Second-Order Processes and Mean-Square Calculus

Suppose $K$ is continuous and $m$ is continuous. By Equation (1), as $t\to t_0$ the right side tends to $K(t_0,t_0)+K(t_0,t_0)-2K(t_0,t_0)+0=0$, so $\E[(X_t-X_{t_0})^2]\to 0$ and the process is mean-square continuous. Conversely, suppose the process is mean-square continuous everywhere. Then the centred variables $Y_t=X_t-m(t)$ form a mean-square continuous curve, since $m$ is continuous, and the inner product is continuous on $L^2$, so $(s,t)\mapsto\ip{Y_s}{Y_t} =K(s,t)$ is continuous as a composition of the continuous map $(s,t)\mapsto(Y_s,Y_t)$ with the continuous inner product. Thus mean-square continuity forces $K$ continuous.

Mean-square continuity makes $m$ continuous, since $\abs{m(t)-m(t_0)}=\abs{\E[X_t-X_{t_0}]}\le\norm{X_t-X_{t_0}}_{L^2}\to 0$ as the expectation is a bounded linear functional of norm one, and then $K$ is continuous by the converse half of Theorem 2, so $g(s,t)=\ip{X_s}{X_t}=K(s,t)+m(s)m(t)$ is continuous. For two partitions, the inner product of their Riemann sums is the double sum $\sum_{i,j}\ip{X_{s_i}}{X_{t_j}}\Delta s_i\Delta t_j$, a Riemann sum for $\int\!\!\int g(s,t)\,ds\,dt =\int\!\!\int(K(s,t)+m(s)m(t))\,ds\,dt=:I$. Since $g$ is continuous on the compact square $[a,b]^2$ it is uniformly continuous there, so any tagged Riemann sums of $g$ with mesh tending to $0$, including those with the mixed tags $(s_i,t_j)$, converge to $I$. The polarisation $\norm{S_\pi-S_{\pi'}}^2=\ip{S_\pi}{S_\pi}-2\ip{S_\pi}{S_{\pi'}}+\ip{S_{\pi'}}{S_{\pi'}}\to I-2I+I=0$ then shows every two sequences of Riemann sums with mesh tending to $0$ are Cauchy and share a limit, by completeness of $L^2$. The mean identity is continuity of the expectation, a bounded linear functional on $L^2$. The inner product of the limit with itself is the second moment $\ip II=\int\!\!\int(K+m m)$; subtracting the squared mean $\big(\int m\big)^2=\int\!\!\int m(s)m(t)\,ds\,dt$ leaves the variance identity $\Var\big(\int X\big)=\int\!\!\int K(s,t)\,ds\,dt$.

The kernel $K$ is continuous and symmetric on the compact square, so by the Mercer setup $T_K$ is compact and self-adjoint. For positivity, take first a continuous $f$. Then $t\mapsto f(t)Y_t$ is mean-square continuous, so the integral $Z_f=\int_a^b f(t)Y_t\,dt$ exists by Theorem 3, and its variance is, by Equation (2) applied to that process with covariance $f(s)K(s,t)f(t)$,

A variance is nonnegative, so $\ip{T_K f}{f}\ge 0$ for every continuous $f$. The quadratic form $f\mapsto \ip{T_K f}{f}$ is continuous on $L^2$, since $\abs{\ip{T_K f}{f}-\ip{T_K g}{g}}\le\norm{T_K}(\norm f+\norm g)\norm{f-g}$ by boundedness of $T_K$, and the continuous functions are dense in $L^2([a,b])$, so nonnegativity passes to the closure and $\ip{T_K f}{f}\ge 0$ for all $f\in L^2$, making $T_K$ positive.

Each $\xi_n$ is a mean-square integral of the centred process, so $\E[\xi_n]=\frac{1}{\sqrt{\lambda_n}}\int \E[Y_t]\varphi_n(t)\,dt=0$. For the cross moment, write $Z_n=\int_a^b Y_s\varphi_n(s)\,ds$ as the $L^2$ limit of Riemann sums; joint continuity of the inner product gives the polarisation of Equation (2), $\E[Z_nZ_m]=\ip{Z_n}{Z_m}=\lim\sum_{i,j}\varphi_n(s_i)\varphi_m(t_j)\ip{Y_{s_i}}{Y_{t_j}}\Delta s_i\Delta t_j=\int\!\!\int\varphi_n(s)K(s,t)\varphi_m(t)\,ds\,dt$, the integrand being continuous. Hence

Since $T_K\varphi_m=\lambda_m\varphi_m$ and the eigenfunctions are orthonormal, the right side is $\frac{\lambda_m}{\sqrt{\lambda_n\lambda_m}}\int\varphi_n\varphi_m=\frac{\lambda_m}{\sqrt{\lambda_n\lambda_m}} \delta_{nm}=\delta_{nm}$.