Gaussian Vectors and Processes

The Gaussian distribution is the fixed point of linear operations. A linear image of a Gaussian is Gaussian, a sum of independent Gaussians is Gaussian, and the limit of normalised sums is Gaussian, which is why it is the universal law of aggregated randomness. Measure-theoretically a Gaussian is determined by two numbers, its mean and its variance, and a Gaussian vector by a mean vector and a covariance matrix, the minimal second-order data. This post builds the Gaussian vector and the Gaussian process from the characteristic function of the previous post [1], [2].

#The Gaussian characteristic function

Definition1

A random variable is standard normal, written $Z\sim N(0,1)$ , when it has density $\frac{1}{\sqrt{2\pi}} e^{-x^2/2}$ . A variable $X=\mu+\sigma Z$ is Gaussian $N(\mu,\sigma^2)$ , including the degenerate constant $\mu$ when $\sigma=0$ .

Proposition2

The standard normal has characteristic function $\varphi_Z(t)=e^{-t^2/2}$ , and $N(\mu,\sigma^2)$ has $\varphi_X(t)=e^{i\mu t-\sigma^2 t^2/2}$ .

Proof

Differentiating $\varphi_Z(t)=\int e^{itx}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}\,dx$ under the integral, licensed because $\E\abs Z<\infty$ , gives $\varphi_Z'(t)=\int ix\,e^{itx}\frac{1}{\sqrt{2\pi}}e^{-x^2/2}\,dx$ . Integrating by parts with $\frac{d}{dx}e^{-x^2/2}=-xe^{-x^2/2}$ moves the $x$ onto the exponential. The boundary term $[-\tfrac{i}{\sqrt{2\pi}}e^{itx}e^{-x^2/2}]_{x=-\infty}^{x=+\infty}$ vanishes because $\abs{e^{itx}}=1$ and $e^{-x^2/2}\to0$ at $\pm\infty$ , leaving $\varphi_Z'(t)=\frac{i}{\sqrt{2\pi}}\int(it) e^{itx}e^{-x^2/2}\,dx=-t\,\varphi_Z(t)$ . With $\varphi_Z(0)=1$ this linear equation has the unique solution $\varphi_Z(t)=e^{-t^2/2}$ . For $X=\mu+\sigma Z$ , $\varphi_X(t)=\E[e^{it(\mu+\sigma Z)}]=e^{i\mu t} \varphi_Z(\sigma t)=e^{i\mu t-\sigma^2t^2/2}$ .

The quadratic exponent is the signature of the Gaussian. The transform of $N(\mu,\sigma^2)$ is an exponential of a quadratic in $t$ , and reading off the coefficients recovers the mean and variance.

#Gaussian vectors

The right definition of a Gaussian vector asks that the distribution survive every projection to a line.

Definition3

A random vector $X\in\R^n$ is Gaussian when every linear combination $a^\top X=\sum_i a_iX_i$ is a univariate Gaussian. Its mean is $\mu=\E[X]$ and its covariance is the matrix $\Sigma_{ij}=\Cov(X_i,X_j) =\E[(X_i-\mu_i)(X_j-\mu_j)]$ .

The covariance is symmetric and positive semidefinite, since $a^\top\Sigma a=\Var(a^\top X)\ge 0$ . Mean and covariance are all the data there is.

Theorem4

A Gaussian vector has characteristic function $\varphi_X(t)=\exp(i\,t^\top\mu-\tfrac12 t^\top\Sigma t)$ , so its law is determined by $\mu$ and $\Sigma$ alone.

Proof

For fixed $t\in\R^n$ the scalar $t^\top X$ is Gaussian by definition, with mean $t^\top\mu$ and variance $\E[(t^\top X-t^\top\mu)^2]=t^\top\Sigma t$ . Evaluating its scalar characteristic function at argument $1$ , $\varphi_X(t)=\E[e^{i\,t^\top X}]=\E[e^{i(t^\top X)}]=\exp(i\,t^\top\mu-\tfrac12 t^\top\Sigma t)$ by Proposition 2. The characteristic function depends only on $\mu$ and $\Sigma$ . If two laws on $\R^n$ share a characteristic function, then for every $t$ the projections $x\mapsto t^\top x$ have equal scalar characteristic functions along the ray, so by the uniqueness theorem all one-dimensional projections coincide, and by the Cramer-Wold device the projections determine the joint law. Hence the law depends only on $\mu$ and $\Sigma$ .

In the Gaussian world the second moment is the whole story, and that collapses the distinction between independence and zero correlation.

Corollary5

The coordinates of a Gaussian vector are independent if and only if they are uncorrelated. More generally, subvectors $X_{(1)}$ and $X_{(2)}$ are independent if and only if their cross-covariance block vanishes.

Proof

Independence always implies zero covariance. Conversely, if the cross-covariance block is zero, then $\Sigma$ is block diagonal, so the quadratic form splits, $t^\top\Sigma t=t_{(1)}^\top\Sigma_{(1)}t_{(1)}+ t_{(2)}^\top\Sigma_{(2)}t_{(2)}$ , and the characteristic function factors, $\varphi_X(t)=\varphi_{X_{(1)}}(t_{(1)})\,\varphi_{X_{(2)}}(t_{(2)})$ . Each factor is a genuine marginal characteristic function, because a subvector $X_{(1)}$ is itself Gaussian (any $a^\top X_{(1)}=(a,0)^\top X$ is a linear combination of all coordinates of $X$ , hence univariate Gaussian by Definition 3) with covariance the leading block $\Sigma_{(1)}$ , so by Theorem 4 its characteristic function is $\exp(i\,t_{(1)}^\top\mu_{(1)}-\tfrac12 t_{(1)}^\top\Sigma_{(1)}t_{(1)})$ , the first factor, and likewise for $X_{(2)}$ . A joint characteristic function that factors into the marginals is the product law, so the subvectors are independent.

In general zero correlation is weaker than independence; the Gaussian collapses the two, which is what makes it the tractable model of dependence.

Conversely, any mean and any positive semidefinite covariance are realised by some Gaussian vector.

Proposition6

For every $\mu\in\R^n$ and symmetric positive semidefinite $\Sigma$ , there is a Gaussian vector with mean $\mu$ and covariance $\Sigma$ .

Proof

Being symmetric positive semidefinite, $\Sigma$ factors as $\Sigma=AA^\top$ , for instance through its spectral decomposition $\Sigma=Q\Lambda Q^\top$ with $A=Q\Lambda ^{1/2}$ . Let $Z=(Z_1,\dots,Z_n)$ have independent standard normal coordinates and set $X=\mu+AZ$ . Then $\E[X]=\mu$ and $\Cov(X)=A\,\Cov(Z)\,A^\top=AA^\top=\Sigma$ , and every $a^\top X=a^\top\mu+(A^\top a)^\top Z$ is a linear combination of independent Gaussians, hence Gaussian, so $X$ is a Gaussian vector with the required mean and covariance.

#Gaussian processes

A process is a family of random variables indexed by a parameter, usually time. The Gaussian property is imposed on every finite subfamily at once.

Definition7

A Gaussian process $(X_t)_{t\in T}$ is a family of random variables for which every finite vector $(X_{t_1},\dots,X_{t_n})$ is a Gaussian vector. It is described by its mean function $m(t)=\E[X_t]$ and its covariance function $K(s,t)=\Cov(X_s,X_t)$ .

The covariance function is symmetric and positive semidefinite, meaning every matrix $(K(t_i,t_j))_{i,j}$ is positive semidefinite, being the covariance of $(X_{t_1},\dots,X_{t_n})$ . These two functions determine the process, and conversely any such pair is realised.

Theorem8

For every function $m$ on $T$ and every symmetric positive semidefinite kernel $K$ on $T\times T$ , there is a Gaussian process with mean function $m$ and covariance function $K$ , unique in law.

Proof

For each finite set $t_1,\dots,t_n$ define $\mu^{(n)}=(m(t_i))_i$ and $\Sigma^{(n)}=(K(t_i,t_j))_{ij}$ , which is symmetric positive semidefinite by hypothesis, and let $P_{t_1,\dots,t_n}$ be the Gaussian law of mean $\mu^{(n)}$ and covariance $\Sigma^{(n)}$ from Proposition 6. This family is consistent, since a marginal of a Gaussian over a subset of coordinates is the Gaussian with the corresponding subvector mean and submatrix covariance, which is exactly $P$ on the smaller index set, and permuting indices permutes the law correspondingly. The Kolmogorov extension theorem, which builds a measure on the product space $\R^T$ from a consistent family of finite-dimensional laws by Caratheodory extension from the algebra of cylinder sets, then produces a process with these finite-dimensional distributions, Gaussian by construction. Uniqueness in law holds because the finite-dimensional distributions determine the law on the cylinder sigma-algebra, again an intersection-closed generating system.

The covariance function carries the entire second-order structure, and when the index set is an interval and the kernel is continuous, it is exactly the Mercer kernel of the previous track. The Karhunen-Loeve expansion applies the spectral decomposition of that kernel to the process itself, writing $X_t=m(t)+\sum_n\sqrt{\lambda_n}\,\xi_n \varphi_n(t)$ with independent standard Gaussian coefficients $\xi_n$ , where the eigenvalues are the variances of the modes and the eigenfunctions their shapes. Brownian motion is the Gaussian process with $m=0$ and $K(s,t)=\min(s,t)$ , and its construction, the foundation of stochastic calculus, is the next step.

[1]

R. Durrett, Probability: Theory and Examples, 5th ed. in Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2019.

[2]

O. Kallenberg, Foundations of Modern Probability, 3rd ed. Springer, 2021.

Explore connections

see in the atlas

referenced by (6)

cite

@misc{gaussian-vectors-and-processes,
  author = {Zac Kienzle},
  title  = {Gaussian Vectors and Processes},
  year   = {2026},
  month  = {06},
  url    = {https://zackienzle.com/blog/gaussian-vectors-and-processes}
}

#The Gaussian characteristic function

Definition1

Proposition2

The standard normal has characteristic function $\varphi_Z(t)=e^{-t^2/2}$ , and $N(\mu,\sigma^2)$ has $\varphi_X(t)=e^{i\mu t-\sigma^2 t^2/2}$ .

Proof

The quadratic exponent is the signature of the Gaussian. The transform of $N(\mu,\sigma^2)$ is an exponential of a quadratic in $t$ , and reading off the coefficients recovers the mean and variance.

#Gaussian vectors

The right definition of a Gaussian vector asks that the distribution survive every projection to a line.

Definition3

The covariance is symmetric and positive semidefinite, since $a^\top\Sigma a=\Var(a^\top X)\ge 0$ . Mean and covariance are all the data there is.

Theorem4

A Gaussian vector has characteristic function $\varphi_X(t)=\exp(i\,t^\top\mu-\tfrac12 t^\top\Sigma t)$ , so its law is determined by $\mu$ and $\Sigma$ alone.

Proof

In the Gaussian world the second moment is the whole story, and that collapses the distinction between independence and zero correlation.

Corollary5

Proof

In general zero correlation is weaker than independence; the Gaussian collapses the two, which is what makes it the tractable model of dependence.

Conversely, any mean and any positive semidefinite covariance are realised by some Gaussian vector.

Proposition6

For every $\mu\in\R^n$ and symmetric positive semidefinite $\Sigma$ , there is a Gaussian vector with mean $\mu$ and covariance $\Sigma$ .

Proof

#Gaussian processes

A process is a family of random variables indexed by a parameter, usually time. The Gaussian property is imposed on every finite subfamily at once.

Definition7

Theorem8

For every function $m$ on $T$ and every symmetric positive semidefinite kernel $K$ on $T\times T$ , there is a Gaussian process with mean function $m$ and covariance function $K$ , unique in law.

Proof

[1]

R. Durrett, Probability: Theory and Examples, 5th ed. in Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2019.

[2]

O. Kallenberg, Foundations of Modern Probability, 3rd ed. Springer, 2021.

Explore connections

see in the atlas

referenced by (6)

cite

@misc{gaussian-vectors-and-processes,
  author = {Zac Kienzle},
  title  = {Gaussian Vectors and Processes},
  year   = {2026},
  month  = {06},
  url    = {https://zackienzle.com/blog/gaussian-vectors-and-processes}
}