Positive Definite Matrices

A positive semidefinite matrix is the matrix version of a nonnegative number, one whose quadratic form $x^\top Ax$ never drops below zero. These are exactly the matrices that arise as covariances, as Gram matrices of inner products, and as the Hessians of convex functions, and the spectral theorem gives them a square root just as a nonnegative number has one. This post proves the characterisations of positive semidefiniteness, the Cholesky factorisation, and the identification of the covariance matrix as the canonical example [1], [2]. Matrices are real symmetric and act on $\R^n$ with $\ip xy=x^\top y$ .

#Characterisations

Definition1

A symmetric matrix $A$ is positive semidefinite, written $A\succeq 0$ , when $x^\top Ax\ge 0$ for all $x$ , and positive definite, $A\succ 0$ , when $x^\top Ax>0$ for all $x\neq 0$ .

The quadratic form, the eigenvalues, and a factorisation all say the same thing.

Theorem2

For a symmetric $A$ the following are equivalent. First, $A\succeq 0$ . Second, every eigenvalue of $A$ is nonnegative. Third, $A=B^\top B$ for some matrix $B$ . Fourth, $A$ has a positive semidefinite square root, a symmetric $A^{1/2}\succeq 0$ with $(A^{1/2})^2=A$ .

Proof

By the spectral theorem write $A=Q\Lambda Q^\top$ with $Q$ orthogonal and $\Lambda=\operatorname{diag}(\lambda_i)$ . For any $x$ , substituting $y=Q^ \top x$ gives $x^\top Ax=y^\top\Lambda y=\sum_i\lambda_i y_i^2$ . If every $\lambda_i\ge 0$ this is nonnegative, so the second condition implies the first. Conversely, taking $x=q_i$ gives $x^\top Ax= \lambda_i$ , so $A\succeq 0$ forces each $\lambda_i\ge 0$ . Given nonnegative eigenvalues, set $A^{1/2}=Q \Lambda^{1/2}Q^\top$ with $\Lambda^{1/2}=\operatorname{diag}(\sqrt{\lambda_i})$ , which is symmetric, has nonnegative eigenvalues hence is positive semidefinite, and squares to $Q\Lambda Q^\top=A$ , giving the fourth condition. The square root is a factorisation $A=B^\top B$ with $B=A^{1/2}$ symmetric, the third condition. Finally $A=B^\top B$ implies $x^\top Ax=x^\top B^\top Bx=\norm{Bx}^2\ge 0$ , the first condition, closing the cycle.

The positive definite case is the same with strict inequalities, $A\succ 0$ exactly when all eigenvalues are positive, equivalently $A=B^\top B$ with $B$ of full column rank, equivalently $A$ invertible and positive semidefinite. The square root is moreover unique among positive semidefinite matrices. Let $S\succeq 0$ with $S^2=A$ , diagonalised $S=U\operatorname{diag}(s_j)U^\top$ with $s_j\ge 0$ and orthonormal columns $u_j$ . For any eigenvector $v$ of $A$ with $Av=\mu v$ , writing $v=\sum_j c_j u_j$ gives $\sum_j c_j s_j^2 u_j=S^2v=Av= \sum_j c_j\mu u_j$ , so $c_j(s_j^2-\mu)=0$ and hence $c_j\neq 0\Rightarrow s_j=\sqrt\mu$ ; therefore $Sv= \sum_j c_j s_j u_j=\sqrt\mu\,v$ . Thus $S$ acts as $\sqrt{\lambda_i}$ on each eigenspace of $A$ and is forced to equal $Q\Lambda^{1/2}Q^\top=A^{1/2}$ . The argument needs only $A\succeq 0$ .

#The Cholesky factorisation

A positive definite matrix factors through a triangular matrix, the form a numerical solver uses.

Theorem3

A positive definite matrix $A$ has a unique factorisation $A=LL^\top$ with $L$ lower triangular and positive diagonal entries.

Proof

Argue by induction on $n$ , the case $n=1$ being $A=(a)$ with $a>0$ and $L=(\sqrt a)$ . For $n>1$ write $A$ in block form with first entry $a=A_{11}>0$ , first-column tail $b$ , and lower block $C$ ,

A=\begin{pmatrix}a & b^\top\\ b & C\end{pmatrix},\qquad L=\begin{pmatrix}\sqrt a & 0\\ b/\sqrt a & L' \end{pmatrix}. \tag{1}

Multiplying out, $LL^\top$ has corner $a$ , off-diagonal blocks $b$ , and lower block $\frac{bb^\top}{a}+L'L'^ \top$ , so $A=LL^\top$ requires $L'L'^\top=C-\frac{bb^\top}{a}$ , the Schur complement. That complement is positive definite of size $n-1$ , because for any $z\neq 0$ the vector $x=(-\frac{b^\top z}{a},z)$ has $x^ \top Ax=z^\top(C-\frac{bb^\top}{a})z$ after expanding, and positivity of $A$ makes it positive. By induction the Schur complement has a Cholesky factor $L'$ , and assembling Equation (1) gives $A=LL^\top$ with positive diagonal. Uniqueness follows because the equations fix $\sqrt a$ , then $b/\sqrt a$ , then recurse on $L'$ , each step determined.

#The covariance matrix

The canonical positive semidefinite matrix is a covariance, and its eigenstructure is the principal component analysis of the underlying randomness.

Proposition4

The covariance matrix $\Sigma=\mathbb E[(X-\mu)(X-\mu)^\top]$ of a random vector $X$ with mean $\mu$ and finite second moments $\mathbb E[\norm{X}^2]<\infty$ is positive semidefinite, and $\Sigma\succ 0$ unless $X$ lies almost surely in a proper affine subspace.

Proof

Each $X_i\in L^2$ , so $X_iX_j\in L^1$ by Cauchy-Schwarz and every $\Sigma_{ij}$ is finite. For any deterministic $a$ the integrand $\sum_{i,j}a_ia_j(X_i-\mu_i)(X_j-\mu_j)$ is a finite sum of integrable terms, so linearity of expectation gives $a^\top\Sigma a=\mathbb E[(a^\top(X-\mu))^2]=\operatorname{Var}(a^\top X)\ge 0$ , so $\Sigma\succeq 0$ . The form vanishes, $a^\top\Sigma a=0$ , exactly when $a^\top X$ is almost surely constant, which confines $X$ to the affine hyperplane $\{x:a^\top x=a^\top\mu\}$ . If no such $a\neq 0$ exists, every $a^\top\Sigma a>0$ and $\Sigma\succ 0$ .

The eigenvectors of $\Sigma$ are the principal axes of the data and its eigenvalues the variances along them, by the variational characterisation, so the spectral decomposition of a covariance is principal component analysis, the same decomposition the Karhunen-Loeve expansion performs in infinite dimensions. The square root $\Sigma^{1/2}$ is the linear map that turns uncorrelated unit-variance noise into a sample with covariance $\Sigma$ , the construction that realises a Gaussian vector of any prescribed covariance. Positive definiteness of $\Sigma$ is the condition that makes the inverse $\Sigma^{-1}$ exist and the quadratic risk $w^\top\Sigma w$ of a portfolio strictly convex, so the mean-variance problem has a unique solution. Positive definiteness is the algebraic form of genuine randomness in every direction.

[1]

R. A. Horn and C. R. Johnson, Matrix Analysis, 2nd ed. Cambridge University Press, 2013.

[2]

G. Strang, Introduction to Linear Algebra, 5th ed. Wellesley-Cambridge Press, 2016.

Explore connections

see in the atlas

referenced by (1)

The Mean-Variance Portfolio

cite

@misc{positive-definite-matrices,
  author = {Zac Kienzle},
  title  = {Positive Definite Matrices},
  year   = {2026},
  month  = {06},
  url    = {https://zackienzle.com/blog/positive-definite-matrices}
}

#Characterisations

Definition1

A symmetric matrix $A$ is positive semidefinite, written $A\succeq 0$ , when $x^\top Ax\ge 0$ for all $x$ , and positive definite, $A\succ 0$ , when $x^\top Ax>0$ for all $x\neq 0$ .

The quadratic form, the eigenvalues, and a factorisation all say the same thing.

Theorem2

Proof

#The Cholesky factorisation

A positive definite matrix factors through a triangular matrix, the form a numerical solver uses.

Theorem3

A positive definite matrix $A$ has a unique factorisation $A=LL^\top$ with $L$ lower triangular and positive diagonal entries.

Proof

Argue by induction on $n$ , the case $n=1$ being $A=(a)$ with $a>0$ and $L=(\sqrt a)$ . For $n>1$ write $A$ in block form with first entry $a=A_{11}>0$ , first-column tail $b$ , and lower block $C$ ,

A=\begin{pmatrix}a & b^\top\\ b & C\end{pmatrix},\qquad L=\begin{pmatrix}\sqrt a & 0\\ b/\sqrt a & L' \end{pmatrix}. \tag{1}

#The covariance matrix

The canonical positive semidefinite matrix is a covariance, and its eigenstructure is the principal component analysis of the underlying randomness.

Proposition4

Proof

[1]

R. A. Horn and C. R. Johnson, Matrix Analysis, 2nd ed. Cambridge University Press, 2013.

[2]

G. Strang, Introduction to Linear Algebra, 5th ed. Wellesley-Cambridge Press, 2016.

Explore connections

see in the atlas

referenced by (1)

The Mean-Variance Portfolio

cite

@misc{positive-definite-matrices,
  author = {Zac Kienzle},
  title  = {Positive Definite Matrices},
  year   = {2026},
  month  = {06},
  url    = {https://zackienzle.com/blog/positive-definite-matrices}
}