Inner Product Spaces

A norm measures length, but it does not by itself measure angle. The extra structure that does is an inner product, and with angle comes orthogonality, the relation the rest of the subject rests on. Two orthogonal vectors are independent in the strongest geometric sense, and projecting onto a subspace, expanding in an orthonormal basis, and diagonalising an operator all reduce to manipulating it. This post builds that geometry from the axioms and assumes the language of metric and normed spaces [1], [2]. We work over the real field throughout; the complex case differs only by conjugations.

#Inner products

Definition1

An inner product on a real vector space $V$ is a function $\ip{\cdot}{\cdot}:V\times V\to\R$ that is symmetric, $\ip{x}{y}=\ip{y}{x}$ , linear in the first argument, $\ip{\alpha x+\beta x'}{y}=\alpha \ip{x}{y}+\beta\ip{x'}{y}$ , and positive definite, $\ip{x}{x}\ge 0$ with equality only at $x=0$ . The pair $(V,\ip{\cdot}{\cdot})$ is an inner product space, and $\norm{x}=\sqrt{\ip{x}{x}}$ is the induced norm.

Euclidean space carries $\ip{x}{y}=\sum_i x_iy_i$ . The space of square-integrable functions carries $\ip{f}{g}=\int fg$ , the model that the Lebesgue space $L^2$ makes rigorous. Symmetry and first-argument linearity together give linearity in the second argument, since $\ip{x}{\alpha y+\beta y'}=\ip{\alpha y+\beta y'}{x}=\alpha\ip{y}{x}+\beta\ip{y'}{x} =\alpha\ip{x}{y}+\beta\ip{x}{y'}$ , so the inner product is bilinear. The decisive consequence of positive definiteness is the following inequality, which binds the inner product to the norm.

#Cauchy-Schwarz and the triangle inequality

Theorem2

For all $x,y$ in an inner product space, $\abs{\ip{x}{y}}\le\norm{x}\,\norm{y}$ , with equality if and only if $x$ and $y$ are linearly dependent.

Proof

If $y=0$ both sides vanish. Otherwise set $t=\ip{x}{y}/\norm{y}^2$ and expand the nonnegative quantity

0\le\norm{x-ty}^2=\norm{x}^2-2t\ip{x}{y}+t^2\norm{y}^2=\norm{x}^2-\frac{\ip{x}{y}^2}{\norm{y}^2}, \tag{1}

using bilinearity. Rearranging gives $\ip{x}{y}^2\le\norm{x}^2\norm{y}^2$ , which is the inequality. Equality holds exactly when $\norm{x-ty}^2=0$ , that is when $x=ty$ , the dependent case.

Proposition3

The induced norm is a norm, in particular $\norm{x+y}\le\norm{x}+\norm{y}$ .

Proof

Positivity and homogeneity are immediate from the definition. For the triangle inequality, note $\ip{x}{y}\le\abs{\ip{x}{y}}\le\norm{x}\norm{y}$ , the first since every real is at most its absolute value and the second by Theorem 2, so with bilinearity

\norm{x+y}^2=\norm{x}^2+2\ip{x}{y}+\norm{y}^2\le\norm{x}^2+2\norm{x}\norm{y}+\norm{y}^2 =(\norm{x}+\norm{y})^2, \tag{2}

and taking square roots finishes it.

So every inner product space is a normed space, and the Cauchy-Schwarz inequality is what makes the inner product jointly continuous, since $\abs{\ip{x}{y}-\ip{x'}{y'}}\le\abs{\ip{x-x'}{y}} +\abs{\ip{x'}{y'-y}}\le\norm{x-x'}\norm{y}+\norm{x'}\norm{y'-y}$ , which tends to $0$ as $(x',y')\to(x,y)$ .

#The parallelogram law

Inner-product norms are special among norms. They satisfy an identity that the supremum norm, for example, does not.

Proposition4

The induced norm satisfies the parallelogram law $\norm{x+y}^2+\norm{x-y}^2=2\norm{x}^2+2\norm{y}^2$ .

Proof

Expand both squares by bilinearity. The sum is $(\norm{x}^2+2\ip{x}{y}+\norm{y}^2)+(\norm{x}^2-2\ip{x}{y}+\norm{y}^2)=2\norm{x}^2+2\norm{y}^2$ , the cross terms cancelling.

The converse holds, so the parallelogram law characterises which norms come from an inner product.

Theorem5

If a norm on a real vector space satisfies the parallelogram law, then the polarisation formula $\ip{x}{y}=\tfrac14\big(\norm{x+y}^2-\norm{x-y}^2\big)$ defines an inner product inducing that norm.

Proof

Symmetry and $\ip{x}{x}=\norm{x}^2$ are read off the formula, and $\ip{x}{x}>0$ for $x\neq 0$ . For additivity in the first argument, apply the parallelogram law to the pairs $(x+z,y)$ and $(x-z,y)$ and subtract, which yields $\ip{x+z}{y}+\ip{x-z}{y}=2\ip{x}{y}$ after the norms recombine. Setting $z=x$ gives $\ip{2x}{y}=2\ip{x}{y}$ since $\ip{0}{y}=0$ , and feeding this back turns the relation into the Cauchy additivity equation $\ip{u}{y}+\ip{v}{y}=\ip{u+v}{y}$ for all $u,v$ . Additivity forces $\ip{qx}{y}=q\ip{x}{y}$ for every rational $q$ , and since $t\mapsto\ip{tx}{y}$ is continuous (it is built from the continuous norm), the identity extends to all real $t$ , giving homogeneity. The form is therefore bilinear, symmetric, and positive definite, an inner product, and it induces the original norm.

Together the two results say a normed space is an inner product space exactly when its norm satisfies the parallelogram law.

#Orthogonality

Vectors $x$ and $y$ are orthogonal, written $x\perp y$ , when $\ip{x}{y}=0$ . Orthogonality turns the triangle inequality into an equality.

Proposition6

If $x\perp y$ then $\norm{x+y}^2=\norm{x}^2+\norm{y}^2$ . More generally, for pairwise orthogonal $x_1,\dots,x_n$ , $\norm{\sum_i x_i}^2=\sum_i\norm{x_i}^2$ .

Proof

Expanding $\norm{\sum_i x_i}^2=\sum_{i,j}\ip{x_i}{x_j}$ by bilinearity, every cross term with $i\neq j$ vanishes by orthogonality, leaving $\sum_i\ip{x_i}{x_i}=\sum_i\norm{x_i}^2$ .

A family $(e_i)$ is orthonormal when $\ip{e_i}{e_j}=\delta_{ij}$ . The coefficients of a vector against an orthonormal family are its projections, and the sum of their squares cannot exceed the vector's squared norm.

Theorem7

For an orthonormal family $e_1,\dots,e_n$ and any $x$ , the coefficients $c_i=\ip{x}{e_i}$ satisfy Bessel's inequality $\sum_{i=1}^n c_i^2\le\norm{x}^2$ , and the residual $x-\sum_i c_i e_i$ is orthogonal to every $e_j$ .

Proof

Let $p=\sum_i c_i e_i$ and $r=x-p$ . For each $j$ , $\ip{r}{e_j}=\ip{x}{e_j}-\sum_i c_i\ip{e_i}{e_j} =c_j-c_j=0$ , so the residual is orthogonal to every $e_j$ and hence to $p$ . Pythagoras applied to $x=p+r$ gives $\norm{x}^2=\norm{p}^2+\norm{r}^2\ge\norm{p}^2=\sum_i c_i^2$ , the orthonormality making $\norm{p}^2=\sum_i c_i^2$ .

Bessel's inequality is the finite shadow of the expansion theory to come. The vector $p=\sum_i c_i e_i$ is the orthogonal projection of $x$ onto the span of the $e_i$ , the closest point of that subspace, and the residual is the projection error. When an orthonormal family is large enough that the residual vanishes for every $x$ , Bessel's inequality becomes the Parseval equality and the family is a basis, the subject of a later post.

#Hilbert spaces

Definition8

A Hilbert space is an inner product space that is complete in the induced norm, meaning every Cauchy sequence converges.

Finite-dimensional inner product spaces are automatically complete, by the equivalence of norms proved for finite-dimensional spaces and the completeness of Euclidean space. The interesting Hilbert spaces are infinite-dimensional, the sequence space $\ell^2$ and the function space $L^2$ , whose completeness is a theorem rather than an automatic fact and is the subject of the next post. Completeness lets an orthogonal series converge to an actual vector, turning Bessel's inequality into a working expansion, the projection theorem into an existence result, and the spectral theorem into a decomposition. Geometry supplies the angles, completeness the limits; their combination is the Hilbert space.

[1]

W. Rudin, Functional Analysis, 2nd ed. McGraw-Hill, 1991.

[2]

J. B. Conway, A Course in Functional Analysis, 2nd ed. Springer, 1990.

Explore connections

see in the atlas

referenced by (8)

cite

@misc{inner-product-spaces,
  author = {Zac Kienzle},
  title  = {Inner Product Spaces},
  year   = {2026},
  month  = {05},
  url    = {https://zackienzle.com/blog/inner-product-spaces}
}

#Inner products

Definition1

#Cauchy-Schwarz and the triangle inequality

Theorem2

For all $x,y$ in an inner product space, $\abs{\ip{x}{y}}\le\norm{x}\,\norm{y}$ , with equality if and only if $x$ and $y$ are linearly dependent.

Proof

If $y=0$ both sides vanish. Otherwise set $t=\ip{x}{y}/\norm{y}^2$ and expand the nonnegative quantity

0\le\norm{x-ty}^2=\norm{x}^2-2t\ip{x}{y}+t^2\norm{y}^2=\norm{x}^2-\frac{\ip{x}{y}^2}{\norm{y}^2}, \tag{1}

using bilinearity. Rearranging gives $\ip{x}{y}^2\le\norm{x}^2\norm{y}^2$ , which is the inequality. Equality holds exactly when $\norm{x-ty}^2=0$ , that is when $x=ty$ , the dependent case.

Proposition3

The induced norm is a norm, in particular $\norm{x+y}\le\norm{x}+\norm{y}$ .

Proof

\norm{x+y}^2=\norm{x}^2+2\ip{x}{y}+\norm{y}^2\le\norm{x}^2+2\norm{x}\norm{y}+\norm{y}^2 =(\norm{x}+\norm{y})^2, \tag{2}

and taking square roots finishes it.

#The parallelogram law

Inner-product norms are special among norms. They satisfy an identity that the supremum norm, for example, does not.

Proposition4

The induced norm satisfies the parallelogram law $\norm{x+y}^2+\norm{x-y}^2=2\norm{x}^2+2\norm{y}^2$ .

Proof

Expand both squares by bilinearity. The sum is $(\norm{x}^2+2\ip{x}{y}+\norm{y}^2)+(\norm{x}^2-2\ip{x}{y}+\norm{y}^2)=2\norm{x}^2+2\norm{y}^2$ , the cross terms cancelling.

The converse holds, so the parallelogram law characterises which norms come from an inner product.

Theorem5

If a norm on a real vector space satisfies the parallelogram law, then the polarisation formula $\ip{x}{y}=\tfrac14\big(\norm{x+y}^2-\norm{x-y}^2\big)$ defines an inner product inducing that norm.

Proof

Together the two results say a normed space is an inner product space exactly when its norm satisfies the parallelogram law.

#Orthogonality

Vectors $x$ and $y$ are orthogonal, written $x\perp y$ , when $\ip{x}{y}=0$ . Orthogonality turns the triangle inequality into an equality.

Proposition6

If $x\perp y$ then $\norm{x+y}^2=\norm{x}^2+\norm{y}^2$ . More generally, for pairwise orthogonal $x_1,\dots,x_n$ , $\norm{\sum_i x_i}^2=\sum_i\norm{x_i}^2$ .

Proof

Expanding $\norm{\sum_i x_i}^2=\sum_{i,j}\ip{x_i}{x_j}$ by bilinearity, every cross term with $i\neq j$ vanishes by orthogonality, leaving $\sum_i\ip{x_i}{x_i}=\sum_i\norm{x_i}^2$ .

Theorem7

Proof

#Hilbert spaces

Definition8

A Hilbert space is an inner product space that is complete in the induced norm, meaning every Cauchy sequence converges.

[1]

W. Rudin, Functional Analysis, 2nd ed. McGraw-Hill, 1991.

[2]

J. B. Conway, A Course in Functional Analysis, 2nd ed. Springer, 1990.

Explore connections

see in the atlas

referenced by (8)

cite

@misc{inner-product-spaces,
  author = {Zac Kienzle},
  title  = {Inner Product Spaces},
  year   = {2026},
  month  = {05},
  url    = {https://zackienzle.com/blog/inner-product-spaces}
}