Metric and Normed Spaces

The convergence theory of the previous post used the real line only through the distance $\abs{a_n-L}$ between a term and its limit. Nothing in the definition of a limit, or in the Cauchy criterion, needs the order of the reals or their arithmetic, only a way to measure how far apart two points are. Abstracting that measurement gives metric spaces, the setting in which the same analysis runs for sequences of vectors, of functions, or of any objects at all, and it is the language every later post speaks. This post uses the completeness of the reals and the Bolzano-Weierstrass theorem from the chapter on sequences and completeness.

#Metric spaces

A metric is a distance function obeying the three properties any reasonable distance must have.

Definition1

A metric space is a set $M$ with a function $d:M\times M\to[0,\infty)$ such that for all $x,y,z\in M$, first $d(x,y)=0$ if and only if $x=y$, second $d(x,y)=d(y,x)$, and third the triangle inequality $d(x,z)\le d(x,y)+d(y,z)$ holds.

The real line is a metric space under $d(x,y)=\abs{x-y}$. Euclidean space $\R^n$ carries the metric $d(x,y)=\big(\sum_i(x_i-y_i)^2\big)^{1/2}$. The set of bounded functions on a domain carries the supremum metric $d(f,g)=\sup_x\abs{f(x)-g(x)}$, under which convergence is uniform convergence. A sequence $(x_n)$ in $M$ converges to $x$ when $d(x_n,x)\to 0$ as real numbers, and is Cauchy when for every $\varepsilon>0$ there is an $N$ with $d(x_m,x_n)<\varepsilon$ for $m,n\ge N$. The definitions are the real-line ones with $\abs{\cdot}$ replaced by $d$, and the proof that limits are unique carries over verbatim from the triangle inequality.

#Open, closed, and continuous

The open ball of radius $r$ about $x$ is $B(x,r)=\{y:d(x,y)<r\}$. A set $U$ is open when every point of $U$ lies in an open ball contained in $U$, and a set $F$ is closed when its complement is open. The characterization of closed sets we use is sequential.

Proposition2

A set $F$ is closed if and only if it contains the limit of every convergent sequence of its points.

Proof

Suppose $F$ is closed and $x_n\to x$ with $x_n\in F$. If $x\notin F$ then $x$ lies in the open complement, so some ball $B(x,r)$ misses $F$ entirely, contradicting $d(x_n,x)\to 0$. Hence $x\in F$. Conversely suppose $F$ contains all such limits. If the complement were not open, some $x\notin F$ has every ball $B(x,1/n)$ meeting $F$, giving points $x_n\in F$ with $d(x_n,x)<1/n$, so $x_n\to x$, contradicting $x\notin F$, since by hypothesis $F$ holds all such limits. The complement is open, so $F$ is closed.

A function $f:M\to M'$ between metric spaces is continuous at $x$ when $x_n\to x$ implies $f(x_n)\to f(x)$, equivalently when for every $\varepsilon>0$ there is a $\delta>0$ with $d'(f(x),f(y))<\varepsilon$ whenever $d(x,y)<\delta$. For the ball form to the sequential form, given $x_n\to x$ and $\varepsilon>0$ pick the matching $\delta$, then $N$ with $d(x_n,x)<\delta$ for $n\ge N$, so $d'(f(x_n),f(x))<\varepsilon$ and $f(x_n)\to f(x)$. For the converse take the contrapositive. If the ball form fails at $x$ there is an $\varepsilon>0$ for which every $\delta=1/n$ admits a point $x_n$ with $d(x_n,x)<1/n$ but $d'(f(x_n),f(x))\ge\varepsilon$, so $x_n\to x$ while $f(x_n)\not\to f(x)$, against the sequential form.

#Completeness and the contraction principle

A metric space is complete when every Cauchy sequence converges. The reals are complete by the Cauchy criterion, and $\R^n$ is complete because a Cauchy sequence is Cauchy in each coordinate, so each coordinate converges and the vector converges to the limit vector. A complete space supports the single most useful existence theorem in analysis.

Theorem3

Let $M$ be a nonempty complete metric space and $T:M\to M$ a contraction, meaning there is a constant $c\in[0,1)$ with $d(Tx,Ty)\le c\,d(x,y)$ for all $x,y$. Then $T$ has a unique fixed point $x^\ast$, and the iterates $x_{n+1}=Tx_n$ converge to it from any start.

Proof

Fix $x_0$ and set $x_{n+1}=Tx_n$. Then $d(x_{n+1},x_n)\le c\,d(x_n,x_{n-1})\le\cdots\le c^n d(x_1,x_0)$. For $m>n$ the triangle inequality and the geometric sum give

$$d(x_m,x_n)\le\sum_{k=n}^{m-1}d(x_{k+1},x_k)\le d(x_1,x_0)\sum_{k=n}^{m-1}c^k \le\frac{c^n}{1-c}\,d(x_1,x_0), \tag{1}$$(1)

which tends to zero as $n\to\infty$, so $(x_n)$ is Cauchy and converges to some $x^\ast$ by completeness. A contraction is Lipschitz, since $d(Tx_n,Tx)\le c\,d(x_n,x)\to 0$, hence continuous, so $Tx^\ast=T\lim x_n=\lim x_{n+1}=x^\ast$, a fixed point. If $x^\ast$ and $y^\ast$ are both fixed, then $d(x^\ast,y^\ast)=d(Tx^\ast,Ty^\ast)\le c\,d(x^\ast,y^\ast)$ with $c<1$ forces $d(x^\ast,y^\ast)=0$, so the fixed point is unique.

The contraction principle is how later posts produce solutions of equations that no formula solves, since the Picard iteration for differential equations is a contraction on a suitable complete space.

#Compactness

A set $K$ is sequentially compact when every sequence in $K$ has a subsequence converging to a point of $K$. In Euclidean space this is exactly the elementary description.

Theorem4

A subset of $\R^n$ is sequentially compact if and only if it is closed and bounded.

Proof

Let $K\subseteq\R^n$ be closed and bounded and let $(x_k)$ be a sequence in $K$. Each coordinate sequence is bounded, so applying the Bolzano-Weierstrass theorem to the first coordinate gives a convergent subsequence, then to the second coordinate along that subsequence, and after $n$ passes a single subsequence converges in every coordinate, hence in $\R^n$. Its limit lies in $K$ because $K$ is closed, by Proposition 2, so $K$ is sequentially compact. Here bounded means $\sup_{x\in K}\norm{x}<\infty$. Conversely, if $K$ is not bounded this supremum is $+\infty$, so for each $k$ there is $x_k\in K$ with $\norm{x_k}>k$, giving $\norm{x_k}\to\infty$, and every subsequence is likewise unbounded and so cannot converge, since a convergent sequence is bounded. If $K$ is not closed it has a sequence converging to a point outside $K$, whose every subsequence has that same outside limit. Either failure breaks sequential compactness, so a sequentially compact set is closed and bounded.

Compactness is preserved by continuous maps, which is the source of every existence-of-extremum result.

Theorem5

The continuous image of a sequentially compact set is sequentially compact. In particular a continuous real function on a sequentially compact set attains a maximum and a minimum.

Proof

Let $f:M\to M'$ be continuous and $K$ sequentially compact, and let $(y_k)$ be a sequence in $f(K)$, say $y_k=f(x_k)$. Some subsequence $x_{k_j}\to x\in K$, and continuity gives $y_{k_j}=f(x_{k_j})\to f(x)\in f(K)$, so $f(K)$ is sequentially compact. When $M'=\R$ the set $f(K)$ is closed and bounded by Theorem 4, so it is bounded, giving a supremum and infimum, and closed, so it contains them, and they are attained.

#Normed spaces

When the set is a vector space the natural metrics come from a length.

Definition6

A norm on a real vector space $V$ is a function $\norm{\cdot}:V\to[0,\infty)$ with $\norm{x}=0$ only at $x=0$, homogeneity $\norm{\alpha x}=\abs{\alpha}\,\norm{x}$, and the triangle inequality $\norm{x+y}\le\norm{x}+\norm{y}$. It induces the metric $d(x,y)=\norm{x-y}$, and a complete normed space is a Banach space.

In finite dimensions the choice of norm does not matter for any convergence question, because all norms are comparable.

Theorem7

Any two norms on a finite-dimensional real vector space are equivalent, meaning there are constants $0<m\le M$ with $m\norm{x}_a\le\norm{x}_b\le M\norm{x}_a$ for all $x$.

Proof

It suffices to compare an arbitrary norm $\norm{\cdot}$ with the Euclidean norm $\norm{\cdot}_2$ in a fixed basis, since equivalence is transitive. By the triangle inequality and homogeneity, $\norm{x}=\norm{\sum_i x_i e_i}\le\sum_i\abs{x_i}\norm{e_i}\le M\norm{x}_2$ with $M=\big(\sum_i\norm{e_i}^2\big)^{1/2}$ by Cauchy-Schwarz, which also shows $\norm{\cdot}$ is continuous with respect to $\norm{\cdot}_2$, since $\abs{\norm{x}-\norm{y}}\le\norm{x-y}\le M\norm{x-y}_2$, where the first step is the reverse triangle inequality $\norm{x}\le\norm{x-y}+\norm{y}$ and its symmetric counterpart. The Euclidean unit sphere $\{x:\norm{x}_2=1\}$ is closed and bounded, hence sequentially compact by Theorem 4, so the continuous function $\norm{\cdot}$ attains a minimum $m$ on it by Theorem 5, and $m>0$ because the norm vanishes only at the origin, which the sphere excludes. Scaling by homogeneity, $m\norm{x}_2\le\norm{x}$ for all $x$, and combined with the upper bound this is the equivalence.

Equivalence of norms is why finite-dimensional analysis is coordinate-free, since convergence, continuity, and compactness do not depend on which norm measures them. It fails in infinite dimensions, where the supremum and integral norms on function spaces are genuinely different, and that failure is the reason the next posts must choose their norms with care.

The completeness of a metric space is what makes the Lebesgue space $L^p$ a Banach space and the inner product space $L^2$ a Hilbert space, the contraction principle constructs solutions of stochastic differential equations, and compactness is the property that the spectral theory of operators turns into eigenvalues. Distance, completeness, and compactness are the three words the rest of the curriculum is built from.