Table of Contents

Short Summary: Norm refers to the magnitude of a vector. It commutes with scaling, obeys a form of the triangle inequality, and is zero only at the origin. Since the definition of the magnitude can vary, there are many different norms.

L0 Norm

The L0 Norm is the number of non-zero elements in a vector. L0 Norm is not a true norm. It satisfies only two of the three necessary properties of a norm: that it is zero only for the all-zero vector, and the triangle inequality. It is not, however, positively homogeneous. And unlike all true norms, it is not convex. Calling it a "norm" sews confusion. L0 Norm is the cardinality function. The L0 norm is popular in compressive sensing which tries to find the sparsest solution to an underdetermined set of equations:

\begin{equation} \lVert X \rVert_0 = \lvert\{x \in X \mid x \neq 0\}\rvert \end{equation}

L1 Norm

The L1 Norm, also known as Manhattan Distance or Taxicab norm, is the sum of the magnitudes of the vectors in a space:

\begin{equation} \lVert X \rVert_1 = \sum_{x \in X} \text{abs}(x) \end{equation}

L2 Norm

The L2 Norm, also known as Euclidean norm, represents the shortest distance to the center:

\begin{equation} \lVert X \rVert_2 = \sqrt{\sum_{x \in X} x^2} \end{equation}

L-Infinity Norm

The L-Infinity Norm is the element of the greatest magnitude:

\begin{equation} \lVert X \rVert_\infty = \max(\text{abs}(X)) \end{equation}