Nullspace property

In compressed sensing, the nullspace property gives necessary and sufficient conditions on the reconstruction of sparse signals using the techniques of ${\displaystyle \ell _{1}}$-relaxation. The term "nullspace property" originates from Cohen, Dahmen, and DeVore.^[1] The nullspace property is often difficult to check in practice, and the restricted isometry property is a more modern condition in the field of compressed sensing.

The non-convex ${\displaystyle \ell _{0}}$-minimization problem,

${\displaystyle \min \limits _{x}\|x\|_{0}}$ subject to ${\displaystyle Ax=b}$,

is a standard problem in compressed sensing. However, ${\displaystyle \ell _{0}}$-minimization is known to be NP-hard in general.^[2] As such, the technique of ${\displaystyle \ell _{1}}$-relaxation is sometimes employed to circumvent the difficulties of signal reconstruction using the ${\displaystyle \ell _{0}}$-norm. In ${\displaystyle \ell _{1}}$-relaxation, the ${\displaystyle \ell _{1}}$ problem,

${\displaystyle \min \limits _{x}\|x\|_{1}}$ subject to ${\displaystyle Ax=b}$,

is solved in place of the ${\displaystyle \ell _{0}}$ problem. Note that this relaxation is convex and hence amenable to the standard techniques of linear programming - a computationally desirable feature. Naturally we wish to know when ${\displaystyle \ell _{1}}$-relaxation will give the same answer as the ${\displaystyle \ell _{0}}$ problem. The nullspace property is one way to guarantee agreement.

An ${\displaystyle m\times n}$ complex matrix ${\displaystyle A}$ has the nullspace property of order ${\displaystyle s}$, if for all index sets ${\displaystyle S}$ with ${\displaystyle s=|S|\leq n}$ we have that: ${\displaystyle \|\eta _{S}\|_{1}<\|\eta _{S^{C}}\|_{1}}$ for all ${\displaystyle \eta \in \ker {A}\setminus \left\{0\right\}}$.

The following theorem gives necessary and sufficient condition on the recoverability of a given ${\displaystyle s}$-sparse vector in ${\displaystyle \mathbb {C} ^{n}}$. The proof of the theorem is a standard one, and the proof supplied here is summarized from Holger Rauhut.^[3]

${\displaystyle {\textbf {Theorem:}}}$ Let ${\displaystyle A}$ be a ${\displaystyle m\times n}$ complex matrix. Then every ${\displaystyle s}$-sparse signal ${\displaystyle x\in \mathbb {C} ^{n}}$ is the unique solution to the ${\displaystyle \ell _{1}}$-relaxation problem with ${\displaystyle b=Ax}$ if and only if ${\displaystyle A}$ satisfies the nullspace property with order ${\displaystyle s}$.

${\displaystyle {\textit {Proof:}}}$ For the forwards direction notice that ${\displaystyle \eta _{S}}$ and ${\displaystyle -\eta _{S^{C}}}$ are distinct vectors with ${\displaystyle A(-\eta _{S^{C}})=A(\eta _{S})}$ by the linearity of ${\displaystyle A}$, and hence by uniqueness we must have ${\displaystyle \|\eta _{S}\|_{1}<\|\eta _{S^{C}}\|_{1}}$ as desired. For the backwards direction, let ${\displaystyle x}$ be ${\displaystyle s}$-sparse and ${\displaystyle z}$ another (not necessary ${\displaystyle s}$-sparse) vector such that ${\displaystyle z\neq x}$ and ${\displaystyle Az=Ax}$. Define the (non-zero) vector ${\displaystyle \eta =x-z}$ and notice that it lies in the nullspace of ${\displaystyle A}$. Call ${\displaystyle S}$ the support of ${\displaystyle x}$, and then the result follows from an elementary application of the triangle inequality: ${\displaystyle \|x\|_{1}\leq \|x-z_{S}\|_{1}+\|z_{S}\|_{1}=\|\eta _{S}\|_{1}+\|z_{S}\|_{1}<\|\eta _{S^{C}}\|_{1}+\|z_{S}\|_{1}=\|-z_{S^{C}}\|_{1}+\|z_{S}\|_{1}=\|z\|_{1}}$, establishing the minimality of ${\displaystyle x}$. ${\displaystyle \square }$