Highly Nonlinear Approximations for Sparse Signal Representation

• Project Overview
• Tutorial [.ps | .pdf]
• Source code
• Examples
• Publications
• EIFS
• SCEIFS
• Acknowledgements

• Home
• maths@Aston
• EPSRC Home

Construction of Oblique Projectors for signal splitting

Oblique projectors in the context of signal reconstruction were introduced in [2] and further analyzed in [3]. The application to signal splitting, also termed structured noise filtering, amongst a number of other applications, is discussed in [4]. For advanced theoretical studies of oblique projector operators in infinite dimensional spaces see [5,6]. We restrict our consideration to numerical constructions in finite dimension, with the aim of addressing the problem of signal splitting when the problem is ill posed.

Given ${\cal{V}}$ and ${\cal{W}^\bot}$ disjoint, i.e., such that ${\cal{V}}\cap {\cal{W}^\bot}= \{0\},$ in order to provide a prescription for constructing $\hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ we proceed as follows. Firstly we define ${\cal{S}}$ as the direct sum of ${\cal{V}}$ and ${\cal{W}^\bot}$, which we express as

$${\cal{S}}= {\cal{V}}\oplus {\cal{W}^\bot}.$$

Let ${\cal{W}}= ({\cal{W}^\bot})^\bot$ be the orthogonal complement of ${\cal{W}^\bot}$ in ${\cal{S}}$. Thus we have

$${\cal{S}}= {\cal{V}}\oplus {\cal{W}^\bot}={\cal{W}}\oplus^\bot {\cal{W}^\bot}.$$

The operations $\oplus$ and $\oplus^\bot$ are termed direct and orthogonal sum, respectively.

Considering that $\{v_i\}_{i=1}^M$ is a spanning set for ${\cal{V}}$ a spanning set for ${\cal{W}}$ is obtained as

$$u_i=v_i - \hat{P}_{{\cal{W}^\bot}} v_i= \hat{P}_{{\cal{W}}} v_i,\; \;i=1,\ldots,M.$$

Denoting as $\{{\mathbf{{e}}}_i\}_{i=1}^M$ the standard orthonormal basis in $\mathbb{F}^M$, i.e., the Euclidean inner product $\langle {\mathbf{{e}}}_i, {\mathbf{{e}}}_j\rangle =\delta_{ij}$, we define the operators $\hat{V}: \mathbb{F}^M \to {\cal{V}}$ and $\hat{U}:\mathbb{F}^M \to {\cal{W}}$ as

$$\hat{V}=\sum_{i=1}^M v_i \langle {\mathbf{{e}}}_i, \cdot \rangle , \;\;\;\;\; \hat{U}=\sum_{i=1}^M u_i \langle {\mathbf{{e}}}_i, \cdot \rangle .$$

Thus the adjoint operators $\hat{U}^\ast$ and $\hat{V} ^\ast$ are

$$\hat{V}^\ast=\sum_{i=1}^M {\mathbf{{e}}}_i\langle v_i , \cdot \ra... ...\;\;\; \hat{U}^\ast=\sum_{i=1}^M {\mathbf{{e}}}_i\langle u_i , \cdot \rangle .$$

It follows that $\hat{P}_{{\cal{W}}} \hat{V}= \hat{U}$ and $\hat{U}^\ast \hat{P}_{{\cal{W}}} = \hat{U}^\ast$ hence, $\hat{G}: \mathbb{C}^M \to \mathbb{C}^M$ defined as:

$$\hat{G}=\hat{U}^\ast \hat{V}= \hat{U}^\ast \hat{U}$$

is self-adjoint and its matrix representation, $G$, has elements

$$g_{i,j}=\langle u_i,v_j\rangle = \langle \hat{P}_{{\cal{W}}} u_i,... ... \hat{P}_{{\cal{W}}} u_j\rangle = \langle u_i,u_j\rangle , \;\;i,j=1,\dots,M.$$

From now on we restrict our signal space to be ${\cal{S}}$, since we would like to build the oblique projector $\hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ onto ${\cal{V}}$ and along ${\cal{W}^\bot}$ having the form

$$\hat{E}_{{\cal{V}}{\cal{W}^\bot}}=\sum_{i=1}^M v_i \langle w_i , \cdot \rangle .$$ (3)

Clearly for the operator to map to zero every vector in ${\cal{W}^\bot}$ the dual vectors $\{w_i\}_{i=1}^M$ must span ${\cal{W}}=({\cal{W}^\bot})^\bot={\mbox{\rm {span}}}\{u_i\}_{i=1}^M$. This entails that for each $w_i$ there exists a set of coefficients $\{b_{i,j}\}_{j=1}^M$ such that

$$w_i= \sum_{i=1}^M b_{i,j} u_i,$$ (4)

which guarantees that every $w_i$ is orthogonal to all vectors in ${\cal{W}^\bot}$ and therefore ${\cal{W}^\bot}$ is included in the null space of $\hat{E}_{{\cal{V}}{\cal{W}^\bot}}$. Moreover, since every signal, $g$ say, in ${\cal{S}}$ can be written as $g=g_{{\cal{V}}} + g_{{\cal{W}^\bot}}$ with $g_{{\cal{V}}} \in {{\cal{V}}}$ and $g_{{\cal{W}^\bot}} \in {\cal{W}^\bot}$, the fact that $\hat{E}_{{\cal{V}}{\cal{W}^\bot}}g = 0$ implies $g_{{\cal{V}}}=0$. Hence, $g=g_{{\cal{W}^\bot}}$, which implies that the null space of $\hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ restricted to ${\cal{S}}$ is ${\cal{W}^\bot}$.

In order for $\hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ to be a projector it is necessary that $\hat{E}_{{\cal{V}}{\cal{W}^\bot}}^2=\hat{E}_{{\cal{V}}{\cal{W}^\bot}}$. As will be shown in the next proposition, if the coefficients $b_{i,j}$ are the matrix elements of a generalised inverse of the matrix $G$ this condition is fulfilled.

Proposition 1   If the coefficients $b_{i,j}$ in (4) are the matrix elements of a generalised inverse of the matrix $G$, which has elements $g_{i,j}= \langle v_i,u_j\rangle , i,j=1,\ldots,M$, the operator in (3) is a projector.

Proof. For the measurement vectors in (4) to yield a projector of the form (3), the corresponding operator should be idempotent, i.e.,

$$\sum_{n=1}^M \sum_{m=1}^M \sum_{i=1}^M\sum_{j=1}^M v_i b_{i,j}\... ... =\sum_{i=1}^M\sum_{j=1}^M v_i \overline {b_{i,j}} \langle u_j , \cdot\rangle .$$ (5)

Defining

$$\hat{B} =\sum_{i=1}^M\sum_{j=1}^M {\mathbf{{e}}}_i\overline {b_{i,j}} \langle \mathbf{e}_j , \cdot \rangle$$ (6)

and using the operators $\hat{V}$ and $\hat{U}^\ast$, as given above, the left hand side in (5) can be expressed as

$$\hat{V} \hat{B}^\ast\hat{U}^\ast \hat{V} \hat{B}^\ast \hat{U}^\ast$$ (7)

and the right hand side as

$$\hat{V} \hat{B}^\ast\hat{U}^\ast.$$ (8)

Assuming that $\hat{B}^\ast$ is a generalised inverse of $(\hat{U}^\ast \hat{V})$ indicated as $\hat{B}^\ast = (\hat{U}\hat{V})^\dagger$ it satisfies (c.f. Section )

$$\hat{B}^\ast(\hat{U}^\ast \hat{V})\hat{B}^\ast=\hat{B}^\ast,$$ (9)

and therefore, from (7), the right hand side of (5) follows. Since $\hat{B}^\ast = (\hat{U}^\ast\hat{V})^\dagger= \hat{G}^\dagger$ and $\hat{G}^\ast= \hat{G}$, we have $\hat{B}= \hat{G}^\dagger$. Hence, if the elements $b_{i,j}$ determining $\hat{B}$ in (6) are the matrix elements of a generalised inverse of the matrix representation of $\hat{G}$, the corresponding vectors $\{w_i\}_{i=1}^n$ obtained by (4) yield an operator of the form (3), which is an oblique projector. $\qedsymbol$

Property 1   Let $\hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ be the oblique projector onto ${\cal{V}}$ and along ${\cal{W}^\bot}$ and $\hat{P}_{{\cal{W}}}$ the orthogonal projector onto ${\cal{W}}= ({\cal{W}^\bot})^\bot$. Then the following relation holds

$$\hat{P}_{{\cal{W}}} \hat{E}_{{\cal{V}}{\cal{W}^\bot}}= \hat{P}_{{\cal{W}}}.$$

Proof. $\hat{E}_{{\cal{V}}{\cal{W}^\bot}}$ given in (3) can be recast, in terms of operator $\hat{V}$ and $\hat{U}^\ast$, as:

$$\hat{E}_{{\cal{V}}{\cal{W}^\bot}}= \hat{V} (\hat{U}^\ast \hat{V})^{\dagger} \hat{U}^\ast.$$

Applying $\hat{P}_{{\cal{W}}}$ both sides of the equation we obtain:

$$\hat{P}_{{\cal{W}}} \hat{E}_{{\cal{V}}{\cal{W}^\bot}}= \hat{P}_{{... ...\dagger} \hat{U}^\ast= \hat{U} (\hat{U}^\ast \hat{U})^{\dagger} \hat{U}^\ast,$$

which is a well known form for the orthogonal projector onto ${\cal{R}}(\hat{U})= {\cal{W}}$. $\qedsymbol$

Remark 2   Notice that the operative steps for constructing an oblique projector are equivalent to those for constructing an orthogonal one. The difference being that in general the spaces ${\mbox{\rm {span}}}\{v_i\}_{i=1}^M ={\cal{V}}\quad \text{and}\quad {\mbox{\rm {span}}}\{w_i\}_{i=1}^M= {\cal{W}}$ are different. For the special case $u_i=v_i$, $i=1,\ldots,M$, both sets of vectors span ${\cal{V}}$ and we have an orthogonal projector onto ${\cal{V}}$ along ${\cal{V}}^\bot$.

This project is supported by EPSRC (EP/D062632/1)