Post on 08-Jul-2018
8/19/2019 Scs Mit Feb18
1/38
Marco F. Duarte
Spectral Compressive Sensing
Portions are joint work with:
Richard G. Baraniuk
(Rice University)
Hamid Dadkhahi
(UMass Amherst)
Karsten Fyhn
(Aalborg University)
8/19/2019 Scs Mit Feb18
2/38
Spectral Compressive Sensing
• Compressive sensing applied to frequency-sparse signals
linearmeasurements
frequency-sparsesignal
Fourier components
[E. Candès, J. Romberg, T. Tao; D. Donoho]
Φ
8/19/2019 Scs Mit Feb18
3/38
Ψ
Spectral Compressive Sensing
• Compressive sensing applied to frequency-sparse signals
linearmeasurements
frequency-sparsesignal
nonzero
DFT coefficients
Φ
DFT Basis forfrequency-sparse signals
8/19/2019 Scs Mit Feb18
4/38
e(f ) = 1√
N
hej2πf/N ej2π2f/N . . . ej2π(N −1)f/N
i
Frequency-Sparse Signalsand the DFT Basis
8/19/2019 Scs Mit Feb18
5/38
0 10 20 30 40 500
0.2
0.4
0.6
0.8
1
Approximation sparsity K
N o r m a l i z e d a p p
r o x . e r r o r
Integral frequencies
Arbitrary frequencies
Signal is sum of 10 sinusoids
Frequency-Sparse Signalsand the DFT Basis
8/19/2019 Scs Mit Feb18
6/38
Compressive Sensing forFrequency-Sparse Signals
100 200 300 400 500
0
10
20
30
40
50
60
Number of measurements M
A v e r a g e S N R ,
d B
Root MUSIC on M signal samples
Standard IHT via DFT (Best)
8/19/2019 Scs Mit Feb18
7/38
Compressive Sensing forFrequency-Sparse Signals
100 200 300 400 500
0
10
20
30
40
50
60
Number of measurements M
A v e r a g e S N R ,
d B
Root MUSIC on M signal samples
Standard IHT via DFT (Average)
Standard IHT via DFT (Best/Worst)
8/19/2019 Scs Mit Feb18
8/38
The Redundant DFT Frame
8/19/2019 Scs Mit Feb18
9/38
The Redundant DFT Frame
, c = 10
8/19/2019 Scs Mit Feb18
10/38
The Redundant DFT Frame
, c = 10
8/19/2019 Scs Mit Feb18
11/38
The Redundant DFT Frame
, c = 10
Recovery
algorithmsoperate similarlyto “matchedfiltering”:
Dirichlet Kernel
8/19/2019 Scs Mit Feb18
12/38
The Redundant DFT Frame
[Candès, Needell, Eldar, Randall 2011]Sparse approximation
algorithms fail
, c = 10
8/19/2019 Scs Mit Feb18
13/38
Sparse Approximation ofFrequency-Sparse Signals
Signal is sum of 10 sinusoids at arbitrary frequencies
0 10 20 30 40 500
1
2
3
Approximation sparsity K
N o r
m a l i z e d a p p r o x . e r r o r
Standard sparse approx. via DFT Basis
Standard sparse approx. via DFT Frame
8/19/2019 Scs Mit Feb18
14/38
Structured Sparse Signals
• A K -sparse signal lives onthe collection of K -dimsubspaces aligned with
coordinate axes
RN
ΣK
• A K -structured sparsesignal lives on a particular
(reduced) collection of
K -dimensional canonicalsubspaces
ΩK ⊆ ΣK
RN
[Baraniuk, Cevher, Duarte, Hegde 2010]
8/19/2019 Scs Mit Feb18
15/38
• Preserve the structure only between sparse signals thatfollow the structure model
RM
Φ
Φx1
Φx2mK K -dim planes
Ω
Structured RestrictedIsometry Property (SRIP)
RN
• Random (iid Gaussian, Rademacher) matrix has the SRIP
with high probability if
[Blumensath, Davies; Lu, Do]
8/19/2019 Scs Mit Feb18
16/38
Many state-of-the-art sparse recovery algorithms(greedy and optimization solvers) rely on
thresholding
RN
ΣK
Leveraging Structure in Recovery
x
[Daubechies, Defrise, and DeMol;Nowak, Figueiredo, and Wright;Tropp and Needell; Blumensath and Davies...]
Thresholding provides the
best approximation of
x within ΣK
8/19/2019 Scs Mit Feb18
17/38
• Modify existing approaches (optimization or greedy-based)to obtain structure-aware recovery algorithms:
replace the thresholding step in IHT, CoSaMP, SP, ... with a
best structured sparse approximation step
that finds the closest point within union of subspaces
RN
x
Ω
Greedy structure-aware recoveryalgorithms inherit guarantees of generic counterparts(even though feasible set may benonconvex)
Structured Recovery Algorithms
8/19/2019 Scs Mit Feb18
18/38
• If x is K -structured frequency-sparse, then there exists a
K -sparse vector such that and the nonzeros
in are spaced apart from each other (band exclusion).
Structured Frequency-Sparse Signals
• A K -structured frequency-sparse signal x consists of K sinusoids that are mutually
incoherent:
R
N
if
8/19/2019 Scs Mit Feb18
19/38
• If x is K -structured frequency-sparse, then there exists a
K -sparse vector such that and the nonzerosin are spaced apart from each other.
Structured Frequency-Sparse Signals
• Preserve the structure only between sparse signals that
follow the structured sparsity model
• Random (iid Gaussian, Bernoulli) matrix has the
structured RIP with high probability if
8/19/2019 Scs Mit Feb18
20/38
Structured Sparse Approximation
ΩK ⊆ ΣK
R
N
Inputs:• Signal vector x• Target sparsity K
• Redundancy factor c• Maximum coherence
Output:• Approximation vector
• Compute coefficients: • Solve support:
• Mask coefficients:
• Return[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 1 1 1 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
Algorithm 1:
Integer Program
8/19/2019 Scs Mit Feb18
21/38
Theorem:Assume we obtain noisy CS measurements of asignal . If has the structured RIPwith , then the output of the structured
IHT algorithm obeys
CS recovery
error
signal K -term
structured sparse approximation error
noise
Recovery with Structured Sparsity
In words, instance optimality based onstructured sparse approximation
[Baraniuk, Cevher, Duarte, Hegde 2010]
8/19/2019 Scs Mit Feb18
22/38
• Compute coefficients:
• Initialize:
• While is nonzero and• Find max abs entry of
• Copy entry
• Inhibit “coherent” entries
• Return
Inputs:• Signal vector x• Target sparsity K
• Redundancy factor c• Maximum coherence
ΩK ⊆ ΣK
R
N
Algorithm 2:
Inhibition Heuristic
Output:• Approximation vector
Structured Sparse Approximation
8/19/2019 Scs Mit Feb18
23/38
DFT Frame + Thresholdingequivalent toMaximum Likelihood Estimate
of amplitudes and frequenciesfor frequency-sparse signalvia Periodogram
Widely-studied problem:Line spectral estimation
amplitudesfrequencies
Structured Sparse Approximation
8/19/2019 Scs Mit Feb18
24/38
Inputs:• Signal vector x • Target sparsity K
Algorithm 3:Line Spectral Estimation
Output:
• Parameter estimates
• Signal estimate
MUSICRoot MUSIC
ESPRITPHD...
x
K
Structured Sparse Approximation
8/19/2019 Scs Mit Feb18
25/38
Sparse Approximation ofFrequency-Sparse Signals
Signal is sum of 10 sinusoids at arbitrary frequencies
0 10 20 30 40 500
1
2
3
Approximation sparsity K
N o r m a l i z e d a p p r o x . e r r o r
Standard sparse approx. via DFT Basis
Standard sparse approx. via DFT Frame
8/19/2019 Scs Mit Feb18
26/38
Sparse Approximation ofFrequency-Sparse Signals
Signal is sum of 10 sinusoids at arbitrary frequencies
0 10 20 30 40 500
1
2
3
Approximation sparsity K
N o r m a l i z e d a p p r o x . e r r o r
Standard sparse approx. via DFT BasisStandard sparse approx. via DFT FrameStructured sparse approx. via Alg. 1Structured sparse approx. via Alg. 2Structured sparse approx. via Alg. 3
8/19/2019 Scs Mit Feb18
27/38
100 200 300 400 500
0
10
20
30
40
50
60
Number of measurements M
A v e r a g e S N R ,
d B
Root MUSIC on M signal samples
Standard IHT via DFT (Average)
Standard IHT via DFT (Best/Worst)
Structured CS: Performance
8/19/2019 Scs Mit Feb18
28/38
Structured CS: Performance
100 200 300 400 500
0
10
20
30
40
50
60
Number of measurements M
A v e r a g e S N R ,
d B
SIHT via Alg. 1
SIHT via Alg. 2
SIHT via Root MUSIC
Root MUSIC on M signal samples
Standard IHT via DFT (Average)
Standard IHT via DFT (Best/Worst)
8/19/2019 Scs Mit Feb18
29/38
From Recovery of Sparse SignalsTo Line Spectral Estimation
• Can “read” indices of nonzero DFTF coefficients to obtain
frequencies of frequency-sparse signal components
• Equivalence: accurate recovery = accurate estimation?
• Algorithms: Alg. 3 essentially combines legacy line
spectral estimation with CS recovery algorithms
100 200 300 400 5000
100
200
300
400
500
Number of measurements M
M e a n F r e q . E s t . E r r o r , H z
Structured sparsity
Multitaper
Root MUSIC
Standard Sparsity • How to change
signal model
to furtherimprove
performance?
8/19/2019 Scs Mit Feb18
30/38
Interpolating the Projections(Dirichlet Kernel)
•Main lobe of Dirichletkernel can be well
approximated by a
quadratic polynomial
(parabola)
•Three samples around peak are
required for
interpolation
8/19/2019 Scs Mit Feb18
31/38
From Discrete to Continuous Models• Both the DFT basis and the DFT frame can be conceived
as samplings from an infinite set of signals e(f ) for a discrete set of values for the frequency
• Since the signal vector e(f ) varies smoothly in each entry
as a function of f , we can represent the signal set as aone-dimensional nonlinear manifold:
e(0)
e(1)e(2)
e(3) ...
f 0 N
Parameter space
e(f ) = 1√
N
hej2πf/N ej2π2f/N . . . ej2π(N −1)f/N
i
8/19/2019 Scs Mit Feb18
32/38
From Discrete to Continuous Models• For computational reasons, we wish to design methods
that allow us to interpolate the manifold from thesamples obtained in the DFT basis/frame to increase the
resolution of the frequency estimates.
• An interpolation-based compressive line spectral
estimation algorithm obtains projection values for sets of
manifold samples and interpolates around peak on therest of the manifold to get frequency estimate
e(0)
e(1)e(2)
e(3) ...
f 0 N
Parameter space
8/19/2019 Scs Mit Feb18
33/38
•All points in manifold haveequal norm; distance b/w
samples is uniform
•Manifold must be containedwithin unit Euclidean ball
(hypersphere)•Project signal estimates
into hypersphere
•Find closest point inmanifold by interpolating
from closest samples withpolar coordinates
• Integrate band exclusion toget Band-Excluding
Interpolating SP (BISP)
e(0)e(1)e(2)
e(3) ...
Interpolating the Manifold:Polar Interpolation
8/19/2019 Scs Mit Feb18
34/38
• In BISP, find closest pointin manifold by interpolating
from closest samples with
polar coordinates:
•Map back from manifold tofrequency estimates
( parameter space)
Interpolating the Manifold:Polar Interpolation
e(f 0-1/c)
e(f 0+1/c)
e(f 0)
f
0 N Akin to Continuous Basis Pursuit (CBP)
[Ekanadham, Tranchina, and Simoncelli 2011]
8/19/2019 Scs Mit Feb18
35/38
Compressive Line Spectral Estimation:Performance Evaluation
20 40 60 80 10010−7
10−4
10−1
102
Number of measurements (M) A v e r a
g e c o s t i n f r e q
u e n c y e s t i m a t
i o n
SIHT
SDP
BOMP
CBP
BISP
`1-analysis
N = 100, K = 4,c = 5, = 0.2 Hz
BOMP [Fannjiang and Liao 2012]
SDP: Atomic Norm Minimization
[Tang, Rhaskar, Shah, Recht 2012]
Number of measurements (M)
8/19/2019 Scs Mit Feb18
36/38
Compressive Line Spectral Estimation:Performance Evaluation (Noise)
0 5 10 15 2010−2
10−1
100
101
102
103
SNR [dB] A v e r a g
e c o s t i n f r e q u
e n c y e s t i m a t i o n
`1-analysis
`1-synthesis
SIHT
SDP
BOMP
CBP
BISP
`1-analysis
N = 100, K = 4, M = 50, c = 5, = 0.2 Hz
8/19/2019 Scs Mit Feb18
37/38
Compressive Line Spectral Estimation:Computational Expense
SIHT 0.2628 0.1499SDP 8.2355 9.9796
BOMP 0.0141 0.0101
CBP 46.9645 40.3477
BISP 5.4265 1.4060
Noiseless Noisy
`1-analysis 9.5245 8.8222
Time (seconds)
8/19/2019 Scs Mit Feb18
38/38
Conclusions• Spectral CS provides significant improvements on
frequency-sparse signal recovery– address coherent dictionaries via structured sparsity
– simple-to-implement modifications to recovery algs
– can leverage decades of work on spectral estimation
– robust to model mismatch, presence of noise• Compressive line spectral estimation:
– recovery via parametric dictionaries providescompressive parameter estimation
– dictionary elements as samples from manifold models
– from dictionaries to manifolds via interpolation techniques
– from recovery to parameter estimation from compressivemeasurements
– localization, bearing estimation, radar imaging, ...
http://www.ecs.umass.edu/~mduarte mduarte@ecs.umass.edu