1108.1022Aug 4, 2011Appears at WITMSE 2011, The Fourth Workshop on Information Theoretic Methods in Science and Engineering, 7-10 August 2011, Helsinki, Finland. Note that the WITMSE version is 4 pages, owing to different formatting
We consider an input x generated by an unknown stationary ergodic source X that enters a signal processing system J, resulting in w=J(x). We observe w
Nearby in the stack
through a noisy channel,
y=z(w)
; our goal is to estimate x from
y
,
J
, and knowledge of
fY∣W
. This is universal estimation, because
fX
is unknown. We provide a formulation that describes a trade-off between information complexity and noise. Initial theoretical, algorithmic, and experimental evidence is presented in support of our approach.