Multi-Source Distributed Data Compression Based on Information Bottleneck Principle

In this article, we focus on a generic multiterminal (remote) source coding scenario in which, via a joint design, several intermediate nodes must locally compress their noisy observations from various sets of user / source signals ahead of forwarding them through multiple error-free and rate-limited channels to a (remote) processing unit. Although different local compressors might receive noisy observations from a / several common source signal(s), each local quantizer should also compress noisy observations from its own, i.e., uncommon source signal(s). This, in turn, yields a highly generalized scheme with most flexibility w.r.t. the assignment of users to the serving nodes, compared to the State-of-the-Art techniques designed exclusively for a common source signal. Following the Information Bottleneck (IB) philosophy, we choose the Mutual Information as the fidelity criterion here, and, by taking advantage of the Variational Calculus, we characterize the form of stationary solutions for two different types of processing flow/ strategy. We utilize the derived solutions as the core of our devised algorithmic approach, the GEneralized Multivariate IB (GEMIB), to (efficiently) address the corresponding design problems. We further provide the respective convergence proofs of GEMIB to a stationary point of the pertinent objective functionals and substantiate its effectiveness by means of numerical investigations over a couple of (typical) digital transmission scenarios.