Optimal Feedback Communication with Information Maximization and Distortion Minimization

Aolin Xu

arxiv: 2606.09698 · v1 · pith:WVPLB2LQnew · submitted 2026-06-08 · 💻 cs.IT · cs.SY· eess.SY· math.IT· math.OC

Optimal Feedback Communication with Information Maximization and Distortion Minimization

Aolin Xu This is my paper

Pith reviewed 2026-06-27 14:38 UTC · model grok-4.3

classification 💻 cs.IT cs.SYeess.SYmath.ITmath.OC

keywords feedback communicationposterior matchingmutual information maximizationMMSE distortion minimizationsymmetric channelschannel with feedback

0 comments

The pith

For symmetric channels with feedback, posterior matching is sufficient and essentially necessary to maximize mutual information while minimizing MMSE distortion at each step.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines the task of sending a real-valued source over multiple uses of a channel with feedback. It first gives conditions sufficient for an encoder to achieve maximal mutual information between the source and the sequence of channel outputs, and shows these conditions are also necessary when the channel is input-identifiable. It then considers the joint objective of maximizing that mutual information while simultaneously minimizing the MMSE of estimating the source from the outputs received so far. For discrete channels possessing k-ary symmetric or k-ary erasure symmetry, the work derives that the posterior matching scheme meets both requirements at once.

Core claim

We show that for such channels the famous posterior matching scheme, while not necessary for information maximization alone, is sufficient and essentially necessary for achieving both information maximization and distortion minimization.

What carries the argument

The posterior matching scheme, which at each step selects the channel input so that its distribution matches the current posterior distribution of the source given all previous outputs.

If this is right

Encoders for these symmetric channels can be constructed directly from the posterior matching rule to satisfy the joint objective.
Information maximization serves as a regularizer that renders the otherwise intractable distortion-minimization problem solvable.
The necessity result shows that any encoder achieving the joint optimum must satisfy the posterior-matching input selection rule.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same symmetry-based argument may extend to approximate symmetries in continuous or non-symmetric channels.
Practical systems could adopt posterior matching as a default when both rate and estimation accuracy matter.
Testing the scheme on channels that violate input-identifiability would clarify the boundary of the necessity claim.

Load-bearing premise

The channel must have specific symmetries such as k-ary symmetric or k-ary erasure and must be input-identifiable.

What would settle it

On a k-ary symmetric channel, exhibit an encoding scheme other than posterior matching that achieves strictly higher mutual information or strictly lower MMSE at some time step.

Figures

Figures reproduced from arXiv: 2606.09698 by Aolin Xu.

read the original abstract

We study the problem of optimally sending a real-valued source through multiple uses of a channel with feedback. First, we state a set of conditions that are sufficient for an encoder to achieve maximal mutual information between the source and all the channel outputs. This set of conditions are also necessary when the channel is input-identifiable, a condition widely satisfied by common channel models. More notably, we further study the information maximization-distortion minimization problem, where the mutual information between the source and all channel outputs still needs to be maximized, while at each step, the MMSE of estimating the source from the channel outputs so far also needs to be minimized. We derive a solution to this problem for discrete channels with certain symmetries, e.g. $k$-ary symmetric or $k$-ary erasure channels. We show that for such channels the famous posterior matching scheme, while not necessary for information maximization alone, is sufficient and essentially necessary for achieving both information maximization and distortion minimization. This work also provides a new perspective of regularizing distortion-minimizing feedback communication through information maximization, which enables us to find the optimal solution that otherwise would be intractable.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper gives a joint info-max plus per-step MMSE-min solution for symmetric feedback channels and claims posterior matching is essentially necessary there, but the necessity step needs close checking on how the distortion term is handled.

read the letter

The main point is that this work combines mutual information maximization with per-step MMSE minimization in a feedback setting and derives an explicit solution for k-ary symmetric and erasure channels, where posterior matching turns out to be sufficient and essentially necessary.

What is new is the regularization framing that treats information maximization as a way to make the otherwise intractable distortion problem solvable, along with the necessity claim under the joint objective. The sufficient conditions for pure information maximization are also laid out and shown to be necessary for input-identifiable channels. That organizes two objectives that had been treated separately and supplies a concrete scheme for the symmetric cases.

The soft spot sits in the necessity argument for the combined criterion. The abstract indicates that necessity holds once symmetries and input-identifiability are assumed, yet it is not clear how the MMSE-minimization condition is isolated from the information-maximization condition when proving that other encoders cannot work. The symmetries are invoked for both parts, but their specific role in the distortion step is not separated out, so it remains possible that the necessity result leans more heavily on the information side. Without the full derivations this cannot be confirmed, and the input-identifiability assumption is stated only for the information-max part.

The paper is aimed at people working on feedback communication, posterior matching, and joint rate-estimation problems. Readers who care about explicit schemes on symmetric channels or about regularizing distortion objectives will get the most out of it. It is worth sending for peer review because the claims are specific and the channel models are standard; a referee can check the necessity separation directly.

Referee Report

1 major / 2 minor

Summary. The paper states sufficient conditions for an encoder to maximize mutual information between a real-valued source and channel outputs with feedback; these conditions are also necessary for input-identifiable channels. It then derives the solution to the joint problem of maximizing mutual information while minimizing per-step MMSE distortion for k-ary symmetric and k-ary erasure channels, showing that the posterior matching scheme is sufficient and essentially necessary for achieving both objectives simultaneously on these channels. The work frames information maximization as a regularizer that renders the joint distortion-minimization problem tractable.

Significance. If the derivations hold, the result strengthens the understanding of posterior matching by showing its role extends from information maximization alone to the joint objective on symmetric channels, and supplies a concrete regularization technique for otherwise intractable feedback communication problems. The explicit separation of sufficiency from necessity under input-identifiability is a clear strength.

major comments (1)

[Abstract] Abstract (necessity claim for joint objective): the statement that posterior matching is 'essentially necessary' for both information maximization and per-step MMSE minimization on k-ary symmetric/erasure channels invokes input-identifiability only for the information-maximization component. The manuscript does not isolate how this assumption interacts with the sequential MMSE-minimization requirement in the necessity direction, leaving open whether other encoders could satisfy the combined criteria; this is load-bearing for the central 'essentially necessary' claim.

minor comments (2)

The abstract refers to 'derivations exist' for the sufficient conditions and the symmetric-channel case; explicit pointers to the sections containing the full proofs (or statements of the lemmas used) would improve readability.
Notation for the per-step MMSE estimator and the input-identifiability condition should be introduced with a short definition or reference to prior literature on first use.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful reading and for identifying a point that merits clarification in the abstract. We address the concern below and will make a targeted revision to improve precision without altering the technical claims.

read point-by-point responses

Referee: [Abstract] Abstract (necessity claim for joint objective): the statement that posterior matching is 'essentially necessary' for both information maximization and per-step MMSE minimization on k-ary symmetric/erasure channels invokes input-identifiability only for the information-maximization component. The manuscript does not isolate how this assumption interacts with the sequential MMSE-minimization requirement in the necessity direction, leaving open whether other encoders could satisfy the combined criteria; this is load-bearing for the central 'essentially necessary' claim.

Authors: The referee correctly notes that input-identifiability is invoked for necessity of information maximization. For the k-ary symmetric and erasure channels under consideration, this property holds. The necessity argument for the joint objective proceeds in two steps that are already present in the manuscript: (i) any encoder achieving maximal mutual information must satisfy the posterior-matching conditions (by the necessity result under input-identifiability), and (ii) among all such encoders, only the posterior-matching scheme additionally satisfies the per-step MMSE-minimization requirement at every time. Consequently, the combined criteria are satisfied if and only if the encoder is the posterior-matching scheme. While this logical structure is used in the proofs, the abstract does not explicitly separate the two steps. We will therefore revise the abstract (and the corresponding paragraph in Section IV) to state that input-identifiability is used to characterize the information-maximizing encoders and that the distortion-minimization condition then selects posterior matching from within that class. This revision clarifies the interaction without changing any technical result. revision: partial

Circularity Check

0 steps flagged

No circularity; derivation relies on channel symmetries without self-referential reductions

full rationale

The paper states sufficient conditions for information maximization (necessary under input-identifiability) and then derives the joint information-maximization plus per-step MMSE-minimization solution specifically for k-ary symmetric and erasure channels. Posterior matching is shown sufficient and essentially necessary for the joint objective on these channels. No step reduces a claimed prediction or necessity result to a fitted parameter, self-definition, or self-citation chain by construction. The input-identifiability condition is external to the target joint result and the symmetries are channel properties used to isolate the MMSE condition. The derivation is self-contained against the stated assumptions.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The work rests on standard definitions of mutual information and MMSE together with domain assumptions about channel symmetry and input-identifiability; no free parameters or new entities are introduced in the abstract.

axioms (2)

standard math Standard definitions of mutual information between source and channel outputs and of MMSE estimation error.
Invoked throughout the problem statement and solution.
domain assumption The channel satisfies input-identifiability (widely true for common models) and possesses k-ary symmetry or erasure symmetry.
Required for necessity of the encoder conditions and for the posterior-matching optimality result.

pith-pipeline@v0.9.1-grok · 5733 in / 1355 out tokens · 25380 ms · 2026-06-27T14:38:50.846970+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Minimum Distortion Quantization with Specified Output Distribution
cs.IT 2026-06 unverdicted novelty 6.0

Derives optimal quantizer form X=σ(F^{-1}(F_W(W))) with permutation σ minimizing MMSE under specified output distribution P_X, using majorization.

Reference graph

Works this paper leans on

15 extracted references · cited by 1 Pith paper

[1]

Optimal feedback communication via posterior matching,

O. Shayevitz and M. Feder, “Optimal feedback communication via posterior matching,”IEEE Transactions on Information Theory, vol. 57, no. 3, 2011

2011
[2]

A coding scheme for additive noise channels with feedback–i: No bandwidth constraint,

J. Schalkwijk and T. Kailath, “A coding scheme for additive noise channels with feedback–i: No bandwidth constraint,”IEEE Transactions on Information Theory, vol. 12, no. 2, 1966. 8

1966
[3]

A coding scheme for additive noise channels with feedback–ii: Band-limited signals,

J. Schalkwijk, “A coding scheme for additive noise channels with feedback–ii: Band-limited signals,”IEEE Transactions on Information Theory, vol. 12, no. 2, 1966

1966
[4]

Sequential transmission using noiseless feedback,

M. Horstein, “Sequential transmission using noiseless feedback,”IEEE Transactions on Information Theory, vol. 9, no. 3, 1963

1963
[5]

Source-channel communication with feedback,

M. Gastpar and B. Rimoldi, “Source-channel communication with feedback,” inProceedings 2003 IEEE Infor- mation Theory Workshop, 2003

2003
[6]

Fast iterative coding techniques for feedback channels,

J. Ooi and G. Wornell, “Fast iterative coding techniques for feedback channels,”IEEE Transactions on Infor- mation Theory, vol. 44, no. 7, 1998

1998
[7]

Dynamic joint source-channel coding with feedback,

T. Javidi and A. Goldsmith, “Dynamic joint source-channel coding with feedback,” in2013 IEEE International Symposium on Information Theory, 2013

2013
[8]

Joint source-channel coding with feedback,

V. Kostina, Y. Polyanskiy, and S. Verd, “Joint source-channel coding with feedback,”IEEE Transactions on Information Theory, vol. 63, no. 6, 2017

2017
[9]

Control-theoretic approach to communication with feedback,

E. Ardestanizadeh and M. Franceschetti, “Control-theoretic approach to communication with feedback,”IEEE Transactions on Automatic Control, vol. 57, no. 10, 2012

2012
[10]

The necessity and sufficiency of anytime capacity for stabilization of a linear system over a noisy communication link—part i: Scalar systems,

A. Sahai and S. Mitter, “The necessity and sufficiency of anytime capacity for stabilization of a linear system over a noisy communication link—part i: Scalar systems,”IEEE Transactions on Information Theory, vol. 52, no. 8, Aug. 2006

2006
[11]

An information and control framework for optimizing user-compliant human–computer interfaces,

J. Tantiongloc, D. A. Mesa, R. Ma, S. Kim, C. H. Alzate, J. J. Camacho, V. Manian, and T. P. Coleman, “An information and control framework for optimizing user-compliant human–computer interfaces,”Proceedings of the IEEE, vol. 105, no. 2, 2017

2017
[12]

Polyanskiy and Y

Y. Polyanskiy and Y. Wu,Information Theory: From Coding to Learning. Cambridge University Press, 2024

2024
[13]

Necessary and sufficient conditions for information-maximizing communication with feedback,

A. Xu, “Necessary and sufficient conditions for information-maximizing communication with feedback,”to ap- pear
[14]

An effective characterization of Schur-convex functions with applications,

C. Stepniak, “An effective characterization of Schur-convex functions with applications,”Heldermann Verlag Journal of Convex Analysis, vol. 14, no. 1, 2007

2007
[15]

Minimum distortion quantization with specified output distribution,

A. Xu, “Minimum distortion quantization with specified output distribution,” inInternational Symposium on Information Theory, 2026. 9

2026

[1] [1]

Optimal feedback communication via posterior matching,

O. Shayevitz and M. Feder, “Optimal feedback communication via posterior matching,”IEEE Transactions on Information Theory, vol. 57, no. 3, 2011

2011

[2] [2]

A coding scheme for additive noise channels with feedback–i: No bandwidth constraint,

J. Schalkwijk and T. Kailath, “A coding scheme for additive noise channels with feedback–i: No bandwidth constraint,”IEEE Transactions on Information Theory, vol. 12, no. 2, 1966. 8

1966

[3] [3]

A coding scheme for additive noise channels with feedback–ii: Band-limited signals,

J. Schalkwijk, “A coding scheme for additive noise channels with feedback–ii: Band-limited signals,”IEEE Transactions on Information Theory, vol. 12, no. 2, 1966

1966

[4] [4]

Sequential transmission using noiseless feedback,

M. Horstein, “Sequential transmission using noiseless feedback,”IEEE Transactions on Information Theory, vol. 9, no. 3, 1963

1963

[5] [5]

Source-channel communication with feedback,

M. Gastpar and B. Rimoldi, “Source-channel communication with feedback,” inProceedings 2003 IEEE Infor- mation Theory Workshop, 2003

2003

[6] [6]

Fast iterative coding techniques for feedback channels,

J. Ooi and G. Wornell, “Fast iterative coding techniques for feedback channels,”IEEE Transactions on Infor- mation Theory, vol. 44, no. 7, 1998

1998

[7] [7]

Dynamic joint source-channel coding with feedback,

T. Javidi and A. Goldsmith, “Dynamic joint source-channel coding with feedback,” in2013 IEEE International Symposium on Information Theory, 2013

2013

[8] [8]

Joint source-channel coding with feedback,

V. Kostina, Y. Polyanskiy, and S. Verd, “Joint source-channel coding with feedback,”IEEE Transactions on Information Theory, vol. 63, no. 6, 2017

2017

[9] [9]

Control-theoretic approach to communication with feedback,

E. Ardestanizadeh and M. Franceschetti, “Control-theoretic approach to communication with feedback,”IEEE Transactions on Automatic Control, vol. 57, no. 10, 2012

2012

[10] [10]

The necessity and sufficiency of anytime capacity for stabilization of a linear system over a noisy communication link—part i: Scalar systems,

A. Sahai and S. Mitter, “The necessity and sufficiency of anytime capacity for stabilization of a linear system over a noisy communication link—part i: Scalar systems,”IEEE Transactions on Information Theory, vol. 52, no. 8, Aug. 2006

2006

[11] [11]

An information and control framework for optimizing user-compliant human–computer interfaces,

J. Tantiongloc, D. A. Mesa, R. Ma, S. Kim, C. H. Alzate, J. J. Camacho, V. Manian, and T. P. Coleman, “An information and control framework for optimizing user-compliant human–computer interfaces,”Proceedings of the IEEE, vol. 105, no. 2, 2017

2017

[12] [12]

Polyanskiy and Y

Y. Polyanskiy and Y. Wu,Information Theory: From Coding to Learning. Cambridge University Press, 2024

2024

[13] [13]

Necessary and sufficient conditions for information-maximizing communication with feedback,

A. Xu, “Necessary and sufficient conditions for information-maximizing communication with feedback,”to ap- pear

[14] [14]

An effective characterization of Schur-convex functions with applications,

C. Stepniak, “An effective characterization of Schur-convex functions with applications,”Heldermann Verlag Journal of Convex Analysis, vol. 14, no. 1, 2007

2007

[15] [15]

Minimum distortion quantization with specified output distribution,

A. Xu, “Minimum distortion quantization with specified output distribution,” inInternational Symposium on Information Theory, 2026. 9

2026