{"id":"e95481d8-a014-457e-ba29-3355f24932a2","arxiv_id":"2507.01038","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A cross-attention transformer decoder, plus ensemble and foundation versions, is reported to outperform belief-propagation and prior neural decoders on several BCH, polar, and LDPC codes.","lead":"This paper proposes a transformer-based decoder that can be trained once and then correct errors for several different channel codes, using cross-attention masks derived from each code's parity-check matrix. The authors report better bit-error rates than belief propagation and previous transformer decoders, and present ensemble and foundation versions aimed at flexible 6G decoding.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The foundation-model claim rests on interpolation, not transfer: every FCrossMPT/FCrossED evaluation code is in its training set, so 'code-agnostic decoding without retraining' is not actually tested.","rationale":"The reader's weakest assumption identifies exactly the same gap: foundation-model generalization is asserted but all evaluated codes are in the training set. This is the most load-bearing issue because the paper's headline contribution is the code-agnostic foundation decoder, not the CrossMPT architecture alone, which is otherwise supported by consistent BER gains over ECCT and BP baselines and by ablation Table I. No internal contradiction in the architecture or training procedure was found; the issue is an unverified empirical claim. The proposed concrete test is a held-out zero-shot evaluation, which directly settles whether 'code-agnostic' means transfer or only multi-task fitting. Because this is an addressable missing experiment rather than a demonstrated failure, the reader's CONDITIONAL verdict is appropriate and unchanged.","tokens_in":17382,"tokens_out":4762,"duration_ms":57232,"concrete_test":"Retrain FCrossMPT under the exact Sec. III-E setup on a subset, e.g., only (63,45) BCH, (49,24) LDPC, and (121,60) LDPC. Then evaluate BER on held-out codes never seen in training: a different rate in the same family, e.g., (121,80) LDPC, a different length/class, e.g., (127,64) BCH or (64,48) polar, and a different length outside the training range, e.g., (128,86) polar. Compare with CrossMPT trained individually on each held-out code at the same Eb/N0 points. If the held-out FCrossMPT BER is within roughly 0.5 dB of the individually trained CrossMPT at BER 1e-4, the foundation claim is supported; otherwise the abstract and conclusion should be softened to multi-code training rather than code-agnostic generalization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the abstract is that a single trained model can decode a broad range of codes without retraining. The experiments do not test this. In Sec. IV-B, FCrossMPT is trained on (121,60), (121,70), and (121,80) LDPC codes and then evaluated on (121,70) and (121,80), with the text explicitly saying 'These codes are included in the training data set.' In the mixed-code experiment, the training set consists of (63,30), (63,45), (49,24), and (121,60) LDPC/BCH codes, and Fig. 9 evaluates exactly those four codes. Sec. IV-D trains FCrossED jointly on (15,7), (31,16), and (63,30) and reports results only for (31,16) and (63,30), both in the training set. Thus every reported 'foundation' result is in-distribution multi-task fitting, not zero-shot transfer to an unseen code, unseen length, or unseen rate. The architecture replaces position-specific embeddings with shared embeddings, which is a plausible mechanism for transfer, but it does not guarantee it: the model can still memorize the small set of PCMs/masks seen in training. Since the abstract and conclusion advertise 'code-agnostic' and 'without retraining,' the missing held-out evaluation is the load-bearing gap in the paper's strongest claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CrossMPT, a transformer-based channel decoder that iteratively updates magnitude and syndrome embeddings through two masked cross-attention blocks, using the parity-check matrix and its transpose as attention masks. It then introduces FCrossMPT, a length/rate/class-invariant variant built from shared embeddings, and CrossED, an ensemble of CrossMPT blocks using complementary parity-check matrices, together with a foundation version FCrossED. Experiments compare BER against ECCT and BP-based neural decoders on BCH, polar, and LDPC codes, and report complexity and memory reductions for CrossMPT over ECCT.","tokens_in":17654,"tokens_out":3579,"duration_ms":38671,"significance":"If validated, the code-agnostic foundation-model claim would be a useful step toward flexible decoders for 6G, and CrossMPT's sparse cross-attention is a principled simplification of ECCT. The ablation in Table I gives evidence that magnitude-syndrome interactions alone are sufficient for the tested codes, and the complexity analysis in Section V-D quantifies a concrete advantage in FLOPs, memory, and training time. However, the paper's headline claim of generalization without retraining is not tested: all foundation-model evaluations use codes from the training set, and the performance comparisons lack error bars. The architectural and complexity contributions are credible, but the foundation-model conclusion currently exceeds the evidence.","major_comments":[{"comment":"The abstract and conclusion claim 'code-agnostic' decoding 'without retraining,' but no experiment evaluates a code, length, or rate outside the training set. In Section IV-B, the (121,70) and (121,80) evaluation codes are explicitly stated to be 'included in the training data set'; the mixed-code experiment trains on all four codes shown in Fig. 9; and Section IV-D trains FCrossED on (15,7), (31,16), and (63,30), then reports results only for (31,16) and (63,30), both in the training set. Every reported foundation-model result is therefore in-distribution multi-task fitting, which cannot distinguish memorization of the small set of parity-check matrices from zero-shot generalization. To support the central claim, the authors should add held-out evaluations on codes, lengths, and rates not seen during training, and should separate a weaker 'multi-task decoding' claim from the stronger 'code-agnostic foundation model' claim.","section":"IV-B, IV-D"},{"comment":"No Monte-Carlo trial counts, error bars, confidence intervals, or multiple-seed results are reported for any BER curve or table entry. For example, in Table I the difference between ECCT+Masking and ECCT at parameter 4 for the (63,45) code is 5.53 vs 5.60; without variance information, the claim of 'no significant performance degradation' and the broader state-of-the-art claim are not statistically grounded. The authors should report the number of Monte-Carlo trials and standard errors or confidence intervals for the main BER comparisons and ablations, or otherwise justify that the differences are beyond training noise.","section":"IV and Table I"}],"minor_comments":[{"comment":"The conclusion contains a duplicated sentence: 'To move toward a code-agnostic foundation model, we developed FCrossMPT by replacing code-dependent parameters with code-invariant embeddings' appears twice in succession.","section":"VI"},{"comment":"The text refers to a theoretical proof in [25] that preprocessing makes performance invariant to the training codeword, but the theorem is not stated; adding a precise statement would help the reader verify the claim.","section":"II-C"},{"comment":"The phrase 'CrossMPT and ECCT are obtained with N=6 and d=128' should be 'configured with' or 'set to'; as written it is slightly misleading.","section":"IV-A"},{"comment":"The 'state-of-the-art among single neural decoders' claim is based on comparisons with ECCT and two BP-based neural decoders on a limited set of codes; the comparison set should be stated explicitly or the claim should be softened.","section":"Abstract and IV-A"}],"recommendation":"major_revision","confidential_remarks":"The foundation-model claim is the paper's headline contribution, and the current experiments cannot support it. If the authors add a genuine held-out-code evaluation and statistical error bars, the paper could be suitable; without those, the central claim should be substantially downgraded. The CrossMPT architecture and complexity analysis are the strongest parts of the work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the strongest claim in the abstract—a single trained model decoding a broad range of codes without retraining—is not tested by the experiments. All FCrossMPT and FCrossED evaluation codes are in the training set. The stress test's main point lands.\n\nWhat is actually new: CrossMPT itself appeared at ICLR 2025; the extensions here are FCrossMPT, CrossED, and FCrossED. They combine two existing ideas—shared embeddings from FECCT and multiple PCM masks from MM-ECCT—with CrossMPT's cross-attention structure. That is incremental but not trivial. The core CrossMPT architecture is well motivated: two masked cross-attention blocks over H and H^T give a sparser attention map, the same parameter count as ECCT, lower FLOPs and memory, and better BER than ECCT and BP-based neural decoders on the tested BCH, polar, and LDPC codes, including a (648,540) LDPC code where ECCT could not run. Table I is a good ablation: masking out magnitude-magnitude and syndrome-syndrome attention costs little, which supports the design. The complexity analysis in Section V-D is careful. CrossED's ensemble with cyclically shifted PCMs gives large BER gains on short BCH codes at no parameter cost and with parallel latency; Fig. 13 shows the mechanism. These are real engineering contributions.\n\nSoft spots, in proportion. First, the missing held-out experiment. Section IV-B explicitly says the evaluated (121,70) and (121,80) codes are in the training set; the mixed-code experiment evaluates exactly the four training codes; Section IV-D trains on (15,7), (31,16), (63,30) and evaluates two of them. So the foundation-model claim rests on in-distribution multi-task fitting, not transfer. That is load-bearing because it is the abstract's headline. Second, there are no error bars, confidence intervals, or Monte-Carlo trial counts anywhere; the BER curves are point estimates. For a stochastic neural decoder that matters. Third, the SOTA claim misses direct comparisons with FECCT and MM-ECCT, the most relevant baselines; FECCT especially should be compared for the foundation experiments. Fourth, no code or data release, which makes replication harder.\n\nOverall, the paper is honest at the experimental level—it says which codes are in the training set—but overreaches in the abstract and conclusion. The CrossMPT result is solid and the extensions are plausible; the gap is addressable. I would send this to peer review. The referees should demand a held-out zero-shot evaluation on unseen codes, lengths, and rates, or a softened claim, plus trial statistics or released artifacts. If the authors can show transfer, this is a useful 6G physical-layer paper; if not, the foundation-model wording should be removed.","headline":"The CrossMPT architecture is a genuine improvement over ECCT, but the code-agnostic foundation-model claim is untested: every FCrossMPT/FCrossED evaluation code is in the training set.","tokens_in":18246,"tokens_out":4264,"would_cite":false,"duration_ms":47678,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["94B05","94B35","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper reports that a masked cross-attention transformer can decode several error-correcting code classes with a single trained model, outperforming belief propagation on the tested codes.","keywords":["code-agnostic decoding","transformer decoder","cross-attention message passing","error correction codes","foundation model","6G networks","belief propagation"],"falsifier":"Train FCrossMPT on one set of codes, for example BCH and LDPC codes of lengths 49, 63, and 121, then test it on a held-out polar code or a BCH code of a length not in training; if the held-out BER at a fixed $E_b/N_0$ is close to random guessing or clearly worse than a code-specific decoder trained for that code, the claim of code-agnostic foundation decoding is refuted.","tokens_in":17172,"feed_emoji":"📡","tokens_out":8075,"duration_ms":79100,"temperature":0.7,"pith_summary":"The paper sets out to show that one transformer architecture can act as a unified, code-agnostic decoder for channel coding in 6G, replacing code-specific decoders that must be redesigned for each code. Its central architecture, CrossMPT, replaces the standard transformer's dense self-attention with two masked cross-attention blocks that pass messages between magnitude and syndrome representations, using the parity-check matrix and its transpose as attention masks. The paper argues this message-passing-like design is why CrossMPT beats belief propagation and prior neural decoders on the tested BCH, polar, and LDPC codes. On top of that, FCrossMPT and FCrossED replace code-dependent parameters with shared embeddings, so the authors claim a single trained model can decode a broad range of codes without retraining. A sympathetic reader would care because if this holds, a single AI decoder could serve the heterogeneous coding scenarios of 6G with lower latency and higher flexibility than per-code hardware.","feed_headline":"One trained cross-attention decoder outperforms belief propagation","feed_subtitle":"A single model covers BCH, polar, and LDPC codes without retraining, heading toward a unified 6G physical layer.","key_machinery":"The central object is the masked cross-attention block pair indexed by the code's parity-check matrix $H$, the matrix whose rows define the parity-check equations. In each CrossMPT layer, the first block uses the magnitude embedding as query and the syndrome embedding as key and value with mask $H^\\top$; the second uses the syndrome embedding as query and the updated magnitude embedding as key and value with mask $H$. Because the masks restrict attention to entries where $H$ or $H^\\top$ has a one, the network only exchanges messages along legitimate variable–check connections, reproducing the information flow of belief propagation. FCrossMPT and FCrossED make this code-agnostic by using one shared embedding vector for all magnitude positions and one for all syndrome positions, and by resizing the syndrome embedding with $H^\\top$ before the output layer; CrossED additionally runs $p$ parallel blocks with complementary parity-check masks generated by cyclically shifting the systematic parity-check matrix.","core_discovery":"CrossMPT's discovery is that decoding can be learned by iteratively alternating two sparse cross-attention operations: magnitude embeddings attend to syndrome embeddings through the mask $H^\\top$, and syndrome embeddings attend to magnitude embeddings through $H$. This mirrors sum-product message passing on the Tanner graph, and the paper shows it is enough to outperform the denser ECCT self-attention and belief propagation on the codes tested, with sparser masks yielding lower FLOPs, faster inference, and much lower memory for long codes. The foundation variants carry the claim further: with position-invariant embeddings, a single model trained jointly on several codes achieves BER comparable to separately trained code-specific CrossMPT and to ECCT on all evaluated codes. The ensemble variant CrossED uses several cyclically shifted parity-check matrices in parallel to fix the weak bit positions left by a single systematic parity-check matrix, giving large BER gains on short blocklength codes without extra parameters or latency.","pith_inferences":["The paper's foundation-model evidence is weaker than the 'code-agnostic' wording suggests: every evaluated code appears in the training set, so the current experiments demonstrate multi-code fitting rather than transfer to unseen parameters; a leave-one-code-out test is the missing experiment.","If genuine transfer holds, one could pre-train a decoder on a small family of codes and then apply it to new 6G code designs without any retraining or weight adaptation, which would change how new physical-layer codes are deployed; the paper does not yet establish this.","The complementary-mask idea is stated for cyclic codes via column shifts; for non-cyclic codes the paper only says partial diagonalization 'as much as possible', leaving a concrete construction and bound on bit coverage as an open extension.","Because attention is restricted to edges of the Tanner graph, scaling to very long codes may need low-rank or sparse attention kernels beyond simple masking; that is an implementation question the paper leaves implicit."],"forward_implications":["If the reported results hold, a single CrossMPT-trained model can replace code-specific decoders across the BCH, polar, and LDPC families tested, cutting the number of decoder implementations in a 6G modem.","CrossMPT's sparser attention means ECCT-sized transformers can scale to longer blocklengths, such as the (648,540) LDPC code, where the original ECCT could not fit in the authors' memory budget.","CrossED's complementary parity-check masks improve short-blocklength BER by over an order of magnitude at the tested SNR points, with no extra inference latency because the parallel blocks run simultaneously.","The foundation variants FCrossMPT and FCrossED imply that one set of weights can be trained jointly on codes of different class, length, and rate, a necessary step toward an AI-native physical layer for 6G."],"supporting_citations":[{"why":"supplies the ECCT baseline and the masked self-attention decoding method that CrossMPT is compared against and modifies.","marker":"[26]"},{"why":"provides the syndrome-based preprocessing that lets model-free neural decoders train on the all-zero codeword, which both ECCT and CrossMPT rely on.","marker":"[25]"},{"why":"introduces the position-invariant shared-embedding principle for a foundation error-correction transformer that FCrossMPT adopts.","marker":"[29]"},{"why":"introduces multiple complementary parity-check matrices and the identity-coverage effect that CrossED's ensemble masks build on.","marker":"[28]"},{"why":"defines belief propagation and sum-product message passing on the Tanner graph that CrossMPT's cross-attention blocks are designed to emulate.","marker":"[11]"},{"why":"supplies the hyper-graph-network BP decoder baseline used in the BCH and polar BER comparisons.","marker":"[17]"},{"why":"supplies the autoregressive belief propagation decoder baseline used in the BCH and polar BER comparisons.","marker":"[18]"},{"why":"provides the parity-check matrices used for the simulated BCH, polar, and LDPC codes.","marker":"[35]"}],"fun_headline_variants":["One transformer decoder beats belief propagation across codes","Cross-attention decoder unifies BCH, polar, LDPC without retraining","Code-agnostic 6G decoder: single model, multiple codes, no retraining","AI decoder handles any code length, rate, and class in 6G","CrossMPT: sparse cross-attention matches sum-product decoding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The foundation-model claim rests on the assumption that training on a small fixed set of codes produces a decoder that works for codes, lengths, and rates outside that set; the experiments only evaluate codes that were included in the training data.","fun_headline_variants_meta":{"raw":{"variants":["One transformer decoder beats belief propagation across codes","Cross-attention decoder unifies BCH, polar, LDPC without retraining","Code-agnostic 6G decoder: single model, multiple codes, no retraining","AI decoder handles any code length, rate, and class in 6G","CrossMPT: sparse cross-attention matches sum-product decoding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001081,"raw_usage":{"total_tokens":4546,"prompt_tokens":992,"completion_tokens":3554,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":608,"completion_tokens_details":{"reasoning_tokens":3458}},"tokens_in":608,"tokens_out":3554,"duration_ms":27220,"temperature":1.0,"reasoning_tokens":3458,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:23:41.032854+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train FCrossMPT on one set of codes, for example BCH and LDPC codes of lengths 49, 63, and 121, then test it on a held-out polar code or a BCH code of a length not in training; if the held-out BER at a fixed $E_b/N_0$ is close to random guessing or clearly worse than a code-specific decoder trained for that code, the claim of code-agnostic foundation decoding is refuted.","supporting_citations":[{"cited_title":"Error correction code transformer,","cited_arxiv_id":null,"evidence_quote":"supplies the ECCT baseline and the masked self-attention decoding method that CrossMPT is compared against and modifies."},{"cited_title":"Deep learning for decoding of linear codes - A syndrome-based approach,","cited_arxiv_id":null,"evidence_quote":"provides the syndrome-based preprocessing that lets model-free neural decoders train on the all-zero codeword, which both ECCT and CrossMPT rely on."},{"cited_title":"A foundation model for error correction codes,","cited_arxiv_id":null,"evidence_quote":"introduces the position-invariant shared-embedding principle for a foundation error-correction transformer that FCrossMPT adopts."},{"cited_title":"Multiple- masks error correction code transformer for short block codes,","cited_arxiv_id":null,"evidence_quote":"introduces multiple complementary parity-check matrices and the identity-coverage effect that CrossED's ensemble masks build on."},{"cited_title":"The capacity of low-density parity check codes under message-passing decoding,","cited_arxiv_id":null,"evidence_quote":"defines belief propagation and sum-product message passing on the Tanner graph that CrossMPT's cross-attention blocks are designed to emulate."},{"cited_title":"Hyper-graph-network decoders for block codes,","cited_arxiv_id":null,"evidence_quote":"supplies the hyper-graph-network BP decoder baseline used in the BCH and polar BER comparisons."},{"cited_title":"Autoregressive Belief Propagation for Decoding Block Codes","cited_arxiv_id":"2103.11780","evidence_quote":"supplies the autoregressive belief propagation decoder baseline used in the BCH and polar BER comparisons."},{"cited_title":"Database of channel codes and ML simulation results,","cited_arxiv_id":null,"evidence_quote":"provides the parity-check matrices used for the simulated BCH, polar, and LDPC codes."}],"review_version":1}