{"id":"bb0b7b63-f8f2-4646-b86f-2e2f53ea0348","arxiv_id":"2502.04535","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"SeqMAP, a beam-search decoding method that marginalizes over Directed Acyclic Transformer paths, beats PathMAP and CTC baselines for length-controlled summarization on Gigaword and DUC2004.","lead":"This paper proposes a new decoding algorithm for a type of non-autoregressive transformer that writes summaries with a strict word limit. It improves summary quality while guaranteeing the requested length, which matters for headlines, tweets, and abstract generation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SeqMAP Eq. (11) marginalizes over paths ending at any step, while DAT training (Eqs. 6–7) marginalizes only over paths ending at the final step S with BOS/EOS; the claimed training-inference alignment is unsupported.","rationale":"The central claim has two parts: a theoretical claim that SeqMAP is a MAP objective marginalizing links and therefore aligns with DAT training, and an empirical claim that SeqMAP with a reranker achieves state-of-the-art length-control summarization. The empirical results are plausible but lack significance tests; the beam-search approximation is an admitted limitation and does not by itself invalidate the comparison. The objective mismatch, by contrast, is not admitted and affects the interpretation of every result: if Eq. 11 is not the trained model's marginal, the algorithm is a heuristic, and the paper's explanation for why it beats PathMAP (better training-inference alignment) is unsupported. This is an internal-consistency concern, not a disagreement with external consensus. The reader listed the same mismatch as a second fragile premise, but their primary weakest assumption was beam-search pruning; I think the objective mismatch is more load-bearing because it concerns the definition of SeqMAP itself. The concrete check—computing the training-consistent marginal with BOS/EOS and the final link—would settle it. If the rankings agree, the mismatch is benign and the framing is acceptable; if they disagree, the theoretical motivation and possibly the empirical attribution need revision. I therefore leave the verdict at CONDITIONAL rather than rejecting the paper outright: the method may still work, but the central theoretical claim needs correction or re-derivation, and the proposed test should be reported.","tokens_in":14482,"tokens_out":13338,"duration_ms":138268,"concrete_test":"Take 100 random Gigaword test examples. For each final beam candidate y from the SeqMAP decoder (T = 20% length), compute (i) the Eq.-11 score and (ii) the training-consistent marginal Σ_{a_2<...<a_{T+1}} w_{1,<bos>} · [∏_{t=2}^{T+1} l_{a_{t-1},a_t} w_{a_t,y_{t-1}}] · l_{a_{T+1},S} w_{S,<eos>}, with a_1=1, using the same trained DAT probabilities. If ranking by (i) and (ii) differs (e.g., top-1 changes on more than 5% of cases), Eq. 11 is not the model's posterior. Then add the final eos term to Algorithm 1's scores and rerun the 25%-length Gigaword experiment; a material R-Sum change would show the reported gain depends on the mismatched objective.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the claim that SeqMAP 'aligns with the training objective' (Sec. 1) because it marginalizes links. In DAT training (Eqs. 3–7), a ground-truth summary y of length Ty is scored by marginalizing over Γ_{Ty,S}, the set of paths of Ty linked steps with a_1=1 and a_{Ty}=S. With special tokens, y includes <bos> at step 1 and <eos> at step S, so a T-word content summary corresponds to a Ty=T+2 path in Γ_{T+2,S}. The SeqMAP objective (Eq. 11), however, marginalizes over Γ_{T,s} for every s∈{T,...,S} and uses no <bos>/<eos> and no final link to S. These are different probability models: Eq. 11 omits the mandatory l_{a_T,S}·w_{S,<eos>} factor and admits paths ending before S, which the trained DAT cannot generate. Thus Eq. 11 is not the model's sequence posterior, and the central theoretical motivation—that SeqMAP matches the training marginalization—is internally inconsistent with the model defined in Sec. 2.1. The Limitations section only concedes beam-search suboptimality, not this objective mismatch. If the reported gains come from this altered objective rather than from the marginalization principle, the central claim is overstated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper adapts the Directed Acyclic Transformer (DAT) to length-control summarization and proposes a Sequence Maximum a Posteriori (SeqMAP) decoding objective that marginalizes over possible linked steps, together with an approximate beam-search dynamic programming algorithm and a BERT/RoBERTa-based reranker. Experiments on Gigaword and DUC2004 report consistent ROUGE improvements over PathMAP and CTC baselines, with exact length control, and the paper releases code and outputs.","tokens_in":14848,"tokens_out":5173,"duration_ms":60600,"significance":"If the central claims hold, the paper makes a useful contribution: it brings a more expressive non-autoregressive model to length-controlled summarization, gives a concrete decoding algorithm, and shows consistent empirical gains. The availability of code and outputs is a strength. However, the theoretical motivation—that SeqMAP aligns with DAT's training objective—is not supported by the equations as written, and the empirical evidence lacks significance testing and a clearly separated validation split for hyperparameter selection. These issues bear directly on the paper's main claims and need to be resolved before the results can be fully credited.","major_comments":[{"comment":"The SeqMAP objective in Eq. (11) is not the marginal sequence probability of the DAT model defined in §2.1. Training in Eq. (7) marginalizes only over paths a in Γ_{Ty,S} with a_1=1 and a_{Ty}=S, and, per footnote 2, every path includes <bos> at step 1 and <eos> at step S. Eq. (11), by contrast, sums over paths ending at any step s in {T,...,S} and omits both the mandatory final link l_{a_T,S} and the <eos> word factor. Thus Eq. (11) is a different probability model from the one used in training. The paper's central motivation in §1 and §2.2—that SeqMAP 'aligns with the training objective' because it marginalizes links—is therefore unsupported. Please either redefine SeqMAP as a proper marginalization over Γ_{T+2,S} with <bos>/<eos> and the final link to step S, or explicitly present Eq. (11) as a heuristic decoding objective and remove the training-inference alignment claim.","section":"§2.2, Eq. (11); §2.1, Eqs. (3)–(7)"},{"comment":"The operating hyperparameters K=20 and V=5 are reported in §3.1, and their selection is supported by Tables 7 and 8, but the paper never states whether these tables report validation or test ROUGE. If the tables are on the Gigaword test set, then K and V have been tuned on the test data, which can inflate the reported gains. This is especially important because the SeqMAP-versus-PathMAP differences in R-Sum are around 1 point. Please specify the split used for hyperparameter selection, report multiple seeds or confidence intervals, and provide significance tests for the main ROUGE comparisons.","section":"§3.2, Tables 7 and 8; §3.1 Implementation Details"},{"comment":"The paper's limitations section correctly concedes that Algorithm 1 does not guarantee finding the exact SeqMAP optimum, but the empirical section interprets the SeqMAP-vs-PathMAP gap as evidence of 'the superiority of the SeqMAP objective.' Because Algorithm 1 combines the new objective with top-K/top-V pruning, the observed gains could in principle come from the search heuristic rather than from marginalization. Please provide a diagnostic that separates objective from search—for example, exact or near-exact computation on small S, or a comparison where both methods use matched search effort—so that the reader can tell which component drives the improvement.","section":"§2.2, Algorithm 1; §6 Limitations"}],"minor_comments":[{"comment":"In the sentence 'our SeqMAP aligns with the training objective better (as it huggalso performs marginalization)', 'huggalso' appears to be a typo for 'also'.","section":"§1"},{"comment":"The caption says 'Reranker performance of different K' but the table varies V with K fixed at 20; the caption should say 'different V'.","section":"§3.2, Table 8 caption"},{"comment":"The text says 'the DUC2024 experiment' but the dataset is DUC2004; please correct the typo.","section":"§3.1"},{"comment":"The notation is ambiguous about whether y already includes <bos> and <eos>; if the special tokens are added, the equations and the SeqMAP objective should be stated consistently in terms of the augmented sequence length.","section":"§2.1, footnote 2"},{"comment":"Line 12 contains an extraneous closing brace in 'top-K{Bt,s}' that should be removed, and the ranking criterion for B_{t,s} should be stated explicitly.","section":"Appendix A, Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical contribution, but the central theoretical claim about training-inference alignment is internally inconsistent with the paper's own model definition, and the experimental protocol needs sharper validation discipline. I do not see a reason to reject outright, because the empirical contribution could stand if the objective is reframed as a heuristic and the statistical reporting is improved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuine, useful contribution—a new decoding objective for length-controlled summarization with DAT, plus a beam-search DP and a reranker. The experiments are consistent and the code link is given. The main soft spot is the stated theoretical motivation: the paper says SeqMAP 'aligns with the training objective' because it marginalizes links, but Eq. (11) marginalizes paths of T content words ending at any step s, while DAT training (Eqs. 6–7) marginalizes paths of T+2 tokens (including bos/eos) that must end at the final step S. So Eq. (11) is not the model's sequence posterior; it omits the final link and eos probability. That said, PathMAP (Eq. 8) makes the same omission, so the empirical comparison between SeqMAP and PathMAP is fair. The alignment claim is overstated, not the results.\n\nWhat's new: SeqMAP is distinct from PathMAP (max over one path) and from CTC length control; the DP with EXPAND/MERGE and marginalization over previous linked steps is a reasonable approximation, and the reranker is a sensible addition. The ROUGE gains over PathMAP (roughly 1–2.5 R-Sum) and over CTC (6–8 points) are consistent across Gigaword and DUC2004, including the no-reranker variant, which strengthens the case that the objective itself helps.\n\nSoft spots, in proportion: (1) no significance tests, no confidence intervals, no multiple seeds; gains of 1–2 ROUGE points can be noise. (2) Hyperparameters K and V are selected from tables that appear to be on the Gigaword test set (Tables 7 and 8), with no mention of a validation split. That is test-set tuning, though the selected values are moderate and the trend is flat, so the overfitting risk is low. (3) The missing eos factor in Eq. (11) is a real modeling inconsistency relative to training, but since it also applies to PathMAP, it does not invalidate the comparison. The paper should say SeqMAP optimizes a surrogate objective, not the exact data likelihood. (4) The LLM evaluation is a nice extra but the 0% loss against AT/AT-LenInfo suggests either a weak baseline or LLM bias; treat that table with caution.\n\nWho is it for: people working on non-autoregressive generation, structured prediction, and length-controlled summarization. It deserves a serious referee. I'd send it to review and ask for significance testing, validation-set hyperparameter selection, and a corrected statement about Eq. (11).","headline":"SeqMAP is a plausible and well-tested decoding improvement over PathMAP for DAT length control, but the paper overstates the training-alignment story and tunes hyperparameters on the test set.","tokens_in":15320,"tokens_out":3832,"would_cite":true,"duration_ms":37293,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes SeqMAP, a decoding objective for Directed Acyclic Transformers that marginalizes over all link paths and selects the most probable summary of a fixed length, reporting higher ROUGE than PathMAP and CTC baselines with…","keywords":["length-control summarization","directed acyclic transformer","non-autoregressive generation","SeqMAP decoding","beam search dynamic programming","sequence reranking","hard length constraint","ROUGE evaluation"],"falsifier":"Take a small synthetic DAT with known word and link probabilities, with vocabulary and steps small enough to enumerate all length-$T$ word sequences and all link paths; compute the exact SeqMAP argmax from Eq. (11) and compare it with the output of the beam-search algorithm. If the approximate algorithm frequently returns a sequence whose SeqMAP score is below the exact optimum, the empirical gains could stem from the search heuristic rather than from the marginalization principle.","tokens_in":14319,"feed_emoji":"✂️","tokens_out":9990,"duration_ms":91779,"temperature":0.7,"pith_summary":"Summaries that must fit a fixed length—web titles, tweets, abstract limits—are hard to generate because most models treat length as a soft constraint and then truncate. This paper adapts the Directed Acyclic Transformer, a non-autoregressive model that writes a summary as a graph of word fragments and links, and proposes SeqMAP: rather than committing to the single most probable path through the graph, it sums the probabilities of every path that spells the same candidate summary and then picks the highest-scoring summary of the target length. That sum cannot be maximized by standard dynamic programming, so the authors develop an approximate beam-search algorithm with expand-and-merge recursion and a learned reranker over the beam. On Gigaword and DUC2004, SeqMAP achieves exact length control with zero truncated outputs and reports higher ROUGE scores than CTC-based length control and the earlier PathMAP decoding.","feed_headline":"Hit exact length caps without truncation","feed_subtitle":"SeqMAP beats path-only decoding by 1–2.5 ROUGE points on Gigaword and DUC2004, with zero truncation.","key_machinery":"The central object is the Directed Acyclic Transformer (DAT), a non-autoregressive model that maintains $S$ prediction steps, each with a word distribution and a link distribution over future steps; a path is a chain of linked steps with predicted words. The SeqMAP objective (Eq. 11) is a sum over every valid path and ending step of the product of link and word probabilities for a given word sequence. The algorithm that carries the argument is an approximate beam-search dynamic program with two operations: EXPAND, which extends length-$(t-1)$ beam sequences with the top-$V$ words at step $s$ and scores each extension by marginalizing over the previous linked step's probability, and MERGE, which keeps the top-$K$ sequences generated at or before step $s$. A reranker built on a pretrained RoBERTa encoder with rank embeddings then picks the best candidate in the final beam.","core_discovery":"The central claim is that the best fixed-length summary under a Directed Acyclic Transformer (DAT) is found not by taking the single most probable path of linked words (the PathMAP objective) but by a Sequence Maximum a Posteriori (SeqMAP) objective: for a candidate word sequence of length $T$, sum the probabilities of all link paths and all valid ending steps that spell that sequence, then choose the sequence with the largest sum. Because DAT's training objective marginalizes over link paths, SeqMAP aligns decoding with training better than PathMAP. The paper develops an approximate beam-search dynamic program for this objective—exact search is intractable because the summation and maximization cannot be swapped—and shows empirically that SeqMAP, with and without a learned reranker, outperforms PathMAP and CTC-based length-control summarization on Gigaword and DUC2004 while producing zero truncated outputs.","pith_inferences":["The same SeqMAP-style sum-over-paths decoding could apply to any non-autoregressive generator whose output space is a DAG or lattice, such as CTC-based models, by replacing a Viterbi-style path choice with sequence-level marginalization; this is not tested in the paper.","Because SeqMAP sums over paths ending at any valid step while DAT training marginalizes over paths ending at the final step $S$, a natural testable extension is to train DAT with a length-agnostic marginalization over all end steps, which might align training and inference even more closely.","The gap between the approximate beam search and the exact SeqMAP argmax can be measured on small synthetic instances (small $S$, small vocabulary, known probabilities) by brute-force enumeration; if the beam frequently misses the true optimum, the reported gains are at least partly due to search, not marginalization alone.","The rank-embedding reranker design is portable to other n-best reranking problems where the base model's ranking carries information; the paper's ablation shows this component alone is worth about 0.6 ROUGE-Sum."],"forward_implications":["Exact length control becomes practical for headline, tweet, and abstract generation: every output meets the mandated length, eliminating the truncation step that soft-constraint models require.","Marginalizing over link paths rather than committing to one path improves ROUGE-Sum by roughly 1 to 2.5 points over PathMAP and by 6 to 8 points over CTC on the two reported datasets.","The alignment between decoding and DAT's marginalization-based training suggests the SeqMAP principle should transfer to other DAT-based generation tasks beyond summarization.","The reranker adds about 0.5 to 1.3 ROUGE-Sum points on top of SeqMAP, and its rank embeddings produce a consistent gain in the ablation, indicating that the model's own ranking is useful signal.","Non-autoregressive decoding with SeqMAP (without the reranker) is faster than an autoregressive Transformer baseline, and even the reranked version remains competitive in speed while enforcing the length budget."],"supporting_citations":[{"why":"Introduces the Directed Acyclic Transformer whose word/link predictions and marginalization-based training form the base decoding space.","marker":"Huang et al. (2022b)"},{"why":"Defines the PathMAP objective that SeqMAP is compared against and improves upon.","marker":"Shao et al. (2022)"},{"why":"Applies CTC to length-control summarization, providing the main non-autoregressive baseline and the experimental setup followed here.","marker":"Liu et al. (2022a)"},{"why":"Provides the Gigaword dataset used for training and testing.","marker":"Graff et al. (2003)"},{"why":"Provides the DUC2004 test set used for transfer evaluation.","marker":"Bommasani and Cardie (2020)"},{"why":"Supplies the pretrained RoBERTa encoder used in the reranker.","marker":"Zhuang et al. (2021)"}],"fun_headline_variants":["Sum paths, not choose them: exact-length summaries","Exact length, zero truncation: new decoding wins","SeqMAP decoding beats path-only for length control","Beam search sums path probabilities to hit length caps","Length-control summarization without truncation via SeqMAP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the beam-search approximation—keeping only the top-$K$ partial sequences and top-$V$ word expansions at each step—still contains the true argmax of the SeqMAP objective, and that the objective itself matches DAT's training marginalization; if either fails, the measured gains could come from the search heuristic rather than from the marginalization principle.","fun_headline_variants_meta":{"raw":{"variants":["Sum paths, not choose them: exact-length summaries","Exact length, zero truncation: new decoding wins","SeqMAP decoding beats path-only for length control","Beam search sums path probabilities to hit length caps","Length-control summarization without truncation via SeqMAP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00066,"raw_usage":{"total_tokens":2978,"prompt_tokens":867,"completion_tokens":2111,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":483,"completion_tokens_details":{"reasoning_tokens":2035}},"tokens_in":483,"tokens_out":2111,"duration_ms":13675,"temperature":1.0,"reasoning_tokens":2035,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T22:23:31.592301+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a small synthetic DAT with known word and link probabilities, with vocabulary and steps small enough to enumerate all length-$T$ word sequences and all link paths; compute the exact SeqMAP argmax from Eq. (11) and compare it with the output of the beam-search algorithm. If the approximate algorithm frequently returns a sequence whose SeqMAP score is below the exact optimum, the empirical gains could stem from the search heuristic rather than from the marginalization principle.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Gigaword dataset used for training and testing."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the DUC2004 test set used for transfer evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained RoBERTa encoder used in the reranker."}],"review_version":1}