{"id":"a7fa3a3d-3a99-4d54-9581-65a3fe9d4a7f","arxiv_id":"2508.18651","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Collaborative Decoding fuses a knowledge-conditioned and a context-only token distribution with confidence- and divergence-based weights plus knowledge-aware reranking, improving faithfulness while keeping expressiveness across six LLMs and three dialogue datasets.","lead":"This paper introduces CoDe, a decoding method for knowledge-grounded chatbots that blends the model's output with and without external knowledge, then re-ranks candidate words by their similarity and attention to the supplied facts. If the results hold, RAG systems get a training-free lever to make grounded answers both accurate and natural.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 9's attentive reward att(y_t,k_j) is undefined for a candidate next token in a single forward pass; if read as last-position attention it is candidate-independent and cannot rerank, and if read as per-candidate forward passes it violates the stated zero extra generation budget.","rationale":"I read the paper as proposing a deterministic decoding-time intervention whose two mechanisms are (i) adaptive interpolation of context-only and knowledge-conditioned distributions (Eqs. 3-8) and (ii) knowledge-aware reranking (Eq. 9). The empirical section is broad: ten baselines, six LLMs, nine metrics, plus human and LLM-as-a-judge evaluations, and ablations in Table 5 that attribute clear gains to both components. Those are real evidence, but they do not resolve a formal ambiguity in the definition of the reranking term. The single most load-bearing point is that Eq. 9 uses att(y_t,k_j), a per-candidate attention weight between a not-yet-generated token and knowledge tokens. In a causal transformer such a weight is not available in the forward pass that produces the candidates; the paper never specifies how it is obtained. If it is the last position's attention, it is constant across candidates and cannot do the work shown in Figure 3 and Table 5. If it is computed by scoring each candidate in its own forward pass, the method is not 'without additional generation budgets'. The same issue applies to h_{y_t} in the cosine term. This is more foundational than the reader's concern about whether attention and hidden-state similarity track parametric preferences: even if those signals are faithful, the equation as written cannot be executed as a reranking over ungenerated tokens unless an extra mechanism is assumed. The reader's identified assumption is real but secondary; my concern is about operational validity. Other issues in the paper - beta and gamma tuned on the FaithDial test set (Appendix D, Figure 7), HalluJudge retrained locally, no code release, and the flawed proof of Theorem 1 in Appendix I - are worth fixing but do not by themselves undermine the central empirical claim as directly. Because the concern is addressable (the authors can specify or release the implementation) and does not prove the method fails, the appropriate status remains CONDITIONAL; my analysis does not move the reader's verdict. If future versions provide code or a precise implementation of Eq. 9 and the cost accounting, the concern can be closed.","tokens_in":27710,"tokens_out":6941,"duration_ms":67367,"concrete_test":"Obtain or reimplement CoDe and instrument the reranking step on FaithDial with Llama2-7B-chat. For each decoding step, record the top-K candidates under Eq. 9 under two implementations: (A) att(y_t,k_j) set to the last-position attention to knowledge tokens (candidate-independent); (B) att computed by a separate forward pass with each candidate appended to the prefix (candidate-specific). Compare selected tokens and downstream F-Critic, H-Judge, and K-BP scores. If (A) reproduces the Table 5 '-Att' ablation drop, the attentive reward must have been computed differently than Eq. 9 states; if only (B) reproduces it, the method requires K extra forward passes per token, falsifying the 'no additional generation budget' claim. Either outcome forces a correction to Eq. 9 or to the cost and mechanism claims.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2, Eq. 9 reranks candidate tokens with (1/2)[max_i sim(h_{y_t}, h_{k_i}) + max_j att(y_t,k_j)]. During autoregressive decoding at step t, y_t is a candidate next token, not an input position. A standard causal transformer therefore has no attention weight 'between token y_t and knowledge element k_j': attention is a function of input positions, and the only prefix-to-knowledge attention available is from the last input position (or previous generated tokens) to the knowledge chunks. Two readings of Eq. 9 are possible. (i) If att(y_t,k_j) is taken from the last input position, it is identical for every candidate y_t, so the attentive reward is constant under argmax and cannot change token selection; the ablation row '-Att' in Table 5 could not then show a real effect from this term. (ii) If att is computed by appending each candidate to the prefix and running a forward pass, each decoding step costs K extra forward passes (K=4 in Appendix D), contradicting the 'without requiring additional ... generation budgets' claim and the plug-and-play framing. The same position ambiguity affects h_{y_t} in the cosine-similarity term. The paper releases no code and gives no implementation note for Eq. 9, so the central faithfulness mechanism is currently under-specified. This is load-bearing because Eq. 9 and its ablations are the principal evidence that CoDe prevents parametric over-reliance (Figure 3, Table 5).","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Collaborative Decoding (CoDe), a decoding-time method for knowledge-grounded generation in LLMs. CoDe has two components. The Adaptive Dual-Stream Fusion module computes two next-token distributions, one conditioned on context alone and one conditioned on context plus external knowledge, and combines them via a weight alpha that depends on model confidence and on the Jensen-Shannon divergence between the two distributions (Eqs. 3-8). The Knowledge-Aware Reranking module re-ranks the top-K candidates using a weighted combination of semantic cosine similarity and attention between candidates and knowledge tokens (Eq. 9). The method is evaluated on six LLMs (Llama-2-7B-chat, Llama-3.1-8B-chat, Mistral-7B-Instruct-v0.2, Qwen-2.5-3B/7B/14B) on FaithDial, HalluDial, and Wizard of Wikipedia, plus Natural Questions, NQ-Swap, and HalluEval, against ten baselines, with automatic, LLM-judge, and human evaluations. The paper reports consistent improvements in faithfulness (K-BP, F-Critic, H-Judge) while preserving or improving expressiveness (DIV, COH, CRE), and claims to break the faithfulness-expressiveness trade-off without additional training or generation budgets.","tokens_in":28065,"tokens_out":8231,"duration_ms":74640,"significance":"If the method works as described, it is a practically appealing contribution: it requires no training, is compatible with any autoregressive LLM, and the empirical evaluation is unusually broad. The paper's strengths include the breadth of models and datasets, the inclusion of human and LLM-based evaluation, and ablations that decompose the contributions of the fusion and reranking components. However, the central reranking mechanism in Eq. 9 is currently under-specified to the point of being ambiguous, and the hyperparameters are tuned on the test set. These issues must be resolved before the claim that CoDe breaks the trade-off can be accepted.","major_comments":[{"comment":"The attentive reward att(y_t, k_j) is not computable as written. In a causal transformer, attention weights are defined over input positions, and the candidate next token y_t is not yet an input position at decoding step t. If att is read as the attention from the last input position to the knowledge tokens, it is identical for every candidate and therefore cannot change the argmax; in that case the ablation row '-Att' in Table 5 could not produce the reported drop in H-Judge from 90.8 to 88.3. If att is instead computed by appending each candidate to the prefix and running a forward pass, each decoding step requires K additional forward passes (K=4 in Appendix D), which contradicts the stated claim of 'without requiring additional ... generation budgets'. The same position ambiguity applies to the hidden state h_{y_t} in the semantic reward. The paper releases no code and gives no implementation note for Eq. 9, so this load-bearing faithfulness mechanism is currently under-specified. Please provide unambiguous pseudocode for one decoding step, state precisely where h_{y_t} and att(y_t, k_j) come from, and reconcile the compute cost with the no-additional-budget claim.","section":"Section 4.2, Eq. (9), and Table 5"},{"comment":"The hyperparameters beta and gamma are selected on the FAITH DIAL test set. Figure 7 reports optimal performance at beta=0.6 and gamma=3 with no held-out validation split, and the same values are then used for all experiments and all datasets. Because the headline results on FAITH DIAL (Tables 1 and 2) are obtained with these test-set-tuned values, the reported improvements may partly reflect overfitting to the evaluation set. This is load-bearing for the generalizability claim. The authors should either tune on a separate validation split (for example, a portion of the training set) or report results across a range of hyperparameters to show that the qualitative conclusions do not depend on the chosen values.","section":"Section 5.3 and Figure 7"},{"comment":"The paper claims CoDe operates 'without requiring additional training, model, or generation budgets' (contributions bullet and Section 4), but the dual-stream fusion in Eq. 3 requires two full forward passes per decoding step, one with and one without external knowledge. If Eq. 9 is implemented with per-candidate forward passes, the compute cost grows further. No wall-clock time, FLOP counts, or forward-pass counts are reported for any method, so the budget claim is unsupported. Please define what counts as a 'generation budget' and quantify the actual overhead of CoDe relative to the baselines, or revise the claim accordingly.","section":"Introduction, Section 4.1, and Appendix D"},{"comment":"The H-Judge metric is a re-trained version of HalluJudge, not the released model, because the original weights were unavailable. The paper states that Meta-Llama-3-8B was trained on HalluDial, but no evidence is given that this retrained judge reproduces the original HalluJudge's judgments, and the training-data size, evaluation accuracy, and agreement with the original model are not reported. Since H-Judge is one of the three headline faithfulness metrics and drives several specific claims, please report agreement on a sample with the original HalluJudge if obtainable, or at minimum give the training details and validation accuracy of the re-trained judge.","section":"Appendix E.1 and Table 1"}],"minor_comments":[{"comment":"There is a typo in the first sentence: 'e evaluated' should be 'We evaluated'.","section":"Section 5.1"},{"comment":"The expression for C_t is ambiguous as typeset: it can be read as 2*sqrt(pmax)/H_t + eta. Since the text says a geometric mean is intended, please write C_t = 2 * sqrt(pmax / (H_t + eta)) with explicit parentheses.","section":"Eq. (7)"},{"comment":"The notation 'topK' is undefined. Please clarify whether the top-K candidates are selected before or after the weighted combination with the semantic and attentive rewards, and whether the resulting distribution is renormalized.","section":"Eq. (9)"},{"comment":"The claim that CoDe 'also achieves top-2 performance in diversity and relevance metrics' is not supported by the reported tables; for example, in Table 6 (WoW-Seen) the CRE of CoDe is 27.7, lower than top-k (33.4), Nucleus (33.1), CD (32.9), DoLa (30.4), CS (29.0), and Greedy (28.7). Please qualify the claim or point to the specific metrics and tables that support it.","section":"Section 5.2"},{"comment":"The sentence 'We conducted three experiments for all methods, using a different set of samples in each experiment' is unclear. Please specify what differs across the three runs and report standard deviations or confidence intervals so the reader can assess the stability of the reported averages.","section":"Appendix D"},{"comment":"Theorem 1 (F |= T) is proved under the assumption that the provided external knowledge K is a proper subset of world knowledge K_w, but this assumption is not stated in Definition 2. If the external knowledge contains outdated or erroneous information, a response can be faithful to K yet not factual with respect to K_w, so the theorem should be restated as conditional on K being a subset of true world knowledge.","section":"Appendix H/I"},{"comment":"No code is released. Given the implementation ambiguity in Eq. 9, a public implementation or detailed pseudocode would greatly aid reproducibility and verification.","section":"General"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about arXiv:2508.18651. The adaptive dual-stream fusion (Eq. 3 with the JSD/confidence weight) is a genuinely different, plausible way to get knowledge-grounded responses that are more faithful without sounding like knowledge-dump. The experiments are broad: ten baselines, six LLMs, three dialogue datasets plus QA/summarization, human eval. The main empirical claim mostly holds up. But the paper's central reranking term (Eq. 9) is under-specified to the point that I can't tell what it computes, and the 'no extra generation budget' claim is not believable as written.\n\nWhat's actually new: adaptive alpha from local confidence and global entropy, and the knowledge-aware top-K reranking with semantic and attentive rewards. The ablations show each component helps, and the qualitative examples are genuinely better than CAD and greedy. That's credit where earned.\n\nWhere it gets shaky. First, Eq. 9. A candidate next token doesn't have attention weights to knowledge positions in a single forward pass. If they take last-position attention, the term is identical for every candidate and cannot change the argmax, yet the -Att ablation shows an effect. If they append each candidate and run a forward pass, each step costs K extra model forwards (K=4), which contradicts 'without requiring additional generation budgets' and the plug-and-play framing. No code, no implementation note. Even under the most charitable implementation, the reranking rewards tokens the model already attends to, so it can reinforce parametric confidence rather than external grounding; the ablations don't isolate that. This is not a nitpick: the reranking is the mechanism that prevents parametric over-reliance, and it's currently unverifiable.\n\nSecond, beta and gamma are tuned on the FaithDial test set (Fig. 7) with no held-out validation reported. That's a real leak, though the smooth parameter curves suggest it probably doesn't reverse the conclusions.\n\nThird, the appendix proof of Theorem 1 is incorrect. The step 'Since T|=F, then Φ⊆K' doesn't follow from the definitions, and the assumption that external knowledge is a subset of world knowledge is itself shaky in RAG. Minor relative to the main paper, but it's a sign of haste. Also: no code, and H-Judge is a locally retrained model without release, so numbers are internally consistent but not directly reproducible.\n\nWho this is for: people working on RAG, knowledge-grounded dialogue, and decoding-time hallucination mitigation. It deserves a serious referee; the empirical package is solid and the idea is worth engaging. I would not cite it in its current form, but I'd be glad to see a revision that pins down Eq. 9, releases code, and moves hyperparameter selection to a validation split.","headline":"CoDe is a solid, well-tested decoding method for knowledge-grounded dialogue with a promising adaptive dual-stream fusion, but its central reranking term (Eq. 9) is under-specified, possibly far more expensive than claimed, and the paper tunes on the test set without code.","tokens_in":28547,"tokens_out":5343,"would_cite":false,"duration_ms":51415,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes Collaborative Decoding (CoDe), a training-free decoding strategy that fuses knowledge-conditioned and context-only output distributions with adaptive weights, claiming to make LLM responses simultaneously more faithful…","keywords":["collaborative decoding","faithfulness","expressiveness","knowledge-grounded dialogue","decoding strategy","hallucination mitigation","retrieval-augmented generation","adaptive fusion"],"falsifier":"Build a set of prompt instances from NQ-Swap, where the provided knowledge contradicts the model's parametric answer, and record, for each reranked token, the attention-to-knowledge and hidden-state similarity scores. If CoDe promotes the knowledge-supported token on confident-conflict cases less often than CAD does, or if the reranking rewards are higher for tokens the model already believed before seeing the passage, the mechanism's core assumption fails.","tokens_in":27511,"feed_emoji":"💬","tokens_out":7240,"duration_ms":62660,"temperature":0.7,"pith_summary":"The paper sets out to break a trade-off that plagues knowledge-grounded language models: deterministic decoding (greedy, beam, contrastive search, CAD) produces responses faithful to provided knowledge but stiff and copy-heavy, while stochastic decoding (top-k, nucleus sampling) produces lively responses that drift into hallucination. It proposes Collaborative Decoding (CoDe), which fuses the model's output distribution with external knowledge and its distribution without it, using an adaptive weight based on model confidence and the Jensen-Shannon divergence between the two distributions. A knowledge-aware reranking step then favors tokens that attend to and semantically resemble the supplied knowledge. Across six LLMs and three dialogue datasets, CoDe reports higher faithfulness than all ten baselines while matching or beating them on expressiveness metrics. If correct, the result shows that the fidelity-expressiveness trade-off is a property of decoding strategy, not an inherent limit, and can be overcome without training or extra generation budgets.","feed_headline":"Fusing two decoding streams keeps LLM answers faithful and lively","feed_subtitle":"The method needs no retraining and works across six LLMs, keeping responses grounded and natural.","key_machinery":"The load-bearing object is the fused distribution in Eq. (3), an interpolation between the prior distribution over next tokens given only the conversation and the posterior distribution given the external knowledge: $p_{\\text{CoDe}}(y_t) \\propto p_\\theta(y_t\\mid x,y_{