{"id":"49f32c5e-0ce7-42f8-8be4-70d0608c1d87","arxiv_id":"2508.18292","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":1,"one_line_summary":"Language models exchanging answers with each other until they agree can, according to this paper, produce more accurate and robust collective reasoning than any single model.","lead":"This paper describes a method where several large language models swap answers and reasoning with each other, like a gossip network, until they settle on one shared response. If the claims hold, engineers could combine existing models to get more accurate and robust answers without training a new one.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Consensus accuracy claim rests on unverified premise that agreement implies correctness; correlated LLM errors may make gossip converge on shared wrong answers.","rationale":"The reader's weakest_assumption precisely identifies the same load-bearing premise: that agreement implies correctness. My stress-test agrees and sharpens it by pointing to the mechanism of correlated errors from shared training data, which makes consensus more likely to amplify systematic biases than to cancel them. The proposed concrete test would empirically settle whether consensus accuracy exceeds the best constituent model on adversarially selected correlated-error items. Because the full text is unavailable, the reader's UNVERDICTED verdict remains appropriate; my concern does not move the verdict, but it reinforces the need for direct empirical validation before the central claim can be accepted.","tokens_in":685,"tokens_out":1876,"duration_ms":23318,"concrete_test":"Construct a benchmark from known LLM hallucination or common-misconception datasets, selecting items where several strong individual LLMs (e.g., GPT, Claude, Llama) independently produce the same wrong answer with high confidence. Run the proposed gossip protocol on this filtered set. Compare final consensus accuracy to (a) the best single model, (b) independent majority voting, and (c) a diversity-preserving ensemble that stops before full agreement. If consensus accuracy is not significantly higher than the best individual and majority vote on these correlated-error items, the claim that gossip consensus overcomes individual weaknesses fails. Additionally, track answer entropy across rounds; if entropy collapses to zero while accuracy does not improve, the process is converging on a shared error rather than toward truth.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's central claim is that gossip-based consensus yields 'robust, resilient, and accurate' multi-agent reasoning. The load-bearing premise is that convergence to agreement is a reliable proxy for correctness. This is not justified. All LLMs in the ensemble are trained on overlapping internet text, so their errors are correlated. A gossip protocol that repeatedly exchanges answers and moves toward consensus will tend to reduce diversity and amplify the most common response, which may be a confidently wrong answer shared by all models. The mechanism described is exactly a consensus protocol on a value; nothing in the abstract indicates that the consensus value is anchored to ground truth. If the initial responses are conditioned on the same prompt, the ensemble may collapse to the majority error rather than 'overcome weaknesses.' The claim that consensus 'brings out collective strengths' therefore requires evidence that the consensus point is more accurate than the best individual model, especially on items where all constituents share the same blind spot. Without such evidence, the central claim is unsupported. Since the full text is unavailable, this concern targets the core premise stated in the abstract, not implementation details.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a multi-agent reasoning framework in which several large language models (LLMs) exchange answers and reasoning traces in a peer-to-peer, gossip-style protocol and iteratively converge on a shared solution. The abstract claims that this 'gossip-based consensus' yields robust, resilient, and accurate reasoning, overcomes individual model weaknesses, and is analogous to human consensus-building. The available material is abstract-only: no protocol specification, datasets, baselines, numerical results, or equations are provided. The central claim is therefore stated but not evidenced in the text available for review.","tokens_in":927,"tokens_out":2169,"duration_ms":26970,"significance":"If substantiated, the idea would be of interest to the multi-agent LLM and distributed-systems communities: a lightweight, decentralized consensus mechanism that improves accuracy over individual models would be a practical contribution, and the claim is empirically falsifiable (e.g., by comparing consensus accuracy with the best constituent model on held-out tasks). However, as presented, the abstract makes a strong empirical assertion without any supporting measurement or protocol detail. The paper's contribution currently rests on an unverified premise—that convergence to agreement is a reliable proxy for correctness—which is especially risky when constituent models share training data and correlated errors. The absence of evidence is the central obstacle; the idea itself is not implausible and merits the requested revision.","major_comments":[{"comment":"The central empirical claim—that gossip-based consensus leads to 'robust, resilient, and accurate multi-agent AI reasoning'—is unsupported in the available text. No accuracy figures, datasets, baselines, or protocol details are given. The abstract states the result rather than reporting it; since the entire contribution is the empirical claim, the reader cannot assess soundness. Please provide the full experimental protocol and results, or clearly mark the claim as a hypothesis.","section":"Abstract"},{"comment":"The method equates convergence to agreement with correctness. This concern is load-bearing: the described process—models exchange answers and 'gradually work toward a shared solution'—will tend to reduce diversity and amplify the most frequent response. Because LLMs are trained on overlapping internet text, their errors are correlated; consensus may converge on a confidently wrong answer shared by all members. No argument or evidence is given that the consensus point is anchored to ground truth or that it surpasses the best individual model, especially on items where all constituents share the same blind spot. Please add a correctness criterion, a comparison against each constituent model, and a failure-mode analysis.","section":"Abstract"},{"comment":"No concrete protocol is specified: number of rounds, peer-selection strategy, convergence threshold, and the exact form of exchanged information ('answers and thought processes') are undefined. Without these, the approach cannot be reproduced or compared against existing ensemble or distributed-reasoning baselines. The analogy to gossip protocols in distributed systems is mentioned but not formalized or cited; please give a precise algorithmic description and position it relative to prior consensus-based multi-agent methods.","section":"Abstract"}],"minor_comments":[{"comment":"The phrase 'robust, resilient, and accurate' bundles three distinct properties. Specify which evaluation metrics address each property, and report them separately.","section":"Abstract"},{"comment":"The sentence 'making AI seem more collaborative and trustworthy' conflates perceived collaboration with measured trustworthiness. Either provide trust-related evaluation or rephrase as a motivation rather than a result.","section":"Abstract"},{"comment":"The opening observation that 'no single model excels in every area' is used as a premise. Cite relevant benchmark evidence or present it as a motivating assumption rather than an established fact.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"This review is based on the abstract only because the full text was not supplied. The abstract asserts the paper's key empirical contribution without any supporting data or protocol specification, so major revision is necessary regardless of the full text's content. The editor may wish to obtain the complete manuscript and verify that the empirical claims include comparisons against each constituent model and an analysis of correlated-error failure cases before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the abstract makes a strong empirical claim and shows no evidence to back it. That is the main issue, but it is not the only one. The idea itself is legitimate: using gossip protocols to coordinate LLM ensembles is a genuine new twist on the multi-agent reasoning line, distinct from debate or mixture-of-agents. I credit the authors for borrowing a sensible distributed systems technique and applying it to LLM exchange, and the framing is clear enough that the method could be implemented tomorrow.\n\nWhat works: the motivation is standard (no single model is best), the mechanism (random peer-to-peer exchange until consensus) is concrete, and the intended benefit - tapping collective strengths - is easy to understand. If the full manuscript reports benchmarks, ablations, convergence behavior, and a comparison to existing methods, this could be a useful contribution.\n\nWhere it falls down: the abstract contains zero numbers, no datasets, no baselines, no protocol parameters, and no comparison to iterative debate or other multi-agent approaches. Worse, the central claim that consensus improves accuracy rests on an unexamined assumption that agreement equals truth. LLMs share training data, so their errors are correlated. A gossip protocol that iteratively averages answers could simply converge on the most common confident mistake. The authors do not address this anywhere in the visible text. They also throw around 'robust, resilient, trustworthy' without defining any of these terms. The lack of citations is also a missed opportunity; positioning against prior work would make the claim more believable.\n\nBottom line: this is a one-paragraph idea, not yet a paper. For someone working in multi-agent LLM reasoning, it might be worth reading the full manuscript if it contains actual evidence and a discussion of failure modes. As it stands, I would not cite it, and I cannot assess its soundness from the abstract alone.\n\nRecommendation: I would not desk reject it outright - the idea is legitimate and the subfield is active. But I would only send it to peer review if the full paper delivers empirical support and directly addresses the correlated-error concern. If the full text is as thin as the abstract, it is not worth referee time.","headline":"The abstract promises strong empirical results but delivers none; the underlying gossip idea is plausible, but the paper needs a full read and real evidence before anyone can judge it.","tokens_in":653,"tokens_out":813,"would_cite":false,"duration_ms":53988,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Gossip-based consensus among LLMs yields shared, more accurate answers","keywords":["gossip protocol","multi-agent reasoning","LLM ensemble","consensus","peer-to-peer","collaborative AI"],"falsifier":"Take a set of questions with known correct answers, including questions specifically chosen to trigger common LLM errors; run the gossip protocol to consensus and compare the consensus answer's accuracy against each single model's accuracy. If the consensus is wrong as often as or more than the best individual model, the claim that agreement brings accuracy is falsified.","tokens_in":603,"feed_emoji":"🤝","tokens_out":2190,"duration_ms":24288,"temperature":0.7,"pith_summary":"This paper proposes treating a team of large language models as nodes in a peer-to-peer network that exchange answers and reasoning steps, then iteratively revise their responses until they converge on a shared answer. The author's claim is that this gossip-based consensus protocol makes the group more accurate than any single member, because models' individual weaknesses cancel out and their collective strengths accumulate. If true, ensembles of off-the-shelf LLMs could be made more reliable without retraining or a central orchestrator.","feed_headline":"Gossip-based consensus lifts LLM team accuracy","feed_subtitle":"Models swap answers and reasoning until they agree, and the group outperforms any single model, the paper claims.","key_machinery":"Gossip-based consensus: a distributed protocol in which each LLM is a node that periodically sends its current answer and reasoning to randomly chosen peers, receives theirs, and updates its own response toward the group. The repeated exchange is what drives individual models out of their isolated errors and into a shared solution.","core_discovery":"The paper's central claim is that consensus building through gossip — each LLM sharing its answer and thought process with peers and revising toward agreement — produces accurate multi-agent reasoning. The author argues that this mimics human consensus and turns the ensemble into a collaborative system rather than a black box.","pith_inferences":["The paper does not justify why agreement tracks truth; because the models share substantial training data, gossip may converge on a confidently wrong answer that all members agree on. An adversarial test with questions designed to trigger common LLM errors would clarify this.","Gossip's random peer selection may behave differently from fixed voting or ensemble averaging, potentially acting as a cheap search over reasoning paths with agreement as the stopping criterion.","The protocol resembles human group discussion, so social-psychology findings about conformity and groupthink could yield testable predictions — for instance, that an early confident answer tends to dominate the consensus.","A natural extension is to vary the peer-sampling rate or the update rule, which could allow the protocol to trade speed of convergence against diversity of explored answers."],"forward_implications":["Multi-model ensembles can improve accuracy without a central aggregator or a fixed voting scheme.","The protocol scales to any number of models, so larger and more heterogeneous groups can participate in the same consensus process.","Exchanging reasoning alongside answers means flawed chains of thought are exposed to peer correction, not just the final outputs.","Because the discussion trail is visible, the resulting AI system is less of a black box and more like a collaborative group.","The approach can be applied directly to existing off-the-shelf models, with no retraining or special architecture required."],"supporting_citations":[],"fun_headline_variants":["LLMs reach consensus via gossip, boosting accuracy","Gossip makes LLM teams smarter","When LLMs gossip, accuracy improves","Peer-to-peer LLM reasoning improves answers","Consensus via gossip boosts LLM accuracy"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The accuracy claim rests on the unstated premise that convergence to agreement is a reliable signal of correctness; because the models share training data, they may converge on a confidently wrong answer.","fun_headline_variants_meta":{"raw":{"variants":["LLMs reach consensus via gossip, boosting accuracy","Gossip makes LLM teams smarter","When LLMs gossip, accuracy improves","Peer-to-peer LLM reasoning improves answers","Consensus via gossip boosts LLM accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000429,"raw_usage":{"total_tokens":1943,"prompt_tokens":573,"completion_tokens":1370,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":317,"completion_tokens_details":{"reasoning_tokens":1314}},"tokens_in":317,"tokens_out":1370,"duration_ms":10535,"temperature":1.0,"reasoning_tokens":1314,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T17:31:24.544218+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of questions with known correct answers, including questions specifically chosen to trigger common LLM errors; run the gossip protocol to consensus and compare the consensus answer's accuracy against each single model's accuracy. If the consensus is wrong as often as or more than the best individual model, the claim that agreement brings accuracy is falsified.","supporting_citations":[],"review_version":1}