Pith. sign in

REVIEW 5 cited by

Learning to Decode Collaboratively with Multiple Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.03870 v2 pith:CXM4YKOD submitted 2024-03-06 cs.CL cs.LG

Shannon Zejiang Shen , Hunter Lang , Bailin Wang , Yoon Kim , David Sontag This is my paper
classification cs.CLcs.LG
keywords modelslanguagelatentmodelbasecollaborationdecodinggenerate
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent variable. By optimizing the marginal likelihood of a training set under our latent variable model, the base LLM automatically learns when to generate itself and when to call on one of the ``assistant'' language models to generate, all without direct supervision. Token-level collaboration during decoding allows for a fusion of each model's expertise in a manner tailored to the specific task at hand. Our collaborative decoding is especially useful in cross-domain settings where a generalist base LLM learns to invoke domain expert models. On instruction-following, domain-specific QA, and reasoning tasks, we show that the performance of the joint system exceeds that of the individual models. Through qualitative analysis of the learned latent decisions, we show models trained with our method exhibit several interesting collaboration patterns, e.g., template-filling. Our code is available at https://github.com/clinicalml/co-llm.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Inference-Time Consensus for Mitigating Hidden Behaviors from LLM Fine-Tuning

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Token-wise minimum and base-relative consensus over per-source fine-tunes suppress hidden and explicit poisoning that union training and weight averaging retain.

  2. PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference

    cs.CL 2026-07 conditional novelty 6.0 of 10

    PyroDash lets a small model decide mid-generation to hand off once to a frozen LLM, achieving 64.04% accuracy (+6.36 points vs LLM) at lambda=0.05 and 96.4% lower estimated cost at lambda=0.6.

  3. Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units

    cs.AI 2025-08 conditional novelty 6.0 of 10

    MCSU-based vocabulary alignment plus distance-based dynamic selection (DDS) lets several LLMs vote token-by-token, beating single models and prior ensemble baselines on multiple reasoning benchmarks without training.

  4. What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding

    cs.CL 2025-06 conditional novelty 6.0 of 10

    FoReaL-Decoding lets a strong reasoning model generate the first few tokens of each sentence and a weaker model complete the sentence, cutting theoretical FLOPs by 30-55% while retaining 86-100% of accuracy on four ma...

  5. Toward Edge General Intelligence with Multiple-Large Language Model (Multi-LLM): Architecture, Trust, and Orchestration

    cs.NI 2025-07 conditional novelty 4.0 of 10

    A survey of multi-LLM systems in edge computing, covering architectures, enabling technologies, trust mechanisms, applications, and open datasets for edge general intelligence.

Pith tools