Pith. sign in

REVIEW 4 major objections 6 minor 52 references

Synergistic Weak-Strong Collaboration by Aligning Preferences

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that a small, domain-fine-tuned model and a large, general-purpose model can be paired so that the small model's specialized drafts, refined by the large model, outperform either model alone, and that aligning the small…

desk verdict Genuinely new preference-construction recipe for weak-strong collaboration, but the alignment gain is not firmly established because the evaluator doubles as the strong model and every reported number is a single run. read the letter →

arxiv 2504.15188 v2 pith:3I734YKJ submitted 2025-04-21 cs.AI

Yizhu Jiao , Xuchao Zhang , Zhaoyang Wang , Yubo Ma , Zhun Deng , Rujia Wang , Chetan Bansal , Saravan Rajmohan
show 2 more authors
Jiawei Han Huaxiu Yao
This is my paper · ORCID
classification cs.AI
keywords weak-strongcollaborationpreferencealignmentdirectoptimizationspecializedlanguagemodelsmodelblack-boxLLMadaptationcounterfactualquestionansweringmedical
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes COWEST, a way to make a small domain-specialized language model and a large general-purpose language model work as a team: the small model writes a first draft with domain knowledge, and the large model rewrites it. The paper's central claim is that this collaboration outperforms either model used alone on three specialized benchmarks, and that tuning the small model with preference data gathered from the collaboration makes the team even better. The preference signal is generated by comparing, for the same query, the large model's solo answer against its refined answer built from the small model's draft, scored by the same large model on a 1-to-10 rubric. If the collaborative answer scores higher, the small model's draft is treated as preferred and used for direct preference optimization. The authors report consistent gains across counterfactual question answering, medical exam questions, and ethical-dialogue classification.

What carries the argument

The machinery is the collaborative preference signal \(\$\Delta$ = E(\pi_s \circ y, x) - E(z, x)\), the difference between the evaluator's score of the full collaborative output and the strong model's solo output. A positive \(\$\Delta$\) turns the weak draft \(y\) into a preferred sample; a non-positive \(\$\Delta$\) turns it into a dispreferred sample. These pairs feed direct preference optimization (DPO), which shifts the weak model's policy toward drafts the strong model can refine successfully. The same component does double duty: it labels training data and, per the paper's theoretical analysis, forces the aligned weak model to avoid drafts that fail to beat the strong-only baseline.

What would settle it

Re-run the preference-data construction with the rubric replaced by ground-truth-exact scoring, such as exact match on the final answer, and compare the DPO-tuned weak model against the SFT-only weak model in collaboration; if the alignment gain disappears or reverses, the reported gain comes from the evaluator rather than from better collaboration.

Watch

Extended reading notes

Core claim

COWEST is a framework for weak-strong model collaboration. The weak model is a small, white-box model fine-tuned on a target domain; the strong model is a large, black-box model with general reasoning ability. At inference time the weak model produces a draft answer and explanation, and the strong model refines that draft into a final answer. The paper's central discovery is that this division of labor — specialized drafting plus generalist refinement — outperforms either model used alone on all three tested domains, and that the improvement is largest when the strong model is genuinely capable of correcting the drafts. To strengthen the partnership, the paper constructs preference pairs by comparing the evaluator score of the strong model's solo answer with the evaluator score of the collaborative answer: if the collaborative answer scores higher, the weak model's draft is a preferred sample; otherwise it is a dispreferred sample. Direct preference optimization over these pairs aligns the weak model with the strong model's preferences, and the paper proves that under a mild assumption the aligned weak model assigns zero probability to drafts whose collaborative score would not beat the strong model's solo baseline.

Load-bearing premise

The load-bearing premise is that the large model's 1-to-10 rubric score actually tracks correct answers; if the evaluator rewards style or shared bias instead of correctness, the preference-tuning step could tune the small model away from what makes the collaboration better.

Editorial extensions

If this is right

  • Specialized capability can be added to a black-box model by fine-tuning only a small companion model, avoiding retraining or parameter access to the large model.
  • The alignment step needs no human preference labels: preference pairs are constructed automatically from the two models plus an evaluator, so the loop can be rerun whenever a new domain or a new strong model appears.
  • The gain is not automatic: the strong model must be capable enough to correct and refine the weak draft; pairing a weak model with a barely-stronger model does not guarantee collaborative improvement.
  • Giving the weak model a chain-of-thought format, with explanation and answer, helps the strong model refine more effectively than a bare answer or background-knowledge-only format.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the evaluator and the strong model are the same model, the DPO signal may encode the large model's stylistic preferences rather than objective correctness; a natural test is to label the same drafts with a different evaluator family and see whether the alignment gain survives.
  • The paper's own theoretical result implies a cheap inference-time rule the authors do not test: sample several weak drafts, estimate which one will yield the largest collaborative score, and pass only that one to the strong model.
  • The signed preference score \(\Delta\) could be reused as a reward for other training algorithms, such as reward-weighted regression or rejection sampling, or as a trigger to decide when the strong model should refine at all rather than answer alone.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes COWEST, a framework for weak-strong LLM collaboration. A specialized small model produces an initial draft and background information, and a general strong model refines this output into the final answer. To improve collaboration, the weak model is aligned via Direct Preference Optimization (DPO) using preference pairs constructed from whether a collaborative output scores higher than a strong-only output under an LLM evaluator. Experiments on IfQA, MedMCQA, and Prosocial-Dialog report that COWEST outperforms single models and prior collaboration methods, and ablations suggest that preference alignment further improves performance. Section 3.5 offers a theoretical argument intended to show that the aligned weak model avoids producing outputs that do not improve on the strong model's standalone performance.

Significance. If the empirical results are robust, COWEST provides a practical recipe for specializing small models in niche domains without updating or even accessing the parameters of the large model, and the idea of using collaborative feedback as a preference signal is a useful contribution. The paper ships publicly available code, evaluates on three domains with diverse baselines, includes informative ablations, and states its limitations clearly. The main weaknesses are that every reported number appears to be a single run with no error bars or significance tests, and the preference signal comes from the same model family that serves as the strong model, so the alignment gains could partly reflect stylistic self-consistency rather than external correctness. The theoretical section also overclaims what follows from the finite-sample DPO objective. These issues are addressable, but they currently prevent the central claims from being fully established.

major comments (4)
  1. [3.5] The conclusion that the optimized weak model satisfies π∗w(y|x)=0 for every y with E(πs∘y,x)≤p(x) does not follow from the finite-sample DPO objective in Eq. (2). DPO trains on a finite set of preference pairs and includes a KL penalty to the reference policy; it does not force zero probability on dispreferred or unobserved outputs unless the implicit reward is infinite and the optimization is exact. Please remove the zero-probability claim, replace it with a finite-sample statement, or prove it under explicit assumptions on the reward model and optimization.
  2. [3.3.1 and Figure 7] The evaluator E is the same model family as the strong model πs (GPT-4), and the text states that this choice ensures consistency in reflecting the strong model's preferences. Because preference labels are computed from this evaluator, the DPO objective in Eq. (2) may align the weak model to the strong model's stylistic self-preferences rather than to external correctness, so the alignment gains in Table 1 and Figure 3 could be an artifact of evaluator bias. Please add an experiment with an independent evaluator (e.g., a different model family or human annotations) to verify that the alignment gains persist under an external correctness signal.
  3. [Table 1 and Figure 3] All reported numbers appear to be from a single run, with no error bars, confidence intervals, or significance tests. The alignment deltas are small on some datasets (e.g., Medicine accuracy 73.8 vs 74.7 and Ethics accuracy 67.1 vs 68.3 in Figure 3), so without variance estimates it is unclear whether the central claim that alignment further enhances performance is supported. Please report multiple random seeds with standard deviations and, where appropriate, paired significance tests.
  4. [3.5, Corollary 1] The corollary uses an undefined object p(z) and assumes p(z)≤E(z,x) for all z, but the sign of ∆=E(πs∘y,x)−E(z,x) in Section 3.3.1 depends on the realized strong-model output z, not on a lower bound of the evaluator score. Furthermore, the theorem's assumption that E(z,x)=p(x) is constant for all z is inconsistent with Algorithm 1, where z is sampled stochastically from πs(z|x). Please revise the theory to match the actual algorithm or remove the corollary.
minor comments (6)
  1. [4.3] Several cross-references in Section 4.3 are inconsistent with the figure captions: the interaction-strategy results are in Figure 3, while the different weak/strong model results are in Figure 5; please correct the in-text references.
  2. [3.3.2 and Appendix B.2] The DPO scaling parameter α in Eq. (2) is not reported in Appendix B.2; please add the value and the range considered.
  3. [3.3.2] In the DPO loss, the sampling notation y+∼pw(·|z,x) should be y+∼p+(·|z,x) to match the definitions of p+ and p− in Section 3.3.1.
  4. [3.3.1] The evaluator is described as "external" in Section 3.3.1, but it is the same model as the strong model; please rephrase to "external to the weak model" or similar to avoid confusion.
  5. [Figure 7] Figure 7 contains a typo ("corrct") in the rubric; please fix it.
  6. [Appendix B.1] Appendix B.1 states that a 2K subset of Prosocial-Dialog is used for testing, but Table 2 lists 25K as the testing size; please clarify in the main text that the reported Ethics results are on the 2K subset.

Circularity Check

1 steps flagged · score 4.0 of 10

Theoretical Insight (Section 3.5) restates the preference-labeling rule by construction; the headline empirical claims are judged on external EM/F1/accuracy and are not circular.

  1. self definitional [Section 3.5, Theoretical Insight; labels defined in Section 3.3.1, Eq. (2) DPO objective.]
    "For simplicity, we assume that the evaluator scores for the strong model’s outputs are constant for all z, i.e. E(z,x) = p(x) for all z when given x. ... As a result, the optimized weak model π∗w allocates zero probability to any output y that results in an evaluator score E(πs◦y,x) ≤ p(x)."

    In Section 3.3.1, y− is designated whenever E(πs◦y,x) ≤ E(z,x) (with Δ = E(πs◦y,x) − E(z,x) ≤ 0). The theory then assumes E(z,x) = p(x), so the set of outputs the 'theorem' says the optimized model avoids is exactly the set of outputs labeled y− in the preference data. The DPO objective in Eq. (2) is trained to downweight those very y−, so the conclusion π∗w(y|x)=0 for E(πs◦y,x) ≤ p(x) is a restatement of the labeling-plus-training procedure, not an independent derivation. Corollary 1 repeats the same equivalence. It does not by itself establish that alignment improves external metrics.

full rationale

The circularity is localized to the theoretical-insight subsection. The empirical headline—collaboration outperforms each model alone, and preference alignment adds gains—is evaluated with external Exact Match, F1, and Accuracy in Table 1 and Figure 3, so those numbers are not forced by the preference construction. There is no load-bearing self-citation chain: DPO and the baseline methods are standard external references. The same-model evaluator issue (GPT-4 as both strong model and evaluator) is a validity/bias concern about the training signal, not a reduction of the final metric to the training objective; therefore it is not counted as an additional circular step. Because one claimed 'theoretical insight' is definitionally equivalent to the preference-labeling rule, the paper is partially circular, but its central empirical claim retains independent content.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The framework leans on standard supervised fine-tuning and DPO, so the main axioms are domain assumptions about evaluator reliability. There are no fitted science constants; the free parameters listed are standard hyperparameters whose values are reported. The theoretical claim in Section 3.5 depends on an explicit constancy assumption that is likely false in practice.

free parameters (6)
  • DPO scaling parameter alpha = not reported in text
    Controls the strength of preference optimization in Equation (2); chosen by hand and not swept.
  • Evaluator rubric anchors = 1, 3, 6, 10
    The GPT-4 evaluator prompt in Figure 7 defines quality anchors by hand; these affect which drafts become positive versus negative examples.
  • Weak-model samples per query K = 5
    Algorithm 1 samples K candidate drafts per training query to build preference pairs; affects balance of Y+ and Y-.
  • Preference dataset size = 2,000 IfQA; 5,000 MedMCQA and Prosocial
    Amount of DPO training data per domain; Figure 4 shows performance varies with this choice.
  • LoRA rank and alpha = 16, 16
    Used for both SFT and DPO stages; standard but hand-chosen.
  • Learning rates = 1.4e-5 SFT; 1.41e-5 DPO
    Reported hyperparameters; not optimized in the paper.
assumptions (5)
  • domain assumption Evaluator scores E(y,x) are a valid proxy for output quality
    Preference pairs in Section 3.3.1 are labeled purely by GPT-4 scores on a 1-10 rubric; no human validation is reported.
  • domain assumption Using the same LLM as strong model for evaluation aligns preferences
    Stated in Section 3.3.1; this conflates the strong model's preference with correctness and is not independently verified.
  • standard math DPO/Bradley-Terry preference model applies to the collaborative preference pairs
    The DPO loss in Section 3.3.2 assumes pairwise preferences are generated by a Bradley-Terry model; no check is given that this holds for y+ and y- constructed by thresholding evaluator scores.
  • ad hoc to paper For the theory, strong model evaluator scores are constant: E(z,x)=p(x)
    Explicitly assumed in Section 3.5 "for simplicity"; it is implausible in practice and is the basis for the zero-probability claim.
  • ad hoc to paper Corollary assumes strong responses are bounded below by p(z)<=E(z,x)
    Second assumption in Section 3.5 used to extend the zero-probability claim; not proven and likely false for arbitrary z.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Synergistic Weak-Strong Collaboration by Aligning Preferences." pith.science (2026). https://pith.science/paper/3I734YKJ

@misc{pith2026250415188,
  author       = {Pith},
  title        = {Pith review of: Synergistic Weak-Strong Collaboration by Aligning Preferences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3I734YKJ}},
  note         = {Machine review of arXiv:2504.15188}
}
read the original abstract

Current Large Language Models (LLMs) excel in general reasoning yet struggle with specialized tasks requiring proprietary or domain-specific knowledge. Fine-tuning large models for every niche application is often infeasible due to black-box constraints and high computational overhead. To address this, we propose a collaborative framework that pairs a specialized weak model with a general strong model. The weak model, tailored to specific domains, produces initial drafts and background information, while the strong model leverages its advanced reasoning to refine these drafts, extending LLMs' capabilities to critical yet specialized tasks. To optimize this collaboration, we introduce a collaborative feedback to fine-tunes the weak model, which quantifies the influence of the weak model's contributions in the collaboration procedure and establishes preference pairs to guide preference tuning of the weak model. We validate our framework through experiments on three domains. We find that the collaboration significantly outperforms each model alone by leveraging complementary strengths. Moreover, aligning the weak model with the collaborative preference further enhances overall performance.

Figures

Figures reproduced from arXiv: 2504.15188 by the authors.

Figure 1
Figure 1. Comparison of our method and the related [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Aligning the Weak Model with Strong Model [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Analysis of different interaction strategies between weak-strong models in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Analysis of different sizes of training data for [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Analysis of adopting different weak or strong [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The task example of three datasets [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: The prompts used in our framework [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Case Study [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 8 canonical work pages

  1. [1]

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

    arXiv 2024

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  3. [3]

    Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, and Jeffrey Wu. 2024. https://openreview.net/forum?id=ghNRg2mEgN Weak-to-strong generalization: Eliciting strong capabilities with weak supervision . In Forty-first International Conferenc...

    work page 2024

  4. [4]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1--45

    2024

  5. [5]

    Moses Charikar, Chirag Pabbaraju, and Kirankumar Shiragur. 2024. https://doi.org/10.48550/ARXIV.2405.15116 Quantifying the gain in weak-to-strong generalization . CoRR, abs/2405.15116

    arXiv doi:10.48550/arxiv.2405.15116 2024

  6. [6]

    Haikang Deng and Colin Raffel. 2023. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.721 Reward-augmented decoding: Efficient controlled text generation with a unidirectional reward model . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023 , pages 11781--11791. Association for...

    doi:10.18653/v1/2023.emnlp-main.721 2023

  7. [7]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  8. [8]

    William Fedus, Barret Zoph, and Noam Shazeer. 2022. https://jmlr.org/papers/v23/21-0998.html Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity . J. Mach. Learn. Res., 23:120:1--120:39

    work page 2022

Show all 52 references
  1. [9]

    Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. 2023. Specializing smaller language models towards multi-step reasoning. In International Conference on Machine Learning, pages 10421--10430. PMLR

    2023

  2. [10]

    Yue Guo and Yi Yang. 2024. Improving weak-to-strong generalization with reliability-aware alignment. arXiv preprint arXiv:2406.19032

    2024 arXiv

  3. [11]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming - Wei Chang. 2020. http://proceedings.mlr.press/v119/guu20a.html Retrieval augmented language model pre-training . In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020,...

    2020

  4. [12]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. 2021. https://arxiv.org/abs/2106.09685 Lora: Low-rank adaptation of large language models . CoRR, abs/2106.09685

    2021 arXiv

  5. [13]

    Gautier Izacard, Patrick S. H. Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi - Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. https://doi.org/10.48550/ARXIV.2208.03299 Few-shot learning with retrieval augmented language models . C...

    2022 doi arXiv

  6. [14]

    Jiaming Ji, Boyuan Chen, Hantao Lou, Donghai Hong, Borong Zhang, Xuehai Pan, Juntao Dai, and Yaodong Yang. 2024. Aligner: Achieving efficient alignment through weak-to-strong correction. arXiv preprint arXiv:2402.02416

    2024 arXiv

  7. [15]

    Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, L \' e lio Renard Lavaud, Lucile Saulnier, Marie...

    2024 doi arXiv

  8. [16]

    Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. 2023 a . https://doi.org/10.18653/V1/2023.ACL-LONG.792 Llm-blender: Ensembling large language models with pairwise ranking and generative fusion . In Proceedings of the 61st Annual Meeting of the Association for Computational Lingu...

    2023 doi

  9. [17]

    Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi - Yu, Yiming Yang, Jamie Callan, and Graham Neubig

    Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi - Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023 b . https://doi.org/10.18653/V1/2023.EMNLP-MAIN.495 Active retrieval augmented generation . In Proceedings of the 2023 Conference on Empirical Me...

    2023 doi

  10. [18]

    Gurusha Juneja, Subhabrata Dutta, Soumen Chakrabarti, Sunny Manchanda, and Tanmoy Chakraborty. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.225 Small language models fine-tuned to coordinate larger language models improve complex reasoning . In Proceedings of the 2023 Con...

    2023 doi

  11. [19]

    Hyunwoo Kim, Youngjae Yu, Liwei Jiang, Ximing Lu, Daniel Khashabi, Gunhee Kim, Yejin Choi, and Maarten Sap. 2022. Prosocialdialog: A prosocial backbone for conversational agents. In EMNLP

    2022

  12. [20]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199--22213

    2022

  13. [21]

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. 2024. https://openreview.net/forum?id=uydQ2W41KO RLAIF vs. RLHF: scaling reinforcement learning from human fe...

    2024

  14. [22]

    Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. 2022. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Syste...

    2022

  15. [23]

    An Liu, Zonghan Yang, Zhenhe Zhang, Qingyuan Hu, Peng Li, Ming Yan, Ji Zhang, Fei Huang, and Yang Liu. 2024. Panda: Preference adaptation for enhancing domain-specific abilities of llms. arXiv preprint arXiv:2402.12835

    2024 arXiv

  16. [24]

    Jinliang Lu, Ziliang Pang, Min Xiao, Yaochen Zhu, Rui Xia, and Jiajun Zhang. 2024. Merge, ensemble, and cooperate! a survey on collaborative strategies in the era of large language models. arXiv preprint arXiv:2407.06089

    2024 arXiv

  17. [25]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36

    2024

  18. [26]

    Sean O'Brien and Mike Lewis. 2023. https://doi.org/10.48550/ARXIV.2309.09117 Contrastive decoding improves reasoning in large language models . CoRR, abs/2309.09117

    2023 doi arXiv

  19. [27]

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. https://proceedings.mlr.press/v174/pal22a.html Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering . In Proceedings of the Conference on Health, Inference, and Lea...

    2022

  20. [28]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html Direct preference optimization: Your language model is sec...

    2023

  21. [29]

    Devendra Singh Sachan, Mike Lewis, Dani Yogatama, Luke Zettlemoyer, Joelle Pineau, and Manzil Zaheer. 2023. Questions are all you need to train a dense passage retriever. Transactions of the Association for Computational Linguistics, 11:600--616

    2023

  22. [30]

    Shannon Zejiang Shen, Hunter Lang, Bailin Wang, Yoon Kim, and David Sontag. 2024. Learning to decode collaboratively with multiple language models. arXiv preprint arXiv:2403.03870

    2024 arXiv

  23. [31]

    Tal Shnitzer, Anthony Ou, M \' rian Silva, Kate Soule, Yuekai Sun, Justin Solomon, Neil Thompson, and Mikhail Yurochkin. 2023. https://doi.org/10.48550/ARXIV.2309.15789 Large language model routing with benchmark datasets . CoRR, abs/2309.15789

    2023 doi arXiv

  24. [32]

    KV Srivatsa, Kaushal Kumar Maurya, and Ekaterina Kochmar. 2024. Harnessing the power of multiple minds: Lessons learned from llm routing. arXiv preprint arXiv:2405.00467

    2024 arXiv

  25. [33]

    Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. 2023. https://openreview.net/forum?id=-cqvvvb-NkI Recitation-augmented language models . In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net

    2023

  26. [34]

    Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, and Chuang Gan. 2024. Easy-to-hard generalization: Scalable alignment beyond human supervision. arXiv preprint arXiv:2403.09472

    2024 arXiv

  27. [35]

    Szymanski and Michael D

    Peter T. Szymanski and Michael D. Lemmon. 1993. https://doi.org/10.1109/ICNN.1993.298760 Adaptive mixtures of local experts are source coding solutions . In Proceedings of International Conference on Neural Networks (ICNN'88), San Francisco, CA, USA, March 28 - April 1, 1993, ...

    1993

  28. [36]

    Leitian Tao and Yixuan Li. 2024. Your weak llm is secretly a strong teacher for alignment. arXiv preprint arXiv:2409.08813

    2024 arXiv

  29. [37]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

    2023 arXiv

  30. [38]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  31. [39]

    Yile Wang, Peng Li, Maosong Sun, and Yang Liu. 2023. Self-knowledge guided retrieval augmentation for large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10303--10315

    2023

  32. [40]

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022 a . Emergent abilities of large language models. arXiv preprint arXiv:2206.07682

    2022 arXiv

  33. [41]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 b . Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  34. [42]

    Canwen Xu, Yichong Xu, Shuohang Wang, Yang Liu, Chenguang Zhu, and Julian J. McAuley. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-ACL.18 Small models are valuable plug-ins for large language models . In Findings of the Association for Computational Linguistics, ACL 2024, B...

    2024 doi

  35. [43]

    Yuqing Yang, Yan Ma, and Pengfei Liu. 2024. https://doi.org/10.48550/ARXIV.2407.13647 Weak-to-strong reasoning . CoRR, abs/2407.13647

    2024 doi arXiv

  36. [44]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36

    2024

  37. [45]

    Wenhao Yu, Meng Jiang, Peter Clark, and Ashish Sabharwal. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.515 I f QA : A dataset for open-domain question answering under counterfactual presuppositions . In Proceedings of the 2023 Conference on Empirical Methods in Natural La...

    2023 doi

  38. [46]

    Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. 2024 a . Tinyllama: An open-source small language model. arXiv preprint arXiv:2401.02385

    2024 arXiv

  39. [47]

    Zihan Zhang, Meng Fang, and Ling Chen. 2024 b . https://doi.org/10.18653/V1/2024.FINDINGS-ACL.415 Retrievalqa: Assessing adaptive retrieval-augmented generation for short-form open-domain question answering . In Findings of the Association for Computational Linguistics, ACL 20...

    2024 doi

  40. [48]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

    2023 arXiv

  41. [49]

    Chujie Zheng, Ziqi Wang, Heng Ji, Minlie Huang, and Nanyun Peng. 2024. Weak-to-strong extrapolation expedites alignment. arXiv preprint arXiv:2404.16792

    2024 arXiv

  42. [50]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595--46623

    2023

  43. [51]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.