{"id":"b63a5eb1-09bd-406f-8fe5-a76030ce380a","arxiv_id":"2505.07460","paper_version":1,"verdict":"UNVERDICTED","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey that classifies LLM-SLM collaboration into pipeline, routing, auxiliary, distillation, and fusion mechanisms, and lists challenges and future directions.","lead":"This paper surveys ways that large and small language models can work together, grouping the methods into five categories. It is a research summary, not a new experiment or derivation.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The §3.1 five-way taxonomy is not a faithful classification: it mixes training-time distillation with inference-time interaction and includes non-collaborative single-model architectures (Hymba, MLLMs), undercutting the survey's central claim of a comprehensive overview.","rationale":"The reader identifies the §3.1 taxonomy and the unsourced quantitative claims as the weakest assumptions. This stress-test focuses on the taxonomy, which is the more load-bearing issue for the central claim of a comprehensive, usable overview. The paper explicitly states the five-way classification, but the examples used to instantiate it include systems that are not LLM-SLM collaboration (Hymba, MLLMs), categories that overlap in practice (CoGenesis, Collab-RAG), and a category (knowledge distillation) that operates at a different stage of the model lifecycle. These are not merely presentation choices; they affect whether a reader can reliably use the survey to understand and compare collaboration mechanisms. The unsourced numbers in §2.3 are a secondary concern: they can be corrected or removed without changing the taxonomy, whereas the taxonomy defines the paper's contribution. Because the central organizational claim is not supported as stated, the survey in its current form cannot be accepted as a reliable guide to the field. The appropriate verdict is REJECT, though a revised version that redefines the categories, removes non-collaborative examples, and sources quantitative claims could become a useful survey.","tokens_in":26623,"tokens_out":4808,"duration_ms":46477,"concrete_test":"Construct a classification matrix: take every framework cited in §3.1 (CoGenesis, CITER, HybridLLM, Collab-RAG, Hymba, MLLMs, and others) and apply the paper's own definitions from §3.1.1–3.1.5. Count how many systems fit exactly one category, more than one category, or none; specifically, verify whether Hymba and MLLMs involve an LLM and an SLM cooperating. If a substantial fraction (e.g., more than 30%) of the cited systems are multi-category or non-collaborative, the taxonomy is not a faithful classification. Additionally, consult the original papers for Hymba and for multimodal LLMs to confirm they are single-model architectures; if so, the integration/fusion category needs redefinition or removal.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that LLM-SLM collaboration 'can be classified into pipeline, hybrid/routing, auxiliary/enhancement, knowledge distillation-driven, and integration/fusion collaboration' (§3.1). This taxonomy is load-bearing because the survey's value is its organization of the field. The taxonomy is not coherent. First, the categories are not mutually exclusive: CoGenesis is presented as pipeline (§3.1.1) but also as a communication-protocol example (§3.2.2); Collab-RAG is classified as auxiliary (§3.1.3) yet is functionally pipeline (SLM decomposes queries, LLM reads the retrieved evidence); CITER's token-level routing could also be described as auxiliary assistance. Second, at least two 'integration/fusion' examples are not LLM-SLM collaboration at all: Hymba is a single small model with a hybrid attention-SSM head (§3.1.5), and multimodal LLMs are single models with multiple input modalities. Third, knowledge distillation is a training-time transfer technique, not an inference-time interaction mechanism, so grouping it with pipeline and routing conflates two different meanings of 'collaboration.' Since the taxonomy is the paper's main contribution, these issues undermine the claim of a comprehensive overview. The unsourced quantitative claims in §2.3 (e.g., 'GPT-4 achieves over 90% accuracy', 'TinyBERT achieves 96% of BERT's performance', energy and cost ratios) reinforce the reliability concern, but the taxonomy problem is the more fundamental one.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper is a survey of mechanisms for collaboration between large language models (LLMs) and small language models (SLMs). It proposes a five-way taxonomy of collaboration modes—pipeline, hybrid/routing, auxiliary/enhancement, knowledge-distillation-driven, and integration/fusion—then describes enabling technologies (task routing, inter-model communication, model fusion, state synchronization, resource scheduling), surveys application scenarios organized around on-device requirements (latency, privacy, personalization, offline operation, energy), and closes with challenges and future directions. The paper is entirely a literature review; it contains no new experiments, derivations, or formal results.","tokens_in":26895,"tokens_out":6915,"duration_ms":59508,"significance":"If the taxonomy and the reported system facts were reliable, the survey would be a useful entry point: it collects a broad set of recent systems (CITER, CoGenesis, Collab-RAG, HybridLLM, FedCoLLM, CE-CoLLM, and others) and organizes them under a small number of intuitive headings, and the application-scenario chapters connect the collaboration literature to concrete deployment constraints. The paper also deserves credit for explicitly naming evaluation, consistency, and security as open problems. However, the value of the survey depends on the classification being coherent and on the reported quantitative claims being traceable; both conditions are currently not met, so the contribution is not yet publishable in its present form. The paper does ship a useful bibliography and cites many relevant works, but the central organizing device needs substantial reworking.","major_comments":[{"comment":"The five-way taxonomy is presented without exclusion rules, and the paper's own examples cross categories. CoGenesis is treated as pipeline collaboration in §3.1.1 and again as an example of structured intermediate representation in §3.2.2; Collab-RAG is classified as auxiliary/enhancement in §3.1.3 even though its described operation—SLM decomposes user queries and the LLM reads the retrieved evidence—is sequential and resembles the pipeline definition in §3.1.1. The manuscript should state whether the modes are mutually exclusive, whether a single system can instantiate several modes, and which criterion (information flow, purpose, or training/inference phase) defines each mode. Without this, the taxonomy cannot be applied to new systems and the 'comprehensive overview' claim is weakened.","section":"§3.1.1, §3.1.3, §3.2.2"},{"comment":"The integration/fusion category includes examples that are not LLM-SLM collaboration under the paper's own definition in §1.2. Hymba is a single small language model with a hybrid attention-SSM head, and multimodal LLMs are single models processing multiple input modalities; neither involves a large model and a small model working together. These examples should either be removed or explicitly presented as adjacent techniques with a reasoned justification for their inclusion. As written, they make the survey's central claim to cover LLM-SLM collaboration internally inconsistent.","section":"§3.1.5"},{"comment":"Knowledge distillation is a training-time transfer method, whereas pipeline, routing, and auxiliary/enhancement are inference-time interaction patterns. Placing them in one taxonomy conflates two different senses of 'collaboration.' The abstract and §3.1 describe all five as 'interaction mechanisms,' and §2.3.3 separately lists distillation as a way small models are constructed. The authors should separate training-time and inference-time mechanisms or explicitly redefine the scope of 'collaboration' to include both, explaining why a single taxonomy is appropriate.","section":"§3.1.4"},{"comment":"Multiple quantitative performance and cost claims are stated without citations or units. Examples include 'GPT-4 achieves over 90% accuracy on complex tasks like mathematical reasoning and code generation' (§2.3.2), 'single inference cost is about 0.1−1' (§2.3.2, no units), 'TinyBERT achieves 96% of BERT's performance on the GLUE benchmark with 75% fewer parameters' (§2.3.3), and 'GPT-4 generating 1000 words consumes about 500 times the energy of TinyBERT' (§2.3.5). A survey's factual claims must be traceable to the cited literature; otherwise readers cannot distinguish the authors' estimates from established results. These claims need either proper citations or explicit labeling as illustrative estimates.","section":"§2.3.2, §2.3.3, §2.3.5"}],"minor_comments":[{"comment":"Raw BibTeX keys 'raffel2020exploring' and 'wei2022emergent' appear in the text and are not present in the reference list; these should be expanded into full citations.","section":"§2.1.1"},{"comment":"The phrase 'single inference cost is about 0.1−1' lacks both a unit and a complete range; it should read, for example, '0.1–1 US dollars per inference' with an appropriate citation.","section":"§2.3.2"},{"comment":"The terminology is inconsistent between the abstract, which says 'pipeline, routing, auxiliary, distillation, fusion,' and §3.1, which uses 'hybrid/routing' and 'auxiliary/enhancement.' Use one set of labels consistently throughout.","section":"§3.1"},{"comment":"The section is titled 'Evaluation Metrics and Benchmarking,' but much of its content concerns LLMOps monitoring, deployment, and debugging; the section should be reorganized so that evaluation metrics and benchmark construction are actually discussed, and repeated observations should be consolidated.","section":"§5.3"},{"comment":"Several references have incomplete metadata: 'Kumar (2025)' lists the venue as 'In.', and 'Xu, Chenning et al. (n.d.)' has no year or venue. These entries should be completed before publication.","section":"References"},{"comment":"The survey claims to be 'comprehensive' but does not describe its search strategy, inclusion criteria, or coverage window. A short methodology paragraph would help readers assess how complete the coverage is.","section":"Introduction"}],"recommendation":"major_revision","confidential_remarks":"The survey overlaps substantially with recent surveys already cited by the authors, particularly Gao, Zhu, and Liu (2025) and Niu et al. (2025). The editor may wish to ask the authors to state explicitly what this survey adds beyond those works. In addition, some quantitative claims are supported only by industry blog posts or press releases (e.g., Aizip, Arcee AI, Arize AI); this should be flagged in the published version so that readers can calibrate the evidence level."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on the survey by Chen et al. (arXiv:2505.07460). The paper's value is as a pointer to recent work: it rounds up CITER, CoGenesis, HybridLLM, Collab-RAG, FedCoLLM, and other systems, and organizes them under five collaboration modes with five application areas. That organization is the main contribution, and it is plausible at first glance. If you are new to the area, this gives you a decent reading list and a vocabulary for talking about routing, pipelining, distillation, and fusion. The application chapters (privacy, offline, energy-constrained) are a nice way to frame why collaboration matters on-device.\n\nThe soft spots, though, are real. The stress-test note is right: the taxonomy is not coherent. Knowledge distillation is a training-time transfer technique, not an inference-time interaction mechanism, so grouping it with pipeline and routing conflates two different meanings of 'collaboration.' And the 'integration/fusion' category includes Hymba (a single hybrid-head SLM) and multimodal LLMs, which are not LLM-SLM collaboration at all. That means the five-way classification is not a faithful map of the field. Also, several quantitative claims in §2.3 appear without citations ('GPT-4 achieves over 90% accuracy', 'TinyBERT achieves 96% of BERT's performance', cost/energy ratios, latency numbers). They might be true, but an unsourced number in a survey is a reliability problem, especially when the survey's job is to be a trustworthy entry point. There are also unfinished citation keys in the text (raffel2020exploring, wei2022emergent) and an unresolved 'C. Xu et al., n.d.' in §5.2.1. These are fixable, but they add to the impression of a hastily assembled manuscript.\n\nIs it worth a serious referee? I think yes, but with the expectation of major revision. The paper's material is useful enough that a careful referee could guide the authors to tighten the taxonomy (either narrow it to inference-time interaction, or explicitly separate training-time distillation from inference-time collaboration) and to either cite or delete the unsourced numbers. The current version's central claim of a comprehensive overview is overstated, but the underlying survey work is not wasted. I would send it to peer review with a request for major revision, not desk-reject it. I would not cite it in my own work yet; I'd cite the primary systems instead.","headline":"A useful but flawed survey of LLM-SLM collaboration: good collection of recent systems, but the five-mode taxonomy mixes training-time and inference-time mechanisms and includes single-model architectures, which undercuts the 'comprehensive' claim.","tokens_in":27459,"tokens_out":3250,"would_cite":false,"duration_ms":26587,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Large and small language models can work together in five distinct modes, and a new survey maps the field and its open problems.","keywords":["large language models","small language models","LLM-SLM collaboration","model routing","knowledge distillation","model fusion","edge computing","on-device AI"],"falsifier":"Sample a set of published LLM-SLM collaboration systems and check whether each cleanly fits one of the five modes; a substantial share of systems that span modes or fit none would falsify the taxonomy. Equally, re-running the cited distillation or routing experiments on a standard benchmark (such as replicating TinyBERT's GLUE score) would settle whether the reported numbers hold.","tokens_in":26391,"feed_emoji":"🤝","tokens_out":6517,"duration_ms":53972,"temperature":0.7,"pith_summary":"This paper is a survey that tries to establish a systematic account of how large language models and small language models can work together, rather than treating that collaboration as a scattered set of tricks. Its central claim is that LLM-SLM collaboration is a coherent paradigm with a stable taxonomy, divisible into five interaction modes: pipeline, hybrid/routing, auxiliary/enhancement, knowledge-distillation-driven, and integration/fusion. A sympathetic reader would care because the survey connects each mode to concrete frameworks, enabling technologies, and on-device applications, and argues that this division of labor is the practical path to low-latency, private, personalized, offline, and energy-efficient AI. It also identifies the open problems—routing overhead, inter-model consistency, task-allocation robustness, evaluation gaps, and security—that the field must solve for the paradigm to deliver on that promise.","feed_headline":"Five modes map how large and small language models team up","feed_subtitle":"Pipeline, routing, auxiliary, distillation, and fusion designs could put capable AI on phones and edge devices.","key_machinery":"The load-bearing object is the five-mode classification itself, defined by how information flows between models. In pipeline mode, one model's output feeds the next; in hybrid/routing mode, a router decides which model handles each query or token; in auxiliary/enhancement mode, one model assists the other with sub-problems, context, or feedback; in knowledge-distillation-driven mode, a teacher large model transfers its knowledge into a small student; and in integration/fusion mode, architectures, parameters, or outputs are combined into one system. The survey supports this taxonomy with named mechanisms—token-level routers such as CITER, structured 'sketch' interfaces from CoGenesis, logits and probability sharing, parameter-efficient fine-tuning, and model fusion or merging techniques—showing that each mode has a distinct engineering vocabulary.","core_discovery":"On its own terms, the paper's contribution is a classification: collaboration between large and small language models can be organized into pipeline, hybrid/routing, auxiliary/enhancement, knowledge-distillation-driven, and integration/fusion modes. It then attaches to each mode representative frameworks (e.g., CoGenesis for pipeline, CITER for token-level routing, Collab-RAG for auxiliary, Hymba for fusion) and identifies cross-cutting enabling technologies: task allocation and routing, inter-model communication and interface design, model fusion and result integration, state synchronization and context management, and dynamic resource scheduling. The paper further claims these mechanisms are driven by on-device needs—low latency, privacy, personalization, offline operation, and energy limits—and that the field still faces unresolved problems in overhead, consistency, task-allocation robustness, evaluation, and security.","pith_inferences":["The taxonomy is probably a simplification: real systems often blend modes (a router that also distills, or a pipeline with fusion), so a combined-mode ontology would be a natural testable extension.","The 'router latency paradox' the survey identifies implies a concrete experiment: measure whether router overhead is negligible compared to the LLM-SLM inference gap; if not, the economic case for routing weakens.","Because the survey's usefulness rests partly on cited numbers, reproducing those figures (e.g., 96% GLUE retention, 40% latency cut, 90% intent accuracy) on a standard suite would test the strength of the paradigm.","The same five-mode lens may transfer to multimodal and embodied settings, where specialized small encoders feeding a central large model look like auxiliary or fusion modes, but the categories would likely need refinement as those fields mature."],"forward_implications":["On-device AI becomes a realistic target: a local small model handles privacy- and latency-critical front-end work while a cloud large model handles deep reasoning, a pattern already deployed in products like on-device keyboard prediction.","Token- or query-level routing can cut both cost and latency by sending only hard cases to the large model; cited frameworks report end-to-end latency reductions of up to 40% in edge-cloud settings.","Knowledge distillation lets small models capture a large fraction of large-model ability, with the survey citing TinyBERT reaching 96% of BERT's GLUE performance with far fewer parameters, making capable AI deployable on constrained hardware.","The field lacks standardized evaluation, so future progress depends on building benchmarks that jointly measure output quality, efficiency, and cost rather than quality alone."],"supporting_citations":[{"why":"Supplies the definition of pipeline collaboration and the general LLM-SLM collaboration scope that the survey's taxonomy builds on.","marker":"F. Wang, L. Zhang, and Jian Hu, 2024"},{"why":"Prior survey establishing SLMs as front-end processors and framing collaboration as a hybrid ecosystem; the present paper extends its classification.","marker":"Gao, K. Zhu, and Z. Liu, 2025"},{"why":"CITER framework exemplifies token-level routing collaboration, showing a router can distribute generation between SLM and LLM.","marker":"W. Zheng et al., 2025"},{"why":"Collab-RAG is the paper's main example of auxiliary collaboration, with an SLM decomposing queries to improve LLM reasoning.","marker":"R. Xu et al., 2025"},{"why":"Survey of knowledge distillation methods, providing the foundation for the knowledge-distillation-driven collaboration mode.","marker":"X. Xu et al., 2024"},{"why":"Hymba hybrid architecture integrates attention and state-space models, illustrating the integration/fusion collaboration mode.","marker":"X. Dong et al., 2024"},{"why":"CoGenesis framework demonstrates privacy-preserving pipeline collaboration and the structured 'sketch' communication interface.","marker":"Kaiyan Zhang, Jianyu Wang, Hua, et al., 2024"},{"why":"Extended survey of routing strategies, used to support task allocation and intelligent routing as a key enabling technology.","marker":"Varangot-Reille et al., 2025"}],"fun_headline_variants":["Five modes for large-small language model collaboration","LLM-SLM teamwork: five mechanisms in a new survey","How big and small AI models can work together","Large and small language models: five collaboration patterns","Survey identifies five modes for LLM-SLM cooperation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The survey's central value depends on the assumption that its five-way classification genuinely reflects how LLM-SLM collaboration systems are built, and that the quantitative performance claims it cites (for instance, TinyBERT achieving 96% of BERT's GLUE performance) are accurate as reported.","fun_headline_variants_meta":{"raw":{"variants":["Five modes for large-small language model collaboration","LLM-SLM teamwork: five mechanisms in a new survey","How big and small AI models can work together","Large and small language models: five collaboration patterns","Survey identifies five modes for LLM-SLM cooperation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000251,"raw_usage":{"total_tokens":1531,"prompt_tokens":895,"completion_tokens":636,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":562}},"tokens_in":511,"tokens_out":636,"duration_ms":6067,"temperature":1.0,"reasoning_tokens":562,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:15:22.420695+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Sample a set of published LLM-SLM collaboration systems and check whether each cleanly fits one of the five modes; a substantial share of systems that span modes or fit none would falsify the taxonomy. Equally, re-running the cited distillation or routing experiments on a standard benchmark (such as replicating TinyBERT's GLUE score) would settle whether the reported numbers hold.","supporting_citations":[],"review_version":1}