{"id":"c61af60e-2d29-4e54-a478-6e999bf07036","arxiv_id":"2505.02309","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":1.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A survey of knowledge distillation, quantization, and pruning for compressing LLMs to run on resource-constrained edge devices, with no new experimental results.","lead":"This paper surveys techniques for making large language models smaller and faster for phones and other low-power devices. It reviews knowledge distillation, quantization, and pruning, and compares their trade-offs.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unsupported 'first survey' claim is the load-bearing risk; the reference errors in Tables I-III are symptoms of the same reliability problem.","rationale":"The reader's weakest assumption is about citation accuracy; I partially agree, but I think the more load-bearing assumption is the novelty claim itself. The survey's contribution statement is the only place where the paper makes a non-organizational assertion. That assertion is externally falsifiable and is not supported by any evidence in the manuscript. The internal citation errors are evidence of the same weakness: if the authors cannot reliably map references, the claim to be a 'first focused survey' is not credible. However, I do not move the verdict: a conditional accept with a mandatory prior-art search and citation fix is appropriate. The taxonomy and method descriptions are standard and mostly correct, so the paper is salvageable after these corrections.","tokens_in":12835,"tokens_out":8235,"duration_ms":92339,"concrete_test":"Run a systematic prior-art search (arXiv, Google Scholar, DBLP, Scopus) for surveys combining 'large language model' with 'compression' and 'edge', 'on-device', or 'resource-constrained', with publication date before May 2025. If any such focused survey is found, the Section I firstness claim is false; if none is found, the novelty claim is provisionally supported and the remaining task is a full citation audit of Tables I-III.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a novelty claim: the paper says it is the first focused survey of LLM compression from a resource-constrained-environment lens. The manuscript offers no operational definition of that lens, no search protocol, and no comparison against prior surveys; worse, it mischaracterizes the closest prior work. Reference [9] is itself a survey of model compression for LLMs covering KD, quantization, and pruning, and [10] is a broad transformer-compression survey, yet Section I lumps them with [9], [10], [13], [32], [55] as works focusing on 'one or two compression techniques.' If an earlier edge-focused or resource-constrained LLM-compression survey exists, the headline contribution fails. The paper's own body does not demonstrate a distinct lens: Sections III-A through III-C are generic method summaries, and the only resource-constrained-specific artifact is the qualitative 'Edge Suitability' column in Table III. The reliability of the survey as a reference is further compromised by clear citation mismatches (Table II cites ZeroQuant V2 as [43], which is the Lottery Ticket paper, and BLOOM as [60], which is an N:M sparsity paper), and by a missing KD loss equation in Section III-A. These are correctable, but they are not cosmetic: the paper's stated value is being a comprehensive, trustworthy reference, and that value is not yet established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript is a survey of techniques for adapting large language models to resource-constrained (mobile and edge) environments. It reviews three primary compression families—knowledge distillation, model quantization, and model pruning—together with their variants and example applications, and briefly discusses complementary efficiency strategies such as mixture-of-experts and early-exit mechanisms. The paper also outlines future directions including newer distillation forms, FP8/NF4 representations, neural architecture search, outlier-aware quantization, and quantized finetuning. The authors state in Section I that this is, to their knowledge, the first focused survey of LLM compression from the perspective of resource-constrained environments. The survey contains three tables: examples of KD on LLMs, examples of model quantization, and a qualitative comparison of the three compression techniques.","tokens_in":13146,"tokens_out":8398,"duration_ms":89332,"significance":"If the novelty claim and the reference apparatus were reliable, this survey would be a useful organizational reference for practitioners and researchers entering the area. The paper is clearly structured, covers the major technique families, and usefully includes complementary strategies (MoE, early exit) and recent topics (FP8, NF4, quantized finetuning). The absence of machine-checked proofs or new technical results is expected for a survey; the contribution is expository. However, the paper's value as a trustworthy reference depends on the accuracy of its technical descriptions and citation pointers, and on the validity of its first-survey claim. Both currently need substantive attention before the paper can be recommended for publication.","major_comments":[{"comment":"The technical core of the knowledge-distillation subsection is incomplete. The text states 'The loss function becomes:' and then jumps to a list of symbol definitions, but the actual equation is absent. Because this equation defines the distillation loss and its temperature scaling, its omission leaves readers without a precise statement of one of the three main techniques. Please insert the full expression (e.g., L = α T² KL(softmax(z_T/T) ‖ softmax(z_S/T)) + (1−α) CE(y, softmax(z_S))) and verify the surrounding notation.","section":"Section III.A.1"},{"comment":"The ZeroQuant V2 row cites '[43]', but reference [43] is the Lottery Ticket Hypothesis paper, not the ZeroQuant-V2 paper; the correct bibliography entry appears to be [41]. The same row cites 'BLOOM Family (560M to 176B) [60]', but [60] is a paper on learned N:M sparsity, not the BLOOM model. These mismatches are not cosmetic: in a survey, the tables are the reference apparatus readers rely on, and incorrect pointers undermine the paper's reliability. Please correct every citation in Tables I–III and cross-check each one against the bibliography.","section":"Table II"},{"comment":"The claim that this is 'the first paper that provides a focused survey of LLM compression techniques from the lens of resource-constrained environments' is not currently supported. The manuscript does not define what the resource-constrained lens is operationally, does not report a search or screening protocol, and does not compare its scope against prior surveys. In addition, the sentence that lists prior work [9], [10], [13], [32], [55] as focusing on 'one or two compression techniques' mischaracterizes [9], which is itself a survey of model compression for LLMs covering knowledge distillation, quantization, and pruning. Please either substantiate the first-survey claim with a concrete lens definition and a comparison against existing surveys (including [9] and [10]), or revise the claim to state the paper's scope without asserting novelty.","section":"Section I"}],"minor_comments":[{"comment":"The passage 'However, reducing the precision even further requires more … framework to decompose the quantization of weights into ternary optimization problems' contains a broken sentence and no citation; the described method appears to be from [26] (Bit-Split), which is in the bibliography but never cited in the text.","section":"Section III.B.1"},{"comment":"The statement that LLaMa 3 405B does not fit on '8 Nvidia H100 GPUs (800 GB of combined memory)' is numerically incorrect for standard 80 GB H100 parts, which total 640 GB; please correct the memory figure or specify the exact GPU variant.","section":"Section I"},{"comment":"The DeepSeek-V3 memory requirement in Section I is cited to [62], which is a HuggingFace GGUF quantization page rather than the DeepSeek-V3 technical report; please cite a primary source for this figure.","section":"References"},{"comment":"The SmoothQuant entry [40] lists the first author as 'G. Xizao'; the correct name is Guangxuan Xiao, so the citation should read 'G. Xiao'.","section":"References"},{"comment":"The direction of the 'High' values is ambiguous: high is favorable for Memory Reduction and Computation Cost Reduction but unfavorable for Accuracy Degradation; add a note or a visual cue indicating the favorable direction in each column.","section":"Table III"},{"comment":"The phrase '2 FLOPS per active parameter per token' should read '2 FLOPs per active parameter per token', and the unit notation should be made consistent throughout the paper.","section":"Section II"}],"recommendation":"major_revision","confidential_remarks":"The editor may wish to independently verify the 'first survey' claim, since the manuscript provides no search protocol or systematic comparison with prior surveys; a quick check for earlier edge-focused or resource-constrained LLM compression surveys would be prudent."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent survey of LLM compression that practitioners will find handy, but it overclaims novelty and has enough citation sloppiness that I would not yet trust it as a reference. The taxonomy is sound: KD variants (soft-target, feature-based, relation-based, self, multi-teacher), quantization (PTQ, QAT, mixed-precision, binary/ternary), pruning (unstructured/structured), plus MoE and early-exit. The tables are useful, particularly the qualitative comparison in Table III. The coverage is good for someone new to the area.\n\nThe main problem is the stated novelty. The introduction claims this is the first focused survey of LLM compression from a resource-constrained-environment lens, but it does not define that lens, report a search protocol, or compare against prior surveys. Worse, it lumps references [9] and [10] — themselves broad surveys of LLM/transformer compression — into works focusing on 'one or two compression techniques.' That mischaracterization undercuts the 'first' claim. In the body, the resource-constrained lens mostly reduces to an 'Edge Suitability' column in Table III. I'd soften the claim or support it with an explicit comparison.\n\nThe citation errors are not cosmetic. Table II cites ZeroQuant V2 as [43], which is the Lottery Ticket paper, and BLOOM as [60], which is an N:M sparsity paper. The actual ZeroQuant-V2 is reference [41]; BLOOM is not in the bibliography at all. The KD loss equation is missing from Section III.A.1: the text says 'the loss function becomes:' and then only defines the variables. For a survey whose value is being a reliable reference, these are exactly the kinds of errors that make readers question every table cell. They are fixable, but they need to be fixed and cross-checked.\n\nOne minor factual quibble: the '2 FLOPS per active parameter per token' figure is attributed to [2] (the GPT-3 paper), but that paper does not state that. It's a reasonable estimate, not a citation.\n\nNone of this is fatal to the paper's purpose. The summaries are largely accurate, and the examples (GKD, MiniLLM, DistiLLM, GPTQ, SmoothQuant) are real and correctly described. The Pith Report's 'soundness 4' is about right. The paper deserves peer review, but it should come back with the citation mismatches fixed, the missing equation supplied, and the novelty claim reworded. With those changes, it is a solid entry-level resource. Without them, I'd hesitate to point a student at it.","headline":"A decent but sloppy survey: useful taxonomy, overstated novelty, and citation errors that need fixing before I'd trust it as a reference.","tokens_in":13598,"tokens_out":4050,"would_cite":false,"duration_ms":41529,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This survey frames LLM compression for edge deployment as three families: distillation, quantization, and pruning.","keywords":["Model compression","Large language models","Knowledge distillation","Quantization","Pruning","Early exit","Edge deployment","Resource-constrained environments"],"falsifier":"Locate a peer-reviewed survey published before this paper that explicitly frames LLM compression techniques around resource-constrained or edge deployment; the existence of such a survey would falsify the paper's claim to be the first focused treatment.","tokens_in":12716,"feed_emoji":"📱","tokens_out":8253,"duration_ms":93822,"temperature":0.7,"pith_summary":"This paper is a survey rather than a new algorithm. Its aim is to establish that the scattered work on making large language models small enough for phones, edge devices, and IoT hardware can be organized into three primary compression families—knowledge distillation, quantization, and pruning—and that a survey focused specifically on resource-constrained deployment is a first in the literature. The paper catalogs the variants inside each family, gives worked examples of each method applied to LLMs, and compares the families on five axes: memory reduction, computation reduction, accuracy degradation, hardware adaptability, and edge suitability. A reader should care because the usefulness of LLMs in everyday devices depends on exactly these trade-offs, and this survey is intended as the reference point for choosing among them.","feed_headline":"Three routes to fit LLMs on edge devices, compared","feed_subtitle":"Knowledge distillation, quantization, and pruning scored on memory, speed, accuracy, and hardware fit.","key_machinery":"The organizing device is a three-way taxonomy of compression families—distillation, quantization, pruning—each with named variants: soft-target, feature-based, relation-based, self-distillation, and multi-teacher distillation; post-training quantization, quantization-aware training, mixed-precision, and binary or ternary quantization; and structured and unstructured pruning. Wrapped around the taxonomy are two tables that collect published LLM compression examples and a comparison table that scores the families on memory reduction, computation reduction, accuracy degradation, hardware adaptability, and edge suitability. The tables and the five trade-off axes do the argumentative work: they turn a list of techniques into a decision-oriented map for someone deploying LLMs under resource constraints.","core_discovery":"The paper's central claim is that the field of LLM compression is best understood through three primary techniques with distinct trade-off profiles. Knowledge distillation transfers behavior from a large teacher to a small student; quantization cuts the bit-width of weights and activations; pruning removes low-importance parameters, heads, or layers. The survey then asserts that looking at these techniques from the explicit standpoint of mobile, edge, and IoT deployment—rather than from general model-efficiency or single-technique viewpoints—yields a new and practically oriented organizing frame. It supports that frame with tables of representative LLM compression results and a qualitative comparison of the three families on memory, computation, accuracy, hardware adaptability, and edge suitability. The discovery, if accepted, is not a new compression method but a structured map of the existing ones from the perspective of deployment constraints.","pith_inferences":["Editorial inference: the five qualitative comparison axes could be turned into a quantitative decision tool by measuring latency, energy, and accuracy for each family on representative mobile and edge hardware; the survey identifies the axes but leaves the measurement to future work.","Editorial inference: if the edge-suitability ranking is correct, the strongest practical deployments will combine structured pruning with low-bit quantization, because that pairing avoids the sparse-matrix penalty on dense accelerators; the survey lists these ingredients without making the combination explicit.","Editorial inference: the same three compression families should transfer to vision-language and multimodal models headed to edge devices, where the teacher-student, bit-width, and sparsity trade-offs are likely to reappear; this is a natural extension the survey does not develop."],"forward_implications":["A practitioner choosing a compression method can use the survey's comparison to narrow options: quantization offers the largest memory and compute reductions with strong hardware adaptability, while distillation is the option most likely to improve a small student model's accuracy.","Post-training quantization is the fastest route because it needs no retraining, but quantization-aware training is the route to choose when the accuracy budget is tight, especially at INT8 and INT4.","Structured pruning is more hardware-friendly than unstructured pruning; unstructured sparsity may shrink the model on paper without speeding up inference on dense accelerators.","Compression techniques compound: distillation and quantization can be combined, as in layer-by-layer distillation with mixed-precision quantization, to get gains neither method gives alone.","Complementary strategies—mixture-of-experts and early exit—can reduce compute per request without shrinking parameter count, so they belong in the same deployment toolkit."],"supporting_citations":[{"why":"A prior survey of model compression for LLMs that the paper uses as the contrast point for its own resource-constrained focus.","marker":"[9]"},{"why":"A prior survey of transformer compression that motivates the need for a broader, deployment-oriented synthesis.","marker":"[10]"},{"why":"The foundational distillation paper whose teacher-student soft-target loss defines the KD section's basic mechanism.","marker":"[11]"},{"why":"The original model-compression work that establishes distillation as a way to shrink an ensemble into a small student.","marker":"[12]"},{"why":"GPTQ supplies a flagship post-training quantization result at 175B scale with INT3-4 weights, grounding the PTQ discussion.","marker":"[30]"},{"why":"ZeroQuant grounds mixed-precision PTQ and the layer-by-layer distillation-quantization hybrid that the survey cites.","marker":"[34]"},{"why":"SmoothQuant shows INT8 quantization working on 500B-parameter models, supporting the claim that quantization scales to large LLMs.","marker":"[40]"},{"why":"The lottery-ticket hypothesis supplies the rationale that pruning can find sparse subnetworks that train to full accuracy.","marker":"[43]"},{"why":"BranchyNet introduces the early-exit mechanism the survey includes as a complementary efficiency strategy.","marker":"[52]"}],"fun_headline_variants":["LLM compression for edge: three key techniques","Shrink LLMs for mobile: distillation, quantization, pruning","The art of squeezing LLMs onto edge devices","Three paths to run LLMs on edge hardware"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The survey's value as a reference depends on the accuracy of its summaries, tables, and citation mapping; the rendered text already shows citation markers pointing to the wrong references, so the reliability of the map cannot be taken on faith.","fun_headline_variants_meta":{"raw":{"variants":["LLM compression for edge: three key techniques","Shrink LLMs for mobile: distillation, quantization, pruning","The art of squeezing LLMs onto edge devices","Three paths to run LLMs on edge hardware"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000489,"raw_usage":{"total_tokens":2345,"prompt_tokens":820,"completion_tokens":1525,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":436,"completion_tokens_details":{"reasoning_tokens":1463}},"tokens_in":436,"tokens_out":1525,"duration_ms":14472,"temperature":1.0,"reasoning_tokens":1463,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:54:19.163294+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Locate a peer-reviewed survey published before this paper that explicitly frames LLM compression techniques around resource-constrained or edge deployment; the existence of such a survey would falsify the paper's claim to be the first focused treatment.","supporting_citations":[{"cited_title":"ZeroQuant: Efficient and Affordable Post -Training Quantization for Large-Scale Transformers,","cited_arxiv_id":null,"evidence_quote":"ZeroQuant grounds mixed-precision PTQ and the layer-by-layer distillation-quantization hybrid that the survey cites."},{"cited_title":"SmoothQuant: Accurate and Efficient Post -Training Quantization for Large Language Models,","cited_arxiv_id":null,"evidence_quote":"SmoothQuant shows INT8 quantization working on 500B-parameter models, supporting the claim that quantization scales to large LLMs."}],"review_version":1}