{"id":"f4d012fc-d678-4eb4-b9c1-d8c47e24f029","arxiv_id":"2504.20482","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"GRKD replaces absolute probability matching with a pairwise ranking loss on teacher-ordered pairs and reports improved LLM distillation results.","lead":"This paper proposes Group Relative Knowledge Distillation (GRKD), a loss that trains a student model to preserve the teacher's pairwise ranking of classes instead of matching absolute probabilities. The authors report large gains on LLM chat benchmarks, but the method's math is written for classification and is not actually connected to the experiments.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 2 is not well-defined for LLM distillation: q_i is called logits but used as log-probabilities, and for sampled responses no q_i is defined, so the reported gains cannot be traced to the stated loss.","rationale":"The paper's framework is coherent at a high level for classification: a pairwise ranking loss over teacher-ordered classes plus an annealed soft-label term is a plausible distillation objective, and Eq. 2 has precedents in RankNet/BPR. The classification version would be implementable if q_i were probabilities. But the manuscript's only experimental support for the central claim is the LLM transfer in Table 1, and there the method is not defined. My stress-test therefore focuses on the link from Eq. 2 to the experiments. The reader's weakest assumption points to the same gap: Section 3.1 says n = 4 sampled responses but never maps a response to q_i. I additionally note the internal logits/probability clash between Eq. 2 and Eq. 3, which makes even the classification version ambiguous. The over-20% claim in Section 3.2 is also unsupported: for LLaMA-3 the AlpacaEval LC gain is 6.0, not 20%, and no confidence intervals or seeds are reported. However, that is secondary; the primary issue is that if q_i is undefined, the table cannot be attributed to GRKD. Because the paper includes no code, no formal proof, and no ablation linking Eq. 2 to the gains, the empirical claim is not verifiable as submitted. I agree with the REJECT verdict; no adjustment is needed.","tokens_in":4496,"tokens_out":6701,"duration_ms":63751,"concrete_test":"Obtain the training code or an exact specification of q_i and s_i for a sampled response, then re-run the Gemma-2B GRKD row with the stated definitions and also with two plausible alternatives (whole-sequence log-likelihood vs. length-normalized per-token log-probability for q_i). If Table 1's AlpacaEval 2.0 LC and Arena-Hard numbers are not reproduced under the authors' definitions, or if the two alternatives give materially different results, Eq. 2 is not the object being tested; if both alternatives reproduce the table, the missing definition remains a reproducibility defect but the empirical claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. 2 is the sole mechanism for the claimed relational transfer, so its variables must be unambiguous. It is not. Section 2.2 says the student produces logits q_i and then computes log(q_i) - log(q_j); logits can be negative, so log(q_i) is generally undefined. Eq. 3 immediately redefines q_i as the student's predicted probability, making Eq. 2 a log-odds objective but contradicting the 'logits' wording. Section 3.1 then applies this classification-style loss to autoregressive models by 'sampling n outputs' with n = 4, but never specifies the score q_i for an entire sampled response (sequence log-likelihood? length-normalized token log-probability? reward-model score?) nor the teacher score s_i (log-probability? reward?). The unexplained 'reward calibration ratio of α = 0.8' appears only in the setup. Without these definitions the loss cannot be computed, so the Gemma/LLaMA rows in Table 1 are not evidence for GRKD as stated; they could reflect any preference or likelihood score. This is a definitional gap in the central construction, not a missing hyperparameter.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Group Relative Knowledge Distillation (GRKD), a distillation framework that trains a student model to preserve pairwise preference orderings from a teacher's outputs rather than matching absolute probability distributions. The method combines a group relative ranking loss (Eq. 2) with an annealed soft-label cross-entropy term (Eq. 3-4). Experiments are reported on Gemma-2 and LLaMA-3 instruction-tuned models trained on UltraFeedback and evaluated on AlpacaEval 2.0, Arena-Hard, MT-Bench, and GSM8K, with claims of consistent improvements over DPO, SimPO, PRO, Standard KD, SeqKD, and MiniLLM.","tokens_in":4672,"tokens_out":3257,"duration_ms":32519,"significance":"The conceptual premise, that relational structure in teacher outputs is more transferable than absolute likelihoods, is plausible and connects to recent preference-optimization methods such as DPO and GRPO. If the loss were rigorously defined for sequence generation and validated with controlled ablations, the idea could constitute a useful baseline for preference-based distillation. However, as presented, the central loss is not defined for the actual experimental setting, the experimental protocol lacks statistical and ablation support, and the abstract's framing contradicts the evaluation setup. The paper therefore does not yet deliver a testable method or credible evidence for its claims.","major_comments":[{"comment":"The group relative loss is defined for classification logits q_i over n classes, but Section 3.1 applies it to autoregressive language models by 'sampling n outputs' with n=4. The quantity q_i is never defined for a sampled response: it is unclear whether q_i is a sequence log-likelihood, a length-normalized token log-probability, a reward-model score, or something else. Similarly, the teacher score s_i for a sampled response is not defined. Moreover, Eq. (2) writes log(q_i) - log(q_j) while Section 2.1 states that q_i are logits, which can be negative and thus make the logarithm undefined; Eq. (3) then treats q_i as a probability. These are not presentation issues: without a precise definition of q_i for responses, Eq. (2) cannot be computed, and the results in Table 1 cannot be attributed to the stated loss.","section":"Section 2.2, Eq. (2)"},{"comment":"The hyperparameters τ, the λ annealing schedule, the reward calibration ratio α=0.8, and the number of sampled responses n=4 are introduced without derivations or sensitivity analyses. The claim that λ starts at 0 and gradually increases to 1, thereby fully removing the soft-label term at the end of training, is load-bearing because the method is defined as a weighted combination of two terms; no ablation isolates the contribution of the relative loss, and no evidence justifies the chosen τ, α, or n. Consequently, Section 3.2's assertion that GRKD 'consistently outperforms' baselines is not supported by controlled experiments.","section":"Section 3.1, Table 1"},{"comment":"All experimental results are reported without error bars, confidence intervals, or multiple independent runs. Several of the reported improvements over the strongest baseline PRO are small (for example, Gemma MT-Bench 7.11 vs 6.96, and LLaMA Arena-Hard 46.92 vs 45.09). Without variance information, the claim of consistent, significant gains is not statistically supported.","section":"Section 3.1 and Table 1"},{"comment":"The abstract and introduction frame GRKD as a classification method targeting fine-grained class differentiation and state that experiments are conducted on 'classification benchmarks,' but Section 3 evaluates generative language models on conversational benchmarks (AlpacaEval, Arena-Hard, MT-Bench, GSM8K). This mismatch means the motivating scenario, relational structure among classes, is not directly tested, and the conclusion's 'average improvement of 9% across all four datasets' is not computable from Table 1, which reports five evaluation metrics.","section":"Abstract and Section 1 vs Section 3"}],"minor_comments":[{"comment":"The sentence 'For each input x, we sample n outputs under the temperature is 1' is grammatically incomplete and should be revised; it also does not clarify whether the temperature refers to teacher logits, student logits, or sampling temperature.","section":"Section 3.1"},{"comment":"The notation is inconsistent: 'GEMMA -2' and 'LL AMA-3' appear in the text and Table 1 with irregular spacing, and the model family names should be typeset consistently.","section":"Throughout"},{"comment":"Several references appear incomplete or inaccurate: the PRO reference (Song et al., 2024) lists authors but no venue, the Tunstall et al. reference for Zephyr is not the original DPO citation, and the Dubey et al. entry titled 'Herding llama 3' does not match the standard LLaMA-3 citation.","section":"References"},{"comment":"The paper states that training data is filtered by context length and that training is conducted for one epoch, but no details are given about optimizer, learning rate, batch size, or computational budget, which are needed for reproducibility.","section":"Section 3.1"}],"recommendation":"reject","confidential_remarks":"The manuscript appears rushed, with undefined variables in the central loss, a mismatch between the classification framing and the LLM experiments, and no code or data release. The core idea may have some merit, but the current submission does not provide a well-defined method or credible evidence. A resubmission would need to fully specify the loss for sequence generation, provide ablations and multiple seeds, and reconcile the abstract with the actual evaluation protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core loss cannot be computed from the text. Eq. 2 takes log(q_i) - log(q_j) with q_i called logits, which is undefined. Eq. 3 then redefines q_i as a probability. When the paper moves to LLMs and says it samples n outputs, it never defines q_i or s_i for a whole sampled response. So the reported gains on Gemma-2 and LLaMA-3 are not evidence for the stated method unless the authors supply those missing definitions. That is a load-bearing gap, not a quibble.\n\nCredit where it is earned: the observation that relative orderings carry useful information beyond absolute probabilities is legitimate, and the recipe of blending a pairwise ranking loss with a soft-label term and an annealing schedule is a sensible variant. The baseline comparison against DPO, SimPO, and PRO is informative, even if nothing beats them by a huge margin. Also, the \"over 20% improvement on AlpacaEval\" claim does hold for both model rows when you look at length-controlled win rates (about 26% for Gemma, 21% for LLaMA). That particular overclaim criticism does not land.\n\nThe soft spots are real but secondary once the definitional issue is fixed: no error bars, no seeds, no ablations for tau, lambda, alpha, or n, no code, and the abstract promises classification experiments while Table 1 is all chat and math. The method is also a modest variant of RankNet plus soft labels, so novelty is limited even if the definitions were clean.\n\nWho gets value from this? Someone thinking about ranking-based distillation for LLMs might find it a pointer, but as written it is not reproducible and I would not cite it. The right move is to desk reject and ask the authors to resubmit with a properly defined loss for autoregressive models, along with ablations and code. That would make it worth a referee's time.","headline":"The core idea is plausible but the central loss is undefined for LLMs as written, so Table 1 cannot be interpreted; this version is not ready for peer review.","tokens_in":574,"tokens_out":1211,"would_cite":false,"duration_ms":46621,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Distilling a teacher's relative class rankings rather than its exact probabilities gives smaller language models consistent gains across instruction-following and reasoning benchmarks.","keywords":["knowledge distillation","relational inductive bias","pairwise ranking loss","preference optimization","large language models","soft labels","exposure bias","fine-grained classification"],"falsifier":"Define the student score $q_i$ for sampled responses in a concrete way (for example, per-token log-likelihood) and rerun GRKD; if no reasonable definition reproduces the reported gains, the method as written is not implementable. Alternatively, train the same student with the same teacher but replace the teacher's ordering in $P$ with random pair order: if the win rates stay unchanged, the ranking signal is not what drives the results.","tokens_in":4200,"feed_emoji":"🎯","tokens_out":9696,"duration_ms":86304,"temperature":0.7,"pith_summary":"The paper argues that knowledge distillation wastes useful information when a student is forced to copy a teacher's exact output probabilities, because the teacher's relative ordering of classes carries the fine-grained distinctions that matter. It proposes Group Relative Knowledge Distillation (GRKD), which trains the student with a pairwise ranking loss over class pairs the teacher ranks highest, plus an annealed soft-label term that keeps early training stable. On Gemma-2 and LLaMA-3 teacher-student pairs, the method reports higher scores than standard KD, SeqKD, MiniLLM, DPO, SimPO, and PRO on AlpacaEval 2.0, Arena-Hard, MT-Bench, and GSM8K. A sympathetic reading is that the paper's central claim is that relational structure, not absolute likelihood, is the transferable part of teacher knowledge.","feed_headline":"Distill teacher rankings, not probabilities, to lift student LLMs","feed_subtitle":"A pairwise ranking loss plus an annealed soft-label term beats six baselines on Gemma and LLaMA.","key_machinery":"The machinery is the group relative loss of Eq. (2), a pairwise ranking loss defined over teacher-ordered class pairs: for every pair $(i,j)$ with teacher score $s_i > s_j$, the student is trained so that $\\log q_i - \\log q_j$ is large, through the logistic term $-\\log\\sigma((\\log q_i - \\log q_j)/\\tau)$. It is paired with the softened cross-entropy $L_{ST}$ and an annealing schedule for $\\lambda$, so the student first imitates the teacher's probability surface and gradually shifts to preserving only its rankings. In the LLM experiments, the same loss is applied by sampling $n=4$ outputs per input and treating the sampled outputs as the items to rank.","core_discovery":"The central claim is that replacing absolute-probability matching with a ranking-preservation objective yields better student models. For each input, GRKD builds the set $P$ of class pairs $(i,j)$ for which the teacher's score $s_i$ exceeds $s_j$, then minimizes $L_{GR} = -\\sum_{(i,j)\\in P} \\log\\sigma((\\log q_i - \\log q_j)/\\tau)$, where $q_i$ and $q_j$ are the student's per-class scores. This forces the student to keep the teacher's pairwise preference order. The full objective is $L_{total} = \\lambda L_{GR} + (1-\\lambda) L_{ST}$, with $L_{ST}$ the softened cross-entropy and $\\lambda$ annealed from 0 to 1, so training begins with probability matching and ends with pure relational alignment. The paper reports that this recipe consistently outperforms six distillation and preference baselines on four benchmarks, with the largest margins on length-controlled win rate and on tasks requiring fine-grained response selection.","pith_inferences":["A natural extension the paper does not test is applying the same group-relative loss to hidden representations or feature embeddings, where pairwise distances have a similar relational structure.","Whether the gains come from preserving pairwise order or merely from an extra smooth training signal is testable by ablating $P$ down to the teacher's top-1 pair only; the paper includes no such ablation.","The undefined mapping from an autoregressive response to a score $q_i$ is the sharpest open question; a reader would need to try definitions such as per-token log-likelihood or reward-model scores to know which version of GRKD the reported numbers describe.","The rank-only endpoint of the annealing schedule predicts that starting at $\\lambda=1$ would be unstable early in training, a consequence that could be checked directly."],"forward_implications":["If GRKD is right, a student trained on teacher rankings should beat the same student trained on teacher probabilities on fine-grained discrimination tasks, and the reported tables claim exactly this.","The annealing schedule implies that pure relational alignment is sufficient at the end of training; the soft-label term is a warm start rather than an ongoing requirement.","The method's gains over DPO and SimPO suggest that when a teacher is available, distilling its preferences directly can be a cheaper alternative to optimizing the student's own preference objective.","Because the paper motivates GRKD as reducing exposure bias and overfitting to spurious teacher outputs, the method is claimed to give students more stable inference behavior."],"supporting_citations":[{"why":"Introduces softened probability distillation, the absolute-matching baseline GRKD is designed to move beyond.","marker":"(Hinton et al., 2015)"},{"why":"Sequence-level KD baseline that trains the student on teacher-generated sequences.","marker":"(Kim and Rush, 2016)"},{"why":"MiniLLM baseline, a reverse-KL distillation method compared in the main tables.","marker":"(Gu et al., 2024)"},{"why":"DPO preference-optimization baseline run under the teacher-as-annotator setup.","marker":"(Tunstall et al., 2024)"},{"why":"SimPO baseline, a simple preference optimization method compared against GRKD.","marker":"(Meng et al., 2024)"},{"why":"PRO preference-ranking baseline that also models relative preferences.","marker":"(Song et al., 2024)"},{"why":"UltraFeedback preference dataset used as the training source for all methods.","marker":"(Cui et al., 2023)"},{"why":"Gemma-2 model family supplies the 9B teacher and 2B student in the first experiment block.","marker":"(Team, 2024)"},{"why":"LLaMA-3 model family supplies the 8B teacher and 3B student in the second experiment block.","marker":"(Dubey et al., 2024)"},{"why":"AlpacaEval 2.0 benchmark provides the length-controlled and raw win-rate headline numbers.","marker":"(Liang et al., 2024)"}],"fun_headline_variants":["Rank over match: distill teacher preferences, not probabilities","Pairwise preference distillation beats six baselines on Gemma and LLaMA","Group relative knowledge: learn teacher's order, not just output","Ranking-based distillation improves student LLMs on fine-grained tasks","Distill teacher rankings, not scores, to boost student models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the student's output scores are well-defined and comparable for every sampled response, so that the pairwise ranking loss can actually be computed; for the LLM experiments the paper samples responses but never specifies what score each response gets.","fun_headline_variants_meta":{"raw":{"variants":["Rank over match: distill teacher preferences, not probabilities","Pairwise preference distillation beats six baselines on Gemma and LLaMA","Group relative knowledge: learn teacher's order, not just output","Ranking-based distillation improves student LLMs on fine-grained tasks","Distill teacher rankings, not scores, to boost student models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1421,"prompt_tokens":900,"completion_tokens":521,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":434}},"tokens_in":516,"tokens_out":521,"duration_ms":6093,"temperature":1.0,"reasoning_tokens":434,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:27:51.642962+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Define the student score $q_i$ for sampled responses in a concrete way (for example, per-token log-likelihood) and rerun GRKD; if no reasonable definition reproduces the reported gains, the method as written is not implementable. Alternatively, train the same student with the same teacher but replace the teacher's ordering in $P$ with random pair order: if the win rates stay unchanged, the ranking signal is not what drives the results.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"AlpacaEval 2.0 benchmark provides the length-controlled and raw win-rate headline numbers."}],"review_version":1}