{"id":"e2cbebff-7b33-4aad-833d-5bc1078a5a68","arxiv_id":"2606.01436","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Quality signals from LLM self-judgments and token entropy yield up to 18.6% gains over base model on saturated arithmetic tasks, outperforming SFT, but produce mixed or negative results on GSM8K depending on the signal used.","lead":"This paper tests whether fine-grained signals like a model's own pairwise judgments of its solutions and token-level entropy can improve LLM training even on questions the model already solves perfectly. A smart generalist might read it to see if saturated benchmarks still contain usable training signal without needing endless new data.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Self-judgment signal reliability unvalidated on the arithmetic task itself despite poor GSM8K agreement","rationale":"The reader's weakest_assumption already isolates the exact premise that must hold for the central claim. Because the abstract itself flags the signal's fragility on a harder task and supplies no counter-evidence on the arithmetic task, the concern is load-bearing and the UNVERDICTED status is appropriate pending full-text verification of any internal validation metrics.","tokens_in":1734,"tokens_out":345,"duration_ms":15369,"concrete_test":"On a held-out sample of 200 arithmetic solutions, obtain pairwise judgments from the same model and from an external judge (e.g., Qwen2.5-72B or GPT-4); compute rank correlation (Kendall tau) or agreement rate between the two; if tau < 0.4 or agreement < 60%, rerun the training loop with random instead of self-judgment rankings to isolate whether the reported 18.6% delta disappears.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline result (18.6% gain on simple arithmetic via quality signals over SFT) requires that pairwise self-judgments and token entropy actually rank solution quality when all answers are empirically correct. The abstract explicitly states poor agreement between self-judgments and a stronger external judge on GSM8K, which directly questions whether the same signal is informative on the arithmetic data. No section in the provided abstract (and by extension the claim) reports agreement, correlation with external judge, or ablation against random ranking on the arithmetic set, leaving open that gains could arise from incidental data filtering rather than quality signal.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that saturated training data (where all solutions have perfect empirical correctness) can still yield useful training signal via fine-grained quality indicators: pairwise LLM self-judgments and token-level entropy. On a simple arithmetic task with Qwen3-1.7B-Base these signals produce up to 18.6% gains over the base model and outperform SFT; on GSM8K the gains are modest, highly signal-dependent, and self-judgments exhibit poor agreement with an external judge, sometimes degrading performance below the base model.","tokens_in":1870,"tokens_out":445,"duration_ms":21201,"significance":"If the quality signals are shown to be reliable on the arithmetic data, the work would be significant for continued progress on saturated benchmarks by moving beyond binary correctness. The arithmetic gains are the strongest element, but the explicit dependence on signal choice and the GSM8K degradation indicate limited generalizability without further calibration. No machine-checked proofs or parameter-free derivations are present.","major_comments":[{"comment":"Abstract: the central 18.6% gain on arithmetic is reported to arise from pairwise self-judgments and token entropy, yet the abstract itself documents poor agreement between self-judgments and a stronger external judge on GSM8K. No corresponding agreement metric, correlation with external judgment, or ablation against random ranking is provided for the arithmetic task, so it remains possible that the observed gains result from incidental data selection rather than the quality signal.","section":"Abstract"},{"comment":"Abstract: the claim that quality-based training 'substantially outperform[s] SFT' on arithmetic rests on the signals being informative when all answers are correct, but the manuscript provides no validation that self-judgments or entropy actually rank solution quality on that specific dataset.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract would be clearer if it stated the number of training runs, presence or absence of error bars, and the precise training objectives used to incorporate the quality signals.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the careful reading and for identifying the need for stronger direct validation of the quality signals on the arithmetic task. We agree that the current evidence relies primarily on downstream performance gains and that explicit agreement metrics, correlations, and random-ranking ablations would make the claims more robust. We will revise the manuscript to incorporate these elements.","responses":[{"response":"We acknowledge the validity of this observation. The abstract already highlights the signal-dependent and sometimes negative results on GSM8K, but the arithmetic results lack parallel validation. In the revised version we will add (i) pairwise agreement statistics between self-judgments and an external judge on a held-out sample of the arithmetic data, (ii) Spearman or Pearson correlations between each signal and external quality ratings, and (iii) a random-ranking ablation that trains on the same number of examples selected uniformly at random. These additions will directly test whether the observed 18.6 % gains exceed what would be expected from incidental selection.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the central 18.6% gain on arithmetic is reported to arise from pairwise self-judgments and token entropy, yet the abstract itself documents poor agreement between self-judgments and a stronger external judge on GSM8K. No corresponding agreement metric, correlation with external judgment, or ablation against random ranking is provided for the arithmetic task, so it remains possible that the observed gains result from incidental data selection rather than the quality signal."},{"response":"We agree that downstream gains alone do not constitute direct proof that the signals rank solution quality. While the performance lift over both the base model and SFT is consistent with the signals carrying useful information, we will strengthen the manuscript by including an explicit validation section for the arithmetic dataset. This will report agreement rates with external judgments, rank correlations between self-judgment scores / entropy and external quality labels, and, where feasible, qualitative examples illustrating the distinctions the signals capture among mathematically correct solutions.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the claim that quality-based training 'substantially outperform[s] SFT' on arithmetic rests on the signals being informative when all answers are correct, but the manuscript provides no validation that self-judgments or entropy actually rank solution quality on that specific dataset."}],"tokens_in":1401,"tokens_out":501,"duration_ms":20077,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main result is that on a simple arithmetic task where the model already gets 100% correct, swapping binary correctness for pairwise self-judgments or token entropy as training signals produces up to 18.6% improvement over the base model and beats plain SFT. That is the concrete new angle: treating saturated data as still having usable variation rather than stopping at accuracy.\n\nThe work is straightforward in its setup. It takes Qwen3-1.7B-Base, applies the two signals inside several training algorithms, and reports the arithmetic gains clearly. The abstract also flags the task dependence without overclaiming, which is useful.\n\nThe soft spot is exactly where the stress-test note lands. The abstract gives no evidence that the self-judgment signal actually ranks quality on the arithmetic examples themselves. We only learn that the same signal agrees poorly with an external judge on GSM8K and can hurt performance there. Without an ablation or correlation check on the arithmetic set, it remains possible the reported gains come from incidental filtering effects rather than the intended quality signal. The circularity of the model judging its own outputs adds to the concern, and the strong dependence on which signal is chosen makes the approach look brittle.\n\nThis is for people working on data-efficient training and alternative supervision signals. The arithmetic result is worth checking in full, but the current evidence is narrow and the signal reliability is unproven on the data that matters for the headline claim.\n\nI would send it to peer review so the experiments can be examined in detail.","headline":"Quality signals from self-judgments and entropy give gains on saturated arithmetic data but lack direct validation there and falter on GSM8K.","tokens_in":2355,"tokens_out":383,"would_cite":false,"duration_ms":17752,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Quality signals from saturated data improve base LLM performance up to 18.6% on arithmetic tasks, outperforming standard fine-tuning.","keywords":["saturated data","LLM training","quality signals","self-judgments","token entropy","arithmetic tasks","GSM8K","supervised fine-tuning"],"falsifier":"An experiment that trains the same base model on the arithmetic task using these quality signals and measures no improvement or a drop relative to the base model or to standard supervised fine-tuning.","tokens_in":2624,"feed_emoji":"📈","tokens_out":694,"duration_ms":21833,"temperature":0.7,"pith_summary":"The paper tests whether data that models already solve with perfect accuracy can still improve training when richer quality signals replace binary correct-or-wrong labels. It replaces those labels with two proxies: the model's own pairwise judgments of which of its solutions is better, and the entropy of its token predictions as a measure of uncertainty. On a simple arithmetic task these signals produce gains of up to 18.6 percent over the untouched base model and beat ordinary supervised fine-tuning. On the harder GSM8K set the same signals give smaller or even negative results that depend on which signal is chosen. The work matters because many current benchmarks and datasets are already saturated, so any method that extracts extra value from them could stretch the life of existing training collections.","feed_headline":"Quality signals lift saturated data training by 18.6% on arithmetic","feed_subtitle":"Pairwise judgments and token entropy outperform binary correctness labels even when every question is already solved perfectly.","key_machinery":"Pairwise LLM self-judgments and token-level entropy used as quality signals in place of binary correctness labels inside several training algorithms.","core_discovery":"When training exclusively on a simple arithmetic task, quality-based signals improve performance by up to 18.6% over the base model, substantially outperforming SFT. On GSM8K, however, gains are more modest and depend strongly on the quality signal. Pairwise LLM self-judgments show poor agreement with a stronger external judge and can even degrade performance below the base model. The central claim is therefore that fine-grained quality signals drawn from saturated questions can still supply useful training signal for base models, provided the signal is well calibrated to the task.","pith_inferences":["The same signals could be tested on other saturated domains such as code generation or multi-step reasoning to see whether calibration remains the main requirement.","Combining self-judgment and entropy into a single training objective might reduce the dependence on any one noisy signal.","The poor agreement between self-judgments and external judges on GSM8K suggests that external verification may still be needed for complex tasks even if self-signals work on simpler ones."],"forward_implications":["Quality signals enable performance gains even when every training example is already solved correctly.","On simple arithmetic the signals outperform binary-label supervised fine-tuning.","On GSM8K the outcome depends on the chosen signal, with some signals harming performance.","Saturated data retains training value once quality rather than correctness is measured."],"fun_headline_variants":["Quality signals outperform correctness on saturated arithmetic by 18.6%","Token entropy and pairwise judgments aid training from saturated questions","Quality based training extracts signal from perfectly solved arithmetic tasks","Saturated data training improves with fine grained quality signals by 18.6%"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That pairwise self-judgments and token entropy reliably mark higher-quality solutions in a way that produces measurable downstream gains.","fun_headline_variants_meta":{"raw":{"variants":["Quality signals outperform correctness on saturated arithmetic by 18.6%","Token entropy and pairwise judgments aid training from saturated questions","Quality based training extracts signal from perfectly solved arithmetic tasks","Saturated data training improves with fine grained quality signals by 18.6%"]},"model":"grok-4.3","cost_usd":0.008889,"raw_usage":{"total_tokens":4015,"prompt_tokens":704,"num_sources_used":0,"completion_tokens":69,"cost_in_usd_ticks":88887000,"prompt_tokens_details":{"text_tokens":704,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3242,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":704,"tokens_out":69,"duration_ms":22467,"temperature":1.0,"reasoning_tokens":3242,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T17:07:55.299308+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An experiment that trains the same base model on the arithmetic task using these quality signals and measures no improvement or a drop relative to the base model or to standard supervised fine-tuning.","supporting_citations":[],"review_version":1}