{"id":"49409638-e252-44be-af78-943b56fbf508","arxiv_id":"2606.05682","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"CKA-guided alignment of layerwise Gram matrices during NVFP4 QAD preserves internal representational geometry and improves downstream accuracy beyond KL-only output matching.","lead":"This paper shows that standard output-matching distillation for low-precision LLMs can allow internal layer representations to drift even when final outputs align with the teacher. It adds a CKA-based regularizer to preserve layerwise geometry during NVFP4 quantization-aware distillation, yielding better reasoning and coding accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Causality between CKA regularizer and accuracy gains not isolated from other training factors","rationale":"The reader's weakest assumption directly names the missing causal link; the proposed test would resolve whether the CKA intervention itself produces the claimed gains.","tokens_in":1750,"tokens_out":274,"duration_ms":16256,"concrete_test":"Re-train the Nemotron 3 Nano and Qwen3-4B models with an otherwise identical QAD setup but replace the CKA term by a dummy regularizer (e.g., L2 penalty on random projections of the same activations) whose coefficient is tuned to produce the same total loss magnitude; if downstream accuracy matches CKA-QAD, the geometry-specific alignment is not required.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that explicitly aligning layerwise Gram matrices via CKA (rather than output KL alone) is what recovers downstream reasoning/coding accuracy. The abstract shows only that KL-only QAD reduces CKA similarity and that this reduction correlates with bottlenecks; it does not establish that the added CKA term is the causal driver. If the reported gains arise from unstated differences in total loss weighting, optimizer schedule, or data ordering instead of the representational regularizer, the 'beyond output matching' contribution does not hold.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that KL-divergence output matching in quantization-aware distillation (QAD) for NVFP4 LLMs masks internal representational degradation, as shown by reduced layerwise CKA similarity to the BF16 teacher (especially severe in RL-post-trained models). This drift correlates with downstream bottlenecks on reasoning and coding tasks. Motivated by the diagnosis, the authors propose CKA-QAD, which adds a lightweight CKA regularizer to align layerwise Gram matrices, claiming it substantially improves representational alignment and downstream accuracy on Nemotron 3 Nano and Qwen3-4B-Thinking-2507 with modest overhead.","tokens_in":1846,"tokens_out":466,"duration_ms":32367,"significance":"If the empirical results hold with proper controls and quantitative validation, the work would be moderately significant for low-precision LLM deployment: it identifies a concrete limitation of output-only distillation and offers a practical, low-overhead complement that targets internal geometry preservation. The use of CKA as a diagnostic and regularizer is a straightforward extension of existing representation similarity tools to the quantization setting.","major_comments":[{"comment":"Abstract: The central claim that adding the CKA regularizer (rather than other training factors) recovers downstream accuracy is not supported by any described ablation studies, controls, or isolation experiments; the text only reports that KL-only QAD reduces CKA similarity and that this correlates with bottlenecks, without establishing causality for the proposed method.","section":"Abstract"},{"comment":"Abstract: The abstract states that CKA-QAD 'substantially improves' reasoning and coding accuracy but supplies no effect sizes, baseline comparisons (e.g., vs. standard QAD or other regularizers), number of runs, variance, or statistical tests, leaving the magnitude and reliability of the claimed gains impossible to evaluate.","section":"Abstract"}],"minor_comments":[{"comment":"Abstract: 'NVFP4' is introduced without a definition or citation to prior work on this specific quantization format.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears preliminary; the absence of any quantitative results or experimental details even in the abstract suggests it may not yet be ready for a full journal submission without substantial additional experiments and reporting."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed and constructive comments. We address each major comment below and indicate the revisions we will make to strengthen the manuscript.","responses":[{"response":"We agree that the abstract does not explicitly describe ablation studies isolating the contribution of the CKA regularizer from other training factors. The manuscript presents direct comparisons of KL-only QAD versus CKA-QAD on both representational similarity (CKA) and downstream tasks across two models, which provides correlational support for the benefit of the added regularizer. However, we acknowledge that stronger causal evidence would require additional controls such as ablating the CKA loss weight or comparing against alternative regularizers. We will add these ablation experiments and corresponding discussion in the revised manuscript to better establish causality.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The central claim that adding the CKA regularizer (rather than other training factors) recovers downstream accuracy is not supported by any described ablation studies, controls, or isolation experiments; the text only reports that KL-only QAD reduces CKA similarity and that this correlates with bottlenecks, without establishing causality for the proposed method."},{"response":"We agree that the abstract would benefit from quantitative details. The full manuscript reports specific accuracy improvements on reasoning and coding benchmarks for Nemotron 3 Nano and Qwen3-4B-Thinking-2507 relative to the KL-only baseline, along with CKA similarity metrics. To address the concern, we will revise the abstract to include effect sizes (e.g., absolute and relative accuracy gains), baseline comparisons, and any available statistics on variance or multiple runs. Where multiple runs were not performed due to computational cost, we will note this limitation explicitly.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The abstract states that CKA-QAD 'substantially improves' reasoning and coding accuracy but supplies no effect sizes, baseline comparisons (e.g., vs. standard QAD or other regularizers), number of runs, variance, or statistical tests, leaving the magnitude and reliability of the claimed gains impossible to evaluate."}],"tokens_in":1431,"tokens_out":459,"duration_ms":17950,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that output-only KL distillation for NVFP4 can allow internal layer geometries to drift from the teacher even when logits match, and this drift correlates with drops on reasoning and coding. They measure it with CKA and add a Gram-matrix alignment regularizer to fix it.\n\nWhat is new is the explicit diagnosis that many internal geometries can produce the same teacher outputs, plus the lightweight CKA term applied inside NVFP4 QAD. The paper does a clean job of motivating why this matters more for RL-post-trained models and claims the overhead stays modest while improving alignment and downstream accuracy on Nemotron 3 Nano and Qwen3-4B.\n\nThe soft spot is exactly the one in the stress-test note. The abstract shows correlation between CKA drift and bottlenecks but does not isolate whether the added regularizer, rather than changes in total loss scale, optimizer, or data, drives the reported gains. No effect sizes, baseline tables, or ablation details appear here, so the causal claim stays plausible rather than demonstrated.\n\nThis is for people working on low-bit LLM deployment and quantization recovery. A reader already running QAD experiments could test the regularizer quickly and see if it helps their setup.\n\nIt deserves peer review because the problem is real for production use and the method is simple enough to verify or refute with standard ablations.","headline":"CKA regularizer for NVFP4 QAD is a reasonable practical idea but the abstract leaves causality and effect sizes unverified.","tokens_in":2329,"tokens_out":348,"would_cite":false,"duration_ms":21965,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Aligning layerwise Gram matrices via CKA during NVFP4 distillation restores internal geometry that output matching alone allows to drift.","keywords":["quantization-aware distillation","representational alignment","CKA","NVFP4","Gram matrix","internal geometry","low-bit LLM"],"falsifier":"Running the full CKA-QAD training loop and finding no increase in layerwise CKA or no gain in reasoning and coding accuracy would falsify the claim that geometry preservation drives the improvement.","tokens_in":2651,"feed_emoji":"","tokens_out":607,"duration_ms":11134,"temperature":0.7,"pith_summary":"Standard quantization-aware distillation matches teacher logits through KL divergence yet permits intermediate activation geometries to diverge from the BF16 teacher, as shown by declining CKA scores across layers. This representational drift is more severe after RL post-training and tracks with degraded performance on reasoning and coding benchmarks. The paper introduces CKA-QAD, which augments the distillation objective with a term that aligns the Gram matrices of corresponding layers between student and teacher. On Nemotron 3 Nano and Qwen3-4B models, the added regularizer raises layerwise similarity and lifts downstream accuracy while adding only modest compute.","feed_headline":"CKA alignment fixes internal drift in quantized LLM distillation","feed_subtitle":"Gram-matrix regularizer during output matching raises reasoning accuracy with modest overhead.","key_machinery":"CKA regularizer that aligns layerwise Gram matrices between quantized student and higher-precision teacher during distillation","core_discovery":"KL-only QAD reduces layerwise CKA similarity to the teacher, especially in RL-post-trained models, and this reduction correlates with downstream bottlenecks; explicitly aligning Gram matrices through an added CKA regularizer during NVFP4 distillation restores similarity and improves reasoning and coding accuracy.","pith_inferences":["The same Gram-matrix alignment idea could be tested in other compression settings such as pruning or knowledge distillation without quantization.","Certain layers may be more critical to preserve for particular task families, suggesting selective rather than uniform CKA weighting as a possible refinement.","If the drift is driven by post-training dynamics, applying CKA-QAD during the RL stage itself rather than only in the distillation phase might yield further gains."],"forward_implications":["Layerwise representational similarity to the BF16 teacher increases across the network.","Downstream accuracy on reasoning and coding tasks rises relative to standard KL-only QAD.","The method applies to NVFP4 quantization with only modest extra training cost.","The gains appear across distinct model families including Nemotron and Qwen variants."],"fun_headline_variants":["CKA-QAD preserves internal geometry in NVFP4 LLM distillation","Layerwise CKA alignment for quantized LLM geometry","Gram matrices aligned via CKA in NVFP4 distillation","Preserving CKA similarity in low bit LLM distillation","CKA regularizer for representational alignment in LLMs"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The observed correlation between lower CKA similarity and accuracy loss is causal, so that explicitly raising CKA via the regularizer will produce the accuracy gains.","fun_headline_variants_meta":{"raw":{"variants":["CKA-QAD preserves internal geometry in NVFP4 LLM distillation","Layerwise CKA alignment for quantized LLM geometry","Gram matrices aligned via CKA in NVFP4 distillation","Preserving CKA similarity in low bit LLM distillation","CKA regularizer for representational alignment in LLMs"]},"model":"grok-4.3","cost_usd":0.00699,"raw_usage":{"total_tokens":3243,"prompt_tokens":679,"num_sources_used":0,"completion_tokens":70,"cost_in_usd_ticks":69899500,"prompt_tokens_details":{"text_tokens":679,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2494,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":679,"tokens_out":70,"duration_ms":18714,"temperature":1.0,"reasoning_tokens":2494,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T01:37:15.851676+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running the full CKA-QAD training loop and finding no increase in layerwise CKA or no gain in reasoning and coding accuracy would falsify the claim that geometry preservation drives the improvement.","supporting_citations":[],"review_version":1}