{"id":"427a57a7-107c-40b9-9130-3602beac98e7","arxiv_id":"2506.13472","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ROSAQ uses PCA projection to find salient transformer weight channels, keeps those in FP16, quantizes the rest to INT3/4, and reports modest gains over previous quantization methods.","lead":"The paper proposes a new way to compress large language models by rotating internal activations with PCA, then storing only the most important channels in full precision and squeezing the others into 3 or 4 bits. It reports small accuracy gains over existing quantization methods and a 2.3x decoding speedup on one GPU setup, though that speedup depends on a special kernel.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PCA eigenvalue saliency ignores weight-scale and loss; a weight-magnitude variant could match or beat ROSAQ, undermining the central claim.","rationale":"The reader's weakest_assumption correctly identifies the load-bearing assumption: saliency is determined solely from the input activation covariance X^T X, with no direct dependence on weight magnitudes or downstream loss. The paper's results show consistent small gains over several baselines, and the Appendix B ablation does show that top-eigenvalue channels outperform random and bottom channels, which is real evidence for the criterion being non-arbitrary. However, that ablation does not compare eigenvalues against weight-aware or sensitivity-based saliency, so the central claim remains under-tested. The concrete test I propose would settle whether the eigenvalue-based selection is actually necessary, or whether a weight-aware variant would perform just as well or better. Since this concern is testable and the paper already lacks release code and error bars, the CONDITIONAL verdict is appropriate; the concern does not by itself warrant rejection but requires resolution before the central claim can be accepted.","tokens_in":13557,"tokens_out":9812,"duration_ms":99950,"concrete_test":"On LLaMA2-7B with the same PCA rotation R from Eq. (4), run two variants of ROSAQ for the FFN layers (W_U/W_G): (a) select the K=128 salient channels as the largest eigenvalues, as in the paper; (b) select the K=128 channels with the largest L2 norm of the corresponding rows of R^T W (weight magnitude in the rotated space). Recompute the INT4g128 WikiText-2 perplexity and MMLU numbers from Table 5 for both variants. If variant (b) matches or beats variant (a), the eigenvalue-based saliency criterion is not necessary for the claimed benefit, and the paper's central claim should be revised to attribute the gain to the rotation itself or to other factors.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assertion is that the top-K eigenvectors of X^T X (Eq. 4) identify the channels whose quantization error matters most. An eigenvalue measures only the input activation variance along one rotated direction; it does not measure the output-relevant sensitivity of that channel, which also depends on the magnitude of the corresponding entries in R^T W and on the downstream loss. A channel with low activation variance but large weight values can incur substantial output error when quantized, yet ROSAQ would leave it in INT4. The ablation in Appendix B (Top vs Bottom vs Random) shows only that top-eigenvalue channels are better than random or bottom choices, not that eigenvalues outperform weight-aware or loss-aware saliency. Because the 'Mixed' baseline in Tables 5–7 changes both the rotation and the selection criterion, the reported gains over AWQ cannot be attributed to the eigenvalue criterion per se. The paper's Limitations even state that 'further analysis is needed' to connect saliency-aware and outlier-free rotation methods, and that both theoretical and empirical investigations are required. Without a test isolating the selection criterion, the central claim that 'projected principal dimensions are naturally considered as salient features' remains unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ROSAQ, a post-training quantization method for LLMs that performs PCA on calibration activations to rotate weight matrices, selects K channels with the largest eigenvalues as salient, stores them in FP16, and quantizes the remaining channels to INT3/INT4 with per-group quantization. For multi-head attention, a head-wise PCA is used. The method is evaluated on LLaMA2-7B, LLaMA3-8B, and Qwen2-7B in terms of WikiText-2 perplexity, zero-shot common-sense reasoning, and MMLU, and is compared with GPTQ, SpinQuant, AWQ, and a rotation-less Mixed baseline. Throughput is measured with the QUICK kernel and reported as speedup over FP16.","tokens_in":13801,"tokens_out":7657,"duration_ms":71774,"significance":"If the central claim holds, ROSAQ offers a simple and computationally light way to identify channels worth protecting during quantization, using only calibration activations. The paper includes useful ablations (top vs bottom vs random salient channels; head-wise vs global PCA) and is transparent about its limitations, including the open question of how PCA saliency relates to outlier-removing rotations. The main contribution is plausible but not fully established because the saliency criterion is not isolated from other design choices, and several reported differences are within the noise of a single run.","major_comments":[{"comment":"The saliency criterion is defined solely by the eigenvalues of X^T X, i.e., the input activation covariance along rotated directions. This ignores the magnitudes of the rotated weights R^T W and the downstream loss; a direction with small activation variance but large weight entries could incur significant quantization error. The ablation in Appendix B (Top vs Bottom vs Random) tests only the choice of top versus bottom eigenvalues within the same criterion, not whether this eigenvalue-based criterion is better than a weight-aware or loss-aware criterion. To support the abstract's statement that 'projected principal dimensions are naturally considered as salient features,' please compare in the same rotated space the eigenvalue criterion against alternatives such as row-norm of R^T W or AWQ-style activation-magnitude selection, with all other components fixed.","section":"Section 3.1, Eq. (4)"},{"comment":"The 'Mixed' baseline is defined as R=I with saliency based on activation magnitudes, but the WD layer in ROSAQ is not rotated and is instead protected by AWQ per-channel scaling. It is not clear whether the Mixed baseline also applies this scaling to WD. If it does not, the comparison conflates the effect of PCA rotation with the effect of the AWQ scaling patch. Please specify the exact configuration of Mixed and include an ablation of ROSAQ without the AWQ scaling on WD so the contribution of the rotation itself is isolated.","section":"Section 4.1 and Appendix G"},{"comment":"The reported improvements are not uniform. For example, Table 5 (LLaMA2-7B, INT4g128) shows SpinQuant achieving lower PPL (5.52 vs 5.57) and the Mixed baseline achieving higher CSR (59.62 vs 59.29) than ROSAQ. The main text's claim that ROSAQ is 'slightly superior' is therefore contradicted by some rows. The authors should qualify the claim, for example by noting that ROSAQ is usually best or tied but with exceptions, and discuss possible reasons for these cases.","section":"Tables 5-7"},{"comment":"No error bars or multiple seeds are reported, and the differences between methods are often small (e.g., PPL differences below 0.1). The number of salient channels K is set per layer type (128 for WQ/WK/WV, 32 per head for WO, 128 for WU/WG in Appendix G) with no sensitivity analysis. Since the performance of the method may depend on K, please report at least a small sweep over K or a justification for the chosen values, and include repeated runs or bootstrap confidence intervals for the main tables.","section":"Section 4.2 and Appendix G"}],"minor_comments":[{"comment":"In Eq. (3), the definitions of W_S and W_N are ambiguous: they are described as sub-blocks of the weight matrix, but after rotation the relevant sub-blocks are rows of R^T W. Please clarify the notation, for example by defining W_S=(R^T W)_S and W_N=(R^T W)_N.","section":"Eq. (3)"},{"comment":"Section 3.1 refers to 'Fig 1' for the activation magnitude plots, but the plots appear in Figure 2; please correct the reference.","section":"Section 3.1"},{"comment":"Appendix B contains the typo 'eigenvaluesl' in place of 'eigenvalues'; please fix it.","section":"Appendix B"},{"comment":"Table 4 is difficult to read because the decode-speed and speedup columns are not clearly separated and the device name is repeated. Also, because ROSAQ, AWQ, and GPTQ use different kernels (QUICK, AutoAWQ, and Marlin), the speed comparisons across methods reflect kernel differences rather than algorithmic differences; please state this caveat in the text.","section":"Table 4"},{"comment":"In Table 5, the FP16 row condenses PPL and the first accuracy score into '- 5.4779.11', making the table hard to parse; please separate these values.","section":"Table 5"},{"comment":"The eigenvalues listed in Table 8 do not appear sorted even though the channels were selected by average magnitude; please clarify that the eigenvalues correspond to the magnitude-selected channels, or sort the rows accordingly.","section":"Appendix F, Table 8"},{"comment":"The Limitations section admits that 'further analysis is needed' to connect saliency-aware and outlier-free rotations; this caveat should also appear in the introduction or conclusion so that the paper's claims are not overstated.","section":"Section 5 and Limitations"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central idea is interesting and within the scope of the journal. The main risk is that the saliency criterion is not properly isolated from other design choices, and the baseline configuration is underspecified. These concerns are addressable with additional ablations and clarifications within the manuscript's scope, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my honest read of ROSAQ. The two things you should know: (1) it's a real, incremental idea—use PCA rotation to define a feature space, then treat top-eigenvalue channels as salient and keep them in FP16 while quantizing the rest to INT3/4. (2) The experiments are directionally positive but the evaluation misses the one ablation that would make the central claim solid.\n\nWhat's new: AWQ picks salient channels by activation magnitude in the original space; SpinQuant rotates to remove outliers but doesn't do mixed precision. ROSAQ combines both: PCA projection plus eigenvalue-based saliency selection, with a head-wise PCA for MHSA. That combination is new, and the head-wise trick looks useful (PPL 5.57 vs 5.94 in Table 3). The main tables show ROSAQ is usually best or tied on PPL and often on MMLU against GPTQ, SpinQuant, AWQ, and the 'Mixed' baseline, across LLaMA2-7B, LLaMA3-8B, and Qwen2-7B. That's a decent empirical package.\n\nWhere the soft spots are: the load-bearing claim—that eigenvalue saliency beats other saliency criteria—is not isolated. The Mixed baseline uses no rotation and magnitude selection, so ROSAQ vs Mixed conflates rotation and selection. To prove the claim, they need a rotated baseline that selects salient channels by, say, weight magnitude or activation magnitude in the rotated space. The stress-test worry about ignoring weight magnitudes is real: a high-variance channel with tiny weights might not be where quantization hurts. Appendix B's Top vs Bottom vs Random only shows top-k is better than random, not that eigenvalues are better than weight-aware or loss-aware criteria. The paper's own Limitations admit this connection needs further analysis. That's honest, but it means the central assertion is under-supported. Also: no error bars (differences are 0.1-0.3 PPL), K is hand-set without sensitivity analysis, and the 2.3x throughput comparison uses different kernels for each method (AutoAWQ, Marlin, QUICK), so that speedup is a feature of the kernel, not of ROSAQ. No code released.\n\nOverall, this is a solid incremental paper that deserves peer review, but the referee should ask for the missing ablation. Who benefits: researchers working on weight-only PTQ for LLMs. I'd send it to review, but expect a revision.","headline":"ROSAQ is a plausible incremental idea—PCA-based saliency for mixed-precision quantization—but its central claim needs an ablation that isolates the eigenvalue criterion.","tokens_in":14345,"tokens_out":3352,"would_cite":false,"duration_ms":29526,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PCA rotation reveals which weight channels deserve full precision, improving LLM quantization.","keywords":["large language models","post-training quantization","weight quantization","saliency-aware quantization","PCA projection","rotational invariance","mixed-precision quantization","LLM inference"],"falsifier":"Take a layer from LLaMA2-7B, compute the PCA rotation, and find a channel with a low eigenvalue whose weights have unusually large absolute values; quantize that channel to INT3 while keeping a top-eigenvalue channel in FP16, and compare perplexity to the ROSAQ assignment. If the low-variance, high-weight channel degrades output more than the top-eigenvalue channel, the top-K saliency rule is false.","tokens_in":13350,"feed_emoji":"🔄","tokens_out":6954,"duration_ms":61505,"temperature":0.7,"pith_summary":"This paper argues that when quantizing a large language model, the channels to protect with higher precision are best found after rotating the weight matrix by the principal components of its input activations, not before. It proposes ROSAQ, which uses the rotational invariance of linear layers to rewrite each layer as $(XR)(R^T W)$, obtains $R$ from PCA on a calibration set, and keeps the $K$ channels with the largest eigenvalues in FP16 while quantizing the rest to INT3 or INT4. Because the top-principal dimensions concentrate activation variance, the paper claims they are naturally the salient features whose quantization error matters most. On LLaMA2-7B, LLaMA3-8B, and Qwen2-7B, ROSAQ reports perplexity and zero-shot accuracy improvements over the activation-magnitude baseline and over GPTQ, SpinQuant, and AWQ, with a fused kernel giving about 2.3x speedup over FP16. The point would matter if true: a cheap, data-only rotation could decide where to spend precision in compressed LLMs without retraining.","feed_headline":"Rotate first, then quantize: PCA picks the channels to keep","feed_subtitle":"Keeping top-variance channels in FP16 while quantizing the rest improves perplexity and speeds up LLaMA/Qwen inference.","key_machinery":"The carrying identity is the rotational invariance of linear layers, $XW = (XR)(R^T W)$, together with the PCA eigendecomposition $X^T X = R\\Lambda R^T$ used to choose $R$. The eigenvalues of the calibration activation covariance rank the rotated channels; the $K$ channels with the largest eigenvalues are designated salient, kept in FP16, while the rest are quantized to INT3/INT4 in groups of 128. This machinery transforms saliency detection from a per-channel magnitude heuristic into a variance-maximizing projection, and the head-wise variant applies the same idea separately to each attention head's representation.","core_discovery":"The central discovery is that saliency for weight quantization should be defined in the PCA-projected feature space rather than the original activation space. Concretely, the paper constructs a rotation matrix $R$ from the eigendecomposition of the calibration covariance $X^T X = R\\Lambda R^T$, rotates the weights to $R^T W$, and declares the channels corresponding to the $K$ largest eigenvalues salient. Those channels are kept in FP16 and the remaining channels are quantized in INT3/INT4 groups of 128. The paper argues that because PCA maximizes variance, the projected principal channels have larger and more separated activation magnitudes than magnitude-based salient channels in the original space, and its experiments support this by showing lower perplexity and higher MMLU than the rotationless Mixed baseline and existing methods. For multi-head attention, head-wise PCA, with a separate rotation per attention head, is claimed to further improve over global PCA.","pith_inferences":["Beyond the paper, the selection rule is testable against weight-aware criteria: one could compare the top-eigenvalue channels with the channels whose INT3 quantization causes the largest output perturbation; if overlap is low, the variance-only rule is incomplete.","Because the PCA rotation is computed from a calibration set, the method's stability under different calibration data and sequence lengths is an open extension; a practical deployment would want the chosen salient channels to be invariant across calibration samples.","The same variance-maximizing selection could be applied to activation quantization or to the key/value cache, where outlier channels also concentrate; the paper itself notes weight-activation quantization and retrieval-augmented generation as future work.","If rotation makes saliency more concentrated, then ROSAQ and orthogonal-projection outlier-removal rotations are likely complementary rather than competing; combining them could let even fewer channels carry FP16."],"forward_implications":["If the PCA-rotation claim holds, a calibration set alone, with no re-training and no gradient information, is enough to locate the precision-critical channels in a transformer.","Protecting only the top-K eigenchannels in FP16 while quantizing the rest in INT3/INT4 gives lower WikiText-2 perplexity and higher zero-shot accuracy than quantizing all channels at low precision or using original-space activation magnitudes.","The head-wise PCA variant for multi-head attention improves over applying one global PCA to the concatenated heads in the paper's LLaMA2-7B comparison.","The mixed-precision layout is compatible with a fused decode kernel, yielding about 2.3x speedup over FP16 at batch size 64 and about 2x speedup at batch size 128.","At the aggressive INT3 setting, the method reports the highest MMLU among the compared methods on several model and task combinations, suggesting the benefit grows as precision drops."],"supporting_citations":[{"why":"Supplies the rotational invariance property of transformers that the method exploits, and provides the SpinQuant learned-rotation baseline.","marker":"Liu et al., 2024b"},{"why":"Defines saliency-aware weight quantization and the activation-magnitude channel selection that ROSAQ replaces, and provides the AWQ comparison and the Mixed rotationless baseline.","marker":"Lin et al., 2024"},{"why":"Provides the GPTQ post-training quantization method used as a primary comparison baseline.","marker":"Frantar et al., 2023"},{"why":"Motivates rotation-based quantization by removing outliers in rotated LLMs, the context ROSAQ builds on.","marker":"Ashkboos et al., 2024b"},{"why":"Contributes orthogonal-projection incoherence processing, an alternative rotation-based approach that ROSAQ distinguishes itself from.","marker":"Chee et al., 2024"},{"why":"Supplies the fused QUICK kernel used in the paper to measure ROSAQ's inference throughput speedup.","marker":"Kim et al., 2024b"}],"fun_headline_variants":["Rotate first, quantize smarter: PCA picks key channels","PCA rotation boosts LLM quantization: 2.3x speedup","Salient channels in rotated space beat original saliency","Head-wise PCA rotation improves mixed-precision quantization","Quantize after rotating: PCA finds the important channels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a channel's importance is fully determined by the variance it explains in the calibration input activations, so a low-variance channel containing large or otherwise sensitive weights would be unprotected.","fun_headline_variants_meta":{"raw":{"variants":["Rotate first, quantize smarter: PCA picks key channels","PCA rotation boosts LLM quantization: 2.3x speedup","Salient channels in rotated space beat original saliency","Head-wise PCA rotation improves mixed-precision quantization","Quantize after rotating: PCA finds the important channels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000743,"raw_usage":{"total_tokens":3315,"prompt_tokens":948,"completion_tokens":2367,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":2286}},"tokens_in":564,"tokens_out":2367,"duration_ms":17693,"temperature":1.0,"reasoning_tokens":2286,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:01:07.857179+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a layer from LLaMA2-7B, compute the PCA rotation, and find a channel with a low eigenvalue whose weights have unusually large absolute values; quantize that channel to INT3 while keeping a top-eigenvalue channel in FP16, and compare perplexity to the ROSAQ assignment. If the low-variance, high-weight channel degrades output more than the top-eigenvalue channel, the top-K saliency rule is false.","supporting_citations":[],"review_version":2}