{"id":"a3bfec8d-8c49-401c-83c4-74c4e6552fc7","arxiv_id":"2501.18154","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A graph neural network trained on Hessian-derived graphs of LLM weights adaptively assigns per-column quantization bit-widths, improving low-bit perplexity over GPTQ on LLaMA models.","lead":"The paper proposes MG-PTQ, a method that trains a graph neural network to assign different quantization bit-widths to different weight columns of large language models. It reports lower perplexity than the GPTQ baseline at 2-bit weight quantization on several LLaMA models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's compensation step is algebraically a no-op, so the stated method does not explain the reported gains over GPTQ.","rationale":"The reader's weakest_assumption already points at Algorithm 1's compensation step. I refine it: the step is not merely underived; it is algebraically self-cancelling. With E = (W−B)/Hc and W := W − E·Hc, any standard interpretation gives E·Hc = W−B, so W_block becomes B_block after B_block has already been fixed. Since later blocks are not modified, lines 15–16 have no effect on the returned B. This makes the stated method equivalent to per-column mixed-precision rounding without the claimed GPTQ-style output compensation. The central empirical claim in the abstract ('outperforms GPTQ') would still be meaningful if the GNN allocator alone produced Table I, but the paper neither says that nor provides code, so the relationship between the described algorithm and the reported numbers is unverifiable. My proposed check (run with and without lines 15–16; compare both to Table I) would settle whether the no-op is real and whether the reported PPL comes from the stated procedure. I therefore keep the reader's CONDITIONAL verdict: the authors must provide a non-cancelling update, a derivation, and preferably code; without these, acceptance is not warranted. This is not an accusation; it is a call for the minimal evidence that would make the central claim assessable.","tokens_in":9150,"tokens_out":12734,"duration_ms":126222,"concrete_test":"Implement Algorithm 1 exactly as written in PyTorch for LLaMA-7b with block size 128, Hc from Eq. 2, and the described t-bit quantizer, then run it twice: once with lines 15–16 and once with those lines deleted. Because of the algebraic cancellation the two runs must produce bit-identical B and identical perplexity; verify this. Then compare the resulting WikiText2 perplexity with Table I's 130.27 for 2.0 bits. If the identical runs do not reproduce 130.27, the stated algorithm is not the one that produced the table; if they do, the reported improvement over GPTQ must be attributed to the bit allocator alone, not to the claimed OBC compensation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that MG-PTQ outperforms GPTQ at low bits (Table I). A load-bearing condition for that claim is that Algorithm 1 is a valid statement of the quantization procedure, since no code is provided and the compensation step is not derived. That condition fails in lines 15–16: E := (W_block − B_block) / Hc_block and then W_block := W_block − E · Hc_block. For any standard meaning of '/' and '·' (right matrix division, elementwise division, or left pseudo-inverse), E·Hc_block = W_block − B_block, so the update sets W_block := B_block. Because B was already fixed on line 14 and columns b+β onward are never touched, the 'Block-wise Output Compensation (OBC)' has no effect on the returned quantized matrix B. The paper claims to follow GPTQ [16], but GPTQ propagates quantization error to the remaining columns using off-diagonal entries of the Hessian inverse; the written update does no such thing. Consequently, the algorithm as written reduces to per-column mixed-precision t-bit rounding with no error compensation, and the Table I numbers are not explained by the stated method. Either the authors ran a different, unspecified update, or the reported gains come solely from the GNN bit allocator and the chosen t-bit quantizer; neither the paper nor the algorithm supports the claim as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MG-PTQ, a post-training quantization method that uses a graph neural network (GNN) to assign per-column mixed-precision bit-widths to the weights of large language models. The GNN takes a Cholesky-decomposed Hessian as the adjacency matrix and weight-derived features as node features, and is trained to minimize blockwise quantization error subject to an average-bit-width penalty. Quantization is performed blockwise following a GPTQ-like procedure. Experiments on five LLaMA models at 1.6-2.5 bits on WikiText2 and C4 report perplexity improvements over GPTQ at 2 bits.","tokens_in":9413,"tokens_out":6569,"duration_ms":61197,"significance":"The central idea - learning a mixed-precision bit allocation with a GNN using second-order information - is timely, and the experimental coverage (five models, two datasets) is reasonably broad. The paper also introduces a concrete mechanism (Gumbel-Softmax approximate gradients) for training through discrete bit-width choices. However, the load-bearing algorithmic claim is not supported by the written procedure: the compensation step in Algorithm 1 cancels algebraically, and the ablation intended to isolate the GCN's contribution changes the input features along with the architecture. If the algorithm description were corrected and the ablations made controlled, the approach could be a useful contribution; as it stands, the reported gains over GPTQ are not explained by the stated method.","major_comments":[{"comment":"The 'Block-wise Output Compensation' step is algebraically a no-op. For any standard meaning of '/' (right matrix division, elementwise division, or left pseudo-inverse), E·Hc_{b:b+β,b:b+β} = W_{:,b:b+β} − B_{:,b:b+β}, so the update W_{:,b:b+β} := W_{:,b:b+β} − E·Hc_{b:b+β,b:b+β} sets the block equal to B_{:,b:b+β}, which was already fixed on line 14. Because columns beyond the current block are never touched, no quantization error is propagated to the remaining weights as in GPTQ's update. Consequently, Algorithm 1 as written reduces to per-column mixed-precision t-bit rounding without any error compensation, and the perplexity improvements over GPTQ reported in Table I cannot be explained by the stated algorithm. The authors must either provide a corrected compensation step (e.g., updating the remaining columns using the off-diagonal blocks of the inverse Hessian, as in GPTQ) together with a derivation, or revise the description and claims to reflect the actual procedure used.","section":"III-B, Algorithm 1 lines 15-16"},{"comment":"The ablation comparing GCN with MLP is not a controlled comparison. The MLP receives as input the 'second-order Hessian matrix' while the GCN receives weight-derived features (X_G^(0) from Eq. 3) as node features and uses the Hessian as adjacency. Thus the performance difference could be due to the different input features rather than the graph architecture. To support the conclusion that 'the GCN module is crucial', the MLP should be given the same node features as the GCN, or the GCN should be ablated by removing the message-passing step while keeping the input features identical.","section":"IV-C-1, Fig. 2(a)"},{"comment":"The baseline set is insufficient to support the claim of state-of-the-art performance. Only RTN, AWQ, and GPTQ are compared; recent low-bit PTQ methods such as BiLLM, PB-LLM, SpQR, and Slim-LLM are not included, and there are no GPTQ results at 1.6 or 1.8 bits to match MG-PTQ's bit levels. The abstract's claim that MG-PTQ 'outperforms previous state-of-the-art PTQ method GPTQ' may be true at 2 bits, but the broader claim of setting new benchmarks requires comparison with current SOTA methods at the same bit widths.","section":"Table I"}],"minor_comments":[{"comment":"The output of Algorithm 1 is described as 'binarized weights' but B contains mixed-precision values; this should be clarified.","section":"Algorithm 1"},{"comment":"The Cholesky factor Hc is lower-triangular and non-symmetric; using it directly as a GCN adjacency matrix is unconventional and the paper does not discuss how directed message passing is handled. The authors should justify this design choice or use a symmetrized version.","section":"III-B, Eq. (2) and (4)"},{"comment":"The table formatting is confusing because the GPTQ rows and MG-PTQ rows are not visually separated, and the bit labels 2.5, 2, 1.8, 1.6 appear to belong to GPTQ; also, GPTQ results at 1.6 and 1.8 bits are missing, which limits comparability.","section":"Table I"},{"comment":"Reproduction details are incomplete: the Gumbel-Softmax temperature schedule is not specified, the value of the Hessian regularizer λ in Eq. (2) is not reported, and the actual average bit-width achieved after training is not given.","section":"IV-A-3"},{"comment":"The reference list contains duplicates: [34] and [40] are the same GCN paper, and AWQ is listed as both [27] and [41] with different bibliographic details.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to have been prepared quickly: pseudocode inconsistencies, a missing derivation for the key update, duplicate references, and an uncontrolled ablation. I would recommend the editors require the authors to release code or provide a precise, corrected algorithm with a full derivation of the compensation step, since the current description is internally inconsistent and does not explain the reported results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper has a genuinely new idea—use a GNN to assign per-column bit-widths for LLM weight quantization—but the written algorithm's error-compensation step is algebraically a no-op, so the reported numbers are not explained by the stated method. It needs rigorous revision before I'd trust it.\n\nWhat's new: Applying a graph neural network to mixed-precision PTQ for LLMs is new relative to the cited literature. The framing of weight-column dependencies via a graph and using Gumbel-Softmax for differentiable bit allocation is sensible. The empirical table shows consistent 2-bit perplexity gains over GPTQ on LLaMA variants, which is a real signal if it reproduces.\n\nWhat it does well: The idea of treating bit allocation as a learned graph task is worth exploring; the loss formulation with quantization error plus bit-width penalty is clean; the efficiency analysis is a nice touch.\n\nSoft spots: The load-bearing problem is Algorithm 1. Lines 15–16 define E := (W−B)/Hc and then W := W − E·Hc. For any standard division/multiplication interpretation, this sets W := B, so the 'Block-wise Output Compensation' has zero effect and the output B is just per-column t-bit rounding. That's not GPTQ's update, and it means the Table I results cannot be explained by the written algorithm. Either the implementation does something different (and the paper must say what), or the gains come solely from the bit allocator and quantizer, which would need a clean experiment. In addition, the ablation compares GCN (node features = mean weight columns) to MLP (input = Hessian matrix), which holds two things different at once; the missing recent low-bit baselines (BiLLM, PB-LLM, Slim-LLM, SpQR) and no same-bit comparisons at 1.6/1.8 bits weaken the SOTA claim. None of these are fatal to the core idea, but the algorithm gap is fatal to the paper as written.\n\nWho it's for: PTQ researchers might take the GNN-for-bit-allocation concept as a starting point, but they can't rely on the results until the algorithm is corrected and code is released. A serious referee could push the authors to fix these issues, so I'd send it to review rather than desk reject, but I wouldn't cite it as-is.","headline":"Novel GNN-based bit-allocation idea, but Algorithm 1's compensation step is a no-op and the reported gains aren't explained by the written method.","tokens_in":9980,"tokens_out":2812,"would_cite":false,"duration_ms":26502,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a graph neural network can assign quantization bit-widths per weight column and beat GPTQ at 2-bit post-training quantization of LLaMA models.","keywords":["Post-Training Quantization","Graph Neural Networks","Mixed-Precision Quantization","Large Language Models","Low-bit Quantization","LLaMA","Hessian-based Quantization"],"falsifier":"Run the same GNN allocator on LLaMA-7b at 2.0 bits but replace the Algorithm 1 update with GPTQ's standard Hessian-inverse compensation; if WikiText2 perplexity jumps back to GPTQ's roughly 152 rather than staying near MG-PTQ's 130, the claimed gain comes from the unusual update, not from the GNN allocation.","tokens_in":8902,"feed_emoji":"⚡","tokens_out":6122,"duration_ms":55510,"temperature":0.7,"pith_summary":"Post-training quantization of large language models degrades sharply below 3 bits per weight. This paper claims that a graph neural network can decide, column by column, how many bits each weight column deserves, and that this mixed-precision allocation beats the standard GPTQ method at the same average bit budget. The supporting evidence is Table I: at 2.0 bits, MG-PTQ reports lower perplexity than GPTQ on every tested LLaMA model on both WikiText2 and C4. The practical stake is that 2-bit deployment of models in the 7B-to-13B range may become usable without retraining. The paper also claims that the graph component itself, not just the bit-allocation classifier, drives the improvement, based on an ablation that replaces the GCN with an MLP.","feed_headline":"GNN bit allocator beats GPTQ for 2-bit LLaMA models","feed_subtitle":"At 2.0 bits, MG-PTQ reports lower perplexity than GPTQ on five LLaMA variants and two datasets.","key_machinery":"The load-bearing object is the feature graph built from the Cholesky factor $H_c = \\mathrm{Cholesky}((2X_F^T X_F+\\lambda I)^{-1})$, where $X_F$ is calibration data. Each column of the target weight matrix is a node; $H_c$ plays two roles at once: as the weighted adjacency matrix in the GCN propagation $X_G^{(l+1)}=\\sigma(H_c X_G^{(l)} W^{(l)})$ and as the divisor and multiplier in the blockwise compensation update $E:=(W-B)/H_c$, then $W:=W-E\\cdot H_c$ (Algorithm 1). The GCN's message passing is meant to let each column's bit assignment reflect dependencies on other columns, and the Gumbel-Softmax approximation supplies gradients through the discrete bit-width choice. The training loss combines quantization error with a mean-squared-error constraint on average bit-width, so the allocator can be tuned to a target bit budget.","core_discovery":"The central claim is that weight-column importance for quantization can be learned by message passing over a graph whose adjacency matrix is the Cholesky factor of the inverse second-order Hessian, and that the resulting per-column bit assignment yields lower perplexity than GPTQ's uniform low-bit quantization. Concretely, MG-PTQ forms one node per column of the target weight matrix, uses mean-pooled column values as node features, propagates them through a two-layer GCN, and maps the output to bit-widths per column. The same Cholesky factor is then used in the blockwise error-compensation update inherited from the GPTQ-style pipeline. Table I reports the headline numbers: at 2.0 bits, MG-PTQ achieves perplexity of 130.27 on LLaMA1-7b on WikiText2 versus GPTQ's 152.31, with similar improvements across LLaMA1-13b, LLaMA2-7b, LLaMA2-13b and LLaMA3-8B on both WikiText2 and C4. The authors interpret these results as showing that GNN-based importance perception captures dependencies among weights that uniform or heuristic schemes miss.","pith_inferences":["The same learned importance scores could be reused for pruning or activation quantization, since the GCN is effectively learning which weight columns are costly to distort; this is a natural follow-up the paper does not test.","A head-to-head against salience-driven mixed-precision heuristics rather than uniform GPTQ would separate what the learned allocator adds over a handcrafted importance rule such as activation magnitudes.","The unproven compensation update is the place to look first: a direct check of layer-wise reconstruction error on a single LLaMA layer would settle whether the gains come from the allocator or from an undocumented error-correcting effect of the Cholesky factor."],"forward_implications":["At 2.0 bits, MG-PTQ reports lower perplexity than GPTQ on every LLaMA variant tested, on both WikiText2 and C4, so the claimed advantage is consistent across models rather than a single favorable case.","Because average bit-width is controlled by an MSE penalty, the same trained allocator can be re-targeted to different budgets such as 1.6, 1.8, 2.0, and 2.5 bits without re-architecting the method.","The ablation with an MLP in place of the GCN shows a significant perplexity drop, which supports the paper's claim that graph propagation itself, not merely per-column classification, drives the improvement.","The efficiency analysis reports quantization time close to GPTQ's, so the method is presented as practical for deployment-scale models."],"supporting_citations":[{"why":"Supplies the blockwise quantization pipeline, the Hessian-based Cholesky preconditioning, and the GPTQ baseline that MG-PTQ claims to outperform.","marker":"[16]"},{"why":"Defines the LLaMA model family and gives the full-precision perplexity anchors used in Table I.","marker":"[2]"},{"why":"Provides the WikiText2 evaluation set.","marker":"[21]"},{"why":"Provides the C4 evaluation set.","marker":"[22]"},{"why":"Serves as one of the baselines whose very high perplexity at 2 bits motivates the low-bit problem.","marker":"[41]"},{"why":"Supplies the Gumbel-Softmax approximation that gives gradients through the discrete bit-width choice.","marker":"[42]"},{"why":"Supplies the two-layer GCN message-passing architecture used for weight-importance perception.","marker":"[34]"}],"fun_headline_variants":["GNN-driven bit assignment beats GPTQ at 2-bit","Graph neural net learns bit widths, tops GPTQ","MG-PTQ: GNN allocates bits, GPTQ beaten","Low-bit LLMs: GNN beats GPTQ on perplexity","Graph-based quantization outscores GPTQ at 2 bits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result stands on the unproven claim that the same Cholesky factor can both carry information in the graph network and serve as the divisor and multiplier in the blockwise error-correction update; if that correction is mathematically unsound, the reported perplexity gains are not explained by the stated algorithm.","fun_headline_variants_meta":{"raw":{"variants":["GNN-driven bit assignment beats GPTQ at 2-bit","Graph neural net learns bit widths, tops GPTQ","MG-PTQ: GNN allocates bits, GPTQ beaten","Low-bit LLMs: GNN beats GPTQ on perplexity","Graph-based quantization outscores GPTQ at 2 bits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000222,"raw_usage":{"total_tokens":1461,"prompt_tokens":959,"completion_tokens":502,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":416}},"tokens_in":575,"tokens_out":502,"duration_ms":4736,"temperature":1.0,"reasoning_tokens":416,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T00:27:32.509798+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same GNN allocator on LLaMA-7b at 2.0 bits but replace the Algorithm 1 update with GPTQ's standard Hessian-inverse compensation; if WikiText2 perplexity jumps back to GPTQ's roughly 152 rather than staying near MG-PTQ's 130, the claimed gain comes from the unusual update, not from the GNN allocation.","supporting_citations":[{"cited_title":"Pointer sentinel mixture models,","cited_arxiv_id":null,"evidence_quote":"Provides the WikiText2 evaluation set."},{"cited_title":"Categorical reparameterization with gumbel-softmax,","cited_arxiv_id":null,"evidence_quote":"Supplies the Gumbel-Softmax approximation that gives gradients through the discrete bit-width choice."}],"review_version":1}