{"id":"8ae5e614-df9a-4751-ac14-5dc181e67216","arxiv_id":"2505.00259","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A Hessian-guided packing scheme for post-training quantization that jointly reconstructs groups of blocks and adds pack-based mixed precision, claiming state-of-the-art low-bit accuracy.","lead":"Pack-PTQ quantizes neural networks down to very low bit widths by grouping network blocks into 'packs' and reconstructing each pack jointly. It reports large accuracy gains over prior post-training quantization methods on image and point cloud classifiers, though its theoretical derivation is flawed and some baseline numbers appear implausibly low.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed SOTA superiority rests on baseline numbers far below published and internally implausible values; corrected baselines would likely erase the reported margins.","rationale":"The reader's stated weakest assumption is the Hessian approximation in Theorem 1. That is a real weakness: Eq. (5) replaces tr(Hσ²I) with μ_H σ² n, which holds only if every diagonal entry equals the global mean, and quantization noise is not Gaussian i.i.d. However, the paper's practical contribution is the SOTA accuracy claim; even a repaired Theorem 1 would not salvage the paper if the baseline numbers are wrong. Conversely, the baseline issue alone is decisive: several reported baseline accuracies (0.01%, 0.14%, 0.59% in W3/A3 ViT) are far below published ranges and internally inconsistent with the same methods' W4/A4 results. The strongest claim therefore lacks a trustworthy empirical foundation. I keep the reader's REJECT verdict. The recommendation is UNCHANGED because this stress-test identifies the same bottom line rather than a different verdict.","tokens_in":12302,"tokens_out":3700,"duration_ms":36446,"concrete_test":"Run the released official implementations of PTQ4ViT, RepQ-ViT, PD-Quant, QDrop, and BRECQ on ImageNet W3/A3 for ViT-B and DeiT-S using their standard calibration protocols (same 1024-sample calibration data and preprocessing), and compare with Pack-PTQ's Table 2 numbers. As a minimal check, reproduce the PTQ4ViT ViT-B W3/A3 result from its own released code; if it is not near 0.01%, the comparison protocol in this paper is invalid and the claimed SOTA margins collapse.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical superiority in W3/A3 and W4/A4. The comparison tables undercut that claim. In Table 2, PTQ4ViT is reported at 0.01% on ViT-B W3/A3, RepQ-ViT at 0.14%, and BRECQ at 0.59%; the same methods reach 30-68% at W4/A4. A one-bit reduction does not plausibly collapse accuracy from 68.48% to 0.14% (RepQ-ViT ViT-B). These near-zero numbers are likely artifacts of the evaluation protocol (e.g., calibration setup, input ranges, or using CNN-oriented quantizers without ViT-specific adaptations), not genuine method failures. If baselines should be in the published 40-70% range, Pack-PTQ's reported advantages of tens of percentage points are an artefact of a broken comparison. The packing ablation in Table 4 shows that grouping blocks helps under the authors' protocol, but that does not validate superiority against correctly run baselines. The theoretical derivation in Theorem 1 is also fragile (i.i.d. Gaussian noise and Hessian mean approximations), but a flawed proof could be corrected or set aside if the empirical comparison were sound; the baseline comparison is the load-bearing part of the paper's headline claim and it is not credible as reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes Pack-PTQ, a post-training quantization method. It computes per-block Hessian-guided importance scores, partitions blocks into contiguous non-overlapping packs, reconstructs the network pack-wise, and assigns mixed precisions to packs. The paper reports experiments on ImageNet with CNN and ViT architectures and on ModelNet40 with PointNet, claiming consistent superiority over existing PTQ methods. The core novelty is the packing granularity and the Hessian-guided scoring; the paper also includes an ablation study and an execution-time analysis.","tokens_in":12628,"tokens_out":6357,"duration_ms":58685,"significance":"If the results were valid, pack-wise reconstruction would be a useful contribution to post-training quantization, and the combination of Hessian-guided packing with mixed precision could improve low-bit performance across architectures. The paper's coverage of CNNs, ViTs, and point-cloud networks, as well as its inclusion of ablations, is appropriate. However, the central empirical claims are not credible as reported: the baseline scores in Table 2 are far below published results and at or below chance level in several cases, and the point-cloud results in Table 3 show quantized models matching or exceeding full-precision accuracy at 2-bit weights without explanation. The theoretical derivation supporting the packing criterion is also mathematically incorrect. These issues affect load-bearing parts of the contribution.","major_comments":[{"comment":"Table 2 reports near-zero W3/A3 accuracies for baselines, e.g., PTQ4ViT at 0.01% on ViT-B and DeiT-S, RepQ-ViT at 0.14% on ViT-B, and BRECQ at 0.42% on ViT-S. These are at or below the 0.1% random-guess level for ImageNet and are orders of magnitude below the same methods' W4/A4 results in the same table (e.g., RepQ-ViT ViT-B reaches 68.48% at W4/A4). A one-bit reduction does not plausibly cause such a collapse. Because the paper's headline claim of superiority is based on comparisons to these baselines, the empirical comparison is not credible as reported.","section":"Table 2, W3/A3 rows"},{"comment":"Table 3 reports Pack-PTQ at W3/A3 with mixed precision (92.28 mAcc, 89.53 OA) and at W4/A4 with mixed precision (92.21 mAcc, 88.56 OA), both at or above the stated full-precision PointNet accuracy of 92.01 mAcc and 88.54 OA. Even W2/A4 without mixed precision loses only 0.45 mAcc and 0.17 OA. Quantization that improves on full precision, especially at 2-bit weights, requires an explanation such as a regularizing effect or an evaluation artifact, but none is provided; this makes the point-cloud evidence difficult to credit.","section":"Table 3"},{"comment":"Equation (5) asserts tr(H(z)σ²I) ≈ μ_H(z)σ²n, but tr(H(z)σ²I)=σ²Σ_i H_ii(z); equating this to μ_H(z)σ²n requires the mean of the diagonal entries to equal the mean of all entries of H, which is not true for a general Hessian. The proof also assumes Δz has i.i.d. Gaussian components, whereas quantization noise is deterministic and input-dependent. Since Eq. (8) uses this approximation to define the importance score S that drives the adaptive packing, the theoretical grounding of the packing mechanism is invalid as stated.","section":"Theorem 1, Eq. (5)"},{"comment":"Equation (11) defines the mixed-precision objective in terms of L_q^j[i], the quantization loss of the i-th block in pack j, but the bit-width assignment problem is solved before the pack reconstruction is performed. The paper does not specify how L_q^j[i] is computed for candidate bit-widths or whether it depends on the very quantizers being optimized, which creates a circularity concern. The memory constraint C and the candidate bit-width set K used in the experiments are also not reported, so the mixed-precision results cannot be interpreted as a constrained optimization.","section":"Section 4.2, Eqs. (10)-(11)"},{"comment":"The abstract and Section 5.2 state that Pack-PTQ consistently outperforms existing methods in both W3/A3 and W4/A4 settings, but Table 2 shows Adalog outperforming Pack-PTQ at W4/A4 on ViT-S (72.13 vs 61.03 w/o MP and 66.83 with MP) and ViT-B (79.16 vs 75.71 w/o MP and 77.89 with MP). The claim should be qualified to match the reported table.","section":"Abstract and Section 5.2"}],"minor_comments":[{"comment":"There are typographical and grammatical errors in this section, including 'quantitation parameters' and 'the block-wise overlooks the cross-block relationships'; these should be corrected.","section":"Section 5.3"},{"comment":"The section heading 'Conculsion' is misspelled and should read 'Conclusion'.","section":"Section 6"},{"comment":"References [19] and [22] are the same QDrop paper listed twice, and the citation formatting is inconsistent between the two entries.","section":"References"},{"comment":"The source of the pre-trained PointNet model and the exact calibration protocol for ModelNet40 should be stated; the reported full-precision accuracy of 92.01 mAcc and 88.54 OA should also be reconciled with commonly reported PointNet accuracy.","section":"Section 5.1.2"},{"comment":"The values in the bar chart are difficult to read in the current figure; numerical labels or a larger figure would improve clarity.","section":"Figure 1"}],"recommendation":"reject","confidential_remarks":"The manuscript is not ready for publication. The baseline comparison in Table 2 is the main pillar of the claimed empirical superiority, and the implausible near-zero baseline accuracies suggest a systematic evaluation problem that would require rerunning the experiments. The point-cloud results also demand a careful explanation. Since the necessary fixes are not local edits but a reworking of core evidence, I recommend rejection rather than major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's core idea is legitimate: grouping adjacent blocks into packs and reconstructing them jointly, with a Hessian-inspired score to choose the grouping, is a natural extension of block-wise PTQ. The ablations show that packing helps under the authors' protocol, and the HAda heuristic beats random or fixed-size packing. That part is credible and worth taking seriously.\n\nThe problems are in the evaluation and the theory. Table 2 reports near-zero accuracies for established ViT methods at W3/A3 (e.g., PTQ4ViT at 0.01%, RepQ-ViT at 0.14%) when the same methods are at 30–68% at W4/A4. A one-bit drop does not produce that collapse; these numbers are almost certainly artifacts of a mismatched evaluation protocol. With baselines that low, the headline 'state-of-the-art' margins are meaningless. The point cloud results have a similar smell: MinMax at W4/A4 beats the full-precision model on OA, which is not credible under a fixed protocol.\n\nThe theory is also fragile. Theorem 1's proof relies on tr(H σ² I) ≈ μ_H σ² n, which equates the mean diagonal of the Hessian with the mean of all entries. That is false for a general Hessian. The i.i.d. Gaussian noise assumption is also hard to justify for quantization error. The mixed-precision objective has a circular flavor: Eq. (11) uses L_q^j[i], the quantization loss of a block, which depends on the bit-width being selected. The paper never says how that loss is computed before bit assignment.\n\nWhat would change my mind: corrected baseline numbers, a fixed or discarded Theorem 1 (the packing idea can stand on empirical ablations alone), and a clarification of the sensitivity metric. If the gains survive honest baselines, this is a useful paper for the PTQ community. As written, the central claim is not credible.\n\nI would not accept this in current form, but I would send it to peer review with a strong request to fix the comparison and the proof. It deserves a serious referee because the packing concept is new and the ablations suggest it works; the flaws are fixable.","headline":"Pack-wise reconstruction is a genuine idea, but the paper's headline results rest on implausible baseline numbers and a shaky Hessian approximation; fixable, but not credible as written.","tokens_in":13112,"tokens_out":3493,"would_cite":false,"duration_ms":32754,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pack-PTQ claims that quantizing networks in Hessian-guided packs, rather than block by block, keeps low-bit models close to full-precision accuracy.","keywords":["post-training quantization","cross-block dependency","Hessian-guided packing","mixed-precision quantization","low-bit quantization","vision transformers","point cloud classification","block reconstruction"],"falsifier":"On ResNet18 at W3/A3, compute each block's importance score with exact Hessian-vector products instead of the Gaussian approximation, run the same packing and reconstruction, and compare top-1 accuracy with the reported 64.46% without mixed precision; if exact-Hessian packing does not reproduce or exceed that number, the approximation, not pack-wise reconstruction itself, is what matters.","tokens_in":12081,"feed_emoji":"🧩","tokens_out":9765,"duration_ms":94739,"temperature":0.7,"pith_summary":"This paper tries to establish that the standard reconstruction unit in post-training quantization—one block of layers—is the wrong granularity for low-bit compression. Its proposal, Pack-PTQ, scores each block with a Hessian-derived importance measure, groups consecutive blocks into non-overlapping packs, and reconstructs each pack so that its quantized output matches the full-precision output, thereby preserving dependencies between blocks. A second component assigns different bit-widths to different packs under a memory constraint. The paper reports ImageNet results across CNNs and vision transformers, including ViT-S at W3/A3 rising from 0.42% with block-wise reconstruction to 55.56% with packing and mixed precision, and near-full-precision accuracy on ModelNet40 point cloud classification with PointNet. The payoff, if the claim holds, is accurate 3-bit and 4-bit deployment without retraining, using only a small calibration set.","feed_headline":"Packing blocks rescues low-bit quantization accuracy","feed_subtitle":"Hessian-guided pack-wise reconstruction outperforms block-wise PTQ across CNNs, ViTs, and PointNet.","key_machinery":"The load-bearing object is the Hessian-guided block importance score $S = \\mu_H(z) \\approx \\mathbb{E}[2(L_q^b - \\Delta z^\\top g(z))]\\,/\\,\\mathbb{E}[\\Delta z^\\top \\Delta z]$, derived from a second-order Taylor expansion of the loss change when a block's output $z$ is perturbed by quantization noise $\\Delta z$. Theorem 1 justifies reading $\\mu_H(z)$ as the mean of all entries of the Hessian $H(z)$ by assuming i.i.d. Gaussian $\\Delta z$ and approximating $\\operatorname{tr}(H(z)\\sigma^2 I)$ by $\\mu_H(z)\\sigma^2 n$. Starting from the last block, the packing algorithm finds the lowest-scoring block in the remaining range and makes a pack from that block to the end, then repeats; this yields non-overlapping packs as the reconstruction unit. The second mechanism is pack-level mixed precision: bit-widths are chosen to maximize $\\sum_j b_j\\Omega_j$ under a memory budget, where $\\Omega_j$ averages block sensitivities and quantization losses in pack $j$. These two mechanisms are what the paper credits for capturing cross-block dependency while keeping the reconstruction tractable.","core_discovery":"The central claim is that cross-block dependency, not just per-block sensitivity, limits low-bit post-training quantization, and that an adaptive packing mechanism can preserve that dependency without reconstructing the entire network. Pack-PTQ computes a Hessian-guided importance score for each block, then repeatedly selects the lowest-scoring block in the not-yet-packed range and groups it with all later blocks, so every pack begins with a relatively insensitive block and contains the blocks that follow it. The paper argues that jointly reconstructing such packs—aligning quantized pack outputs with full-precision pack outputs—yields more accurate quantization parameters than block-wise reconstruction. On top of this, pack-level mixed precision solves a constrained optimization that gives more bits to high-sensitivity packs. The reported experiments claim consistent gains at W3/A3 and W4/A4 on ImageNet CNN and vision transformer models, with especially large margins on transformers, and stable near-full-precision results on ModelNet40.","pith_inferences":["The authors do not isolate whether the Hessian approximation or the optimization effect of jointly reconstructing more layers drives the gains; a version using exact Hessian-vector products, or scores replaced by random ones under the same pack sizes, would separate the two.","Because every pack starts at a low-importance block and includes all later blocks up to the previous pack boundary, the method may implicitly place quantization 'buffers' at pack boundaries; the paper does not test whether boundary position, rather than internal dependency, explains part of the benefit.","The packing idea should extend naturally to detection or segmentation backbones and to large language models, where cross-layer dependencies are also strong; that is an extrapolation, not a claim in the paper."],"forward_implications":["If the central claim is right, the reconstruction granularity for PTQ shifts from fixed single blocks to adaptive, variable-size packs, so methods no longer need to choose between block-wise efficiency and network-wise dependency capture.","Low-bit vision transformers become practical: the paper reports W3/A3 ViT accuracy far above prior PTQ baselines, for example DeiT-S at 58.38% with mixed precision versus 29.33% for PD-Quant.","Pack-level mixed precision gives a principled way to spend a memory budget: high-sensitivity packs get more bits, low-sensitivity packs get fewer, under an explicit constraint.","The same recipe transfers to a new domain, 3D point cloud classification, where the paper reports W3/A3 accuracy within about half a point of full precision and even above it with mixed precision."],"supporting_citations":[{"why":"Supplies the block-wise reconstruction baseline and the Hessian approximation that Pack-PTQ extends to packs.","marker":"[14]"},{"why":"Provides a QDrop baseline for low-bit PTQ, including vision transformer results that Pack-PTQ compares against.","marker":"[19]"},{"why":"Provides PTQ4ViT, a vision-transformer-specific quantization baseline that Pack-PTQ must beat in low-bit settings.","marker":"[17]"},{"why":"Supplies PD-Quant, a global-dependency-aware PTQ method that Pack-PTQ contrasts with its pack-wise dependency capture.","marker":"[25]"},{"why":"Supplies the Hessian-spectrum sensitivity idea that motivates pack-level mixed-precision bit assignment.","marker":"[28]"},{"why":"Frames mixed-precision assignment as a constrained optimization problem, the form Pack-PTQ adopts for packs.","marker":"[27]"},{"why":"Provides the MinMax baseline and the adaptive-rounding framework used in the point cloud classification experiments.","marker":"[18]"},{"why":"Sets the training protocol (batch size, iterations, learning-rate schedule) that Pack-PTQ follows in its evaluations.","marker":"[37]"}],"fun_headline_variants":["Pack-wise reconstruction preserves cross-block info in PTQ","Hessian-guided packing boosts low-bit quantization accuracy","Adaptive packs cut PTQ error by keeping block dependencies","Pack-PTQ: Group blocks by sensitivity for sharper low-bit models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The packing mechanism rests on the approximation that a block's Hessian can be summarized by its average entry, computed as if quantization noise were independent Gaussian; real quantization noise is not Gaussian and general Hessians need not satisfy that equality, so the scores that decide pack boundaries are only as sound as that shortcut.","fun_headline_variants_meta":{"raw":{"variants":["Pack-wise reconstruction preserves cross-block info in PTQ","Hessian-guided packing boosts low-bit quantization accuracy","Adaptive packs cut PTQ error by keeping block dependencies","Pack-PTQ: Group blocks by sensitivity for sharper low-bit models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000236,"raw_usage":{"total_tokens":1484,"prompt_tokens":906,"completion_tokens":578,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":522,"completion_tokens_details":{"reasoning_tokens":510}},"tokens_in":522,"tokens_out":578,"duration_ms":5913,"temperature":1.0,"reasoning_tokens":510,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:46:52.290469+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On ResNet18 at W3/A3, compute each block's importance score with exact Hessian-vector products instead of the Gaussian approximation, run the same packing and reconstruction, and compare top-1 accuracy with the reported 64.46% without mixed precision; if exact-Hessian packing does not reproduce or exceed that number, the approximation, not pack-wise reconstruction itself, is what matters.","supporting_citations":[{"cited_title":"BRECQ: Pushing the limit of post-training quantization by block reconstruction","cited_arxiv_id":null,"evidence_quote":"Supplies the block-wise reconstruction baseline and the Hessian approximation that Pack-PTQ extends to packs."},{"cited_title":"PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization","cited_arxiv_id":null,"evidence_quote":"Provides PTQ4ViT, a vision-transformer-specific quantization baseline that Pack-PTQ must beat in low-bit settings."},{"cited_title":"PD-Quant: Post-training quantization based on prediction difference metric","cited_arxiv_id":null,"evidence_quote":"Supplies PD-Quant, a global-dependency-aware PTQ method that Pack-PTQ contrasts with its pack-wise dependency capture."},{"cited_title":"HAWQ: Hessian aware quantization of neural networks with mixed-precision","cited_arxiv_id":null,"evidence_quote":"Supplies the Hessian-spectrum sensitivity idea that motivates pack-level mixed-precision bit assignment."},{"cited_title":"Towards mixed-precision quantization of neural networks via constrained optimization","cited_arxiv_id":null,"evidence_quote":"Frames mixed-precision assignment as a constrained optimization problem, the form Pack-PTQ adopts for packs."},{"cited_title":"Up or down? adaptive rounding for post-training quantization","cited_arxiv_id":null,"evidence_quote":"Provides the MinMax baseline and the adaptive-rounding framework used in the point cloud classification experiments."}],"review_version":1}