{"id":"1e0e5542-0507-4ba5-acd5-49d5b87e8bd8","arxiv_id":"2511.04805","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A training-free method that merges pairs of MoE experts at the individual-weight level and packs the required masks into unused exponent bits, cutting expert memory by 50% with minimal accuracy loss.","lead":"PuzzleMoE cuts the memory of large mixture-of-experts language models by pairing up similar experts and keeping only the important weights, with the bookkeeping hidden in unused exponent bits. It reports roughly 50% expert-memory reduction with under two points of average accuracy loss, and faster inference on fewer GPUs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Bit-packing losslessness is validated only for Mixtral/DeepSeek via WikiText-2; for Qwen models the exponent-window assumption is supported only by plots, so the metadata-free 50% compression claim is not established across the claimed model zoo.","rationale":"The central claim is that PuzzleMoE achieves 50% expert compression with no metadata overhead while maintaining accuracy. The merging algorithm itself appears internally consistent: Eq. (7)–(8) are coherent, routing can remain unchanged because each original expert is decoded from its merged pair, and the multi-seed ablations in Section 5 show robustness to grouping. The load-bearing weak point is the bit-packing losslessness. Section 3.2.2 requires all merged-weight exponents to lie in [112,143] and clamps smaller exponents to 112. Table 1 validates this only for Mixtral and DeepSeek using WikiText-2 perplexity, an aggregate metric that may miss task-specific perturbations. For Qwen1.5 and Qwen3, only exponent-distribution plots are provided, with no before/after packing measurement. This is exactly the reader's weakest_assumption, so I agree with the reader's identification. The gap is concrete: if the Qwen models have nontrivial exponent mass outside the window, the metadata-free 50% compression claim fails for them, and the reported Qwen accuracy may not correspond to the actual deployed artifact. This does not require rejecting the paper, but it does require additional validation. The reader's CONDITIONAL verdict already captures this, so I recommend no change. A single focused test—measuring out-of-window exponent fractions and comparing packed vs. unpacked task accuracy on the Qwen models—would either remove the concern or force a revised compression claim.","tokens_in":21543,"tokens_out":16251,"duration_ms":150348,"concrete_test":"For Qwen1.5-MoE-A2.7B and Qwen3-MoE-30B-A3B at 50% sparsity, instrument the packing step to compute the fraction and maximum magnitude of merged-weight entries whose exponent is <112 or >143 before packing. Then run the full seven-benchmark suite and WikiText-2 in two configurations: (a) exact reconstruction with masks/signs stored as separate tensors, and (b) the end-to-end bit-packed kernel. If per-task accuracy differences stay within the seed-level noise reported in Table 2 (±0.2–0.4 average) and the out-of-window fraction is negligible (e.g., <0.1%), the packing assumption is validated; otherwise the compression ratio or accuracy claim must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim (Section 3.2.2, Algorithm 1) is that masks and signs for two experts can be embedded in the freed exponent/sign bits of a single bfloat16 merged weight, so 50% expert sparsity costs no extra metadata. This is only lossless if every merged-weight exponent lies in [112,143] after the Eq. (7) construction, and if clamping smaller exponents to 112 is harmless. Table 1 reports WikiText-2 perplexity before/after packing for Mixtral-8x7B and DeepSeek-MoE only. For Qwen1.5-MoE-A2.7B and Qwen3-MoE-30B-A3B, Appendix B.1 gives only exponent-distribution plots; no numeric perplexity or task-accuracy comparison before/after packing is provided. The merging operation can produce merged weights with different tail statistics than the original expert weights, so the packing may introduce non-negligible reconstruction error on those models. If so, the reported Qwen results either do not reflect the bit-packed deployment artifact, or the 'no metadata overhead' claim fails because auxiliary storage would be needed to preserve accuracy. The concern is concrete and testable, not a demonstrated failure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"PuzzleMoE proposes a training-free, task-agnostic MoE compression method. Pairs of experts are merged element-wise using a similarity mask and an activation-saliency mask; the merged weights are stored in bfloat16, with the mask and sign bits packed into the freed exponent/sign bits, and a custom GEMV kernel decodes them on the fly. Experiments on four MoE models (Mixtral-8x7B, DeepSeek-MoE, Qwen1.5-MoE-A2.7B, Qwen3-MoE-30B-A3B) across WikiText-2, seven zero-shot tasks, and math benchmarks report near-baseline accuracy at 25% and 50% expert sparsity, outperforming prior merging/dropping methods by up to 16.7 MMLU points, with compression taking minutes and inference speedup of 1.19–1.28x.","tokens_in":21840,"tokens_out":12394,"duration_ms":118154,"significance":"If the reported results are reproducible, PuzzleMoE is a practically important contribution: it is training-free, uses a single C4 calibration pass, shows consistently small accuracy drops across four different MoE families, and avoids storing mask/sign metadata. The evaluation is stronger than typical for this area: 16 seeds with reported standard deviations, code release, and several baselines. The core merging equations are coherent. The main technical risk is the bit-packing validity on two of the four models, and the abstract contains an unsupported 45x compression-speedup number.","major_comments":[{"comment":"The metadata-free 50% compression claim is load-bearing and depends on every exponent of the merged tensor W_merged (Eq. 7) lying in [112,143] after shifting, so Algorithm 1 can pack two masks and two signs into freed bits. Table 1 validates packing only for Mixtral-8x7B and DeepSeek-MoE via WikiText-2 perplexity. For Qwen1.5-MoE and Qwen3-MoE, Appendix B.1 shows only exponent plots of original expert weights, not numeric before/after packing perplexity or task accuracy, and not exponent statistics of the merged tensors. Merging can change tail statistics, so the Table 2 Qwen results do not yet establish that the bit-packed artifact is lossless. Please report before/after packing perplexity and accuracy for all four models, the count/percentage of W_merged exponents outside [112,143], and state explicitly whether Table 2 used packed or un-packed inference.","section":"§3.2.2, Table 1, Appendix B.1"},{"comment":"The '45x faster compression' claim is not supported by the reported data. The text states PuzzleMoE takes 2 minutes for Mixtral-8x7B and D2 takes 55 minutes, i.e. a 27.5x reduction (55/2), not 45x. No other timing pair in Fig. 4(a) yields 45x. Please correct the claim or supply the measurement that supports 45x.","section":"§4.4, Fig. 4(a), abstract and contribution bullet"}],"minor_comments":[{"comment":"The passage says PuzzleMoE retains 99%, 92%, and 84% of baseline accuracy on Math-500, AIME24, and AIME25. The actual ratios are 96.2/97.2=99.0%, 71.1/83.3=85.4%, and 61.5/72.9=84.4%, respectively. The 92% figure for AIME24 should be corrected to about 85%.","section":"Table 4"},{"comment":"The bit positions for the packed masks/signs ('expert pos' mapping to bits 13/12 and 15/14) are not defined in the text; the diagram is hard to read. Please add a clear bit-layout table or explanation.","section":"Algorithm 1, Fig. 3"},{"comment":"Typo: 'highly attribute to' should be 'highly attributed to'.","section":"§5"},{"comment":"Saying that negligible correlation implies the weight entries 'can be treated as independent' is not rigorous. Since this section is an explanatory post-hoc analysis, rephrase as a modeling assumption rather than a conclusion from the correlation values.","section":"Appendix B.2"},{"comment":"Spelling of 'DeepSeek' is inconsistent ('Deepseek-MoE' appears in several places). Please unify.","section":"Table 1 caption"}],"recommendation":"major_revision","confidential_remarks":"The two major issues are fixable. I would ask the authors to (i) numerically validate bit-packing for Qwen1.5-MoE and Qwen3-MoE, including merged-tensor exponent ranges, and (ii) correct or support the 45x speedup number. If the packing validation turns out negative, the Qwen accuracy results would need to be re-run with the actual packed format or with auxiliary metadata, which changes the compression-ratio claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The accuracy story holds up. PuzzleMoE's central idea—merge experts entry-wise, keep a similarity mask for shared weights and a saliency mask for expert-specific ones, and reconstruct at inference from one merged tensor plus masks/signs—is genuinely new in the MoE compression literature. Prior work drops whole experts or merges at coarse/low-rank level, and dense-model sparse merging does not target MoE expert storage. The dual-mask design makes sense, and the experiments are the best part: 16 seeds, tiny std, consistent gains over HC-SMoE/Sub-MoE/Wanda across Mixtral, DeepSeek, and both Qwen MoEs, plus robustness to calibration dataset and random pairing. That's real evidence, and the method is training-free and task-agnostic.\n\nThe weak parts are all in the efficiency claims. The abstract's \"45× faster compression\" is not supported by the reported data: 2 minutes vs 55 minutes is 27.5×, not 45×. The 1.28× inference speedup is presented without matched-hardware detail, so it mixes whatever kernel gain the bit-packed decoding gives with the benefit of fitting on one GPU instead of two. Those numbers need re-baselining or explicit clarification.\n\nThe stress-test concern about bit-packing is also legitimate. The no-loss exponent-shift claim is verified only for Mixtral and DeepSeek, via WikiText-2 perplexity; for Qwen1.5 and Qwen3 the paper shows only exponent-distribution plots. Since merged tensors can have different tail behavior than original expert weights, the claim that 50% compression carries no metadata overhead needs per-model packing validation. This is a fixable gap—add before/after perplexity or task accuracy for the Qwen models—not a demonstrated failure.\n\nMinor: τ_sim=0.4 is chosen from a validation sweep and then reported as fixed. That's mild selection, not circular, and the ablation shows 0.3–0.5 all work. Appendix B.2's probability analysis is post-hoc but harmless.\n\nOverall: the central method and accuracy results are credible; the paper overstates efficiency in the abstract and under-validates packing on two of four models. Those are exactly the things a referee should push on. Send it to review.","headline":"Entry-wise dual-mask expert merging is a real, well-tested contribution; the efficiency claims and bit-packing validation need correction before the paper is release-ready.","tokens_in":22374,"tokens_out":2153,"would_cite":true,"duration_ms":20090,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PuzzleMoE compresses Mixture-of-Experts models by 50% by merging expert pairs element-wise and hiding masks and signs inside bfloat16 exponent bits.","keywords":["Mixture-of-Experts","MoE compression","expert merging","sparse masks","bit-packing","bfloat16","post-training compression","GPU inference"],"falsifier":"For any target MoE model, compute the exponent histogram of each merged expert tensor after 50% merging; if the minimum exponent is below 112 or the maximum after subtracting 112 exceeds 31 (i.e., original exponent above 143), then the bit-packing is lossy. Then run WikiText-2 perplexity for the packed vs unpacked version; any difference above 1e-3 contradicts the paper's 'no perplexity degradation' claim. The paper only reports this check for Mixtral and DeepSeek-MoE, not for the Qwen models.","tokens_in":21393,"feed_emoji":"🧩","tokens_out":8702,"duration_ms":81171,"temperature":0.7,"pith_summary":"PuzzleMoE tries to show that the biggest obstacle to running Mixture-of-Experts language models — storing all expert weights even though only a few are used per token — can be removed without retraining and with almost no accuracy loss. It merges experts in pairs at the individual weight-entry level, using one mask to find entries where the two experts agree in magnitude and another to keep the entries most important to each expert, then reconstructs each expert as its sign pattern times its mask times a shared merged weight. The masks and signs are not stored separately: they are packed into the exponent bits of bfloat16 weights, which the paper observes are mostly idle because expert weight exponents concentrate in a narrow range. A custom GPU kernel unpacks them on the fly during matrix-vector multiplication. If the method works as reported, a 47-billion-parameter model like Mixtral-8x7B can be halved in expert memory, compressed in minutes, and run on a single GPU with a 1.28x speedup, which would change the practical deployment cost of MoE models.","feed_headline":"PuzzleMoE cuts Mixture-of-Experts models to half size","feed_subtitle":"Merging expert pairs keeps accuracy at half size by hiding masks inside floating-point exponent bits","key_machinery":"The key object is the element-wise reconstruction identity (Eq. 8), Ŵ_i = (−1)^{S_i} ⊙ M_i ⊙ W_merged. M_i is the union of a similarity mask M_sim (entry magnitudes agreeing within threshold τ_sim) and a saliency mask M_sal_i (weights whose magnitude times input-activation norm is larger for expert i); S_i is the sign bit pattern of expert i; W_merged is the averaged/saliency-selected magnitude tensor. The second half of the machinery is a bfloat16 bit-packing scheme: because expert weight exponents concentrate around 112–128, all exponents are shifted down by 112 (with values below 112 clamped), fitting into 5 bits and leaving 3 exponent bits plus the sign bit free; two mask bits and one si","core_discovery":"On its own terms, the paper's central claim is that expert weights in MoE models split cleanly into shared entries (similar magnitudes across a pair of experts) and specialized entries (large magnitude and high activation saliency for one expert), so two binary masks plus a sign pattern are enough to reconstruct either expert from a single merged tensor. The reconstruction formula is Ŵ_i = (−1)^{S_i} ⊙ M_i ⊙ W_merged, computed element-wise. The authors claim this reconstruction is accurate enough that at 50% expert sparsity, average zero-shot accuracy drops only 0.2–1.4 points across four MoE families, and the bit-packing step that hides masks and signs in the bfloat16 exponent field introdu","pith_inferences":["The exponent-bit packing is a general data-layout idea, not MoE-specific; the same freed-bits trick could store per-tensor metadata (masks, signs, even small calibration constants) inside any bfloat16 tensor whose dynamic range is narrow, which suggests applications to dense pruning or multi-task model merging.","Since random pairwise grouping is nearly as good as the searched grouping, the method's headroom likely sits in the grouping strategy rather than the merge rule; a cheap affinity-based grouping (e.g., using router statistics already collected during calibration) could recover the small observed gain without a search.","The fixed shift of 112 is a brittle constant across model generations; a natural testable extension is per-layer or per-tensor learned shifts selected at compression time, which would let the packing adapt to models with wider exponent ranges.","The method's accuracy at 50% sparsity on Mixtral (72.6 average vs 74.1 full) suggests that expert-level redundancy in MoE is much larger than previously assumed, implying that future MoE training could intentionally produce mergeable experts to make compression a first-class goal."],"forward_implications":["At 50% expert sparsity, Mixtral-8x7B fits on one A100-80GB and Qwen3-MoE-30B fits on one A100-40GB, instead of two GPUs.","Compression is training-free and takes minutes (2 minutes for Mixtral, 10 for DeepSeek-MoE), so the method supports rapid re-compression for different deployments.","Calibration data choice barely matters: C4 and MATH give nearly identical downstream accuracy, so task-specific calibration is not needed.","The merge step composes with quantization: 3-bit quantization of merged weights yields about 4.8x total compression with around 1.7% max accuracy drop, so it is a complementary rather than competing method.","On reasoning tasks, 25% sparsity keeps Qwen3-MoE within about 12 points of baseline on AIME24 and within 1 point on Math-500, compared with baseline collapse to zero for coarse merging."],"fun_headline_variants":["PuzzleMoE halves MoE models without retraining","Hiding masks in exponent bits shrinks MoE size 50%","Sparse merging plus bit-packing compresses MoEs","MoE models cut to half size, accuracy intact","PuzzleMoE packs expert masks into floats, saves memory"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole no-overhead trick depends on merged expert weights using only a narrow range of exponent values — roughly values 112 to 143 in bfloat16 — so that after clamping and shifting, the freed bits really are free; if a model's weights fall outside that range, the stored masks and signs get corrupted or small weights get destroyed.","fun_headline_variants_meta":{"raw":{"variants":["PuzzleMoE halves MoE models without retraining","Hiding masks in exponent bits shrinks MoE size 50%","Sparse merging plus bit-packing compresses MoEs","MoE models cut to half size, accuracy intact","PuzzleMoE packs expert masks into floats, saves memory"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000345,"raw_usage":{"total_tokens":1746,"prompt_tokens":777,"completion_tokens":969,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":885}},"tokens_in":521,"tokens_out":969,"duration_ms":9245,"temperature":1.0,"reasoning_tokens":885,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T23:34:12.487409+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"For any target MoE model, compute the exponent histogram of each merged expert tensor after 50% merging; if the minimum exponent is below 112 or the maximum after subtracting 112 exceeds 31 (i.e., original exponent above 143), then the bit-packing is lossy. Then run WikiText-2 perplexity for the packed vs unpacked version; any difference above 1e-3 contradicts the paper's 'no perplexity degradation' claim. The paper only reports this check for Mixtral and DeepSeek-MoE, not for the Qwen models.","supporting_citations":[],"review_version":1}