{"id":"6439f95f-534f-4c69-9895-9bfe5042491a","arxiv_id":"2605.29350","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"ConMoE consolidates MoE experts into a smaller prototype pool via deterministic remapping based on contribution and replaceability, matching or beating pruning/merging baselines at 25-50% reduction on three models.","lead":"ConMoE is a train-free method that compresses MoE models by keeping fewer experts as prototypes and remapping the rest to them using signals from calibration data. This could lower memory needs for serving large language models without retraining.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Calibration signals from small dataset may fail to identify substitutable prototypes for deterministic remapping","rationale":"The reader's weakest assumption directly identifies the same load-bearing step. The abstract-level description supplies no additional evidence (such as cross-domain calibration results or expert similarity metrics) that would strengthen it, so the UNVERDICTED status remains appropriate.","tokens_in":1794,"tokens_out":279,"duration_ms":15321,"concrete_test":"Recompute the reported average scores on deepseek-moe-16b-base using a fresh calibration set drawn from a disjoint domain (e.g., code instead of the original calibration distribution) while keeping all other hyperparameters fixed; if the 25% and 50% reduction scores drop by more than 2 points relative to the published numbers, the signal sufficiency assumption does not hold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that contribution and replaceability scores computed on a limited calibration set identify a reusable expert subset such that deterministic remapping (no weight updates, no fine-tuning) preserves downstream behavior at 25-50% reduction. This is least secure because the signals are proxies whose correlation with actual functional equivalence is unproven; expert activations can be highly input-dependent, and a small calibration corpus may miss rare but high-impact contexts where remapped experts diverge.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes ConMoE, a train-free post-training compression framework for Mixture-of-Experts (MoE) language models. It reformulates compression as expert-pool consolidation: a smaller set of pretrained experts is retained as prototypes based on calibration-derived contribution and replaceability signals, after which each original expert reference is deterministically remapped to one of the prototypes. This preserves the original router interface, permits local layer-scope prototype sharing, and requires no weight updates or fine-tuning. Experiments on DeepSeek-MoE-16B-Base, Qwen3-30B-A3B, and OLMoE-1B-7B-0125 report that ConMoE matches or exceeds strong pruning and merging baselines at 25% and 50% routed-expert reduction, attaining the best average score on DeepSeek-MoE-16B-Base at both ratios; ablations indicate deterministic reassignment is the most stable component while cross-layer sharing and weight fusion are model-dependent.","tokens_in":1877,"tokens_out":667,"duration_ms":22150,"significance":"If the empirical claims hold under detailed scrutiny, the work supplies a lightweight, training-free compression technique that decouples the reduced expert pool from the reuse mapping and keeps the router unchanged. The explicit separation of prototype selection from remapping structure, together with the reported stability of the deterministic component, constitutes a practical contribution for memory-constrained MoE deployment. The purely empirical, calibration-driven nature and the component-wise ablations are strengths that allow direct comparison with existing pruning/merging methods.","major_comments":[{"comment":"Abstract and §4 (Experiments): the central claim that ConMoE 'matches or outperforms' baselines and achieves 'the best average score' on DeepSeek-MoE-16B-Base rests on high-level summary only; no numerical tables, exact baseline scores, dataset sizes, number of runs, or error bars are referenced, so the magnitude and statistical reliability of the reported gains cannot be assessed.","section":"Abstract and §4"},{"comment":"§3 (Method): the replaceability and contribution signals are computed on an unspecified small calibration corpus; because expert activations are known to be input-dependent, the manuscript must demonstrate that these proxies remain predictive of functional equivalence under deterministic remapping on held-out data, or the train-free preservation claim is unsupported.","section":"§3"},{"comment":"§4.3 (Ablations): the statement that 'deterministic reassignment is the most stable component' is presented without quantitative comparison of performance variance across random seeds or across different calibration-set sizes; this directly affects the reliability of the core design choice.","section":"§4.3"}],"minor_comments":[{"comment":"Notation for the contribution and replaceability scores should be defined once with explicit formulas rather than described only in prose.","section":"§3"},{"comment":"The manuscript should state the exact calibration corpus (size, domain, number of tokens) used for all reported runs.","section":"§4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed feedback. We address each major comment below and will incorporate revisions to improve clarity, specificity, and empirical support in the manuscript.","responses":[{"response":"We agree that the abstract and opening of §4 would benefit from explicit references to numerical results. The full results appear in Tables 1–3 of §4 (including exact scores for ConMoE and all baselines on the three models, with dataset details in §4.1). In the revision we will (i) insert key numerical values and table references into the abstract and §4 introduction, (ii) state that all scores are averaged over three independent evaluation runs, and (iii) ensure standard deviations are reported in the tables where they were previously omitted from the summary text.","revision_made":"yes","referee_comment":"[Abstract and §4] Abstract and §4 (Experiments): the central claim that ConMoE 'matches or outperforms' baselines and achieves 'the best average score' on DeepSeek-MoE-16B-Base rests on high-level summary only; no numerical tables, exact baseline scores, dataset sizes, number of runs, or error bars are referenced, so the magnitude and statistical reliability of the reported gains cannot be assessed."},{"response":"We will revise §3.2 to explicitly state the calibration corpus details (size, source, and sampling procedure). To address input dependence, we will add a controlled experiment that recomputes prototype selection on a held-out portion of the same data distribution and reports the resulting end-to-end performance after deterministic remapping. This will directly test whether the signals remain predictive outside the calibration set.","revision_made":"yes","referee_comment":"[§3] §3 (Method): the replaceability and contribution signals are computed on an unspecified small calibration corpus; because expert activations are known to be input-dependent, the manuscript must demonstrate that these proxies remain predictive of functional equivalence under deterministic remapping on held-out data, or the train-free preservation claim is unsupported."},{"response":"We accept that the current ablation lacks quantitative variance metrics. In the revised §4.3 we will report mean and standard deviation of downstream scores across five random seeds for deterministic reassignment versus the compared alternatives, and we will repeat the ablation for calibration-set sizes of 256, 512, and 1024 samples. These additions will provide the requested quantitative support for the stability claim.","revision_made":"yes","referee_comment":"[§4.3] §4.3 (Ablations): the statement that 'deterministic reassignment is the most stable component' is presented without quantitative comparison of performance variance across random seeds or across different calibration-set sizes; this directly affects the reliability of the core design choice."}],"tokens_in":1599,"tokens_out":606,"duration_ms":27765,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper treats post-training MoE compression as expert-pool consolidation: pick a smaller set of prototypes using calibration signals for contribution and replaceability, then remap the rest deterministically while keeping the original router and allowing local-layer sharing. No weight updates or fine-tuning required.\n\nWhat stands out is the clean separation between the reduced pool and the reuse structure. That framing lets them avoid the usual merging or pruning trade-offs and stay fully train-free, which is useful for quick deployment on existing models. The experiments cover three pretrained MoEs and report that the method matches or beats strong baselines, with the best average on deepseek-moe-16b-base at both 25% and 50% reduction.\n\nThe soft spot is the evidence level. Everything rests on high-level claims with no tables, exact numbers, error bars, or dataset details visible. The central assumption—that small calibration sets produce reliable replaceability signals—remains untested in the summary we have, and expert activations can be input-dependent enough that this could break in practice. The stress-test concern about missing rare contexts lands here.\n\nThis is aimed at people working on efficient serving of production MoE models. A reader focused on post-training compression would get value from the formulation and the train-free angle. It deserves a serious referee to examine the full experimental setup and check whether the calibration signals actually deliver the claimed preservation of behavior.","headline":"ConMoE frames MoE compression as train-free prototype selection and deterministic remapping, with competitive summary results on DeepSeek-MoE but thin evidence overall.","tokens_in":2371,"tokens_out":365,"would_cite":false,"duration_ms":20037,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"MoE compression succeeds by keeping fewer experts as reusable prototypes and remapping the rest deterministically without weight updates.","keywords":["mixture of experts","model compression","expert consolidation","prototype reassignment","train-free compression","post-training methods","language model deployment"],"falsifier":"A clear performance drop on a held-out evaluation set after applying the remapping to a new pretrained MoE model, compared with the original model and with pruning baselines, would show the method does not preserve behavior.","tokens_in":2699,"feed_emoji":"","tokens_out":759,"duration_ms":24236,"temperature":0.7,"pith_summary":"The paper aims to establish that post-training compression of Mixture-of-Experts language models can be achieved by consolidating the expert pool into a smaller set of retained prototypes and then deterministically remapping every original expert slot to one of those prototypes. This formulation keeps the router unchanged and permits sharing of prototypes within each layer while avoiding any weight updates or fine-tuning after compression. A reader would care if true because it offers a direct way to cut the memory cost of storing and serving all experts in large MoE models. The selection of prototypes relies on contribution and replaceability signals computed from a small calibration set.","feed_headline":"MoE models compress by remapping to fewer expert prototypes","feed_subtitle":"Train-free method selects prototypes via calibration signals and redirects calls, matching baselines on three models at 25-50 percent reduct","key_machinery":"expert-pool consolidation via prototype reassignment, which uses calibration signals to choose retained experts and then applies deterministic remapping of all original expert calls","core_discovery":"We formulate post-training MoE compression as expert-pool consolidation: retaining a smaller set of pretrained experts as reusable prototypes and deterministically remapping each original expert reference to one selected prototype. This view separates the reduced expert pool from the reuse structure that represents the original expert slots, and allows prototype sharing within local layer scopes while preserving the original router interface. We propose ConMoE, a train-free prototype remapping framework that selects retained experts using calibration-based contribution and replaceability signals, then redirects original expert calls to the selected prototypes without weight updates or post-c","pith_inferences":["The results imply that many expert behaviors in these models overlap enough to be captured by a smaller prototype pool selected from the original set.","If the calibration signals prove reliable across more models, the same selection step could be reused for other compression ratios or for combining with quantization.","The separation of pool size from reuse structure opens a path to layer-specific sharing ratios without changing the router.","Applying the same remapping logic at inference time on hardware with limited expert cache could reduce loading costs."],"forward_implications":["ConMoE matches or outperforms strong pruning and merging baselines in several settings on three pretrained MoE models.","It achieves the best average score on deepseek-moe-16b-base at both 25 percent and 50 percent routed-expert reduction.","It remains competitive on Qwen3-30B-A3B and OLMoE-1B-7B-0125 under the same reductions.","Deterministic reassignment is the most stable component of the method.","Broader cross-layer sharing and post-hoc weight fusion show model-dependent effects."],"fun_headline_variants":["ConMoE remaps experts to prototypes for MoE compression","Train-free prototype reassignment consolidates MoE expert pools","Deterministic remapping of MoE experts to selected prototypes","ConMoE compresses MoE by consolidating experts via calibration"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Calibration-based contribution and replaceability signals from a small dataset are enough to pick a reusable prototype set whose deterministic remapping preserves model behavior without weight updates or fine-tuning.","fun_headline_variants_meta":{"raw":{"variants":["ConMoE remaps experts to prototypes for MoE compression","Train-free prototype reassignment consolidates MoE expert pools","Deterministic remapping of MoE experts to selected prototypes","ConMoE compresses MoE by consolidating experts via calibration"]},"model":"grok-4.3","cost_usd":0.006837,"raw_usage":{"total_tokens":3209,"prompt_tokens":733,"num_sources_used":0,"completion_tokens":67,"cost_in_usd_ticks":68374500,"prompt_tokens_details":{"text_tokens":733,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2409,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":733,"tokens_out":67,"duration_ms":16903,"temperature":1.0,"reasoning_tokens":2409,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T07:53:40.669153+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A clear performance drop on a held-out evaluation set after applying the remapping to a new pretrained MoE model, compared with the original model and with pruning baselines, would show the method does not preserve behavior.","supporting_citations":[],"review_version":1}