{"id":"3abd0649-27ee-42ea-b2c5-a29678a6b7e4","arxiv_id":"2607.23860","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"low","formal_verification":"none","parameter_count":6,"one_line_summary":"Softmax-temperature regularisation of sigmoid-bounded normalisation scales controls member diversity in implicit ensembles, matching deep ensembles cheaply across CNNs and transformers.","lead":"A shared-backbone ensemble controls how much members disagree by tuning one temperature on normalisation scales. It matches deep ensembles on accuracy and calibration at far lower parameter cost, and can turn a pretrained model into a calibrated ensemble with a short fine-tune.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The flagship result — σN-Ens \"peaks on WideResNet/CIFAR-100,\" where its margin over deep ensembles \"widens with capacity\" — leans on a transcribed deep-ensemble baseline (NLL 0.903, ECE 22.9%, AUROC 67.9) that contradicts established deep-ensemble behaviour and was never rerun under the authors' own","rationale":"The reader's verdict and most of its conditions stand; I would keep CONDITIONAL but attach one more concrete condition: an in-protocol rerun of the deep-ensemble baseline on the flagship cell before the \"outperforms deep ensembles / widening with capacity\" framing is accepted at face value. Agreement is partial: the reader's weakest_assumption pinned the load-bearing risk on the modulation-uncertainty premise (§3.4), and mentioned transcribed baselines only as a general soundness caveat (\"some transcribed baselines and protocol-conditioned comparisons\"). My read inverts the priority. The modulation-uncertainty premise is honestly hedged in the paper — the authors state it as an empirical premise, derive only structural identities (Eq. 6/10 are chain-rule tautologies, correctly presented), and their OOD weakness on pretrained ViT is reported rather than hidden, with a plausible mechanism (least perturbation of the inherited solution). That premise could shift the magnitude of the calibration-under-shift story but not the existence of the mechanism. By contrast, the anomalous transcribed deep-ensemble row is checkable arithmetic sitting directly under the abstract's strongest comparative sentence, and the pattern across the other cells (DE wins on C10 RN50 and ViT/ImageNet, parity on C100 RN50) means the WRN/C100 cell is doing outsized work in the \"peaks with capacity\" narrative. Credit where due: the paper runs its own rows in a shared framework, states limitations plainly, reports per-seed spreads on the transformer tables, and the τ-control / member-scaling / λ-robustness evidence is internally generated and reproducible in principle — none of that is implicated by this concern. If the proposed rerun confirms the baseline anomaly, the fix is a wording downgrade plus a corrected table, not a rejection; if it dispels the anomaly, the paper's headline is actually strengthened. Hence CONDITIONAL, unchanged in letter from the reader but with the verification target redirected.","tokens_in":26594,"tokens_out":4090,"duration_ms":104913,"concrete_test":"Rerun deep ensembles (M=4) on WideResNet-28-10 / CIFAR-100 inside TorchUncertainty under the exact σN-Ens protocol (dropout 0, same schedule/augmentation), 3 seeds, and additionally rerun σN-Ens with protocol-default dropout to isolate the §A.4 deviation. If rerun deep ensembles recover to NLL ≲ 0.70 / ECE ≲ 5 / AUROC ≳ 80 (per Ovadia et al.), the \"advantage widens with capacity\" claim reduces to efficiency parity and the abstract should be reworded; if the anomaly reproduces in-protocol, the transcribed-baseline concern is dispelled and the headline stands. Also rerun the DE rows on CIFAR-10/C100 RN50 as a sanity cross-check.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The strongest claim's marquee evidence is the WideResNet-28-10 / CIFAR-100 cell of Table 1: it is the one setting where σN-Ens clearly beats deep ensembles (NLL 0.584 vs 0.903, ECE 5.1 vs 22.9, AUROC 82.6 vs 67.9), and §4.4 builds the capacity-trend narrative on it (\"the advantage over deep ensembles widening at higher capacity... peaking on WideResNet / CIFAR-100\"). But that deep-ensemble row is transcribed from Laurent et al. (2023), and it is anomalous: deep ensembles with ECE 22.9% — worse-calibrated than the Single model (15.6) and far worse than any ensemble method in the block, with AUROC 67.9 barely above the single model — contradicts the well-replicated finding (Ovadia et al. 2019; Lakshminarayanan et al. 2017) that deep ensembles are the best-calibrated, strongest-OOD baseline. A deep ensemble scoring NLL 0.903 where Packed, Masksembles, FiLM and σN-Ens all land at 0.58–0.94 with far better ECE strongly suggests a protocol artefact in the source table (e.g. fewer members, different augmentation, or no temperature-appropriate training), not genuine deep-ensemble behaviour. Meanwhile the authors reran only their own rows, and deviated from the shared protocol by setting dropout to zero (§A.4) — so the comparison is cross-paper, asymmetric, and the deviation's effect on σN-Ens is unmeasured. Elsewhere the claim is honest but weaker: on CIFAR-10 RN50 deep ensembles beat σN-Ens on accuracy and NLL (96.0/0.136 vs 95.4/0.156); on C100 RN50 it's parity (80.9/0.713 vs 80.8/0.701); on ViT/ImageNet deep ensembles win accuracy and NLL (83.67/0.593 vs 82.32/0.655). So \"matches or outperforms deep ensembles, margin widening with capacity\" currently rests on the single cell whose baseline looks broken. If that baseline is repaired, the headline likely degrades to \"parity with deep ensembles at a fraction of the parameter cost\" — still valuable, and the τ-control, member-scaling and modulation-uncertainty contributions survive untouched, but the abstract's competitive ph","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The manuscript introduces σN-Ens, an implicit ensemble in which M members share a backbone and differ only in per-normalisation-layer sigmoid-bounded scale vectors (σNorm, building on the authors' σBN/TSσBN primitives), plus a softmax-temperature regulariser L_div (Eq. 2) that pushes each feature's across-member importance distribution toward a τ-parametrised equilibrium. A single temperature τ is claimed to select the operating point on the accuracy–calibration frontier, with λ only setting the strength of the pull. The paper further defines \"modulation uncertainty\" (§3.4, Eq. 6) as the conditional-mutual-information component a shared-backbone ensemble expresses, and predicts — then confirms — graceful calibration degradation under corruption and weaker OOD detection. Experiments cover ResNets/WideResNet on CIFAR-10/100 (Packed Ensembles protocol), ViT-B/16 on CIFAR-100/ImageNet, and BERT on SST-2, with ablations over M (2–16), λ (three orders of magnitude), τ frontiers, parameter-vs-output diversity correlations, and CIFAR-100-C/ImageNet-C corruption curves.","tokens_in":26994,"tokens_out":5968,"duration_ms":111631,"significance":"If the results hold, this is a useful contribution: to my knowledge it is the first implicit-ensemble method with an explicit, single-knob control over member diversity during training, and the τ/λ separation (equilibrium vs. strength) is both conceptually clean and empirically supported (Table 4). The derivation in Appendix A.5 that the penalty's attractors are k-owner corners — interpolating between hard partitioning (k=1) and full sharing — is a genuinely parameter-free piece of analysis that unifies partitioning and modulation methods on one axis. The output-preserving conversion of pretrained backbones (Appendix A.3) is practically valuable and the pretrained ViT calibration results (Table 3, Fig. 3) are strong and fairly reported, including the acknowledged OOD weakness. The member-scaling study (Fig. 4, Table 7) is a well-designed falsifiable contrast against Packed Ensembles, and the parameter-space/output-space diversity correspondence (Table 6, |ρ| ≥ 0.83) supports the monitoring claim. The evaluation is multi-architecture, multi-regime, with seed statistics on the transformer runs — above the field's typical standard.","major_comments":[{"comment":"Table 1, WRN-28-10/CIFAR-100 block: the deep-ensemble row (Acc 82.5, NLL 0.903, ECE 22.9, AUROC 67.9) is transcribed from Laurent et al. (2023) and is anomalous — the deep ensemble is worse calibrated than the Single model (ECE 22.9 vs 15.6) and barely better on AUROC (67.9 vs 64.2), contradicting well-replicated deep-ensemble behaviour (Lakshminarayanan et al. 2017; Ovadia et al. 2019) and every other ensemble row in the block. This cell is load-bearing: it is the only convolutional setting where σN-Ens clearly dominates deep ensembles, and §4.4 builds the capacity-trend narrative ('advantage... widening at higher capacity, peaking on WideResNet / CIFAR-100') on it. Elsewhere the comparison is parity (C100/RN50: 80.8/0.701 vs 80.9/0.713) or a loss (C10/RN50: 95.4/0.156 vs 96.0/0.136). Compounding this, §A.4 states the authors deviated from the shared protocol by setting dropout to zero,","section":"Table 1 / §4.4"},{"comment":"while the transcribed baselines were trained under the original protocol — so the comparison in this block is cross-paper and asymmetric, and the effect of the dropout deviation on the σN-Ens rows is unmeasured. Relatedly, Table 8's WRN deep-ensemble shows ECE 5.6 at corruption severity 1 versus 22.9 on clean data, which is implausible for the same model; this suggests the corruption-eval deep ensemble was rerun under a different protocol. The authors should (i) rerun deep ensembles (and ideally Packed/FiLM) on WRN/C100 under their exact protocol, or rerun their own rows with protocol dropout; (ii) state explicitly which rows of Tables 8–9 were rerun and reconcile the clean/corruption ECE discrepancy; and (iii) temper the capacity-trend claim until the cell is validated.","section":"Table 8 / §A.10"},{"comment":"The claim that τ is 'the only hyperparameter a practitioner must set' (§5.2) requires a clean selection protocol, which is not described. The benchmark tables use τ=0.1 for all CNNs, yet the frontier sweeps (Fig. 2, Fig. 5) show τ∈{0.1..1.0} with the interior optimum varying by backbone (e.g., WRN optimum near τ=0.7–0.9 by eye). How was τ=0.1 chosen for Table 1 — on validation data, or by inspecting test metrics? If the latter, the main-table numbers benefit from test-set tuning. Additionally, the frontier in Fig. 2(Left) spans only ~0.8 points of accuracy and ~1.5 points of ECE over 3 seeds with no error bars shown; the claimed 'interior optimum' may be within seed noise. Please report seed spread on the frontier points and describe the τ-selection procedure.","section":"§3.3 Fig. 2 / §5.2"},{"comment":"§3.4's modulation-uncertainty account is presented as explaining both the corruption robustness and the OOD weakness, but the two 'predictions' rest on an empirical premise (backbone component small on corrupted inputs, large off-distribution) that is asserted from Fort et al./Ovadia et al. rather than tested, and members' reinitialised heads confound the attribution: some of the measured disagreement comes from the M independent heads, not from the σNorm scales the regulariser controls. A direct ablation — shared head vs. reinitialised heads, at fixed τ — would isolate how much of U_mod and of the corruption robustness is actually carried by the modulation mechanism the paper proposes. Without it, 'explain why its calibration holds' (abstract) overstates what is shown. The reachable-set framing (Eq. 9–11) itself is sound and honestly scoped; the gap is evidential.","section":"§3.4 / Appendix A.6"},{"comment":"The abstract's 'matches or outperforms deep ensembles' is not supported as a blanket claim. On ImageNet/ViT (Table 3) deep ensembles are clearly better on accuracy (83.67 vs 82.32) and NLL (0.593 vs 0.655); on C10/RN50 deep ensembles win accuracy and NLL; on SST-2 σN-Ens is below the Single model on accuracy. The accurate statement is: best-in-class calibration and NLL on the pretrained ViT and on hard from-scratch settings, parity elsewhere, at ~4–6× fewer parameters — but with identical multiply-adds (Table 1: σN-Ens 23.83 MAdds = deep ensembles 23.83), so the efficiency gain is memory/parameters, not inference compute. The abstract and §6 should be qualified accordingly, and the equal-MAdds point stated explicitly.","section":"Abstract / §6"}],"minor_comments":[{"comment":"Table 4 (λ robustness) is single-seed; given the claim that λ need not be tuned, 3 seeds would close the point cheaply.","section":"§5.2 / Table 4"},{"comment":"Fig. 3's legend lists FiLM-Ensemble for the ViT-B/16 panel, but §4.3 states FiLM cannot wrap a pretrained backbone and does not appear in that benchmark. Presumably a shared legend; please clean up.","section":"Fig. 3"},{"comment":"A temperature-scaled Single baseline (Guo et al. 2017) in the calibration columns would sharpen the contribution: post-hoc TS is near-free and would show whether σN-Ens's ECE advantage survives the cheapest competitor, particularly on the pretrained ViT rows.","section":"§4.2–4.3"},{"comment":"Eq. (2): state explicitly that L_div is added to the task loss and that minimising Σ_m log p_m sharpens the across-member distribution; the gradient direction (Eq. 8) only appears in A.5. Also 'Equation equation 6' typo in §3.4, and 'regardless initialisation' in §3.1.","section":"§3.2 / typos"},{"comment":"Notation: γ is overloaded for the trainable scale logits (Eq. 1) and the pretrained affine scale γ^pre (Eq. 7); a distinct symbol for one would ease reading of A.3.","section":"Appendix A.3"},{"comment":"Ensemble size is inconsistent across tables (M=4 vision, M=3 SST-2) and LoRA-Ensemble is run at M=4 versus M=16 in its own paper; the disclosure in A.2/A.4 is good, but a sentence in §4.3 itself would prevent misreading of Table 3.","section":"§4.3"},{"comment":"Fig. 1(Center) is a synthetic simulation from N(0,1) initialisations; this is stated in the text but should also appear in the caption, since it is easy to read as measured data.","section":"Fig. 1"},{"comment":"LoRA-Ensemble's ImageNet ECE std of ±9.21 (Table 3) indicates a diverged seed; please note how seeds were handled rather than reporting the raw spread.","section":"Table 3"},{"comment":"Packed's WRN rows show Params 36.62 (equal to the full model) versus 14.55–19.35 elsewhere; the widening factor α used per cell should be tabulated, since Packed's parameter efficiency is part of the comparison.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is well written and the core mechanism (τ-controlled diversity equilibrium, k-owner-corner derivation, member-scaling contrast) is novel and, as far as I can check, correct. My one blocking concern is evidentiary rather than conceptual: the flagship WRN-28-10/CIFAR-100 comparison rests on a transcribed deep-ensemble row from Laurent et al. that behaves unlike any published deep ensemble (ECE worse than the single model), and the authors' own rows deviate from that paper's protocol (dropout=0). This is fixable with one rerun and should be required before acceptance. The paper also leans heavily on the authors' prior σBN/TSσBN line for both the primitive and the MTL framing; these are used as tools rather than claimed as contributions, so I do not consider this problematic, but the editor may wish to note that the 100× LR multiplier and other defaults are inherited untuned from that prior work."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The real advance here is a single temperature on sigmoid-bounded norm scales that holds member diversity at a chosen equilibrium during training, plus a clean conversion path that turns a pretrained checkpoint into a calibrated implicit ensemble. That is new relative to FiLM (init-seeded), Packed/MIMO (hard partition), and LoRA-Ens (adapter init). The τ frontier, σNorm-cosine ↔ Jensen-gap tracking, member-scaling vs Packed, and λ-insensitivity are the parts I trust most.\n\nExperiments are broad: ResNets and ViT/BERT, from-scratch and fine-tune, CIFAR/ImageNet/SST-2, corruption curves, and honest framing of “modulation uncertainty” as the reachable epistemic slice. Calibration under shift is the strongest empirical story; OOD weakness on frozen vision backbones is predicted rather than papered over. Citation pattern is fair to the family; self-cites to σBN/TSσBN are tools, not the claim.\n\nSoft spot, in proportion: the abstract’s “matches or outperforms deep ensembles, margin widening with capacity” leans hard on the WRN-28-10/CIFAR-100 cell where the transcribed DE row (NLL 0.903, ECE 22.9, AUROC 67.9) is worse than the single model and contradicts standard DE behaviour. Authors reran only their rows and zeroed dropout. Elsewhere the picture is parity or DE ahead on accuracy/NLL. If that baseline is repaired, the headline becomes “parity at a fraction of the parameters,” which is still useful; τ-control, scaling, and the modulation story survive. LoRA under a non-native shared recipe is a secondary fairness nit. No code drop is a practical minus, not a conceptual one.\n\nWho it’s for: people building efficient ensembles or adapting pretrained models who want one interpretable diversity dial. Worth a serious referee. I’d engage, cite the mechanism and the uncertainty framing, and discount the capacity-trend rhetoric until DE is rerun.","headline":"Solid methods paper with a real training-time diversity knob; the abstract oversells deep-ensemble wins because the marquee WRN/CIFAR-100 DE baseline looks broken.","tokens_in":27765,"tokens_out":501,"would_cite":true,"duration_ms":11282,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A single temperature knob turns shared-backbone ensembles into deep-ensemble competitors at a fraction of the parameter cost.","keywords":["implicit ensembles","uncertainty quantification","normalisation layers","member diversity","softmax temperature","modulation uncertainty","calibration under shift","pretrained adaptation"],"falsifier":"Train σN-Ens and matched deep ensembles on the same backbone and data, then measure calibration (ECE) across corruption severities and near/far OOD detection: if σN-Ens’s ECE collapses under moderate corruption while OOD detection matches or exceeds the deep ensemble, the modulation-uncertainty split and its predictions fail.","tokens_in":27209,"feed_emoji":"🎛️","tokens_out":929,"duration_ms":21955,"temperature":0.7,"pith_summary":"Deep ensembles give strong uncertainty estimates but cost scales with every extra member. Implicit ensembles share one backbone to cut that cost, yet until now none could set how much members disagree during training—diversity was fixed at startup or baked into the architecture. This paper introduces σN-Ens: members share every convolution or attention weight and differ only through sigmoid-bounded scales on normalisation layers, treated like tasks in a multi-task setup. A softmax-temperature regulariser pulls those scales toward a chosen sharing equilibrium, so one hyperparameter traces the accuracy–calibration tradeoff. Because only normalisation is replicated, the same recipe wraps ResNets and transformers, and can turn a pretrained checkpoint into a calibrated ensemble with a short fine-tune. Across CIFAR, ImageNet, and SST-2 the method matches or beats full deep ensembles on accuracy and likelihood while using far fewer parameters, keeps scaling as members are added where hard-partition methods collapse, and holds calibration under input corruption—while explaining why out-of-distribution detection stays weaker.","feed_headline":"One temperature builds cheap ensembles that rival deep ones","feed_subtitle":"Shared-backbone members stay calibrated under corruption and scale without collapsing","key_machinery":"Softmax-temperature regularisation on σNorm importance vectors: for each feature, members’ sigmoid scales are softmaxed across the ensemble at temperature τ; minimising the resulting penalty drives features toward a τ-chosen soft M-choose-k ownership pattern, holding diversity at a chosen equilibrium throughout training rather than fixing it at initialisation or architecture design.","core_discovery":"Member diversity in a normalisation-based implicit ensemble can be controlled during training by a softmax-temperature regulariser on sigmoid-bounded per-feature scales. That single temperature selects the equilibrium level of feature sharing, traces an accuracy–calibration frontier, and yields ensembles that match or outperform independently trained deep ensembles at a fraction of the parameter count across convolutional and transformer backbones.","pith_inferences":["If modulation uncertainty is the dominant usable epistemic signal in large pretrained models, many deployment settings may not need full independent backbones for calibration—only for hard OOD gates.","Pairing controllable normalisation diversity with a small amount of member-specific feature capacity (for example light adapters) is a direct route to recover the forgone OOD component without returning to linear parameter cost.","A label-free parameter-space diversity metric that tracks output disagreement could be monitored online during training as a cheap early-stopping or τ-selection signal in other multi-head systems."],"forward_implications":["Practitioners can convert a single pretrained model into a calibrated multi-member ensemble by swapping normalisation layers and fine-tuning briefly, without retraining full members from scratch.","Ensemble size can be increased without hard channel partitioning, so accuracy and likelihood improve or saturate instead of collapsing when members multiply.","One temperature τ becomes the practical control for navigating the accuracy–calibration frontier to match an application’s risk tolerance.","Calibration under input corruption can be maintained by design in shared-backbone ensembles even when full backbone diversity is forgone.","The same normalisation-only wrap applies unchanged to both convolutional and transformer families."],"fun_headline_variants":["Softmax temperature sets member diversity in shared-backbone ensembles","One temperature traces the accuracy-calibration frontier for implicit ensembles","Sigmoid-bounded scales let a single temperature control ensemble sharing","Normalisation scalers plus temperature match deep ensembles at lower cost","Temperature-tuned σN-Ens stays calibrated under corruption across backbones"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The paper assumes that disagreement from reweighting one shared backbone is enough for calibrated uncertainty on corrupted but still-representable inputs, while the disagreement that would come from different feature extractors is what mainly drives out-of-distribution detection.","fun_headline_variants_meta":{"raw":{"variants":["Softmax temperature sets member diversity in shared-backbone ensembles","One temperature traces the accuracy-calibration frontier for implicit ensembles","Sigmoid-bounded scales let a single temperature control ensemble sharing","Normalisation scalers plus temperature match deep ensembles at lower cost","Temperature-tuned σN-Ens stays calibrated under corruption across backbones"]},"model":"grok-4.5","effort":"low","cost_usd":0.003456,"raw_usage":{"total_tokens":1131,"prompt_tokens":778,"num_sources_used":0,"completion_tokens":69,"cost_in_usd_ticks":34564000,"prompt_tokens_details":{"text_tokens":778,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":284,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":778,"tokens_out":69,"duration_ms":5231,"temperature":1.0,"reasoning_tokens":284,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T14:03:10.104962+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train σN-Ens and matched deep ensembles on the same backbone and data, then measure calibration (ECE) across corruption severities and near/far OOD detection: if σN-Ens’s ECE collapses under moderate corruption while OOD detection matches or exceeds the deep ensemble, the modulation-uncertainty split and its predictions fail.","supporting_citations":[],"review_version":2}