Pith. sign in

REVIEW 4 major objections 5 minor 43 references

Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A mixture of experts can produce well-calibrated predictive uncertainty in semantic segmentation without architectural modifications.

desk verdict The paper's new contribution is the adaptation of standard ensemble uncertainty metrics to unmodified model-level MoEs, with code and transparent caveats, but the headline claim that MoEs beat ensembles on OOD is not supported by the weak baseline and accuracy-metric confound. read the letter →

arxiv 2509.04816 v1 pith:QHA3EXIY submitted 2025-09-05 cs.CV cs.LG

Svetlana Pavlitska , Beyza Keskin , Alwin Fa{ss}bender , Christian Hubschneider , J. Marius Zöllner This is my paper · ORCID
classification cs.CVcs.LG
keywords mixtureofexpertspredictiveuncertaintysemanticsegmentationcalibrationout-of-distributiondetectiongateentropyexpertvarianceautonomousdriving
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a mixture-of-experts (MoE) model trained for semantic segmentation of traffic scenes already contains what is needed for reliable predictive uncertainty, and that extracting it requires no architectural changes. Taking the MoE's gate-weighted probability output as an aggregated predictive distribution, the authors compute predictive entropy, mutual information, and a new expert-variance score, producing per-pixel uncertainty maps. On a highway/urban split of a driving dataset, these MoE-based uncertainty estimates outperform a two-expert averaged ensemble and a Monte-Carlo dropout baseline on conditional correctness metrics, especially on out-of-distribution and corrupted inputs. The paper also finds that simple gates give better-calibrated routing uncertainty than classwise gates, and that increasing the number of experts on Cityscapes slightly improves calibration. If right, this means safety-critical perception can obtain calibrated uncertainty per pixel from a single unmodified model instead of paying the training and inference cost of an ensemble.

What carries the argument

The load-bearing object is the gate-weighted predictive distribution of a model-level mixture of experts, $\bar{p}(c|x)=\sum_{e=1}^{N} w_e(x)\,p_e(c|x)$, where $w_e(x)$ is the input-dependent gate weight for expert $e$ and $p_e(c|x)$ is that expert's per-class probability. From this one distribution the paper derives predictive entropy, mutual information, and a proposed expert-variance score $\mathrm{EV}(x,c)=\frac{1}{N}\sum_e (p_e(c|x)-\bar{p}(c|x))^2$, without modifying the trained architecture. Gate entropy, computed on the routing distribution over experts, forms a separate uncertainty channel. The argument's load is carried by the fact that the gate's input-dependent weights already encode where each expert is competent, so weighted expert disagreement becomes a usable uncertainty signal.

What would settle it

Train a deep ensemble with independent initializations on the combined highway-urban training data and compare p(uncertain|inaccurate) and AU-PAvPU on the ambiguous A2D2 test split and on corrupted Cityscapes; if the MoE no longer outperforms that ensemble on the conditional metrics, the central claim fails. Independently, run the same MoE uncertainty extraction on a second OOD set, such as night or rain images, to see whether the advantage transfers or is specific to the ambiguous A2D2 split.

Watch

Extended reading notes

Core claim

The central discovery is that the gating mechanism of a model-level MoE, normally used only to improve segmentation output, doubles as an uncertainty estimator. For each input, the authors form the aggregate predictive distribution $\bar{p}(c|x)=\sum_e w_e(x)p_e(c|x)$ from gate weights and expert probabilities, then derive three uncertainty scores: the Shannon entropy of $\bar{p}$, the mutual information between $\bar{p}$ and the individual experts' predictions, and the variance of the experts around the MoE output. On out-of-distribution A2D2 data and under increasing corruption severity, these MoE uncertainty maps align with misclassification better than the averaged ensemble and the Monte-Carlo dropout baseline on conditional correctness metrics such as $p(\mathrm{accurate}|\mathrm{certain})$, $p(\mathrm{uncertain}|\mathrm{inaccurate})$, and Patch Accuracy vs Patch Uncertainty (PAvPU). The paper also reports that simple gate architectures give better-calibrated routing uncertainty (gate entropy) than classwise gates, and that scaling to 10 experts on Cityscapes improves negative log-likelihood without consistent gains on other calibration metrics.

Load-bearing premise

The central comparison is an ensemble that merely averages the same two domain-specialized experts, not a deep ensemble with independently trained members; on top of that, the out-of-distribution results rely on one ambiguous A2D2 split standing in for out-of-distribution data.

Editorial extensions

If this is right

  • An unmodified MoE can emit a calibrated per-pixel uncertainty map alongside its segmentation prediction, so downstream systems can threshold on uncertainty without extra forward passes or retraining.
  • MoE uncertainty can serve as an out-of-distribution or anomaly cue: the same ambiguous A2D2 scenes that decrease segmentation accuracy produce higher uncertainty under the MoE estimates.
  • Routing uncertainty and prediction uncertainty can be reported as separate numbers, giving a two-channel view of whether the model is unsure about which expert should act (gate entropy) or about what the output should be (predictive entropy).
  • Adding more experts to an MoE is a plausible but modest lever for uncertainty calibration, improving NLL in the paper's Cityscapes experiments even when segmentation accuracy plateaus.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the MoE advantage is real, then uncertainty estimation becomes a by-product of expert diversity: any collection of pretrained specialists can be fused by a trained gate and immediately yield uncertainty maps, an option the paper does not compare against a full deep ensemble.
  • The metric split the paper observes, with conditional correctness favoring MoEs while ECE and Brier often favor ensembles, suggests that calibration error and decision-useful uncertainty are different quantities; safety evaluations should report both.
  • A direct test of the paper's thesis would be to train a deep ensemble with independently initialized members on the same combined data and compare AU-PAvPU; the paper's ensemble baseline is only an average of the two domain experts, so the main claim is not yet tested against the strongest ensemble baseline.
  • Gate entropy's near-identical distribution on in-distribution and out-of-distribution data hints that routing confidence is not inherently distribution-aware; an extension would be to feed gate entropy into a supervised OOD detector rather than using it as a standalone signal.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies extraction of predictive uncertainty from model-level mixtures of experts (MoEs) for semantic segmentation. Three estimators are considered: predictive entropy (PE), mutual information (MI), and a proposed expert variance (EV), computed from the gate-weighted combination of pretrained expert outputs. Routing uncertainty is measured by gate entropy. Experiments on A2D2 with two semantically disjoint experts compare MoE variants against a two-model averaging ensemble and MC dropout baselines; additional Cityscapes experiments vary the number of experts. The main claimed finding is that MoEs provide more reliable uncertainty estimates than ensembles in conditional-correctness metrics under out-of-distribution data, without architectural modification. The paper also finds that simple gates give better-calibrated routing uncertainty than classwise gates, and that scaling experts on Cityscapes gives modest NLL improvement but no consistent calibration gain.

Significance. If the central claim were established, the paper would offer a practical recipe: an unmodified, already-trained MoE can yield uncertainty maps that track its own errors better than a uniform two-expert ensemble, at no extra inference cost beyond the MoE's own forward pass. The paper is honest about several limitations: the ensemble baseline is not a deep ensemble (Section 4.2), the stacked PE/MI construction includes the MoE output as an extra component (Section 3.2), and the Cityscapes results show no consistent MoE advantage (Section 5.2). Nevertheless, the main comparative claim is currently supported only in a narrow, accuracy-confounded setting and against a weak baseline; the contribution is therefore incremental rather than definitive, and the paper would need substantial additional experiments to justify the abstract's general wording. The release of code and the evaluation on two datasets are strengths, as is the explicit acknowledgment of the degeneracy of 100% peak values in Table 2.

major comments (4)
  1. [§4.2, Fig. 6, Fig. 7, Table 2] The ensemble baseline is the uniform average of the highway and urban experts, which the paper itself notes is not a deep ensemble in the sense of Lakshminarayanan et al. [25]. Because these experts were trained on semantically disjoint subsets, the ensemble reaches only 0.342 mIoU on A2D2 while the MoE reaches 0.429–0.444 (Fig. 6). The conditional-correctness metrics in Fig. 7 and Table 2 score how well an uncertainty map tracks the model's own errors, so a model with substantially higher segmentation accuracy can achieve better values even if its uncertainty estimates are not intrinsically better. The claim that MoEs yield more reliable uncertainty estimates than ensembles is therefore not supported by this comparison. The paper should add a proper deep ensemble baseline trained on the same data with random initializations, or otherwise control for the accuracy difference (e.g., by reporting normalized or accuracy-matched conditional metrics).
  2. [§3.2, Eq. (2)] The stacked approach to computing PE and MI adds the MoE output pMoE as an extra component in the averaged predictive distribution, giving the MoE N+1 terms while the ensemble averages only N expert outputs. This makes the PE/MI values non-comparable across the two models, and the paper acknowledges that including the MoE output 'may, however, artificially reduce perceived variance.' Since the weighted formulation (which has N components) is only applicable to simple gates, the comparison in Fig. 7 and Table 2 is not symmetric across model types. The paper should either restrict the headline comparison to the weighted formulation where possible or provide a principled justification for why the stacked formulation is a fair equivalent of the ensemble estimate.
  3. [§4.4, Table 2] The 100% peak values in Table 2 are flagged as degenerate ('Note that a 100 % peak occurs when there are very few inaccurate or very few certain pixels at the respective threshold'), yet these peaks are presented as evidence that MoEs achieve the 'highest peak values' for conditional metrics and the AU-PAvPU score. No error bars or seed variance are reported anywhere in the paper, and the threshold-dependence of these metrics is not analyzed. Consequently, the claimed superiority in conditional correctness under data shift is not statistically supported. The paper should report variance over multiple training seeds and preferably provide confidence intervals or significance tests for the AU-PAvPU differences.
  4. [§5.2, Table 3, §6] The Cityscapes experiments directly contradict the abstract's sweeping claim that MoEs yield more reliable uncertainty estimates than ensembles. Table 3 shows that ensembles match or outperform MoEs on ECE, MCE, and Brier score, and the paper states that 'MoEs have slightly worse mIoU and worse calibration metrics overall.' The conclusion nonetheless repeats that MoEs produce 'competitive and often superior uncertainty estimates compared to standard ensembles.' This overgeneralization needs to be corrected: the advantage observed in conditional-correctness metrics is specific to the A2D2 semantically disjoint expert configuration, and the paper should explicitly state the boundary conditions under which the claimed benefit transfers.
minor comments (5)
  1. [§6] In the final paragraph, 'interoperability' appears to be a typo for 'interpretability' in the phrase 'improve uncertainty robustness and interoperability.'
  2. [§4.1 and Abstract] 'Semantical split' is used in both the abstract and Section 4.1; the standard term is 'semantic split.'
  3. [Table 2] The asterisk markers for the 100% values are not explained in the table caption; the explanation appears only in the body text. Add a note to the caption to avoid ambiguity.
  4. [§3.2, Eq. (3)] EV is defined per class c, but Figures 7 and 8 appear to treat EV as a per-pixel scalar. Please clarify how the class-wise variances are aggregated to produce the reported uncertainty maps and metric values.
  5. [§4.4] The selection of the classwise gate without the additional convolutional layer for the data-shift experiments is justified by 'consistently showed strong calibration on OOD data (see Figure 4),' but Figure 4 does not show this variant dominating all calibration metrics (e.g., MCE). Please clarify the selection criterion.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: uncertainty estimates are computed from frozen MoE outputs without fitting to evaluation metrics.

full rationale

No circular step is present in the derivation chain. The claimed contribution is an empirical evaluation: predictive entropy, mutual information, and expert variance (Eqs. 1-3) are computed directly from the frozen experts' softmax outputs and the gate weights; no parameter is fitted to ECE, PAvPU, or any other evaluation metric, and the gate is trained only for segmentation accuracy, so the uncertainty estimates are not 'predictions' of quantities already contained in the training objective. The stacked PE/MI construction in Eq. (2) does include pMoE in the averaged distribution, and the paper itself cautions that this 'may, however, artificially reduce perceived variance'; this is a caveat about variance reduction in the estimator, not a reduction of the claim to its inputs, and the weighted and EV variants are evaluated without that construction. The self-citations to [31]-[33] supply the pretrained experts, the highway/urban split, and the gate architectures; these are transparent experimental inputs, not an unverified theorem that forces the uncertainty result. The comparison against the average-of-two-experts 'ensemble' may be a weak baseline, but a weak baseline is a fairness or correctness concern, not circularity, and the paper explicitly notes that its ensemble differs from deep ensembles [25]. The Cityscapes experiments provide an external benchmark where ensembles often match or beat MoEs, showing the authors did not engineer the evaluation to force the MoE-favorable conclusion.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central comparison relies on the assumption that the ambiguous A2D2 subset is genuinely out-of-distribution, that softmax probabilities are meaningful inputs for entropy and variance measures, and that the accuracy-trained gate remains useful for uncertainty on OOD inputs. The MC dropout baseline uses hand-selected hyperparameters. No invented entities are introduced.

free parameters (2)
  • MC dropout rate = 0.1
    Selected from the sweep in Figure 3 as a trade-off between calibration and segmentation accuracy; affects the MC dropout baseline, not the MoE itself.
  • Number of MC dropout forward passes = 2
    Chosen for a fair comparison with an ensemble or two-expert MoE; the paper states up to 20 forward passes did not improve calibration metrics.
assumptions (4)
  • domain assumption The ambiguous A2D2 test subset is a valid out-of-distribution evaluation set.
    Section 4.1 defines OOD as the ambiguous subset containing scenes absent from training; the OOD superiority claim depends on this subset being distributionally distinct from highway and urban images.
  • domain assumption Softmax outputs of the experts and the MoE are meaningful inputs for entropy, mutual information, and variance-based uncertainty.
    Section 3.2 computes PE, MI, and EV over class probabilities; if these probabilities are poorly calibrated or not comparable across experts, the extracted uncertainties are not meaningful.
  • domain assumption The gate trained only for segmentation accuracy generalizes to OOD inputs well enough to support uncertainty extraction.
    The MoE gate is trained on combined highway-urban data (Section 4.1) and then used on the ambiguous OOD set; the paper assumes the routing weights remain sensible out-of-distribution.
  • domain assumption MC dropout with two forward passes and dropout rate 0.1 is a fair baseline.
    Section 4.2 selects two forward passes for fairness and dropout rate 0.1 based on Figure 3; if the MC dropout baseline had been tuned differently, the comparison could change.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation." pith.science (2026). https://pith.science/paper/QHA3EXIY

@misc{pith2026250904816,
  author       = {Pith},
  title        = {Pith review of: Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QHA3EXIY}},
  note         = {Machine review of arXiv:2509.04816}
}
read the original abstract

Estimating accurate and well-calibrated predictive uncertainty is important for enhancing the reliability of computer vision models, especially in safety-critical applications like traffic scene perception. While ensemble methods are commonly used to quantify uncertainty by combining multiple models, a mixture of experts (MoE) offers an efficient alternative by leveraging a gating network to dynamically weight expert predictions based on the input. Building on the promising use of MoEs for semantic segmentation in our previous works, we show that well-calibrated predictive uncertainty estimates can be extracted from MoEs without architectural modifications. We investigate three methods to extract predictive uncertainty estimates: predictive entropy, mutual information, and expert variance. We evaluate these methods for an MoE with two experts trained on a semantical split of the A2D2 dataset. Our results show that MoEs yield more reliable uncertainty estimates than ensembles in terms of conditional correctness metrics under out-of-distribution (OOD) data. Additionally, we evaluate routing uncertainty computed via gate entropy and find that simple gating mechanisms lead to better calibration of routing uncertainty estimates than more complex classwise gates. Finally, our experiments on the Cityscapes dataset suggest that increasing the number of experts can further enhance uncertainty calibration. Our code is available at https://github.com/KASTEL-MobilityLab/mixtures-of-experts/.

Figures

Figures reproduced from arXiv: 2509.04816 by the authors.

Figure 1
Figure 1. We extract uncertainty estimates from an MoE combin [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. MoE architecture with two experts and an additional [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Semantic segmentation accuracy (dark blue) and uncer [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Calibration evaluation of the predictive uncertainty on in-distribution (highway-urban) and out-of-distribution (ambiguous) A2D2 test data. Lower values indicate better-calibrated uncertainty estimates; comparison models are shown in red. Gate Entropy In-distribution G…
Figure 5
Figure 5. Figure 5: Calibration evaluation of the routing uncertainty on in-distribution (highway-urban) and out-of-distribution (ambiguous) A2D2 test data. Lower values indicate better-calibrated uncertainty estimates. with the classwise gate leading to higher mIoU values. We then evalua…
Figure 6
Figure 6. Figure 6: Semantic segmentation performance on A2D2 test data. [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Conditional correctness metrics for the A2D2 models [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Uncertainty calibration of A2D2 models under data shift. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 36 canonical work pages

  1. [25]

    Simple and scalable predictive uncertainty esti- mation using deep ensembles

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty esti- mation using deep ensembles. In Advances in Neural Infor- mation Processing Systems (NIPS), 2017. 2, 3, 4

    work page 2017

  2. [1]

    Network of experts for large-scale image categoriza- tion

    Karim Ahmed, Mohammad Haris Baig, and Lorenzo Torre- sani. Network of experts for large-scale image categoriza- tion. In European Conference on Computer Vision (ECCV) - Workshops. Springer, 2016. 1

    work page 2016

  3. [2]

    Weight uncertainty in neural networks

    Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. CoRR, abs/1505.05424, 2015. 1, 2, 3

    arXiv 2015

  4. [3]

    Verification of forecasts expressed in terms of probability

    Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly weather review, 1950. 3

    work page 1950

  5. [4]

    Uncertainty prediction and cal- ibration using multi-expert gating mechanism

    Kun Cao and Zongxia Xie. Uncertainty prediction and cal- ibration using multi-expert gating mechanism. In Interna- tional Joint Conference on Neural Networks (IJCNN), 2022. 2

    work page 2022

  6. [5]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In European Conference on Computer Vision (ECCV) - Work- shops, 2018. 4

    work page 2018

  7. [6]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Confer- ence on Computer Vision and Pattern Recognition (CVPR) ,

    work page

  8. [7]

    Closer look at the uncertainty estimation in semantic segmentation un- der distributional shift

    Sebastian Cygert, Bartlomiej Wr ´oblewski, Karol Wozniak, Radoslaw Slowinski, and Andrzej Czyzewski. Closer look at the uncertainty estimation in semantic segmentation un- der distributional shift. In International Joint Conference on Neural Networks (IJCNN), 2021. 2

    work page 2021

Show all 43 references
  1. [8]

    Bayesian active learning for se- mantic segmentation

    Sima Didari, Wenjun Hu, Jae Oh Woo, Heng Hao, Hankyu Moon, and Seungjai Min. Bayesian active learning for se- mantic segmentation. CoRR, abs/2408.01694, 2024. 1

    2024 arXiv

  2. [9]

    Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity. The Journal of Machine Learning Research, 2022. 1

    2022

  3. [10]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning

    Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning (ICML), 2016. 2, 4

    2016

  4. [11]

    Modeling multimodal aleatoric uncertainty in segmentation with mixture of stochastic experts

    Zhitong Gao, Yucong Chen, Chuyu Zhang, and Xuming He. Modeling multimodal aleatoric uncertainty in segmentation with mixture of stochastic experts. In International Confer- ence on Learning Representations (ICLR), 2023. 2

    2023

  5. [12]

    Jakob Geyer, Yohannes Kassahun, Mentar Mahmudi, Xavier Ricou, Rupesh Durgesh, Andrew S. Chung, Lorenz Hauswald, Viet Hoang Pham, Maximilian M ¨uhlegg, Sebas- tian Dorn, Tiffany Fernandez, Martin J ¨anicke, Sudesh Mi- rashi, Chiragkumar Savani, Martin Sturm, Oleksandr V oro- bi...

    2004 arXiv

  6. [13]

    On calibration of modern neural networks

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning (ICML), 2017. 2

    2017

  7. [14]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 4

    2016

  8. [15]

    Deepme: Deep mixture experts for large-scale image classification

    Ming He, Guangyi Lv, Weidong He, Jianping Fan, and Gui- hua Zeng. Deepme: Deep mixture experts for large-scale image classification. In International Joint Conference on Artificial Intelligence (IJCAI), 2021. 1

    2021

  9. [16]

    Dietterich

    Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common corruptions and per- turbations. In International Conference on Learning Repre- sentations (ICLR), 2019. 6

    2019

  10. [17]

    Holder and Muhammad Shafique

    Christopher J. Holder and Muhammad Shafique. Efficient uncertainty estimation in semantic segmentation via distil- lation. In International Conference on Computer Vision (ICCV) - Workshops, 2021. 3

    2021

  11. [18]

    Bayesian active learning for classification and pref- erence learning

    Neil Houlsby, Ferenc Huszar, Zoubin Ghahramani, and M´at´e Lengyel. Bayesian active learning for classification and pref- erence learning. CoRR, abs/1112.5745, 2011. 3

    2011 arXiv

  12. [19]

    Harder tasks need more experts: Dynamic routing in moe models

    Quzhe Huang, Zhenwei An, Nan Zhuang, Mingxu Tao, Chen Zhang, Yang Jin, Kun Xu, Liwei Chen, Songfang Huang, and Yansong Feng. Harder tasks need more experts: Dynamic routing in moe models. CoRR, 2024. 1

    2024

  13. [20]

    Marius Z¨ollner

    Christian Hubschneider, Robin Hutmacher, and J. Marius Z¨ollner. Calibrating uncertainty models for steering angle estimation. In International Conference on Intelligent Trans- portation Systems (ITSC). IEEE, 2019. 3

    2019

  14. [21]

    Jacobs, Michael I

    Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. Adaptive mixtures of local experts.Neu- ral computation, 1991. 1

    1991

  15. [22]

    Mixture of experts with uncertainty voting for imbalanced deep regression problems

    Yuchang Jiang, Vivien Sainte Fare Garnot, Konrad Schindler, and Jan Dirk Wegner. Mixture of experts with uncertainty voting for imbalanced deep regression problems. CoRR, abs/2305.15178, 2023. 2

    2023 arXiv

  16. [23]

    What uncertainties do we need in bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems (NIPS) , 2017

    Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems (NIPS) , 2017. 1, 2

    2017

  17. [24]

    Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding

    Alex Kendall, Vijay Badrinarayanan, and Roberto Cipolla. Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. In British Machine Vision Conference (BMVC), 2017. 2

    2017

  18. [26]

    Training of neural networks with uncertain data, A mixture of experts approach.CoRR, abs/2312.08083,

    Lucas Luttner. Training of neural networks with uncertain data, A mixture of experts approach.CoRR, abs/2312.08083,

    arXiv

  19. [27]

    Evaluating bayesian deep learning methods for semantic segmentation

    Jishnu Mukhoti and Yarin Gal. Evaluating bayesian deep learning methods for semantic segmentation. CoRR, abs/1811.12709, 2018. 2, 3

    2018 arXiv

  20. [28]

    Cooper, and Milos Hauskrecht

    Mahdi Pakdaman Naeini, Gregory F. Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In AAAI Conference on Artificial Intel- ligence (AAAI), 2015. 3

    2015

  21. [29]

    Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran

    Jeremy Nixon, Michael W. Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In Conference on Computer Vision and Pat- tern Recognition (CVPR) - Workshops, 2019. 3

    2019

  22. [30]

    Marius Z¨ollner

    Svetlana Pavlitska, Christian Hubschneider, Lukas Struppek, and J. Marius Z¨ollner. Sparsely-gated mixture-of-expert lay- ers for cnn interpretability. InInternational Joint Conference on Neural Networks (IJCNN), 2023. 1

    2023

  23. [31]

    Towards adversarial robustness of model-level mixture-of- experts architectures for semantic segmentation

    Svetlana Pavlitska, Enrico Eisen, and J Marius Z ¨ollner. Towards adversarial robustness of model-level mixture-of- experts architectures for semantic segmentation. In Inter- national Conference on Machine Learning and Applications (ICMLA), 2024. 4

    2024

  24. [32]

    Mar- ius Z¨ollner

    Svetlana Pavlitskaya, Christian Hubschneider, Michael We- ber, Ruby Moritz, Fabian H¨uger, Peter Schlicht, and J. Mar- ius Z¨ollner. Using mixture of expert models to gain insights into semantic segmentation. In Conference on Computer Vi- sion and Pattern Recognition (CVPR) - ...

    2020

  25. [33]

    Evaluating mixture-of-experts architectures for net- work aggregation

    Svetlana Pavlitskaya, Christian Hubschneider, and Michael Weber. Evaluating mixture-of-experts architectures for net- work aggregation. In Deep Neural Networks and Data for Automated Driving: Robustness, Uncertainty Quantifica- tion, and Insights Towards Safety. Springer, 2022. 2, 4

    2022

  26. [34]

    Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey E. Hinton. Regularizing neural net- works by penalizing confident output distributions. In In- ternational Conference on Learning Representations (ICLR),

  27. [35]

    Deepspeed-moe: Advanc- ing mixture-of-experts inference and training to power next- generation AI scale

    Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, and Yuxiong He. Deepspeed-moe: Advanc- ing mixture-of-experts inference and training to power next- generation AI scale. InInternational Conference on Machine Lear...

    2022

  28. [36]

    Scaling vision with sparse mix- ture of experts

    Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, Andr ´e Susano Pinto, Daniel Keysers, and Neil Houlsby. Scaling vision with sparse mix- ture of experts. Advances in Neural Information Processing Systems, 34:8583–8595, 2021. 1

    2021

  29. [37]

    Bernstein, Alexander C

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition chal- lenge. Int. J. Comput. Vis., 2015. 4

    2015

  30. [38]

    Le, Geoffrey E

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V . Le, Geoffrey E. Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In International Conference on Learning Representations (ICLR), 2017. 1

    2017

  31. [39]

    Sculley, Joshua V

    Jasper Snoek, Yaniv Ovadia, Emily Fertig, Balaji Lakshmi- narayanan, Sebastian Nowozin, D. Sculley, Joshua V . Dil- lon, Jie Ren, and Zachary Nado. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information ...

    2019

  32. [40]

    Rethinking the inception ar- chitecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception ar- chitecture for computer vision. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 2

    2016

  33. [41]

    Gonzalez

    Xin Wang, Fisher Yu, Lisa Dunlap, Yi-An Ma, Ruth Wang, Azalia Mirhoseini, Trevor Darrell, and Joseph E. Gonzalez. Deep mixture of experts via shallow embedding. In Con- ference on Uncertainty in Artificial Intelligence, UAI. AUAI Press, 2019. 1

    2019

  34. [42]

    Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, Yuan Du, and Shanghang Zhang

    Rongyu Zhang, Yulin Luo, Jiaming Liu, Huanrui Yang, Zhen Dong, Denis A. Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, Yuan Du, and Shanghang Zhang. Ef- ficient deweather mixture-of-experts with uncertainty-aware feature-wise linear modulation. CoRR, abs/2312.16610,

    arXiv

  35. [43]

    Self-supervised mixture-of-experts by uncertainty estimation

    Zhuobin Zheng, Chun Yuan, Xinrui Zhu, Zhihui Lin, Yangyang Cheng, Cheng Shi, and Jiahui Ye. Self-supervised mixture-of-experts by uncertainty estimation. In AAAI Con- ference on Artificial Intelligence (AAAI), 2019. 2

    2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.