ExpertSim: Fast Particle Detector Simulation Using Mixture-of-Generative-Experts

Filip Szatkowski; Jan Dubi\'nski; Kamil Deja; Patryk B\k{e}dkowski; Przemys{\l}aw Rokita; Tomasz Trzci\'nski

REVIEW 4 major objections 5 minor 57 references

ExpertSim: Fast Particle Detector Simulation Using Mixture-of-Generative-Experts

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Three expert generators beat single-network particle simulation

desk verdict Solid, reproducible MoE-GAN result for ZDC simulation with an overclaimed headline comparison set. read the letter →

arxiv 2508.20991 v1 pith:KDMY4AWI submitted 2025-08-28 cs.CV cs.AI

Patryk Będkowski , Jan Dubiński , Filip Szatkowski , Kamil Deja , Przemys{l}aw Rokita , Tomasz Trzciński This is my paper

classification cs.CVcs.AI

keywords particledetectorsimulationmixtureofexpertsgenerativeadversarialnetworksZeroDegreeCalorimeterresponsegenerationrouterspecializationWassersteindistancefast

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ExpertSim tests whether a mixture of small generative experts can simulate a particle calorimeter's response more faithfully than a single large generator, without paying a speed penalty. The paper's answer is yes: a lightweight router sends each collision's conditional data — energy, mass, charge, positions, momenta — to one of three GAN-based experts, and a differentiation loss pushes those experts to specialize by response intensity. On the Zero Degree Calorimeter's proton and neutron channels, ExpertSim reduces the Wasserstein distance between real and simulated responses from 2.07/1.89 (best single-model baseline) to 1.59/1.34, a more-than-15% improvement, while the router adds only about 2% inference overhead. If this holds, high-fidelity detector simulation becomes dramatically cheaper than the Monte Carlo methods that currently consume large CPU fleets at collider experiments.

What carries the argument

The load-bearing mechanism is a Mixture-of-Generative-Experts: a fully connected router network assigns each input to one of three small GAN experts. The router's differentiation loss, Ldiff, maximizes pairwise squared differences between the experts' mean generated-image intensities, which is what forces specialization into low-, medium-, and high-intensity response regimes; the utilization entropy loss Lutil keeps the load balanced. Each expert's generator is further anchored by diversity regularization, intensity regularization, and shower-center regression.

What would settle it

Retrain ExpertSim with the router's assignments replaced by random three-way splits that preserve the same load balance, keeping every expert loss fixed. If the Wasserstein distance does not rise substantially, the learned routing is not the cause of the improvement. Equivalently, compute each expert's conditional Wasserstein distance against only the true responses for the inputs the router assigns to it; if any routed subset is no closer than the single-model baseline, the experts are not specializing along the true response modes.

Watch

Extended reading notes

Core claim

The central claim is that the distribution of Zero Degree Calorimeter responses — which falls naturally into low-, medium-, and high-intensity groups — is modeled better by routing each input to one of three specialized generators than by any single generator. Each expert is an SDI-GAN equipped with an intensity-regularization term that matches total deposited photon counts and an auxiliary regressor that predicts the shower-center location. The router is trained with two terms: an entropy-based utilization loss for balanced load and a differentiation loss that maximizes pairwise differences in the mean intensity of each expert's generated images. The resulting specialization is visible in t

Load-bearing premise

The whole scheme rests on the assumption that the nine input variables (energy, mass, charge, positions, momenta) carry enough information to predict whether a collision's detector response will be low, medium, or high intensity, so that routing by those variables separates the real distribution instead of distorting it.

Editorial extensions

If this is right

Replacing one monolithic generator with three routed experts improves fidelity on every intensity quartile, not just on average.
Inference stays near single-model speed: on GPU the three-expert model runs in 3.27 s vs 3.20 s (about 2% overhead), and on CPU it takes 314 s vs 273 s, still roughly 35x faster than Monte Carlo's 11,172 s.
Three experts is the empirically optimal count: ablations show 2, 4, and 5 experts all give worse Wasserstein distances, matching the physical three-mode structure of the calorimeter responses.
Both router losses are necessary: disabling the differentiation loss raises WS to 6.80/8.01, and disabling the utilization loss raises it to 2.10/1.89, showing that specialization alone or balance alone is not enough.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension is to compare the router's learned assignment against a physics-derived label of shower intensity; if they coincide, the differentiation loss could be replaced by a supervised classifier, potentially improving sample efficiency.
The same router-plus-differentiated-experts recipe should transfer to other detectors with multimodal response distributions; the decisive diagnostic would be whether each expert's within-subset Wasserstein distance stays below the single-model baseline after training on shifted energy spectra.
The intensity-difference separation signal ignores spatial shape: two responses with the same total intensity but different shower profiles would not be distinguished by Ldiff. Augmenting it with a distributional distance per expert would test whether spatial modes also separate.
The reported evaluation uses per-channel and per-quartile Wasserstein distances; checking the joint distribution over all five photomultiplier channels would reveal whether marginal improvements hide residual cross-channel correlation errors.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take. The paper's core contribution is real: a Mixture-of-Generative-Experts architecture for fast ZDC simulation, where a router assigns events to one of three GAN experts, each specializing in different intensity regimes. The authors show a clear, low-variance improvement over their tested GAN baselines (Wasserstein 1.59/1.34 vs. 2.07/1.89 for the best single-model variant), and the ablations are genuinely informative. They vary the number of experts, disable the differentiation and utilization losses, and report robustness to hyperparameters. Code is available. That is a solid empirical package.

The main soft spot is the claim in Section 1 that ExpertSim “significantly outperforms all existing approaches.” That is not supported by the comparison set. The paper cites flow-matching ZDC simulators (refs 52–54) but never benchmarks them. So the headline should be “outperforms single-generator GAN variants,” not “all existing approaches.” The stress-test note is right on this. The abstract also reports a Wasserstein distance of 1.70, which does not match the 1.59/1.34 in Table 2. That inconsistency needs fixing before publication.

One more concern is the router training. The differentiation loss maximizes differences in mean intensities between experts, which could in principle push experts away from the real distribution rather than toward complementary parts of it. But the ablation shows that disabling Ldiff degrades performance severely (WS 6.80/8.01), and Table 3 shows the experts do end up with different mean intensities. So empirically the concern is largely addressed, though the mechanism is under-explained. I would have liked more detail on how the router is trained end-to-end, especially how the straight-through estimator or soft assignment works, but that is a reproducibility issue rather than a fatal flaw.

Overall, this is a useful paper for people working on generative models for detector simulation, and the empirical methodology is better than much of what I see in this area. The overclaim is real but fixable; the abstract inconsistency is sloppy but not indicative of a deeper problem. If I were handling this, I would send it to peer review and require the authors to either benchmark the cited flow-matching methods or soften the claim, and to align the abstract with the actual table values. A serious referee can definitely engage with this constructively.

Referee Report

4 major / 5 minor

Summary. ExpertSim proposes a Mixture-of-Generative-Experts framework for fast simulation of Zero Degree Calorimeter responses in the ALICE experiment at CERN. The method routes each collision, described by nine conditional particle variables, to one of three GAN-based experts, each trained with diversity, intensity, and auxiliary-regression losses. The router is trained with an entropy-based utilization loss and a differentiation loss that encourages different mean output intensities across experts. The paper reports a Wasserstein distance of 1.59 (Proton) and 1.34 (Neutron), outperforming SDI-GAN+IR+AR (2.07/1.89) at roughly single-model inference speed, and provides ablations on the number of experts and router hyperparameters.

Significance. If the reported results are robust, ExpertSim is a practically useful contribution to fast calorimeter simulation, with a sensible architectural idea — specializing generative experts through a physically motivated router — and a reproducible code release. The authors provide multi-run statistics, ablations, and inference-time benchmarks, which is good experimental practice. However, the central claim of outperforming 'all existing approaches' is not supported by the comparison set, and the headline Wasserstein number in the abstract is inconsistent with the tables. The core architecture is plausible and the reported gains over the included GAN baselines are meaningful, but the scope of the empirical validation must be corrected before the paper can be considered acceptable.

major comments (4)

[Abstract and Section 1 vs. Table 2] The abstract and Section 1 state a Wasserstein distance of 1.70 and claim ExpertSim significantly outperforms all existing approaches. Table 2 reports 1.59 (Proton) and 1.34 (Neutron) for ExpertSim, and the improvement is measured against only SDI-GAN+IR+AR (2.07/1.89). The value 1.70 does not appear anywhere in the results, and the comparison set does not justify 'all existing approaches'. Please reconcile the reported number and restrict the claim to the evaluated baselines, or benchmark the cited non-GAN simulators (e.g., refs [12], [52]–[54]).
[Section 5.1 and Related Work] The paper's headline claim ('outperforms all existing approaches') is unsupported because the experiments in Tables 1 and 2 include only GAN and SDI-GAN variants. The Related Work and Section 2 cite Sinkhorn autoencoders [12] and flow-matching/normalizing-flow ZDC simulators [52]–[54], yet none are evaluated. If these methods are indeed applicable to the same ZDC task and metric, they must be included in the comparison or the claim must be weakened to 'outperforms the tested GAN baselines'. This is load-bearing for the paper's central contribution.
[Section 5.3, Table 5] The text states that 'our method remains robust across a wide range of λutil and λdiff values'. The table does not support this: the default (0.01, 0.0001) yields WS 1.59/1.34, but λutil=0.1 gives 1.86/4.14, λutil=0.001 gives 2.34/2.52, λdiff=0.001 gives 1.92/1.91, and λdiff=0.000001 gives 1.78/1.74. Several of these are substantially worse than the default, especially for the neutron channel. The sensitivity is not negligible for the proton and is severe for the neutron. Please report variance across runs and temper the robustness claim.
[Sections 4.2.2 and 5.2, Table 3] The specialization evidence in Table 3 is partly circular: the router's differentiation loss Ldiff (Eq. 7) explicitly maximizes differences in mean generated intensities between experts, and Table 3 reports exactly mean intensities of each expert. Showing that experts have different mean intensities therefore only confirms that the loss achieved its stated objective. To demonstrate that experts specialize on meaningful and complementary data subsets, the authors should report per-expert conditional distributions or per-expert Wasserstein distances on held-out data, or show that the routing is stable across test samples rather than merely reflecting the training objective.

minor comments (5)

[Table 1] The caption says 'mean WS metric across 4 quartiles' but the table appears to report WS per quartile. Please clarify whether these are single-run values or averaged over multiple runs, and add standard deviations or a note on significance.
[Table 6] The text claims router overhead is 2%, but this is only true for GPU time (3.20 vs 3.27 s). CPU time increases from 273 to 314 s (about 15%). Please report both values clearly and adjust the claim accordingly.
[Equation (5)] The adversarial loss Ladv is used but not explicitly defined. Define it or cite the DCGAN formulation. Also, the notation N in Eq. (6) (number of experts) conflicts with the batch size N used in Eq. (4).
[Section 4.2.2] Equation (7) defines Ldiff as a negative sum of squared differences; the text says the loss 'encourages diversity' and 'maximizes differences'. This is correct when minimizing Ldiff, but the sign convention may confuse readers. Please state explicitly that Ldiff is minimized and that this maximizes the squared differences.
[References] Ref. [17] appears as a preprint with no venue; if it has since been published, please update. In addition, refs. [52]–[54] are closely related to the ZDC simulation task and should be discussed in the experimental comparison if not evaluated.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: central WS claim is evaluated on held-out MC data; minor by-construction observations are not load-bearing.

full rationale

The central empirical claim—ExpertSim achieves lower Wasserstein distance than GAN/SDI-GAN variants—is established by comparing generated samples against a held-out Monte Carlo test set (Section 5, Tables 1–2). The router training objectives (Eqs. 6–7) and the auxiliary losses (Eqs. 3–4) are not re-statements of the WS metric: they are proxies (load balance, intensity separation, spatial regression), and the WS numbers are computed on real data. The intensity differentiation loss does make expert specialization by intensity a direct consequence of the objective, and Table 3 simply confirms this designed behavior; but the paper does not present that table as an independent prediction, and the global WS advantage does not reduce to the loss. Self-citations [3], [17], [18] supply components, hyperparameters, and data, but the comparison against single-generator baselines is self-contained, and the cited works are not invoked to rule out alternatives. The headline 'all existing approaches' is overbroad because cited flow-matching ZDC simulators ([52]–[54]) are not benchmarked, and the abstract's WS 1.70 does not match Table 2's 1.59/1.34; these are scope/consistency concerns, not circularity. No circular step found.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities, particles, forces, or conserved quantities. The 'experts' and 'router' are architectural components, not invented entities. The free parameters are loss weights and the number of experts, which are standard ML hyperparameters tuned on validation data.

free parameters (4)

lambda_util (router utilization loss weight) = 0.01 (default)
Tuned via logarithmic grid search on validation data (Table 5); central to load balancing.
lambda_diff (router differentiation loss weight) = 0.0001 (default)
Tuned via grid search (Table 5); controls intensity specialization among experts.
lambda_div, lambda_in, lambda_aux (expert loss weights) = Not specified in this paper, inherited from ref [3]
Taken from prior work on the same ZDC task; specific values are omitted, yet they scale the expert objectives.
Number of experts = 3 (chosen by ablation)
Selected because ablations with 2, 4, or 5 experts give worse WS (Table 4), although the choice is also motivated by the three physical intensity groups.

assumptions (4)

standard math GAN adversarial training converges sufficiently to produce valid generator outputs with the given loss weighting.
The empirical results assume that the adversarial objective plus regularizations reaches a useful optimum; no convergence guarantee is provided.
domain assumption The ZDC response distribution is naturally decomposed into three intensity groups (low, medium, high) that are separable by the 9 conditional particle variables.
Section 5.3 states the calorimeters generally produce three response types, and the router is expected to partition inputs accordingly (Section 4.2).
domain assumption The total intensity fin(x) (sum of pixel values) is a sufficient summary for expert specialization and for the intensity regularization.
Eq. 2 defines fin and it is used both in the generator loss (Eq. 3) and the router differentiation loss (Eq. 7). This assumes intensity captures the dominant response variation.
ad hoc to paper Maximizing differences in mean generated-image intensities between experts yields experts that collectively match the real distribution better.
The differentiation loss Ldiff (Eq. 7) is a heuristic objective; the paper does not prove that it preserves intra-expert fidelity or prevents mode collapse to extreme intensities.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ExpertSim: Fast Particle Detector Simulation Using Mixture-of-Generative-Experts." pith.science (2026). https://pith.science/paper/KDMY4AWI

@misc{pith2026250820991,
  author       = {Pith},
  title        = {Pith review of: ExpertSim: Fast Particle Detector Simulation Using Mixture-of-Generative-Experts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KDMY4AWI}},
  note         = {Machine review of arXiv:2508.20991}
}

read the original abstract

Simulating detector responses is a crucial part of understanding the inner workings of particle collisions in the Large Hadron Collider at CERN. Such simulations are currently performed with statistical Monte Carlo methods, which are computationally expensive and put a significant strain on CERN's computational grid. Therefore, recent proposals advocate for generative machine learning methods to enable more efficient simulations. However, the distribution of the data varies significantly across the simulations, which is hard to capture with out-of-the-box methods. In this study, we present ExpertSim - a deep learning simulation approach tailored for the Zero Degree Calorimeter in the ALICE experiment. Our method utilizes a Mixture-of-Generative-Experts architecture, where each expert specializes in simulating a different subset of the data. This allows for a more precise and efficient generation process, as each expert focuses on a specific aspect of the calorimeter response. ExpertSim not only improves accuracy, but also provides a significant speedup compared to the traditional Monte-Carlo methods, offering a promising solution for high-efficiency detector simulations in particle physics experiments at CERN. We make the code available at https://github.com/patrick-bedkowski/expertsim-mix-of-generative-experts.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 50 canonical work pages

[12]

End-to-end Sinkhorn Autoencoder with Noise Generator

K. Deja et al. End-to-end Sinkhorn Autoencoder with Noise Genera- tor. 2020. arXiv: 2006.06704 [cs.LG]

work page Pith review arXiv 2020
[52]

Applying generative neural networks for fast simulations of the ALICE (CERN) experiment

M. Wojnar. “Applying generative neural networks for fast sim- ulations of the ALICE (CERN) experiment”. In: arXiv preprint arXiv:2407.16704 (2024)

work page Pith review arXiv 2024
[54]

Fast simulation of the Zero Degree Calorimeter responses with generative neural networks

M. Wojnar, E. Majerz, and W. Dzwinel. “Fast simulation of the Zero Degree Calorimeter responses with generative neural networks”. In: Computing and Software for Big Science 1 (2025), p. 1

work page 2025
[1]

The ALICE Zero Degree Calorimeters

ALICE Collaboration. The ALICE Zero Degree Calorimeters. CERN- ALI-99-17; CERN-ALICE-PUB-99-17. CERN, Apr. 1999

work page 1999
[2]

PonderNet: Learning to Pon- der

A. Banino, J. Balaguer, and C. Blundell. “PonderNet: Learning to Pon- der”. In: 8th ICML Workshop on Automated Machine Learning (Au- toML). 2021

work page 2021
[3]

Deep Generative Models for Proton Zero Degree Calorimeter Simulations in ALICE, CERN

P. B˛ edkowski et al. “Deep Generative Models for Proton Zero De- gree Calorimeter Simulations in ALICE, CERN”. In: arXiv preprint arXiv:2406.03263 (2024)

work page Pith review arXiv 2024
[4]

Estimating or propagating gradients through stochastic neurons for conditional computation

Y . Bengio, N. Léonard, and A. Courville. “Estimating or propagating gradients through stochastic neurons for conditional computation”. In: arXiv preprint arXiv:1308.3432 (2013)

arXiv 2013
[5]

Worldwide LHC Computing Grid Resources Report

CERN. Worldwide LHC Computing Grid Resources Report . Remote access (16.01.2024). 2024

work page 2024

Show all 57 references

[6]

Mod-Squad: Designing Mixtures of Experts As Mod- ular Multi-Task Learners

Z. Chen et al. “Mod-Squad: Designing Mixtures of Experts As Mod- ular Multi-Task Learners”. In: Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR). June 2023

2023
[7]

Unified scaling laws for routed language models

A. Clark et al. “Unified scaling laws for routed language models”. In: International Conference on Machine Learning. 2022

2022
[8]

CaloMan: Fast generation of calorimeter show- ers with density estimation on learned manifolds

J. C. Cresswell et al. “CaloMan: Fast generation of calorimeter show- ers with density estimation on learned manifolds”. In: arXiv preprint arXiv:2211.15380 (2022)

2022 arXiv
[9]

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

D. Dai et al. “DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models”. In: Annual Meeting of the Association for Computational Linguistics (2024)

2024
[10]

Mobile V-MoEs: Scaling Down Vision Transformers via Sparse Mixture-of-Experts

E. Daxberger et al. “Mobile V-MoEs: Scaling Down Vision Transformers via Sparse Mixture-of-Experts”. In: arXiv preprint arXiv:2309.04354 (2023)

2023 arXiv
[11]

Universal Transformers

M. Dehghani et al. “Universal Transformers”. In: International Con- ference on Learning Representations. 2018

2018
[13]

Generative models for fast cluster simulations in the TPC for the ALICE experiment

K. Deja et al. “Generative models for fast cluster simulations in the TPC for the ALICE experiment”. In:Conference on Information Tech- nology, Systems Research and Computational Physics. 2018

2018
[14]

ALICE technical design report of the zero degree calorimeter (ZDC)

G. e. a. Dellacasa. “ALICE technical design report of the zero degree calorimeter (ZDC)”. In: (1999)

1999
[15]

DijetGAN: a generative-adversarial network ap- proach for the simulation of QCD dijet events at the LHC

R. Di Sipio et al. “DijetGAN: a generative-adversarial network ap- proach for the simulation of QCD dijet events at the LHC”. In:Journal of high energy physics (2019)

2019
[16]

Glam: Efficient scaling of language models with mixture- of-experts

N. Du et al. “Glam: Efficient scaling of language models with mixture- of-experts”. In: International Conference on Machine Learning. 2022

2022
[17]

Machine Learning methods for simulating particle response in the Zero Degree Calorimeter at the ALICE experiment, CERN

J. Dubi ´nski, K. Deja, S. Wenzel, et al. “Machine Learning methods for simulating particle response in the Zero Degree Calorimeter at the ALICE experiment, CERN”. In: (2023)

2023
[18]

Selectively Increasing the Diversity of GAN- Generated Samples

J. Dubi ´nski et al. “Selectively Increasing the Diversity of GAN- Generated Samples”. In: Neural Information Processing. 2023. DOI: 10.1007/978-3-031-30105-6_22

2023 doi
[19]

Depth-Adaptive Transformer

M. Elbayad et al. “Depth-Adaptive Transformer”. In: International Conference on Learning Representations. 2019

2019
[20]

Precise simulation of electromagnetic calorimeter showers using a Wasserstein Generative Adversarial Network

M. Erdmann, J. Glombitza, and T. Quast. “Precise simulation of electromagnetic calorimeter showers using a Wasserstein Generative Adversarial Network”. In: Computing and Software for Big Science (2019)

2019
[21]

Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity

W. Fedus, B. Zoph, and N. Shazeer. “Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity”. In: The Journal of Machine Learning Research (2022)

2022
[22]

Spatially adaptive computation time for residual networks

M. Figurnov et al. “Spatially adaptive computation time for residual networks”. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017

2017
[23]

Generative Adversarial Nets

I. Goodfellow et al. “Generative Adversarial Nets”. In: Advances in Neural Information Processing Systems. 2014

2014
[24]

Adaptive computation time for recurrent neural net- works

A. Graves. “Adaptive computation time for recurrent neural net- works”. In: arXiv preprint arXiv:1603.08983 (2016)

2016 arXiv
[25]

Gururangan et al

S. Gururangan et al. DEMix Layers: Disentangling Domains for Mod- ular Language Modeling. 2021. arXiv: 2108.05036 [cs.CL]

2021 arXiv
[26]

Channel selection us- ing gumbel softmax

C. Herrmann, R. S. Bowen, and R. Zabih. “Channel selection us- ing gumbel softmax”. In: European Conference on Computer Vision. Springer. 2020

2020
[27]

Distilling the knowledge in a neural network

G. Hinton, O. Vinyals, and J. Dean. “Distilling the knowledge in a neural network”. In: arXiv preprint arXiv:1503.02531 (2015)

2015 arXiv
[28]

Learning to simulate high energy particle colli- sions from unlabeled data

J. N. Howard et al. “Learning to simulate high energy particle colli- sions from unlabeled data”. In: Scientific Reports (2022)

2022
[29]

Three Dimensional Energy Parametrized Generative Adversarial Networks for Electro- magnetic Shower Simulation

G. R. Khattak, S. Vallecorsa, and F. Carminati. “Three Dimensional Energy Parametrized Generative Adversarial Networks for Electro- magnetic Shower Simulation”. In:2018 25th IEEE International Con- ference on Image Processing (ICIP). 2018

2018
[30]

Auto-Encoding Variational Bayes

D. P. Kingma and M. Welling. “Auto-Encoding Variational Bayes”. In: 2014. DOI: 10.48550/arXiv.1312.6114

2014 doi arXiv
[31]

Kita et al

M. Kita et al. Generative Diffusion Models for Fast Simula- tions of Particle Collisions at CERN . 2024. arXiv: 2406 . 03233 [physics.data-an]. URL: https://arxiv.org/abs/2406.03233

2024 arXiv
[32]

Krishnamurthy, C

Y . Krishnamurthy, C. Watkins, and T. Gaertner. Improving Expert Spe- cialization in Mixture of Experts. 2023. arXiv: 2302.14703[cs.LG]. URL: https://arxiv.org/abs/2302.14703

2023 arXiv
[33]

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

D. Lepikhin et al. “GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding”. In: International Conference on Learning Representations. 2020

2020
[34]

Pruning and quantization for deep neural network ac- celeration: A survey

T. Liang et al. “Pruning and quantization for deep neural network ac- celeration: A survey”. In: Neurocomputing (2021)

2021
[35]

Runtime Neural Pruning

J. Lin et al. “Runtime Neural Pruning”. In: Advances in Neural Infor- mation Processing Systems
[36]

Learning instance-wise sparsity for accelerating deep models

C. Liu et al. “Learning instance-wise sparsity for accelerating deep models”. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. 2019

2019
[37]

Split computing and early exiting for deep learning applications: Survey and research chal- lenges

Y . Matsubara, M. Levorato, and F. Restuccia. “Split computing and early exiting for deep learning applications: Survey and research chal- lenges”. In: ACM Computing Surveys (2022)

2022
[38]

Evomoe: An evolutional mixture-of-experts train- ing framework via dense-to-sparse gate

X. Nie et al. “Evomoe: An evolutional mixture-of-experts train- ing framework via dense-to-sparse gate”. In: arXiv preprint arXiv:2112.14397 (2021)

2021 arXiv
[39]

CaloGAN: Simulat- ing 3D high energy particle showers in multilayer electromagnetic calorimeters

M. Paganini, L. de Oliveira, and B. Nachman. “CaloGAN: Simulat- ing 3D high energy particle showers in multilayer electromagnetic calorimeters”. In: Physical Review D (2018)

2018
[40]

MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation

D. Park et al. “MEGAN: Mixture of Experts of Generative Adversarial Networks for Multimodal Image Generation”. In: July 2018

2018
[41]

Scaling vision with sparse mixture of experts

C. Riquelme, J. Puigcerver, B. Mustafa, et al. “Scaling vision with sparse mixture of experts”. In: Advances in Neural Information Pro- cessing Systems (2021)

2021
[42]

Rogozi ´nski et al

K. Rogozi ´nski et al. Particle physics DL-simulation with control over generated data properties. 2024. arXiv: 2405.14049 [cs.LG]. URL: https://arxiv.org/abs/2405.14049

2024 arXiv
[43]

Hash layers for large sparse models

S. Roller, S. Sukhbaatar, J. Weston, et al. “Hash layers for large sparse models”. In: Advances in Neural Information Processing Sys- tems (2021)

2021
[44]

Why should we add early exits to neural net- works?

S. Scardapane et al. “Why should we add early exits to neural net- works?” In: Cognitive Computation (2020)

2020
[45]

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

N. Shazeer et al. “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer”. In: International Confer- ence on Learning Representations. 2016

2016
[46]

Variational Mixture-of-Experts Autoencoders for Multi- Modal Deep Generative Models

Y . Shi et al. “Variational Mixture-of-Experts Autoencoders for Multi- Modal Deep Generative Models”. In:Advances in Neural Information Processing Systems. 2019

2019
[47]

Efficientnet: Rethinking model scaling for con- volutional neural networks

M. Tan and Q. Le. “Efficientnet: Rethinking model scaling for con- volutional neural networks”. In: International conference on machine learning. PMLR. 2019

2019
[48]

Tolstikhin et al

I. Tolstikhin et al. Wasserstein Auto-Encoders. 2017. arXiv: 1711 . 01558 [stat.ML]

2017
[49]

Dynamic convolutions: Exploiting spa- tial sparsity for faster inference

T. Verelst and T. Tuytelaars. “Dynamic convolutions: Exploiting spa- tial sparsity for faster inference”. In: Proceedings of the ieee/cvf con- ference on computer vision and pattern recognition. 2020

2020
[50]

Skipnet: Learning dynamic routing in convolutional networks

X. Wang et al. “Skipnet: Learning dynamic routing in convolutional networks”. In: Proceedings of the European Conference on Computer Vision (ECCV). 2018

2018
[51]

Zero time waste in pre-trained early exit neural net- works

B. Wójcik et al. “Zero time waste in pre-trained early exit neural net- works”. In: Neural Networks (2023)

2023
[53]

M. Wojnar. Even Faster Simulations with Flow Matching: A Study of Zero Degree Calorimeter Responses . 2025. arXiv: 2507 . 18811 [cs.LG]

2025
[55]

MoEfication: Transformer Feed-forward Layers are Mixtures of Experts

Z. Zhang et al. “MoEfication: Transformer Feed-forward Layers are Mixtures of Experts”. In: Findings of the Association for Computa- tional Linguistics: ACL 2022. 2022

2022
[56]

Mixture-of-experts with expert choice routing

Y . Zhou et al. “Mixture-of-experts with expert choice routing”. In: Advances in Neural Information Processing Systems (2022)

2022
[57]

MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation

S. Zuo et al. “MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation”. In: NAACL 2022. July 2022

2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [12]

End-to-end Sinkhorn Autoencoder with Noise Generator

K. Deja et al. End-to-end Sinkhorn Autoencoder with Noise Genera- tor. 2020. arXiv: 2006.06704 [cs.LG]

work page Pith review arXiv 2020

[2] [52]

Applying generative neural networks for fast simulations of the ALICE (CERN) experiment

M. Wojnar. “Applying generative neural networks for fast sim- ulations of the ALICE (CERN) experiment”. In: arXiv preprint arXiv:2407.16704 (2024)

work page Pith review arXiv 2024

[3] [54]

Fast simulation of the Zero Degree Calorimeter responses with generative neural networks

M. Wojnar, E. Majerz, and W. Dzwinel. “Fast simulation of the Zero Degree Calorimeter responses with generative neural networks”. In: Computing and Software for Big Science 1 (2025), p. 1

work page 2025

[4] [1]

The ALICE Zero Degree Calorimeters

ALICE Collaboration. The ALICE Zero Degree Calorimeters. CERN- ALI-99-17; CERN-ALICE-PUB-99-17. CERN, Apr. 1999

work page 1999

[5] [2]

PonderNet: Learning to Pon- der

A. Banino, J. Balaguer, and C. Blundell. “PonderNet: Learning to Pon- der”. In: 8th ICML Workshop on Automated Machine Learning (Au- toML). 2021

work page 2021

[6] [3]

Deep Generative Models for Proton Zero Degree Calorimeter Simulations in ALICE, CERN

P. B˛ edkowski et al. “Deep Generative Models for Proton Zero De- gree Calorimeter Simulations in ALICE, CERN”. In: arXiv preprint arXiv:2406.03263 (2024)

work page Pith review arXiv 2024

[7] [4]

Estimating or propagating gradients through stochastic neurons for conditional computation

Y . Bengio, N. Léonard, and A. Courville. “Estimating or propagating gradients through stochastic neurons for conditional computation”. In: arXiv preprint arXiv:1308.3432 (2013)

arXiv 2013

[8] [5]

Worldwide LHC Computing Grid Resources Report

CERN. Worldwide LHC Computing Grid Resources Report . Remote access (16.01.2024). 2024

work page 2024