Pith. sign in

REVIEW 3 major objections 4 minor 47 references

Replacing fixed top-k routing in MoE-LoRA with a confidence-adaptive nucleus rule improves accuracy at matched compute and yields a free uncertainty signal.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 03:15 UTC pith:MVVSYM2A

load-bearing objection Solid, modest contribution: adaptive expert-count routing via router-mass thresholding is genuinely new to MoE-LoRA and broadly evaluated, but the optimality proofs rely on unproven monotonicity and the empirics need error bars and a real artifact. the 3 major comments →

arxiv 2607.26052 v2 pith:MVVSYM2A submitted 2026-07-28 cs.LG

Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA

Tom Saliencro , Rohan Desai , Priya Nair , Maya Lindqvist , Daniel Whitmore This is my paper
classification cs.LG
keywords MoE-LoRAadaptive routinguncertainty estimationnucleus samplingout-of-distribution detectionconditional computationlow-rank adaptationmixture of experts
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

MoE-LoRA adapters route every token to a fixed number of experts, wasting compute on easy tokens and starving hard ones. CARE exploits the router's own softmax distribution as a per-token confidence signal: peaked mass means confident, flat means ambiguous. It admits experts in nucleus fashion until cumulative router mass reaches a budget-calibrated threshold, with an extension when admitted experts disagree. Across eight commonsense benchmarks and math, code, and knowledge tasks on two backbones, CARE improves accuracy over fixed top-k at matched average compute and matches fixed-k=4 accuracy while activating 12% fewer experts. The same signals give single-pass OOD detection that beats MSP, entropy, and multi-pass proxies.

Core claim

The central claim is that the router distribution in a trained MoE-LoRA is already a per-token uncertainty signal, and that using it to adapt the number of active experts—admitting experts in decreasing router weight until cumulative mass reaches a threshold, plus a small extension when admitted experts disagree—strictly improves the accuracy–compute frontier over fixed top-k routing. CARE is a drop-in, parameter-free, single-forward-pass rule that replaces only the gate. At matched average compute it improves accuracy (e.g., +0.5 on LLaMA commonsense), and at matched accuracy it needs 3.5 experts on average versus 4.0 for fixed-k. The same concentration and disagreement signals, blended as

What carries the argument

The carrying mechanism is the nucleus admission rule: sort router weights, accumulate them, and stop at the first k whose cumulative mass C_k reaches threshold τ; optionally extend by up to γ experts scaled by disagreement D(h;S) beyond δ. The budget thermostat (Eq. 5) calibrates τ by bisection on a held-out set to match an average budget B, so the average number of active experts is controlled. Disagreement D is the weighted per-coordinate variance of admitted expert outputs normalized by mean squared magnitude. Proposition 3 connects the thermostat to an optimal budget allocation under concavity and monotonicity of marginal gains.

Load-bearing premise

The central premise is that a token's chance of being answered correctly increases as the router's top-1 mass increases, and that each additional expert helps in proportion to the router mass it covers; if either monotonicity fails, CARE's optimality guarantees and potentially its accuracy gains weaken.

What would settle it

Measure the empirical correctness rate of tokens as a function of top-1 router mass p(1) on a given MoE-LoRA checkpoint. If the curve is non-monotonic (e.g., tokens with p(1) near 0.9 are less accurate than those near 0.6), then the confidence ranking guarantee (Prop. 2) fails and CARE's allocation may not be optimal. Similarly, plot marginal accuracy gain from the (k+1)-th expert against cumulative mass C_k; if the gain is not non-decreasing in C_k, the budget thermostat may not implement the claimed optimal allocation.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Any trained MoE-LoRA checkpoint can be wrapped with CARE at inference time by replacing the gate; no retraining, no extra parameters, no extra forward passes.
  • The accuracy–compute frontier shifts upward: at any average budget CARE is more accurate than fixed top-k, and it reaches the fixed-k=4 accuracy with 12% fewer active experts.
  • CARE's confidence and disagreement signals yield a single-pass OOD detection score (AUROC 0.668) that outperforms max-softmax, entropy, and multi-pass proxies like MC-dropout and deep ensembles.
  • Under distribution shift, CARE retains accuracy better than fixed top-k at the same average compute (53.1% vs 50.3%), because shifted inputs flatten the router and trigger more expert admissions.
  • The thermostat provides a single knob τ that trades compute for accuracy along the frontier, letting deployments select an operating point.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The optimality guarantees (Props 2 and 3) rely on monotonicity of correctness probability in top-1 mass and monotonicity of marginal gain in cumulative mass; these are only partially validated. If they fail on a target distribution, CARE's allocation may not be budget-optimal, though the empirical gains could persist.
  • CARE's benefit should scale with per-token difficulty heterogeneity; on homogeneous tasks it converges to fixed-k. A practitioner can pre-screen by measuring the variance of routing entropy across a small sample.
  • The disagreement signal is defined over classification-style outputs; extending it to free-form generation (e.g., via semantic equivalence) is a natural next step that the paper leaves open.
  • Per-layer thermostats, mentioned as an extension, could improve allocation further; a testable hypothesis is that layerwise calibration with per-layer budgets yields additional accuracy gains at the same total compute.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes CARE, a test-time routing rule for MoE-LoRA that replaces fixed top-k expert selection with per-token nucleus admission: experts are added in decreasing router weight until cumulative mass exceeds a threshold tau, with a disagreement-based extension, and tau is calibrated so that the average number of active experts matches a target budget. The same router signals (concentration and expert disagreement) are used as an uncertainty score for OOD detection and abstention. The paper claims that CARE improves accuracy over fixed top-k at matched compute (e.g., +0.5 on LLaMA commonsense and +0.9 on math/code/knowledge), matches fixed-k=4 accuracy with 12% fewer experts, and improves OOD-AUROC from 0.640 to 0.668 in a single forward pass. Theoretical support is provided via four propositions: nucleus fidelity, confidence ranking, budget optimality, and epistemic disagreement.

Significance. The contribution is potentially significant: if the empirical claims hold, CARE is a zero-retraining, single-pass improvement applicable to any MoE-LoRA model, and it yields a useful uncertainty signal as a by-product. The router-as-uncertainty idea is simple and well motivated, and the experimental suite spans two backbones and four task families. The reported gains are small but consistent in direction. However, the strength of the claims is undercut by an incomplete budget-optimality proof and by a lack of statistical detail in the empirical comparisons; these are fixable, so the paper merits revision rather than rejection.

major comments (3)
  1. [A.3 / Prop. 3] The proof of budget optimality does not establish that the nucleus threshold rule of Eq. (3) implements the optimal allocation. The greedy marginal-gain argument for a separable concave objective is standard, but the final step asserts that a threshold on marginal gain corresponds to a single threshold on cumulative mass C_k. This requires a cross-token homogeneity condition: the marginal gain of the last admitted expert must be a function of C_k that is comparable across tokens. The manuscript states this only as a modeling assumption and provides no direct evidence; Fig. 4(c) plots marginal gain against expert disagreement, not against C_k. Without a precise sufficient condition or a direct empirical test, the advertised 'budget optimality' is not supported. Please either prove a concrete condition under which the equivalence holds, add a test of monotonicity in C_k, or weaken the clai
  2. [Tables 1-4 / App. B] All headline numbers are three-seed means with no confidence intervals, error bars, or per-seed results. The key differences are small (+0.5 on LLaMA commonsense, +0.9 on math/code/knowledge) and the text describes the top fixed-k baselines as a 'tight band.' Without variance estimates it is impossible to judge whether the improvements are statistically distinguishable from noise. Since the central empirical claim is that CARE improves accuracy at matched compute, this is load-bearing. Please report per-seed values or error bars, and state whether the differences are significant.
  3. [§6.5 / Table 3] The OOD split is described only as 'a harder-and-flatter distribution shift of the same domain,' which is too vague to reproduce or to rule out selection bias. Moreover, the 'MC-dropout (proxy)' and 'Deep ensemble (proxy)' rows report AUROC 0.636/0.638, below the simple MSP baseline of 0.640. This is anomalous and suggests the proxy implementations are not faithful to the methods they approximate, undermining the claim that CARE beats multi-pass uncertainty methods. Specify exactly how the OOD split is constructed and how the proxies are computed, or remove the comparison to multi-pass methods.
minor comments (4)
  1. [Abstract / §4] CARE is described as 'no extra parameters' and 'parameter-free,' but Algorithm 1 depends on hyperparameters gamma, delta, w, kmin, kmax, and the calibrated tau*. These are not learned parameters, but the phrasing is overstated and could mislead readers about tuning burden.
  2. [Figure 4(b)] The x-axis label 'token difficulty' is not defined. It should be made explicit how difficulty is measured (e.g., empirical error rate, router entropy, or something else).
  3. [Table 2] The column header 'SV AMP' should be 'SVAMP'.
  4. [Eq. (6)] The notation for the sequence-level uncertainty score is ambiguous: H(p) and D are introduced as per-token quantities, but the equation appears to define a sequence-level average. Clarify the averaging.

Circularity Check

1 steps flagged

Empirical claims are externally benchmarked and non-circular, but the theoretical budget-optimality proof partly reduces to its own monotonicity assumption.

specific steps
  1. other [§5 Prop. 3 and App. A.3 (proof)]
    "If the marginal gain ∆gj(k) is monotone in the covered routing mass Ck(pj) (the modeling assumption that extra experts help in proportion to the mass they add), then the threshold on marginal gain corresponds to a single threshold τ(λ) on cumulative mass, which is exactly the nucleus rule of Eq. (3) with the thermostat τ⋆ of Eq. (5)."

    Proposition 3 is presented as proving that CARE's single-threshold nucleus rule is budget-optimal. The proof's final step does not derive this from concavity; it assumes that a threshold on cumulative mass C_k is equivalent to a threshold on marginal gain. That equivalence is precisely the property needed to make Eq. (3)/Eq. (5) optimal. A single global τ equalizes marginal gains only if marginal gain is the same monotone function of C_k across tokens, and the paper provides no direct evidence for this—Figure 4(c) plots marginal gain against disagreement, not C_k. The advertised 'budget optimality' is therefore encoded in the 'monotone in covered mass' assumption rather than established from the stated concavity premise.

full rationale

The central empirical claim—CARE improves accuracy at matched average compute and matches fixed-k=4 with 12% fewer experts—is evaluated against external fixed top-k and other MoE-LoRA baselines (Tables 1–3, Figure 3), so it is not circular. The budget thermostat is calibrated to a target average expert count by construction, but the accuracy-compute frontier is then measured, not defined. The self-citation to FRAME (Saliencro et al., 2026) is merely a related-work mention and is not load-bearing. Proposition 1 (nucleus fidelity) and Proposition 4 (disagreement as ensemble variance) are straightforward derivations from definitions. Proposition 2 is explicitly conditional ('If the correctness likelihood ratio is monotone...'), and while its conclusion is a restatement of the assumption, it is presented as a conditional theorem rather than a hidden premise. The one genuine circularity is in Proposition 3: the proof's final step assumes the very monotone relationship between marginal gain and cumulative mass that would make a single cumulative-mass threshold implement the optimal greedy allocation; without that unstated cross-token equivalence, the theorem does not follow. This partially undermines the theoretical support for budget optimality but leaves the externally benchmarked accuracy and OOD results intact.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

CARE introduces no learned parameters, but its behavior is controlled by hand-set hyperparameters (tau*, gamma, delta, w, kmax) and by three modeling assumptions that do much of the theoretical work. The empirical accuracy gain has independent benchmark support; the optimality and confidence-ranking proofs are conditional on assumptions that resemble the design principle itself.

free parameters (5)
  • tau* (budget thermostat threshold) = ~0.8 for B=4 (Table 5: tau=0.8 gives 3.96 avg experts)
    Calibrated once by bisection on a held-out set so the average expert count equals the target budget B=4. All main results and the accuracy-compute frontier depend on this calibration.
  • gamma (disagreement extension budget) = 2
    Hand-set; controls how many extra experts are admitted when disagreement exceeds delta. No tuning curve is shown for gamma.
  • delta (disagreement threshold) = 0.55
    Hand-set; the epistemic extension activates only above this disagreement value. No sensitivity analysis is provided.
  • w (uncertainty blend weight) = 0.5
    Blends routing entropy and disagreement in Eq. (6). Appendix C Table 8 shows the empirical optimum is closer to 0.75, so 0.5 is a chosen default rather than a derived value.
  • kmax (max active experts cap) = 8
    Caps the per-token expert count. Table 7 shows accuracy saturates by kmax=6, so the choice of 8 shapes hard-token allocation.
axioms (4)
  • domain assumption Correctness probability eta(h) is non-decreasing in top-1 router mass p(1) (Prop. 2's monotone-likelihood assumption).
    Needed for the claim that routing concentration ranks confidence optimally. Only supported by a pooled density plot (Fig. 4a), not by a per-domain or statistical test.
  • ad hoc to paper Per-token accuracy gain g_j(k) is non-decreasing and concave in k, and marginal gain is monotone in cumulative routing mass C_k (Prop. 3).
    This monotonicity-in-mass assumption essentially stipulates that the largest-weight expert order is the right order for budget allocation. The paper provides no direct evidence that marginal accuracy gain is monotone in C_k.
  • domain assumption Disagreement D(h;S) is an estimator of epistemic (reducible) uncertainty.
    Prop. 4 proves only that D equals the ensemble predictive variance by definition; calling that variance 'epistemic' and treating more experts as reducing it is an interpretive assumption, not a derivation.
  • domain assumption The held-out set used to calibrate tau* has router statistics representative of the test and shifted splits.
    The thermostat is calibrated once on a held-out split. Under distribution shift the same tau* may no longer deliver the target average budget; the §6.7 shifted-split results assume this calibration transfers.

pith-pipeline@v1.3.0-alltime-deepseek · 15003 in / 14002 out tokens · 343292 ms · 2026-08-04T03:15:49.684025+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA." pith.science (2026). https://pith.science/paper/MVVSYM2A

@misc{pith2026260726052,
  author       = {Pith},
  title        = {Pith review of: Spend Experts Where You Are Unsure: Confidence-Adaptive Routing for Mixture-of-Experts LoRA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MVVSYM2A}},
  note         = {Machine review of arXiv:2607.26052}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Mixture-of-Experts (MoE) variants of Low-Rank Adaptation (LoRA) route every token to a fixed number of experts $k$. Tokens differ in how uncertain the model is about them, so a single k over-spends on easy tokens and under-serves hard ones. We observe that the router's output distribution is already a per-token uncertainty signal: peaked mass indicates confidence, while a flat distribution indicates ambiguity. We introduce CARE (Confidence-Adaptive Routing of Experts), which admits experts in a nucleus fashion. Experts are activated in decreasing router weight until their cumulative mass reaches a threshold, with a small extension when the admitted experts disagree. A budget thermostat calibrates the threshold so that the average number of active experts matches any target. CARE is a drop-in, single-forward-pass rule with no extra parameters. Across eight commonsense benchmarks on LLaMA-3.1-8B and Qwen2.5-7B, as well as math, code, and knowledge tasks, CARE improves over fixed top-k MoE-LoRA at matched compute and matches the fixed-k=4 baseline while activating fewer experts. The same confidence and disagreement signals also improve out-of-distribution detection over MSP, entropy, and multi-pass proxies. We support the design with nucleus fidelity, budget optimality, and an epistemic reading of disagreement, and we release code.

Figures

Figures reproduced from arXiv: 2607.26052 by Daniel Whitmore, Maya Lindqvist, Priya Nair, Rohan Desai, Tom Saliencro.

Figure 1
Figure 1. Figure 1: Fixed top-k spends the same budget on every token; CARE spends it where uncertainty de￾mands. Each row is a token and each square an expert slot. Left: every token activates k=4 experts, over￾spending on easy (green) tokens and under-serving hard (orange) ones. Right: CARE reallocates the same aver￾age budget per token according to uncertainty read off the router. 2024; Zou et al., 2025b). These MoE-LoRA m… view at source ↗
Figure 2
Figure 2. Figure 2: CARE pipeline. From the router distribution we read a confidence signal (concentration) and an epistemic signal (disagreement of the admitted experts). A single budget-calibrated threshold turns them into a per-token expert count k(h), activating exactly that many experts in one forward pass with no added parameters. apply the epistemic extension: if the admitted ex￾perts disagree beyond a level δ, admit u… view at source ↗
Figure 3
Figure 3. Figure 3: Accuracy–compute trade-off (pooled). CARE dominates the fixed top-k frontier; it matches fixed-k=4 accuracy using 12% fewer active experts (ar￾row). in the mid-budget regime, where reallocating ex￾perts across heterogeneous tokens matters most. 6.4 Mechanism analysis [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Why CARE works. (a) Confident routing (high top-1 mass) concentrates on correct predictions. (b) CARE spends more experts on harder tokens, crossing the fixed-k=4 line. (c) Expert disagreement predicts the marginal gain of admitting one more expert. Method ECE OOD-AUROC ↑ LoRA + MSP 0.267 0.640 HydraLoRA + MSP 0.173 0.640 MixLoRA + MSP 0.173 0.640 DynMoLE + entropy 0.174 0.651 FlyLoRA + MSP 0.175 0.640 Fly… view at source ↗
Figure 5
Figure 5. Figure 5: Reliability diagram (pooled), raw scores. Both methods sit off the diagonal before recalibration; CARE’s primary UQ gain is OOD detection ( [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Component ablation at matched budget ¯k=4. (a) The confidence/nucleus term drives the accuracy gain. (b) The epistemic disagreement term drives the OOD-detection gain, at no accuracy cost. Variant Acc. OOD-AUROC ECE Fixed-k (=4) 52.8 0.640 0.175 Random-k 52.1 0.640 0.171 Entropy routing 53.7 0.651 0.192 Nucleus only 53.7 0.640 0.203 CARE (full) 53.7 0.668 0.186 [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 15 linked inside Pith

  1. [1]

    and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=

    Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=. 2022 , eprint=

    2022

  2. [2]

    Utility-Diversity Aware Online Batch Selection for

    Zou, Heming and Mao, Yixiu and Qu, Yun and Wang, Qi and Ji, Xiangyang , journal=. Utility-Diversity Aware Online Batch Selection for. 2025 , url=

    2025

  3. [3]

    2023 , eprint=

    Zhang, Qingru and Chen, Minshuo and Bukharin, Alexander and Karampatziakis, Nikos and He, Pengcheng and Cheng, Yu and Chen, Weizhu and Zhao, Tuo , booktitle=. 2023 , eprint=

    2023

  4. [4]

    International Conference on Learning Representations (ICLR) , year=

    Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer , author=. International Conference on Learning Representations (ICLR) , year=. 1701.06538 , archivePrefix=

    Pith/arXiv arXiv

  5. [5]

    Journal of Machine Learning Research , volume=

    Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity , author=. Journal of Machine Learning Research , volume=. 2022 , eprint=

    2022

  6. [6]

    Gao, Chongyang and Chen, Kezhen and Rao, Jinmeng and Sun, Baochen and Liu, Ruibo and Peng, Daiyi and Zhang, Yawen and Guo, Xiaoyuan and Yang, Jie and Subrahmanian, V. S. , journal=. Higher Layers Need More. 2024 , url=

    2024

  7. [7]

    2024 , eprint=

    Dou, Shihan and Zhou, Enyu and Liu, Yan and Gao, Songyang and Zhao, Jun and Shen, Wei and Zhou, Yuhao and Xi, Zhiheng and Wang, Xiao and Fan, Xiaoran and Pu, Shiliang and Zhu, Jiang and Zheng, Rui and Gui, Tao and Zhang, Qi and Huang, Xuanjing , booktitle=. 2024 , eprint=

    2024

  8. [8]

    2024 , url=

    Li, Dengchun and Ma, Yingzi and Wang, Naizheng and Ye, Zhengmao and Cheng, Zhiyuan and Tang, Yinghao and Zhang, Yan and Duan, Lei and Zuo, Jie and Yang, Cal and Tang, Mingjie , journal=. 2024 , url=

    2024

  9. [9]

    International Conference on Learning Representations (ICLR) , year=

    Energy-based Automated Model Evaluation , author=. International Conference on Learning Representations (ICLR) , year=. 2401.12689 , archivePrefix=

    Pith/arXiv arXiv

  10. [10]

    2025 , url=

    Li, Dengchun and Wang, Naizheng and Zhang, Zihao and Yin, Haoyang and Duan, Lei and Xiao, Meng and Tang, Mingjie , journal=. 2025 , url=

    2025

  11. [11]

    2026 , eprint=

    Saliencro, Tom and Lindqvist, Maya and Desai, Rohan and Nair, Priya and Whitmore, Daniel , journal=. 2026 , eprint=

    2026

  12. [12]

    Functional-level Uncertainty Quantification for Calibrated Fine-tuning on

    Niu, Ruijia and Wu, Dongxia and Yu, Rose and Ma, Yi-An , journal=. Functional-level Uncertainty Quantification for Calibrated Fine-tuning on. 2024 , url=

    2024

  13. [13]

    2023 , url=

    Wang, Xi and Aitchison, Laurence and Rudolph, Maja , journal=. 2023 , url=

    2023

  14. [14]

    Transactions on Machine Learning Research , year=

    M. Transactions on Machine Learning Research , year=. 2405.14438 , archivePrefix=

    arXiv

  15. [15]

    International Conference on Learning Representations (ICLR) , year=

    Bayesian Low-rank Adaptation for Large Language Models , author=. International Conference on Learning Representations (ICLR) , year=. 2308.13111 , archivePrefix=

    Pith/arXiv arXiv

  16. [16]

    arXiv preprint arXiv:2511.08968 , year=

    Bayesian Mixture of Experts for Large Language Models , author=. arXiv preprint arXiv:2511.08968 , year=

    arXiv

  17. [17]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Energy-based Out-of-distribution Detection , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=. 2010.03759 , archivePrefix=

    Pith/arXiv arXiv 2010

  18. [18]

    Proceedings of the 34th International Conference on Machine Learning (ICML) , year=

    On Calibration of Modern Neural Networks , author=. Proceedings of the 34th International Conference on Machine Learning (ICML) , year=. 1706.04599 , archivePrefix=

    Pith/arXiv arXiv

  19. [19]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=. 1612.01474 , archivePrefix=

    Pith/arXiv arXiv

  20. [20]

    Proceedings of the 33rd International Conference on Machine Learning (ICML) , year=

    Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , author=. Proceedings of the 33rd International Conference on Machine Learning (ICML) , year=. 1506.02142 , archivePrefix=

    Pith/arXiv arXiv

  21. [21]

    International Conference on Learning Representations (ICLR) , year=

    Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation , author=. International Conference on Learning Representations (ICLR) , year=. 2302.09664 , archivePrefix=

    Pith/arXiv arXiv

  22. [22]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Selective Classification for Deep Neural Networks , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=. 1705.08500 , archivePrefix=

    Pith/arXiv arXiv

  23. [23]

    International Conference on Learning Representations (ICLR) , year=

    The Curious Case of Neural Text Degeneration , author=. International Conference on Learning Representations (ICLR) , year=. 1904.09751 , archivePrefix=

    Pith/arXiv arXiv 1904

  24. [24]

    International Conference on Learning Representations (ICLR) , year=

    Measuring Massive Multitask Language Understanding , author=. International Conference on Learning Representations (ICLR) , year=. 2009.03300 , archivePrefix=

    Pith/arXiv arXiv 2009

  25. [25]

    arXiv preprint arXiv:2110.14168 , year=

    Training Verifiers to Solve Math Word Problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2107.03374 , year=

    Evaluating Large Language Models Trained on Code , author=. arXiv preprint arXiv:2107.03374 , year=

    Pith/arXiv arXiv

  27. [27]

    Think you have Solved Question Answering? Try

    Clark, Peter and Cowhey, Isaac and Etzioni, Oren and Khot, Tushar and Sabharwal, Ashish and Schoenick, Carissa and Tafjord, Oyvind , journal=. Think you have Solved Question Answering? Try. 2018 , url=

    2018

  28. [28]

    2022 , eprint=

    Lin, Stephanie and Hilton, Jacob and Evans, Owain , booktitle=. 2022 , eprint=

    2022

  29. [29]

    Grattafiori, Aaron and Dubey, Abhimanyu and Jauhri, Abhinav and Pandey, Abhinav and Kadian, Abhishek and Al-Dahle, Ahmad and Letman, Aiesha and Mathur, Akhil and Schelten, Alan and Vaughan, Alex and others , journal=. The

  30. [30]

    Yang, An and Yang, Baosong and Zhang, Beichen and Hui, Binyuan and Zheng, Bo and Yu, Bowen and Li, Chengyuan and Liu, Dayiheng and Huang, Fei and Wei, Haoran and others , journal=

  31. [31]

    2024 , eprint=

    Liu, Shih-Yang and Wang, Chien-Yi and Yin, Hongxu and Molchanov, Pavlo and Wang, Yu-Chiang Frank and Cheng, Kwang-Ting and Chen, Min-Hung , booktitle=. 2024 , eprint=

    2024

  32. [32]

    2024 , eprint=

    Tian, Chunlin and Shi, Zhan and Guo, Zhijiang and Li, Li and Xu, Cheng-Zhong , booktitle=. 2024 , eprint=

    2024

  33. [33]

    Clark, Christopher and Lee, Kenton and Chang, Ming-Wei and Kwiatkowski, Tom and Collins, Michael and Toutanova, Kristina , booktitle=

  34. [34]

    Bisk, Yonatan and Zellers, Rowan and Gao, Jianfeng and Choi, Yejin , booktitle=

  35. [35]

    Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin , booktitle=

  36. [36]

    Sakaguchi, Keisuke and Le Bras, Ronan and Bhagavatula, Chandra and Choi, Yejin , booktitle=

  37. [37]

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    Can a suit of armor conduct electricity? A new dataset for open book question answering , author=. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages=

    2018

  38. [38]

    Measuring mathematical problem solving with the

    Hendrycks, Dan and Burns, Collin and Kadavath, Saurav and Arora, Akul and Basart, Steven and Tang, Eric and Song, Dawn and Steinhardt, Jacob , journal=. Measuring mathematical problem solving with the

  39. [39]

    arXiv preprint arXiv:2108.07732 , year=

    Program synthesis with large language models , author=. arXiv preprint arXiv:2108.07732 , year=

    Pith/arXiv arXiv

  40. [40]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=. 2209.09513 , archivePrefix=

    Pith/arXiv arXiv

  41. [41]

    Sap, Maarten and Rashkin, Hannah and Chen, Derek and Le Bras, Ronan and Choi, Yejin , booktitle=. Social

  42. [42]

    Patel, Arkil and Bhattamishra, Satwik and Goyal, Navin , journal=. Are

  43. [43]

    2025 , eprint=

    Zou, Heming and Zang, Yunliang and Xu, Wutong and Zhu, Yao and Ji, Xiangyang , booktitle=. 2025 , eprint=

    2025

  44. [44]

    Koncel-Kedziorski, Rik and Roy, Subhro and Amini, Aida and Kushman, Nate and Hajishirzi, Hannaneh , booktitle=

  45. [45]

    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year=

    Program induction by rationale generation: Learning to solve and explain algebraic word problems , author=. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year=

  46. [46]

    2025 , url=

    Liao, Mengqi and Chen, Wei and Shen, Junfeng and Guo, Shengnan and Wan, Huaiyu , booktitle=. 2025 , url=

    2025

  47. [47]

    Parameter-Efficient Fine-Tuning with Discrete

    Gao, Ziqi and Wang, Qichao and Chen, Aochuan and Liu, Zijing and Wu, Bingzhe and Chen, Liang and Li, Jia , booktitle=. Parameter-Efficient Fine-Tuning with Discrete. 2024 , eprint=

    2024