Pith. sign in

REVIEW 5 major objections 6 minor 81 references

The central claim is that higher-layer expert activation patterns in MoE language models differ between factual and non-factual outputs across all MoE architectures, and a contrastive decoding method built on this pattern reduces hallucinat

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 14:31 UTC pith:RJLQ544Z

load-bearing objection New MoE decoding method with real accuracy gains but a weakly supported mechanism; deserves peer review with revisions. the 5 major comments →

arxiv 2607.20426 v1 pith:RJLQ544Z submitted 2026-05-08 cs.CL cs.AI

Knowledge Injection Exists in MoE? Exploring Expert-Aware Contrast Decoding in MoE for Mitigating LLMs'Hallucinations

Xinyue Fang , Zhiliang Tian , Zhen Huang , Ziyi Pan , Zhihua Wen , Xi Wang , Quntian Fang , Dongsheng Li This is my paper
classification cs.CL cs.AI
keywords mixture-of-expertshallucination mitigationcontrastive decodingknowledge injectionexpert activation patternslarge language modelsquestion answering
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Contrastive decoding for LLM hallucinations has relied on the 'knowledge injection' pattern in dense transformers, where higher layers progressively sharpen factual predictions. This paper asks whether that pattern exists in mixture-of-experts models and finds it is architecture-dependent: shared-expert MoEs show almost no layer-wise prediction change, while MoEs without shared experts show a sharp late-layer shift. The paper then identifies a broader signal: in every MoE family tested, higher-layer expert activation patterns differ measurably between factual and non-factual outputs. EAACD builds on this by clustering final-layer experts, ranking them by confidence and consistency, amplifying hallucinations in lower-ranked experts via attention-masked inputs, and adaptively contrasting the top group against those amplified references to recalibrate the model's original predictions. The method reports higher accuracy than all compared decoding baselines on four QA/factuality datasets for both MoE architectures.

Core claim

Across all tested MoE architectures, higher-layer expert activation patterns differ measurably between factual and non-factual outputs, while the layer-wise 'knowledge injection' seen in dense transformers appears only in MoEs without shared experts. EAACD clusters final-layer experts by prediction similarity, ranks clusters by entropy/L2-norm confidence and consistency, amplifies lower-ranked experts' hallucinations via attention-masked prompts, and subtracts them from the top cluster's prediction with KL-based adaptive penalties and entropy-weighted interpolation. It outperforms all compared baselines on FACTOR, HellaSwag, StrategyQA, and MathQA for both a no-shared-expert and a shared-exp

What carries the argument

Expert activation divergence between factual and non-factual outputs, measured as L2 distance on per-layer expert activation frequency vectors, is the signal EAACD uses. The carrying mechanism is a reliability-weighted expert contrast: agglomerative clustering of final-layer expert logits; a reliability score per group combining entropy, logit L2 norm, and cluster-size consistency; attention-guided masking of salient tokens to amplify low-reliability hallucinations; and adaptive KL-divergence penalties per low-reliability group, blended into the original prediction with an entropy-based interpolation weight.

Load-bearing premise

That a single final-layer grouping, scored by confidence and group-size consistency, reliably identifies experts whose next-token predictions are more factual, and that masking salient context tokens reliably makes the complementary groups more hallucinated, without needing labeled factual examples at inference; the paper validates this only indirectly via ablations and one probability analysis.

What would settle it

Run the expert-partitioning module alone on a labeled QA set and compare next-token accuracy of the higher-reliability group against each lower-reliability group; if the higher group is not more accurate, the contrast can remove correct information. Also, measure whether masking the highest-attention tokens lowers the lower groups' accuracy relative to the unmasked prompt; if not, the negative references are not actually hallucination-amplified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If higher-layer expert activation differences track factuality, then MoE hallucination mitigation can be done at decoding time from the model's own internals, without external knowledge bases or fine-tuning.
  • The absence of the 'knowledge injection' layer progression in shared-expert MoEs explains why layer-contrast decoding baselines underperform there, and indicates that expert-level contrasts are the right unit for these models.
  • Grouping experts by reliability—rather than by router weight alone—and amplifying low-reliability hallucinations before contrast improves accuracy across datasets, supporting the two-step design.
  • The method's adaptive penalties and entropy-based calibration keep it applicable to both no-shared-expert and shared-expert MoE families with a single recipe.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If expert activation patterns are reliable proxies for factuality, router statistics alone might serve as a hallucination detector for MoE models, flagging uncertain generations before output.
  • Because the amplification step masks the most-attended tokens, the method implies that factual MoE behavior depends on those tokens routing to the reliable experts; a testable extension is whether pruning those tokens from deployment inputs reproduces the hallucination pattern.
  • The architecture-dependent knowledge-injection finding suggests design choices about shared experts shift where factual knowledge resides; that could inform pretraining decisions for future MoE architectures.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper asks whether Mixture-of-Experts (MoE) LLMs exhibit the 'knowledge injection' phenomenon used by layer-wise contrastive decoding, and whether expert-level differences can support hallucination mitigation. The authors report that the Jensen–Shannon divergence between lower- and final-layer next-token distributions drops sharply in MoE without shared experts (LLaMA-MoE, Mixtral) but stays flat in MoE with shared experts (DeepSeek-MoE, Qwen-MoE), and that higher-layer router activation patterns differ between factual and non-factual outputs. They then propose EAACD, which clusters final-layer expert logits, scores groups by confidence (entropy, L2 norm) and size-based consistency, selects one high-reliability group, masks salient context tokens to amplify low-reliability predictions, and contrasts the high-reliability prediction against the amplified low-reliability predictions with KL-derived weights and entropy-based calibration. On FACTOR subsets, HellaSwag, StrategyQA, and MathQA, EAACD outperforms greedy, CD, DoLa, SCMoE, END, and Self-Endorsement on LLaMA-MoE and Qwen-MoE.

Significance. If the central premise holds—that the confidence/consistency grouping separates factual from hallucination-prone experts—EAACD would be a useful training-free hallucination mitigation method for MoE models and one of the first to exploit expert diversity for contrastive decoding. The paper includes several good practices: a public code link, a control experiment in Appendix K intended to rule out depth/scale confounds for the shared-expert finding, a case study, and ablations of the main components. However, the direct evidence for the factuality–expert link is thin, the evaluation is narrow, and the method's connection to the router-activation analysis is not established. These issues are substantial but appear fixable with additional analysis and experiments.

major comments (5)
  1. [§3.3 and §4.1] The empirical motivation and the proposed method operate on different objects. RQ2 (Figs. 3–4) measures expert activation patterns—router activation frequencies—and concludes that higher layers show divergent expert activation between factual and non-factual outputs. EAACD, however, does not use router activations; it clusters experts by final-layer logits (§4.1.1) and scores them by entropy/L2 norm and group size (§4.1.2). No result in the paper connects the two: an expert that is activated differently under factual versus non-factual conditions is never shown to be the same expert that lands in the higher- or lower-reliability group. The central claim that EAACD exploits the §3.3 finding is therefore not directly supported. Please add a direct analysis (e.g., comparing router-activation changes with group membership, or a version of EAACD conditioned on router statistics).
  2. [§5.4, Fig. 6] The only direct evidence that higher-reliability expert groups favor factual answers is Fig. 6. The mean probability differences between correct and incorrect answers are on the order of 1e-5 to 1e-6, computed without error bars, confidence intervals, or significance tests, and the metric is not top-1 accuracy. Such probability-mass differences are within the noise of a softmax over a large vocabulary. The Appendix H case study (Table 5) is a single example. The ablations in Table 2 show that removing the confidence/consistency components hurts accuracy, but that only establishes a contribution to the objective, not that the grouping separates factual from non-factual experts. This is load-bearing because the 'knowledge injection'/factual-expert interpretation depends on it. Please provide a direct factual-accuracy measurement of the groups (e.g., top-1 accuracy on labeled FACTOR example
  3. [§5.1, Table 1] The main evaluation only covers LLaMA-MoE and Qwen-MoE; Mixtral and DeepSeek-MoE appear in the empirical study but not in the performance table. The abstract and conclusion claim effectiveness 'across different MoEs' and 'all MoE architectures,' but no EAACD result is reported for Mixtral, DeepSeek, or any shared-expert model besides Qwen. In addition, Table 1 reports only accuracy with no standard errors or repeated runs, and Table 3's paired t-tests are aggregated and do not show per-dataset or per-model breakdowns. Please add EAACD results on at least Mixtral and DeepSeek (or a justified subset) and report variance.
  4. [§4.1.2, Eq. (1)–(2)] The reliability score T_j = mean confidence + |c_j|/(k m) adds a term that is purely group size. For any fixed m, the largest cluster receives an automatic reliability boost independent of prediction consistency; the term is better called a size prior than a consistency score. If the largest cluster is typically the one with the most 'default' predictions, this could bias the high-reliability group selection in a way that is not tied to factuality. Please justify or replace this term, and report the sensitivity of the results to the cluster count m, which is not specified in the paper (nor are the clustering distance/linkage choices in §4.1.1).
  5. [§3.3 and App. B] Non-factual outputs in RQ2 are generated using a malicious system prompt ('generate incorrect answers'). Under this design, the difference between factual and non-factual expert activation patterns may reflect compliance with an instruction to be incorrect rather than factuality per se. The same confound carries into the motivation for EAACD. A less instruction-bound way to induce non-factual outputs (e.g., entity substitution, adversarial distractors, or known hallucination triggers without an explicit 'be incorrect' instruction) would make the finding more convincing.
minor comments (6)
  1. [App. I] The β-selection text appears logically reversed: 'If β is too large, an excessive number of tokens are masked... Conversely, if β is too small, only a few tokens are masked.' Since tokens are masked when importance exceeds β, a larger β masks fewer tokens. Please correct.
  2. [Eq. (2)–(3)] The display of Eq. (2) is garbled (the D_KL notation and softmax arguments are difficult to parse), and Eq. (3) writes p′ = p_r − p_l although p_r and p_l are logits. Clarify the distinction between logits and probabilities throughout §4.3.
  3. [§4.1.1] The number of clusters m is not specified, and no sensitivity analysis is given. Since the reliability grouping depends on m, please report the value(s) used and, ideally, a small robustness check.
  4. [Appendix K] The OpenMoE control experiment is a useful check, but it is presented only on a single input sample. Dataset-level averages, as in Fig. 22, would strengthen the claim that the shared-expert effect is not due to model depth or scale.
  5. [§7 Limitations] The Limitations section only discusses the possibility that MoE architectures may become non-mainstream. It does not acknowledge the lack of direct expert-factuality validation, the instruction-following confound in RQ2, or the limited model coverage of the main evaluation. These are more immediate limitations of the current work.
  6. [Fig. 6] The figure legend uses colors and line markers that may be hard to distinguish in grayscale; please add clearer labels or hatching. Also indicate the number of samples used to compute the mean probability differences.

Circularity Check

0 steps flagged

No significant circularity: EAACD's reliability grouping is not fitted to test labels and the contrastive decoding is a genuine compositional manipulation.

full rationale

The paper's central chain is: (i) observe JSD layer trajectories and expert-activation L2 differences; (ii) define an unsupervised reliability score T_j = mean expert confidence + group consistency, where confidence is entropy/L2 based and is not a function of the correctness labels; (iii) cluster experts by prediction similarity; (iv) amplify lower-reliability group hallucinations by attention masking; (v) contrast p_r and p_l and calibrate p_o via an entropy weight. None of these steps fits a parameter to the test labels or defines the target quantity in terms of itself. The reliability score is a proxy (confidence/consistency), and the paper's Fig. 6 is an external check of that proxy against FACTOR labels rather than a derivation of the labels from the proxy. The ablation table shows components contribute, but that is an empirical claim, not a tautology. Self-citations (Fang et al. 2025c; Fang et al. 2025a,b; Pan et al. 2026; Sun et al. 2025) appear only as background or societal-impact citations and do not carry the load-bearing argument. Weaknesses such as the tiny Fig. 6 probability differences, the prompt-induced 'non-factual' condition in §3.3, and the factually incorrect Table 7 case are validity/correctness concerns, not evidence that the derivation reduces to its inputs. The method would be circular only if high-reliability status were assigned using the same correct-answer probabilities it is then used to predict; the paper assigns it with entropy/L2/consistency, independent of labels. The central claim therefore has independent empirical content, and the self-citations are not load-bearing.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No new physical or architectural entities are postulated. The main load-bearing assumptions are about reliability signals (entropy/L2, group consensus, attention masking) and the transfer of activation-pattern statistics to decoding decisions. These are domain assumptions rather than free parameters strictly, though the method contains several unspecified hyperparameters.

free parameters (4)
  • mask threshold β = not specified numerically
    Chosen via validation to balance hallucination amplification and semantic preservation (§4.2, App. I); no exact value or selection protocol reported.
  • number of expert clusters m = not specified
    Agglomerative clustering requires a stopping criterion or cluster count; not given in main text or appendix.
  • final-layer selection = final layer only
    Choice justified in App. G as a trade-off, but it is a modeling decision not derived.
  • reliability formula weights α = adaptive via σH/(σH+σL)
    The adaptive combination of entropy and L2 norm is a hand-designed heuristic; its standard deviations are computed per-step from experts, but the functional form is not derived.
axioms (4)
  • domain assumption Entropy and L2 norm of expert logits measure expert reliability.
    Stated in §4.1.2 and used to partition experts; no calibration against factual accuracy except the FACTOR analysis in §5.4.
  • domain assumption Higher layers' expert activation divergence between factual/non-factual outputs implies expert capability differences usable for decoding.
    Bridges RQ2 to the method; the paper does not prove that activation divergence is causally related to factual reliability.
  • domain assumption Masking high-attention context tokens amplifies hallucinations in lower-reliability experts.
    Argued with one case study (App. I) and ablation; no general validation across datasets.
  • domain assumption Agglomerative clustering of expert logits produces meaningful reliability groups.
    Used to define groups; no justification that cluster structure corresponds to factual vs non-factual behavior.

pith-pipeline@v1.3.0-alltime-deepseek · 101072 in / 6108 out tokens · 52597 ms · 2026-08-02T14:31:41.230295+00:00 · methodology

0 comments
read the original abstract

Existing LLM hallucination mitigation methods, including prompt engineering and model optimization, either hardly alter models'internal knowledge or have poor cross-domain generalization. Contrastive decoding mitigates hallucinations by using layer-wise differences in LLMs. However, prior studies only explore transformer-based models (e.g., GPT), ignoring other effective frameworks like mixture-of-experts (MoE) models. Since MoE alters the traditional transformer architecture, we conduct empirical studies to investigate whether similar layer-wise differences exist in MoEs. Our results show that they do not exist in MoE with shared experts; nevertheless, across different MoEs, higher layers exhibit distinct expert activation patterns between factual and non-factual outputs. Building on these, we propose EAACD, an expert-aware adaptive contrast decoding that uses expert differences in MoE's higher layers to mitigate hallucinations on QA tasks. EAACD splits high-layer experts into a higher-reliability group and several lower-reliability groups based on their confidence and consistency. It contrasts the higher-reliability group's prediction with each lower-reliability group's prediction to calibrate the model's original predictions. To strengthen this contrast, EAACD amplifies hallucinations from lower-reliability experts via attention and masking to provide stronger negative references. EAACD outperforms all baselines on four datasets.

Figures

Figures reproduced from arXiv: 2607.20426 by Dongsheng Li, Quntian Fang, Xinyue Fang, Xi Wang, Zhen Huang, Zhihua Wen, Zhiliang Tian, Ziyi Pan.

Figure 1
Figure 1. Figure 1: Overview of prior intra-model contrast decoding and EAACD (ours). Prior intra-model contrast decoding [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: JSD (scaled by 105 ) between the final layer and each lower layer across different models. Column labels indicate tokens in each step. Row labels indicate layer indices. The input sample is from TruthfulQA dataset; see App. A for results on other datasets. 3.1 Experimental setting We investigate two mainstream MoE architectures: LLaMA-MoE (Zhu et al., 2024) and Mixtral (Jiang et al., 2024) for MoE without … view at source ↗
Figure 4
Figure 4. Figure 4: Differences in expert activation patterns be [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Overview of EAACD. Given an MoE input (left), we cluster final-layer expert predictions and evaluate [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Mean probability difference and mean re [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 9
Figure 9. Figure 9: JSD (scaled by 105 ) between the final layer and each lower layer across Mixtral. Column labels indicate tokens in each step. Row labels indicate layer indices. The input sample is from GSM8K dataset. The candle burns for 5 - 1 = << 51 -1 11=1 4>> 41 hours . Thus , the candle1willbe 2 * 3 42=2<<1 21 *1 43=3 8>>1 81 cent imeters shorter .1 1 #### 82 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22… view at source ↗
Figure 8
Figure 8. Figure 8: JSD (scaled by 105 ) between the final layer and each lower layer across LLaMA MoE. Column labels indicate tokens in each step. Row labels indicate layer indices. The input sample is from GSM8K dataset. B Malicious System Prompt in RQ2 We use this malicious system prompt to steer the model into generating false outputs: You are an AI model specifically trained to generate incorrect answers. Be misleading i… view at source ↗
Figure 13
Figure 13. Figure 13: Expert activation patterns across layers in [PITH_FULL_IMAGE:figures/full_fig_p013_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Expert activation patterns across layers in [PITH_FULL_IMAGE:figures/full_fig_p013_14.png] view at source ↗
Figure 12
Figure 12. Figure 12: Expert activation patterns across layers in [PITH_FULL_IMAGE:figures/full_fig_p013_12.png] view at source ↗
Figure 16
Figure 16. Figure 16: Expert activation patterns across layers in [PITH_FULL_IMAGE:figures/full_fig_p014_16.png] view at source ↗
Figure 20
Figure 20. Figure 20: Expert activation patterns across layers in [PITH_FULL_IMAGE:figures/full_fig_p014_20.png] view at source ↗
Figure 18
Figure 18. Figure 18: Expert activation patterns across layers in [PITH_FULL_IMAGE:figures/full_fig_p014_18.png] view at source ↗
Figure 22
Figure 22. Figure 22: Layer-wise average JSD across different datasets for various MoE models. [PITH_FULL_IMAGE:figures/full_fig_p016_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: JSD (scaled by 105 ) between the final layer and each lower layer in Mixtral. Column labels indicate tokens in each step. Row labels indicate layer indices. The input sample is What color is the sun when viewed from space? The sun is all colors mixed together . 0 4 8 12 16 20 24 28 Layer 3.25 4.14 3.80 3.47 2.84 3.11 4.18 3.41 3.25 4.12 3.82 3.44 2.84 3.09 4.21 3.37 3.23 4.14 3.80 3.46 2.82 3.10 4.19 3.40… view at source ↗
Figure 24
Figure 24. Figure 24: JSD (scaled by 105 ) between the final layer and each lower layer in OpenMoE. Column labels in￾dicate tokens in each step. Row labels indicate layer indices. The input sample is What color is the sun when viewed from space? [PITH_FULL_IMAGE:figures/full_fig_p017_24.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

81 extracted references · 13 linked inside Pith

  1. [1]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  2. [2]

    arXiv preprint arXiv:2210.09150 , year=

    Prompting gpt-3 to be reliable , author=. arXiv preprint arXiv:2210.09150 , year=

    Pith/arXiv arXiv

  3. [3]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  4. [4]

    Advances in neural information processing systems , volume=

    Tree of thoughts: Deliberate problem solving with large language models , author=. Advances in neural information processing systems , volume=

  5. [5]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Graph of thoughts: Solving elaborate problems with large language models , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  6. [6]

    arXiv preprint arXiv:2310.06271 , year=

    Towards mitigating hallucination in large language models via self-reflection , author=. arXiv preprint arXiv:2310.06271 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2212.10560 , year=

    Self-instruct: Aligning language models with self-generated instructions , author=. arXiv preprint arXiv:2212.10560 , year=

    Pith/arXiv arXiv

  8. [8]

    arXiv preprint arXiv:2501.17295 , year=

    Mitigating hallucinated translations in large language models with hallucination-focused preference optimization , author=. arXiv preprint arXiv:2501.17295 , year=

    Pith/arXiv arXiv

  9. [9]

    arXiv preprint arXiv:2212.12017 , year=

    Opt-iml: Scaling language model instruction meta learning through the lens of generalization , author=. arXiv preprint arXiv:2212.12017 , year=

    Pith/arXiv arXiv

  10. [10]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Hallucination mitigation in natural language generation from large-scale open-domain knowledge graphs , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  11. [11]

    arXiv preprint arXiv:2406.11267 , year=

    Mitigating large language model hallucination with faithful finetuning , author=. arXiv preprint arXiv:2406.11267 , year=

    Pith/arXiv arXiv

  12. [12]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Can We Edit Factual Knowledge by In-Context Learning? , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  13. [13]

    Advances in Neural Information Processing Systems , volume=

    Adversarial representation engineering: A general model editing framework for large language models , author=. Advances in Neural Information Processing Systems , volume=

  14. [14]

    arXiv preprint arXiv:2301.09785 , year=

    Transformer-patcher: One mistake worth one neuron , author=. arXiv preprint arXiv:2301.09785 , year=

    Pith/arXiv arXiv

  15. [15]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Editing Common Sense in Transformers , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  16. [16]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  17. [17]

    Advances in neural information processing systems , volume=

    Locating and editing factual associations in gpt , author=. Advances in neural information processing systems , volume=

  18. [18]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Contrastive Decoding: Open-ended Text Generation as Optimization , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  19. [19]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    Alleviating Hallucinations of Large Language Models through Induced Hallucinations , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  20. [20]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Improving factuality in large language models via decoding-time hallucinatory and truthful comparators , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  21. [21]

    Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) , pages=

    A Frustratingly Simple Decoding Method for Neural Text Generation , author=. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) , pages=

    2024

  22. [22]

    2024 IEEE International Conference on Smart Internet of Things (SmartIoT) , pages=

    Freeze-CD: Alleviating Hallucination of Large Language Models via Contrastive Decoding with Local Freezing Training , author=. 2024 IEEE International Conference on Smart Internet of Things (SmartIoT) , pages=. 2024 , organization=

    2024

  23. [23]

    The Twelfth International Conference on Learning Representations , year=

    DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models , author=. The Twelfth International Conference on Learning Representations , year=

  24. [24]

    Advances in Neural Information Processing Systems , volume=

    Unchosen experts can contribute too: Unleashing moe models’ power by self-contrast , author=. Advances in Neural Information Processing Systems , volume=

  25. [25]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  26. [26]

    Proceedings of the 31st International Conference on Computational Linguistics , pages=

    Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models , author=. Proceedings of the 31st International Conference on Computational Linguistics , pages=

  27. [27]

    Neural computation , volume=

    Adaptive mixtures of local experts , author=. Neural computation , volume=. 1991 , publisher=

    1991

  28. [28]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  29. [29]

    Qwen1.5-MoE: Matching 7B Model Performance with 1/3 Activated Parameters" , url =

    Qwen Team , month =. Qwen1.5-MoE: Matching 7B Model Performance with 1/3 Activated Parameters" , url =

  30. [30]

    Findings of the Association for Computational Linguistics ACL 2024 , pages=

    XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection , author=. Findings of the Association for Computational Linguistics ACL 2024 , pages=

    2024

  31. [31]

    CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-Experts , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2025

  32. [32]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  33. [33]

    arXiv preprint arXiv:2401.04088 , year=

    Mixtral of experts , author=. arXiv preprint arXiv:2401.04088 , year=

    Pith/arXiv arXiv

  34. [34]

    Journal of Machine Learning Research , volume=

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity , author=. Journal of Machine Learning Research , volume=

  35. [35]

    The Twelfth International Conference on Learning Representations , year=

    From Sparse to Soft Mixtures of Experts , author=. The Twelfth International Conference on Learning Representations , year=

  36. [36]

    CoRR , year=

    MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts , author=. CoRR , year=

  37. [37]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-Training , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  38. [38]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Null-shot prompting: rethinking prompting large language models with hallucination , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  39. [39]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers) , pages=

    Trusting your evidence: Hallucinate less with context-aware decoding , author=. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers) , pages=

    2024

  40. [40]

    2008 IEEE Symposium on Interactive Ray Tracing , pages=

    Fast agglomerative clustering for rendering , author=. 2008 IEEE Symposium on Interactive Ray Tracing , pages=. 2008 , organization=

    2008

  41. [41]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    A Closer Look into Mixture-of-Experts in Large Language Models , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  42. [42]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    TruthfulQA: Measuring How Models Mimic Human Falsehoods , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  43. [43]

    arXiv preprint arXiv:2110.14168 , year=

    Training verifiers to solve math word problems , author=. arXiv preprint arXiv:2110.14168 , year=

    Pith/arXiv arXiv

  44. [44]

    Transactions of the Association for Computational Linguistics , volume=

    Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies , author=. Transactions of the Association for Computational Linguistics , volume=

  45. [45]

    2016 23rd international conference on pattern recognition (ICPR) , pages=

    Branchynet: Fast inference via early exiting from deep neural networks , author=. 2016 23rd international conference on pattern recognition (ICPR) , pages=. 2016 , organization=

    2016

  46. [46]

    Advances in Neural Information Processing Systems , volume=

    Exploiting activation sparsity with dense to dynamic-k mixture-of-experts conversion , author=. Advances in Neural Information Processing Systems , volume=

  47. [47]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

    HellaSwag: Can a Machine Really Finish Your Sentence? , author=. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

  48. [48]

    Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Generating Benchmarks for Factuality Evaluation of Language Models , author=. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  49. [49]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    Uni-moe: Scaling unified multimodal llms with mixture of experts , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  50. [50]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Kam-cot: Knowledge augmented multimodal chain-of-thoughts reasoning , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  51. [51]

    CoRR , year=

    Think More, Hallucinate Less: Mitigating Hallucinations via Dual Process of Fast and Slow Thinking , author=. CoRR , year=

  52. [52]

    arXiv preprint arXiv:2409.00159 , year=

    LLMs Prompted for Graphs: Hallucinations and Generative Capabilities , author=. arXiv preprint arXiv:2409.00159 , year=

    Pith/arXiv arXiv

  53. [53]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  54. [54]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  55. [55]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  56. [56]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

    1980

  57. [57]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  58. [58]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

    doi:10.1016/s0020-7373(84)80003-6 1984

  59. [59]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  60. [60]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  61. [61]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

  62. [62]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  63. [63]

    2017 , eprint=

    Attention Is All You Need , author=. 2017 , eprint=

    2017

  64. [64]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  65. [65]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? , author=. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages=

    2024

  66. [66]

    arXiv preprint arXiv:2504.10020 , year=

    The Mirage of Performance Gains: Why Contrastive Decoding Fails to Address Multimodal Hallucination , author=. arXiv preprint arXiv:2504.10020 , year=

    Pith/arXiv arXiv

  67. [67]

    arXiv preprint arXiv:2501.12948 , year=

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning , author=. arXiv preprint arXiv:2501.12948 , year=

    Pith/arXiv arXiv

  68. [68]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

  69. [69]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    Improve Decoding Factuality by Token-wise Cross Layer Entropy of Large Language Models , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  70. [70]

    MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages=

    2019

  71. [71]

    IEEE Geoscience and Remote Sensing Letters , year=

    Heterogeneous mixture of experts for remote sensing image super-resolution , author=. IEEE Geoscience and Remote Sensing Letters , year=

  72. [72]

    Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V

    Uncertainty quantification and confidence calibration in large language models: A survey , author=. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 , pages=

  73. [73]

    arXiv preprint arXiv:2506.09684 , year=

    Inv-Entropy: A Fully Probabilistic Framework for Uncertainty Quantification in Language Models , author=. arXiv preprint arXiv:2506.09684 , year=

    arXiv

  74. [74]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Ualign: Leveraging uncertainty estimations for factuality alignment on large language models , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  75. [75]

    Findings of the Association for Computational Linguistics ACL 2024 , pages=

    Improving LLM Generations via Fine-Grained Self-Endorsement , author=. Findings of the Association for Computational Linguistics ACL 2024 , pages=

    2024

  76. [76]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Zero-resource hallucination detection for text generation via graph-based contextual knowledge triples modeling , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  77. [77]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Mitigating hallucinations in lm-based TTS models via distribution alignment using gflownets , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  78. [78]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    DPGA-TextSyn: Differentially Private Genetic Algorithm for Synthetic Text Generation , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  79. [79]

    GTA: Towards generative text-to-audio retrieval via multi-scale tokenizer , author=. Proc. Interspeech , pages=

  80. [80]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    CART: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

Showing first 80 references.