pith. sign in

arxiv: 2606.28725 · v1 · pith:5VJD7P3Mnew · submitted 2026-06-27 · 💻 cs.CL

DriftGuard: Safety-Aware Multi-Monitor Detection and Selective Adaptation for Evolving Toxicity Moderation

Pith reviewed 2026-06-30 10:07 UTC · model grok-4.3

classification 💻 cs.CL
keywords toxicity moderationdrift detectionselective model adaptationsafety-aware monitoringfalse-negative reductiontemporal shiftcross-dataset evaluationhard-mix updating
0
0 comments X

The pith

DriftGuard detects safety-relevant toxicity shifts via five specialized monitors and selectively updates models on hard-mix high-risk examples to raise toxic recall.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Existing drift detection in toxicity moderation relies on global distributional change, which can overlook localized harm patterns such as coded language or shifting identity targets. DriftGuard adds four safety-specific monitors for identity-harm drift, model uncertainty, toxic-risk drift, and false-negative-risk drift alongside the global monitor. When any safety monitor triggers, the system assembles a hard-mix adaptation set that prioritizes likely false negatives, identity-related high-risk cases, false-positive risks, and boundary examples. On Civil Comments temporal shift this raises toxic recall to 0.8777; on Jigsaw-to-DynaHate cross-dataset shift recall rises from 0.7107 to 0.8523 with a 0.0781 drop in false-negative prevalence. The framework therefore ties targeted detection directly to lightweight, safety-focused model updates rather than blanket retraining.

Core claim

DriftGuard is a safety-aware adaptive moderation framework that tracks five drift signals: global text drift, identity-harm drift, model uncertainty, toxic-risk drift, and false-negative-risk drift. Detection of safety-relevant change triggers selective updating on a hard-mix adaptation set that prioritizes likely false negatives, identity-related high-risk examples, false-positive-risk examples, and uncertain boundary cases. Experiments on Civil Comments temporal shift and Jigsaw-to-DynaHate cross-dataset shift demonstrate that the safety-aware monitors surface risks missed by global drift alone, while hard-mix adaptation improves toxic recall and accuracy over no-update and random-balanced

What carries the argument

The multi-monitor drift detection system (global text drift plus identity-harm, uncertainty, toxic-risk, and false-negative-risk monitors) paired with hard-mix adaptation selection that assembles a prioritized update set from likely false negatives and high-risk boundary cases.

If this is right

  • Safety-aware monitors surface risks missed by global drift detection alone.
  • Hard-mix adaptation raises toxic recall to 0.8777 on Civil Comments temporal shift and from 0.7107 to 0.8523 on Jigsaw-to-DynaHate shift.
  • Bootstrap analysis shows stable DynaHate safety gains with toxic recall up 0.1418 and false-negative prevalence down 0.0781.
  • The framework links safety-aware detection directly to targeted lightweight model updating for evolving moderation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The monitor-plus-hard-mix design could be tested on other evolving content-moderation domains such as misinformation or hate-speech variants to check whether the same localized-shift logic applies.
  • If the hard-mix selection inadvertently over-weights certain identity subgroups, an auxiliary fairness monitor could be added without changing the core detection logic.
  • Production deployments might measure the reduction in full-retraining frequency achieved by triggering updates only when safety monitors fire.
  • Extending the false-negative-risk monitor to track emerging coded-language patterns would be a direct next measurement on new shift datasets.

Load-bearing premise

The five monitors accurately identify safety-relevant localized shifts that merit updating, and the hard-mix selection of adaptation examples yields genuine generalization improvements rather than overfitting to the chosen subsets.

What would settle it

An experiment on a fresh temporal or cross-dataset toxicity shift in which applying the hard-mix adaptation set produces no statistically significant gain in toxic recall or reduction in false-negative rate compared with a random-balanced update baseline would falsify the selective-adaptation benefit.

read the original abstract

Automated toxicity moderation systems operate in dynamic online environments where harmful behavior evolves through coded language, shifting targets, and strategic adaptation to enforcement. Existing drift detection methods often focus on global distributional change, but such signals may miss safety-relevant shifts that emerge in localized harm subspaces or high-risk model-error regions. This paper introduces DriftGuard, a safety-aware adaptive moderation framework that combines multi-monitor drift detection with selective model updating. The framework tracks global text drift, identity-harm drift, model uncertainty, toxic-risk drift, and false-negative-risk drift. When safety-relevant change is detected, the model is updated using a hard-mix adaptation set that prioritizes likely false negatives, identity-related high-risk examples, false-positive-risk examples, and uncertain boundary cases. Experiments on Civil Comments temporal shift and Jigsaw-to-DynaHate cross-dataset shift show that safety-aware monitors detect risks missed by global drift alone. Hard-mix adaptation improves toxic recall and accuracy over no-update and random-balanced baselines, raising toxic recall to 0.8777 on Civil Comments and from 0.7107 to 0.8523 on DynaHate. Bootstrap analysis further shows stable DynaHate safety gains, with toxic recall increasing by 0.1418 and false-negative prevalence decreasing by 0.0781. Overall, DriftGuard links safety-aware drift detection to targeted, lightweight model updating for more robust adaptive toxicity moderation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The manuscript introduces DriftGuard, a framework for toxicity moderation that combines five safety-aware monitors (global text drift, identity-harm drift, model uncertainty, toxic-risk drift, false-negative-risk drift) with selective model updating via a hard-mix adaptation set prioritizing false negatives, identity high-risk examples, false-positive-risk cases, and boundary examples. On Civil Comments temporal shift and Jigsaw-to-DynaHate cross-dataset shift, it reports that the monitors detect localized risks missed by global drift, and hard-mix adaptation raises toxic recall to 0.8777 (Civil Comments) and from 0.7107 to 0.8523 (DynaHate), with bootstrap analysis showing stable gains and reduced false-negative prevalence.

Significance. If the monitors and hard-mix procedure can be shown to isolate genuine safety-relevant shifts and produce non-overfit gains, the work would address a practical gap in adaptive content moderation by moving beyond global distributional signals. The explicit connection between multi-monitor detection and targeted lightweight updating is a coherent direction for handling evolving coded language and shifting targets in online toxicity.

major comments (3)
  1. [Abstract] Abstract: the central empirical claims rest on concrete recall numbers (0.8777; 0.7107 o0.8523) and bootstrap deltas (toxic recall +0.1418, false-negative prevalence -0.0781), yet the text supplies no equations, scoring functions, or pseudocode defining any of the five monitors, their thresholds, or how they differ from global drift. This prevents verification that the monitors isolate localized safety shifts rather than re-expressing the adaptation targets.
  2. [Abstract] Abstract: the hard-mix adaptation set is described as prioritizing false negatives, identity-related high-risk examples, false-positive-risk examples, and uncertain boundary cases, but no sampling weights, selection algorithm, or dataset statistics are given. Without these, it is impossible to assess whether the reported gains over no-update and random-balanced baselines reflect generalization or selection bias on the chosen subsets.
  3. [Abstract] Abstract: the experiments invoke temporal shift on Civil Comments and cross-dataset shift from Jigsaw to DynaHate, yet provide no details on split construction, label distributions, or how the adaptation examples are drawn from the target distribution. This leaves open the possibility of leakage or non-stationarity artifacts that could inflate the observed improvements.
minor comments (1)
  1. [Abstract] The abstract is concise but would benefit from a single sentence clarifying the relationship between the five monitors and the hard-mix selection criteria to aid readers in assessing independence of the detection and adaptation stages.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each comment below. While the abstract is necessarily concise, the full manuscript provides the requested details in the methods and experiments sections; we will revise the abstract to improve verifiability and cross-referencing.

read point-by-point responses
  1. Referee: [Abstract] Abstract: the central empirical claims rest on concrete recall numbers (0.8777; 0.7107 to 0.8523) and bootstrap deltas (toxic recall +0.1418, false-negative prevalence -0.0781), yet the text supplies no equations, scoring functions, or pseudocode defining any of the five monitors, their thresholds, or how they differ from global drift. This prevents verification that the monitors isolate localized safety shifts rather than re-expressing the adaptation targets.

    Authors: The abstract summarizes the framework at a high level. Equations, scoring functions (e.g., KL divergence for global text drift, targeted identity-term analysis for identity-harm drift, entropy-based uncertainty, and risk-specific drift measures), thresholds (calibrated on validation data), and pseudocode distinguishing the monitors from global drift appear in Section 3.1. We will revise the abstract to include a short reference to these formulations and the section number. revision: yes

  2. Referee: [Abstract] Abstract: the hard-mix adaptation set is described as prioritizing false negatives, identity-related high-risk examples, false-positive-risk examples, and uncertain boundary cases, but no sampling weights, selection algorithm, or dataset statistics are given. Without these, it is impossible to assess whether the reported gains over no-update and random-balanced baselines reflect generalization or selection bias on the chosen subsets.

    Authors: The prioritization logic, sampling weights, selection algorithm, and dataset statistics for the hard-mix set are specified in Section 3.2. We will update the abstract to briefly note the selection criteria and direct readers to the methods for the full algorithm and statistics. revision: yes

  3. Referee: [Abstract] Abstract: the experiments invoke temporal shift on Civil Comments and cross-dataset shift from Jigsaw to DynaHate, yet provide no details on split construction, label distributions, or how the adaptation examples are drawn from the target distribution. This leaves open the possibility of leakage or non-stationarity artifacts that could inflate the observed improvements.

    Authors: Split construction, label distributions, and sampling procedures for adaptation examples (with explicit steps to prevent leakage) are detailed in Section 4.1. We will add a concise clause to the abstract describing the shift setups and referencing the experimental section. revision: yes

Circularity Check

0 steps flagged

No circularity detected; purely empirical evaluation on held-out shifts

full rationale

The paper describes a multi-monitor drift detection framework and hard-mix adaptation, with all central claims supported by direct experimental measurements (toxic recall 0.8777 on Civil Comments temporal shift; 0.7107→0.8523 on Jigsaw-to-DynaHate). No equations, derivations, fitted parameters renamed as predictions, or self-citation chains appear in the provided text. The reported gains are framed as outcomes on external held-out datasets rather than quantities defined by construction from the monitors or adaptation rules themselves. This is the standard case of an empirical ML paper whose validity rests on data splits and metrics, not on internal definitional reduction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No details available from abstract to identify free parameters, axioms, or invented entities.

pith-pipeline@v0.9.1-grok · 5795 in / 1159 out tokens · 30669 ms · 2026-06-30T10:07:19.879410+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 17 canonical work pages · 9 internal anchors

  1. [1]

    A survey on concept drift adaptation,

    J. Gama, I. ˇZliobait˙e, A. Bifet, M. Pechenizkiy, and A. Bouchachia, “A survey on concept drift adaptation,”ACM computing surveys (CSUR), vol. 46, no. 4, pp. 1–37, 2014

  2. [2]

    Ranking Abuse via Strategic Pairwise Data Perturbations

    J. Yao, Z. Zheng, and J. Long, “Ranking abuse via strategic pairwise data perturbations,” 2026. [Online]. Available: https://arxiv.org/abs/ 2604.17805

  3. [3]

    Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection

    L. Lin, J. You, Y . Li, L. Lin, Y . Wang, Z. Zhang, and M. Zheng, “Reflect-guard: Enhancing llm safeguards against adversarial prompts via logical self-reflection,”arXiv preprint arXiv:2605.24834, 2026. [Online]. Available: https://arxiv.org/abs/2605.24834

  4. [4]

    Failing loudly: An empir- ical study of methods for detecting dataset shift,

    S. Rabanser, S. G ¨unnemann, and Z. Lipton, “Failing loudly: An empir- ical study of methods for detecting dataset shift,”Advances in Neural Information Processing Systems, vol. 32, 2019

  5. [5]

    Nu- anced metrics for measuring unintended bias with real data for text classification,

    D. Borkan, L. Dixon, J. Sorensen, N. Thain, and L. Vasserman, “Nu- anced metrics for measuring unintended bias with real data for text classification,” inCompanion proceedings of the 2019 world wide web conference, 2019, pp. 491–500

  6. [6]

    The risk of racial bias in hate speech detection,

    M. Sap, D. Card, S. Gabriel, Y . Choi, and N. A. Smith, “The risk of racial bias in hate speech detection,” inProceedings of the 57th annual meeting of the association for computational linguistics, 2019, pp. 1668– 1678

  7. [7]

    Beyond math: Stories as a testbed for memorization-constrained reasoning in llms,

    Y . Jiang and F. Ferraro, “Beyond math: Stories as a testbed for memorization-constrained reasoning in llms,” inProceedings of the 19th Conference of the European Chapter of the Association for Computa- tional Linguistics (Volume 1: Long Papers), 2026, pp. 5590–5607

  8. [8]

    Measuring whether llm tutors teach or solve: A diagnostic for educational impact,

    J. Yao, Z. Zheng, and B. Li, “Measuring whether llm tutors teach or solve: A diagnostic for educational impact,” 2026. [Online]. Available: https://arxiv.org/abs/2606.16206

  9. [9]

    A unified framework for dataset shift diagnostics,

    F. M. Polo, R. Izbicki, E. G. Lacerda Jr, J. P. Ibieta-Jimenez, and R. Vi- cente, “A unified framework for dataset shift diagnostics,”Information Sciences, vol. 649, p. 119612, 2023

  10. [10]

    Hatecheck: Functional tests for hate speech detection models,

    P. R ¨ottger, B. Vidgen, D. Nguyen, Z. Talat, H. Margetts, and J. Pierre- humbert, “Hatecheck: Functional tests for hate speech detection models,” inProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 41–58

  11. [11]

    When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems

    S. Wang, P. Qian, Y . Chen, J. You, X. Wang, X. Jiang, L. Liu, H. Yu, and J. Xu, “When safe skills collide: Measuring compositional risk in agent skill ecosystems,” 2026. [Online]. Available: https://arxiv.org/abs/2606.00448

  12. [12]

    Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAG

    P. Qian, S. Wang, X. Wang, Y . Chen, W. Xu, Q. Yu, S. Lin, S. Zhang, J. You, and X. Wei, “Relevant is not warranted: Evidence-force calibration for cited rag,” 2026. [Online]. Available: https://arxiv.org/abs/2605.28044

  13. [13]

    Active learning literature survey,

    B. Settles, “Active learning literature survey,” 2009

  14. [14]

    From concept drift to model degradation: An overview on performance-aware drift detectors,

    F. Bayram, B. S. Ahmed, and A. Kassler, “From concept drift to model degradation: An overview on performance-aware drift detectors,” Knowledge-Based Systems, vol. 245, p. 108632, 2022

  15. [15]

    SHAP stability in credit risk management: A case study in credit card default model,

    L. Lin and Y . Wang, “SHAP stability in credit risk management: A case study in credit card default model,”Risks, vol. 13, no. 12, p. 238,

  16. [16]

    Available: https://doi.org/10.3390/risks13120238

    [Online]. Available: https://doi.org/10.3390/risks13120238

  17. [17]

    Detecting and correcting for label shift with black box predictors,

    Z. Lipton, Y .-X. Wang, and A. Smola, “Detecting and correcting for label shift with black box predictors,” inInternational conference on machine learning. PMLR, 2018, pp. 3122–3130

  18. [18]

    Wilds: A benchmark of in-the-wild distribution shifts,

    P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsub- ramani, W. Hu, M. Yasunaga, R. L. Phillips, I. Gaoet al., “Wilds: A benchmark of in-the-wild distribution shifts,” inInternational conference on machine learning. PMLR, 2021, pp. 5637–5664

  19. [19]

    A mon- itoring framework for deployed machine learning models with supply chain examples,

    B. Eck, D. Kabakci-Zorlu, Y . Chen, F. Savard, and X. Bao, “A mon- itoring framework for deployed machine learning models with supply chain examples,” in2022 IEEE International Conference on Big Data (Big Data). IEEE, 2022, pp. 2231–2238

  20. [20]

    Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict

    Y . Chen, P. Qian, S. Wang, S. Zhang, H. Xu, S. Lin, and X. Wei, “Does rag know when retrieval is wrong? diagnosing context compliance under knowledge conflict,” 2026. [Online]. Available: https://arxiv.org/abs/2605.14473

  21. [21]

    Discrepancy learning guided hierarchical fusion network for multi-modal recommen- dation,

    Y . Dang, Z. Pan, X. Zhang, W. Chen, F. Cai, and H. Chen, “Discrepancy learning guided hierarchical fusion network for multi-modal recommen- dation,”Knowledge-Based Systems, vol. 317, p. 113496, 2025

  22. [22]

    Finsentllm: Multi-llm and structured semantic signals for enhanced financial sentiment forecasting,

    Z. Zhang, R. Fu, Y . He, X. Shen, Y . Wang, X. Du, H. You, K. Jin, J. Shi, and S. Fong, “Finsentllm: Multi-llm and structured semantic signals for enhanced financial sentiment forecasting,” inICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2026, pp. 17 682–17 686

  23. [23]

    A systematic review of hate speech au- tomatic detection using natural language processing,

    M. S. Jahan and M. Oussalah, “A systematic review of hate speech au- tomatic detection using natural language processing,”Neurocomputing, vol. 546, p. 126232, 2023

  24. [24]

    Toxicity detection: Does context really matter?

    J. Pavlopoulos, J. Sorensen, L. Dixon, N. Thain, and I. Androutsopoulos, “Toxicity detection: Does context really matter?” inProceedings of the 58th annual meeting of the association for computational linguistics, 2020, pp. 4296–4305

  25. [25]

    Racial bias in hate speech and abusive language detection datasets,

    T. Davidson, D. Bhattacharya, and I. Weber, “Racial bias in hate speech and abusive language detection datasets,” inProceedings of the third workshop on abusive language online, 2019, pp. 25–35

  26. [26]

    Medequalizer: A framework investigating bias in synthetic medical data and mitigation via augmentation,

    S. Salarian, Y . Zhang, S. Padhee, and S. Parthasarathy, “Medequalizer: A framework investigating bias in synthetic medical data and mitigation via augmentation,”arXiv preprint arXiv:2511.01054, 2025

  27. [27]

    Hatexplain: A benchmark dataset for explainable hate speech detection,

    B. Mathew, P. Saha, S. M. Yimam, C. Biemann, P. Goyal, and A. Mukherjee, “Hatexplain: A benchmark dataset for explainable hate speech detection,” inProceedings of the AAAI conference on artificial intelligence, vol. 35, no. 17, 2021, pp. 14 867–14 875

  28. [28]

    Learning from the worst: Dynamically generated datasets to improve online hate detection,

    B. Vidgen, T. Thrush, Z. Talat, and D. Kiela, “Learning from the worst: Dynamically generated datasets to improve online hate detection,” inProceedings of the 59th annual meeting of the Association for Computational Linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), 2021, pp. 1667– 1682

  29. [29]

    A survey of active learning for natural language processing,

    Z. Zhang, E. Strubell, and E. Hovy, “A survey of active learning for natural language processing,” inProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 6166– 6190

  30. [30]

    A survey on deep active learning: Recent advances and new frontiers,

    D. Li, Z. Wang, Y . Chen, R. Jiang, W. Ding, and M. Okumura, “A survey on deep active learning: Recent advances and new frontiers,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 4, pp. 5879–5899, 2024

  31. [31]

    Training region-based object detectors with online hard example mining,

    A. Shrivastava, A. Gupta, and R. Girshick, “Training region-based object detectors with online hard example mining,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 761– 769

  32. [32]

    Focal loss for dense object detection,

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, “Focal loss for dense object detection,” inProceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988

  33. [33]

    PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence

    Y . Xu, H. Sang, Z. Zhou, R. He, and Z. Wang, “Paced: Distillation and on-policy self-distillation at the frontier of student competence,” 2026. [Online]. Available: https://arxiv.org/abs/2603.11178

  34. [34]

    TIP: Token Importance in On-Policy Distillation

    Y . Xu, H. Sang, Z. Zhou, R. He, Z. Wang, and A. Geramifard, “Tip: Token importance in on-policy distillation,” 2026. [Online]. Available: https://arxiv.org/abs/2604.14084

  35. [35]

    Lora: Low-rank adaptation of large language models

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, W. Chenet al., “Lora: Low-rank adaptation of large language models.” Iclr, vol. 1, no. 2, p. 3, 2022

  36. [36]

    Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

    Z. Han, C. Gao, J. Liu, J. Zhang, and S. Q. Zhang, “Parameter-efficient fine-tuning for large models: A comprehensive survey,”arXiv preprint arXiv:2403.14608, 2024

  37. [37]

    Parameter-efficient fine-tuning in large models: A survey of method- ologies,

    L. Wang, S. Chen, L. Jiang, S. Pan, R. Cai, S. Yang, and F. Yang, “Parameter-efficient fine-tuning in large models: A survey of method- ologies,”arXiv preprint arXiv:2410.19878, 2024

  38. [38]

    Study of exam- ples effect on the llm performance,

    A. Ainiwaer, Q. Liu, and M. Lily, “Study of exam- ples effect on the llm performance,” 2026, available: https://https://doi.org/10.13140/RG.2.2.20382.40007

  39. [39]

    Gsq-tuning: Group-shared exponents integer in fully quantized training for llms on- device fine-tuning,

    S. Zhou, S. Wang, Z. Yuan, M. Shi, Y . Shang, and D. Yang, “Gsq-tuning: Group-shared exponents integer in fully quantized training for llms on- device fine-tuning,” inFindings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 22 971–22 988

  40. [40]

    Medtri: A platform for structured medical report normalization to enhance vision-language pretraining,

    Y . Chu, X. Ma, X. Jin, G. Luo, and X. Gao, “Medtri: A platform for structured medical report normalization to enhance vision-language pretraining,”arXiv preprint arXiv:2602.22143, 2026

  41. [41]

    Drp: Distilled reasoning pruning with skill-aware step decomposition for efficient large reasoning models,

    Y . Jiang, D. Li, and F. Ferraro, “Drp: Distilled reasoning pruning with skill-aware step decomposition for efficient large reasoning models,”

  42. [42]
  43. [43]

    Netsenseml: Network-adaptive compression for efficient distributed machine learning,

    Y . Wang, X. Li, R. Wu, H. Chen, and D. Kutscher, “Netsenseml: Network-adaptive compression for efficient distributed machine learning,” inEuro-Par 2025: Parallel Processing: 31st European Conference on Parallel and Distributed Processing, Dresden, Germany, August 25–29, 2025, Proceedings, Part III. Berlin, Heidelberg: Springer-Verlag, 2025, p. 283–297. [...

  44. [44]

    A comprehensive anal- ysis of indicator effect on llm performance,

    A. Ainiwaer, Q. Liu, and M. Lily, “A comprehensive anal- ysis of indicator effect on llm performance,” 2026, available: https://doi.org/10.13140/RG.2.2.29554.98248