REVIEW 4 major objections 6 minor 38 references
An LLM that reads school calendars, weather, and official reports into a single transmission-impact number can match strong numerical forecasters on weekly HFMD point accuracy while adding calibrated prediction intervals and human-readable
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 19:32 UTC pith:IQHACK4Y
load-bearing objection Nice architectural idea, shaky uncertainty claims — the coverage is likely an artifact of LLM self-assessed width, and one key result contradicts its own table. the 4 major comments →
Auditable Context-Aware HFMD Forecasting with Structured LLM Agents
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On the paper's terms, the central discovery is that a neuro-symbolic division of labor works: let an LLM event interpreter read qualitative signals and compress them into a bounded transmission-impact score, and let a separate probabilistic forecaster consume that score alongside recent case counts. The authors show that this architecture matches or nearly matches the best numerical-only models on mean absolute error while producing substantially better-calibrated 90% intervals than classical statistical approaches, and it supplies an auditable rationale for every adjustment. The interpreter operates at inference time, so a new school closure or policy announcement can change the forecast im
What carries the argument
The load-bearing object is the Transmission Impact Score, a single number in [-1,1] produced by the LLM Event Interpreter that summarizes the net expected effect of all external context on HFMD transmission in the coming week, with a confidence score attached. The Forecast Generator then maps that score plus recent counts and an estimated volatility into a median forecast and an uncertainty score, and the pair is converted into a negative-binomial predictive distribution by moment matching, using variance constructed as the square of the product of mean, volatility, and one-plus-uncertainty. The 90% interval is read from the inverse CDF of that distribution. The score is what carries context
Load-bearing premise
The load-bearing premise is that the LLM's scalar transmission-impact score and its self-declared uncertainty faithfully encode real epidemiological drivers and calibrated uncertainty — if the score merely echoes hints in the prompt and the uncertainty is uncalibrated, the context channel adds no genuine forecasting value and the interval coverage numbers become artifacts.
What would settle it
Run the pipeline on the Lishui data twice with identical case counts, once with the real school calendar and once with the calendar shifted four weeks later; if point error and interval coverage do not materially change, the context channel is not carrying epidemiological information. A second check is a long held-out calibration: if empirical 90% coverage diverges sharply from the nominal level when the LLM's self-declared uncertainty is high, the variance formula is not a valid uncertainty model.
If this is right
- Weekly HFMD forecasts can come with calibrated 90% intervals and a short rationale naming the dominant driver (school term, weather, policy), which is directly useful for hospital-capacity and public-health planning.
- New contextual information — a closure, a vaccination campaign, an extreme-weather event — can change the forecast immediately at inference time, with no retraining, because the interpreter runs at query time.
- Ablations indicate school-calendar status is the strongest short-term driver in the Lishui data: removing it raises mean absolute error from about 4.1 to 6.7, implying systems that ignore school calendars will miss post-holiday surges.
- Grounding the interpreter in retrieved official guidelines stabilizes its reasoning; without retrieval, it becomes cautious and underestimates peaks, so the retrieval step is part of the mechanism rather than a wrapper.
- Because the two agents communicate through a single scalar, either side can be upgraded independently — a different language model or a different numerical forecaster — without rearchitecting the system.
Where Pith is reading between the lines
- Beyond the paper: the same two-agent template likely transfers to other environmentally sensitive infections such as influenza or dengue, since it only needs a disease-specific knowledge base and lag policy; the scalar interface keeps the forecasting core untouched.
- Beyond the paper: the framework's advantage may be largest in short, irregular series like Lishui, where classical models under-cover and foundation models smooth peaks; a natural test is a multi-year rolling evaluation to see whether the coverage advantage persists once seasonal structure is well learned.
- Beyond the paper: the 0.85–1.00 coverage hangs on the LLM's self-reported uncertainty being informative; an independent conformal calibration on held-out weeks would tell whether the intervals are statistically valid or partly an artifact of the moment-matching formula.
- Beyond the paper: since each forecast costs several LLM calls, deployment would likely need a distilled, cheaper interpreter that approximates the transmission-impact score; the scalar target makes such distillation straightforward.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a two-agent neuro-symbolic framework for weekly HFMD forecasting. Agent 1, an LLM-based Event Interpreter, ingests heterogeneous context (school calendars, weather, government reports, retrieved guidelines) and outputs a scalar transmission-impact score I_t along with a confidence score and natural-language rationale. Agent 2, an LLM-based Forecast Generator, combines I_t with recent case history and an estimated volatility v to produce point forecasts, which are mapped to negative-binomial/90% prediction intervals via moment matching σ² = (μ·v·(1+u))², where u is the LLM's self-declared uncertainty. The model is evaluated on Lishui (33 weeks) and Hong Kong (90 weeks) against classical, ML, LLM, and time-series foundation-model baselines. The central claims are that the approach achieves competitive point accuracy while providing robust 90% intervals and human-readable rationales, demonstrating that LLM-based context interpretation can match strong numerical forecasters.
Significance. If the claims are fully supported, the paper offers a useful modular design for incorporating unstructured domain knowledge into epidemiological forecasting while retaining a probabilistic output and an auditable explanation layer. The manuscript has concrete strengths: public code, full prompt templates in the appendix, two real datasets with different characteristics, and ablations isolating school calendars, climate, and RAG retrieval. However, the probabilistic-calibration claim is currently not established: the 90% intervals are driven by an uncalibrated LLM self-reported uncertainty, and the point-accuracy claim is overstated in the text relative to the reported tables. The contribution is still valuable as a qualitative-context injection mechanism, but the headline robustness claim needs recalibration or reframing.
major comments (4)
- [§3.3.1, Table 2] The text states that the Qwen3-based NSF-LLM 'achieves the best overall point-forecast accuracy, attaining the lowest MAE and RMSE among all models.' Table 2 directly contradicts this: TimesFM has MAE 3.972 vs. 4.124 for Qwen NSF-LLM, and Chronos/Moirai have lower CRPS (2.105/1.785 vs. 2.319). On Hong Kong (Table 3), Prophet/XGBoost/ARIMA/LSTM have MAE 3.49–3.53 vs. 3.49–4.32 for NSF-LLM variants. The 'lowest' claim should be replaced with 'competitive' and any remaining differences should be qualified with variance estimates given single-run evaluations.
- [§2.3.3, Eq. (σ² = (μ·v·(1+u))²)] The 90% interval width is a multiplicative function of the LLM's self-reported uncertainty u and a hand-clipped volatility v, with no recalibration step. Coverage is then measured on the same data used to generate the forecasts. The pattern in Table 2—Qwen NSF-LLM coverage 1.000 with CRPS 2.319, while Moirai has coverage 0.794 and CRPS 1.785—is exactly what one would expect from interval widening rather than calibrated predictive distributions. The abstract's 'robust 90% intervals (coverage approximately 0.85–1.00)' is therefore not supported. The authors should either calibrate u (e.g., conformal prediction, isotonic regression, or temperature scaling) or explicitly state that the intervals are conditional on LLM self-assessed uncertainty and not statistically calibrated.
- [§2.6.3, Tables 2–3] All experiments appear to be single-run rolling forecasts with provider-specific temperatures (0.6, 1.0, 0.7) and no repeated sampling. The reported MAE differences are often small (e.g., 4.124 vs. 3.972 on Lishui), so without repeated runs or seed/temperature sensitivity analysis, the 'competitive' point-accuracy claim is not distinguishable from LLM stochasticity. The authors should report means/standard deviations over at least several repeated runs, or justify why single runs are sufficient for their API configuration.
- [§3.2.2, Tables 2–3] The baseline list includes context-free LLM forecasters (Qwen3, GPT-5.1, Gemini Pro, DeepSeek-V3), but none of these are reported in Tables 2 or 3. The 'No-Agent1' ablation is not equivalent because the Forecast Generator still receives the full history and a different prompt. Without the context-free LLM results, the manuscript cannot isolate whether Agent 1's context interpretation is what improves over a pure LLM time-series forecaster. These baselines should be added to the tables or explicitly reported in an appendix.
minor comments (6)
- [§2.3.1] Typo: 'V olatility' should be 'Volatility'.
- [§4.4] The sentence 'Lower CRPS values from Moirai and the proposed Qwen-based framework' is misleading on Lishui: Qwen NSF-LLM CRPS 2.319 is higher than Chronos 2.105 and Moirai 1.785. Clarify that the Qwen variant has competitive but not lowest CRPS.
- [§2.3.2 vs Appendix A.4] Notation is inconsistent: §2.3.2 says the LLM outputs a 'median forecast' and 'uncertainty score u', while the prompt in A.4 asks for 'forecast_mean' and 'uncertainty_scale'. Reconcile the terminology.
- [§2.2.4 vs §2.6.3] The temperature is stated as 0.6 in §2.2.4, but §2.6.3 gives provider-specific temperatures (0.6 for Qwen3, 1.0 for DeepSeek, 0.7 for GPT-5.1). Clarify which temperature applies to which agent and whether Agent 2 uses a different value (the setup text mentions 0.2 elsewhere).
- [Figures 2–3] The figures are referenced as trajectory plots but not discussed panel-by-panel; a short pointer to which panels illustrate peak-tracking or over-smoothing would help.
- [Various] The repeated 'PRIME AI paper' header appears to be a template artifact and should be removed. The GitHub link should be verified before final submission.
Circularity Check
No significant circularity; the LLM-uncertainty interval concern is a calibration/validity issue, not a circular reduction.
full rationale
The derivation chain is a two-stage pipeline: Agent 1 converts external context into I_t and C_t; Agent 2 converts history plus (I_t, C_t) into a median forecast and uncertainty score u; Section 2.3.3 maps (mu, v, u) to negative-binomial parameters by moment matching, and coverage/CRPS are then computed on rolling out-of-sample origins against observed counts. No step is defined in terms of the quantity it claims to predict, and no load-bearing self-citation or imported uniqueness theorem appears. The potential weakness—that sigma^2 = (mu·v·(1+u))^2 makes interval width a direct function of the LLM's self-reported u, with no explicit recalibration to observed coverage—is a real validity and reproducibility concern (the paper itself concedes 'computational and reproducibility limits of commercial LLM APIs'), but it is not circularity: coverage is an empirical out-of-sample statistic that could in principle be poor, and CRPS is reported alongside it. The central accuracy claims are benchmarked against external baselines (TimesFM, Chronos, Moirai, LSTM, etc.), so those comparisons have independent content.
Axiom & Free-Parameter Ledger
free parameters (4)
- volatility clip bounds (volatility_min, volatility_max) =
0.05, 0.50
- recent_window =
8 weeks
- RAG retrieval top-k =
2
- LLM temperatures =
0.6 (Qwen3), 0.7 (GPT-5.1), 1.0 (DeepSeek)
axioms (5)
- domain assumption HFMD has a 1-week lag between transmission events and reported case counts (incubation + reporting delay)
- domain assumption School status is the strongest driver, followed by temperature/humidity, then other news
- ad hoc to paper A single scalar I_t ∈ [-1,1] can adequately summarize heterogeneous contextual drivers
- ad hoc to paper LLM-generated uncertainty u_{t+k} is a meaningful basis for probabilistic intervals
- domain assumption Retrieved China CDC guidelines are applicable to both Hong Kong and Lishui
invented entities (1)
-
Transmission Impact Score I_t
no independent evidence
read the original abstract
Effective HFMD surveillance requires forecasts capturing both time-series patterns and contextual drivers such as school calendars, weather, and policy or surveillance reports. In clinical settings, forecasts must be trusted and actionable; thus, beyond point accuracy, decision-makers require concise, auditable explanations of why risk is expected to rise or fall. Classical models (e.g., ARIMA and Prophet) and foundation models (e.g., Chronos, Moirai, and TimesFM) treat external covariates as numerical inputs, lacking semantic reasoning to reflect epidemiological mechanisms or resolve conflicting signals. We propose a two-agent neuro-symbolic framework that decouples contextual interpretation from probabilistic forecasting. An LLM-based Event Interpreter ingests heterogeneous signals -- school schedules, weather summaries, government reports, and clinical guidelines -- and outputs a scalar transmission-impact signal. A Forecast Generator combines this signal with historical case counts to produce point forecasts that are mapped to probabilistic predictions through Poisson/negative-binomial moment matching. We focus on one-week-ahead rolling forecasts, aligning with weekly hospital-capacity planning and the rapid, context-driven inflections typical of HFMD. We evaluate on two datasets: Hong Kong surveillance (90 target weeks in 2023--2024) and Lishui hospital visits (33 target weeks in 2024). Against traditional and foundation-model baselines, our approach achieves competitive point accuracy while providing robust 90\% intervals (coverage approximately 0.85--1.00) and concise rationales. This demonstrates that integrating domain knowledge through LLM-based agents can match strong numerical forecasters while yielding interpretable, context-aware forecasts aligned with public-health decision-making.
Figures
Reference graph
Works this paper leans on
-
[1]
Update on hand-foot-and-mouth disease.Clinics in dermatology, 33(3):340–346, 2015
Daniel Ventarola, Lindsey Bordone, and Nanette Silverberg. Update on hand-foot-and-mouth disease.Clinics in dermatology, 33(3):340–346, 2015
2015
-
[2]
Shih-Perng Chen, Yhu-Chering Huang, Wen-Chen Li, Cheng-Hsun Chiu, Chung-Guei Huang, Kuo-Chien Tsao, and Tzou-Yien Lin. Comparison of clinical features between coxsackievirus a2 and enterovirus 71 during the enterovirus outbreak in taiwan, 2008: a children’s hospital experience.Journal of Microbiology, Immunology and Infection, 43(2):99–104, 2010
2008
-
[3]
Epidemiology and clinical characteristics of hand foot, and mouth disease in a shenzhen sentinel hospital from 2009 to 2011
Yan-rong Wang, Lu-lu Sun, Wan-ling Xiao, Li-yun Chen, Xian-feng Wang, and Dong-ming Pan. Epidemiology and clinical characteristics of hand foot, and mouth disease in a shenzhen sentinel hospital from 2009 to 2011. BMC infectious diseases, 13(1):539, 2013
2009
-
[4]
Hand, foot, and mouth disease in china, 2008–12: an epidemiological study.The Lancet infectious diseases, 14(4):308–318, 2014
Weijia Xing, Qiaohong Liao, Cécile Viboud, Jing Zhang, Junling Sun, Joseph T Wu, Zhaorui Chang, Fengfeng Liu, Vicky J Fang, Yingdong Zheng, et al. Hand, foot, and mouth disease in china, 2008–12: an epidemiological study.The Lancet infectious diseases, 14(4):308–318, 2014
2008
-
[5]
Shigui Yang, Jie Wu, Cheng Ding, Yuanxia Cui, Yuqing Zhou, Yiping Li, Min Deng, Chencheng Wang, Kaijin Xu, Jingjing Ren, et al. Epidemiological features of and changes in incidence of infectious diseases in china in the first decade after the sars outbreak: an observational trend study.The Lancet Infectious Diseases, 17(7):716–725, 2017
2017
-
[6]
Current status of hand-foot-and-mouth disease.Journal of biomedical science, 30(1):15, 2023
Peiyu Zhu, Wangquan Ji, Dong Li, Zijie Li, Yu Chen, Bowen Dai, Shujie Han, Shuaiyin Chen, Yuefei Jin, and Guangcai Duan. Current status of hand-foot-and-mouth disease.Journal of biomedical science, 30(1):15, 2023
2023
-
[7]
Detecting spatial-temporal clusters of hfmd from 2007 to 2011 in shandong province, china.PloS one, 8(5):e63447, 2013
Yunxia Liu, Xianjun Wang, Yanxun Liu, Dapeng Sun, Shujun Ding, Bingbing Zhang, Zhaohui Du, and Fuzhong Xue. Detecting spatial-temporal clusters of hfmd from 2007 to 2011 in shandong province, china.PloS one, 8(5):e63447, 2013
2007
-
[8]
Epidemiological analysis, detection, and comparison of space-time patterns of beijing hand-foot-mouth disease (2008–2012).PLoS one, 9(3):e92745, 2014
Jiaojiao Wang, Zhidong Cao, Daniel Dajun Zeng, Quanyi Wang, Xiaoli Wang, and Haikun Qian. Epidemiological analysis, detection, and comparison of space-time patterns of beijing hand-foot-mouth disease (2008–2012).PLoS one, 9(3):e92745, 2014
2008
-
[9]
The epidemiology of hand, foot and mouth disease in asia: a systematic review and analysis.The Pediatric infectious disease journal, 35(10):e285–e300, 2016
Wee Ming Koh, Tiffany Bogich, Karen Siegel, Jing Jin, Elizabeth Y Chong, Chong Yew Tan, Mark IC Chen, Peter Horby, and Alex R Cook. The epidemiology of hand, foot and mouth disease in asia: a systematic review and analysis.The Pediatric infectious disease journal, 35(10):e285–e300, 2016
2016
-
[10]
Research on hand, foot and mouth disease incidence forecasting using hybrid model in mainland china.BMC Public Health, 23(1):619, 2023
Daren Zhao, Huiwu Zhang, Ruihua Zhang, and Sizhang He. Research on hand, foot and mouth disease incidence forecasting using hybrid model in mainland china.BMC Public Health, 23(1):619, 2023
2023
-
[11]
Applying infectious disease forecasting to public health: a path forward using influenza forecasting examples.BMC Public Health, 19(1):1659, 2019
Chelsea S Lutz, Mimi P Huynh, Monica Schroeder, Sophia Anyatonwu, F Scott Dahlgren, Gregory Danyluk, Danielle Fernandez, Sharon K Greene, Nodar Kipshidze, Leann Liu, et al. Applying infectious disease forecasting to public health: a path forward using influenza forecasting examples.BMC Public Health, 19(1):1659, 2019
2019
-
[12]
Princeton university press, 2020
James D Hamilton.Time series analysis. Princeton university press, 2020. 20 PRIME AI paper
2020
-
[13]
A contribution to the mathematical theory of epidemics
William Ogilvy Kermack and Anderson G McKendrick. A contribution to the mathematical theory of epidemics. Proceedings of the royal society of london. Series A, Containing papers of a mathematical and physical character, 115(772):700–721, 1927
1927
-
[14]
Predicting the incidence of hand, foot and mouth disease in sichuan province, china using the arima model.Epidemiology & Infection, 144(1):144–151, 2016
Lei Liu, RS Luan, F Yin, XP Zhu, and Q Lü. Predicting the incidence of hand, foot and mouth disease in sichuan province, china using the arima model.Epidemiology & Infection, 144(1):144–151, 2016
2016
-
[15]
Short-term effects of weather on hand, foot, and mouth disease
Yien Ling Hii, Joacim Rocklöv, and Nawi Ng. Short-term effects of weather on hand, foot, and mouth disease. Epidemiology, 22(1):S18, 2011
2011
-
[16]
Cui Guo, Jun Yang, Yuming Guo, Qiao-Qun Ou, Shuang-Quan Shen, Chun-Quan Ou, and Qi-Yong Liu. Short- term effects of meteorological factors on pediatric hand, foot, and mouth disease in guangdong, china: a multi-city time-series analysis.BMC infectious diseases, 16(1):524, 2016
2016
-
[17]
Spatio-temporal analysis of the relationship between meteorological factors and hand-foot-mouth disease in beijing, china.BMC infectious diseases, 18(1):158, 2018
Lin Tian, Fengchao Liang, Meimei Xu, Lei Jia, Xiaochuan Pan, and Archie CA Clements. Spatio-temporal analysis of the relationship between meteorological factors and hand-foot-mouth disease in beijing, china.BMC infectious diseases, 18(1):158, 2018
2018
-
[18]
Trend analysis and forecast of daily reported incidence of hand, foot and mouth disease in hubei, china by prophet model
Cong Xie, Haoyu Wen, Wenwen Yang, Jing Cai, Peng Zhang, Ran Wu, Mingyan Li, and Shuqiong Huang. Trend analysis and forecast of daily reported incidence of hand, foot and mouth disease in hubei, china by prophet model. Scientific reports, 11(1):1445, 2021
2021
-
[19]
Deep learning for time series forecasting: a survey.Big data, 9(1):3–21, 2021
José F Torres, Dalil Hadjout, Abderrazak Sebaa, Francisco Martínez-Álvarez, and Alicia Troncoso. Deep learning for time series forecasting: a survey.Big data, 9(1):3–21, 2021
2021
-
[20]
Forecasting at scale.The American Statistician, 72(1):37–45, 2018
Sean J Taylor and Benjamin Letham. Forecasting at scale.The American Statistician, 72(1):37–45, 2018
2018
-
[21]
A decoder-only foundation model for time-series forecasting
Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. InForty-first International Conference on Machine Learning, 2024
2024
-
[22]
Xgboost: A scalable tree boosting system
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. InProceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794, 2016
2016
-
[23]
Lightgbm: A highly efficient gradient boosting decision tree.Advances in neural information processing systems, 30, 2017
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree.Advances in neural information processing systems, 30, 2017
2017
-
[24]
Catboost: unbiased boosting with categorical features.Advances in neural information processing systems, 31, 2018
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr V orobev, Anna Veronika Dorogush, and Andrey Gulin. Catboost: unbiased boosting with categorical features.Advances in neural information processing systems, 31, 2018
2018
-
[25]
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis.arXiv preprint arXiv:2210.02186, 2022
Pith/arXiv arXiv 2022
-
[26]
Y Nie. A time series is worth 64words: Long-term forecasting with transformers.arXiv preprint arXiv:2211.14730, 2022
Pith/arXiv arXiv 2022
-
[27]
Chronos-2: From univariate to universal forecasting.arXiv e-prints, pages arXiv–2510, 2025
Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sun- dar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, et al. Chronos-2: From univariate to universal forecasting.arXiv e-prints, pages arXiv–2510, 2025
2025
-
[28]
Chronos: Learning the language of time series.arXiv preprint arXiv:2403.07815, 2024
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, et al. Chronos: Learning the language of time series.arXiv preprint arXiv:2403.07815, 2024
Pith/arXiv arXiv 2024
-
[29]
Xu Liu, Juncheng Liu, Gerald Woo, Taha Aksu, Yuxuan Liang, Roger Zimmermann, Chenghao Liu, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Moirai-moe: Empowering time series foundation models with sparse mixture of experts.arXiv preprint arXiv:2410.10469, 2024
Pith/arXiv arXiv 2024
-
[30]
Neuro-symbolic models for interpretable time series classification using temporal logic description
Ruixuan Yan, Tengfei Ma, Achille Fokoue, Maria Chang, and Agung Julius. Neuro-symbolic models for interpretable time series classification using temporal logic description. In2022 IEEE International Conference on Data Mining (ICDM), pages 618–627. IEEE, 2022
2022
-
[31]
React: Synergizing reasoning and acting in language models
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InThe eleventh international conference on learning representations, 2022
2022
-
[32]
Logicgame: Benchmarking rule-based reasoning abilities of large language models
Jiayi Gui, Yiming Liu, Jiale Cheng, Xiaotao Gu, Xiao Liu, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. Logicgame: Benchmarking rule-based reasoning abilities of large language models. InFindings of the Association for Computational Linguistics: ACL 2025, pages 1474–1491, 2025
2025
-
[33]
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anand- kumar. V oyager: An open-ended embodied agent with large language models.arXiv preprint arXiv:2305.16291, 2023. 21 PRIME AI paper
Pith/arXiv arXiv 2023
-
[34]
Reflexion: Language agents with verbal reinforcement learning.Advances in Neural Information Processing Systems, 36:8634–8652, 2023
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning.Advances in Neural Information Processing Systems, 36:8634–8652, 2023
2023
-
[35]
Assessment of large language models in clinical reasoning: A novel benchmarking study.NEJM AI, 2(10):AIdbp2500120, 2025
Liam G McCoy, Rajiv Swamy, Nidhish Sagar, Minjia Wang, Stephen Bacchi, Jie Ming Nigel Fong, Nigel CK Tan, Kevin Tan, Thomas A Buckley, Peter Brodeur, et al. Assessment of large language models in clinical reasoning: A novel benchmarking study.NEJM AI, 2(10):AIdbp2500120, 2025
2025
-
[36]
Livebench: A challenging, contamination-free LLM benchmark
Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Benjamin Feuer, Siddhartha Jain, Ravid Shwartz- Ziv, Neel Jain, Khalid Saifullah, Sreemanti Dey, Shubh-Agrawal, Sandeep Singh Sandha, Siddartha Venkat Naidu, Chinmay Hegde, Yann LeCun, Tom Goldstein, Willie Neiswanger, and Micah Goldblum. Livebench: A challenging, contamination-free LLM benchmark. InTh...
2025
-
[37]
Halogen: Fantastic llm hallucinations and where to find them.arXiv preprint arXiv:2501.08292, 2025
Abhilasha Ravichander, Shrusti Ghela, David Wadden, and Yejin Choi. Halogen: Fantastic llm hallucinations and where to find them.arXiv preprint arXiv:2501.08292, 2025
Pith/arXiv arXiv 2025
-
[38]
Siren’s song in the ai ocean: A survey on hallucination in large language models.Computational Linguistics, pages 1–46, 2025
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. Siren’s song in the ai ocean: A survey on hallucination in large language models.Computational Linguistics, pages 1–46, 2025. 22 PRIME AI paper (a) ARIMA (b) LSTM (c) Prophet (d) XGBoost (e) TimesFM (f) Chronos (g) NSF-LLM (GPT-5.1) (h...
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.