REVIEW 5 major objections 5 minor 25 references
Evaluating LLMs as Interpretable Controllers for Dynamical Systems
T0 review · 5 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read This paper shows that large language models can act as interpretable closed-loop controllers for a physical system, provided the model is sufficiently large and the prompt grounds the actuator semantics.
desk verdict A useful scaling probe with an overclaimed grounding result: the prediction tool is the same fitted model as the plant, so the 'domain knowledge' improvement is structurally expected. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the closed-loop LLM controller: a chat agent prompted with the current temperature, ambient, setpoint, and (when applicable) an actuator-usage penalty, which must emit a structured JSON plan (heater float, fan binary) plus a natural-language rationale. On top of this, a forward-simulation tool ('simulate_plan') lets the model run candidate action sequences against the physics-based model before acting. The interpretive lens that carries the scale-dependence argument is the five-dimension reasoning taxonomy used to classify every explanation; it is the lens through which the authors convert control trajectories into evidence about what the models understand.
What would settle it
Re-run the five models over multiple seeds and sampling temperatures in the same enclosure, and have independent raters (or an automated rubric) re-code a sample of explanations. If Qwen-3 14B's MAE varies by more than about 0.5 °C across runs, or if the same model's explanation changes from 'cohesive' to 'misinterpreted' across trials, the claim that high-complexity models maintain stable, physically grounded reasoning would be falsified.
Extended reading notes
Core claim
On the authors' account, an LLM can act as a viable closed-loop controller for a dynamical system when two conditions hold: the model is large enough to maintain a stable internal representation of the causal actuator effects, and the natural-language prompt plus, optionally, a physics-based forward simulator, grounds the mapping from states to actions. In their testbed—a 50 cm cuboidal enclosure with a discrete-step heater duty cycle and a binary fan, tracking a 360-minute reference profile at 60-second steps—Qwen-3 14B and GPT-4o keep mean absolute error below 0.1–0.2 °C across all penalty scenarios and produce heater cycles that anticipate the reference, whereas Gemma-3 1B and Phi-4 Mini
Load-bearing premise
The load-bearing premise is that each reported trajectory is a representative single run and that the qualitative reasoning-taxonomy labels are reproducible; if LLM outputs are stochastic across runs or the taxonomy labels would not be assigned the same way by independent raters, the scale-dependent conclusion—small models misread dynamics, large models reason cohesively—is unsupported.
Editorial extensions
If this is right
- If the central claim holds, LLM-based controllers are viable for slow, well-specified physical processes—at least when the model is sufficiently capable and the prompt is unambiguous about actuator semantics.
- Natural-language penalties (e.g., 'minimize heater use') are effectively communicated only to high-complexity models; mid-scale models may 'understand' the penalty but break the tracking objective, so prompt-language multi-objective control is not yet reliable across scales.
- Giving a capable LLM a physics-based forward simulator yields smoother trajectories, fewer overshoots, and lower energy use—evidence that hybrid model-based + language-driven control can improve both performance and interpretability.
- Explanation quality correlates with control performance, so the reasoning taxonomy could serve as a coarse diagnostic: if a controller's rationale is physically coherent and temporally aware, its actions are more likely to be sound.
- The results suggest that parameter count is not the only driver: Qwen-3 14B outperformed the much larger GPT-4o in both precision and energy efficiency, pointing to architectural and training differences.
Reading between the lines
- A natural next test, not run here, is whether giving the forward-simulation tool to weaker models (e.g., Mistral 7B) rescues their closed-loop performance; if it does not, the bottleneck is causal reasoning rather than missing predictive information, which would sharpen the scale-dependence story.
- Because each trajectory appears to be a single run, the numerical MAE claims are fragile; repeating the experiment across seeds and sampling temperatures would show whether the 0.086 °C figure for Qwen-3 is a stable property or a lucky draw.
- The reasoning taxonomy is a qualitative instrument; encoding it as an automated rubric (or measuring inter-rater agreement) would let the claimed progression from 'hallucinated physics' to 'temporally aware reasoning' be tested quantitatively, and would make the interpretability claim more than anecdotal.
- If LLM controllers are to be used in safety-critical settings, the same prompt-engineering and tool-based grounding could be combined with classical MPC or PID as a fallback—an architecture that keeps the LLM's explanations but adds formal stability guarantees.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper evaluates five LLMs (Gemma-3 1B, Phi-4 Mini 3.8B, Mistral 7B, Qwen-3 14B, GPT-4o) as closed-loop controllers for a simulated thermal enclosure described by a first-order physics-based model (Eq. 1). Controllers receive state/setpoint information in natural language and return heater duty-cycle and fan commands with textual reasoning. Experiments cover three settings (no penalty, heater penalty, fan penalty), plus a tool-assisted setting in which selected models may call a forward simulator before acting. The paper reports MAE and actuator-effort results, claims that high-complexity models track the reference well and produce physically coherent explanations, and argues that providing the physics-based prediction tool substantially improves control smoothness and energy efficiency. A qualitative reasoning taxonomy is used to support a scale-dependent progression in causal, temporal, and cohesive reasoning.
Significance. If the claims were fully supported, the paper would be a useful step toward LLM-based interpretable control and hybrid LLM+PBM pipelines. The manuscript is transparent about its prompt design, model equation, and tuning procedure, and the trace tables in the appendix give concrete examples of model behavior. However, the current evidence for the central scale-dependency and tool-grounding conclusions is substantially weaker than the abstract suggests: the tool experiment is close to circular, the quantitative results are single-run, and the reasoning taxonomy lacks coding discipline. With additional experiments and more rigorous analysis, the core idea could be publishable, but the present version overstates what has been demonstrated.
major comments (5)
- [Section 4.2 / Sections 2.2, 3.1, 3.3] The tool-assisted improvement claim is structurally guaranteed with the current setup. The plant is the fitted physics-based model (Section 3.1: "continuous updates of values from the simulated model"; Sections 2.2 and 3.3 use Eq. 1 with fitted H'_max and F'_max), and the tool is a forward simulator of the same Eq. 1 with the same fitted parameters. When the prediction tool is an exact oracle of the plant, any model that performs even a small amount of candidate-plan search will appear to improve; this does not test whether LLMs can incorporate model-based knowledge under model mismatch, sensor noise, or uncertainty. The conclusion in the abstract and Section 5 that "incorporating a physics-based model significantly improves control smoothness and energy efficiency" is therefore only supported in the oracle setting. Please quantify the tool-on vs tool-off difference (MAE, actuator usage,
- [Section 4.1 / Figure 6] All quantitative performance results appear to be single trajectories. The text reports MAE values (e.g., 0.086 °C for Qwen-3, 0.098 °C with heater penalty) without stating sampling temperature, seeds, number of repetitions, or run-to-run variability. LLM generation is stochastic, and the observed differences between mid- and high-complexity models could easily be within the noise of a single run. The central scale-dependent conclusion and the penalty comparisons need repeated runs with confidence intervals, or at minimum a clear statement that these are illustrative single-run observations rather than statistically supported results.
- [Section 4.3 / Tables 2-14, Figure 8] The reasoning taxonomy is assigned by the authors without a coding rubric, inter-rater reliability check, or quantitative grounding. The radar chart in Figure 8 has no numerical basis in the text, and the reported "progression from causal misinterpretation in smaller models to cohesive, temporally aware reasoning in larger ones" is supported only by selected examples. This is load-bearing for the interpretability claims. Please provide a explicit rubric, blind coding by multiple annotators with agreement statistics (e.g., Cohen's kappa), and preferably classification counts across all collected explanations rather than illustrative quotes.
- [Section 3.2 / System prompt] The system prompt explicitly states the physical semantics and a control rule: "The fan will make T smaller, the heater will make T larger. Let e = T−Tsp and use smaller control values when |e| is small." This hands the models both the actuator causal structure and a proportional-control heuristic. Consequently, the observation that high-complexity models give "coherent explanations aligned with physical principles" may in large part reflect the model reciting or following the prompt, rather than exhibiting emergent physical reasoning. This does not invalidate the engineering value of prompt-grounding, but it weakens the paper's broader claim about model understanding. Please discuss this explicitly and separate 'ability to follow explicitly stated semantics' from 'ability to infer or reason about physics'.
- [Section 4.2 / Figure 7] The tool comparison is also presented without quantitative summary. The text describes "smoother and more stable" behavior and lower heater duty cycle, but no MAE, overshoot count, or actuator-usage numbers are given for tool-on vs tool-off for either model. Even under the oracle assumption, the magnitude of the claimed improvement is therefore not established. Please add a small table with these metrics for both conditions.
minor comments (5)
- [Throughout] The prompt text repeatedly writes "JASON" instead of "JSON" (e.g., in the system prompt block). This appears to be a typo and should be corrected.
- [Table 1] The table has "V ocabulary" and "Unkonwn" (should be "Vocabulary" and "Unknown"). Also, the GPT-4o reference [22] is the GPT-4 Technical Report, not a GPT-4o-specific reference; please cite the appropriate source if one exists.
- [Appendix] The "MPC.txt" example in the appendix is presented without context. It appears to be an illustrative prompt with an SQL-style tool trace, but it is not clearly connected to the main experimental workflow. Please either integrate it into the main text or explain its role.
- [Section 5 / Conclusion] The bullet "Does incorporating prior model-based knowledge improve control performance?" answers affirmatively, but given the oracle-tool circularity noted in Major Comment 1, the answer should be qualified to the exact-model case. The future-work paragraph already lists imperfect model parameters and sensor noise; these are precisely what the current experiment omits.
- [Figures 3-5] The multi-panel figures are dense; the caption does not state which subplot corresponds to which model or condition. Adding clear panel labels and a legend consistent with the text would improve readability.
Circularity Check
Tool-assisted improvement is guaranteed by construction: the plant and the physics-based prediction tool are the same fitted Eq. 1, reducing the third-axis result to oracle access.
-
self definitional
[Sections 3.2–3.3 and 4.2 (tool-assisted experiment; Eq. 1)]
"In the tool-assisted setting, the models can call a forward simulator to evaluate candidate action sequences over a 10-minute planning horizon ... The physics-based model is based on Equation 1. ... these new values resulted in improved model performance. ... This message feature has been used to provide the model with continuous updates of values from the simulated model ... uses the predictive model to simulate the outcome of candidate control sequences before acting, yielding smoother and more stable temperature tracking."
The closed-loop plant is the simulated model (Sec. 3.1), which is the fitted PBM of Eq. 1 with H'_max and F'_max estimated from the measured dataset (Sec. 3.3). The 'prediction tool' is a forward simulator of the same Eq. 1 with the same fitted constants. Therefore the tool's predicted temperature profile is the plant's exact trajectory by construction, i.e., an oracle. The reported improvement in smoothness and energy efficiency is a logical consequence of consulting the plant's own equations, not an empirical demonstration that LLMs incorporate domain knowledge under model uncertainty or plant-model mismatch. The third-axis conclusion reduces to the definition of the tool.
full rationale
The core scale-comparison experiments (Sec. 4.1) are self-contained: LLM outputs are fed to the simulated plant and MAE/actuator usage are measured directly; no fitted parameter is relabeled as a prediction there. The reasoning taxonomy is subjective but not circular. However, the tool-assisted axis is partially circular: the plant and the physics-based prediction tool are the same Equation 1 with the same fitted parameters, so the tool is an exact oracle of the plant. The improvement with the tool is therefore guaranteed whenever the LLM consults it and does not test grounding under mismatch, sensor noise, or uncertainty. This reduces one headline claim to a construction. Self-citations [15,16] are background and not load-bearing. No other circular steps were found.
Assumptions & free parameters
free parameters (3)
- H'_max (heater power) =
not reported numerically in text
- F'_max (fan airflow) =
not reported numerically in text
- Planning horizon (HORIZON_MIN) =
10 minutes
assumptions (3)
- domain assumption Energy balance ODE (Eq. 1) describes the enclosure temperature dynamics
- domain assumption Fitted PBM parameters from the same dataset are representative of the evaluation scenario
- domain assumption A single run per model/scenario is representative of model behavior
Cite this review
Pith. "Pith review of Evaluating LLMs as Interpretable Controllers for Dynamical Systems." pith.science (2026). https://pith.science/paper/53GC5IMX
@misc{pith2026260722609,
author = {Pith},
title = {Pith review of: Evaluating LLMs as Interpretable Controllers for Dynamical Systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/53GC5IMX}},
note = {Machine review of arXiv:2607.22609}
}
read the original abstract
Large Language Models (LLMs) are increasingly used for decision-making and reasoning tasks, yet their potential as controllers for physical systems remains largely unexplored. This work investigates whether LLMs can function as interpretable controllers for a dynamic thermal environment, examining their ability to follow setpoints, interpret natural-language commands, reason about actuator effects, and incorporate prior model-based knowledge. Five LLMs of varying scales are evaluated under multiple scenarios, including settings with penalties on heater or fan usage and cases where the models have access to a physics-based prediction tool. The results show that control performance depends on model complexity: while low- and mid-scale models frequently misinterpret actuator dynamics or generate inconsistent reasoning, high-complexity models such as Qwen-3~14B and GPT-4o achieve accurate temperature tracking, stable actuator usage, and coherent explanations aligned with physical principles. Incorporating a physics-based model significantly improves control smoothness and energy efficiency by enabling anticipatory decision-making. A detailed reasoning taxonomy further reveals a clear progression from causal misinterpretation in smaller models to cohesive and temporally aware reasoning in larger ones. The findings demonstrate that LLMs can act as interpretable controllers when sufficiently capable and appropriately grounded in domain knowledge, highlighting promising opportunities for hybrid model-based and language-driven control strategies that can provide plausible explanations.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin, Attention Is All You Need, arXiv:1706.03762 (2023)
arXiv 2023
-
[2]
Ichter, A
B. Ichter, A. Brohan, Y . Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian, D. Kalashnikov, S. Levine, Y . Lu, C. Parada, K. Rao, P. Sermanet, A. T. Toshev, V . Vanhoucke, F. Xia, T. Xiao, P. Xu, M. Yan, N. Brown, M. Ahn, O. Cortes, N. Sievers, C. Tan, S. Xu, D. Reyes, J. Rettinghouse, J. Quiambao, P. Pastor, L. Luu,...
2023
-
[3]
Zitkovich, T
B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, Q. Vuong, V . Vanhoucke, H. Tran, R. Soricut, A. Singh, J. Singh, P. Sermanet, P. R. Sanketi, G. Salazar, M. S. Ryoo, K. Reymann, K. Rao, K. Pertsch, I. Mordatch, H. Michalewski, Y . Lu, S. Levine, L. Lee, T.-W. E. Lee, I. Leal, Y . Kuang, D. Kalashnikov, R. Julia...
2023
-
[4]
Huang, F
W. Huang, F. Xia, T. Xiao, H. Chan, J. Liang, P. Florence, A. Zeng, J. Tompson, I. Mordatch, Y . Chebotar, P. Ser- manet, T. Jackson, N. Brown, L. Luu, S. Levine, K. Haus- man, B. Ichter, Inner Monologue: Embodied Reasoning through Planning with Language Models, inProceedings of The 6th Conference on Robot Learning, PMLR, 2023, pp. 1769–1782, ISSN: 2640-3498
2023
-
[5]
M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brock- man, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Win- ter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbe...
arXiv 2021
- [6]
-
[7]
E. O. Pyzer-Knapp, M. Manica, P. Staar, L. Morin, P. Ruch, T. Laino, J. R. Smith, A. Curioni, Foundation models for materials discovery – current state and future directions, npj Computational Materials, 11:61 (2025), Publisher: Na- ture Publishing Group
2025
-
[8]
Singhal, S
K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl, P. Payne, M. Seneviratne, P. Gamble, C. Kelly, A. Babiker, N. Schärli, A. Chowdhery, P. Mansfield, D. Demner- Fushman, B. Agüera y Arcas, D. Webster, G. S. Cor- rado, Y . Matias, K. Chou, J. Gottweis, N. Tomasev, Y . Liu, A. Rajkomar, J. Barral...
2023
Show all 25 references
-
[9]
Huang, P
W. Huang, P. Abbeel, D. Pathak, I. Mordatch, Lan- guage Models as Zero-Shot Planners: Extracting Action- able Knowledge for Embodied Agents, inProceedings of the 39th International Conference on Machine Learning, PMLR, 2022, pp. 9118–9147, ISSN: 2640-3498
2022
-
[10]
C. H. Song, B. M. Sadler, J. Wu, W.-L. Chao, C. Washing- ton, Y . Su, LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models, in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 2986–2997, ISSN: 2380-7504
2023
-
[11]
Stiennon, L
N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. V oss, A. Radford, D. Amodei, P. F. Christiano, Learn- ing to summarize with human feedback, inAdvances in Neural Information Processing Systems, vol. 33, Curran Associates, Inc., 2020, pp. 3008–3021
2020
-
[12]
H. Hu, D. Sadigh, Language Instructed Reinforcement Learning for Human-AI Coordination, inProceedings of the 40th International Conference on Machine Learning, PMLR, 2023, pp. 13584–13598, ISSN: 2640-3498
2023
-
[13]
J. Lin, Y . Du, O. Watkins, D. Hafner, P. Abbeel, D. Klein, A. Dragan, Learning to Model the World with Language, arXiv:2308.01399 [cs] (2024)
2024 arXiv
-
[14]
J. Wang, E. Shi, H. Hu, C. Ma, Y . Liu, X. Wang, Y . Yao, X. Liu, B. Ge, S. Zhang, Large language models for robotics: Opportunities, challenges, and perspectives,Jour- nal of Automation and Intelligence, 4:52–64 (2025)
2025
-
[15]
Rasheed, O
A. Rasheed, O. Ravik, O. San, Large Language Models for Control, arXiv:2511.00337 [eess] (2025)
2025
-
[16]
Rasheed, O
A. Rasheed, O. Ravik, O. San, Hybrid Modeling, Sim-to- Real Reinforcement Learning, and Large Language Model Driven Control for Digital Twins, arXiv:2510.23882 [cs] (2025)
2025
-
[17]
Turpin, J
M. Turpin, J. Michael, E. Perez, S. R. Bowman, Lan- guage Models Don’t Always Say What They Think: Un- faithful Explanations in Chain-of-Thought Prompting, arXiv:2305.04388 [cs] (2023)
2023 arXiv
-
[18]
G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Riv- ière, L. Rouillard, T. Mesnard, G. Cideron, J.-b. Grill, S. Ramos, E. Yvinec, M. Casbon, E. Pot, I. Penchev, G. Liu, F. Visin, K. Kenealy, L. Beyer, X. Zhai, A. Tsit...
2025 arXiv
-
[19]
Abouelenin, A
Microsoft, A. Abouelenin, A. Ashfaq, A. Atkinson, H. Awadalla, N. Bach, J. Bao, A. Benhaim, M. Cai, V . Chaudhary, C. Chen, D. Chen, D. Chen, J. Chen, W. Chen, Y .-C. Chen, Y .-l. Chen, Q. Dai, X. Dai, R. Fan, M. Gao, M. Gao, A. Garg, A. Goswami, J. Hao, A. Hendy, Y . Hu, X. J...
2025 arXiv
-
[20]
A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, W. E. Sayed, Mistral 7B (2023)
2023
-
[21]
A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, 12 B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M...
2025 arXiv
-
[22]
OpenAI, GPT-4 Technical Report, arXiv:2303.08774 [cs] (2024)
2024 arXiv
-
[23]
Hinton, O
G. Hinton, O. Vinyals, J. Dean, Distilling the Knowledge in a Neural Network, arXiv:1503.02531 [cs, stat] (2015)
2015 arXiv
-
[24]
J. Yang, Z. Wang, Y . Lin, Z. Zhao, Problematic Tokens: Tokenizer Bias in Large Language Models, in2024 IEEE International Conference on Big Data (BigData), IEEE Computer Society, Washington, DC, USA, 2024, pp. 6387– 6393, arXiv:2406.11214 [cs] version: 3
2024 arXiv
-
[25]
Virtanen, R
P. Virtanen, R. Gommers, T. E. Oliphant, M. Haber- land, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wil- son, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, I. Polat, Y . ...
2020
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.