Enhancing LLMs for Power System Simulations: A Feedback-driven Multi-agent Framework

Gabriela Hug; Mengshuo Jia; Zeyu Cui

REVIEW 4 major objections 7 minor 1 cited by

Enhancing LLMs for Power System Simulations: A Feedback-driven Multi-agent Framework

T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A feedback-driven multi-agent framework lets GPT-4o write executable power-system simulations with 93–97% success on 69 tasks, up from below 30% for baselines.

desk verdict A well-engineered, unusually honest agentic-RAG paper whose headline success rates are single-run point estimates on a self-built benchmark; worth serious review. read the letter →

arxiv 2411.16707 v3 pith:BDWRIS2L submitted 2024-11-21 cs.CL cs.AIcs.MAcs.SYeess.SY

Mengshuo Jia , Zeyu Cui , Gabriela Hug This is my paper · ORCID

classification cs.CLcs.AIcs.MAcs.SYeess.SY

keywords powersystemsimulationlargelanguagemodelsmulti-agentframeworkretrieval-augmentedgenerationchain-of-thoughtpromptingerrorfeedbackMATPOWERDALINE

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Power-system simulations are a bottleneck for LLM-based research assistants because simulation code demands precise function names, option values, and dependencies that models rarely memorize. This paper argues that the fix is not more model parameters or fine-tuning but a feedback-driven multi-agent framework: an enhanced retrieval module that decomposes a natural-language simulation request into function-related and option-related sub-queries, a reasoning module that uses few-shot chain-of-thought and retrieved knowledge to write code, and an acting module that executes the code, reads error messages, and loops corrections back into retrieval and reasoning. On 69 tasks across DALINE (a tool the model has never seen) and MATPOWER (a familiar tool), the framework reports success rates of 93.13% and 96.85%, respectively, while ChatGPT 4o, o1-preview, and fine-tuned GPT-4o all stay below 30% on complex tasks. If the result holds, researchers could delegate simulation implementation to natural-language assistants at roughly 30 seconds and 0.014 USD per task.

What carries the argument

The load-bearing object is the triple-based structured option document, built by parsing a tool's manual with an LLM and expert review, in which each entry links an option's name/default format to its dependent function and to its description and value choices. Around this document the framework runs three interacting agents: a retrieval agent that splits a request into function- and option-related sub-queries via few-shot chain-of-thought; a coding agent prompted with static tool knowledge, retrieved details, and a four-step reasoning path; and an environmental acting module that executes code, detects errors, and feeds structured error reports back into retrieval and reasoning. The loop continues until the code runs cleanly or the attempt budget is exhausted.

What would settle it

Apply the same framework to a simulation tool whose interface is not reducible to a function–option list (for example, a GUI-only or equation-heavy tool) or whose error reporting is vague; if success does not fall far below the reported 93–97%, the paper's claim that documentation structure and error-reporting quality are load-bearing would be called into question. A simpler check: rerun GPT4o-Full on the same 69 tasks with the triple-based document replaced by the raw manual while keeping all other modules, and compare against the reported 81.37% (DALINE) figure for the scheme missing that document.

Watch

Extended reading notes

Core claim

The paper's central claim is that the key to LLM-based simulation is correctly identifying and combining functions and options, and that this can be achieved by structured retrieval plus iterative execution feedback. Standard RAG, which retrieves on the whole request, conflates function and option information; supervised fine-tuning compresses away the fine-grained parameter details coding requires. The proposed framework instead splits each request into function-related and option-related sub-queries, maps them to precise keywords, and retrieves from a triple-based option document that records each option's name and default format, the function it depends on, and its description and value choices. The coding agent then follows a universal four-step reasoning path—identify functions, learn syntax, extract option information, generate code—and the environmental acting module runs the code and feeds error reports back into the loop. The reported outcome is that GPT-4o achieves 93.13% success on DALINE and 96.85% on MATPOWER, with complex and standard tasks nearly closing their performance gap.

Load-bearing premise

The framework assumes that a simulation tool's usability can be captured by a structured document of functions and options derived from its manual, and that the tool returns error messages detailed enough for the feedback loop to correct mistakes.

Editorial extensions

If this is right

A natural-language interface to power-system simulation is feasible now: a researcher can describe a task and receive executable code plus a simulation result.
The success depends on the cumulative combination of query planning, the triple-based knowledge base, few-shot chain-of-thought, and feedback; dropping any one of these visibly lowers success rates.
Supervised fine-tuning is not a substitute: even when trained on the full test set, the fine-tuned model stays below 60%, whereas the retrieval-based framework exceeds 95%.
Simulation tools with weak error reporting will need layered checking or external code-extraction strategies, because the feedback loop's power depends on error-message quality.
Because the function/option decomposition is tool-independent, the framework can be ported to newly released simulation tools without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same function/option decomposition could transfer to other engineering simulation ecosystems, such as circuit or chemical-process simulators, where manuals likewise define functions and options.
A testable extension is to add an LLM-based code-checking agent that inspects successfully executed code for wrong-but-runnable outputs, addressing the non-execution-bug failures the paper flags as open.
The framework's success suggests a general recipe for code generation on specialized tools: retrieve structured relational facts about API elements rather than raw manual prose.
Coupling the feedback loop with an interactive clarification stage could extend the framework to underspecified or ambiguous requests, which the paper identifies as a remaining challenge.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take. The framework is real engineering—the results come from executing code in actual simulators—and the paper is more honest than most. The genuinely new bit is the adaptive query planning that splits a simulation request into function and option sub-queries, plus the triple-based option document that encodes option-function dependencies. The combination with few-shot CoT and an error-feedback loop is well described, and the ablation is unusually systematic: thirteen schemes, first/final attempt breakdowns, an extended 50-attempt budget, and an SFT comparison. I thought the SFT diagnostic was actually clever—the authors fine-tune on the test tasks and still lose to retrieval, which supports their 'retrieval beats memorization for precise API details' argument. The paper also flags its own limits: Remark 2 admits the whole approach assumes documentation exists, Remark 4 discusses weak error-reporting tools, and the conclusion acknowledges non-execution-bug failures.

The soft spots are the usual ones for this genre, and one is load-bearing. The headline numbers—93.13% and 96.85%—are single-run point estimates on 34 and 35 tasks, with no variance or confidence intervals. The stress-test note is right: at those task counts a couple of flipped outcomes move the aggregate by several points. I agree that the missing repeated-run quantification is more load-bearing than the functions/options schema worry. Given GPT-4o stochasticity, the exact figures in the abstract should not be treated as established until repeated runs or bounds are reported. Second, the evaluation is self-referential in a way that matters: DALINE is the authors' own toolbox, and the tasks, knowledge documents, prompts, and scoring all come from the same group. The 'unseen tool' claim is about LLM pretraining, not evaluator neutrality. That doesn't invalidate the result, but it does mean the adaptability claim rests on two tools, one of which is in-house. Third, the scoring is manual by the authors, with no inter-rater reliability check. Minor: no code or data released yet, and [10] and [18] are the same paper.

Net: this is a solid engineering contribution for LLM-based scientific computing and power-system assistants. The central argument holds qualitatively; I just wouldn't defend the precise success rates yet. Send it to serious peer review—it deserves referee time—but ask for repeated runs, confidence bounds, and open artifacts. With those, this is a conditional accept.

Referee Report

4 major / 7 minor

Summary. The paper proposes a feedback-driven multi-agent framework to help LLMs write executable power-system simulation code. The framework combines an enhanced RAG module with adaptive query planning and a triple-based knowledge base, a few-shot chain-of-thought reasoning module, and an environmental acting module that executes code and iteratively corrects it from error messages. The authors evaluate the framework on 34 DALINE tasks and 35 MATPOWER tasks, reporting success rates of 93.13% and 96.85% for GPT4o-Full, compared with much lower rates for ChatGPT-4o, o1-preview, standard RAG variants, and a supervised fine-tuned model. They also report per-task execution time of about 30 seconds and token cost of about 0.014 USD.

Significance. If the reported success rates are reliable, the framework would be a practically useful contribution: it demonstrates that a modular combination of retrieval, structured reasoning, and error feedback can enable a general LLM to operate both a newly released tool (DALINE) and a well-established tool (MATPOWER) at near-perfect accuracy on a diverse task suite. The evaluation is grounded in actual execution in the simulators rather than static code inspection, which is a genuine strength, as are the explicit ablations across framework components and the cost analysis. The promise to release prompts, knowledge bases, tasks, and generated code further supports reproducibility if fulfilled.

major comments (4)

[Section V.A, Eqs. (1)-(3)] The reported "success rate" is not a conventional success rate. Because unused attempts are assigned the last executed score, a task solved on the first attempt contributes 100×N_max^t points, whereas the same task solved only on the last attempt contributes only 100 points. With N_max=3 for DALINE and N_max=5 for MATPOWER, early success is weighted 3× and 5× more heavily than late success. Thus the headline "93.13%" and "96.85%" values conflate correctness with iteration efficiency and are not directly comparable across the two environments. Please report the per-task binary success rate (solved at least once within the attempt budget) as a primary metric, or explicitly rename the current measure an "attempt-weighted score" and justify its use for the abstract's claim.
[Section V.A, Figs. 8 and 12] All reported success rates are single-run point estimates with no repeated runs, no random seeds, and no confidence intervals. Since per-attempt scores are essentially ternary (0/50/100) and the task counts are only 34 and 35, one task flipping from 100 to 0 changes the aggregate by roughly 2.9 percentage points, and two flips would move the MATPOWER rate from 96.85% below 92%. LLM API outputs are stochastic even at temperature 0.1, and the paper itself shows model-version sensitivity (main results use gpt-4o-2024-05-13, while Section V.D uses gpt-4o-2024-08-06 with a different DALINE success rate for the same scheme). The near-perfect quantitative claim in the abstract is therefore not yet established. Please provide multiple independent runs or bootstrap confidence intervals, and include per-task score tables so readers can recompute the aggregates.
[Section V.D, DALINE experiments] The DALINE SFT comparison uses the full testing dataset as the SFT training dataset ("all 34 testing simulation tasks, augmented with 16 additional tasks, were employed for fine-tuning"). This means the SFT model is not being evaluated on unseen tasks. The experiment may show that SFT cannot memorize the training set, but it does not support the broader claim that SFT underperforms the framework on equal terms. In addition, these experiments use gpt-4o-2024-08-06 for all schemes, while the main DALINE results in Figs. 7–9 use gpt-4o-2024-05-13, and Table III reports GPT4o-Full at 95.098% versus 93.13% in Fig. 8. The version sensitivity should be acknowledged and bounded.
[Section V.A, scoring] The paper states that "the scoring was conducted manually by human experts during the evaluation process," but no inter-annotator agreement, blinded evaluation, or detailed scoring protocol is reported. Because the task set, knowledge bases, prompts, and rubric were all designed by the same group, the quantitative results should be accompanied by the raw per-task score sheets and the generated code and benchmark outputs for independent verification; ideally a second annotator should score a subset of tasks.

minor comments (7)

[Throughout] The name of the DALINE toolbox is inconsistently rendered as "DALINE", "Daline", and "D ALINE" in footnotes, captions, and text; please standardize.
[Table I] The notation "CGPT4o R" is not expanded in the table or its caption; please define it explicitly (presumably ChatGPT with OpenAI's built-in RAG).
[Section V.A] The scoring equations use N_max^(t) before it is formally introduced; please define all notation before the equations.
[Footnotes and Table II] Several hrefs contain unresolved "[ here]" placeholders (e.g., the DALINE/MATPOWER links in Section V and the embedding-model link in Table II); provide the actual URLs.
[References] Reference [18] duplicates reference [10]; please merge or renumber.
[Conclusion] The list of key findings includes two items labeled "(iv)"; renumber the items sequentially.
[Table IV and abstract] Table IV shows per-task expenses of 0.014 USD (DALINE) and 0.013 USD (MATPOWER), while the abstract says "an average cost of 0.014 USD"; clarify the rounding and averaging method.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline success rates are grounded in external simulator execution, and no fitted parameter or self-citation is renamed as a prediction.

full rationale

The paper's central claim is empirical: GPT4o-Full achieves 93.13% and 96.85% success on 34 DALINE and 35 MATPOWER tasks. These numbers are produced by executing generated code in actual simulation environments, not by a derivation that reduces to its inputs. No equation in the paper defines a predicted quantity in terms of the same quantity used as input. The RAG module retrieves function/option documentation, the reasoning module writes code, and the environmental module executes that code and iterates on error messages; the success rate is then scored by human experts against task requirements. This is an external, falsifiable evaluation. The DALINE tool and its manual are authored by the same group, and the evaluation tasks are also self-constructed, but this is a benchmarking-choice limitation rather than circularity: the simulator itself is an independent execution target, and the paper's claims are not derived from those artifacts by construction. The comparison with SFT, including the control where the full test set is used for fine-tuning, further shows that the framework's advantage is not simply a restatement of the training data. The admitted limitation in Remark 2 — that the framework assumes tools are learnable from documentation — is an assumption about scope, not a circular step. Similarly, the dependence on well-developed error-reporting systems, evidenced by the GPT4o-RSRNW drop to 78.43%, is an empirical boundary condition. The absence of repeated runs and confidence intervals concerns statistical robustness of the point estimates, not logical circularity. Overall, no load-bearing step reduces to its own inputs, and no cited result is invoked as a substitute for the framework's empirical validation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 2 invented entities

The central claim rests on hand-chosen configuration parameters and untested domain assumptions. The framework introduces no physical entities, but it does introduce two agent roles implemented via LLM calls. The evaluation's validity depends on assumptions about documentation learnability, the functions/options decomposition, the accuracy of the generated knowledge document, and the representativeness of the self-designed tasks.

free parameters (5)

return_num = 20
Number of top relevant chunks returned per retrieval query (Table II); chosen by hand, affects retrieval quality and cost.
chunk_size = 30 words for .txt, 50 words for .pdf
Chunking sizes for the knowledge base (Table II); affect retrieval granularity and the content available to the coding agent.
temperature = 0.1
Sampling temperature for all LLM calls (Table II); reduces randomness but does not eliminate run-to-run variance.
max_attempts = 3 for DALINE, 5 for MATPOWER
Stopping criteria per task (Table II); the success-rate metric depends heavily on this because unused attempts inherit the last executed score.
SFT hyperparameters = 3 epochs, batch size 1, learning rate multiplier 2
Used only in the SFT comparison; chosen via OpenAI defaults/recommendations, not tuned to the test set, but they affect the fairness of the comparison.

assumptions (5)

domain assumption Simulation tools are learnable by humans through sufficient documentation.
Remark 2 states the framework is built on this premise; if false for a given tool, the RAG and reasoning modules lack a basis.
domain assumption All simulation coding knowledge can be decomposed into functions and options with logical dependencies.
Section II-A and III-B; the adaptive query planning and triple-based knowledge base rely on this schema.
domain assumption The triple-based option document generated by ChatGPT-4o and reviewed by domain experts is accurate and complete.
Section II-B Step 4; errors in this document propagate directly to retrieved knowledge and generated code.
ad hoc to paper The 69 tasks and manual scoring rubric are a valid operationalization of successful simulation performance.
Section V-A; scoring was conducted manually by human experts and the tasks were designed by the authors, so this validity assumption is not externally established.
domain assumption GPT-4o API behavior is stable across calls.
The evaluation uses single runs at temperature 0.1; no repeated sampling or variance is reported (Table II, Section V).

invented entities (2)

Retrieval agent
purpose: Performs semantic recognition and keyword mapping to generate function and option retrieval queries from simulation requests and error reports.
A role assigned to a general LLM; no standalone artifact or external falsifiable handle is provided beyond the reported experiments.
Coding agent
purpose: Generates and revises simulation code using static knowledge, retrieved knowledge, and error feedback.
A role assigned to an LLM; depends on the closed API and the authors' prompts, so it is not independently verifiable without released artifacts.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing LLMs for Power System Simulations: A Feedback-driven Multi-agent Framework." pith.science (2026). https://pith.science/paper/BDWRIS2L

@misc{pith2026241116707,
  author       = {Pith},
  title        = {Pith review of: Enhancing LLMs for Power System Simulations: A Feedback-driven Multi-agent Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BDWRIS2L}},
  note         = {Machine review of arXiv:2411.16707}
}

read the original abstract

The integration of experimental technologies with large language models (LLMs) is transforming scientific research. It positions AI as a versatile research assistant rather than a mere problem-solving tool. In the field of power systems, however, managing simulations -- one of the essential experimental technologies -- remains a challenge for LLMs due to their limited domain-specific knowledge, restricted reasoning capabilities, and imprecise handling of simulation parameters. To address these limitations, this paper proposes a feedback-driven, multi-agent framework. It incorporates three proposed modules: an enhanced retrieval-augmented generation (RAG) module, an improved reasoning module, and a dynamic environmental acting module with an error-feedback mechanism. Validated on 69 diverse tasks from Daline and MATPOWER, this framework achieves success rates of 93.13% and 96.85%, respectively. It significantly outperforms ChatGPT 4o, o1-preview, and the fine-tuned GPT-4o, which all achieved a success rate lower than 30% on complex tasks. Additionally, the proposed framework also supports rapid, cost-effective task execution, completing each simulation in approximately 30 seconds at an average cost of 0.014 USD for tokens. Overall, this adaptable framework lays a foundation for developing intelligent LLM-based assistants for human researchers, facilitating power system research and beyond.

Figures

Figures reproduced from arXiv: 2411.16707 by the authors.

**Figure 2.** Enhanced RAG module for simulation tasks. (a) The retrieval agent decomposes simulation requests into function-related and option-related sub-queries, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** General RAG diagram, including the process of external knowledge [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (13 more)

**Figure 4.** Figure 4: Enhanced reasoning module for simulation code generation: (a) Core Concept, defining the coding agent’s role, assigned tasks, reasoning path, and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Environmental acting module with an error-feedback mechanism that integrates with both the RAG and reasoning modules. TABLE I: Evaluated schemes by distinct combinations of the proposed strategies; a checkmark indicates inclusion of the corresponding strategy. (GPT4o: …

**Figure 6.** Figure 6: Representative examples of simulation tasks for D [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 7.** Figure 7: Distribution of scores achieved across attempts for each evaluated scheme. For the definitions of scheme notations (e.g., GPT4o-Full), refer to Table I; the notations apply hereafter. (simulation environment: DALINE; GPT4o version: gpt-4o-2024-05-13). GPT4o Full GPT4o …

**Figure 8.** Figure 8: Success rates for each scheme. “All” refers to the aggregated success rate across all tasks; “All-Complex” and “All-Standard” report the success rates calculated exclusively for complex and standard tasks, respectively. “First Attempt” represents the success rate achie…

**Figure 9.** Figure 9: Distribution of feedback loops (attempts) required for successful task completion in complex (left) and standard (right) tasks. (simulation environment: DALINE; GPT4o version: gpt-4o-2024-05-13) [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

**Figure 10.** Figure 10: Scores achieved by each evaluated scheme in individual attempts when handling complex tasks. The automatic error correction mechanism aids [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]

**Figure 11.** Figure 11: Distribution of scores achieved across attempts for each evaluated scheme, separated by complex and standard tasks [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]

**Figure 12.** Figure 12: Success rates for each scheme, broken down by all tasks combined, [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]

**Figure 13.** Figure 13: Distribution of feedback loops (attempts) required for successful task completion in complex (upper) and standard (lower) tasks. (simulation environment: MATPOWER; GPT4o version: gpt-4o-2024-05-13; o1p-Sole is only tested by complex tasks). This retrieval mechanism ef…

**Figure 14.** Figure 14: Scores achieved by each evaluated scheme in individual attempts when handling standard tasks (simulation environment: MATPOWER; GPT4o version: gpt-4o-2024-08-06). GPT4o Full GPT4o Sole SFT GPT4o Sole 0 50 100 Score of Each Attempt Complex Tasks GPT4o Full GPT4o Sole S…

**Figure 15.** Figure 15: Distribution of scores achieved across attempts for each evaluated scheme (simulation environment: DALINE; GPT4o version: gpt-4o-2024-08- 06). achieve over 95% accuracy across both complex and standard tasks. E. Evaluation under Increased Attempt Budget To further ana…

**Figure 16.** Figure 16: Scores achieved by each evaluated scheme in individual attempts when handling tasks with different difficulty levels; the bar on the right represents the score scale, where 100, 50, and 0 indicate different score values (simulation environment: MATPOWER; GPT4o version…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Large Language Model-Empowered Interactive Load Forecasting
cs.LG 2025-05 conditional novelty 5.0 of 10

A multi-agent LLM framework enables interactive, natural-language steering of load forecasting pipelines and reports improved accuracy when users provide informed guidance.

Reference graph

Works this paper leans on

27 extracted references · 19 canonical work pages · cited by 1 Pith paper

[18]

Large foundation models for power systems,

C. Huang, S. Li, R. Liu, H. Wang, and Y . Chen, “Large foundation models for power systems,” in 2024 IEEE Power & Energy Society General Meeting (PESGM) . IEEE, 2024, pp. 1–5

2024
[1]

Autonomous chemical research with large language models,

D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes, “Autonomous chemical research with large language models,” Nature, vol. 624, no. 7992, pp. 570–578, 2023

2023
[2]

Mathematical discoveries from program search with large language models,

B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi et al. , “Mathematical discoveries from program search with large language models,” Nature, vol. 625, no. 7995, pp. 468–475, 2024

2024
[3]

Solving olympiad geometry without human demonstrations,

T. H. Trinh, Y . Wu, Q. V . Le, H. He, and T. Luong, “Solving olympiad geometry without human demonstrations,” Nature, vol. 625, no. 7995, pp. 476–482, 2024

work page 2024
[4]

Large language models streamline automated machine learning for clinical studies,

S. Tayebi Arasteh, T. Han, M. Lotfinia, C. Kuhl, J. N. Kather, D. Truhn, and S. Nebelung, “Large language models streamline automated machine learning for clinical studies,” Nature Communications, vol. 15, no. 1, p. 1603, 2024

work page 2024
[5]

On the potential of chatgpt to generate distribution systems for load flow studies using opendss,

R. S. Bonadia, F. C. Trindade, W. Freitas, and B. Venkatesh, “On the potential of chatgpt to generate distribution systems for load flow studies using opendss,” IEEE Transactions on Power Systems , 2023

work page 2023
[6]

Exploring the capabilities and limitations of large language models in the electric energy sector,

S. Majumder, L. Dong, F. Doudi, Y . Cai, C. Tian, D. Kalathil, K. Ding, A. A. Thatte, N. Li, and L. Xie, “Exploring the capabilities and limitations of large language models in the electric energy sector,” Joule, vol. 8, no. 6, pp. 1544–1549, 2024

2024
[7]

How do large language models acquire factual knowledge during pretraining?

H. Chang, J. Park, S. Ye, S. Yang, Y . Seo, D.-S. Chang, and M. Seo, “How do large language models acquire factual knowledge during pretraining?” in Advances in Neural Information Processing Systems , 2024

work page 2024

Show all 27 references

[8]

Real-time optimal power flow with linguistic stipulations: integrating gpt-agent and deep reinforcement learning,

Z. Yan and Y . Xu, “Real-time optimal power flow with linguistic stipulations: integrating gpt-agent and deep reinforcement learning,” IEEE Transactions on Power Systems , 2023

2023
[9]

Large language model assisted optimal bidding of bess in fcas market: An ai-agent based approach,

B. Zhang, C. Li, G. Chen, and Z. Dong, “Large language model assisted optimal bidding of bess in fcas market: An ai-agent based approach,” arXiv preprint arXiv:2406.00974 , 2024

2024 arXiv
[11]

Carbon footprint accounting driven by large language models and retrieval- augmented generation,

H. Wang, Z. Chen, N. Shang, S. Yao, Z. Pan, F. Wen, and J. Zhao, “Carbon footprint accounting driven by large language models and retrieval- augmented generation,” arXiv preprint arXiv:2408.09713 , 2024

2024 arXiv
[12]

Chatgpt and other large language models for cybersecurity of smart grid applications,

A. Zaboli, S. L. Choi, T.-J. Song, and J. Hong, “Chatgpt and other large language models for cybersecurity of smart grid applications,” in 2024 IEEE Power & Energy Society General Meeting (PESGM) . IEEE, 2024, pp. 1–5

2024
[13]

From news to forecast: Integrating event analysis in llm-based time series forecasting with reflection,

X. Wang, M. Feng, J. Qiu, J. Gu, and J. Zhao, “From news to forecast: Integrating event analysis in llm-based time series forecasting with reflection,” Advances in Neural Information Processing Systems , vol. 37, pp. 58 118–58 153, 2025

2025
[14]

Global, regional, and local acceptance of solar power,

K. Nuortimo, J. Harkonen, and K. Breznik, “Global, regional, and local acceptance of solar power,” Renewable and Sustainable Energy Reviews , vol. 193, p. 114296, 2024

2024
[15]

Elecbench: a power dispatch evaluation benchmark for large language models,

X. Zhou, H. Zhao, Y . Cheng, Y . Cao, G. Liang, G. Liu, and J. Zhao, “Elecbench: a power dispatch evaluation benchmark for large language models,” arXiv preprint arXiv:2407.05365 , 2024

2024 arXiv
[16]

Applying large language models to power systems: Potential security threats,

J. Ruan, G. Liang, H. Zhao, G. Liu, X. Sun, J. Qiu, Z. Xu, F. Wen, and Z. Y . Dong, “Applying large language models to power systems: Potential security threats,” IEEE Transactions on Smart Grid , 2024

2024
[17]

Exploration of generative intelligent application mode for new power systems based on large language models,

D. Lifu, C. Ying, X. Tannan, H. Shaowei, and S. Chen, “Exploration of generative intelligent application mode for new power systems based on large language models,” Automation of Electric Power Systems , 2024. [Online]. Available: https://github.com/xxh0523/llm4power

2024
[19]

Retrieval-augmented generation for knowledge-intensive NLP tasks,

P. S. H. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W. Yih, T. Rockt ¨aschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Advances in Neural Information Processing Systems , 2020

2020
[20]

Enabling large language models to perform power system simulations with previously unseen tools: A case of daline,

M. Jia, Z. Cui, and G. Hug, “Enabling large language models to perform power system simulations with previously unseen tools: A case of daline,” arXiv preprint arXiv:2406.17215 , 2024

2024 arXiv
[21]

Chain-of-thought prompting elicits reasoning in large language models,

J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24 824–24 837, 2022

2022
[22]

Language models are few-shot learners,

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., “Language models are few-shot learners,” in Advances in neural information processing systems, vol. 33, 2020, pp. 1877–1901

2020
[23]

Daline: A data-driven power flow linearization toolbox for power systems research and education,

M. Jia, W. Y . Chan, and G. Hug, “Daline: A data-driven power flow linearization toolbox for power systems research and education,” 2024. [Online]. Available: https://doi.org/10.3929/ethz-b-000681867

2024 doi
[24]

Matpower: Steady-state operations, planning, and analysis tools for power systems research and education,

R. D. Zimmerman, C. E. Murillo-S ´anchez, and R. J. Thomas, “Matpower: Steady-state operations, planning, and analysis tools for power systems research and education,” IEEE Transactions on power systems , vol. 26, no. 1, pp. 12–19, 2010

2010
[25]

User manual for daline 1.1.5,

M. Jia, W. Y . Chan, and G. Hug, “User manual for daline 1.1.5,” 2024. [Online]. Available: https://doi.org/10.3929/ethz-b-000680438

2024 doi
[26]

Matpower 8.0 user’s manual,

R. D. Zimmerman and C. E. Murillo-S ´anchez, “Matpower 8.0 user’s manual,” 2024. [Online]. Available: https://matpower.org/docs/ MATPOWER-manual-8.0.pdf

2024
[27]

Finetuned language models are zero-shot learners,

J. Wei, M. Bosma, V . Y . Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V . Le, “Finetuned language models are zero-shot learners,” in 2022 International Conference on Learning Representations , 2022

2022
[28]

Chatgpt-4o,

OpenAI, “Chatgpt-4o,” language model used for language polishing in this manuscript. [Online]. Available: https://openai.com/

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [18]

Large foundation models for power systems,

C. Huang, S. Li, R. Liu, H. Wang, and Y . Chen, “Large foundation models for power systems,” in 2024 IEEE Power & Energy Society General Meeting (PESGM) . IEEE, 2024, pp. 1–5

2024

[2] [1]

Autonomous chemical research with large language models,

D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes, “Autonomous chemical research with large language models,” Nature, vol. 624, no. 7992, pp. 570–578, 2023

2023

[3] [2]

Mathematical discoveries from program search with large language models,

B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi et al. , “Mathematical discoveries from program search with large language models,” Nature, vol. 625, no. 7995, pp. 468–475, 2024

2024

[4] [3]

Solving olympiad geometry without human demonstrations,

T. H. Trinh, Y . Wu, Q. V . Le, H. He, and T. Luong, “Solving olympiad geometry without human demonstrations,” Nature, vol. 625, no. 7995, pp. 476–482, 2024

work page 2024

[5] [4]

Large language models streamline automated machine learning for clinical studies,

S. Tayebi Arasteh, T. Han, M. Lotfinia, C. Kuhl, J. N. Kather, D. Truhn, and S. Nebelung, “Large language models streamline automated machine learning for clinical studies,” Nature Communications, vol. 15, no. 1, p. 1603, 2024

work page 2024

[6] [5]

On the potential of chatgpt to generate distribution systems for load flow studies using opendss,

R. S. Bonadia, F. C. Trindade, W. Freitas, and B. Venkatesh, “On the potential of chatgpt to generate distribution systems for load flow studies using opendss,” IEEE Transactions on Power Systems , 2023

work page 2023

[7] [6]

Exploring the capabilities and limitations of large language models in the electric energy sector,

S. Majumder, L. Dong, F. Doudi, Y . Cai, C. Tian, D. Kalathil, K. Ding, A. A. Thatte, N. Li, and L. Xie, “Exploring the capabilities and limitations of large language models in the electric energy sector,” Joule, vol. 8, no. 6, pp. 1544–1549, 2024

2024

[8] [7]

How do large language models acquire factual knowledge during pretraining?

H. Chang, J. Park, S. Ye, S. Yang, Y . Seo, D.-S. Chang, and M. Seo, “How do large language models acquire factual knowledge during pretraining?” in Advances in Neural Information Processing Systems , 2024

work page 2024