Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysis

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that turning beam analysis into a code-generation task—with chain-of-thought, a complete worked example, and automatic execution of the generated solver code—makes the Llama-3.3 70B Instruct model accurate on…

desk verdict A clean, honest ablation shows the agent's >99% reliability is real but almost entirely inherited from a few-shot template that closely matches the test problems; the paper deserves peer review but should be asked to release artifacts and scope the claim. read the letter →

arxiv 2507.02938 v1 pith:LCTSLPYJ submitted 2025-06-27 cs.CL cs.AI

Jiachen Liu , Ziheng Geng , Ran Cao , Lu Cheng , Paolo Bocchini , Minghui Cheng This is my paper
classification cs.CLcs.AI
keywords largelanguagemodelsLLMagentsstructuralanalysisbeamstaticscodegenerationfew-shotpromptingchain-of-thoughtreliabilityandrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that an LLM can be made engineering-grade for elementary beam statics if it is told to write executable analysis code instead of producing calculations in prose. On a benchmark of eight beam problems with moving loads, the code-writing agent keeps reliability at or above 0.990 across every load location, with robustness between 0.996 and 1.000, while the same model answering directly falls to reliability as low as 0.060 on combined-load overhang cases. The core move is to hand the arithmetic to a finite-element solver and let the language model act only as translator from problem statements to code. The paper's own ablation shows that the margin comes almost entirely from the complete worked example and function usage examples in the prompt: removing the worked example collapses reliability to zero.

What carries the argument

The central object is the agent's five-part prompt template: role specification, chain-of-thought reasoning steps, one complete worked example with full solver code, several function usage examples showing node placement and load application, and prescriptive constraints that force mandatory code blocks. The agent executes the generated code automatically in a finite-element solver and visualizes the results. The complete example and function usage examples do most of the work, since removing the complete example drops reliability to 0.000 and removing the function usage examples drops it to between 0.026 and 0.623 on the extended tasks.

What would settle it

Give the agent beam problems whose support and load pattern cannot be matched to any example in its prompt—for instance, a beam with an internal hinge or an inclined point load—and measure reliability over 500 runs; if it stays near 1.0 the agent generalizes, and if it falls toward the raw model's levels, the claimed robustness is an artifact of the hand-selected example library.

Watch

Extended reading notes

Core claim

In the paper's own terms, the discovery is that reframing structural analysis from open-ended text generation to structured code generation converts a model with qualitative understanding but unreliable numerics into a consistent engineering tool. Reliability, defined as the fraction of correct outputs over 500 independent runs, reaches or exceeds 0.990 in every benchmark condition, and robustness, defined through the coefficient of variation of reliability across load positions, stays between 0.996 and 1.000; the same raw model scores as low as 0.060. The paper also establishes, through ablation, that the complete worked example and the function usage examples are the load-bearing components of this success.

Load-bearing premise

The near-perfect reliability numbers depend on each prompt carrying a complete worked example and function usage examples that closely match the target problem; the paper's own ablation shows that removing the complete example drops reliability to 0.000 on all three extended tasks.

Editorial extensions

If this is right

  • A language model that currently fails the hardest beam cases can be made engineering-grade for statically determinate beams by letting it write code instead of doing arithmetic.
  • The same code-generation shift should apply to a wider class of structural problems, because it removes arithmetic and sign conventions from the model's failure surface.
  • The ablation result is a deployment warning: any practical tool built this way must ship a library of worked examples covering each structural topology, since the complete example alone is worth the entire performance margin.
  • The two documented failure modes—hallucinated boundary conditions and misapplied load ranges—define exactly the cases a safety review must sample before release.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the complete example in the prompt is nearly identical to the benchmark tasks, the claimed 99 percent is best understood as the reliability of a code-template-adaptation pipeline, not of a general structural-analysis capability.
  • Testing on topologically novel beams—continuous beams with three supports, frames, or supports not matching any example—would likely drop reliability close to the raw model's level, since the removal experiments point to example matching as the source of accuracy.
  • The robustness metric averages over load positions; a deployment that needs certifiable per-case correctness should report the worst-case reliability position, not the mean.
  • A natural next step the paper does not test is execution-feedback self-correction: feeding the solver's error message back into the model could catch hallucinated supports and boundary conditions without larger example libraries.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper evaluates the reliability and robustness of Llama-3.3 70B Instruct on eight beam structural-analysis problems and proposes an LLM-empowered agent that reframes the task as OpenSeesPy code generation. The agent uses a structured prompt with role specification, chain-of-thought, a complete worked example, function-usage examples, and prescriptive constraints, then automatically executes the generated code. The authors report that the agent achieves reliability above 0.990 and robustness above 0.996 on the benchmark, outperforming the baseline LLM, and they present ablation results on three extended tasks showing that removing the complete example or the function-usage examples sharply degrades performance.

Significance. If the central claim were established, this would be a useful demonstration that a code-generation agent with a deterministic solver can make a general-purpose LLM reliable for a narrow class of structural-analysis problems. The paper has several strengths: it adopts a concrete reliability definition based on repeated sampling, uses a deterministic solver for the executed code, and includes an ablation study that honestly exposes which prompt components matter. The visualization module and the reframing of text output into executable code are sensible contributions. However, the paper's headline claim of an agent that is 'reliable and robust across diverse conditions' is not supported by the evidence as presented, because the high performance is inherited from few-shot exemplars that closely match the test distribution.

major comments (3)
  1. [Section 4.3, Table 1] The ablation results undermine the central claim that the agent is reliable and robust. Removing the complete example drops reliability to 0.000 on all three extended tasks, and removing the function-usage examples drops reliability to 0.026, 0.296, and 0.623. Since the complete example and function-usage examples in Section 3.3 (Fig. 7) explicitly demonstrate the same load configuration that dominates the benchmark and extended tasks—a distributed load overlapping a point load—the reported >99% reliability is only established for problems that are near-duplicates of the prompt's worked examples. The abstract and Section 4.1 should be revised to scope the claim to template-matched problems, or the agent should be evaluated on problems whose load/geometry patterns are not present in the prompt.
  2. [Section 2.2, Eq. (1); Section 4.1, Fig. 8] No confidence intervals or sampling parameters (temperature, seed, decoding strategy) are reported for the 500-run reliability estimates. For n=500, the binomial standard error is about 0.004 to 0.022 depending on the proportion; several differences between reported reliability values in Fig. 4 and Fig. 8 are comparable to or smaller than this noise. The statement in Section 4.1 that the agent 'significantly outperforms' the baseline therefore lacks statistical support. The authors should report the sampling configuration and provide confidence intervals or hypothesis tests for the reliability proportions.
  3. [Section 2.2, Eq. (2); Section 4.1] The robustness metric is computed from the coefficient of variation of reliability across load locations. For the agent, reliability is at or near 1.0 in almost every subplot of Fig. 8, so the robustness values (0.996 to 1.000) largely reflect the saturation of the reliability measure rather than a meaningful invariance property. The interpretation of 'robustness over 0.996' should be tempered, and the metric should be discussed in light of its degeneracy when reliability is near its upper bound.
minor comments (5)
  1. [Abstract] The abstract contains a typo: 'OpeeSeesPy' should read 'OpenSeesPy'.
  2. [Section 2.2] The heading 'Workflows to assess reliabity and robustness' contains a typo: 'reliabity' should be 'reliability'.
  3. [Section 2.1] The phrase 'as an first step' should be 'as a first step'.
  4. [Throughout] The model name is inconsistently spelled as 'Llama' and 'LLaMA'; please standardize.
  5. [Section 4.1, Fig. 8] The figure labels alternate between 'Reliability = 1.000' and 'Reliability ≥ 0.990' without explaining why the lower bound differs across subplots; a consistent annotation would improve clarity.

Circularity Check

2 steps flagged · score 6.0 of 10

The >99% reliability claim is an in-sample evaluation: the prompt is iteratively tuned on the benchmark, and Table 1 shows reliability collapses to 0.000 without the complete example, which itself mirrors the test loading pattern.

  1. fitted input called prediction [Sec. 2.2 (Workflows to assess reliability and robustness), Sec. 4.1]
    "Recognizing the sensitivity of LLM performance to prompt formulation, a series of prompt variants are designed and tested in a preliminary step. These variants explore different levels of descriptive detail and structural formatting. The prompt presented here represents the most effective formulation identified through iterative experiments, consistently yielding the highest reliability across various problems."

    The prompt is explicitly selected to maximize reliability on the problems under study, then Sec. 4.1 reports 'the proposed agent consistently achieves reliability over 0.990' on the same benchmark. The reported reliability is therefore an in-sample score of a tuned input, not an out-of-sample prediction. The paper's own ablation (Table 1) confirms that the two tuned components—complete example and function usage examples—are the primary drivers of this score, so the 99% result is forced by prompt selection rather than by the model's general structural-analysis ability.

  2. fitted input called prediction [Sec. 3.3 (Fig. 7), Sec. 4.2, Sec. 4.3 (Table 1)]
    "This is followed by three applied examples that demonstrate how to construct models in which a concentrated load lies within the span of a distributed load."

    The complete example in Fig. 7 is a 10 kN point load at 7.5 m inside a 10 kN/m distributed load from 6 to 10 m, and the function usage examples teach exactly the same pattern. All three extended tasks in Fig. 9 combine a point load inside a distributed load. Table 1 shows that removing the complete example drops reliability to 0.000 on all three extended tasks, and removing the function usage examples drops it to 0.026/0.296/0.623. Thus the reported 'generalization' (1.000, 0.998, 0.996) is essentially the model copying the embedded template; the prediction reduces to the few-shot input by construction.

full rationale

No self-citation load-bearing chain or imported uniqueness theorem is present; the references are external and the OpenSeesPy executions are machine-checkable. The circularity concern is centered on the agent's evaluation protocol. Section 2.2 states the prompt was iteratively selected for the highest reliability on the benchmark problems, and Section 4.1 then reports that same benchmark as the agent's headline result; this is an in-sample, tuned-input evaluation. The stronger structural issue is template matching: the complete example and the function usage examples encode the exact combined-load configuration (a concentrated load within the span of a distributed load) that appears in benchmark conditions III/IV and in all three extended tasks. Table 1 shows that without these components reliability falls to 0.000 or to 0.026-0.623. Therefore the >99% accuracy is largely inherited from few-shot examples that are near-duplicates of the test distribution, and the 'robust and reliable across diverse conditions' claim is established only for template-matched problems. The paper is transparent about the ablation, which is why this is partial circularity (score 6) rather than a fully forced result (8 or 10).

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the correctness of static equilibrium as the ground-truth model, on the assumption that OpenSeesPy's deterministic outputs are the reference solution, on the representativeness of the 8 plus 3 beam tasks, and on the hand-selected prompt components (especially the complete example) being treated as part of the system rather than as fitted parameters.

assumptions (5)
  • domain assumption Static equilibrium equations (sum of forces and moments equal zero) are the correct and sufficient model for the beam reaction forces in the benchmark.
    The benchmark tasks are solved analytically using these equations, and OpenSeesPy results are checked against them (Sections 2.1 and 2.2).
  • domain assumption OpenSeesPy with a linear elastic beam-column element and static analysis produces the correct ground-truth reactions for these problems.
    The agent's outputs are taken from OpenSeesPy execution without independent verification (Section 3.2, Appendix).
  • ad hoc to paper The eight benchmark problems and three extended tasks are representative of structural analysis sufficient to support claims of reliability and robustness.
    The tasks are simple statically determinate beams with fixed load magnitudes; the paper calls this a first step (Section 2.1).
  • ad hoc to paper Llama-3.3 70B Instruct is representative of current LLMs for drawing general conclusions about LLM reliability.
    A single model is used; the paper selects it for open-source availability and instruction-following capability (Section 1).
  • ad hoc to paper The baseline prompt selected through iterative experiments is a fair and typical prompt for the baseline model.
    The prompt was tuned on the same benchmark problems, which may inflate baseline reliability (Section 2.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysis." pith.science (2026). https://pith.science/paper/LCTSLPYJ

@misc{pith2026250702938,
  author       = {Pith},
  title        = {Pith review of: A Large Language Model-Empowered Agent for Reliable and Robust Structural Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LCTSLPYJ}},
  note         = {Machine review of arXiv:2507.02938}
}
read the original abstract

Large language models (LLMs) have exhibited remarkable capabilities across diverse open-domain tasks, yet their application in specialized domains such as civil engineering remains largely unexplored. This paper starts bridging this gap by evaluating and enhancing the reliability and robustness of LLMs in structural analysis of beams. Reliability is assessed through the accuracy of correct outputs under repetitive runs of the same problems, whereas robustness is evaluated via the performance across varying load and boundary conditions. A benchmark dataset, comprising eight beam analysis problems, is created to test the Llama-3.3 70B Instruct model. Results show that, despite a qualitative understanding of structural mechanics, the LLM lacks the quantitative reliability and robustness for engineering applications. To address these limitations, a shift is proposed that reframes the structural analysis as code generation tasks. Accordingly, an LLM-empowered agent is developed that (a) integrates chain-of-thought and few-shot prompting to generate accurate OpeeSeesPy code, and (b) automatically executes the code to produce structural analysis results. Experimental results demonstrate that the agent achieves accuracy exceeding 99.0% on the benchmark dataset, exhibiting reliable and robust performance across diverse conditions. Ablation studies highlight the complete example and function usage examples as the primary contributors to the agent's enhanced performance.

Figures

Figures reproduced from arXiv: 2507.02938 by the authors.

Figure 1
Figure 1. Benchmark dataset consisting of eight types of representative beam structural analysis kN/m uniformly distributed load. These loads are applied at varying locations along the beam span to reflect a range of [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Workflow for assessing the reliability of LLM for structural analysis problems. ustness metric, a normalized robustness score is defined with reference to the inverse t f iti(CV) (Ht l2009)dfid iE(2) [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Workflow for assessing the robustness of LLM for structural analysis problems. Following the evaluation workflows described above, the reliability and robustness of the Llama-3.3 70B Instruct model [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Performance of Llama-3.3 on the benchmark dataset. Fig. 4: Reliability and robustness of Llama-3.3 70B Instruct model on the benchmark dataset. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Illustrative examples of incorrect solutions produced by the Llama-3.3 70B model in structural analysis tasks. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: A paradigm shift from text generation to code generation for structural analysis tasks using a LLM-empowered [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: A prompt template for reliable and robust structural analysis. [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Performance of the proposed agent on the benchmark dataset. Fig. 8: Reliability and robustness of the proposed LLM-empowered agent on the benchmark dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Generalization performance of the proposed LLM-empowered agent on extended beam analysis tasks. [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Visualization of the proposed LLM-empowered agent for extended beam analysis problems. [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Examples of simply supported beam under various load conditions. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Examples of overhang beam under various load conditions. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Automating structural reliability analysis with a multi-agent large language model framework

    stat.AP 2026-07 conditional novelty 6.0 of 10

    A multi-agent LLM framework automates structural reliability analysis from natural-language input to interpreted β and Pf, with a QLoRA-tuned method-planning agent.

Reference graph

Works this paper leans on

40 extracted references · 25 canonical work pages · cited by 1 Pith paper

  1. [1]

    Ahn, J., Verma, R., Lou, R., Liu, D., Zhang, R., and Yin, W. (2024). ``Large language models for mathematical reasoning: Progresses and challenges.''\ arXiv preprint arXiv:2402.00157

    arXiv 2024

  2. [2]

    and Arora, A

    Arora, A. and Arora, A. (2023). ``The promise of large language models in health care.''\ The Lancet , 401(10377), 641

    work page 2023

  3. [3]

    Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H., and Wang, H. (2023). ``Retrieval-augmented generation for large language models: A survey.''\ arXiv preprint arXiv:2312.10997 , 2, 1

    arXiv 2023

  4. [4]

    G., Soltani, H., and Isiaq, O

    Graham, S. G., Soltani, H., and Isiaq, O. (2023). ``Natural language processing for legal document review: categorising deontic modalities in contracts.''\ Artificial Intelligence and Law , 1--22

    work page 2023

  5. [5]

    ``The llama 3 herd of models.''\ arXiv preprint arXiv:2407.21783

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al.\@ (2024). ``The llama 3 herd of models.''\ arXiv preprint arXiv:2407.21783

    arXiv 2024

  6. [6]

    ``Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.''\ arXiv preprint arXiv:2501.12948

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al.\@ (2025). ``Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.''\ arXiv preprint arXiv:2501.12948

    arXiv 2025

  7. [7]

    L., Loomans, M

    Hoes, P., Hensen, J. L., Loomans, M. G., de Vries, B., and Bourgeois, D. (2009). ``User behavior in whole building simulation.''\ Energy and buildings , 41(3), 295--302

    work page 2009

  8. [8]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.\@ (2022)

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.\@ (2022). ``Lora: Low-rank adaptation of large language models..''\ ICLR , 1(2), 3

    work page 2022

Show all 40 references
  1. [9]

    ``A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.''\ ACM Transactions on Information Systems , 43(2), 1--55

    Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al.\@ (2025). ``A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.''\ ACM Transactions on Information Systems , 43(2), 1--55

    2025

  2. [10]

    Imani, S., Du, L., and Shrivastava, H. (2023). ``Mathprompter: Mathematical reasoning using large language models.''\ arXiv preprint arXiv:2303.05398

    2023 arXiv

  3. [11]

    Jeoung, J., Jung, S., and Hong, T. (2025). ``Zero-shot framework for construction equipment task monitoring.''\ Computer-Aided Civil and Infrastructure Engineering

    2025

  4. [12]

    Ji, B., Duan, X., Zhang, Y., Wu, K., and Zhang, M. (2024). ``Zero-shot prompting for llm-based machine translation using in-domain target sentences.''\ IEEE/ACM Transactions on Audio, Speech, and Language Processing

    2024

  5. [13]

    Joffe, I., Felobes, G., Elgouhari, Y., Talebi Kalaleh, M., Mei, Q., and Chui, Y. H. (2025). ``The framework and implementation of using large language models to answer questions about building codes and standards.''\ Journal of Computing in Civil Engineering , 39(4), 05025004

    2025

  6. [14]

    Kim, M.-Y., Rabelo, J., Babiker, H. K. B., Rahman, M. A., and Goebel, R. (2024). ``Legal information retrieval and entailment using transformer-based approaches.''\ The Review of Socionetwork Strategies , 18(1), 101--121

    2024

  7. [15]

    S., and Suh, H

    Kim, T., Yun, T. S., and Suh, H. S. (2025). ``Can chatgpt implement finite element models for geotechnical engineering applications?.''\ International Journal for Numerical and Analytical Methods in Geomechanics

    2025

  8. [16]

    Kokot, S. (2024). ``Opsvis documentation. Accessed: 2025-06-07

    2024

  9. [17]

    and Shi, C

    Li, H. and Shi, C. (2025). ``Few-shot learning of geological cross-sections from sparse data using large language model.''\ Geodata and AI , 2, 100010

    2025

  10. [18]

    B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al.\@ (2023)

    Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al.\@ (2023). ``Starcoder: may the source be with you!.''\ arXiv preprint arXiv:2305.06161

    2023 arXiv

  11. [19]

    and Tang, Y

    Lopez-Lira, A. and Tang, Y. (2023). ``Can chatgpt forecast stock price movements? return predictability and large language models.''\ arXiv preprint arXiv:2304.07619

    2023

  12. [20]

    Lu, Y., Zhu, W., Li, L., Qiao, Y., and Yuan, F. (2024). ``Llamax: Scaling linguistic horizons of llm by enhancing translation capabilities beyond 100 languages.''\ arXiv preprint arXiv:2407.05975

    2024 arXiv

  13. [21]

    Lv, K., Yang, Y., Liu, T., Gao, Q., Guo, Q., and Qiu, X. (2023). ``Full parameter fine-tuning for large language models with limited resources.''\ arXiv preprint arXiv:2306.09782

    2023 arXiv

  14. [22]

    ``Gpt-4o: Openai's multimodal flagship model

    OpenAI (2024). ``Gpt-4o: Openai's multimodal flagship model. Accessed: 2025-05-21

    2024

  15. [23]

    Pandey, S., Xu, R., Wang, W., and Chu, X. (2025). ``Openfoamgpt: A retrieval-augmented large language model (llm) agent for openfoam-based computational fluid dynamics.''\ Physics of Fluids , 37(3)

    2025

  16. [24]

    and Shi, C

    Qian, Z. and Shi, C. (2025). ``Large language model-empowered paradigm for automated geotechnical site planning and geological characterization.''\ Automation in Construction , 173, 106103

    2025

  17. [25]

    E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al.\@ (2023)

    Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al.\@ (2023). ``Code llama: Open foundation models for code.''\ arXiv preprint arXiv:2308.12950

    2023 arXiv

  18. [26]

    B., Alvarez-Melis, D., and Fusi, N

    Shen, J., Tenenholtz, N., Hall, J. B., Alvarez-Melis, D., and Fusi, N. (2024). ``Tag-llm: Repurposing general-purpose llms for specialized domains.''\ arXiv preprint arXiv:2402.05140

    2024 arXiv

  19. [27]

    H., Wu, J., Washington, C., Sadler, B

    Song, C. H., Wu, J., Washington, C., Sadler, B. M., Chao, W.-L., and Su, Y. (2023). ``Llm-planner: Few-shot grounded planning for embodied agents with large language models.''\ Proceedings of the IEEE/CVF international conference on computer vision , 2998--3009

    2023

  20. [28]

    J., Ting, D

    Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., and Ting, D. S. W. (2023). ``Large language models in medicine.''\ Nature medicine , 29(8), 1930--1940

    2023

  21. [29]

    Tonmoy, S., Zaman, S., Jain, V., Rani, A., Rawte, V., Chadha, A., and Das, A. (2024). ``A comprehensive survey of hallucination mitigation techniques in large language models.''\ arXiv preprint arXiv:2401.01313 , 6

    2024 arXiv

  22. [30]

    V., Zhou, D., et al.\@ (2022)

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al.\@ (2022). ``Chain-of-thought prompting elicits reasoning in large language models.''\ Advances in neural information processing systems , 35, 24824--24837

    2022

  23. [31]

    Wei, J., Zhuo, L., Fu, X., Zeng, X., Wang, L., Zou, Q., and Cao, D. (2024). ``Drugrealign: a multisource prompt framework for drug repurposing based on large language models.''\ BMC biology , 22(1), 226

    2024

  24. [32]

    Wu, T., Terry, M., and Cai, C. J. (2022). ``Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts.''\ Proceedings of the 2022 CHI conference on human factors in computing systems , 1--22

    2022

  25. [33]

    Yang, X., Li, Y., Zhang, X., Chen, H., and Cheng, W. (2023). ``Exploring the limits of chatgpt for query or aspect-based text summarization.''\ arXiv preprint arXiv:2302.08081

    2023 arXiv

  26. [34]

    Yu, T., Jing, Y., Zhang, X., Jiang, W., Wu, W., Wang, Y., Hu, W., Du, B., and Tao, D. (2025). ``Benchmarking reasoning robustness in large language models.''\ arXiv preprint arXiv:2503.04550

    2025 arXiv

  27. [35]

    Zhang, B., Yang, H., Zhou, T., Ali Babar, M., and Liu, X.-Y. (2023). ``Enhancing financial sentiment analysis via retrieval augmented large language models.''\ Proceedings of the fourth ACM international conference on AI in finance , 349--356

    2023

  28. [36]

    Zhang, T., Ladhak, F., Durmus, E., Liang, P., McKeown, K., and Hashimoto, T. B. (2024a). ``Benchmarking large language models for news summarization.''\ Transactions of the Association for Computational Linguistics , 12, 39--57

    2024

  29. [37]

    Zhang, W., Li, Y., Dong, Z., Wu, Y., Zhou, Y., Wang, D., Xing, S., Zhou, C., and Shen, D. (2024b). ``Renaissance of literate programming in the era of llms: Enhancing llm-based code generation in large-scale projects.''\ arXiv preprint arXiv:2502.17441

    2024 arXiv

  30. [38]

    and Goodfellow, S

    Zhong, Y. and Goodfellow, S. D. (2024). ``Domain-specific language models pre-trained on construction management systems corpora.''\ Automation in Construction , 160, 105316

    2024

  31. [39]

    Zhu, M., McKenna, F., and Scott, M. H. (2018). ``Openseespy: Python library for the opensees finite element framework.''\ SoftwareX , 7, 6--11

    2018

  32. [40]

    Zhu, W., Liu, H., Dong, Q., Xu, J., Huang, S., Kong, L., Chen, J., and Li, L. (2023). ``Multilingual machine translation with large language models: Empirical results and analysis.''\ arXiv preprint arXiv:2304.04675

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.