Pith. sign in

REVIEW 5 major objections 5 minor 37 references

Harnessing Multi-Agent LLMs for Complex Engineering Problem-Solving: A Framework for Senior Design Projects

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A team of role-playing LLM agents evaluates capstone proposals more like faculty than a single LLM does, cutting error by 89.3%.

desk verdict Useful applied paper on multi-agent LLM feedback for capstone design, but the headline 89.3% number is a mislabeled MAE ratio over six proposals, so the superiority claim is not yet established. read the letter →

arxiv 2501.01205 v1 pith:3IN5KYYH submitted 2025-01-02 cs.MA cs.AIcs.CLcs.LG

Abdullah Mushtaq , Muhammad Rafay Naeem , Ibrahim Ghaznavi , Muhammad Imran Taj , Imran Hashmi , Junaid Qadir This is my paper · ORCID
classification cs.MAcs.AIcs.CLcs.LG
keywords multi-agentLLMsseniordesignprojectsengineeringeducationwisdomofcrowdsLLMagentsfacultyevaluationpromptTree-of-Thoughts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Senior design projects, the capstone of engineering education, force students to balance technical performance against ethical, social, and environmental concerns. This paper proposes that a multi-agent large language model (LLM) system—where each agent plays a different expert role, from problem formulation to societal impact—can evaluate such proposals more the way faculty would than a single LLM asked to cover all perspectives at once. On six real capstone proposals judged against the average of four faculty members' ratings, the multi-agent system achieved a mean absolute error of 0.205 versus 0.388 for the single-agent baseline, which the authors call an 89.3% accuracy improvement. The paper reads this as evidence that dividing an evaluation task among specialized, communicating LLM agents yields richer, more holistic, and more human-aligned feedback than a single model, and it suggests the framework could serve as a co-pilot for students and supervisors.

What carries the argument

The central mechanism is the multi-agent architecture itself: eight LLM agents, each given a distinct expert persona and a rubric of evaluation points, coordinated by a supervisor agent that decomposes the project proposal into focused tasks, routes them through a shared task channel, checks outputs, and synthesizes a final evaluation. The agents process tasks sequentially and can use the previous agents' outputs as inputs, which the paper argues produces interdependent, layered assessment rather than independent verdicts. A single-agent baseline is prompted with a Tree-of-Thoughts template that asks it to imagine multiple experts thinking stepwise, isolating whether any advantage comes from genuine multi-agent collaboration or merely from richer prompting.

What would settle it

Take a larger set of proposals and a larger, more diverse faculty panel (say 20+ raters), compute inter-rater variance, and rerun the same MAE comparison; if the single-agent system matches or beats the multi-agent system once the ground truth is less noisy, or if the MAS error falls within the faculty disagreement band, the paper's central claim is not supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that a multi-agent LLM system, built by assigning distinct expert personas to eight coordinated agents, can evaluate open-ended engineering design proposals more consistently with faculty judgment than a single LLM prompted to simulate multiple experts. The comparison uses faculty evaluations as the reference standard: four faculty scored six real senior design proposals across seven aspects, and the multi-agent and single-agent systems scored the same proposals on the same rubric. The multi-agent system's mean absolute error relative to the faculty average was 0.205, against 0.388 for the single-agent system, an 89.3% reduction in error relative to the single-agent baseline. The single-agent system beat the multi-agent system only on the Breadth and Depth aspect, while the multi-agent system was markedly better on Ambiguity and Uncertainty, Societal and Ethical Considerations, and Technical Innovation and Risk Management. Beyond the error numbers, the paper reports that multi-agent outputs scored higher on clause density, lexical cohesion, and readability in the Flesch-Kincaid grade range ideal for senior students, supporting the reading that the MAS produces more detailed, integrated, and accessible feedback.

Load-bearing premise

The comparison treats the average of four faculty members' subjective ratings as the ground truth for proposal quality; if those ratings are noisy, biased, or unrepresentative, the measured error gap between the multi-agent and single-agent systems may not reflect genuine quality differences.

Editorial extensions

If this is right

  • If the 89.3% error reduction generalizes, multi-agent LLM systems could be deployed as co-pilots in capstone courses, giving students structured, multi-perspective feedback before they meet their supervisor.
  • The observed advantage on societal, ethical, and uncertainty aspects suggests MAS is especially useful where proposals involve conflicting objectives that a single model tends to oversimplify.
  • The framework could be adapted to other formative assessment tasks, such as peer review of project reports or accreditation self-studies, using the same role-decomposition pattern.
  • Because both systems rest on the same underlying LLM, the result isolates the value of dividing labor among specialized agents rather than the value of a better model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One implication the paper leaves implicit: if the error gap stems from the diversity of perspectives rather than the LLM itself, then the framework's value should grow with the number and heterogeneity of expert roles, a hypothesis one could test by ablating agents one at a time.
  • Another testable extension is whether the MAS also aligns better with a larger, more diverse faculty panel; the current ground truth rests on only four raters and six proposals, so the 89.3% figure is a proof-of-concept estimate, not a stable population statistic.
  • The multi-agent design could be transferred directly to other complex evaluation tasks, such as peer review of research proposals or interdisciplinary design reviews in industry, where checklists and multiple stakeholder viewpoints already matter.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a multi-agent LLM (MAS) framework to support senior design projects (SDPs) in engineering and computer science. The system consists of eight role-specialized agents—such as Problem Formulation, System Complexity, and Societal and Ethical Consideration agents—coordinated by a Coordinator Agent and a Task Agent, built on the Camel AI framework with GPT-4o as the backend. The authors compare this MAS against a single-agent LLM baseline that uses Tree-of-Thoughts prompting. Evaluation is based on six student proposals scored by four faculty members; the primary quantitative claim is that the MAS achieves a mean absolute error (MAE) of 0.205 versus 0.388 for the single-agent system, reported as an '89.3% accuracy improvement.' Secondary NLP-based metrics (clause density, lexical cohesion, Flesch-Kincaid score, average sentence length) are also reported. The paper concludes that the MAS provides a richer, more inclusive problem-solving environment than single-agent systems, with pedagogical benefits for students and supervisors.

Significance. If properly supported, this would be a useful contribution to engineering education technology: it offers a concrete, reusable framework for automated SDP feedback and clearly describes agent personas and coordination mechanisms. The choice of a single-agent baseline using the same backend LLM is a reasonable starting point. However, the central empirical claim is currently not statistically substantiated. The reported '89.3% accuracy improvement' is actually a relative reduction in MAE, and the comparison relies on only six proposals, no confidence intervals or significance tests, aspect-level errors that are not independent, and a faculty benchmark whose acknowledged variability is not propagated into the analysis. The NLP evaluation is purely descriptive. The paper is therefore a promising proof-of-concept rather than a validated comparison, and the strength of the abstract's and conclusion's claims exceeds what the evidence supports.

major comments (5)
  1. [IV-A, Figure 4] The headline '89.3% accuracy improvement' is the relative change in MAE, computed as (0.388 − 0.205)/0.205 ≈ 0.893. This is an error reduction, not an accuracy gain, and calling it 'accuracy' is misleading. More importantly, the comparison is reported without confidence intervals, significance tests, or any uncertainty estimate. With only six proposals and seven evaluation aspects per proposal, the aspect-level errors are not independent; a few outlier proposals could dominate the gap. The authors should report per-proposal paired differences (MAS vs. single-agent), the distribution of those differences, and a permutation test or a mixed-effects model with proposal as a random effect to establish whether the MAS advantage is distinguishable from noise.
  2. [III-B.1 and Figure 6] The paper acknowledges that faculty evaluations are subjective and naturally vary among evaluators, and Figure 6 shows substantial standard deviations across the four raters. However, the analysis treats the mean faculty rating as a fixed ground truth and does not propagate rater variance into the MAE comparison. To support the claim that the MAS aligns better with faculty, the authors should quantify the uncertainty in the reference scores—for example, by reporting MAE computed against each individual faculty member, or by bootstrapping over raters and proposals—and show that the 0.183 MAE gap persists. Without this, the observed gap may be within the noise of the benchmark, especially given the authors' own statement that the faculty scores are not an absolute ground truth.
  3. [IV-B, Figure 5] The NLP-based evaluation is reported descriptively, with no statistical tests, effect sizes, or confidence intervals. The statement that 'the MAS approach consistently outperforms the single-agent system across all evaluated metrics' is therefore not supported. Moreover, the text in Section IV-A claims broad MAS superiority while also reporting that the single-agent system performs better on Breadth and Depth (MAE 0.208 vs. 0.292); these two characterizations need to be reconciled. The NLP claims should either be given appropriate statistical support or explicitly reframed as observations without inferential force.
  4. [V-A and Conclusion] The phrase '89% more accuracy' in Section V-A and 'overall accuracy of 89%' in the Conclusion misrepresent the reported result; the underlying number is a relative reduction in MAE, not an accuracy measure. This is a load-bearing error because the abstract and conclusion use the 'accuracy' framing to support the paper's central claim. The authors should correct all occurrences, state the result in terms of alignment or error reduction, and qualify it with the uncertainty analysis requested above.
  5. [V-B] The paper states that the code link 'will be released after paper acceptance,' and no evaluation data are provided. Given that the central comparison rests on aggregate MAE numbers, reproducibility requires the anonymized proposals, agent outputs, faculty scores, and the scripts used to compute the reported MAE values. The authors should provide these artifacts or explain why they cannot be released; at minimum, the data needed to verify the headline numbers should be available to reviewers.
minor comments (5)
  1. [Figure 3 caption] The caption claims that 'multi-agent system scores are more aligned with faculty evaluation scores,' but the figure itself does not provide a quantitative basis for this assertion; consider referring the reader to Figure 4 or adding summary statistics to the caption.
  2. [Box 2] The Tree-of-Thoughts prompt template ends with 'The question is...' without showing how the complete prompt was constructed for the single-agent baseline; please include the full prompt used in the experiments.
  3. [III-B.1] The text refers to 'X University' without stating whether this anonymization is intentional or required; if so, please say so explicitly in a footnote.
  4. [References] Reference [23] is listed only as 'CamelAI-Documentation' without a title, access date, or URL details; please complete the citation.
  5. [IV-B] The abbreviation 'SA' is used in the text ('both MAS and SA responses') but is not defined; please define it at first use or replace it with 'single-agent.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the MAE comparison against external faculty ratings is an independent empirical benchmark, and self-citations are background only.

full rationale

The central quantitative claim (MAS MAE 0.205 vs single-agent 0.388 in Section IV-A) is an empirical comparison between system-generated scores and faculty evaluations. The faculty scores are external human judgments collected in Section III-B.1 and are not defined in terms of the MAS outputs; no parameter is fitted to the MAE or to the faculty ratings. The agent personas and evaluation criteria are designed before scoring and are not derived from the faculty scores used in the comparison. The paper acknowledges faculty subjectivity (Figure 6) and treats faculty ratings as a comparative benchmark, not as a construction of the system's outputs. The self-citations ([2]-[5], [35]) are used for background motivation or general educational claims and do not carry the derivation of the reported results. No equation reduces a predicted quantity to an input by definition, and no uniqueness theorem or ansatz is imported from the authors' prior work. The fragility of the MAE comparison due to small sample size and lack of uncertainty quantification is a statistical validity concern, not circularity.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central claim relies on hand-chosen design choices (rubric scale, message window, aspect definitions) and domain assumptions about faculty scores, GPT-4o, and the validity of NLP metrics. No new physical or conceptual entity is introduced; the agents are software components built from existing frameworks. The most fragile assumption is the use of subjective faculty ratings as ground truth with no uncertainty propagation.

free parameters (2)
  • Evaluation rubric scale = 1 to 5
    The 1-5 scoring scale for each aspect is chosen by hand in Box 1 and Figure 2; all MAE comparisons depend on this scale.
  • Message history window = 10 responses
    Section III-A states agents keep a history window of the last 10 responses; the effect on output quality is untested.
assumptions (5)
  • domain assumption Faculty scores are a valid reference standard for system quality
    Section III-B.1 uses faculty evaluations as the benchmark for MAS and single-agent quality, despite acknowledging their subjectivity and variability.
  • domain assumption GPT-4o is a suitable backend for both MAS and single-agent
    Section III-A selects GPT-4o without comparing to other models, assuming it is the best choice for this evaluation task.
  • domain assumption NLP metrics measure feedback quality
    Section III-B.2 uses clause density, lexical cohesion, Flesch-Kincaid score, and sentence length to infer that MAS responses are 'richer' and more 'cohesive', but no validation links these text statistics to learning outcomes.
  • domain assumption Wisdom of crowds applies to LLM agents
    Section I motivates the framework with Surowiecki's wisdom of crowds, but the paper does not test whether the eight agents are sufficiently diverse or independent to produce the claimed collective benefit.
  • domain assumption ABET definition of complex engineering problems is the correct evaluation target
    The evaluation criteria are derived from ABET's definition (Section II-B); the paper assumes this is the right standard for judging both student proposals and LLM feedback quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Harnessing Multi-Agent LLMs for Complex Engineering Problem-Solving: A Framework for Senior Design Projects." pith.science (2026). https://pith.science/paper/3IN5KYYH

@misc{pith2026250101205,
  author       = {Pith},
  title        = {Pith review of: Harnessing Multi-Agent LLMs for Complex Engineering Problem-Solving: A Framework for Senior Design Projects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3IN5KYYH}},
  note         = {Machine review of arXiv:2501.01205}
}
read the original abstract

Multi-Agent Large Language Models (LLMs) are gaining significant attention for their ability to harness collective intelligence in complex problem-solving, decision-making, and planning tasks. This aligns with the concept of the wisdom of crowds, where diverse agents contribute collectively to generating effective solutions, making it particularly suitable for educational settings. Senior design projects, also known as capstone or final year projects, are pivotal in engineering education as they integrate theoretical knowledge with practical application, fostering critical thinking, teamwork, and real-world problem-solving skills. In this paper, we explore the use of Multi-Agent LLMs in supporting these senior design projects undertaken by engineering students, which often involve multidisciplinary considerations and conflicting objectives, such as optimizing technical performance while addressing ethical, social, and environmental concerns. We propose a framework where distinct LLM agents represent different expert perspectives, such as problem formulation agents, system complexity agents, societal and ethical agents, or project managers, thus facilitating a holistic problem-solving approach. This implementation leverages standard multi-agent system (MAS) concepts such as coordination, cooperation, and negotiation, incorporating prompt engineering to develop diverse personas for each agent. These agents engage in rich, collaborative dialogues to simulate human engineering teams, guided by principles from swarm AI to efficiently balance individual contributions towards a unified solution. We adapt these techniques to create a collaboration structure for LLM agents, encouraging interdisciplinary reasoning and negotiation similar to real-world senior design projects. To assess the efficacy of this framework, we collected six proposals of engineering and computer science of...

Figures

Figures reproduced from arXiv: 2501.01205 by the authors.

Figure 1
Figure 1. Workflow Overview: 1) Input Submission: The student enters the project title and proposal PDF. 2) Proposal Processing: Coordinator Agent forwards details to Tasks Agent to generate focused tasks. 3) Task Generation: Tasks Agent returns a list of tasks. 4) Task Distribution: Tasks are sent to the Tasks Channel. 5) Task Assignment: Tasks are assigned to relevant agents. 6) Output Generation: Agents produce outputs and… view at source ↗
Figure 2
Figure 2. Personas for each agent in the proposed MAS, detailing their tasks, objectives, and evaluation points for evaluating [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Visual representation of scores to evaluate the performance of both of our proposed multi-agent and single-agent systems [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Mean Absolute Error comparison between multi-agent [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: NLP-based performance evaluation of original student proposals, MAS responses, and single-agent system responses. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Mean Faculty Evaluation Scores with standard de [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: User interface for the proposed LLM-based MAS co-pilot, enabling students to interact with a graphical interface. Students [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 28 canonical work pages

  1. [1]

    Work in progress: Perception of complex engineering problem among capstone design students,

    A. R. Fernando, J. G. U. Vergara, and C. A. D. Canlapan, “Work in progress: Perception of complex engineering problem among capstone design students,” in 2022 IEEE Global Engineering Education Conference (EDUCON). IEEE, 2022, pp. 14–16

    work page 2022

  2. [2]

    Engineering education, moving into 2020s: Essential competencies for effective 21st century electrical & computer engineers,

    J. Qadir, K.-L. A. Yau, M. A. Imran, and A. Al-Fuqaha, “Engineering education, moving into 2020s: Essential competencies for effective 21st century electrical & computer engineers,” in 2020 IEEE Frontiers in Education Conference (FIE) . IEEE, 2020, pp. 1–9

    work page 2020

  3. [3]

    A student primer on how to thrive in en- gineering education during and beyond COVID-19,

    J. Qadir and A. Al-Fuqaha, “A student primer on how to thrive in en- gineering education during and beyond COVID-19,” Education Sciences, vol. 10, no. 9, p. 236, 2020

    work page 2020

  4. [4]

    Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education,

    J. Qadir, “Engineering education in the era of ChatGPT: Promise and pitfalls of generative AI for education,” in 2023 IEEE Global Engineering Education Conference (EDUCON) . IEEE, 2023, pp. 1–9

    work page 2023

  5. [5]

    Generative artificial in- telligence and engineering education

    A. Johri, A. S. Katz, J. Qadir, and A. Hingle, “Generative artificial in- telligence and engineering education.” Journal of Engineering Education , vol. 112, no. 3, 2023

    work page 2023

  6. [6]

    The process of solving complex problems,

    A. Fischer, S. Greiff, and J. Funke, “The process of solving complex problems,” The Journal of Problem Solving , vol. 4, no. 1, pp. 19–42, 2012

    work page 2012

  7. [7]

    Definitions, traditions, and a general frame- work for understanding complex problem solving,

    P. A. Frensch and J. Funke, “Definitions, traditions, and a general frame- work for understanding complex problem solving,” in Complex problem solving: The European perspective . Lawrence Erlbaum Associates, Hillsdale, NJ, 1995, pp. 24–43

    work page 1995

  8. [8]

    P. M. Senge, The Fifth Discipline: The Art and Practice of the Learning Organization. Doubleday, 1990

    work page 1990

Show all 37 references
  1. [9]

    Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations

    J. Surowiecki, The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies, and Nations . Anchor, 2004

    2004

  2. [10]

    Wooldridge, An introduction to multiagent systems

    M. Wooldridge, An introduction to multiagent systems . John wiley & sons, 2009

    2009

  3. [11]

    S. E. Page, The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies . Princeton University Press, 2007

    2007

  4. [12]

    Minsky, The Society of Mind

    M. Minsky, The Society of Mind . Simon and Schuster, 1986

    1986

  5. [13]

    A survey on large language model based autonomous agents,

    L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y . Lin et al., “A survey on large language model based autonomous agents,” Frontiers of Computer Science , vol. 18, no. 6, p. 186345, 2024

    2024

  6. [14]

    Large language model based multi-agents: A survey of progress and challenges,

    T. Guo, X. Chen, Y . Wang, R. Chang, S. Pei, N. V . Chawla, O. Wiest, and X. Zhang, “Large language model based multi-agents: A survey of progress and challenges,” arXiv preprint arXiv:2402.01680 , 2024

    2024 arXiv

  7. [15]

    Camel: Communicative agents for “mind

    G. Li, H. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem, “Camel: Communicative agents for “mind”’ exploration of large language model society,” Advances in Neural Information Processing Systems , vol. 36, pp. 51 991–52 008, 2023

    2023

  8. [16]

    Megaagent: A prac- tical framework for autonomous cooperation in large-scale LLM agent systems,

    Q. Wang, T. Wang, Q. Li, J. Liang, and B. He, “Megaagent: A prac- tical framework for autonomous cooperation in large-scale LLM agent systems,” arXiv preprint arXiv:2408.09955 , 2024

    2024 arXiv

  9. [17]

    Unleashing the emergent cognitive synergy in large language models: A task- solving agent through multi-persona self-collaboration,

    Z. Wang, S. Mao, W. Wu, T. Ge, F. Wei, and H. Ji, “Unleashing the emergent cognitive synergy in large language models: A task- solving agent through multi-persona self-collaboration,” arXiv preprint arXiv:2307.05300, 2023

    2023 arXiv

  10. [18]

    Crew.ai: An open-source multi-agent orchestration frame- work,

    J. Moura, “Crew.ai: An open-source multi-agent orchestration frame- work,” https://www.crewai.com/, 2024, accessed: October 12, 2024

    2024

  11. [19]

    Very large-scale multi-agent simulation in agentscope,

    X. Pan, D. Gao, Y . Xie, Z. Wei, Y . Li, B. Ding, J.-R. Wen, and J. Zhou, “Very large-scale multi-agent simulation in agentscope,” arXiv preprint arXiv:2407.17789, 2024

    2024 arXiv

  12. [20]

    Openagents: An open platform for language agents in the wild,

    T. Xie, F. Zhou, Z. Cheng, P. Shi, L. Weng, Y . Liu, T. J. Hua, J. Zhao, Q. Liu, C. Liu et al., “Openagents: An open platform for language agents in the wild,” arXiv preprint arXiv:2310.10634 , 2023

    2023 arXiv

  13. [21]

    Agent lumos: Unified and modular training for open- source language agents,

    D. Yin, F. Brahman, A. Ravichander, K. Chandu, K.-W. Chang, Y . Choi, and B. Y . Lin, “Agent lumos: Unified and modular training for open- source language agents,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Paper...

    2024

  14. [22]

    [Online]

    OpenAI. [Online]. Available: https://openai.com/index/hello-gpt-4o

  15. [23]

    [Online]

    CamelAI-Documentation. [Online]. Available: https://docs.camel-ai.org/ camel.html

  16. [24]

    Using tree-of-thought prompting to boost chatgpt’s reasoning,

    D. Hulbert, “Using tree-of-thought prompting to boost chatgpt’s reasoning,” https://github.com/dave1010/tree-of-thought-prompting, May

  17. [25]

    Prompt Engineering Guide,

    E. Saravia, “Prompt Engineering Guide,” https://github.com/dair- ai/Prompt-Engineering-Guide, 12 2022

    2022

  18. [26]

    Should we use characteristics of conversation to measure grammatical complexity in L2 writing develop- ment?

    D. Biber, B. Gray, and K. Poonpon, “Should we use characteristics of conversation to measure grammatical complexity in L2 writing develop- ment?” Tesol Quarterly, vol. 45, no. 1, pp. 5–35, 2011

    2011

  19. [27]

    M. A. K. Halliday and R. Hasan, Cohesion in english . Routledge, 2014

    2014

  20. [28]

    Derivation of new readability formulas (automated readability index, fog count and Flesch reading ease formula) for navy enlisted personnel,

    J. Kincaid, “Derivation of new readability formulas (automated readability index, fog count and Flesch reading ease formula) for navy enlisted personnel,” Chief of Naval Technical Training , 1975. 11

    1975

  21. [29]

    The prompt report: A systematic survey of prompting techniques,

    S. Schulhoff, M. Ilie, N. Balepur, K. Kahadze, A. Liu, C. Si, Y . Li, A. Gupta, H. Han, S. Schulhoff et al. , “The prompt report: A systematic survey of prompting techniques,” arXiv preprint arXiv:2406.06608 , 2024

    2024 arXiv

  22. [30]

    Mrkl systems: A modular, neuro-symbolic architecture that combines large language mod- els, external knowledge sources and discrete reasoning,

    E. Karpas, O. Abend, Y . Belinkov, B. Lenz, O. Lieber, N. Ratner, Y . Shoham, H. Bata, Y . Levine, K. Leyton-Brownet al., “Mrkl systems: A modular, neuro-symbolic architecture that combines large language mod- els, external knowledge sources and discrete reasoning,” arXiv prep...

    2022 arXiv

  23. [31]

    Creating large language model applica- tions utilizing LangChain: A primer on developing LLM apps fast,

    O. Topsakal and T. C. Akinci, “Creating large language model applica- tions utilizing LangChain: A primer on developing LLM apps fast,” in International Conference on Applied Engineering and Natural Sciences , vol. 1, no. 1, 2023, pp. 1050–1056

    2023

  24. [32]

    AutoGen: Enabling next-gen LLM applications via multi-agent conversation framework,

    Q. Wu, G. Bansal, J. Zhang, Y . Wu, S. Zhang, E. Zhu, B. Li, L. Jiang, X. Zhang, and C. Wang, “AutoGen: Enabling next-gen LLM applications via multi-agent conversation framework,” arXiv preprint arXiv:2308.08155, 2023

    2023 arXiv

  25. [33]

    ReAct: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “ReAct: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022

    2022 arXiv

  26. [34]

    A framework to develop and implement role- play case studies to teach responsible technology use,

    A. Hingle and A. Johri, “A framework to develop and implement role- play case studies to teach responsible technology use,” IEEE Transactions on Technology and Society , 2024

    2024

  27. [35]

    Leveraging the force of formative assessment & feedback for effective engineering education,

    J. Qadir, A.-E. M. Taha, K.-L. A. Yau, J. Ponciano, S. Hussain, A. Al- Fuqaha, and M. A. Imran, “Leveraging the force of formative assessment & feedback for effective engineering education,” 2020

    2020

  28. [36]

    The prompting guide: LLM agents research,

    Prompting Guide, “The prompting guide: LLM agents research,” 2024, accessed: 2024-11-13. [Online]. Available: https://www.promptingguide. ai/research/llm-agents

    2024

  29. [2023]

    Available: https://doi.org/10.5281/zenodo.10323452

    [Online]. Available: https://doi.org/10.5281/zenodo.10323452

    doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.