Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a multi-agent teacher–learner–evaluator workflow can generate complete, diverse, tutorial-aligned simulated tutoring conversations at scale, and that a combined computational and rubric-based protocol can compare…

desk verdict Large open dataset of simulated tutoring conversations is the real contribution; the evaluation protocol's human-alignment check is too selective to support the effectiveness claim. read the letter →

arxiv 2507.05528 v2 pith:4RYD2RPY submitted 2025-07-07 cs.AI cs.CL

Jiahuan Pei , Fanghua Ye , Xin Sun , Wentao Deng , Koen Hindriks , Junxiao Wang This is my paper
classification cs.AIcs.CL
keywords multi-agentLLMprocedurallearningeducationalconversationsimulationpedagogicqualityassessmentevaluationrubricmetricsAIineducationhow-totutorials
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes WikiHowAgent, a multi-agent workflow in which a teacher agent and a learner agent converse step-by-step through public how-to tutorials while an interaction manager directs the flow and an evaluator scores the result. The goal is to establish that LLM-driven simulation can produce complete, diverse, and semantically faithful teaching–learning dialogues at scale, and that the resulting conversations plus a combined computational-and-rubric evaluation protocol provide a practical way to compare LLMs as tutors. If this holds, educational assessment no longer requires hand-crafted dialogues or live human participants for every test, because pedagogy can be probed automatically across many domains and models.

What carries the argument

The load-bearing mechanism is the conversational graph: a loop with three node types—Teacher, Learner, and END—plus an interaction manager that tracks the current tutorial step, decides whether the learner needs clarification, and advances the state. The teacher supplies instructions grounded in the step text, the learner responds with either a clarification question or an acknowledgment that triggers the next step, and completion is signalled by a FINISHED token. An evaluator then scores the transcript with computational metrics (BLEU, METEOR, ROUGE, BERTScore, Diversity, Question, Completion) and with an LLM judge using an eight-criterion rubric (Clarity, Truthfulness, Engagement, Coherence, Depth, Relevance, Progress, Naturalness).

What would settle it

Record a sample of real students attempting to learn the same how-to procedures in one-on-one tutoring sessions, then compare their questions, misunderstandings, and completion patterns to the LLM learner transcripts under the same rubric; a substantial divergence in engagement or clarification behaviour, or a completion rate far below 95%, would undermine the claim that the workflow measures pedagogic quality.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a state-machine loop with separate teacher, learner, and evaluator agents, prompted by tutorial steps, reliably generates conversation graphs that reach the FINISHED token in 95.16% of cases, reach 80.14% bigram diversity, and stay semantically close to the source tutorial (BERTScore 70.42%) across eight LLMs. The paper further finds that heterogeneous setups, where the learner model differs from the teacher, improve diversity and tutorial alignment while reducing variance, and that LLM rubric judgments align well with human raters on clarity, progress, coherence, and relevance but less well on depth, engagement, naturalness, and truthfulness. These results are presented as evidence that the workflow is an effective instrument for procedural learning simulation and pedagogic quality assessment.

Load-bearing premise

The load-bearing premise is that an LLM-generated learner behaves enough like a real human student that its clarification questions, acknowledgments, and comprehension signals are representative of actual learners; if simulated learners are systematically more polite, more passive, or more consistent than humans, the measured pedagogic quality will not transfer to real education.

Editorial extensions

If this is right

  • If the workflow is sound, the 114,296-conversation corpus grounded in 14,287 tutorials across 17 domains can serve as a public benchmark for tutoring-dialogue research.
  • LLM-based evaluator scores can be used as a coarse-grained proxy for human pedagogical ratings on the more objective rubric dimensions, reducing the cost of large-scale assessment.
  • Heterogeneous agent pairings—mixing learner model with teacher model—can be used to tune conversational diversity and tutorial alignment without retraining.
  • The observed variance in engagement and question generation across models identifies specific failure modes in simulating active learners that future learner-agent designs must address.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not demonstrate that simulated learners generalize to real students; an obvious next test is to replace the learner agent with actual human learners on a sample of tutorials and compare the resulting pedagogical dynamics.
  • The domain-wise metric variance suggests that model choice for an educational tool may need to be domain-dependent, which could be tested with a domain-stratified leaderboard.
  • The same state-machine scaffolding could be applied beyond how-to articles to any structured procedural text, such as medical instructions or hardware manuals, effectively turning the workflow into a general-purpose procedural-training simulator.
  • Because the evaluator is the same model family as the teacher in homogeneous setups, the rubric scores may be partly self-congratulatory; using a third-party judge model in all setups would test for this bias.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces WikiHowAgent, a multi-LLM agent workflow that simulates teaching–learning conversations grounded in WikiHow procedural tutorials. The workflow consists of teacher, learner, interaction-manager, and evaluator agents, and is used to generate a dataset of 114,296 teacher–learner conversations across 17 domains and 727 topics. The authors propose an evaluation protocol combining computational metrics (e.g., BLEU, ROUGE, BERTScore, Completion, Diversity) with LLM-judge rubric scores across eight quality dimensions, and validate the protocol by comparing LLM-judge scores with human annotations on a small subset. The paper reports high Completion (mean 95.16%) and Diversity (80.14%) for eight LLMs in homogeneous settings, and compares homogeneous vs. heterogeneous learner configurations. The main claims are that the workflow is effective at scale and that the evaluation protocol yields insights into LLM capabilities for procedural teaching.

Significance. If the claims hold, the dataset and workflow would be a useful resource for AI4Education: the open-sourced release of 114k simulated conversations is a substantial empirical contribution, and the workflow itself is modular and reproducible, with prompt templates and code provided. The paper's computational metrics show clear strengths in conversation completion and diversity, and the comparison across eight LLMs of different sizes and origins provides a useful benchmark. However, the significance of the pedagogical and evaluative conclusions is limited by methodological issues in the human-alignment study and by the absence of control conditions, which I detail below. The core dataset and workflow contributions remain valuable even if the evaluative protocol needs revision.

major comments (4)
  1. [§3.2, §5.3 (Human Judgment Alignment)] The human-evaluation protocol selects only the 25 tutorials with the top average LLM-judged scores, yielding 175 conversations for human rating. Because the selection criterion is the very LLM evaluator being validated, the correlation coefficients in Figure 7 are computed on a truncated, non-representative sample. This range restriction can inflate or deflate correlations, and it cannot detect systematic bias in LLM rubric scores on average or low-quality conversations. The claim of alignment between automatic metrics and human judgment (RQ3) is therefore not established for the full dataset. I recommend re-running the human evaluation on a random or stratified sample of conversations across the full quality range, and reporting inter-annotator agreement (e.g., Cohen's kappa) for the two judges.
  2. [§5.1, Tables 1–2] The effectiveness claim in the abstract and RQ1 lacks any control or baseline condition. There is no comparison against a single-agent system (e.g., one LLM prompted to produce a full tutorial dialogue directly), a non-interactive setup (e.g., a static lecture generated from the tutorial), or an existing multi-agent educational framework. Without such baselines, high Completion and Diversity values are difficult to interpret; a single model could plausibly achieve similar values on this task. I request the addition of at least one simple baseline (e.g., a zero-shot single-prompt conversation generator) or a reference to existing comparable systems, and a statistical comparison of the workflow's metrics against that baseline.
  3. [§5.1.1, Figures 3 and 6] In homogeneous learning, teacher, learner, and evaluator share the same LLM, so the rubric-based scores in Figure 3 are self-assessments of the model's own generated conversations. Figure 6 confirms that LLM judges rate higher and more tightly than human judges, but the paper does not analyze how this self-evaluation bias affects the cross-LLM comparisons in Figure 3. For example, a model that is merely confident or fluent may receive inflated self-scores, confounding the reported 'well-rounded performer' conclusions about OpenChat and OLMo2. I recommend either using a separate evaluator model in the homogeneous setting or explicitly analyzing the self-evaluation bias and adjusting the conclusions accordingly.
  4. [§5.2, Figure 5] The domain analysis in Section 5.2 is internally inconsistent with the previous section. Section 5.1.2 defines heterogeneous learning as OpenChat serving as teacher/evaluator with four different learner models, and homogeneous learning as using all eight LLMs. However, Section 5.2 says 'We present the evaluation results of the proposed multi-LLM agent workflow with 8 backbone LLMs in heterogeneous learning' and then reports rubric scores across domains. Since Figure 5 plots all eight LLMs, it appears to show homogeneous rather than heterogeneous results, or it uses an undefined 'heterogeneous' configuration. Please clarify which learning setting is used for the domain analysis and correct the text and figure caption accordingly.
minor comments (6)
  1. [§3.1.1] METEOR is described as evaluating '4-gram similarity,' but METEOR is based on unigram matching with stemming and synonymy; the description is incorrect and should be fixed.
  2. [Appendix B, Table 3] The prompt templates contain spacing and formatting errors, e.g., 'Step{current_step_index}' and 'Scorefrom' should be 'Step {current_step_index}' and 'Score from'.
  3. [Table 1, Table 2] The notation 'µ±δ' is used for mean and standard deviation, but δ conventionally denotes a small difference; use 'μ ± σ' to avoid ambiguity.
  4. [§5.2] The opening sentence of Section 5.2 says the results are 'evaluated by computational metrics,' but the text then discusses rubric-based scores such as Clarity and Engagement; this mismatch should be corrected.
  5. [§3.2] Only two human annotators were employed, and no inter-rater reliability statistic is reported; at least Cohen's kappa or Krippendorff's alpha should be added to support the reliability of the human ratings.
  6. [Appendix A] The parameter count for GPT-4 is reported as '1.76TB,' which is likely a typo for 1.76T (trillion) parameters; please correct the unit.

Circularity Check

2 steps flagged · score 6.0 of 10

Rubric scores in homogeneous setups are self-graded by the same LLM, and the human-alignment check is run only on the evaluator's top-scored subset, so the evaluation protocol is partially circular.

  1. other [Section 5.1.1 Homogeneous Learning and Section 5.1.2 Heterogeneous Learning (Figure 3 and following text)]
    "In this setting, all three agents (teacher, learner, and evaluator) share the same model. ... OpenChat and OLMo2 are the most well-rounded performers, excelling in the majority of the metrics. Hence, in the heterogeneous learning setups, we equip OpenChat with the teacher and evaluator agents."

    The rubric-based metrics used to identify 'well-rounded' performance are produced by the same LLM that generated the conversation. The paper then uses those self-scores to select OpenChat as teacher and evaluator for the heterogeneous setup, creating a closed loop: OpenChat's high rubric scores make it the chosen teacher, and the same OpenChat evaluator subsequently grades its own teaching. A model that systematically overrates its own output will be selected and then appear to perform well, so the heterogeneous comparison is not independent evidence of effectiveness.

  2. other [Section 3.2 Human Judgment Alignment]
    "First, we select 25 tutorials based on the top average LLM-judged scores across all metrics, resulting in a set of 175 model-generated conversations."

    Human ratings are collected only for conversations that the LLM evaluator already scored highest, so the correlation coefficients in Figure 7 are computed on a truncated, non-representative sample. This cannot validate alignment on typical or low-quality conversations, where Figures 6 and 8 show human and LLM ratings diverge. The RQ3 conclusion that automatic metrics 'align with human judgment' is therefore conditional on the evaluator's own top-ranked subset; the validation is not independent of the measure being validated.

full rationale

The dataset construction, computational metrics (Completion, Diversity, BLEU, METEOR, ROUGE, BERTScore, Question), and open-source release are independent of the evaluation loop and give the workflow genuine external content. No load-bearing self-citation or imported uniqueness theorem appears; the Pei et al. (2024) citation is contextual only. The circularity is confined to the rubric-based assessment: in homogeneous learning the same LLM is teacher, learner, and evaluator, and the model selection for the heterogeneous setup is based on those self-scores. The human-alignment study, which would otherwise break the loop, is weakened because human judges rate only the 175 conversations with top LLM-judged scores, so the correlation evidence does not cover the full quality range. The Limitations section's admission that simulated learners 'may not fully reflect the variability and learning behavior of real human learners' is an external-validity caveat rather than a circularity. These issues prevent the rubric-based 'insights into LLM capabilities' from being fully independent, but they do not make the workflow's construction or computational evaluation circular.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest primarily on domain assumptions about the validity of simulated learners and LLM judges, plus a small number of hand-set experimental parameters. No new physical or conceptual entities are introduced; the agents are software components built from existing LLMs.

free parameters (4)
  • Max interactions M = 40
    Algorithm 1 and Appendix A fix the conversation cap at 40 interactions. This truncates longer tutorials and directly affects Completion and length-sensitive metrics.
  • Sampling temperature = 0.0
    Appendix A sets temperature to 0.0 for deterministic outputs. This suppresses stochastic diversity and likely affects Diversity and Question metrics.
  • Top-25 tutorial selection for human evaluation = 25
    Section 3.2 selects 25 tutorials with the top average LLM-judged scores. This post-hoc selection biases the human alignment sample toward conversations the LLM evaluator already rated highly.
  • Number of human annotators = 2
    Section 3.2 hires two graduate students and averages their ratings. Two annotators without reported agreement makes the gold standard noisy.
assumptions (4)
  • domain assumption Simulated learner responses are a valid proxy for real learner behavior.
    The paper measures Engagement, Depth, and other pedagogical rubrics on conversations where the learner is an LLM. The Limitations section explicitly concedes this proxy may not reflect real human learners, so the proxy is load-bearing for all pedagogical conclusions.
  • domain assumption WikiHow articles are an accurate and sufficient ground truth for procedural learning.
    Computational metrics (BLEU, ROUGE, METEOR, BERTScore) and the Truthfulness rubric compare generated text to the WikiHow tutorial as reference (Sections 3.1.1 and 3.1.2). If a tutorial is inaccurate or incomplete, the quality scores inherit that error.
  • domain assumption LLM-as-judge rubric scores measure pedagogic quality.
    The evaluator agent assigns 1-5 rubric scores (Section 3.1.2), and these scores drive all model comparisons and the selection of tutorials for human evaluation. They are validated only against two human judges on a selected subset, not against learning outcomes.
  • domain assumption The average of two human judges is a reliable gold standard.
    Section 3.2 uses the mean of two annotators as the reference for correlation coefficients, but no inter-annotator agreement statistic is reported, so the reference may itself be noisy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment." pith.science (2026). https://pith.science/paper/4RYD2RPY

@misc{pith2026250705528,
  author       = {Pith},
  title        = {Pith review of: Conversational Education at Scale: A Multi-LLM Agent Workflow for Procedural Learning and Pedagogic Quality Assessment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4RYD2RPY}},
  note         = {Machine review of arXiv:2507.05528}
}
read the original abstract

Large language models (LLMs) have advanced virtual educators and learners, bridging NLP with AI4Education. Existing work often lacks scalability and fails to leverage diverse, large-scale course content, with limited frameworks for assessing pedagogic quality. To this end, we propose WikiHowAgent, a multi-agent workflow leveraging LLMs to simulate interactive teaching-learning conversations. It integrates teacher and learner agents, an interaction manager, and an evaluator to facilitate procedural learning and assess pedagogic quality. We introduce a dataset of 114,296 teacher-learner conversations grounded in 14,287 tutorials across 17 domains and 727 topics. Our evaluation protocol combines computational and rubric-based metrics with human judgment alignment. Results demonstrate the workflow's effectiveness in diverse setups, offering insights into LLM capabilities across domains. Our datasets and implementations are fully open-sourced.

Figures

Figures reproduced from arXiv: 2507.05528 by the authors.

Figure 1
Figure 1. The proposed multi-LLM agent workflow for [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Hierarchical distribution of topics across 17 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Evaluation results of homogeneous learning [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Evaluation results of heterogeneous learning [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Multiple rubric evaluation scores of all [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Comparison between human annotation and [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Human alignment correlation coefficient Pear [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Distribution of model evaluations compared to human annotations across all metrics. [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Visualization of the knowledge graph com [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Lect\=uraAgents: A Multi-Agent Framework for Adaptive Personalized AI-Assisted Learning and Embodied Teaching

    cs.CL 2026-06 unverdicted novelty 5.0 of 10

    LectūraAgents proposes a hierarchical multi-agent system with adaptive embodied teaching and the TASA algorithm for personalized AI-assisted learning, reporting gains in content quality, teaching actions, and personal...

Reference graph

Works this paper leans on

36 extracted references · 22 canonical work pages · cited by 1 Pith paper

  1. [1]

    Alaa Abd-Alrazaq, Rawan AlSaad, Dari Alhuwail, Arfan Ahmed, Padraig Mark Healy, Syed Latifi, Sarah Aziz, Rafat Damseh, Sadam Alabed Alrazak, and Javaid Sheikh. 2023. Large language models in medical education: opportunities, challenges, and future directions. JMIR Medical Education, 9(1):e48291

    work page 2023

  2. [2]

    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. 2024. Researchagent: Iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738

    arXiv 2024

  3. [3]

    Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In ACL workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65--72

    work page 2005

  4. [4]

    Angxuan Chen, Yuang Wei, Huixiao Le, and Yan Zhang. 2024 a . Learning-by-teaching with chatgpt: The effect of teachable chatgpt agent on programming education. arXiv preprint arXiv:2412.15226

    arXiv 2024

  5. [5]

    Yulin Chen, Ning Ding, Hai-Tao Zheng, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2024 b . Empowering private tutoring by chaining large language models. In CIKM, pages 354--364

    work page 2024

  6. [6]

    Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S Yu, and Qingsong Wen. 2025. Llm agents for education: Advances and applications. arXiv preprint arXiv:2503.11733

    arXiv 2025

  7. [7]

    Alireza Ghafarollahi and Markus J Buehler. 2024. Protagents: protein discovery via large language model multi-agent collaborations combining physics and machine learning. Digital Discovery, 3(7):1389--1409

    work page 2024

  8. [8]

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. 2024. Tora: A tool-integrated reasoning agent for mathematical problem solving. In ICLR

    work page 2024

Show all 36 references
  1. [9]

    Neel Guha, Julian Nyarko, Daniel Ho, Christopher R \'e , Adam Chilton, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel Rockmore, Diego Zambrano, and 1 others. 2023. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. v...

    2023

  2. [10]

    Bihao Hu, Jiayi Zhu, Yiying Pei, and Xiaoqing Gu. 2025. Exploring the potential of llm to enhance teaching plans through teaching simulation. npj Science of Learning, 10(1):7

    2025

  3. [11]

    Huazhen Huang, Xianguo Shi, Hongyang Lei, Fan Hu, and Yunpeng Cai. 2024. Protchat: An ai multi-agent for automated protein analysis leveraging gpt-4 and protein language model. Journal of Chemical Information and Modeling, 65(1):62--70

    2024

  4. [12]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  5. [13]

    Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. 2024. Mapcoder: Multi-agent code generation for competitive problem solving. In ACL, pages 4912--4944

    2024

  6. [14]

    Shi Jinxin, Zhao Jiabao, Wang Yilei, Wu Xingjiao, Li Jiawen, and He Liang. 2023. Cgmi: Configurable general multi-agent interaction framework. arXiv preprint arXiv:2308.12503

    2023 arXiv

  7. [15]

    Paraskevas Lagakis and Stavros Demetriadis. 2024. Evaai: a multi-agent framework leveraging large language models for enhanced automated grading. In International Conference on Intelligent Tutoring Systems, pages 378--385. Springer

    2024

  8. [16]

    Yu-Ju Lan and Nian-Shing Chen. 2024. Teachers’ agency in the era of llm and generative ai. Educational Technology & Society, 27(1):I--XVIII

    2024

  9. [17]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  10. [18]

    Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller

    Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. 2024. Augmenting large language models with chemistry tools. Nature Machine Intelligence, 6(5):525--535

    2024

  11. [19]

    Pingchuan Ma, Tsun-Hsuan Wang, Minghao Guo, Zhiqing Sun, Joshua B Tenenbaum, Daniela Rus, Chuang Gan, and Wojciech Matusik. 2024 a . Llm and simulation as bilevel optimizers: a new paradigm to advance physical scientific discovery. In ICML, pages 33940--33962

    2024

  12. [20]

    Yiping Ma, Shiyu Hu, Xuchen Li, Yipei Wang, Shiqing Liu, and Kang Hao Cheong. 2024 b . Students rather than experts: A new ai for education pipeline to model more human-like and personalised early adolescences. arXiv preprint arXiv:2410.15701

    2024 arXiv

  13. [21]

    Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, and Aixin Sun. 2024 c . Sciagent: Tool-augmented language models for scientific reasoning. In EMNLP, pages 15701--15736

    2024

  14. [22]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL, pages 311--318

    2002

  15. [23]

    Minju Park, Sojung Kim, Seunghyun Lee, Soonwoo Kwon, and Kyuseok Kim. 2024. Empowering personalized learning through a conversation-based tutoring system with student modeling. In CHI, pages 1--10

    2024

  16. [24]

    Jiahuan Pei, Irene Viola, Haochen Huang, Junxiao Wang, Moonisa Ahsan, Fanghua Ye, Yiming Jiang, Yao Sai, Di Wang, Zhumin Chen, Pengjie Ren, and Pablo Cesar. 2024. Autonomous workflow for multimodal fine-grained training assistants towards mixed reality. In Findings of ACL

    2024

  17. [25]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In EMNLP-IJCNLP, pages 3982--3992

    2019

  18. [26]

    Henry W Sprueill, Carl Edwards, Khushbu Agarwal, Mariefel V Olarte, Udishnu Sanyal, Conrad Johnston, Hongbin Liu, Heng Ji, and Sutanay Choudhury. 2024. Chemreasoner: Heuristic search over a large language model's knowledge space using quantum-chemical feedback. In ICML, pages ...

    2024

  19. [27]

    Yi Wang, Jiafei Duan, Dieter Fox, and Siddhartha Srinivasa. 2023. Newton: Are large language models capable of physical reasoning? In Findings of EMNLP, pages 9743--9758

    2023

  20. [28]

    Songlin Xu, Hao-Ning Wen, Hongyi Pan, Dallas Dominguez, Dongyin Hu, and Xinyu Zhang. 2025 a . Classroom simulacra: Building contextual student generative agents in online education for learning behavioral simulation. arXiv preprint arXiv:2502.02780

    2025 arXiv

  21. [29]

    Songlin Xu, Xinyu Zhang, and Lianhui Qin. 2024. Eduagent: Generative student agents in learning. arXiv preprint arXiv:2404.07963

    2024 arXiv

  22. [30]

    Songlin Xu, Xinyu Zhang, and Lianhui Qin. 2025 b . Llm-powered multi-agent framework for goal-oriented learning in intelligent tutoring system. arXiv preprint arXiv:2501.15749

    2025 arXiv

  23. [31]

    John Yang, Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024 a . Swe-agent: Agent-computer interfaces enable automated software engineering. In NeurIPS, volume 37, pages 50528--50652

    2024

  24. [32]

    Kaiqi Yang, Yucheng Chu, Taylor Darwin, Ahreum Han, Hang Li, Hongzhi Wen, Yasemin Copur-Gencturk, Jiliang Tang, and Hui Liu. 2024 b . Content knowledge identification with multi-agent large language models. In International Conference on Artificial Intelligence in Education, p...

    2024

  25. [33]

    Ziqi Yang, Xuhai Xu, Bingsheng Yao, Ethan Rogers, Shao Zhang, Stephen Intille, Nawar Shara, Guodong Gordon Gao, and Dakuo Wang. 2024 c . Talk2care: An llm-based voice assistant for communication between healthcare providers and older adults. volume 8, pages 1--35. ACM New York...

    2024

  26. [34]

    Kechi Zhang, Jia Li, Ge Li, Xianjie Shi, and Zhi Jin. 2024. Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges. In ACL, pages 13643--13658

    2024

  27. [35]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  28. [36]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.