Pith. sign in

REVIEW 5 major objections 5 minor 35 references

SurgBox: Agent-Driven Operating Room Sandbox with Surgery Copilot

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read An LLM-driven Surgery Copilot inside a simulated operating room matches expert neurosurgeons' route and plan choices in 88% of real cases.

desk verdict A useful engineering demo of an LLM-agent OR sandbox, but the cognitive-enhancement claims outrun the proxy-accuracy evaluation. read the letter →

arxiv 2412.05187 v1 pith:QK76DHBN submitted 2024-12-06 cs.AI cs.CVcs.HCcs.RO

classification cs.AIcs.CVcs.HCcs.RO
keywords surgicalsimulationoperatingroomsandboxLLMagentsSurgeryCopilotretrieval-augmentedgenerationlong-shortmemoryneurosurgerycognitiveload
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes SurgBox, an agent-driven sandbox that recreates a neurosurgical operating room as a multi-agent simulation: each team member, from chief surgeon to scrub nurse and anesthetist, is an LLM-based agent with a role-specific knowledge base. Its central claim is that this environment, together with a Surgery Copilot agent that coordinates information and gives real-time guidance, can systematically enhance surgeons' cognitive capabilities while reducing cognitive workload. To support the claim, the authors compare the Copilot's surgical route selection and surgical planning against the decisions recorded by experienced neurosurgeons in 128 real operative reports. The Copilot matches the recorded decisions 88.00% of the time on route selection and 88.02% on planning, outperforming general LLM baselines. If correct, this gives surgeons a risk-free training ground for rare and high-stakes scenarios, and a live assistant that could support decision-making during actual operations.

What carries the argument

The load-bearing mechanism is the Surgery Copilot with its Long-Short Memory design, supported by role-specific retrieval-augmented generation (RAG). Short memory tracks the current operation's live information—patient context, stage, dialogues, instrument events—so the Copilot can respond to immediate needs; long memory stores summaries and lessons from earlier simulated procedures so the Copilot can draw on cumulative experience. Around it, each simulated surgical role has a dedicated knowledge base retrieved through RAG, and the Copilot orchestrates role interactions across preoperative, intraoperative, and postoperative phases. The combination is what the paper credits with raising route-selection accuracy from the 72% baseline to 88% and with maintaining accuracy as procedures become more complex.

What would settle it

Run a randomized trial in which residents train in SurgBox versus standard methods and then perform a simulated procedure while cognitive-load metrics and performance are recorded; if the SurgBox-trained group shows no lower load and no better performance than controls, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's discovery, stated on its own terms, is that a simulated operating room driven by LLM agents plus an orchestration agent can reproduce expert-level surgical decision-making as recorded in real clinical reports. The Surgery Copilot—an LLM agent with a Long-Short Memory mechanism and access to role-specific retrieval-augmented knowledge—selects the surgical route and generates the surgical plan that match the human-recorded ground truth at 88.00% and 88.02% accuracy respectively, above the general LLM baselines reported in the paper. The authors further report that this advantage holds across all four surgical stages, degrades less than baselines as cases grow more complex, and remains strong with small training samples. They present this as evidence that SurgBox can both train surgeons through deliberate practice and support clinical decision-making in real time.

Load-bearing premise

The load-bearing premise is that the surgical routes and step-by-step plans recorded in the 128 retrospective reports are the correct decisions, so matching them measures a surgeon's cognitive capability and decision-support quality.

Editorial extensions

If this is right

  • Residents can rehearse complete surgical workflows—from patient transfer through anesthesia, operation, and postoperative care—in a risk-free environment, building automatic responses to complications before meeting them in real patients.
  • In a live operating room, the Copilot could filter the multiple information streams the team must track and surface stage-appropriate guidance, which is the mechanism the authors credit with lowering cognitive load.
  • Because the Copilot's accuracy holds across later, more complex surgical stages and with small training samples, the approach may transfer to rare procedures and resource-limited settings.
  • The domain-specific knowledge banks reduce the model's tendency to invent nonexistent procedures or misread uncommon findings, directly addressing a known failure mode of general LLMs in surgery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Untested by the paper: whether 88% agreement with retrospective reports means better patient outcomes; a prospective study that tracks complications or expert-blinded ratings of the Copilot's plans would be the next discriminating test.
  • The Long-Short Memory architecture is a general recipe for procedural copilots in other high-stakes fields—emergency medicine, aviation, or intensive care—where a live context buffer plus a long case memory is exactly the needed balance.
  • The 1,000 simulated reports generated by the framework could be repurposed as synthetic training data for surgical phase-recognition models, a useful by-product even before the cognitive-training claim is validated.
  • A stronger evaluation would compare Copilot plans against a panel of independent neurosurgeons' consensus rather than a single recorded route, since retrospective reports may reflect one team's habits rather than an objectively best choice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper proposes SurgBox, an LLM-based multi-agent simulation of an operating room, with role-specific RAG knowledge bases and a 'Surgery Copilot' agent that uses a long-short memory mechanism to coordinate the surgical team. The authors report experiments on a corpus of 128 real neurosurgical (pituitary adenoma) reports, comparing route-selection and plan-accuracy against general LLMs, and claim 88.00% and 88.02% accuracy for surgery route and surgical plan tasks, respectively. The paper also includes an ablation study, a failure analysis, and a qualitative comparison with GPT-4 and LLaMA-3-70B, and states that code is publicly available.

Significance. If the performance claims held, SurgBox would be a useful training sandbox and decision-support prototype for neurosurgery. The paper has several strengths: it applies multi-agent LLM simulation to a new clinical domain, integrates task-specific RAG, includes an ablation and error analysis, and makes code available. However, the evidence as presented does not establish the headline claims about enhancing cognitive capabilities or reducing cognitive workload. The evaluation is a proxy task on a small, single-source dataset with no error bars or statistical tests, and the gold standard is not independently validated. The significance is therefore conditional on substantially stronger validation.

major comments (5)
  1. [V-A, Table I] The headline numbers in Table I are reported without error bars, confidence intervals, or significance tests, and the test-set size is not stated. With only 128 total records and likely a small test split, the 9-point route-accuracy gap over GPT-4 (88.00 vs 79.00) is borderline once sampling noise is considered, and the 2.34-point plan-accuracy gap (88.02 vs 85.68) is well within plausible noise. Please report the exact test-set N, per-metric confidence intervals, and paired significance tests (e.g., McNemar) for every comparison in Tables I–III.
  2. [Abstract, V-A, VI] The abstract and conclusion claim that SurgBox 'systematically enhance[s] the cognitive capabilities of surgeons' and that Surgery Copilot 'diminish[es] the cognitive workload,' but no experiment measures cognition, learning, situation awareness, or workload. The route and plan accuracies in Section V-A are at best proxies for decision quality, not for cognitive capability. As the paper stands, these central claims are unsupported; either add a human-subject evaluation (e.g., training transfer, NASA-TLX, or structured expert assessment) or revise the claims to say the system demonstrates promising proxy performance in simulation.
  3. [IV-A, V-A] The evaluation uses the surgical routes and plans recorded in 128 retrospective reports as ground truth, but the paper does not report inter-rater reliability, clinician adjudication, or any independent validation that these recorded choices are the correct or best decisions. A single team's retrospective report may reflect local practice rather than a consensus gold standard. This matters because route and plan accuracy is the only quantitative evidence for the system's decision-support capability.
  4. [III-C, Table III] The long-short memory mechanism is presented as a key contribution, yet the ablation study in Table III does not isolate it. The row labeled 'w/ Copilot [35]' cites a chain-of-thought paper, not the memory mechanism, and no row removes long-short memory from the full system. Therefore the contribution of the memory mechanism to the reported 88.00/88.02 results is not demonstrated.
  5. [V-A] The implementation section says the system was iteratively optimized on a limited number of training records with 'continuous adjustment of model parameters and decision logic based on expert feedback,' but it does not state how many optimization iterations were performed, whether the same experts provided both the gold labels and the feedback, or how test-set contamination was prevented. This protocol makes the reported gains difficult to interpret and reproduce; please specify the tuning and selection procedure and, ideally, run a locked evaluation after tuning.
minor comments (5)
  1. [Table II] The Surgery Copilot entry reads '8 8.00'; this is almost certainly a typo for '88.00'.
  2. [Table III] The row label 'w/ Copilot [35]' is confusing because reference [35] is chain-of-thought prompting; rename it to 'w/ CoT' or otherwise clarify what component is being ablated.
  3. [IV-B] Section IV-B describes a 1,000-report simulated dataset, but this dataset is not used in the evaluation; state its role explicitly or remove it.
  4. [V-A] The prompt templates, knowledge-base contents, and hyperparameters (including the random seed value) are not provided despite the code link; including them would improve reproducibility.
  5. [References] Several references are duplicated (e.g., [1] and [2]); please consolidate them.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SurgBox's evaluation is a standard held-out benchmark, and the reported accuracies are empirical comparisons rather than constructed identities.

full rationale

The derivation chain in SurgBox is not circular. The paper's central results—88.00% surgical route accuracy and 88.02% surgical plan accuracy (Table I)—are empirical outcomes of a supervised evaluation: Section V-A states that the 128 real neurosurgery records are divided into training and test sets, with the training set used for initialization and optimization and the test set used for evaluation. The Surgery Copilot's outputs are LLM generations conditioned on RAG retrieval and short/long-term memory, not algebraic or definitional transforms of the ground-truth route/plan labels; nothing in the paper forces the predicted route or plan to equal the recorded one by construction. The Long-Short Memory mechanism is presented as a system design rather than a derived theorem, and no uniqueness theorem is invoked. Self-citations [3]–[5] and [7] appear only as background context on surgical workflow and phase recognition, and they are not load-bearing for the system's design or evaluation. The main weakness—that the claim of 'enhancing cognitive capabilities' is validated only by proxy text-matching accuracies against retrospective reports, with no direct human-subject or cognitive-load measurement—is a validity and overclaim concern, not circularity; an evaluation can be weak or unpersuasive without being self-referential. No step meets the required standard of a quoted reduction of a prediction to its own input.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The system's performance depends on multiple undocumented design choices (knowledge base content, prompts, decision weights) that are tuned on the training data. The evaluation assumes recorded surgical plans are the correct answer and that matching them reflects cognitive competence. These assumptions are not validated, and the cognitive enhancement claim is unsupported.

free parameters (4)
  • Enhanced decision weights for Surgery Copilot = Not specified
    The Copilot is given higher decision weights than other roles; the exact values and how they are optimized are not disclosed (Section V-A).
  • Knowledge base content for each role = Not specified
    Tailored RAG knowledge bases are constructed for each role; their content, size, and source are not disclosed, and they are likely curated from the training data (Section III-A).
  • Long-short memory update rules = Not specified
    The iterative learning process updates long-term memory with summaries; the update frequency, retention criteria, and summarization method are not formalized (Section III-C).
  • Prompt engineering templates = Not specified
    Specific prompts are engineered to optimize medical terminology comprehension; these are tuned on the training set and not provided (Section V-A).
assumptions (4)
  • domain assumption The surgical records and MRI reports are accurate and representative of real clinical decision-making.
    The dataset of 128 real reports is the ground truth for evaluation; no independent verification of expert judgments is provided (Section IV-A).
  • domain assumption Recorded surgical routes and plans are the correct plans.
    The evaluation compares model output to recorded surgical routes and plans, assuming these are optimal and that matching them indicates competence (Section V-A).
  • ad hoc to paper LLM agents with RAG can authentically replicate surgical roles.
    The entire training framework rests on the assumption that LLM role-play is a faithful simulation of operating room dynamics; no validation of fidelity is given (Section III-A).
  • domain assumption The training and test split is random and free of leakage.
    The paper states a split but does not describe how it was performed or whether the same patient could appear in both sets (Section V-A).

how reviews work

0 comments
Cite this review

Pith. "Pith review of SurgBox: Agent-Driven Operating Room Sandbox with Surgery Copilot." pith.science (2026). https://pith.science/paper/QK76DHBN

@misc{pith2026241205187,
  author       = {Pith},
  title        = {Pith review of: SurgBox: Agent-Driven Operating Room Sandbox with Surgery Copilot},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QK76DHBN}},
  note         = {Machine review of arXiv:2412.05187}
}
read the original abstract

Surgical interventions, particularly in neurology, represent complex and high-stakes scenarios that impose substantial cognitive burdens on surgical teams. Although deliberate education and practice can enhance cognitive capabilities, surgical training opportunities remain limited due to patient safety concerns. To address these cognitive challenges in surgical training and operation, we propose SurgBox, an agent-driven sandbox framework to systematically enhance the cognitive capabilities of surgeons in immersive surgical simulations. Specifically, our SurgBox leverages large language models (LLMs) with tailored Retrieval-Augmented Generation (RAG) to authentically replicate various surgical roles, enabling realistic training environments for deliberate practice. In particular, we devise Surgery Copilot, an AI-driven assistant to actively coordinate the surgical information stream and support clinical decision-making, thereby diminishing the cognitive workload of surgical teams during surgery. By incorporating a novel Long-Short Memory mechanism, our Surgery Copilot can effectively balance immediate procedural assistance with comprehensive surgical knowledge. Extensive experiments using real neurosurgical procedure records validate our SurgBox framework in both enhancing surgical cognitive capabilities and supporting clinical decision-making. By providing an integrated solution for training and operational support to address cognitive challenges, our SurgBox framework advances surgical education and practice, potentially transforming surgical outcomes and healthcare quality. The code is available at https://github.com/franciszchen/SurgBox.

Figures

Figures reproduced from arXiv: 2412.05187 by the authors.

Figure 1
Figure 1. The overview of the SurgBox framework. In the simulated operating room sandbox, each surgical role is automatically [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Examples of surgical roles in SurgBox include the chief [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The surgical workflow in the SurgBox framework. The SurgBox framework simulates the patient’s entire surgical closed [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: The interaction of Surgery Copilot and surgical roles [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: The example of MRI diagnostic reports. Surgery date: 2021-04-19 Preoperative diagnosis: Recurrent invasive pituitary adrenocorticotroph macroadenoma (right Knosp grade 4) Cushing’s syndrome Postoperative diagnosis: Recurrent invasive pituitary adrenocorticotroph macroa…
Figure 7
Figure 7. Figure 7: The example of neurosurgical procedure records. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: As the sample size increases, Surgery Copilot’s per [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Comparison in specific diseases. D-1: Primary non [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 8
Figure 8. Figure 8: The performance of Surgery Copilot with different [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 10
Figure 10. Figure 10: Example comparison of surgical video question-answering. We provided the answers generated by the Llama-3-70B [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 19 canonical work pages

  1. [35]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” NeurIPS, vol. 35, pp. 24 824–24 837, 2022

  2. [1]

    Artificial intelligence in surgery,

    C. Varghese, E. M. Harrison, G. O’Grady, and E. J. Topol, “Artificial intelligence in surgery,” Nature Medicine, vol. 30, no. 5, pp. 1257–1268, May 2024

  3. [2]

    Artificial intelligence in surgery,

    C. Varghese, E. M. Harrison, G. O’Grady, and E. J. Topol, “Artificial intelligence in surgery,” Nature Medicine, pp. 1–12, 2024

  4. [3]

    Surgical video captioning with mutual-modal concept alignment,

    Z. Chen, Q. Guo, L. K. Yeung, D. T. Chan, Z. Lei, H. Liu, and J. Wang, “Surgical video captioning with mutual-modal concept alignment,” in MICCAI, 2023, pp. 24–34

  5. [4]

    Surgplan: Surgical phase localization network for phase recognition,

    X. Luo, Y . Pang, Z. Chen, J. Wu, Z. Zhang, Z. Lei, and H. Liu, “Surgplan: Surgical phase localization network for phase recognition,” in ISBI. IEEE, 2024, pp. 1–5

  6. [5]

    Surgical temporal action- aware network with sequence regularization for phase recognition,

    Z. Chen, Y . Zhai, J. Zhang, and J. Wang, “Surgical temporal action- aware network with sequence regularization for phase recognition,” in BIBM, 2023

  7. [6]

    Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: a randomized trial,

    J. G. Frederiksen, S. M. D. Sørensen, L. Konge, M. B. S. Svendsen, M. Nobel-Jørgensen, F. Bjerrum, and S. A. W. Andersen, “Cognitive load and performance in immersive virtual reality versus conventional virtual reality simulation training of laparoscopic surgery: a randomized trial,” Surgical endoscopy, vol. 34, pp. 1244–1252, 2020

  8. [7]

    Asi-seg: Audio-driven surgical instrument segmentation with surgeon intention understanding,

    Z. Chen, Z. Zhang, W. Guo, X. Luo, L. Bai, J. Wu, H. Ren, and H. Liu, “Asi-seg: Audio-driven surgical instrument segmentation with surgeon intention understanding,” in IROS, 2024

Show all 35 references
  1. [8]

    Global variation in postoperative mortality and complications after cancer surgery: a multicentre, prospective cohort study in 82 countries,

    S. R. Knight, C. A. Shaw, R. Pius, T. M. Drake, L. Norman, A. O. Ademuyiwa, A. O. Adisa, M. L. Aguilera, S. W. Al-Saqqa, I. Al-Slaibi et al. , “Global variation in postoperative mortality and complications after cancer surgery: a multicentre, prospective cohort study in 82 cou...

  2. [9]

    Immersive virtual reality for surgical training: a systematic review,

    R. Q. Mao, L. Lan, J. Kay, R. Lohre, O. R. Ayeni, D. P. Goel et al. , “Immersive virtual reality for surgical training: a systematic review,” Journal of Surgical Research , vol. 268, pp. 40–58, 2021

  3. [10]

    Patient safety culture as perceived by operating room professionals: a mixed-methods study,

    W. Aouicha, M. A. Tlili, J. Sahli, A. Mtiraoui, T. Ajmi, H. Said Latiri, S. Chelbi, M. Ben Rejeb, and M. Mallouli, “Patient safety culture as perceived by operating room professionals: a mixed-methods study,” BMC Health Services Research , vol. 22, no. 1, p. 799, 2022

  4. [11]

    Generative agents: Interactive simulacra of human behavior,

    J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, “Generative agents: Interactive simulacra of human behavior,” in Proceedings of the 36th annual acm symposium on user interface software and technology , 2023, pp. 1–22

  5. [12]

    Metagpt: Meta programming for a multi-agent collaborative framework,

    S. Hong, M. Zhuge, J. Chen, X. Zheng, Y . Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin et al. , “Metagpt: Meta programming for a multi-agent collaborative framework,” in ICLR, 2024

  6. [13]

    Ai hospital: Interactive evaluation and collaboration of llms as intern doctors for clinical diagnosis,

    Z. Fan, J. Tang, W. Chen, S. Wang, Z. Wei, J. Xi, F. Huang, and J. Zhou, “Ai hospital: Interactive evaluation and collaboration of llms as intern doctors for clinical diagnosis,” arXiv preprint arXiv:2402.09742 , 2024

  7. [14]

    Medagents: Large language models as collaborators for zero-shot medical reasoning,

    X. Tang, A. Zou, Z. Zhang, Y . Zhao, X. Zhang, A. Cohan, and M. Gerstein, “Medagents: Large language models as collaborators for zero-shot medical reasoning,” arXiv preprint arXiv:2311.10537 , 2023

  8. [15]

    Agent hospital: A simulacrum of hospital with evolvable medical agents,

    J. Li, S. Wang, M. Zhang, W. Li, Y . Lai, X. Kang, W. Ma, and Y . Liu, “Agent hospital: A simulacrum of hospital with evolvable medical agents,” arXiv preprint arXiv:2405.02957 , 2024

  9. [16]

    Debug like a human: A large language model debugger via verifying runtime execution step by step,

    L. Zhong, Z. Wang, and J. Shang, “Debug like a human: A large language model debugger via verifying runtime execution step by step,” in Findings of ACL , 2024, pp. 851–870

  10. [17]

    Determinants of llm-assisted decision- making,

    E. Eigner and T. H ¨andler, “Determinants of llm-assisted decision- making,” arXiv preprint arXiv:2402.17385 , 2024

  11. [18]

    Interactive ai with retrieval-augmented generation for next generation networking,

    R. Zhang, H. Du, Y . Liu, D. Niyato, J. Kang, S. Sun, X. Shen, and H. V . Poor, “Interactive ai with retrieval-augmented generation for next generation networking,” IEEE Network , 2024

  12. [19]

    Tree of thoughts: Deliberate problem solving with large language models,

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y . Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” NeurIPS, vol. 36, 2024

  13. [20]

    Re- flexion: Language agents with verbal reinforcement learning,

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Re- flexion: Language agents with verbal reinforcement learning,” NeurIPS, vol. 36, 2024

  14. [21]

    Exploring large language models for communication games: An empirical study on werewolf,

    Y . Xu, S. Wang, P. Li, F. Luo, X. Wang, W. Liu, and Y . Liu, “Exploring large language models for communication games: An empirical study on werewolf,” arXiv preprint arXiv:2309.04658 , 2023

  15. [22]

    Metagpt: Meta programming for multi-agent collaborative framework,

    S. Hong, X. Zheng, J. Chen, Y . Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou et al. , “Metagpt: Meta programming for multi-agent collaborative framework,” arXiv preprint arXiv:2308.00352 , 2023

  16. [23]

    S3: Social-network simulation system with large language model- empowered agents,

    C. Gao, X. Lan, Z. Lu, J. Mao, J. Piao, H. Wang, D. Jin, and Y . Li, “S3: Social-network simulation system with large language model- empowered agents,” arXiv preprint arXiv:2307.14984 , 2023

  17. [24]

    Closing the loop for patients with parkinson disease: where are we?

    H. Teymourian, F. Tehrani, K. Longardner, K. Mahato, T. Podhajny, J.- M. Moon, Y . G. Kotagiri, J. R. Sempionatto, I. Litvan, and J. Wang, “Closing the loop for patients with parkinson disease: where are we?” Nature Reviews Neurology , vol. 18, no. 8, pp. 497–507, 2022

  18. [25]

    Exploring autonomous agents through the lens of large language models: A review,

    S. Barua, “Exploring autonomous agents through the lens of large language models: A review,” arXiv preprint arXiv:2404.04442 , 2024

  19. [26]

    Larp: Language-agent role play for open-world games,

    M. Yan, R. Li, H. Zhang, H. Wang, Z. Yang, and J. Yan, “Larp: Language-agent role play for open-world games,” arXiv preprint arXiv:2312.17653, 2023

  20. [27]

    Characterchat: Learning towards conversational ai with personalized social support,

    Q. Tu, C. Chen, J. Li, Y . Li, S. Shang, D. Zhao, R. Wang, and R. Yan, “Characterchat: Learning towards conversational ai with personalized social support,” arXiv preprint arXiv:2308.10278 , 2023

  21. [28]

    Roleinteract: Evaluating the social interaction of role-playing agents,

    H. Chen, H. Chen, M. Yan, W. Xu, X. Gao, W. Shen, X. Quan, C. Li, J. Zhang, F. Huang et al., “Roleinteract: Evaluating the social interaction of role-playing agents,” arXiv preprint arXiv:2403.13679 , 2024

  22. [29]

    Lamp: When large language models meet personalization,

    A. Salemi, S. Mysore, M. Bendersky, and H. Zamani, “Lamp: When large language models meet personalization,” arXiv preprint arXiv:2304.11406, 2023

  23. [30]

    Exploring the role of simulation training in improving surgical skills among residents: a narrative review,

    S. A. Cardoso, J. Suyambu, J. Iqbal, D. C. C. Jaimes, A. Amin, J. T. Sikto, M. Valderrama, S. S. Aulakh, V . Ramana, B. Shaukat et al. , “Exploring the role of simulation training in improving surgical skills among residents: a narrative review,” Cureus, vol. 15, no. 9, 2023

  24. [31]

    Internlm2 technical report,

    Z. Cai, M. Cao, H. Chen, K. Chen, K. Chen, X. Chen, X. Chen, Z. Chen, Z. Chen, P. Chu et al. , “Internlm2 technical report,” arXiv preprint arXiv:2403.17297, 2024

  25. [32]

    Introducing meta llama 3: The most capable openly available llm to date,

    A. Meta, “Introducing meta llama 3: The most capable openly available llm to date,” Meta AI , 2024

  26. [33]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

  27. [34]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y . Cao, “React: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.