Pith. sign in

REVIEW 4 major objections 5 minor 38 references

Large Language Model-Based Agents for Automated Research Reproducibility: An Exploratory Study in Alzheimer's Disease

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a team of autonomous large language model agents, given only an abstract, a methods section, and a data dictionary, can approximately reproduce about half of the key findings of published Alzheimer's disease…

desk verdict A genuinely new, honestly reported exploratory benchmark for LLM-agent reproducibility, whose headline percentage should be read as alignment, not validated reproduction. read the letter →

arxiv 2505.23852 v1 pith:K2J7G7O5 submitted 2025-05-29 cs.CL cs.AIcs.MAstat.AP

Nic Dobbins , Christelle Xiong , Kristine Lan , Meliha Yetisgen This is my paper
classification cs.CLcs.AIcs.MAstat.AP
keywords largelanguagemodelsLLMagentsresearchreproducibilityAlzheimer'sdiseaseautomatedscientificvalidationnaturalprocessingagent-basedreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether large language models, organized as an autonomous team of agents that can write and execute code, can reproduce the findings of published research studies on their own. The authors took five highly cited Alzheimer's disease studies that all use the same shared registry dataset, fed the agents only each paper's abstract, methods section, and relevant data-dictionary entries, and let the agents plan, code, run, and revise their analyses. Across 35 key findings extracted from the five abstracts, the agents approximately reproduced 53.2% of findings per study on average, with Boolean and directional claims aligning much more often than numeric or range-based values. In none of the five studies did the agents use exactly the same statistical methods as the original paper, and in 6 of 14 method comparisons the divergence was serious enough that the reproduced result was not justifiable by the agent's own approach. The paper concludes that structured agent-based systems are a promising route to scalable reproducibility checking, but that missing methodological detail and shallow implementation of domain-specific methods currently block full automation.

What carries the argument

The central object is the simulated research team: a set of large language model agents with distinct roles — a planner, an engineer, a scientist, a critic, a code executor, and a manager — coordinated through a multi-agent conversation loop. Given one prompt containing the study's abstract, methods section, relevant data-dictionary definitions, and ground rules for handling missing values and cohort criteria, the team iterates through plan, write-code, execute, critique, and revise cycles until it judges that the abstract's findings have been addressed. The mechanism that carries the argument is this plan-write-execute-critique loop: it converts an open-ended reproducibility goal into concrete, checkable outputs that can be compared against manually extracted abstract assertions, and it lets the authors attribute successes and failures to specific steps in the loop.

What would settle it

Obtain the exact historical data freezes used by each of the five studies, such as the data requested in November 2020 with a September 2020 freeze for the racial-disparities study, and rerun the same agent pipeline on those versions; if the reproduction rate moves materially away from 53.2%, the dataset-equivalence assumption is the reason, and the agents' apparent success or failure was not measuring what the authors intended.

Watch

Extended reading notes

Core claim

The paper's central claim is that an LLM-based agent pipeline can partially automate study reproduction in a realistic biomedical setting: given only the abstract, methods text, and data dictionary entries for a study, a team of role-specialized agents produced results aligned with 53.2% of the study's abstract findings on average. The agents recovered the direction and significance of most Boolean or comparative findings, but numeric and range-based outputs frequently differed from the published values. The authors report that the agents' statistical methods matched the originals exactly in 0 of 5 studies; 8 of 14 method comparisons were deemed justifiable, mostly generic tests such as chi-square, t-tests, and z-scores, while domain-specific methods such as demographically corrected regression were either skipped or replaced with simpler unadjusted analyses. They counted 26 code-execution failures in 55 attempts, mostly from wrong column names, all eventually corrected through iterative feedback. On the authors' reading, the failures stem less from the model's coding ability than from absent methodological detail in the original papers and from the agents' lack of domain-specific statistical knowledge.

Load-bearing premise

The load-bearing assumption is that the local copy of the shared dataset is structurally identical to the versions the five original studies used and only slightly newer; if the historical versions differed in coding, cohort composition, or variable availability, the observed match rates could reflect data differences rather than the agents' capabilities.

Editorial extensions

If this is right

  • A structured LLM agent pipeline can serve as a first-pass reproducibility screen: studies whose abstract findings are mostly reproduced on a shared dataset are likely to be more methodologically transparent and robust.
  • Numeric and range-based findings are the weak point of automated reproduction; any future benchmark should weight these more heavily than Boolean assertions, which can align even when the underlying values differ.
  • Domain-specific statistical methods, such as demographically adjusted regression and network robustness checks, are the main source of unjustifiable divergence; adding access to cited methodological references or a domain-knowledge module should be a priority.
  • The high frequency of correctable code-execution errors (26 of 55 attempts) suggests that operational failures are a solvable engineering problem rather than a fundamental limit of LLM agents.
  • Evaluation protocols for automated reproducibility should report not only finding-level alignment but also method-level justifiability, since the two can diverge sharply.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the agents were compared only against abstract findings and given a slightly newer dataset, the 53.2% alignment is not a clean estimate of reproducibility; it is an upper bound for exact-value matching and a lower bound for trend reproduction, so it should be read as a proof of concept rather than a rate.
  • The same pipeline could be run in reverse: systematically vary the information given to agents (abstract only, methods only, plus cited references, plus tables and figures) to measure how much of the reproduction gap is due to missing inputs versus model capability.
  • A concrete testable extension would be to give agents access to the cited methodological papers, such as the regression-based normative scoring procedure, and see whether the justifiability rate rises from 8 of 14; this would separate prompt incompleteness from genuine LLM limitations.
  • The observed slippage from demographically adjusted to unadjusted regression is a caution for any automated or LLM-assisted statistical analysis: without explicit verification, a confident agent can produce a numerically different and statistically invalid answer that still looks plausible.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents an exploratory study of using GPT-4o-based multi-agent systems (built with Autogen) to reproduce findings from five highly cited Alzheimer's disease studies using the NACC Quick Access dataset. The authors manually extracted 35 key findings from the abstracts, provided the agents with the abstracts, methods sections, and data dictionary descriptions, and then evaluated whether the agents' outputs aligned with those findings under a tolerance rule (within 1.0 for numeric values, Boolean match otherwise). The paper reports that on average 53.2% of findings per study were approximately reproduced, that in 3/5 studies a majority of findings were replicated, and that in 0/5 studies were the statistical methods exactly the same. The authors provide a detailed error analysis and discuss limitations including dataset version differences and methodological divergences.

Significance. If the central quantitative claim is made robust, this is a valuable early baseline for automated reproducibility assessment. The paper is the first to our knowledge to apply LLM-based multi-agent systems to reproduce findings from real published biomedical studies, using a real public dataset. The methodology is clearly described, the authors are appropriately hedged, and the detailed error analysis (Table 2, Figure 4) is a strength. The availability of transcripts and code (once released) would further support reproducibility of the agent study itself. The main contribution is an empirical demonstration of both promise and limitations, which can guide future work in this emerging area.

major comments (4)
  1. [Section 3.4 and Table 2] The headline reproduction rate (abstract: 53.2% average per study; Section 2.3: 25/35) is computed solely from Table 1's alignment rule, which checks whether agent output numerically or Boolean-matches the abstract assertion. It does not require that the agent's statistical method be justifiable as defined in Table 2. For example, in the 'Examining racial...' study [33], the agent's unadjusted z-scores are labeled 'Not justifiable' in Table 2 because demographic adjustment was omitted, yet the Table 1 rows for 'Whites higher rate low scores: memory/attention/processing speed/verbal fluency' and 'Whites more likely meet actuarial criteria' are marked aligned because the Boolean outcomes match. A Boolean True/True can occur even when the analysis is invalid. Because the paper itself separates method justifiability from outcome alignment, the reported reproduction rate conflates two different senses of reproduction. Please recompute and report the reproduction rate conditioned on method justifiability, and restate the abstract's headline with this caveat.
  2. [Section 3.2] The dataset equivalence assumption — that the local spring 2023 Quick Access dataset is structurally identical to the data versions used by the original studies but contains slightly more rows — is explicitly acknowledged but never validated. The original studies used earlier data freezes with specific cohort definitions (e.g., [33] used a September 2020 freeze and a final sample of 7,201 participants after exclusions). Differences in coding, cohort composition, or variable availability could materially affect both numeric and Boolean comparisons. The authors should validate this assumption (e.g., by comparing summary statistics or requesting historical data versions) or present a sensitivity analysis showing that the reported reproduction rate is robust to plausible cohort differences. Without this, the observed pattern of successes and failures cannot be cleanly attributed to LLM agent capability.
  3. [Section 3.4] The alignment rubric is underspecified. The rule 'within a value of 1' is stated for means, medians, or percentages, but Table 1 contains ranges (e.g., 'MBR of low scores range 1.40 to 79.2%' vs. '10.9 to 51.3%', 'Median presentation age 74 to 75' vs. '73 to 74.5') and p-values (e.g., 'Network structure p-value 0.71' vs. '0.42') without a clear rule for how these are compared. This ambiguity directly affects the count of aligned findings and thus the headline percentage. Please provide a complete, pre-specified scoring rule for ranges and p-values, or report how the results change under alternative reasonable rules.
  4. [Section 3.3 and Data Availability] The paper does not provide the full prompts, agent role definitions, or the complete transcripts needed to reproduce the agent study itself. The Appendix contains only one example reproduction prompt (for [33]), and the GitHub link in footnote 2 says the code 'will be made available upon article acceptance,' while Section 10 states 'There are no new data associated with this article.' Given that the paper is about reproducibility, the authors should either include the full agent system prompts, role definitions, and transcripts as supplementary material, or clarify their availability and update the Data Availability statement accordingly.
minor comments (5)
  1. [Section 3.4] The sentence 'We considered a finding aligned if, if a mean, median, or percentage, agent findings were within a value of 1' contains a grammatical error ('if, if') and does not specify whether the tolerance is absolute or relative; please clarify.
  2. [Abstract vs. Section 2.3] The abstract reports 'approximately reproduced 53.2% of findings per study' while Section 2.3 reports '25/35 (71.4%) of key findings automatically reproduced.' These two numbers are not reconciled in the text; please define the computation that yields 53.2% and explain the relationship to the 71.4% figure.
  3. [Section 5.1] The statement that differences in statistical techniques 'are unlikely to have led to unjustified improvements in the results' is not demonstrated and appears to conflict with Table 2, where six of fourteen method comparisons are labeled 'Not justifiable' (e.g., unadjusted regression in [32] omitting key covariates). Please either provide evidence or soften this claim.
  4. [Throughout] There are several typographical and minor wording issues, including 'intergration' (Section 2.1), 'keywork search' (Section 3.3), '5 studies appeared reproducible' (missing 'that'), and 'producibility' in Section 5.1 (likely 'reproducibility'). These should be corrected.
  5. [Section 3.3] The paper does not report the temperature, random seed, or other sampling parameters for GPT-4o, yet LLM outputs are stochastic; please state these parameters or explain why they are not expected to affect the qualitative conclusions.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the reproduction rate is an empirical measurement under the paper's stated alignment rule and could have failed (and partially did); the only self-citations are background and non-load-bearing.

full rationale

This paper contains no circular derivation. Its central output, the 53.2% average reproduction rate per study (abstract) and the 25/35 (71.4%) figure (Section 2.3), is an empirical measurement: the authors manually extracted 35 key findings from study abstracts (Section 3.4), prompted GPT-4o agents with each study's Abstract, Methods, and a curated list of NACC variables (Section 3.3), and scored agent outputs under a stated alignment rule (Section 3.4: numeric findings within 1; Boolean findings matched true-to-true or false-to-false). The rate is not equivalent to its inputs by construction: the rule could have been failed, and frequently was, since 0/5 studies had exactly matching statistical methods and many numeric values diverged (e.g., Whites meeting actuarial MCI criteria: 71.6 in the original versus 48.9 from the agent). No parameter was fitted to a subset of the target data and then renamed as a prediction; prompt design was tuned only on a separate development study [37]. The self-citations in the introduction (refs [3] and [6], the first author's prior work on concept normalization and cohort discovery) support only background claims about general LLM capabilities and are not load-bearing for the reproduction-rate claim, which rests on the executed agent experiments and archived transcripts. The legitimate concerns the paper itself raises, the assumed schema equivalence of the locally held NACC dataset to historical versions (Section 3.2), the counting as 'aligned' of findings whose agent methods Table 2 labels 'Not justifiable' (e.g., unadjusted z-scores or omitted demographic adjustment in [32,33]), and the internal tension between 53.2% (abstract) and 71.4% (Section 2.3), are threats to the validity or interpretation of the metric, not reductions of the claim to its own inputs; the authors explicitly acknowledge the alignment-versus-reproduction conflation in Section 5.1. The scoring rubric is self-defined, but the outcome is empirically contingent, so no circular step is exhibited.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No new physical or theoretical entities are postulated. The central claim rests on domain assumptions about data equivalence, sufficiency of abstract and methods text, and the validity of manual assertion extraction, plus one hand-chosen alignment tolerance. The agent roles are software configurations, not invented entities.

free parameters (1)
  • alignment tolerance (reproduction threshold) = 1.0 units for means, medians, and percentages
    Hand-chosen in Section 3.4 as the criterion for considering a numeric finding aligned. This threshold directly determines the reported 53.2% reproduction rate and is not justified by any distributional or practical argument.
assumptions (3)
  • domain assumption The local NACC Quick Access dataset is structurally identical in schema to the data used in the original studies and contains only slightly more rows.
    Stated in Section 3.2; if false, observed failures may reflect data mismatch rather than LLM agent limitations.
  • domain assumption Information in abstracts and methods sections, without figures and tables, is sufficient for reproducing the abstract-level findings.
    Stated in Section 3.3; the authors explicitly assume figures and tables are not essential.
  • domain assumption Manual extraction of key assertions from abstracts is a valid and adequate benchmark for evaluating reproduction.
    Section 3.4; the 35 key findings are manually curated, and no inter-rater reliability or external benchmark is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large Language Model-Based Agents for Automated Research Reproducibility: An Exploratory Study in Alzheimer's Disease." pith.science (2026). https://pith.science/paper/K2J7G7O5

@misc{pith2026250523852,
  author       = {Pith},
  title        = {Pith review of: Large Language Model-Based Agents for Automated Research Reproducibility: An Exploratory Study in Alzheimer's Disease},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K2J7G7O5}},
  note         = {Machine review of arXiv:2505.23852}
}
read the original abstract

Objective: To demonstrate the capabilities of Large Language Models (LLMs) as autonomous agents to reproduce findings of published research studies using the same or similar dataset. Materials and Methods: We used the "Quick Access" dataset of the National Alzheimer's Coordinating Center (NACC). We identified highly cited published research manuscripts using NACC data and selected five studies that appeared reproducible using this dataset alone. Using GPT-4o, we created a simulated research team of LLM-based autonomous agents tasked with writing and executing code to dynamically reproduce the findings of each study, given only study Abstracts, Methods sections, and data dictionary descriptions of the dataset. Results: We extracted 35 key findings described in the Abstracts across 5 Alzheimer's studies. On average, LLM agents approximately reproduced 53.2% of findings per study. Numeric values and range-based findings often differed between studies and agents. The agents also applied statistical methods or parameters that varied from the originals, though overall trends and significance were sometimes similar. Discussion: In some cases, LLM-based agents replicated research techniques and findings. In others, they failed due to implementation flaws or missing methodological detail. These discrepancies show the current limits of LLMs in fully automating reproducibility assessments. Still, this early investigation highlights the potential of structured agent-based systems to provide scalable evaluation of scientific rigor. Conclusion: This exploratory work illustrates both the promise and limitations of LLMs as autonomous agents for automating reproducibility in biomedical research.

Figures

Figures reproduced from arXiv: 2505.23852 by the authors.

Figure 1
Figure 1. Process for determining eligible studies for inclusion. Our initial review of citing papers for the NACC version 3 manuscript [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. 1. Planner - suggests and revises a plan for the agent team to follow based on instructions from the user. 3https://scholar.google.com/scholar?cites=15070434224497513322&as_sdt= 5,48&sciodt=0,48&hl=en 4https://github.com/microsoft/autogen/tree/0.2/notebook 6 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 2
Figure 2. Agent roles in this study. Individual agent instructions (shown in quotes) are abbreviated for readability. The entire agent [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: Process for prompting agents and progressing the conversation simulation. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png]
Figure 4
Figure 4. Figure 4: Sequence of speakers for each study replication attempt. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 25 canonical work pages

  1. [33]

    Examining racial disparities in the diagnosis of mild cognitive impairment

    Andrew M Kiselica, Ellen Johnson, Kaleea R Lewis, and Kate Trout. Examining racial disparities in the diagnosis of mild cognitive impairment. Applied Neuropsychology: Adult, 30(6):749–756, 2023

    work page 2023

  2. [1]

    Large language models for generative information extraction: A survey

    Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng, Yang Wang, and Enhong Chen. Large language models for generative information extraction: A survey. Frontiers of Computer Science, 18(6):186357, 2024

    2024

  3. [2]

    Llms4ol: Large language models for ontol- ogy learning

    Hamed Babaei Giglou, Jennifer D’Souza, and S ¨oren Auer. Llms4ol: Large language models for ontol- ogy learning. In International Semantic Web Conference, pages 408–427. Springer, 2023

    work page 2023

  4. [3]

    Generalizable and Scalable Multistage Biomedical Concept Normalization Leveraging Large Language Models

    Nicholas J Dobbins. Generalizable and scalable multistage biomedical concept normalization leverag- ing large language models. arXiv preprint arXiv:2405.15122, 2024

    work page Pith review arXiv 2024

  5. [4]

    Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine

    Thomas Savage, Ashwin Nayak, Robert Gallo, Ekanath Rangan, and Jonathan H Chen. Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine. NPJ Digital Medicine, 7(1):20, 2024

    2024

  6. [5]

    Potential use cases for chatgpt in radiology reporting

    Asser Abou Elkassem and Andrew D Smith. Potential use cases for chatgpt in radiology reporting. American Journal of Roentgenology, 221(3):373–376, 2023

    work page 2023

  7. [6]

    Leafai: query generator for clinical cohort discovery rivaling a human programmer

    Nicholas J Dobbins, Bin Han, Weipeng Zhou, Kristine F Lan, H Nina Kim, Robert Harrington, ¨Ozlem Uzuner, and Meliha Yetisgen. Leafai: query generator for clinical cohort discovery rivaling a human programmer. Journal of the American Medical Informatics Association, 30(12):1954–1964, 2023

    work page 1954

  8. [7]

    Codeagent: Enhancing code generation with tool- integrated agent systems for real-world repo-level coding challenges.arXiv preprint arXiv:2401.07339, 2024

    Kechi Zhang, Jia Li, Ge Li, Xianjie Shi, and Zhi Jin. Codeagent: Enhancing code generation with tool- integrated agent systems for real-world repo-level coding challenges.arXiv preprint arXiv:2401.07339, 2024

    arXiv 2024

Show all 38 references
  1. [8]

    Autonomous chemical research with large language models

    Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models. Nature, 624(7992):570–578, 2023

    2023

  2. [9]

    Reproducibility crisis

    Monya Baker. Reproducibility crisis. nature, 533(26):353–66, 2016

    2016

  3. [10]

    Reproducibility in science: improving the standard for basic and preclinical research

    C Glenn Begley and John PA Ioannidis. Reproducibility in science: improving the standard for basic and preclinical research. Circulation research, 116(1):116–126, 2015

    2015

  4. [11]

    Empowering biomedical discovery with ai agents

    Shanghua Gao, Ada Fang, Yepeng Huang, Valentina Giunchiglia, Ayush Noori, Jonathan Richard Schwarz, Yasha Ektefaie, Jovana Kondic, and Marinka Zitnik. Empowering biomedical discovery with ai agents. Cell, 187(22):6125–6151, 2024

    2024

  5. [12]

    Autogen: Enabling next-gen llm applications via multi-agent conver- sation framework

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen llm applications via multi-agent conver- sation framework. arXiv preprint arXiv:2308.08155, 2023. 17

    2023 arXiv

  6. [13]

    The national alzheimer’s coordinating center (nacc) database: the uniform data set

    Duane L Beekly, Erin M Ramos, William W Lee, Woodrow D Deitrich, Mary E Jacka, Joylee Wu, Janene L Hubbard, Thomas D Koepsell, John C Morris, Walter A Kukull, et al. The national alzheimer’s coordinating center (nacc) database: the uniform data set. Alzheimer Disease & Asso- c...

    2007

  7. [14]

    Is it an agent, or just a program?: A taxonomy for autonomous agents

    Stan Franklin and Art Graesser. Is it an agent, or just a program?: A taxonomy for autonomous agents. In International workshop on agent theories, architectures, and languages , pages 21–35. Springer, 1996

    1996

  8. [15]

    Modeling adaptive autonomous agents

    Pattie Maes. Modeling adaptive autonomous agents. Artificial life, 1(1 2):135–162, 1993

    1993

  9. [16]

    Intelligent agents: Theory and practice

    Michael Wooldridge and Nicholas R Jennings. Intelligent agents: Theory and practice. The knowledge engineering review, 10(2):115–152, 1995

    1995

  10. [17]

    Integrating web resources and lexicons into a natural language query system

    Boris Katz, Deniz Yuret, Jimmy Lin, Sue Felshin, Rebecca Schulman, Adnan Ilik, Ali Ibrahim, and Philip Osafo-Kwaako. Integrating web resources and lexicons into a natural language query system. In Proceedings IEEE International Conference on Multimedia Computing and Systems, v...

    1999

  11. [18]

    A survey on large language model based autonomous agents

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024

    2024

  12. [19]

    Chain of hindsight aligns language models with feedback

    Hao Liu, Carmelo Sferrazza, and Pieter Abbeel. Chain of hindsight aligns language models with feedback. arXiv preprint arXiv:2302.02676, 2023

    2023 arXiv

  13. [20]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. arxiv. arXiv preprint ArXiv:2304.03442, 2023

    2023 arXiv

  14. [21]

    The socialai school: Insights from developmental psychology towards artificial socio-cultural agents

    Grgur Kova ˇc, R ´emy Portelas, Peter Ford Dominey, and Pierre-Yves Oudeyer. The socialai school: Insights from developmental psychology towards artificial socio-cultural agents. arXiv preprint arXiv:2307.07871, 2023

    2023 arXiv

  15. [22]

    Chatlaw: Open-source legal large language model with integrated external knowledge bases

    Jiaxi Cui, Zongjian Li, Yang Yan, Bohua Chen, and Li Yuan. Chatlaw: Open-source legal large language model with integrated external knowledge bases. CoRR, 2023

    2023

  16. [23]

    Blind judgement: Agent-based supreme court modelling with gpt

    Sil Hamilton. Blind judgement: Agent-based supreme court modelling with gpt. arXiv preprint arXiv:2301.05327, 2023

    2023 arXiv

  17. [24]

    Can large language models transform computational social science? Computational Linguistics, 50(1):237–291, 2024

    Caleb Ziems, William Held, Omar Shaikh, Jiaao Chen, Zhehao Zhang, and Diyi Yang. Can large language models transform computational social science? Computational Linguistics, 50(1):237–291, 2024. 18

    2024

  18. [25]

    Can generative ai improve social science? Proceedings of the National Academy of Sciences, 121(21):e2314021121, 2024

    Christopher A Bail. Can generative ai improve social science? Proceedings of the National Academy of Sciences, 121(21):e2314021121, 2024

    2024

  19. [26]

    Reproducibility in automated chemistry laboratories using computer science abstractions

    Richard B Canty and Milad Abolhasani. Reproducibility in automated chemistry laboratories using computer science abstractions. Nature Synthesis, pages 1–13, 2024

    2024

  20. [27]

    Reproducibility of computational workflows is automated using continuous analysis

    Brett K Beaulieu-Jones and Casey S Greene. Reproducibility of computational workflows is automated using continuous analysis. Nature biotechnology, 35(4):342–346, 2017

    2017

  21. [28]

    Sierra: A modular framework for research automation and reproducibil- ity

    John Harwell and Maria Gini. Sierra: A modular framework for research automation and reproducibil- ity. arXiv preprint arXiv:2208.07805, 2022

    2022 arXiv

  22. [29]

    Autonomous llm-driven re- search—from data to human-verifiable research papers

    Tal Ifargan, Lukas Hafner, Maor Kern, Ori Alcalay, and Roy Kishony. Autonomous llm-driven re- search—from data to human-verifiable research papers. NEJM AI, 2(1):AIoa2400555, 2025

    2025

  23. [30]

    Version 3 of the na- tional alzheimer’s coordinating center’s uniform data set

    Lilah Besser, Walter Kukull, David S Knopman, Helena Chui, Douglas Galasko, Sandra Wein- traub, Gregory Jicha, Cynthia Carlsson, Jeffrey Burns, Joseph Quinn, et al. Version 3 of the na- tional alzheimer’s coordinating center’s uniform data set. Alzheimer Disease & Associated D...

    2018

  24. [31]

    Development and preliminary validation of standardized regression-based change scores as measures of transitional cognitive decline

    Andrew M Kiselica, Alyssa N Kaser, Troy A Webber, Brent J Small, and Jared F Benge. Development and preliminary validation of standardized regression-based change scores as measures of transitional cognitive decline. Archives of Clinical Neuropsychology, 35(7):1168–1181, 2020

    2020

  25. [32]

    Using multivariate base rates of low scores to understand early cognitive declines on the uniform data set 3.0 neuropsychological battery

    Andrew M Kiselica, Troy A Webber, and Jared F Benge. Using multivariate base rates of low scores to understand early cognitive declines on the uniform data set 3.0 neuropsychological battery. Neuropsy- chology, 34(6):629, 2020

    2020

  26. [34]

    Network structures and temporal stability of self-and informant-rated affective symptoms in alzheimer’s disease

    Toni T Saari, Ilona Hallikainen, Taina Hintsa, and Anne M Koivisto. Network structures and temporal stability of self-and informant-rated affective symptoms in alzheimer’s disease. Journal of affective disorders, 276:1084–1092, 2020

    2020

  27. [35]

    Ethnoracial differ- ences in lewy body diseases with cognitive impairment

    Andrea M Kurasz, Glenn E Smith, Maria G McFarland, and Melissa J Armstrong. Ethnoracial differ- ences in lewy body diseases with cognitive impairment. Journal of Alzheimer’s Disease, 77(1):165– 174, 2020

    2020

  28. [36]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 19

    2024 arXiv

  29. [37]

    Robust reference group normative data for neuropsychological tests accounting for primary language use in asian american older adults

    Arunima Kapoor, Jean K Ho, Jung Yun Jang, and Daniel A Nation. Robust reference group normative data for neuropsychological tests accounting for primary language use in asian american older adults. Journal of the International Neuropsychological Society, 30(4):402–409, 2024. 20

    2024

  30. [38]

    NACCAGE" - Subject’s age at visit. Form:

    Appendix 11.1. Appendix A: Prompt for Study Reproduction 11.1.1. Prompt Reproduce the results of the following study: Examining racial disparities in the diagnosis of mild cognitive impairment. Methods: Sample. We requested data through the National Alzheimer’s Coordinating We...

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.