Pith. sign in

REVIEW 3 major objections 4 minor 33 references

Score geospatial dataset FAIRness for $0.054 each

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:19 UTC pith:XIZSAQNL

load-bearing objection AgentFAIR is a genuinely useful, honestly limited feasibility study of LLM-based FAIR scoring for geospatial data; the 82% expert-alignment number is real but not yet independently testable, and the paper knows it. the 3 major comments →

arxiv 2607.15781 v2 pith:XIZSAQNL submitted 2026-07-17 cs.AI cs.ETcs.MA

AgentFAIR: A Multi-Agent Collaborative Framework for FAIRness Evaluation of Geospatial Datasets

Ming Chen , Pranav Pai This is my paper
classification cs.AI cs.ETcs.MA
keywords FAIR principlesgeospatial datasetsmulti-agent systemslarge language modelsmetadata qualityFAIRness evaluationmaturity rubriccritic agent
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that a multi-agent LLM framework — one specialized evaluator per FAIR sub-principle, plus a critic that checks evidence and consistency — can produce auditable, low-cost FAIR-compliance scores for geospatial datasets. On a 50-dataset, 10-repository sample it reports strong Findability (79.7%) and Reusability (72.0%), moderate Accessibility (70.4%), and weak Interoperability (45.3%), with the biggest gaps in vocabulary registration and typed references. Repeated runs agree on 89% of sub-principle scores with the critic versus 71% without, and the system matches the expert majority on 82% of 195 sub-principle judgments at roughly $0.05 per dataset. The authors offer this as evidence of feasibility and auditability, while cautioning that the small benchmark and single-model validation limit claims about accuracy.

Core claim

AgentFAIR is a three-stage pipeline that renders each dataset's landing page, extracts machine-readable metadata together with provenance-linked snippets, and then runs 13 sub-principle-specific LLM evaluators, each applying the same explicit 0–3 maturity rubric with geospatial indicators. Every non-zero score must cite observable evidence; a critic agent performs hard logical checks and cross-principle consistency checks and can trigger a targeted re-evaluation when evidence is missing or low-confidence. The paper reports mean dimension scores of 79.7% for Findability, 70.4% for Accessibility, 45.3% for Interoperability, and 72.0% for Reusability across 50 datasets; sub-principle agreement

What carries the argument

The load-bearing mechanism is the critic-driven feedback loop: 13 sub-principle-specific LLM agents, each governed by an explicit 0–3 maturity rubric (with geospatial indicators such as CRS/EPSG identifiers, ISO 19115 elements, and OGC endpoints), must attach provenance-linked evidence to every non-zero score. A critic agent then checks evidence sufficiency and cross-principle consistency, triggering targeted re-evaluation when hard checks fail or confidence is low. This critic is what lifts repeated-run sub-principle agreement from 71% to 89% in the reported ablation.

Load-bearing premise

The scores are only as valid as the author-defined 0–3 rubric and the LLM's reading of page snippets; if that rubric is not a faithful operationalization of FAIR for geospatial data, the whole measurement is off.

What would settle it

Run the same pipeline on a held-out set of geospatial datasets and have independent auditors score them with a rubric not designed by the authors. If AgentFAIR's alignment with the external rubric drops far below the reported 82%, or if repeated runs on JavaScript-heavy pages diverge by more than the reported standard deviation, the feasibility and auditability claims would be falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If AgentFAIR's results hold across more datasets, geospatial repositories could deploy continuous, evidence-linked FAIR monitoring for roughly $0.05 per dataset, making stewardship decisions auditable and cheap.
  • The consistently weak Interoperability scores (mean 45.3%, with I2 and I3 near the bottom) imply that geospatial FAIRness improvements depend on ecosystem-level vocabulary registration and typed-reference exposure, not just on dataset-level metadata quality.
  • The critic's 18-percentage-point improvement in repeated-run agreement suggests that evidence-based consistency checking is a key ingredient for reliable LLM-based evaluation, not a minor add-on.
  • The 82% agreement with expert consensus (experts themselves at Fleiss' kappa 0.71) suggests the 0–3 rubric captures a shared interpretation of geospatial FAIRness, at least within the paper's sample.
  • The diagnostic finding that normalized scores across four existing evaluators vary by a mean per-dataset standard deviation of 15.0 points means FAIR scores remain tool-dependent; AgentFAIR provides one auditable signal rather than a universal ground truth.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the expert study and the automated system used the same author-defined rubric, the 82% alignment may partly reflect shared criteria rather than an external ground truth; a more decisive test would score the same datasets with an independent rubric.
  • The paper's own I2 registry-count criterion is likely under-crediting valid community vocabularies; extending the rubric to include vocabulary documentation, resolvability, and governance could raise Interoperability scores without loosening standards.
  • The critic's high trigger rate (96%) and the absence of trigger-type breakdown leave open whether it mainly corrects errors or mostly enforces consistency; a natural follow-up is to compare critic-revised scores against expert labels on the triggered cases alone.
  • With mean processing time around 1,054 seconds per dataset, the pipeline is batch-oriented rather than interactive; caching evidence and using smaller models could make near-real-time FAIR monitoring feasible, though the paper does not explore this.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents AgentFAIR, a multi-agent LLM-based framework for evaluating the 13 FAIR sub-principles of geospatial datasets. The pipeline combines Playwright-based crawling of JavaScript-rendered landing pages, structured metadata extraction, per-sub-principle GPT-4o-mini evaluators using a 0–3 maturity rubric, and a critic agent that checks evidence sufficiency and cross-principle consistency and may trigger re-evaluation. On 50 datasets across 10 repositories, the authors report mean Findability 79.7%, Accessibility 70.4%, Interoperability 45.3%, and Reusability 72.0%; a mean across-tool per-dataset standard deviation of 15.0 points; 89% repeated-run exact sub-principle agreement with the critic versus 71% without; 82% agreement with expert majority on 195 judgments across 15 datasets (Fleiss' kappa 0.71); and a mean API cost of USD 0.054 per dataset. The paper is careful to frame the cross-tool comparison as a disagreement and failure-mode analysis rather than as comparative accuracy, and it explicitly lists unresolved validity limitations.

Significance. If supported, the framework is a useful and honest contribution to automated FAIR assessment: it provides an evidence-linked audit trail, a transparent rubric, deterministic checks, and very low marginal cost. The open-source release and archived snapshot are clear strengths, as is the paper's repeated insistence that tool-score normalization does not establish accuracy. The main significance, however, is feasibility and auditability. The claimed accuracy link to expert judgment is weakened by the fact that the experts applied the same author-constructed rubric that the LLM prompts implement, and because the raw expert labels and repeated-run outputs are not released, the headline agreement statistics cannot be independently recomputed. The paper's own limitation sections acknowledge many of these issues; the task for revision is to make the validity claims match the available evidence.

major comments (3)
  1. [Sec. 5.6, 5.10, B.2, App. F] The 82% expert-alignment result is the only quantitative bridge between AgentFAIR outputs and an external criterion, yet it is not independently verifiable. The expert protocol used the same 0–3 rubric (Table 2) that the LLM prompts implement, and B.2 states that prompts were iteratively refined during development. Consequently, the agreement may partially reflect shared rubric and prompt tuning rather than independent FAIR correctness. Moreover, Appendix F states that raw expert labels, repeated-run outputs, ablation runs, and analysis notebooks are not released. The paper should either release the raw expert labels and confirm that prompts were frozen before any expert labels were collected, or explicitly reframe the 82% as agreement with expert application of the paper's rubric, not as validation of FAIR accuracy. As written, the abstract's accuracy narrative overstates what the evide
  2. [Sec. 5.2, Tables 3 and 5, Sec. 5.7] All dataset-level scores and the resulting means, correlations, and cost figures are based on a single representative run of an LLM pipeline at temperature 0.1. Repeated-run agreement is measured only on a stratified 10-dataset subset, so no run-level uncertainty is attached to the 50-dataset means (e.g., Findability 79.7%, Interoperability 45.3%) or to the Spearman correlations in Table 6. Because LLM stochasticity is acknowledged as inherent to the system, the manuscript should either provide multi-run interval estimates for the full 50-dataset sample or explicitly present all numerical summaries as estimates from one representative run rather than as stable population statistics.
  3. [Table 2 (II2), Sec. 5.3, 5.10, B.2] The headline finding that Interoperability is the weakest dimension is substantially driven by the I2 rubric, which counts vocabulary registry presence as the main evidence of FAIR vocabularies. The paper itself concedes (Sec. 5.10 and B.2) that this criterion can under-credit valid community vocabularies absent from generic registries, and the expert agreement on I2 is the lowest (kappa = 0.58). The mean Interoperability score of 45.3% should therefore be presented as rubric-dependent, and the authors should either provide a sensitivity analysis with a broader I2 operationalization or soften the cross-repository conclusion that geospatial interoperability is systematically weak.
minor comments (4)
  1. [Sec. 5.7] The sentence 'We omit the earlier confidence interval because the retained summary statistics are insufficient to reconstruct it correctly' is a self-referential editorial note about the writing process. It should be replaced with a clean statement of the reported summary statistics.
  2. [Sec. 4.4] The critic confidence thresholds (theta_p; 0.5 generally, 0.4 for I1) and the retry budget are free parameters. No sensitivity analysis is provided, so the reported 89% vs. 71% consistency difference could depend on these specific values. A brief discussion or supplementary experiment on threshold robustness would strengthen the ablation.
  3. [Table 3] The 50-column maturity matrix is visually dense. A machine-readable table in the supplement (beyond the raw code release) would make the per-dataset scores easier to audit and reuse.
  4. [App. F] The 'Missing research artifacts' list is important and should be surfaced in the main text or in a formal data-availability statement, since it directly affects whether the reported inferential statistics can be reproduced.

Circularity Check

0 steps flagged

No significant circularity: reported results are empirical measurements, not derivations from their inputs; the shared expert rubric is an acknowledged validity limitation, not a constructional identity.

full rationale

The paper's central claims are empirical: AgentFAIR produces 0-3 maturity scores from landing-page evidence via the rubric in Section 3.2/Table 2, and these scores are then compared with expert labels, baseline tools, and repeated runs. No equation defines the reported outputs in terms of expert labels or baseline scores, and no parameter is fitted to the target claims. The strongest possible circularity candidate is the 82% expert-alignment result in Section 5.7, since experts and LLM prompts both use the same author-constructed rubric. But this is not circularity in the required sense: the expert labels are independent human applications of the rubric, and agreement is not guaranteed by construction — LLM outputs could and do diverge from expert majority on 18% of judgments. The paper itself repeatedly disclaims this evidence as a definitive ground truth (B.2: 'the preliminary expert sample is not a definitive ground truth'), frames cross-tool comparisons as disagreement rather than accuracy (Section 5.5), and acknowledges that prompts were iteratively refined during development (B.2). Those are genuine validity and reproducibility limitations, not reductions of outputs to inputs. There are no load-bearing self-citations, no imported uniqueness theorems, no ansatz smuggled in via citation, and no renaming of a known result. Therefore no circular step meets the evidentiary bar required here.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The reported scores depend on author-chosen rubric semantics, thresholds, and prompt design rather than on a derived law. The main cost and consistency numbers are empirical measurements; the validity of the rubric and of expert agreement as ground truth are assumptions, not externally anchored. No invented natural entities appear.

free parameters (3)
  • critic confidence thresholds theta_p = 0.5 (0.4 for I1)
    Hand-chosen trigger for re-evaluation; 96% of sub-principle assessments trigger, and no trigger-type breakdown or threshold ablation is released.
  • LLM temperature = 0.1
    Chosen to permit lexical variation while stabilizing scores; no 0.0 versus 0.1 ablation was conducted.
  • retry budget = not specified
    Limits critic-triggered re-evaluations and affects scores and cost; the value is not reported.
axioms (5)
  • domain assumption The Table 2 rubric is a faithful operationalization of the FAIR Guiding Principles for geospatial data.
    Every score inherits this. The authors note I2's registry-count criterion can under-credit valid community vocabularies (Sec. 5.10, B.2).
  • domain assumption LLM outputs constrained by rubric and evidence rules are meaningful compliance measurements.
    This is the method's central mechanism; it is validated only via a 15-dataset expert pilot and repeated-run agreement, not an external ground truth.
  • domain assumption Expert majority labels using the same rubric are a valid reference for alignment.
    Human labels are used for the 82% alignment claim; experts did not see tool outputs but were calibrated on the authors' rubric, so agreement is partly internal to the rubric design.
  • domain assumption Normalized 0-100 scores from heterogeneous tools can diagnose disagreement.
    The authors explicitly state that normalization does not make metrics equivalent and is used only for disagreement/failure-mode analysis (Sec. 5.2).
  • standard math Statistical tests (Spearman, paired t-test, Fleiss kappa) are valid on the small samples used.
    Used for correlations, the critic ablation p-value, and expert kappa; sample sizes are n=50, n=10, and n=15.

pith-pipeline@v1.3.0-alltime-deepseek · 20134 in / 12692 out tokens · 86203 ms · 2026-08-01T22:19:38.959584+00:00 · methodology

0 comments
read the original abstract

Geospatial datasets support applications from urban planning to climate modeling, yet consistent assessment of FAIR compliance is difficult. Existing evaluators use different rubrics and evidence sources and may fail on JavaScript-rendered pages or repository-specific identifiers. For 50 datasets from 10 repositories, the standard deviation of normalized scores across available tools averages 15.0 percentage points and reaches 30.3 for one dataset. Because these outputs are not equivalent measurements, we use them to characterize disagreement and failure modes, not comparative accuracy. We present AgentFAIR, a multi-agent framework combining structured metadata extraction with 13 sub-principle-specific LLM evaluators. Each produces a 0-3 maturity score, cited evidence, and recommendations; a critic checks evidence and consistency and can request targeted re-evaluation. Mean Findability, Accessibility, Interoperability, and Reusability scores are 79.7%, 70.4%, 45.3%, and 72.0%. Rank correlations with four baseline tools range from 0.31 to 0.61; the FAIR-enough comparison is not statistically significant. On a 10-dataset repeated-run subset, sub-principle agreement averages 89% (standard deviation: 3 percentage points), versus 71% without the critic. A preliminary 15-dataset expert study yields Fleiss' kappa of 0.71 and 82% alignment with expert consensus. API cost is approximately USD 0.054 per dataset. These results support auditability and feasibility, while the limited benchmark, incomplete ablations, and single-model-family validation constrain claims about accuracy and generalization.

Figures

Figures reproduced from arXiv: 2607.15781 by Ming Chen, Pranav Pai.

Figure 1
Figure 1. Figure 1: The proposed AgentFAIR framework for multi-agent collaborative FAIRness evaluation. service endpoints), which can systematically depress Interoperabil￾ity—and, downstream, Reusability—scores for spatial datasets. 2.3 LLM Agent for Data Management and Evaluation Large language models (LLMs) have shown promise for extracting, normalizing, and enriching metadata from heterogeneous sources [2, 30]. Recent syst… view at source ↗
Figure 2
Figure 2. Figure 2: Cost-efficiency quadrant: datasets positioned by [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: LangGraph state machine orchestrating FAIR evaluation workflows with conditional edges and retry logic. [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Processing time distribution by FAIR dimension. [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: API cost breakdown by FAIR dimension across 50 datasets. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: FAIR dimension coverage radar chart across tools. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: FAIR principle maturity landscape: mean score vs. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Dataset controversy index: scoring variance across tools. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 6 canonical work pages · 1 internal anchor

  1. [1]

    Christiane Bahlo, Siddeswara Guru, Nicholas Car, and Lesley Wyborn. 2024. Ad- vancing FAIR Agricultural Data: The AgReFed FAIR Assessment Tool.Data Sci- ence Journal23, 1 (2024), 15. doi:10.5334/dsj-2024-018

    work page doi:10.5334/dsj-2024-018 2024

  2. [2]

    Lennart Busch, Daniel Tebernum, and Gissel Velarde. 2025. Exploring LLM Ca- pabilities in Extracting DCAT-Compatible Metadata for Data Cataloging.arXiv preprint arXiv:2507.05282(2025). doi:10.48550/arXiv.2507.05282

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2507.05282 2025

  3. [3]

    Leonardo Candela, Dario Mangione, and Gina Pavone. 2024. The FAIR Assess- ment Conundrum: Reflections on Tools and Metrics.Data Science Journal23, 1 (2024). doi:10.5334/dsj-2024-033

    work page doi:10.5334/dsj-2024-033 2024

  4. [4]

    Daniel J. B. Clarke, Lily Wang, Alex Jones, Megan L. Wojciechowicz, Denis Torre, Kathleen M. Jagodnik, Sherry L. Jenkins, Peter McQuilton, Zachary Flamholz, Moshe C. Silverstein, et al. 2019. FAIRshake: Toolkit to Evaluate the FAIRness of Research Digital Resources.Cell Systems9, 5 (2019), 417–421. doi:10.1016/j. cels.2019.09.011

    doi:10.1016/j 2019

  5. [5]

    Anusuriya Devaraju and Robert Huber. 2020. F-UJI FAIR Assessment Tool. https: //www.f-uji.net/ Accessed: 2026-01-08

    2020

  6. [6]

    Vincent Emonet and Michel Dumontier. 2022. FAIR-enough: A Community- Governed FAIR Maturity Assessment Framework. https://github.com/ MaastrichtU-IDS/fair-enough

    2022

  7. [7]

    Alban Gaignard, Thomas Rosnet, Frédéric De Lamotte, Vincent Lefort, and Marie-Dominique Devignes. 2023. FAIR-Checker: Supporting Digital Resource Findability and Reuse with Knowledge Graphs and Semantic Web Standards. Journal of Biomedical Semantics14, 1 (2023), 7. doi:10.1186/s13326-023-00289-5

    work page doi:10.1186/s13326-023-00289-5 2023

  8. [8]

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber

  9. [9]

    Zezhou Huang and Eugene Wu. 2024. Cocoon: Semantic Table Profiling Using Large Language Models. InProceedings of the 2024 Workshop on Human-In-the- Loop Data Analytics (HILDA ’24). ACM, 1–7. doi:10.1145/3665939.3665957

    arXiv 2024

  10. [10]

    International Organization for Standardization. 2014. ISO 19115-1:2014 Geo- graphic Information — Metadata Part 1: Fundamentals. https://www.iso.org/ standard/53798.html Published Edition 1, last reviewed and confirmed in 2019, with 2 amendments

    2014

  11. [11]

    N. A. Krans, A. Ammar, P. Nymark, E. L. Willighagen, M. I. Bakker, and J. T. K. Quik. 2022. FAIR Assessment Tools: Evaluating Use and Performance.NanoIm- pact27 (2022), 100402. doi:10.1016/j.impact.2022.100402

    arXiv 2022

  12. [12]

    Lang Chain Authors. 2026. LangGraph: Agent Orchestration Framework for Reliable AI Agents. https://www.langchain.com/langgraph. Accessed on: 2026- 02-08

    2026

  13. [13]

    Lan Li, Liri Fang, and Vetle I. Torvik. 2024. Autodcworkflow: LLM-Based Data Cleaning Workflow Auto-Generation and Benchmark.arXiv preprint arXiv:2412.06724(2024). doi:10.48550/arXiv.2412.06724

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.06724 2024

  14. [14]

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Preprint, 2026, Ming Chen and Pranav Pai Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. 2024. AgentBench: Evaluating LLMs as Agents. InInt...

    2026

  15. [15]

    Gengchen Mai, Chris Cundy, Kristy Choi, Yingjie Hu, Ni Lao, and Stefano Ermon

  16. [16]

    Avanika Narayan, Ines Chami, Laurel Orr, Simran Arber, Pengyu Rong, Mon- ica Shen, and Christopher Ré. 2024. Can Foundation Models Wrangle Your Data?Proceedings of the VLDB Endowment17, 5 (2024), 1212–1225. doi:10.14778/ 3641204.3641227

    arXiv 2024

  17. [17]

    On the Opportunities and Challenges of Foundation Models for GeoAI. Comput. Surveys56, 10 (2024), 1–41. doi:10.1145/3653070

    doi:10.1145/3653070 2024

  18. [18]

    Dominik Paprotny and Matthias Mengel. 2023. Population, Land Use and Eco- nomic Exposure Estimates for Europe at 100 m Resolution from 1870 to 2020. Scientific Data10, 1 (2023), 372. doi:10.1038/s41597-023-02282-0

    work page doi:10.1038/s41597-023-02282-0 2023

  19. [19]

    Open Geospatial Consortium. 2022. OGC Disaster Pilot 2022: Integrating ISO and OGC Standards for FAIR Geospatial Data. https://www.ogc.org/initiatives/ disaster-pilot/ OGC Innovation Program

    2022

  20. [20]

    Sakib, Kallol Naha, Sajratul Y

    Syed N. Sakib, Kallol Naha, Sajratul Y. Rubaiat, and Hasan M. Jamil. 2025. A GenAI System for Improved FAIR Independent Biological Database Integration. ACM Journal of Data and Information Quality17, 4, Article 26 (2025), 29 pages. doi:10.1145/3770753

    work page doi:10.1145/3770753 2025

  21. [21]

    2018.Cost-benefit analysis for FAIR research data

    PwC EU Services. 2018.Cost-benefit analysis for FAIR research data. Techni- cal Report. European Commission. https://op.europa.eu/en/publication-detail/- /publication/d375368c-1a0a-11e9-8d04-01aa75ed71a1

    2018

  22. [22]

    TKFDM. 2023. FAIR Data Assessment Tool. https://www.ubs.uzh.ch/de/ TKFDM/Services/FAIR-Assessment.html University of Zurich

    2023

  23. [23]

    Travis Thompson, Sweta Ojha, Christian D

    P. Travis Thompson, Sweta Ojha, Christian D. Powell, Kelly G. Pennell, and Hunter N. B. Moseley. 2023. A Proposed FAIR Approach for Disseminating Geospatial Information System Maps.Scientific Data10, 1 (2023), 389. doi:10. 1038/s41597-023-02281-1

    2023

  24. [24]

    Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E

    Mark D. Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E. Bourne, et al. 2016. The FAIR Guiding Principles for Scientific Data Management and Stewardship.Scientific Data3, 1 (2016), 1–9. doi:10.1038/sdata.2016.18

    doi:10.1038/sdata.2016.18 2016

  25. [25]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. 2024. A Survey on Large Language Model Based Autonomous Agents.Frontiers of Computer Science18, 6 (2024), 186345. doi:10.1007/s11704- 024-40231-1

    doi:10.1007/s11704- 2024

  26. [26]

    Mark D. Wilkinson, Michel Dumontier, Susanna-Assunta Sansone, Luiz Olavo Bonino da Silva Santos, Mario Prieto, Dominique Batista, Peter McQuilton, To- bias Kuhn, Philippe Rocca-Serra, Mercè Crosas, et al. 2019. Evaluating FAIR Ma- turity Through a Scalable, Automated, Community-Governed Framework.Sci- entific Data6, 1 (2019), 174. doi:10.1038/s41597-019-0184-5

    doi:10.1038/s41597-019-0184-5 2019

  27. [27]

    Wilkinson, Michel Dumontier, and Luiz Olavo Bonino da Silva Santos

    Mark D. Wilkinson, Michel Dumontier, and Luiz Olavo Bonino da Silva Santos

  28. [28]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xi- angyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huang,...

    Pith/arXiv arXiv doi:10.48550/arxiv.2309.07864 2024

  29. [29]

    Xu, Yufan Wang, Boxuan Sharma, Hoang Peng, Hailey Hyunji Liu, Jerry Yang Chen, Shuyan Lin, Yunzhe Yang, Zijian Sun, Lun Du Zheng, Sida I

    Frank F. Xu, Yufan Wang, Boxuan Sharma, Hoang Peng, Hailey Hyunji Liu, Jerry Yang Chen, Shuyan Lin, Yunzhe Yang, Zijian Sun, Lun Du Zheng, Sida I. Ding, Ningxin Hou, Sophia Ni, John Hsu, Raghav Ramanujan, Yongjae Kim, Tan- ishq Bhasker, Chen Wu, and Xiang Xu. 2024. TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. doi:10.48550/ar...

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.14161 2024

  30. [30]

    White, Doug Burger, and Chi Wang

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadal- lah, Ryen W. White, Doug Burger, and Chi Wang. 2024. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation.arXiv preprint arXiv:2308.08155(2024). doi:10.48550/arXiv.2308.08155 Microsoft Research

    Pith/arXiv arXiv doi:10.48550/arxiv.2308.08155 2024

  31. [33]

    principle

    Shuo Zhang, Zezhou Huang, and Eugene Wu. 2025. Data Cleaning Using Large Language Models. In2025 IEEE 41st International Conference on Data Engineering Workshops (ICDEW). IEEE, 28–32. doi:10.48550/arXiv.2410.15547 A Open-source artifact and prompt specifications A.1 Zenodo release and contents An archived software snapshot is available on Zenodo (https://...

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.15547 2025

  32. [2019]

    https://fairsharing.github

    FAIR Evaluator: The FAIR Evaluation Services. https://fairsharing.github. io/FAIR-Evaluator-FrontEnd/ GO FAIR Foundation

  33. [2024]

    InInternational Conference on Learning Representations (ICLR)

    MetaGPT: Meta Programming for A Multi-Agent Collaborative Frame- work. InInternational Conference on Learning Representations (ICLR). https: //openreview.net/forum?id=VtmBAGCN7o