Pith. sign in

REVIEW 4 major objections 5 minor 43 references

Harnessing multiple LLMs for Information Retrieval: A case study on Deep Learning methodologies in Biodiversity publications

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A majority vote of five open-source LLMs extracts deep-learning methodological details from biodiversity papers, agreeing with human annotations on 417 of 600 comparisons (69.5%), even though the models see only the article text.

desk verdict A transparent, reproducible incremental case study whose 69.5% headline overstates what is measured: the gold standard saw code and figures and used different criteria, so the number mixes extraction error with annotation-scheme mismatch. read the letter →

arxiv 2411.09269 v1 pith:R3HRFCIW submitted 2024-11-14 cs.IR cs.AI

Vamsi Krishna Kommineni , Birgitta König-Ries , Sheeba Samuel This is my paper · ORCID
classification cs.IRcs.AI
keywords multi-LLMensembleretrieval-augmentedgenerationinformationextractiondeeplearningreproducibilitybiodiversityliteraturevotingclassifiercompetencyquestionscategoricalannotation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deep-learning methods in scientific papers are usually described in unstructured prose, which makes it hard to find out how a model was built, trained, and evaluated. This paper tries to show that a pipeline of five open-source large language models, each combined with retrieval-augmented generation, can automatically turn the text of a paper into categorical yes/no answers about reproducibility variables. Tested on 464 biodiversity publications, a majority-vote ensemble of the five models matched the manual annotations from the authors' earlier study on 417 of 600 comparisons, or 69.5%, across six variables such as model architecture, datasets, hyperparameters, and source-code links. This agreement is notable because the human annotators had access to code, figures, and tables, while the LLMs saw only the text. If the claim is right, the pipeline offers a scalable way to screen research papers for methodological transparency and to help authors and reviewers check whether a deep-learning study is reproducible.

What carries the argument

The load-bearing mechanism is a hard voting classifier built from five open-source LLMs: Llama-3 70B, Llama-3.1 70B, Mixtral-8x22B-Instruct-v0.1, Mixtral 8x7B, and Gemma 2 9B. Each LLM is paired with retrieval-augmented generation, a setup that retrieves the most relevant chunks of a paper's full text and feeds them to the model so that answers are grounded in the source. Each model answers 28 competency questions covering the whole deep-learning pipeline, and a second pass converts each textual answer into a categorical 'yes' or 'no' using the same model. The five categorical votes are then combined by simple majority rule, which is what makes the final decision an ensemble verdict rather than the output of any single model. A final RAG-assisted filtering step removes publications that only mention DL keywords without actually presenting a DL pipeline.

What would settle it

Run the same five-LLM voting pipeline on a fresh sample of biodiversity deep-learning papers whose six variables are annotated by a second, independent team applying the same text-only criteria, and check whether the majority vote's agreement with those annotations is significantly above the per-variable majority-class baseline; if it is not, the 69.5% figure was an artifact of the earlier annotation criteria rather than a measure of extraction quality.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the majority vote of five different open-source LLMs, each fed a paper's full text through a retrieval-augmented pipeline, is a usable partial proxy for human judgment about whether six deep-learning reproducibility variables are present in a biodiversity publication. On a curated set of 100 papers from earlier work, the voting classifier matched the prior manual annotations in 417 out of 600 question-answer pairs, or 69.5%; agreement by variable ranged from 53/100 for open-source frameworks to 89/100 for model architecture. The authors further show that filtering out papers that mention DL keywords without actually employing a DL pipeline raises the fraction of answered queries from 27.12% to 35.77%, and that the filter itself achieved 93% accuracy on the 100-paper set. They present the results as a proof of concept that a multi-LLM voting pipeline can retrieve DL methodology from text alone and can be transferred to other scientific domains.

Load-bearing premise

The 69.5% accuracy number assumes the earlier manual annotations are correct ground truth for the six reproducibility variables, even though those annotations were made by partially overlapping authors who could see code, figures, and tables and who used criteria, such as requiring persistent identifiers for datasets, that differ from what a text-only LLM can observe.

Editorial extensions

If this is right

  • The majority-vote outputs of five LLMs can serve as a partial proxy for human annotation in large-scale reproducibility audits, letting researchers screen thousands of papers for DL methodological reporting without reading each one.
  • The filtering step that removes non-DL papers is itself a usable component: it reached 93% accuracy on the 100-paper evaluation set and raised the pipeline's positive response rate from 27.12% to 35.77%.
  • Because the pipeline relies only on text and not on biodiversity-specific features, the same RAG-plus-multi-LLM procedure should transfer to other scientific domains where methodological reporting matters.
  • Individual LLMs diverge, with pairwise Cohen's kappa between 0.53 and 0.79, so an ensemble is more trustworthy than picking one model; Llama 3.1 70B and Llama 3 70B answer most alike, while Gemma 2 9B and Mixtral 8x22B diverge most.
  • The pipeline can report its own environmental footprint as a routine byproduct: 177.55 kWh and 60.14 kg CO2e for RAG text responses, and 50.63 kWh and 17.15 kg CO2e for categorical conversion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 69.5% figure probably understates the models' text-extraction ability on the dataset variable, because the human gold standard counted a dataset only when a persistent identifier was given, while the LLMs counted the dataset by name; a criterion-aligned evaluation would be a fairer test.
  • Because the evaluation covered only the six of 28 competency questions that overlap with the earlier study, the ensemble's accuracy on the remaining 22 questions, such as deployment platforms, bias handling, and randomness strategies, is still unmeasured; a full 28-question gold standard would show whether the proxy extends beyond the best-matched variables.
  • The pipeline could be turned into a preprint-time reproducibility checklist: authors could run their own manuscripts through the same 28 competency questions, and the unanswered ones would flag missing methodological details before review.
  • A sharp test of ensemble value would be to measure agreement between the five models on the six variables and re-run the evaluation with each model alone; the paper's moderate pairwise kappas suggest the majority vote should beat the best single model, but that comparison is not explicitly reported.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper presents an automated information-extraction pipeline that uses five open-source LLMs (Llama-3 70B, Llama-3.1 70B, Mixtral-8x22B, Mixtral 8x7B, and Gemma 2 9B) with retrieval-augmented generation (RAG) to answer 28 competency questions about deep-learning methodology from 464 biodiversity publications. Textual answers are converted to binary yes/no responses by the same LLMs and combined via majority voting. The main evaluation compares the voting output with manual annotations from the authors' prior work (Ahmed et al., 2024b) for six reproducibility variables over 100 publications, reporting 417/600 agreements (69.5%). The paper additionally reports per-LLM agreement with human labels for the text-to-categorical conversion step on 30 publications, an analysis of publication filtering, pairwise LLM cosine similarity and inter-annotator agreement, and an environmental-footprint estimate.

Significance. The work is a useful and transparent case study: the pipeline is described in enough detail to be reproduced, the model parameters (temperature=0, fixed majority voting, chunking settings) are fixed rather than tuned on the evaluation set, and code and data are provided. If the 69.5% agreement were shown to reflect text-only extraction quality under a commensurate annotation scheme, the result would be practically valuable for reproducibility screening and knowledge-graph construction. However, the current evidence does not establish the 'enhances retrieval' claim, because there is no single-LLM baseline on the same task and the gold standard is incommensurate with the text-only input in exactly the areas of lowest agreement.

major comments (4)
  1. [Abstract; §Results (Table 3)] The claim that the multi-LLM, RAG-assisted pipeline 'enhances the retrieval' of DL methodological information is not supported by a same-task comparison against a single-LLM baseline. The 417/600 result in Table 3 is computed for the majority-vote outputs only; Table 2 evaluates the text-to-categorical conversion step on 30 publications per LLM, which is a different task. To support the 'enhances' wording, the authors should report accuracy of each individual LLM on the same six competency questions over the same 100 evaluation publications, or otherwise revise the claim to state that the voting outputs agree with prior annotations at a certain rate.
  2. [§Discussion; §Results (Table 3)] The headline 69.5% agreement is not a clean measure of text-only extraction accuracy. As the Discussion concedes, the prior human annotations in Ahmed et al. (2024b) were produced with access to code, figures, and tables, and used criteria that differ from the LLM competency questions: datasets were recorded only when a persistent identifier was given, whereas the LLM counts a named dataset without an identifier; frameworks and hyperparameters could be inferred from accompanying code. Every disagreement therefore mixes genuine extraction errors with annotation-scheme and input-modality mismatch, and Table 3 shows the lowest agreement precisely on Dataset (63/100) and Open source framework (53/100), the two variables with divergent criteria. The authors should either construct a text-only, same-criteria gold standard for at least a subset of the 100 publications, or explicitly present 69.5% as agreement under a mismatched protocol rather than as retrieval accuracy.
  3. [§Evaluation; §Results (Table 3)] The evidence base is narrow for the strength of the conclusions: only 6 of the 28 competency questions have human ground truth, only the 100 prior-work publications are scored, and no uncertainty quantification is provided for the 417/600 estimate or for the per-variable counts. A binomial 95% confidence interval for 417/600 is approximately 65.7% to 73.1%, and the per-variable intervals (e.g., 53/100 to 89/100) overlap substantially, so the variable-level ordering in Table 3 should not be over-interpreted. Adding confidence intervals, a majority-class or chance baseline, and a per-competency-question breakdown would make the reliability claim commensurate with the evidence.
  4. [§Additional analysis (Publication filtering); §Discussion] The 93% accuracy reported for the publication-filtering component is not informative without additional statistics, because the evaluation set consists of 100 publications that all use DL methods and therefore contains no true negatives. Reporting only accuracy on a one-class sample cannot distinguish sensitivity from specificity; the authors should report precision, recall, and the false-positive and false-negative counts, ideally on a sample that includes non-DL papers, before using the filtering result to support the conclusion that filtering 'improved our findings.'
minor comments (5)
  1. [§Methods (Prompt for converting textual to categorical responses)] In the in-context conversion prompt, Example 1 asks about 'methods are utilized for collecting raw data', but the answer discusses whether a code repository is available; the question-answer pair is mismatched and should be corrected, since in-context examples directly influence the conversion step.
  2. [§Evaluation] The text says 'randomly selected 30 publications' twice, but it is not clear whether the same 30 publications were used for all five LLMs or a different random sample was drawn for each LLM; please clarify, as this affects the comparability of the per-model kappa scores in Table 2.
  3. [§Evaluation] The two annotators who produced the ground truth for the 30-publication conversion evaluation are the first and last authors, but no inter-annotator agreement between them is reported; reporting a kappa value for the human annotations would strengthen the validity of the conversion evaluation.
  4. [§Results (Table 2)] Please report confidence intervals or standard errors alongside the Cohen's kappa values; with 840 comparisons the differences between models (e.g., 0.7128 vs. 0.7708) may not be statistically meaningful.
  5. [§Methods (Dataset from Ecological Informatics)] The description of the retrieval query states that keywords were divided into five sets due to the platform's limit of 8 boolean connectors per search, but it is not stated whether the five result sets were unioned after deduplication or how the 991 citation records were combined; a sentence clarifying the union and deduplication step would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

The reported agreements rest on incommensurate prior annotations, but the derivation contains no circular reduction of the pipeline's outputs to its evaluation inputs.

full rationale

The paper's central claim is an empirical accuracy number: the hard-voting classifier over five LLM categorical responses agrees with 417/600 human-annotated variable values from Ahmed et al. (2024a/b). Nothing in the pipeline is fitted to those 600 comparisons: temperature is 0, chunk size (1000), overlap (50), token limit (1200), and majority voting are all fixed before evaluation. The CQs are reused from the authors' earlier KG work, but they are natural-language queries, not definitions of the six target variables; Table 3's mapping is asserted, not derived, so any mismatch is a validity threat, not a circular one. The main weakness—that the gold standard had access to code, figures, and tables and used different criteria (e.g., datasets counted only with persistent identifiers)—means the 69.5% comparison mixes extraction error with annotation-scheme mismatch. The paper explicitly owns this in the Discussion, noting that the prior work 'also considered the accompanying code' and 'considered dataset availability only when persistent identifiers were provided,' so it is not a hidden reduction. Because no step is defined in terms of its output and no parameter is fitted to the benchmark, the derivation chain is not circular.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The pipeline introduces no fitted parameters; the reported accuracy is a measurement against an internal gold standard. The main a priori commitments are the hand-picked chunking and generation settings, the deferred RAG retrieval mechanism, and the assumption that the authors' prior human annotations are an appropriate benchmark.

free parameters (5)
  • chunk_size = 1000 tokens
    Hand-selected chunk size for RAG retrieval; not optimized against the evaluation benchmark.
  • chunk_overlap = 50 chunks
    Hand-selected overlap between chunks.
  • max_retrieval_tokens = 1200 tokens
    Cap on tokens retrieved per query.
  • max_response_words = 400 words
    Instruction to LLMs to keep answers concise.
  • temperature = 0
    Deterministic generation; reasonable, but a design choice.
assumptions (5)
  • domain assumption Human annotations in Ahmed et al. (2024b) are a valid ground truth for DL variable presence.
    The 69.5% accuracy is computed against these annotations, which were produced by the same research group with access to code, figures, and tables.
  • domain assumption The RAG retrieval component from Kommineni et al. (2024b) supplies the relevant context to the LLMs.
    The paper defers RAG implementation details to prior work and does not measure retrieval quality.
  • domain assumption The mapping of six competency questions to six reproducibility variables is semantically exact.
    Table 3 assumes each CQ corresponds cleanly to one variable in Ahmed et al. (2024b).
  • domain assumption LLM conversion from textual answers to yes/no is faithful.
    Validated only on 30 publications per LLM with moderate-to-strong kappa (0.5583 to 0.7708); the conversion prompt is hand-crafted.
  • domain assumption The 28 CQs cover the relevant aspects of a DL pipeline.
    CQs were designed in the authors' prior ontology work; no external validation of completeness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Harnessing multiple LLMs for Information Retrieval: A case study on Deep Learning methodologies in Biodiversity publications." pith.science (2026). https://pith.science/paper/R3HRFCIW

@misc{pith2026241109269,
  author       = {Pith},
  title        = {Pith review of: Harnessing multiple LLMs for Information Retrieval: A case study on Deep Learning methodologies in Biodiversity publications},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R3HRFCIW}},
  note         = {Machine review of arXiv:2411.09269}
}
read the original abstract

Deep Learning (DL) techniques are increasingly applied in scientific studies across various domains to address complex research questions. However, the methodological details of these DL models are often hidden in the unstructured text. As a result, critical information about how these models are designed, trained, and evaluated is challenging to access and comprehend. To address this issue, in this work, we use five different open-source Large Language Models (LLMs): Llama-3 70B, Llama-3.1 70B, Mixtral-8x22B-Instruct-v0.1, Mixtral 8x7B, and Gemma 2 9B in combination with Retrieval-Augmented Generation (RAG) approach to extract and process DL methodological details from scientific publications automatically. We built a voting classifier from the outputs of five LLMs to accurately report DL methodological information. We tested our approach using biodiversity publications, building upon our previous research. To validate our pipeline, we employed two datasets of DL-related biodiversity publications: a curated set of 100 publications from our prior work and a set of 364 publications from the Ecological Informatics journal. Our results demonstrate that the multi-LLM, RAG-assisted pipeline enhances the retrieval of DL methodological information, achieving an accuracy of 69.5% (417 out of 600 comparisons) based solely on textual content from publications. This performance was assessed against human annotators who had access to code, figures, tables, and other supplementary information. Although demonstrated in biodiversity, our methodology is not limited to this field; it can be applied across other scientific domains where detailed methodological reporting is essential for advancing knowledge and ensuring reproducibility. This study presents a scalable and reliable approach for automating information extraction, facilitating better reproducibility and knowledge transfer across studies.

Figures

Figures reproduced from arXiv: 2411.09269 by the authors.

Figure 1
Figure 1. Workflow of the pipeline. The solid arrows represent the main process flow, while the dotted line indicates the evaluation phase for the categorical responses from 100 publications retrieved from our previous research (Ahmed et al., 2024a). “yes” if the information was present and “no” if it was absent. In the current study, these 100 publications serve as an evaluation dataset, supporting the comparison and validat… view at source ↗
Figure 2
Figure 2. LLM-Human optimized 25 DL-related keywords from 44 AI-related session abstracts at the Biodiversity Information Standards (TDWG) conferences. Publication citation data extraction: Using the 25 refined keywords identified from TDWG abstracts with the assistance of both the LLM and domain experts, we queried the Ecological Informatics journal. The query applied the following filters: publication years from 2016 to Aug… view at source ↗
Figure 3
Figure 3. Number of publications selected from Ecological Informatics Journal (364 publications) us to extract, organize, and link information from unstructured text into structured, queryable data within the KG framework. By semi-automating the construction of KGs, we streamlined the process of mapping complex domain knowledge, which is crucial for advancing research in areas that require high levels of detail, such as biodi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 36 canonical work pages

  1. [1]

    o ffler, F., Feddoul, L., Algergawy, A., Samuel, S., Gaikwad, J., Kazem, A., and K \

    Abdelmageed, N., L \"o ffler, F., Feddoul, L., Algergawy, A., Samuel, S., Gaikwad, J., Kazem, A., and K \"o nig-Ries, B. (2022). BiodivNERE : Gold standard corpora for named entity recognition and relation extraction in the biodiversity domain. Biodiversity Data Journal , 10

    work page 2022

  2. [2]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  3. [3]

    K., Koenig-ries, B., and Samuel, S

    Ahmed, W., Kommineni, V. K., Koenig-ries, B., and Samuel, S. (2023). How reproducible are the results gained with the help of deep learning methods in biodiversity research? Biodiversity Information Science and Standards , 7

    work page 2023

  4. [4]

    K., K \"o nig-Ries, B., Gaikwad, J., Gadelha, L., and Samuel, S

    Ahmed, W., Kommineni, V. K., K \"o nig-Ries, B., Gaikwad, J., Gadelha, L., and Samuel, S. (2024a). Evaluating the method reproducibility of deep learning models in the biodiversity research. PeerJ Computer Science, Under review

    work page 2024

  5. [5]

    Evaluating the method reproducibility of deep learning models in the biodiversity domain

    Ahmed, W., Kommineni, V. K., K \" o nig - Ries, B., Gaikwad, J., Jr., L. M. R. G., and Samuel, S. (2024b). Evaluating the method reproducibility of deep learning models in the biodiversity domain. CoRR , abs/2407.07550

    work page Pith review arXiv 2024

  6. [6]

    A., Pescott, O

    August, T. A., Pescott, O. L., Joly, A., and Bonnet, P. (2020). AI naturalists might hold the key to unlocking biodiversity data in social media imagery. Patterns , 1(7):100116

    work page 2020

  7. [7]

    Beltagy, I., Lo, K., and Cohan, A. (2019). S ci BERT : A pretrained language model for scientific text. In Inui, K., Jiang, J., Ng, V., and Wan, X., editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pages 3615--3620, Hon...

    work page 2019

  8. [8]

    and Stodden, V

    Bhaskar, A. and Stodden, V. (2024). Reproscreener: Leveraging llms for assessing computational reproducibility of machine learning pipelines. In Proceedings of the 2nd ACM Conference on Reproducibility and Replicability , ACM REP '24, page 101–109, New York, NY, USA. Association for Computing Machinery

    work page 2024

Show all 43 references
  1. [9]

    Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert - Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E.,...

    2020 arXiv

  2. [10]

    Christin, S., Hervet, \'E ., and Lecomte, N. (2019). Applications for deep learning in ecology. Methods in Ecology and Evolution , 10(10):1632--1644

    2019

  3. [11]

    S., Ceder, G., Persson, K

    Dagdelen, J., Dunn, A., Lee, S., Walker, N., Rosen, A. S., Ceder, G., Persson, K. A., and Jain, A. (2024). Structured information extraction from scientific text with large language models. Nature Communications , 15(1):1418

    2024

  4. [12]

    S., Ceder, G., Persson, K., and Jain, A

    Dunn, A., Dagdelen, J., Walker, N., Lee, S., Rosen, A. S., Ceder, G., Persson, K., and Jain, A. (2022). Structured information extraction from complex scientific text with fine-tuned large language models. arXiv preprint arXiv:2212.05238

    2022 arXiv

  5. [13]

    and Hamdy, M

    El-Amir, H. and Hamdy, M. (2020). Deep learning pipeline. Apress: Berkeley, CA, USA

    2020

  6. [14]

    S., Walker, C., Peterson, A

    Feng, X., Park, D. S., Walker, C., Peterson, A. T., Merow, C., and Pape s , M. (2019). A checklist for maximizing reproducibility of ecological niche models. Nature Ecology & Evolution , 3(10):1382--1395

    2019

  7. [15]

    Frandsen, P., Dikow, R., Trizna, M., and White, A. (2019). SS86 - Machine learning: an emerging toolkit for biodiversity science using museum collections . Biodiversity Information Science and Standards

    2019

  8. [16]

    Gougherty, A. V. and Clipp, H. L. (2024). Testing the reliability of an ai-based large language model to extract ecological information from the scientific literature. npj Biodiversity , 3

    2024

  9. [17]

    Biodiversity and artificial intelligence, opportunities and recommendations report

    GPAI (2022). Biodiversity and artificial intelligence, opportunities and recommendations report

    2022

  10. [18]

    and Ellwood, E

    Groom, Q. and Ellwood, E. (2021). SYM01 - Applications of machine learning in biodiversity image analysis . Biodiversity Information Science and Standards

    2021

  11. [19]

    and Fox, M

    Gr \"u ninger, M. and Fox, M. S. (1995). The role of competency questions in enterprise engineering. In Benchmarking—Theory and practice , pages 22--31. Springer

    1995

  12. [20]

    Gundersen, O. E. and Kjensmo, S. (2018). State of the art: Reproducibility in artificial intelligence. Proceedings of the AAAI Conference on Artificial Intelligence , 32(1)

    2018

  13. [21]

    E., Shamsaliei, S., and Isdahl, R

    Gundersen, O. E., Shamsaliei, S., and Isdahl, R. J. (2022). Do machine learning platforms provide out-of-the-box reproducibility? Future Generation Computer Systems , 126:34--47

    2022

  14. [22]

    Haddaway, N. R. and Verhoeven, J. T. (2015). Poor methodological detail precludes experimental repeatability and hampers synthesis in ecology. Ecology and Evolution , 5(19):4451--4454

    2015

  15. [23]

    D., Gutierrez, C., Kirrane, S., Gayo, J

    Hogan, A., Blomqvist, E., Cochez, M., d’Amato, C., Melo, G. D., Gutierrez, C., Kirrane, S., Gayo, J. E. L., Navigli, R., Neumaier, S., et al. (2021). Knowledge graphs. ACM Computing Surveys (Csur) , 54(4):1--37

    2021

  16. [24]

    Johaadien, R., Lewers, K., and Torma, M. (2023). SYM05 AI Contributions to biodiversity data & data standardisation: Opportunities and challenges . Biodiversity Information Science and Standards

    2023

  17. [25]

    K., Ahmed, W., Koenig-Ries, B., and Samuel, S

    Kommineni, V. K., Ahmed, W., Koenig-Ries, B., and Samuel, S. (2024a). Automating information retrieval from biodiversity literature using large language models: A case study. Biodiversity Information Science and Standards , 8:e136735

    2024

  18. [26]

    K., Groom, Q., and Panda, R

    Kommineni, V. K., Groom, Q., and Panda, R. (2022). SYM12 - Information extraction from digital specimen images using Artificial Intelligence . Biodiversity Information Science and Standards

    2022

  19. [27]

    K., K \" o nig - Ries, B., and Samuel, S

    Kommineni, V. K., K \" o nig - Ries, B., and Samuel, S. (2024b). From human experts to machines: An LLM supported approach to ontology and knowledge graph construction. CoRR , abs/2403.08345

    2024 arXiv

  20. [28]

    Lannelongue, L., Grealey, J., Bateman, A., and Inouye, M. (2021a). Ten simple rules to make your computing more environmentally sustainable. PLOS Computational Biology , 17(9):e1009324

    2021

  21. [29]

    Lannelongue, L., Grealey, J., and Inouye, M. (2021b). Green algorithms: Quantifying the carbon footprint of computation. Advanced Science , 8(12):2100707

    2021

  22. [30]

    u ttler, H., Lewis, M., Yih, W., Rockt \

    Lewis, P. S. H., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K \" u ttler, H., Lewis, M., Yih, W., Rockt \" a schel, T., Riedel, S., and Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. CoRR , abs/2005.11401

    2020 arXiv

  23. [31]

    L., Chiang, C.-C., and Shah, N

    Lozano, A., Fleming, S. L., Chiang, C.-C., and Shah, N. (2023). Clinfo. ai: An open-source retrieval-augmented large language model system for answering medical questions using scientific literature. In PACIFIC SYMPOSIUM ON BIOCOMPUTING 2024 , pages 8--23. World Scientific

    2023

  24. [32]

    K., Brunt, J

    Michener, W. K., Brunt, J. W., Helly, J. J., Kirchner, T. B., and Stafford, S. G. (1997). Nongeospatial metadata for the ecological sciences. Ecological Applications , 7(1):330--342

    1997

  25. [33]

    S., Nguyen, A

    Norouzzadeh, M. S., Nguyen, A. T., Kosmala, M., Swanson, A., Palmer, M. S., Packer, C., and Clune, J. (2017). Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. Proceedings of the National Academy of Sciences of the Unite...

    2017

  26. [34]

    Pando, F., Mata, E., Carranza-Rojas, J., Lloret, L., and GOEAU, H. (2018). W14 Deep Learning for Biodiversity . Biodiversity Information Science and Standards

    2018

  27. [35]

    Pineau, J., Vincent-Lamarre, P., Sinha, K., Larivi \`e re, V., Beygelzimer, A., d'Alch \'e Buc, F., Fox, E., and Larochelle, H. (2021). Improving reproducibility in machine learning research (a report from the neurips 2019 reproducibility program). The Journal of Machine Learn...

    2021

  28. [36]

    o ffler, F., and K \

    Samuel, S., L \" o ffler, F., and K \" o nig - Ries, B. (2021). Machine learning pipelines: Provenance, reproducibility and FAIR data principles. In Glavic, B., Braganholo, V., and Koop, D., editors, Provenance and Annotation of Data and Processes - 8th and 9th International P...

    2021

  29. [37]

    and Mietchen, D

    Samuel, S. and Mietchen, D. (2024). Computational reproducibility of jupyter notebooks from biomedical publications. GigaScience , 13:giad113

    2024

  30. [38]

    Stark, P. B. (2018). Before reproducibility must come preproducibility. Nature , 557(7706):613--614

    2018

  31. [39]

    G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ramé, A., Ferret, J., Liu, P., Tafti, P., Friesen, A., Casbon, M., Ramos, S., Kumar, R., Lan, C

    Team, G., Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ramé, A., Ferret, J., Liu, P., Tafti, P., Friesen, A., Casbon, M., Ramos, S., Kumar, R., Lan, C. L., Jerome, S., Tsitsulin, A., Vieillard, N., Stanczyk, P., ...

    2024

  32. [40]

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G. (2023). Llama: Open and efficient foundation language models

    2023

  33. [41]

    B., Brunt, J

    Waide, R. B., Brunt, J. W., and Servilla, M. S. (2017). Demystifying the landscape of ecological data repositories in the United States . BioScience , 67(12):1044--1051

    2017

  34. [42]

    Whitlock, M. C. (2011). Data archiving in ecology and evolution: best practices. Trends in ecology & evolution , 26(2):61--65

    2011

  35. [43]

    Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Dou, Z., and Wen, J.-R. (2023). Large language models for information retrieval: A survey. arXiv preprint arXiv:2308.07107

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.