Pith. sign in

REVIEW 2 major objections 3 minor 3 cited by

A Multi-Task Evaluation of LLMs' Processing of Academic Text Input

T0 review · 2 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A four-task evaluation of Gemini finds reliable summaries but unreliable ranking, grading, and reflection, concluding LLMs should not be used unchecked in peer review.

desk verdict This submission is two different papers: the title/abstract describe a Gemini academic-text evaluation, the body is an unrelated biomedical query expansion paper, so the central claim is unsupported as submitted. read the letter →

arxiv 2508.11779 v1 pith:DQCKP33H submitted 2025-08-15 cs.CL econ.GNq-fin.EC

Tianyi Li , Yu Qin , Olivia R. Liu Sheng This is my paper
classification cs.CLecon.GNq-fin.EC
keywords largelanguagemodelsacademictextprocessingpeerreviewGeminimulti-taskevaluationsummarizationscoring
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Taking Google's Gemini as the test model, the paper tries to determine how far a leading large language model can be trusted to process academic text for scholarly work. It structures the test as four tasks—content reproduction, comparison, scoring, and reflection—that ask the model to act as oracle, judgmental arbiter, knowledgeable arbiter, and collaborator, respectively. The recorded result is a split profile: summary and paraphrase are acceptably reliable, while pairwise ranking is only faintly scalable, grading shows poor discrimination, and reflection is self-consistent but not insightful. Because the pattern persists across internal linguistic metrics, external ground-truth comparisons, and human evaluation, and under prompt variations, the paper's bottom line is that unchecked use of LLMs in constructing peer reviews is not advisable.

What carries the argument

The carrying mechanism is a four-task evaluation workflow in which each task assigns the LLM a distinct role—oracle (content reproduction), judgmental arbiter (comparison), knowledgeable arbiter (scoring), and collaborator (reflection)—and supplies the model with academic articles from top information systems journals. The assessment layers three forms of evidence: internal linguistic metrics, external comparison against ground truth, and human evaluation, all repeated across prompt variations. This design is what lets the paper attribute the observed failures to genuine limits in text understanding rather than to a single flawed test.

What would settle it

An independent evaluation on a different corpus of academic articles using the same four-task design that shows Gemini (or another LLM) producing pairwise rankings that agree with expert judgments and essay scores that correlate strongly with human grades would count against the paper's central claim. Concretely: take 100 papers with known expert ratings; have the model rank every pair; if the resulting order matches the expert order on most pairs, then the 'faintly scalable' and 'poor discrimination' findings would fail to replicate.

Watch

Extended reading notes

Core claim

The paper's central claim is that Gemini's ability to process academic text degrades sharply as tasks move from content reproduction to judgment. Acting as an oracle for content reproduction, its summary and paraphrase are acceptably reliable. Acting as a judgmental arbiter for comparison, its pairwise ranking of texts is faintly scalable. Acting as a knowledgeable arbiter for scoring, it is prone to poor discrimination, meaning it cannot separate good from bad texts with useful precision. Acting as a collaborator for reflection, its output is self-consistent yet hardly insightful, offering little to inspire meaningful research. The authors support this with internal linguistic metrics, exte

Load-bearing premise

The verdict rests on the assumption that the four chosen tasks and the selected top-journal information systems articles are a fair stand-in for what an LLM must do with scientific text, and that the text metrics and ground truth used are meaningful and calibrated.

Editorial extensions

If this is right

  • LLM-generated peer review drafts should not be used without substantial human correction, especially for scoring and ranking judgments.
  • LLMs might still be useful in peer-review pipelines for summarization and paraphrasing tasks, where the model is reliable.
  • The four-task evaluation template can be applied to other LLMs and other academic domains to map where each model's reliable capabilities end.
  • If the result holds, automated review systems that assign grades or rankings based on LLM output would systematically produce poorly discriminated, low-insight assessments.
  • The pattern suggests that performance on surface-level tasks does not predict performance on judgment-heavy tasks, so benchmarks should test the full hierarchy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's finding about Gemini likely extends to other general-purpose LLMs, because the failure mode is tied to task difficulty rather than to a specific model quirk; a similar four-task test on other models would reveal where they also plateau.
  • The task hierarchy could be used to build staged human-in-the-loop workflows: let LLMs prepare summaries and evidence, but route scoring, ranking, and novelty judgments to human reviewers.
  • The evaluation method could be adapted to non-English or interdisciplinary academic corpora to test whether the reliability boundary shifts with text genre.
  • The paper's 'faintly scalable' comparison result suggests that pairwise voting schemes might amplify a weak ranker into adequate rankings when aggregated, which is a testable hypothesis the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The submission's metadata (title, abstract, arXiv number) advertises a multi-task evaluation of LLMs' processing of academic text input, centered on a claimed finding that Google's Gemini performs acceptably at summary/paraphrase but poorly at pairwise comparison, grading, and insightful reflection. The full text, however, is an entirely different paper: 'Ontology-Guided Query Expansion for Biomedical Document Retrieval using Large Language Models' (arXiv:2508.11784), which proposes and evaluates the BMQExpander pipeline for biomedical information retrieval. None of the four advertised tasks, the 'first-rate Information Systems articles at three top journals,' the text-metrics suite, the ground-truth comparison, or the human evaluation appear anywhere in the body. The only mention of Gemini is as one of several LLM backbones for query expansion in Table 5. As submitted, the manuscript contains zero evidentiary support for its abstract's central empirical claims.

Significance. If a rigorous multi-task evaluation of LLM academic-text processing, with triangulated metric-based, ground-truth, and human evaluation and prompt-robustness checks, were actually presented, it would be a useful contribution to the debate on LLMs in peer review. The body of this submission, taken on its own, is a coherent biomedical IR paper with released code, new paraphrased benchmarks, and extensive baseline comparisons. However, those strengths do not substantiate the abstract: the advertised study is not merely incomplete, it is entirely missing. The manuscript as submitted cannot be evaluated for the validity of its central claim, and no amount of significance of the abstract's topic can compensate for the absence of the corresponding methods and results.

major comments (2)
  1. [Entire manuscript (Abstract vs. Full Text)] The abstract's central claim—that Gemini's processing of academic text is 'compromised' across four tasks, with consistency across metric-based internal, ground-truth external, and human evaluation, and robustness to prompt variation—has no corresponding evidence in the full text. The body is a different paper with its own abstract, methodology, experiments, and references. None of the four tasks (content reproduction/comparison/scoring/reflection), the three-journal Information Systems corpus, the text metrics, or the human evaluation appear in Sections 1–6. The only occurrence of Gemini is as a query-expansion backbone in Table 5, evaluated by NDCG@10 on NFCorpus/TREC-COVID/SciFact. This is not a missing derivation or a presentation issue; the entire empirical study advertised in the abstract is absent, so the central claim is unsupported as submitted.
  2. [Section 5, Table 5] Even if the body were interpreted as the submitted paper, the experimental results in Table 5 concern biomedical document retrieval (BMQExpander against sparse/dense/query-expansion baselines), not summary/paraphrase reliability, pairwise text comparison, grading discrimination, or qualitative reflection on academic texts. The abstract's practical recommendation against 'an unchecked use of LLMs in constructing peer reviews' therefore cannot be derived from any result in this manuscript.
minor comments (3)
  1. [Metadata] The arXiv identifier/title/abstract (2508.11779) do not match the full-text article (2508.11784). This needs to be corrected, either by supplying the correct full text for 2508.11779 or by rewriting the abstract/metadata to match the body.
  2. [Section 6 / Conclusion] The body's conclusion discusses BMQExpander and biomedical query expansion only; it does not address academic-text processing or peer review. The advertised scope and the actual content are disjoint.
  3. [Limitations / disclosure] The body states that a larger qualitative study is left as future work, but it does not disclose the mismatch with the abstract's claimed evaluation. If this is a submission error, it should be flagged prominently; if not, the absence of the advertised study is a fundamental issue.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation identified; the submission's abstract and body are two different papers, so the abstract's Gemini-evaluation claim has no derivational support in the text at all.

full rationale

I inspected the claimed derivation chain. The abstract promises a multi-task evaluation of Gemini on academic texts, but the full text is an unrelated biomedical query-expansion paper (Ontology-Guided Query Expansion for Biomedical Document Retrieval using Large Language Models, arXiv:2508.11784). None of the four tasks, the Gemini results, the top-journal corpus, or the human-evaluation protocol appears in the body. This is a completeness/identity failure, not circularity: there is no equation, fitted parameter, or self-citation chain in which the abstract's conclusion is assumed as input. Within the body that is present, the BMQExpander results are empirical comparisons against external benchmarks and baselines; the alpha=5 weighting is adopted from prior work (Query2doc/LLM-QE), and the paraphrased benchmarks are independently reviewed. I find no step where a 'prediction' reduces by construction to a fitted input or to a self-citation. The appropriate score is therefore 0 for circularity; the unsupported-claim concern belongs to correctness/verifiability, not to the circularity dimension.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

Abstract-only review due to full-text mismatch. No derivations or fitted parameters are present; the central claim rests on the assumed validity of the task design and the representativeness of the chosen model and corpus.

assumptions (2)
  • domain assumption The four evaluation tasks (content reproduction, comparison, scoring, reflection) are a valid and sufficiently complete operationalization of LLM processing of academic text.
    Entered in the abstract's description of the tasks; if the tasks underrepresent 'academic text processing,' the negative conclusion about Gemini does not generalize.
  • domain assumption Gemini, described as the leading LLM, is representative of the best available LLM text-processing capability at the time of the study.
    The abstract generalizes from a single model; if Gemini is not representative, the practical recommendation against 'unchecked use of LLMs in peer review' would be overbroad.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Multi-Task Evaluation of LLMs' Processing of Academic Text Input." pith.science (2026). https://pith.science/paper/DQCKP33H

@misc{pith2026250811779,
  author       = {Pith},
  title        = {Pith review of: A Multi-Task Evaluation of LLMs' Processing of Academic Text Input},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DQCKP33H}},
  note         = {Machine review of arXiv:2508.11779}
}
read the original abstract

How much large language models (LLMs) can aid scientific discovery, notably in assisting academic peer review, is in heated debate. Between a literature digest and a human-comparable research assistant lies their practical application potential. We organize individual tasks that computer science studies employ in separate terms into a guided and robust workflow to evaluate LLMs' processing of academic text input. We employ four tasks in the assessment: content reproduction/comparison/scoring/reflection, each demanding a specific role of the LLM (oracle/judgmental arbiter/knowledgeable arbiter/collaborator) in assisting scholarly works, and altogether testing LLMs with questions that increasingly require intellectual capabilities towards a solid understanding of scientific texts to yield desirable solutions. We exemplify a rigorous performance evaluation with detailed instructions on the prompts. Adopting first-rate Information Systems articles at three top journals as the input texts and an abundant set of text metrics, we record a compromised performance of the leading LLM - Google's Gemini: its summary and paraphrase of academic text is acceptably reliable; using it to rank texts through pairwise text comparison is faintly scalable; asking it to grade academic texts is prone to poor discrimination; its qualitative reflection on the text is self-consistent yet hardly insightful to inspire meaningful research. This evidence against an endorsement of LLMs' text-processing capabilities is consistent across metric-based internal (linguistic assessment), external (comparing to the ground truth), and human evaluation, and is robust to the variations of the prompt. Overall, we do not recommend an unchecked use of LLMs in constructing peer reviews.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Stability vs. Manipulability: Evaluating Robustness Under Post-Decision Interaction in LLM Judges

    cs.AI 2026-06 unverdicted novelty 7.0 of 10

    LLM judges exhibit high stability under neutral re-evaluation but substantial reversibility under targeted post-decision challenges, quantified via a new Evaluation Robustness Score (ERS).

  2. Whose Name Comes Up? III: Persona Prompting Effects in LLM-Based Scholar Recommendation

    cs.IR 2026-05 unverdicted novelty 6.0 of 10

    Audits of 43 LLMs show that varying persona prompts (language, location, role-and-task) and context affects technical quality and social representativeness of scholar recommendations, with location impacting diversity...

  3. LLM-based Schema-Guided Extraction and Validation of Missing-Person Intelligence from Heterogeneous Data Sources

    cs.CL 2026-04 unverdicted novelty 5.0 of 10

    The Guardian Parser Pack pipeline extracts structured intelligence from heterogeneous missing-person documents using schema-guided LLM assistance, achieving F1 of 0.866 on 75 cases versus 0.258 for a deterministic baseline.

Reference graph

Works this paper leans on

60 extracted references · 46 canonical work pages · cited by 3 Pith papers

  1. [1]

    Mohannad ALMasri, Catherine Berrut, and Jean-Pierre Cheval- let. 2016. A comparison of deep learning based query expansion with pseudo-relevance feedback and mutual information. InAd- vances in Information Retrieval: 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20–23, 2016. Proceedings 38. Springer, 709–715

    work page 2016

  2. [2]

    Negar Arabzadeh, Radin Hamidi Rad, Maryam Khodabakhsh, and Ebrahim Bagheri. 2023. Noisy perturbations for estimating query difficulty in dense retrievers. InProceedings of the 32nd ACM International Conference on Information and Knowledge Management, 3722–3727

    work page 2023

  3. [3]

    Graph-based Model of Smart Grid Architectures

    Alan R. Aronson and François-Michel Lang. 2010. An overview of metamap: historical perspective and recent advances.Journal of the American Medical Informatics Association (JAMIA), 17, 3, 229–236. doi:10.1136/jamia.2009.002733

    work page Pith review arXiv 2010

  4. [4]

    Elham Asgari, Nina Montaña-Brown, Magda Dubois, Saleh Khalil, Jasmine Balloch, Joshua Au Yeung, and Dominic Pi- menta. 2025. A framework to assess clinical safety and halluci- nation rates of llms for medical text summarisation.npj Digital Medicine, 8, 1, 274

    work page 2025

  5. [5]

    Hiteshwar Kumar Azad and Akshay Deepak. 2019. Query expan- sion techniques for information retrieval: a survey.Information Processing & Management, 56, 5, 1698–1735

    work page 2019

  6. [6]

    Jagdev Bhogal, Andrew MacFarlane, and Peter Smith. 2007. A review of ontology based query expansion.Information process- ing & management, 43, 4, 866–886

    work page 2007

  7. [7]

    Olivier Bodenreider. 2004. The unified medical language sys- tem (umls): integrating biomedical terminology.Nucleic acids research, 32, suppl_1, D267–D270

    work page 2004

  8. [8]

    Vera Boteva, Demian Gholipour, Artem Sokolov, and Stefan Riezler. 2016. A full-text learning to rank dataset for medical information retrieval. InAdvances in Information Retrieval: 38th European Conference on IR Research, ECIR 2016, Padua, Italy, March 20–23, 2016. Proceedings 38. Springer, 716–722

    2016

Show all 60 references
  1. [9]

    Courtni Byun, Piper Vasicek, and Kevin Seppi. 2024. This refer- ence does not exist: an exploration of llm citation accuracy and relevance. InProceedings of the Third Workshop on Bridging Human–Computer Interaction and Natural Language Process- ing, 28–39

    2024

  2. [10]

    Xinran Chen, Xuanang Chen, Ben He, Tengfei Wen, and Le Sun

  3. [11]

    Hao Cheng, Hao Fang, Xiaodong Liu, and Jianfeng Gao. 2022. Task-aware specialization for efficient and robust dense retrieval foropen-domainquestionanswering. arXiv preprint arXiv:2210.05156

    2022 arXiv

  4. [12]

    Zhuyun Dai and Jamie Callan. 2019. Context-aware sentence/- passage term importance estimation for first stage retrieval. arXiv preprint arXiv:1910.10687

    2019 arXiv

  5. [13]

    Sherri de Coronado, Lyubov Remennik, and Peter L Elkin

  6. [14]

    Giorgio Maria Di Nunzio, Stefano Marchesin, Maristella Agosti, et al. 2019. Exploring how to combine query reformulations for precision medicine. InTREC

    2019

  7. [15]

    Kevin Donnelly et al. 2006. Snomed-ct: the advanced terminol- ogy and coding system for ehealth.Studies in health technology and informatics, 121, 279

    2006

  8. [16]

    Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant

  9. [17]

    Kailash A Hambarde and Hugo Proenca. 2023. Information retrieval: recent advances and beyond.IEEE Access, 11, 76581– 76604

    2023

  10. [18]

    Jie Huang and Kevin Chen-Chuan Chang. 2023. Citation: a key to building responsible and accountable large language models. arXiv preprint arXiv:2307.02185

    2023 arXiv

  11. [19]

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave

  12. [20]

    Rolf Jagerman, Honglei Zhuang, Zhen Qin, Xuanhui Wang, and Michael Bendersky. 2023. Query expansion by prompting large language models.arXiv preprint arXiv:2305.03653

    2023 arXiv

  13. [21]

    Qiao Jin, Won Kim, Qingyu Chen, Donald C Comeau, Lana Yeganova, W John Wilbur, and Zhiyong Lu. 2023. Medcpt: con- trastive pre-trained transformers with large-scale pubmed search logs for zero-shot biomedical information retrieval.Bioinfor- matics, 39, 11, btad651

    2023

  14. [22]

    Jia-Huei Ju, Sheng-Chieh Lin, Ming-Feng Tsai, and Chuan- Ju Wang. 2023. Improving conversational passage re-ranking with view ensemble. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2077–2081

    2023

  15. [23]

    arXiv preprint arXiv:2112.09118

    Unsupervised dense information retrieval with contrastive learning. arXiv preprint arXiv:2112.09118

    arXiv

  16. [24]

    Yibin Lei, Yu Cao, Tianyi Zhou, Tao Shen, and Andrew Yates

  17. [25]

    Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021. Pyserini: a python toolkit for reproducible information retrieval research with sparse and dense representations. InProceedings of the 44th International ACM SIGIR Conference o...

    2021

  18. [26]

    Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, and Xilun Chen. 2023. How to train your dragon: diverse augmentation towards generalizable dense retrieval.arXiv preprint arXiv:2302.07452

    2023 arXiv

  19. [27]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In EMNLP (1), 6769–6781

    2020

  20. [28]

    Sumbal Malik, Umar Shoaib, Syed Ahmad Chan Bukhari, Hes- ham El Sayed, and Manzoor Ahmed Khan. 2022. A hybrid query expansion framework for the optimal retrieval of the biomedical literature. Smart Health, 23, 100247

    2022

  21. [29]

    In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Vol- ume 2: Short Papers)

    Corpus-steered query expansion with large language mod- els. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Vol- ume 2: Short Papers). Yvette Graham and Matthew Purver, (Eds.) Association for Computational Lingui...

    2024

  22. [30]

    Monica Munnangi, Sergey Feldman, Byron C Wallace, Silvio Amir, Tom Hope, and Aakanksha Naik. 2024. On-the-fly defini- tion augmentation of llms for biomedical ner.arXiv preprint arXiv:2404.00152

    2024 arXiv

  23. [31]

    Jianmo Ni et al. 2021. Large dual encoders are generalizable retrievers. arXiv preprint arXiv:2112.07899

    2021 arXiv

  24. [32]

    Carolyn E Lipscomb. 2000. Medical subject headings (mesh). Bulletin of the Medical Library Association, 88, 3, 265

    2000

  25. [33]

    National Institutes of Health et al. 2002. Computer retrieval of information on scientific projects.Available at: http: www- commons. cit. nih. gov/crisp. Accessed September, 6

    2002

  26. [34]

    Niklas Muennighoff. 2022. Sgpt: gpt sentence embeddings for semantic search.arXiv preprint arXiv:2202.08904

    2022 arXiv

  27. [35]

    Gustavo Penha, Arthur Câmara, and Claudia Hauff. 2022. Eval- uating the robustness of retrieval pipelines with query variation generators. InEuropean conference on information retrieval. Springer, 397–412

    2022

  28. [36]

    Chen Qu, Liu Yang, Cen Chen, Minghui Qiu, W Bruce Croft, and Mohit Iyyer. 2020. Open-retrieval conversational question answering. InProceedings of the 43rd International ACM SI- GIR conference on research and development in Information Retrieval, 539–548

    2020

  29. [37]

    Kosuke Nishida, Naoki Yoshinaga, and Kyosuke Nishida. 2023. Sparse neural retrieval model for efficient cross-domain retrieval- based question answering. InProceedings of the 37th Pacific Asia Conference on Language, Information and Computation, 819–830

    2023

  30. [38]

    Amanpreet Singh, Mike D’Arcy, Arman Cohan, Doug Downey, and Sergey Feldman. 2022. Scirepeval: a multi-format bench- mark for scientific document representations.arXiv preprint arXiv:2211.13308

    2022 arXiv

  31. [39]

    Rituraj Ojha and Gerard Deepak. 2021. Metadata driven seman- tically aware medical query expansion. InKnowledge Graphs and Semantic Web: Third Iberoamerican Conference and Sec- ond Indo-American Conference, KGSWC 2021, Kingsville, Texas, USA, November 22–24, 2021, Proceedings ...

    2021

  32. [40]

    Sonish Sivarajkumar, Haneef Ahamed Mohammad, David Oni- ani, Kirk Roberts, William Hersh, Hongfang Liu, Daqing He, Shyam Visweswaran, and Yanshan Wang. 2024. Clinical in- formation retrieval: a literature review.Journal of healthcare informatics research, 8, 2, 313–352

    2024

  33. [41]

    Hongjin Su et al. 2022. One embedder, any task: instruction- finetuned text embeddings.arXiv preprint arXiv:2212.09741

    2022 arXiv

  34. [42]

    Sina Shool, Sara Adimi, Reza Saboori Amleshi, Ehsan Bitaraf, Reza Golpira, and Mahmood Tara. 2025. A systematic review of large language model (llm) evaluations in clinical medicine. BMC Medical Informatics and Decision Making, 25, 1, 117

    2025

  35. [43]

    Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Sri- vastava, and Iryna Gurevych. 2021. BEIR: a heterogeneous benchmark for zero-shot evaluation of information retrieval models. InThirty-fifth Conference on Neural Information Pro- cessing Systems Datasets and Benchmarks ...

    2021

  36. [44]

    Karan Singhal et al. 2025. Toward expert-level medical question answering with large language models.Nature Medicine, 1–8

    2025

  37. [45]

    Ellen Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner- Fushman, William R Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, and Lucy Lu Wang. 2021. Trec-covid: constructing a pandemic information retrieval test collection. InACM SIGIR Forum number 1. Vol. 54. ACM New York, NY, USA, 1–12

    2021

  38. [46]

    DavidWadden,ShanchuanLin,KyleLo,LucyLuWang,Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact Ontology-Guided Query Expansion for Biomedical Document Retrieval using Large Language Models or fiction: verifying scientific claims. InProceedings of the 2020 Co...

    2020 doi

  39. [47]

    Lynda Tamine and Lorraine Goeuriot. 2021. Semantic informa- tion retrieval on medical texts: research challenges, survey, and open issues.ACM Computing Surveys (CSUR), 54, 7, 1–38

    2021

  40. [48]

    Xiao Wang, Craig Macdonald, Nicola Tonellotto, and Iadh Ounis

  41. [49]

    Mathias Vast, Yuxuan Zong, Benjamin Piwowarski, and Laure Soulier. 2024. Simple domain adaptation for sparse retrievers. In European Conference on Information Retrieval. Springer, 403–412

    2024

  42. [50]

    Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen

  43. [51]

    Zhi Zheng, Kai Hui, Ben He, Xianpei Han, Le Sun, and An- drew Yates. 2020. Bert-qe: contextualized query expansion for document re-ranking.arXiv preprint arXiv:2009.07258

    2020 arXiv

  44. [52]

    Liang Wang, Nan Yang, and Furu Wei. 2023. Query2doc: query expansion with large language models.arXiv preprint arXiv: 2303 . 07678

    2023

  45. [53]

    Kaiwen Zuo and Yirui Jiang. 2024. Medhallbench: a new bench- mark for assessing hallucination in medical large language mod- els. arXiv preprint arXiv:2412.18947

    2024 arXiv

  46. [54]

    Colbert-prf: semantic pseudo-relevance feedback for dense passage and document retrieval.ACM Transactions on the Web, 17, 1, 1–39

  47. [55]

    Ran Xu, Wenqi Shi, Yue Yu, Yuchen Zhuang, Yanqiao Zhu, May D Wang, Joyce C Ho, Chao Zhang, and Carl Yang. 2024. Bmretriever: tuning large language models as better biomedical text retrievers.arXiv preprint arXiv:2404.18443

    2024 arXiv

  48. [57]

    Dense text retrieval based on pretrained language models: a survey.ACM Transactions on Information Systems, 42, 4, 1–60

  49. [59]

    Aleksandar Zivaljevic, Koray Atalag, and James Warren. 2022. Utility of snomed ct in automated expansion of clinical terms in discharge summaries: testing issues of coverage.Health In- formation Management Journal, 51, 1, 23–31

    2022

  50. [2021]

    In Proceedings of the 44th International ACM SI- GIR Conference on Research and Development in Information Retrieval, 2288–2292

    Splade: sparse lexical and expansion model for first stage ranking. In Proceedings of the 44th International ACM SI- GIR Conference on Research and Development in Information Retrieval, 2288–2292

  51. [2023]

    InTerminology, Ontology and their Implementations

    National cancer institute thesaurus (ncit). InTerminology, Ontology and their Implementations. Springer, 395–441

  52. [2024]

    InFindings of the Association for Computational Linguistics ACL 2024, 11908–11922

    Analyze, generate and refine: query expansion with llms for zero-shot open-domain qa. InFindings of the Association for Computational Linguistics ACL 2024, 11908–11922

    2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.