Pith. sign in

REVIEW 3 major objections 6 minor 41 references

Aligning LLMs for the Classroom with Knowledge-Based Retrieval -- A Comparative RAG Study

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that no single retrieval-augmented generation setup is best for classroom question answering: vector RAG wins on quick factual queries, GraphRAG on thematic depth, and a router that combines both raises faithfulness.

desk verdict A useful new dataset and a plausible RAG comparison for education, but the unvalidated LLM-judge loop means the headline rankings are conditional, not proven. read the letter →

arxiv 2509.07846 v1 pith:URFJNYM6 submitted 2025-09-09 cs.AI

Amay Jain , Liu Cui , Si Chen This is my paper · ORCID
classification cs.AI
keywords retrievalaugmentedgenerationclassroomquestionansweringvectorgraphLLM-as-a-judgeknowledgeshifteducationaltechnologyEduScopeQA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to give educators a practical answer to which retrieval-augmented generation (RAG) setup should power classroom question answering. It constructs EduScopeQA, a 3,176-question dataset spanning four subjects and three levels of question scope, and pairs it with KnowShiftQA, a set of textbooks whose facts have been deliberately altered. The central finding is that no single RAG method wins everywhere: OpenAI's vector-based RAG is low-cost and best for specific factual lookups, GraphRAG Global produces richer answers to thematic questions, and GraphRAG Local is most accurate when the corpus is large, dense, and must override the model's outdated knowledge. The authors then show that a dynamic branching system that routes each question to the method best suited to its scope can improve average faithfulness over any single system, at lower cost than running graph RAG on everything.

What carries the argument

The machinery that carries the argument is the scope-resolved evaluation design plus a routing prompt. EduScopeQA labels each question specific, sectional, or thematic, so retrieval quality is measured where the demands differ; KnowShiftQA provides a control where correctness requires trusting the corpus over the model's memory. Win rates are computed by an LLM judge with AB-BA swapping to suppress position bias. A final GPT-4.1-Nano prompt, shown in the paper as a branching rule, inspects each query and routes it to OpenAI RAG, GraphRAG Local, or GraphRAG Global. The branch router is what converts the observed per-method strengths into a single deployable system.

What would settle it

Take a random sample of EduScopeQA questions and ask human teachers to rank the same answer pairs on the same four criteria; if the teachers' ordering across the three systems does not reproduce the LLM judge's win-rate ordering, the central claim is refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is a division of labor among retrieval paradigms. Across the 3,176 EduScopeQA pairs, GraphRAG Global won the highest faithfulness, comprehensiveness, and learnability scores on sectional and thematic questions, because its global graph summaries let the generator synthesize dispersed material. OpenAI RAG won directness and accuracy on specific questions, because such answers live in one retrieved snippet. GraphRAG Local sat in between and, in the altered-textbook study, achieved the highest accuracy on the largest, densest textbooks, where it stayed faithful to the provided material even when it contradicted the LLM's latent knowledge. On smaller corpora, OpenAI RAG matched or beat graph methods. The paper concludes that a lightweight branch router, which sends quick factual queries to vector RAG and broader queries to graph RAG, improves average faithfulness and avoids the worst failures of any single system.

Load-bearing premise

The rankings rest on GPT-4.1-family judges: reference answers were generated by GPT-4.1 and every comparison was scored by GPT-4.1-Nano, with no human-validated subset, so if those judges rank responses differently from real teachers and students, the deployment guidance does not follow.

Editorial extensions

If this is right

  • A school deploying a general chatbot can default to vector RAG for homework help and glossary-style questions, avoiding graph indexing cost.
  • For seminar discussions or essay prompts on a single text, GraphRAG Global's higher indexing and query cost is justified by markedly higher pedagogical criteria.
  • When curricula use large textbooks with revised facts, GraphRAG Local is the safer choice for exam-aligned multiple-choice accuracy.
  • The branching system's faithfulness beat every standalone system, so routing by question scope is a viable low-overhead deployment strategy.
  • GraphRAG's indexing cost can be amortized by building the index once and sharing it across cohorts, making it practical for courses that reuse a text.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that the size of the branching gain is likely overstated if the judge model shares the generator's blind spots, since every score in the comparison comes from one model family; a teacher-rated subset would be the sharp test.
  • The routing decision can probably be made without an LLM: query scope, length, and corpus size are features the brancher appears to rely on, so a cheap classifier or rules could replicate much of the gain.
  • The same cost–accuracy trade-off probably extends to image- or video-based classroom material, but the graph-construction overhead would be different; testing visual RAG under the same scope-resolved rubric would be a natural next step.
  • The paper's cost numbers count indexing overhead for every routed query, so persistent indexed corpora across terms would narrow the gap to pure vector RAG in practice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript compares two RAG paradigms for classroom question answering: OpenAI Vector Search RAG (vector-based) and Microsoft GraphRAG in Local and Global modes (graph-based). Using a new dataset, EduScopeQA, of 3,176 open-ended questions across Literature, History, Computer Science, and Science, it evaluates answers with an LLM-as-a-judge protocol across four criteria. A second case study uses the KnowShiftQA dataset of systematically altered textbooks to measure whether each system follows the provided corpus over the model's latent knowledge. The paper reports that OpenAI RAG excels at specific fact retrieval, GraphRAG Global provides richer thematic answers, GraphRAG Local is most faithful on dense altered textbooks, and a proof-of-concept branching router combines these strengths with modest efficiency gains.

Significance. If the reported rankings are valid, the paper would provide actionable deployment guidance for educators and system designers, and the EduScopeQA dataset would be a useful resource for future classroom-RAG research. The manuscript has several genuine strengths: it releases a new multi-subject dataset, it measures indexing and query costs in a deployable way, it controls for position bias with an AB-BA swap, and it tests knowledge-shift robustness with an altered-corpus design. However, the central evaluation loop—GPT-4.1-generated reference answers, GPT-4.1-Mini-generated candidate answers, and GPT-4.1-Nano as judge—is not validated against any human or external standard, and the reported win rates lack uncertainty estimates. The headline qualitative findings therefore remain plausible but not established.

major comments (3)
  1. [Section III-A, III-C, and Limitations] The validity of all Case Study 1 quality rankings rests on GPT-4.1-Nano judgments of answers produced by GPT-4.1-Mini against reference answers generated by GPT-4.1. No human-validated subset, no inter-annotator agreement, and no independent measure of pedagogical quality are reported, and the Limitations paragraph explicitly concedes that alignment with actual educational outcomes remains unvalidated. Because the criteria 'Faithfulness' and 'Learnability' are operationalized only as this judge's preferences, the patterns in Table II and Fig. 2 could be produced by a systematic preference for longer, more narrative outputs or for style similarity to the GPT-4.1 reference summaries. The AB-BA swap controls position bias only; it does not establish that the judge's notion of quality matches what teachers or students need. The authors should calibrate the judge against human expert ratings on a subset, or provide an external outcome measure, before the central deployment claims can be accepted.
  2. [Table I, Table II, and Eq. (1)] Win rates are reported without confidence intervals or significance tests despite small cell sizes. For example, Science has only 20 thematic questions and History has 38 thematic questions (Table I), so a single judgment flip changes a thematic win rate by 5 percentage points or more. The discipline-level differences discussed in Section III-D (e.g., Computer Science vs. Literature faithfulness gaps) may be within the noise of the pairwise LLM judgments. Please provide bootstrapped confidence intervals, exact per-cell counts, and significance tests for the key pairwise comparisons that support the abstract's qualitative claims.
  3. [Section V-A and Table IV] The branching-system evaluation inherits the same unvalidated LLM judge, and the abstract's claim that the router 'boosts fidelity and efficiency' is not supported by significance testing or a cost-benefit analysis that accounts for judge uncertainty. Table IV is also difficult to interpret: the caption says 'Branching System Vs. The Rest,' but the columns list the three standalone systems, and it is unclear whether the percentages are the branching system's win rates against each method or vice versa. The reported values lack cell counts, confidence intervals, and a clear definition of the comparison direction. Please clarify the table and add uncertainty measures before using it as evidence for the routing framework.
minor comments (6)
  1. [Section I] There is a typo in the introduction: 'question anwering' should be 'question answering.'
  2. [Table II vs. Section III-C] The evaluation section defines four criteria (Comprehensiveness, Directness, Faithfulness, Learnability), but Table II labels the fourth row 'Accuracy' instead of 'Faithfulness.' The text elsewhere refers to faithfulness; please standardize the terminology.
  3. [Table IV and Fig. 6] The branching-system results would be much easier to interpret if the authors stated explicitly how win percentages are computed, how ties are handled, and how many questions contribute to each reported value.
  4. [Section III-A and GitHub release] The dataset release is stated to cover History, Literature, and Science only, yet the paper analyzes Computer Science texts. Please clarify whether the Computer Science portion is available or excluded for licensing reasons, since this affects reproducibility.
  5. [References] Reference [9] is listed as 'in press' without a venue or year; please provide the full bibliographic details if available.
  6. [Experimental details] Please specify the exact model versions and access dates for GPT-4.1, GPT-4.1-Mini, and GPT-4.1-Nano, as well as the embedding model used by OpenAI Vector Search, since these details affect reproducibility and the interpretation of cost measurements.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the evaluation-loop limitations are external-validity concerns, not definitional reductions.

full rationale

The paper's central results are empirical rankings produced by a defined pipeline: GPT-4.1 generates EduScopeQA reference answers, GPT-4.1-Mini answers queries, GPT-4.1-Nano performs pairwise AB-BA judgments, and win rates are aggregated by the explicit formula W_A = (w_A + 0.5 t_A)/n. No parameter is fitted to a subset and then renamed as a prediction, and no conclusion is defined in terms of the quantity it is supposed to establish. The fact that reference answers, candidate answers, and judge all come from the GPT-4.1 model family is a legitimate external-validity threat, and the paper acknowledges this in its Limitations paragraph ('classroom pilots and co-design studies with teachers and students to validate our evaluation’s alignment with actual educational outcomes'), but it is not a circularity: the judge is not given the intended ranking, and the win-rate formula does not presuppose which system wins. Case Study 2 additionally relies on the external KnowShiftQA dataset with systematically altered facts, which provides an independent accuracy signal. The branching results are explicitly offered as a proof of concept, not as a derived theorem, and the router is evaluated against the same benchmark rather than being constructed to force the reported outcome. Citations to GraphRAG, LightRAG, and LLM-as-judge work are external to the authors and are not used to smuggle in an unverified uniqueness claim. Accordingly, no circular step meets the evidence bar of a definitional equivalence or a fitted-parameter-renamed-as-prediction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No mathematical derivation is present; the central claims rest on empirical evaluation. The main free parameters are hand-chosen dataset generation thresholds. The key assumptions are the validity of LLM-generated ground truth and LLM-as-judge, and the representativeness of the two specific RAG products.

free parameters (3)
  • Section size = 10 chunks
    Dataset pipeline groups chunks into sections of ten; chosen by hand and affects the distribution of specific versus sectional questions.
  • Summarization character threshold = 35,000 characters
    Recursive summarization splits content above this threshold; affects the granularity of generated QA pairs.
  • Question sampling fraction = Not specified
    A subset of screened sections was randomly sampled for QA generation; the exact ratio is not reported.
assumptions (4)
  • domain assumption GPT-4.1-generated QA pairs are valid ground truth for faithfulness and answerability
    EduScopeQA questions and reference answers were generated by GPT-4.1 without human validation; the paper relies on this for all Case Study 1 faithfulness scores (Section III-A).
  • domain assumption LLM-as-a-judge with GPT-4.1-Nano produces human-level quality orderings
    Section III-C cites prior work for this, but no human calibration subset or inter-annotator agreement is reported for this specific setup.
  • domain assumption OpenAI Vector Search and Microsoft GraphRAG are representative of vector-based and graph-based RAG classes
    The paper draws general conclusions about RAG paradigms from two turnkey products; implementation-specific differences are not controlled.
  • domain assumption KnowShiftQA's altered textbooks are internally consistent and questions are aligned
    Case Study 2 relies entirely on the external in-press KnowShiftQA dataset [9]; the paper does not audit the dataset.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligning LLMs for the Classroom with Knowledge-Based Retrieval -- A Comparative RAG Study." pith.science (2026). https://pith.science/paper/URFJNYM6

@misc{pith2026250907846,
  author       = {Pith},
  title        = {Pith review of: Aligning LLMs for the Classroom with Knowledge-Based Retrieval -- A Comparative RAG Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/URFJNYM6}},
  note         = {Machine review of arXiv:2509.07846}
}
read the original abstract

Large language models like ChatGPT are increasingly used in classrooms, but they often provide outdated or fabricated information that can mislead students. Retrieval Augmented Generation (RAG) improves reliability of LLMs by grounding responses in external resources. We investigate two accessible RAG paradigms, vector-based retrieval and graph-based retrieval to identify best practices for classroom question answering (QA). Existing comparative studies fail to account for pedagogical factors such as educational disciplines, question types, and practical deployment costs. Using a novel dataset, EduScopeQA, of 3,176 questions across academic subjects, we measure performance on various educational query types, from specific facts to broad thematic discussions. We also evaluate system alignment with a dataset of systematically altered textbooks that contradict the LLM's latent knowledge. We find that OpenAI Vector Search RAG (representing vector-based RAG) performs well as a low-cost generalist, especially for quick fact retrieval. On the other hand, GraphRAG Global excels at providing pedagogically rich answers to thematic queries, and GraphRAG Local achieves the highest accuracy with the dense, altered textbooks when corpus integrity is critical. Accounting for the 10-20x higher resource usage of GraphRAG (representing graph-based RAG), we show that a dynamic branching framework that routes queries to the optimal retrieval method boosts fidelity and efficiency. These insights provide actionable guidelines for educators and system designers to integrate RAG-augmented LLMs into learning environments effectively.

Figures

Figures reproduced from arXiv: 2509.07846 by the authors.

Figure 1
Figure 1. EduScopeQA Dataset Question Generation Pipeline [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Case Study 1 Results: Average Win Rates across question types and criteria [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. LLM Prompt for Case Study 2 B. Evaluation We checked if the chosen answer was the correct (altered) one by performing a fuzzy matching using a GPT-4.1-Nano evaluation. We measured accuracy in terms of percentage of questions answered correctly. C. Discussion Our findings ( [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Case Study 2 Results: Average Accuracy by Subject and Retrieval Scope. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Branching prompt to choose the optimal retrieval method [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Case Study 2 Results with Branching System, averaged across subjects [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 25 canonical work pages

  1. [1]

    New era of artificial intelligence in education: Towards a sustainable multifaceted revolution,

    F. Kamalov, D. S. Calonge, and I. Gurrib, “New era of artificial intelligence in education: Towards a sustainable multifaceted revolution,” Sustainability, vol. 15, no. 16, 2023

    work page 2023

  2. [2]

    Siren’s song in the ai ocean: A survey on hallucination in large language models,

    Y . Zhanget al., “Siren’s song in the ai ocean: A survey on hallucination in large language models,” 2023, arXiv preprint arXiv:2309.01219

    arXiv 2023

  3. [3]

    What is the impact of chatgpt on education? a rapid review of the literature,

    C. K. Lo, “What is the impact of chatgpt on education? a rapid review of the literature,”Education Sciences, vol. 13, no. 4, 2023. [Online]. Available: https://www.mdpi.com/2227-7102/13/4/410

    work page 2023

  4. [4]

    Retrieval-augmented generation for knowledge-intensive nlp tasks,

    P. Lewiset al., “Retrieval-augmented generation for knowledge-intensive nlp tasks,”CoRR, vol. abs/2005.11401, 2020

    arXiv 2005

  5. [5]

    Evaluating retrieval-augmented generation models for financial report question and answering,

    I. Iaroshev, R. Pillai, L. Vaglietti, and T. Hanne, “Evaluating retrieval-augmented generation models for financial report question and answering,”Applied Sciences, vol. 14, no. 20, 2024. [Online]. Available: https://www.mdpi.com/2076-3417/14/20/9318

    work page 2024

  6. [6]

    Retrieval-augmented generation for large language models: A survey,

    Y . Gaoet al., “Retrieval-augmented generation for large language models: A survey,” 2024, arXiv preprint arXiv:2312.10997

    arXiv 2024

  7. [7]

    From local to global: A graph rag approach to query- focused summarization,

    D. Edgeet al., “From local to global: A graph rag approach to query- focused summarization,” 2024, arXiv preprint arXiv:2404.16130

    arXiv 2024

  8. [8]

    Retrieval - openai platform documentation,

    OpenAI, “Retrieval - openai platform documentation,” 2024, [Online]. Available: https://platform.openai.com/docs/guides/retrieval# vector-stores

    work page 2024

Show all 41 references
  1. [9]

    Knowshiftqa: How robust are rag systems when textbook knowledge shifts in k-12 education?

    T. Zheng, W. Li, J. Bai, W. Wang, and Y . Song, “Knowshiftqa: How robust are rag systems when textbook knowledge shifts in k-12 education?” 2025, in press

    2025

  2. [10]

    Retrieval-augmented generation to improve math question-answering: Trade-offs between groundedness and human pref- erence,

    Z. Levonianet al., “Retrieval-augmented generation to improve math question-answering: Trade-offs between groundedness and human pref- erence,” 2023, arXiv preprint arXiv:2310.03184

    2023 arXiv

  3. [11]

    Enhancing llm-based short answer grading with retrieval-augmented generation,

    Y . Chuet al., “Enhancing llm-based short answer grading with retrieval-augmented generation,” 2025, arXiv preprint arXiv:2504.05276, doi:10.48550/arXiv.2504.05276

    2025 doi arXiv

  4. [12]

    Does chatgpt enhance student learning? a systematic review and meta-analysis of experimental studies,

    R. Deng, M. Jiang, X. Yu, Y . Lu, and S. Liu, “Does chatgpt enhance student learning? a systematic review and meta-analysis of experimental studies,”Computers & Education, vol. 227, p. 105224, 2025

    2025

  5. [13]

    Lightrag: Simple and fast retrieval-augmented generation,

    Z. Guo, L. Xia, Y . Yu, T. Ao, and C. Huang, “Lightrag: Simple and fast retrieval-augmented generation,” 2025, arXiv preprint arXiv:2410.05779

    2025 arXiv

  6. [14]

    Rag vs. graphrag: A systematic evaluation and key insights,

    H. Hanet al., “Rag vs. graphrag: A systematic evaluation and key insights,” 2025, arXiv preprint arXiv:2502.11371

    2025

  7. [15]

    How significant are the real performance gains? an unbiased evaluation framework for graphrag,

    Q. Zenget al., “How significant are the real performance gains? an unbiased evaluation framework for graphrag,” 2025, arXiv preprint arXiv:2506.06331

    2025 arXiv

  8. [16]

    Natural questions: A benchmark for question answering research,

    T. Kwiatkowskiet al., “Natural questions: A benchmark for question answering research,”Transactions of the Association of Computational Linguistics, 2019

    2019

  9. [17]

    Hotpotqa: A dataset for diverse, explainable multi-hop question answering,

    Z. Yanget al., “Hotpotqa: A dataset for diverse, explainable multi-hop question answering,”CoRR, vol. abs/1809.09600, 2018

    2018 arXiv

  10. [18]

    Multihop-rag: Benchmarking retrieval- augmented generation for multi-hop queries,

    Y . Tang and Y . Yang, “Multihop-rag: Benchmarking retrieval- augmented generation for multi-hop queries,” 2024, arXiv preprint arXiv:2401.15391

    2024 arXiv

  11. [19]

    Are you smarter than a sixth grader? textbook question answer- ing for multimodal machine comprehension,

    A. Kembhavi, M. Seo, D. Schwenk, J. Choi, A. Farhadi, and H. Ha- jishirzi, “Are you smarter than a sixth grader? textbook question answer- ing for multimodal machine comprehension,” in2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 5376– 5384

    2017

  12. [20]

    Can a suit of armor conduct electricity? a new dataset for open book question answering,

    T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal, “Can a suit of armor conduct electricity? a new dataset for open book question answering,” CoRR, vol. abs/1809.02789, 2018

    2018 arXiv

  13. [21]

    Gutenberg, [Online]

    P. Gutenberg, [Online]. Available: https://www.gutenberg.org/

  14. [22]

    Available: https://arxiv.org/

    arXiv, [Online]. Available: https://arxiv.org/

  15. [23]

    Moby-dick; or, the whale,

    H. Melville, “Moby-dick; or, the whale,” 1851, [Online]. Available: https://www.gutenberg.org/ebooks/2701

  16. [24]

    Little women; or, meg, jo, beth, and amy,

    L. M. Alcott, “Little women; or, meg, jo, beth, and amy,” 1868, [Online]. Available: https://www.gutenberg.org/ebooks/514

  17. [25]

    The north pole: Its discovery in 1909 under the auspices of the peary arctic club,

    R. E. Peary, “The north pole: Its discovery in 1909 under the auspices of the peary arctic club,” 1910, [Online]. Available: https://www.gutenberg. org/ebooks/18975

    1909

  18. [26]

    A history of the philippines,

    D. P. Barrows, “A history of the philippines,” 1905, [Online]. Available: https://www.gutenberg.org/ebooks/38269

    1905

  19. [27]

    The autobiography of benjamin franklin,

    B. Franklin, “The autobiography of benjamin franklin,” 1791, [Online]. Available: https://www.gutenberg.org/ebooks/20203

  20. [28]

    The economic consequences of the peace,

    J. M. Keynes, “The economic consequences of the peace,” 1919, [Online]. Available: https://www.gutenberg.org/ebooks/15776

    1919

  21. [29]

    Narrative of the life of frederick douglass, an american slave,

    F. Douglass, “Narrative of the life of frederick douglass, an american slave,” 1845, [Online]. Available: https://www.gutenberg.org/ebooks/23

  22. [30]

    Common sense,

    T. Paine, “Common sense,” 1776, [Online]. Available: https://www. gutenberg.org/ebooks/147

  23. [31]

    Foundations of vector retrieval,

    S. Bruch, “Foundations of vector retrieval,” 2024, arXiv preprint arXiv:2401.09350

    2024 arXiv

  24. [32]

    Common information, noise stability, and their extensions,

    L. Yu and V . Y . F. Tan, “Common information, noise stability, and their extensions,” 2022, arXiv preprint arXiv:2211.01788

    2022 arXiv

  25. [33]

    A modern introduction to online learning,

    F. Orabona, “A modern introduction to online learning,” 2019, arXiv preprint arXiv:1912.13213

    2019 arXiv

  26. [34]

    A brief introduction to machine learning for engineers,

    O. Simeone, “A brief introduction to machine learning for engineers,” 2017, arXiv preprint arXiv:1709.02840

    2017 arXiv

  27. [35]

    Community detection and stochastic block models: Recent developments,

    E. Abbe, “Community detection and stochastic block models: Recent developments,” 2017, arXiv preprint arXiv:1703.10146

    2017 arXiv

  28. [36]

    Convex optimization: Algorithms and complexity,

    S. Bubeck, “Convex optimization: Algorithms and complexity,” 2014, arXiv preprint arXiv:1405.4980

    2014 arXiv

  29. [37]

    System architecture optimization strategies: Dealing with expensive hierarchical problems,

    J. H. Bussemaker, P. Saves, N. Bartoli, T. Lefebvre, and R. Lafage, “System architecture optimization strategies: Dealing with expensive hierarchical problems,” 2025, arXiv preprint arXiv:2502.00838

    2025 arXiv

  30. [38]

    Microbiology,

    N. Parker, M. Schneegurt, A.-H. T. Tu, B. M. Forster, and P. Lister, “Microbiology,” 2016, [Online]. Available: https://openstax.org/books/ microbiology

    2016

  31. [39]

    Can large language models be an alternative to human evaluations?

    C.-H. Chiang and H.-Y . Lee, “Can large language models be an alternative to human evaluations?” inProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), A. Rogers, J. Boyd-Graber, and N. Okazaki, Eds. Toronto, Canada:...

    2023

  32. [40]

    Judging the judges: A systematic study of position bias in llm-as-a-judge,

    L. Shi, C. Ma, W. Liang, X. Diao, W. Ma, and S. V osoughi, “Judging the judges: A systematic study of position bias in llm-as-a-judge,” 2025, arXiv preprint arXiv:2406.07791

    2025

  33. [41]

    Humans or llms as the judge? a study on judgement biases,

    G. H. Chen, S. Chen, Z. Liu, F. Jiang, and B. Wang, “Humans or llms as the judge? a study on judgement biases,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Miami, Florida, USA: Association for Computational Linguistics, Nov. 2024, ...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.