Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on a Math Textbook

Chuangji Li; Eason Chen; Eric Li; Jionghao Lin; Kenneth R. Koedinger; Zimo Xiao

REVIEW 4 major objections 6 minor 2 cited by

Page-level embedding RAG beats GraphRAG on math-textbook retrieval, with 99.4% top-10 accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 16:03 UTC pith:X3XFOMA3

load-bearing objection Useful new dataset and a systematic embedding comparison, but the RAG-beats-GraphRAG claim is not apples-to-apples and the abstract overstates what the body shows. the 4 major comments →

arxiv 2509.16780 v3 pith:X3XFOMA3 submitted 2025-09-20 cs.IR cs.AIcs.HC

Comparing RAG and GraphRAG for Page-Level Retrieval Question Answering on a Math Textbook

Eason Chen , Chuangji Li , Eric Li , Zimo Xiao , Jionghao Lin , Kenneth R. Koedinger This is my paper

classification cs.IR cs.AIcs.HC

keywords retrieval-augmented generationGraphRAGpage-level retrievalmathematical textbook QAAI tutoringknowledge graphsquestion answeringretrieval accuracy

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether an AI retrieval system can point a student to the exact page of a math textbook where a question is answered. On 477 page-tied questions, standard embedding-based RAG beats GraphRAG for page-level retrieval: the best embedding model finds the correct page 68.6% of the time at top-1 and 99.4% at top-10, while GraphRAG's accuracy reaches 91.4% only with far more context. The paper's explanation is structural: GraphRAG expands entity neighborhoods and returns roughly 47,000 tokens per question, whereas page-chunked RAG returns about 3,700 tokens, and that excess context drags down answer-generation F1. The result matters for AI tutors because a system that cites exact pages is easier for students to verify, and the paper shows that simpler page-based indexing beats graph-based complexity for this goal.

Core claim

On the paper's own terms, the central discovery is that the natural unit for a textbook-answering task is the page, not the knowledge graph. Dense embeddings chunked by page retrieve the correct page for 99.4% of questions when ten candidates are allowed; GraphRAG, which links entities across pages, retrieves broader neighborhoods that include the right content but also drown the generator in roughly 47,000 tokens per query. The authors adapt GraphRAG to attach document IDs to entities so pages can be referenced, yet it still produces lower F1 answers (around 0.52) than page-based RAG (around 0.53–0.55). They also find that re-ranking retrieved pages with an LLM does not reliably improve acc

What carries the argument

The load-bearing mechanism is page-level chunking for retrieval: the textbook is split page by page, each page is embedded as a vector, and a question retrieves the top-k pages by cosine similarity. GraphRAG is the contrast: it builds an entity-relation graph and retrieves related entities with document IDs attached, which breaks the book's linear page structure. The mismatch between graph fragmentation and page-level organization is the mechanism the paper blames for excessive context length and lower generation quality. Evaluation uses two tools: retrieval accuracy (is the ground-truth page among the retrieved pages) and token-overlap F1 between the generated answer and the reference answe

Load-bearing premise

The 477 test questions were written by an LLM from the very page they are supposed to retrieve, so the benchmark may be easier than real student questions; if students ask differently, the retrieval advantage could shrink.

What would settle it

Ask a group of students to write questions about this textbook, run the same voyage-3-large page retrieval on those questions, and compare top-1 accuracy with the paper's 68.6%; a large drop, or a case where GraphRAG then beats page-level RAG on answer F1, would undercut the central conclusion.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

System builders should index textbook pages as individual retrieval units; page-level RAG gives near-perfect coverage at top-10, so AI tutors can confidently offer a short list of candidate pages.
Retrieval accuracy is not the same as answer quality: GraphRAG can locate relevant content but its oversized context lowers generated-answer F1, so context length must be controlled separately.
LLM-based page re-ranking is not a reliable enhancement; it often hurts top-1 accuracy and can invent nonexistent page references, so it should be deployed with caution.
Weaker open-source models benefit proportionally more from RAG than stronger models do, which makes retrieval augmentation especially valuable for cost-sensitive deployments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Because the 477 questions were generated from page text by an LLM and then filtered by the authors, a natural next test is to run the same retrieval stack on questions written by students; if top-1 accuracy drops, the reported advantage may not transfer to classrooms.
Token-overlap F1 is a weak proxy for mathematical correctness; a human panel comparing GraphRAG and RAG answers could reveal whether the extra graph context adds pedagogical value that F1 misses, or whether it actively confuses.
A direct test of the paper's diagnosis would be to impose page-level constraints on GraphRAG's graph expansion (for example, stop entity traversal at page boundaries) and see whether its F1 rises to meet embedding RAG.
The token-count gap implies a cost consequence: GraphRAG consumes over 12 times more input tokens per question, so page-level RAG is likely cheaper at scale as well as more accurate.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful new dataset and a systematic embedding comparison, but the RAG-beats-GraphRAG claim is not apples-to-apples and the abstract overstates what the body shows.

read the letter

The useful thing here is the new resource: 477 page-linked QA pairs from a real math textbook, plus a fairly careful comparison of five embedding models on a page-level retrieval task. The re-ranking experiments are a nice addition, and the authors are honest about the synthetic-data limitation in Section 5.7. If you work on educational retrieval, this dataset and the embedding rankings are worth a look.

The headline claim—embedding RAG outperforms GraphRAG—does not hold up as stated. The stress-test note is right: Table 1 compares top-k page lists (a few pages, ranked) against GraphRAG accuracy measured as "does the target page appear anywhere in the retrieved entity dump," which averages about 47K tokens per query. That is a high-recall, uncontrolled-context measurement. A method allowed to return ten to fifty times more context will trivially look better on inclusion. And the F1 gap separating them is tiny (0.52 vs. 0.54), so the qualitative conclusion could flip under a matched token or page budget. The authors do flag GraphRAG's excessive context as a practical cost, but that is not the same as a controlled retrieval comparison.

There are also several abstract claims that never show up in the body: the BM25 baseline, the Qwen3.5-35B-A3B replication, the bootstrap CI, and the 63.3% same-chapter error analysis are all absent. That is a real problem. The F1 calculation example also has a set-size error (the intersection set has three elements but is written as 2), which makes the equations internally inconsistent. No significance tests or error bars accompany the main tables, so the differences between embedding models may be noise.

The dataset generation process is another soft spot: gpt-4o-mini wrote each question from the target page itself, so retrieval may be artificially easy. The authors acknowledge this, but it still limits how far the absolute accuracies can be trusted.

Who is this for? People building page-level AI tutors or evaluating embedding models on textbook-style content will get value from the dataset and the general pattern. But the paper needs a matched-budget GraphRAG comparison, removal or correction of the unsupported abstract claims, and error bars before the central comparison is established. As it stands, it's a legitimate empirical contribution with one load-bearing comparison that is not yet properly controlled. I'd send it to review, but I'd expect serious revision and would not cite the GraphRAG conclusion as is.

Referee Report

4 major / 6 minor

Summary. The paper reports an empirical comparison of standard embedding-based RAG against GraphRAG for page-level retrieval and question answering on an undergraduate mathematics textbook. The authors build a dataset of 477 question–answer pairs by prompting gpt-4o-mini with each textbook page, then filtering manually. They evaluate five embedding models at top-1/3/5/10, a closed-book baseline, and GraphRAG, reporting retrieval accuracy (whether the target page appears in the retrieved set) and answer-generation F1. Their headline finding is that embedding-based RAG outperforms GraphRAG: voyage-3-large reaches 0.994 top-10 accuracy, while GraphRAG's accuracy is measured as 0.914 (with o3-mini) but returns far more context (~47K tokens per query). They also report that LLM-based re-ranking gives mixed results. The abstract additionally claims a BM25 baseline, a Qwen3.5-35B-A3B replication, a bootstrap confidence interval, and a same-chapter error analysis, none of which appear in the body.

Significance. If the comparison were methodologically sound, this would be a useful contribution to educational RAG: it addresses page-level grounding, a practically important requirement for AI tutoring systems, and the authors have open-sourced their data and code. The retrieval-accuracy finding for voyage-3-large, the token-cost observation for GraphRAG, and the re-ranking results are potentially informative for practitioners. However, the central RAG-versus-GraphRAG claim is currently undermined by an apples-to-oranges retrieval comparison, the benchmark's synthetic nature is acknowledged but not resolved, and several abstract-level claims are unsupported in the body. The paper's practical implications are real, but the headline comparison needs substantial reworking before the conclusions can be accepted as stated.

major comments (4)

[§4.1, Table 1, §5.3] The central claim that embedding-based RAG outperforms GraphRAG at page-level retrieval is not established by the reported measurements. In §4.1, GraphRAG accuracy is computed by checking whether the target page appears anywhere in the entity-based retrieval output, with no top-k or page budget; §5.3 states that GraphRAG returns 46,949 tokens per question on average, versus 3,743 tokens for voyage-3-large top-5 and 899 for top-1. Table 1 therefore compares a high-recall inclusion metric on a ~47K-token context against precision-oriented top-k accuracy on a small page list. The 0.914 GraphRAG value shows only that GraphRAG returns more context, not that it is better or worse at page retrieval. The paper should report GraphRAG under matched page/token budgets, or provide precision/recall at a controlled context size, before drawing the headline conclusion.
[§3.2, §5.7] The benchmark's validity for the claimed educational use is not established. Each question–answer pair is generated by gpt-4o-mini from the page content that is also the retrieval target, and the same model family (gpt-4o-mini) is used as the generator in the F1 evaluation. This setup can artificially inflate both retrieval accuracy and F1, because the generated questions likely reuse the page's vocabulary and formulation. The authors acknowledge in §5.7 that the content was LLM-generated and call for human-authored future work, but the paper's conclusions depend on these 477 items being a reasonable proxy for genuine student questions. Please provide at least a small human-authored validation set, or explicitly restrict the conclusions to LLM-generated questions.
[Abstract vs. Sections 3–5] Several results stated in the abstract are not present in the body: (i) the BM25 baseline; (ii) the bootstrap 95% CI [0.644, 0.728] for voyage-3-large top-1; (iii) the Qwen3.5-35B-A3B replication and the +39% vs +16% relative F1 improvements; (iv) the error analysis that 63.3% of top-1 failures retrieve same-chapter content. None of these appear in Sections 3–5, yet they are used in the abstract to support the paper's claims. Either add the corresponding experiments/analyses or remove these statements from the abstract.
[§4.2, Table 2, §5.3] The generative-quality conclusion rests on F1 differences that are small and not shown to be significant. GraphRAG with gpt-4o-mini scores 0.525, GraphRAG with o3-mini 0.524, while the best RAG row is 0.552 (OpenAI text-embedding-3-large top-3) but several RAG conditions are below GraphRAG (e.g., intfloat/multilingual-e5-large-instruct top-1 at 0.514, voyage-3-large top-1 at 0.523). No confidence intervals or paired tests are reported for Table 2. With 477 items a 0.02 difference may be real, but the paper does not demonstrate it. Minimum action: report bootstrap CIs or paired significance tests before interpreting the F1 gap as evidence that GraphRAG's context reduces generation quality.

minor comments (6)

[§3.5] The F1 example contains an arithmetic inconsistency: it writes |R∩G| = |{0, natural, number}| = 2, but the set has three elements, and the subsequent precision/recall calculations correctly use 3. Please fix the notation.
[§2.1] Typo: 'mdoel' should be 'model'.
[§3.5] 'Base on the fact computed above' should be 'Based on'.
[Table 3] The table formatting is confusing: the 'Retrieved Top 5' header and the arrow notation (e.g., 0.686→0.593) should be explained explicitly in the caption or text.
[§5.2] The random baseline '0.16%' assumes 628 pages, but the final dataset has 477 pages; if retrieval is over the filtered corpus the random baseline is ~0.21%. Please clarify the corpus size used in retrieval.
[Table 2] For GraphRAG rows, clarify which component uses gpt-4o-mini versus o3-mini. The text says generation is always done by gpt-4o-mini, but the table labels are ambiguous.

Circularity Check

0 steps flagged

No significant circularity: the comparison is empirical and self-contained, with only background self-citations.

full rationale

This paper reports an empirical benchmark comparison rather than a derivation, so the central RAG-vs-GraphRAG claims are not equivalent to their inputs by construction. The 477 QA pairs were generated from individual textbook pages with gpt-4o-mini and filtered by two authors (Section 3.2); while this is a validity limitation acknowledged in Section 5.7, it does not make retrieval accuracy a fitted parameter or a tautology—the embedding models are external MTEB models and are not optimized on the test set. The GraphRAG retrieval protocol differs from RAG (inclusion in uncontrolled ~47K-token output vs. top-k ranked pages, Sections 4.1 and 5.3), but this is a measurement/experimental-design confound, not a circular reduction; the reported numbers still come from direct evaluation. Self-citations [2,5,12,17] appear only in related-work motivation and are corroborated by non-author references [9,28,30]; they are not load-bearing for any result. No equation defines a predicted quantity in terms of an outcome, no fitted value is renamed as a prediction, and no conclusion reduces to a self-citation chain. Therefore no circular step is present.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The paper does not introduce new theoretical entities or fitted models; it reuses existing embedding models and a public GraphRAG implementation. Its main hidden assumptions are about the benchmark's construct validity and the unspecified GraphRAG configuration.

free parameters (2)

GraphRAG entity extraction and context-building thresholds = not reported
The authors modified GraphRAG to include document_ids and entity_ids (Section 3.4) but do not report entity confidence thresholds, number of entities, or context-building parameters. These hand choices affect the retrieved pages and token counts, and thus the GraphRAG comparison.
Top-k retrieval depth = 1, 3, 5, 10
These are standard evaluation hyperparameters rather than fitted values; they are not used to optimize the central claim, only to report accuracy curves.

axioms (3)

domain assumption Each question in the dataset is answerable from and grounded in exactly one textbook page, and the source page is the correct retrieval target.
Section 3.2 constructs one QA per page and evaluates retrieval against that page. The paper itself notes in Section 5.2 that alternative pages may be equally relevant, so this premise is load-bearing for the accuracy metric.
domain assumption LLM-generated questions are representative of real student questions for retrieval evaluation.
Section 3.2 generated all questions with gpt-4o-mini; Section 5.7 acknowledges this and calls for future human-authored content. The reported accuracies depend on this assumption.
domain assumption Token-level F1 between generated and reference answers measures answer quality.
Section 3.5 uses shared-word F1. For math answers, variations in notation and phrasing can give low F1 even for correct answers; the paper acknowledges only in passing that it is a similarity metric.

pith-pipeline@v1.3.0-alltime-deepseek · 10464 in / 12615 out tokens · 97779 ms · 2026-08-04T16:03:35.650127+00:00 · methodology

0 comments

read the original abstract

Large language models (LLMs) show promise as educational aids but often lack alignment with specific course materials. We investigate Retrieval-Augmented Generation (RAG) and GraphRAG for page-level question answering on an undergraduate mathematics textbook. Using a curated dataset of 477 question-answer pairs, each tied to a specific textbook page, we compare five embedding-based RAG models, a BM25 baseline, and GraphRAG across two metrics: retrieval accuracy (whether the correct page is retrieved) and answer quality (F1 score). Our results show that embedding-based RAG outperforms GraphRAG for page-level retrieval, with voyage-3-large achieving 99.4% accuracy at top-10 (bootstrap 95% CI for top-1: [.644, .728]). BM25 proves a strong baseline, outperforming several embedding models. Error analysis reveals that 63.3% of top-1 failures retrieve same-chapter content, suggesting pedagogical relevance even in failure cases. GraphRAG retrieves excessive context (~47K tokens vs. ~3.7K for RAG), reducing generation quality. We further replicate key experiments using an open-source local LLM (Qwen3.5-35B-A3B), finding that RAG benefits are proportionally larger for weaker models (+39% vs. +16% relative F1 improvement), an important result for cost-sensitive educational deployments. These findings inform the design of AI tutoring systems that reference specific textbook pages.

Figures

Figures reproduced from arXiv: 2509.16780 by Chuangji Li, Eason Chen, Eric Li, Jionghao Lin, Kenneth R. Koedinger, Zimo Xiao.

discussion (0)

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

When Confidence Takes the Wrong Path: Diagnosing Retrieval-State Lock-In in RAG
cs.CL 2026-06 unverdicted novelty 6.0

Retrieval-state lock-in causes zero-dispersion errors in 42% of KG-RAG and 59% of dense-retrieval failures; a three-object check rule reaches 91.9% pooled precision at 7.7% coverage.
AI Economist Agent: An Agentic Framework for Model-Grounded Economic Analysis with RAG, Knowledge Graphs, and Large Language Models
econ.GN 2026-06 unverdicted novelty 4.0

Proposes an agentic RAG framework with knowledge graphs and LLMs to produce model-grounded economic reports, evaluated on U.S. inflation persistence and commercial real estate stress-test narratives.

Reference graph

Works this paper leans on

30 extracted references · 4 linked inside Pith · cited by 2 Pith papers

[1]

Advances in neural information processing systems33, 1877–1901 (2020)

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems33, 1877–1901 (2020)

1901
[2]

arXiv preprint arXiv:2410.11123 (2024)

Chen, E., Wang, D., Xu, L., Cao, C., Fang, X., Lin, J.: A systematic review on prompt engineering in large language models for k-12 stem education. arXiv preprint arXiv:2410.11123 (2024)

Pith/arXiv arXiv 2024
[3]

Expert systems with applications141, 112948 (2020)

Chen, X., Jia, S., Xiang, Y.: A review: Knowledge reasoning over knowledge graph. Expert systems with applications141, 112948 (2020)

2020
[4]

In: Rogers, A., Boyd-Graber, J., Okazaki, N

Chiesurin,S.,Dimakopoulos,D.,SobrevillaCabezudo,M.A.,Eshghi,A.,Papaioan- nou, I., Rieser, V., Konstas, I.: The dangers of trusting stochastic parrots: Faithful- ness and trust in open-domain conversational question answering. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Findings of the Association for Computational Linguistics: ACL 2023. pp. 947–95...

doi:10.18653/v1/2023.findings-acl.60 2023
[5]

Computers and Education: Artificial Intelligence7, 100299 (2024)

Dai, W., Tsai, Y.S., Lin, J., Aldino, A., Jin, H., Li, T., Gašević, D., Chen, G.: As- sessing the proficiency of large language models in automatic feedback generation: An evaluation study. Computers and Education: Artificial Intelligence7, 100299 (2024). https://doi.org/10.1016/j.caeai.2024.100299

arXiv 2024
[6]

Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Larson, J.: From local to global: A graph rag approach to query-focused summarization (2024)

2024
[7]

https://huggingface.co/spaces/mteb/leaderboard (2025), [Accessed 20-12-2024]

Face, H.: MTEB Leaderboard - a Hugging Face Space by mteb — huggingface.co. https://huggingface.co/spaces/mteb/leaderboard (2025), [Accessed 20-12-2024]

2025
[8]

Feng, T., Liu, S., Ghosal, D.: Courseassist: Pedagogically appropriate ai tutor for computer science education (2024)

2024
[9]

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., Wang, H.: Retrieval-augmented generation for large language models: A survey (2024)

2024
[10]

In: International Conference on Theory and Practice of Digital Libraries

Ghiriti, A., Göderle, W., Kern, R.: Exploring the capabilities of gpt4-vision as ocr engine. In: International Conference on Theory and Practice of Digital Libraries. pp. 3–12. Springer (2024)

2024
[11]

arXiv preprint arXiv:2401.15901 (2024) Title Suppressed Due to Excessive Length 15

Guo, K., Liu, Y., Zhang, X., Wang, C., et al.: Graphrag: Mitigating hallu- cination via graph-augmented retrieval for question answering. arXiv preprint arXiv:2401.15901 (2024) Title Suppressed Due to Excessive Length 15

Pith/arXiv arXiv 2024
[12]

arXiv preprint arXiv:2402.14594 (2024)

Han, Z.F., Lin, J., Gurung, A., Thomas, D.R., Chen, E., Borchers, C., Gupta, S., Koedinger, K.R.: Improving assessment of tutoring practices using retrieval- augmented generation. arXiv preprint arXiv:2402.14594 (2024)

Pith/arXiv arXiv 2024
[13]

Advances in Neural Information Processing Systems37, 132876–132907 (2024)

He, X., Tian, Y., Sun, Y., Chawla, N., Laurent, T., LeCun, Y., Bresson, X., Hooi, B.: G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. Advances in Neural Information Processing Systems37, 132876–132907 (2024)

2024
[14]

In: 25th International Conference on Artificial Intelligence in Education (AIED)

Henkel, S., Robertson, D., Chen, A.: Using retrieval-augmented generation for personalized math tutoring in middle schools. In: 25th International Conference on Artificial Intelligence in Education (AIED). Springer (2024)

2024
[15]

ACM computing surveys55(12), 1–38 (2023)

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y.J., Madotto, A., Fung, P.: Survey of hallucination in natural language generation. ACM computing surveys55(12), 1–38 (2023)

2023
[16]

In: 12th ACM Conference on Learning @ Scale (L@S)

Lang, M., Gürpınar, B.: Leveraging retrieval-augmented generation in an online R programming course chatbot. In: 12th ACM Conference on Learning @ Scale (L@S). ACM (2025)

2025
[17]

Levonian, Z., Li, C., Zhu, W., Gade, A., Henkel, O., Postle, M.E., Xing, W.: Retrieval-augmented generation to improve math question-answering: Trade-offs between groundedness and human preference (2023)

2023
[18]

In: 7th Inter- national Conference on Educational Technology Management

Lin, J., Mai, S., Bu, B., He, M., Wang, X.: Research on the application of stem practical teaching based on rag knowledge graph and large models. In: 7th Inter- national Conference on Educational Technology Management. pp. 520–527 (2024)

2024
[19]

In: Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24) (2024)

Mitra, C., Miroyan, M., Jain, R., Kumud, V., Ranade, G., Norouzi, N.: Retllm-e: Retrieval-prompt strategy for question-answering on student discussion forums. In: Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24) (2024)

2024
[20]

In: European conference on technology enhanced learning

Moore, S., Nguyen, H.A., Bier, N., Domadia, T., Stamper, J.: Assessing the quality of student-generated short answer questions using gpt-3. In: European conference on technology enhanced learning. pp. 243–257. Springer (2022)

2022
[21]

arXiv preprint arXiv:2210.07316 (2022)

Muennighoff, N., Tazi, N., Magne, L., Reimers, N.: Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316 (2022)

Pith/arXiv arXiv 2022
[22]

Newstead, C.: An Infinite Descent into Pure Mathematics (2024), https://infinitedescent.xyz/, last updated on Wednesday 29th May 2024

2024
[23]

In: European conference on technology enhanced learning

Nguyen, H.A., Stec, H., Hou, X., Di, S., McLaren, B.M.: Evaluating chatgpt’s decimal skills and feedback generation in a digital learning game. In: European conference on technology enhanced learning. pp. 278–293. Springer (2023)

2023
[24]

https://platform.openai.com/docs/guides/vision (2024), [Accessed 20-02-2025]

OpenAI: Openai vision model documentation. https://platform.openai.com/docs/guides/vision (2024), [Accessed 20-02-2025]

2024
[25]

OpenAI, e.a.: Gpt-4 technical report (2024)

2024
[26]

In: 42nd ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)

Salminen, J., Park, G., Kim, D., Jung, J.: Cipherbot: A retrieval-augmented teach- ing assistant for business courses. In: 42nd ACM SIGCHI Conference on Human Factors in Computing Systems (CHI). ACM (2024)

2024
[27]

In: International Conference on Artificial Intelligence in Education

Scarlatos,A.,Smith,D.,Woodhead,S.,Lan,A.:Improvingthevalidityofautomat- ically generated feedback via reinforcement learning. In: International Conference on Artificial Intelligence in Education. pp. 280–294. Springer (2024)

2024
[28]

Wang, S., Xu, T., Li, H., Zhang, C., Liang, J., Tang, J., Yu, P.S., Wen, Q.: Large language models for education: A survey and outlook (2024)

2024
[29]

ACM Computing Surveys (2025), forthcoming

Zhang, Q., Li, F., Zhao, W., et al.: A survey of retrieval-augmented generation with knowledge graphs. ACM Computing Surveys (2025), forthcoming

2025
[30]

In: 25th In- ternational Conference on Artificial Intelligence in Education

Zheng, Y., Carter, M., Huang, J., Patel, S.: Resolving knowledge conflicts in retrieval-augmented llms: A case study in k-12 science education. In: 25th In- ternational Conference on Artificial Intelligence in Education. Springer (2024)

2024

[1] [1]

Advances in neural information processing systems33, 1877–1901 (2020)

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Nee- lakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems33, 1877–1901 (2020)

1901

[2] [2]

arXiv preprint arXiv:2410.11123 (2024)

Chen, E., Wang, D., Xu, L., Cao, C., Fang, X., Lin, J.: A systematic review on prompt engineering in large language models for k-12 stem education. arXiv preprint arXiv:2410.11123 (2024)

Pith/arXiv arXiv 2024

[3] [3]

Expert systems with applications141, 112948 (2020)

Chen, X., Jia, S., Xiang, Y.: A review: Knowledge reasoning over knowledge graph. Expert systems with applications141, 112948 (2020)

2020

[4] [4]

In: Rogers, A., Boyd-Graber, J., Okazaki, N

Chiesurin,S.,Dimakopoulos,D.,SobrevillaCabezudo,M.A.,Eshghi,A.,Papaioan- nou, I., Rieser, V., Konstas, I.: The dangers of trusting stochastic parrots: Faithful- ness and trust in open-domain conversational question answering. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Findings of the Association for Computational Linguistics: ACL 2023. pp. 947–95...

doi:10.18653/v1/2023.findings-acl.60 2023

[5] [5]

Computers and Education: Artificial Intelligence7, 100299 (2024)

Dai, W., Tsai, Y.S., Lin, J., Aldino, A., Jin, H., Li, T., Gašević, D., Chen, G.: As- sessing the proficiency of large language models in automatic feedback generation: An evaluation study. Computers and Education: Artificial Intelligence7, 100299 (2024). https://doi.org/10.1016/j.caeai.2024.100299

arXiv 2024

[6] [6]

Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Larson, J.: From local to global: A graph rag approach to query-focused summarization (2024)

2024

[7] [7]

https://huggingface.co/spaces/mteb/leaderboard (2025), [Accessed 20-12-2024]

Face, H.: MTEB Leaderboard - a Hugging Face Space by mteb — huggingface.co. https://huggingface.co/spaces/mteb/leaderboard (2025), [Accessed 20-12-2024]

2025

[8] [8]

Feng, T., Liu, S., Ghosal, D.: Courseassist: Pedagogically appropriate ai tutor for computer science education (2024)

2024

[9] [9]

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., Wang, H.: Retrieval-augmented generation for large language models: A survey (2024)

2024

[10] [10]

In: International Conference on Theory and Practice of Digital Libraries

Ghiriti, A., Göderle, W., Kern, R.: Exploring the capabilities of gpt4-vision as ocr engine. In: International Conference on Theory and Practice of Digital Libraries. pp. 3–12. Springer (2024)

2024

[11] [11]

arXiv preprint arXiv:2401.15901 (2024) Title Suppressed Due to Excessive Length 15

Guo, K., Liu, Y., Zhang, X., Wang, C., et al.: Graphrag: Mitigating hallu- cination via graph-augmented retrieval for question answering. arXiv preprint arXiv:2401.15901 (2024) Title Suppressed Due to Excessive Length 15

Pith/arXiv arXiv 2024

[12] [12]

arXiv preprint arXiv:2402.14594 (2024)

Han, Z.F., Lin, J., Gurung, A., Thomas, D.R., Chen, E., Borchers, C., Gupta, S., Koedinger, K.R.: Improving assessment of tutoring practices using retrieval- augmented generation. arXiv preprint arXiv:2402.14594 (2024)

Pith/arXiv arXiv 2024

[13] [13]

Advances in Neural Information Processing Systems37, 132876–132907 (2024)

He, X., Tian, Y., Sun, Y., Chawla, N., Laurent, T., LeCun, Y., Bresson, X., Hooi, B.: G-retriever: Retrieval-augmented generation for textual graph understanding and question answering. Advances in Neural Information Processing Systems37, 132876–132907 (2024)

2024

[14] [14]

In: 25th International Conference on Artificial Intelligence in Education (AIED)

Henkel, S., Robertson, D., Chen, A.: Using retrieval-augmented generation for personalized math tutoring in middle schools. In: 25th International Conference on Artificial Intelligence in Education (AIED). Springer (2024)

2024

[15] [15]

ACM computing surveys55(12), 1–38 (2023)

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y.J., Madotto, A., Fung, P.: Survey of hallucination in natural language generation. ACM computing surveys55(12), 1–38 (2023)

2023

[16] [16]

In: 12th ACM Conference on Learning @ Scale (L@S)

Lang, M., Gürpınar, B.: Leveraging retrieval-augmented generation in an online R programming course chatbot. In: 12th ACM Conference on Learning @ Scale (L@S). ACM (2025)

2025

[17] [17]

Levonian, Z., Li, C., Zhu, W., Gade, A., Henkel, O., Postle, M.E., Xing, W.: Retrieval-augmented generation to improve math question-answering: Trade-offs between groundedness and human preference (2023)

2023

[18] [18]

In: 7th Inter- national Conference on Educational Technology Management

Lin, J., Mai, S., Bu, B., He, M., Wang, X.: Research on the application of stem practical teaching based on rag knowledge graph and large models. In: 7th Inter- national Conference on Educational Technology Management. pp. 520–527 (2024)

2024

[19] [19]

In: Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24) (2024)

Mitra, C., Miroyan, M., Jain, R., Kumud, V., Ranade, G., Norouzi, N.: Retllm-e: Retrieval-prompt strategy for question-answering on student discussion forums. In: Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24) (2024)

2024

[20] [20]

In: European conference on technology enhanced learning

Moore, S., Nguyen, H.A., Bier, N., Domadia, T., Stamper, J.: Assessing the quality of student-generated short answer questions using gpt-3. In: European conference on technology enhanced learning. pp. 243–257. Springer (2022)

2022

[21] [21]

arXiv preprint arXiv:2210.07316 (2022)

Muennighoff, N., Tazi, N., Magne, L., Reimers, N.: Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316 (2022)

Pith/arXiv arXiv 2022

[22] [22]

Newstead, C.: An Infinite Descent into Pure Mathematics (2024), https://infinitedescent.xyz/, last updated on Wednesday 29th May 2024

2024

[23] [23]

In: European conference on technology enhanced learning

Nguyen, H.A., Stec, H., Hou, X., Di, S., McLaren, B.M.: Evaluating chatgpt’s decimal skills and feedback generation in a digital learning game. In: European conference on technology enhanced learning. pp. 278–293. Springer (2023)

2023

[24] [24]

https://platform.openai.com/docs/guides/vision (2024), [Accessed 20-02-2025]

OpenAI: Openai vision model documentation. https://platform.openai.com/docs/guides/vision (2024), [Accessed 20-02-2025]

2024

[25] [25]

OpenAI, e.a.: Gpt-4 technical report (2024)

2024

[26] [26]

In: 42nd ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)

Salminen, J., Park, G., Kim, D., Jung, J.: Cipherbot: A retrieval-augmented teach- ing assistant for business courses. In: 42nd ACM SIGCHI Conference on Human Factors in Computing Systems (CHI). ACM (2024)

2024

[27] [27]

In: International Conference on Artificial Intelligence in Education

Scarlatos,A.,Smith,D.,Woodhead,S.,Lan,A.:Improvingthevalidityofautomat- ically generated feedback via reinforcement learning. In: International Conference on Artificial Intelligence in Education. pp. 280–294. Springer (2024)

2024

[28] [28]

Wang, S., Xu, T., Li, H., Zhang, C., Liang, J., Tang, J., Yu, P.S., Wen, Q.: Large language models for education: A survey and outlook (2024)

2024

[29] [29]

ACM Computing Surveys (2025), forthcoming

Zhang, Q., Li, F., Zhao, W., et al.: A survey of retrieval-augmented generation with knowledge graphs. ACM Computing Surveys (2025), forthcoming

2025

[30] [30]

In: 25th In- ternational Conference on Artificial Intelligence in Education

Zheng, Y., Carter, M., Huang, J., Patel, S.: Resolving knowledge conflicts in retrieval-augmented llms: A case study in k-12 science education. In: 25th In- ternational Conference on Artificial Intelligence in Education. Springer (2024)

2024