Pith. sign in

REVIEW 5 major objections 10 minor 60 references

Personalizing Education through an Adaptive LMS with Integrated LLMs

T0 review · 5 major / 10 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A benchmark of ten language models finds that small self-hosted models perform about as well as large proprietary API models across most educational tasks, making low-cost, private adaptive tutoring feasible.

desk verdict Useful benchmark with a clearly unsupported hardware-utilization claim and an arithmetic slip in the data count; the problem-solving half is plausible and worth a careful revision. read the letter →

arxiv 2502.08655 v1 pith:CLRXY45S submitted 2025-01-24 cs.AI

Kyle Spriggs , Meng Cheng Lau , Kalpdrum Passi This is my paper · ORCID
classification cs.AI
keywords adaptivelearningmanagementsystemlargelanguagemodelsself-hostedLLMsretrievalaugmentedgenerationpersonalizedstandardizedtestbenchmarkingLLMresourceutilizationhallucinationmitigation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that an adaptive learning management system (ALMS) can deliver personalized education by combining a small set of large language models rather than relying on one general-purpose chatbot. To support that, it benchmarks ten LLMs--several small self-hosted models and larger proprietary API models--on 3,780 responses across mathematics, reading, writing, reasoning, and coding, using questions drawn from standardized tests. The paper's central conclusion is that self-hosted 7-billion-parameter models perform about as well as proprietary API models on most tasks while using comparable hardware resources, so a system can delegate each task to the cheapest suitable model without sacrificing quality. If right, this makes privacy-preserving, low-cost adaptive tutoring feasible for schools and institutions with limited instructor availability.

What carries the argument

The load-bearing mechanism is task-specialized model delegation inside an expert-system-style pipeline: instead of one general LLM, the ALMS routes each question or learning task to the LLM best suited to it. The paper's evidence for this mechanism is a stratified benchmark battery of ten models across five categories and three difficulty levels, with grading automated for multiple-choice items, rubric-based for essays, and unit-test-based for code. Supporting machinery includes retrieval-augmented generation with vector embedding, which lets the system answer from a curated knowledge base rather than from the model's training memory, and resource-utilization tracking that compares CPU, memory, and execution time between self-hosted and API models. The benchmark is what carries the claim that delegation plus RAG can match proprietary API quality at lower cost and higher privacy.

What would settle it

Administer the same 150-question battery to a matched group of human students under the identical text-only, no-diagram, forced-choice conditions and compare their scores against the published norms the paper relies on; if the human sample deviates substantially from those norms, the paper's relative rankings of LLM strengths and weaknesses lose their anchor. A second check is to rerun the ten models on a fresh set of never-published standardized questions and see whether the self-hosted/proprietary parity and the sensitivity-to-wording effects persist.

Watch

Extended reading notes

Core claim

According to the paper, when ten LLMs were tested over three rounds on questions from EQAO, ACT, LSAT, OSSLT, and custom coding problems, the models collectively came out weak in mathematics, strong in reading and writing, strong in CS1-level coding, and moderate in reasoning relative to published human norms. The key observation was not just the ranking but the parity: small self-hosted models such as a 7-billion-parameter Mistral and the 2.3-billion-parameter Phi-2 scored close to, and sometimes above, proprietary models with far larger parameter counts, and resource-utilization measurements showed no systematic hardware penalty for self-hosting. The paper therefore concludes that an ALMS can rely on self-hosted models for most subtasks, reserving APIs only where needed, while using retrieval-augmented generation to curb hallucination and keep answers grounded in curated sources.

Load-bearing premise

The comparison of LLM performance to human performance rests on the assumption that published norm statistics for tests like EQAO, ACT, and LSAT are directly comparable to LLM scores obtained under a different protocol, with text-only input, no diagrams, single-letter output constraints, and volunteer rubric grading.

Editorial extensions

If this is right

  • An adaptive LMS can be built around a small set of self-hosted models, with proprietary APIs reserved for specific tasks, preserving data privacy and cutting operating cost.
  • Mathematics is the category where LLM tutoring needs the most scaffolding, since all tested models, including GPT-4, performed poorly relative to human norms.
  • Retrieval-augmented generation from a curated knowledge base is a practical guard against hallucination, since RAG tests produced 'not found' answers instead of confident fabrications.
  • Wording sensitivity is a systematic failure mode: models answered consistently but were strongly influenced by phrasing, so prompt engineering should be part of any ALMS.
  • Because self-hosted 7B models matched API models on hardware metrics, institutions with mid-range hardware can realistically host their own tutoring stack.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's benchmark, the observed self-hosted/proprietary parity suggests the per-student cost of AI tutoring could drop sharply, since self-hosted models avoid per-token API fees.
  • The near-identical wording across different models on the easy writing prompts points to shared fine-tuning data or model distillation; a similarity analysis over response corpora could test this directly.
  • A stronger test of the paper's design would deploy the ALMS in a real classroom and measure learning gains against a standard LMS, since the present study measures model aptitude rather than student outcomes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 10 minor

Summary. The paper describes the design and development of an adaptive learning management system (ALMS) called Komodo, built by integrating an expert-system-style test-bank search with multiple large language models (LLMs). The system was developed in three phases: a Django/React expert system with OCR and question matching, an LLM question-answering interface with retrieval-augmented generation, and a benchmark study comparing ten LLMs (including proprietary GPT-3.5/GPT-4 and self-hosted 7B models plus Phi-2) on mathematics, reading, writing, reasoning, and coding questions drawn from standardized tests. The authors conclude that self-hosted LLMs are comparable to proprietary API models in both problem-solving competency and hardware utilization, and that LLMs are relatively strong in reading, writing, and coding, moderate in reasoning, and weak in mathematics.

Significance. The paper's practical claim that small self-hosted models can substitute for expensive proprietary APIs in an adaptive LMS is potentially valuable for privacy-conscious, low-cost deployments. The work contributes a concrete system architecture and a benchmark that uses real standardized-test items, and it has several strengths: the benchmark is empirical with no fitted parameters, the essay grading was blinded with model names omitted, and the authors explicitly discuss confounding variables such as responses without identifiers. However, the significance of the empirical conclusions is currently limited by methodological gaps, as detailed in the major comments: the reported dataset size is inconsistent with the stated test design, the human-performance comparisons rely on published norms rather than a matched control group, the hardware-utilization claims conflate local client overhead with model inference, and the mathematics difficulty paradox is explained with an internally contradictory guess-probability argument.

major comments (5)
  1. [Sections V-A, V-H, VI-C] The reported total of 3,780 data points is inconsistent with the question counts in Table II and the statement in Section V-A that each model was tested on the entire battery three times over. Summing the question-round products for all categories (Math: 90, Reading: 90, Writing: 61, Reasoning: 90, Coding: 15, all multiplied by 10 LLMs) gives 3,460 data points, not 3,780. The dataset size should be corrected in all three locations and the void-response rate in Section VI-C recomputed.
  2. [Sections VI-A6 and VII] The comparison to human performance is based on published norm statistics from EQAO, ACT, OSSLT, and LSAT rather than on a human control group that sat the identical test battery under the same conditions (text-only input, no diagrams, single-letter response format, and volunteer-graded essays). Because the administration protocol differs substantially, the qualitative labels 'weak in mathematics,' 'strong in reading,' and similar claims relative to humans are not established. The authors should either add a matched human control condition or explicitly limit these conclusions to 'performance relative to published norms collected under different administration conditions.'
  3. [Sections V-G, VI-B, VII] The hardware utilization metrics (CPU%, memory%, system/user time, execution time) are measured on the local client and its process tree. For API-based models (GPT-3.5, GPT-3.5 Instruct, GPT-4), the actual model inference runs on remote servers and is not included in these counters; the measurements capture the HTTP client, JSON handling, and local polling. Consequently, the conclusion in Section VII that self-hosted LLMs are 'very capable ... in terms of hardware utilization' relative to proprietary API models is not supported. The authors should distinguish 'local client resource usage' from 'model inference resource usage' and restrict their hardware claims accordingly.
  4. [Section VI-A1] The explanation of the mathematics difficulty paradox is internally contradictory. The authors state that hard questions have five multiple-choice options whereas easy and medium questions have four, and they claim this leads to a 'higher number of lucky guesses' for the hard set. In fact, a larger number of options reduces the probability of a correct random guess, so this mechanism cannot explain the observed inverse relationship between difficulty and score. The passage should be corrected or the paradox left explicitly unresolved.
  5. [Sections VI-A and VI-B] Model rankings and comparative statements are presented without any measure of variability or statistical testing. For example, Section VI-A5 states that Falcon 7B 'scored the highest, but only by a single point' on the basis of five questions per difficulty, and Section VI-A3 reports that GPT-4 scored 33 versus Phi-2's 30 on a single essay per model. Because the number of items per cell is small (often five or ten, and one for the hard writing test), these differences may not be reliable. The authors should report standard errors, confidence intervals, or significance tests, or temper the comparative conclusions.
minor comments (10)
  1. [Section IV-B] The phrase 'Informal experimentation was performed on the self-hosted and proprietary models' capabilities in system prompt modification' is awkward; consider 'with' instead of 'in' and rephrasing to make the object of experimentation clear.
  2. [Table III] The entries 'GPT3 5' and 'GPT3 5 I' are unclear; they likely denote GPT-3.5 and GPT-3.5 Instruct but should be written out. The CodeLlama 7B row is missing an entry in the Parameters column, and the 'HDD Space' column for proprietary models uses '-' without explanation.
  3. [Section V-A] The sentence 'Each model was tested on the entire battery three times over' conflicts with Table II, which shows one round for the hard writing question and the coding questions; clarify that the 'three times' applies only to the 10-question sets.
  4. [Section VI-A3] The phrase 'anti-perfect scores' is nonstandard; it presumably means that every question was answered incorrectly, but the terminology should be defined or replaced.
  5. [Section VI-B] The sentence 'the range between the highest and lowest-performing models was so small that this was not a significant concern' uses 'significant' informally; since no statistical test is performed, avoid the statistical term or provide a quantitative basis for the claim.
  6. [Section VI-C, Table V] The header 'Total V oid Responses' contains a stray space, and the column label 'Explanation Only' is unclear; it appears to mean 'provided an explanation but no identifier,' which should be stated explicitly.
  7. [Section VII] The phrase 'hardware intensive' should be hyphenated as 'hardware-intensive' for consistency with standard usage.
  8. [Section V-D] In Section VI-A3, the claim 'All models had perfect scores on the easy test' is ambiguous because the easy category was not a single test; clarify that it refers to the easy writing questions.
  9. [Section V-G] The description of cgroup usage says that getrusage() 'accesses cgroup usage data,' but getrusage() returns process resource usage statistics and does not directly access cgroups; the explanation should be clarified.
  10. [Section II-C, Table I] Table I is titled 'Requirements by Parameter Count' but only lists memory demand; since the text discusses hardware requirements more broadly, either rename the table or add other resources such as GPU/CPU considerations.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark results are empirical, externally sourced, and independent of any fitted parameters or self-referential derivations.

full rationale

The paper reports direct measurements of LLM performance and resource usage rather than deriving predictions from fitted inputs. No parameters are fit to a subset of the data and then renamed as predictions; no self-citations appear in the reference list; and no uniqueness theorem, ansatz, or definition is imported from the authors' prior work. The human-comparison claim in Section VI-A6 relies on published norm statistics from EQAO, ACT, and LSAT tests, which is an external comparability assumption rather than a circular reduction: the LLM scores are measured independently and then compared with pre-existing human statistics. The Phase II RAG experiments are described informally and are not presented as predictions, so they carry no circularity burden. The skeptic concern about Section VII's hardware-utilization claim—that API-model resource metrics reflect the local client process while self-hosted metrics include local Ollama inference—is a measurement-validity issue, not a circularity pattern: the conclusion does not reduce to its inputs by construction. The paper's central claims are therefore self-contained empirical findings, and no load-bearing step is equivalent to its own inputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fitted and no new entities are postulated. The central claims rest on validity assumptions about tests, human norms, and grading rather than on mathematical derivations.

assumptions (3)
  • domain assumption Standardized test performance is a valid proxy for the educational tutoring capability of an LLM in an ALMS.
    The paper draws system-design conclusions from test scores without connecting scores to learning outcomes or engagement.
  • domain assumption Published human norm statistics are directly comparable to LLM scores obtained under a different administration protocol.
    Section VI-A6 compares LLMs to 'statistics of human subjects who have taken similar tests' without a matched human control group.
  • domain assumption Manual and regex-based grading produces consistent, unbiased scores across models and questions.
    The paper relies on manual grading for coding and essays; no inter-rater reliability or grading consistency check is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Personalizing Education through an Adaptive LMS with Integrated LLMs." pith.science (2026). https://pith.science/paper/CLRXY45S

@misc{pith2026250208655,
  author       = {Pith},
  title        = {Pith review of: Personalizing Education through an Adaptive LMS with Integrated LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CLRXY45S}},
  note         = {Machine review of arXiv:2502.08655}
}
read the original abstract

The widespread adoption of large language models (LLMs) marks a transformative era in technology, especially within the educational sector. This paper explores the integration of LLMs within learning management systems (LMSs) to develop an adaptive learning management system (ALMS) personalized for individual learners across various educational stages. Traditional LMSs, while facilitating the distribution of educational materials, fall short in addressing the nuanced needs of diverse student populations, particularly in settings with limited instructor availability. Our proposed system leverages the flexibility of AI to provide a customizable learning environment that adjusts to each user's evolving needs. By integrating a suite of general-purpose and domain-specific LLMs, this system aims to minimize common issues such as factual inaccuracies and outdated information, characteristic of general LLMs like OpenAI's ChatGPT. This paper details the development of an ALMS that not only addresses privacy concerns and the limitations of existing educational tools but also enhances the learning experience by maintaining engagement through personalized educational content.

Figures

Figures reproduced from arXiv: 2502.08655 by the authors.

Figure 2
Figure 2. Backend DFD [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 1
Figure 1. Screen capture of user interface. A: Query input, B: ContentPanel-1 [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 4
Figure 4. Mean score across all LLMs [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Total correct answers across all rounds. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Mean score across all LLMs [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Total correct answers across all rounds. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Mean score across all LLMs. 4) Reasoning Test Results: Scores in this category were somewhat inconsistent, with a high degree of uniformity per each question, but also high variability in success rate throughout each difficulty. Given the challenging nature of the hard…
Figure 9
Figure 9. Figure 9: Mean score across all LLMs [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Scores per each grading metric. were low as shown in [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 13
Figure 13. Figure 13: Total correct answers across all rounds. [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 16
Figure 16. Figure 16: Mean resource usage across all tests. Since this is such a small portion of the overall sample size, it should not have any meaningful effect on the study’s results. TABLE V RESPONSES WITHOUT IDENTIFIERS “No valid option” Explanation Only Total Void Responses 12 20 35…
Figure 17
Figure 17. Figure 17: Mean CPU time across all tests [PITH_FULL_IMAGE:figures/full_fig_p013_17.png]
Figure 18
Figure 18. Figure 18: Mean execution time across all tests. statistics of human subjects who have attempted similar tests, modern LLMs demonstrate strong reading, writing, and CS1- level coding abilities, and moderate reasoning performance, but tend to struggle in mathematics. Considering …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 45 canonical work pages

  1. [1]

    Learning management systems, an overview,

    D. Turnbull, R. Chugh, and J. Luck, “Learning management systems, an overview,” Encyclopedia of education and information technologies , pp. 1052–1058, 2020

    work page 2020

  2. [2]

    Software as a service (saas) cloud computing: An empirical investi- gation on university students’perception

    G. A.-M. Taufiq-Hail, A. R. A. Alanzi, S. A. M. Yusof, and M. Alruwaili, “Software as a service (saas) cloud computing: An empirical investi- gation on university students’perception.” Interdisciplinary Journal of Information, Knowledge & Management , vol. 16, no. 2021, 2021

    work page 2021

  3. [3]

    The impact of number of students per teacher on student achievement,

    N. Koc and B. Celik, “The impact of number of students per teacher on student achievement,” Procedia - Social and Behavioral Sciences, vol. 177, p. 65–70, Apr. 2015. [Online]. Available: http://dx.doi.org/10.1016/j.sbspro.2015.02.335

    work page doi:10.1016/j.sbspro.2015.02.335 2015

  4. [4]

    Do spacing and self-testing predict learning outcomes?

    F. Rodriguez, S. Kataoka, M. Janet Rivas, P. Kadandale, A. Nili, and M. Warschauer, “Do spacing and self-testing predict learning outcomes?” Active Learning in Higher Education , vol. 22, no. 1, p. 77–91, May 2018. [Online]. Available: http://dx.doi.org/10.1177/ 1469787418774185

    work page 2018

  5. [5]

    Woolf, Building Intelligent Interactive Tutors, Student-Centered Strategies for Revolutionizing E-Learning

    B. Woolf, Building Intelligent Interactive Tutors, Student-Centered Strategies for Revolutionizing E-Learning . Elsevier, 01 2008

    work page 2008

  6. [6]

    Play and intrinsic rewards,

    M. Csikszentmihalyi, “Play and intrinsic rewards,” Journal of Humanistic Psychology , vol. 15, no. 3, p. 41–63, Jul. 1975. [Online]. Available: http://dx.doi.org/10.1177/002216787501500306

    work page doi:10.1177/002216787501500306 1975

  7. [7]

    Alienation, agency and authenticity: a synthesis of the literature on student engagement,

    K. Wimpenny and M. Savin-Baden, “Alienation, agency and authenticity: a synthesis of the literature on student engagement,” Teaching in Higher Education , vol. 18, no. 3, p. 311–326, Apr. 2013. [Online]. Available: http://dx.doi.org/10.1080/13562517.2012.725223

    arXiv 2013

  8. [8]

    LLM lies: Hallucinations are not bugs, but features as adversarial examples,

    J.-Y . Yao, K.-P. Ning, Z. Liu, M. Ning, and L. Yuan, “LLM lies: Hallucinations are not bugs, but features as adversarial examples,”

    work page

Show all 60 references
  1. [9]

    A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning,

    S. Steenbergen-Hu and H. Cooper, “A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning,” J. Educ. Psychol. , vol. 106, no. 2, pp. 331–347, May 2014

    2014

  2. [10]

    The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems,

    K. VanLEHN, “The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems,” Educ. Psychol., vol. 46, no. 4, pp. 197–221, Oct. 2011

    2011

  3. [11]

    J. H. McMillan, Classroom assessment . Elsevier, 2023, p. 519–528. [Online]. Available: http://dx.doi.org/10.1016/B978-0-12-818630-5. 10064-8

    2023 doi

  4. [12]

    Ai technologies for education: Recent research & future directions,

    K. Zhang and A. B. Aslan, “Ai technologies for education: Recent research & future directions,” Computers and Education: Artificial Intelligence, vol. 2, no. 100025, p. 100025, 2021

    2021

  5. [13]

    Examining the applications of intelligent tutoring systems in real educational contexts: A systematic literature review from the social experiment perspective,

    H. Wang, A. Tlili, R. Huang, Z. Cai, M. Li, Z. Cheng, D. Yang, M. Li, X. Zhu, and C. Fei, “Examining the applications of intelligent tutoring systems in real educational contexts: A systematic literature review from the social experiment perspective,” Education and Information...

    2023 doi

  6. [14]

    A digital game- based assessment of middle-school and college students’ choices to seek critical feedback and to revise,

    M. Cutumisu, D. B. Chin, and D. L. Schwartz, “A digital game- based assessment of middle-school and college students’ choices to seek critical feedback and to revise,” Br. J. Educ. Technol., vol. 50, no. 6, pp. 2977–3003, Nov. 2019

    2019

  7. [15]

    Systematic review of research on artificial intelligence in K-12 education (2017–2022),

    F. Martin, M. Zhuang, and D. Schaefer, “Systematic review of research on artificial intelligence in K-12 education (2017–2022),” Computers and Education: Artificial Intelligence , vol. 6, no. 100195, p. 100195, Jun. 2024

    2017

  8. [16]

    Identifying opportunities for artificial intelligence in the evolution of training and development practices,

    S. Maity, “Identifying opportunities for artificial intelligence in the evolution of training and development practices,” J. Manag. Dev. , vol. 38, no. 8, pp. 651–663, Sep. 2019

    2019

  9. [17]

    Sciassess: Benchmarking llm proficiency in scientific literature analysis,

    H. Cai, X. Cai, J. Chang, S. Li, L. Yao, C. Wang, Z. Gao, H. Wang, Y . Li, M. Lin, S. Yang, J. Wang, M. Xu, J. Huang, F. Xi, J. Zhuang, Y . Yin, Y . Li, C. Chen, Z. Cheng, Z. Zhao, L. Zhang, and G. Ke, “Sciassess: Benchmarking llm proficiency in scientific literature analysis,...

    2024 arXiv

  10. [18]

    Large language models are state-of- the-art evaluators of translation quality,

    T. Kocmi and C. Federmann, “Large language models are state-of- the-art evaluators of translation quality,” 2023. [Online]. Available: https://arxiv.org/abs/2302.14520

    2023 arXiv

  11. [19]

    Individualized ai tutor based on develop- mental learning networks,

    W.-H. Kim and J.-H. Kim, “Individualized ai tutor based on develop- mental learning networks,” IEEE Access, vol. 8, pp. 1–1, 02 2020

    2020

  12. [20]

    Examining student use of ai in cs1 and cs2,

    E. D. Manley, T. Urness, A. Migunov, and M. A. Reza, “Examining student use of ai in cs1 and cs2,” J. Comput. Sci. Coll. , vol. 39, no. 6, p. 41–51, may 2024

    2024

  13. [21]

    Measuring mathematical problem solving with the math dataset,

    D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt, “Measuring mathematical problem solving with the math dataset,” 2021. [Online]. Available: https://arxiv.org/abs/2103.03874

    2021 arXiv

  14. [22]

    State of what art? a call for multi-prompt llm evaluation,

    M. Mizrahi, G. Kaplan, D. Malkin, R. Dror, D. Shahaf, and G. Stanovsky, “State of what art? a call for multi-prompt llm evaluation,” 2024. [Online]. Available: https://arxiv.org/abs/2401.00595

    2024 arXiv

  15. [23]

    Active retrieval augmented generation,

    Z. Jiang, F. F. Xu, L. Gao, Z. Sun, Q. Liu, J. Dwivedi-Yu, Y . Yang, J. Callan, and G. Neubig, “Active retrieval augmented generation,”

  16. [24]

    How to build an ai tutor that can adapt to any course and provide accurate answers using large language model and retrieval-augmented generation,

    C. Dong, “How to build an ai tutor that can adapt to any course and provide accurate answers using large language model and retrieval-augmented generation,” 2024. [Online]. Available: https: //arxiv.org/abs/2311.17696

    2024 arXiv

  17. [25]

    A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,

    Y . Yao, J. Duan, K. Xu, Y . Cai, Z. Sun, and Y . Zhang, “A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,” High-Confidence Computing , vol. 4, no. 2, p. 100211, 2024. [Online]. Available: https://www.sciencedirect.com/ science/arti...

    2024

  18. [26]

    D. A. Waterman, A guide to expert systems . USA: Addison-Wesley Longman Publishing Co., Inc., 1985

    1985

  19. [27]

    Expert system methodologies and applications—a decade review from 1995 to 2004,

    Shu-Hsien Liao, “Expert system methodologies and applications—a decade review from 1995 to 2004,” Expert Systems with Applications , vol. 28, no. 1, pp. 93–103, 2005. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S0957417404000934

    1995

  20. [28]

    The knowledge-based expert system: A tutorial,

    Hayes-Roth, “The knowledge-based expert system: A tutorial,” Computer, vol. 17, no. 9, p. 11–28, Sep. 1984. [Online]. Available: http://dx.doi.org/10.1109/mc.1984.1659242

    1984

  21. [29]

    The nlp cookbook: Modern recipes for transformer based deep learning architectures,

    S. Singh and A. Mahmood, “The nlp cookbook: Modern recipes for transformer based deep learning architectures,” IEEE Access, vol. 9, pp. 68 675–68 702, 01 2021

    2021

  22. [30]

    Deep learning,

    Y . LeCun, Y . Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, p. 436–444, May 2015. [Online]. Available: http://dx.doi.org/10.1038/nature14539

    2015 doi

  23. [31]

    A survey of transformers,

    T. Lin, Y . Wang, X. Liu, and X. Qiu, “A survey of transformers,” AI Open , vol. 3, pp. 111–132, 2022. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S2666651022000146

    2022

  24. [32]

    The nlp cookbook: Modern recipes for transformer based deep learning architectures,

    S. Singh and A. Mahmood, “The nlp cookbook: Modern recipes for transformer based deep learning architectures,” p. 68675–68702, 2021. [Online]. Available: http://dx.doi.org/10.1109/access.2021.3077350

    2021

  25. [33]

    Talking about large language models,

    M. Shanahan, “Talking about large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2212.03551

    2023 arXiv

  26. [34]

    A survey of large language models,

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Dong, Y . Du, C. Yang, Y . Chen, Z. Chen, J. Jiang, R. Ren, Y . Li, X. Tang, Z. Liu, P. Liu, J.-Y . Nie, and J.-R. Wen, “A survey of large language models,” 2023. [Online]. Available: https:...

    2023 arXiv

  27. [35]

    Interactive analysis of word vector embeddings,

    F. Heimerl and M. Gleicher, “Interactive analysis of word vector embeddings,” Computer Graphics Forum , vol. 37, no. 3, p. 253–265, Jun. 2018. [Online]. Available: http://dx.doi.org/10.1111/cgf.13417

    2018 doi

  28. [36]

    A survey on retrieval-augmented text generation,

    H. Li, Y . Su, D. Cai, Y . Wang, and L. Liu, “A survey on retrieval-augmented text generation,” 2022. [Online]. Available: https://arxiv.org/abs/2202.01110

    2022 arXiv

  29. [37]

    Llm lies: Hallucinations are not bugs, but features as adversarial examples,

    J.-Y . Yao, K.-P. Ning, Z.-H. Liu, M.-N. Ning, and L. Yuan, “Llm lies: Hallucinations are not bugs, but features as adversarial examples,”

  30. [38]

    Hallucination is inevitable: An innate limitation of large language models,

    Z. Xu, S. Jain, and M. Kankanhalli, “Hallucination is inevitable: An innate limitation of large language models,” 2024. [Online]. Available: https://arxiv.org/abs/2401.11817

    2024 arXiv

  31. [39]

    What is devops?: A systematic mapping study on definitions and practices,

    R. Jabbari, N. bin Ali, K. Petersen, and B. Tanveer, “What is devops?: A systematic mapping study on definitions and practices,” in Proceedings of the Scientific Workshop Proceedings of XP2016 , ser. XP ’16 Workshops. ACM, May 2016. [Online]. Available: http://dx.doi.org/10.11...

    2016

  32. [40]

    Available: https://arxiv.org/abs/2310.01469

    [Online]. Available: https://arxiv.org/abs/2310.01469

    arXiv

  33. [41]

    Grade 3 assessment of reading, writing and mathematics, primary division - student booklet - mathematics,

    E. Quality and A. Office, “Grade 3 assessment of reading, writing and mathematics, primary division - student booklet - mathematics,” https://www.google.com/url?sa=t&source= web&rct=j&opi=89978449&url=https://schoolweb.tdsb.on.ca/ Portals/thorncliffepark/docs/3e Math 0612 web....

    2012

  34. [42]

    Resource: Released questions - assessment of reading, writing and mathematics - junior division,

    ——, “Resource: Released questions - assessment of reading, writing and mathematics - junior division,” https://www.eqao.com/ math-resource-released-questions-g6-2023/, 2023, [Accessed 24-03- 2024]

    2023

  35. [43]

    Math - sample test questions,

    I. ACT, “Math - sample test questions,” https://www.act. org/content/act/en/products-and-services/the-act/test-preparation/ math-practice-test-questions.html?page=0&chapter=0, 2024, [Accessed 26-03-2024]

    2024

  36. [44]

    Released selections and test questions - ontario secondary school literacy test (osslt),

    E. Quality and A. Office, “Released selections and test questions - ontario secondary school literacy test (osslt),” https://schoolweb.tdsb. on.ca/Portals/northviewheights/docs/osslt-question-bklt-2015.pdf, 2015, [Accessed 24-03-2024]

    2015

  37. [45]

    Reading comprehension sample questions,

    I. Law School Admission Council, “Reading comprehension sample questions,” https://www.lsac.org/lsat/taking-lsat/test-format/ reading-comprehension/reading-comprehension-sample-questions, 2024, [Accessed 24-03-2024]

    2024

  38. [46]

    Reading - sample test questions,

    I. ACT, “Reading - sample test questions,” https://www.act. org/content/act/en/products-and-services/the-act/test-preparation/ reading-practice-test-questions.html?page=0&chapter=0, 2024, [Accessed 27-03-2024]

    2024

  39. [47]

    Released selections and test questions - ontario sec- ondary school literacy test (osslt),

    ——, “Released selections and test questions - ontario sec- ondary school literacy test (osslt),” https://sites.google.com/view/ wlmacossltprep/practice-tests#h.9877qckx670w, 2019, [Accessed 24- 03-2024]

    2019

  40. [48]

    Ontario secondary school practice literacy test,

    ——, “Ontario secondary school practice literacy test,” https://drive. google.com/file/d/1t22MraoSJHRA36wVVTYPCWxo755gb5OY/view, 2023, [Accessed 24-03-2024]

    2023

  41. [49]

    Ontario secondary school literacy test 2015 - student answer key: Released questions,

    E. Quality and A. Office, “Ontario secondary school literacy test 2015 - student answer key: Released questions,” https://schoolweb.tdsb.on.ca/Portals/northviewheights/docs/ osslt-multiple-choice-answer-key-2015.pdf, 2015, [Accessed 24- 03-2024]

    2015

  42. [50]

    Writing test scores,

    ——, “Writing test scores,” https://web.archive.org/web/ 20160217162357/http://www.actstudent.org/writing/writing-scores.html, 2016, [Accessed 24-03-2024]

    2016

  43. [51]

    The act profile report – national, graduating class 2023,

    ——, “The act profile report – national, graduating class 2023,” https://www.act.org/content/dam/act/unsecured/documents/ 2023-National-ACT-Profile-Report.pdf, 2023, [Accessed 25-03-2024]

    2023

  44. [52]

    Writing sample essays,

    I. ACT, “Writing sample essays,” https://www.act.org/content/act/en/ products-and-services/the-act/test-preparation/writing-sample-essays. html?page=0&chapter=0, 2024, [Accessed 23-03-2024]

    2024

  45. [53]

    Logical reasoning: Verbal classification,

    ——, “Logical reasoning: Verbal classification,” https://www.indiabix. com/logical-reasoning/verbal-classification/, 2024, [Accessed 24-03- 2024]

    2024

  46. [54]

    Logical reasoning sample questions,

    I. Law School Admission Council, “Logical reasoning sample questions,” https://www.lsac.org/lsat/taking-lsat/test-format/ logical-reasoning/logical-reasoning-sample-questions, 2024, [Accessed 24-03-2024]

    2024

  47. [55]

    Logical reasoning: Number series,

    I. Technologies, “Logical reasoning: Number series,” https://www. indiabix.com/logical-reasoning/number-series/, 2024, [Accessed 24-03- 2024]

    2024

  48. [56]

    Searching algorithms,

    S. E. P. Limited, “Searching algorithms,” https://www.geeksforgeeks.org/ searching-algorithms/, 2024, accessed 24-03-2024

    2024

  49. [57]

    Python examples,

    R. Data, “Python examples,” https://www.w3schools.com/python/ python examples.asp, 2024, [Accessed 24-03-2024]

    2024

  50. [58]

    Cormen, C

    T. Cormen, C. Leiserson, R. Rivest, and C. Stein, Introduction To Algorithms, ser. Mit Electrical Engineering and Computer Science. MIT Press, 2001

    2001

  51. [2023]

    Available: https://arxiv.org/abs/2305.06983

    [Online]. Available: https://arxiv.org/abs/2305.06983

    arXiv

  52. [2024]

    Available: https://openreview.net/forum?id=Rh1aThKliu

    [Online]. Available: https://openreview.net/forum?id=Rh1aThKliu

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.