Pith. sign in

REVIEW 4 major objections 7 minor 39 references

Fine-tuning a large language model on ability-labeled responses can reconstruct item characteristic curves well enough to predict item difficulty and discrimination from text alone.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 12:29 UTC pith:Q4ZPZD2P

load-bearing objection Useful incremental extension of SMART to MCQs, but the headline ELA transfer correlations rest on an unspecified cross-validation split. the 4 major comments →

arxiv 2601.02580 v2 pith:Q4ZPZD2P submitted 2026-01-05 cs.CL cs.AI

Reconstructing Item Characteristic Curves using Fine-Tuned Large Language Models

Christopher Ormerod This is my paper
classification cs.CL cs.AI
keywords Item Response TheoryItem Characteristic CurveItem difficulty predictionLarge Language ModelsLoRAStudent response simulationNominal Response ModelQwen-3
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the token probabilities a large language model assigns to multiple-choice options, after fine-tuning on student responses organized by ability level, can be read as synthetic Item Characteristic Curves. From those curves the author recovers Item Response Theory parameters (difficulty and discrimination) for items the model has never seen, using only the item text. On a Grade 6 English Language Arts item set the method reaches a 0.503 Pearson correlation with calibrated 1PL difficulty and 0.446 with 2PL discrimination, outperforming text-feature and encoder baselines; on the BEA 2024 shared task it beats the reported comparison models with a 0.381 correlation. The author frames this as an implicit, simulation-based route to item pretesting that is especially strong for discrimination, which text-only methods usually miss. The caveat is that both evaluation sets are small and the BEA result relies on assuming incorrect choices are guessed uniformly at random.

Core claim

The central claim is that the conditional distribution over answer options, not the generated answer itself, carries the psychometric signal. By prompting the model with an ability descriptor (e.g., 'Proficient' bounded by θ∈(0.6,0.9)) and a question, then reading the softmax over the option tokens, the author obtains a discrete ICC per item. A second fine-tuning stage—'distribution correction'—matches those token probabilities to the Nominal Response Model (softmax of linear functions of θ), and the resulting NRM parameters map back to IRT difficulty and discrimination. This turns the LLM into a simulated student cohort that can be queried at any ability level.

What carries the argument

The load-bearing object is the conditional token distribution over answer options, treated as a discrete ICC, together with the two-stage fine-tuning procedure: supervised fine-tuning teaches the model to emit a plausible answer; then distribution correction minimizes the squared difference between the softmax of the option logits and the NRM probabilities softmax(a_i(θ-b_i)). The ability scale is discretized into 20 labeled intervals with bound values c_k; Eq. (4) supplies the uniform-random-distractor correspondence between NRM and 1PL parameters used for the BEA dataset, where no response vectors are available.

Load-bearing premise

The whole procedure hinges on the assumption that the token probabilities a fine-tuned LLM assigns to answer options on unseen items mirror the response probabilities of real students at each ability level; for the BEA dataset it further assumes that students who don't know the answer pick uniformly among the wrong options.

What would settle it

Run the fine-tuned model on a set of items with known field-test responses and ability estimates, reconstruct the ICC for each item, and compare the model's distractor probabilities to the observed distractor frequencies; if a clearly attractive distractor accounts for far more incorrect responses than 1/(n-1), the uniform assumption fails and predicted 1PL difficulties will be systematically off.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Item parameters for new multiple-choice questions can be estimated from item text alone, sidestepping the field-testing stage that typically supplies IRT calibration data.
  • The method's stronger showing on discrimination (a) than on difficulty (b) suggests text-driven simulation can recover the slope of the ICC, a quantity most text-based difficulty models omit.
  • The observed scaling trend across model sizes indicates that as base-model capabilities improve, the fidelity of simulated student responses is likely to improve, increasing the practical ceiling of the approach.
  • For assessments that use the BEA-style 1PL calibration, the method can produce a full NRM response curve from a single b parameter (via the uniform-guessing assumption), enabling the same LLM pipeline without any response data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The uniform-random-distractor assumption can be stress-tested directly: on items with real response data, compare the LLM's reconstructed probabilities for each distractor against the observed choice rates; any systematic preference for a particular wrong answer would make the BEA-style estimates biased in a predictable direction.
  • Because the pipeline reads token probabilities, it could be extended to constructed-response items (where the relevant token is the generated response) and to response-time simulation, provided a mapping from ability to timing is defined.
  • A practical extension is to use the LLM's generated rationales (chain-of-thought) conditioned on ability level; if the rationales are consistent with the simulated errors, the method would produce diagnostic explanations of why items are hard, not only difficulty values.
  • The transfer-from-text caveat implies the method should be re-validated on out-of-distribution items (different grade, subject, or format); without such a check, claims about 'unseen items' rest on the assumption that the fine-tuned model generalizes beyond its training distribution.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes a method to predict IRT item parameters by fine-tuning Qwen-3 LLMs with LoRA on prompts conditioned on discrete ability-level descriptors, then reading out the token probabilities assigned to the MCQ options. The method is evaluated on two datasets: a proprietary ELA dataset with 275 items and 754,000 real student responses, and the BEA 2024 Shared Task dataset which provides only 1PL difficulty labels. The paper reports Pearson correlations up to 0.503 for 1PL difficulty (Qwen-14B) and 0.381 for BEA test-set difficulty, and claims that after LoRA fine-tuning and a 'distribution correction' stage, the LLM token probabilities reconstruct ICCs for unseen items.

Significance. If the transfer-to-unseen-items claim holds, this is a meaningful contribution to item difficulty modeling: it offers a parameter-efficient way to generate synthetic response curves without field testing, and it explicitly models discrimination, a quantity that many text-based baselines omit. The use of open-weight models, the explicit reporting of hyperparameters, and the comparison against ModernBERT and feature-based baselines are strengths. The BEA held-out test set provides a clean test-bed, although its synthetic training targets weaken the interpretation. The overall significance, however, rests entirely on whether the ELA five-fold evaluation is item-level and free of development-set leakage; the current manuscript does not establish this, and the reported correlations could be inflated by either leakage or sampling noise.

major comments (4)
  1. [§3.3.1, Table 4] The five-fold cross-validation is underspecified. State explicitly whether the folds split items or student responses. If folds are response-level, the same items appear in both train and test, so the LLM can memorize each item's NRM curve during distribution correction and the Table 4 correlations measure fit rather than prediction. If folds are item-level, report per-fold correlations and standard errors; with roughly 55 items per fold, the gap between Qwen-14B (r=0.503) and ModernBERT (r=0.239) could be sampling noise. This is the load-bearing evidence for the paper's central claim and must be documented with fold assignments or explicit split specifications.
  2. [§3.2] The linear calibration step — 'fit the ability level estimates on the development set to the true ability level values with a linear regression' — and the early stopping on the development set are described without stating whether the development set is inside or outside each ELA fold. If the dev set overlaps with the training items, the linear correction and early stopping leak target information into the test set. Specify the partition of the 275 items into train/dev/test (or folds) and the exact role of the development set in each fold.
  3. [§3.3.2, Eq. (4)] The BEA experiment converts 1PL difficulty b into full NRM response curves by assuming incorrect options are selected uniformly at random when a student does not know the answer. This assumption is acknowledged in the text but untested; if high-ability students systematically prefer certain distractors, the synthetic training targets are misspecified and the resulting b predictions inherit that bias. The BEA result therefore mainly demonstrates text-to-b regression under a synthetic-response model, not reconstruction of ICCs from real response patterns. Please temper the abstract/conclusion accordingly or provide evidence that the uniform-distractor assumption is reasonable for this item pool.
  4. [§3.1, Eqs. (12)–(14)] There are index errors in the core equations. Eq. (12) lists (C_{1j1}/C_{j1}, ..., C_{1j1}/C_{j1}) with all entries identical; it should be C_{ijk}/C_{jk}. Eq. (14) defines P_{jk} on the left-hand side but the right-hand side depends on the free index i; it should be P_{ijk}. Eq. (16) likewise equates a vector with a softmax without proper indices. As written, the NRM fitting procedure cannot be implemented without guesswork. Correct these and re-derive the weighting in Eq. (13) consistently.
minor comments (7)
  1. [Abstract and §1] The phrase 'reconstruct the probability of a correct response as a function of student ability' overstates the BEA evidence, which is based entirely on synthetic response curves; consider softening the wording to reflect the assumptions.
  2. [§3.3.2, Table 5] The Dummy Regressor row reports only a single value (0.31) while the discussion states an RMSE of 0.31. Clarify which column this value belongs to and provide the missing RMSE if available.
  3. [§4.1, Table 4] The caption says 'across five folds' but no per-fold values are reported. Add per-fold correlations/RMSE or state explicitly that only fold-averaged values are shown.
  4. [§3.1, Eq. (4)] The derivation of b-hat involves a correction log(n-1)/a1; the notation b_hat and b_j should be defined consistently to avoid confusion with the general NRM parameters.
  5. [Figure 3] The caption refers to blue, green, and red curves, but the figure appears to be in grayscale. Use distinguishable line styles or markers for accessibility.
  6. [§3.2] Minor language issues: 'an answer an item' in §3, 'we are now looking to equate' in §3.2, and 'Qwen-14B followed closely with a correlation with RMSE of 0.294' in §4.2 are awkward. A careful proofread is recommended.
  7. [§3.1, Table 2] The ability descriptor cut points and the normal ability distribution parameters for BEA are free parameters chosen by the authors; providing a brief justification or a sensitivity analysis would strengthen the method's credibility.

Circularity Check

0 steps flagged

No significant circularity: the LLM-to-NRM matching is by design, and evaluation is framed around held-out items; the main weakness is an unspecified ELA fold construction, not a definitional reduction.

full rationale

The central derivation chain is: (i) fit a Nominal Response Model to observed student responses (Eq. 13); (ii) fine-tune the LLM so its option logits approximate the NRM probabilities (Eq. 16); (iii) for items not used in that fine-tuning, read the LLM's option probabilities and solve Eq. 11 for IRT parameters. Step (ii) does make the LLM output on training items equal, by construction, to the fitted NRM curve; this would be circular if the paper claimed to 'reconstruct' ICCs for the same items used in distribution correction. But the paper explicitly frames the goal as 'unseen' items: 'This approach aims to reconstruct the probability that a student correctly answers an unseen question as a function of their ability.' The BEA evaluation uses a genuine external test set of 201 items, and the ELA evaluation is described as five-fold cross-validation. The one concrete weakness is that the ELA fold construction is not specified (item-level vs response-level splits), so the text alone does not let a reader verify that the same items never appear in both training and test folds. That is an omitted reporting detail and a potential leakage risk, not an exhibited equation-level circularity. The BEA uniform-distractor assumption (Eq. 4) is stated openly as an assumption. Self-citations to SMART [22] and other prior work are contextual and are not used as a load-bearing uniqueness or justification argument. The paper is therefore not circular in the sense of deriving its conclusions from its own fitted inputs; it is under-specified on evaluation hygiene for the ELA experiment.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The method rests on several unvalidated domain assumptions: the NRM functional form, uniform random guessing for BEA, normal ability distribution, and the identification of LM token probabilities with student response probabilities. None are tested; the uniform-guessing assumption is acknowledged but not analyzed. The free parameters are mostly design choices (ability bins, calibration mapping) rather than physically motivated constants.

free parameters (4)
  • Ability descriptor cut points (Table 2) = 19 cutoffs: -3, -2.7, ..., 2.7
    The θ intervals for the 20 ability labels were generated by prompting a language model, not derived from data; they define the discrete ICC grid and the weights in Eq. (10).
  • Normal ability distribution parameters for BEA = Unspecified; implicitly N(0,1)
    The mapping from labels to expected θ and weights ω_k assumes θ~N(μ,σ²). For ELA, μ=0.13 and σ²=1.15 are externally calibrated; for BEA the paper does not state what μ and σ are used.
  • Linear calibration slope/intercept on development set = Not reported
    Section 3.2: 'fit the ability level estimates on the development set to the true ability level values with a linear regression'; this is a fitted post-processing applied to test predictions.
  • LoRA rank and training hyperparameters = rank=64, lr 5e-5 and 5e-6
    Chosen by the author; not swept. These affect whether the distribution-correction loss converges and how well the model adheres to ability descriptors.
axioms (5)
  • domain assumption MCQ response probabilities follow the Nominal Response Model: P(v_i)=softmax(a_i(θ-b_i)).
    Eq. (3). The entire training target and the mapping from LLM probabilities to item parameters assume this functional form; if distractor utilities are non-linear in θ, the reconstructed ICCs are misspecified.
  • ad hoc to paper For BEA, when a student does not know the answer, all incorrect options are chosen with equal probability.
    Section 4.2 and Eq. (4). Used to convert 1PL difficulty into full option probabilities because response data are unavailable; this is known to be false for real examinees and is not validated.
  • domain assumption Student ability is normally distributed (θ~N(μ,σ²)).
    Section 3.1. Used to compute expected θ_k and weights ω_k; ELA parameters are externally calibrated, BEA parameters are not specified.
  • domain assumption LLM token probabilities over the option tokens can be treated as student response probabilities after fine-tuning.
    Section 3.2, Eq. (16). The method equates softmax of LM logits with NRM probabilities; this identification is the core of the approach and is not independently verified.
  • domain assumption A model fine-tuned on training items transfers to unseen items based on item text.
    The evaluation in Tables 4–5 only makes sense if the LLM generalizes across items; no analysis of item-level leakage or text-similarity confounding is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 13537 in / 16574 out tokens · 154166 ms · 2026-08-03T12:29:50.165109+00:00 · methodology

0 comments
read the original abstract

Traditional methods for determining assessment item parameters, such as difficulty and discrimination, rely heavily on expensive field testing to collect student performance data for Item Response Theory (IRT) calibration. This study introduces a novel approach that implicitly models these psychometric properties by fine-tuning Large Language Models (LLMs) to simulate student responses across a spectrum of latent abilities. Leveraging the Qwen-3 dense model series and Low-Rank Adaptation (LoRA), we train models to generate responses to multiple choice questions conditioned on discrete ability descriptors. We reconstruct the probability of a correct response as a function of student ability, effectively generating synthetic Item Characteristic Curves (ICCs) to estimate IRT parameters. Evaluation on a dataset of Grade 6 English Language Arts (ELA) items and the BEA 2024 Shared Task dataset demonstrates that this method competes with or outperforms baseline approaches. This simulation-based technique seems particularly effective at modeling item discrimination.

Figures

Figures reproduced from arXiv: 2601.02580 by Christopher Ormerod.

Figure 1
Figure 1. Figure 1: Architectural Schematic of the Qwen3 Dense Model Series. This diagram illustrates the trans [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Distribution of Ability-Level Descriptors. The bar chart illustrates the frequency of student [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of Item Characteristic Curves (ICCs). This plot illustrates the relationship between [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 18 linked inside Pith

  1. [1]

    Hewett, Jamie Huynh, Mojan Javaheripi, Xin Jin, Piero Kauffmann, Nikos Karampatziakis, Dongwoo Kim, Mahoud Khademi, Lev Kurilenko, James R

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha 14 Bilenko, Johan Bjorck, S´ ebastien Bubeck, Qin Cai, Martin Cai, Caio C´ esar Teodoro Mendes, Weizhu Chen, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Yen-Chun Chen, Yi-Li...

    Pith/arXiv arXiv 2024

  2. [2]

    Llama 3 Model Card

    AI@Meta. Llama 3 Model Card. 2024

    2024

  3. [3]

    Casabianca, Sharon C

    Okan Bulut, Maggie Beiting-Parrish, Jodi M. Casabianca, Sharon C. Slater, Hong Jiao, Dan Song, Christopher M. Ormerod, Deborah Gbemisola Fabiyi, Rodica Ivan, Cole Walsh, Oscar Rios, Joshua Wilson, Seyma N. Yildirim-Erbasli, Tarid Wongvorachan, Joyce Xinle Liu, Bin Tan, and Polina Mo- rilova. The Rise of Artificial Intelligence in Educational Measurement: ...

    Pith/arXiv arXiv 2024

  4. [4]

    Item Difficulty and Response Time Prediction with Large Language Models: An Empirical Analysis of USMLE Items

    Okan Bulut, Guher Gorgun, and Bin Tan. Item Difficulty and Response Time Prediction with Large Language Models: An Empirical Analysis of USMLE Items. In Ekaterina Kochmar, Marie Bexte, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Ana ¨ ıs Tack, Victoria Yaneva, and Zheng Yuan, editors,Proceedings of the 19th Workshop on Innovative Use of NLP for ...

    2024

  5. [5]

    QLoRA: Efficient Finetuning of Quantized LLMs.Advances in Neural Information Processing Systems, 36:10088–10115, December 2023

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient Finetuning of Quantized LLMs.Advances in Neural Information Processing Systems, 36:10088–10115, December 2023

    2023

  6. [7]

    Prompt Engineering with ChatGPT: A Guide for Academic Writers.Annals of Biomedical Engineering, 51(12):2629–2633, December 2023

    Louie Giray. Prompt Engineering with ChatGPT: A Guide for Academic Writers.Annals of Biomedical Engineering, 51(12):2629–2633, December 2023

    2023

  7. [8]

    Hambleton, Hariharan Swaminathan, and H

    Ronald K. Hambleton, Hariharan Swaminathan, and H. Jane Rogers.Fundamentals of Item Response Theory. SAGE, 1991. Google-Books-ID: gW05DQAAQBAJ. 15

    1991

  8. [9]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-Rank Adaptation of Large Language Models, October 2021. arXiv:2106.09685 [cs]

    Pith/arXiv arXiv 2021

  9. [10]

    Item Diffi- culty Modeling Using Fine-Tuned Small and Large Language Models

    Ming Li, Hong Jiao, Tianyi Zhou, Nan Zhang, Sydney Peters, and Robert W Lissitz. Item Diffi- culty Modeling Using Fine-Tuned Small and Large Language Models. In Joshua Wilson, Christopher Ormerod, and Magdalen Beiting Parrish, editors,Proceedings of the Artificial Intelligence in Measure- ment and Education Conference (AIME-Con): Coordinated Session Paper...

    2025

  10. [11]

    Yunting Liu, Shreya Bhandari, and Zachary A. Pardos. Leveraging LLM respondents for item evaluation: A psychometric analysis.British Journal of Educational Technology, 56(3):1028–1052, 2025. eprint: https://bera-journals.onlinelibrary.wiley.com/doi/pdf/10.1111/bjet.13570

    doi:10.1111/bjet.13570 2025

  11. [12]

    Decoupled Weight Decay Regularization, January 2019

    Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization, January 2019. arXiv:1711.05101 [cs, math]

    Pith/arXiv arXiv 2019

  12. [13]

    GPT-4 Technical Report, March 2023

    OpenAI. GPT-4 Technical Report, March 2023. arXiv:2303.08774 [cs]

    Pith/arXiv arXiv 2023

  13. [14]

    Automated Text Scoring in the Age of Generative AI for the GPU-poor.Chinese/English Journal of Educational Measurement and Evaluation, 5(3), December 2024

    Chris Ormerod and Alexander Kwako. Automated Text Scoring in the Age of Generative AI for the GPU-poor.Chinese/English Journal of Educational Measurement and Evaluation, 5(3), December 2024

    2024

  14. [15]

    Argumentation Element Annotation Modeling using XLNet, November 2023

    Christopher Ormerod, Amy Burkhardt, Mackenzie Young, and Sue Lottridge. Argumentation Element Annotation Modeling using XLNet, November 2023. arXiv:2311.06239 [cs]

    Pith/arXiv arXiv 2023

  15. [16]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback,...

    Pith/arXiv arXiv 2022

  16. [17]

    Text-Based Ap- proaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review, September

    Sydney Peters, Nan Zhang, Hong Jiao, Ming Li, Tianyi Zhou, and Robert Lissitz. Text-Based Ap- proaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review, September

  17. [18]

    Improving Language Under- standing by Generative Pre-training, 2018

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving Language Under- standing by Generative Pre-training, 2018

    2018

  18. [19]

    MESA Press, 5835 S,

    Georg Rasch.Probabilistic Models for Some Intelligence and Attainment Tests. MESA Press, 5835 S,

  19. [20]

    Pedro Uria Rodriguez, Amir Jafari, and Christopher M. Ormerod. Language models and Automated Essay Scoring, September 2019. Number: arXiv:1909.09482 arXiv:1909.09482 [cs, stat]

    Pith/arXiv arXiv 2019

  20. [21]

    LaFlair, Yena Park, and Jacqueline Church

    Andrew Runge, Yigal Attali, Geoffrey T. LaFlair, Yena Park, and Jacqueline Church. A generative AI-driven interactive listening assessment task.Frontiers in Artificial Intelligence, 7, November 2024. Publisher: Frontiers

    2024

  21. [22]

    SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction, September 2025

    Alexander Scarlatos, Nigel Fernandez, Christopher Ormerod, Susan Lottridge, and Andrew Lan. SMART: Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction, September 2025. arXiv:2507.05129 [cs]. 16

    arXiv 2025

  22. [23]

    Proximal Policy Optimization Algorithms, August 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms, August 2017. arXiv:1707.06347 [cs]

    Pith/arXiv arXiv 2017

  23. [24]

    DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines, February 2024

    Arnav Singhvi, Manish Shetty, Shangyin Tan, Christopher Potts, Koushik Sen, Matei Zaharia, and Omar Khattab. DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines, February 2024. arXiv:2312.13382 [cs]

    Pith/arXiv arXiv 2024

  24. [25]

    Dhamecha, Swarnadeep Saha, Tengfei Ma, V

    Chul Sung, Tejas I. Dhamecha, Swarnadeep Saha, Tengfei Ma, V. Reddy, and R. Arora. Pre-Training BERT on Domain Resources for Short Answer Grading. InEMNLP, 2019

    2019

  25. [26]

    Attention is All you Need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is All you Need. InAdvances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017

    2017

  26. [27]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. Technical Report arXiv:1804.07461, arXiv, February 2019. arXiv:1804.07461 [cs] type: article

    Pith/arXiv arXiv 2019

  27. [28]

    Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference, December 2024

    Benjamin Warner, Antoine Chaffin, Benjamin Clavi´ e, Orion Weller, Oskar Hallstr¨ om, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference...

    Pith/arXiv arXiv 2024

  28. [29]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent Abilities of Large Language Models, October 2022. arXiv:2206.07682 [cs]

    Pith/arXiv arXiv 2022

  29. [30]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, January

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, January

  30. [31]

    Parameter-Efficient Fine- Tuning Methods for Pretrained Language Models: A Critical Review and Assessment, December 2023

    Lingling Xu, Haoran Xie, Si-Zhao Joe Qin, Xiaohui Tao, and Fu Lee Wang. Parameter-Efficient Fine- Tuning Methods for Pretrained Language Models: A Critical Review and Assessment, December 2023. arXiv:2312.12148 [cs]

    Pith/arXiv arXiv 2023

  31. [32]

    Findings from the First Shared Task on Automated Prediction of Difficulty and Response Time for Multiple-Choice Questions

    Victoria Yaneva, Kai North, Peter Baldwin, Le An Ha, Saed Rezayi, Yiyun Zhou, Sagnik Ray Choud- hury, Polina Harik, and Brian Clauser. Findings from the First Shared Task on Automated Prediction of Difficulty and Response Time for Multiple-Choice Questions. In Ekaterina Kochmar, Marie Bexte, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Ana ¨ ıs T...

    2024

  32. [33]

    Qwen3 Technical Report, May

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, ...

  33. [34]

    Qwen2 Technical Report, September 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei...

    Pith/arXiv arXiv 2024

  34. [35]

    Critical

    Eric Zelikman, Wanjing Ma, Jasmine Tran, Diyi Yang, Jason Yeatman, and Nick Haber. Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sen- tence Reading Efficiency. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pa...

    2023

  35. [38]

    arXiv:2505.09388 [cs]

    Pith/arXiv arXiv

  36. [1993]

    ERIC Number: ED419814

  37. [2018]

    arXiv:1810.04805 [cs] type: article

    Pith/arXiv arXiv

  38. [2023]

    arXiv:2201.11903 [cs]

    Pith/arXiv arXiv

  39. [2025]

    arXiv:2509.23486 [cs]

    arXiv