Pith. sign in

REVIEW 3 major objections 5 minor 39 references

GENCAT selects adaptive-test questions by generating the code a student would write, and this predicted-response signal yields significantly more accurate knowledge estimates than correctness-only CAT in early test stages, with up to 4.32%

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 21:24 UTC pith:TGPCSF2J

load-bearing objection GENCAT's core idea — using generated open-ended responses to drive CAT item selection — is genuinely new and worth engaging, but the headline 4.32% gain is a non-significant cell measured through a 1PL proxy, not through GENCAT's own knowledge estimates. the 3 major comments →

arxiv 2602.20020 v2 pith:TGPCSF2J submitted 2026-02-23 cs.CL

CodeGENCAT: Generative Computerized Adaptive Testing for Open-ended Coding Problems

Wanyong Feng , Alexander Scarlatos , Ruochen Sun , Andrew Lan This is my paper
classification cs.CL
keywords computerized adaptive testinggenerative item response theoryopen-ended programming responsesquestion selectionlarge language modelsdirect preference optimizationknowledge estimationcode generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces GENCAT, a computerized adaptive testing framework that uses a generative language model to predict not just whether a student will answer a coding question correctly, but what code they would write. The central claim is that this richer predicted response carries diagnostic information—specific bugs, misconceptions, partial understanding—that lets the test pick more informative next questions than traditional correctness-only selection. On two real programming-course datasets, the best selection strategies improve the AUC of knowledge-estimate quality by up to 4.32% over strong baselines in the early test stages, when accurate estimates matter most. A sympathetic reader would care because shorter, sharper adaptive tests are the practical payoff: if GENCAT works, open-ended responses can be used to reduce test length without losing measurement accuracy.

Core claim

GENCAT replaces the binary correctness prediction at the heart of CAT with full open-ended response generation. Student ability is represented as a low-dimensional latent vector mapped to knowledge-component mastery levels, which are injected into a language model's prompt as soft embeddings interpolating between the semantic tokens for TRUE and FALSE; the model is trained to generate the student's actual code conditioned on those mastery levels, first by supervised fine-tuning and then by direct preference optimization to force generated responses to depend on mastery. At each test step, the framework samples several predicted code responses for every candidate question and selects the next

What carries the argument

The load-bearing object is the Generative Item Response Theory (GIRT) model—a generative LLM that outputs full student code conditioned on a structured prompt containing each question's knowledge components and the student's mastery level for each, encoded as a linear interpolation between the embeddings of the words TRUE and FALSE. The mastery levels come from a low-dimensional latent ability vector projected through an MLP with a sigmoid, so the generative objective and a knowledge-correctness alignment loss are trained end to end; a direct preference optimization stage then prevents mode collapse and makes generation genuinely sensitive to mastery. Around this model, three question-select

Load-bearing premise

The load-bearing premise, stated in the paper's Section 3.3, is that the pre-trained one-parameter logistic item-response model, used to convert GENCAT's selected question sequences into held-out response predictions, is a valid proxy for true student knowledge; if that model mis-measures ability, the reported AUC gains may reflect better alignment with that specific model rather than genuinely more accurate knowledge estimates.

What would settle it

Train GENCAT on simulated students with known true ability levels and responses generated from a known item-response process; if the question sequences GENCAT selects do not yield ability estimates closer to the true simulated ability than correctness-only selection does (measured by correlation or mean squared error), the central claim collapses. A complementary check is to replace the one-parameter proxy with a multidimensional item-response model and see whether the early-stage AUC advantage persists.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Adaptive test question selection can be driven by predicted open-ended responses rather than only predicted correctness, opening the door to better early-stage ability estimates with fewer questions.
  • The diversity-based selection strategy is most informative when the ability estimate is still close to the population prior; information-based selection takes over once the estimate becomes precise.
  • Aligning the generative model with knowledge via direct preference optimization is necessary: without it the model collapses to nearly identical responses regardless of mastery, and question selection loses its signal.
  • GENCAT maintains competitive question-exposure and test-overlap rates while improving knowledge estimates, so the gains do not come at the cost of item overuse.
  • Response similarity between generated and actual student code improves as the test progresses, suggesting the estimated ability becomes increasingly faithful.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: If GENCAT's gains reproduce in non-programming open-ended domains (short-answer math, essays, science explanations), the framework's central idea generalizes to any CAT setting where response text carries diagnostic signal; the paper itself notes it lacks such datasets.
  • Editorial inference: The evaluation's reliance on a pre-trained one-parameter logistic item-response model as a proxy means the reported AUC gains could partly reflect better matching with that model's question-difficulty assumptions; testing with multidimensional or 2PL/3PL proxies would clarify how much of the gain is intrinsic to knowledge estimation.
  • Editorial inference: The Fisher-information criterion used here depends on the generative model's gradient with respect to the latent vector; a natural extension would be to compute expected information gain after observing the actual response rather than on sampled responses, which might sharpen later-stage selection.
  • Editorial inference: Sampling cost is the main deployment barrier; with fewer sampled responses the gains drop only slightly while inference time drops substantially, suggesting a practical latency-accuracy trade-off that the paper measures but does not optimize.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CodeGENCAT, a generative CAT framework that uses LLMs to condition question selection on predicted open-ended code responses. It introduces a GIRT model trained with SFT and DPO, three question-selection algorithms (Uncertainty, Diversity, Information), and evaluates on two real-world programming datasets (CodeWorkout and ProgFeed). The main claim is that CodeGENCAT achieves more accurate knowledge estimates than existing CAT baselines, with an AUC improvement of up to 4.32% in early testing stages.

Significance. The paper addresses an important limitation of standard CAT, which discards the rich textual information in open-ended responses. The GIRT model and the three selection criteria are novel and reasonably motivated. Strengths include public code (Section 1.1), real-world datasets, ablations (N=3 vs 10, DPO vs SFT), and a test-security analysis. However, the central claim of improved knowledge estimation is not directly supported by the reported evaluation, which relies on a pre-trained 1PL proxy rather than GENCAT's own latent knowledge estimates. The paper also overstates its results in the abstract and Section 1.1 relative to Table 1. If the evaluation is strengthened or the claims are appropriately qualified, this could be a useful contribution to the field.

major comments (3)
  1. [Section 3.3, Table 1] The Accuracy/AUC results are computed by feeding GENCAT's selected question sequence to a pre-trained 1PL IRT model, which updates a scalar ability and predicts correctness. GENCAT's own latent knowledge z_i is never used in these metrics. Thus Table 1 evaluates question selection, not the GIRT knowledge estimator claimed in Section 1.1. If 1PL is misspecified for open-ended programming responses, the gains may reflect selection policies favorable to 1PL rather than improved knowledge estimates. The authors acknowledge this ("We leave studying... to future work"), but the claim "significantly more accurate knowledge estimate" is stronger than the evidence. Please report metrics using GIRT's own estimates (e.g., via predicted code and a scoring model) or qualify the claim.
  2. [Abstract, Section 1.1, Table 1] The abstract says CodeGENCAT "outperforms all CAT baselines", but Table 1 shows GENCAT (Uncertainty) is below LACAT/NCAT on ProgFeed Accuracy at t=1 and other cells. The headline 4.32% improvement at t=1 on CodeWorkout (Diversity AUC 0.7073 vs 1PL_IRT 0.6641) has no significance star; stars appear only at t=3,5,7 on CodeWorkout, and none on ProgFeed. No error bars or confidence intervals are reported. Please state results per variant/dataset/step and provide measures of uncertainty.
  3. [Section 4.1, Appendix C] The CodeBLEU similarity results (Table 5) show only small differences among selection algorithms and the authors note that CodeBLEU struggles to distinguish due to structural overlap. The qualitative analysis (Appendix F) is a single problem. This additional evidence does not strongly support the central claim of improved knowledge estimation. Stronger validation (e.g., correlation of GIRT ability estimates with external outcomes, or higher-resolution response similarity metrics) is needed.
minor comments (5)
  1. [Title/Abstract] The title and abstract use "CodeGENCAT" while the body uses "GENCAT"; please choose one consistent name throughout.
  2. [Section 2.3] Typo: "stduent lantent knowledge" should be "student latent knowledge".
  3. [Section 3.3] Typo: "respones" should be "responses".
  4. [Section 3.1] The new dataset ProgFeed is private and IRB approval is pending; reproducibility would be improved by releasing anonymized data or by describing how to obtain it.
  5. [Abstract, Table 1] The claim "up to 4.32%" should specify that this is an absolute percentage-point difference in AUC, not a relative improvement, to avoid ambiguity.

Circularity Check

0 steps flagged

No significant circularity: the main results are held-out comparisons against external baselines, though the 1PL-proxy evaluation and minor self-citations make the central claim somewhat stronger than the direct evidence.

full rationale

The paper is largely self-contained against external benchmarks. The GIRT model is trained on a separate 80% student group, and the CAT evaluation group is held out with 5-fold cross-validation; the question-selection algorithms (Uncertainty, Diversity, Information) are fixed criteria rather than learned to optimize the reported AUC. The reported AUC/Accuracy in Table 1 are computed by a pre-trained 1PL IRT model used as an explicit proxy, as stated in Section 3.3: 'we use the pre-trained 1PL IRT model as a proxy item response model to update student knowledge estimates and predict correctness.' This proxy applies uniformly to GENCAT and LACAT, and the held-out predictions are not refits of GENCAT's own parameters, so the evaluation is an indirect validity check rather than a definitional reduction. The DPO preference pairs in Section 2.4 are constructed using the model's own predicted correctness likelihood, which is self-referential in data construction, but the preferred response is the student's actual ground-truth response, and the central experimental result does not rest on a formal derivation from that construction. The self-citations to [8], [28], and [29] are inspirational or technical (KC vector representation, logit-interpolation scoring, and preference-pair construction) and are not load-bearing uniqueness claims or forced alternatives. The main weakness is that the headline 'significantly more accurate knowledge estimate' is evaluated through a fixed 1PL proxy rather than through GIRT's own latent knowledge estimates, and some starred early-stage gains appear on one dataset only; this is a limitation, not circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central claim rests primarily on the LLM's ability to generate knowledge-dependent code and on the validity of the 1PL proxy evaluation. The free parameters are standard hyperparameters; there are no new physical or conceptual entities.

free parameters (6)
  • λ (loss balancing coefficient) = 0.2
    Chosen by validation; balances SFT and KC alignment losses (Section 2.3, Appendix D).
  • D (latent knowledge dimension) = 2
    Chosen; the latent knowledge vector dimension (Section 2.2, Appendix D.1.1).
  • M (number of dispreferred responses per DPO pair) = 3
    Chosen for DPO preference construction (Appendix D.1.2).
  • τ (likelihood difference threshold for DPO pairs) = 0.1
    Responses are paired only if the correctness-likelihood difference exceeds τ (Section 2.4).
  • N (number of sampled responses per candidate question) = 10
    Chosen; sampling number for question selection (Section 2.5, Appendix D.2).
  • β (DPO KL penalty) = 0.5
    DPO hyperparameter (Appendix D.1.2).
axioms (5)
  • domain assumption Human-annotated knowledge components (KCs) are the correct decomposition of programming skills
    The GIRT model conditions on KC mastery levels; if the KC set is wrong or incomplete, the knowledge estimate is mis-specified (Section 2.2).
  • domain assumption Monotonicity: higher KC mastery leads to higher probability of correct response
    Enforced via the L_KC alignment loss; the paper assumes this is the desired interpretation of knowledge (Section 2.3).
  • domain assumption The LLM can generate plausible student code conditioned on soft-prompted KC mastery
    The GIRT model relies on a fine-tuned Llama-3.2-1B to produce realistic, knowledge-dependent code (Section 2.2, Appendix H).
  • domain assumption Student responses are conditionally independent given the latent knowledge z
    The total loss sums over individual responses without modeling dependence; standard in IRT but unstated (Section 2.3).
  • standard math Continuous Bernoulli likelihood models response correctness for DPO pair selection
    Used to rank preferred vs. dispreferred responses; follows [19] (Section 2.4).

pith-pipeline@v1.3.0-alltime-deepseek · 15686 in / 15418 out tokens · 125524 ms · 2026-08-02T21:24:42.429068+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of CodeGENCAT: Generative Computerized Adaptive Testing for Open-ended Coding Problems." pith.science (2026). https://pith.science/paper/TGPCSF2J

@misc{pith2026260220020,
  author       = {Pith},
  title        = {Pith review of: CodeGENCAT: Generative Computerized Adaptive Testing for Open-ended Coding Problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TGPCSF2J}},
  note         = {Machine review of arXiv:2602.20020}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Existing Computerized Adaptive Testing (CAT) frameworks typically select questions based on the predicted likelihood that the student will answer correctly. This design ignores information contained in students' open-ended responses, especially in domains such as programming education, where code structures and bugs contain rich information on student knowledge. In this work, we propose \textbf{Code} \textbf{GEN}erative \textbf{CAT} (\textbf{CodeGENCAT}), a generative CAT framework that selects questions using predicted student code responses. First, we develop a Generative Item Response Theory (GIRT) model that generates code responses conditioned on estimated student knowledge, trained with supervised fine-tuning followed by direct preference optimization for knowledge-response alignment. Second, we introduce three question-selection algorithms that measure uncertainty, coding style diversity, and information from predicted student code responses. Experiments on two real-world programming education datasets show that CodeGENCAT outperforms all CAT baselines, achieving an AUC improvement of up to 4.32\% over the strongest baseline in the early stages of adaptive testing.

Figures

Figures reproduced from arXiv: 2602.20020 by Alexander Scarlatos, Andrew Lan, Ruochen Sun, Wanyong Feng.

Figure 1
Figure 1. Figure 1: Overview of one GENCAT cycle with Diversity as the question selection algorithm. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

39 extracted references · 7 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2303.08774 (2023)

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    Pith/arXiv arXiv 2023

  2. [2]

    arXiv preprint arXiv:2402.16827 (2024)

    Albalak, A., Elazar, Y ., Xie, S.M., Longpre, S., Lambert, N., Wang, X., Muennighoff, N., Hou, B., Pan, L., Jeong, H., et al.: A survey on data selection for language models. arXiv preprint arXiv:2402.16827 (2024)

    Pith/arXiv arXiv 2024

  3. [3]

    In: Noise reduction in speech processing, pp

    Benesty, J., Chen, J., Huang, Y ., Cohen, I.: Pearson correlation coefficient. In: Noise reduction in speech processing, pp. 1–4. Springer (2009)

    2009

  4. [4]

    Online (2021), https://sites.google.com/ncsu.edu/ csedm-dc-2021/, accessed: 2025-12-17

    Challenge Organizers: The 2nd csedm data challenge. Online (2021), https://sites.google.com/ncsu.edu/ csedm-dc-2021/, accessed: 2025-12-17

    2021

  5. [5]

    Chang, H.S., Hsu, H.J., Chen, K.T., et al.: Modeling exercise relationships in e-learning: A unified approach. In: EDM. pp. 532–535 (2015)

    2015

  6. [6]

    In: Findings of the Association for Computational Linguistics: EMNLP 2024

    Cheng, C., Zhao, G., Huang, Z., Zhuang, Y ., Pan, Z., Liu, Q., Li, X., Chen, E.: Towards explainable computerized adaptive testing with large language model. In: Findings of the Association for Computational Linguistics: EMNLP 2024. pp. 2655–2672 (2024)

    2024

  7. [7]

    Oxford University Press (2010)

    DeMars, C.: Item response theory. Oxford University Press (2010)

    2010

  8. [8]

    arXiv preprint arXiv:2502.18632 (2025)

    Duan, Z., Fernandez, N., Narayanan, A.B.L., Hassany, M., de Alencar, R.S., Brusilovsky, P., Akram, B., Lan, A.: Automated knowledge component generation and knowledge tracing for coding problems. arXiv preprint arXiv:2502.18632 (2025)

    Pith/arXiv arXiv 2025

  9. [9]

    arXiv e-prints pp

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al.: The llama 3 herd of models. arXiv e-prints pp. arXiv–2407 (2024)

    2024

  10. [10]

    In: Second Joint Conference on Lexical and Computational Semantics (*SEM), V olume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013)

    Dzikovska, M.O., Nielsen, R.D., Brew, C., Leacock, C., Giampiccolo, D., Bentivogli, L., Clark, P., Dagan, I., Dang, H.T.: Semeval-2013 task 7: The joint student response analysis and 8th recognizing textual entailment challenge. In: Second Joint Conference on Lexical and Computational Semantics (*SEM), V olume 2: Proceedings of the Seventh International W...

    2013

  11. [11]

    In: Proceedings of the 2017 ACM conference on innovation and technology in computer science education

    Edwards, S.H., Murali, K.P.: Codeworkout: short programming exercises with built-in data collection. In: Proceedings of the 2017 ACM conference on innovation and technology in computer science education. pp. 188–193 (2017)

    2017

  12. [12]

    arXiv preprint arXiv:2002.08155 (2020)

    Feng, Z.: Codebert: A pre-trained model for program-ming and natural languages. arXiv preprint arXiv:2002.08155 (2020)

    Pith/arXiv arXiv 2002

  13. [13]

    arXiv preprint arXiv:2108.07386 (2021)

    Ghosh, A., Lan, A.: Bobcat: Bilevel optimization-based computerized adaptive testing. arXiv preprint arXiv:2108.07386 (2021)

    Pith/arXiv arXiv 2021

  14. [14]

    Journal of educational evaluation for health professions15(2018)

    Han, K.C.T.: Components of the item selection algorithm in computerized adaptive testing. Journal of educational evaluation for health professions15(2018)

    2018

  15. [15]

    In: Acm Sigir Forum

    Lewis, D.D.: A sequential algorithm for training text classifiers: Corrigendum and additional data. In: Acm Sigir Forum. vol. 29, pp. 13–19. ACM New York, NY , USA (1995)

    1995

  16. [16]

    Psychometrika63(2), 201–216 (1998)

    van der Linden, W.J.: Bayesian item selection criteria for adaptive testing. Psychometrika63(2), 201–216 (1998)

    1998

  17. [17]

    Van der Linden, W.J., Glas, C.A., et al.: Computerized adaptive testing: Theory and practice, vol. 13. Springer (2000)

    2000

  18. [18]

    In: Elements of adaptive testing, pp

    Van der Linden, W.J., Pashley, P.J.: Item selection and ability estimation in adaptive testing. In: Elements of adaptive testing, pp. 3–30. Springer (2009)

    2009

  19. [19]

    In: Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)

    Loginova, E., Benedetto, L., Benoit, D., Cremonesi, P.: Towards the application of calibrated transformers to the unsupervised estimation of question difficulty from text. In: Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021). pp. 846–855 (2021)

    2021

  20. [20]

    Routledge (2012)

    Lord, F.M.: Applications of item response theory to practical testing problems. Routledge (2012)

    2012

  21. [21]

    research report 2011-12

    Luecht, R.M., Sireci, S.G.: A review of models for computer-based testing. research report 2011-12. College Board (2011)

    2011

  22. [22]

    https://nces.ed.gov/nationsreportcard/ (2022), u.S

    National Center for Education Statistics: National assessment of educational progress (naep) data explorer. https://nces.ed.gov/nationsreportcard/ (2022), u.S. Department of Education

    2022

  23. [23]

    SIAM (2006) 10 arXivTemplateA PREPRINT

    Pukelsheim, F.: Optimal design of experiments. SIAM (2006) 10 arXivTemplateA PREPRINT

    2006

  24. [24]

    Advances in neural information processing systems36, 53728–53741 (2023)

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C.D., Ermon, S., Finn, C.: Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems36, 53728–53741 (2023)

    2023

  25. [25]

    ERIC (1993)

    Rasch, G.: Probabilistic models for some intelligence and attainment tests. ERIC (1993)

    1993

  26. [26]

    In: Multidimensional item response theory, pp

    Reckase, M.D.: Historical background for multidimensional item response theory (mirt). In: Multidimensional item response theory, pp. 57–77. Springer (2009)

    2009

  27. [27]

    arXiv preprint arXiv:2009.10297 (2020)

    Ren, S., Guo, D., Lu, S., Zhou, L., Liu, S., Tang, D., Sundaresan, N., Zhou, M., Blanco, A., Ma, S.: Codebleu: a method for automatic evaluation of code synthesis. arXiv preprint arXiv:2009.10297 (2020)

    Pith/arXiv arXiv 2009

  28. [28]

    In: Proceedings of the 15th International Learning Analytics and Knowledge Conference

    Scarlatos, A., Baker, R.S., Lan, A.: Exploring knowledge tracing in tutor-student dialogues using llms. In: Proceedings of the 15th International Learning Analytics and Knowledge Conference. pp. 249–259 (2025)

    2025

  29. [29]

    In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing

    Scarlatos, A., Fernandez, N., Ormerod, C., Lottridge, S., Lan, A.: Smart: Simulated students aligned with item response theory for question difficulty prediction. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. pp. 25082–25105 (2025)

    2025

  30. [30]

    Journal of Educational Measurement53(3), 332–351 (2016)

    Schmidt, S., Zlatkin-Troitschanskaia, O., Fox, J.P.: Pretest-posttest-posttest multilevel irt modeling of competence growth of students in higher education in germany. Journal of Educational Measurement53(3), 332–351 (2016)

    2016

  31. [31]

    Journal of Writing Assessment1(1) (2012)

    Shermis, M.D., Hamner, B.: Contrasting state-of-the-art automated scoring of essays: Analysis. Journal of Writing Assessment1(1) (2012)

    2012

  32. [32]

    In: Proceedings of the 15th International Conference on Educational Data Mining

    Shi, Y ., Chi, M., Barnes, T., Price, T.: Code-dkt: A code-based knowledge tracing model for programming tasks. In: Proceedings of the 15th International Conference on Educational Data Mining. p. 50 (2022)

    2022

  33. [33]

    In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

    Wang, H., Long, T., Yin, L., Zhang, W., Xia, W., Hong, Q., Xia, D., Tang, R., Yu, Y .: Gmocat: A graph-enhanced multi-objective method for computerized adaptive testing. In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 2279–2289 (2023)

    2023

  34. [34]

    arXiv preprint arXiv:2210.10109 (2022)

    Zhang, Z., Strubell, E., Hovy, E.: A survey of active learning for natural language processing. arXiv preprint arXiv:2210.10109 (2022)

    Pith/arXiv arXiv 2022

  35. [35]

    In: Proceedings of the AAAI conference on artificial intelligence

    Zhuang, Y ., Liu, Q., Huang, Z., Li, Z., Shen, S., Ma, H.: Fully adaptive framework: Neural computerized adaptive testing for online education. In: Proceedings of the AAAI conference on artificial intelligence. vol. 36, pp. 4734–4742 (2022)

    2022

  36. [36]

    Correct” (1) and “Incorrect

    Zhuang, Y ., Liu, Q., Zhao, G., Huang, Z., Huang, W., Pardos, Z., Chen, E., Wu, J., Li, X.: A bounded ability estimation for computerized adaptive testing. Advances in Neural Information Processing Systems36, 2381–2402 (2023) 11 arXivTemplateA PREPRINT A Scoring Model for Student Response For the scoring model used in Uncertainty question selection algori...

    2023

  37. [37]

    If a student is unlikely to have knowledge of KCs, the generated code may be incomplete

    Output only one complete method or function in Java. If a student is unlikely to have knowledge of KCs, the generated code may be incomplete

  38. [38]

    Do not include any explanations, comments, or formatting symbols (such as backticks)

  39. [39]

    The goal is to produce plausible student code influenced by the provided knowledges

    Correctness is not required. The goal is to produce plausible student code influenced by the provided knowledges. Inefficiency, minor bugs, and convoluted logic are expected if the student is unlikely to have knowledge of KCs. This rule is the most important!!!! The Java programming problem is:<question> The KC 1 is: <kc1>. The student has knowledge of <k...