Pith. sign in

REVIEW 1 major objections 170 references

PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback

T0 review · 1 major / 0 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PsyScore links essay scoring to ability-adapted feedback through a shared psychometric parameter.

desk verdict PsyScore sketches a shared latent ability link between GPCM-based scoring and ZPD feedback but shows no ablation or numbers confirming the conditioning step adds value. read the letter →

arxiv 2606.20287 v1 pith:5GYPWPUH submitted 2026-06-18 cs.CL

Wei Xia , Jin Wu , Haoran Shi , Xiangyu Wang , Chanjin Zheng This is my paper · ORCID
classification cs.CL
keywords automatedessayscoringitemresponsetheoryadaptivefeedbackpsychometricmodelingzoneofproximaldevelopmentlargelanguagemodelseducationalassessment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces PsyScore to overcome the split between accurate automated essay scoring and feedback that actually matches a learner's current level. It does this by deriving one latent ability value from a neural implementation of the graded partial credit model and then using that same value to steer a multi-agent feedback generator toward zone-of-proximal-development scaffolds. On the ASAP++ dataset the resulting scores stay competitive with existing models while the generated feedback receives higher ratings for pedagogical fit in both pairwise preference tests and simulated revision tasks.

What carries the argument

The shared latent ability parameter produced by the Trait-Adaptive Neural IRT Scorer, which directly conditions the ZPD-Scaffolded Feedback Generator.

What would settle it

A controlled trial in which feedback generated without conditioning on the estimated ability parameter yields equal or higher student revision quality and preference scores than the ability-conditioned version.

Watch

Extended reading notes

Core claim

PsyScore comprises a Trait-Adaptive Neural IRT Scorer that embeds the Graded Partial Credit Model to produce both essay scores and an interpretable ability parameter, a ZPD-Scaffolded Feedback Generator that conditions multi-agent strategies on that parameter, and a Multi-Perspective Feedback Evaluation Strategy that measures quality through preference judgments and revision simulations; the shared ability representation thereby unifies diagnostic assessment with level-specific instructional support.

Load-bearing premise

The ability parameter estimated by the Trait-Adaptive Neural IRT Scorer can be directly used to condition multi-agent feedback strategies so that instructional focus adapts effectively across proficiency levels.

Editorial extensions

If this is right

  • Scoring accuracy on ASAP++ remains competitive with prior neural AES systems.
  • Feedback becomes more aligned with learner proficiency as judged by preference and revision metrics.
  • A single ability estimate supports both assessment and scaffolding without separate models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same ability-conditioned loop could be tested on short-answer or programming tasks to check whether the unification generalizes.
  • If the ability parameter proves stable across multiple essay prompts, the framework offers a route to longitudinal tracking of skill growth inside one system.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 0 minor

Summary. The manuscript proposes PsyScore, a framework integrating three modules: a Trait-Adaptive Neural IRT Scorer that embeds the Graded Partial Credit Model (GPCM) into a neural architecture for essay scoring and student ability estimation, a ZPD-Scaffolded Feedback Generator that conditions multi-agent LLM feedback strategies on the estimated ability parameter to adapt instructional focus, and a Multi-Perspective Feedback Evaluation Strategy that uses pairwise preference judgments and student revision simulations to assess feedback quality. Experiments on the ASAP++ dataset are reported to show competitive scoring performance alongside more pedagogically aligned feedback than prior approaches.

Significance. If the central results hold after addressing the evaluation gap, the work would offer a concrete bridge between psychometric models and LLM-based feedback systems, potentially improving interpretability and adaptivity in automated essay scoring. The shared latent ability representation is a clear conceptual strength that could influence future designs in educational NLP if the conditioning effect is isolated and quantified.

major comments (1)
  1. [Multi-Perspective Feedback Evaluation Strategy] Multi-Perspective Feedback Evaluation Strategy: the reported experiments do not include an ablation isolating the effect of conditioning the ZPD-Scaffolded Feedback Generator on the GPCM-derived ability parameter (e.g., ability-conditioned multi-agent feedback versus fixed-prompt or unconditioned baselines). This comparison is required to substantiate the claim that the shared latent representation yields measurably more pedagogically aligned output or revision gains; without it, improvements cannot be attributed to the ability-parameter mechanism rather than other design choices.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive comment on evaluation design. We agree that isolating the contribution of the ability-parameter conditioning is necessary to strengthen the central claim and will add the requested ablation in revision.

read point-by-point responses

  1. Referee: [Multi-Perspective Feedback Evaluation Strategy] Multi-Perspective Feedback Evaluation Strategy: the reported experiments do not include an ablation isolating the effect of conditioning the ZPD-Scaffolded Feedback Generator on the GPCM-derived ability parameter (e.g., ability-conditioned multi-agent feedback versus fixed-prompt or unconditioned baselines). This comparison is required to substantiate the claim that the shared latent representation yields measurably more pedagogically aligned output or revision gains; without it, improvements cannot be attributed to the ability-parameter mechanism rather than other design choices.

    Authors: We agree that the current experiments do not contain a direct ablation isolating the conditioning of the ZPD-Scaffolded Feedback Generator on the GPCM-derived ability parameter. The manuscript reports overall competitive scoring and feedback alignment but does not compare ability-conditioned multi-agent strategies against fixed-prompt or unconditioned baselines. In the revised version we will add this ablation on the ASAP++ dataset, reporting pairwise preference judgments and revision-simulation outcomes for the three conditions. This will allow quantification of the incremental effect attributable to the shared latent ability representation. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: framework integrates independent modules via shared latent variable

full rationale

The derivation chain estimates student ability via the Trait-Adaptive Neural IRT Scorer (incorporating GPCM), then uses that parameter to condition the separate ZPD-Scaffolded Feedback Generator, and evaluates the output with an independent Multi-Perspective Feedback Evaluation Strategy (pairwise preferences and revision simulations). None of these steps reduce to self-definition, fitted-input-as-prediction, or self-citation load-bearing; the shared latent representation is an explicit design choice rather than a tautology, and no equations or claims in the abstract or described modules collapse the output back to the input by construction. The paper remains self-contained against external benchmarks on ASAP++.

Assumptions & free parameters 1 free parameters · 1 assumptions · 0 invented entities

The framework rests on standard IRT modeling assumptions and LLM generation capabilities. The central ability parameter is a fitted latent variable whose quality determines both scoring and feedback.

free parameters (1)
  • student ability parameter
    Latent trait estimated from essay responses via the neural GPCM model; used for both scoring and feedback conditioning.
assumptions (1)
  • domain assumption The Graded Partial Credit Model can be incorporated into a neural architecture while preserving psychometric interpretability.
    Invoked as the basis for the Trait-Adaptive Neural IRT Scorer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback." pith.science (2026). https://pith.science/paper/5GYPWPUH

@misc{pith2026260620287,
  author       = {Pith},
  title        = {Pith review of: PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5GYPWPUH}},
  note         = {Machine review of arXiv:2606.20287}
}
read the original abstract

Effective Automated Essay Scoring (AES) are expected to support both reliable assessment and actionable instructional feedback. However, existing approaches often treat scoring and feedback as separate components: neural scoring models provide limited interpretability, while Large Language Model (LLM)-based feedback is typically insensitive to learners proficiency levels. To address this fragmentation, this work proposes PsyScore, a psychometrically-aware framework that integrates diagnostic assessment with instructional scaffolding through a shared latent ability representation. PsyScore comprises three key modules: a Trait-Adaptive Neural IRT Scorer that incorporates the Graded Partial Credit Model (GPCM) into a neural architecture, enabling the precise estimation of student ability while maintaining psychometric interpretability, a ZPD-Scaffolded Feedback Generator, which conditions multi-agent feedback strategies on the diagnosed ability parameter to adapt instructional focus across different proficiency levels, and a Multi-Perspective Feedback Evaluation Strategy that assesses feedback quality via pairwise preference judgements and student revision simulations. Experiments on the ASAP++ dataset demonstrate that PsyScore achieves competitive scoring performance while providing more pedagogically aligned feedback.

Figures

Figures reproduced from arXiv: 2606.20287 by the authors.

Figure 1
Figure 1. Comparison of traditional approaches and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the PsyScore framework. (a) Trait-Adaptive GPCM Scorer estimates the student’s latent ability (θ) and outputs a diagnostic vector (Dx). (b) ZPD-Conditional Feedback Generator synthesizes consensus feedback (ff inal) by mapping θ to adaptive strategies via multi-agent fusion. (c) Multi-Perspective Evaluation validates quality via intrinsic LLM-based comparison and extrinsic simulated revision. ability (θ)… view at source ↗
Figure 3
Figure 3. Pairwise preference evaluation results across four baselines. The bars represent the number of wins awarded by judges. PsyScore demonstrates consistent superiority across both open-source (a-c) and closed-source (d) models, particularly in Actionability and Adaptivity [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

170 extracted references · 72 canonical work pages

  1. [1]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  2. [2]

    Publications Manual , year = "1983", publisher =

    1983

  3. [3]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    work page doi:10.1145/322234.322243 1981

  4. [4]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  5. [5]

    Dan Gusfield , title =. 1997

    1997

  6. [6]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  7. [7]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

  8. [8]

    Dual-scale

    Cho, Minsoo and Huang, Jin-Xia and Kwon, Oh-Woog , year =. Dual-scale. ETRI Journal , volume =

Show all 170 references
  1. [10]

    2025 , issn =

    An interpretable polytomous cognitive diagnosis framework for predicting examinee performance , journal =. 2025 , issn =. doi:https://doi.org/10.1016/j.ipm.2024.103913 , url =

    2025 doi

  2. [11]

    2025 , eprint=

    MetaCD: A Meta Learning Framework for Cognitive Diagnosis based on Continual Learning , author=. 2025 , eprint=

    2025

  3. [12]

    2020 , publisher =

    A Trait-based Deep Learning Automated Essay Scoring System with Adaptive Feedback , journal =. 2020 , publisher =. doi:10.14569/IJACSA.2020.0110538 , url =

    2020 doi

  4. [13]

    Unleashing

    Lee, Sanwoo and Cai, Yida and Meng, Desong and Wang, Ziyang and Wu, Yunfang , year =. Unleashing. doi:10.48550/arXiv.2404.04941 , note =

    doi

  5. [15]

    doi:10.48550/ARXIV.2502.11916 , note =

    Su, Jiamin and Yan, Yibo and Fu, Fangteng and Zhang, Han and Ye, Jingheng and Liu, Xiang and Huo, Jiahao and Zhou, Huiyu and Hu, Xuming , year =. doi:10.48550/ARXIV.2502.11916 , note =

    doi

  6. [16]

    T - MES : Trait-Aware Mix-of-Experts Representation Learning for Multi-trait Essay Scoring

    Wang, Jiong and Liu, Jie. T - MES : Trait-Aware Mix-of-Experts Representation Learning for Multi-trait Essay Scoring. Proceedings of the 31st International Conference on Computational Linguistics. 2025

    2025

  7. [17]

    Automated

    Ormerod, Christopher , month =. Automated. 2025 , note =. doi:10.48550/arXiv.2505.22771 , publisher =

    2025 doi

  8. [18]

    2025 , note =

    Jordan, Joaquin and Yin, Xavier and Fabros, Melissa and Ranade, Gireeja and Norouzi, Narges , month =. 2025 , note =. doi:10.48550/arXiv.2506.13037 , publisher =

    2025 doi

  9. [19]

    Zafar, Samra and Minhas, Shaheer and Zaidi, Syed Ali Hassan and Naeem, Arfa and Ali, Zahra , month =. ". 2025 , note =. doi:10.48550/arXiv.2506.08221 , publisher =

    2025 doi

  10. [20]

    doi:10.1145/3726302.3730143 , author =

    2025 , note =. doi:10.1145/3726302.3730143 , author =

    2025 doi

  11. [22]

    Transfer

    Morris, Oscar , month =. Transfer. 2025 , note =. doi:10.48550/arXiv.2503.11836 , publisher =

    2025 doi

  12. [23]

    2025 , note =

    Liu, Zhexiong and Litman, Diane and Wang, Elaine and Li, Tianwen and Gobat, Mason and Matsumura, Lindsay Clare and Correnti, Richard , month =. 2025 , note =. doi:10.48550/arXiv.2501.00715 , publisher =

    2025 doi

  13. [24]

    Advancing

    Zeinalipour, Kamyar and Mehak, Mehak and Parsamotamed, Fatemeh and Maggini, Marco and Gori, Marco , month =. Advancing. 2025 , note =. doi:10.48550/arXiv.2501.07740 , publisher =

    2025 doi

  14. [25]

    2024 , note =

    Kim, Minsun and Kim, SeonGyeom and Lee, Suyoun and Yoon, Yoosang and Myung, Junho and Yoo, Haneul and Lim, Hyunseung and Han, Jieun and Kim, Yoonsu and Ahn, So-Yeon and Kim, Juho and Oh, Alice and Hong, Hwajung and Lee, Tak Yeon , month =. 2024 , note =. doi:10.48550/arXiv.241...

    2024 doi

  15. [26]

    Wang, Yupei and Hu, Renfen and Zhao, Zhe , month =. Beyond. 2024 , note =. doi:10.48550/arXiv.2405.19433 , publisher =

    2024 doi

  16. [27]

    Investigating

    Katuka, Gloria Ashiya and Gain, Alexander and Yu, Yen-Yun , month =. Investigating. 2024 , note =. doi:10.48550/arXiv.2405.00602 , publisher =

    2024 doi

  17. [28]

    , month =

    Karizaki, Mahsa Sheikhi and Gnesdilow, Dana and Puntambekar, Sadhana and Passonneau, Rebecca J. , month =. How. 2024 , note =. doi:10.48550/arXiv.2404.11682 , publisher =

    2024 doi

  18. [29]

    Wang, Izia Xiaoxiao and Wu, Xihan and Coates, Edith and Zeng, Min and Kuang, Jiexin and Liu, Siliang and Qiu, Mengyang and Park, Jungyeul , month =. Neural. 2024 , note =. doi:10.48550/arXiv.2402.17613 , publisher =

    2024 doi

  19. [30]

    , month =

    Yoon, Su-Youn and Miszoglad, Eva and Pierce, Lisa R. , month =. Evaluation of. 2023 , note =. doi:10.48550/arXiv.2310.06505 , publisher =

    2023 doi

  20. [31]

    2023 , note =

    Solopova, Veronika and Gruszczynski, Adrian and Rostom, Eiad and Cremer, Fritz and Witte, Sascha and Zhang, Chengming and Plößl, Fernando Ramos López Lea and Hofmann, Florian and Romeike, Ralf and Gläser-Zikuda, Michaela and Benzmüller, Christoph and Landgraf, Tim , month =. 2...

    2023 doi

  21. [32]

    Review of feedback in

    Jong, You-Jin and Kim, Yong-Jin and Ri, Ok-Chol , month =. Review of feedback in. 2023 , note =. doi:10.48550/arXiv.2307.05553 , publisher =

    2023 doi

  22. [33]

    Predicting the

    Liu, Zhexiong and Litman, Diane and Wang, Elaine and Matsumura, Lindsay and Correnti, Richard , month =. Predicting the. 2023 , note =. doi:10.48550/arXiv.2306.00667 , publisher =

    2023 doi

  23. [34]

    British Journal of Educational Technology , author =

    Practical and. British Journal of Educational Technology , author =. 2024 , note =. doi:10.1111/bjet.13370 , number =

    2024 doi

  24. [35]

    Transformer

    Abhishek, Tushar and Rawat, Daksh and Gupta, Manish and Varma, Vasudeva , month =. Transformer. 2022 , note =. doi:10.48550/arXiv.2109.02176 , publisher =

    2022 doi

  25. [36]

    Effective

    Afrin, Tazin and Kashefi, Omid and Olshefski, Christopher and Litman, Diane and Hwa, Rebecca and Godley, Amanda , month =. Effective. 2021 , note =. doi:10.1145/3411764.3445683 , booktitle =

    2021 doi

  26. [37]

    Hong, Shengxin and Cai, Chang and Du, Sixuan and Feng, Haiyue and Liu, Siyuan and Fan, Xiuyi , month =. ". 2024 , note =. doi:10.48550/arXiv.2409.07453 , publisher =

    2024 doi

  27. [38]

    2025 , note =

    Shibata, Takumi and Miyamura, Yuichi , month =. 2025 , note =. doi:10.48550/arXiv.2505.08498 , publisher =

    2025 doi

  28. [39]

    Operationalizing

    Plasencia-Calaña, Yenisel , month =. Operationalizing. 2025 , note =. doi:10.48550/arXiv.2506.21603 , publisher =

    2025 doi

  29. [40]

    Yoshida, Lui , month =. Do. 2025 , note =. doi:10.48550/arXiv.2505.01035 , publisher =

    2025 doi

  30. [41]

    and Kwong, Theresa and Atif, Amara , month =

    Kamalov, Firuz and Calonge, David Santandreu and Smail, Linda and Azizov, Dilshod and Thadani, Dimple R. and Kwong, Theresa and Atif, Amara , month =. Evolution of. 2025 , note =. doi:10.48550/arXiv.2504.20082 , publisher =

    2025 doi

  31. [42]

    Cai, Yida and Liang, Kun and Lee, Sanwoo and Wang, Qinghan and Wu, Yunfang , month =. Rank-. 2025 , note =. doi:10.48550/arXiv.2504.05736 , publisher =

    2025 doi

  32. [43]

    and Yang, Yi and Abbasi, Ahmed , month =

    Oketch, Kezia and Lalor, John P. and Yang, Yi and Abbasi, Ahmed , month =. Bridging the. 2025 , note =. doi:10.48550/arXiv.2503.11827 , publisher =

    2025 doi

  33. [44]

    Teach-to-

    Do, Heejin and Ryu, Sangwon and Lee, Gary Geunbae , month =. Teach-to-. 2025 , note =. doi:10.48550/arXiv.2502.20748 , publisher =

    2025 doi

  34. [45]

    How well can

    Ghazawi, Rayed and Simpson, Edwin , month =. How well can. 2025 , note =. doi:10.48550/arXiv.2501.16516 , publisher =

    2025 doi

  35. [46]

    Wendlinger, Lorenz and Braun, Christian and Zubaer, Abdullah Al and Nonn, Simon Alexander and Großkopf, Sarah and Fellicious, Christofer and Granitzer, Michael , month =. On the. 2024 , note =. doi:10.48550/arXiv.2412.15902 , publisher =

    2024 doi

  36. [47]

    Evaluating

    Zhong, Yang and Hao, Jiangang and Fauss, Michael and Li, Chen and Wang, Yuan , month =. Evaluating. 2024 , note =. doi:10.48550/arXiv.2410.17439 , publisher =

    2024 doi

  37. [48]

    Kundu, Anindita and Barbosa, Denilson , month =. Are. 2024 , note =. doi:10.48550/arXiv.2409.13120 , publisher =

    2024 doi

  38. [49]

    Kim, Seungju and Jo, Meounggun , month =. Is. 2024 , note =. doi:10.1145/3657604.3664703 , booktitle =

    2024 doi

  39. [50]

    Attention-based

    Dong, Fei and Zhang, Yue and Yang, Jie , year =. Attention-based. doi:10.18653/v1/K17-1017 , booktitle =

    doi

  40. [51]

    Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays , booktitle =

    Rahul Kumar and Sandeep Mathias and Sriparna Saha and Pushpak Bhattacharyya , editor =. Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays , booktitle =. 2022 , url =

    2022

  41. [52]

    Automated essay scoring with string kernels and word embeddings

    Cozma, M a d a lina and Butnaru, Andrei and Ionescu, Radu Tudor. Automated essay scoring with string kernels and word embeddings. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 2018. doi:10.18653/v1/P18-2080

    2018 doi

  42. [53]

    2023 , doi =

    Document. 2023 , doi =

    2023

  43. [54]

    Linking essay-writing tests using many-facet models and neural automated essay scoring , url =

    Uto, Masaki and Aramaki, Kota , month =. Linking essay-writing tests using many-facet models and neural automated essay scoring , url =. doi:10.3758/s13428-024-02485-2 , journal =

    doi

  44. [55]

    Uto, Masaki and Takahashi, Yuto , editor =. Neural. Artificial. 2024 , doi =

    2024

  45. [56]

    Difficulty-

    Tomikawa, Yuto and Uto, Masaki , editor =. Difficulty-. Artificial. 2024 , doi =

    2024

  46. [57]

    Artificial

    Shindo, Naoki and Uto, Masaki , editor =. Artificial. 2024 , doi =

    2024

  47. [58]

    Behavior Research Methods , author =

    A. Behavior Research Methods , author =. 2022 , pages =. doi:10.3758/s13428-022-01997-z , number =

    2022 doi

  48. [59]

    Yamaura, Misato and Fukuda, Itsuki and Uto, Masaki , editor =. Neural. Artificial. 2023 , doi =

    2023

  49. [60]

    Behavior Research Methods , author =

    Accuracy of performance-test linking based on a many-facet. Behavior Research Methods , author =. 2021 , pages =. doi:10.3758/s13428-020-01498-x , number =

    2021 doi

  50. [61]

    Behaviormetrika , author =

    Special issue: e-testing from artificial intelligence approach , volume =. Behaviormetrika , author =. 2021 , pages =. doi:10.1007/s41237-021-00146-8 , number =

    2021 doi

  51. [62]

    Behaviormetrika , author =

    A multidimensional generalized many-facet. Behaviormetrika , author =. 2021 , pages =. doi:10.1007/s41237-021-00144-w , number =

    2021 doi

  52. [63]

    Behaviormetrika , author =

    A review of deep-neural automated essay scoring models , volume =. Behaviormetrika , author =. 2021 , pages =. doi:10.1007/s41237-021-00142-y , number =

    2021 doi

  53. [64]

    Integration of

    Aomi, Itsuki and Tsutsumi, Emiko and Uto, Masaki and Ueno, Maomi , editor =. Integration of. Artificial. 2021 , doi =

    2021

  54. [65]

    Uto, Masaki , editor =. A. Artificial. 2021 , doi =

    2021

  55. [66]

    Estimating

    Nakayama, Minoru and Sciarrone, Filippo and Uto, Masaki and Temperini, Marco , editor =. Estimating. Methodologies and. 2021 , doi =

    2021

  56. [67]

    Behaviormetrika , author =

    A generalized many-facet. Behaviormetrika , author =. 2020 , pages =. doi:10.1007/s41237-020-00115-7 , number =

    2020 doi

  57. [68]

    International Journal of Artificial Intelligence in Education , author =

    Time- and. International Journal of Artificial Intelligence in Education , author =. 2020 , pages =. doi:10.1007/s40593-019-00189-9 , number =

    2020 doi

  58. [69]

    Automated

    Uto, Masaki and Uchida, Yuto , editor =. Automated. Artificial. 2020 , doi =

    2020

  59. [70]

    Uto, Masaki and Okano, Masashi , editor =. Robust. Artificial. 2020 , doi =

    2020

  60. [71]

    Uto, Masaki , editor =. Rater-. Artificial. 2019 , doi =

    2019

  61. [72]

    Social constructivist approach of motivation: social media messages recommendation system , url =

    Louvigné, Sébastien and Uto, Masaki and Kato, Yoshihiro and Ishii, Takatoshi , month =. Social constructivist approach of motivation: social media messages recommendation system , url =. doi:10.1007/s41237-017-0043-7 , journal =

    doi

  62. [73]

    Uto, Masaki and Ueno, Maomi , editor =. Item. Artificial. 2018 , doi =

    2018

  63. [74]

    and Yancey, Kevin and von Davier, Alina A

    Burstein, Jill and LaFlair, Geoffrey T. and Yancey, Kevin and von Davier, Alina A. and Dotan, Ravit , year =. Responsible. doi:10.48550/ARXIV.2409.07476 , publisher =

    doi

  64. [75]

    Collaborative

    Aramaki, Kota and Uto, Masaki , editor =. Collaborative. Artificial. 2024 , doi =

    2024

  65. [76]

    Ridley, Robert and He, Liang and Dai, Xinyu and Huang, Shujian and Chen, Jiajun , month =. Prompt. 2020 , note =

    2020

  66. [77]

    Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024) , month =

    Automated Essay Scoring Using Grammatical Variety and Errors with Multi-Task Learning and Item Response Theory , author =. Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024) , month =. 2024 , address =

    2024

  67. [78]

    IEEE Transactions on Learning Technologies , author =

    Integration of. IEEE Transactions on Learning Technologies , author =. 2023 , pages =. doi:10.1109/TLT.2023.3253215 , number =

    2023 doi

  68. [79]

    IEEE Transactions on Learning Technologies , author =

    Learning. IEEE Transactions on Learning Technologies , author =. 2021 , pages =. doi:10.1109/TLT.2022.3145352 , number =

    2021 doi

  69. [80]

    Analytic

    Shibata, Takumi and Uto, Masaki , year =. Analytic

  70. [81]

    doi:https://doi.org/10.1016/j.eswa.2023.123043 , journal =

    Liu, Yuanchao and Han, Jiawei and Sboev, Alexander and Makarov, Ilya , year =. doi:https://doi.org/10.1016/j.eswa.2023.123043 , journal =

    2023 doi

  71. [82]

    1990 , publisher =

    Item Response Theory , author =. 1990 , publisher =

    1990

  72. [83]

    1991 , publisher =

    Fundamentals of Item Response Theory , author =. 1991 , publisher =

    1991

  73. [84]

    and Fayers, Peter M

    Reeve, Bryce B. and Fayers, Peter M. , editor =. Applying Item Response Theory Modelling for Evaluating Questionnaire Item and Scale Properties , booktitle =. 2005 , publisher =

    2005

  74. [85]

    2016 , publisher =

    Handbook of Item Response Theory , author =. 2016 , publisher =

    2016

  75. [86]

    The Computer Journal , author =

    Automated. The Computer Journal , author =. 2014 , pages =. doi:10.1093/comjnl/bxt117 , number =

    2014 doi

  76. [87]

    Taghipour, Kaveh and Ng, Hwee Tou , year =. A. doi:10.18653/v1/D16-1193 , booktitle =

    doi

  77. [88]

    Automatic

    Dong, Fei and Zhang, Yue , year =. Automatic. doi:10.18653/v1/D16-1115 , booktitle =

    doi

  78. [89]

    and Ionescu, Radu Tudor , month =

    Cozma, Mădălina and Butnaru, Andrei M. and Ionescu, Radu Tudor , month =. Automated essay scoring with string kernels and word embeddings , url =. 2018 , note =

    2018

  79. [90]

    Psych , author =

    Automated. Psych , author =. 2021 , pages =. doi:10.3390/psych3040056 , number =

    2021 doi

  80. [91]

    and Crossley, Scott A

    McNamara, Danielle S. and Crossley, Scott A. and Roscoe, Rod D. and Allen, Laura K. and Dai, Jianmin , month =. A hierarchical classification approach to automated essay scoring , volume =. 2015 , pages =. doi:10.1016/j.asw.2014.09.002 , journal =

    2015 doi

  81. [92]

    and Bau, David and Yuan, Ben Z

    Gilpin, Leilani H. and Bau, David and Yuan, Ben Z. and Bajwa, Ayesha and Specter, Michael and Kagal, Lalana , month =. Explaining. 2018 , pages =. doi:10.1109/DSAA.2018.00018 , booktitle =

    2018 doi

  82. [93]

    Proceedings of the National Academy of Sciences , author =

    Definitions, methods, and applications in interpretable machine learning , volume =. Proceedings of the National Academy of Sciences , author =. 2019 , pages =. doi:10.1073/pnas.1900654116 , number =

    2019 doi

  83. [94]

    ACM Computing Surveys , author =

    A. ACM Computing Surveys , author =. 2019 , pages =. doi:10.1145/3236009 , number =

    2019 doi

  84. [95]

    EMNLP , author =

    Modeling. EMNLP , author =. 2010 , file =

    2010

  85. [96]

    2018 , publisher =

    Mathias, Sandeep and Bhattacharyya, Pushpak , booktitle =. 2018 , publisher =

    2018

  86. [97]

    2023 , volume=

    Designing feedback activities to help low-performing students , author=. 2023 , volume=

    2023

  87. [98]

    International Journal of Learning, Teaching and Educational Research , year=

    Teachers’ Feedback Practice and Students’ Academic Achievement: A Systematic Literature Review , author=. International Journal of Learning, Teaching and Educational Research , year=

  88. [99]

    Assessing Writing , year=

    Fostering student engagement with feedback: An integrated approach , author=. Assessing Writing , year=

  89. [100]

    Applied Sciences , year=

    Large Language Models as Evaluators in Education: Verification of Feedback Consistency and Accuracy , author=. Applied Sciences , year=

  90. [101]

    Transactions of the Association for Computational Linguistics , year=

    State of What Art? A Call for Multi-Prompt LLM Evaluation , author=. Transactions of the Association for Computational Linguistics , year=

  91. [102]

    2024 , pages=

    A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations , author=. 2024 , pages=

    2024

  92. [103]

    2024 , pages=

    Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives , author=. 2024 , pages=

    2024

  93. [104]

    IEEE Transactions on Emerging Topics in Computational Intelligence , year=

    A Survey on Neural Network Interpretability , author=. IEEE Transactions on Emerging Topics in Computational Intelligence , year=

  94. [105]

    IEEE Access , year=

    Interpretability-Aware Industrial Anomaly Detection Using Autoencoders , author=. IEEE Access , year=

  95. [106]

    Psychometrika , volume=

    Estimation of latent ability using a response pattern of graded scores , author=. Psychometrika , volume=. 1969 , publisher=

    1969

  96. [107]

    ETS Research Report Series , volume=

    A generalized partial credit model: Application of an EM algorithm , author=. ETS Research Report Series , volume=. 1992 , publisher=

    1992

  97. [108]

    Handbook of item response theory , pages=

    Generalized partial credit model , author=. Handbook of item response theory , pages=. 2016 , publisher=

    2016

  98. [109]

    Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI , journal =

    Alejandro. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI , journal =. 2020 , issn =. doi:https://doi.org/10.1016/j.inffus.2019.12.012 , url =

    2020 doi

  99. [110]

    Rationale Behind Essay Scores: Enhancing S - LLM ' s Multi-Trait Essay Scoring with Rationale Generated by LLM s

    Chu, SeongYeub and Kim, Jong Woo and Wong, Bryan and Yi, Mun Yong. Rationale Behind Essay Scores: Enhancing S - LLM ' s Multi-Trait Essay Scoring with Rationale Generated by LLM s. Findings of the Association for Computational Linguistics: NAACL 2025. 2025. doi:10.18653/v1/202...

    2025 doi

  100. [111]

    BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina. BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Hum...

    2019 doi

  101. [112]

    Autoregressive Score Generation for Multi-trait Essay Scoring

    Do, Heejin and Kim, Yunsu and Lee, Gary. Autoregressive Score Generation for Multi-trait Essay Scoring. Findings of the Association for Computational Linguistics: EACL 2024. 2024

    2024

  102. [113]

    Autoregressive Multi-trait Essay Scoring via Reinforcement Learning with Scoring-aware Multiple Rewards

    Do, Heejin and Ryu, Sangwon and Lee, Gary. Autoregressive Multi-trait Essay Scoring via Reinforcement Learning with Scoring-aware Multiple Rewards. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.917

    2024 doi

  103. [114]

    TDNN : A Two-stage Deep Neural Network for Prompt-independent Automated Essay Scoring

    Jin, Cancan and He, Ben and Hui, Kai and Sun, Le. TDNN : A Two-stage Deep Neural Network for Prompt-independent Automated Essay Scoring. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018. doi:10.18653/v1/P18-1100

    2018 doi

  104. [115]

    e R evise+ RF : A Writing Evaluation System for Assessing Student Essay Revisions and Providing Formative Feedback

    Liu, Zhexiong and Litman, Diane and Wang, Elaine L and Li, Tianwen and Gobat, Mason and Matsumura, Lindsay Clare and Correnti, Richard. e R evise+ RF : A Writing Evaluation System for Assessing Student Essay Revisions and Providing Formative Feedback. Proceedings of the 2025 C...

    2025 doi

  105. [116]

    Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions

    Nair, Inderjeet Jayakumar and Tan, Jiaye and Su, Xiaotian and Gere, Anne and Wang, Xu and Wang, Lu. Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions. Proceedings of the 2024 Conference on Empirical Methods in Natural Langua...

    2024 doi

  106. [117]

    Does Writing with Language Models Reduce Content Diversity? , url =

    Padmakumar, Vishakh and He, He , booktitle =. Does Writing with Language Models Reduce Content Diversity? , url =

  107. [118]

    Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation

    Stahl, Maja and Biermann, Leon and Nehring, Andreas and Wachsmuth, Henning. Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation. Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024). 2024

    2024

  108. [119]

    CoRR , volume =

    Jiamin Su and Yibo Yan and Zhuoran Gao and Han Zhang and Xiang Liu and Xuming Hu , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2505.13965 , eprinttype =. 2505.13965 , timestamp =

    2025 doi

  109. [120]

    Beyond Agreement: Diagnosing the Rationale Alignment of Automated Essay Scoring Methods based on Linguistically-informed Counterfactuals

    Wang, Yupei and Hu, Renfen and Zhao, Zhe. Beyond Agreement: Diagnosing the Rationale Alignment of Automated Essay Scoring Methods based on Linguistically-informed Counterfactuals. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024. doi:10.18653/v1/2024...

    2024 doi

  110. [121]

    Proceedings of the 15th International Learning Analytics and Knowledge Conference , pages =

    Xiao, Changrong and Ma, Wenxing and Song, Qingping and Xu, Sean Xin and Zhang, Kunpeng and Wang, Yufang and Fu, Qi , title =. Proceedings of the 15th International Learning Analytics and Knowledge Conference , pages =. 2025 , isbn =. doi:10.1145/3706468.3706507 , abstract =

    2025 doi

  111. [122]

    Does the Prompt-Based Large Language Model Recognize Students' Demographics and Introduce Bias in Essay Scoring?

    Yang, Kaixun and Rakovi \' c , Mladen and Ga s evi \' c , Dragan and Chen, Guanliang. Does the Prompt-Based Large Language Model Recognize Students' Demographics and Introduce Bias in Essay Scoring?. Artificial Intelligence in Education. 2025

    2025

  112. [123]

    eRevis(ing): Students’ revision of text evidence use in an automated writing evaluation system , journal =

    Elaine Lin Wang and Lindsay Clare Matsumura and Richard Correnti and Diane Litman and Haoran Zhang and Emily Howe and Ahmed Magooda and Rafael Quintana , keywords =. eRevis(ing): Students’ revision of text evidence use in an automated writing evaluation system , journal =. 202...

    2020 doi

  113. [124]

    Learning to work with the black box: Pedagogy for a world with artificial intelligence , author=. Br. J. Educ. Technol. , year=

  114. [125]

    IEEE Access , year=

    A Systematic Review of Pretrained Models in Automated Essay Scoring , author=. IEEE Access , year=

  115. [126]

    Mathematics , volume=

    Hybrid approach to automated essay scoring: Integrating deep learning embeddings with handcrafted linguistic features for improved accuracy , author=. Mathematics , volume=. 2024 , publisher=

    2024

  116. [127]

    Frontiers in education , volume=

    Explainable automated essay scoring: Deep learning really has pedagogical value , author=. Frontiers in education , volume=. 2020 , organization=

    2020

  117. [128]

    Educational psychology review , volume=

    Formative assessment: Assessment is for self-regulated learning , author=. Educational psychology review , volume=. 2012 , publisher=

    2012

  118. [129]

    European Journal of Engineering Education , volume=

    Formative feedback and scaffolding for developing complex problem solving and modelling outcomes , author=. European Journal of Engineering Education , volume=. 2018 , publisher=

    2018

  119. [130]

    2024 , school=

    AI-Driven Assessment and Scaffolding for Mathematical Modeling and Explanations During Science Investigations , author=. 2024 , school=

    2024

  120. [131]

    South African Journal of Higher Education , volume=

    The use of an online learning platform: A step towards e-learning , author=. South African Journal of Higher Education , volume=. 2021 , publisher=

    2021

  121. [132]

    Education and Information Technologies , pages=

    A comparative study on sustainable development of online education platforms at home and abroad since the twenty-first century based on big data analysis , author=. Education and Information Technologies , pages=. 2025 , publisher=

    2025

  122. [133]

    Chinese/English Journal of Educational Measurement and Evaluation , year=

    An Early Review of Generative Language Models in Automated Writing Evaluation: Advancements, Challenges, and Future Directions for Automated Essay Scoring and Feedback Generation , author=. Chinese/English Journal of Educational Measurement and Evaluation , year=

  123. [134]

    Artificial Intelligence Review , year=

    An automated essay scoring systems: a systematic literature review , author=. Artificial Intelligence Review , year=

  124. [135]

    Artificial Intelligence Review , year=

    A survey on deep learning-based automated essay scoring and feedback generation , author=. Artificial Intelligence Review , year=

  125. [136]

    Journal of Educational Computing Research , year=

    A Multi-Strategy Computer-Assisted EFL Writing Learning System With Deep Learning Incorporated and Its Effects on Learning: A Writing Feedback Perspective , author=. Journal of Educational Computing Research , year=

  126. [137]

    Vygotsky’s educational theory in cultural context , volume=

    The zone of proximal development in Vygotsky’s analysis of learning and instruction , author=. Vygotsky’s educational theory in cultural context , volume=. 2003 , publisher=

    2003

  127. [138]

    AI , year=

    The Promises and Pitfalls of Large Language Models as Feedback Providers: A Study of Prompt Engineering and the Quality of AI-Driven Feedback , author=. AI , year=

  128. [139]

    Behaviormetrika , year=

    A review of deep-neural automated essay scoring models , author=. Behaviormetrika , year=

  129. [140]

    PeerJ Computer Science , year=

    Automated language essay scoring systems: a literature review , author=. PeerJ Computer Science , year=

  130. [141]

    Language Testing , year=

    More efficient processes for creating automated essay scoring frameworks: A demonstration of two algorithms , author=. Language Testing , year=

  131. [142]

    ArXiv , year=

    Deep Learning Architecture for Automatic Essay Scoring , author=. ArXiv , year=

  132. [143]

    International Journal of Advanced Computer Science and Applications , year=

    A Trait-based Deep Learning Automated Essay Scoring System with Adaptive Feedback , author=. International Journal of Advanced Computer Science and Applications , year=

  133. [144]

    Artificial Intelligence in Education , year=

    Automated Short-Answer Grading Using Deep Neural Networks and Item Response Theory , author=. Artificial Intelligence in Education , year=

  134. [145]

    Psych , volume=

    Automated Essay Scoring Using Transformer Models , author=. Psych , volume=. 2021 , doi=

    2021

  135. [146]

    Proceedings of the 29th International Conference on Computational Linguistics (COLING) , pages=

    Analytic Automated Essay Scoring Based on Deep Neural Networks Integrating Multidimensional Item Response Theory , author=. Proceedings of the 29th International Conference on Computational Linguistics (COLING) , pages=

  136. [147]

    Choi , journal =

    Haile Misgna and Byung-Won On and Ingyu Lee and G. Choi , journal =. A survey on deep learning-based automated essay scoring and feedback generation , volume =

  137. [148]

    Palermo and Ruitao Liu and Yong He , journal =

    Yue Huang and C. Palermo and Ruitao Liu and Yong He , journal =. An Early Review of Generative Language Models in Automated Writing Evaluation: Advancements, Challenges, and Future Directions for Automated Essay Scoring and Feedback Generation , year =

  138. [149]

    and Jang E

    Hannah L. and Jang E. E. and Shah M. and Gupta V. , issn =. Validity Arguments for Automated Essay Scoring of Young Students’ Writing Traits , volume =. Language Assessment Quarterly , number =

  139. [150]

    Comparison of traditional machine learning and neural network approaches for automated scoring of second language English essays , volume =

    Erik Voss , journal =. Comparison of traditional machine learning and neural network approaches for automated scoring of second language English essays , volume =

  140. [151]

    Fairness in Automated Essay Scoring: A Comparative Analysis of Algorithms on German Learner Essays from Secondary Education , year =

    Nils-Jonathan Schaller and Yuning Ding and Andrea Horbach and Jennifer Meyer and Thorben Jansen , pages =. Fairness in Automated Essay Scoring: A Comparative Analysis of Algorithms on German Learner Essays from Secondary Education , year =

  141. [152]

    Gierl , pages =

    Jinnie Shin and Qi Guo and Mark J. Gierl , pages =. Automated Essay Scoring Using Deep Learning Algorithms , year =

  142. [153]

    Holistic and analytic assessments of the TOEFL iBT

    ONO, Masumi and YAMANISHI, Hiroyuki and HIJIKATA, Yuko , journal =. Holistic and analytic assessments of the TOEFL iBT

  143. [154]

    Imbler and S

    Angenette C. Imbler and S. Clark and T. Young and Erika Feinauer , journal =. Teaching second-grade students to write science expository text: Does a holistic or analytic rubric provide more meaningful results? , year =

  144. [155]

    A Multi-Strategy Computer-Assisted EFL Writing Learning System With Deep Learning Incorporated and Its Effects on Learning: A Writing Feedback Perspective , volume =

    Chen Binbin and Bao Lina and Zhang Rui and Zhang Jingyu and Liu Feng and Wang Shuai and Li Mingjiang , issn =. A Multi-Strategy Computer-Assisted EFL Writing Learning System With Deep Learning Incorporated and Its Effects on Learning: A Writing Feedback Perspective , volume =....

  145. [156]

    Exploring

    Stahl, Maja and Biermann, Leon and Nehring, Andreas and Wachsmuth, Henning , language =. Exploring

  146. [157]

    P. G. Policar and Martin Špendl and Tomaž Curk and Blaz Zupan , journal =. Automated assignment grading with large language models: insights from a bioinformatics course , volume =

  147. [158]

    Jacobsen and Kira Elena Weber , journal =

    L. Jacobsen and Kira Elena Weber , journal =. The Promises and Pitfalls of Large Language Models as Feedback Providers: A Study of Prompt Engineering and the Quality of AI-Driven Feedback , year =

  148. [159]

    Does the Prompt-Based Large Language Model Recognize Students' Demographics and Introduce Bias in Essay Scoring? , year =

    Yang, Kaixun and Rakovi. Does the Prompt-Based Large Language Model Recognize Students' Demographics and Introduce Bias in Essay Scoring? , year =. Artificial Intelligence in Education , editor =

  149. [160]

    Analytic automated essay scoring based on deep neural networks integrating multidimensional item response theory , year =

    Shibata, Takumi and Uto, Masaki , booktitle =. Analytic automated essay scoring based on deep neural networks integrating multidimensional item response theory , year =

  150. [161]

    Automated Essay Scoring Using Grammatical Variety and Errors with Multi-Task Learning and Item Response Theory , year =

    Doi, Kosuke and Sudoh, Katsuhito and Nakamura, Satoshi , booktitle =. Automated Essay Scoring Using Grammatical Variety and Errors with Multi-Task Learning and Item Response Theory , year =

  151. [162]

    SMART : Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction

    Scarlatos, Alexander and Fernandez, Nigel and Ormerod, Christopher and Lottridge, Susan and Lan, Andrew. SMART : Simulated Students Aligned with Item Response Theory for Question Difficulty Prediction. Proceedings of the 2025 Conference on Empirical Methods in Natural Language...

    2025 doi

  152. [163]

    1978 , publisher=

    Mind in society: The development of higher psychological processes , author=. 1978 , publisher=

    1978

  153. [164]

    , author=

    The role of tutoring in problem solving. , author=. Journal of child psychology and psychiatry, and allied disciplines , year=

  154. [165]

    The power of feedback , volume =

    Hattie, John and Timperley, Helen , journal =. The power of feedback , volume =

  155. [166]

    Review of educational research , volume=

    Focus on formative feedback , author=. Review of educational research , volume=. 2008 , publisher=

    2008

  156. [167]

    2001 , publisher=

    The basics of item response theory , author=. 2001 , publisher=

    2001

  157. [168]

    1987 , publisher=

    Politeness: Some universals in language usage , author=. 1987 , publisher=

    1987

  158. [169]

    IFlyEA: A Chinese essay assessment system with automated rating, review generation, and recommendation , author=. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing:...

  159. [170]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    CEAES: Bidirectional Reinforcement Learning Optimization for Consistent and Explainable Essay Assessment , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  160. [171]

    Cognitive science , volume=

    Cognitive load during problem solving: Effects on learning , author=. Cognitive science , volume=. 1988 , publisher=

    1988

  161. [172]

    Educational researcher , volume=

    Those who understand: Knowledge growth in teaching , author=. Educational researcher , volume=. 1986 , publisher=

    1986

  162. [173]

    1982 , publisher=

    Principles and practice in second language acquisition , author=. 1982 , publisher=

    1982

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.