FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedback

Jongwoo Kim; Munyong Yi; Seongyeub Chu

REVIEW 3 major objections 5 minor 8 references

FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedback

T0 review · 3 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper tries to establish that unvalidated LLM-generated essay feedback is a hidden source of noise, and that a three-dimension small-model filter can replace costly expert checks.

desk verdict FeedEval's human-alignment results are real, but the downstream filtering benefit is not yet established because the key comparison lacks an unfiltered baseline. read the letter →

arxiv 2601.04574 v2 pith:SYWY57NQ submitted 2026-01-08 cs.CL

Seongyeub Chu , Jongwoo Kim , Munyong Yi This is my paper · ORCID

classification cs.CL

keywords FeedEvalautomatedessayscoringfeedbackevaluationspecificityhelpfulnessvalidityLLM-as-judgerevision

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the main bottleneck in using LLM-generated essay feedback is not generation but validation: synthetic feedback is often added to training data without checking whether it refers to the essay, gives actionable guidance, and agrees with the scoring rubric. It proposes FeedEval, three small fine-tuned evaluators that score feedback on specificity, helpfulness, and validity, then pick the best of several candidates. The central claim is that this filter produces feedback that human teachers judge better than GPT-5.1's or Gemini-2.5-Pro's direct judgments, and that training essay-scoring models on the filtered feedback yields more accurate scoring and more effective student revisions. If true, this gives a cheap, reproducible substitute for expert annotation of feedback labels.

What carries the argument

Three dimension-specific fine-tuned LLM evaluators: specificity and helpfulness are trained as pairwise reward models, and validity is trained as a natural-language-inference model that treats rubric score descriptions as the premise and feedback as the hypothesis. Algorithm 1 selects the feedback candidate with the highest equal-weighted average of softmax-normalized scores.

What would settle it

Take a set of essays with expert-validated high-quality feedback and compute its probability of entailing the rubric text of the true score. If a substantial fraction of clearly valid feedback scores no better than random or contradicts a neighboring score level, the validity evaluator's premise is wrong. Alternatively, retrain the framework without the validity dimension; if scoring and revision gains are unchanged, validity is not load-bearing.

Watch

Extended reading notes

Core claim

FeedEval turns feedback-quality evaluation into three supervised scoring tasks. Specialized small-LLM evaluators are trained on a new SpecEval specificity dataset, adapted helpfulness pairwise data, and an NLI-style validity dataset built from rubric/feedback pairs, and combined through equal-weight softmax selection (Algorithm 1). The paper's central finding is that this filter selects feedback that human teachers judge better than unfiltered candidates, and that using the filtered high-quality feedback as training labels for 8B essay scorers improves multi-trait scoring (quadratic weighted kappa) over low-quality labels, while revisions guided by high-quality feedback produce larger score

Load-bearing premise

The load-bearing premise is that valid feedback is exactly feedback entailed by the rubric description for the essay's true score, so pairing the same feedback with a different score level always marks it as a contradiction.

Editorial extensions

If this is right

Using FeedEval-filtered high-quality feedback as training labels improves multi-trait essay scoring over low-quality labels: average gains of 3.22% (Llama3-8B) and 1.58% (Qwen3-8B) on ASAP++, with gains growing as more dimensions are included.
Revisions by small LLMs guided by FeedEval-high feedback receive higher automated scores than low-feedback revisions, and human experts preferred the high-feedback revisions 70.3% versus 29.7%.
FeedEval's 3B-scale evaluators match teacher judgments better than GPT-5.1 and Gemini-2.5-Pro used directly as judges, e.g., specificity pairwise accuracy 0.820 versus 0.729 for GPT-5.1.
The pattern generalizes to the ASAP-SAS dataset, with average scoring gains of 4.55% (Llama3-8B) and 4.26% (Qwen3-8B) from high- versus low-quality feedback.
Feedback generated by models trained on FeedEval-high labels also produces larger revision gains than feedback from models trained on low-quality labels, suggesting the filter's value propagates through trained systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The validity dimension looks nearly non-discriminative in the final filtered sets: Table 17 reports validity scores of 0.202 (high-quality) versus 0.189 (low-quality), while specificity and helpfulness show large gaps. This suggests most of the downstream gain may come from the first two dimensions, and an ablation that drops validity would test that.
Because validity is defined as entailment from rubric text for the true score, the framework assumes rubric descriptions are complete and unambiguous. The paper's own observation that Organization feedback is hard to distinguish—rubric descriptions are 'broadly defined'—is exactly the failure mode this assumption predicts.
A natural extension is to reuse the SpecEval-style specificity labelling and the NLI validity formulation in other synthetic-data pipelines, such as filtering feedback for other languages or STEM tutoring where rubric-like score descriptions exist.
The revision experiments use small LLMs as student proxies; whether real students revise better with FeedEval-high feedback remains an open, testable question that the paper acknowledges.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: the headline claim—that FeedEval-filtered feedback improves downstream essay scoring—is not actually demonstrated. Table 5 lacks an unfiltered or random-feedback condition, so we can't tell whether gains come from filtering or from adding any feedback at all. The more credible contribution is the human-alignment result: fine-tuned 3B evaluators match teacher pairwise judgments at reasonable accuracy (0.82–0.86 F1) and beat GPT-5.1 and Gemini-2.5-Pro used as direct judges. That part looks real and is worth building on.

The framework is clean: three trained evaluators (specificity reward model on a new 41K SpecEval dataset, helpfulness reward model on adapted datasets, validity NLI on a 99K paired set), softmax normalization, equal-weight fusion, pick best per trait. The writing is clear and the limitations section is honest, though there's an inconsistency: the abstract says code/data are released at a GitHub URL, while the body says they will be released upon acceptance. As of now, the code is not available, which matters for reproducibility.

Soft spots, in proportion. First, the missing baseline is load-bearing. For Llama3-8B, both low- and high-quality feedback conditions beat score-only (0.562/0.580 vs 0.528), so simply adding feedback supervision may explain much of the gain. For Qwen3-8B, the high-minus-low average gain is 0.016 QWK, within the reported fold SDs, and no significance test is reported. Same pattern in the ASAP-SAS appendix. This is fixable: add an arm with randomly sampled feedback from the same candidate pool, and report paired tests across folds.

Second, the validity dimension barely discriminates: Table 17 shows validity scores of 0.202 vs 0.189 for high- vs low-quality feedback, with identical ranges. So the claim that validity drives the downstream effect is weakly supported, even if equal-weight selection still works in practice. The NLI construction (rubric entails feedback) is a reasonable operationalization and does have human alignment evidence, but the 450 pairwise judgments are the only external check and they cover all three dimensions jointly.

Overall, the paper is a solid engineering contribution with a promising dataset and an honest evaluation that falls short of its strongest claim. It deserves a serious referee, not a desk reject. My recommendation: send it to peer review, with the request that the authors add the unfiltered baseline and significance tests before publication.

Referee Report

3 major / 5 minor

Summary. This paper proposes FeedEval, a framework for evaluating LLM-generated essay feedback along three dimensions: specificity, helpfulness, and validity. Dimension-specific 3B-scale reward/NLI models are trained on newly constructed or adapted datasets (SpecEval, RECIPE4U/FEAT/ASAP++ based helpfulness pairs, and a Prometheus-derived NLI set). Given eight GPT-5.1 feedback candidates per essay/trait, FeedEval scores each candidate, softmax-normalizes per dimension, and selects the candidate with the highest equal-weight average. The paper reports (i) human-expert alignment of FeedEval judgments on 450 pairwise comparisons, exceeding GPT-5.1 and Gemini-2.5-Pro; (ii) essay scoring results where Qwen3-8B/Llama3-8B trained with FeedEval-high feedback outperform FeedEval-low and GPT-5.1-high feedback on ASAP++ (with ASAP-SAS in the appendix); and (iii) small-LLM revision experiments plus human ratings indicating high-quality feedback yields more effective revisions.

Significance. The contribution is timely and potentially valuable: if validated, FeedEval offers a cheap, teacher-aligned filter for synthetic feedback, addressing a real bottleneck in training essay-assessment models with LLM-generated rationales. The paper's strengths are its external human-alignment check with three MA-level teachers, the construction and planned release of SpecEval, and evaluation across two backbones and two benchmarks. However, the central downstream claim—that FeedEval's filtering itself improves essay scoring—is not currently established. The experiments lack an unfiltered/random-feedback training arm, and the reported gains are small relative to the five-fold SDs with no significance tests. A second, secondary concern is that the validity dimension appears nearly non-discriminative in the selected datasets. These are fixable with additional experiments and analyses.

major comments (3)

[§5.2.1, Table 5; Appendix E, Table 9] The paper's headline downstream claim is that 'essay scoring models trained with FeedEval-filtered high-quality feedback achieve superior scoring performance.' The design only compares FeedEval-high vs FeedEval-low and FeedEval-high vs GPT-5.1-high; it never trains on feedback sampled from the same candidate pool without quality filtering (e.g., random selection or all candidates). This missing arm is load-bearing. For Llama3-8B, score-only QWK is 0.528, FeedEval-low is 0.562, and FeedEval-high is 0.580; for Qwen3-8B, score-only is 0.689 and FeedEval-high is 0.692. The score-only comparison shows that low-quality feedback also improves over score-only for Llama3, and that the incremental gain of FeedEval-high over score-only for Qwen3 is only 0.3%, well within the reported SD of 0.020. Without an unfiltered/random-feedback arm, the observed advantage can be explained by 'any feedback hel
[§5.2, Tables 5 and 12; §5.2.2, Figure 3; Appendix E, Table 9] No significance testing is reported for any of the scoring comparisons. The paper uses 'consistently outperform,' but the reported differences are small relative to fold-level SDs. Example: Qwen3-8B FeedEval-high vs low average QWK is 0.692 vs 0.676 (delta 0.016, SD 0.020); per-trait deltas range from 0.30% to 5.79%, and most are smaller than the corresponding SDs. In Figure 3, the one-, two-, and three-dimension averages differ by at most 0.009 (0.683, 0.683, 0.692), again within the reported SDs. The ASAP-SAS results in Appendix E, Table 9 show the same pattern. The superiority claim requires significance tests across folds or paired per-prompt tests, or the claims should be tempered.
[§3.4; Table 17] The validity evaluator is trained by treating feedback as entailed by the rubric score description for the essay's true score and as contradicted by a randomly chosen different score level. This assumes (i) valid feedback is exactly feedback inferable from rubric text, and (ii) feedback cannot be consistent with multiple score levels. Both assumptions are strong and unvalidated; teachers in the human study might accept feedback that is not a strict NLI entailment. More importantly, Table 17 shows the validity dimension is nearly non-discriminative in the actual selection: high-quality feedback's mean validity score is 0.202 vs 0.189 for low-quality, with near-identical min/max ranges. Thus Algorithm 1's equal-weight average is essentially a specificity/helpfulness filter, and Figure 3's improvement from adding validity may be noise. Please report validity score distributions by quality,

minor comments (5)

[§5.2.1] The paragraph after Table 5 says 'models trained on high-quality feedback filtered by FeedEval consistently outperform those trained on low-quality feedback across all traits.' This is true for high-vs-low, but vs score-only the Overall trait is worse in both backbones (Llama3: 0.451 vs 0.476; Qwen3: 0.661 vs 0.712). Please clarify that the improvement claim is relative to low-quality feedback, not to score-only, and note that Overall uses 'NAN' feedback.
[§4.2 and Appendix C.1] The human alignment result rests on only 450 pairwise judgments (150 per dimension) from three teachers. This is a genuine external check and the reported inter-rater agreement is good, but the paper should report confidence intervals or per-dimension pair counts so readers can gauge the precision of the pairwise accuracy/F1 numbers.
[Appendix D.2] The training schedule is described as '5 epochs (100 steps)' with batch size 4. For SpecEval (41,730 pairs) and helpfulness (14,158 pairs), 100 steps with batch 4 equals 400 examples, far less than one epoch. Please clarify whether the models are trained on a subsample or whether the epoch count is a typo.
[Abstract and Section 6] The abstract states 'We release our code and curated datasets at [URL]' while Section 6 says 'We will release ... upon accepted.' Please align these statements and ensure the provided repository contains or documents the SpecEval, helpfulness-pair, and validity-pair data if the release claim is to be verifiable.
[§5.3.2; Appendix H.3] Small presentation issues: the heading 'Reivison' is misspelled; Table 15 has a spacing error ('improvereadability'); and Appendix H.3 says formatting-error cases are excluded from QWK, but no analysis is given of whether the error rate differs between high- and low-quality conditions, which could affect the comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: downstream claims are anchored to external expert judgments and independent QWK scoring.

full rationale

FeedEval's derivation chain is not circular. The three evaluators are trained on constructed or adapted datasets (GPT-4o-based SpecEval, student-acceptance/ranking datasets for helpfulness, Prometheus-based NLI for validity), and Section 5.1 tests them against held-out pairwise judgments from three educational experts; the alignment numbers in Table 2 are an external check, not a re-statement of the training objective. Downstream essay-scoring comparisons in Section 5.2 are measured by QWK against human-annotated ASAP++/ASAP-SAS scores, so the outcome is independent of FeedEval's own quality scores. Revision experiments in Section 5.3 use a score-only Qwen3-8B scoring model and human expert pairwise preferences, again external to the filtering step. The self-citations to Chu et al. (2025a,b) are background references and are not load-bearing: the specificity computation is also validated against human experts (footnote 6: accuracy 89.2%, F1 92.0%, kappa 0.72), and the downstream conclusions do not rest on those citations. The absence of an unfiltered-feedback arm in Table 5 is a methodological limitation—it means the paper cannot distinguish filtering effects from the mere presence of feedback supervision—but that is a missing control, not a case where the prediction is equal to the input by construction. No equation or label is defined in terms of the outcome it is used to predict.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The framework rests on several modeling choices that define what 'quality' means: the NLI formulation of validity, the GPT-4o-dependent F1 measure of specificity, and acceptance/ranking proxies for helpfulness. These are not fitted parameters in the usual sense, but they are hand-chosen operationalizations that the central claim inherits. The only hand-set numeric hyperparameters are N=8, margin=0.5, temperature=0.7, and equal fusion weights.

free parameters (4)

Equal fusion weights for three dimensions = 1/3 each
Algorithm 1 averages softmax-normalized specificity, helpfulness, validity scores with equal weights; no learned or validated weighting.
Number of feedback candidates N = 8
Section 4.3: eight candidates per essay/trait via temperature=0.7; selection quality may depend on N, not varied.
Ranking-loss margin m = 0.5
Appendix D.1 sets margin=0.5 for specificity/helpfulness evaluators; standard hyperparameter, not justified by data.
Temperature for candidate generation = 0.7
Section 4.3 and D.3; chosen by hand.

assumptions (6)

domain assumption Validity is operationalized as NLI entailment from the rubric score description of the true score.
Section 3.4; if this premise fails, validity scores are mismeasured.
domain assumption Specificity can be measured as F1 of GPT-4o-extracted essay references (faithfulness and coverage).
Section 3.2.1; GPT-4o extraction and the F1 formula are assumed to capture pedagogical specificity.
domain assumption Feedback accepted/used by students (RECIPE4U), ranked by annotators (FEAT), or rewritten by GPT-5.1 over human feedback (ASAP++) is a valid proxy for helpfulness.
Appendix B.1; chosen-rejected pair construction rests on these proxies.
domain assumption Small LLMs revising essays are a valid proxy for student revision behavior.
Section 4.5; the authors acknowledge real-student validation is still needed.
domain assumption ASAP++/ASAP-SAS human scores and rubric descriptions are reliable ground truth.
Used without independent verification throughout the experiments.
ad hoc to paper Generating feedback with the true human score in the prompt (Score+Rubric) is an acceptable setup for studying feedback quality.
Section 4.3 adopts Score+Rubric after Table 4; this introduces the true score into candidate generation and may affect downstream distillation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedback." pith.science (2026). https://pith.science/paper/SYWY57NQ

@misc{pith2026260104574,
  author       = {Pith},
  title        = {Pith review of: FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SYWY57NQ}},
  note         = {Machine review of arXiv:2601.04574}
}

read the original abstract

Going beyond the prediction of numerical scores, recent research in automated essay scoring has increasingly emphasized the generation of high-quality feedback that provides justification and actionable guidance. To mitigate the high cost of expert annotation, prior work has commonly relied on LLM-generated feedback to train essay assessment models. However, such feedback is often incorporated without explicit quality validation, resulting in the propagation of noise in downstream applications. To address this limitation, we propose FeedEval, an LLM-based framework for evaluating LLM-generated essay feedback along three pedagogically grounded dimensions: specificity, helpfulness, and validity. FeedEval employs dimension-specialized LLM evaluators trained on datasets curated in this study to assess multiple feedback candidates and select high-quality feedback for downstream use. Experiments on the ASAP++ benchmark show that FeedEval closely aligns with human expert judgments and that essay scoring models trained with FeedEval-filtered high-quality feedback achieve superior scoring performance. Furthermore, revision experiments using small LLMs show that the high-quality feedback identified by FeedEval leads to more effective essay revisions. We release our code and curated datasets at: https://github.com/BBeeChu/FeedEval.git.

Figures

Figures reproduced from arXiv: 2601.04574 by the authors.

**Figure 2.** Overview of the proposed FeedEval frame [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Average essay scoring performance across [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (7 more)

**Figure 4.** Figure 4: Average essay score improvement across traits [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Prompt template for feedback generation using both human-annotated [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 6.** Figure 6: Prompt template for feedback generation using human-annotated [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: Prompt template for feedback generation using the [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: Prompt template for filtering essay feedback by using GPT-5.1. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

**Figure 9.** Figure 9: Average essay score improvement across prompts on ASAP-SAS after revisions guided by feedback of high- and low-quality identified by FeedEval and GPT-5.1 [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]

**Figure 10.** Figure 10: Average essay score improvement across [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 4 linked inside Pith

[1]

The feedback {should/should not} quote parts of the essay that are relevant to evaluating the given traits
[2]

Ronen Eldan and Yuanzhi Li

Teach-to-reason with scoring: Self-explainable rationale-driven multi-trait essay scoring.arXiv preprint arXiv:2502.20748. Ronen Eldan and Yuanzhi Li. 2023. Tinystories: How small can language models be and still speak coherent english?arXiv preprint arXiv:2305.07759. Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Tak Yeon Lee, So-Yeon Ahn, and Alice Oh....

arXiv 2023
[3]

Narrativity

The feedback {should/should not} align with the score descriptions in the rubric. (end of [Condition]) [Essay] {essay text} (end of [Essay]) [Scores] Narrativity: 3 Language: 2 (. . . ) (end of [Scores]) [Rubric descriptions] [Trait] Narrativity (end of [Trait]) The following is a rubric description in terms of the “Narrativity” trait. Score 3: The respon...

arXiv 2025
[7]

The feedback {should/should not} include actionable revision suggestions for improving the essay
[2018]

A broad-coverage challenge corpus for sen- tence understanding through inference. InProceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122. Association for Computational Linguistics. Ruosong Yang, Jiannong Cao, Zhiyuan Wen, ...

arXiv 2018
[2022]

arXiv preprint arXiv:2205.03835

On the use of bert for automated essay scor- ing: Joint learning of multi-scale essay representation. arXiv preprint arXiv:2205.03835. Zhaoyang Wang, Jinqi Jiang, Huichi Zhou, Wen- hao Zheng, Xuchao Zhang, Chetan Bansal, and Huaxiu Yao. 2025. Verifiable format control for large language model generations.arXiv preprint arXiv:2502.04498. Adina Williams, Ni...

arXiv 2025
[2023]

Narrativity

Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information pro- cessing systems, 36:46595–46623. A Prompt Templates A.1 Prompt Design for Feedback Generation We employ GPT-5.1 to generate essay feedback us- ing multiple sources of information. For each essay, we provide the essay text, the associated prompt, and an excerpt when ...

2024
[2025]

InProceedings of the 2025 Conference on Empirical Methods in Natural Lan- guage Processing, pages 272–292, Suzhou, China

From problem-solving to teaching problem- solving: Aligning LLMs with pedagogy using re- inforcement learning. InProceedings of the 2025 Conference on Empirical Methods in Natural Lan- guage Processing, pages 272–292, Suzhou, China. Association for Computational Linguistics. Heejin Do, Yunsu Kim, and Gary Geunbae Lee. 2023. Prompt-and trait relation-aware...

arXiv 2025

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [1]

The feedback {should/should not} quote parts of the essay that are relevant to evaluating the given traits

[2] [2]

Ronen Eldan and Yuanzhi Li

Teach-to-reason with scoring: Self-explainable rationale-driven multi-trait essay scoring.arXiv preprint arXiv:2502.20748. Ronen Eldan and Yuanzhi Li. 2023. Tinystories: How small can language models be and still speak coherent english?arXiv preprint arXiv:2305.07759. Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Tak Yeon Lee, So-Yeon Ahn, and Alice Oh....

arXiv 2023

[3] [3]

Narrativity

The feedback {should/should not} align with the score descriptions in the rubric. (end of [Condition]) [Essay] {essay text} (end of [Essay]) [Scores] Narrativity: 3 Language: 2 (. . . ) (end of [Scores]) [Rubric descriptions] [Trait] Narrativity (end of [Trait]) The following is a rubric description in terms of the “Narrativity” trait. Score 3: The respon...

arXiv 2025

[4] [7]

The feedback {should/should not} include actionable revision suggestions for improving the essay

[5] [2018]

A broad-coverage challenge corpus for sen- tence understanding through inference. InProceed- ings of the 2018 Conference of the North American Chapter of the Association for Computational Lin- guistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122. Association for Computational Linguistics. Ruosong Yang, Jiannong Cao, Zhiyuan Wen, ...

arXiv 2018

[6] [2022]

arXiv preprint arXiv:2205.03835

On the use of bert for automated essay scor- ing: Joint learning of multi-scale essay representation. arXiv preprint arXiv:2205.03835. Zhaoyang Wang, Jinqi Jiang, Huichi Zhou, Wen- hao Zheng, Xuchao Zhang, Chetan Bansal, and Huaxiu Yao. 2025. Verifiable format control for large language model generations.arXiv preprint arXiv:2502.04498. Adina Williams, Ni...

arXiv 2025

[7] [2023]

Narrativity

Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in neural information pro- cessing systems, 36:46595–46623. A Prompt Templates A.1 Prompt Design for Feedback Generation We employ GPT-5.1 to generate essay feedback us- ing multiple sources of information. For each essay, we provide the essay text, the associated prompt, and an excerpt when ...

2024

[8] [2025]

InProceedings of the 2025 Conference on Empirical Methods in Natural Lan- guage Processing, pages 272–292, Suzhou, China

From problem-solving to teaching problem- solving: Aligning LLMs with pedagogy using re- inforcement learning. InProceedings of the 2025 Conference on Empirical Methods in Natural Lan- guage Processing, pages 272–292, Suzhou, China. Association for Computational Linguistics. Heejin Do, Yunsu Kim, and Gary Geunbae Lee. 2023. Prompt-and trait relation-aware...

arXiv 2025