Using Large Language Models for Automated Grading of Student Writing about Science

Chris Impey; Matthew Wenger; Nikhil Garuda; Sarah Stamer; Shahriar Golchin

REVIEW 4 major objections 5 minor 1 cited by

Using Large Language Models for Automated Grading of Student Writing about Science

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GPT-4 can match an instructor's grades on short science essays — and beat peer grading — when its prompt includes a model answer and a rubric.

desk verdict New data on an old claim: the paper repackages the prior GPT-4 grading result with per-student comparisons, but the 'match' still rests on null results and a single gold-standard grader. read the letter →

arxiv 2412.18719 v1 pith:2NVEKVJX submitted 2024-12-25 cs.CL

Chris Impey , Matthew Wenger , Nikhil Garuda , Shahriar Golchin , Sarah Stamer This is my paper · ORCID

classification cs.CL

keywords GPT-4gradingautomatedessayscoringpeermassiveopenonlinecoursessciencewritingassessmentrubricgenerationlargelanguagemodelsastronomyeducation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a large language model can grade short, content-based science writing as reliably as the instructor who designed the assignment, and more reliably than the peer grading that massive open online courses currently rely on. Across 120 student answers to 12 questions from three astronomy-themed MOOCs, GPT-4 produced grade distributions that were not statistically different from the instructor's whenever the prompt included both a model answer and a grading rubric, whether the rubric was the instructor's own or generated by GPT-4 itself. When the model received only a model answer, its grades differed significantly from the instructor's, and so did peer grades. If the result holds, automated grading could replace or lighten peer grading in low-stakes online courses and large introductory science classes, where a human cannot grade thousands of essays.

What carries the argument

The mechanism is a prompt template with three conditions that control how much grading guidance GPT-4 receives. Every condition embeds the student response and the assignment's total point value; the first adds only the instructor's model answer, the second adds the model answer plus the instructor's rubric, and the third asks GPT-4 to write its own rubric from the course description, the question, and the model answer before scoring. The rubric is the load-bearing element: leaving it out makes GPT-4's grades differ significantly from the instructor's, while adding either the instructor's rubric or an LLM-generated one brings the grades into statistical agreement ($p = 1.000$). The statistical framework is non-parametric — the Friedman test with Conover post-hoc comparisons and Bonferroni correction — chosen because the score distributions violate normality and homogeneity of variance, with bootstrap resampling used for per-question standard errors and p-values.

What would settle it

Take the same 120 essays and have two independent instructors who did not write the rubrics grade them, then run GPT-4 with the same prompt template. If GPT-4 sits no closer to the rubric-writing instructor than the two outside instructors sit to each other, the claim that the LLM 'matches the instructor' collapses into the weaker claim that it matches one instructor's standards. A sharper version: re-run the study with rubrics and model answers written by a different instructor; if the agreement pattern flips, rubric authorship, not grading skill, is what the LLM is reproducing.

Watch

Extended reading notes

Core claim

The paper's central claim is that GPT-4, given an instructor's model answer plus a grading rubric, produces grades that are not statistically different from the instructor's on short science writing assignments, and that this beats peer grading. When the prompt supplied the instructor's rubric (the second condition) or a GPT-4-generated rubric (the third condition), the LLM's grades were statistically indistinguishable from the instructor's ($p = 1.000$ after a Bonferroni-corrected Conover post-hoc test on the Friedman analysis), while both peer grades and rubric-free GPT-4 grading differed significantly from the instructor ($p < 0.001$). The same pattern held for individual students: the mean instructor-minus-GPT-4 gap was near zero with smaller dispersion than the instructor-minus-peer gap in all three courses. An intraclass correlation coefficient of 0.92 across graders is offered as evidence of consistency, and the authors conclude that with a model answer and a rubric in hand, an LLM can stand in for the instructor in low-stakes settings and that LLM-generated rubrics match the utility of instructor rubrics.

Load-bearing premise

The load-bearing premise is that a single instructor's grades are the gold standard, and that same instructor wrote the model answers and rubrics given to GPT-4, so 'matching the instructor' may amount to reproducing one person's scoring style rather than grading accurately in general; the paper itself concedes that instructors are fallible.

Editorial extensions

If this is right

Grading of writing in low-stakes MOOCs can be automated: an LLM equipped with a model answer and rubric can score thousands of essays in real time, removing the peer-grading burden that currently caps assignment frequency.
Assignments that lack rubrics need not be redesigned, because GPT-4-generated rubrics produced grades statistically indistinguishable from instructor-rubric grades, extending the approach to archival courses.
The method's accuracy degrades on open-ended, speculative questions, as the history and philosophy course showed the weakest agreement, so content-based factual prompts are the near-term application.
The paper states the same pipeline transfers to large university general-education science courses, where it plans to use the approach for formative assessment with grade reasoning attached.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the reference instructor wrote the model answers and rubrics, part of what GPT-4 matches may be that instructor's scoring style rather than an objective standard; an immediate test would be having a second independent instructor grade the same 120 essays and comparing how far GPT-4 sits from each of them.
A workflow the paper does not design but its data support is a human-audit loop: let the LLM grade everything with a short written justification, and have the instructor review only borderline cases, since the disagreement histograms imply the audit load would be small.
A testable extension beyond the paper: agreement between LLM and instructor should track rubric granularity, with point-by-point analytic rubrics yielding tighter agreement than holistic scales; the astronomy-versus-history-and-philosophy contrast is consistent with this but too small to prove it.
The paper's own concession that instructors are fallible implies the next benchmark should be agreement with a consensus of multiple expert graders, not any single instructor; if two experts disagree with each other by as much as GPT-4 disagrees with the reference instructor, 'matches the instructor' stops being evidence of accuracy.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the headline result is not new. It's a re-analysis and extension of the same group's Golchin et al. 2024, and Table 3 is explicitly adapted from it. What is new is the per-student comparison, the dispersion plots, and the test of LLM-generated rubrics. Those are legitimate additions, but the core claim—GPT-4 with a model answer and rubric 'approximately matches' instructor grades—still leans on a null result.

The study does several things right. It uses real MOOC data: 120 students, 12 questions across three courses, graded by an instructor, peers, and GPT-4 under three prompt conditions. The statistical approach (Friedman with Conover post-hoc, bootstrap CIs) is appropriate for the skewed distributions. The authors are transparent about limitations: purposeful sampling, small per-question samples, and the fallibility of the instructor as gold standard. The LLM-generated rubric condition is a genuine extension and worth reporting.

The soft spots are real and central. 'Not statistically different' (p=1.000) is not evidence of agreement. No equivalence margin was specified. With 10 answers per question, the per-question tests have low power, and the paper itself notes this. More importantly, the descriptive statistics contradict the abstract's 'approximately matched for all three' claim. In Table 2, HPA Q2 instructor mean is 2.39 vs Prompt 2's 3.10 on a 4-point scale—a 0.71-point gap. HPA RMS in Table 4 is 0.48, just under half a point, which on a 4-point rubric is a meaningful grade-category shift. The text concedes agreement is 'poor' on HPA Q3 and Q4. The aggregate Friedman test can mask these gaps because it ranks within each answer and pools across heterogeneous questions. Similarly, 'GPT-4 performs better than peer grading' is inferred indirectly: peers differ from the instructor, GPT-4 doesn't. That is not a direct test of whether GPT-4's absolute disagreement is smaller. The per-student scatterplot (Fig. 2) is descriptive, with no test on the error distributions.

The circularity concern is also fair: the instructor who wrote the rubrics and model answers is the same person whose grades serve as the gold standard. So 'matching the instructor' may just mean GPT-4 learned that instructor's idiosyncrasies. That's acknowledged in the Discussion but not mitigated.

Who is this for? Someone in educational assessment or MOOC research will want to read it as a proof-of-concept. It deserves a serious referee—the data are real, the question matters, and the limitations are addressable. But the authors should be pushed to reframe the claims as 'no detectable difference' with an equivalence margin, or better, to report direct paired comparisons of absolute errors and to release the data. As is, the abstract overstates what the design can support.

Referee Report

4 major / 5 minor

Summary. The manuscript reports an experiment in which GPT-4 graded short science writing assignments from three astronomy-related MOOCs, using three prompt conditions: instructor model answer only; instructor answer plus instructor-written rubric; and instructor answer plus an LLM-generated rubric. Grades from 120 students across 12 questions were compared with the original instructor grades and with Coursera peer grades, using Friedman/Conover tests, bootstrap confidence intervals, RMS differences, and per-student scatterplots. The paper claims that GPT-4 with an answer plus a rubric produced grades that were not statistically different from the instructor's and that GPT-4 outperformed peer grading in matching the instructor.

Significance. If the claim holds, the result is practically useful for large-scale MOOC assessment and for large introductory science courses, where instructor grading of writing is infeasible. The paper's methodological strengths include the use of non-parametric tests with post-hoc corrections, bootstrap-based standard errors, and a per-student analysis that goes beyond aggregate means. The appendices provide full question texts and rubrics, which helps reproducibility. However, the central claim rests on a single instructor as the gold standard, on null-hypothesis tests rather than equivalence tests, and on a small purposeful sample; these do not refute the claim but substantially limit its current evidentiary strength.

major comments (4)

[Results, Tables 1 and 2; Large Language Model vs. Instructor] The claim that GPT-4 'approximately matched' instructor grades is based on non-significant Friedman/Conover comparisons (p = 1.000 in Table 1), with no pre-specified equivalence margin. A null result from a rank-based test does not demonstrate agreement, and the descriptive statistics in Table 2 show educationally meaningful gaps, e.g., History and Philosophy Q2 instructor mean 2.39 versus Prompt 2 mean 3.10 on a 4-point scale (0.71 points, about 18% of the maximum), and HPA Q3 2.70 versus 3.20. The text itself notes 'poor' agreement on HPA Q3 and Q4. The authors should report an equivalence test with a justified margin (e.g., ±0.5 rubric points or a Cohen's d bound) and should temper the 'approximately matched for all three online courses' conclusion where the data do not support it.
[Author Contributions and Discussion (gold-standard instructor)] The instructor who created the rubrics and model answers is also the sole human grader whose grades serve as the gold standard. As the Author Contributions state, 'Evaluation material for the MOOCs was created by Matthew Wenger, who also acted as the instructor grader for this project.' This creates a favorable alignment: the LLM is prompted with the same instructor's rubric and answer and is then compared with that same instructor's scores. The Discussion concedes 'we have assumed instructors to be perfect, when in fact they are fallible.' The manuscript should explicitly frame the result as reproducing one instructor's grading, not as accurate grading in general, and should discuss how rubric-derived idiosyncrasy could inflate the apparent agreement.
[Research Data and Results, Table 3] Per-question samples of 10 answers (12 questions total) with purposeful sampling to spread peer grades provide very low power for the per-question bootstrap p-values in Table 3, all of which exceed 0.05. The paper acknowledges that per-class tests were underpowered, but then uses the non-significant per-question results to support the claim of no difference. A non-significant p-value with n=10 cannot support 'no statistically significant difference' as evidence of agreement. The authors should either report effect sizes with confidence intervals for each question, or explicitly restrict their generalizability claim to the aggregate course-level comparison and present the per-question results only descriptively.
[Comparisons for Individual Students and Reliability of Large Language Models and Peer Grading] The statement 'GPT-4 performs better than peer grading' is an indirect comparison: peer grades differ significantly from the instructor while GPT-4 grades do not (Table 1), and descriptive dispersions in Figures 2 and 3 are smaller for GPT-4 in some courses. No direct statistical test compares the absolute or squared errors of GPT-4 versus peer graders relative to the instructor. Also, the reported ICC of 0.92 is computed across graders without clarifying which graders are included, and the LLM was run once per prompt, so no estimate of LLM run-to-run variability is given. A direct paired comparison of |GPT-4 - instructor| versus |peer - instructor|, with appropriate clustering, would directly support the claimed superiority over peer grading.

minor comments (5)

[Methods, Research Data] The sentence 'The text for all writing assignment questions and grading rubrics used in this research study are provided in Appendix A' is followed by 'Model answers are available upon request'; the model answers should be included in the appendices or supplementary material for full reproducibility.
[Results, Table 4 and surrounding text] Table 4 is introduced as 'Table 3' in the text ('shown in Table 3'), and the caption numbering is inconsistent; this should be corrected.
[Appendix A, ETS rubrics] Several rubric entries contain typographical errors, e.g., 'The write only includes one wavelength instead of two' and 'The writer correctly answers the question correctly'; these should be proofread.
[Results, Figure 2 caption] The caption says 'Dashed lines in the histograms indicate the means of the three classes for the two measures of grade difference,' but it is unclear which classes the dashed lines correspond to; please clarify or label the lines.
[Introduction, Previous Work] Some references are incomplete (e.g., Bojic, Kovacevic, & Caparkapa, 2023 lacks publication details), and the relationship between this manuscript and the authors' earlier arXiv paper (Golchin et al., 2024) should be stated more explicitly to avoid duplicate-publication concerns.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: GPT-4 grading is benchmarked empirically against an instructor; no fitted parameters or constructed equivalences make the result true by definition.

full rationale

This paper is an empirical benchmark, not a derivation. The central result—that GPT-4 grades are not statistically different from the instructor's when the prompt includes an instructor model answer and rubric—is an experimental outcome. GPT-4 could have failed to match; no parameter is fitted to the instructor grades, and no equation is constructed to force agreement. The same instructor wrote the model answers and rubrics used in the prompts and also supplied the reference grades, so the benchmark measures replication of that instructor's criteria rather than absolute grading accuracy. The paper openly acknowledges this in the Discussion: 'we have assumed instructors to be perfect, when in fact they are fallible.' That is a validity limitation, not circularity. Self-citations to prior work (e.g., Formanek et al. 2017 on peer grading) are contextual and are not load-bearing for the current comparison. The LLM-generated rubric condition (Prompt 3) partially breaks the dependence on the instructor's rubric while still using the instructor's model answer. No circular step can be identified under the required standard of exhibiting a specific reduction.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The study is empirical and fits no parameters. Its conclusions rest on two domain assumptions: that instructor grades are a valid gold standard, and that a single instructor's rubrics and model answers are appropriate inputs for evaluating the LLM on the same questions those inputs describe. The purposeful sampling is an explicit assumption about representativeness.

assumptions (3)

domain assumption Instructor grades are treated as the gold standard for grading accuracy.
The entire evaluation compares GPT-4 and peer grades against a single instructor's scores; if the instructor is biased or inconsistent, matching the instructor is not the same as accurate grading. Acknowledged in Discussion: 'we have assumed instructors to be perfect, when in fact they are fallible.'
domain assumption GPT-4 is evaluated on the same assignments whose model answers and rubrics were provided in the prompt.
The model answers and rubrics were written by the same instructor for the same questions being graded; generalization to new questions or instructors is not tested.
domain assumption The sample of 120 answers, purposefully selected to span the peer grade range, is representative enough for the aggregate comparison.
Purposeful sampling can introduce selection bias; the paper acknowledges this in Limitations. The aggregate Friedman test may not generalize to the full distribution of student writing.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Using Large Language Models for Automated Grading of Student Writing about Science." pith.science (2026). https://pith.science/paper/2NVEKVJX

@misc{pith2026241218719,
  author       = {Pith},
  title        = {Pith review of: Using Large Language Models for Automated Grading of Student Writing about Science},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2NVEKVJX}},
  note         = {Machine review of arXiv:2412.18719}
}

read the original abstract

Assessing writing in large classes for formal or informal learners presents a significant challenge. Consequently, most large classes, particularly in science, rely on objective assessment tools such as multiple-choice quizzes, which have a single correct answer. The rapid development of AI has introduced the possibility of using large language models (LLMs) to evaluate student writing. An experiment was conducted using GPT-4 to determine if machine learning methods based on LLMs can match or exceed the reliability of instructor grading in evaluating short writing assignments on topics in astronomy. The audience consisted of adult learners in three massive open online courses (MOOCs) offered through Coursera. One course was on astronomy, the second was on astrobiology, and the third was on the history and philosophy of astronomy. The results should also be applicable to non-science majors in university settings, where the content and modes of evaluation are similar. The data comprised answers from 120 students to 12 questions across the three courses. GPT-4 was provided with total grades, model answers, and rubrics from an instructor for all three courses. In addition to evaluating how reliably the LLM reproduced instructor grades, the LLM was also tasked with generating its own rubrics. Overall, the LLM was more reliable than peer grading, both in aggregate and by individual student, and approximately matched instructor grades for all three online courses. The implication is that LLMs may soon be used for automated, reliable, and scalable grading of student science writing.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Automated Assignment Grading with Large Language Models: Insights From a Bioinformatics Course
cs.LG 2025-01 reject novelty 6.0 of 10

In a semester-long bioinformatics course, LLM graders with hand-tuned rubrics and examples graded written answers in 85 to 90 percent agreement with human TAs, but the accuracy was measured on the same set used to des...

Reference graph

Works this paper leans on

13 extracted references · 9 canonical work pages · cited by 1 Pith paper

[1]

Alseddiqi, M., Al-Mofleh, A., Albalooshi, L, & Najam, O. (2023). Revolutionizing Online Learning: The Potential of ChatGPT in Massive Open Online Courses. European Journal of Education and Pedagogy, 4(4), 1-5. https://doi.org/10.24018/ejedu.2023.4.4.686. Anft, M. (2023). Perspectives on Generative AI: College Leaders Assess the Promise and the Threat of a...

work page doi:10.24018/ejedu.2023.4.4.686 2023
[2]

3 points: The author answers the question thoroughly and thoughtfully and references information from the course to support their perspective or claims

it does not reference course information to support the claims. 3 points: The author answers the question thoroughly and thoughtfully and references information from the course to support their perspective or claims. HPA Question 4: Imagine if the universe was eternal and unchanging as proposed by the steady state model in the 1950’s. What are the philoso...

work page 1950
[7]

Ouyang, L., et al. (2022). Training Language Models to Follow Instructions with Human Feedback. https://arxiv.org/abs/2203.02155. Pereira, D. G., Afonso, A., & Medeiros, F. M. (2015). Overview of Friedman’s test and post-hoc analysis. Communications in Statistics-Simulation and Computation, 44(10), 2636-2653. Pisano, A., Crawford, A., Huffman, H., Graham,...

arXiv 2022
[13]

3 points: The author answers the question thoroughly and thoughtfully and references information from the course to support their perspective or claims

it does not reference course information to support the claims. 3 points: The author answers the question thoroughly and thoughtfully and references information from the course to support their perspective or claims. 30 Appendix B. Example Rubrics Generated for Knowing the Universe: History and Philosophy of Astronomy (HPA) by GPT-4 Rubric for HPA Questio...

work page 1950
[158]

Shapiro, S

https://doi.org/10.1016/j.autcon.2023.105200. Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3-4), 591-611. 20 Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological bulletin, 86(2),

arXiv 1965
[243]

Formanek, M., Buxner, S., Impey, C., & Wenger, M. (2019). Relationship between Learners’ Motivation and Course Engagement in an Astronomy Massive Open Online Course. Physical Review Physics Education Research, 15, 020140. Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the america...

work page arXiv 2019
[420]

Stevens, D.D., & Levi, A.J. (2012). Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning (2nd ed.). Routledge. https://doi.org/10.4324/9781003445432. Thirunavukasaru, A.J., Ting, D.S.J., Elangovan, K., Gutierrez, L., Tan, T.F., & Ting, D.S.W. (2023). Large Language Models in Medicine. Na...

work page doi:10.4324/9781003445432 2012
[692]

Hand, B., Lawrence, C., & Yore, L.D. (1999). A Writing in Science Framework Designed to Enhance Science Literacy, International Journal of Science Education 21(10), 1021-1035. Holmes, W., & Tuomi, I. (2022). State of the Art and Practice in AI in Education. European Journal of Education, 57(4), 542-570. Impey, C.D., Wenger, M.C., & Austin, C.L. (2015). As...

arXiv 1999

Show all 13 references

[746]

Biango-Daniels, M., & Sarvary, M. (2020). A Challenge in Teaching Scientific Communication: Academic Experience Does Not Improve Undergraduates’ Ability to Assess Their or Their Peers’ Writing. Assessment and Evaluation in Higher Education, 46(5), 809-820. Bojic, L., Kovacevic...

2020
[754]

ancient astronauts

Springer, Singapore. https://doi.org/10.1007/978-981-99-4932-8_8. Wang, Y., & Baker, R. (2015). Content or platform: Why do students complete MOOCs. MERLOT Journal of Online Learning and Teaching 11(1), 17-30. Wang, Y., Reddy, R.G., Mujahid, Z.M., Arora, A., Rubashevskii, A., ...

2015 arXiv
[2020]

Lecture Notes in Computer Science, Vol. 12615. Springer, Cham. https://doi.org/10.1007/978-3-030-68449-5_25. Brown, M. B., & Forsythe, A. B. (1974). Robust tests for the equality of variances. Journal of the American statistical association, 69(346), 364-367. Conover, W. J., &...

1974 doi
[2021]

https://www.classcentral.com/report/mooc-stats-2021/

Class Central. https://www.classcentral.com/report/mooc-stats-2021/. Shah, V., Murthy, S., Warriem, J., Saharasbudhe, S., Banergee, G., & Iyer, S. (2022). Learner-centric MOOC Model: A Pedagogical Design Model Towards Active Learner Participation and Higher Completion Rates. E...

2022
[2024]

Koneru, S., Wu, J, & Rajtmajer, S. (2023). Can Large Language Models Discern Evidence for Scientific Hypotheses? Case Studies in the Social Sciences. https://arxiv.org/abs/2309.06578. Krathwohl, D.R. (2002). A Revision of Bloom's Taxonomy: An Overview. Theory into Practice 41(...

2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Alseddiqi, M., Al-Mofleh, A., Albalooshi, L, & Najam, O. (2023). Revolutionizing Online Learning: The Potential of ChatGPT in Massive Open Online Courses. European Journal of Education and Pedagogy, 4(4), 1-5. https://doi.org/10.24018/ejedu.2023.4.4.686. Anft, M. (2023). Perspectives on Generative AI: College Leaders Assess the Promise and the Threat of a...

work page doi:10.24018/ejedu.2023.4.4.686 2023

[2] [2]

3 points: The author answers the question thoroughly and thoughtfully and references information from the course to support their perspective or claims

it does not reference course information to support the claims. 3 points: The author answers the question thoroughly and thoughtfully and references information from the course to support their perspective or claims. HPA Question 4: Imagine if the universe was eternal and unchanging as proposed by the steady state model in the 1950’s. What are the philoso...

work page 1950

[3] [7]

Ouyang, L., et al. (2022). Training Language Models to Follow Instructions with Human Feedback. https://arxiv.org/abs/2203.02155. Pereira, D. G., Afonso, A., & Medeiros, F. M. (2015). Overview of Friedman’s test and post-hoc analysis. Communications in Statistics-Simulation and Computation, 44(10), 2636-2653. Pisano, A., Crawford, A., Huffman, H., Graham,...

arXiv 2022

[4] [13]

3 points: The author answers the question thoroughly and thoughtfully and references information from the course to support their perspective or claims

it does not reference course information to support the claims. 3 points: The author answers the question thoroughly and thoughtfully and references information from the course to support their perspective or claims. 30 Appendix B. Example Rubrics Generated for Knowing the Universe: History and Philosophy of Astronomy (HPA) by GPT-4 Rubric for HPA Questio...

work page 1950

[5] [158]

Shapiro, S

https://doi.org/10.1016/j.autcon.2023.105200. Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3-4), 591-611. 20 Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological bulletin, 86(2),

arXiv 1965

[6] [243]

Formanek, M., Buxner, S., Impey, C., & Wenger, M. (2019). Relationship between Learners’ Motivation and Course Engagement in an Astronomy Massive Open Online Course. Physical Review Physics Education Research, 15, 020140. Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the america...

work page arXiv 2019

[7] [420]

Stevens, D.D., & Levi, A.J. (2012). Introduction to Rubrics: An Assessment Tool to Save Grading Time, Convey Effective Feedback, and Promote Student Learning (2nd ed.). Routledge. https://doi.org/10.4324/9781003445432. Thirunavukasaru, A.J., Ting, D.S.J., Elangovan, K., Gutierrez, L., Tan, T.F., & Ting, D.S.W. (2023). Large Language Models in Medicine. Na...

work page doi:10.4324/9781003445432 2012

[8] [692]

Hand, B., Lawrence, C., & Yore, L.D. (1999). A Writing in Science Framework Designed to Enhance Science Literacy, International Journal of Science Education 21(10), 1021-1035. Holmes, W., & Tuomi, I. (2022). State of the Art and Practice in AI in Education. European Journal of Education, 57(4), 542-570. Impey, C.D., Wenger, M.C., & Austin, C.L. (2015). As...

arXiv 1999