Pith. sign in

REVIEW

Could an Artificial-Intelligence agent pass an introductory physics course?

Not yet reviewed by Pith; the record is open.

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2301.12127 v2 pith:VIZEJUS2 submitted 2023-01-28 physics.ed-ph

classification physics.ed-ph
keywords physicsresponsescourseattentioncontentcontroversyhuman-likeintroductory
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Massive pre-trained language models have garnered attention and controversy due to their ability to generate human-like responses: attention due to their frequent indistinguishability from human-generated phraseology and narratives, and controversy due to the fact that their convincingly presented arguments and facts are frequently simply false. Just how human-like are these responses when it comes to dialogues about physics, in particular about the standard content of introductory physics courses? This study explores that question by having ChatGTP, the pre-eminent language model in 2023, work through representative assessment content of an actual calculus-based physics course and grading the responses in the same way human responses would be graded. As it turns out, ChatGPT would narrowly pass this course while exhibiting many of the preconceptions and errors of a beginning learner.

Discussion (0). Continue with ORCID to comment.

Pith tools