Leveraging AI for Rapid Generation of Physics Simulations in Education: Building Your Own Virtual Lab

(2) Department of Physics; Bar-Ilan University; Boulder; Colorado; Israel; Joshua Glazer (1); Noah D. Finkelstein (2) ((1) Department of Physics; Ramat Gan; Roi Einhorn Zarzecki (1); University of Colorado Boulder

REVIEW 4 major objections 6 minor 2 cited by

Leveraging AI for Rapid Generation of Physics Simulations in Education: Building Your Own Virtual Lab

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A natural-language prompt is enough to generate a working interactive physics simulation—pendulum, Ising model, or random walker—and plain-language dialogue with the model can validate and correct it without programming skill.

desk verdict A practical demonstration that LLMs can generate usable physics simulations from a reusable prompt template; the case is made, but the evidence is qualitative and the pedagogical claims outrun the data. read the letter →

arxiv 2412.07482 v1 pith:56LD53QF submitted 2024-12-10 physics.ed-ph

Yossi Ben-Zion (1) , Roi Einhorn Zarzecki (1) , Joshua Glazer (1) , Noah D. Finkelstein (2) ((1) Department of Physics , Bar-Ilan University , Ramat Gan , Israel , (2) Department of Physics

show 4 more authors

University of Colorado Boulder Boulder Colorado USA)

This is my paper · ORCID

classification physics.ed-ph

keywords AI-generatedsimulationsphysicseducationlargelanguagemodelsinteractivevirtuallabspromptengineeringsimplependulumIsingmodelrandomwalk

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that current large language models, when given a detailed natural-language prompt, can produce working interactive HTML and JavaScript simulations of physical systems, and that non-programmers can validate and refine these simulations through ordinary conversation. The authors demonstrate this with three representative systems: the simple pendulum, the 2D Ising model, and a 2D/3D random walker. Their workflow pairs a structured prompt template with a battery of technical and physical tests, then fixes discrepancies by re-prompting the model in plain language. The value, if true, is that tailored virtual labs become cheap and fast enough to create on demand for classroom use.

What carries the argument

The load-bearing mechanism is a reusable prompt template—a structured block specifying animation, slider controls, graphs, axis labels, and the required numerical method—combined with an iterative 'prompt, test, re-prompt' loop. The template constrains the model's output to land close to a correct simulation, while the loop allows a user to detect discrepancies against known physics (for example, the pendulum period $T = 2\pi\sqrt{L/g}$) and send targeted correction messages. This two-part mechanism is what converts a probabilistic text generator into a reliable builder of educational physics simulations.

What would settle it

Run the paper's three prompts through current LLMs and, without human editing, check the outputs against the paper's own physical tests—pendulum period matching $T = 2\pi\sqrt{L/g}$ at small angles, Ising magnetization falling sharply near $T_c = 2.269$, and random-walk mean distance scaling as $\sqrt{\pi n/4}$; if most unfixed outputs fail these checks, the central claim of rapid valid simulation generation fails. Alternatively, ask students who have not yet learned the pendulum period formula to validate the simulation and see whether they catch the halved-period error the authors had to fix.

Watch

Extended reading notes

Core claim

The paper establishes that a carefully structured prompt handed to a leading language model yields a runnable, interactive simulation of a physical system, and that the loop of testing against known analytical results and re-prompting in plain language repairs the residual errors. The authors document three worked examples: a pendulum solved with Runge-Kutta and compared with the small-angle analytical solution, an Ising model using Metropolis-Hastings and checked against the critical temperature $T_c \approx 2.269$, and a random walker whose mean distance is compared with $\langle r\rangle = \sqrt{\pi n/4}$. Each case passes the stated physical tests after iterative natural-language corrections, such as the instruction that the analytical pendulum graph 'displays a period that is exactly half of the correct time.'

Load-bearing premise

The workflow's reliability depends on the user being able to recognize when a simulation is physically wrong by inspecting graphs and reasoning about the model's answers; a user who cannot spot an error like a halved pendulum period will not know to correct it.

Editorial extensions

If this is right

Teachers can generate a custom simulation for a specific lesson, edge case, or student misconception within minutes instead of searching or waiting for a pre-built one.
Students can build and test their own models, turning simulation creation into an active exercise that reinforces conceptual understanding of the underlying physics.
The same template-plus-refinement loop should transfer to other disciplines such as chemistry or biology by substituting domain-specific prompts and validation targets.
Because human oversight remains necessary, the method works best in settings where educators can tell correct physics from incorrect physics, making validation itself a pedagogical activity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's debugging example shows a latent risk: the model silently produced an analytical curve with half the correct period, and the fix depended on the human noticing it. A quantitative benchmark that runs the stated prompts many times and measures how often the unfixed output passes the physical tests would reveal how much oversight is really required.
The 'no expertise needed' claim likely holds for producing a plausible-looking simulation, but not for certifying its correctness. The user's physics knowledge is the effective safety net, so the method's reliability scales with the user's ability to judge the output.
A natural next step is to automate the validation oracle: have a script check the pendulum period, Ising magnetization curve, and random-walk scaling law automatically, then measure the number of re-prompting rounds needed to reach a passing simulation without any human judgment.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new thing here is the reusable prompt template and the three worked case studies—pendulum, Ising model, random walker. That is a useful extension of the existing LLM-in-education program, and the paper is honest that it complements rather than replaces PhET-style tools. The writing is clear, the workflow is plausible, and the iterative debugging example is instructive. The incremental approach to the 3D random walker is a practical tip worth copying.

The central capability claim—that an educator can generate a working simulation by prompting without writing code—is believable for the systems shown. I have no serious doubt that this works, and the paper's own three examples support it. The choice of three distinct system types (ODE, many-body, stochastic) gives the method breadth.

Soft spots: the evidence is almost entirely qualitative. No simulation files are shipped, no quantitative comparison to analytical solutions is given, and there is no user study. The validation procedures are described but not shown in detail, so a referee cannot independently check that the simulations are actually correct. The educational-benefit claims (critical thinking, metacognition, engagement) are asserted in the abstract and conclusion but are not measured. I would not call these fatal—this is a practice paper, not a controlled study—but the conclusions should be scaled back to what the evidence supports.

The stress-test note worries that non-experts cannot validate without physics knowledge. That concern does not fully land. The paper says validation requires no additional programming expertise, not no physics knowledge. Comparing to known analytical solutions requires knowing those solutions, which is exactly the pedagogical point. The example where the authors catch the period error does require recognizing the correct period, but that is not a weakness of the method; it is the intended use of domain knowledge in validation. The paper would benefit from saying this explicitly.

The citation pattern is appropriate and does not overclaim novelty. The prior LLM-in-education papers are cited, and the unique contribution is the prompt template and examples.

Who this is for: physics educators and those developing teacher-training materials. It deserves a serious referee for a practitioner-oriented journal like The Physics Teacher or a physics education research venue. I would recommend acceptance with moderate revisions: ship at least one HTML simulation file as supplementary material, add a short quantitative validation note (e.g., a table of measured periods vs. analytic values), and soften the broad pedagogical claims. A serious referee should engage with it rather than desk-reject.

Referee Report

4 major / 6 minor

Summary. The paper describes a method for using large language models (ChatGPT's O1 and Claude 3.5 Sonnet) with a structured prompt template to generate interactive HTML/JavaScript physics simulations for three systems: a simple pendulum, a 2D Ising model, and 2D/3D random walkers. For each system, the paper reports the initial prompt, a set of technical and physical validation criteria, and — for the pendulum — a debugging session in which the authors identify and correct a period-2 error in the analytical overlay. The central claim, stated in the abstract and conclusion, is that this workflow enables educators and students to rapidly create custom simulations tailored to educational needs without requiring advanced programming skills.

Significance. If fully supported, the proposed workflow would be a valuable, low-cost complement to curated simulations such as PhET, with plausible benefits for active learning, prompt engineering practice, and metacognitive reflection on physics models. The paper has genuine strengths: it spans three distinct classes of physical systems (deterministic ODE, many-body statistical, and stochastic), provides full prompts that are likely reusable, documents a concrete iterative debugging episode, and explicitly frames the approach as complementary rather than a replacement for established tools. However, the evidence as presented is anecdotal: there are no simulation artifacts, no quantitative comparisons to analytical results beyond the author's own visual inspection, no failure statistics, and no data on the target population of non-programming educators or students. The load-bearing assumption that non-experts can reliably judge physical correctness through natural-language dialogue is asserted (Section I.A) but not demonstrated.

major comments (4)

[Section I.A and Section II.C] The claim that validation can be conducted 'through natural language dialogue with the model, requiring no additional programming expertise' is not supported by the only documented debugging example. In Section II.C, the authors diagnose the pendulum's analytical curve by telling the model that it 'displays a period that is exactly half of the correct time,' which requires the user to know the correct period formula T=2π√(L/g) and to compare it against the displayed curve. A student who lacks that domain knowledge cannot formulate such a diagnosis. Since the stated audience is non-programming educators and students, the paper needs either a user study (even a small think-aloud session) or a significantly narrowed claim about who can perform this validation.
[Section III.C] The physical validation of the Ising model is qualitative and incomplete. The paper asserts that near Tc=2.269 the magnetization should sharply decrease, but no magnetization-versus-temperature data, no error bars, no lattice-size or Monte Carlo step convergence checks, and no quantitative comparison are provided. Moreover, because the prompt itself supplies the value Tc=2.269, the observed agreement (if any) only demonstrates that the model implemented the supplied number, not that the physics of the 2D Ising model is correctly reproduced. The physical validity of the Ising simulation is therefore not established.
[Section IV.C and Section IV.D] The random-walk validation is similarly lacking in quantitative evidence. The theoretical expected distances (√(πn/4) in 2D and √(8n/(3π)) in 3D) are given in the prompts, so the test amounts to checking whether the AI coded the supplied formula, not whether the simulation's statistical behavior is physically correct. The paper reports no comparisons with error bars, no number of walkers or time steps used in the validation, and no quantitative measure of agreement. The claim that the simulation 'confirm[s] the model's consistency with 2D random walk analytics as the number of walkers increases' is unsupported.
[Throughout (no artifacts)] The paper provides no simulation files, URLs, or code repository. The described simulations cannot be inspected, run, or reproduced by readers. For a method that is essentially about generating concrete artifacts (HTML files), this is a substantial omission that weakens the paper's reproducibility and makes it impossible to verify the physical validation claims. The authors should provide the generated code or a public demo for at least the three main examples.

minor comments (6)

[Section II.C] There is a typo: 'opporutnity' should be 'opportunity'.
[Abstract] The phrase 'We share a new class of opportunities' is vague; consider rephrasing to something more concrete, such as 'We present a workflow for...'.
[Section I.B and Reference [11]] Reference [11] is a footnote-style comment rather than a formal citation; it should be converted to a proper reference or integrated into the text.
[Section II.B] The analytical solution in the prompt is printed with the same symbol for time and the product; consider making the formula clearer (e.g., θ(t)=θ0 cos(t√(g/L)) + (ω0/√(g/L)) sin(t√(g/L))) and noting that it is the small-angle approximation.
[Figures 1–5] The figure captions do not identify which language model (O1 or Claude 3.5 Sonnet) generated each displayed interface; specify this, as the paper suggests model differences matter.
[Section III.C] The unit of the critical temperature Tc=2.269 should be stated explicitly (e.g., in units of J/k_B) for readers unfamiliar with the 2D Ising model.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's simulations are validated against externally supplied analytical results, and the sole self-citation is contextual background.

full rationale

The paper makes no derived prediction that reduces to its own inputs. Its workflow is: (i) a user writes a detailed prompt containing the physical model and any analytical comparison curves; (ii) an LLM generates HTML/JavaScript implementing the simulation; (iii) the user validates the simulation against known analytical solutions or established physical behavior. The analytical expressions (small-angle pendulum solution, Ising critical temperature, random-walk mean displacement) are supplied by the user as benchmarks, not derived by the AI. The agreement between the generated simulation and these benchmarks is therefore an empirical test of whether the AI correctly implemented the physics, not a conclusion forced by construction. The numerical pendulum solver is instructed to use Runge-Kutta integration, so agreement with the small-angle formula is a genuine check of the integration code. The Ising and random-walk validations likewise compare independently simulated stochastic dynamics against theoretical expectations. The only self-citation is reference [6] (Finkelstein et al., PhET project) in a general statement about the educational value of simulations; that point is contextual and does not support the paper's central claim. No uniqueness theorem, ansatz, or fitted parameter is imported from the authors' prior work. The paper's weaker point—that non-experts can reliably validate subtle simulation errors—is a legitimate empirical concern about the strength of the evidence, but it is a question of validity and evidence, not circularity. Under the stated standards, no circular step can be exhibited with a specific reduction, so the score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper's central claim depends on two domain assumptions: LLMs can generate working simulation code from prompts, and users can validate the output through qualitative comparison with known physics. No free parameters are fitted and no new entities are posited.

assumptions (4)

domain assumption LLMs can produce working interactive HTML/JavaScript physics simulations from natural-language prompts, with errors correctable through iterative re-prompting.
Central premise of the methodology; introduced in Section I and demonstrated anecdotally in Sections II-IV.
domain assumption Qualitative visual comparison of simulation output to known analytical solutions (small-angle formula, Tc=2.269, sqrt(pi n/4)) is sufficient to certify physical validity.
Used in the testing procedures (Sections II.C, III.C, IV.C); no quantitative error metrics are reported.
domain assumption Simulation-based learning is effective, as established by the cited literature.
Invoked in the introduction with references 6-8; the paper's educational rationale depends on it.
standard math The random-walk expectation formulas <r>=sqrt(pi n/4) (2D) and <r>=sqrt(8n/(3pi)) (3D) correctly describe the simulated lattice walks.
Used as overlay curves in the prompts and validation (Sections IV.B, IV.D); these are asymptotic results for random walks.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Leveraging AI for Rapid Generation of Physics Simulations in Education: Building Your Own Virtual Lab." pith.science (2026). https://pith.science/paper/56LD53QF

@misc{pith2026241207482,
  author       = {Pith},
  title        = {Pith review of: Leveraging AI for Rapid Generation of Physics Simulations in Education: Building Your Own Virtual Lab},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/56LD53QF}},
  note         = {Machine review of arXiv:2412.07482}
}

read the original abstract

Seemingly we are not so far from Star Trek's food replicator. Generative artificial intelligence is rapidly becoming an integral part of both science and education, offering not only automation of processes but also the dynamic creation of complex, personalized content for educational purposes. With such advancement, educators are now crafting exams, building tutors, creating writing partners for students, and developing an array of other powerful tools for supporting our educational practices and student learning. We share a new class of opportunities for supporting learners and educators through the development of AI-generated simulations of physical phenomena and models. While we are not at the stage of "Computer: make me a mathematical simulation depicting the quantum wave functions of electrons in the hydrogen atom", we are not far off.

Figures

Figures reproduced from arXiv: 2412.07482 by the authors.

**Figure 2.** FIG. 2. Enhanced pendulum simulation featuring large-angle dynamics, friction effects, pendulum [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** FIG. 3. 2D Ising model simulation with heat map visualization, parameter controls, and real-time [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: FIG. 4. Multiple random walkers on a 2D lattice with adjustable walker count. The graph compares [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: FIG. 5. 3D random walk simulation with adjustable particle count and viewing options. The graph [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

A principled way to think about AI in education: guidance for educators and policy makers based on goals, models of human learning, and use of technologies
cs.CY 2025-10 unverdicted novelty 5.0 of 10

A principles-based framework links educational goals, socio-cultural learning theory, and human/technology roles to guide generative AI use in higher education.
Using LLMs to Detect Growth in Computational Thinking in Introductory Physics
physics.ed-ph 2026-08 conditional novelty 4.0 of 10

An LLM scored students' written computational thinking responses in an introductory physics course with human-level agreement on well-defined practices and reproduced pre-post growth trends at scale.

Reference graph

Works this paper leans on

11 extracted references · 11 canonical work pages · cited by 2 Pith papers

[1]

Slider Interactivity: Ensure that adjusting the mass, length, initial angle, and angular velocity sliders updates the pendulum’s animation and graph in real-time

work page
[2]

Axis Reset on Parameter Change: Verify that the graph resets and recalibrates when any of the sliders are adjusted, ensuring that the simulation reflects the new initial conditions accurately

work page
[3]

Graphical Representation: Ensure the graph displays numerical values on both axes 8 and includes a clear legend differentiating between the numerical and analytical solu- tions

work page
[4]

Pendulum Length at Maximum Extension: Ensure the entire pendulum, including the pendulum, remains visible within the canvas at its maximum length

work page
[5]

Period Accuracy for Small Angles: Confirm that for small initial angles, the period of the pendulum matches the analytical solution T = 2π q L g

work page
[6]

Numerical vs. Analytical Solution for Larger Angles: Ensure that for small initial angles, the numerical and analytical solutions match, but as the initial angle increases, the numerical solution diverges from the analytical one, illustrating the limits of the small-angle approximation. Notably, this physical test becomes a great opporutnity for students ...

work page
[7]

Technical Tests As with the simple pendulum simulation, the Ising model simulation underwent technical checks to ensure correct functionality. This included verifying that the display of the spin orientations, the interaction sliders, and the real-time graphs were properly synchronized and responsive to changes in user inputs

work page
[8]

Physical Tests The physical tests focused on ensuring that the simulation reflects the expected physical behaviors of the 2D Ising model under various conditions. Again, the particular areas of attention to physical validation could be the focus of an instructor prompting students, as they seek to focus on specific aspects of the Ising model and its appli...

work page

Show all 11 references

[9]

The magnetization should sharply decrease as the temper- ature approaches and exceeds this critical point

Phase Transition and Critical Temperature: Near the critical temperature Tc = 2.269 for J = 1, H= 0, the system should undergo a transition from a magnetized (ordered) state to a disordered state. The magnetization should sharply decrease as the temper- ature approaches and ex...
[10]

This phenomenon should occur even without an external magnetic field

Magnetization at Low Temperatures: At temperatures significantly lower than Tc, the system is expected to exhibit spontaneous magnetization, where most of the spins align in the same direction. This phenomenon should occur even without an external magnetic field. The simulatio...
[11]

This test was performed at both high and low temperatures to ensure that the simulation reflects the expected physical response to external fields

Response to External Magnetic Field: When a magnetic field is applied ( H >0), the spins should align with the direction of the field, causing an increase in magnetization proportional to the strength of the field. This test was performed at both high and low temperatures to e...

2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Slider Interactivity: Ensure that adjusting the mass, length, initial angle, and angular velocity sliders updates the pendulum’s animation and graph in real-time

work page

[2] [2]

Axis Reset on Parameter Change: Verify that the graph resets and recalibrates when any of the sliders are adjusted, ensuring that the simulation reflects the new initial conditions accurately

work page

[3] [3]

Graphical Representation: Ensure the graph displays numerical values on both axes 8 and includes a clear legend differentiating between the numerical and analytical solu- tions

work page

[4] [4]

Pendulum Length at Maximum Extension: Ensure the entire pendulum, including the pendulum, remains visible within the canvas at its maximum length

work page

[5] [5]

Period Accuracy for Small Angles: Confirm that for small initial angles, the period of the pendulum matches the analytical solution T = 2π q L g

work page

[6] [6]

Numerical vs. Analytical Solution for Larger Angles: Ensure that for small initial angles, the numerical and analytical solutions match, but as the initial angle increases, the numerical solution diverges from the analytical one, illustrating the limits of the small-angle approximation. Notably, this physical test becomes a great opporutnity for students ...

work page

[7] [7]

Technical Tests As with the simple pendulum simulation, the Ising model simulation underwent technical checks to ensure correct functionality. This included verifying that the display of the spin orientations, the interaction sliders, and the real-time graphs were properly synchronized and responsive to changes in user inputs

work page

[8] [8]

Physical Tests The physical tests focused on ensuring that the simulation reflects the expected physical behaviors of the 2D Ising model under various conditions. Again, the particular areas of attention to physical validation could be the focus of an instructor prompting students, as they seek to focus on specific aspects of the Ising model and its appli...

work page