T2MM: An LLM Supported Architecture For Inquiry-Based Modeling
Pith reviewed 2026-07-04 19:30 UTC · model glm-5.2
The pith
Action sequences beat full code generation for LLM-driven model building
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central finding is that an LLM asked to output a short list of validated actions on an existing model state is substantially more accurate than an LLM asked to output the entire model representation from scratch. On 975 test cases, T2MM compiled 100% of the time and produced structurally identical models 92.7% of the time, compared to 54.5% and 57.4% for zero-shot and few-shot full-code-generation baselines. The gap was most pronounced for model creation from an empty canvas, where the code-generation baselines failed almost entirely (0% and 8.3% success), while T2MM succeeded 79.4% of the time. The paper attributes this to the fact that action sequences are compact, can be checked and,,
What carries the argument
The mechanism is a four-stage pipeline: (1) the learner's current model is retrieved from the VERA database as XML and converted to a minimal JSON representation; (2) this JSON, the learner's natural language request, and a description of the VERA ontology are sent to the LLM (GPT-4.1-mini); (3) the LLM returns a list of structured actions such as 'add node', 'remove relationship', or 'update parameter', which are validated against the VERA ontology rules; (4) validated actions are mapped to VERA API calls that write changes to the model. The key contrast is with baselines that send the same prompt but ask the LLM to return complete XML, which introduces more tokens, more hallucination risk,
If this is right
- Action-sequence architectures could generalize to other interactive learning environments where models must conform to a domain-specific ontology and remain editable across multiple learner turns.
- The finding that full code generation fails on blank-canvas creation suggests that LLM-assisted tools relying on raw code output may be unsuitable for scaffolding the initial stages of open-ended modeling tasks.
- The action-sequence approach could reduce cognitive load for learners by letting them describe desired changes in natural language while the system handles structural correctness, potentially freeing learners to focus on the scientific reasoning behind their models.
- Ontology validation as a post-LLM checkpoint could become a standard design pattern for deploying LLMs in domains where outputs must satisfy formal constraints beyond what the LLM can guarantee on its own.
Load-bearing premise
The evaluation dataset is procedurally generated from five expert models using templated natural language phrases that closely mirror the system's action vocabulary (e.g., 'Add bear and owl, and make bear consume the owl'). This means the test requests are structurally simple and linguistically narrow. The authors acknowledge that real learner requests will likely be more ambiguous and varied, and the performance gap may narrow when the system faces genuine learner language.
What would settle it
Test T2MM against the baselines using natural language requests collected from actual learners using VERA, rather than procedurally generated templates. If the performance gap between T2MM and full-code-generation baselines narrows to statistical insignificance under real learner language, the central claim that action sequences are superior would be weakened.
Figures
read the original abstract
Model Construction is a foundational practice in science learning that relies on visualization and interactivity. Large Language Models, increasingly augmented with multimodal capabilities, have been integrated in education contexts to support learning. However, these tools lack visual interactivity that is required by some learning contexts. We introduce Text to Multimodal Model (T2MM), a robust, dynamic LLM supported architecture that assists in model construction within the open inquiry ecology-based modeling software Virtual Experimental Research Assistant (VERA). T2MM accounts for the current context of the learner's model and creates interactive models, rather than static images, enabling the model to remain responsive to manual adjustment. To measure technical feasibility, we evaluate T2MM through a custom procedurally generated dataset of natural language learner modeling requests and target models within the VERA system. T2MM outperforms a baseline model generation architecture implemented through LLM-supported full code generation, common in the literature, across all measured success metrics. Our contribution not only outlines LLM integration into a inquiry-based learning modeling tool, but also describes a possible architecture through which more interactive multimodal LLM tools can be created.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Text to Multimodal Model (T2MM), an LLM-supported architecture for interactive model construction within the VERA ecology modeling platform. Rather than having the LLM generate full XML representations of models, T2MM prompts the LLM to produce structured action sequences (e.g., create node, edit relationship) that are validated against the VERA ontology and applied via API calls to update the learner's model. The system is evaluated on a custom procedurally generated dataset of 975 natural language modeling requests paired with target models, against two baselines: 0-Shot and N-Shot full XML code generation. T2MM achieves 100% compilation and 92.7% exact structural match (graph edit distance = 0), compared to 54.5% and 57.4% for the baselines. The paper positions this as the first AI-in-Education architecture to use LLM-generated action sequences grounded in a live learner model state for interactive conceptual modeling.
Significance. The core architectural idea—using LLM-generated action sequences validated against an ontology rather than full code generation—is a reasonable and well-motivated design choice for interactive learning environments. The authors provide reproducible prompts and materials on OSF, which is commendable. The comparison against two code-generation baselines on four metrics provides useful evidence that the action-sequence approach yields higher structural fidelity. The system addresses a genuine gap: existing multimodal LLM architectures for education lack iterative, context-aware model construction. However, the significance of the evaluation is tempered by the procedurally generated nature of the dataset, which uses templated natural language closely tied to the action vocabulary, limiting the strength of claims about natural language understanding.
major comments (1)
- §3.3 (Dataset): The evaluation dataset is procedurally generated by decomposing expert models into intermediate states and pairing each transition with a templated natural language phrase tied to the action vocabulary (e.g., 'Add bear and owl, and make bear consume the owl'). The evaluation then measures whether T2MM can recover the target model from these phrases. This creates a risk of circularity: the NL input is generated from the same action vocabulary that T2MM expects as output, making the parsing task near-deterministic on simple cases. The paper acknowledges this limitation in §5 ('textual requests by real learners will present a unique challenge'), but the headline claims in the abstract and conclusion do not adequately contextualize that the 92.7% structural match is achieved on templated input that mirrors the system's action schema. The authors should either (a) reframe the
minor comments (7)
- §4.3, Table 3: The distribution of target action sequence lengths is heavily skewed: 600 of 975 cases (61.5%) have length 1, and some lengths (e.g., 4) have only 4 cases. Figure 3's success rates for lengths 4, 6, and 8 are based on very small samples. Consider adding sample sizes to Figure 3 or noting the small-n caveat in the caption.
- §4.2: The N-Shot baseline performs worse than 0-Shot (mean GED 3.572 vs. 1.638), which is counterintuitive. The paper attributes this to 'random parameter variation' but leaves it uninvestigated. While this does not affect T2MM's own results, it weakens the baseline comparison. A brief investigation or at least a more explicit acknowledgment that the N-Shot configuration may be suboptimal would strengthen the evaluation.
- §3.2: The JSON examples contain formatting artifacts (e.g., 'p r o p e r t i e s', 'so ur ce _i d', 'ta rg et _i d', 'r e m o v e _ r e l a t i o n s h i p'). These should be cleaned up for publication.
- §1: 'a inquiry-based learning modeling tool' should be 'an inquiry-based learning modeling tool'.
- §3.4: The graph edit distance metric is described qualitatively but not formally defined. A brief formal definition (or at least a clear statement of the cost model for node/edge/parameter edits) would improve reproducibility.
- §4.3: 'All architectures achieved full success for action sequence length of four, however there were only four cases in this category.' This sentence is somewhat misleading because it implies a strength, but n=4 provides no statistical power. Consider rephrasing to avoid overinterpretation.
- References: Reference [24] (Yu & Jiang, 2026) and [25] (Zheng et al., 2025) appear to be very recent or forthcoming; ensure citation details are complete and accurate.
Simulated Author's Rebuttal
The referee acknowledges the core architectural contribution is well-motivated and the comparison against baselines is useful, but raises a major concern about the procedurally generated dataset: the templated NL input mirrors the system's action vocabulary, creating a risk of circularity that is not adequately reflected in the headline claims. The referee requests either reframing the claims or otherwise addressing this limitation more prominently.
read point-by-point responses
-
Referee: §3.3 (Dataset): The evaluation dataset is procedurally generated by decomposing expert models into intermediate states and pairing each transition with a templated natural language phrase tied to the action vocabulary... This creates a risk of circularity: the NL input is generated from the same action vocabulary that T2MM expects as output, making the parsing task near-deterministic on simple cases. The paper acknowledges this limitation in §5 but the headline claims in the abstract and conclusion do not adequately contextualize that the 92.7% structural match is achieved on templated input that mirrors the system's action schema. The authors should either (a) reframe the claims or otherwise address this.
Authors: We thank the referee for this thoughtful and accurate observation. The referee is correct that our dataset generation process creates a correspondence between the natural language phrasing and the action vocabulary, and that this limits the strength of claims we can make about natural language understanding. We agree that the abstract and conclusion do not sufficiently contextualize this limitation, and we will revise the manuscript accordingly. Specifically, we will: (1) Add an explicit caveat in the abstract noting that the evaluation uses procedurally generated, templated requests rather than natural learner language. (2) Revise the conclusion to state that the 92.7% structural match demonstrates technical feasibility of the action-sequence architecture on structured input, not robustness to the linguistic variability of real learner requests. (3) Expand the §5 limitations discussion to more directly acknowledge the circularity risk the referee identifies. We want to note two points of context, however. First, the dataset was designed intentionally to evaluate representational correctness and action sequencing — whether the architecture can reliably produce ontology-valid model edits — rather than NL understanding per se. The conclusion already states that 'this evaluation isolates representational correctness and action sequencing as prerequisites for learner-facing deployment, leaving linguistic variability and visual evaluation to future work,' but we agree this framing needs to be brought forward into the abstract. Second, the baselines (0-Shot and N-Shot) are evaluated on the identical dataset, so the relative comparison between action sequences and full code generation is not affected by the templating concern. The finding that action sequences yield higher ont revision: no
Circularity Check
Mild circularity in dataset construction: NL inputs are generated from the action vocabulary that T2MM is evaluated on recovering, but the paper is transparent about this and frames it as a feasibility test.
specific steps
-
fitted input called prediction
[Section 3.3 (Dataset), paragraph 2]
"To generate the natural language request, we tied each of the actions described above to a natural language phase. For example, if the learner wanted to create a model consisting of a bear consuming an owl, the generated natural language request would be “Add bear and owl, and make bear consume the owl.” This gave us a sample of 975 model and natural language pairs."
The evaluation input (NL request) is constructed by mapping each action to a fixed phrase. T2MM is then evaluated on whether it can recover the action sequence from the NL. Since the NL is generated FROM the action vocabulary that T2MM outputs, the task reduces to inverting a near-deterministic mapping. The 92.7% exact structural match is partly an artifact of this construction: the input encodes the output in templated form. However, the paper is transparent about this — it explicitly labels the dataset as 'procedurally generated,' acknowledges real learner requests 'will present a unique challenge,' and frames the evaluation as 'technical feasibility' rather than naturalistic performance. This is an evaluation validity concern more than a derivation that reduces to its inputs by formal定义
full rationale
The paper's central architectural contribution — generating structured action sequences rather than full XML code — has independent content and is not defined in terms of its evaluation output. The dataset construction does create a mild circularity: NL inputs are generated from the action vocabulary that T2MM is evaluated on recovering, making the inversion task near-deterministic on single-action cases (61.5% of the dataset). However, the paper is transparent about this construction, explicitly frames it as a feasibility test, and acknowledges the limitation with real learner language. Self-citations (refs [1], [20] involving Goel) are for background context, not load-bearing premises. No parameter is fitted to evaluation data and then 'predicted' back. The circularity is real but minor and honestly disclosed, warranting a score of 2.
Axiom & Free-Parameter Ledger
free parameters (2)
- N (number of few-shot examples in N-Shot baseline) =
Not explicitly stated; described as examples for node creation, edge creation, and parameter changes
- Prompt design for T2MM, 0-Shot, and N-Shot =
Not specified in paper; hosted on OSF
axioms (4)
- domain assumption LLM-generated action sequences are more reliable than LLM-generated full XML for ontology-constrained model construction.
- domain assumption Graph edit distance is an adequate proxy for model correctness in science learning contexts.
- ad hoc to paper Procedurally generated templated requests are representative enough of learner behavior to demonstrate technical feasibility.
- domain assumption GPT 4.1-mini offers sufficient performance for this task at a suitable price point.
invented entities (2)
-
T2MM Architecture
independent evidence
-
Procedurally generated VERA dataset
no independent evidence
Reference graph
Works this paper leans on
-
[1]
https://doi.org/10.48550/arXiv.2209.02576, http://arxiv.org/abs/2209.02576, arXiv:2209.02576 [cs]
An, S., Bates, R., Rugaber, S., Hammock, J., Weigel, E., Goel, A.K.: Cognitive Assistance for Inquiry-Based Modeling (Sep 2022). https://doi.org/10.48550/arXiv.2209.02576, http://arxiv.org/abs/2209.02576, arXiv:2209.02576 [cs]
-
[2]
Bauer, A., Flatten, J., Popovic, Z.: Analysis of Problem-Solving Behavior in Open- Ended Scientific-Discovery Game Challenges. Tech. rep., International Educational Data Mining Society (Jun 2017), https://eric.ed.gov/?id=ED596582, eRIC Num- ber: ED596582
work page 2017
-
[3]
Learning and Individual Dif- ferences118, 102601 (Feb 2025)
Bewersdorff, A., Hartmann, C., Hornberger, M., Seßler, K., Bannert, M., Kas- neci, E., Kasneci, G., Zhai, X., Nerdel, C.: Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education. Learning and Individual Dif- ferences118, 102601 (Feb 2025). https://doi.org/10.1016/j.lind...
-
[4]
Applied Artificial Intelligence19(3- 4), 363–392 (Mar 2005)
Biswas, G., Leelawong, K., Schwartz, D., Vye, N., The Teachable Agents Group At Vande: LEARNING BY TEACHING: A NEW AGENT PARADIGM FOR EDUCATIONAL SOFTWARE. Applied Artificial Intelligence19(3- 4), 363–392 (Mar 2005). https://doi.org/10.1080/08839510590910200, http://www.tandfonline.com/doi/abs/10.1080/08839510590910200
-
[5]
Cognitive Research: Principles and Implications 1(1), 27 (Dec 2016)
Bobek, E., Tversky, B.: Creating visual explanations im- proves learning. Cognitive Research: Principles and Implications 1(1), 27 (Dec 2016). https://doi.org/10.1186/s41235-016-0031-6, https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235- 016-0031-6
-
[6]
International Journal of Human-Computer Studies64(11), 1099–1114 (Nov 2006)
Bridewell, W., Sánchez, J.N., Langley, P., Billman, D.: An interac- tive environment for the modeling and discovery of scientific knowl- edge. International Journal of Human-Computer Studies64(11), 1099–1114 (Nov 2006). https://doi.org/10.1016/j.ijhcs.2006.06.006, https://www.sciencedirect.com/science/article/pii/S1071581906001030
-
[7]
https://doi.org/10.48550/arXiv.2407.17544, http://arxiv.org/abs/2407.17544, arXiv:2407.17544 [cs]
Bulusu, A., Man, B., Jagmohan, A., Vempaty, A., Mari-Wyka, J., Akkil, D.: MathViz-E: A Case-study in Domain-Specialized Tool-Using Agents (Jul 2024). https://doi.org/10.48550/arXiv.2407.17544, http://arxiv.org/abs/2407.17544, arXiv:2407.17544 [cs]
-
[8]
Bybee, R.W.: Achieving scientific literacy. The Science Teacher62(7), 28 (Oct 1995),https://www.proquest.com/docview/214631067/abstract/63C7A5E3113D496BPQ/1, num Pages: 6
-
[9]
Christie, S., Rafferty, A., Lee, Z., Cutler, E., Tian, Y., Almoubayyed, H.: An Agen- tic Framework for Real-Time Pedagogical Plot Generation. pp. 309–316 (Jul 2025). https://doi.org/10.1007/978-3-031-98462-4_39
-
[10]
Journal of Science Teacher Education4(1), 1–8 (Mar 1993)
Flick, L.B.: The meanings of hands-on science. Journal of Science Teacher Education4(1), 1–8 (Mar 1993). https://doi.org/10.1007/BF02628851, https://www.tandfonline.com/doi/full/10.1007/BF02628851
-
[11]
In: Cristea, A.I., Walker, E., Lu, Y., Santos, O.C., Isotani, S
Gupta, A., Reddig, J., Calò, T., Weitekamp, D., MacLellan, C.: Beyond Final Answers: Evaluating Large Language Models for Math Tutoring. In: Cristea, A.I., Walker, E., Lu, Y., Santos, O.C., Isotani, S. (eds.) Artificial Intelli- gence in Education. pp. 323–337. Springer Nature Switzerland, Cham (2025). https://doi.org/10.1007/978-3-031-98414-3_23 14 John ...
-
[12]
In: Cristea, A.I., Walker, E., Lu, Y., Santos, O.C., Isotani, S
Hou, X., Forsyth, C., Andrews-Todd, J., Rice, J., Cai, Z., Jiang, Y., Zapata-Rivera, D., Graesser, A.: An LLM-Enhanced Multi-agent Architecture for Conversation- Based Assessment. In: Cristea, A.I., Walker, E., Lu, Y., Santos, O.C., Isotani, S. (eds.) Artificial Intelligence in Education. pp. 119–134. Springer Nature Switzer- land, Cham (2025). https://do...
-
[13]
Educational Psycholo- gist41(2), 75–86 (Jun 2006)
Kirschner, P.A., Sweller, J., Clark, R.E.: Why Minimal Guidance During Instruc- tion Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching. Educational Psycholo- gist41(2), 75–86 (Jun 2006)
work page 2006
-
[14]
Bibliovault OAI Repository, the University of Chicago Press87(Jan 2005)
Kuhn, D.: Education for Thinking. Bibliovault OAI Repository, the University of Chicago Press87(Jan 2005)
work page 2005
-
[15]
Computers in Human Behavior25(2), 284–289 (Mar 2009)
Leutner, D., Leopold, C., Sumfleth, E.: Cognitive load and science text comprehen- sion: Effects of drawing and mentally imagining text content. Computers in Human Behavior25(2), 284–289 (Mar 2009). https://doi.org/10.1016/j.chb.2008.12.010, https://www.sciencedirect.com/science/article/pii/S0747563208002197
-
[16]
In: Cristea, A.I., Walker, E., Lu, Y., Santos, O.C., Isotani, S
Li, W., Pea, R., Haber, N., Subramonyam, H.: CogGen: A Learner-Centered Gen- erative AI Architecture for Intelligent Tutoring with Programming Videos. In: Cristea, A.I., Walker, E., Lu, Y., Santos, O.C., Isotani, S. (eds.) Artificial In- telligence in Education. pp. 11–18. Springer Nature Switzerland, Cham (2025). https://doi.org/10.1007/978-3-031-98462-4_2
-
[17]
Educational Psychologist38(1), 43–52 (Jan 2003)
Mayer, R.E., Moreno, R.: Nine Ways to Reduce Cognitive Load in Multimedia Learning. Educational Psychologist38(1), 43–52 (Jan 2003)
work page 2003
-
[18]
Chelsea Green Publishing (2008)
Meadows, D.: Thinking in Systems: A Primer. Chelsea Green Publishing (2008)
work page 2008
-
[19]
In: Cristea, A.I., Walker, E., Lu, Y., Santos, O.C., Isotani, S
Mou,Y.,Fathi,F.,Thillen,B.,Decker,S.:WILLM:ASystemforAcademicWriting Improvement Based on Large Language Models. In: Cristea, A.I., Walker, E., Lu, Y., Santos, O.C., Isotani, S. (eds.) Artificial Intelligence in Education. pp. 36– 43.SpringerNatureSwitzerland,Cham(2025).https://doi.org/10.1007/978-3-031- 98462-4_5
-
[20]
In: Cristea, A.I., Walker, E., Lu, Y., San- tos, O.C., Isotani, S
Taneja, K., Singh, A., Goel, A.K.: Towards a Multimodal Document-Grounded Conversational AI System for Education. In: Cristea, A.I., Walker, E., Lu, Y., San- tos, O.C., Isotani, S. (eds.) Artificial Intelligence in Education. pp. 92–99. Springer Nature Switzerland, Cham (2025). https://doi.org/10.1007/978-3-031-98462-4_12
-
[21]
Tisue, S., Wilensky, U.: NetLogo: A Simple Environment for Modeling Complexity
-
[22]
Interactive Learning Environments21(4), 371–413 (Aug 2013)
VanLehn, K.: Model construction as a learning activity: a design space and review. Interactive Learning Environments21(4), 371–413 (Aug 2013). https://doi.org/10.1080/10494820.2013.803125
-
[23]
Cognition and Instruction24(2), 171–209 (Jun 2006)
Wilensky, U., Reisman, K.: Thinking Like a Wolf, a Sheep, or a Firefly: Learning Biology Through Constructing and Testing Computational Theories—An Embod- ied Modeling Approach. Cognition and Instruction24(2), 171–209 (Jun 2006). https://doi.org/10.1207/s1532690xci2402_1
-
[24]
https://doi.org/10.48550/arXiv.2601.05162, http://arxiv.org/abs/2601.05162, arXiv:2601.05162 [cs]
Yu, J., Jiang, D.: GenAI-DrawIO-Creator: A Framework for Automated Diagram Generation (Jan 2026). https://doi.org/10.48550/arXiv.2601.05162, http://arxiv.org/abs/2601.05162, arXiv:2601.05162 [cs]
-
[25]
https://doi.org/10.48550/arXiv.2510.20229, http://arxiv.org/abs/2510.20229, arXiv:2510.20229 [cs]
Zheng, G., Qian, J., Tang, J., Yang, S.: Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context (Oct 2025). https://doi.org/10.48550/arXiv.2510.20229, http://arxiv.org/abs/2510.20229, arXiv:2510.20229 [cs]
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.