Pith. sign in

REVIEW 3 major objections 6 minor 4 cited by

AI2T: Building Trustable AI Tutors by Interactively Teaching a Self-Aware Learning Agent

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A non-programmer can author a complete model-tracing tutor in 20-30 minutes by demonstrating solutions and grading the agent's attempts, with certainty scores as the stopping signal.

desk verdict Process-learning and the study 2 results are worth taking seriously, but the certainty-based stopping claim is preliminary and needs a controlled study before it can carry the weight the abstract puts on it. read the letter →

arxiv 2411.17924 v1 pith:A377EVRB submitted 2024-11-26 cs.HC cs.AIcs.LG

Daniel Weitekamp , Erik Harpstead , Kenneth Koedinger This is my paper · ORCID
classification cs.HCcs.AIcs.LG
keywords interactivetasklearningauthoring-by-tutoringmodel-tracingtutorshierarchicalnetworksprogrammingbydemonstrationcertaintyestimationintelligenttutoringsystemsself-aware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AI2T is an interactively teachable agent that lets non-programmers author intelligent tutoring systems by doing what tutors do: demonstrating step-by-step solutions and then grading the agent's own attempts. The paper's central claim is that from roughly 20-30 minutes of such interaction, AI2T can induce hierarchical production-rule programs that are model-tracing complete, meaning they permit every correct next action in a problem state and no incorrect ones. The second claim is that STAND, the agent's precondition-learning algorithm, produces certainty estimates that reliably track actual learning progress, so an author can use low certainty scores as a signal to keep training. If these claims hold, the most difficult part of building a model-tracing tutor, traditionally estimated at 200-300 programming hours per hour of instruction, becomes a short, programming-free teaching session.

What carries the argument

The load-bearing mechanism is STAND, a self-aware precondition induction algorithm that maintains an approximate version space: rather than fitting one classifier, it compresses and tracks the space of all good generalizations consistent with the author's positive and negative feedback, and it derives an instance certainty score from how much those generalizations disagree on a candidate action. Around STAND sits process-learning, a fourth learning mechanism that induces hierarchical task networks from the author's action sequences, so solution ordering starts sequential and is relaxed only when demonstrations show alternative orders, which removes the need for dozens of negative examples to teach ordering constraints.

What would settle it

Run the paper's exact simulation protocol but inject a single incorrect correctness label or one wrong formula annotation at an early training problem; if STAND plus HTN induction then fails to reach 100% holdout completeness in most repetitions, the completeness result depends on flawless teaching rather than on the learning mechanism itself.

Watch

Extended reading notes

Core claim

On the paper's own terms, AI2T succeeds where prior authoring-by-tutoring systems fell short: it induces correct and complete model-tracing behavior, not merely high-accuracy problem solving. The agent learns hierarchical task networks from demonstrated action sequences, and STAND learns the preconditions that gate when each skill may apply. In simulation, STAND combined with HTN induction reached 100% holdout model-tracing completeness in all 40 repetitions for multicolumn addition and 38 of 40 for fraction arithmetic, outperforming decision trees, random forests, and XGBoost. In user study 2, 8 of 10 untrained participants (six of them self-reported non-programmers) produced 100% complete multicolumn-addition tutors, and 5 of 10 produced 100% complete fraction tutors, with median authoring times around 22 minutes per topic. The paper also claims that STAND's instance certainty is a reliable heuristic: unlike random-forest or XGBoost prediction probabilities, which tracked holdout performance only at chance levels, STAND's certainty scores moved with actual learning gains.

Load-bearing premise

The strongest efficiency and completeness results assume an ideal author who always demonstrates correct actions, always selects the correct formula to explain a demonstration, and grades every proposed action accurately; the paper's own user studies show real authors make uncaught mistakes, so the 20-30 minute numbers depend on that flawless teaching behavior.

Editorial extensions

If this is right

  • If the central claim is correct, non-programmers can author two representative model-tracing tutors (multicolumn addition and fraction arithmetic) in about 20-30 minutes per topic, compared with the 200-300 programming hours per hour of instruction traditionally estimated for such tutors.
  • If certainty scores track learning progress, then an author who waits until every proposed action reaches 100% certainty has a principled stopping rule for training, rather than guessing how many problems are enough.
  • Because AI2T induces inspectable hierarchical task networks rather than opaque statistical models, the resulting tutor behavior is checkable and does not carry the hallucination risk of LLM-based tutoring.
  • The same process-learning mechanism that organizes skills into unordered groups (and conditional or optional steps) makes the authoring interface simpler and reduces the number of states an author must grade.
  • If the tool's scope expands to more domains with adequate primitive libraries, the authoring-by-tutoring approach could replace hand-programming for a wider class of procedural skills.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our inference: if STAND's certainty is really a measure of version-space ambiguity, then the same stopping heuristic could be lifted out of AI2T and applied to any interactive task learner that maintains a version space, giving those systems a self-assessment signal they currently lack.
  • Our inference: the simulation's ideal-user assumption is the fragile link; real authors make uncaught errors, and the paper's own data show that two study-2 participants fell below 50% completeness because of mistakes they never found, so error-detection support is the next bottleneck rather than raw learning efficiency.
  • Our inference: the paper's comparison against LLMs suggests a complementary architecture in which LLMs generate or parse interface primitives and natural-language explanations while AI2T supplies the trustworthy rule induction, a division of labor neither system achieves alone.
  • Our inference: a randomized experiment that turns the certainty display on and off for matched authors would directly test whether the display causes completeness, which the paper notes it has not yet run.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents AI2T, an interactive authoring-by-tutoring system for building model-tracing intelligent tutoring systems (ITSs). Authors train the agent by demonstrating step-by-step solutions and grading the agent's proposed actions; the system induces hierarchical task networks (HTNs) via a process-learning mechanism and learns preconditions with STAND, a version-space-based algorithm that also outputs instance-certainty scores. The authors report simulation experiments in multicolumn addition and fraction arithmetic comparing STAND with decision trees, random forests, and XGBoost on productive monotonicity, precision at high certainty, and holdout completeness, and two user studies (n=10 each) in which participants authored tutors. The paper claims that authors can achieve correct and complete model-tracing behavior in about 20-30 minutes per topic and can use STAND's certainty scores as a heuristic to decide when training is sufficient.

Significance. If the central claims hold, this is a substantial contribution to ITS authoring and interactive task learning: it demonstrates a concrete path for non-programmers to author flexible model-tracing tutors, and it provides a self-aware certainty signal that could address a long-standing gap in authoring-by-tutoring. The simulation comparison against strong baselines, the detailed interaction-design contributions (skill application window, behavior graph, unordered groups), and the honest reporting of study 1's failures and study 2's limitations are all strengths. The user-study results, with 8/10 participants reaching 100% completeness in multicolumn addition in study 2, are encouraging and clearly improve on prior work. However, the evidence for the certainty-based stopping heuristic is preliminary, and several load-bearing quantitative claims need strengthening before the paper's strongest conclusions can be accepted.

major comments (3)
  1. [Section 6.3.1 and Section 6.4 (also Abstract and Section 3.5)] The claim that 'our user study shows that authors can use STAND's certainty heuristic to estimate when AI2T has been trained on enough diverse problems' is not supported by the study 2 evidence as reported. Only 5 of 10 participants noticed the certainty scores and 4 of 10 reported using them; there is no control condition without scores, no random assignment, and the reported use is self-reported. Section 6.4 itself concedes that a future randomized experiment is needed. Because the abstract and Section 3.5 make the certainty heuristic a central contribution, the manuscript should either substantially soften this claim or provide an additional quantitative analysis (e.g., comparing stopping outcomes for participants who did and did not use the scores) that directly supports it.
  2. [Section 5.1, Figure 11, and Table 4] In multicolumn addition, STAND's productive monotonicity is below 50% for approximately the first 60 training problems, yet study 2 participants stopped after a median of 11 problems and 22 minutes. The certainty-based stopping heuristic is therefore being applied in exactly the regime where the simulation suggests certainty changes are not yet reliably aligned with holdout performance. The paper should report productive monotonicity at early training sizes (e.g., problems 5-20) for both domains and either show that early-problem monotonicity is adequate for the observed stopping behavior or explicitly restrict the certainty-heuristic claim to later training.
  3. [Section 5, automated training system paragraph] The simulation experiments assume an ideal user who always selects the correct formula annotation and grades every proposed action. This assumption is load-bearing for the comparative data-efficiency claims in Tables 1 and 2, because the user studies demonstrate that real authors make uncaught training mistakes, and two study 2 participants ended below 50% completeness in fraction arithmetic. The paper should at minimum quantify the sensitivity of the STAND-vs-baseline comparisons to annotation and grading noise (e.g., a noise-injection experiment), and the conclusions should state that the strong data-efficiency numbers assume a perfect trainer.
minor comments (6)
  1. [Section 3.3] The heading 'Supporting Completene Correctness Feedback for Each Problem State' contains a typo; it should be 'Supporting Complete Correctness Feedback for Each Problem State.'
  2. [Section 5, first paragraph] The text says 'in the two domains that we had participants author in our user study (section 5)', but the user studies are described in Section 6, not Section 5.
  3. [Sections 2 and 4] There are repeated spelling errors: 'heirarchical' should be 'hierarchical', and 'processs-learning' should be 'process-learning'.
  4. [Figures 11 and 12] The simulation results are averaged over 40 repetitions but no error bars or confidence intervals are shown; adding them would help the reader assess the stability of the reported differences, especially the early-training comparisons that are central to the certainty-stopping claim.
  5. [Section 5, classifier descriptions] The hyperparameters for the comparison classifiers are only partially specified (e.g., random forest size is given but XGBoost details are not); please provide the exact settings and any hyperparameter tuning procedure for reproducibility.
  6. [Section 4.2] STAND is the core algorithmic contribution but is only summarized here and defined in the authors' prior preprint [57]; the paper would be more self-contained if it included a concise formal description of the version-space representation and the certainty computation, or a precise pointer to the relevant equations in [57].

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: STAND is imported from the authors' prior work but is independently re-evaluated on holdout data and with human users; the certainty heuristic is an empirical claim, not a definitional identity.

full rationale

The paper's load-bearing machinery, STAND and its instance-certainty measure, comes from the authors' prior paper [57]. That is a self-citation, but it is not circular here: Section 5 re-tests STAND against decision trees, random forests, and XGBoost on two held-out domains using productive monotonicity, precision at high certainty, and per-problem completeness, and Section 6 tests the certainty heuristic with human authors. These evaluations use data not fitted into the claim and would be falsifiable if STAND's certainty did not track holdout performance. No equation or definition reduces the target result to its inputs: productive monotonicity is a measured correlation between certainty changes and correct labels; instance certainty is not definitionally tied to ground truth (indeed, Figure 2 shows 97%-certainty actions that are incorrect). The simulation's ideal-user assumption is stated as a replication of an ideal author (Section 5) and limits external validity, but it is not a circular reduction. Section 6.4 explicitly concedes that a future randomized experiment is needed to establish the causal effect of the certainty display; this is a statistical-strength limitation, not a circularity. Overall, the derivation chain is self-contained against external benchmarks, so no circular step is identified.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hand-authored ground-truth behavior, an ideal-user simulation, and a finite holdout metric. These are reasonable for a systems paper but should be stated explicitly, and they limit the generality of the reported completeness numbers.

free parameters (1)
  • Comparison classifier hyperparameters = RF: 100 trees; all tree methods: no depth or leaf-size limits
    The authors state 'In all models no limits are set on tree depth or leaf size' and use 100 trees for random forests. These hand choices can affect the baseline comparison, but they are standard and not tuned to data.
assumptions (4)
  • domain assumption The hand-authored ground-truth model-tracing behavior is the correct target for both domains.
    Section 2 defines what counts as correct, such as allowing carry actions in either order for multicolumn addition. All completeness metrics are measured against this manually specified target.
  • ad hoc to paper The simulated ideal user provides perfect formula annotations and grades every proposed action.
    Section 5 states the training system 'replicates the behavior of an ideal user who always selects the correct formula to explain each demo.' This removes the main source of human error from the learning-efficiency simulation.
  • domain assumption Condition learning is noiseless and perfectly separable for the tree-based classifiers.
    Section 5 states 'Since condition learning is noiseless the trees will already tend to not become more complex than the ideal solution,' which justifies the absence of noise handling in the comparison.
  • domain assumption A 100-problem holdout set adequately measures model-tracing completeness.
    Section 6 reports that agents are 'scored their agents' model-tracing performance on a large holdout set of 100 problems.' This operationalizes completeness on a finite sample rather than proving it for all possible problems.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AI2T: Building Trustable AI Tutors by Interactively Teaching a Self-Aware Learning Agent." pith.science (2026). https://pith.science/paper/A377EVRB

@misc{pith2026241117924,
  author       = {Pith},
  title        = {Pith review of: AI2T: Building Trustable AI Tutors by Interactively Teaching a Self-Aware Learning Agent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A377EVRB}},
  note         = {Machine review of arXiv:2411.17924}
}
read the original abstract

AI2T is an interactively teachable AI for authoring intelligent tutoring systems (ITSs). Authors tutor AI2T by providing a few step-by-step solutions and then grading AI2T's own problem-solving attempts. From just 20-30 minutes of interactive training, AI2T can induce robust rules for step-by-step solution tracking (i.e., model-tracing). As AI2T learns it can accurately estimate its certainty of performing correctly on unseen problem steps using STAND: a self-aware precondition learning algorithm that outperforms state-of-the-art methods like XGBoost. Our user study shows that authors can use STAND's certainty heuristic to estimate when AI2T has been trained on enough diverse problems to induce correct and complete model-tracing programs. AI2T-induced programs are more reliable than hallucination-prone LLMs and prior authoring-by-tutoring approaches. With its self-aware induction of hierarchical rules, AI2T offers a path toward trustable data-efficient authoring-by-tutoring for complex ITSs that normally require as many as 200-300 hours of programming per hour of instruction.

Figures

Figures reproduced from arXiv: 2411.17924 by the authors.

Figure 1
Figure 1. Authors tutor AI2T in HTML interfaces and grade its step-by-step solutions. Certainty scores help authors determine when [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. AI2T’s interface. The user has just demonstrated two actions. They show up as blue dashed edges in the behavior graph [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. (a) The user has demonstrated the converted fraction 6. (b) The agent displays several possible explanations for this demon [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: (a) The user selects the interface elements that they used as arguments to compute the demonstrated 4. They first select the 8 [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The skill application window shows each of the actions proposed by the agent. Currently, action 3 of 4 is selected. The proposed [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Interface element overlays have an action count indicator when one or more proposed actions apply to that element. (Top) [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: (a) A generated behavior graph and (b) a behavior graph zoomed into the current state. Of 3 proposed actions, the 1st and 3rd [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: (a) A behavior graph with two visible unordered groups. (b) The Move On button moves the author to a new state by applying [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Three actions proposed for the initial state of 1/9+1/3. Certainty scores are shown over edges in the behavior graph (a) and in [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: A possible HTN for a fraction arithmetic ITS. A task trace (dotted-purple) and action trace (dashed-green) are shown of the [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Productive Monotonicity By Problem Our results show that STAND’s instance certainty measure has considerably higher overall productive monotonicity than the two ensemble methods’ prediction probabilities. Thus, increases in instance certainty reflect actual increases …
Figure 12
Figure 12. Figure 12: Average holdout completeness by problem. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Study 1 Interface. An intermediate state of problem 597+346. Two of three actions (outgoing graph edges in top-left) have [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Normal AI2T interface (study 2) in an intermediate state of 189+542. There are 4 proposed actions, shown as skill application [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Decomposed Inductive Procedure Learning: Learning Academic Tasks with Human-Like Data Efficiency

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A three-mechanism symbolic learner reaches under 10 percent error in about 20 problems on two arithmetic tutor tasks, versus thousands for reinforcement learning and 2,000 to 8,000 for a single decision tree.

  2. TutorGym: A Testbed for Evaluating AI Agents as Tutors and Students

    cs.AI 2025-05 conditional novelty 6.0 of 10

    TutorGym connects LLMs, RL agents, and computational learning models to 223 existing tutoring domains, and initial evaluations show LLMs grade steps poorly but can imitate human learning curves with in-context learning.

  3. Model Human Learners: Computational Models to Guide Instructional Design

    cs.HC 2025-02 conditional novelty 6.0 of 10

    A parameter-free computational model of learning predicted the main effects of two human A/B tutoring experiments, a first for the Model Human Learner concept.

  4. Knowledge Distillation for Automated AI Tutor Evaluation

    cs.CL 2026-07 conditional novelty 4.5 of 10

    Knowledge distillation from Claude Opus 4.7 into Llama 3.1 8B yields FATE, which scores AI tutors on four BEA pedagogical dimensions and ranks commercial models.

Reference graph

Works this paper leans on

61 extracted references · 54 canonical work pages · cited by 4 Pith papers

  1. [1]

    Vincent Aleven, Bruce M McLaren, Jonathan Sewall, Martin Van Velsen, Octav Popescu, Sandra Demi, Michael Ringenberg, and Kenneth R Koedinger

    work page

  2. [2]

    Vincent Aleven, Elizabeth A McLaughlin, R Amos Glenn, and Kenneth R Koedinger. 2016. Instruction based on adaptive learning technologies. Handbook of research on learning and instruction 2 (2016), 522–560

    work page 2016

  3. [3]

    Stephen B Blessing. 1997. A programming by demonstration authoring tool for model-tracing tutors. International Journal of Artificial Intelligence in Education 8 (1997), 233–261

    work page 1997

  4. [4]

    Leo Breiman. 1996. Bagging predictors. Machine learning 24 (1996), 123–140

    1996

  5. [5]

    Leo Breiman. 2001. Random forests. Machine learning 45 (2001), 5–32

    2001

  6. [6]

    Leo Breiman. 2017. Classification and Regression Trees. Routledge

    work page 2017

  7. [7]

    Tommaso Calo and Christopher Maclellan. 2024. Towards Educator-Driven Tutor Authoring: Generative AI Approaches for Creating Intelligent Tutor Interfaces. In Proceedings of the Eleventh ACM Conference on Learning@ Scale . 305–309

    work page 2024

  8. [8]

    Paulo Carvalho, Napol Rachatasumrit, and Kenneth R Koedinger. 2022. Learning depends on knowledge: The benefits of retrieval practice vary for facts and skills. In Proceedings of the Annual Meeting of the Cognitive Science Society , Vol. 44

    work page 2022

Show all 61 references
  1. [9]

    Hao Cen, Kenneth Koedinger, and Brian Junker. 2006. Learning Factors Analysis–A General Method for Cognitive Model Evaluation and Improvement. In International Conference on Intelligent Tutoring Systems . Springer, 164–175

    2006

  2. [10]

    Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining . 785–794

    2016

  3. [11]

    Michelene TH Chi and Ruth Wylie. 2014. The ICAP framework: Linking cognitive engagement to active learning outcomes.Educational psychologist 49, 4 (2014), 219–243

    2014

  4. [12]

    Albert T Corbett and John R Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge.User modeling and user-adapted interaction 4, 4 (1994), 253–278

    1994

  5. [13]

    Allen Cypher and Daniel Conrad Halbert. 1993. Watch What I Do: Programming by Demonstration . MIT press

    1993

  6. [14]

    Rui Dong, Zhicheng Huang, Ian Iong Lam, Yan Chen, and Xinyu Wang. 2022. WebRobot: web robotic process automation using interactive programming-by-demonstration. In Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implemen- tation. 152–167

    2022

  7. [15]

    Jerry Alan Fails and Dan R Olsen Jr. 2003. Interactive machine learning. In Proceedings of the 8th international conference on Intelligent user interfaces . 39–45

    2003

  8. [16]

    Bhavya Ghai, Q Vera Liao, Yunfeng Zhang, Rachel Bellamy, and Klaus Mueller. 2021. Explainable active learning (xal) toward ai explanations as interfaces for machine teachers. Proceedings of the ACM on Human-Computer Interaction 4, CSCW3 (2021), 1–28

    2021

  9. [17]

    Arthur C Graesser, Mark W Conley, and Andrew Olney. 2012. Intelligent tutoring systems. (2012)

    2012

  10. [18]

    Sumit Gulwani, William R Harris, and Rishabh Singh. 2012. Spreadsheet data manipulation using examples. Commun. ACM 55, 8 (2012), 97–105

    2012

  11. [19]

    Yu Guo, Joseph E Beck, and Neil T Heffernan. 2008. Trying to Reduce Bottom-out hinting: Will telling student how many hints they have left help?. In International Conference on Intelligent Tutoring Systems . Springer, 774–778

    2008

  12. [20]

    Erik Harpstead, Christopher J MacLellan, Daniel Weitekamp, and Kenneth R Koedinger. [n. d.]. The use simulated learners in adaptive education. AIAED-19: AI+ Adaptive Education ([n. d.]), 1–3

  13. [21]

    Neil T Heffernan and Cristina Lindquist Heffernan. 2014. The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education 24...

    2014

  14. [22]

    Chad Hogg, Héctor Munoz-Avila, and Ugur Kuter. 2008. HTN-MAKER: Learning HTNs with Minimal Additional Knowledge Engineering Required.. In AAAI. 950–956

    2008

  15. [23]

    Scott B Huffman and John E Laird. 1995. Flexibly instructable agents. Journal of Artificial Intelligence Research 3 (1995), 271–324

    1995

  16. [24]

    Viswanathan Kodaganallur, Rob R Weitz, and David Rosenthal. 2005. A comparison of model-tracing and constraint-based intelligent tutoring paradigms. International Journal of Artificial Intelligence in Education 15, 2 (2005), 117–144

    2005

  17. [25]

    Kenneth R Koedinger and Vincent Aleven. 2007. Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review 19 (2007), 239–264

    2007

  18. [26]

    Kenneth R Koedinger, Jihee Kim, Julianna Zhuxin Jia, Elizabeth A McLaughlin, and Norman L Bier. 2015. Learning is not a spectator sport: Doing is better than watching for learning from a MOOC. In Proceedings of the second (2015) ACM conference on learning@ scale . 111–120

    2015

  19. [27]

    James A Kulik and JD Fletcher. 2016. Effectiveness of intelligent tutoring systems: a meta-analytic review. Review of educational research 86, 1 (2016), 42–78

    2016

  20. [28]

    John E Laird, Kevin Gluck, John Anderson, Kenneth D Forbus, Odest Chadwicke Jenkins, Christian Lebiere, Dario Salvucci, Matthias Scheutz, Andrea Thomaz, Greg Trafton, Robert E Wray, Shiwali Mohan, and James R Kirk. 2017. Interactive Task Learning. IEEE Intelligent Systems 32, ...

    2017

  21. [29]

    Tessa Lau, Steven A Wolfman, Pedro Domingos, and Daniel S Weld. 2003. Programming by demonstration using version space algebra. Machine Learning 53, 1 (2003), 111–156

    2003

  22. [30]

    Lane Lawley and Christopher Maclellan. 2024. VAL: Interactive Task Learning with GPT Dialog Parsing. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–18

    2024

  23. [31]

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436–444

    2015

  24. [32]

    Nan Li, William W Cohen, Kenneth R Koedinger, and Noboru Matsuda. 2011. A machine learning approach for automatic student model discovery.. In Edm. ERIC, 31–40

    2011

  25. [33]

    Nan Li, Noboru Matsuda, William W Cohen, and Kenneth R Koedinger. 2015. Integrating representation learning and skill learning in a human-like intelligent agent. Artificial Intelligence 219 (2015), 67–91

    2015

  26. [34]

    Toby Jia-Jun Li, Amos Azaria, and Brad A Myers. 2017. SUGILITE: Creating Multimodal Smartphone Automation by Demonstration. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems . ACM, 6038–6049

    2017

  27. [35]

    Christopher J MacLellan. 2017. Computational Models of Human Learning: Applications for Tutor Development, Behavior Prediction, and Theory Testing. Ph. D. Dissertation. Carnegie Mellon University

    2017

  28. [36]

    Christopher J Maclellan, Erik Harpstead, Rony Patel, and Kenneth R Koedinger. 2016. The Apprentice Learner Architecture: Closing the Loop between Learning Theory and Educational Data. International Educational Data Mining Society (2016)

    2016

  29. [37]

    Christopher J MacLellan and Kenneth R Koedinger. 2020. Domain-General Tutor Authoring with Apprentice Learner Models. International Journal of Artificial Intelligence in Education (2020), 1–42

    2020

  30. [38]

    Noboru Matsuda. 2022. Teachable agent as an interactive tool for cognitive task analysis: A case study for authoring an expert model. International Journal of Artificial Intelligence in Education 32, 1 (2022), 48–75

    2022

  31. [39]

    Noboru Matsuda, William W Cohen, and Kenneth R Koedinger. 2015. Teaching the Teacher: Tutoring SimStudent Leads to More Effective Cognitive Tutor Authoring. International Journal of Artificial Intelligence in Education 25, 1 (2015), 1–34

    2015

  32. [40]

    Richard G McDaniel and Brad A Myers. 1997. Gamut: demonstrating whole applications. In Proceedings of the 10th annual ACM symposium on User interface software and technology . 81–82

    1997

  33. [41]

    Katherine L McNeill, David J Lizotte, Joseph Krajcik, and Ronald W Marx. 2006. Supporting students’ construction of scientific explanations by fading scaffolds in instructional materials. The Journal of the Learning Sciences 15, 2 (2006), 153–191

    2006

  34. [42]

    Tom M Mitchell. 1982. Generalization as Search. Artificial Intelligence 18, 2 (1982), 203–226

    1982

  35. [43]

    Brad A Myers. 1988. Creating user interfaces by demonstration . Academic Press Professional, Inc

    1988

  36. [44]

    Hyacinth S Nwana. 1990. Intelligent tutoring systems: an overview. Artificial Intelligence Review 4, 4 (1990), 251–277

    1990

  37. [45]

    Benjamin D Nye, Dillon Mee, and Mark G Core. 2023. Generative Large Language Models for Dialog-Based Tutoring: An Early Consideration of Opportunities and Concerns.. In LLM@ AIED. 78–88

    2023

  38. [46]

    Scott Ososky, Keith Brawner, Benjamin Goldberg, and Robert Sottilare. 2016. GIFT Cloud: Improving usability of adaptive tutor authoring tools within a web-based application. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting , Vol. 60. SAGE Publications ...

    2016

  39. [47]

    Zachary A Pardos, Matthew Tang, Ioannis Anastasopoulos, Shreya K Sheel, and Ethan Zhang. 2023. Oatutor: An open-source adaptive tutoring system and curated content library for learning sciences research. In Proceedings of the 2023 chi conference on human factors in computing s...

    2023

  40. [48]

    Kevin Pu, Jim Yang, Angel Yuan, Minyi Ma, Rui Dong, Xinyu Wang, Yan Chen, and Tovi Grossman. 2023. DiLogics: Creating Web Automation Programs With Diverse Logics. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology . 1–15

    2023

  41. [49]

    Sadhana Puntambekar and Roland Hubscher. 2005. Tools for scaffolding students in a complex learning environment: What have we gained and what have we missed? Educational psychologist 40, 1 (2005), 1–12

    2005

  42. [50]

    Momin N Siddiqui, Adit Gupta, Jennifer M Reddig, and Christopher J MacLellan. 2024. HTN-Based Tutors: A New Intelligent Tutoring Framework Based on Hierarchical Task Networks. In Proceedings of the Eleventh ACM Conference on Learning@ Scale . 491–495

    2024

  43. [51]

    Kurt VanLehn. 1987. Learning one subprocedure per lesson. Artificial Intelligence 31, 1 (1987), 1–40

    1987

  44. [52]

    Kurt VanLehn. 1990. Mind bugs: The origins of procedural misconceptions . MIT press

    1990

  45. [53]

    Kurt VanLehn. 2011. The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist 46, 4 (2011), 197–221

    2011

  46. [54]

    2020 in press

    Daniel Weitekamp, Erik Harpstead, and Kenneth Koedinger. 2020 in press. An Interaction Design for Machine Teaching to Develop AI Tutors. CHI (2020 in press)

    2020

  47. [55]

    Daniel Weitekamp, Erik Harpstead, and Kenneth Koedinger. 2021. Toward Stable Asymptotic Learning with Simulated Learners. InInternational Conference on Artificial Intelligence in Education . Springer, 390–394

    2021

  48. [56]

    Daniel Weitekamp and Kenneth Koedinger. 2023. Computational models of learning: Deepening care and carefulness in AI in education. In International Conference on Artificial Intelligence in Education . Springer, 13–25

    2023

  49. [57]

    Daniel Weitekamp and Kenneth Koedinger. 2024. STAND: Data-Efficient and Self-Aware Precondition Induction for Interactive Task Learning. arXiv:2409.07653 [cs.LG] https://arxiv.org/abs/2409.07653

    2024

  50. [58]

    Daniel Weitekamp, Napol Rachatasumrit, Rachael Wei, Erik Harpstead, and Kenneth Koedinger. 2023. Simulating Learning from Language and Examples. In International Conference on Artificial Intelligence in Education . Springer, 580–586. AI2T: Building Trustable AI Tutors by Inter...

    2023

  51. [59]

    Daniel Weitekamp III, Erik Harpstead, Christopher J MacLellan, Napol Rachatasumrit, and Kenneth R Koedinger. 2019. Toward Near Zero-Parameter Prediction Using a Computational Model of Student Learning. International Educational Data Mining Society (2019)

    2019

  52. [60]

    Xiaojin Zhu. 2015. Machine teaching: An inverse problem to machine learning and an approach toward optimal education. In Proceedings of the AAAI conference on artificial intelligence , Vol. 29

    2015

  53. [2016]

    International Journal of Artificial Intelligence in Education 26, 1 (2016), 224–269

    Example-Tracing Tutors: Intelligent Tutor Development for Non-Programmers. International Journal of Artificial Intelligence in Education 26, 1 (2016), 224–269

    2016

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.