{"id":"ed1ce1dd-8bbc-4342-a3dc-a99728f437ea","arxiv_id":"2602.16026","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"A calculus teacher proposes using Maxima to make algebra and calculus expressions into visual, manipulable objects with blanks, to help students with weak procedural foundations.","lead":"This education paper uses Maxima to turn algebra and calculus expressions into visual, manipulable objects, such as proof steps with blanks, to help students with very weak math foundations. It describes the tools and classroom anecdotes, but offers no controlled evidence that they improve learning.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central pedagogical claim rests on an untested learning-transfer assumption; Section 7 admits only small-class, non-rigorous testing, so the proposal remains a set of Maxima macros rather than a validated teaching strategy.","rationale":"The reader's verdict of CONDITIONAL is appropriate. The paper openly acknowledges its lack of rigorous evaluation, so it does not overclaim. However, the entire value of the proposal hinges on the learning-transfer hypothesis. The stress-test confirms the reader's weakest-assumption identification: no empirical evidence connects the manipulable objects to improved student outcomes. The concrete test I propose is a minimal controlled experiment that would directly probe this transfer. If the test is not run or fails, the paper remains a suggestive tool showcase, not a validated methodology. I considered other possible concerns—such as the incompleteness of the Maxima code or the internal typo in the chain-rule example (g'(x) = -x vs. -1)—but these are secondary to the central educational claim. The code and typos could be fixed without addressing the fundamental question of whether the approach helps students. Therefore, the reader's CONDITIONAL verdict stands unchanged, grounded in exactly the learning-transfer assumption they identified.","tokens_in":13088,"tokens_out":4820,"duration_ms":52145,"concrete_test":"Run a small controlled pilot with students who fit the profile of 'E' (can solve x+2=5 by the inverse method but cannot substitute x=3 into x+2=5). Randomly assign them to two groups: one receives two sessions of the proposed Maxima manipulable-object exercises (e.g., working with 'holes' on chain-rule examples), the other receives the same amount of conventional substitution/derivative practice. Pre- and post-test both groups on structurally novel tasks: substituting a number into an expression, justifying each '=' step, and decomposing a composite derivative. If the treatment group does not show significantly greater gains on these transfer tasks, the central pedagogical claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Section 1) is that presenting expressions and trees as basic objects and introducing an explicit substitution operation 'very early in the course, before theorems and quantifiers' will help students with extremely bad foundations. The mechanism is that manipulable Maxima objects—especially 'holes'—will let such students see how substitution and justification work, and eventually reach Mason's 'splitting of attention' (Section 7). This is a learning-transfer hypothesis: from interacting with tree diagrams and substitution syntax, students will acquire pattern-matching skills and logical understanding that transfer to novel calculus problems. The author explicitly disclaims any supporting evidence: 'I have only a few examples ready, and they were tested only on small classes, and without a rigorous methodology' (Section 7). No data, control group, or even a systematic observational protocol is provided. Moreover, the intervention may presuppose the very competence it aims to teach: a student like 'E', who cannot substitute x=3 into x+2=5, must first understand that a symbolic expression is a parse tree with named child positions before the tree/matrix visualization becomes meaningful. The paper does not address this prerequisite. Without a demonstrated causal link from using the tool to improved substitution/justification performance, the strongest claim is unsupported; the paper is a collection of illustrative macros and anecdotal classroom observations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that calculus instructors facing many students with 'extremely bad foundations'—students who know mathematics only as memorized procedures—should introduce expressions and trees as foundational objects and teach an explicit substitution operation very early, before theorems and quantifiers (Section 1). The author proposes Maxima as the vehicle: custom operators ('s', 'V', 'sss', 'ssu'), Lisp-tree visualizations ('lisptreem', 'lisptree2'), and 'holes' ('mkholes') turn symbolic expressions, equalities, and justifications into manipulable, expandable objects (Sections 2, 4, 6). The paper includes Maxima input/output examples, a discussion of why Lean is not used (Section 5), and a closing section (7) that explicitly labels the approach a work in progress with only informal classroom trials.","tokens_in":13435,"tokens_out":7350,"duration_ms":65724,"significance":"The paper is genuinely useful as an exposition of a pedagogical idea: it connects a concrete computational tool (Maxima) to documented difficulties with procedural versus conceptual knowledge (Engelbrecht-Bergsten), substitution, and equality (Fischbein, Thomas, Sfard), and it introduces a distinctive visual language for substitutions and 'holes' that could be adapted by other instructors. The Maxima examples are mostly self-consistent and give a clear sense of the intended interaction. The 'holes' idea—rewriting a true expression with unknown parts—is a novel and promising exercise format. However, the paper's central evaluative claim is not supported by evidence; it is a proposal with anecdotal illustrations, not a validated teaching strategy. If the paper is revised to frame its contribution as a testable hypothesis and to provide either pilot data or a detailed evaluation plan, its value would be considerably clearer.","major_comments":[{"comment":"The load-bearing assertion in Section 1—that 'when we have lots of students like the ones that I described above it is a good idea to present expressions and trees as some of our most basic objects'—is stated as a recommendation but is not supported by any empirical evidence. Section 7 explicitly concedes: 'I am treating as a work in progress; I have only a few examples ready, and they were tested only on small classes, and without a rigorous methodology.' There is no control group, no measured learning outcome, no systematic classroom protocol, and no analysis of transfer. The authors should either (a) present this as a conjecture with explicit falsifiable predictions and a plan for evaluation, or (b) include a pilot study, however small, with pre/post measures. In its current form, the abstract and introduction overclaim relative to the evidence.","section":"Section 1; Section 7"},{"comment":"The paper states at the end of Section 6: 'I haven't shown (yet!) how to transform our expandable proof from section 1 into a manipulable object – I am just showing how to manipulate objects that look like its justifications.' This is a significant missing component. The Section 1 L–M–R proof layout is central to the paper's motivation, but the claimed manipulable-object version of that proof is not provided. The paper would need either to implement and show such a version, or to restrict its claims to the objects that are actually demonstrated (substitution and justifications).","section":"Section 6"},{"comment":"A deeper conceptual issue: the target student E cannot substitute x=3 into x+2=5. The proposed visualization relies on the student recognizing that an expression is a parse tree with named child positions (e.g., lisptreem(f(a,b)) shows f applied to two children). For a student who does not already see substitution as a meaningful operation, it is not clear why the tree diagram is more transparent than the original equation. The paper does not address this prerequisite or offer a diagnostic. A concrete test would be to check whether E, after working with lisptree2 and holes, can identify the 'x' in 'x+2=5' as a leaf and can trace how substitution changes that leaf. Without such a bridge, the intervention risks presupposing the very competence it aims to teach.","section":"Sections 4 and 6"}],"minor_comments":[{"comment":"In the notation example at the bottom of p. 10, '(ax+b) \\binom{a:=-3}{b:=5} = -3x+6' contradicts the Maxima output in (%o11), which is 5-3x (i.e., -3x+5). Since b=5, the result should be -3x+5. This error in a central substitution illustration should be corrected.","section":"Section 4.1, p. 10"},{"comment":"The phrase 'In the notation that we used in section 2' is likely a cross-reference error; the substitution notation was introduced in Section 1 (the chain rule example), not Section 2.","section":"Section 4.1"},{"comment":"The dedication 'Para Walter Machado Pinheiro, que não leu e não vai ler documento nenhum, e se ler não vai entender' is unprofessional and potentially offensive to a named individual; it should be removed or rewritten.","section":"Front matter"},{"comment":"Several custom Maxima functions ('s', 'V', 'sss', 'ssu', 'mkholes', 'lisptreem', 'lazy +.') are used but their definitions are not included in the paper. For reproducibility, provide the code as an appendix or a clear link to a repository/URL. Some references to the author's own materials (OchsNSC2026, OchsMD, etc.) are given, but the main paper should be self-contained at least for the central macros.","section":"Sections 4 and 6"},{"comment":"Minor typos: 'obtaing' (Section 4.1) and 'reconstruced' (Section 7) should be corrected.","section":"Section 4.1; Section 7"}],"recommendation":"major_revision","confidential_remarks":"The paper is a borderline teaching note: the central idea is attractive and the Maxima examples are plausible, but the evidence is anecdotal and the paper itself disclaims rigorous testing. The dedication is inappropriate for a journal and should be removed. If the journal welcomes pedagogical proposals, the manuscript could be acceptable after major revision; otherwise it may be better suited to a less formal venue. I recommend major revision, with the main requirement being a reframing of the central claim as a testable hypothesis or the inclusion of a pilot study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper is that it's a thoughtful, honest position/tool paper. Ochs describes specific Maxima functions (lisptreem, lisptree2, the 's' substitution family, holes) that let you visualize expressions as trees and substitutions as operations. The concrete examples are real and I suspect they'd help some students see what 'substituting x=3' means. The author is also unusual in his transparency: Section 7 flatly says the method is 'a work in progress', tested on small classes without rigorous methodology. That honesty is worth taking seriously.\n\nWhat's actually new is the specific Maxima implementation — the lazy '+' operator, the pretty-printed substitution macros, the hole generator — rather than the underlying ideas. The paper builds on known didactic literature (e.g., Freudenthal, Sfard, Mason) and his own prior work, which he cites. As a piece of engineering, the code sketches look plausible, though I could not verify them without the full definitions, which are not included.\n\nThe soft spots are in the pedagogy, not the code. The central claim — that presenting trees and substitution early, before quantifiers, will help students with extremely bad foundations learn to reason — rests entirely on an assumption about learning transfer. The author provides no evidence that interaction with these manipulables produces the 'aha' moment he hopes for. In fact, to understand the tree visualization of f(g(x)), a student like E, who can't substitute x=3 into x+2=5, has to first understand that an expression is a parsed structure with named child positions. That is precisely the competence the tool is meant to build. It's possible the tool creates that understanding through use, but that's the untested hypothesis. So the paper is not a validated teaching strategy; it's a set of promising prototypes.\n\nThat said, the paper doesn't overclaim. It is explicitly exploratory. For a reader in math education or CAS-based teaching, the examples are concrete enough to try in a classroom and adapt. For a general research reader, it's a minor piece, but it deserves serious referee time because it's a genuine, clearly-presented proposal that could generate empirical studies. I'd send it to a referee with expertise in both Maxima and calculus pedagogy.\n\nRecommendation: engage with it conditionally — treat it as a source of classroom tools, not as evidence. If the author publishes the code in full and some controlled classroom data, the contribution would be significant.","headline":"Honest, clearly-presented Maxima manipulables for weak calculus students; the tools are promising but the learning-transfer claim is untested.","tokens_in":13918,"tokens_out":2484,"would_cite":false,"duration_ms":26028,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that calculus courses with many poorly-prepared students should treat expressions and trees as basic objects and teach substitution as a visible, manipulable operation before theorems and quantifiers.","keywords":["bad foundations","substitution","calculus teaching","Maxima","manipulable objects","expandable objects","procedural knowledge","computer algebra system"],"falsifier":"A controlled trial with two groups of under-prepared calculus students, one taught with these Maxima manipulable objects early and the other with the standard syllabus, measuring their ability to substitute values into expressions and to justify steps in a chain rule problem after one semester. If the manipulable-object group shows no significant gain over the control group, the central claim is refuted. More directly: if a student who has used the tools still cannot compute d/dx sin(cos(tan(42x))) or cannot explain why (ax+b)[a:=−3,b:=5] = 5−3x, the transfer that the paper hopes for has not o","tokens_in":12963,"feed_emoji":"🧮","tokens_out":6428,"duration_ms":59961,"temperature":0.7,"pith_summary":"This paper is a teaching-strategy proposal for calculus courses that contain many students with what it calls 'extremely bad foundations'—students who can solve x+2=5 by a memorized move but cannot substitute 3 for x to verify it. The author argues that such students cannot benefit from the usual assumption that 'taking a particular case' is obvious, and that the right move is to introduce expressions and trees as basic objects and teach the substitution operation explicitly and early, before theorems and quantifiers. To make substitution visible, the paper develops a set of Maxima macros that display expressions as lisp trees, allow substitution to be annotated as a pretty-printed ':=', and create 'holes' in true expressions so they become exercises. The payoff, if true, is a concrete, low-cost way to help weak students see what mathematicians actually do when they justify a step.","feed_headline":"Make 'taking a particular case' visible in calculus","feed_subtitle":"For students who can't plug x=3 back into x+2=5, these Maxima tools show the step that was missing.","key_machinery":"The key machinery is the substitution operation written as (expr)[var:=value], which the author implements in Maxima as the operator 's'. To handle substitutions of functions, 's' converts function definitions to lambda-expressions on the fly, so that f(g(t))[f(x):=g(g(x)), g(x):=f(f(x))] correctly yields g(g(f(f(t)))). Around this core sits a set of Maxima macros: 'lisptree2' draws the internal Lisp representation of an expression as a two-dimensional tree; 'sss' pretty-prints a substitution in the standard mathematical 'a:=...' notation; 'dpart' and 'substpart' mark sub-expressions, and 'mkholes' replaces chosen sub-expressions by '?' to turn a true expression into an exercise. These toget","core_discovery":"The central claim is that 'when we have lots of students like the ones that I described above it is a good idea to present expressions and trees as some of our most basic objects, and present this substitution operation ... very early in the course, before theorems and quantifiers.' The author offers Maxima-based 'manipulable objects' as a way to make taking a particular case easy to visualize: expression trees drawn from internal Lisp representations, a custom substitution operator 's' that pretty-prints assignments in square brackets, and 'holes' that turn a true equality into an exercise. The discovery is not an empirical result but a designed pedagogical toolset and a specific ordering o","pith_inferences":["If the central claim is true, the same 'substitution as a first-class object' might be tested in other subjects, like algebra or physics problem solving, where students also struggle with plugging in numbers.","The 'holes' mechanism could be extended to automatically generate many variants of an exercise from one template, creating an unlimited supply of practice problems for drill.","The paper's emphasis on visual, manipulable objects suggests a testable prediction: students who interact with the Maxima tools will outperform peers on tasks that require substituting specific values into functions, even on paper exams that do not allow the tools.","The author's choice of Maxima over a proof assistant can be read as a broader point: for this population, immediate visual feedback in a light-weight REPL may be more valuable than formal type-checking, a hypothesis that could be examined by comparing outcomes with a Lean-based intervention."],"forward_implications":["If the proposal is right, the calculus course can present substitution as a primary object early on, giving under-prepared students a concrete handle on what 'taking a particular case' means.","The Maxima macros provide a ready way for instructors to turn standard textbook exercises (like chain-rule problems) into manipulable objects with holes, without rewriting the whole course.","Students with extremely bad foundations could see, in a REPL, how changing the substitution changes the result, which may help them shift from memorized procedures to pattern-matching and justification.","The author's 'archetypal case' method for separable ODEs suggests that teaching a single visual example alongside the general method can make the general method believable.","Because the tools run in a free computer algebra system and the non-textbook material is a small part of each class, the approach is feasible even on a crowded syllabus."],"fun_headline_variants":["Visualize substitution with expression trees","Maxima tools make 'taking a particular case' visible","Making substitution obvious for weak foundations","Turn equations into manipulable objects","Show the missing step: plug x=3 into x+2=5"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that students with extremely bad foundations who interact with these visual, manipulable objects will develop the pattern-matching and logical skills they lack—the paper explicitly states this has not been rigorously tested, and if it fails, the tools remain just Maxima macros, not a viable teaching strategy.","fun_headline_variants_meta":{"raw":{"variants":["Visualize substitution with expression trees","Maxima tools make 'taking a particular case' visible","Making substitution obvious for weak foundations","Turn equations into manipulable objects","Show the missing step: plug x=3 into x+2=5"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00018,"raw_usage":{"total_tokens":1125,"prompt_tokens":711,"completion_tokens":414,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":344}},"tokens_in":455,"tokens_out":414,"duration_ms":4124,"temperature":1.0,"reasoning_tokens":344,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T22:41:39.816068+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled trial with two groups of under-prepared calculus students, one taught with these Maxima manipulable objects early and the other with the standard syllabus, measuring their ability to substitute values into expressions and to justify steps in a chain rule problem after one semester. If the manipulable-object group shows no significant gain over the control group, the central claim is refuted. More directly: if a student who has used the tools still cannot compute d/dx sin(cos(tan(42x))) or cannot explain why (ax+b)[a:=−3,b:=5] = 5−3x, the transfer that the paper hopes for has not o","supporting_citations":[],"review_version":1}