REVIEW 4 major objections 4 minor 22 references
A cross-context look at upper-division student difficulties with integration
T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Integration mistakes split into persistent, fading, and new across physics courses.
desk verdict Useful contribution with real data, but the headline 'new difficulty' with current density is partly built into the prompt differences, and the numbers need cleaning. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The ACER framework, operationalized for direct integration, is the central mechanism. It separates expert problem solving into four components—Activation of the mathematical tool, Construction of the integral model, Execution of the mathematics, and Reflection on the result—and specifies elements within each, such as C2 (express the differential element) and C4 (express the difference vector). The paper adapts the framework to gravitation and magnetostatics and uses the resulting coding scheme to make exam and interview data from different courses commensurable. The pseudo-longitudinal design, comparing cohorts at different curriculum points rather than tracking individuals, supplies the cross-context comparison that the framework then interprets.
What would settle it
Give the same direct-integration prompt—identical geometry, identical requested quantity, no supplied formula—to students in the classical-mechanics, electrostatics, and magnetostatics courses, and measure the frequency of differential-element and difference-vector errors; the persistent/fading taxonomy would be refuted if, with prompts controlled, error frequencies no longer track curriculum level or the current-density difficulty appears in earlier contexts.
Extended reading notes
Core claim
On the authors' own terms, the central discovery is a taxonomy of how student difficulties with direct integration change across the undergraduate curriculum. Using the ACER framework—Activate, Construct, Execute, Reflect—to code exam solutions and paired think-aloud interviews, they find that difficulties with activation (recognizing that a potential calls for an integral) and with construction of the difference vector $|\vec{r}-\vec{r}'|$ persist across all three contexts, appearing in roughly a quarter to a half of students at each level. Difficulties with expressing differential line, area, and volume elements decline from about 14% of classical mechanics solutions to 9% in later courses, suggesting these fade with practice. In magnetostatics, a new and largely conceptual difficulty appears around the volume current density $\vec{J}$ and the meaning of $\vec{J}\,dV'$, which has no simple physical interpretation as a 'chunk' of current. Spontaneous reflection—checking units or limiting behavior—remains rare at every level, and when prompted students can check units but struggle with limiting forms requiring expansion.
Load-bearing premise
The paper assumes the three data sets are comparable enough that differences in difficulty rates reflect students' developing experience rather than differences in the exam prompts themselves, for example one vector-potential version supplied the integral formula and only the scalar-potential questions required actually computing an integral.
Editorial extensions
If this is right
- Because activation errors persist at roughly the same rate across contexts, instructors at multiple levels should explicitly discuss when integration is the right tool and when simpler methods such as Gauss's law or point-mass approximations do not apply.
- Because differential-element errors fade, curricular time may be better spent on construction elements that do not fade, such as the difference vector.
- The current-density difficulty implies that instruction on the vector potential should explicitly address the physical meaning of $\vec{J}\,dV'$ rather than treating it as purely algebraic.
- Because spontaneous reflection is rare at all levels, prompting reflection with explicit checks of units and limits is likely necessary; instructors should expect students to find limiting-form checks harder than units checks.
- The ACER operationalization developed here can be applied to other mathematical tools, such as Dirac delta functions or separation of variables, to build cross-context difficulty taxonomies.
Reading between the lines
- The observed decline in differential-element errors could partly be an artifact of prompt differences: later-course questions asked only for setup, and one vector-potential prompt supplied the formula. A direct test would be to give the same rod-potential prompt to all three populations and compare error frequencies.
- The persistent difference-vector difficulty may be notation-driven: the 'script-r' shorthand requires students to decode which vector is source and which is field before they can reason geometrically. A testable extension is to compare performance when the integrand is written explicitly in terms of $\vec{r}$ and $\vec{r}'$ versus shorthand.
- The pattern generalizes beyond integration: any mathematical tool that reappears with a new physical quantity may generate context-specific construction difficulties, for example interpreting the polarization density $\vec{P}$ in a bound-charge integral.
- The pseudo-longitudinal method could be strengthened by a true longitudinal study tracking the same students through two or more courses, which would distinguish persistent individual difficulty from cohort-level differences.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper investigates upper-division student difficulties with direct integration in three physics contexts: gravitational potential (middle-division classical mechanics), scalar electric potential (upper-division electrostatics), and magnetic vector potential (upper-division magnetostatics). Using the ACER analytical framework (activation, construction, execution, reflection), the authors code exam solutions and think-aloud interviews, compare the frequencies of difficulties across contexts, and categorize them as persistent, fading, or new over a pseudo-longitudinal sequence. They report that difficulties with activating integration and expressing the difference vector persist across all contexts, difficulties with differential line/area/volume elements appear to fade, and difficulties with interpreting and expressing the current density appear as a new challenge in magnetostatics. The paper argues that the ACER framework provides a useful standardized structure for cross-context and cross-study comparisons of student problem solving.
Significance. If the findings are valid, this work provides one of the few cross-context, pseudo-longitudinal maps of student difficulties with a mathematical tool, and it demonstrates the utility of the ACER framework for organizing such comparisons. The study combines quantitative exam data (over 300 solutions) with qualitative interview data, and it extends prior PER work to two new curricular points. The persistent/fading/new taxonomy is a clear and directly actionable claim for instructors and researchers, and it is falsifiable: for example, the claim that difference-vector difficulties persist while differential-element difficulties fade can be tested in other contexts. The paper also provides a detailed operationalization of ACER for integration problems, which is a reusable methodological contribution. These strengths make the paper a potentially valuable contribution to physics education research, provided the load-bearing empirical comparisons are placed on firmer footing.
major comments (4)
- [IV A, first paragraph] The sample size for activation is internally inconsistent. The text states "representing N = 325 exam solutions" but later in the same section reports "none of the N = 50 vector potential solutions" and "only one of the N = 50 vector potential solutions." Given that the activation analyses include gravitation (N = 77) and scalar potential (N = 160), the implied vector-potential contribution is either 88 (to reach 325) or 50 (as stated), and these do not reconcile. This inconsistency affects the denominator for all activation percentages and must be corrected before publication.
- [IV B and V] The claim that "interpreting and expressing the current density" is a new difficulty that appears in magnetostatics is confounded by unmatched prompt demands. In the gravitation and scalar-potential rod prompts, the linear density λ was provided in the problem statement (Fig. 1a), so students never had to construct the density from a total mass or charge. In the line-current vector-potential version, students were required to translate a total current I into a volume current density J, a step absent from the other prompts. The paper itself notes that the surface-current version "bypassed the need for students to express this quantity" (Section IV B). The reported spike in density-related C2 errors (90% of C2 difficulties in that version) is therefore attributable to task structure rather than solely to a context-emergent conceptual difficulty with the physics. Because the abstract and conclusions explicitly advertise current density as the exemplar of a new difficulty, this confound threatens the central taxonomy. The authors should either re-analyze the data using matched prompts (e.g., comparing only prompts that require constructing the density, or comparing the two vector-potential versions) or substantially temper the claim.
- [III A and IV B] The "fading" claim for differential line/area elements is based on a pseudo-longitudinal comparison across cohorts, semesters, and prompts that differ in task demands. For example, the gravitation prompt and the scalar-potential prompt differ in orientation signal, in whether the density is supplied, and in whether students were required to execute the integral; the vector-potential prompts include a strip version and a line version. The decrease from 14% (N = 10 of 72) to 9% (N = 20 of 218) in line-element errors could be an artifact of prompt construction rather than a genuine curriculum-driven improvement. The paper acknowledges the pseudo-longitudinal nature of the data but does not address prompt comparability for this specific frequency comparison. The "fading" category should be either supported by a matched analysis or presented as a tentative hypothesis.
- [III B and IV] The coding reliability is not reported. The quantitative frequencies in Tables I-III derive from codings by the authors using an operationalized ACER scheme, but no inter-rater reliability (e.g., Cohen's kappa or percent agreement) is provided, and no mention is made of how disagreements were resolved. For a study whose central claims are frequency comparisons across contexts, the absence of reliability evidence is a notable methodological gap. The authors should report reliability statistics or clearly justify why they are unnecessary.
minor comments (4)
- [Abstract] The word "accross" should be "across."
- [IV B, paragraph on differential line element] The word "exibit" should be "exhibit."
- [III A, exam question description] The phrase "constant current current" contains a duplicated word and should be "constant current."
- [V, throughout] Several instances of "students difficulties" should be "students' difficulties."
Circularity Check
No significant circularity: the ACER framework is a coding lens, and the empirical taxonomy is supported by disclosed coded data rather than derived from the framework.
full rationale
This paper is an empirical physics-education-research study rather than a formal derivation, so the circularity patterns based on equations or fitted inputs largely do not apply. The central claim, that some integration difficulties persist while others fade or appear anew, is grounded in coded exam solutions and interviews (Tables I–III and Section IV), not entailed by the ACER framework. The ACER framework and its operationalization, drawn substantially from the authors' earlier work (Refs. [6,7]), function as a coding and comparison lens; the paper presents the frequency data and qualitative excerpts needed to check each classification, and the framework itself is described as arising from task analysis of expert problem solving rather than from the target findings. The paper also explicitly discloses the relevant prompt asymmetries: one vector-potential version supplied Eq. 3 and bypassed activation, and one version supplied K while the other required students to construct J (Sec. IV.B). These disclosures identify validity and confound concerns for the pseudo-longitudinal comparison, but they are not circularity: the 'new difficulty' with current density is an interpretation of observed coding patterns, not a value fitted to guarantee that conclusion. No equation in the paper is defined in terms of the result it is used to support, and no load-bearing claim reduces by construction to a fitted parameter or to an unverified self-citation chain. Therefore no significant circularity is present.
Assumptions & free parameters
assumptions (4)
- domain assumption The ACER framework is a valid and sufficiently complete taxonomy for capturing student difficulties with direct integration.
- domain assumption The exam prompts across the three contexts are comparable enough to support pseudo-longitudinal inferences.
- domain assumption Student solutions and think-aloud interview statements are reliable evidence of underlying reasoning difficulties.
- domain assumption The pseudo-longitudinal comparison across different cohorts is a meaningful proxy for longitudinal development.
Cite this review
Pith. "Pith review of A cross-context look at upper-division student difficulties with integration." pith.science (2026). https://pith.science/paper/XCTLADT7
@misc{pith2026190800474,
author = {Pith},
title = {Pith review of: A cross-context look at upper-division student difficulties with integration},
year = {2026},
howpublished = {\url{https://pith.science/paper/XCTLADT7}},
note = {Machine review of arXiv:1908.00474}
}
read the original abstract
We investigate upper-division student difficulties with direct integration in multiple contexts involving the calculation of a potential from a continuous distribution (e.g., mass, charge, or current). Integration is a tool that has been historically studied at several different points in the curriculum including introductory and upper-division levels. We build off of these prior studies and contribute additional data around student difficulties with multi-variable integration at two new points in the curriculum: middle-division classical mechanics, and upper-division magnetostatics. To facilitate comparisons across prior studies as well as the current work, we utilize an analytical framework that focuses on how students activate, construct, execute, and reflect on mathematical tools during physics problem solving (i.e., the ACER framework). Using a mixed-methods approach involving coded exam solutions and student problem-solving interviews, we identify and compare students' difficulties in these two different context and relate them to what has been found previously in other levels and contexts. We find that some of the observed student difficulties were persistent accross all three contexts (e.g., identifying integration as the appropriate tool, and expressing the difference vector), while other difficulties seemed to fade as students advanced through the curriculum (e.g., expressing differential line, area, and volume elements). We also identified new difficulties that appear in different contexts (e.g., interpreting and expressing the current density).
Figures
Reference graph
Works this paper leans on
-
[1]
Resource letter rps-1: Research in problem solv- ing,
Leon Hsu, Eric Brewe, Thomas Foster, and Kathleen Harper, “Resource letter rps-1: Research in problem solv- ing,” Am. J. Phys. 72, 1147–1156 (2004)
work page 2004
-
[2]
as a point mass, representing a failure in A3. One student argued that any rigid body could be treated as a point at its center of mass; the other was less confident in this approach, but said that they did not know how to account for the details of this particular mass distribution. When prompted to consider approaching the problem via direct integration,...
-
[3]
Stephanie V. Chasteen, Bethany Wilcox, Marcos D. Caballero, Katherine K. Perkins, Steven J. Pol- lock, and Carl E. Wieman, “Educational transfor- mation in upper-division physics: The science educa- tion initiative model, outcomes, and lessons learned,” Phys. Rev. ST Phys. Educ. Res. 11, 020110 (2015)
work page 2015
-
[4]
Resource let- ter alip–1: Active-learning instruction in physics,
David E Meltzer and Ronald K Thornton, “Resource let- ter alip–1: Active-learning instruction in physics,” Am. J. Phys. 80, 478 (2012)
work page 2012
-
[5]
N. D. Finkelstein and S. J. Pollock, “Replicat- ing and understanding successful innovations: Implementing tutorials in introductory physics,” Phys. Rev. ST Phys. Educ. Res. 1, 010101 (2005)
work page 2005
-
[6]
Lillian C McDermott and Peter S Shaffer, Tutorials in in- troductory physics and homework package (Prentice Hall, 2001)
work page 2001
-
[7]
Analytic framework for students use of mathematics in upper-division physics,
Bethany R. Wilcox, Marcos D. Caballero, Daniel A. Rehn, and Steven J. Pollock, “Analytic framework for students use of mathematics in upper-division physics,” Phys. Rev. ST Phys. Educ. Res. 9, 020119 (2013)
work page 2013
-
[8]
Unpacking students use of mathematics in upper-division physics: where do we go from here?
Marcos D Caballero, Bethany R Wilcox, Leanne Doughty, and Steven J Pollock, “Unpacking students use of mathematics in upper-division physics: where do we go from here?” European Journal of Physics 36, 065004 (2015)
work page 2015
Show all 22 references
-
[9]
Student resources for learning introduc- tory physics,
David Hammer, “Student resources for learning introduc- tory physics,” Am. J. Phys. 68, S52–S59 (2000)
2000
-
[10]
Task analysis by problem solving (taps): Uncovering expert knowledge to develop high- quality instructional materials and training,
Richard Catrambone, “Task analysis by problem solving (taps): Uncovering expert knowledge to develop high- quality instructional materials and training,” in 2011 Learning and Technology Symposium (Columbus, Geor- gia, 2011)
2011
-
[11]
Stu- dents’ difficulties with integration in electricity,
Dong-Hai Nguyen and N. Sanjay Rebello, “Stu- dents’ difficulties with integration in electricity,” Phys. Rev. ST Phys. Educ. Res. 7, 010113 (2011)
2011
-
[12]
Bing, An Epistemic Framing Analysis of Upper-Level Physics Students’ Use of Mathematics , Dis- sertation, University of Maryland (2008)
Thomas J. Bing, An Epistemic Framing Analysis of Upper-Level Physics Students’ Use of Mathematics , Dis- sertation, University of Maryland (2008)
2008
-
[13]
Understanding stu- dent use of differentials in physics integration problems,
Dehui Hu and N. Sanjay Rebello, “Understanding stu- dent use of differentials in physics integration problems,” Phys. Rev. ST Phys. Educ. Res. 9, 020108 (2013)
2013
-
[14]
How stu- dents use mathematical resources in an electrostatics con- text,
Dawn C. Meredith and Karen A. Marrongelle, “How stu- dents use mathematical resources in an electrostatics con- text,” American Journal of Physics 76, 570–578 (2008)
2008
-
[15]
point-source
used problem sorting tasks with introductory physics students which asked them to determine the correct ap- proach to various problems relating to electricity and magnetism; several of these tasks included problems for which direct integration via Coulomb’s law or the Biot- Sa...
-
[16]
Medi- ating relationship of differential products in un- derstanding integration in introductory physics,
Nathaniel Amos and Andrew F. Heckler, “Medi- ating relationship of differential products in un- derstanding integration in introductory physics,” Phys. Rev. Phys. Educ. Res. 14, 010105 (2018)
2018
-
[17]
Choosing the right solution approach: The crucial role of situa- tional knowledge in electricity and magnetism,
Elwin R. Savelsbergh, Ton de Jong, and Mon- ica G. M. Ferguson-Hessler, “Choosing the right solution approach: The crucial role of situa- tional knowledge in electricity and magnetism,” Phys. Rev. ST Phys. Educ. Res. 7, 010103 (2011)
2011
-
[18]
Taylor, Classical mechanics (University Science Books, 2005)
John R. Taylor, Classical mechanics (University Science Books, 2005)
2005
-
[19]
Boas, Mathematical Methods in the Physical Sciences (John Wiley & Sons, 2006)
M.L. Boas, Mathematical Methods in the Physical Sciences (John Wiley & Sons, 2006)
2006
-
[20]
Griffiths, Introduction to electrodynamics (Prentice Hall, 1999)
David J. Griffiths, Introduction to electrodynamics (Prentice Hall, 1999)
1999
-
[21]
Eric Mazur, Peer Instruction: A User’s Manual , Series in Educational Innovation (Prentice Hall, Upper Saddle River, 1997)
1997
-
[22]
Transforming the junior level: Outcomes from instruction and research in e&m,
Stephanie V Chasteen, Steven J Pollock, Rachel E Pep- per, and Katherine K Perkins, “Transforming the junior level: Outcomes from instruction and research in e&m,” Phys. Rev. ST Phys. Educ. Res. 8, 020107 (2012)
2012
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.