Physics teaching assistants' views of different types of introductory problems: Challenge of perceiving the instructional benefits of context-rich and multiple-choice problems
Pith reviewed 2026-05-25 11:50 UTC · model grok-4.3
The pith
Many physics TAs list no pros for context-rich or multiple-choice problems and see the latter only as high-stakes tests.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
TAs regard context-rich problems as overly challenging, unnecessarily wordy, and too time-consuming to benefit their students, while they view multiple-choice questions nearly exclusively as instruments for high-stakes summative assessment rather than as possible tools for formative assessment such as clicker questions in large classes. Many TAs listed no advantages for either type despite being explicitly prompted to do so. These perceptions held for problems built on identical scenarios that TAs rated more favorably in other formats.
What carries the argument
TAs' written lists of pros and cons for problem types derived from the same scenario, which exposed their limited recognition of formative-assessment or active-learning uses.
If this is right
- TA professional development programs would need to include explicit practice in designing and deploying context-rich problems to build problem-solving skills.
- Training would need to demonstrate concrete ways multiple-choice items can supply real-time feedback even in large lectures.
- Without shifts in these views, introductory courses may continue to under-use active-engagement methods that rely on these problem types.
- The same scenario-comparison interview method could be reused to track whether views on other formats change after training.
Where Pith is reading between the lines
- Departments could test whether sections led by TAs who recognize these benefits show higher rates of student participation or conceptual gains.
- Similar gaps in recognizing benefits may appear for other active-learning tools, suggesting the issue is broader than problem format alone.
- Some TAs' skepticism may stem from real constraints such as grading time or class size, so training would need to address those constraints directly.
Load-bearing premise
That the instructional benefits of well-designed context-rich and multiple-choice problems are substantial enough that TAs' failure to name them mainly reflects a training gap rather than practical limits or doubts about their fit in ordinary courses.
What would settle it
A follow-up study that measures whether TAs who complete targeted training on these formats later list more pros, report using the formats in their sections, and produce measurable gains in student engagement compared with untrained TAs.
Figures
read the original abstract
Given a physics scenario, different problem types presenting that scenario in various ways can emphasize different instructional goals. In this investigation, we examined the views of physics graduate teaching assistants (TAs) about the instructional benefits of different types of introductory problems based upon the same problem scenario. Here we report on TAs' views about two of these problem types that were regarded by TAs as the least instructionally beneficial of all problem types--the context rich and multiple-choice formats. Many TAs listed no pros at all for these problem types, despite being explicitly asked for at least one pro. They viewed multiple-choice questions nearly exclusively as tools for high-stakes summative assessment rather than their possible use as formative assessment tools, e.g., as clicker questions even in large classes. Similarly, TAs viewed context-rich problems as overly challenging, unnecessarily wordy, and too time-consuming to be instructionally beneficial to their students. While TAs' concerns have obvious validity and value, the benefits of well-designed multiple-choice questions as a formative assessment tool was not readily identified by them, nor did the TAs recognize the learning benefits associated with solving context-rich problems. Given the powerful ways multiple-choice and context-rich problems can be used for active engagement and formative assessment in different instructional contexts to meet diverse instructional goals, the lack of enthusiasm for these types of problems has implications for future TA professional development programs.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript examines physics graduate teaching assistants' (TAs) perceptions of the instructional benefits of context-rich and multiple-choice problems in introductory physics, based on the same underlying scenario. The central finding is that many TAs identified no advantages for these formats despite being prompted, viewing multiple-choice questions exclusively as high-stakes summative tools and context-rich problems as excessively challenging, wordy, and time-consuming. The authors conclude that this indicates a need for enhanced TA professional development to highlight the formative assessment and active engagement potential of these problem types.
Significance. This work addresses an important aspect of physics education research by exploring TAs' views on problem design, which can inform training programs. If the results are robust, they suggest that TAs may underappreciate certain instructional tools, potentially affecting student learning in recitation sections. The qualitative approach allows for rich insights into TA reasoning, but its value depends on methodological transparency.
major comments (2)
- [Abstract/Methods] The manuscript does not report the sample size of TAs interviewed, details of the interview protocol, the coding method used to analyze responses, or any measure of inter-rater reliability. These omissions make it difficult to evaluate the prevalence and reliability of the reported views, which are central to the claim that 'many TAs listed no pros at all' and the implications for professional development.
- [Discussion] The interpretation that the lack of identified benefits primarily reflects a training gap assumes that the instructional benefits of well-designed context-rich and multiple-choice problems are substantial and readily transferable to typical TA-led sections. However, the paper provides no within-study evidence or validation that these formats deliver net gains after accounting for preparation time, class size, and other constraints mentioned by TAs. This makes the attribution to insufficient training an external premise rather than a data-driven conclusion.
minor comments (1)
- [Abstract] The abstract states the main observations but could benefit from including the number of participants or a brief note on methodology to strengthen the summary.
Simulated Author's Rebuttal
We thank the referee for their constructive comments, which help clarify the presentation of our qualitative study on TA perceptions. We respond to each major comment below.
read point-by-point responses
-
Referee: [Abstract/Methods] The manuscript does not report the sample size of TAs interviewed, details of the interview protocol, the coding method used to analyze responses, or any measure of inter-rater reliability. These omissions make it difficult to evaluate the prevalence and reliability of the reported views, which are central to the claim that 'many TAs listed no pros at all' and the implications for professional development.
Authors: We agree that these details are necessary for evaluating the findings and will add them to the methods section in revision. The expanded description will include the number of participants, the semi-structured interview protocol, the thematic analysis procedure, and inter-rater reliability measures. revision: yes
-
Referee: [Discussion] The interpretation that the lack of identified benefits primarily reflects a training gap assumes that the instructional benefits of well-designed context-rich and multiple-choice problems are substantial and readily transferable to typical TA-led sections. However, the paper provides no within-study evidence or validation that these formats deliver net gains after accounting for preparation time, class size, and other constraints mentioned by TAs. This makes the attribution to insufficient training an external premise rather than a data-driven conclusion.
Authors: The manuscript reports TAs' perceptions and explicitly notes that their concerns have validity; the professional-development implication follows from the observed absence of recognized benefits that are documented elsewhere in the PER literature on these formats. We will revise the discussion to state this distinction more explicitly and to avoid any implication of within-study validation of net gains. We maintain that the data on perceptions directly support the call for training to address the identified gap in views. revision: partial
Circularity Check
No circularity: qualitative empirical report with no derivations or self-referential modeling
full rationale
The paper is a qualitative interview/survey study reporting TAs' stated views on problem formats. It contains no equations, no fitted parameters, no predictions derived from inputs, and no load-bearing self-citations that reduce the central claims to prior author work by construction. The interpretive framing (that unrecognized benefits indicate a training gap) rests on external literature assumptions rather than any internal reduction of the reported data to itself. This is a standard non-circular empirical report.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
E. Yerushalmi, C. Henderson, K. Heller, P. Heller, and V . Kuo, Physics faculty beliefs and values about the teaching and learning of problem solving I. Mapping the common core Phys. Rev. ST PER 3, 020109 (2007)
work page 2007
-
[2]
B. Eylon and F. Reif, Effects of knowledge organization on task performance Cognition Instruction 1, 5 (1984)
work page 1984
-
[3]
E. F. Redish, The Implications of Cognitive Studies for Teaching Physics, Am. J. Phys. 62, 796 (1994)
work page 1994
- [4]
-
[5]
Mestre, Is transfer ubiquitous or rare: New paradigms for studying transfer, Proc
J. Mestre, Is transfer ubiquitous or rare: New paradigms for studying transfer, Proc. Phys. Educ. Res. Conf., AIP Conf. Proc. 790, 3 Melville, NY (2005)
work page 2005
-
[6]
M. Chi, P. Feltovich, and R. Glaser, Characterization and representation of physics problems by experts and novices, Cognitive Science 5, 121(1981)
work page 1981
-
[7]
Reif, Millikan Lecture 1994: Understanding and teaching important scientific thought processes, Am
F. Reif, Millikan Lecture 1994: Understanding and teaching important scientific thought processes, Am. J. Phys. 63, 17 (1995)
work page 1994
-
[8]
Reif, Teaching problem solving–A scientific approach The Phys
F. Reif, Teaching problem solving–A scientific approach The Phys. Teach. 19, 310 (1981)
work page 1981
-
[9]
B. Eylon and F. Reif F, Effects of knowledge organization on task performance, Cogn. Instr. 1, 5 (1984)
work page 1984
-
[10]
K. K. Mashood and V . Singh, Large-scale studies on the transferability of general problem-solving skills and the pedagogic potential of physics, Phys. Educ. 48, 629 (2013)
work page 2013
-
[11]
J. Bolton and S. Ross, Developing students physics problem-solving skills, Phys. Educ. 32, 176 (1997)
work page 1997
-
[12]
E. Yerushalmi, C. Henderson, K. Heller, P. Heller, and V . Kuo, Physics faculty beliefs and values about the teaching and learning of problem solving I. Mapping the common core, Phys. Rev. ST PER 3, 020109 (2007)
work page 2007
-
[13]
Singh, When physical intuition fails, Am
C. Singh, When physical intuition fails, Am. J. Phys. 70, 1103 (2002)
work page 2002
-
[14]
Singh Assessing student expertise in introductory physics with isomorphic problems
C. Singh Assessing student expertise in introductory physics with isomorphic problems. I. Performance on a nonintuitive problem pair from introductory physics, Phys. Rev. ST PER 4, 010104 (2008)
work page 2008
-
[15]
Singh, Assessing student expertise in introductory physics with isomorphic problems
C. Singh, Assessing student expertise in introductory physics with isomorphic problems. II. Effect of some potential factors on problem solving and transfer, Phys. Rev. ST PER 4, 010105 (2008)
work page 2008
-
[16]
S. Y . Lin and C. Singh, Using isomorphic problems to learn introductory physics, Phys. Rev. ST PER7, 020104 (2011)
work page 2011
-
[17]
S. Y . Lin and C. Singh, Using isomorphic problem pair to learn introductory physics: Transferring from a two-step problem to a three-step problem, Phys Rev ST PER 9, 020114 (2013)
work page 2013
-
[18]
S. Y . Lin and C. Singh, Effect of scaffolding on helping introductory physics students solve quantitative problems involving strong alternative conceptions, Phys. Rev. ST PER11, 020105 (2015)
work page 2015
-
[19]
E. Yerushalmi, E. Cohen, A. Mason and C. Singh, What do students do when asked to diagnose their mistakes? Does it help them? I. An atypical quiz context , Phys. Rev. ST PER 8, 020109 (2012)
work page 2012
-
[20]
E. Yerushalmi, E. Cohen, A. Mason and C. Singh, What do students do when asked to diagnose their mistakes? Does it help them? II. A more typical quiz context, Phys. Rev. ST PER 8, 020110 (2012)
work page 2012
-
[21]
A. Mason and C. Singh, Using categorization of problems as an instructional tool to help introductory students learn physics, Phys. Educ. 51, 025009 (2016)
work page 2016
-
[22]
C. Henderson et al., Instructional Goals and Grading Practices of Graduate Students after One Semester of Teaching Experience, Proc. Phys. Educ. Res. Conf. (2015) doi: 10.1119/perc.2014.pr.024
-
[23]
Hake, Interactive-engagement vs
R. Hake, Interactive-engagement vs. traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses, Am. J. Phys. 66, 64 (1998)
work page 1998
-
[24]
D. Nicol and D. Macfarlane-Dick, Formative assessment and selfregulated learning: a model and seven principles of good feedback practice, Studies in Higher Educ., 31, 2 (2006)
work page 2006
-
[25]
J. Faust, and D. Paulson, Active learning in the college classroom, J. Excellence in College Teaching 9, 3 (1998)
work page 1998
-
[26]
For example, see http://www.aps.org/programs/education/graduate/conf2008/index.cfm
-
[27]
J. Nyquist, R. Abbott, and D. Wulff, The challenge of TA training in the 1990s, in New Directions for Teaching and Learning 39: pp. 7-13, Kendall/Hunt publishing Company, Dubuque, IA, 1989
work page 1989
-
[28]
E. Yerushalmi, K. Heller, P. Heller, C. Henderson, Instructors’ reasons for choosing problem features in a calculus-based introductory physics course, Phys. Rev. ST PER 6, 020108 (2010)
work page 2010
-
[29]
F. Lawrenz, P. Heller, and R. Keith, Training the teaching assistant J. Col. Sci. Teach.22 (1992)
work page 1992
-
[30]
E. Yerushalmi, E. Marshman, A. Maries, C. Henderson, and C. Singh, Grading practices and considerations of graduate students at the beginning of their teaching assignment, Proc. Phys. Educ. Res. Conf. (2015) doi: 10.1119/perc.2014.pr.068
-
[31]
E. Yerushalmi, R. Sayer, E. Marshman, C. Henderson, and C. Singh, Physics graduate teaching assistants’ beliefs about a grading rubric: Lessons learned, Proc. Phys. Educ. Res. Conf. Proc. (2016) doi: 10.1119/perc.2016.pr.097
-
[32]
S. Y . Lin, C. Henderson, W. Mamudi, C. Singh, and E. Yerashalmi, Teaching assistants beliefs regarding example solutions in introduc- tory physics, Phys. Rev. ST PER 9, 010120 (2013)
work page 2013
-
[33]
E. Marshman, R. Sayer, C. Henderson, E. Yerushalmi, and C. Singh, The challenges of changing teaching assistants grading practices: Requiring students to show evidence of understanding, Can. J. Phys. 96 (4), 420 (2018)
work page 2018
-
[34]
E. Marshman, R. Sayer, C. Henderson, and C. Singh, Contrasting grading approaches in introductory physics and quantum mechanics: The case of graduate teaching assistants, Phys. Rev. ST PER 13, 010120 (2017)
work page 2017
-
[35]
R. Goertzen, R. Scherr, and A. Elby, Respecting tutorial instructors’ beliefs and experiences: A case study of a physics teaching assistant, Phys. Rev. ST PER 6 (2), 020125 (2010)
work page 2010
-
[36]
J. Chini and A. Al-Rawi, Alignment of TAs’ beliefs with practice and student perception Proc. Phys. Educ. Res. Conf. (2012) doi: 10.1063/1.4789661
-
[37]
Fuhrman, Developing good multiple-choice tests and test questions, J
M. Fuhrman, Developing good multiple-choice tests and test questions, J. Geoscience Educ. 44, 379 (1996)
work page 1996
-
[38]
K. K. Mashood and V . A. Singh, An inventory on rotational kinematics of a particle: unraveling misconceptions and pitfalls in reasoning, Euro. J. Phys. 33, (2012)
work page 2012
-
[39]
P. Engelhardt, An introduction to classical test theory as applied to conceptual multiple-choice tests, Getting Started in PER 2 (2009)
work page 2009
- [40]
-
[41]
L. Ding, R. Chabay, B. Sherwood, and R. Beichner, Valuating an assessment tool: Brief Electricity and Magnetism Assessment, Phys Rev ST PER 1 ,10105 (2006)
work page 2006
-
[42]
A. Savinainen and P. Scott, Using the Force Concept Inventory to monitor student learning and to plan teaching, Phys. Educ. 37, 53 (2002)
work page 2002
-
[43]
C. Singh and D. Rosengrant, Multiple-choice test of energy and momentum concepts, Am. J. Phys, 71, 607 (2003)
work page 2003
-
[44]
Singh, Student understanding of symmetry and Gauss’s law of electricity, Am
C. Singh, Student understanding of symmetry and Gauss’s law of electricity, Am. J. Phys., 74 (10), 923 (2006)
work page 2006
-
[45]
L. Rimoldini and C. Singh, Student understanding of rotational and rolling motion concepts, Phys. Rev. ST PER 1, 010102 (2005)
work page 2005
- [46]
-
[47]
D. F. Treagust, Development and use of diagnostic tests to evaluate students’ misconceptions in scienceInt. J. Sci. Educ. 10 159 (1988)
work page 1988
-
[48]
Mazur, Peer Instruction: A User’s Manual Prentice Hall, Upper Saddle River, NJ (1997)
E. Mazur, Peer Instruction: A User’s Manual Prentice Hall, Upper Saddle River, NJ (1997)
work page 1997
-
[49]
C. Keller, N. Finkelstein, K. Perkins, S. Pollock, C. Turpen, and M. Dubson, Research-based practices for effective clicker use, Proc. Phys. Educ. Res. Conf., AIP Conf. Proc. doi: 10.1063/1.2820913
-
[50]
Singh, Impact of peer interaction on conceptual test performance, Am
C. Singh, Impact of peer interaction on conceptual test performance, Am. J. Phys. 73 (5), 446 (2005)
work page 2005
-
[51]
C. Singh and G. Zhu, Improving students’ understanding of quantum mechanics by using peer instruction tools, Proc. Phys. Ed. Res. Conf., AIP Conf. Proc., Melville, New York1413, 77-80 (2012) doi:10.1063/1.3679998
-
[52]
R. Sayer, E. Marshman, and C. Singh, The impact of peer interaction on the responses to clicker questions in an upper-level quantum mechanics course, Proc. Phys. Educ. Res. Conf. (2016) doi: 10.1119/perc.2016.pr.071
- [53]
-
[54]
James, The effect of grading incentive on student discourse in Peer Instruction, Am
M. James, The effect of grading incentive on student discourse in Peer Instruction, Am. J. Phys. 74, 689 (2006) doi: 10.1119/1.2198887
-
[55]
S. Willoughby and E. Gustafson, Technology talks: Clickers and grading incentive in the large lecture hall,Am. J. Phys. 77, 180 (2009). doi: 10.1119/1.3013542
- [56]
- [57]
- [58]
-
[59]
M. Lorenzo, C. Crouch and E. Mazur, Reducing the gender gap in the physics classroom, Am. J. Phys 74, 118 (2006)
work page 2006
- [60]
-
[61]
R. Beichner, The Student-Centered Activities for Large Enrollment Undergraduate Programs (SCALE-UP) Project Research-Based Reform of University Physics V ol. 1 (2007)
work page 2007
-
[62]
D. Sokoloff and R. Thornton, Using interactive lecture demonstrations to create an active learning environment, The Phys. Teach. 35, 340 (1997)
work page 1997
-
[63]
K. Manivannan and D. Meltzer, Use of in-class physics demonstrations in highly interactive format, Proc. Phys. Educ. Res. Conf. (2001). doi: 10.1119/perc.2001.pr.011
- [64]
-
[65]
Putt, Testing the mastery concept of self-paced learning in physics, Am
G. Putt, Testing the mastery concept of self-paced learning in physics, Am. J. Phys. 45, 472 (1977)
work page 1977
-
[66]
C. Keebaugh, E. Marshman and C. Singh, Investigating and addressing student difficulties with the corrections to the energies of the hydrogen atom for the strong and weak field Zeeman effect, Euro. J. Phys. 39, 045701 (2018)
work page 2018
-
[67]
Singh, Interactive learning tutorials on quantum mechanics, Am
C. Singh, Interactive learning tutorials on quantum mechanics, Am. J. Phys., 76, 400 (2008)
work page 2008
- [68]
-
[69]
J. Saul, Beyond problem solving: Evaluating introductory physics courses through the hidden curriculum, PhD Dissertation University of Maryland (1998)
work page 1998
-
[70]
C. Singh, Interactive video tutorials for enhancing problem solving, reasoning, and meta-cognitive skills of introductory physics stu- dents, AIP Conf. Proc. 720, 177 (2004) doi:10.1063/1.1807283
-
[71]
C. Singh, Computer-based tutorials to develop expertise in introductory physics students APS Forum on Education https://www.aps.org/units/fed/newsletters/summer2008/singh.cfm (2008)
work page 2008
-
[72]
Developing problem solving skills of students taking introductory physics via web-based tutorials, C. Singh and D. Haileselassie, J. Coll. Sci. Teach. 39 (4), 42 (2010)
work page 2010
-
[73]
S. DeV ore, E. Marshman, and C. Singh, Challenge of engaging all students via self-paced interactive electronic learning tutorials for introductory physics, Phys. Rev. STPER 13, 010127 (2017)
work page 2017
-
[74]
S. DeV ore, Using the tutorial approach to improve physics learning from introductory to graduate levelPhD Dissertation University of Pittsburgh (2010) http://d-scholarship.pitt.edu/23484/
work page 2010
-
[75]
E. Marshman, S. DeV ore and C. Singh, Challenge of helping introductory physics students transfer their learning by engaging with a self-paced learning tutorial, Frontiers in Science 1, (2018)
work page 2018
- [76]
-
[77]
P. Heller and M. Hollabaugh, Teaching problem solving through cooperative grouping. Part 1: Group versus individual problem solving, Am. J. Phys. 60(7), 627 (1992); P. Heller and M. Hollabaugh, Teaching problem solving through cooperative grouping. Part 2: Designing problems and structuring groups, Am. J. Phys. 60, 7 (1992)
work page 1992
-
[78]
C. A. Ogilvie, Impact of context-rich, multifaceted problems on students’ attitudes towards problem-solving, arXiv:0809.1081 (2008)
work page internal anchor Pith review Pith/arXiv arXiv 2008
-
[79]
C. A. Ogilvie, Changes in students’ problem-solving strategies in a course that includes context-rich, multifaceted problems Phys. Rev. ST PER 5, 020102 (2009)
work page 2009
-
[80]
P. Antonenko, C. Ogilvie, D. Niederhauser, J. Jackman, P. Kumsaikaew, R. Marathe, S. Ryan, Understanding student pathways in context-rich problems, Educ. and Info. Tech. 16, 323 (2010)
work page 2010
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.