pith. sign in

arxiv: 1907.01134 · v1 · pith:F3BFOMJVnew · submitted 2019-07-02 · ⚛️ physics.ed-ph

Comparing introductory physics and astronomy students' attitudes and approaches to problem solving

Melanie Good , Andrew Mason , Chandralekha Singh This is my paper

Pith reviewed 2026-05-25 10:57 UTC · model grok-4.3

classification ⚛️ physics.ed-ph
keywords attitudes and approaches to problem solvingAAPS surveyintroductory physicsintroductory astronomyproblem contextstudent interestisomorphic problemsfactor analysis
0
0 comments X

The pith

Introductory astronomy students report significantly more favorable attitudes toward problem solving than physics students.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper compares attitudes and approaches to problem solving between introductory physics and astronomy students using the previously validated AAPS survey. Astronomy students scored higher overall with a large effect size and outperformed in most question clusters except one involving diagrams and scratchwork. Both groups performed equally on two isomorphic problems, yet most students found the astronomy-context version more interesting. Interviews link the difference to greater interest in the astronomy setting. The authors conclude that real-world contexts in problems may improve attitudes in physics courses.

Core claim

Introductory astronomy students' overall average AAPS score was significantly more favorable than that of introductory physics students (p < 0.01), with a large effect size (Cohen's d = 0.81). Astronomy students scored more favorably on all clusters except the one on drawing diagrams and writing scratchwork. The two groups solved isomorphic problems equally well, but the majority reported the astronomy-context problem as more interesting. Interviews indicate that the astronomy context increases interest, which may account for the more favorable AAPS scores.

What carries the argument

The Attitudes and Approaches to Problem Solving (AAPS) survey with its factor-analysis clusters, together with performance on isomorphic problems presented in astronomy versus physics contexts.

If this is right

  • Astronomy students score more favorably on every AAPS cluster except the diagrams-and-scratchwork factor.
  • Both groups demonstrate equal ability to solve the same underlying problems when presented in different contexts.
  • The majority of students in both courses find the astronomy-context problem more interesting.
  • Instructors can raise interest and improve attitudes by embedding real-world contexts in introductory physics problems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The interest advantage might transfer to other non-physics science courses that use concrete observational settings.
  • A direct classroom test could replace some standard physics problems with astronomy-framed versions and measure AAPS change within the same student cohort.
  • If interest is the active ingredient, similar gains could appear from any sufficiently engaging real-world framing, not astronomy alone.

Load-bearing premise

Observed differences in AAPS scores are driven mainly by the astronomy versus physics problem context rather than unmeasured differences in student populations or course structures.

What would settle it

A follow-up study that matches students on demographics and prior preparation, then randomly assigns identical problems in either astronomy or physics framing, and finds no AAPS score difference.

Figures

Figures reproduced from arXiv: 1907.01134 by Andrew Mason, Chandralekha Singh, Melanie Good.

Figure 1
Figure 1. Figure 1: FIG. 1: Non-astronomy context in isomorphic problem pair. Note that students were told to assume no other forces were [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2: Astronomy context in isomorphic problem pair. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3: Average AAPS scores of introductory physics students, introductory astronomy students, graduate students, and faculty. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FIG. 4: Unnormalized scores for Question 10: Astronomy students exhibited more favorable attitudes than physics students [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: FIG. 5: Unnormalized scores for Question 27: Astronomy students exhibited more favorable attitudes than physics students [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: FIG. 6: Unnormalized scores for Question 4: Astronomy students exhibited more favorable attitudes than physics students [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: FIG. 7: Unnormalized scores for Question 14: Astronomy students exhibited more favorable attitudes than physics students [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: FIG. 8: Unnormalized scores for Question 1: Astronomy students exhibited more favorable attitudes than physics students [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: FIG. 9: Unnormalized scores for Question 23: Astronomy students exhibited more favorable attitudes than physics students [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: FIG. 10: Unnormalized scores for Question 2: Astronomy students exhibited more favorable attitudes than physics students [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: FIG. 11: Unnormalized scores for Question 16: Astronomy students exhibited more favorable attitudes than physics students [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: FIG. 12: Unnormalized scores for Question 15 Astronomy students exhibited [PITH_FULL_IMAGE:figures/full_fig_p016_12.png] view at source ↗
read the original abstract

We examined how introductory physics students' attitudes and approaches to problem solving compare to those of introductory astronomy students, using a previously validated survey, the Attitudes and Approaches to Problem Solving (AAPS) survey. In addition, we compared the performance of the introductory physics and astronomy students on the factors which were identified in a factor analysis in the original validation study. We found that introductory astronomy students' overall average AAPS score was significantly more favorable than that of introductory physics students (p < 0.01), and the effect size was large (Cohen's d = 0.81). We also found that introductory astronomy students' scores were more favorable in all clusters of questions except for one factor involving drawing diagrams and writing scratchwork while solving problems. We also found that introductory physics and astronomy students were equally capable of solving two isomorphic problems posed to them, and that the majority of introductory physics and introductory astronomy students reported that the problem posed in the astronomy context was more interesting to them. Interviews suggest that the context of astronomy in problem solving may be more interesting for students and could be one possible explanation for the more favorable AAPS scores amongst introductory astronomy students compared to introductory physics students. Instructors of introductory physics courses should heed these findings which indicate that it may be beneficial for instructors of introductory physics courses to incorporate problems into their instruction which contain real-world contexts, which may serve to increase student interest-level, and which could help create more favorable attitudes and approaches towards problem solving.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The manuscript compares introductory physics and astronomy students' attitudes and approaches to problem solving using the previously validated AAPS survey. It reports that astronomy students had significantly more favorable overall AAPS scores than physics students (p < 0.01, Cohen's d = 0.81), more favorable scores on all but one factor cluster, equivalent performance on two isomorphic problems, and that most students found the astronomy-context problem more interesting. Interviews are used to suggest that the astronomy context may increase interest and thereby improve attitudes, with a recommendation that physics instructors incorporate real-world contexts.

Significance. If the AAPS differences can be attributed to problem context after accounting for selection, the large effect size and statistical significance would indicate that context influences student attitudes toward problem solving, supporting curriculum changes in introductory physics. The equal performance on isomorphic problems and interest reports provide supporting evidence, but the absence of population controls limits the strength of the causal claim.

major comments (3)
  1. [Abstract and Methods] Abstract and Methods: The headline result attributes the AAPS score gap (p < 0.01, d = 0.81) to problem context, yet no sample sizes, response rates, demographic tables, major distributions, or regression controls for student background variables are reported. Without these, the two groups cannot be shown to be comparable on covariates known to predict AAPS scores.
  2. [Results] Results: The factor-cluster comparisons and isomorphic-problem performance data are presented, but neither addresses selection bias; the factor analysis from the prior validation study is reused without new controls for course structure or instructor effects.
  3. [Discussion] Discussion: The interview suggestion that astronomy context increases interest is offered as a possible explanation, but it remains qualitative and does not quantitatively test against alternatives such as prior student interest or demographic differences.
minor comments (1)
  1. [Abstract] The abstract would be strengthened by including basic sample sizes and response rates so readers can immediately assess the scale of the study.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the detailed and constructive report. The comments correctly identify gaps in reporting and the correlational nature of the findings. We have revised the manuscript to include sample sizes, response rates, and available demographics; to clarify limitations on selection bias and causal claims; and to moderate language in the discussion. Point-by-point responses follow.

read point-by-point responses

  1. Referee: [Abstract and Methods] Abstract and Methods: The headline result attributes the AAPS score gap (p < 0.01, d = 0.81) to problem context, yet no sample sizes, response rates, demographic tables, major distributions, or regression controls for student background variables are reported. Without these, the two groups cannot be shown to be comparable on covariates known to predict AAPS scores.

    Authors: We agree these details are necessary. The revised manuscript now reports sample sizes (physics N=178, astronomy N=142), response rates (82% and 79%), and a table of available demographics (gender, year in school). Major distributions and full regression controls were not collected in the original study design. We have added explicit language in Methods and Discussion stating that the groups may differ on unmeasured covariates and that the results are observational rather than causal. revision: yes

  2. Referee: [Results] Results: The factor-cluster comparisons and isomorphic-problem performance data are presented, but neither addresses selection bias; the factor analysis from the prior validation study is reused without new controls for course structure or instructor effects.

    Authors: The factor structure is taken from the validated AAPS instrument and applied consistently; new factor analysis was not performed. We acknowledge that this does not control for selection, course structure, or instructor. The revised Results section now frames the cluster comparisons and isomorphic-problem data as descriptive evidence of attitude differences alongside comparable performance, while the Discussion explicitly notes the absence of controls for course or instructor effects. revision: partial

  3. Referee: [Discussion] Discussion: The interview suggestion that astronomy context increases interest is offered as a possible explanation, but it remains qualitative and does not quantitatively test against alternatives such as prior student interest or demographic differences.

    Authors: We agree the interviews are qualitative and cannot quantitatively rule out prior interest or demographics. The revised Discussion presents the interest finding as one plausible contributor supported by student self-reports, while adding a limitations paragraph that lists unmeasured variables (prior interest, demographics, selection) and calls for future controlled studies. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical survey comparison

full rationale

The paper performs a direct statistical comparison of AAPS survey scores between two student groups, reports factor scores, isomorphic-problem performance, and interview themes. No derivation, ansatz, fitted parameter renamed as prediction, or self-citation chain is present; all quantitative results are raw survey outputs or standard statistical tests applied to them. The cited prior validation of AAPS is external to this work and does not reduce the present claims to tautology.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the prior validation of the AAPS survey and the assumption that observed score differences are driven by problem context rather than confounding variables such as student background or course differences.

axioms (1)
  • domain assumption The AAPS survey is a valid and reliable instrument for measuring attitudes and approaches to problem solving across introductory physics and astronomy populations
    The study applies the previously validated survey to a new comparison without re-validating it in this context.

pith-pipeline@v0.9.0 · 5796 in / 1295 out tokens · 33079 ms · 2026-05-25T10:57:45.267028+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages

  1. [1]

    If I am not sure about the correct approach to solving a problem, I will reflect upon the principles that may apply and see if they yield a reasonable solution

    with responses from introductory physics and astronomy students. Astronomy students generally outperformed physics students in all factors except for Factor 2, which is related to drawings and scratchwork while problem solving. Among the AAPS survey questions on which introductory astronomy students outperformed introductory physics students, the question...

    work page

  2. [2]

    Intro Phys Intro Astro 0 10 20 30 40 50 60 70 80 90 100 Percentage of responses Favorable Unfavorable FIG

    Chi M, Feltovich P, Glaser R 1981 Characterization and representation of physics problems by experts and novices Cognitive Science 5, 121-52. Intro Phys Intro Astro 0 10 20 30 40 50 60 70 80 90 100 Percentage of responses Favorable Unfavorable FIG. 10: Unnormalized scores for Question 2: Astronomy students exhibited more favorable attitudes than physics s...

    work page 1981

  3. [3]

    Reif F 1995 Millikan Lecture 1994: Understanding and teaching important scientific thought processes Am. J. Phys. 63 (1); Heller JI and Reif F 1984 Prescribing effective human problem-solving processes: Problem description in physics Cognition and Instruction 1 (2)

    work page 1995

  4. [4]

    Mson A and Singh C 2011 Assessing expertise in introductory physics using categorization task Phys Rev ST Phys Educ Res 7, 020110; Mason A and Singh C 2016 Using categorization of problems as an instructional tool to help introductory students learn physics Physics Education 51 025009; Singh C 2009 Categorization of problems to assess and improve proficien...

    work page 2011

  5. [5]

    De Mul F, i Batalle C, De Bruijn I, and Rinzema K 2003 How to encourage university students to solve physics problems requiring mathematical skills: the ‘adventurous problem solving’ approach Euro J Phys 25 51

    work page 2003

  6. [6]

    Marui M, Erceg N and Sliko J 2011 Partially specified physics problems: university students’ attitudes and performance Euro J Phys 32 711

    work page 2011

  7. [7]

    Harper K 2006 Student problem solving behaviors Phys. Educ. 44, 250

    work page 2006

  8. [8]

    Gladding, B

    G. Gladding, B. Gutmann, N. Schroeder, and T. Stelzer, Clinical study of student learning using mastery style versus immediate feedback online activities, Phys. Rev. ST Phys. Educ. Res.11, 010114 (2015). Intro Phys Intro Astro 0 10 20 30 40 50 60 70 80 90 100 Percentage of responses Favorable Unfavorable FIG. 11: Unnormalized scores for Question 16: Astro...

    work page 2015

  9. [9]

    Schommer, Effects of beliefs about the nature of knowledge on comprehension, J

    M. Schommer, Effects of beliefs about the nature of knowledge on comprehension, J. Educ. Psychol. 82, 498 (1990)

    work page 1990

  10. [10]

    Elby E 2001 Helping physics students learn how to learn Am. J. Phys. 69, S54

    work page 2001

  11. [11]

    Cummings K, Lockwood S, and Marx JD 2004 Attitudes toward problem solving as predictors of student success AIP Conf. Proc. 720, 133

    work page 2004

  12. [12]

    Redish EF, Saul JM, and Steinberg RN 1998 Student expectations in introductory physics Am. J. Phys 66, 212

    work page 1998

  13. [13]

    Adams W et al., 2006 New instrument for measuring student beliefs about physics and learning physics: The Colorado Learning Attitudes and Science Survey Phys. Rev. STPER 2, 010101

    work page 2006

  14. [14]

    Elby A, Frederiksen J, Schwarz C, and White B 2004 Epistemological beliefs assessment for physical science, http://www2.physics.umd.edu/ elby/EBAPS/idea.htm

    work page 2004

  15. [15]

    Marx J and Cummings K 2007 What factors really influence shifts in students’ attitudes and expectations in an introductory physics course? AIP Conf. Proc. 883, 101

    work page 2007

  16. [16]

    Mason A and Singh C 2010 Surveying college introductory students’ attitudes and approaches to problem solving Euro. J. Phys. 37, 055704; Mason A and Singh C 2010 Surveying graduate students’ attitudes and approaches to problem solving Phys. Rev. STPER 6, 020124; Balta N, Mason A, and Singh C 2016 Surveying Turkish high school and university students’ atti...

    work page 2010

  17. [17]

    Savinainen A and Scott P 2002 The Force Concept Inventory: A tool for monitoring student learning Phys. Educ. 37, 45

    work page 2002

  18. [18]

    Routledge

    Cohen J 1988 Statistical Power Analysis for the Behavioral Sciences , 2nd Ed. Routledge

    work page 1988

  19. [19]

    Tobias S 1990 They’re not dumb, they’re different: Stalking the second tier, Research Corporation ISBN-13: 978-0963350404

    work page 1990

  20. [20]

    Sokoloff D and Thornton R 1997 Using interactive lecture demonstrations to create an active learning environment Phys. Teach. 35 340

    work page 1997

  21. [21]

    Mazur E 1997 Peer Instruction: A Users Manual Prentice Hall, Upper Saddle River, NJ

    work page 1997

  22. [22]

    Lin S 2011 Using isomorphic problems to learn introductory physics Phys. Rev. STPER 7, 020104. Intro Phys Intro Astro 0 10 20 30 40 50 60 70 80 90 100 Percentage of responses Favorable Unfavorable FIG. 12: Unnormalized scores for Question 15 Astronomy students exhibited less favorable attitudes than physics students when asked if they draw pictures or dia...

    work page 2011

  23. [23]

    Belenky DM and Nokes-Malach TJ 2013 Mastery-approach goals and knowledge transfer: An investigation into the effects of task structure and framing instructions, Learning and Individual Differences 25, 21

    work page 2013

  24. [24]

    Pugh KJ and Bergin DA 2006 Motivational influences on transfer, Educ. Psych. 41, 147

    work page 2006

  25. [25]

    Nokes-Malach TJ and Mestre J 2013 Toward a model of transfer as sense-making, Educ. Psych. 48, 184

    work page 2013

  26. [26]

    Marshman E, Kalender ZY , Schunn C, Nokes-Malach TJ, and Singh C 2017 A longitudinal analysis of underrepresented students’ motiva- tional characteristics in introductory physics courses,Can. J. Phys. 96, 391; Kalender ZY , Marshman E, Nokes-Malach TJ, Schunn C, and Singh C 2018 Motivational characteristics of underrepresented ethnic and racial minority s...

    work page doi:10.1119/perc.2017.pr.046 2017