pith. sign in

arxiv: 2605.22657 · v1 · pith:MYRHGHE2new · submitted 2026-05-21 · 💻 cs.HC

Student programming behavior with and without phone notification suppression

Gavin Eddington , Christopher Warren , Seth Poulsen , John Edwards This is my paper

Pith reviewed 2026-05-22 03:36 UTC · model grok-4.3

classification 💻 cs.HC
keywords notification suppressionprogramming behaviorstudent attentionCS1 educationphone distractionsfocus intervalsreplication studywithin-subject design
0
0 comments X

The pith

Suppressing phone notifications lowers break rates and lengthens focus intervals for many but not all programming students.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether blocking phone notifications changes how students engage with programming assignments. In a CS1 course, 22 students completed selected tasks under a within-subject design where suppression was randomly enabled or left in the control state. Phone state logs were aligned with millisecond keystroke records from the IDE to quantify break frequency and sustained focus periods. Results indicated lower break rates and longer focus stretches under suppression for many students, with a clear bimodal pattern: most improved, a minority worsened, and very few showed no change. The work positions notification tools such as Do Not Disturb as a potential practical intervention for supporting attention in coding work.

Core claim

Assignments completed with notification suppression enabled significantly lower break rates and longer intervals of focus compared to assignments completed in the control condition for many, but not all, students. The study also reports a remarkable bimodality in the effect across students, with many positively affected, a small number negatively affected, and very few experiencing little or no effect.

What carries the argument

Within-subject random assignment of programming tasks to notification-suppression or control conditions, measured through synchronized phone state logs and millisecond-resolution IDE keystroke data.

If this is right

  • For many students, enabling notification suppression improves measurable attention during programming tasks.
  • Do-not-disturb settings can function as a targeted intervention in computing education courses.
  • Individual differences in response to notifications must be accounted for when studying or applying distraction-reduction techniques.
  • The observed bimodality matches patterns reported in studies of phone use across other disciplines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Predictors of which students will benefit or be harmed by suppression could be identified through additional student-level variables.
  • Similar notification controls might be tested for their effect on focus in non-programming academic tasks.
  • Learning platforms could incorporate optional automatic suppression features during assigned work sessions.

Load-bearing premise

The synchronization of phone state logs with keystroke data produces an accurate, unbiased measure of when students are focused versus taking breaks.

What would settle it

A replication that measures focus with an independent method such as eye-tracking or self-report and finds no reduction in breaks under suppression would falsify the central association.

Figures

Figures reproduced from arXiv: 2605.22657 by Christopher Warren, Gavin Eddington, John Edwards, Seth Poulsen.

Figure 1
Figure 1. Figure 1: (a) Break rates (Breaks per 1,000 keystrokes) for non [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Break rates (breaks per 1,000 keystrokes) are shown [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Each bar represents a student’s percent change in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Histogram of percent reduction in break rate for [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Kernel density estimates of break duration (breaks [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
read the original abstract

Background and Context. Computer programming often involves extended periods of sustained activity and mobile phone notifications introduce frequent opportunities for interruption. Prior work demonstrates that suppressing phone notifications may reduce these disruptions. Objectives. Our primary research question is: How does suppressing phone notifications affect students' task engagement and productivity while programming? Method. We report on a replication and methodological extension study conducted in a CS1 course involving 22 students. Using a within-subject design, selected programming assignments were randomly designated for enabling notification suppression. Phone state logs were synchronized with millisecond-resolution IDE keystroke data to measure student attention and focus when in the control and notification-suppression conditions. Findings. Assignments completed with notification suppression enabled significantly lower break rates and longer intervals of focus compared to assignments completed in the control condition for many, but not all, students. This study provides evidence that notification suppression is associated with measurable differences in programming engagement and behavior. We also find a remarkable bimodality in the effect across students -- many students are positively affected, a small number are negatively affected, and very few experience little or no effect. This finding is consistent with other studies in diverse disciplines. Implications. Our results show that, for many students, phone notification suppression tools, such as Do Not Disturb, can improve attention and focus. Implications apply to educational settings (do-not-disturb as an intervention) and scholarship (understanding the effects of phone distraction).

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper reports results from a within-subject experiment with 22 CS1 students in which selected programming assignments were randomly assigned to a notification-suppression condition or a control condition. Phone-state logs were synchronized with millisecond-resolution IDE keystroke data to derive measures of break rates and focus intervals. The central claim is that suppression is associated with lower break rates and longer focus intervals for many (but not all) students, with a bimodal pattern of individual effects.

Significance. If the reported differences survive controls for assignment heterogeneity, the work supplies logged behavioral evidence that notification suppression can measurably alter programming engagement in an authentic educational setting. The documented bimodality is consistent with findings from other domains and supports the possibility of individualized interventions.

major comments (1)
  1. Method section: the within-subject random assignment of distinct programming assignments to suppression versus control does not include assignment identity, length, or difficulty as a covariate or fixed effect. Because CS1 assignments routinely differ in expected duration and cognitive demand, any average difference in break rates or focus intervals could reflect task properties rather than the suppression manipulation, undermining causal attribution even with perfect synchronization of phone and keystroke logs.
minor comments (2)
  1. Abstract and Findings: operational definitions of 'break' and 'focus interval' are not stated; explicit criteria (e.g., idle-time threshold, keystroke-gap cutoff) should be provided so readers can evaluate the derived metrics.
  2. Findings: the abstract asserts 'significantly lower break rates' yet supplies no test statistic, p-value, effect size, or error bars; these details are required for assessing the strength of the reported differences.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive comments. The concern about potential confounding from assignment heterogeneity in our within-subject design is a valid methodological point. We address it directly below, agreeing that additional controls will strengthen causal attribution, and we outline the planned revisions.

read point-by-point responses

  1. Referee: Method section: the within-subject random assignment of distinct programming assignments to suppression versus control does not include assignment identity, length, or difficulty as a covariate or fixed effect. Because CS1 assignments routinely differ in expected duration and cognitive demand, any average difference in break rates or focus intervals could reflect task properties rather than the suppression manipulation, undermining causal attribution even with perfect synchronization of phone and keystroke logs.

    Authors: We agree that explicitly accounting for assignment characteristics would improve the robustness of our causal claims. Although random assignment of distinct assignments to conditions within each participant balances task properties in expectation, we acknowledge that this does not substitute for direct statistical control. In the revised manuscript we will add assignment identity as a fixed effect in the mixed-effects models used to analyze break rates and focus intervals. We have re-run the analyses with this control; the key results—lower break rates and longer focus intervals under suppression for many students, together with the bimodal pattern of individual effects—remain consistent in direction and significance. We will also report the assignment coefficients to allow readers to assess the magnitude of task-related variation. revision: yes

Circularity Check

0 steps flagged

No circularity: direct empirical comparison from randomized within-subject logs

full rationale

The paper describes a within-subject randomized experiment that assigns programming tasks to notification-suppression or control conditions, then compares observed break rates and focus intervals derived from synchronized phone-state logs and millisecond IDE keystroke timestamps. No equations, fitted parameters, or derivations are presented; the reported differences are raw empirical contrasts. No self-citation load-bearing steps, uniqueness theorems, or ansatzes appear in the provided text. The design is self-contained against external benchmarks (logged behavioral data) and does not reduce any claim to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The central claim rests on empirical observation of logged behavior rather than theoretical derivations; no free parameters, unstated axioms, or newly postulated entities are introduced beyond standard assumptions of behavioral measurement.

pith-pipeline@v0.9.0 · 5782 in / 1208 out tokens · 51794 ms · 2026-05-22T03:36:18.042093+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages

  1. [1]

    B. S. Bloom. Time and learning.American psychologist, 29(9):682, 1974

    work page 1974

  2. [2]

    J. B. Carroll. A model of school learning.Teachers college record, 64(8):1–9, 1963

    work page 1963

  3. [3]

    Q. Chen, Z. Yan, M. Moeyaert, and R. Bangert-Drowns. Mobile multitasking in learning: A meta-analysis of effects of mobilephone distraction on young adults’ immediate recall.Computers in Human Behavior, 162:108432, 2025

    work page 2025

  4. [4]

    A. W. Chickering and Z. F. Gamson. Seven principles for good practice in undergraduate education.Biochemical Education, 17(3):140–141, 1989

    work page 1989

  5. [5]

    Z. Deng, Z. A. Cheng, P. Ferreira, and P. A. Pavlou. From smartphones to smart students: Learning vs. distraction using smartphones in the classroom. Information Systems Research, 2025

    work page 2025

  6. [6]

    J. Edwards. JetBrains Marketplace; ShowYourWork Plugin, 2025

    work page 2025

  7. [7]

    Edwards, K

    J. Edwards, K. Hart, and C. Warren. A practical model of student engagement while programming. InProceedings of the 2022 ACM SIGCSE technical symposium on computer science education, 2022

    work page 2022

  8. [8]

    Edwards, J

    J. Edwards, J. Leinonen, and A. Hellas. A study of keystroke data in two contexts: Written language and programming language influence predictability of learning outcomes. InProceedings of the 51st ACM Technical Symposium on Computer Science Education, pages 413–419, 2020

    work page 2020

  9. [9]

    C. M. End, S. Worthman, M. B. Mathews, and K. Wetterau. Costly cell phones: The impact of cell phone rings on academic performance.Teaching of Psychology, 37(1):55–57, 2009

    work page 2009

  10. [10]

    N. Fitz, K. Kushlev, R. Jagannathan, T. Lewis, D. Paliwal, and D. Ariely. Batching smartphone notifications can improve well-being.Computers in Human Behavior, 101:84–94, 2019

    work page 2019

  11. [11]

    P. E. Fortin and J. Cooperstock. Understanding smartphone notifications’ activity disruption via in situ wrist motion monitoring. InCHI Conference on Human Factors in Computing Systems Extended Abstracts, pages 1–6, 2022

    work page 2022

  12. [12]

    Graben, B

    K. Graben, B. K. Doering, and A. Barke. Receiving push-notifications from smartphone games reduces students learning performance in a brief lecture: An experimental study.Computers in Human Behavior Reports, 5:100170, 2022

    work page 2022

  13. [13]

    K. Hart, C. Mano, and J. Edwards. Plagiarism deterrence in cs1 through keystroke data. InProceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1, pages 493–499, 2023

    work page 2023

  14. [14]

    K. Hart, C. Warren, and J. Edwards. Accurate estimation of time-on-task while programming. InProceedings of the 2023 ACM SIGCSE technical symposium on computer science education, 2023

    work page 2023

  15. [15]

    K. Hart, C. M. Warren, S. Poulsen, and J. Edwards. Phone use while program- ming. In B. Paaßen and C. D. Epp, editors,Proceedings of the 17th International Conference on Educational Data Mining, pages 270–278, Atlanta, Georgia, USA, July 2024. International Educational Data Mining Society

    work page 2024

  16. [16]

    Hellas, J

    A. Hellas, J. Leinonen, and P. Ihantola. Plagiarism in take-home exams: help- seeking, collaboration, and systematic cheating. InProceedings of the 2017 ACM conference on innovation and technology in computer science education, pages 238–243, 2017

    work page 2017

  17. [17]

    R. Hoong. Self control and smartphone use: An experimental study of soft commitment devices.European Economic Review, 140:103924, 2021

    work page 2021

  18. [18]

    N. Karweit. Time-on-task reconsidered: Synthesis of research on time and learning.Educational leadership, 41(8):32–35, 1984

    work page 1984

  19. [19]

    Y.-H. Kim, J. H. Jeon, E. K. Choe, B. Lee, K. Kim, and J. Seo. Timeaware: Leveraging framing effects to enhance personal productivity. InProceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 272–283, 2016

    work page 2016

  20. [20]

    D. J. Kruger and J. M. Djerf. High ringxiety: Attachment anxiety predicts ex- periences of phantom cell phone ringing.Cyberpsychology, behavior, and social networking, 19(1):56–59, 2016

    work page 2016

  21. [21]

    D. J. Kruger and J. M. Djerf. Bad vibrations? cell phone dependency predicts phantom communication experiences.Computers in Human Behavior, 70:360–364, 2017

    work page 2017

  22. [22]

    Leinonen, F

    J. Leinonen, F. E. V. Castro, and A. Hellas. Time-on-task metrics for predicting performance. InProceedings of the 53rd ACM Technical Symposium on Computer Science Education, pages 871–877, 2022

    work page 2022

  23. [23]

    Liao and S

    M. Liao and S. S. Sundar. Sound of silence: Does muting notifications reduce phone use?Computers in Human Behavior, 134:107338, 2022

    work page 2022

  24. [24]

    Longi, J

    K. Longi, J. Leinonen, H. Nygren, J. Salmi, A. Klami, and A. Vihavainen. Iden- tification of programmers from typing patterns. InProceedings of the 15th Koli Calling conference on computing education research, pages 60–67, 2015

    work page 2015

  25. [25]

    Martin, S

    F. Martin, S. Long, K. Haywood, and K. Xie. Digital distractions in education: A systematic review of research on causes, consequences and prevention strategies. Educational Technology Research and Development, 2025

    work page 2025

  26. [26]

    N. Moray. Attention in dichotic listening: Affective cues and the influence of instructions.Quarterly journal of experimental psychology, 11(1):56–60, 1959

    work page 1959

  27. [27]

    Oberst, E

    U. Oberst, E. Wegmann, B. Stodt, M. Brand, and A. Chamarro. Negative conse- quences from heavy social networking in adolescents: The mediating role of fear of missing out.Journal of adolescence, 55:51–60, 2017

    work page 2017

  28. [28]

    Pielot and L

    M. Pielot and L. Rello. Productive, anxious, lonely: 24 hours without push notifications. InProceedings of the 19th International Conference on Human- Computer Interaction with Mobile Devices and Services, pages 1–11, 2017

    work page 2017

  29. [29]

    A. K. Przybylski, K. Murayama, C. R. DeHaan, and V. Gladwell. Motivational, emotional, and behavioral correlates of fear of missing out.Computers in human behavior, 29(4):1841–1848, 2013

    work page 2013

  30. [30]

    F. I. Rahmillah, A. Tariq, M. King, and O. Oviedo-Trespalacios. Evaluating the ef- fectiveness of apps designed to reduce mobile phone use and prevent maladaptive mobile phone use: Multimethod study, aug 2023

    work page 2023

  31. [31]

    R. D. Rogers and S. Monsell. Costs of a predictible switch between simple cognitive tasks.Journal of experimental psychology: General, 124(2):207, 1995

    work page 1995

  32. [32]

    A. Roye, T. Jacobsen, and E. Schröger. Personal significance is encoded auto- matically by the human brain: an event-related potential study with ringtones. European Journal of Neuroscience, 26(3):784–790, 2007

    work page 2007

  33. [33]

    L. Saad. Americans have close but wary bond with their smartphone, May 2023. Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al

    work page 2023

  34. [34]

    Stallings

    J. Stallings. Allocated academic learning time revisited, or beyond time on task. Educational researcher, 9(11):11–16, 1980

    work page 1980

  35. [35]

    Stothart, A

    C. Stothart, A. Mitchum, and C. Yehnert. The attentional cost of receiving a cell phone notification.Journal of experimental psychology: human perception and performance, 41(4):893, 2015

    work page 2015

  36. [36]

    Thornton, A

    B. Thornton, A. Faires, M. Robbins, and E. Rollins. The mere presence of a cell phone may be distracting.Social Psychology, 2014

    work page 2014

  37. [37]

    Urry and J

    J. Urry and J. Edwards. A Framework that Explores the Cognitive Load of CS1 Assignments Using Pausing Behavior. InACM Technical Symposium on Computing Science Education (SIGCSE), 2024

    work page 2024

  38. [38]

    A. F. Ward, K. Duke, A. Gneezy, and M. W. Bos. Brain drain: The mere presence of one’s own smartphone reduces available cognitive capacity.Journal of the Association for Consumer Research, 2(2):140–154, 2017

    work page 2017

  39. [39]

    Yoon and K.-p

    S. Yoon and K.-p. Lee. A study on notification system design of smartphone messenger considering the user’s stress.Archives of design research, 28(2):75–88, 2015

    work page 2015

  40. [40]

    Zamani, K

    M. Zamani, K. Hall, A. Cunningham, N. Chin, S. Kent-Ferguson, and V. Wadhwa. Effectiveness of ‘do not disturb’strategies in reducing errors during discharge prescription writing.Journal of Pharmacy Practice and Research, 49(5):433–438, 2019

    work page 2019