Pith. sign in

REVIEW 2 major objections 6 minor 1 cited by

Human Attention During Localization of Memory Bugs in C Programs

T0 review · 2 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Eye-tracking study of C memory bugs ties success to rereading and tight gaze.

desk verdict First eye-tracking dataset for C memory-bug localization, with a solid descriptive result; the success/failure claims need nested statistics before they land. read the letter →

arxiv 2506.00693 v3 pith:HAQAYX7Z submitted 2025-05-31 cs.SE

Emory Michaels , Robert Wallace , Matthew Robison , Yu Huang , Collin McMillan This is my paper
classification cs.SE
keywords eyetrackingvisualattentionbuglocalizationmemorybugsCprogramsprogramcomprehensionregressionratefixationanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports an eye-tracking study of 21 programmers who spent up to two hours each trying to locate memory bugs (leaks, overflows, double frees) in three real C programs. It tries to establish that success and failure in bug localization are visibly different in the eyes: successful localizers re-fixate already-read tokens at a higher rate and keep successive fixations closer together, while unsuccessful ones often look directly at the correct line but do not register it as the answer. The study also documents that visual attention is highly concentrated, with 75% of fixations landing on 25% of the functions viewed. If these patterns hold, gaze data could become a real-time signal for when a programmer is about to miss a bug they have already seen.

What carries the argument

The argument runs on gaze-based metrics computed from eye-tracker fixations. A fixation is a brief pause of the eyes that lets the brain process what is seen; the study detects fixations with an IVT filter using a velocity threshold of 50 and a minimum duration of 80 ms. Two metrics carry the central comparison: regression rate, the percentage of fixations whose token was fixated more than once, and mean Euclidean distance, the average pixel distance from each fixation to the next. These are compared between tasks graded high (accuracy 4 or 5) and low (accuracy 1 to 3) on a rubric for locating the bug, with the high and low groups defined by the authors' graded 'where accuracy' scores. The same fixation data also yield the concentration result that about 75% of visual effort falls on 25% of the functions viewed.

What would settle it

Re-analyze the released eye-tracking data with a mixed-effects model that includes random intercepts for participant and bug; if the regression-rate and fixation-distance differences disappear or drop below significance once individual skill and bug difficulty are accounted for, the central claim would be refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that success and failure in locating C memory bugs are associated with distinguishable patterns of visual attention. On tasks where participants pinpointed the correct location, their eyes returned to already-read tokens more often (higher regression rate) and traveled shorter distances between successive fixations than on low-accuracy tasks; those two differences were statistically significant in Mann-Whitney U tests. At the same time, participants in failing tasks often did fixate the line that would have been the correct answer, so failure was not simply failure to look. The authors interpret the combination as evidence that what separates success from failure is not finding the buggy line but registering it: successful attempts involve rereading and dwelling, while unsuccessful attempts glance at the right line and move on.

Load-bearing premise

The paper's strongest statistical claim assumes each participant-task contributes an independent observation, even though the same people and the same bugs appear many times; if success-related attention is really driven by who the participant is or how hard the bug is, the reported differences would shrink.

Editorial extensions

If this is right

  • If success is marked by rereading and tight scanning, then exposing programmers to the relevant code is not enough; interventions should target recognition and confirmation of a suspicious line.
  • Gaze metrics like regression rate and fixation distance can be computed in real time, so they could be used to flag when a developer has looked at a correct line but is about to move on.
  • The 75/25 concentration result quantifies how little code context humans actually use, which can inform how much context debuggers or AI assistants should surface.
  • Because self-reported attention was high (91% of probe responses indicated focus), the observed success/failure differences are not primarily explained by mind wandering or external distraction in this setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's between-group comparison mixes participants and bugs, so a natural next step is a within-subject analysis: does the same programmer show a higher regression rate on bugs they solve than on bugs they miss, holding the bug fixed? The released data would allow this check.
  • If rereading causes recognition, then training or user-interface nudges that encourage deliberate revisits of candidate lines would be a testable intervention; the paper does not test that.
  • The 'looked but did not see' pattern hints that failed localization is partly a recognition or confirmation failure rather than a pure search failure, which could be explored with retrospective interviews or think-aloud protocols.
  • The attention metrics might transfer to other bug types or programming languages, but the paper only studies three C programs, so that transfer is an open question.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. This manuscript reports an eye-tracking study of 21 programmers attempting to localize eight memory bugs (leaks, overflows, double frees) in three C programs, with 31 hours of observed effort. The paper addresses three research questions: task outcomes, overall patterns of visual attention, and differences in attention between successful and failed bug localization attempts. The principal findings are that participants concentrate about 75% of fixations on 25% of viewed functions; that even unsuccessful participants often fixate on the correct buggy line; that higher 'where accuracy' is associated with higher regression rate and lower mean Euclidean fixation distance (p=0.003 and p=0.0004 in Table 11); and that successful cases show more time on or regressions to the buggy line in illustrative examples. The authors release their data and provide transparent descriptions of the procedure, grading rubrics, and threats to validity.

Significance. If the results hold, this is a useful first step in characterizing human visual attention specifically for memory-bug localization in C, a task type that has received less eye-tracking attention than Java program comprehension. The study's strengths include a clearly described experimental protocol, a reproducibility package with raw and processed data, explicit rubrics for grading 'what' and 'where' answers, and appropriately cautious language about the preliminary nature of the findings. The observation that unsuccessful participants often fixate on the correct line is an interesting and falsifiable result that could motivate future work on why programmers dismiss correct locations. However, the central quantitative claim linking attention metrics to success is currently supported by pooled tests that ignore the nested structure of the data, and one of the four key findings rests on anecdotal examples rather than a statistical comparison.

major comments (2)
  1. [Section 4.3, Table 11] The Mann-Whitney U tests for regression rate and mean Euclidean distance treat each participant-task as an independent observation, but the 21 participants contribute up to eight tasks each and the tasks are drawn from only eight bug reports. Figure 11 shows that high and low outcome groups are imbalanced by participant, and Table 7 shows strong bug-level variation in regression rate (e.g., weevil 67% versus silverfish 43%). Under this analysis, the reported p-values (0.003 and 0.0004) cannot distinguish the claim that successful localizers genuinely regress more and scan less from the confound that less skilled participants, or more difficult bugs, produce both lower accuracy and a different eye-movement signature. The text acknowledges the imbalance in the Figure 11 discussion but does not model it; without random intercepts for participant and bug, or a participant-level aggregation analysis, the evidence for Key Finding 3 is overstated.
  2. [Section 4.3, Key Finding 4 and Figures 10a-10d] Key Finding 4 states that in success cases participants 'often spend more time on the buggy line or regress back to the buggy line several times,' but this claim is supported only by four selected fixation timelines. No quantitative definition of 'time on the buggy line' or 'regressions to the buggy line' is given, and no test compares these quantities across the high and low accuracy groups. If this is intended as a key finding, it needs a metric and a statistical comparison; if it is intended as anecdotal motivation, it should be labeled as such rather than presented as a finding alongside Key Findings 1-3.
minor comments (6)
  1. [Section 4.3, paragraph 1] The phrase 'statically significant results' should be 'statistically significant results'.
  2. [Section 5.2] The sentence 'To our knowledge, this first study to examine human visual attention during memory bug localization in C programs' is missing a verb; it should read 'this is the first study.'
  3. [Table 11 and text throughout] The metric name 'mean euclidean distance' should be capitalized as 'mean Euclidean distance' for consistency with standard terminology.
  4. [Figures 5a and 5b] The y-axis tick labels (100, 101, 102) appear to be a logarithmic scale, but the caption does not state this; please clarify the scale in the caption or use linear ticks.
  5. [Table 10] Several entries are marked 'UNK' without a footnote explaining how to interpret them; the caption mentions that participants changed regions, but the current explanation is too terse to allow readers to interpret the table.
  6. [Sections 3.2 and 3.5] Section 3.2 says 'Three bugs were the same for all participants' while Section 3.5 says 'The first two bug reports were the same for everyone'; these statements should be reconciled.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the key findings are direct group comparisons of independently measured eye-tracking metrics, with no fitted-input-called-prediction or self-citation chain doing the work.

full rationale

The paper's central claims are empirical observations from a controlled eye-tracking study, not derivations from fitted equations. Key metrics such as fixation count, fixation duration, regression rate, mean Euclidean distance, unique lines read, and unique methods read are computed from raw eye-tracking fixations independently of the outcome variable, and RQ3 then compares these pre-computed metrics between outcome groups via Mann-Whitney U tests (Section 4.3, Table 11). The high/low accuracy grouping is an explicit analytic choice that the authors stress-test: 'if we choose to group the scores in different ways ... the key findings and statically significant results do not change.' No parameter is fitted to the outcome and then renamed as a prediction, and no result is defined in terms of the quantity it is claimed to estimate. The citations to the authors' prior work are limited to operational definitions, such as 'regression rate' credited to Wallace et al. [81], and to reusable scripts or data pipelines; they are not invoked as a uniqueness theorem or as an imported ansatz that supplies the paper's conclusions. The caveat in the Figure 11 discussion that the low and high outcome groups are not balanced by participant is a statistical inference concern about potential nesting or confounding, not a circularity: the eye-movement metrics are measured independently of the accuracy labels, so comparing them across groups does not make the result equivalent to its inputs by construction. Therefore no circular step meets the evidentiary bar, and the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claims rest on an eye-tracking pipeline (iTrace fixation settings), a subjective grading rubric, and a pooled statistical model. No new entity or fitted prediction is introduced; the main burden is interpretive (gaze equals attention) and statistical (independence of observations).

free parameters (2)
  • iTrace IVT fixation thresholds = Velocity threshold 50, duration threshold 80 ms
    Hand-chosen detection settings inherited from iTrace tool defaults; all fixation-derived metrics depend on these thresholds, though they are not fitted to the outcome.
  • High/low accuracy split = where accuracy >= 4 is high, <= 3 is low
    Post-hoc grouping chosen after inspecting the data; authors state that alternative groupings do not change the key findings.
assumptions (3)
  • domain assumption Fixations reliably index cognitive attention to the token or line being viewed.
    The entire study interprets gaze fixations as evidence of what information the programmer processed (Sections 3.6 and 4.3).
  • domain assumption The rubric-graded 'where' accuracy score is a valid measure of bug-localization success.
    Accuracy scores from two author-graded holistic rubrics (Tables 2 and 3) are the outcome variable that defines high/low performance groups; rubric validity is asserted but not independently validated.
  • domain assumption Mann-Whitney U tests with pooled participant-task observations are valid for repeated measures.
    Section 4.3 pools all tasks for testing; within-participant correlation is not modeled, and the authors acknowledge group imbalance in the Figure 11 discussion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Human Attention During Localization of Memory Bugs in C Programs." pith.science (2026). https://pith.science/paper/HAQAYX7Z

@misc{pith2026250600693,
  author       = {Pith},
  title        = {Pith review of: Human Attention During Localization of Memory Bugs in C Programs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HAQAYX7Z}},
  note         = {Machine review of arXiv:2506.00693}
}
read the original abstract

This paper presents a study of human visual attention during localization of memory bugs in C. Human visual attention refers to the mechanical processes by which we selectively process and prioritize information. Visual attention is important to study because it is central to what information people (who are sighted) use to solve a particular problem. Meanwhile, memory bugs are among the most common types of bugs in C programs that manifest as a variety of program faults. In this paper, we study human visual attention while people attempt to locate memory bugs in code. We recruit 21 programmers to locate between one and eight memory bugs in three C programs for 1.5-2 hours each. In total we collected observations of 31 hours of programmer effort. The bugs in our study cover memory leaks, overflows, and double frees, which are among the most common memory bugs. We analyze the task outcomes in terms of success rate and related factors, patterns of visual attention overall such as what lines and functions are read, and finally we explore differences of visual attention patterns during success versus failure cases.

Figures

Figures reproduced from arXiv: 2506.00693 by the authors.

Figure 1
Figure 1. This is the bug report for ladybug. This report is adapted from issue #6861 in the sway repository. 3.4 Study Participants We recruited 21 participants for this study. The participants had between 3 and 13 years of programming experience. Eleven were graduate students, nine were undergraduates, and one was a professional software engineer. Eight of the graduate students worked in the software engineering industry be… view at source ↗
Figure 2
Figure 2. The interface presented to participants. Area A is the bug report, area B is the code, area C is the file explorer, and area D is a [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Histograms of confidence, difficulty, and accuracy scores from all participants and tasks. Confidence and difficulty are [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Official fix for ladybug. See [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Bar charts showing the distribution of fixations over functions and lines of code. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Percent fixations on code for each bug. Each point represents one participant’s percent of time looking at code versus other [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Percent fixations on a “correct bug line” from lowest to highest “where accuracy” score for each participant-task. More fixations [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: As ‘Where Accuracy” increases, the mean distance between fixations tends to decrease, and the regression rate tends to [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Boxplots to accompany statistical summary of eye metrics. Low group means accuracy scores [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Comparing successful and failure for cases for bug [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 10
Figure 10. Figure 10: Comparing successful and failure for cases for bug [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]
Figure 11
Figure 11. Figure 11: Distribution of bugs per participant grouped by “Where Accuracy.” Accuracy level 2 was most common, but participants were [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Predicting Human Visual Attention on Words in Source Code

    cs.SE 2026-07 conditional novelty 4.0 of 10

    A Transformer fine-tuned with a loss that matches its internal attention to human eye-tracking gaze time predicts programmer visual attention better than the compared baselines on three datasets.

Reference graph

Works this paper leans on

89 extracted references · 68 canonical work pages · cited by 1 Pith paper

  1. [1]

    Nahla J Abid, Bonita Sharif, Natalia Dragan, Hend Alrasheed, and Jonathan I Maletic. 2019. Developer reading behavior while summarizing java methods: Size and context matters. In2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 384–395

    2019

  2. [2]

    Richard Andersson, Linnea Larsson, Kenneth Holmqvist, Martin Stridh, and Marcus Nyström. 2017. One algorithm to rule them all? An evaluation and discussion of ten eye movement event-detection algorithms.Behavior research methods49 (2017), 616–637

    2017

  3. [3]

    Christoph Aschwanden and Martha Crosby. 2006. Code scanning patterns in program comprehension. InProceedings of the 39th hawaii international conference on system sciences. Citeseer

    2006

  4. [4]

    Aslina Baharum, Rozita Ismail, Suhaida Halamy, Emelia Abdul Rahim, Noorsidi Aizuddin Mat Noor, and Farhana Diana Deris. 2024. Enhancing UX Design Through Eye-Tracking and Image Processing: Practical Insights and Applications. In2024 International Conference on Platform Technology and Service (PlatCon). IEEE, 1–4

    2024

  5. [5]

    Aakash Bansal, Bonita Sharif, and Collin McMillan. 2023. Towards modeling human attention from eye movements for neutral source code summarization.Proceedings of ACM Human-Computer Interaction, ETRA Vol. 7(2023)

    2023

  6. [6]

    Aakash Bansal, Chia-Yi Su, Zachary Karas, Yifan Zhang, Yu Huang, Toby Jia-Jun Li, and Collin McMillan. 2023. Modeling Programmer Attention as Scanpath Prediction. InProceedings of The 38th IEEE/ACM International Conference on Automated Software Engineering ASE 2023 - NIER track

    2023

  7. [7]

    Roman Bednarik and Markku Tukiainen. 2006. An eye-tracking methodology for characterizing program comprehension processes. InProceedings of the 2006 symposium on Eye tracking research & applications. 125–132

    2006

  8. [8]

    Roman Bednarik and Markku Tukiainen. 2008. Temporal eye-tracking data: Evolution of debugging strategies with multiple representations. In Proceedings of the 2008 symposium on Eye tracking research & applications. 99–102

    2008

Show all 89 references
  1. [9]

    Guarnera, Bonita Sharif, and Jonathan I

    Joshua Behler, Praxis Weston, Drew T. Guarnera, Bonita Sharif, and Jonathan I. Maletic. 2023. iTrace-Toolkit: A Pipeline for Analyzing Eye- Tracking Data of Software Engineering Studies. In2023 IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedin...

    2023

  2. [10]

    Ted J Biggerstaff, Bharat G Mitbander, and Dallas Webster. 1993. The concept assignment problem in program understanding. InProceedings of the 15th international conference on Software Engineering. IEEE Computer Society Press, 482–498

    1993

  3. [11]

    Robert Bixler and Sidney D’Mello. 2016. Automatic gaze-based user-independent detection of mind wandering during computerized reading.User Modeling and User-Adapted Interaction26 (2016), 33–68. doi:10.1007/s11257-015-9167-1

    2016 doi

  4. [12]

    Yoram Braw, Motti Ratmansky, and Itay Goor-Aryeh. 2023. Integrating the Numerical Pain Rating Scale (NPRS) with an Eye Tracker: Feasibility and Initial Validation. (2023)

    2023

  5. [13]

    Adrian Carmichael, Adam Larson, Elizabeth Gire, Lester Loschky, and N Sanjay Rebello. 2010. How does visual attention differ between experts and novices on physics problems?. InAIP Conference Proceedings, Vol. 1289. American Institute of Physics, 93–96

    2010

  6. [14]

    Marisa Carrasco. 2011. Visual attention: The past 25 years.Vision research51, 13 (2011), 1484–1525

    2011

  7. [15]

    Shiwei Cheng and Ying Liu. 2012. Eye-tracking based adaptive user interface: implicit human-computer interaction for preference indication. Journal on Multimodal User Interfaces5 (2012), 77–84

    2012

  8. [16]

    Collard, Michael John Decker, and Jonathan I

    Michael L. Collard, Michael John Decker, and Jonathan I. Maletic. 2013. srcML: An Infrastructure for the Exploration, Analysis, and Manipulation of Source Code: A Tool Demonstration. In2013 IEEE International Conference on Software Maintenance. 516–519. doi:10.1109/ICSM.2013.85

    2013 doi

  9. [17]

    Bas Cornelissen, Andy Zaidman, Arie Van Deursen, Leon Moonen, and Rainer Koschke. 2009. A systematic survey of program comprehension through dynamic analysis.IEEE Transactions on Software Engineering35, 5 (2009), 684–702

    2009

  10. [18]

    Domenico Cotroneo, Roberto Pietrantuono, Stefano Russo, and Kishor Trivedi. 2016. How do bugs surface? A comprehensive study on the characteristics of software bugs manifestation.Journal of Systems and Software113 (2016), 27–43

    2016

  11. [19]

    Anthony Cox and Maryanne Fisher. 2009. Programming style: Influences, factors, and elements. In2009 second international conferences on advances in computer-human interactions. IEEE, 82–89

    2009

  12. [20]

    James Dominic, Charles Ritter, and Paige Rodeghero. 2020. Onboarding bot for newcomers to software engineering. InProceedings of the International Conference on Software and System Processes. 91–94. Manuscript submitted to ACM Human Attention During Localization of Memory Bugs...

    2020

  13. [21]

    James Dominic, Brock Tubre, Jada Houser, Charles Ritter, Deborah Kunkel, and Paige Rodeghero. 2020. Program comprehension in virtual reality. In Proceedings of the 28th International Conference on Program Comprehension. 391–395

    2020

  14. [22]

    Zakir Durumeric, Frank Li, James Kasten, Johanna Amann, Jethro Beekman, Mathias Payer, Nicolas Weaver, David Adrian, Vern Paxson, Michael Bailey, et al. 2014. The matter of heartbleed. InProceedings of the 2014 conference on internet measurement conference. 475–488

    2014

  15. [23]

    Shahram Eivazi and Roman Bednarik. 2011. Predicting problem-solving behavior and performance levels from visual attention data. InProc. workshop on eye gaze in intelligent human machine interaction at IUI. 9–16

    2011

  16. [24]

    Ralf Engbert, Antje Nuthmann, Eike M Richter, and Reinhold Kliegl. 2005. SWIFT: a dynamical model of saccade generation during reading. Psychological review112, 4 (2005), 777–813. doi:10.1037/0033-295X.112.4.777

    2005 doi

  17. [25]

    Svana Esche. 2024. Rubric for the Quality of Answers to Student Queries about Code. InProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1. 331–337

    2024

  18. [26]

    Myrthe Faber, Kristina Krasich, Robert E Bixler, James R Brockmole, and Sidney K D’Mello. 2020. The eye–mind wandering link: Identifying gaze indices of mind wandering across tasks.Journal of experimental psychology: human perception and performance46, 10 (2020), 1201–1221. do...

    2020 doi

  19. [27]

    Sue Fitzgerald, Brian Hanks, Raymond Lister, Renee McCauley, and Laurie Murphy. 2013. What are we thinking when we grade programs?. In Proceeding of the 44th ACM technical symposium on Computer science education. 471–476

    2013

  20. [28]

    Ruth C Fong, Walter J Scheirer, and David D Cox. 2018. Using human brain activity to guide machine learning.Scientific reports8, 1 (2018), 1–10

    2018

  21. [29]

    Tom Foulsham, James Farley, and Alan Kingstone. 2013. Mind wandering in sentence reading: decoupling the link between mind and eye.Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale67, 1 (2013), 51–59. doi:10.1037/a0030217

    2013 doi

  22. [30]

    David J Frank, Brent Nara, Michela Zavagnin, Dayna R Touron, and Michael J Kane. 2015. Validating older adults’ reports of less mind-wandering: An examination of eye movements and dispositional influences.Psychology and Aging30, 2 (2015), 266–278. doi:10.1037/pag0000031

    2015 doi

  23. [31]

    Thomas Fritz, Andrew Begel, Sebastian C Müller, Serap Yigit-Elliott, and Manuela Züger. 2014. Using psycho-physiological measures to assess task difficulty in software development. InProceedings of the 36th international conference on software engineering. 402–413

    2014

  24. [32]

    Lisa Grabinger, Naser Al Madi, Roman Bednarik, Teresa Busjahn, Fabian Engl, Timur Ezer, Hans Gruber, Florian Hauser, Jonathan I Maletic, Unaizah Obaidellah, et al. 2025. A Cookbook for Eye Tracking in Software Engineering. InProceedings of the 6th European Conference on Softwa...

    2025

  25. [33]

    Lisa Grabinger, Florian Hauser, Christian Wolff, and Jürgen Mottok. 2024. On eye tracking in software engineering.SN Computer Science5, 6 (2024), 729

    2024

  26. [34]

    Yann-Gaël Guéhéneuc. 2006. TAUPE: towards understanding program comprehension. InProceedings of the 2006 conference of the Center for Advanced Studies on Collaborative research. 1–es

    2006

  27. [35]

    Yi He et al. 2025. A Comparative Study of the Significance of Different Programming Languages.Academic Journal of Computing & Information Science8, 2 (2025), 45–50

    2025

  28. [36]

    Prateek Hejmady and N Hari Narayanan. 2012. Visual attention patterns during program debugging with an IDE. Inproceedings of the symposium on eye tracking research and applications. 197–200

    2012

  29. [37]

    Ting-Yun Hou, Yu-Tzu Lin, Yu-Chih Lin, Chia-Hu Chang, and Miao-Hsuan Yen. 2013. Exploring the gender effect on cognitive processes in program debugging based on eye-movement analysis. InInternational Conference on Computer Supported Education, Vol. 2. SCITEPRESS, 469–473

    2013

  30. [38]

    Derrick Hylton, Shannon Hsianghan-huang Sung, Xiaotong Ding, and Mary Johanna Van Vleet. 2023. Board 196: A Framework to Assess Debugging Skills for Computational Thinking in Science and Engineering. In2023 ASEE Annual Conference & Exposition

    2023

  31. [39]

    Huibin Jin, Zhanyao Hu, Kun Li, Mingjian Chu, Guoliang Zou, Guihua Yu, and Jianlei Zhang. 2021. Study on how expert and novice pilots can distribute their visual attention to improve flight performance.IEEE Access9 (2021), 44757–44769

    2021

  32. [40]

    An Ju, Hitesh Sajnani, Scot Kelly, and Kim Herzig. 2021. A case study of onboarding in software teams: Tasks and strategies. In2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 613–623

    2021

  33. [41]

    Zachary Karas, Aakash Bansal, Yifan Zhang, Toby Li, Collin McMillan, and Yu Huang. 2024. A Tale of Two Comprehensions? Analyzing Student Programmer Attention during Code Summarization.ACM Transactions on Software Engineering and Methodology(2024)

    2024

  34. [42]

    Alan E. Kazdin. 2016. Experimental and observational designs: An overview. InMethodological issues and strategies in clinical research(4 ed.), Alan E. Kazdin (Ed.). American Psychological Association, 155–180

    2016

  35. [43]

    ES Kiseleva, EE Gudoshnik, AV Orlov, and AI Rustemova. 2020. Study of the possibility of using pupillography for personnel selection at hiring. In Journal of physics: conference series, Vol. 1519. IOP Publishing, 012023

    2020

  36. [44]

    Kristina Krasich, Robert McManus, Stephen Hutt, Myrthe Faber, Sidney K D’Mello, and James R Brockmole. 2018. Gaze-based signatures of mind wandering during real-world scene processing.Journal of Experimental Psychology: General147, 8 (2018), 1111–1124. doi: 10.1037/ xge0000411

    2018

  37. [45]

    Xinyu Li, Wei Liu, Huitong Liu, Jing Xu, and Wenqing Cheng. 2021. Task-oriented Analysis on Debugging Process Based on Eye Movements and IDE Interactions. In2021 16th International Conference on Computer Science & Education (ICCSE). IEEE, 379–384

    2021

  38. [46]

    Angela Lopez-Cardona, Carlos Segura, Alexandros Karatzoglou, Sergi Abadal, and Ioannis Arapakis. 2024. Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models.arXiv preprint arXiv:2410.01532(2024)

    2024 arXiv

  39. [47]

    Walid Maalej, Rebecca Tiarks, Tobias Roehm, and Rainer Koschke. 2014. On the comprehension of program comprehension.ACM Transactions on Software Engineering and Methodology (TOSEM)23, 4 (2014), 1–37. Manuscript submitted to ACM 28 Emory Smith, Robert Wallace, Matthew Robison, ...

    2014

  40. [48]

    Christopher Mole. 2025. Attention. InThe Stanford Encyclopedia of Philosophy(Summer 2025 ed.), Edward N. Zalta and Uri Nodelman (Eds.). Metaphysics Research Lab, Stanford University

    2025

  41. [49]

    Jakub Štěpán Novák, Jan Masner, Petr Benda, Pavel Šimek, and Vojtěch Merunka. 2024. Eye tracking, usability, and user experience: A systematic review.International Journal of Human–Computer Interaction40, 17 (2024), 4484–4500

    2024

  42. [50]

    Unaizah Obaidellah, Mohammed Al Haek, and Peter C.-H. Cheng. 2018. A Survey on the Usage of Eye-Tracking in Computer Programming.ACM Comput. Surv.51, 1 (2018), 5:1–5:58. doi:10.1145/3145904

    2018 doi

  43. [51]

    Yvette Oortwijn, Thijs Ossenkoppele, and Arianna Betti. 2021. Interrater disagreement resolution: A systematic procedure to reach consensus in annotation tasks. InProceedings of the Workshop on Human Evaluation of NLP Systems (HumEval). 131–141

    2021

  44. [52]

    H. Orman. 2003. The Morris worm: a fifteen-year perspective.IEEE Security & Privacy1, 5 (2003), 35–43. doi: 10.1109/MSECP.2003. 1236233

    2003 doi

  45. [53]

    Norman Peitek, Annabelle Bergum, Maurice Rekrut, Jonas Mucke, Matthias Nadig, Chris Parnin, Janet Siegmund, and Sven Apel. 2022. Correlates of programmer efficacy and their link to experience: A combined EEG and eye-tracking study. InProceedings of the 30th ACM Joint European ...

    2022

  46. [54]

    Mahshid Pourhosein and Mehdi Sabokro. 2025. Unveiling the gaze: deciphering key factors in selecting knowledge workers through eye-tracking analysis.European Journal of Management Studies(2025)

    2025

  47. [55]

    Erik D Reichle, Alexander Pollatsek, and Keith Rayner. 2012. Using EZ Reader to simulate eye movements in nonreading tasks: A unified framework for understanding the eye–mind link.Psychological Review119, 1 (2012), 155–185. doi:10.1037/a0026473

    2012 doi

  48. [56]

    Erik D Reichle, Keith Rayner, and Alexander Pollatsek. 2003. The EZ Reader model of eye-movement control in reading: Comparisons to other models.Behavioral and brain sciences26, 4 (2003), 445–476. doi:10.1017/S0140525X03000104

    2003 doi

  49. [57]

    Erik D Reichle, Andrew E Reineberg, and Jonathan W Schooler. 2010. Eye movements during mindless reading.Psychological science21, 9 (2010), 1300–1310. doi:10.1177/0956797610378686

    2010 doi

  50. [58]

    Remain Software. 2025. Remain AI Chat (ChatGPT). https://marketplace.eclipse.org/content/remain-ai- chat-chatgpt#detailsAccessed: 2025-02-03

    2025

  51. [59]

    2011.Reinforcement learning for adaptive dialogue systems: a data-driven methodology for dialogue management and natural language generation

    Verena Rieser and Oliver Lemon. 2011.Reinforcement learning for adaptive dialogue systems: a data-driven methodology for dialogue management and natural language generation. Springer Science & Business Media

    2011

  52. [60]

    Matthew K Robison, Ashley L Miller, and Nash Unsworth. 2019. Examining the effects of probe frequency, response options, and framing within the thought-probe method.Behavior research methods51 (2019), 398–408. doi:10.3758/s13428-019-01212-6

    2019 doi

  53. [61]

    Paige Rodeghero, Cheng Liu, Paul W McBurney, and Collin McMillan. 2015. An eye-tracking study of java programmers and application to source code summarization.IEEE Transactions on Software Engineering41, 11 (2015), 1038–1054

    2015

  54. [62]

    Paige Rodeghero and Collin McMillan. 2015. An empirical study on the patterns of eye movement during summarization tasks. In2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). 1–10

    2015

  55. [63]

    Paige Rodeghero, Collin McMillan, Paul W McBurney, Nigel Bosch, and Sidney D’Mello. 2014. Improving automated source code summarization via an eye-tracking study of programmers. InProceedings of the 36th international conference on Software engineering. ACM, 390–401

    2014

  56. [64]

    Ripon K Saha, Matthew Lease, Sarfraz Khurshid, and Dewayne E Perry. 2013. Improving bug localization using structured information retrieval. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 345–355

    2013

  57. [65]

    Ivonne Schröter, Jacob Krüger, Janet Siegmund, and Thomas Leich. 2017. Comprehending studies on program comprehension. In2017 IEEE/ACM 25th International Conference on Program Comprehension (ICPC). IEEE, 308–311

    2017

  58. [66]

    Zohreh Sharafi, Ian Bertram, Michael Flanagan, and Westley Weimer. 2020. Eyes on code: A study on developers’ code navigation strategies.IEEE Transactions on Software Engineering48, 5 (2020), 1692–1704

    2020

  59. [67]

    Zohreh Sharafi, Bonita Sharif, Yann-Gaël Guéhéneuc, Andrew Begel, Roman Bednarik, and Martha E. Crosby. 2020. A practical guide on conducting eye tracking studies in software engineering.Empir. Softw. Eng.25, 5 (2020), 3128–3174. doi:10.1007/s10664-020-09829-4

    2020 doi

  60. [68]

    Zohreh Sharafi, Zéphyrin Soh, and Yann-Gaël Guéhéneuc. 2015. A systematic literature review on the usage of eye-tracking in software engineering. Information and Software Technology67 (2015), 79–107

    2015

  61. [69]

    Bonita Sharif, Michael Falcone, and Jonathan I Maletic. 2012. An eye-tracking study on the role of scan time in finding source code defects. In Proceedings of the Symposium on Eye Tracking Research and Applications. 381–384

    2012

  62. [70]

    Bonita Sharif and Jonathan I. Maletic. 2010. The Effects of Layout on Detecting the Role of Design Patterns. InProceedings of the 2010 23rd IEEE Conference on Software Engineering Education and Training (CSEET ’10). IEEE Computer Society, Washington, DC, USA, 41–48. http: //dx...

    2010 doi

  63. [71]

    Janet Siegmund, Norbert Siegmund, and Sven Apel. 2015. Views on internal and external validity in empirical software engineering. In2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, Vol. 1. IEEE, 9–19

    2015

  64. [72]

    Jonathan Smallwood and Jonathan W Schooler. 2015. The science of mind wandering: Empirically navigating the stream of consciousness.Annual review of psychology66, 1 (2015), 487–518. doi:10.1146/annurev-psych-010814-015331

    2015 doi

  65. [73]

    Sean Stapleton, Yashmeet Gambhir, Alexander LeClair, Zachary Eberhart, Westley Weimer, Kevin Leach, and Yu Huang. 2020. A Human Study of Comprehension and Code Summarization. InProceedings of the 28th International Conference on Program Comprehension. 2–13

    2020

  66. [74]

    Ben Steichen, Cristina Conati, and Giuseppe Carenini. 2014. Inferring visualization task properties, user performance, and user cognitive abilities from eye gaze data.ACM Transactions on Interactive Intelligent Systems (TiiS)4, 2 (2014), 1–29. Manuscript submitted to ACM Human...

    2014

  67. [75]

    Klaas-Jan Stol and Brian Fitzgerald. 2020. Guidelines for conducting software engineering research. InContemporary Empirical Methods in Software Engineering. Springer, 27–62

    2020

  68. [76]

    The Chromium Project. [n. d.]. Memory Safety. https://www.chromium.org/Home/chromium-security/memory- safety/. Accessed: 2025-05-15

    2025

  69. [77]

    Ferdian Thung, Tien-Duy B Le, Pavneet Singh Kochhar, and David Lo. 2014. Buglocalizer: Integrated tool support for bug localization. InProceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 767–770

    2014

  70. [78]

    Rachel Turner, Michael Falcone, Bonita Sharif, and Alina Lazar. 2014. An eye-tracking study assessing the comprehension of C++ and Python source code. InProceedings of the Symposium on Eye Tracking Research and Applications. 231–234

    2014

  71. [79]

    Victor Van der Veen, Nitish Dutt-Sharma, Lorenzo Cavallaro, and Herbert Bos. 2012. Memory errors: The past, the present, and the future. InResearch in Attacks, Intrusions, and Defenses: 15th International Symposium, RAID 2012, Amsterdam, The Netherlands, September 12-14, 2012....

    2012

  72. [80]

    van Oorschot

    Paul C. van Oorschot. 2023. Memory Errors and Memory Safety: C as a Case Study.IEEE Security & Privacy21, 2 (2023), 70–76. doi: 10.1109/ MSEC.2023.3236542

    2023

  73. [81]

    Robert Wallace, Aakash Bansal, Zachary Karas, Ningzhi Tang, Yu Huang, Toby Jia-Jun Li, and Collin McMillan. 2025. Programmer Visual Attention During Context-Aware Code Summarization.IEEE Transactions on Software Engineering(2025), 1–13. doi: 10.1109/TSE.2025.3554990

    2025

  74. [82]

    Yana Weinstein. 2018. Mind-wandering, how do I measure thee with probes? Let me count the ways.Behavior research methods50 (2018), 642–661. doi:10.3758/s13428-017-0891-9

    2018 doi

  75. [83]

    W Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization.IEEE Transactions on Software Engineering42, 8 (2016), 707–740

    2016

  76. [84]

    Andrew Wood, Paige Rodeghero, Ameer Armaly, and Collin McMillan. 2018. Detecting speech act types in developer question/answer conversations during bug repair. InProceedings of the 2018 26th ACM joint meeting on european software engineering conference and symposium on the fou...

    2018

  77. [85]

    Jiayi Zeng, Wei Liu, Ying Zhou, Xinyu Li, and Wenqing Cheng. 2022. Attention Cloud Map: Visualizing Attention Migration of Students’ Debugging Process in Semantic Code Space. InInternational Conference on Computer Science and Education. Springer, 481–491

    2022

  78. [86]

    Han Zhang, Kevin F Miller, Xin Sun, and Kai S Cortina. 2020. Wandering eyes: Eye movements during mind wandering in video lectures.Applied Cognitive Psychology34, 2 (2020), 449–464. doi:10.1002/acp.3632

    2020 doi

  79. [87]

    Yifan Zhang, Chen Huang, Zachary Karas, Dung Thuy Nguyen, Kevin Leach, and Yu Huang. 2025. Enhancing Code LLM Training with Programmer Attention.arXiv preprint arXiv:2503.14936(2025)

    2025 arXiv

  80. [88]

    Yifan Zhang, Jiliang Li, Zachary Karas, Aakash Bansal, Toby Jia-Jun Li, Collin McMillan, Kevin Leach, and Yu Huang. 2024. EyeTrans: Merging Human and Machine Attention for Neural Code Summarization. InProceedings of The ACM Joint European Software Engineering Conference and Sy...

    2024

  81. [89]

    Guarnera, Cole S

    Vlas Zyrianov, Drew T. Guarnera, Cole S. Peterson, Bonita Sharif, and Jonathan I. Maletic. 2020. Automated Recording and Semantics-Aware Replaying of High-Speed Eye Tracking and Interaction Data to Support Cognitive Studies of Software Engineering Tasks. In2020 IEEE Internatio...

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.