KOALA: a Configurable Tool for Collecting IDE Data When Solving Programming Tasks

Anastasiia Birillo; Daniil Karol; Elizaveta Artser; Hieke Keuning; Ilya Vlasov; Yaroslav Golubev

REVIEW 4 major objections 5 minor 56 references

KOALA: a Configurable Tool for Collecting IDE Data When Solving Programming Tasks

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read KOALA is a configurable plugin for JetBrains IDEs that records students' code, actions, and hotkeys during programming tasks, with output convertible to the standard ProgSnap2 format.

desk verdict A useful, well-described IDE data collection tool whose novelty is real but whose event-capture accuracy is asserted, not yet demonstrated. read the letter →

arxiv 2506.21266 v1 pith:4GK2LSHU submitted 2025-06-26 cs.SE cs.CY

Daniil Karol , Elizaveta Artser , Ilya Vlasov , Yaroslav Golubev , Hieke Keuning , Anastasiia Birillo This is my paper · ORCID

classification cs.SEcs.CY

keywords datacollectionIDEinteractioncodesnapshotsJetBrainsIDEsProgSnap2computingeducationactivitytrackingin-IDElearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

KOALA is an open-source plugin-and-server system for JetBrains IDEs that records what students do while solving programming tasks: code snapshots taken at a configurable frequency (down to every keystroke), IDE actions such as running and debugging, hotkey usage, file-focus switches, tool-window changes, and survey answers. The paper's claim is that this tool overcomes the two main limitations of earlier data-collection tools: it is configurable through YAML files without touching source code, and it captures event types (hotkeys, focus changes) that previous JetBrains tools did not record. The collected data is stored on a bundled server and can be converted to the standard ProgSnap2 format, so researchers can reuse existing analysis tools. To show it works, the paper reports a case study with 28 students solving tasks in two in-IDE courses, collecting over 127,000 code snapshots and 585,000 activity events. A sympathetic reader would take the central contribution to be a ready-to-use, standardized measurement instrument for computing-education research.

What carries the argument

Central to the design is the YAML configuration layer, which controls the plugin's behavior through several files—scenario, task content, IDE settings and inspections, survey, activity data, code tracking, and research metadata. A plugin built on the IntelliJ Platform Plugin SDK reads these files and subscribes to IDE events, so a single codebase can capture data across all JetBrains IDEs without recompilation. The same plugin-server architecture used by earlier JetBrains trackers is reused, with the addition of a converter that exports the raw CSV records into the ProgSnap2 format, making the tool's output interoperable with existing analysis tools.

What would settle it

Record a session in a JetBrains IDE on video or with an independent keylogger while KOALA runs at its finest granularity, then compare the logs event by event; any unlogged keystroke, hotkey, or focus switch, or any phantom event, would show that the capture is not complete.

Watch

Extended reading notes

Core claim

KOALA is a plugin for all JetBrains IDEs, built on the IntelliJ Platform SDK, that lets a researcher or educator define a whole study through YAML configuration files: the sequence of tasks and surveys, which files the student will work in, which IDE settings and inspections to enable or disable (e.g., code completion), how often code snapshots should be taken, and which activities to track. During a session, the plugin records the student's code at the configured granularity—in the case study, the full file after every keystroke—together with all IDE actions, run/debug invocations, keyboard shortcuts, file-editor focus changes, and tool-window openings, writing them to CSV files. The data is uploaded to a Ktor-based server (provided as a Docker image) when the student submits or pauses, and can then be inspected on an interactive dashboard and converted into ProgSnap2, the standard format for programming-process data. The paper demonstrates the pipeline on 28 students from two courses using the JetBrains Academy in-IDE learning plugin, obtaining tens of thousands of snapshots and hundreds of thousands of events, and uses the newly available hotkey data to show which refactoring shortcuts students favored. The tool's stated purpose is to make fine-grained IDE interaction data easy to collect, standardize, and share for computing-education research.

Load-bearing premise

The plugin's event capture is complete and does not disturb students, meaning the IDE delivers every relevant keystroke, hotkey, action, and focus change to the plugin, and the plugin's presence does not alter how students solve tasks.

Editorial extensions

If this is right

Researchers can run the same configuration across IntelliJ IDEA, PyCharm, and CLion, so a single study design can capture Java, Python, and C++ problem solving without rewriting the tool.
Studies can now measure IDE feature adoption directly—hotkeys, run/debug invocations, focus switches—rather than inferring it from code snapshots alone.
Because data converts to ProgSnap2, datasets collected with KOALA can be combined with existing shared datasets and analyzed with tools already built for that format.
The configurable snapshot granularity lets a study trade data volume against detail, from full-file-per-keystroke histories to coarse summaries such as function names on save.
The case study's hotkey ranking in a refactoring course (Reformat, Inline, Move, Introduce constant) gives a concrete example of insights that were not obtainable from earlier JetBrains trackers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If event capture is validated against ground truth, such as screen recordings or an independent keylogger, KOALA could become a shared measurement benchmark that lets different research groups compare IDE interaction patterns quantitatively.
The YAML-driven design points toward a natural extension in which IDE settings are toggled per student cohort, enabling causal experiments on how features like code completion affect learning outcomes.
Because only IntelliJ-platform events are logged, anything happening outside the IDE—browser searches, notes, OS-level distractions—remains invisible, so a complete picture would require supplementing KOALA with other sensors.
The dataset's long-term value depends on the promised public release of both the anonymized data and the plugin source code, since reproducibility of the instrumentation is what would let other groups build on the measurements.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Karol et al. present KOALA, a JetBrains IDE plugin for collecting code snapshots and activity data from students. The tool fills a genuine gap: TaskTracker is unmaintained and requires extra plugins, Test My Code is effectively dead on the IDE side, and none of the existing tools capture hotkey usage or focus switches as first-class data types. KOALA's YAML configuration is a real step forward—researchers can control task order, IDE settings, snapshot granularity, surveys, and even third-party log files without touching plugin code. The architecture is straightforward and the ProgSnap2 converter plus dashboard make the tool practical. The paper is clearly written, the comparison table is informative, and the case study gives a concrete sense of the data volume and format.

The soft spots are real. The central claim that the plugin reliably captures hotkeys and focus events is not validated. Section 4.1 says what is collected, but there is no check against ground truth, no precision/recall for action or focus events, no measurement of plugin overhead, and no discussion of known failure modes (custom keymaps, programmatic action triggers, focus changes via tool windows). The 21k hotkey usages in the case study are just counts. For a tool whose selling point is fine-grained IDE interaction data, that is a meaningful gap, though not a fatal one—it can be fixed with a small validation study.

I also note that the manuscript says the plugin will be fully open-sourced "upon acceptance" and the dataset shared "upon acceptance", despite a GitHub link in the introduction. That mismatch should be cleaned up; ideally the code and data are available at submission so reviewers can audit the capture logic. The claim that the tool is "convenient" is not empirically supported—no usability data, just configuration examples—but that is a minor wording issue.

The circularity burden is low: the tool is not being fitted to explain its own data, and the previous use in Birillo et al. is an external application, not a validation. The case study is limited but appropriately described as a showcase. The dataset, once released, will be a contribution on its own.

Who is this for? Computing education researchers who need a configurable JetBrains-based data collection pipeline, and tool builders looking for a reference architecture. A serious referee should engage with it, mainly to push on the validation and availability points. I would support sending it to review, with the expectation of a major revision that adds event-capture validation and makes code/data available.

Referee Report

4 major / 5 minor

Summary. The paper presents KOALA, a tool for collecting fine-grained IDE interaction data from students solving programming tasks in JetBrains IDEs. KOALA consists of an IDE plugin, a data-collection server, a visualization dashboard, and a converter to the ProgSnap2 format. The plugin is configured through YAML files that control task content, IDE settings, surveys, code-snapshot granularity, and activity-event frequency. The authors report a case study with 28 students completing tasks in two JetBrains Academy in-IDE courses, yielding over 585 thousand activity events and 127 thousand code snapshots, and they illustrate the data with a ranking of the most-used hotkeys in a refactoring course.

Significance. If the tool performs as described, KOALA would be a valuable contribution to computing-education research by providing a configurable, standardized way to collect code snapshots and IDE activity data across JetBrains IDEs, including data types such as hotkey usage and file-focus changes that are not commonly available in prior tools. The paper's strengths include a clear architecture, a detailed configuration taxonomy, built-in privacy controls, a dashboard, and ProgSnap2 compatibility, as well as a non-trivial dataset from 28 students. The prior use of KOALA in a published study is also a positive indicator. However, the central claim of accurate and non-intrusive event capture is not yet validated, and the source code and dataset are only promised upon acceptance.

major comments (4)

[Section 4.1, 'Collected data'] The central claim that KOALA completely and accurately records IDE actions, hotkeys, and focus changes is not supported by any validation. Hotkey detection appears to depend on inferring the trigger from an action event's InputEvent, and focus changes are reconstructed from editor/file listeners; both are vulnerable to systematic gaps from non-default keymaps, programmatic action invocations, popups, split editors, and tool-window focus. No precision/recall measurement, ground-truth comparison, or event-loss analysis is reported for any of the collected event types. Please add an accuracy evaluation, for example by replaying scripted action sequences in a controlled IDE session and measuring detection rates, and document the assumptions and failure modes of the event handlers.
[Section 5, 'Study setup' and 'Collected data'] The case study reports collecting 127 thousand code snapshots at 'the most granular setting possible' (after each keystroke) and 585 thousand activity events, but it provides no measurement of the plugin's performance overhead or its effect on student behavior. Since the paper motivates the tool partly by the need for non-intrusive data collection, the absence of CPU, memory, latency, or user-experience measurements leaves the non-intrusiveness claim untested. Please include at least basic overhead measurements or a qualitative comparison of IDE responsiveness with and without the plugin.
[Sections 4.1 and 5, 'Output format'] The manuscript is internally inconsistent about artifact availability: footnote 1 lists a GitHub repository for KOALA, but Section 4.1 states that 'upon acceptance, the plugin will be fully open-sourced,' and Section 5 states that the full dataset will be shared 'upon acceptance.' For a tool paper, the event-capture logic and the raw data are necessary to independently assess the main claims. Please make the repository and dataset available at review time, or provide a sufficiently detailed technical appendix covering the event-handler implementations, the CSV schema, and the anonymization procedure.
[Section 5, 'Collected data' and 'Possible applications'] The illustrative analysis in Figure 4 shows counts of hotkey usage, but it does not demonstrate that the newly collected data types (hotkeys and focus switches) are accurate or that they enable analyses beyond what existing tools' logs already provide. The claim that these data are 'not collected in prior works' is plausible, but it would be substantially strengthened by a concrete comparison with data obtainable from a prior tool such as TaskTracker, or by showing that the focus/hotkey data yield a non-trivial behavioral insight rather than a simple ranking.

minor comments (5)

[Section 4.1, 'Collected data'] The CSV output schema is not described in the text; readers are left to infer it from Figure 2. A table or appendix listing each data type, its fields, and the triggering condition would make the tool easier to adopt and verify.
[Figure 4] The hotkey labels in Figure 4 appear garbled in the provided version, making it difficult to read which shortcuts are being counted; please ensure the figure renders clearly in the final PDF.
[Section 2.1, 'Data Collection Tools'] The description of Test My Code states that the server side is 'actively supported' while the IDE plugins are no longer updated; please clarify whether this is a fully maintained option or a legacy system, since Table 1 lists it as a current comparison point.
[Section 4.1, 'Privacy policy'] The privacy section says students explicitly consent to collection of personal data such as name and email, while Section 5 says the distributed data is 'without students' personal information'; please clarify whether personal information is never sent to the server or is removed during the export step.
[Section 3, 'KOALA: Usage Pipeline'] The step numbering in Figure 1 appears to be inconsistent with the accompanying textual list (steps are labeled 1, 2, 3, 4, 6, 5); please check the figure labels and the narrative order.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: KOALA's functionality is described from its architecture and demonstrated with a case study, not derived from its own outputs.

full rationale

The paper makes no derived quantitative prediction and contains no fitted model or equations whose output reduces to an input. The central claim is that KOALA is a configurable tool for collecting code snapshots and IDE feature usage; Section 4.1 supports this by describing the IntelliJ Platform SDK implementation, configuration files, and collected data categories, and Section 5 demonstrates the capability by reporting counts from a 28-student case study. The case study is a demonstration of the tool in use, not a prediction statistically forced by a fit, so it is not circular. The only self-referential element is the sentence 'KOALA has also already shown its practical usefulness in one of our previous studies [24]' in Section 1, where [24] has overlapping authors; however, this is a supporting anecdote rather than a load-bearing justification, because the tool's functionality is described from its design and independently illustrated by the case-study dataset. The manuscript's explicit statement that the plugin will be fully open-sourced only 'upon acceptance' affects external auditability and verification of event-capture completeness, which is a correctness or validation concern, not a circularity concern. No circular step can be quoted and exhibited, so the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The central claim is about a tool's capability; it rests on the reliability of the underlying IDE event API and the representativeness of the case study. No free parameters or invented entities are involved.

assumptions (2)

domain assumption The IntelliJ Platform SDK provides reliable event hooks for all tracked IDE actions and focus changes.
Invoked implicitly in Section 4.1 when stating the plugin handles IDE events and collects actions; no validation is reported.
domain assumption The case-study participants and courses are representative enough to illustrate the tool's usefulness.
Section 5 uses 28 students from two universities; no claim of generalizability is made, so this underpins only the demonstration, not a universal claim.

how reviews work

0 comments

Cite this review

Pith. "Pith review of KOALA: a Configurable Tool for Collecting IDE Data When Solving Programming Tasks." pith.science (2026). https://pith.science/paper/4GK2LSHU

@misc{pith2026250621266,
  author       = {Pith},
  title        = {Pith review of: KOALA: a Configurable Tool for Collecting IDE Data When Solving Programming Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4GK2LSHU}},
  note         = {Machine review of arXiv:2506.21266}
}

read the original abstract

Collecting data of students solving programming tasks is incredibly valuable for researchers and educators. It allows verifying that the students correctly apply the features and concepts they are taught, or finding students' misconceptions. However, existing data collection tools have limitations, e.g., no control over the granularity of the collected code, not collecting the specific events of the programming environment used, and overall being hard to configure. To overcome these limitations, we propose KOALA, a convenient and highly configurable tool for collecting code snapshots and feature usage from students solving programming tasks in JetBrains IDEs. The plugin can be installed in IDEs and configured to provide the students with the necessary tasks, enable or disable certain IDE features like code completion, and run surveys. During problem solving, the plugin collects code snapshots at the configured granularity, all IDE actions like running and debugging, as well as some data not collected in prior works, like employed hotkeys and switching focus between files. The collected data is sent to the server that comes with the tool, where it is stored and can be converted to the standardized ProgSnap2 format. To showcase the tool, we collected data from 28 students solving tasks in two courses within the IDE, highlighting some insights from this data.

Figures

Figures reproduced from arXiv: 2506.21266 by the authors.

**Figure 2.** Examples of data collected using KOALA: (1) activities performed in the IDE, such as running, debugging, etc., (2) current student code, (3) opening, closing, and refocusing of files, (4) opened tool windows, and (5) survey data. On the right, you can see how different data is written to different CSV files, you can find the full dataset in supplementary materials [15]. and refocusing of files; 4 opened tool windows… view at source ↗

**Figure 3.** Example of a Scenario Config. disabling automatic code completion, changing the color theme, etc. This config also controls which code quality inspections should be enabled. Survey Config describes all the surveys that are used in a scenario configuration. Each of these surveys contains a series of different questions, which can be multiple-choice, single-choice, open-ended, etc. A question can be marked as required… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Top-15 most used hotkeys when solving tasks in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 56 canonical work pages

[1]

A++ Courses

2025. A++ Courses. Retrieved June 26, 2025 from https://plugins.jetbrains.com/ plugin/13634-a-courses

work page 2025
[2]

Activity Tracker plugin

2025. Activity Tracker plugin . Retrieved June 26, 2025 from https://plugins. jetbrains.com/plugin/8126-activity-tracker

work page 2025
[3]

Apache NetBeans IDE

2025. Apache NetBeans IDE . Retrieved June 26, 2025 from https://netbeans. apache.org/front/main/index.html

work page 2025
[4]

2025. CLion. Retrieved June 26, 2025 from https://www.jetbrains.com/clion/

work page 2025
[5]

Dev Event Tracker GitHub

2025. Dev Event Tracker GitHub . Retrieved June 26, 2025 from https://github. com/web-cat/web-cat-subsystem-DevEventTracker

work page 2025
[6]

IntelliJ IDEA

2025. IntelliJ IDEA. Retrieved June 26, 2025 from https://www.jetbrains.com/idea/

work page 2025
[7]

Introduction to IDE Code Refactoring in Kotlin course

2025. Introduction to IDE Code Refactoring in Kotlin course . Retrieved June 26 2025 from https://plugins.jetbrains.com/plugin/23048-introduction-to-ide-code- refactoring-in-kotlin

work page 2025
[8]

2025. JavaFX. Retrieved June 26, 2025 from https://openjfx.io/

work page 2025

Show all 56 references

[9]

JavaFX plugin

2025. JavaFX plugin. Retrieved June 26, 2025 from https://plugins.jetbrains.com/ plugin/20225-javafx

2025
[10]

2025. JCEF. Retrieved June 26, 2025 from https://github.com/ chromiumembedded/java-cef

2025
[11]

JetBrains

2025. JetBrains. Retrieved June 26, 2025 from https://www.jetbrains.com/

2025
[12]

JetBrains Academy Marketplace

2025. JetBrains Academy Marketplace . Retrieved June 26, 2025 from https: //plugins.jetbrains.com/education

2025
[13]

JetBrains Academy Plugin

2025. JetBrains Academy Plugin. Retrieved June 26, 2025 from https://plugins. jetbrains.com/plugin/10081-jetbrains-academy

2025
[14]

JetBrains Free Educational Licenses

2025. JetBrains Free Educational Licenses . Retrieved June 26, 2025 from https: //www.jetbrains.com/community/education/#students

2025
[15]

KOALA Dataset

2025. KOALA Dataset. Retrieved June 26, 2025 from https://zenodo.org/records/ 15553341

2025
[16]

Kotlin Onboarding: Introduction course

2025. Kotlin Onboarding: Introduction course . Retrieved June 26, 2025 from https://plugins.jetbrains.com/plugin/21067-kotlin-onboarding-introduction

2025
[17]

Ktor Framework

2025. Ktor Framework. Retrieved June 26, 2025 from https://ktor.io

2025
[18]

Orion - Artemis Programming Exercise Integration

2025. Orion - Artemis Programming Exercise Integration . Retrieved June 26, 2025 from https://plugins.jetbrains.com/plugin/13217-orion--artemis-programming- exercise-integration

2025
[19]

2025. PyCharm. Retrieved June 26, 2025 from https://www.jetbrains.com/ pycharm/

2025
[20]

Task Tracker GitHub

2025. Task Tracker GitHub. Retrieved June 26, 2025 from https://github.com/ JetBrains-Research/task-tracker-plugin

2025
[21]

Test My Code JetBrains IDEs Plugin GitHub

2025. Test My Code JetBrains IDEs Plugin GitHub . Retrieved June 26, 2025 from https://github.com/testmycode/tmc-intellij

2025
[22]

Test My Code NetBeans Plugin GitHub

2025. Test My Code NetBeans Plugin GitHub . Retrieved June 26, 2025 from https://github.com/testmycode/tmc-netbeans

2025
[23]

Test My Code Server GitHub

2025. Test My Code Server GitHub . Retrieved June 26, 2025 from https://github. com/testmycode/tmc-server?tab=readme-ov-file

2025
[24]

Anastasiia Birillo, Elizaveta Artser, Anna Potriasaeva, Ilya Vlasov, Katsiaryna Dzialets, Yaroslav Golubev, Igor Gerasimov, Hieke Keuning, and Timofey Bryksin
[25]

Anastasiia Birillo, Mariia Tigina, Zarina Kurbatova, Anna Potriasaeva, Ilya Vlasov, Valerii Ovchinnikov, and Igor Gerasimov. 2024. Bridging Education and Develop- ment: IDEs as Interactive Learning Platforms. In Proceedings of the 1st ACM/IEEE Workshop on Integrated Developmen...

2024
[26]

Neil Christopher Charles Brown. 2014. Introduction to analysing the BlueJ blackbox data. In Proceedings of the 45th ACM technical symposium on Computer science education. 748–748

2014
[27]

Yue Cao, Liang Wang, Zhiwen Zheng, and Xianping Tao. 2020. A tool for non- intrusive and privacy-preserving developers’ programming activity data collec- tion. In 2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, 953–962

2020
[28]

Robert Eckstein, Marc Loy, and Dave Wood. 1998. Java swing . O’Reilly & Associates, Inc

1998
[29]

John Edwards, Kaden Hart, Raj Shrestha, et al. 2023. Review of csedm data and introduction of two public cs1 keystroke datasets. Journal of Educational Data Mining 15, 1 (2023), 1–31

2023
[30]

Stephen H Edwards and Krishnan Panamalai Murali. 2017. CodeWorkout: short programming exercises with built-in data collection. In Proceedings of the 2017 ACM conference on innovation and technology in computer science education . 188– 193

2017
[31]

Markus Fuchs, Markus Heckner, Felix Raab, and Christian Wolff. 2014. Monitoring students’ mobile app coding behavior data analysis based on IDE and browser interaction logs. 2014 IEEE Global Engineering Education Conference (EDUCON) (2014), 892–899. https://api.semanticscholar...

2014
[32]

Petri Ihantola, Arto Vihavainen, Alireza Ahadi, Matthew Butler, Jürgen Börstler, Stephen H Edwards, Essi Isohanni, Ari Korhonen, Andrew Petersen, Kelly Rivers, et al. 2015. Educational data mining and learning analytics in programming: Literature review and case studies. Proce...

2015
[33]

Vseslav Kasatskii, Agnia Sergeyuk, Anastasiia Serova, Sergey Titov, and Timofey Bryksin. 2023. The Effect of Perceptual Load on Performance Within IDE in People with ADHD Symptoms. In International Conference on Human-Computer Interaction. Springer, 122–141

2023
[34]

Ayaan M Kazerouni, Stephen H Edwards, T Simin Hall, and Clifford A Shaffer
[35]

Natalie Kiesler, John Impagliazzo, Katarzyna Biernacka, Amanpreet Kapoor, Zain Kazmi, Sujeeth Goud Ramagoni, Aamod Sane, Keith Tran, Shubbhi Taneja, and Zihan Wu. 2023. Where’s the Data? Exploring Datasets in Computing Education. In Proceedings of the ACM Conference on Global ...

2023
[36]

Michael Kölling, Bruce Quig, Andrew Patterson, and John Rosenberg. 2003. The BlueJ system and its pedagogy. Computer Science Education 13, 4 (2003), 249–268

2003
[37]

Zarina Kurbatova, Yaroslav Golubev, Vladimir Kovalenko, and Timofey Bryksin
[38]

Antti Leinonen, Henrik Nygren, Nea Pirttinen, Arto Hellas, and Juho Leinonen
[39]

Juho Leinonen. 2022. Open ide action log dataset from a cs1 mooc. In Proceed- ings of the 6th Educational Data Mining in Computer Science Education (CSEDM) Workshop. Zenodo, Virtual

2022
[40]

Let Them Try to Figure It Out First

Dominic Lohr, Natalie Kiesler, Hieke Keuning, and Johan Jeuring. 2024. "Let Them Try to Figure It Out First" - Reasons Why Experts (Do Not) Provide Feedback to Novice Programmers. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1. 38–44

2024
[41]

Elena Lyulina, Anastasiia Birillo, Vladimir Kovalenko, and Timofey Bryksin
[42]

Roberto Minelli, Andrea Mocci, Romain Robbes, and Michele Lanza. 2016. Tam- ing the ide with fine-grained interaction data. In 2016 IEEE 24th International Conference on Program Comprehension (ICPC) . IEEE, 1–10

2016
[43]

Eduardo Oliveira, Hieke Keuning, and Johan Jeuring. 2023. Student code refac- toring misconceptions. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 . 19–25

2023
[44]

José Carlos Paiva, José Paulo Leal, and Álvaro Figueira. 2022. Automated assess- ment in computer science education: A state-of-the-art review.ACM Transactions on Computing Education (TOCE) 22, 3 (2022), 1–40

2022
[45]

Andrei Papancea, Jaime Spacco, and David Hovemeyer. 2013. An open platform for managing short programming exercises. In Proceedings of the ninth annual international ACM conference on International computing education research . 47– 52

2013
[46]

In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education

TaskTracker-tool: A toolkit for tracking of code snapshots and activity data during solution of programming tasks. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education . 495–501
[47]

Thomas W Price, David Hovemeyer, Kelly Rivers, Ge Gao, Austin Cory Bart, Ayaan M Kazerouni, Brett A Becker, Andrew Petersen, Luke Gusukuma, Stephen H Edwards, et al . 2020. Progsnap2: A flexible format for program- ming process data. In Proceedings of the 2020 ACM Conference o...

2020
[48]

Yizhou Qian and James Lehman. 2017. Students’ misconceptions and other difficulties in introductory programming: A literature review. ACM Transactions on Computing Education (TOCE) 18, 1 (2017), 1–24

2017
[49]

Lianne Roest, Hieke Keuning, and Johan Jeuring. 2024. Next-Step Hint Generation for Introductory Programming Using Large Language Models. In Proceedings of the 26th Australasian Computing Education Conference . 144–153

2024
[50]

Vivek Sinha, Frederic Doucet, Chuck Siska, Rajesh Gupta, Stan Liao, and Abhijit Ghosh. 2000. YAML: a tool for hardware design visualization and capture. In Proceedings 13th International Symposium on System Synthesis . IEEE, 9–14

2000
[51]

Martin Pärtel, Matti Luukkainen, Arto Vihavainen, and Thomas Vikberg. 2013. Test my code. International Journal of Technology Enhanced Learning 2 5, 3-4 (2013), 271–283

2013
[56]

Benjamin Xie, Jared Ordona Lim, Paul KD Pham, Min Li, and Amy J Ko. 2023. Developing novice programmers’ self-regulation skills with code replays. In Proceedings of the 2023 ACM Conference on International Computing Education Research-Volume 1. 298–313

2023
[2017]

In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education

DevEventTracker: Tracking development events to assess incremental development and procrastination. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education . 104–109

2017
[2019]

In Proceedings of the 50th ACM Technical Symposium on Computer Science Education

Exploring the applicability of simple syntax writing practice for learning programming. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. 84–90
[2021]

In 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW)

The intellij platform: a framework for building plugins and mining software data. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW). IEEE, 14–17

2021
[2024]

In Proceedings of the 24th Koli Calling International Conference on Computing Education Research

One Step at a Time: Combining LLMs and Static Analysis to Generate Next-Step Hints for Programming Tasks. In Proceedings of the 24th Koli Calling International Conference on Computing Education Research . 1–12

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

A++ Courses

2025. A++ Courses. Retrieved June 26, 2025 from https://plugins.jetbrains.com/ plugin/13634-a-courses

work page 2025

[2] [2]

Activity Tracker plugin

2025. Activity Tracker plugin . Retrieved June 26, 2025 from https://plugins. jetbrains.com/plugin/8126-activity-tracker

work page 2025

[3] [3]

Apache NetBeans IDE

2025. Apache NetBeans IDE . Retrieved June 26, 2025 from https://netbeans. apache.org/front/main/index.html

work page 2025

[4] [4]

2025. CLion. Retrieved June 26, 2025 from https://www.jetbrains.com/clion/

work page 2025

[5] [5]

Dev Event Tracker GitHub

2025. Dev Event Tracker GitHub . Retrieved June 26, 2025 from https://github. com/web-cat/web-cat-subsystem-DevEventTracker

work page 2025

[6] [6]

IntelliJ IDEA

2025. IntelliJ IDEA. Retrieved June 26, 2025 from https://www.jetbrains.com/idea/

work page 2025

[7] [7]

Introduction to IDE Code Refactoring in Kotlin course

2025. Introduction to IDE Code Refactoring in Kotlin course . Retrieved June 26 2025 from https://plugins.jetbrains.com/plugin/23048-introduction-to-ide-code- refactoring-in-kotlin

work page 2025

[8] [8]

2025. JavaFX. Retrieved June 26, 2025 from https://openjfx.io/

work page 2025