pith. machine review for the scientific record. sign in

arxiv: 2604.15963 · v1 · submitted 2026-04-17 · 💻 cs.SE

Recognition: unknown

Supporting the Comprehension of Data Analysis Scripts

Florian Sihler , Oliver Gerstl , Lars Pfrenger , Julian Schubert , Matthias Tichy
Authors on Pith no claims yet

Pith reviewed 2026-05-10 08:36 UTC · model grok-4.3

classification 💻 cs.SE
keywords dataflowrscriptsanalysisgraphanalysescodedataflow
0
0 comments X

The pith

flowR is an IDE extension that performs incremental interprocedural dataflow analysis on R scripts to provide interactive comprehension support including graphs and inline values.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Data analysis scripts written in R are common in research but often become difficult to understand over time, especially when they involve cleaning, processing, and plotting data. The authors built flowR as a plugin for popular code editors that automatically examines these scripts. It creates a graph showing how data moves through the code, highlights dependencies, and lets users see values inline without running the script. The system handles R's flexible and dynamic style by combining data flow and control flow information across functions. It also includes a way for other developers to add new checks or views. The tool runs quickly enough on real projects to give feedback while the user works. This approach aims to reduce errors and make it simpler for analysts to reuse or share their code with others.

Core claim

flowR incrementally analyzes R projects by intertwining interprocedural data- and control-flow analyses to build a comprehensive dataflow graph, incorporating R's dynamic and explorative features, and requires an average of 576ms to calculate the full dataflow graph of real-world projects.

Load-bearing premise

That the static backward program slicer and dataflow graph construction accurately capture the behavior of real R scripts despite the language's dynamic features, without requiring runtime execution or missing critical paths.

Figures

Figures reproduced from arXiv: 2604.15963 by Florian Sihler, Julian Schubert, Lars Pfrenger, Matthias Tichy, Oliver Gerstl.

Figure 1
Figure 1. Figure 1: Dependency View To help researchers un￾derstand the structure of a given data analysis, we provide an automatically generated and updated overview of all script dependencies and out￾puts (cf [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Excerpt of linting results These rules check for various issues such as(1) absolute paths which are usually not portable to different systems, (2) invalid file paths indicating missing or misplaced inputs, (3) accessing non-existent columns in data frames hinting at wrong or incomplete data process￾ing, or (4) using a random number source without initializing it with a fixed seed, as indicators for non-exe… view at source ↗
Figure 3
Figure 3. Figure 3: Hover-over values To help users understand the values of variables and data shapes at specific program points, flowR provides value information in the code editor. When hovering over a vari￾able or expression in the code editor, a tooltip appears show￾ing the computed value or data shape at that point in the program as exemplified in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: below exemplifies the presentation of a simplified CFG: x <- 0 while (x < 20) { x <- x + 1 } x <- 0 x < 20 x <- x + 1 ⊤ ⊥ [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: A simplified view of flowR’s processes and results. All important steps are hyperlinked to their documentation. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example REPL Session Please note that while the REPL can access the file system using the file:// prefix, the corresponding directories have to be mounted into the docker container (this is also exemplified in the video). Besides the REPL, the docker image also supports starting a TCP server using the --server flag, as well as a websocket server by additionally passing the --ws flag. Consult the Interface … view at source ↗
read the original abstract

A lot of research relies on data analysis scripts to process, clean, and visualize data. However, recent studies show that these scripts are often hard to comprehend and maintain, hindering reproducibility and reuse, accompanied by a lack of tool support for handling such scripts. In this work, we focus on the R programming language, addressing this problem by presenting flowR as an extension for the common data analysis IDEs Positron and VS Code. Alongside a previously presented static backward program slicer, flowR provides an overview of data analysis scripts, interactive graph visualizations, linting, and inline value annotations to support data analysts. FlowR incrementally analyzes R projects by intertwining interprocedural data- and control-flow analyses to build a comprehensive dataflow graph, incorporating R's dynamic and explorative features. Additionally, flowR offers a plugin system and interfaces, allowing the integration of further analyses, such as new linting rules or custom visualizations. Requiring an average of 576ms to calculate the full dataflow graph of real-world projects, this enables near real-time feedback. The demonstration video is available at https://youtu.be/hJzr-r-NmMg . For the full source code and extensive documentation, refer to https://github.com/flowr-analysis/flowr . To try the docker image, use `docker run --rm -it eagleoutice/flowr`.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper presents flowR, a VS Code and Positron extension for R data analysis scripts. It builds on a prior static backward program slicer with additional features including interactive dataflow graph visualizations, linting, inline value annotations, and a plugin system for extensibility. The core technical contribution is an incremental interprocedural analysis that interleaves data- and control-flow to construct a comprehensive dataflow graph while accommodating R's dynamic and exploratory constructs; the tool reports an average of 576 ms to compute the full graph on real-world projects.

Significance. If the static analysis correctly captures R script behavior and the interface features prove usable, flowR could meaningfully improve script comprehension, maintenance, and reproducibility in data analysis workflows. The open-source release, plugin architecture, and near-real-time performance are concrete strengths that lower barriers to adoption and further research. However, the absence of any empirical validation (user studies, controlled experiments, or baseline comparisons) in the manuscript limits the assessed impact to the level of a promising prototype description.

major comments (2)
  1. [Abstract and §4] Abstract and §4 (analysis description): the claim that the intertwined interprocedural analyses 'incorporate R's dynamic and explorative features' and produce a 'comprehensive' dataflow graph is load-bearing for the central contribution, yet the manuscript provides no concrete examples, soundness argument, or handling strategy for well-known R constructs such as non-standard evaluation, lazy evaluation, or metaprogramming. Without such detail it is impossible to assess whether critical paths are missed, directly undermining the weakest assumption identified in the review.
  2. [Abstract and evaluation section] Abstract and any evaluation section: the usability and comprehension-support claims rest on feature descriptions and a single performance number (576 ms average), but the manuscript contains no user study, controlled experiment, or comparison against existing R analysis tools or IDE features. This absence makes the central claim that flowR 'supports the comprehension of data analysis scripts' unverified and therefore not yet ready for acceptance.
minor comments (2)
  1. The demonstration video link and GitHub repository are helpful, but the manuscript should include at least one self-contained example script together with the corresponding dataflow graph and slice to illustrate the claimed features.
  2. Clarify the relationship to the 'previously presented static backward program slicer': is the current work an incremental extension, a re-implementation, or a packaging of prior work? A short related-work paragraph would help readers situate the contribution.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for their constructive and detailed review. The comments highlight important areas for clarification and strengthening of the manuscript. We respond to each major comment below, indicating planned revisions where appropriate.

read point-by-point responses

  1. Referee: [Abstract and §4] Abstract and §4 (analysis description): the claim that the intertwined interprocedural analyses 'incorporate R's dynamic and explorative features' and produce a 'comprehensive' dataflow graph is load-bearing for the central contribution, yet the manuscript provides no concrete examples, soundness argument, or handling strategy for well-known R constructs such as non-standard evaluation, lazy evaluation, or metaprogramming. Without such detail it is impossible to assess whether critical paths are missed, directly undermining the weakest assumption identified in the review.

    Authors: We agree that additional concrete detail is needed to support the claims regarding R's dynamic features. In the revised manuscript we will expand §4 with specific examples illustrating how the incremental interprocedural analysis handles non-standard evaluation (e.g., quote/unquote patterns common in data-analysis scripts) and lazy evaluation within function calls. We will also add an explicit limitations subsection discussing the current handling of metaprogramming and the absence of a full formal soundness proof, which lies outside the scope of this tool-description paper. These changes will allow readers to better evaluate the scope of the 'comprehensive' dataflow graph. revision: yes

  2. Referee: [Abstract and evaluation section] Abstract and any evaluation section: the usability and comprehension-support claims rest on feature descriptions and a single performance number (576 ms average), but the manuscript contains no user study, controlled experiment, or comparison against existing R analysis tools or IDE features. This absence makes the central claim that flowR 'supports the comprehension of data analysis scripts' unverified and therefore not yet ready for acceptance.

    Authors: We acknowledge that the manuscript provides no user studies, controlled experiments, or direct tool comparisons to empirically validate the comprehension-support claims. The current evaluation is limited to performance measurements on real-world projects. In revision we will temper the abstract and evaluation sections to describe the features as intended to support comprehension, add a limitations paragraph explicitly noting the lack of usability validation, and include brief comparisons to related static-analysis tools for R drawn from the literature. A full user study remains future work. revision: partial

standing simulated objections not resolved
  • The absence of user studies or controlled experiments validating the comprehension-support claims cannot be addressed by textual revision alone and would require new empirical work.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is an engineering and tool-development paper with no mathematical derivations, fitted constants, or new theoretical entities; all components rest on standard static analysis concepts from prior literature.

pith-pipeline@v0.9.0 · 5547 in / 1090 out tokens · 35562 ms · 2026-05-10T08:36:14.365836+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 9 canonical work pages

  1. [1]

    ACM SIGSOFT Empirical Standards.https:// www2.sigsoft.org/Empirical Standards/ docs/?standard=RepositoryMining

    2022. ACM SIGSOFT Empirical Standards.https:// www2.sigsoft.org/Empirical Standards/ docs/?standard=RepositoryMining

    2022

  2. [2]

    Find and fix problems in your JavaScript code - ESLint - Pluggable JavaScript Linter — eslint.org.https:// eslint.org/

    2026. Find and fix problems in your JavaScript code - ESLint - Pluggable JavaScript Linter — eslint.org.https:// eslint.org/

    2026

  3. [3]

    E. D. Berger, S. M. Blackburn, M. Hauswirth, and M. Hicks. 2018. SIGPLAN Empirical Evaluation Checklist.https:// raw.githubusercontent.com/ SI GPLAN/ empirical-evaluation/master/ checklist/ checklist.pdf 1Using a Linux PC with a 5 GHz Intel i9-9900K CPU and sufficient memory. Supporting the Comprehension of Data Analysis Scripts Conference’17, July 2017, ...

    2018

  4. [4]

    Greta Dolcetti, Agostino Cortesi, Caterina Urban, and Enea Zaffanella. 2024. Towards a High Level Linter for Data Science. InProceedings of the 10th ACM SIGPLAN International Workshop on Numerical and Symbolic Abstract Domains, NSAD 2024, Pasadena, CA, USA, 22 October 2024, Vincenzo Arceri and Michele Pasqua (Eds.). ACM, 18–25. doi:10.1145/ 3689609.3689996

    work page arXiv 2024

  5. [5]

    Stefan Fischer, Callum Duffield, Amanda Davidson, Rhiannon Bolton, Jane Hurst, and Paula Stockley. 2022. Data for: Fitness costs of female competition linked to resource defence and relatedness of competitors. doi:10.5061/ DRYAD.J0ZPC86H 1

    2022

  6. [6]

    GitHub. 2026. CodeQL.https:// codeql.github.com/

    2026

  7. [7]

    2021.Language server protocol and implementation

    Nadeeshaan Gunasinghe and Nipuna Marcus. 2021.Language server protocol and implementation. Springer

    2021

  8. [8]

    Nakshatra Gupta, Ashutosh Rajput, and Sridhar Chimalakonda. 2022. COSPEX: A Program Comprehension Tool for Novice Programmers. In44th IEEE/ACM International Conference on Software Engineering: Companion Proceedings, ICSE Companion 2022, Pittsburgh, PA, USA, May 22-24, 2022. ACM/IEEE, 41–45. doi:10 .1145/ 3510454.3516842

    work page arXiv 2022

  9. [9]

    Sara Hamilton, Vienna Saccomanno, Walter Heady, Alyssa-Lois Gehman, Steve Lonhart, Rodrigo Beas-Luna, Fiona Francis, Lynn Lee, Laura Rogers-Bennett, Anne Salomon, and Sarah Gravem. 2021. Disease-driven mass mortality event leads to widespread extirpation and variable recovery potential of a marine predator across the eastern Pacific. doi:10.5281/ ZENODO.5116546

    2021

  10. [10]

    Jim Hester, Florent Angly, Michael Chirico, Russ Hyde, Ren Kun, Indrajeet Patil, and Alexander Rosenstock. 2025. Static Code Analysis for R.Journal of Open Source Software10, 108 (2025), 7240. doi:10.21105/joss.07 240

    work page doi:10.21105/joss.07 2025

  11. [11]

    Anaytul Islam, Muhammad Asaduzzaman, and Shaowei Wang

    Md. Anaytul Islam, Muhammad Asaduzzaman, and Shaowei Wang. 2024. On the Executability of R Markdown Files. In21st IEEE/ACM International Conference on Mining Software Repositories, MSR 2024, Lisbon, Portugal, April 15-16, 2024, Diomidis Spinellis, Alberto Bacchelli, and Eleni Constantinou (Eds.). ACM, 254–

    2024

  12. [13]

    Daniel Jackson and Eugene J. Rollins. 1994. A New Model of Program De- pendences for Reverse Engineering. InProceedings of the Second ACM SIG- SOFT Symposium on Foundations of Software Engineering, SIGSOFT 1994, New Orleans, Louisiana, USA, December 6-9, 1994, David S. Wile (Ed.). ACM, 2–10. doi:10.1145/ 193173.195281

    work page arXiv 1994

  13. [14]

    Kang and Philip J

    Hyeonsu B. Kang and Philip J. Guo. 2017. Omnicode: A Novice-Oriented Live Programming Environment with Always-On Run-Time Value Visualizations. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, UIST 2017, Quebec City, QC, Canada, October 22 - 25, 2017, Krzysztof Gajos, Jennifer Mankoff, and Chris Harrison (Eds.). A...

    work page arXiv 2017

  14. [15]

    Kadiray Karakaya, Stefan Schott, Jonas Klauke, Eric Bodden, Markus Schmidt, Linghui Luo, and Dongjie He. 2024. SootUp: A Redesign of the Soot Static Analysis Framework. InTACAS 2024, Luxembourg City, Luxembourg, April 6-11, 2024, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 14570). Springer, 229–247. doi:10.1007/ 97 8-3-031-57 246-3_13

    2024

  15. [16]

    2023.languageserver: Language Server Protocol

    Randy Lai. 2023.languageserver: Language Server Protocol. doi:10.32614/CRAN. package.languageserverR package version 0.3.16

    work page doi:10.32614/cran 2023

  16. [17]

    statcheck

    Michèle B. Nuijten and Joshua R. Polanin. 2020. “statcheck”: Automatically detect statistical reporting inconsistencies to increase reproducibility of meta-analyses. Research Synthesis Methods11, 5 (2020), 574–579. doi:10.1002/ jrsm.1408

    2020

  17. [18]

    2025.Computational Reproducibility of R Code Supplements on OSF

    Lorraine Saju, Tobias Holtdirk, Meetkumar Pravinbhai Mangroliya, and Arnim Bleier. 2025.Computational Reproducibility of R Code Supplements on OSF. ICWSM, US. doi:10.36190/2025.49

    work page doi:10.36190/2025.49 2025

  18. [19]

    Florian Sihler, Lukas Pietzschmann, Raphael Straub, Matthias Tichy, Andor Diera, and Abdelhalim Hafedh Dahou. 2024. On the Anatomy of Real-World R Code for Static Analysis. In21st IEEE/ACM International Conference on Mining Software Repositories, MSR 2024, Lisbon, Portugal, April 15-16, 2024, Diomidis Spinellis, Alberto Bacchelli, and Eleni Constantinou (...

    work page arXiv 2024

  19. [20]

    Florian Sihler and Matthias Tichy. 2024. flowR: A Static Program Slicer for R. InProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, ASE 2024, Sacramento, CA, USA, October 27 - November 1, 2024, Vladimir Filkov, Baishakhi Ray, and Minghui Zhou (Eds.). ACM, 2390–2393. doi:10.1145/ 3691620.3695359

    work page arXiv 2024

  20. [21]

    Florian Sihler and Matthias Tichy. 2025. Statically Analyzing the Dataflow of R Programs, In OOPSLA ’25.Proceedings of the ACM on Programming Languages9, OOPSLA2, 1034–1062. doi:10.1145/ 37 63087

    2025

  21. [22]

    Lau, Thomas Pasquier, and Mercè Crosas

    Ana Trisovic, Matthew K. Lau, Thomas Pasquier, and Mercè Crosas. 2022. A Large-Scale Study on Research Code Quality and Execution. 9, 1 (2022), 60. Issue

    2022

  22. [23]

    doi:10.1038/ s41597 -022-01143-6

  23. [24]

    Alexi Turcotte and Zheyuan Wu. 2025. Expressing and Checking Statistical Assumptions.Proc. ACM Softw. Eng.2, FSE (2025), 2735–2758. doi:10.1145/ 37 29391

    2025

  24. [25]

    Tobias Weck and Matthias Tichy. 2016. Visualizing Data-Flows in Functional Programs. InIEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering, SANER 2016, Suita, Osaka, Japan, March 14-18, 2016 - Volume

    2016

  25. [26]

    doi:10.1109/ SANER.2016.82

    IEEE Computer Society, 293–303. doi:10.1109/ SANER.2016.82

    2016

  26. [27]

    R Code Analyzer (flowR)

    Hadley Wickham, Garrett Grolemund, et al . 2017.R for data science. Vol. 2. O’Reilly Sebastopol. A Demonstration Walkthrough This appendix contains the walkthrough of our demonstration video. For the required information on the tool, please refer to Section B. The walkthrough is organized chronologically alongside the different parts in the video. A.1 The...

    2017