Pith. sign in

REVIEW 3 major objections 5 minor 70 references

How Wrangling Tools Shape Wrangling: A Technical Dimensions Analysis

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Tool affordances steer data-wrangling strategies but do not determine outcomes, a 40-person study across four interface paradigms finds.

desk verdict A worthwhile exploratory study with a new TDoPS-based comparison; the unmeasured AI co-pilot confound means the central claim is more suggestive than causal. read the letter →

arxiv 2607.26198 v1 pith:5DFV6SMN submitted 2026-07-28 cs.HC

Shiyi He , El Kindi Rezig , Paul Rosen , Andrew M. McNutt This is my paper
classification cs.HC
keywords datawranglinginterfaceparadigmsuserstudytechnicaldimensionsspreadsheetsnotebooksconversationalAIvisualwranglers
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the interface paradigm of a data-wrangling tool—spreadsheet, notebook, conversational AI, or visual wrangler—shapes how people approach cleaning tasks, not just how well they finish. Across 40 participants using Jupyter, Excel, ChatGPT, or OpenRefine, no tool consistently produced better results, and outputs within each tool varied widely. The study's key finding is a set of trade-offs: data-centered interfaces like Excel and OpenRefine invite opportunistic, visible-error-driven cleaning, while abstraction-centered interfaces like Jupyter push toward systematic transformations at higher cognitive cost. The authors argue that tool design structures the process and subjective experience of wrangling, and that recognizing these trade-offs can make tool design more intentional.

What carries the argument

The Technical Dimensions of Programming Systems (TDoPS) framework, a set of seven clusters of dimensions for comparing programming systems—interaction, errors, conceptual structure, notation, complexity, customizability, and adoptability. The study uses it as a coding scheme for deductive thematic analysis of think-aloud and screen-capture data, mapping each tool onto trade-off axes such as visibility versus abstraction, predictability versus simplicity, big versus small steps, and structured reuse versus ad hoc repetition. The framework does the work of turning observed behaviors into comparable design tensions.

What would settle it

An experiment that assigns the same participants to the same two paradigms with hosting and versions matched (e.g., desktop Excel versus hosted JupyterLab) and measures both process (operation sequences, planning versus opportunistic actions) and outcome would settle whether paradigm-level differences persist; if operation sequences become indistinguishable across paradigms once hosting and external help are controlled, the central claim fails.

Watch

Extended reading notes

Core claim

Within the context of this study, tool affordances steer user strategies but do not determine outcomes: no single tool offers a consistent advantage, and results do not converge within tools. The key tension is between data-centered and abstraction-centered interfaces. Data-centered interfaces (Excel, OpenRefine) encourage opportunistic cleaning driven by visible data issues rather than systematic, planned transformations, but they come with a cognitive burden; abstraction-centered interfaces (Jupyter) support systematic cleaning but require more procedural effort. AI interfaces (ChatGPT) reduce reported effort and frustration but obscure intermediate states, inviting automation bias and mis

Load-bearing premise

The load-bearing premise is that observed differences in strategy and experience are caused by the interface paradigm of the assigned tool, rather than by the specific hosting configuration (web-based Excel, JupyterLite, VM-based ChatGPT, OpenRefine version) or by participants' external use of AI and other resources.

Editorial extensions

If this is right

  • If tool paradigms shape process but not outcome quality, then evaluating wrangling tools solely by final data quality misses most of what design affects; user experience and strategy should be explicit design targets.
  • The trade-off between opportunistic and systematic cleaning suggests designers could combine data visibility with structured transformations—for example, spreadsheet-like direct manipulation with notebook-like abstractions—to get the best of both.
  • The finding that ChatGPT users reported lower frustration but produced no better outputs, and that participants in all conditions turned to AI for help, implies AI is best positioned as an ambient co-pilot inside existing tools rather than a primary interface.
  • The pattern of tool-induced sensemaking—users redefine their task in terms of what the tool makes easy—implies that adding a prominent feature (e.g., OpenRefine's faceting) can steer attention toward that operation, for better or worse.
  • Since task constraints narrow the space of viable strategies, tool differences matter most in open-ended wrangling; benchmarking and design guidance should be task-dependent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If tool-induced sensemaking is real, a testable extension is that the same cleaning task given to the same user in two paradigms should produce measurably different sequences of operations (e.g., opportunistic versus systematic), which could be verified with interaction logs.
  • The paper's outcome measure (Jaccard similarity to silver tables) mainly captures row and column structure, not semantic correctness; future work could use multi-set semantic matching to see whether hidden quality differences lie beneath the 'no consistent advantage' claim.
  • The browser-hosted configurations and permitted external AI use introduce confounds; a replication that isolates the interface paradigm from hosting environment (e.g., desktop Excel versus Excel for the web, hosted Jupyter versus JupyterLite) would strengthen the attribution to paradigm.
  • The study hints that AI assistance may flatten strategy differences between tools; a direct comparison of the same tools with and without ambient AI help would test whether AI acts as a great equalizer of wrangling process.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper presents a between-subjects observational study (N=40) in which participants performed two data-cleaning tasks in one of four tools—Jupyter, Excel, ChatGPT, and OpenRefine—while thinking aloud under screen capture. The authors evaluate outcomes with Jaccard similarity against author-constructed 'silver tables' and analyze process qualitatively through the Technical Dimensions of Programming Systems (TDoPS) framework. They conclude that, within this study, tool affordances steer wrangling strategies but do not determine outcomes: no tool consistently outperformed others, and the main differences are trade-offs between data-centered and abstraction-centered interfaces. Data-centered tools are said to encourage opportunistic cleaning, abstraction-centered tools to support systematic transformation at higher cognitive cost, and ChatGPT to reduce effort while obscuring intermediate states and risking automation bias. The paper explicitly forgoes statistical inference, is exploratory in tone, and provides open materials, analysis code, and replays.

Significance. If its central qualitative claim survives the confounding concerns, the paper would be a useful empirical contribution to HCI and data-wrangling research: it demonstrates a systematic way to compare interface paradigms via TDoPS, supplies rich observation-based accounts of how feedback immediacy, error handling, notational structure, and reuse mechanisms shape behavior, and responsibly avoids overclaiming performance differences. The strongest features are the openly available study materials and analysis code, the use of screen recordings and think-aloud data, the explicit acknowledgment of the exploratory status, and several concrete, falsifiable observations that future work can test. The main risk is that the central causal attribution—interface paradigm steers strategy—is not yet isolated from external AI use and deployment differences. This is a fixable weakness rather than a fundamental flaw, provided the authors can re-analyze their recordings or soften the claim accordingly.

major comments (3)
  1. [§V-B, §VII, §III] Section V-B states that 'most participants needed external help (e.g., AI or docs) to finish,' and Section VII reports that non-AI participants 'usually turned to AI (such as ChatGPT and Claude)' when writing formulas or expressions. Because the paper's central claim is that the assigned tool's interface paradigm steers wrangling strategy, uncontrolled external AI use is a direct confound: the effective treatment in three of four arms is 'tool X plus an AI assistant.' The limitation is acknowledged but not quantified or conditioned on. I request a post hoc analysis of the screen recordings and think-aloud data: (i) count and characterize external-AI use by condition; (ii) compare the coded strategy patterns (opportunistic vs. systematic cleaning, inspection behavior, error handling) for participants with little or no external AI help against those with substantial help; and (iii) state w
  2. [§IV, Appendix B] The silver tables are author-constructed reference solutions, and the clustering in Fig. 4 is used to support claims such as 'tools alone do not drive successful outcomes' and the observation that a ChatGPT-heavy cluster appears in Task 1. The paper correctly disclaims statistical inference, but the cluster-level reading is used in substantive arguments. The cluster labels ('partial/no cleaning', 'aggressively delete columns') were assigned after inspecting outputs with the same team's coding; no inter-rater reliability or sensitivity analysis is reported. With a modest sample and Jaccard similarity on raw rows (plus manual column canonicalization for Task 2), the clusters may be sensitive to distance measure, preprocessing, and the particular silver-table set. Please report robustness of the clustering (e.g., alternative distance/similarity measures, subsets of silver tables), provide p
  3. [§III, Fig. 7, Appendix: Experimental Configuration Details] The paper does not describe the assignment mechanism (random vs. balanced by experience) and recruitment used tool-specific advertisements targeting users familiar with each tool. Fig. 7 suggests that self-reported experience differs visibly across conditions. In addition, the compared conditions are not 'the paradigm' but specific deployments: Excel for the web, ChatGPT-5.2 through a Windows VM via noVNC, OpenRefine 3.10.0, and JupyterLite rather than hosted JupyterLab. Because the qualitative comparisons attribute observed differences to interface paradigm, baseline differences in expertise and the differing deployment contexts are confounds. At minimum, state the assignment procedure, report condition-level experience descriptively, and discuss how the deployment differences could mimic or mask paradigm effects. Ideally, incorporate experience into the quantitative description and int
minor comments (5)
  1. [Throughout §V] Participant IDs are formatted inconsistently (e.g., P AI7 vs. PAI8, POR5 vs. P OR5, PXL1 vs. P XL1). Please standardize the spacing convention.
  2. [§V-E] The phrase '5.2±0.8 on a 7-point scale' lacks a clear referent: what exactly was rated, and in which post-task survey item? Please specify.
  3. [§IV] The sentence 'For Task 2, we canonicalized the columns via a manual alignment, including inserting columns of nulls for those that had been deleted and enforcing a consistent ordering' is grammatically awkward; consider rephrasing for clarity.
  4. [Fig. 4] The figure is dense, especially for Task 2. Larger fonts, separate panels per task, or explicit per-tool row/column annotations would improve legibility.
  5. [§V introduction] Because the thematic analysis is deductive with codes drawn from TDoPS, the finding that TDoPS dimensions describe observed behavior is partly by construction. Please frame the section accordingly (e.g., as an application of the framework rather than an independent validation of it).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study's central claims are empirical, and its use of TDoPS is interpretive rather than derivational.

full rationale

The paper is an observational user study, not a derivation, so the standard circularity reductions (equation equals input, fitted parameter renamed as prediction) do not apply. The central claim that tool affordances steer user strategies without determining outcomes rests on independent observations: screen captures, think-aloud data, task outputs, and similarity analyses. TDoPS is used as a coding scaffold via deductive thematic analysis, which means the presentation is organized by TDoPS dimensions; however, the specific findings (e.g., data-centered interfaces encouraging opportunistic cleaning, abstraction-centered workflows supporting systematic transformation) are grounded in participant behavior and outcome measures, not entailed by the framework itself. The silver tables are author-constructed reference points, but they are not fitted to participant data and are not presented as predictions; the Jaccard comparisons are empirical. Self-citations such as ReVISit and the Buckaroo work are methodological or related-work references, not load-bearing uniqueness theorems or unverified premises. The acknowledged confounds, especially external AI use in non-ChatGPT conditions and browser/VM hosting, are genuine validity threats to causal attribution, but they are not circularity by construction. The paper explicitly lists these limitations and does not reduce its conclusions to its own definitions, so no significant circularity is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented entities; the paper is an empirical qualitative study. The load-bearing assumptions are representativeness of tool configurations, internal silver-table baselines, think-aloud validity, and the appropriateness of TDoPS as an analytic lens.

assumptions (4)
  • domain assumption The four selected tools (Excel web, JupyterLite, ChatGPT 5.2, OpenRefine 3.10.0) are representative instances of spreadsheet, notebook, conversational-AI, and visual-wrangler paradigms.
    Section III selects tools to span paradigms and the qualitative claims are about paradigms, so non-representative configurations or version-specific quirks would undermine the generalization.
  • ad hoc to paper The author-constructed silver tables are a valid reference space for judging output quality and for interpreting the clustering of participant results.
    Section IV and Appendix B: six hand-built variants per task define the baseline; the Jaccard clustering and conclusions about 'close to silver tables' depend on these internal references.
  • domain assumption Think-aloud verbalizations and screen recordings accurately reflect participants' reasoning and strategies.
    Section III uses think-aloud and video as evidence of internal process; reactivity, incomplete verbalization, or social desirability could bias the coded themes.
  • domain assumption TDoPS dimensions are an appropriate coding frame for data-wrangling systems and can be applied deductively.
    Section V states codes were drawn from TDoPS; the framework's categories constrain what is noticed, making dimension-level findings partly framework-driven.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How Wrangling Tools Shape Wrangling: A Technical Dimensions Analysis." pith.science (2026). https://pith.science/paper/5DFV6SMN

@misc{pith2026260726198,
  author       = {Pith},
  title        = {Pith review of: How Wrangling Tools Shape Wrangling: A Technical Dimensions Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5DFV6SMN}},
  note         = {Machine review of arXiv:2607.26198}
}
read the original abstract

Wrangling consumes a disproportionate share of the effort associated with any data project. While a variety of tools support it, relatively little is known about how their differing interface forms shape the way people actually wrangle. We conduct a between-subjects (N=40) observational study of data cleaning tasks performed in tools spanning distinct interface paradigms: Jupyter (notebook), Excel (spreadsheet), ChatGPT (conversational AI), and OpenRefine (visual wranglers). We situate our observations within the Technical Dimensions of Programming Systems framework, which we use as a conceptual scaffold for comparing across interface paradigms. Within the context of our study, the results suggest that tool affordances steer user strategies but do not determine outcomes. There is no consistent advantage of any single tool, nor convergence of results within tools observed across our outcome measures. Instead, we identify trade-offs and connect them with observed practice. For example, a key tension is between data- and abstraction-centered interfaces, where data-centered interfaces encourage opportunistic cleaning rather than systematic, planned transformations found in abstraction-focused tools (but come with a cognitive burden). Tool design, beyond mere functionality, plays a structuring role in how data work unfolds.

Figures

Figures reproduced from arXiv: 2607.26198 by the authors.

Figure 1
Figure 1. For each TDoPs [1] dimensional cluster, we observe trade-offs that data wrangling tools approach in different ways. While we place the tools in these spaces, their placement is less valuable than the spaces that they occupy, as special cases may change that positioning. We forgo Adoptability because our study focuses on short-term usage by practitioners, which limits commentary on factors like learnability. Abstract… view at source ↗
Figure 2
Figure 2. Our study was deployed using ReVISit [29], with a tool￾specific environment embedded for each condition. Similarly, Sultanum et al. [27] conducted an interview study to understand how trust is mediated within data products, highlighting how validation and verification interact. Petricek et al. [28] explored AI agents as data wrangling tools, a consideration we echo by studying ChatGPT. In analyzing their tool Dango,… view at source ↗
Figure 3
Figure 3. Completion time by condition. study, but we focus on mature, paradigmatically representative tools (e.g., OpenRefine has ∼12k GitHub stars). Participants completed a pair of representative data wran￾gling tasks. In the first, they wrangled a Game Rating Dataset [31], requiring them to filter, standardize, and remove irregularities (e.g., type errors, outliers). In the second, more open-ended task, they wrangled a Us… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Similarity between participant outputs, the original dataset, [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Participants held a range of opinions about the tools they used, [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: A version of [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Participant familiarity, confidence, and experience by tool. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

70 extracted references · 1 canonical work pages

  1. [1]

    Technical Dimensions of Programming Systems,

    J. Jakubovic, J. Edwards, and T. Petricek, “Technical Dimensions of Programming Systems,”The Art, Science, and Engineering of Programming, vol. 7, no. 3, 2023. [Online]. Available: https: //doi.org/10.22152/PROGRAMMING-JOURNAL.ORG/2023/7/13

    doi:10.22152/programming-journal.org/2023/7/13 2023

  2. [2]

    Self-Service Data Preparation: Research to Practice,

    J. M. Hellerstein, J. Heer, and S. Kandel, “Self-Service Data Preparation: Research to Practice,”IEEE Data Engineering Bulletin, vol. 41, pp. 23– 34, 2018

    2018

  3. [3]

    Wrangler: Interactive Visual Specification of Data Transformation Scripts,

    S. Kandel, A. Paepcke, J. Hellerstein, and J. Heer, “Wrangler: Interactive Visual Specification of Data Transformation Scripts,” in ACM SIGCHI, 2011, pp. 3363–3372. [Online]. Available: https: //doi.org/10.1145/1978942.1979444

    arXiv 2011

  4. [4]

    Towards Reliable Interactive Data Cleaning: A User Survey and Recommendations,

    S. Krishnan, D. Haas, M. J. Franklin, and E. Wu, “Towards Reliable Interactive Data Cleaning: A User Survey and Recommendations,” in ACM HILDA, 2016, p. 9. [Online]. Available: https://doi.org/10.1145/ 2939502.2939511

    arXiv 2016

  5. [5]

    Research Directions in Data Wrangling: Visualizations and Transformations for Usable and Credible Data,

    S. Kandel, J. Heer, C. Plaisant, J. Kennedy, F. Van Ham, N. H. Riche, C. Weaver, B. Lee, D. Brodbeck, and P. Buono, “Research Directions in Data Wrangling: Visualizations and Transformations for Usable and Credible Data,”Information visualization, vol. 10, no. 4, pp. 271–288,

  6. [6]

    Jupyterlite: Wasm Powered Jupyter Running in the Browser,

    JupyterLite Contributors, “Jupyterlite: Wasm Powered Jupyter Running in the Browser,” https://github.com/jupyterlite/jupyterlite, 2021

    2021

  7. [7]

    Verborgh and M

    R. Verborgh and M. De Wilde,Using Openrefine. Packt, 2013, vol. 1

    2013

  8. [8]

    Cognitive Psychology Meets Data Management: State of the Art and Future Directions,

    S. S Bhowmick, S. A. Chen, and D. Srivastava, “Cognitive Psychology Meets Data Management: State of the Art and Future Directions,” in Companion of ACM SIGMOD, 2024, pp. 590–596. [Online]. Available: https://doi.org/10.1145/3626246.3654682

    arXiv 2024

Show all 70 references
  1. [9]

    Predictive Interaction for Data Transformation,

    J. Heer, J. M. Hellerstein, and S. Kandel, “Predictive Interaction for Data Transformation,” inCIDR, 2015

    2015

  2. [10]

    Potter’s Wheel: An Interactive Data Cleaning System,

    V . Raman and J. M. Hellerstein, “Potter’s Wheel: An Interactive Data Cleaning System,” inPVLDB, 2001, p. 381–390. [Online]. Available: https://doi.org/10.5555/645927.672045

    2001

  3. [11]

    Tableau Prep Builder,

    Tableau Software, “Tableau Prep Builder,” 2026, accessed: 2026-04-06. [Online]. Available: https://www.tableau.com/products/prep

    2026

  4. [12]

    Alteryx Designer,

    Alteryx, Inc., “Alteryx Designer,” 2026, accessed: 2026-04-06. [Online]. Available: https://www.alteryx.com/

    2026

  5. [13]

    Knime Analytics Platform,

    KNIME AG, “Knime Analytics Platform,” 2026, open source with commercial extensions. Accessed: 2026-04-06. [Online]. Available: https://www.knime.com/

    2026

  6. [14]

    Talend Data Preparation,

    Talend, “Talend Data Preparation,” 2026, acquired by Qlik in 2023. Accessed: 2026-04-06. [Online]. Available: https://www.talend.com/

    2026

  7. [15]

    Profiler: Integrated Statistical Analysis and Visualization for Data Quality Assessment,

    S. Kandel, R. Parikh, A. Paepcke, J. M. Hellerstein, and J. Heer, “Profiler: Integrated Statistical Analysis and Visualization for Data Quality Assessment,” inAVI, 2012, pp. 547–554. [Online]. Available: https://doi.org/10.1145/2254556.2254659

    2012

  8. [16]

    Unravel: A Fluent Code Explorer for Data Wrangling,

    N. Shrestha, T. Barik, and C. Parnin, “Unravel: A Fluent Code Explorer for Data Wrangling,” inACM UIST, 2021, pp. 198–207. [Online]. Available: https://doi.org/10.1145/3472749.3474744

    2021

  9. [18]

    Transforming Spreadsheets with Data Noodles,

    M. I. Gorinova, A. Sarkar, A. F. Blackwell, and K. Prince, “Transforming Spreadsheets with Data Noodles,” inIEEE VL/HCC, 2016, pp. 236–237. [Online]. Available: https://doi.org/10.1109/vlhcc.2016.7739694

    2016

  10. [19]

    Dango: A Mixed- Initiative Data Wrangling System Using Large Language Model,

    W.-H. Chen, W. Tong, A. Case, and T. Zhang, “Dango: A Mixed- Initiative Data Wrangling System Using Large Language Model,” inACM SIGCHI, 2025. [Online]. Available: https://doi.org/10.1145/ 3706598.3714135

    2025

  11. [20]

    Towards Scalable Visual Data Wrangling via Direct Manipulation,

    E. K. Rezig, M. M. Mohammad, N. Baret, R. Mayerhofer, A. McNutt, and P. Rosen, “Towards Scalable Visual Data Wrangling via Direct Manipulation,” inCIDR, 2026

    2026

  12. [21]

    Arachnid: Generalized Visual Data Cleaning,

    C. L. Shou and A. Shukla, “Arachnid: Generalized Visual Data Cleaning,” inACM SIGMOD, 2019, pp. 1850–1852. [Online]. Available: https://doi.org/10.1145/3299869.3300108

    2019

  13. [22]

    Dirty data in the newsroom: comparing data preparation in journalism and data science,

    S. Kasica, C. Berret, and T. Munzner, “Dirty data in the newsroom: comparing data preparation in journalism and data science,” inACM SIGCHI, 2023, pp. 1–18

    2023

  14. [23]

    Table Scraps: An Actionable Framework for Multi-Table Data Wrangling from an Artifact Study of Computational Journalism,

    ——, “Table Scraps: An Actionable Framework for Multi-Table Data Wrangling from an Artifact Study of Computational Journalism,” IEEE TVCG, vol. 27, no. 2, pp. 957–966, 2020. [Online]. Available: https://doi.org/10.1109/tvcg.2020.3030462

    2020

  15. [24]

    Enterprise Data Analysis and Visualization: An Interview Study,

    S. Kandel, A. Paepcke, J. M. Hellerstein, and J. Heer, “Enterprise Data Analysis and Visualization: An Interview Study,”IEEE TVCG, vol. 18, no. 12, pp. 2917–2926, 2012. [Online]. Available: https: //doi.org/10.1109/tvcg.2012.219

    2012 doi

  16. [25]

    Preliminary guidelines for combining data integration and visual data analysis,

    A. Coscia, A. Suh, R. Chang, and A. Endert, “Preliminary guidelines for combining data integration and visual data analysis,”IEEE TVCG, vol. 30, no. 10, pp. 6678–6690, 2023. [Online]. Available: https://doi.org/10.1109/TVCG.2023.3334513

    2023

  17. [26]

    The End-User Programming Challenge of Data Wran- gling,

    M. Gorinova, K. Prince, S. Meakins, A. Vuylsteke, M. Jones, and A. Blackwell, “The End-User Programming Challenge of Data Wran- gling,” inPsychology of Programming Interest Group, 2016, pp. 140– 149

    2016

  18. [27]

    Data Guards: Challenges and Solutions for Fostering Trust in Data,

    N. Sultanum, D. Bromley, and M. Correll, “Data Guards: Challenges and Solutions for Fostering Trust in Data,” inIEEE Visualization, 2024, pp. 56–60. [Online]. Available: https://doi.org/10.1109/vis55277. 2024.00019

    2024

  19. [28]

    AI Assistants: A Framework for Semi-Automated Data Wrangling,

    Petricek, Tomas and van Den Burg, Gerrit JJ and Naz ´abal, Alfredo and Ceritli, Taha and Jim ´enez-Ruiz, Ernesto and Williams, Christopher KI, “AI Assistants: A Framework for Semi-Automated Data Wrangling,” IEEE KDD, vol. 35, no. 9, pp. 9295–9306, 2022. [Online]. Available: ht...

    2022

  20. [29]

    Revisit 2: A Full Experiment Life Cycle User Study Framework,

    Z. Cutler, J. Wilburn, H. Shrestha, Y . Ding, B. Bollen, K. A. Nadib, T. He, A. McNutt, L. Harrison, and A. Lex, “Revisit 2: A Full Experiment Life Cycle User Study Framework,”IEEE TVCG, 2025. [Online]. Available: https://doi.org/10.1109/tvcg.2025.3633896

    2025

  21. [30]

    Revealing the Semantics of Data Wrangling Scripts with Comantics,

    K. Xiong, Z. Luo, S. Fu, Y . Wang, M. Xu, and Y . Wu, “Revealing the Semantics of Data Wrangling Scripts with Comantics,”IEEE TVCG, vol. 29, no. 1, pp. 117–127, 2022. [Online]. Available: https://doi.org/10.1109/TVCG.2022.3209470

    2022

  22. [31]

    Video Game Sales with Ratings,

    R. Kirubi, “Video Game Sales with Ratings,” https://www.kaggle.com/ datasets/rush4ratio/video-game-sales-with-ratings, 2025

    2025

  23. [32]

    Used Cars Dataset,

    A. Reese, “Used Cars Dataset,” https://www.kaggle.com/datasets/ austinreese/craigslist-carstrucks-data, 2025

    2025

  24. [33]

    Novnc: Vnc Client Web Appli- cation,

    Martin, Joel and noVNC Contributors, “Novnc: Vnc Client Web Appli- cation,” https://github.com/novnc/novnc

  25. [34]

    Towards Efficient Data Wrangling with Llms Using Code Generation,

    X. Li and T. D ¨ohmen, “Towards Efficient Data Wrangling with Llms Using Code Generation,” inWorkshop on Data Management for End- to-End Machine Learning, 2024, pp. 62–66

    2024

  26. [35]

    Thematic Analysis,

    V . Clarke and V . Braun, “Thematic Analysis,”The Journal of Positive Psychology, vol. 12, no. 3, pp. 297–298, 2017. [Online]. Available: https://doi.org/10.1080/17439760.2016.1262613

    2017

  27. [37]

    Baseline: Operation-Based Evolution and Versioning of Data,

    J. Edwards and T. Petricek, “Baseline: Operation-Based Evolution and Versioning of Data,”arXiv, 2025. [Online]. Available: https: //doi.org/10.48550/arXiv.2512.09762

    2025 doi arXiv

  28. [38]

    Projectional Editors for Json-Based Dsls,

    A. McNutt and R. Chugh, “Projectional Editors for Json-Based Dsls,” inIEEE VL/HCC, 2023. [Online]. Available: https://doi.org/10.1109/ vl-hcc57772.2023.00015

    2023

  29. [39]

    A Usability Analysis of Blocks-Based Programming Editors Using Cognitive Dimensions,

    R. Holwerda and F. Hermans, “A Usability Analysis of Blocks-Based Programming Editors Using Cognitive Dimensions,” inIEEE VL/HCC, 2018, pp. 217–225. [Online]. Available: https://doi.org/10.1109/vlhcc. 2018.8506483

    2018

  30. [40]

    Xlblocks: A Block-Based Formula Editor for Spreadsheet Formulas,

    B. Jansen and F. Hermans, “Xlblocks: A Block-Based Formula Editor for Spreadsheet Formulas,” inIEEE VL/HCC, 2019, pp. 55–63. [Online]. Available: https://doi.org/10.1109/vlhcc.2019.8818748

    2019

  31. [41]

    Gridlets: Reusing Spreadsheet Grids,

    N. Joharizadeh, A. Sarkar, A. D. Gordon, and J. Williams, “Gridlets: Reusing Spreadsheet Grids,” inACM SIGCHI Extended Abstracts, 2020, pp. 1–7. [Online]. Available: https://doi.org/10.1145/3334480.3382806

    2020

  32. [42]

    Norman,The Design of Everyday Things

    D. Norman,The Design of Everyday Things. Basic books, 1988

    1988

  33. [43]

    Cognitive Dimensions of Notations,

    T. R. Green, “Cognitive Dimensions of Notations,”People and comput- ers V, pp. 443–460, 1989

    1989

  34. [44]

    Does Automation Bias Decision-Making?

    L. J. Skitka, K. L. Mosier, and M. Burdick, “Does Automation Bias Decision-Making?”International Journal of Human-Computer Studies, vol. 51, no. 5, pp. 991–1006, 1999. [Online]. Available: https://doi.org/10.1006/ijhc.1999.0252

    1999

  35. [45]

    Patina: Dynamic Heatmaps for Visualizing Application Usage,

    J. Matejka, T. Grossman, and G. Fitzmaurice, “Patina: Dynamic Heatmaps for Visualizing Application Usage,” inACM SIGCHI, 2013, pp. 3227–3236

    2013

  36. [46]

    Data Wrangler Extension for Visual Studio Code,

    Microsoft, “Data Wrangler Extension for Visual Studio Code,” https://marketplace.visualstudio.com/items?itemName=ms-toolsai. datawrangler, 2023, VS Code Extension

    2023

  37. [47]

    Tasks and Visualizations Used for Data Profiling: A Survey and Interview Study,

    R. A. Ruddle, J. Cheshire, and S. J. Fernstad, “Tasks and Visualizations Used for Data Profiling: A Survey and Interview Study,”IEEE TVCG,

  38. [48]

    Surfacing Visualization Mirages,

    A. McNutt, G. Kindlmann, and M. Correll, “Surfacing Visualization Mirages,” inACM SIGCHI, 2020, pp. 1–16. [Online]. Available: https://doi.org/10.1145/3313831.3376420

    2020

  39. [49]

    Visegpt: Towards Better Alignment of Llm-Generated Data Wrangling Scripts and User Prompts,

    J. Zhu, X. Cheng, Z. Luo, Y . Zhou, X. Shu, D. Weng, and Y . Wu, “Visegpt: Towards Better Alignment of Llm-Generated Data Wrangling Scripts and User Prompts,” inACM UIST, 2025, pp. 1–16. [Online]. Available: https://doi.org/10.1145/3746059.3747689

    2025

  40. [50]

    Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research,

    B. A. Becker, P. Denny, R. Pettit, D. Bouchard, D. J. Bouvier, B. Harrington, A. Kamil, A. Karkare, C. McDonald, P.-M. Osera, J. L. Pearce, and J. Prather, “Compiler Error Messages Considered Unhelpful: The Landscape of Text-Based Programming Error Message Research,”Working Gr...

    2019

  41. [51]

    Excelint: Automatically Finding Spreadsheet Formula Errors,

    D. W. Barowy, E. D. Berger, and B. Zorn, “Excelint: Automatically Finding Spreadsheet Formula Errors,”ACM OOPSLA, vol. 2, pp. 1–26,

  42. [52]

    Jupyterlab in Retrograde: Contextual Notifications That Highlight Fairness and Bias Issues for Data Scientists,

    G. Harrison, K. Bryson, A. E. B. Bamba, L. Dovichi, A. H. Binion, A. Borem, and B. Ur, “Jupyterlab in Retrograde: Contextual Notifications That Highlight Fairness and Bias Issues for Data Scientists,” inACM SIGCHI, 2024, pp. 1–19. [Online]. Available: https://doi.org/10.1145/3...

    2024

  43. [53]

    Untidy Data: The Unreasonable Effectiveness of Tables,

    L. Bartram, M. Correll, and M. Tory, “Untidy Data: The Unreasonable Effectiveness of Tables,”IEEE TVCG, vol. 28, no. 1, pp. 686–696,

  44. [54]

    Polite Computing,

    B. Whitworth, “Polite Computing,”Behaviour & Information Technol- ogy, vol. 24, no. 5, pp. 353–363, 2005

    2005

  45. [55]

    papermill,

    Interact, “papermill,” 2026, accessed: 2026-05-01. [Online]. Available: https://github.com/nteract/papermill

    2026

  46. [56]

    Exploration and explanation in computational notebooks,

    A. Rule, A. Tabard, and J. D. Hollan, “Exploration and explanation in computational notebooks,” inACM SIGHCHI, 2018, pp. 1–12

    2018

  47. [57]

    Leakage and the reproducibility crisis in machine-learning-based science,

    S. Kapoor and A. Narayanan, “Leakage and the reproducibility crisis in machine-learning-based science,”Patterns, vol. 4, no. 9, 2023

    2023

  48. [58]

    Overreliance on Ai Literature Review,

    S. Passi and M. V orvoreanu, “Overreliance on Ai Literature Review,” Microsoft Research, vol. 339, p. 340, 2022

    2022

  49. [59]

    Vibe Coding: Programming Through Con- versation with Artificial Intelligence,

    A. Sarkar and I. Drosos, “Vibe Coding: Programming Through Con- versation with Artificial Intelligence,” inPsychology of Programming Interest Group, 2025

    2025

  50. [60]

    The Impact of Generative Ai on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects from a Survey of Knowledge Workers,

    H.-P. Lee, A. Sarkar, L. Tankelevitch, I. Drosos, S. Rintel, R. Banks, and N. Wilson, “The Impact of Generative Ai on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects from a Survey of Knowledge Workers,” inACM SIGCHI, 2025, pp. 1–22. [Onli...

    2025

  51. [61]

    No Silver Bullet: Essence and Accidents of Software Engineering,

    J. Brooks, Frederick P., “No Silver Bullet: Essence and Accidents of Software Engineering,”Computer, vol. 20, no. 4, pp. 10–19, 1987. [Online]. Available: https://doi.org/10.1109/MC.1987.1663532

    1987

  52. [62]

    On the Design of Ai-Powered Code Assistants for Notebooks,

    A. M. McNutt, C. Wang, R. A. Deline, and S. M. Drucker, “On the Design of Ai-Powered Code Assistants for Notebooks,” inACM SIGCHI, 2023, pp. 1–16. [Online]. Available: https: //doi.org/10.1145/3544548.3580940

    2023

  53. [63]

    Empirical Analysis of Program- ming Language Adoption,

    L. A. Meyerovich and A. S. Rabkin, “Empirical Analysis of Program- ming Language Adoption,” inACM OOPSLA, 2013, pp. 1–18

    2013

  54. [64]

    We Entered the World of Microsoft Excel Esports & Got Our Asses Beat,

    People Make Games, “We Entered the World of Microsoft Excel Esports & Got Our Asses Beat,” 2026, accessed: 2026-05-01. [Online]. Available: https://www.youtube.com/watch?v=N2QC6VQXo8U Fig. 6. A version of Figure 4 with task 2 presented using a linear color map. Compared with t...

    2026

  55. [69]

    Connect" and enter the VNC Passcode(You can find it in the sidebar) 2.Click the toolbar, then click the button labeled A. Select the last option to send

    Task 1: Wrangling Game Rating Dataset:You are a data analyst at a gaming media outlet, and you need to analyze how User Scores have trended by Genre since the year 2000. The raw data is currently too messy for the pipeline. Please clean it using the assigned tool. In this task...

    2000

  56. [70]

    Car Value Retention Model

    Task 2: Wrangling Used Cars Dataset:You are a data analyst at a used car trading platform, and you need to prepare a dataset to train a "Car Value Retention Model" for various manufacturers. The raw data is currently too messy for the pipeline. Please clean it using the assign...

  57. [71]

    The variants differ only in how they handle missing values inUser_Score

    Silver Tables Generation Process for Game Dataset: a) Global Preprocessing:All silver tables share the following preprocessing steps: •Remove games released before 2000 •StandardizeUser_Score: –Convert non-numeric values such as‘‘tbd’’to missing values (NaN) –Cast the column t...

    2000

  58. [72]

    it’s been around forever

    Silver Tables Generation Process for Car Dataset:We note that the decision space for this task was relatively large. Decisions such as which columns to remove and how aggressively to handle missing or anomalous values often vary based on analyst judgment and prior experience. ...

    2012

  59. [2011]

    Available: https://doi.org/10.1177/1473871611415994

    [Online]. Available: https://doi.org/10.1177/1473871611415994

    doi

  60. [2018]

    Available: https://doi.org/10.1145/3276518

    [Online]. Available: https://doi.org/10.1145/3276518

    doi

  61. [2021]

    Available: https://doi.org/10.1109/tvcg.2021.3114830

    [Online]. Available: https://doi.org/10.1109/tvcg.2021.3114830

    2021

  62. [2023]

    Available: https://doi.org/10.1109/tvcg.2023.3234337

    [Online]. Available: https://doi.org/10.1109/tvcg.2023.3234337

    2023

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.