{"id":"b017cf5d-f5df-4278-bd52-ba60a9b87526","arxiv_id":"1908.07544","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An interactive ASCII DAG visualization called graphterm is preferred by command-line users over existing GraphViz PDF and git-style ASCII views of package dependencies, despite more errors and lower confidence.","lead":"The authors built graphterm, an interactive ASCII drawing of software dependency graphs that runs inside a terminal, and tested it with 15 command-line users. Most participants preferred it over the existing GraphViz PDF and git-style ASCII views, even though the ASCII views produced more errors and lower confidence.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'viable' claim rests on a post hoc attribution of graphterm's extra errors to insufficient training, which the study never measures.","rationale":"The reader's conditional verdict already lists the post hoc error attribution as condition (1), so my concern is not a new verdict; it strengthens the existing condition and identifies it as the most load-bearing soft spot. I do not adopt the reader's weakest_assumption about the remote scenario as the primary concern: adding file-transfer or X11-forwarding overhead to the GraphViz PDF workflow would, if anything, plausibly increase preference for a terminal-native tool, so that limitation is more likely to understate than overstate the effect. The error attribution is different: it is the only evidence that graphterm's accuracy is acceptable, and it is an unfalsified causal claim presented as fact in the discussion. The paper deserves credit for disclosing the limitation in Sec. 6.3 and for reporting raw counts, but the leap from 'may indicate insufficient training' to 'the error rate in practice may be comparable' is exactly where the viability part of the central claim is least secure. The concrete test I propose would settle whether the six errors attributed to training actually disappear once comprehension is verified, or whether they reflect a property of the ASCII visualization itself.","tokens_in":24623,"tokens_out":12623,"duration_ms":579324,"concrete_test":"Run a small follow-up (or re-contact the existing participants if logs exist) with a comprehension check inserted immediately after the graphterm training block: display a 5-node ASCII DAG and ask two direction questions, e.g., 'Which node is a dependency of X?' and 'Which node depends on X?', before any timed trials. Then analyze graphterm error rates only for participants who answer both comprehension questions correctly. If these participants still commit significantly more graphterm errors than GraphViz errors, the 'insufficient training' attribution is falsified and the viability claim must be re-scoped. If the authors can instead provide per-participant training-phase logs showing that P5 and P6 failed equivalent direction questions in the tutorial, that would settle the attribution without a new study.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has a preference component and a viability component. Preference is supported by the in-sample counts (11/15 direct, 9/15 free-choice). Viability, however, depends heavily on the argument in Sec. 6.2 that graphterm's accuracy deficit is largely a training artifact: 'two of the graphterm errors can be attributed to misread questions and six to insufficient training (see Sec. 6.3 below), indicating the error rate in practice may be comparable.' But Sec. 6.3 only says the error pattern 'may indicate' insufficient training; no measure of training comprehension was taken. The training was a short explanation plus two interactive practice questions per tool, with no check that participants understood that vertical position implies edge direction in ASCII renderings. Participants P5 and P6 made six of the ten graphterm errors and four of the 23 git-like errors, which is equally consistent with an intrinsic difficulty of the ASCII representation (e.g., the o-node and diagonal/vertical mark conventions are not self-explanatory) rather than a correctable training gap. If those six errors are real, graphterm's error count is 8 versus 3 for GraphViz, and the paper's main accuracy mitigation collapses. Preference can coexist with lower accuracy, but the paper explicitly uses the training attribution to argue that graphterm is a viable alternative, so this post hoc causal explanation is load-bearing.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents graphterm, an interactive ASCII DAG visualization for the Spack package manager, together with a task analysis based on interviews with Spack maintainers, a layout adaptation of Tulip's hierarchical placement to an ASCII grid, and a within-subjects user study (n=15) comparing graphterm against Spack's existing git-like ASCII graph and the GraphViz-rendered PDF workflow. The quantitative results show that ASCII workflows are faster (with a weak overall effect, p<0.1, and a significant post-hoc git-like vs. GraphViz difference), less accurate, and lower in confidence than GraphViz PDFs. The central preference finding is that 11 of 15 participants preferred graphterm directly and 9 of 15 used only graphterm in the free-choice block, including all three Spack-experienced participants. The paper argues from these results that graphterm is a viable alternative for command-line users despite the accuracy and confidence costs.","tokens_in":24789,"tokens_out":5754,"duration_ms":56768,"significance":"The paper makes a concrete, falsifiable empirical claim: interactive ASCII dependency graphs can be preferred in command-line workflows even when they carry accuracy and confidence costs. The in-sample preference result is well supported, the study is reported with appropriate statistical hedging on the weaker effects, and the authors are candid about limitations. Strengths include the open-source graphterm tool, the systematic GitHub survey of dependency visualization practice, the task-analysis grounding, and the honesty about participant-recruitment constraints. The main significance is conditional on scoping: the measured preference is for a local or pre-configured terminal setting, and the broader 'viable alternative' claim depends on a post hoc attribution of graphterm's extra errors to insufficient training rather than intrinsic ASCII difficulty. Neither issue invalidates the preference result, but both need to be addressed before the paper's central conclusion can be accepted at face value.","major_comments":[{"comment":"The claim in Sec. 6.2 that 'the error rate in practice may be comparable' is load-bearing for the paper's viability conclusion, but it rests on an unmeasured causal attribution. The paper classifies six of the ten graphterm errors as due to insufficient training, yet Sec. 6.3 only observes that the error pattern 'may indicate' insufficient training; no training-comprehension check was administered. The same participants (P5 and P6) committed 6 of the 10 graphterm errors and 4 of the 23 git-like errors, which is equally consistent with an intrinsic difficulty of the ASCII orientation convention (e.g., the o-node and diagonal/vertical mark conventions) rather than a correctable training gap. I ask the authors to either (a) report the error comparison with and without P5, P6, P7, and P10, (b) add an objective training-comprehension measure, or (c) remove the 'may be comparable' claim and reframe the contribution around the supported preference result. Note also that the accounting in Sec. 6.2 is incomplete as written: 2 + 6 = 8, not 10, of the graphterm errors are accounted for by the stated attributions.","section":"Sec. 6.2 (Discussion) and Sec. 6.3 (Limitations)"},{"comment":"The motivating scenario of the paper is remote command-line access, where file transfer and X11 forwarding impose overhead (Sec. 3.2, Workflow). However, Sec. 6.3 states that participants ran locally or were pre-warned to enable X11 forwarding, and that 'participants did not experience the scenario where a separate login operation or file copy was required to view a PDF or image file.' This removes exactly the cost that the proposed tool is designed to eliminate. The in-sample preference is not invalidated, but the abstract's claim that participants preferred graphterm 'when approached from a command line interface workflow' should be scoped to local or pre-configured terminals; as written, the paper over-generalizes the preference to the remote scenario that motivates it. A remote-condition study or a more conservative statement of scope is needed.","section":"Sec. 6.3 (Effect of Study Setup on Response Time)"}],"minor_comments":[{"comment":"In the paragraph 'ASCII Visualizations Resulted in Lower Response Confidence,' the reported p-value 'p < 0.5' should read 'p < 0.05' to be consistent with the chi-square test used and with the surrounding language.","section":"Sec. 6 (Results and Analysis)"},{"comment":"In reference [8], the journal title is printed as 'IEEE Transactions on Visualization and Computer Grahpics'; this should be corrected to 'Graphics.'","section":"References"},{"comment":"The sentence claiming the git-like layout is 'an unambiguous representation: when one edge is routed into another, they both exit at the same terminus' is unclear; if two edges share routing, the representation may still be ambiguous to a viewer, so this phrasing should be revised.","section":"Sec. 3.1 (Visualizing Dependencies in Spack)"},{"comment":"The x-axis label 'Maximum Bracket Lengths (Labels)' would be clearer as 'Maximum number of labels in a bracketed list,' since the histogram measures label counts rather than physical lengths.","section":"Fig. 11"}],"recommendation":"major_revision","confidential_remarks":"To the editor: This is a reasonable design-study contribution with a well-supported in-sample preference result. The main gap is between that result and the broader viability/remote-scenario claims, which are fixable with robustness analyses and careful scoping rather than requiring rejection. The author overlap with the Spack maintainer community is disclosed and is a natural part of such a design study; I do not see it as a disqualifying conflict. The paper fits the scope of TVCG."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a solid HCI study with a real new artifact, and the central preference claim survives a skeptical read. The numbers are there: 11/15 direct preference, 9/15 used only graphterm in the free-choice block, all three Spack-experienced users preferred it. The paper is honest about the trade-offs. What is softer is the secondary argument that graphterm is \"viable\" despite the accuracy deficit. That rests on a post hoc attribution of six of ten graphterm errors to insufficient training, and the study never actually measures whether participants understood the edge-direction convention. It could be true, but it is speculation; P5 and P6's error pattern is equally consistent with the ASCII representation being genuinely harder. Remove that attribution and graphterm's error count is 8 vs 3, which is a real concern for build decisions.\n\nWhat's new: graphterm is, as far as I can tell, the first interactive ASCII DAG visualizer aimed at package dependency graphs. The layout-to-ASCII conversion heuristics (edge bundling, rerouting, glyph priority) are described as algorithms and the code is on GitHub, so the recipe is reusable. The GitHub survey is a useful snapshot of what dependency visualizations actually look like in practice. The task analysis is modest but sensible.\n\nThe study has real strengths. The within-subjects design, randomized order, free-choice block, and command-line-centered interface are appropriate. The statistical claims are properly hedged; the time effect is flagged as weak, the error and confidence effects as significant. The limitations section is unusually candid: the paper states that participants did not experience the remote file-copy scenario that motivates the work. That is the right call, but it also means the headline preference result is strictly about local command-line use. The motivating remote scenario remains untested.\n\nMy main requests for a revision: (1) replace the training attribution with a direct check of comprehension, or drop the \"comparable error rate\" wording; (2) either run a remote-condition variant or re-scope the conclusion to local use; (3) justify the exclusion criteria and the 300-second cutoff a priori. These are fixable. The core result—that interactive ASCII is preferred by command-line users despite its limitations—is well supported in-sample.\n\nI would send this out. It deserves a serious referee, and with those revisions it would be a useful reference for anyone working on terminal-based visualization or package management UX.","headline":"Solid preference result in a well-run study; the training-attribution argument for error parity is post hoc and needs a check or a softer claim.","tokens_in":25436,"tokens_out":2178,"would_cite":true,"duration_ms":146867,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Interactive ASCII dependency graphs win over command-line users in a head-to-head study","keywords":["ASCII visualization","package dependency graphs","command line interface","DAG visualization","graphterm","Spack","layered graph layout","user study"],"falsifier":"A remote-session study would settle it: participants log into a cluster through a plain terminal, must generate and view the Spack dot output as a PDF on their own, including any file transfer, and then repeat the same task battery; if graphterm's preference margin disappears or reverses under that overhead, the central claim fails.","tokens_in":24299,"feed_emoji":"⌨️","tokens_out":5922,"duration_ms":57804,"temperature":0.7,"pith_summary":"The paper aims to show that package-dependency analysis can stay inside the command line: an interactive ASCII drawing of a dependency DAG is workable and, for terminal-centric users, preferable to the usual generate-a-PDF route. To that end the authors built graphterm, which renders a layered layout using only the characters {|, -, /, \\, X, o} and adds search, highlight, and panning interactions modeled on tools like less. In a within-subjects study of 15 command-line users, 11 said they preferred graphterm over Spack's git-like ASCII view and the GraphViz-PDF workflow, 9 used only graphterm in a free-choice block, and all three Spack-experienced participants preferred it, even though ASCII workflows produced more errors and lower confidence. The conclusion is that for this audience the convenience of staying in the terminal outweighs the visual limitations of ASCII.","feed_headline":"Interactive ASCII graphs beat PDF views for command-line users","feed_subtitle":"Eleven of 15 users preferred the terminal-only dependency graph despite lower accuracy.","key_machinery":"The load-bearing object is graphterm, an interactive terminal renderer that converts a layered (Sugiyama-style) graph layout, which arranges nodes in rows so edges flow in one direction, into a compact ASCII grid using only the glyphs {|, -, /, \\, X, o}. The conversion begins from the Tulip hierarchical layout, then re-routes and bundles crossing diagonal edges to keep the grid compact, maps layout positions to a discrete grid, and places labels either beside nodes or in a bracketed right-side list. Interactivity, including slash-search, n/p node traversal, arrow-key or WASD panning, and two highlighting modes for direct versus all reachable neighbors, resolves the edge ambiguity that the compact ASCII drawing introduces.","core_discovery":"The central claim, stated on the paper's own terms, is that interactive ASCII visualization is a viable alternative to graphical dependency views for command-line users, and that users prefer it when the whole workflow happens in the terminal. The study found a weak effect of visualization on response time, with the git-like ASCII workflow faster than the GraphViz-PDF workflow (p < 0.05 post hoc), while both ASCII workflows produced more errors and lower confidence than the PDF workflow. Preference did not track accuracy: eleven of fifteen participants reported preferring graphterm, nine used only graphterm during the free-choice block, and all three Spack-experienced participants chose it. The authors interpret this as evidence that users accept a sub-optimal visualization when it is convenient to their workflow, but also note the tradeoff is not absolute, since the git-like ASCII view was not preferred.","pith_inferences":["If the preference persists on real remote logins, which the study did not test, graphterm-style views could plausibly become the default dependency view on HPC login nodes, where file transfer is the dominant cost.","The study's neutralization of PDF overhead may bias the comparison toward GraphViz; a remote-session replication with unmodified login and file-copy steps could reveal an even larger preference for graphterm.","The six-character glyph set and bundling strategy suggest a readability-versus-compactness trade-off worth measuring directly; varying the degree of bundling across graphs while holding tasks fixed could identify when ambiguity starts costing accuracy.","Because most Spack dependency graphs have under 50 nodes, the ASCII approach is well matched to the data; applying it to larger graphs would require aggregation or focus+context, not just layout tuning."],"forward_implications":["Spack can ship graphterm as a third graph view, letting users inspect dependency topology without leaving the shell or copying files.","Other terminal-bound analysis tools, such as build logs, profiler output, or git histories, can adopt the same pattern: interactive ASCII for quick topology questions, with a high-fidelity export reserved for precise reading.","Because the git-like ASCII view was the least preferred despite being fastest, the result is not 'ASCII is good' but 'interactive, compact, terminal-native interaction is what users reward'.","The higher error rate in ASCII workflows means package managers should keep a precise graphical view for decisions where a missed dependency would break a build; graphterm is better framed as the quick-look tool.","The paper attributes several graphterm errors to training and question misreading, so improving the interactive instructions could narrow the accuracy gap in actual deployments."],"supporting_citations":[{"why":"Defines Spack and its dependency-graph workflow, the motivating application and the source of the graphs used in the study.","marker":"[14]"},{"why":"Supplies the dot layered layout algorithm behind the GraphViz-PDF baseline workflow against which graphterm is compared.","marker":"[15]"},{"why":"Provides the Tulip hierarchical layout that graphterm converts to an ASCII grid.","marker":"[3]"},{"why":"Establishes the layered graph layout paradigm that the paper matches to users' conceptual model of dependencies.","marker":"[44]"},{"why":"Supplies the graph task taxonomy used to structure the task analysis and study questions.","marker":"[27]"},{"why":"Provides the evaluation taxonomy referenced to justify the study's design choices.","marker":"[40]"},{"why":"Contributes the linear mixed-effects modeling approach used to analyze response time, error rate, and confidence.","marker":"[28]"}],"fun_headline_variants":["ASCII dep graphs preferred by CLI users over PDF","Terminal users favor ASCII dependency graphs despite errors","Spack users choose interactive ASCII graphs for workflow","CLI workflow favors ASCII dep graphs, study finds","ASCII visualization preferred for CLI users, despite accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the measured preference transfers to the remote, GUI-free scenario that motivates the work: the study ran locally or with graphical forwarding pre-enabled, so participants never faced a separate login step, a file copy, or viewer launch lag when using the PDF route.","fun_headline_variants_meta":{"raw":{"variants":["ASCII dep graphs preferred by CLI users over PDF","Terminal users favor ASCII dependency graphs despite errors","Spack users choose interactive ASCII graphs for workflow","CLI workflow favors ASCII dep graphs, study finds","ASCII visualization preferred for CLI users, despite accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000643,"raw_usage":{"total_tokens":2942,"prompt_tokens":912,"completion_tokens":2030,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":1958}},"tokens_in":528,"tokens_out":2030,"duration_ms":14786,"temperature":1.0,"reasoning_tokens":1958,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:04:46.562921+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A remote-session study would settle it: participants log into a cluster through a plain terminal, must generate and view the Spack dot output as a PDF on their own, including any file transfer, and then repeat the same task battery; if graphterm's preference margin disappears or reverses under that overhead, the central claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Tulip hierarchical layout that graphterm converts to an ASCII grid."},{"cited_title":"Sensalire, P","cited_arxiv_id":null,"evidence_quote":"Provides the evaluation taxonomy referenced to justify the study's design choices."}],"review_version":1}