{"id":"26de86a1-4c0a-4aa6-971b-095768d72106","arxiv_id":"2607.14579","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Skeleton is a visual authoring tool that exposes the hidden navigation structure of accessible charts, and an eight-practitioner study finds this visibility shifts accessibility work from compliance toward design.","lead":"Skeleton turns the invisible navigation structure of accessible data visualizations into a visible, editable graph that practitioners can inspect and test. A study with eight practitioners suggests this visibility shifts accessibility work from a compliance task to a design problem.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The causal claim that visibility, rather than task novelty, tool novelty, or demand characteristics, changed practitioner engagement is underdetermined: the study lacks a control condition and relies on reconstructed notes.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: note-based evidence and the absence of a control condition. My concern narrows it to the causal attribution in the abstract. I considered whether a stronger internal problem exists—for example, that the Dimensions API is circular or that the lack of export functionality undermines the contribution—but those are not load-bearing for the stated claim. The paper is careful, self-aware, and includes independent support: longitudinal co-design grounding, deterministic scaffolding with a CV padding-recovery pass, and CD2's expert screen-reader evaluation that surfaced and fixed real bugs. These strengthen the system contribution but do not fix the engagement claim. A controlled replication with recordings is feasible and would settle the matter. Until then, CONDITIONAL remains the appropriate verdict; the current verdict already captures this, so no change is needed.","tokens_in":25917,"tokens_out":3660,"duration_ms":43210,"concrete_test":"Pre-register and run a controlled replication with, say, 16 new visualization practitioners (8 per arm, matched on self-rated accessibility expertise and role). Arm A uses Skeleton exactly as in Phase 2; Arm B performs the same task on the same bar chart using the Data Navigator code API with the Inspector graph (structure visible only as a developer tool, not manipulable over the chart). Record full audio/video and screen capture. Have independent raters blind to arm code three prespecified outcomes from the recordings: (1) number of structural revisions initiated without prompting, (2) unsolicited statements questioning the participant's own chart architecture, and (3) language coded as design-oriented vs compliance-oriented, with inter-rater reliability reported. If Arm A does not exceed Arm B on these outcomes, the causal attribution to visible, manipulable navigation structure fails","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim ('Making navigation structure visible changed how practitioners engaged with accessible design') is causal and specific: visibility caused a shift from compliance-oriented to design-oriented engagement. The evidence in Section 5 cannot support that causal attribution. Phase 2 (Section 5.2) gives every participant the same tool, an intentionally problematic default, and a think-aloud prompt to 'design an accessible navigation experience.' There is no arm in which navigation structure is invisible (e.g., code-only Data Navigator) or in which the same default is presented without the visible node-link graph. Participants' corrections of the extra numerical dimension may reflect simple responsiveness to an obvious UI defect, or expectations during think-aloud, rather than a changed orientation to accessibility. Section 5.3 admits that 'Participant quotes reported in the results are reconstructed from these researcher notes, not verbatim transcripts'; without recordings there is no audit trail, and thematic coding of notes written by the researchers who built the tool is vulnerable to selective emphasis. The design-probe framing (Section 8) explicitly disclaims evaluation of output quality, but the paper's leverage is the qualitative engagement claim, and that claim is exactly what the uncontrolled, note-based design cannot isolate. This is an evidence gap, not internal inconsistency: the authors' own Limitations anticipate part of it, but the gap is load-bearing because the headline result would need a controlled comparison to rule out confounds.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the problem that sighted practitioners who author accessible data visualizations build non-visual navigation structures (nodes, edges, input bindings) entirely in code, with no visual representation to inspect or iterate on. The authors propose that this invisibility prevents practitioners from developing judgment about navigation quality. They contribute: an Inspector that renders navigation graphs as interactive node-link diagrams; a Dimensions API that lets authors specify navigation in terms of data dimensions rather than hand-wiring every node and edge; and Skeleton, a direct-manipulation authoring environment with a dual-view editor, a scaffolding engine that repurposes Vega for automatic spatial placement, a live label-template editor with screen-reader preview, and a testing mode. The evaluation consists of longitudinal co-design across three projects (geologic map, Adobe's chart library, Bokeh) with 12 practitioners, plus an in-situ think-aloud study with 8 visualization practitioners using Skeleton as a design probe. The central qualitative claim is that making navigation structure visible changed how practitioners engaged with accessible design: they reconsidered the architecture of their own visualizations, attended to a broader range of input modalities, and shifted from treating accessibility as a compliance task to treating it as a design problem.","tokens_in":26131,"tokens_out":5599,"duration_ms":59612,"significance":"If the results hold, this is a significant and timely contribution to accessible data visualization. The open-source implementation and the longitudinal co-design with three communities are concrete strengths, as is the practice-based validation by a blind expert (CD2) reported in Section 4.3. The design-probe framing is appropriate, and the paper's own Limitations section anticipates several threats to validity. The main risk is that the abstract's causal claim is stronger than the evidence: the study is uncontrolled, note-based, and uses an intentionally problematic default in a task that may elicit the observed behavior regardless of visibility. A careful revision that tempers the causal language and makes the analysis transparent could make this a valuable paper.","major_comments":[{"comment":"The abstract's central claim—'Making navigation structure visible changed how practitioners engaged with accessible design'—is causal, but the study design cannot support it. In Phase 2 (Section 5.2) every participant used the same visible tool with an intentionally problematic default; there was no arm with invisible structure (e.g., code-only Data Navigator) or a neutral default. Participants' removal of the extra numerical dimension may reflect responsiveness to an obvious defect or demand characteristics during think-aloud, not a shift in orientation. I recommend either adding a comparison condition or, at minimum, reframing the abstract and conclusion as an exploratory design-probe finding rather than a demonstrated causal effect.","section":"Abstract, §5.2, §5.3"},{"comment":"The qualitative evidence rests on researcher notes, not verbatim transcripts; audio/video were not recorded. This creates a missing audit trail for the central themes, and thematic coding by researchers who built the tool risks selective emphasis. Quotes in Sections 6.1–6.5 are presented as evidence but are explicitly reconstructions. To support the load-bearing claim, the paper should provide analysis materials (e.g., anonymized notes, codebook, member checks) or explicitly mark quotes as illustrative and de-emphasize their role in establishing the shift.","section":"§5.3"},{"comment":"The claim that the scaffold 'dramatically improved authoring speed' rests on two pilot tests (a research team member: 8:22 vs. 0:56; a co-designer: 13:07 incorrect vs. 2:44 correct). These are single, non-controlled trials, one by a co-author. This is insufficient to support a quantitative speed claim and should be reported as anecdotal, not as evidence for DG3.","section":"§4.2 (Scaffolding speed)"},{"comment":"The Limitations section appropriately restricts the finding: Skeleton was used as a design probe, output quality was not evaluated, and end-user validation is future work. The abstract and conclusion, however, assert a causal change in engagement. To make the paper internally consistent, the abstract and conclusion should be revised to match the stated limitations, e.g., 'may change how practitioners engage' or 'our observations suggest...'.","section":"§8 vs. Abstract"}],"minor_comments":[{"comment":"Figure 6 is referenced before it appears; the figure numbering should be reordered or the reference should be to a later figure.","section":"§4.2"},{"comment":"The heading 'Alternative Dimensions' is unclear; consider renaming to 'Dimensions API' to match the section content.","section":"§3.4.2"},{"comment":"The opening sentence 'In this appendix subsection, we specifically outline the final work we did before publication of this project to improve it' is informal and first-person; align with the paper's academic style.","section":"Appendix B.3"},{"comment":"The sentence 'Coincidentally, none of our co-designers were crafting visualizations using Vega or Vega-Lite...' is odd; the coincidence is not relevant and the wording should be clarified.","section":"§4.2"},{"comment":"Participants' self-reported accessibility expertise on the 1–5 Likert scale is not reported anywhere in the results. Consider providing a participant table with demographics and expertise levels.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely on the right track, but the gap between the abstract's causal claim and the uncontrolled, note-based study is the main risk. I would not reject it; a careful revision that tempers the claims and makes the evidence base transparent could make it acceptable. The open-source artifacts and co-design work are valuable and should be highlighted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plain-English take: this is a real system and a genuinely new tool category—direct-manipulation authoring of navigation structures for accessible visualizations. The Inspector and Dimensions API are concrete and well documented, with a full appendix showing generated nodes, edges, and keyboard rules. The scaffolding engine that repurposes Vega as a coordinate oracle is clean, and the deterministic padding-estimation CV pass is a nice touch. The co-design work is substantial: three longitudinal engagements with outside teams, including blind co-designers, and the five design goals are grounded in those projects.\n\nThe paper is honest. It explicitly says quotes are reconstructed from researcher notes, not verbatim transcripts; that there was no control condition; that expert validation was in-house; and that the speed improvement figures come from two informal pilot runs. The Limitations section reads like the authors know where the weak points are.\n\nThe soft spot is the central claim in the abstract: 'Making navigation structure visible changed how practitioners engaged with accessible design... shifted from compliance to design.' That causal statement is not established by the study. Every participant used the same tool with a visible node-link graph; there was no condition where the navigation structure was invisible, and no arm that controlled for tool novelty, task novelty, or demand characteristics. The intentionally problematic default (the extra numerical dimension) may explain much of the behavior without invoking a changed orientation to accessibility. The lack of recordings means there is no audit trail for the quotes, and note-based coding by the researchers who built the tool can drift toward confirmation. The speed numbers are anecdotes.\n\nI want to be clear: these are evidence gaps, not internal flaws. The design-probe framing in Section 8 is appropriate, and the authors already concede that they evaluated whether visibility stimulated design consideration, not whether the resulting designs were good. What's missing is discipline in the abstract and conclusion, which assert the causal shift as a finding rather than as a suggestive pattern.\n\nWho is this for? Researchers and tool builders in accessible visualization; also anyone working on authoring tools for non-visual structure. The system itself, the Dimensions API, and the co-design findings are worth engaging with. It deserves a serious referee—the systems contribution alone justifies it. I would send it to peer review, with the expectation that the authors either soften the causal language or add a comparison condition in a future evaluation.\n\nRecommendation: engage with the work; cite it for the system and API; treat the qualitative finding as promising but unproven.","headline":"Skeleton is a solid systems contribution with a genuinely new authoring paradigm, but the headline causal claim about visibility changing practitioner engagement outruns the uncontrolled, note-based study.","tokens_in":26664,"tokens_out":3150,"would_cite":true,"duration_ms":34521,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Giving sighted chart authors a visible, editable navigation structure turns accessibility from compliance into design.","keywords":["accessibility","data visualization","navigation structure","direct manipulation","screen reader","authoring tool","non-visual design","co-design"],"falsifier":"Replicate the study with verbatim audio recording and an equivalent code-only control condition for the same charts. If practitioners in the hidden-structure condition show comparable self-directed iteration and chart reconsideration, or if the note-based findings do not reproduce, the claim that visible structure caused the shift is not supported.","tokens_in":25761,"feed_emoji":"🧭","tokens_out":5277,"duration_ms":53316,"temperature":0.7,"pith_summary":"The paper argues that sighted practitioners build accessible data visualizations blind: the navigation structure that assistive technologies traverse exists only as code, with no visual representation to inspect, debug, or iterate on. It introduces Skeleton, an authoring environment that renders navigation structure as an interactive node-link diagram overlaid on the chart, plus a Dimensions API that lets authors describe traversal in terms of data fields. In a study with eight practitioners, making structure visible changed how they worked: they caught structural errors, iterated on labels and topology, reconsidered the design of their own charts, and moved from asking 'does it pass' to asking 'is it good.' A sympathetic reader cares because this identifies a concrete lever—visible, manipulable structure—for improving the quality ceiling of non-visual data experiences.","feed_headline":"A visible navigation map turns chart accessibility into design","feed_subtitle":"Eight practitioners caught structural errors and rethought their charts once they could see the screen-reader structure.","key_machinery":"The central object is the navigation structure—the nodes, edges, focus states, and input bindings that govern how assistive technology traverses a chart. The argument is carried by Skeleton's dual representation: an abstract schema view of the hierarchy and a spatial canvas overlay on the chart image, linked bidirectionally, so a practitioner sees both the model and the experience. Scaffolding automates node placement by using a visualization rendering engine as a coordinate oracle, and a testing mode instantiates real keyboard navigation with focus highlighting, making traversal sequence visible. These techniques translate the invisible structure into objects a sighted author can perceive a","core_discovery":"The central claim is that invisibility, not lack of skill or care, is what keeps accessible navigation structure from being designed well. Skeleton makes the nodes, edges, spatial positions, and announced text of a navigation structure visible and directly manipulable, closing the feedback loop that sighted authors already have for every other aspect of a visualization. The study of eight practitioners found that with this representation, participants could see problems such as a redundant numerical dimension, refined labels by editing templates, restructured dimensions after keyboard testing, and five of eight reconsidered the architecture of their own charts—some concluding that a chart wa","pith_inferences":["If visibility is the causal lever, then adding an analogous inspector to code-only accessibility workflows (e.g., an accessibility tree viewer) should produce similar shifts; this is directly testable.","The paper leaves open whether the resulting designs are actually better for blind users; the authors say as much. A natural next step is to pair the visible-structure authoring loop with end-user evaluation of the produced structures.","The image-based workflow implies that any 2D graphic, not just charts with tabular data, can be given a navigable structure; this could extend accessible design to infographics, diagrams, and maps that currently lack tooling.","The 'designable vs. understood' distinction the authors draw suggests visibility alone is insufficient; mixed-ability co-design becomes more productive when both parties share the same manipulable representation."],"forward_implications":["Accessibility authoring tools should render non-visual structure as a first-class visual design material, not leave it as code.","With such a representation, practitioners can verify and debug navigation without manual screen-reader passes, shortening iteration loops.","Making structure visible prompts reconsideration of the visualization itself, linking non-visual design quality to visual design decisions.","The same principle should transfer to other domains where sighted authors build non-visual structure without feedback, such as PDF reading order, web page structure, and application layouts.","The Dimensions API suggests a grammar of navigation in data terms, which could make accessible navigation patterns reusable across chart types and libraries."],"fun_headline_variants":["Seeing the invisible: chart accessibility becomes design","Making screen-reader paths visible changes how charts are built","Visible navigation structure turns accessibility into a design challenge","When authors see the screen-reader graph, they redesign charts","Skeleton: visual authoring makes accessible charts a design task"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the observed shift from compliance-oriented to design-oriented engagement was caused by making navigation structure visible—an attribution based on researcher notes rather than verbatim transcripts, with no control condition.","fun_headline_variants_meta":{"raw":{"variants":["Seeing the invisible: chart accessibility becomes design","Making screen-reader paths visible changes how charts are built","Visible navigation structure turns accessibility into a design challenge","When authors see the screen-reader graph, they redesign charts","Skeleton: visual authoring makes accessible charts a design task"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1222,"prompt_tokens":797,"completion_tokens":425,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":347}},"tokens_in":541,"tokens_out":425,"duration_ms":4464,"temperature":1.0,"reasoning_tokens":347,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T01:39:59.256166+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replicate the study with verbatim audio recording and an equivalent code-only control condition for the same charts. If practitioners in the hidden-structure condition show comparable self-directed iteration and chart reconsideration, or if the note-based findings do not reproduce, the claim that visible structure caused the shift is not supported.","supporting_citations":[],"review_version":1}