{"id":"8c1969d5-ab2c-4562-a9a4-f313a50148dd","arxiv_id":"2608.11581","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"RAGE-Vis generates editing panels with target fields and relation-derived affected fields, letting users edit charts from natural language while preserving cross-component consistency.","lead":"RAGE-Vis turns natural-language chart editing requests into interactive panels that also show related chart fields, so edits stay consistent across legends, labels, and marks. A user study with 32 participants suggests it reduces back-and-forth language requests compared with a widget-only baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Relation-template completeness is untested: the offline evaluation reports only a false-positive rate, never recall, so the central consistency claim rests on an unmeasured assumption.","rationale":"The reader's weakest assumption is precisely that the hand-authored relation template file contains the dependencies that matter, and Section 6.3 admits this is incomplete. This is the most load-bearing concern because every downstream benefit attributed to Affected Fields and coordinated controls depends on the relation analysis retrieving the right additional fields. The false-positive ratio reported in Section 5.2.3 is a precision-only metric and cannot detect missing relations; a broken or empty template would pass it. The fixed condition order and the combined removal of Design Presets and Affected Fields in the control condition further weaken attribution, but those issues are secondary to the unmeasured recall of the core mechanism. I do not see a reason to reject the paper; the system is a plausible and well-structured contribution with case studies, a user study, and an offline precision check. However, the conditional verdict is appropriate: the central consistency claim should not be accepted as established until recall over an independent gold set is reported and the relation-template scope is made explicit. My recommendation is therefore no change to the reader's CONDITIONAL verdict.","tokens_in":33019,"tokens_out":3438,"duration_ms":40950,"concrete_test":"Build an independent gold set of expected affected fields for the four user-study tasks and the two case-study charts by having at least two experts annotate, for each sub-intent, every field that should change together with the target field, explicitly including dependency types listed in Section 6.3 as omitted (cross-view consistency, high-level design constraints). Run RAGE-Vis's Coordinated Relationship Analysis on exactly those requests and compute recall@k, the fraction of gold affected fields surfaced, alongside the reported false-positive ratio. If recall is substantially below the precision figure, or below a pre-specified threshold such as 0.9 for the covered relation types, the consistency claim should be scoped to the covered relations and the user-study benefits should be re-examined under that scoping.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that RAGE-Vis maintains cross-component consistency by surfacing affected fields through typed relations. Section 4.2.2 states that Coordinated Relationship Analysis \"determines dependencies based on a relation template file,\" but the paper never reports what the template contains, how it was authored, or how complete it is. Section 5.2.3 validates affected-field expansion only by a false-positive ratio of 0.84%, which measures precision, not recall. A system that returned no affected fields at all would achieve a 0% false-positive rate, so this metric cannot support the claim that the relations that matter are actually retrieved. Section 6.3 explicitly concedes that the three predefined relation types omit complex dependencies such as cross-view consistency and high-level design constraints. If the template omits a dependency that is needed for the study's own tasks (e.g., subtitle text tied to the data range, or category color shared between marks and legend), the system silently fails to surface it, and the observed reductions in additional natural language requests and the high TQ6/TQ7 ratings may reflect only the covered subset of relations. The user study also removes Design Presets and Affected Fields together in the control condition, so the specific contribution of the relation-aware component cannot be separated from the contribution of alternative presets. The concern is not that the system does nothing useful; it is that the completeness of the relation model is both load-bearing and unmeasured.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents RAGE-Vis, a system for natural-language chart editing that converts bitmap or HTML charts into a parameterized intermediate representation, decomposes composite and underspecified requests into sub-intents, and generates hierarchical editing panels containing Design Presets, Target Fields, and Affected Fields. Cross-component dependencies are modeled as three typed field-level relations: visual encoding relations, structural relations, and expressive consistency relations. The authors report two case studies (a mirrored bar chart and a multi-category radar chart) and a within-subjects user study (N=32) comparing the full system with an ablation that removes Design Presets and Affected Fields. The reported results include fewer additional natural-language requests (4.28 vs. 6.25, p<.001), lower accumulated model response time, comparable objective final-chart quality (8.20 vs. 8.36), and higher subjective ratings for alternative comparison (TQ4), related-component discovery (TQ6), and consistency maintenance (TQ7). An offline pipeline evaluation reports an intent-decomposition precision of 82.02% and an affected-field false-positive ratio of 0.84%.","tokens_in":33261,"tokens_out":6928,"duration_ms":66133,"significance":"If the central claims hold, RAGE-Vis would be a useful step toward letting non-experts express composite, underspecified chart-editing goals in natural language while keeping cross-component consistency inspectable and controllable. The paper's conceptual contributions—the decomposition of editing into sub-intents, the display of affected fields with dependency reasons, and the three relation types—are clearly articulated and align with real design requirements. The user study is more rigorous than many systems papers in reporting effect sizes and Holm-adjusted p-values, and the two case studies demonstrate the system on structurally different charts. The offline precision of affected-field expansion is a positive signal, and the authors deserve credit for identifying the setting of the study honestly, including a fixed condition order and the proprietary-model dependency. However, as detailed below, the evidence for the load-bearing relation-aware contribution is incomplete, and two experimental confounds make the current comparative claims stronger than the design supports.","major_comments":[{"comment":"The central claim that relation-aware expansion maintains cross-component consistency is not supported because the offline evaluation measures only a precision-side metric, never recall. The paper states that Coordinated Relationship Analysis \"retrieves relevant fields and determines dependencies based on a relation template file,\" but the contents, size, authoring process, and coverage of that template are not reported. The affected-field false-positive ratio of 0.84% is consistent with a system that returns very few or no affected fields, so it cannot establish that the fields that should change together are actually retrieved. Section 6.3 explicitly concedes that the three predefined relation types omit complex dependencies such as cross-view consistency and high-level design constraints. The paper should report recall or a ground-truth completeness analysis over a labeled set of editing requests, and it should describe the relation template and its coverage; without this, the observed reductions in natural-language requests and the high TQ6/TQ7 ratings may reflect only the subset of relations covered by the template.","section":"Section 4.2.2 / Section 5.2.3"},{"comment":"The within-subjects user study is confounded by a fixed system order: \"all participants used RAGE-Vis before the control condition.\" Learning effects, task familiarity, and fatigue in the second condition can reduce the number of additional natural-language requests, accumulated model response time, and subjective workload independently of the interface's merits. The authors invoke the fixed order to explain the slightly lower objective quality of the full system (8.20 vs. 8.36), but if order can plausibly explain that decrement, it can equally explain the observed efficiency and subjective gains. The comparative claims require counterbalancing of system order or, failing that, a quantitative order-effect analysis showing that the measured differences are not explained by session position.","section":"Section 5.2.1 / Section 5.2.2"},{"comment":"The ablation removes Design Presets and Affected Fields simultaneously, so the user-study results cannot isolate the contribution of the relation-aware Affected Fields component from that of Design Presets. For example, TQ4 (alternative comparison) is naturally driven by Design Presets, while TQ6 and TQ7 are intended to measure the affected-field mechanism; a reduction in the number of natural-language requests could plausibly be caused by the presets alone. The paper should add a condition that removes only one of the two components, or it should use per-panel interaction logs to show that the affected-field widgets themselves, rather than the preset alternatives, produce the measured benefits.","section":"Section 5.2.1 / Section 4.3"},{"comment":"The offline pipeline evaluation is too thinly reported to support the reliability conclusions drawn from it. Intent-decomposition is summarized only as a precision of 82.02%, with no recall, no inter-annotator agreement measure, and no error taxonomy, even though two experts were used for annotation. The affected-field expansion is summarized only by a false-positive ratio, which, as noted above, says nothing about missed affected fields. Without recall, an inter-annotator agreement statistic, and a description of how the gold annotations were constructed, the reader cannot judge whether the pipeline reliably finds the intents and dependencies that the user study depends on.","section":"Section 5.2.3"}],"minor_comments":[{"comment":"The text contains long corrupted glyph sequences (the \"/uni000000...\" strings) inside the figure captions and between the numerical results in Section 5.2.2, making part of the quantitative results unreadable. The manuscript source should be regenerated so that the captions and the total-task-time values appear as normal text.","section":"Figures 5 and 6 / Section 5.2.2"},{"comment":"The total task time for the full-system condition is reported as \"13 min 51 s±\" and is then interrupted by the corrupted glyph block; the mean and standard deviation should be stated explicitly in readable form.","section":"Section 5.2.2"},{"comment":"The expressive consistency relation is defined as an intent-specific relation template, but no concrete example of such a template or of an instantiation is provided; a small worked example would greatly improve reproducibility and reader understanding.","section":"Section 4.1.3"},{"comment":"The claim that the workflow \"could be transferred\" to Vega-Lite, ECharts, SVG, infographics, and interface editing is plausible but currently unsubstantiated; consider either providing a concrete transfer argument or softening the claim to a research opportunity.","section":"Section 6.2"},{"comment":"Several bibliography entries contain awkward line breaks inside DOIs (for example, references [2], [5], and [13]); these should be cleaned up in the camera-ready version.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and makes a plausible systems contribution, but the evidence for the relation-aware component is the main weakness. The most load-bearing gaps are the absence of recall in the offline evaluation of affected-field expansion, the fixed system order, and the ablation that removes both Design Presets and Affected Fields at once. These are fixable in a revision with additional experiments or a reanalysis of existing logs, so I do not recommend rejection. I also suggest ensuring that the final PDF does not contain the corrupted glyph sequences seen in Figures 5/6 and Section 5.2.2."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know first: this is a solid systems paper with a real new idea—moving NL chart editing from single-widget mappings to dependency-aware panels via typed field-level relations (visual encoding, structural, expressive consistency). It also operates on bitmap charts, which is practically useful. But the headline claim about maintaining cross-component consistency has a load-bearing evaluation gap: the offline affected-field metric is only a false-positive ratio (0.84%), never recall. A system that returned no affected fields would score a perfect 0% false-positive rate. So the completeness of the hand-authored relation template file is never actually tested.\n\nWhat it does well: RAGE-Vis decomposes composite requests, generates design presets for underspecified intents, and exposes affected fields with brief reasons in inspectable widgets—a clean extension of DynaVis-style dynamic widgets. The case studies on mirrored bar and radar charts demonstrate coordinated edits convincingly. The user study with 32 participants shows large subjective gains on alternative comparison (TQ4, r_rb=.90), related-component discovery (TQ6, r_rb=.94), and consistency maintenance (TQ7, r_rb=.97), plus significantly fewer NL requests (4.28 vs 6.25, p<.001). Offline intent-decomposition precision of 82% supports the parsing pipeline.\n\nSoft spots, in proportion: the fixed condition order (RAGE-Vis always first) is a real confound, and the paper itself admits a learning effect may explain the comparable objective quality (8.20 vs 8.36). The control condition removes Design Presets and Affected Fields together, so the specific contribution of the relation-aware component is not separated from the presets. There are no released artifacts, and the relation templates are authored by the same team and validated only by their own experts. Section 6.3 honestly concedes that the three relation types omit complex dependencies like cross-view consistency—honest, but also an admission that the consistency guarantee is scoped to whatever the template file covers.\n\nNone of this kills the contribution. For the charts and tasks tested, the system clearly works, and the subjective effects are large and internally consistent. But the paper would be much stronger with a recall-oriented evaluation of relation retrieval, an artifact release, and a counterbalanced study that separates presets from affected fields.\n\nWho this is for: visualization and HCI researchers working on natural-language interfaces and generative UI. It deserves a serious referee. I'd send it out with requests for those three additions; this is a conditional accept, not a desk reject.","headline":"A genuinely useful systems paper on relation-aware chart editing whose central consistency claim rests on an unmeasured relation-template completeness assumption, but the contribution still warrants serious peer review.","tokens_in":33806,"tokens_out":1613,"would_cite":true,"duration_ms":18390,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RAGE-Vis turns vague, multi-part chart editing requests into coordinated, inspectable control panels that keep related chart elements in sync.","keywords":["chart editing","natural language interfaces","generative user interfaces","large language models","cross-component consistency","visual encoding relations","user study","chart image reconstruction"],"falsifier":"Run a broad set of chart edits (style change, data insertion, legend rearrangement, axis rescaling) across varied chart types and measure whether the Affected Fields panel omits a component that visibly must change for the chart to remain coherent; if such omissions occur regularly, the template-based relation model is incomplete. Concretely, rescale the y-axis of a dual-axis chart and check whether the second axis, its gridlines, and any linked annotations are exposed as affected fields.","tokens_in":32791,"feed_emoji":"📊","tokens_out":4201,"duration_ms":40894,"temperature":0.7,"pith_summary":"RAGE-Vis claims that the right way to handle vague, multi-part chart editing instructions is to convert them into interactive panels that expose both the fields the user asked about and the fields those edits will silently disturb. The paper argues that existing natural-language chart tools fail when a request such as \"make the chart more coordinated\" touches marks, legends, labels, and titles at once. Its proposed system decomposes such requests into sub-intents, generates candidate presets for underspecified goals, and uses a typed relation model over chart fields to surface cross-component dependencies as adjustable widgets. A user study with 32 participants reports that this design significantly cuts the number of follow-up language requests (4.28 vs 6.25) and accumulated model response time, while preserving final chart quality. If the result holds, non-experts gain a practical route from loose intentions to coherent multi-component chart edits.","feed_headline":"Natural-language chart edits become coordinated control panels","feed_subtitle":"A relation-aware system cuts follow-up requests from 6.25 to 4.28 per task and keeps related chart elements in sync.","key_machinery":"The machinery is a typed field-level relation model over the chart's parameterized intermediate representation, resolved through a hand-authored relation template file. Each editable field carries source-data path, chart component, visual channel, semantic role, and dependency metadata; three relation types—visual encoding, structural, and expressive consistency—connect fields that should change together. This model is what lets Coordinated Relationship Analysis start from the target fields of a decomposed intent and return the Affected Fields that appear as coordinated widgets, turning cross-component consistency from an afterthought into a retrievable structure.","core_discovery":"The central claim is that chart components are not isolated editing targets: every field in a rendered chart participates in visual encoding relations (e.g., mark colors and legend swatches sharing a mapping), structural relations (e.g., data rows generating marks, legend height constraining plot area), and expressive consistency relations (e.g., a subtitle that states the data range should change when the range changes). RAGE-Vis encodes these as typed field-level relations in the chart's intermediate representation and, for each natural-language request, retrieves all fields connected to the direct targets. The system then organizes retrieved fields into Affected Fields panels with brief explanations, alongside Design Presets and Target Fields, so that a single high-level request becomes an inspectable, coordinated editing space instead of a single widget.","pith_inferences":["A natural next step is to expand the relation model beyond three categories, e.g., learned or instance-specific relations for cross-view consistency and design constraints, which the paper lists as future work; if effective, the same Affected Fields mechanism could serve multi-chart dashboards and infographics.","The measured reduction in follow-up requests suggests a general interaction principle: explicitly exposing the side effects of an edit can substitute for conversational repair turns in other generative-UI settings.","One testable extension is to vary the completeness of the relation template across conditions and measure how often users must manually correct a coordinated edit, which would quantify how much of the benefit comes from relation coverage versus panel design.","The system's reliance on a proprietary LLM for intent parsing means reproducibility and quality may shift with the underlying model; the offline intent-decomposition precision of 82.02% indicates headroom for validation rules that catch mis-parsed intents before panels are generated."],"forward_implications":["Users can issue one underspecified request and get several candidate schemes to compare rather than a single interpretation committed by the model.","Fewer additional natural-language clarifications are needed during a session, which also reduces accumulated model response time.","Editing operations propagate consistently across marks, legends, labels, titles, and annotations, so local fixes do not produce globally inconsistent charts.","Because the input can be a bitmap chart image reconstructed into editable code, existing static charts become editable without their original source files.","The workflow is not tied to D3 specifically, and the paper argues it can transfer to Vega-Lite, ECharts, SVG, or other structured visualization representations."],"supporting_citations":[{"why":"Supplies the dynamic-widget paradigm that the paper extends and that the control condition in the user study is inspired by.","marker":"[47]"},{"why":"Provides the target-matching evaluation protocol used to score final edited charts for correctness and preservation of non-target elements.","marker":"[61]"},{"why":"Represents the declarative visualization grammar tradition whose structured, parameterizable charts motivate the intermediate representation.","marker":"[36]"},{"why":"Establishes the ambiguity-management challenge in natural language interfaces for visualization that RAGE-Vis addresses through presets and panels.","marker":"[10]"},{"why":"Exemplifies style-reuse chart editing whose dependence on reference examples is contrasted with text-driven underspecified intent handling.","marker":"[54]"},{"why":"Demonstrates mixed-input natural language visualization authoring, an antecedent for combining language with interactive controls.","marker":"[51]"}],"fun_headline_variants":["Chart edits that keep related components in sync automatically","Single natural-language request becomes a coordinated editing panel","Relation-aware chart editing synchronizes cross-component changes","From one request to a full panel of coordinated chart edits","RAGE-Vis: connected chart parts update together from a single request"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The system's consistency guarantee depends on the hand-written relation template file containing all the dependencies that actually matter for a given editing task; the paper itself notes that its three predefined relation types omit complex dependencies such as cross-view consistency and high-level design constraints.","fun_headline_variants_meta":{"raw":{"variants":["Chart edits that keep related components in sync automatically","Single natural-language request becomes a coordinated editing panel","Relation-aware chart editing synchronizes cross-component changes","From one request to a full panel of coordinated chart edits","RAGE-Vis: connected chart parts update together from a single request"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000951,"raw_usage":{"total_tokens":4057,"prompt_tokens":947,"completion_tokens":3110,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":3032}},"tokens_in":563,"tokens_out":3110,"duration_ms":23153,"temperature":1.0,"reasoning_tokens":3032,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:33:56.019079+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a broad set of chart edits (style change, data insertion, legend rearrangement, axis rescaling) across varied chart types and measure whether the Affected Fields panel omits a component that visibly must change for the chart to remain coherent; if such omissions occur regularly, the template-based relation model is incomplete. Concretely, rescale the y-axis of a dual-axis chart and check whether the second axis, its gridlines, and any linked annotations are exposed as affected fields.","supporting_citations":[],"review_version":1}