{"id":"ae59e936-2b9e-4a03-b763-16b59b86f4bc","arxiv_id":"2605.25489","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"ATWL is a new domain-agnostic language that decomposes VA workflows into typed artifacts and intent-labeled transforms, enabling extraction of 17 workflows from papers, identification of structural regularities, and more structured LLM recommendations than prose descriptions.","lead":"The paper introduces ATWL, a declarative formal language using an ontology of eight artifact types and standardized transform intents to represent visual analytics workflows. A smart generalist might read it because it shows how turning narrative descriptions into structured, machine-readable forms could improve reuse of analytical strategies and integration with AI tools.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Sufficiency of the fixed eight-artifact ontology for arbitrary VA workflows is asserted but only demonstrated on the 17-paper library","rationale":"The reader's weakest_assumption already isolates the precise point on which the generality claim rests. The full-text description of the ontology and the LLM extraction experiment do not add an independent check of completeness, so the concern remains load-bearing and the provisional UNVERDICTED status is appropriate.","tokens_in":1761,"tokens_out":316,"duration_ms":17438,"concrete_test":"Take one VA paper published after the library construction and outside the original 17; have an independent analyst attempt to encode its workflow using only the published eight artifact types and intent vocabulary; record whether any analytical step or intent must be omitted, merged, or invented to fit the schema.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim requires that the chosen modular ontology (exactly eight artifact types plus fixed transform intents) captures structure and analytical intent of arbitrary VA workflows without material loss or distortion. The paper shows that 17 workflows extracted from published papers fit the ontology, that regularities become visible, and that an LLM produces more structured recommendations when given ATWL rather than prose. However, the extraction process itself presupposes that the ontology is adequate; no independent test is reported of a workflow that would require a ninth artifact type or additional intent. If such a workflow exists, the formal representation would either omit or distort the original analytical knowledge, undermining the transition from narrative to reusable formal knowledge.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces the Artifact-Transform Workflow Language (ATWL), a domain-agnostic declarative language for formally representing visual analytics workflows. ATWL is defined via a modular ontology of exactly eight artifact types (entities, features, arrangements, visualisations, patterns, models, knowledge, specifications) together with standardised transform intents (e.g., define-unit, characterise, contextualise, abstract). The authors describe an LLM-supervised extraction process that reduces human effort to review and refinement, apply it to construct a library of 17 workflows drawn from published VA papers, identify cross-workflow structural regularities invisible in prose, and report a controlled experiment in which the same LLM produces recommendations when supplied either the original papers or the ATWL library, claiming systematic advantages for the formal form in iteration structure, typed data flow, adaptation provenance, and compactness.","tokens_in":1950,"tokens_out":574,"duration_ms":30780,"significance":"If the ontology proves adequate, ATWL would support systematic comparison, reuse, and formalisation of analytical knowledge across visual analytics, with potential benefits for training and cross-domain transfer. The work merits explicit credit for the construction of a 17-workflow library and for the controlled LLM experiment that isolates differences in output structure and provenance between prose and formal representations. The absence of quantitative metrics in the experiment, however, limits the strength of the utility claims.","major_comments":[{"comment":"Abstract (ontology description): the central claim that ATWL enables a transition to formally represented, comparable, and reusable analytical knowledge without material loss or distortion rests on the assertion that the fixed eight-artifact ontology plus fixed transform intents is sufficient for arbitrary VA workflows. This is demonstrated only by successful extraction of the 17 workflows; the extraction process itself presupposes adequacy, and no independent test or search for a workflow requiring a ninth artifact type or additional intent is reported. If such a workflow exists, the representation would omit or distort the original intent, directly undermining the transition claim.","section":"Abstract (ontology description)"},{"comment":"Evaluation experiment (controlled LLM comparison): the experiment reports that ATWL representations yield more explicit iteration structure, typed data flow, fragment-level adaptation provenance, and compactness, yet supplies no quantitative metrics, controls, statistical tests, or raw data. Without these, it is impossible to assess whether the reported advantages are robust or sensitive to post-hoc choices in prompt construction or output interpretation.","section":"Evaluation experiment (controlled LLM comparison)"}],"minor_comments":[{"comment":"The manuscript would benefit from an explicit statement of the precise criteria used to decide that a workflow had been successfully captured by the eight artifact types.","section":"Extraction process"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed comments, which help clarify the evidential basis for our claims. We respond point-by-point to the two major comments below.","responses":[{"response":"We agree that the sufficiency of the eight-artifact ontology is foundational to the claim of lossless formalisation. The extraction of 17 workflows from diverse published VA papers, without requiring additional artifact types, supplies empirical support for adequacy within the sampled set. We acknowledge that this does not constitute an exhaustive or independent search for counterexamples. In revision we will add an explicit discussion of ontology extensibility (how a ninth type could be introduced modularly) and will qualify the central claim to state that the ontology has been shown adequate for the 17 extracted workflows rather than asserted as sufficient for all possible VA workflows.","revision_made":"partial","referee_comment":"[Abstract (ontology description)] Abstract (ontology description): the central claim that ATWL enables a transition to formally represented, comparable, and reusable analytical knowledge without material loss or distortion rests on the assertion that the fixed eight-artifact ontology plus fixed transform intents is sufficient for arbitrary VA workflows. This is demonstrated only by successful extraction of the 17 workflows; the extraction process itself presupposes adequacy, and no independent test or search for a workflow requiring a ninth artifact type or additional intent is reported. If such a workflow exists, the representation would omit or distort the original intent, directly undermining the transition claim."},{"response":"The referee is correct that the experiment is presented qualitatively and lacks quantitative metrics or statistical tests. The design intentionally emphasised observable structural differences (iteration explicitness, typed flows, provenance granularity) that are difficult to reduce to single scalar metrics. We will revise the section to include: (i) the precise prompts employed, (ii) tabulated counts of explicit iteration steps and data-flow connections across the paired outputs, and (iii) the full set of LLM responses as supplementary material. These additions will allow readers to evaluate robustness and sensitivity to prompt choices.","revision_made":"yes","referee_comment":"[Evaluation experiment (controlled LLM comparison)] Evaluation experiment (controlled LLM comparison): the experiment reports that ATWL representations yield more explicit iteration structure, typed data flow, fragment-level adaptation provenance, and compactness, yet supplies no quantitative metrics, controls, statistical tests, or raw data. Without these, it is impossible to assess whether the reported advantages are robust or sensitive to post-hoc choices in prompt construction or output interpretation."}],"tokens_in":1601,"tokens_out":532,"duration_ms":34163,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's core contribution is ATWL, a declarative language built on exactly eight artifact types and a fixed set of transform intents, plus an LLM-supervised pipeline that turns paper text into formal workflow representations. They applied it to 17 published VA papers, identified recurring meta-structures and motifs across them, and ran a controlled test where the same LLM produced recommendations from either the original papers or the ATWL versions.\n\nThe extraction process and the resulting library are new in this combination, and the cross-workflow analysis does surface regularities that prose descriptions hide. The experiment also shows concrete differences: ATWL versions led to outputs with explicit iteration, typed flows, and provenance that the prose versions lacked, plus better compactness for context limits.\n\nThe main limitation is that the ontology's sufficiency is only shown by successful extraction on those 17 cases. No workflow that would require a ninth artifact type or extra intent is tested, so the claim that the language supports reuse without material loss or distortion remains unexamined. The experiment description gives no quantitative scores, statistical tests, or raw data, which leaves the reported advantages hard to assess for robustness.\n\nThis work is for researchers in visual analytics and HCI who care about formalizing workflows for comparison and AI assistance. It deserves peer review because the language and extraction method are specific enough to evaluate, even if the evaluation needs strengthening on coverage and metrics.","headline":"ATWL gives a concrete ontology and extraction method for VA workflows that reveals some patterns, but its claim to capture arbitrary workflows rests only on fitting 17 examples with no test for coverage gaps.","tokens_in":2443,"tokens_out":363,"would_cite":false,"duration_ms":14914,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"ATWL formally represents visual analytics workflows as sequences of eight artifact types and standardized transform intents so they can be compared and reused.","keywords":["visual analytics","workflow representation","formal language","artifact ontology","transform intents","workflow reuse","LLM-assisted extraction","analytical knowledge"],"falsifier":"A published visual analytics workflow whose structure or intent cannot be expressed using only the eight defined artifact types and the listed transform intents without adding new types or intents.","tokens_in":2675,"feed_emoji":"🔄","tokens_out":708,"duration_ms":22374,"temperature":0.7,"pith_summary":"Visual analytics workflows are typically described in unstructured prose that hinders comparison, reuse, and training. ATWL supplies a declarative language that encodes both workflow structure and analytical intent through a fixed ontology of eight artifact types and transforms labelled by intent. The authors extract seventeen workflows from published papers via LLM assistance with human review, then demonstrate that the formal versions expose recurring meta-structures and motifs hidden in the original text. A controlled experiment shows that an LLM given the ATWL library produces recommendations containing explicit iteration, typed data flow, and adaptation provenance that the source papers alone did not supply as clearly.","feed_headline":"ATWL turns prose VA workflows into comparable formal objects","feed_subtitle":"Seventeen extracted examples expose hidden regularities and give LLMs clearer iteration and data-flow structure than original papers.","key_machinery":"The modular ontology of exactly eight artifact types paired with a fixed vocabulary of transform intents that together encode both the data-flow structure and the analytical purpose of each workflow step.","core_discovery":"ATWL is a domain-agnostic declarative language built on a modular ontology of eight artifact types (entities, features, arrangements, visualisations, patterns, models, knowledge, specifications) and transforms characterised by standardised intents such as define-unit, characterise, contextualise, and abstract. It captures the structure and underlying analytical intent of VA workflows, allowing systematic comparison across papers and domains plus reuse of proven strategies. Extraction of seventeen workflows from research papers reveals a recurrent meta-structure, recurring motifs, reusable building blocks, diverse iterative strategies, and cross-domain equivalences that remain invisible in pr","pith_inferences":["A shared repository of ATWL workflows could support automated recommendation or synthesis of new analysis pipelines.","The same ontology approach might be tested on machine-learning or data-science pipelines outside visual analytics.","Standardised workflow encodings could become a reporting requirement that improves reproducibility in visual analytics publications.","Domain experts might later propose controlled extensions to the eight-type ontology once a larger corpus has been encoded."],"forward_implications":["Workflows become directly comparable across different papers and application domains.","Proven analytical strategies can be extracted, stored, and adapted as modular fragments.","LLM-assisted extraction reduces the human effort needed to build libraries of formal workflows.","Formal representations supply explicit iteration structure, typed data flow, and fragment provenance that prose descriptions omit.","Libraries of ATWL workflows remain compact enough to fit inside LLM context windows where full paper collections do not."],"fun_headline_variants":["ATWL formalizes VA workflows for systematic comparison","ATWL captures analytical intent in eight artifact types","Seventeen ATWL workflows expose hidden structural motifs","ATWL representations add explicit iteration to LLM analysis"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The eight artifact types together with the chosen set of transform intents are enough to represent the structure and intent of any visual analytics workflow without significant loss or distortion.","fun_headline_variants_meta":{"raw":{"variants":["ATWL formalizes VA workflows for systematic comparison","ATWL captures analytical intent in eight artifact types","Seventeen ATWL workflows expose hidden structural motifs","ATWL representations add explicit iteration to LLM analysis"]},"model":"grok-4.3","cost_usd":0.003515,"raw_usage":{"total_tokens":1903,"prompt_tokens":779,"num_sources_used":0,"completion_tokens":58,"cost_in_usd_ticks":35149500,"prompt_tokens_details":{"text_tokens":779,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1066,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":779,"tokens_out":58,"duration_ms":10150,"temperature":1.0,"reasoning_tokens":1066,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T21:50:57.923774+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A published visual analytics workflow whose structure or intent cannot be expressed using only the eight defined artifact types and the listed transform intents without adding new types or intents.","supporting_citations":[],"review_version":1}