{"id":"8a164819-783f-41a1-b391-8ec079f72427","arxiv_id":"1908.02619","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Artistic practice can make the boundaries and reasoning of AI systems visible and decipherable, and the Experiential AI agenda organizes artist-scientist residencies to test that claim against the limits of technical explanations.","lead":"This paper proposes a research agenda called Experiential AI, in which artists and AI scientists collaborate to make algorithms more transparent and understandable. A smart generalist can read it to see why artistic practice is argued to fill gaps left by technical explainability methods in AI.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central causal mechanism—visibility of a system's operation yields comprehension and accountability—is asserted rather than argued; the paper's own Ananny and Crawford citation says visibility is not sufficient for accountability, and no evidence links experiential art to durable comprehension.","rationale":"The reader's weakest assumption is correct: the paper's visibility-to-accountability premise is the hinge of the whole agenda. My stress-test confirms that concern and sharpens it by locating an internal tension: the same Section that asserts visibility as necessary for accountability also cites Ananny and Crawford (2018), whose central argument is that transparency and visibility are insufficient because accountability is distributed across sociotechnical systems. The paper borrows their call for deeper engagement without confronting their negative finding about visibility. The other potential concerns, such as the lack of experimental data or the absence of a baseline against existing XAI methods, are consequences of the paper being a position paper; the abstract labels the central claim a hypothesis, so the lack of evaluation is not, by itself, a soundness defect. However, Section 3 shifts from hypothesis to assertion ('helps to viscerally understand'), which increases correctness risk beyond what a proposal would normally carry. The conditional verdict remains appropriate: the agenda is a legitimate contribution to the explainable-AI conversation, but it should not be accepted as an established route to transparency and accountability until the proposed empirical evaluations are published. My recommendation is therefore UNCHANGED rather than a move to ACCEPT or REJECT: the reader's conditional framing already matches the actual epistemic status of the paper, and my analysis does not identify a reason to strengthen or weaken it.","tokens_in":5055,"tokens_out":2534,"duration_ms":30107,"concrete_test":"Run a pre-registered between-subjects experiment: one group experiences a representative Experiential AI installation (e.g., an interactive artwork exposing a classifier's failure modes), a second receives a standard LIME/SHAP-style explanation of the same classifier, and a third receives no explanation. Measure (1) objective comprehension, such as the ability to predict classifier outputs on held-out cases, (2) perceived transparency and trust, and (3) the ability to identify a contestable error or construct a counterfactual. If the art condition does not significantly outperform the standard XAI baseline on at least one of (1)–(3) without degrading the others, the load-bearing premise that art is needed beyond current explanations is falsified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2 states: 'unless the operation of a system is visible, and people can access comprehensible explanations, it cannot be held to account.' The Experiential AI agenda inherits this premise and extends it: Section 3 claims that artists making algorithmic artifacts tangible 'helps to build literacy in those systems' and 'brings to life and question what an algorithm does.' The load-bearing step is not the existence of such artworks, which are amply exemplified, but the inference from a tangible artifact to audience comprehension, and from comprehension to accountability. The paper offers no mechanism, no measurement, and no baseline; Kolb's experiential learning is a general theory of reflection, not evidence about AI explanations. Worse, the same paragraph cites Ananny and Crawford (2018), who argue that accountability fails across distributed sociotechnical systems regardless of visibility; using their call for 'deeper engagement' as support does not neutralize their direct challenge to the visibility-accountability premise. The central claim is therefore plausible but unsupported at its hinge. This is not an internal formal inconsistency, since the abstract honestly labels it a hypothesis; but Section 3 presents the efficacy as already achieved ('helps to viscerally understand...'), which overstates the available evidence and makes the agenda's credibility depend entirely on future empirical work.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This position paper proposes 'Experiential AI' as a new research agenda in which artists and AI scientists collaborate to make algorithmic systems tangible, visible, and interpretable to non-expert audiences. The abstract states the central hypothesis: art can mediate between computer code and human comprehension, overcoming the limitations of conventional explainable-AI (XAI) techniques. The paper surveys limitations of current accountability and explainability efforts, offers examples of artistic practice (Klingemann, Barrat, DeepDream, Buolamwini, Paglen), and announces an artist-in-residence program launched in 2019 as the first concrete instantiation of the agenda. The manuscript has no experiments or measurements; it argues by illustration and precedent.","tokens_in":5192,"tokens_out":2412,"duration_ms":27706,"significance":"If the central hypothesis is correct, Experiential AI would fill a genuine gap in the XAI literature, which the paper correctly identifies as primarily technical and detached from human, legal, and institutional factors. The paper's strength is that it names a real problem—explanations alone do not yield comprehension or accountability—and it gives the hypothesis an honest label in the abstract. The proposal is also timely and actionable: the announced residency program is a concrete, falsifiable testbed. However, the significance is conditional, because the load-bearing causal chain from artistic artifacts to audience comprehension, and from comprehension to accountability, is asserted without evidence. The manuscript currently overstates what artistic practice has already achieved, presenting plausible hopes as established facts.","major_comments":[{"comment":"The load-bearing premise is stated as 'unless the operation of a system is visible, and people can access comprehensible explanations, it cannot be held to account.' This premise is immediately followed by a citation to Ananny and Crawford (2018), whose central argument is that transparency and visibility are not sufficient for accountability because accountability is distributed across sociotechnical systems. The manuscript does not reconcile these two positions; rather, it uses their call for deeper engagement as support while ignoring their direct challenge to the visibility-accountability equation. Please either weaken the premise to a research hypothesis or engage with Ananny and Crawford's argument that visibility is not the bottleneck.","section":"Section 2, 'Accountability and explainability in AI'"},{"comment":"Several sentences assert outcomes as fact rather than hypothesis: artistic approaches 'enable the character of machine reasoning and vision to be made explicit' and 'helping to build literacy in those systems,' and artistic experiments 'brings to life and question what an algorithm does.' No evidence is provided that any audience actually achieves comprehension, literacy, or the ability to question algorithms through these works. The cited examples (Klingemann, Barrat, DeepDream) demonstrate artistic production, not measurable effects on viewers. Kolb's experiential learning (Kolb, 2014) is a general educational theory, not empirical support for AI-specific literacy gains. Please reframe these claims as hypotheses to be tested in the residency program, or add empirical evidence if any exists.","section":"Section 3, 'Artists addressing such AI challenges'"},{"comment":"The claim that 'Experiential approaches (Kolb, 2014) can act as a powerful mechanism' for internalizing ethical standards is unsupported. The mechanism from a role-play or narrative experience to durable ethical internalization in the specific context of AI systems is not established, and the reference to Boal's Forum Theatre is evocative but not evidential. Since ethical internalization is one of the promised benefits of Experiential AI, this step needs either a more explicit argument with intermediate evidence or a clear statement that it is an open research question.","section":"Section 3, paragraph on ethics"}],"minor_comments":[{"comment":"Typographical error: 'his has led' should read 'This has led.'","section":"Section 2, first paragraph"},{"comment":"The reference entry beginning 'Sharif, M., Bhagavatula, S., Bauer, L., & Reiter, M. K.' is followed by a stray '(2018).' on its own line, likely a formatting error.","section":"References"},{"comment":"The entry 'Donnarumma, M. (n.d.). Is artificial intelligence set to become arts next medium?' lacks a source or URL; please complete it or remove it.","section":"References"},{"comment":"The citation style for the Reddit reference is inconsistent with the rest of the reference list; the author name and date are missing from the in-text citation.","section":"Section 3, 'Plugging 50,000 portraits into facial recognition'"}],"recommendation":"major_revision","confidential_remarks":"This is a short position paper rather than a full empirical study. If the journal expects submissions to report novel results, the paper's fit may be a concern; however, as a research-agenda paper it is within the scope of venues that publish position pieces. The main issue is not the absence of experiments per se, but the mismatch between the strong claims in Section 3 and the abstract's honest labeling of the hypothesis. A revision that consistently frames the agenda as a hypothesis with a concrete evaluation plan would resolve this mismatch."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a position paper proposing a research agenda, not a report of a result. The abstract is honest about that: the central claim is a hypothesis. The paper's value is in naming and bundling existing threads—artists making algorithmic distortions tangible (Klingemann, Barrat, Buolamwini, Paglen), the limits of technical explainability (Edwards and Veale, Ananny and Crawford), and experiential learning—into a coherent program with a concrete residency call. That bundling is genuinely useful for people working on non-technical routes to accountability.\n\nThe soft spots are real but proportionate. Section 3 slides from \"these practices exist\" to \"this enables... makes explicit... helps build literacy\" as if the effects were established. They are not. There is no measurement, no baseline against existing XAI methods, no user study. The authors do not dishonestly hide this—the abstract says hypothesis—but the body repeatedly asserts the same efficacy as fact. A careful editor should ask them to mark each such sentence as open or to pilot an evaluation.\n\nThe deeper tension is the visibility-accountability premise. Section 2 states that unless a system's operation is visible, it cannot be held to account. That is a strong claim, and the same paragraph cites Ananny and Crawford, who argue the opposite direction: transparency is necessary but nowhere near sufficient, and accountability fails across distributed systems regardless of visibility. Using their call for deeper engagement as support does not neutralize their direct challenge. The paper inherits this premise without arguing for it. That is the hinge on which the whole agenda turns, and it is the least supported step.\n\nMinor mechanical note: there is an orphaned reference entry \"(2018).\" after the Sharif et al. entry—probably a dangling author. The evidence cited for the value of art also leans on the authors' own previous publication and the residency they are launching; that is fine for an agenda, but it is not external validation.\n\nWho should read it: people who design XAI research programs, evaluators of art-science collaborations, and anyone arguing about the scope of explainability. It is a framing document, not evidence. A serious referee should engage it because the agenda is legitimate, clearly argued, and testable; the right outcome is likely \"revise with a commitment to empirical evaluation,\" not rejection.","headline":"A clearly written agenda paper for artistic approaches to AI transparency; the central hypothesis is honestly labeled, but Section 3 overstates what is known and the visibility-accountability premise is shakier than the authors acknowledge.","tokens_in":5833,"tokens_out":2252,"would_cite":true,"duration_ms":25211,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Art can mediate between computer code and human comprehension to make AI's reasoning visible when technical explanations fall short.","keywords":["Experiential AI","artificial intelligence transparency","explainable AI","algorithmic accountability","art and technology","human-centred machine learning","algorithmic bias","public understanding of AI"],"falsifier":"A controlled study could settle it: one group encounters an experiential artwork that exposes a known bias in a classifier, another receives a written technical explanation of the same bias, and both groups are then tested on their ability to name the bias, describe the causal chain, and propose a challenge to the decision.","tokens_in":4751,"feed_emoji":"🎨","tokens_out":6734,"duration_ms":67323,"temperature":0.7,"pith_summary":"The paper argues that technical explainability—local explanations, simpler models, and fairness metrics—does not by itself make AI accountable, because it does not address how people actually experience, understand, or contest systems in practice. It proposes Experiential AI as a research agenda in which artists and AI scientists work together to make algorithms' mechanisms vividly apparent through tangible artifacts, performances, and interactive experiences. The hypothesis is that art can mediate between code and human comprehension, revealing the boundaries of systems, the causal chain of decisions, and the ways data shapes algorithmic behavior. A sympathetic reader would care because this offers a route to transparency that complements—and may overcome the limits of—standard explanation methods, and gives audiences concrete objects to discuss and question.","feed_headline":"Art can open AI's black box when explanations fail","feed_subtitle":"A new agenda pairs artists with AI researchers to turn opaque algorithms into experiences audiences can question.","key_machinery":"The central object is the Experiential AI artwork—a concrete artifact, performance, or interactive installation in which an algorithm's operation is made visible and tangible to a user or audience. It carries the argument by turning abstract code into an encounter: the audience sees what the system attends to, what it distorts, where its boundaries lie, and how a decision might have been configured. This staged visibility is what the paper claims creates comprehension, discussion, and the possibility of challenge. The mechanism is named by the authors as art's capacity to make boundaries visible and to explore inter-agencies between people and machines.","core_discovery":"The central claim is that artistic practice can serve as a bridge between opaque computer code and human understanding, making AI's reasoning and its social consequences decipherable in ways that technical explanations often are not. The paper positions this as an answer to the call to look 'across systems' rather than inside a single technical object, and to engage with the material and ideological realities of computation. On these terms, artworks such as those that expose distortions in machine vision or bias in classifiers are not merely illustrations of a problem; they are instruments that make the problem visible and contestable. The intended outcome is greater accountability, public literacy, and new configurations of collaboration between humans and machines.","pith_inferences":["The paper does not test whether vivid experience durably changes what audiences understand or contest; a natural next step is measuring that effect.","If visibility is not the binding constraint on accountability—a caveat the paper's own cited critique raises—the agenda's value would shift toward shaping institutional scrutiny of system-wide design choices rather than individual system comprehension.","A controlled comparison between an experiential artwork and a conventional textual explanation for the same algorithm would indicate whether the art adds comprehension or only emotional impact."],"forward_implications":["AI scientists gain a practical channel for exposing a model's failure modes and biases to people who will never read a technical explanation.","Artworks become accountability artifacts: stable objects that audiences can revisit, discuss, and use to press for answers about automated decisions.","Experiential art can surface the causal chain behind a decision—data choice, algorithm selection, configuration—rather than only a local rationale for a single output.","Co-creation between artists and scientists can reframe AI design questions, such as what data to collect and who to collect it about, as aesthetic and ethical choices."],"supporting_citations":[{"why":"Frames accountability as needing a view across systems rather than inside one technical object; the paper positions experiential art as answering this call.","marker":"Ananny & Crawford, 2018"},{"why":"Supports the claim that providing an explanation is not a sufficient remedy, motivating more intuitive interventions.","marker":"Edwards & Veale, 2017"},{"why":"Defines the explainable AI program that the paper argues treats explainability as purely technical.","marker":"Gunning, 2017"},{"why":"Exemplifies local explanation methods that the paper says do not give the human-level causal rationale people want.","marker":"Ribeiro, Singh, & Guestrin, 2016"},{"why":"Provides the call to restrict black-box systems in high-stakes settings, grounding the urgency of the agenda.","marker":"Campolo, Sanfilippo, Whittaker, & Crawford, 2017"},{"why":"Establishes human-centred machine learning and the need for a science and an art to be developed around it.","marker":"Fiebrink & Gillies, 2018"},{"why":"Identifies fairness and accountability design needs in public-sector decision-making that experiential approaches are meant to address.","marker":"Veale, Van Kleek, & Binns, 2018"}],"fun_headline_variants":["Art cracks AI's black box","Art makes AI reasoning visible","Art bridges AI code to human insight","Artists turn AI opacity into experience"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The agenda rests on the premise that making a system's working visible and giving people comprehensible explanations is the key to holding it to account—and that artistic renderings deliver that comprehension in a way that changes what audiences understand and contest.","fun_headline_variants_meta":{"raw":{"variants":["Art cracks AI's black box","Art makes AI reasoning visible","Art bridges AI code to human insight","Artists turn AI opacity into experience"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000568,"raw_usage":{"total_tokens":2623,"prompt_tokens":816,"completion_tokens":1807,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":1759}},"tokens_in":432,"tokens_out":1807,"duration_ms":16300,"temperature":1.0,"reasoning_tokens":1759,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:54:16.065886+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled study could settle it: one group encounters an experiential artwork that exposes a known bias in a classifier, another receives a written technical explanation of the same bias, and both groups are then tested on their ability to name the bias, describe the causal chain, and propose a challenge to the decision.","supporting_citations":[{"cited_title":"\\ Crawford, K","cited_arxiv_id":null,"evidence_quote":"Frames accountability as needing a view across systems rather than inside one technical object; the paper positions experiential art as answering this call."},{"cited_title":"\\ Veale, M","cited_arxiv_id":null,"evidence_quote":"Supports the claim that providing an explanation is not a sufficient remedy, motivating more intuitive interventions."},{"cited_title":"APACrefauthors \\ 2017","cited_arxiv_id":null,"evidence_quote":"Defines the explainable AI program that the paper argues treats explainability as purely technical."},{"cited_title":", Singh, S","cited_arxiv_id":null,"evidence_quote":"Exemplifies local explanation methods that the paper says do not give the human-level causal rationale people want."},{"cited_title":", Sanfilippo, M","cited_arxiv_id":null,"evidence_quote":"Provides the call to restrict black-box systems in high-stakes settings, grounding the urgency of the agenda."},{"cited_title":"\\ Gillies, M","cited_arxiv_id":null,"evidence_quote":"Establishes human-centred machine learning and the need for a science and an art to be developed around it."},{"cited_title":", Van Kleek, M","cited_arxiv_id":null,"evidence_quote":"Identifies fairness and accountability design needs in public-sector decision-making that experiential approaches are meant to address."}],"review_version":1}