{"id":"9fa3fc48-a2c4-4487-9c17-9a5d1014e6a8","arxiv_id":"1908.00559","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Design by immersion is a formalized methodology for problem-driven visualization that blurs expert-user boundaries by having researchers participate in the target domain.","lead":"A team of visualization researchers defines design by immersion, a method where the people who build data pictures take part in the other field's actual work (or invite those experts into visualization work). The paper draws on six of their own collaborations to catalogue the activities, benefits, and risks of this way of working.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Generalizability rests solely on six successful self-authored projects; a reusable methodology is asserted but never tested on independent cases.","rationale":"The reader's weakest assumption identifies exactly the point I find most load-bearing: the methodology is derived entirely from the authors' own six successful projects. My independent reading confirms that Section 3 bases the definition on these cases and Section 5 derives the three opportunities from the same set. The participants are also the coders, so the activity tables could reflect a consistent retrospective narrative rather than an independently observable structure. This is an empirical generalization problem, not a logical contradiction: the paper is coherent and clearly written, but its central claim of transferability is not yet supported. A prospective application by non-authors is the natural and decisive test. Since the reader already assigned CONDITIONAL on this basis, I do not recommend changing the verdict; the conditional remains appropriate until such independent validation appears.","tokens_in":18559,"tokens_out":2177,"duration_ms":24754,"concrete_test":"Run a prospective replication where 2–3 research groups with no author involvement apply the Section 6 trajectory framework to a new problem-driven visualization project, pre-registering which Table 1 activities they select and which of the three opportunities they expect to encounter, then debriefing afterward. If external groups can follow the trajectory, recognize the same opportunities, and produce contributions to both visualization and the domain, the generalization lands. If they cannot, the framework is an author-specific retrospective rather than a reusable methodology.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that design by immersion is a reusable methodology whose activities and opportunities (Tables 1–3, Section 6) transfer beyond the authors' own projects. The only empirical support is the retrospective analysis of six case studies, all successful and all involving at least one author as participant-observer. Section 3 explicitly states that the definition is based on these cases and that extracting immersion from other studies 'would be difficult without insider knowledge' (Section 3). This creates two coupled problems. First, selection on outcome: every case 'worked', so failure modes for immersion are not observable; the framework cannot distinguish activities that enabled success from activities that merely co-occurred in successful projects. Second, participant-observer circularity: the authors both lived the experiences and coded them into the activity/opportunity tables, so the Table 1 mappings may reflect post-hoc narrative rather than a public, repeatable classification. The paper's hedges and challenge list (Section 7) are honest but do not supply independent evidence. This does not make the argument internally inconsistent; it means the load-bearing assumption for a 'reusable methodology' is an empirical generalization from n=6 positive, self-reported cases.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces and defines a methodological paradigm called \"design by immersion\" for problem-driven visualization work. The core claim is that visualization researchers and/or domain experts can engage in and participate in the work of another domain, so that visualization design, solutions, and knowledge emerge from these transdisciplinary experiences. The paper grounds this definition in the authors' own collaborative projects and presents six case studies spanning computational linguistics, medicine, literary analysis, transport studies, and chemistry. From these cases, the authors derive three opportunities of design by immersion: enriching domain understanding through personal experience, exploring new domain-inspired design spaces, and building strong transdisciplinary relationships. They also provide a taxonomy of design activities (Table 1), a staged trajectory framework for constructing immersion experiences (Table 2), and reflective questions inspired by experiential learning (Table 3). The paper closes with a discussion of challenges including time commitment, research identity navigation, translational work, and systemic barriers.","tokens_in":18713,"tokens_out":4216,"duration_ms":44576,"significance":"If the framework holds, this paper makes a useful contribution by giving a name and structure to a practice that is visible in several successful transdisciplinary visualization projects but has not been systematically articulated. The activity taxonomy, the staged guidance, and the honest treatment of challenges provide practitioners with a starting vocabulary for planning and reflecting on immersive collaborations. The paper connects the approach to established literatures such as participatory design, action design research, and experiential learning, which helps situate it within the field. The main strength is the richness and diversity of the six case studies, and the authors are explicit that the definition grows out of their own experiences. However, the evidence base is limited to six successful, self-reported projects, and the transferability of the methodology to new projects is asserted rather than demonstrated. The three opportunities in Section 5 and the tables in Section 6 are interpretations of the same cases that motivated the definition, so the framework currently functions as a reflective taxonomy rather than an externally validated methodology.","major_comments":[{"comment":"The definition of design by immersion is explicitly built from the authors' own projects (Section 3 states: \"This definition is based on our own transdisciplinary visualization projects\"), and the three opportunities in Section 5 are then derived from the same six case studies. This creates a circular evidentiary chain: Table 1 is a post-hoc coding of the authors' experiences rather than an independent classification of external material. Because the authors both lived the experiences and coded them into the activity/opportunity tables, the mappings may reflect narrative construction rather than a public, repeatable analysis. I recommend either reframing the paper as a reflective framework grounded in six cases, with the transferability claim presented as a hypothesis for future work, or adding a systematic analysis protocol (selection criteria, coding scheme, source documents, and ideally independent coding) to support the current strength of the claims.","section":"Section 3 and Section 5"},{"comment":"All six case studies are successful projects, and Section 7's challenges are presented without any case in which immersion failed, was abandoned, or produced net-negative outcomes. This selection-on-outcome makes it impossible to distinguish activities that enabled success from activities that merely co-occurred in successful projects. The framework cannot currently identify boundary conditions under which design by immersion is inappropriate or counterproductive. Please add negative or borderline cases, or explicitly scope the framework to successful transdisciplinary collaborations and state that failure modes remain an open question.","section":"Section 4 and Section 7"},{"comment":"The three-stage trajectory in Section 6.2 is described as \"transferable across immersion trajectories,\" but the supporting evidence is limited to the two example trajectories in Section 6.1 and the authors' own recollections. The paper does not operationalize how a researcher should decide between stopping at Stage #2 and proceeding to Stage #3, nor how activities in Table 1 should be selected or ordered for a particular domain. Because Section 6 is the paper's practical \"how to\" contribution, these criteria need to be made more explicit, or the stages should be clearly labeled as an illustrative rather than prescriptive template.","section":"Section 6.2 and Table 2"}],"minor_comments":[{"comment":"There are several typos: \"transdiciplinary\" in Section 3 and \"multifacted\" in Section 4 should be \"transdisciplinary\" and \"multifaceted,\" respectively.","section":"Section 3 and Section 4"},{"comment":"The text refers to the orange square at the lower bound of the knowledge space, but the figure caption does not label this element; please add a label or adjust the reference.","section":"Figure 1"},{"comment":"The numeric entries in the table (e.g., \"1,4,5,6\") indicate case-study links, but the table does not distinguish activities that recurred in many cases from those that occurred in only one. Adding a frequency or centrality indicator would help readers assess the generalizability of each activity.","section":"Table 1"},{"comment":"The sentence \"the medical collaborator spent considerable time with the visualization group to understand the importance of pre-design empirical work [2]\" cites Brehmer et al.'s methods paper rather than a case study; please clarify whether this detail comes from that reference or from the authors' own project experience.","section":"Section 4.2"},{"comment":"The phrase \"pre-design studies aimed at gaining a understanding of a domain situation\" needs the article corrected to \"an understanding.\"","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a reflective methodology paper rather than an empirical validation study. Its core contribution is a named, well-structured framework that many in the visualization community will find useful, and the authors are transparent about the experiential basis of their claims. My recommendation for major revision is driven by the gap between the strength of the claims (\"methodology,\" \"transferable stages\") and the evidence (six successful self-authored cases). This gap is fixable within the scope of the paper by reframing the contribution as a provisional framework and adding explicit limitations, or by strengthening the analysis protocol and including non-successful cases. I do not see this as a case for rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe thing to know: this paper gives a name and a formal structure to something many of us do informally—immersing yourself in a collaborator's domain rather than studying it from the outside. The authors define design by immersion, describe four themes, a skills space, and a table of 20 activities, and map six of their own projects onto it. The formalization is the new contribution, and it is genuinely useful: it gives practitioners a vocabulary for planning and reflecting on transdisciplinary work.\n\nIt is also well situated. The paper draws fair distinctions between immersion and design study methodology, participatory design, and action design research. The six case studies are diverse (computational linguistics, medicine, literary studies, transport, chemistry) and each connects to specific activities. The prose is clear and the authors are appropriately modest: they do not claim immersion is better, and they list time, identity, and systemic barriers as challenges. The underlying case-study publications include some external validation—notably a pre-clinical trial in the medical case—so this is not pure anecdote.\n\nThe soft spot is the evidence base, and it is load-bearing. The definition is built from the authors' own projects (Section 3), and the three opportunities come from the same six cases (Section 5). All six are successful, and each involves an author as participant-observer. That means failure modes are not observable, and the activity/opportunity tables may reflect post-hoc narrative as much as a repeatable classification. The authors acknowledge this in a backhanded way when they say extracting immersion from others' work \"would be difficult without insider knowledge.\" This limitation is real, but it is not disqualifying: the paper presents itself as a starting point, the reflective questions are provisional, and the challenges are honestly described. Still, the reusable-methodology claim rests on n=6 positive, self-authored cases.\n\nMinor: the multidisciplinary/interdisciplinary/transdisciplinary distinction is thin, and some activities in Table 1 overlap. These are small.\n\nThis paper is for visualization researchers and HCI/digital humanities people doing problem-driven collaborations. It deserves a serious referee. For a revision, I would ask the authors to directly address the selection issue—ideally by applying the framework to an independent project or by specifying a systematic reflective protocol. As a synthesis with practical guidance, it's solid.\n\nSend it to review.","headline":"A useful, honest synthesis that names and organizes a common way of working; the load-bearing weakness is that the framework rests on six successful, self-authored cases, but that is not disqualifying.","tokens_in":19274,"tokens_out":3212,"would_cite":true,"duration_ms":32014,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Problem-driven visualization can be organized as design by immersion, where researchers participate in a partner domain and let designs and knowledge emerge from that participation.","keywords":["design by immersion","transdisciplinarity","problem-driven visualization","design studies","collaborative visualization","experiential learning","domain experts","visualization methodology"],"falsifier":"A concrete test would be to apply the paper's activity catalogue to a new, unfamiliar domain and compare it with a matched project run in the conventional role-separated design-study style: if the immersed team produces no more domain-inspired designs, no deeper domain understanding, and no stronger collaboration, the claimed opportunities fail to materialize. A simpler disconfirming observation would be a documented immersion trajectory that follows all three stages yet yields no design or knowledge traced to transdisciplinary interaction.","tokens_in":18337,"feed_emoji":"📊","tokens_out":8544,"duration_ms":75596,"temperature":0.7,"pith_summary":"This paper proposes that problem-driven visualization work can be organized around a paradigm the authors name design by immersion: instead of keeping visualization experts and domain experts in fixed, separate roles, researchers from one side participate in the work of the other side, so that visualization design, solutions, and knowledge emerge from those transdisciplinary experiences. The authors argue that existing design-study frameworks tend to assign roles and separate domain characterization from ideation, whereas design by immersion treats personal participation in the domain as a generative design resource. They build the argument from six of their own transdisciplinary projects, spanning computational linguistics, medicine, literary studies, transport, and chemistry, and from them derive a catalogue of reusable design activities and a three-stage immersion trajectory. If the paradigm is right, a visualization researcher can deliberately and methodically become part of a partner community, and the designs that come back are more domain-grounded, the relationships more productive, and the process personally transformative.","feed_headline":"Work inside a partner domain to design better visualizations","feed_subtitle":"Immersing in a partner field yields novel designs, deeper understanding, and lasting collaboration.","key_machinery":"The central object is the design-by-immersion paradigm itself, made operational through three instruments. The first is the immersive skills space, a two-axis diagram with visualization expertise on one axis and target-domain expertise on the other, along which immersion moves a researcher from a novice position toward domain skill and, in the deepest cases, dual citizenship. The second is a table of more than twenty design activities—domain data analysis, study methods, prototyping, learning about the other domain, and communicating across domains—each linked to the three opportunities it supports. The third is a three-stage trajectory for planning an immersion: establishing basic domain knowledge, gaining domain research exposure, and establishing a broader domain reputation. Together these instruments let a researcher construct a customized immersion trajectory rather than follow a single prescribed procedure.","core_discovery":"The paper's central claim is that design by immersion is an identifiable, reusable methodology for problem-driven visualization. The methodology's definition is: visualization researchers (or target domain experts) engage with and participate in the work of the other domain, and visualization design, solutions, and knowledge emerge from these transdisciplinary experiences and interactions. Four themes characterize the approach—communal, personal, active, and emergent—and the paper shows how these themes play out in six case studies. Across those cases, the authors identify three opportunities that the method provides: enriching domain understanding through personal experiences, exploring new domain-inspired design spaces, and building strong transdisciplinary relationships. The claim is not that design by immersion is better than existing practices, but that it offers a distinct and currently missing route into problem-driven visualization work.","pith_inferences":["If design by immersion produces the claimed benefits, a testable extension would be a comparative study of otherwise similar visualization projects run with and without immersion, measuring novelty of encodings, persistence of collaborations, and domain uptake; the paper does not provide this comparison.","The underlying mechanism—participating in another discipline's work so that designs emerge from that participation—may transfer beyond visualization to other transdisciplinary design fields such as human-computer interaction, data journalism, or science communication, though the paper's evidence is limited to visualization projects.","Because personal experience serves as evidence of domain understanding, the method's credibility depends heavily on the researcher's documentation habits; the paper's own recommendation to keep journals and sketches is therefore load-bearing rather than optional.","The paper notes systemic barriers but does not develop their consequences: if PhD programs and funding structures remain single-discipline, design by immersion will likely remain an individual choice rather than an institutionalized methodology."],"forward_implications":["Researchers can treat immersion as a deliberate, first-class design activity, selecting activities from the paper's catalogue to match the project's domain, team, and goals.","Domain-inspired visual encodings become a legitimate outcome of the design process, since immersion exposes concepts beyond the immediate problem characterization.","Open-ended and evolving collaborations, where research questions change as the work proceeds, become a natural fit for the method rather than a risk to be contained.","Committed immersion can open channels beyond the immediate project, including domain publications, conference participation, access to domain experts, and impact across sectors.","The approach reframes role blurring and personal relationships as load-bearing mechanisms for trust and communication, complementing role-based methods such as the Liaison concept."],"supporting_citations":[{"why":"Defines the nine-stage design-study methodology and the role-separation assumptions that design by immersion is positioned against.","marker":"[40]"},{"why":"Supplies the nested model of visualization design and validation, used to situate where immersion can aid pre-design and evaluation.","marker":"[33]"},{"why":"Provides the Design Activity Framework whose activity vocabulary and ordered progression design by immersion extends and questions.","marker":"[27]"},{"why":"Introduces Action Design Research and its build-intervene-evaluate cycles, an artifact-centric alternative that design by immersion contrasts with.","marker":"[41]"},{"why":"Defines the Liaison role, the nearest role-based mechanism that an immersed researcher is well positioned to perform.","marker":"[43]"},{"why":"Contributes the dual-citizen case in transport studies, a core demonstration of immersion at its deepest level.","marker":"[46]"},{"why":"Contributes the reciprocal-immersion case in literary studies, showing the approach in an evolving, open-ended collaboration.","marker":"[12]"},{"why":"Contributes the apprenticeship case in computational linguistics, where immersion led to the Bubble Sets design.","marker":"[6]"},{"why":"Contributes the humanities researcher immersing in visualization case, used to show immersion from the domain side.","marker":"[1]"},{"why":"Provides the experiential-learning theory that grounds why transforming experience produces knowledge, a foundation for immersion.","marker":"[21]"}],"fun_headline_variants":["Immerse in partner domains for better visualization design","Design by immersion: a new methodology for visualization projects","Immersion in partner domains yields novel visualization designs","Six case studies reveal how immersion aids visualization design","Transdisciplinary immersion: a fresh route for problem-driven visualization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework rests on the assumption that the six transdisciplinary projects drawn from the authors' own experience, interpreted through their own reflection, are representative enough to define a general methodology and its activity tables.","fun_headline_variants_meta":{"raw":{"variants":["Immerse in partner domains for better visualization design","Design by immersion: a new methodology for visualization projects","Immersion in partner domains yields novel visualization designs","Six case studies reveal how immersion aids visualization design","Transdisciplinary immersion: a fresh route for problem-driven visualization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000575,"raw_usage":{"total_tokens":2684,"prompt_tokens":887,"completion_tokens":1797,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":503,"completion_tokens_details":{"reasoning_tokens":1723}},"tokens_in":503,"tokens_out":1797,"duration_ms":13356,"temperature":1.0,"reasoning_tokens":1723,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:46:28.312189+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test would be to apply the paper's activity catalogue to a new, unfamiliar domain and compare it with a matched project run in the conventional role-separated design-study style: if the immersed team produces no more domain-inspired designs, no deeper domain understanding, and no stronger collaboration, the claimed opportunities fail to materialize. A simpler disconfirming observation would be a documented immersion trajectory that follows all three stages yet yields no design or knowledge traced to transdisciplinary interaction.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Action Design Research and its build-intervene-evaluate cycles, an artifact-centric alternative that design by immersion contrasts with."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes the humanities researcher immersing in visualization case, used to show immersion from the domain side."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the experiential-learning theory that grounds why transforming experience produces knowledge, a foundation for immersion."}],"review_version":1}