{"id":"15457468-cf7d-4205-8ab3-815e10f68665","arxiv_id":"2502.09048","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Interviews with 17 visualization practitioners show that designing race and gender visualizations is value-laden, political work in which designers navigate tensions around neutrality, power, and their own positionalities.","lead":"This paper reports 17 interviews with designers who build visualizations of race and gender data, finding that their values, identities, and views of power and neutrality shape how they work. It is one of the first studies to focus on practitioners rather than audiences as the locus of the social construction of visualizations.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core claim is supported mainly by prompted self-reports: the protocol asks directly about identity, politics, values, and bias, while no artifact-level analysis verifies that these actually shape the resulting visualizations.","rationale":"The paper's strongest claim is empirical: visualizing protected demographic data is value-laden and political because of interactions among designers' experiences, values, understandings of power, neutrality, and politics. For this to hold, participants' reports must accurately trace how design decisions were made, and those decisions must be reflected in the artifacts. The most insecure condition is the self-report link. The protocol directly asks for the vocabulary of the conclusions, which invites socially desirable and prompt-shaped answers. I do not think this is fatal: many quotes describe concrete decisions (e.g., P12's deliberate blank spaces for absent Asian women in §5.1.2; P13's account of being forced to 'push down the chart' in §5.4.3), and the authors are transparent about their method and limitations. However, the article does not close the gap between prompted accounts and observed practice or artifacts. This is essentially the reader's weakest assumption, so I agree with the conditional verdict: the concern justifies caution but not rejection. The recommended split-transcript re-coding would quantify spontaneous versus prompted mentions, and the artifact-inference check would directly test whether the 'resulting data visualizations' actually carry the values the paper attributes to them.","tokens_in":26233,"tokens_out":4457,"duration_ms":48675,"concrete_test":"Re-code each transcript into two segments: the pre-prompt portion (background, tools, design process, before the first identity/politics question) and the post-prompt portion. Count design decisions attributed to personal values, identity, positionality, or politics in each portion, using the study's own codebook. If the pre-prompt rate is near zero while the post-prompt rate is high, the reported themes are largely protocol-induced rather than grounded in unprompted practice. To test the artifact link, additionally collect the consented visualizations and have two blinded coders infer the designer's stated positionality/values from the artifacts alone; accuracy near chance would show that the paper has not demonstrated that the 'resulting data visualizations' carry those values.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (§6: 'visualizing protected demographic data is a value-laden and political process as a result of interactions between these aforementioned factors') requires that designers' positionality genuinely shapes both their design process and the resulting artifacts. The only direct evidence is interview self-report, and the protocol primes that evidence: Appendix A explicitly asks 'How would you describe the identities that are most important to you and your politics?' and 'How do your personal beliefs, values, biases or politics influence your design process?' These questions supply the very constructs that the Findings then report in §5.3 and §5.5. Participants may be producing coherent narratives demanded by the interview situation rather than describing their actual practices. The paper also solicits artifacts ('can you share the visualization...?'), but no artifact-level analysis appears in the Findings, so RQ3's reference to 'resulting data visualizations' is not empirically checked. The acknowledged sample skew (12 White, 9 male, US-based volunteers) further limits the breadth of the central claim, though the authors flag this. This is a validity concern about the evidence base, not an internal inconsistency; the paper is transparent and quotes participants richly.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a qualitative interview study of 17 US-based visualization practitioners who design visualizations of race and gender demographic data. The authors analyze semi-structured interviews to identify challenges around socially constructed demographic categories, tensions between neutrality and other design values, experiences of power and politics in workplace settings, and the role of designers' identities and positionality. They argue in Section 6 that visualizing protected demographic data is a value-laden and political process shaped by interactions among these factors, and they propose implications for safeguarding public visualizations, prioritizing designer safety, and designing visualization tools for epistemological pluralism.","tokens_in":26419,"tokens_out":3592,"duration_ms":38202,"significance":"If the central claim is accepted, the paper provides empirical grounding for feminist and critical visualization research by extending the concept of situated knowledges to visualization practitioners, a population that prior work has largely treated as neutral conduits. The study is transparently reported, includes rich verbatim participant accounts, and builds directly on prior work on anthropographics and critical visualization. The paper's strengths are its focus on practitioner perspectives, its use of a feminist analytical lens, and its explicit acknowledgment of sample limitations. However, the central claim rests primarily on self-reported accounts that the interview protocol itself invites, and the findings do not analyze the visualization artifacts mentioned in the research questions, so the empirical support is narrower than the Discussion suggests.","major_comments":[{"comment":"The interview protocol explicitly invites the paper's central vocabulary: Appendix A asks participants \"How would you describe the identities that are most important to you and your politics?\" and \"How do your personal beliefs, values, biases or politics influence your design process?\", and the neutrality questions directly elicit the construct analyzed in §5.3. The themes in §5.3, §5.4, and §5.5 therefore may be partly an artifact of the instrument rather than emergent from practice. The manuscript should distinguish prompted from unprompted evidence (e.g., P12's Pulse shooting account, P17's Gaza remarks) or report the prevalence of each theme across participants, and the Discussion in §6 should temper the claim accordingly.","section":"Appendix A, §5.3–§5.5"},{"comment":"RQ3 asks about influence on \"the resulting data visualizations\", and the protocol in §4.3 asks participants to share visualization artifacts, yet the Findings present no analysis of any artifact. All quoted evidence concerns designers' accounts of their process, not verified properties of the products. The central claim in §6 that visualizing protected demographic data \"is a value-laden and political process\" is therefore supported only at the level of perceived influence. Either add a small artifact-level analysis (e.g., comparing shared visualizations with the designers' descriptions) or revise RQ3 and the §6 claim to refer to designers' perceived influence on their process.","section":"RQ3, §4.3, §5.1"},{"comment":"The sample is 17 self-selected US-based volunteers, with 12 White and 9 male participants, recruited through visualization subreddits, the Data Visualization Society, and targeted journalism outreach. The authors acknowledge this skew in §6.1, but the generalizing formulation in §6 (\"visualizing protected demographic data is...\") goes beyond what this sample can establish. Please add an explicit transferability argument or qualify the central claim to the practitioners studied.","section":"§4.1–§4.2, §6.1"}],"minor_comments":[{"comment":"The phrase \"we highlight a smaller sample of these responses for brevity\" is vague; please specify how many participants were asked about neutrality and how many distinct definitions were collected, so readers can assess the range of views.","section":"§5.3.1"},{"comment":"A table summarizing the final themes, the number of participants contributing to each, and the distribution of prompted versus unprompted instances would substantially improve the transparency of the thematic analysis.","section":"§4.4"},{"comment":"The statement that the visualization research community \"has yet to critically engage\" with researcher positionality is too strong given the critical and feminist work cited in §2.1; suggest \"has not yet fully engaged\" or similar.","section":"§2.2"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about circularity is real but addressable: the authors can reframe the central claim as being about designers' perceived and narrated influence, add recurrence counts, and explicitly separate prompted from unprompted themes. The lack of artifact-level analysis is the more difficult gap, but it can be handled by scoping the claims rather than by adding a costly new study. The paper is otherwise within the scope of a CHI-style qualitative contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe takeaway: this is a genuinely useful empirical study, but its central claim—that visualizing race and gender data is a value-laden, political process—is best read as a description of what 17 practitioners told the authors, not as a demonstrated causal story about the artifacts themselves. That distinction matters more than the paper admits.\n\nWhat's new: this is the first interview study I know of that specifically asks visualization practitioners about how their positionality, politics, and views of neutrality shape their work with protected demographic data. Prior critical/feminist visualization work has mostly focused on audiences, researchers, or artifacts. The paper fills a real gap. It also does qualitative methods properly: the full interview protocol is in the appendix, the authors quote participants extensively, and they acknowledge their sample skew (12 White, 9 male, US-based) and the limits of reproducibility. That's honest.\n\nThe soft spots are real but not fatal. The protocol explicitly asks 'How would you describe the identities that are most important to you and your politics?' and 'How do your personal beliefs, values, biases or politics influence your design process?' So the finding that identity, politics, and bias matter is partly invited by the instrument. The stress-test note is right about that. Also, the authors ask participants to share visualizations, but no artifact-level analysis appears in the findings; RQ3's reference to 'resulting data visualizations' is not empirically checked. So the causal framing in Section 6 and the conclusion ('influence the end product') goes beyond what self-reports alone can establish.\n\nThat said, the concern is not devastating. Some of the most powerful material—P12's account of mapping the Pulse shooting as a queer journalist, P17's reflections on Gaza and trust in data—was clearly unprompted. And in a semi-structured interview, asking directly about values is normal practice. The paper would be stronger if it consistently framed the findings as practitioners' perceptions rather than as established effects on artifacts. That's a revision, not a rejection.\n\nBottom line: this paper deserves serious peer review. It is a solid empirical contribution for visualization researchers working on critical/feminist approaches, for tool designers, and for data journalism practitioners. I'd send it to a qualified qualitative HCI reviewer and ask for the claims to be calibrated to the evidence. I would not desk reject.","headline":"Useful first empirical study of visualization practitioners' own accounts of how identity and politics shape their work, but the central claim should be read as self-report, not demonstrated artifact-level causation.","tokens_in":26928,"tokens_out":2683,"would_cite":true,"duration_ms":27647,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Visualizing race and gender data is a value-laden, political process shaped by the designer's own identity, values, and views of neutrality, according to interviews with 17 visualization practitioners.","keywords":["visualization designers","values","feminist epistemology","demographic data","race and gender","positionality","neutrality","social construction"],"falsifier":"A content analysis of a large corpus of race/gender visualizations that found no systematic association between designers' stated positionality (for instance, explicit value-laden versus neutrality-claiming) and their design choices—icon style, color, category ordering, annotations, treatment of absent groups—would undercut the claim that positionality shapes artifacts. So would an ethnographic study of a visualization team's design process that found personal beliefs and politics playing no role in their decisions.","tokens_in":26030,"feed_emoji":"📊","tokens_out":6398,"duration_ms":55753,"temperature":0.7,"pith_summary":"The paper sets out to establish that the production of visualizations of protected demographic data—race and gender—is not a neutral technical act but a value-laden and political process. Drawing on 17 semi-structured interviews with US-based visualization practitioners, it argues that designers' personal experiences, values, beliefs, positions in power structures, and understandings of neutrality interact to shape the charts they make. It interprets these accounts through the feminist concept of situated knowledges, which holds that all knowledge is produced from a particular position. If correct, the claim matters because it moves the debate about social construction in visualization from audiences and researchers to the practitioners who actually build the artifacts, and because it implies that tools, workflows, and best practices that assume designer neutrality are inadequate for race and gender data.","feed_headline":"Visualizing race and gender data is a political act","feed_subtitle":"Seventeen designers describe how their values, identities, and views of neutrality shape the charts they make.","key_machinery":"The analytical engine is the feminist concept of situated knowledges, applied to visualization practitioners: the idea that what a designer can see, know, and communicate is limited and shaped by their position in the world. The empirical machinery is a 17-participant semi-structured interview study with a protocol that prompts designers to describe their identities, politics, values, and biases, followed by reflexive thematic analysis to synthesize themes about neutrality, power, politics, and positionality.","core_discovery":"In the paper's own terms, the central discovery is that visualizing protected demographic data is a value-laden and political process as a result of interactions between designers' personal experiences, values, their understandings of power, politics, and neutrality, and the socially constructed nature of demographic categories themselves. The interview data show practitioners navigating tensions: they are acutely aware that census categories are socially constructed and can erase nonbinary and non-White people; they make consequential choices about how to visualize absence; they hold divergent definitions of neutrality and objectivity; they experience power as relational and political; and their identities—especially minoritized identities—shape how they approach and are treated in their work. The authors frame these findings as an empirical study of situated knowledges among visualization designers.","pith_inferences":["If the paper is right, 'neutral' visualizations may systematically default to majority-group perspectives; one testable implication is that a corpus of visualizations labeled neutral would show systematic choices that align with dominant demographic positions.","A direct experimental extension would ask designers with different stated positionalities to encode the same race/gender dataset and compare their outputs on iconography, color, ordering, annotations, and handling of absent categories.","The paper's practitioner-side findings could be joined to audience-side research: in an experiment, adding a designer's positionality disclosure to a visualization could change audience trust, comprehension, or perceived bias.","The focus on self-report suggests a complementary observational study: shadowing designers in their workplaces to see whether stated values actually appear in their design processes and final artifacts."],"forward_implications":["Visualization tools and workflows that assume designer neutrality are poorly matched to protected demographic data; tools should support disclosure of the designer's positionality and the data's context.","Designers working with race and gender data need institutional safeguards against online harassment and against their visualizations being misappropriated as misinformation.","The visualization research community should treat designers' situated knowledges as a resource for equitable practice, including through participatory and co-design work with the groups being visualized.","The same dynamics likely extend to other politically consequential demographic categories, such as migration data used in public discourse."],"supporting_citations":[{"why":"Supplies the feminist lens of situated knowledges that frames the analysis of designers' perspectives.","marker":"[36]"},{"why":"Provides the definition of power as structural privilege and oppression used to interpret designer experiences.","marker":"[27]"},{"why":"Provides the definition of politics as arrangements of power and authority in human associations.","marker":"[69]"},{"why":"Provides the definition of positionality used to describe designers' experiences and backgrounds.","marker":"[43]"},{"why":"Demonstrates applying feminist epistemologies, including situated knowledges, to visualization research, which this paper extends to practitioners.","marker":"[3]"},{"why":"Prior work outlining the challenges of visualizing racial demographic data and diverse anthropographics that this study builds on.","marker":"[25]"},{"why":"Shows that designers engage in reflexive, situated design processes, providing the practitioner-design literature this study extends.","marker":"[58]"}],"fun_headline_variants":["Designers: demographic charts are political, not neutral","How designers' values shape race and gender data visuals","Designers say visualizing race and gender is a political choice","No neutral charts: 17 designers on race and gender data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on the assumption that interviewees' direct, prompted answers about their identities, politics, values, and neutrality accurately reflect how they actually design and what they actually build, rather than socially desirable narratives; it also assumes that 17 mostly White, male, US-based volunteers can stand in for visualization practitioners broadly.","fun_headline_variants_meta":{"raw":{"variants":["Designers: demographic charts are political, not neutral","How designers' values shape race and gender data visuals","Designers say visualizing race and gender is a political choice","No neutral charts: 17 designers on race and gender data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000647,"raw_usage":{"total_tokens":2919,"prompt_tokens":837,"completion_tokens":2082,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":453,"completion_tokens_details":{"reasoning_tokens":2016}},"tokens_in":453,"tokens_out":2082,"duration_ms":13969,"temperature":1.0,"reasoning_tokens":2016,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T22:47:15.575452+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A content analysis of a large corpus of race/gender visualizations that found no systematic association between designers' stated positionality (for instance, explicit value-laden versus neutrality-claiming) and their design choices—icon style, color, category ordering, annotations, treatment of absent groups—would undercut the claim that positionality shapes artifacts. So would an ethnographic study of a visualization team's design process that found personal beliefs and politics playing no role in their decisions.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the definition of politics as arrangements of power and authority in human associations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior work outlining the challenges of visualizing racial demographic data and diverse anthropographics that this study builds on."}],"review_version":1}