{"id":"0f3fe1f8-cad0-4e9e-8005-ae3a56de8516","arxiv_id":"2501.14886","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A systematic review of 202 papers shows equity is usually mentioned without definition and is studied through different, often unconnected approaches across HCI and fairness venues.","lead":"This paper systematically reviewed 202 research papers about equity and technology in human-computer interaction and fairness venues. It maps why researchers study equity, how they define it, and what interventions and trade-offs they report, resulting in a four-part framework for future work.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 'only 27% of papers explicitly define equity' is computed over all 202 papers, including 42 papers (21%) the authors themselves label as merely mentioning equity as motivation; this denominator choice deflates the definition rate and may overstate the claimed definitional gap.","rationale":"I read the paper in good faith and agree with the reader's conditional verdict: the review is transparent and the qualitative synthesis is reasonable, but the quantitative backbone has unresolved methodological sensitivities. The reader's weakest assumption focuses on the keyword/venue search excluding relevant work without the literal term. I see a related but more pointed problem that is internal to the reported statistics: the definition-rate denominator includes papers the authors themselves classify as only mentioning equity as a motivation, which are not equity-focused studies. This conflation directly affects the paper's signature finding and is easily testable. Additional issues noted by the reader — the venue-count inconsistency between Table 1 and Section 4.1, the unreported reliability for the definition coding, and the restrictive search — reinforce the need for conditional acceptance but do not, in my view, invalidate the core contribution. The proposed sensitivity analysis would either confirm the finding's robustness or require qualifying the 'only 27%' framing; either way it strengthens the paper. I therefore recommend keeping the CONDITIONAL verdict (unchanged).","tokens_in":37645,"tokens_out":7729,"duration_ms":67588,"concrete_test":"Recompute the equity-definition rates reported in Section 5.1 after excluding the 42 papers coded as 'equity merely as motivation' in Table 2 (Section 7.1), for the full corpus and for HCI and Fairness separately. Also compute the rate including motivation-only papers as a baseline. If the overall rate rises above 34% or if the HCI–Fairness gap narrows by more than 5 percentage points in the equity-focused subset, the claim that 'only 27%' of equity research explicitly defines equity should be revised to distinguish equity-focused studies from papers that merely use the term as a motivation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 5.1, the authors report that only 27% (54/202) of the corpus explicitly define equity, a statistic that anchors the paper's central claim that the lack of explicit definitions hinders a shared equity framework. Yet Section 7.1 (Table 2) identifies 42 papers (21% of the corpus, 29 HCI and 13 Fairness) as 'equity merely as motivation' — papers that mention equity only to motivate the work without engaging in studying or advancing equity. These papers are included in the 202-paper denominator for the definition-rate calculation, even though they are not equity-focused studies. Motivation-only papers are unlikely to contain a formal definition of equity, so their inclusion mechanically depresses the reported rate. If none of the 42 motivation-only papers define equity, the rate among the 160 genuinely equity-focused papers (monitoring/surfacing or intervention) rises to 54/160 ≈ 34%; if even a few define it, the rate rises further. The paper does not report this sensitivity analysis, nor does it report definition rates separately for equity-focused vs. motivation-only papers. The HCI/fairness gap (21% vs. 40%) may persist directionally, but the headline 'only 27%' and the associated conclusion about widespread definitional absence are materially dependent on this denominator choice. This is not merely a stylistic issue: the central claim is about equity research, yet a fifth of the corpus is not equity research by the authors' own classification.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript reports a systematic literature review, following the PRISMA workflow, of 202 research articles from HCI venues (SIGCHI/SIGACCESS sponsored) and Fairness-focused venues (FAccT, AIES, EAAMO, SaTML) that mention 'equit*' or 'inequit*' in the title, abstract, or keywords. The review characterizes why researchers study equity (existing societal and technological inequities, failures of prior efforts), how they define equity (need-based, contribution-based, equality-based), how they conduct equity research (monitoring/surfacing inequities, three types of interventions), and tensions among values. Based on these findings, the authors propose a four-dimension equity framework with seven recommendations. A central finding is that only 27% of the corpus explicitly defines equity, with a lower rate in HCI (21%) than in Fairness venues (40%).","tokens_in":37924,"tokens_out":5191,"duration_ms":43086,"significance":"If the findings hold, this review provides a useful and much-needed map of an emerging, rapidly growing area at the intersection of equity and technology. Strengths include a transparent PRISMA-based process with explicit search strings, venue lists, inclusion criteria, and coding categories; the authors report inter-rater reliability (moderate Cohen's kappa) and openly discuss limitations, including the exclusion of relevant work that does not use the literal term 'equit*'. The typologies of definitions, motivations, interventions, and tensions are genuinely useful for future researchers, and the proposed framework and recommendations are actionable. The paper also makes a valuable empirical contribution by quantifying the HCI/Fairness gap in definitional clarity and methodological approach.","major_comments":[{"comment":"The headline statistic that only 27% (54/202) of the corpus explicitly defines equity is computed over all 202 papers, including the 42 papers (21%) the authors themselves classify as 'equity merely as motivation' (Table 2). These papers are described as mentioning equity only to motivate the work without engaging with the concept, so they are unlikely to contain formal definitions; their inclusion in the denominator mechanically depresses the reported rate. If none of the 42 motivation-only papers define equity, the rate among the 160 equity-focused papers would be 54/160 ≈ 34%; if some do, the rate is higher still. The paper neither reports this sensitivity analysis nor breaks down definition rates by the papers' equity engagement category. Because this statistic anchors the central claim that the lack of explicit definitions hinders a shared equity framework, the authors should report definition rates separately for motivation-only and equity-focused papers, and re-state the central claim accordingly.","section":"§5.1, §7.1 (Table 2)"},{"comment":"The reliability of the coding that produces the 'equity merely as motivation' category and the intervention-type percentages is not fully established. Open coding was conducted without inter-rater reliability checks, and the closed-coding kappa for the entire corpus is only 0.66 (moderate). Since the size of the motivation-only category directly enters the definition-rate denominator (Major Comment 1) and the intervention-type percentages in Table 2 are a core contribution, the authors should provide per-category reliability or a consensus process demonstrating that these categories are stable. Without this, the magnitude of the motivation-only category, and therefore the corrected definition rate, remains uncertain.","section":"§3.3.1, §3.3.2, §7.1"},{"comment":"The search string requires 'equit*' or 'inequit*' in the title, abstract, or keywords, which excludes relevant equity research like Branham et al. [21], as the authors acknowledge in Section 9. As a result, the corpus constitutes papers that explicitly use the term 'equity,' not the full universe of equity-related research in HCI and Fairness venues. The central claims about growth rates, the definitional gap, and the HCI/Fairness contrast are therefore conditional on this operationalization. The authors should either narrow the title and abstract claims accordingly, or add a sensitivity analysis using related terms (e.g., 'inequality,' 'social justice,' 'accessibility') to test whether the reported patterns persist.","section":"§3.1, §9"}],"minor_comments":[{"comment":"The text states that the authors 'categorized all papers into six categories' but then lists five categories (Empirical-Qualitative, Empirical-Quantitative, Technical/Systems, Design, Theoretical); please correct the count.","section":"§5.3"},{"comment":"There is a duplicated word in 'responsibility for the space space'; please fix the typo.","section":"§8.2"},{"comment":"The PRISMA flow diagram is referenced as Figure 1, but the text does not report the exact numbers at each PRISMA stage (identified, screened, excluded, included). Please add a completed flow diagram with counts.","section":"§3.1, Figure 1"},{"comment":"The list of the 202 included papers is not provided in the appendix or as supplementary material; making the corpus list available would improve reproducibility and allow readers to verify the coding.","section":"Appendix"},{"comment":"The paragraph on growth correctly excludes 2023 papers from Figure 2, but the text could be clearer about the data collection cutoff (September 2023) and that the figure's x-axis ends at 2022.","section":"§4.1.1"},{"comment":"The term 'equality-based definition' might be confused with the equality/equity distinction used elsewhere in the paper; consider adding a brief clarification that this category refers to papers that operationalize equity as statistical parity or equal treatment.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this review is worth refereeing. It gives the HCI/fairness community something it didn't have: a map of how 'equity' is actually used across 202 papers, with a definition taxonomy and intervention typology. But treat the headline 'only 27% define equity' with caution. The denominator includes 42 papers the authors label 'equity merely as motivation' (Section 7.1). Those papers were never about equity, and they're unlikely to define it. Recomputing over the 160 equity-focused papers gives roughly 34% (54/160), still low but less dramatic. The HCI/fairness gap (21% vs 40%) may persist, but the paper doesn't report this sensitivity analysis, and it matters because the central claim leans on it.\n\nThe real contributions are the qualitative ones: the three definition categories (need/contribution/equality), the four intervention types (monitoring/surfacing, system/algorithm, framework, stakeholder participation), and the seven recommendations. These are induced from the corpus in a transparent way, and they ring true. The PRISMA workflow is documented carefully, including the iterative keyword development.\n\nSoft spots, in order of severity. (1) The keyword search (equit*/inequit* in title/abstract/keyword) is narrow and the authors know it; they name Branham et al. as a missed paper. That's fine for a scoping review, but it means every percentage is about a subset, not the field. (2) Closed-coding reliability is 0.66, moderate; open-coding reliability is unreported. For some claims (e.g., 47% vs 10% monitoring) that's a real limitation. (3) Table 1 and the text disagree on AIES/EAAMO counts (text: AIES 9, EAAMO 12; table: AIES 12, EAAMO 9). Sloppy but fixable. (4) The 'only 6 authors overlap' statistic is interesting but not interpreted much.\n\nWho benefits: anyone entering equity research in HCI or fairness, and anyone teaching it. The framework is a good checklist. I'd accept for peer review; the revisions are minor (fix the counts, report sensitivity on the definition rate, acknowledge coding reliability). The central argument - equity is fragmented and often undefined, and the two communities approach it differently - holds up.","headline":"Useful map of a fragmented field, but the 'only 27% define equity' headline overstates the gap once you exclude the 42 motivation-only papers the authors themselves classify as not equity-focused.","tokens_in":38464,"tokens_out":2391,"would_cite":true,"duration_ms":24877,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Only 27% of equity-tech papers define the term, review finds","keywords":["equity","technology","systematic literature review","HCI","algorithmic fairness","equity definitions","participatory design","value tensions"],"falsifier":"Re-run the same review with broader search stems (e.g., \"accessib*\", \"social justice\", \"inclusi*\", \"fairness\") across the same venues, or with full-text search instead of title/abstract/keywords, and check whether the 27% explicit-definition rate and the HCI/fairness definition gap survive; if the rate rises substantially or the gap narrows, the paper's central portrait describes only a subset of the field.","tokens_in":37446,"feed_emoji":"⚖️","tokens_out":5815,"duration_ms":49456,"temperature":0.7,"pith_summary":"Across 202 papers from HCI and fairness-focused venues, this review argues that equity research is growing quickly but is fragmented because researchers rarely say what they mean by equity. Only 27% of the papers give an explicit definition, and the gap is wider in HCI (21%) than in fairness venues (40%). The paper maps the definitions that do appear into three types—need-based, contribution-based, and equality-based—and shows that the two communities approach equity differently: HCI mostly surfaces inequities through qualitative work, while fairness venues mostly build systems, algorithms, and frameworks. It concludes that the missing shared vocabulary hampers the development of a common framework for equity in technology, and it offers a four-dimension framework with seven recommendations for future research.","feed_headline":"Only 27% of equity-tech papers define the term, review finds","feed_subtitle":"HCI and fairness researchers split on methods and values, and the missing definition blocks shared progress.","key_machinery":"The carrying mechanism is the systematic literature review itself, built on a structured review protocol: a corpus of 202 papers retrieved by searching for \"equit*\" or \"inequit*\" in titles, abstracts, or keywords of the selected HCI and fairness venues, then filtered through title-and-abstract screening. The argument is carried by the coding scheme the authors apply to that corpus: closed coding for definitions, methods, and publication data, and open coding for motivations, interventions, and tensions. The three-way partition of equity definitions (need-based, contribution-based, equality-based), the four-part taxonomy of research approaches (equity as motivation, surfacing inequities, interventions, participation), and the resulting four-dimension equity framework are the concrete objects that support the paper's conclusions.","core_discovery":"The paper's central discovery is a portrait of an emerging field that has not yet cohered. Using a systematic literature review of 202 research articles, the authors find a surge in equity-related publications from 2018 to 2022, alongside a persistent absence of explicit equity definitions: 73% of the corpus uses \"equity\" without defining it, and HCI papers are about half as likely as fairness papers to define the term. Where definitions exist, they fall into need-based, contribution-based, and equality-based categories that are not interchangeable. The authors also show that researchers' motivations cluster around existing societal and technological inequities, their methods split sharply by community, and their efforts face tensions between equity and utility values, social values, and equity itself. On this basis, the paper argues that the lack of clear and explicit definitions—especially in HCI—may hinder the common framework and shared understanding the field needs.","pith_inferences":["Going beyond the paper, the definition gap it measures may be partly a vocabulary artifact: work that advances equity under labels like accessibility or social justice without using \"equit*\" was excluded by design, so the true share of definition-rich equity work could be higher than 27%.","A testable extension would be to code the same corpus for which stakeholder groups are centered in each definition type; need-based definitions may associate with participatory methods and equality-based definitions with statistical parity metrics.","If the field adopted the three-type definitional taxonomy, one could ask whether definition type predicts intervention choice—for instance, whether equality-based definitions lead to fairness metrics and need-based definitions lead to resource-allocation systems.","The authors' caution against techno-solutionism leaves open a harder question: whether some equity goals are better served by non-technological policy changes, which the paper gestures at but does not systematically analyze."],"forward_implications":["If the pattern holds, a paper claiming to advance \"equity\" is currently doing so without a shared referent, so readers and reviewers cannot tell whether findings are comparable across studies.","Explicitly choosing among need-based, contribution-based, or equality-based definitions would make equity claims checkable and let results accumulate instead of colliding.","HCI's strength in surfacing inequities and fairness venues' strength in building interventions could be combined, since only 6 authors in the corpus publish in both communities.","Researchers who report value tensions and trade-offs—equity vs. utility, social values, or other equity—can use the four-dimension framework to state what they prioritized and why.","Treating equity as a mere motivation, as 21% of the corpus does, is identified as a gap that the framework's recommendations target."],"supporting_citations":[{"why":"Supplies the systematic review workflow used to build and screen the corpus.","marker":"[97]"},{"why":"Provides the input-versus-outcome ratio conception behind the contribution-based definition category.","marker":"[3]"},{"why":"Supplies the feminist HCI principles used as a lens and framework category.","marker":"[11]"},{"why":"Serves as an example of a need-based definition combined with a participatory algorithmic-governance framework.","marker":"[81]"},{"why":"Illustrates the equity-equality distinction in an AI fairness framework.","marker":"[121]"},{"why":"Provides an intervention example built on a need-based definition of equity.","marker":"[99]"},{"why":"Offers an algorithmic intervention designed to reduce racial inequity.","marker":"[32]"},{"why":"Supplies an equality-based definition of equity as equal statistics across groups.","marker":"[124]"}],"fun_headline_variants":["Most equity-tech papers skip defining equity, review finds","Equity research spikes but lacks shared definition, study shows","HCI lags fairness in defining equity in tech papers","202-paper review reveals equity's definition gap in tech","Equity tech papers surge, but definitions remain scarce"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole quantitative portrait depends on the search assumption that papers about equity and technology will use the literal stems \"equit*\" or \"inequit*\" in the title, abstract, or keywords of the selected venues—an assumption the authors concede misses relevant work such as Branham et al. [21].","fun_headline_variants_meta":{"raw":{"variants":["Most equity-tech papers skip defining equity, review finds","Equity research spikes but lacks shared definition, study shows","HCI lags fairness in defining equity in tech papers","202-paper review reveals equity's definition gap in tech","Equity tech papers surge, but definitions remain scarce"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000229,"raw_usage":{"total_tokens":1473,"prompt_tokens":935,"completion_tokens":538,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":551,"completion_tokens_details":{"reasoning_tokens":459}},"tokens_in":551,"tokens_out":538,"duration_ms":4849,"temperature":1.0,"reasoning_tokens":459,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:49:02.770191+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same review with broader search stems (e.g., \"accessib*\", \"social justice\", \"inclusi*\", \"fairness\") across the same venues, or with full-text search instead of title/abstract/keywords, and check whether the 27% explicit-definition rate and the HCI/fairness definition gap survive; if the rate rises substantially or the gap narrows, the paper's central portrait describes only a subset of the field.","supporting_citations":[],"review_version":1}