The Hardness of Achieving Impact in AI for Social Impact Research: A Ground-Level View of Challenges & Opportunities
Pith reviewed 2026-05-21 23:45 UTC · model grok-4.3
The pith
Structural, organizational, and collaboration challenges often block AI for social impact projects from achieving real-world deployment.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Through thematic analysis of interviews with twenty-six AI4SI researchers and the authors' own experiences, this paper identifies structural, organizational, communication, collaboration, and operational challenges that hinder socially impactful AI4SI deployments. It synthesizes best practices and actionable strategies from these sources, offering a practical guide for researchers and organizations seeking to pursue effective collaborations despite the absence of easy fixes.
What carries the argument
Thematic analysis applied to semi-structured interviews with AI4SI researchers, used to surface and categorize barriers and to derive strategies for overcoming them.
If this is right
- More AI4SI projects could progress to production-level deployment if teams address the identified collaboration and operational challenges using the suggested strategies.
- Academic organizations can improve support for socially impactful work by tackling the structural and organizational issues highlighted.
- Researchers may find it easier to identify suitable collaborators by following the communication best practices outlined.
- Overall, the field of AI4SI can reduce the number of stalled projects by incorporating these insights into planning and execution.
Where Pith is reading between the lines
- The challenges described may vary in emphasis for projects based in the global south or led by governmental or startup teams, suggesting value in targeted follow-up studies.
- Similar patterns of deployment difficulties could appear in other applied AI areas, such as AI for education or public policy, pointing to shared interdisciplinary hurdles.
- Testing the proposed strategies in actual AI4SI projects would provide evidence on their effectiveness in practice.
Load-bearing premise
The challenges identified from the sample of mostly academic researchers in the global north represent the main obstacles faced by AI4SI projects more broadly.
What would settle it
Interviewing a more diverse set of AI4SI practitioners including those from startups, government agencies, and the global south and discovering markedly different primary barriers would indicate that the current findings do not fully capture the range of issues.
read the original abstract
AI for Social Impact (AI4SI) is an emergent field harnessing interdisciplinarities between the fields of artificial intelligence (AI), machine learning (ML), and the social sciences to address societal issues aligned with the United Nations Sustainable Development Goals (UN SDGs), such as universal healthcare, climate action, etc. Despite AI4SI's rising popularity, achieving tangible, on-the-ground impact remains a significant challenge. In particular, identifying collaborators open to co-designing and deploying AI4SI-based solutions in real-world settings is often difficult. Thus, many projects stall at the proof-of-concept stage, unable to scale to production-level deployment. Drawing on twenty-six AI4SI researchers' interviews, primarily from academic institutions though also including some industry researchers and practitioners, and the authors' own lived experiences, this paper employs thematic analysis to highlight structural, organizational, communication, collaboration, and operational challenges hindering socially impactful AI4SI deployments. While there are no easy fixes, the authors synthesize best practices and actionable strategies from interviews and personal experiences, positioning this paper as a practical guide for AI4SI researchers and organizations pursuing socially impactful collaborations$^1$. $^1$We note that our findings are most directly applicable to academic research groups in the global north, as governmental, startup, and global south researchers' perspectives are underrepresented in our sample.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that thematic analysis of interviews with 26 AI4SI researchers (primarily academic from global north institutions, with some industry/practitioner input) plus the authors' lived experiences can surface structural, organizational, communication, collaboration, and operational challenges that prevent scaling AI projects from proof-of-concept to real-world impact on UN SDGs; it synthesizes best practices and actionable strategies from this data, positioning the work as a practical guide while explicitly qualifying in Footnote 1 that findings apply most directly to academic research groups in the global north due to underrepresentation of governmental, startup, and global south perspectives.
Significance. If the bounded findings hold, the work provides a useful empirical contribution to the emerging AI4SI field by documenting persistent ground-level barriers and offering concrete strategies drawn from interviews and experience. The transparent scope qualification in Footnote 1 directly addresses potential overgeneralization concerns, and the use of standard thematic analysis on external interview data (rather than self-referential logic) lends credibility. This can help academic teams improve collaborator identification and deployment success without claiming universality.
major comments (1)
- [Methods] Methods section (thematic analysis description): The paper should provide additional detail on how themes were iteratively developed, how authors' personal experiences were kept distinct from interview-derived themes during coding, and any steps taken for validation or member-checking; without this, the synthesis of challenges and best practices rests on a process whose rigor is harder to assess even within the scoped sample.
minor comments (2)
- [Abstract] Abstract: The parenthetical description of the sample could be tightened for precision (e.g., explicitly noting the 26 interviews and the footnote qualification) to better foreground the bounded nature of the claims.
- [Findings and Discussion] Throughout: Ensure consistent use of 'AI4SI' abbreviation after first definition and verify that all synthesized strategies are clearly linked back to either interview excerpts or authors' experiences to prevent reader conflation.
Simulated Author's Rebuttal
We thank the referee for their constructive review and recommendation for minor revision. We address the major comment on the methods section below.
read point-by-point responses
-
Referee: [Methods] Methods section (thematic analysis description): The paper should provide additional detail on how themes were iteratively developed, how authors' personal experiences were kept distinct from interview-derived themes during coding, and any steps taken for validation or member-checking; without this, the synthesis of challenges and best practices rests on a process whose rigor is harder to assess even within the scoped sample.
Authors: We agree that additional methodological transparency will strengthen the paper. In the revised manuscript we will expand the Methods section to describe the iterative coding process in greater detail, including the sequence of open coding, theme grouping, and refinement across multiple passes on the 26 interview transcripts. We will clarify that authors' lived experiences were not incorporated into the initial coding or theme generation; they were used only after theme identification to help interpret findings and synthesize strategies. We will also add that formal member-checking was not performed due to participant time constraints and the exploratory scope of the study, but that the author team conducted internal peer debriefing sessions to review and validate emerging themes against the raw data. These additions will make the analytical rigor more readily assessable within the acknowledged sample boundaries. revision: yes
Circularity Check
No significant circularity; empirical qualitative study grounded in external data
full rationale
This paper is a qualitative empirical study that performs thematic analysis on interview data from 26 external AI4SI researchers plus authors' lived experiences. It contains no equations, derivations, predictions, fitted parameters, or mathematical claims that could reduce to self-referential inputs. The central claims about challenges and best practices are synthesized directly from the interview transcripts and personal accounts rather than from internal definitions or self-citation chains. Footnote 1 explicitly bounds the applicability to academic groups in the global north due to sample limitations, preventing any unsupported universality. The derivation chain is therefore self-contained against external benchmarks with no load-bearing steps that collapse by construction.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Thematic analysis of semi-structured interviews with AI4SI researchers can reliably identify key structural and operational challenges in the field.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.