From Attribution to Action: A Human-Centered Application of Activation Steering

Referee: [Section 4] Section 4 (User Study / Methodology): The description of the semi-structured interview protocol, participant selection criteria, exact debugging tasks, interview guide, qualitative coding process, and any bias-mitigation steps (e.g., inter-rater reliability) is insufficiently detailed. Because the central claims consist of specific fractions (8/8, 6/8, 7/8) derived from these interviews, the absence of this information prevents verification that the reported patterns are not artifacts of task framing or analysis choices.

Authors: We agree that the current description of the methodology is insufficient for independent verification. In the revised manuscript we will substantially expand Section 4 to include: (1) the complete semi-structured interview protocol and the full interview guide with example questions; (2) explicit participant selection criteria, including recruitment channels, years of experience in ML interpretability, and prior exposure to vision-language models; (3) precise descriptions of the three debugging tasks, including the images, target concepts, and success criteria given to participants; (4) the qualitative analysis procedure, specifying the thematic coding approach, how the 8/8, 6/8, and 7/8 counts were derived, and any inter-rater reliability assessment (we will report Cohen’s kappa or describe the consensus process used); and (5) bias-mitigation steps such as pilot interviews, question ordering to avoid leading participants, and steps taken to reduce confirmation bias during coding. These additions will allow readers to evaluate whether the observed patterns could be artifacts of task framing or analysis choices. revision: yes

Referee: [Findings and Discussion] Findings and Discussion sections: The extrapolation from N=8 experts performing CLIP-specific debugging to general practitioner reasoning about activation steering is load-bearing for the paper's broader utility claim, yet the manuscript provides no additional validation (e.g., larger sample, different models, or quantitative measures) and only limited caveats regarding selection bias and task specificity. This weakens the link between the observed behaviors and the asserted shift toward actionable interpretability.

Authors: We acknowledge that the manuscript’s language occasionally implies broader applicability than the N=8, CLIP-specific evidence strictly supports. While qualitative studies of this size are common in human-centered XAI for generating initial insights, we agree that stronger caveats and clearer scoping are needed. In the revision we will: (1) expand the Limitations subsection to explicitly address selection bias (participants were drawn from a convenience sample of interpretability researchers), task specificity (debugging tasks were limited to CLIP on natural-image classification), and the absence of quantitative or cross-model validation; (2) revise the Findings and Discussion sections to use more qualified language (e.g., “in this study, all participants…” rather than generalizing to “practitioners”); and (3) add a forward-looking paragraph outlining concrete next steps for larger-scale or quantitative follow-up studies. We maintain that the observed shift from inspection to intervention-based reasoning is a substantive finding for the studied setting and provides actionable design implications, but we will no longer present it as a general claim without further evidence. revision: partial