Can AI Make Conflicts Worse? An Alignment Failure in LLM Deployment Across Conflict Contexts

Referee: Methods section on scenario construction: The manuscript provides no details on how the 90 multi-turn scenarios were developed, including selection criteria for conflict contexts, diversity across regions or conflict types, or any expert validation of their realism. This is load-bearing for the central claim, as the reported failure rates (including 80-100% under balance prompts) and the inference that outputs can worsen conflicts depend on these scenarios accurately representing deployment risks.

Authors: We agree that additional details on scenario construction are necessary to allow readers to assess the validity of our test cases. In the revised manuscript, we will add a dedicated subsection in Methods describing the development process. This will include: (1) selection criteria based on conflicts with documented international legal findings (e.g., ICJ or ICC rulings); (2) efforts to ensure diversity across regions and conflict types (e.g., including examples from the Middle East, sub-Saharan Africa, and Eastern Europe); and (3) the process of grounding scenarios in publicly available reports and court documents to enhance realism. We note that while informal internal review was conducted, we will clarify this in the revision. These additions will strengthen the transparency of our evaluation framework. revision: yes

Referee: Evaluation and scoring subsection: No information is given on how outputs were scored for misalignment, the statistical methods for computing failure rates, inclusion of error bars, or inter-annotator agreement if human judgment was used. Without these, the model comparisons (6% to 47% range) and the claim of systematic alignment failure cannot be fully assessed for robustness.

Authors: We acknowledge the need for greater methodological transparency in the evaluation process. In the revised version, we will expand the Evaluation subsection to detail: the criteria used for scoring misalignment (with examples of aligned vs. misaligned responses for each behavior type); the statistical approach for calculating failure rates (simple proportions with binomial confidence intervals); and confirmation that scoring was performed by the authors with a second reviewer for a subset to assess consistency. If human judgment was involved, we will report inter-annotator agreement metrics. This will allow better evaluation of the robustness of our findings. revision: yes

Referee: Discussion of real-world implications: The paper assumes without additional evidence that misaligned outputs in the tested scenarios would translate to measurable societal harm or deepened divisions when deployed in journalism or humanitarian work. This causal link is central to the significance but remains an untested modeling assumption rather than a demonstrated result.

Authors: We appreciate this point and agree that direct empirical evidence of societal harm from these specific outputs is not provided, as conducting such field studies would be beyond the scope of this initial evaluation paper. However, the discussion is grounded in established research on the role of media and information in conflict escalation (e.g., studies on hate speech and false equivalence in reporting). In the revision, we will revise the Discussion to more explicitly frame the implications as potential risks based on logical mechanisms and prior literature, while adding a limitations section acknowledging the inferential nature of the harm claim and calling for future empirical work on deployment effects. This addresses the concern without overstating the current results. revision: partial