Quantifying Harm

Hana Chockler; Joseph Y. Halpern; Sander Beckers

arxiv: 2209.15111 · v3 · submitted 2022-09-29 · 💻 cs.AI

Quantifying Harm

Sander Beckers , Hana Chockler , Joseph Y. Halpern This is my paper

Pith reviewed 2026-05-24 11:01 UTC · model grok-4.3

classification 💻 cs.AI

keywords harm quantificationsocietal harmexpected harmaggregationdecision theoryprecision medicineuncertaintycausal reasoning

0 comments

The pith

The naive approach of summing expected individual harms can lead to counterintuitive or inappropriate measures of societal harm.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a quantitative definition of harm, beginning with a deterministic setting for a single individual. It then incorporates uncertainty about the context and extends the notion to societal harm by aggregating across multiple individuals. The authors show that the straightforward method of computing expected harm for each person and then adding those values across people can produce results that conflict with intuitive judgments about overall harm. They examine alternative aggregation approaches drawn from decision theory to address these issues. The work connects these ideas to ongoing debates about measuring harm in precision medicine.

Core claim

The paper defines a quantitative notion of harm in a deterministic context for a single individual, extends the definition to handle uncertainty, and demonstrates that the obvious aggregation method of taking expected harm per individual and summing over individuals can lead to counterintuitive or inappropriate answers for societal harm, while discussing alternatives from the decision-theory literature and linking the results to precision medicine.

What carries the argument

The quantitative harm measure, first defined for deterministic single-individual cases and then extended via aggregation functions to handle uncertainty and multiple individuals.

If this is right

Choosing the least harmful intervention requires selecting an aggregation method that avoids the counterintuitive results of simple summation.
Societal harm calculations benefit from considering decision-theoretic alternatives rather than defaulting to expected-harm summation.
In precision medicine, the choice of harm aggregation affects which treatments are identified as least harmful overall.
Handling uncertainty in harm assessment calls for methods beyond direct expectation before aggregation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The identified aggregation problems may require incorporating risk attitudes or distributional concerns when defining societal harm.
Testing the alternatives in specific medical case studies could reveal whether they better match prevailing ethical intuitions about total harm.
The framework could be applied to algorithmic decision systems to ensure their harm calculations align with non-naive aggregation principles.

Load-bearing premise

That a single quantitative measure of harm exists that remains meaningful when aggregated across individuals and uncertainty without requiring additional normative commitments beyond the definition itself.

What would settle it

A concrete multi-individual scenario with outcome uncertainty in which the ranking of interventions by summed expected harm differs from the ranking produced by at least one decision-theoretic alternative, and where independent ethical judgment aligns with the alternative ranking.

read the original abstract

In earlier work we defined a qualitative notion of harm: either harm is caused, or it is not. For practical applications, we often need to quantify harm; for example, we may want to choose the least harmful of a set of possible interventions. In this work, which is an expanded version of an earlier conference paper, we develop a quantitative notion of harm. We first present a quantitative definition of harm in a deterministic context involving a single individual, then we consider the issues involved in dealing with uncertainty regarding the context and going from a notion of harm for a single individual to a notion of "societal harm", which involves aggregating the harm to individuals. We show that the "obvious" way of doing this (just taking the expected harm for an individual and then summing the expected harm over all individuals) can lead to counterintuitive or inappropriate answers, and discuss alternatives, drawing on work from the decision-theory literature. Finally, we connect our work to a recent debate over harm within the context of precision medicine.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper extends the authors' qualitative harm definition to a quantitative version for single agents and shows why simply summing expected harms across people can give odd results, but stays at the level of definitions and intuitions.

read the letter

The main takeaway is that Beckers, Chockler, and Halpern move from their earlier qualitative account of harm to a quantitative one. They start with a deterministic single-individual case, then add uncertainty, and finally address aggregation into societal harm. The useful part is their observation that taking expected harm per person and adding across individuals can produce counterintuitive totals, and they point to decision-theory options as possible fixes. They also note the link to precision-medicine debates. This is a direct follow-on from their conference version, so the quantitative step and the aggregation warning are the actual increments. The aggregation critique is worth having on record because anyone applying harm measures to groups or policies will run into the same issue. The paper does a clean job of laying out why the obvious summation fails without overclaiming a full solution. The soft spots are that the work remains almost entirely definitional. There are no formal theorems, no worked numerical examples with explicit models, and no external checks on whether the proposed quantitative measure behaves as intended under realistic uncertainty. The idea that a single harm number can be aggregated without bringing in further normative choices looks optimistic on the evidence given; the paper itself ends up appealing to outside literature for alternatives. It also rests heavily on the authors' own prior qualitative papers, so the advance is incremental rather than a fresh framework. This is for readers already working in causal models, responsibility, or AI ethics who know the qualitative version and want to see the quantitative extension. Someone outside that circle will find it thin on concrete guidance. I would send it to peer review because the aggregation point is a legitimate caution that deserves discussion in the subfield, even if the paper needs more development to be self-contained.

Referee Report

0 major / 3 minor

Summary. The paper extends the authors' prior qualitative definition of harm to a quantitative version. It first presents a quantitative definition for harm in a deterministic single-individual context, then examines extensions to uncertainty and aggregation across individuals to define societal harm. The central observation is that the naive approach of taking expected harm per individual and summing across individuals can produce counterintuitive or inappropriate results; alternatives are discussed drawing on decision-theory literature. The work concludes by relating the framework to ongoing debates in precision medicine.

Significance. If the quantitative definition and the identified aggregation issues hold, the paper could contribute a more careful framework for harm assessment in AI ethics, policy, and medical decision-making by underscoring the normative choices involved in aggregation under uncertainty. Credit is given for explicitly engaging decision-theoretic alternatives rather than stopping at the critique of summation. As a primarily definitional and conceptual exercise without new empirical tests, formal theorems, or falsifiable predictions, its significance rests on whether the distinctions prove useful in downstream applications.

minor comments (3)

[Societal harm / aggregation section] The aggregation discussion would benefit from one or two fully worked numerical examples (with explicit utilities or probabilities) showing precisely where summation diverges from the proposed alternatives.
[Precision medicine connection] The final section connecting to precision medicine would be strengthened by citing one or two specific papers or positions in that debate and indicating exactly which aspect of the quantitative harm measure bears on them.
[Introduction / quantitative definition section] Clarify in the introduction or §2 how the new quantitative definition relates formally to the authors' earlier qualitative definition (e.g., whether the quantitative version is a conservative extension or introduces new parameters).

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their review and for recommending minor revision. We appreciate the acknowledgment that the paper engages decision-theoretic alternatives and recognize the note that significance will depend on downstream usefulness.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper begins from the authors' prior qualitative definition of harm (explicitly cited as earlier work) and extends it by introducing new quantitative definitions for deterministic single-agent cases, then addresses uncertainty and aggregation by drawing on external decision-theory literature. No load-bearing step reduces a claimed prediction or theorem to a fitted parameter, self-referential equation, or unverified self-citation chain; the quantitative notions are presented as definitional extensions rather than derived outputs that loop back to inputs by construction. The work remains conceptual and self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Paper is definitional and extends authors' prior qualitative framework; relies on standard counterfactual notions of harm and decision-theoretic aggregation principles without introducing new fitted parameters or invented entities in the abstract.

axioms (2)

domain assumption Harm is defined relative to a counterfactual baseline of what would have occurred otherwise.
Invoked when moving from qualitative to quantitative harm for a single individual.
domain assumption Aggregation of individual harms requires normative choices drawn from decision theory rather than pure summation.
Central to the critique of the obvious aggregation method.

pith-pipeline@v0.9.0 · 5699 in / 1216 out tokens · 27912 ms · 2026-05-24T11:01:45.833726+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/AbsoluteFloorClosure.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

quantitative harm ... max(0, min(d, u(o')) − u(o)) ... probability weighting ... fairness penalty α

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages

[1]

Beckers, S. (2021). Causal sufficiency and actual causation. Journal of Philosophical Logic\/ 50 , 1341--1374

work page 2021
[2]

Beckers, S., Chockler, H., and Halpern, J.Y. (2022). A Causal Analysis of Harm. Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS)\/

work page 2022
[3]

Bradley, B. (2012). Doing away with harm. Philosophy and Phenomenological Research\/ 85 , 390–412

work page 2012
[4]

Johansson, and O

Carlson, E., J. Johansson, and O. Risberg (2021). Causal accounts of harming. Pacific Philosophical Quarterly\/

work page 2021
[5]

European Commission (2021). Proposal for a regulation of the E uropean parliament and of the council laying down harmonised rules on artificial intelligence ( A rtificial I ntelligence A ct) and amending certain U nion legislative acts. h ttps://artificialintelligenceact.eu/the-act/; accessed Aug. 8, 2021

work page 2021
[6]

G\"ardenfors, P. and N. Sahlin (1982). Unreliable probabilities, risk taking, and decision making. Synthese\/ 53 , 361--386

work page 1982
[7]

Gigerenzer, G. (2006). Out of the frying pan into the fire: behavioral reactions to terrorist attacks. Risk Analysis\/ 26\/ (2), 347--351

work page 2006
[8]

Gilboa, I. and D. Schmeidler (1989). Maxmin expected utility with a non-unique prior. Journal of Mathematical Economics\/ 18 , 141--153

work page 1989
[9]

Glymour, C. and F. Wimberly (2007). Actual causes and thought experiments. In J. Campbell, M. O'Rourke, and H. Silverstein (Eds.), Causation and Explanation , pp.\ 43--67. Cambridge, MA: MIT Press

work page 2007
[10]

Hall, N. (2007). Structural equations and causation. Philosophical Studies\/ 132 , 109--136

work page 2007
[11]

Halpern, J. Y. (2015). A modification of the H alpern- P earl definition of causality. In Proc. 24th International Joint Conference on Artificial Intelligence (IJCAI 2015) , pp.\ 3022--3033

work page 2015
[12]

Halpern, J. Y. (2016). Actual Causality . Cambridge, MA: MIT Press

work page 2016
[13]

Halpern, J. Y. and J. Pearl (2005). Causes and explanations: a structural-model approach. P art I : C auses. British Journal for Philosophy of Science\/ 56\/ (4), 843--887

work page 2005
[14]

Barocas, J

Heidari, H., S. Barocas, J. M. Kleinberg, and K. Levy (2021). On modeling human perceptions of allocation policies with uncertain outcomes. In EC '21: The 22nd ACM Conference on Economics and Computation , pp.\ 589--609. ACM

work page 2021
[15]

Barron, E

Hertwig, R., G. Barron, E. Weber, and I. Erev (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science\/ 15\/ (8), 534--539

work page 2004
[16]

Hitchcock, C. (2001). The intransitivity of causation revealed in equations and graphs. Journal of Philosophy\/ XCVIII\/ (6), 273--299

work page 2001
[17]

Hitchcock, C. (2007). Prevention, preemption, and the principle of sufficient reason. Philosophical Review\/ 116 , 495--532

work page 2007
[18]

Jenni, K. and G. Loewenstein (1997). Explaining the identifiable victim effect. Journal of Risk and Uncertainty\/ 14\/ (3), 235--257

work page 1997
[19]

Kahneman, D. and A. Tversky (1979). Prospect theory, an analysis of decision under risk. Econometrica\/ 47\/ (2), 263--292

work page 1979
[20]

Niehans, J. (1948). Zur preisbildung bei ungewissen erwartungen. Schweizerische Zeitschrift f \" u r Volkswirtschaft und Statistik\/ 84\/ (5), 433--456

work page 1948
[21]

Norcross, A. (1998). Great harms from small benefits grow: how death can be outweighed by headaches. Analysis\/ 58\/ (2), 152--158

work page 1998
[22]

Prelec, D. (1998). The probability weighting function. Econometrica\/ 66\/ (3), 497--527

work page 1998
[23]

Quiggin, J. (1993). Generalized Expected Utility Theory: The Rank-Dependent Expected Utility Model . Boston: Kluwer

work page 1993
[24]

Richens, J. G., R. Beard, and D. H. Thompson (2022). Counterfactual harm. a rXiv preprint https://arxiv.org/abs/2204.12993, version v2

work page arXiv 2022
[25]

Savage, L. J. (1951). The theory of statistical decision. Journal of the American Statistical Association\/ 46 , 55--67

work page 1951
[26]

Wald, A. (1950). Statistical Decision Functions . New York: Wiley

work page 1950
[27]

Weslake, B. (2015). A partial theory of actual causation. British Journal for the Philosophy of Science\/ . To appear

work page 2015
[28]

Woodward, J. (2003). Making Things Happen: A Theory of Causal Explanation . Oxford, U.K.: Oxford University Press

work page 2003
[29]

Zielonka, P. and T. Tyszka (Eds.) (2017). Large Risks with Low Probabilities . IWA Publishing

work page 2017

[1] [1]

Beckers, S. (2021). Causal sufficiency and actual causation. Journal of Philosophical Logic\/ 50 , 1341--1374

work page 2021

[2] [2]

Beckers, S., Chockler, H., and Halpern, J.Y. (2022). A Causal Analysis of Harm. Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS)\/

work page 2022

[3] [3]

Bradley, B. (2012). Doing away with harm. Philosophy and Phenomenological Research\/ 85 , 390–412

work page 2012

[4] [4]

Johansson, and O

Carlson, E., J. Johansson, and O. Risberg (2021). Causal accounts of harming. Pacific Philosophical Quarterly\/

work page 2021

[5] [5]

European Commission (2021). Proposal for a regulation of the E uropean parliament and of the council laying down harmonised rules on artificial intelligence ( A rtificial I ntelligence A ct) and amending certain U nion legislative acts. h ttps://artificialintelligenceact.eu/the-act/; accessed Aug. 8, 2021

work page 2021

[6] [6]

G\"ardenfors, P. and N. Sahlin (1982). Unreliable probabilities, risk taking, and decision making. Synthese\/ 53 , 361--386

work page 1982

[7] [7]

Gigerenzer, G. (2006). Out of the frying pan into the fire: behavioral reactions to terrorist attacks. Risk Analysis\/ 26\/ (2), 347--351

work page 2006

[8] [8]

Gilboa, I. and D. Schmeidler (1989). Maxmin expected utility with a non-unique prior. Journal of Mathematical Economics\/ 18 , 141--153

work page 1989

[9] [9]

Glymour, C. and F. Wimberly (2007). Actual causes and thought experiments. In J. Campbell, M. O'Rourke, and H. Silverstein (Eds.), Causation and Explanation , pp.\ 43--67. Cambridge, MA: MIT Press

work page 2007

[10] [10]

Hall, N. (2007). Structural equations and causation. Philosophical Studies\/ 132 , 109--136

work page 2007

[11] [11]

Halpern, J. Y. (2015). A modification of the H alpern- P earl definition of causality. In Proc. 24th International Joint Conference on Artificial Intelligence (IJCAI 2015) , pp.\ 3022--3033

work page 2015

[12] [12]

Halpern, J. Y. (2016). Actual Causality . Cambridge, MA: MIT Press

work page 2016

[13] [13]

Halpern, J. Y. and J. Pearl (2005). Causes and explanations: a structural-model approach. P art I : C auses. British Journal for Philosophy of Science\/ 56\/ (4), 843--887

work page 2005

[14] [14]

Barocas, J

Heidari, H., S. Barocas, J. M. Kleinberg, and K. Levy (2021). On modeling human perceptions of allocation policies with uncertain outcomes. In EC '21: The 22nd ACM Conference on Economics and Computation , pp.\ 589--609. ACM

work page 2021

[15] [15]

Barron, E

Hertwig, R., G. Barron, E. Weber, and I. Erev (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science\/ 15\/ (8), 534--539

work page 2004

[16] [16]

Hitchcock, C. (2001). The intransitivity of causation revealed in equations and graphs. Journal of Philosophy\/ XCVIII\/ (6), 273--299

work page 2001

[17] [17]

Hitchcock, C. (2007). Prevention, preemption, and the principle of sufficient reason. Philosophical Review\/ 116 , 495--532

work page 2007

[18] [18]

Jenni, K. and G. Loewenstein (1997). Explaining the identifiable victim effect. Journal of Risk and Uncertainty\/ 14\/ (3), 235--257

work page 1997

[19] [19]

Kahneman, D. and A. Tversky (1979). Prospect theory, an analysis of decision under risk. Econometrica\/ 47\/ (2), 263--292

work page 1979

[20] [20]

Niehans, J. (1948). Zur preisbildung bei ungewissen erwartungen. Schweizerische Zeitschrift f \" u r Volkswirtschaft und Statistik\/ 84\/ (5), 433--456

work page 1948

[21] [21]

Norcross, A. (1998). Great harms from small benefits grow: how death can be outweighed by headaches. Analysis\/ 58\/ (2), 152--158

work page 1998

[22] [22]

Prelec, D. (1998). The probability weighting function. Econometrica\/ 66\/ (3), 497--527

work page 1998

[23] [23]

Quiggin, J. (1993). Generalized Expected Utility Theory: The Rank-Dependent Expected Utility Model . Boston: Kluwer

work page 1993

[24] [24]

Richens, J. G., R. Beard, and D. H. Thompson (2022). Counterfactual harm. a rXiv preprint https://arxiv.org/abs/2204.12993, version v2

work page arXiv 2022

[25] [25]

Savage, L. J. (1951). The theory of statistical decision. Journal of the American Statistical Association\/ 46 , 55--67

work page 1951

[26] [26]

Wald, A. (1950). Statistical Decision Functions . New York: Wiley

work page 1950

[27] [27]

Weslake, B. (2015). A partial theory of actual causation. British Journal for the Philosophy of Science\/ . To appear

work page 2015

[28] [28]

Woodward, J. (2003). Making Things Happen: A Theory of Causal Explanation . Oxford, U.K.: Oxford University Press

work page 2003

[29] [29]

Zielonka, P. and T. Tyszka (Eds.) (2017). Large Risks with Low Probabilities . IWA Publishing

work page 2017