RECORD
Stacking interventions for equitable outcomes
Methodology · Submitted 8 Oct 2021
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim By repeatedly observing and updating the intervention and model, we show that this scheme leads to convergence or almost-convergence of eventual outcome risk to an equivocal value for any initial value of covariates.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Predictive risk scores estimating probabilities for a binary outcome on the basis of observed covariates are common across the sciences. They are frequently developed with the intent of avoiding the outcome in question by intervening in response to estimated risks. Since risk scores are usually developed in complex systems, interventions usually take the form of expert agents responding to estimated risks as they best see fit. In this case, interventions may be complex and their effects difficult to observe or infer, meaning that explicit specification of interventions in response to risk scores is impractical. Scope to modulate the aggregate model-intervention scheme so as to optimise an objective is hence limited. We propose an algorithm by which a model-intervention scheme can be developed by 'stacking' possibly unknown intervention effects. By repeatedly observing and updating the intervention and model, we show that this scheme leads to convergence or almost-convergence of eventual outcome risk to an equivocal value for any initial value of covariates. Our approach deploys a series of risk scores to expert agents, with instructions to act on them in succession according to their best judgement. Our algorithm uses only observations of pre-intervention covariates and the eventual outcome as input. It is not necessary to know or infer the effect of the intervention, other than making a general assumption that it is 'well-intentioned'. The algorithm can also be used to safely update risk scores in the presence of unknown interventions and concept drift. We demonstrate convergence of expectation of outcome in a range of settings and show robustness to errors in risk estimation and to concept drift. We suggest several practical applications and demonstrate a potential implementation by simulation, showing that the algorithm leads to a fair distribution of outcome risk across a population.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Predictive risk scores estimating probabilities for a binary outcome on the basis of observed covariates are common across the sciences. | background 1.00 | 0.04 | 0.01 | asserted 1.00 |
| s1 | They are frequently developed with the intent of avoiding the outcome in question by intervening in response to estimated risks. | background 1.00 | 0.04 | 0.02 | asserted 0.89 |
| s2 | Since risk scores are usually developed in complex systems, interventions usually take the form of expert agents responding to estimated risks as they best see fit. | background 1.00 | 0.05 | 0.02 | asserted 0.67 |
| s3 | In this case, interventions may be complex and their effects difficult to observe or infer, meaning that explicit specification of interventions in response to risk scores is impractical. | background 0.75 | 0.07 | 0.02 | hedged possibility 0.87 |
| s4 | Scope to modulate the aggregate model-intervention scheme so as to optimise an objective is hence limited. | gap 0.50 | 0.08 | 0.03 | asserted 0.96 |
| s5 | We propose an algorithm by which a model-intervention scheme can be developed by 'stacking' possibly unknown intervention effects. | contribution 0.95 | 0.83 | 0.28 | asserted 0.93 |
| s6 | By repeatedly observing and updating the intervention and model, we show that this scheme leads to convergence or almost-convergence of eventual outcome risk to an equivocal value for any initial value of covariates. | result 0.96 | 0.93 | 0.59 | asserted 0.99 |
| s7 | Our approach deploys a series of risk scores to expert agents, with instructions to act on them in succession according to their best judgement. | method 0.97 | 0.83 | 0.06 | asserted 1.00 |
| s8 | Our algorithm uses only observations of pre-intervention covariates and the eventual outcome as input. | method 0.99 | 0.81 | 0.04 | asserted 1.00 |
| s9 | It is not necessary to know or infer the effect of the intervention, other than making a general assumption that it is 'well-intentioned'. | method 0.85 | 0.54 | 0.08 | asserted 0.97 |
| s10 | The algorithm can also be used to safely update risk scores in the presence of unknown interventions and concept drift. | contribution 0.85 | 0.55 | 0.08 | asserted 0.73 |
| s11 | We demonstrate convergence of expectation of outcome in a range of settings and show robustness to errors in risk estimation and to concept drift. | result 0.99 | 0.96 | 0.21 | asserted 1.00 |
| s12 | We suggest several practical applications and demonstrate a potential implementation by simulation, showing that the algorithm leads to a fair distribution of outcome risk across a population. | result 0.82 | 0.93 | 0.38 | hedged possibility 0.58 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Predictive risk scores estimating probabilities for a binary outcome on the basis of observed covariates are common across the sciences. They are frequently developed with the intent of avoiding the outcome in question by intervening in response to estimated risks. Since risk scores are usually developed in complex systems, interventions usually take the form of expert agents responding to estimated risks as they best see fit. In this case, interventions may be complex and their effects difficult to observe or infer, meaning that explicit specification of interventions in response to risk scores is impractical. Scope to modulate the aggregate model-intervention scheme so as to optimise an objective is hence limited. We propose an algorithm by which a model-intervention scheme can be developed by 'stacking' possibly unknown intervention effects. By repeatedly observing and updating the intervention and model, we show that this scheme leads to convergence or almost-convergence of eventual outcome risk to an equivocal value for any initial value of covariates. Our approach deploys a series of risk scores to expert agents, with instructions to act on them in succession according to their best judgement. Our algorithm uses only observations of pre-intervention covariates and the eventual outcome as input. It is not necessary to know or infer the effect of the intervention, other than making a general assumption that it is 'well-intentioned'. The algorithm can also be used to safely update risk scores in the presence of unknown interventions and concept drift. We demonstrate convergence of expectation of outcome in a range of settings and show robustness to errors in risk estimation and to concept drift. We suggest several practical applications and demonstrate a potential implementation by simulation, showing that the algorithm leads to a fair distribution of outcome risk across a population.
Human review
How the literature uses it
1 arXiv paper cite this in 2 sentences
What it cites · 9 sentences without a confident label
Cites as background 5 papers
-
Repeatedly updating a risk model is a common practice, and generally required due to `drift', a gradual change in the joint distribution of $(X,Y)$ [ref] .
DriftSurf: A Risk-competitive Learning Algorithm under Concept Drift · Related work · 0.88
-
Other algorithms for model updating in which the aim is convergence to performative stability are given in [ref] , [ref] , [ref] and [ref] .
Stochastic Optimization for Performative Prediction · Related work · 0.90
-
Other algorithms for model updating in which the aim is convergence to performative stability are given in [ref] , [ref] , [ref] and [ref] .
Stochastic optimization with decision-dependent distributions · Related work · 0.90
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.