{"id":"9b95d7ff-848d-4460-818f-7ae6df8b68bf","arxiv_id":"2505.19317","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new fairness metric measures effort as inertia (historical disadvantage) times acceleration (improvement over time), and a pre-registered human experiment shows people value such trajectory over aggregate values.","lead":"This paper introduces a new fairness metric, Effort-aware Fairness, that judges whether people who put in similar effort over time get similar treatment from AI, using changes in income or arrest history and historical disadvantage. It reports a pre-registered experiment showing that laypeople weigh the trajectory of a person's features more than the final total when judging fairness, which could help auditors detect decisions that ignore genuine improvement.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The EaGF conditioning variable embeds a race-level inertia multiplier, so the demonstration could be measuring a mechanical race effect rather than effort; a counterfactual recomputation is needed to check.","rationale":"The paper's central contribution is the force-based Effort metric E = m * sigma(A_avg) and its use in EaIF and EaGF. The pre-registered human experiment provides credible evidence that people weight temporal trajectory more than aggregate feature value in individual fairness judgments. That result independently supports the acceleration component of the formulation. However, the full Effort metric and the EaGF audit also depend on the inertia multiplier m, and the operational choice of m is race-level childhood poverty. Because EaGF conditions directly on E, and E includes a race-constant m, the conditioning can partially condition on race. This is not merely a philosophical worry: within an Effort bin, different racial groups are systematically associated with different acceleration levels, so the within-bin parity comparison is not comparing equally effortful histories in a race-neutral sense. The paper acknowledges that race is an imperfect proxy and notes fairwashing risks, but it does not quantify how much of the reported EaGF patterns is attributable to the race-m link. A counterfactual recomputation with m set to a constant or permuted across individuals would settle whether the audit conclusions are robust or are an artifact of the proxy's group structure. This concern reinforces the reader's CONDITIONAL verdict: the metric proposal and perception experiment are solid, but the EaGF demonstration needs this robustness check before the audit claims can be accepted as measuring effort rather than race.","tokens_in":23286,"tokens_out":3839,"duration_ms":44200,"concrete_test":"On the SHED demonstration (N=704, Equations 9 and 3), recompute Figure 3a and 3b under two counterfactual Effort definitions: (i) m = 1 for all households, so E = sigma(A_avg); and (ii) m permuted randomly across households within each race, breaking only the race-m link while preserving the marginal distribution of m. If the within-bin parity curves or the model fairness ranking (XGBoost vs. Logistic Regression vs. Random Forest) shift by more than a pre-specified margin, for example 0.05 in parity or any change in model ordering, the race-derived inertia is driving EaGF conclusions; if the curves are essentially unchanged, the concern is mitigated. Also report, for each Effort bin, the within-bin race composition and the mean A_avg per race to verify whether bins compare like-for-like acceleration histories across groups.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Equation (9), Effort is computed as E = m * sigma(A_avg), where m is assigned by race from NCES childhood poverty rates scaled to [0,1]. Because m is constant within each racial group, E is a deterministic function of both the individual's acceleration and their race's poverty rate. EaGF, defined in Equation (3) as Y_hat independent of G given E, then conditions on a variable that contains a race-level component. This can mechanically alter parity: within any Effort bin [e, e+epsilon], individuals from a group with higher m need a lower sigma(A_avg) to fall in the bin than individuals from a group with lower m. The two groups are therefore not being compared at the same acceleration history; they are being compared at different effort-relevant histories that happen to produce the same race-adjusted product. If m were instead individual-level or race-invariant, the within-bin parity curves in Figure 3 and Appendix G could shift substantially or reverse. The paper explicitly calls race an 'imperfect proxy' for childhood poverty but does not analyze this statistical consequence, and the human subjects experiment validates only the acceleration/trajectory component, not the inertia multiplier. Thus the load-bearing link between the force-based notion of Effort and the EaGF audit results rests on a group-structured proxy whose effect is unquantified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Effort-aware Fairness (EaF), a framework that incorporates a force-based notion of effort into algorithmic fairness metrics. Effort is defined as E = m · σ(A_avg), where m is an inertia term proxied by race-level childhood poverty rates and A_avg is the average acceleration of a cumulative task-relevant feature. The paper also proposes Effort-aware Individual Fairness (EaIF), which modifies the input-space distance to include effort differences, and Effort-aware Group Fairness (EaGF), which conditions group parity on similar-effort bins. The empirical contributions include a pre-registered human subjects experiment (N=149) showing that trajectory distance correlates more strongly than aggregate distance with overall input-space and output-space distances in both criminal justice (CLUE) and personal finance (SHED) contexts, and demonstration pipelines applying EaIF and EaGF to CLUE and SHED datasets with several predictive models.","tokens_in":23594,"tokens_out":6697,"duration_ms":62509,"significance":"If the claims hold, the paper introduces a novel and philosophically grounded way to incorporate effort into algorithmic fairness audits, with experimental evidence that laypeople prioritize trajectories over aggregate values when judging fairness. The pre-registration, the release of code and a record-linked SHED dataset, and the real-world demonstrations are notable strengths. However, the operationalization of inertia via a race-level poverty proxy is a critical weakness: because m is constant within each racial group, the effort variable implicitly encodes protected attributes, which can mechanically affect the EaGF audit results. The human experiment validates only the trajectory component, not the inertia multiplier, so the link between the human-centered claim and the complete effort metric is incomplete. A counterfactual analysis with a race-invariant inertia term is needed to establish the robustness of the audit conclusions.","major_comments":[{"comment":"The inertia term m is assigned at the racial-group level from NCES childhood poverty rates, making m constant within each group. Consequently, E = m · σ(A_avg) is a deterministic function of both the individual's acceleration and their race's poverty rate. EaGF, defined in Eq. (3) as conditional independence of the predicted outcome G and the protected attribute given E, therefore conditions on a variable that encodes race. Within any effort bin, individuals from a group with higher m require a lower σ(A_avg) to fall in the bin than individuals from a group with lower m, so the two groups are compared at different acceleration histories. The paper calls race an 'imperfect proxy' but does not analyze this statistical consequence. A counterfactual recomputation of EaGF with m set to a constant (e.g., m=1) or to an individual-level proxy, if available, is necessary to show that the within-bin parity curves in Figures 3 and 6 are not artifacts of the group-level proxy rather than reflections of genuine effort-based disparities.","section":"Computing Effort, Eqs. (9)–(10)"},{"comment":"The pre-registered experiment demonstrates that participants weigh trajectory (acceleration) more heavily than aggregate feature values in their individual fairness judgments. However, the full Effort metric E = m · σ(A_avg) includes the inertia multiplier m, which is never tested in the experiment. The paper's conclusion correctly states that the acceleration/trajectory component aligns with laypeople's perception, but the abstract and introduction imply that the entire Effort-aware formulation is human-centered. Because the inertia term is a race-level proxy for childhood poverty, the experiment provides no evidence that laypeople endorse multiplying the trajectory component by such a proxy. This gap should be explicitly acknowledged as a limitation, and ideally tested in a follow-up study that varies m independently of acceleration.","section":"Human Subjects Experiment"},{"comment":"The EaIF pipeline on SHED uses the regression coefficients from the SHED condition of the human study (α1 = 0.6114, α2 = 0.3182, normalized to 0.6577 and 0.3423) as the weight α in the distance function, and then applies these weights to the same SHED dataset. Because the human participants viewed SHED-based stimuli, the weights are not independent of the data used to evaluate the EaIF metric. This is not a fatal circularity, but the paper should clarify whether the weights are intended to be context-specific or generalizable, and it would strengthen the analysis to validate the findings using the CLUE condition's coefficients or a held-out sample.","section":"Computing Effort-aware Individual Fairness"},{"comment":"The regression analyses in Table 5 treat each pairwise rating as an independent observation (900 CLUE and 888 SHED observations from 149 participants). Because each participant provided multiple ratings, the observations are non-independent, and the reported standard errors and p-values may be underestimated. To confirm that the significant differences between trajectory and aggregate effects are robust, the authors should use a mixed-effects model with participant random intercepts or cluster standard errors by participant.","section":"Analysis Methods and Tables 5"}],"minor_comments":[{"comment":"The text states that the sigmoid function 'eliminates the possibility of a negative sign.' This would be clearer if phrased as ensuring that the product m·σ(A_avg) is always positive, whereas m·A_avg could be negative when A_avg is negative.","section":"Computing Effort"},{"comment":"Equation (14) operationalizes EaGF using a min/max ratio of mean predicted risks, which is a specific choice; the paper should note that this is one of several possible parity metrics for conditional independence in Eq. (3).","section":"Computing Effort-aware Group Fairness"},{"comment":"The SHED outcome is described first as 'the respondent's frequency of unpaid credit card balance' and later as 'whether a household missed a payment on a credit card balance at least once.' The conversion from the ordinal frequency variable to a binary outcome should be stated explicitly in the main text.","section":"Appendix D"},{"comment":"The reference to 'V on K¨ugelgen' contains a LaTeX spacing artifact; it should be formatted as 'von Kügelgen' or 'Von Kügelgen.'","section":"References"},{"comment":"The row 'raggr,traj' (aggregate vs. trajectory correlation) is not discussed in the text; consider adding a brief interpretation or removing it to avoid confusion.","section":"Table 2"},{"comment":"The caption notes that the x-axes differ across subplots; it would be helpful to state this in the main text as well, as it affects the interpretation of the parity curves.","section":"Figure 3 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper uses race as a proxy for childhood poverty, which is ethically sensitive and could be seen as essentializing race. The authors should be encouraged to consult with social scientists on the appropriateness of this proxy and to include a clear discussion of its limitations. The main result's reliance on this proxy is the key weakness; if the counterfactual analysis shows stability, the paper would make a solid contribution to the fairness literature. The paper is well-written and the pre-registered experiment is a strength, but the operationalization of inertia needs substantial additional analysis before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this paper has one genuinely solid result and one load-bearing assumption that needs work. The solid result is the pre-registered experiment showing laypeople weigh trajectory (acceleration) more than aggregate value when judging individual fairness. The assumption is that race-level childhood poverty rates can stand in for individual 'inertia' in the effort formula. That assumption drives both audit pipelines and it doesn't hold up as-is.\n\nThe experiment is the best part. 149 Prolific participants, pre-registered, BTS incentives, reasonable analyses (Spearman, Hotelling and Steiger comparisons, bootstrapped CIs, regression with Wald tests). The effect is consistent across CLUE and SHED: trajectory distance correlates more strongly with both input-space and output-space distances than aggregate distance. The qualitative coding reinforces it. I would trust this finding. It also cleanly distinguishes the paper from the recourse-based effort literature: that work is counterfactual and cross-sectional; this is realized temporal trajectory. The philosophical grounding is not window-dressing—it shapes the metric.\n\nNow the soft spot. Effort is E = m * sigma(A_avg), where m comes from NCES childhood poverty rates by race, scaled to [0,1]. The paper calls race an 'individual-level but imperfect proxy'; in reality, m is a group-level ecological variable assigned uniformly to every person of a race. Individual variation is erased. Because m is constant within race, E is a deterministic function of race and acceleration. In EaGF, the paper bins on E and compares demographic groups within bins. For race parity, this is close to circular: a bin of E mixes people from high-poverty-race groups with low acceleration and people from low-poverty-race groups with high acceleration. The within-bin comparison is not comparing like with like on trajectory; it's comparing different acceleration histories that happen to produce the same race-adjusted product. The parity curves in Figures 3 and 6 could shift or reverse if m were individual-level or race-invariant. The paper never analyzes this statistical consequence, and the human experiment validates only the acceleration term, not the inertia multiplier. So the load-bearing link between the validated perception result and the audit numbers is unquantified.\n\nEaIF is less damaged because same-race pairs cancel m in the distance; cross-race pairs still carry the proxy.\n\nThe paper honestly acknowledges other limitations—normative directionality and fairwashing—which is more than most fairness papers do. Missing the group-structured proxy issue is a real gap.\n\nVerdict: the conceptual contribution and the perception experiment deserve serious peer review. The EaGF/EaIF demonstrations should not be cited as validated audit metrics until the authors either get individual-level inertia data or show robustness under alternative m specifications (race-invariant, individual proxies, sensitivity sweeps). I'd bring it to a reading group and would cite the experiment. Send it to referees, but expect a major revision focused on the inertia proxy.","headline":"Trajectory perception result is solid, but the race-level inertia proxy makes the EaGF demonstration untrustworthy without a counterfactual check.","tokens_in":24163,"tokens_out":4110,"would_cite":true,"duration_ms":38479,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Algorithmic fairness should count the effort a person has already exerted, modeled as inertia times life-course acceleration, and a pre-registered experiment shows laypeople weigh trajectory over aggregate values.","keywords":["effort-aware fairness","algorithmic fairness","individual fairness","group fairness","temporal trajectory","inertia proxy","human fairness perception","recidivism risk"],"falsifier":"Recompute the EaGF parity curves with a race-independent inertia proxy, such as individual-level childhood poverty or disability status, in place of race-scaled poverty rates: if the within-effort-bin parity curves and the model rankings they imply change materially or vanish, the reported audit results are artifacts of conditioning on a race-derived variable rather than evidence about effort. Separately, an experiment that matches pairs on aggregate value while varying only the steepness of the positive trajectory would test whether the continuous acceleration term, not just the sign of the trajectory, is what drives lay fairness judgments.","tokens_in":23069,"feed_emoji":"⚖️","tokens_out":11441,"duration_ms":89744,"temperature":0.7,"pith_summary":"Algorithmic fairness metrics typically compare people by their current features or outcomes and ignore how they got there. This paper tries to change that by proposing an \"effort-aware\" fairness that credits the temporal trajectory of a person's features: how much their income or arrest history has moved, weighted by how much societal disadvantage they started with. The paper formalizes effort as force ($E = m \\cdot \\sigma(A_{\\text{avg}})$), with inertia $m$ standing for childhood poverty and $A_{\\text{avg}}$ the average acceleration of the cumulative feature, and builds both an individual and a group fairness constraint around it. A pre-registered human-subjects experiment supports the core premise: in both criminal-justice and personal-finance scenarios, people's similarity judgments and assigned risk scores track trajectory differences more strongly than aggregate differences. If the approach holds, AI auditors could flag decisions that penalize people who improved against the grain of systemic disadvantage.","feed_headline":"Trajectory, not totals, drives fairness judgments","feed_subtitle":"A 149-person experiment supports a force-based fairness metric that credits effort in AI bail and credit audits.","key_machinery":"The load-bearing object is the Effort-as-Force score $E = m \\cdot \\sigma(A_{\\text{avg}})$, built from two analogies: inertia $m$, drawn from race-level childhood poverty rates scaled to $[0,1]$ to represent societal holding-back beyond individual control, and acceleration $A_{\\text{avg}}$, the average second-order difference of the cumulative version of the task feature (cumulative income or cumulative arrests), with the sigmoid $\\sigma$ keeping effort one-signed and pooling extreme accelerations. This score is inserted into a Dwork-style Lipschitz individual-fairness constraint, $D(M\\mathbf{x}_i, M\\mathbf{x}_j) \\le d(\\mathbf{x}_i, \\mathbf{x}_j)$, through a weighted distance $d$ that mixes effort difference with aggregate feature difference, and into a conditional group-fairness constraint $\\hat{Y} \\perp G \\mid E$ computed by binning individuals by effort. The paper's empirical check that people actually judge by trajectory uses a simplified two-step evaluation of individual fairness: pairs are rated for trajectory and aggregate distance, then for overall similarity and for \"fair\" risk scores, with Spearman correlations, Hotelling and Steiger tests, and regressions that compare which distance drives the judgments.","core_discovery":"The paper's central claim is that effort should enter fairness comparisons, and that effort is best modeled as force: $E = m \\cdot \\sigma(A_{\\text{avg}})$, where $m$ is the societal \"holding-back\" an individual faced (operationalized by race-scaled childhood poverty rates) and $A_{\\text{avg}}$ is the average acceleration of the cumulative task-relevant feature (prior arrests or income), with the sigmoid flipped for undesirable features such as arrests. On this basis the paper proposes Effort-aware Individual Fairness, requiring that pairwise output distance obey $D(M\\mathbf{x}_i, M\\mathbf{x}_j) \\le d(\\mathbf{x}_i, \\mathbf{x}_j)$ with $d$ a weighted Euclidean combination of effort difference and aggregate feature difference, and Effort-aware Group Fairness, requiring $\\hat{Y} \\perp G \\mid E$, group parity within bins of similar effort. The paper also reports a pre-registered experiment with 149 lay participants in which trajectory distance correlated significantly more strongly than aggregate distance with both overall input-space distance (CLUE $r = 0.706$ vs $0.502$; SHED $r = 0.788$ vs $0.655$) and output-space distance (CLUE $0.422$ vs $0.209$; SHED $0.499$ vs $0.251$), which it reads as validating the trajectory and acceleration component of the metric.","pith_inferences":["Because $E = m \\cdot \\sigma(A_{\\text{avg}})$ and $m$ is scaled from race-level poverty rates, conditioning fairness on effort is partly conditioning on the protected attribute: effort bins are partially separated by race, so within-bin parity could rise mechanically even if individuals of different races with identical trajectories are treated differently. The paper does not analyze this statistic","A decisive test of the framework is to swap in individual-level inertia measures such as actual childhood poverty or disability status; if audit conclusions change materially, the race-level proxy is carrying the results, and if they do not, the proxy is a fair stand-in.","The experiment contrasts increasing versus decreasing trajectories, but the metric rewards continuous acceleration; whether laypeople credit a steep climb more than a gradual one at equal aggregate value is an open question the current design does not resolve.","The moderation finding, that trajectory's weight relative to aggregate was smaller in the criminal-history condition, implies the weight $\\alpha$ in the EaIF distance is domain-dependent, so applying a single fixed weight across contexts needs justification."],"forward_implications":["An AI model can now be audited for effort-sensitivity: EaIF averages pairwise violations of $D(M\\mathbf{x}_i, M\\mathbf{x}_j) \\le d(\\mathbf{x}_i, \\mathbf{x}_j)$, and EaGF plots within-effort-bin group parity, surfacing unfairness that overall demographic parity hides.","The experiment implies that aggregate-only individual-fairness distances diverge from lay fairness judgment: in both the criminal-justice and personal-finance conditions, trajectory differences outweighed aggregate differences at both the similarity and risk-scoring stages.","The metric ranks models differently from traditional fairness: on SHED, random forest leads EaIF while XGBoost wins highest-effort sex parity; on CLUE, Light GBM leads EaIF while decision tree leads traditional parity, so an effort-aware audit can change model selection.","The EaGF definition $\\hat{Y} \\perp G \\mid E$ embeds the philosophical claim that people who exerted similar effort deserve similar predicted outcomes regardless of demographics, making the fairness debate one about effort rather than outcome alone."],"supporting_citations":[{"why":"Supplies the philosophical force-based account of effort (resistance, fatigue, praiseworthiness) that the paper operationalizes into a computational metric.","marker":"Massin 2017"},{"why":"Defines the individual-fairness Lipschitz constraint $D(Mx_i,Mx_j)\\le d(x_i,x_j)$ that Effort-aware Individual Fairness extends by inserting effort into the input distance.","marker":"Dwork et al. 2012"},{"why":"Provides the empirical link between childhood family income and violent criminality that justifies childhood poverty as the inertia term in the criminal-justice context.","marker":"Sariaslan et al. 2014"},{"why":"Documents large detrimental effects of early-childhood poverty on adult earnings, justifying the same inertia proxy in the personal-finance context.","marker":"Duncan, Ziol-Guest, and Kalil 2010"},{"why":"Supplies the Bayesian Truth Serum incentive used to elicit truthful subjective judgments in the human-subjects experiment.","marker":"Prelec 2004"},{"why":"The counterfactual equal-improvability line of effort fairness that the paper contrasts with its actual, historical-effort formulation.","marker":"Guldogan et al. 2023"},{"why":"Causal-recourse fairness, cited to show why recourse-based effort is incompatible with the Dwork individual-fairness regime.","marker":"Von Kügelgen et al. 2022"},{"why":"The criminal-career paradigm whose acceleration concept grounds the temporal-trajectory term of effort.","marker":"Piquero, Farrington, and Blumstein 2003"}],"fun_headline_variants":["Force-based fairness: effort matters more than totals","Fairness via force: trajectory outweighs aggregate scores","Effort-aware fairness: physics-inspired metric for audits","Trajectory beats totals: effort-aware fairness validated","Fairness that credits effort: force-based metric tested"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that race-level childhood poverty rates, scaled to $[0,1]$, validly measure each individual's inertia, the societal holding-back beyond their control; if that proxy does not track individual disadvantage, every effort score and both audit metrics built on it lose their justification, and since inertia varies by race, conditioning fairness on effort implicitly conditions on the protected attribute.","fun_headline_variants_meta":{"raw":{"variants":["Force-based fairness: effort matters more than totals","Fairness via force: trajectory outweighs aggregate scores","Effort-aware fairness: physics-inspired metric for audits","Trajectory beats totals: effort-aware fairness validated","Fairness that credits effort: force-based metric tested"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000186,"raw_usage":{"total_tokens":1368,"prompt_tokens":1033,"completion_tokens":335,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":260}},"tokens_in":649,"tokens_out":335,"duration_ms":10009,"temperature":1.0,"reasoning_tokens":260,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:17:01.504144+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the EaGF parity curves with a race-independent inertia proxy, such as individual-level childhood poverty or disability status, in place of race-scaled poverty rates: if the within-effort-bin parity curves and the model rankings they imply change materially or vanish, the reported audit results are artifacts of conditioning on a race-derived variable rather than evidence about effort. Separately, an experiment that matches pairs on aggregate value while varying only the steepness of the positive trajectory would test whether the continuous acceleration term, not just the sign of the trajectory, is what drives lay fairness judgments.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the philosophical force-based account of effort (resistance, fatigue, praiseworthiness) that the paper operationalizes into a computational metric."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the empirical link between childhood family income and violent criminality that justifies childhood poverty as the inertia term in the criminal-justice context."},{"cited_title":"J.; Ziol-Guest, K","cited_arxiv_id":null,"evidence_quote":"Documents large detrimental effects of early-childhood poverty on adult earnings, justifying the same inertia proxy in the personal-finance context."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Bayesian Truth Serum incentive used to elicit truthful subjective judgments in the human-subjects experiment."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The counterfactual equal-improvability line of effort fairness that the paper contrasts with its actual, historical-effort formulation."},{"cited_title":"u gelgen, J.; Karimi, A.-H.; Bhatt, U.; Valera, I.; Weller, A.; and Sch \\","cited_arxiv_id":null,"evidence_quote":"Causal-recourse fairness, cited to show why recourse-based effort is incompatible with the Dwork individual-fairness regime."},{"cited_title":"R.; Farrington, D","cited_arxiv_id":null,"evidence_quote":"The criminal-career paradigm whose acceleration concept grounds the temporal-trajectory term of effort."}],"review_version":1}