Pith. sign in

REVIEW 1 major objections 5 minor 55 references

Designing Algorithmic Delegates: The Role of Indistinguishability in Human-AI Handoff

T0 review · 1 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The optimal algorithmic delegate is not the most accurate standalone model; it is the one tuned to the categories of tasks that humans actually hand off, and finding it is NP-hard in general.

desk verdict A genuinely new and mostly clean model of optimal delegation under human categorization, but the proof of the main polynomial-time result has a load-bearing algebraic error in Lemma 4 that needs fixing before the theorem can be trusted. read the letter →

arxiv 2506.03102 v1 pith:I4GJJR7T submitted 2025-06-03 cs.GT cs.AIcs.CY

Sophie Greenwood , Karen Levy , Solon Barocas , Hoda Heidari , Jon Kleinberg This is my paper
classification cs.GTcs.AIcs.CY MSC 68Q1790C27
keywords human-AIdelegationalgorithmicdelegatescategorizationteamperformancevarianceassignmentNP-hardnesshandoffdesign
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that an AI agent that takes over a task from a human should not be designed to be the most accurate on its own; it should be designed to be the best teammate, anticipating that the human will delegate only in situations she can recognize. The authors model a human who sees coarse categories of task instances and hands off exactly when the machine's expected squared loss is lower than her own, and they prove that optimizing the machine for this handoff can yield arbitrarily large gains over optimizing it in isolation. The central discovery is that the optimal delegate is characterized by a subset of human categories to “retain”: the machine is tuned to do well only there, and the human adopts it exactly there. Choosing this subset is NP-hard in general but has polynomial algorithms when the optimal action and distribution separate across human and machine features, or when one side sees few features. A designer who iteratively re-trains on where the machine is adopted generally reaches only a local optimum, though experiments suggest it often performs near optimally.

What carries the argument

The central object is the retained set $R$ of human categories together with the induced machine policy $f^R_M(K) = \mathbb{E}[f^* \mid X(R) \cap K]$, the conditional mean of the ground-truth action over the states that lie both in retained human categories and in machine category $K$. Proposition 1 shows that optimal delegation reduces to choosing $R$; Proposition 2 rewrites the team loss as the sum of the conditional variance of $f^*$ over each yielded human category plus the conditional variance over the retained states in each machine category. This variance decomposition is what makes the problem combinatorial: the designer is really selecting which categories' variance the machine will absorb. It is also the object through which the tractability results are obtained and through which the NP-hardness reduction, via a variance-assignment variant of the densest-subgraph problem, is proved.

What would settle it

Use the two-feature family of Section 3.1 with ground-truth actions $(0,1,0,N)$ on the four states and equal probabilities. Compare the team loss of the optimal delegate that retains only the second human category with the team loss of the oblivious machine that averages within each machine category: the paper's Fact 1 predicts a gap growing like $N^2$, with the optimal delegate's loss constant in $N$. If the gap fails to grow or a standalone-optimized machine matches the tailored machine's team performance, the central separation claim would be refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that the right design target for an algorithmic delegate is not standalone performance but team performance under delegation, and that this target has a clean but combinatorially hard form. In the model, a human category is a set of states with identical human-observable features; a machine category is a set of states with identical machine-observable features. Given the human's optimal action $f^*_H$, which is the mean of the ground-truth action $f^*$ within each human category, the human delegates in category $C$ exactly when the machine's squared loss in $C$ is smaller. The optimal delegate is any machine function minimizing the resulting team loss, and the paper proves in Propositions 1 and 2 that it can be found by choosing a subset $R$ of human categories to retain and setting the machine's action in each machine category to the mean of $f^*$ over the retained states in that category. The team loss decomposes into within-category variances: the human absorbs variance in the unretained categories, and the machine absorbs variance over the retained states in each machine category. The authors also show that the oblivious machine, which minimizes standalone loss, can be arbitrarily worse than this optimal delegate, that a delegate designed to maximize adoption can be arbitrarily worse than the optimal delegate, and that computing the optimal retained set is NP-hard in general, with efficient algorithms for separable settings and for settings where one side has a constant number of features.

Load-bearing premise

The load-bearing premise is that the human decides to delegate by comparing exact expected squared losses and hands off exactly when the machine's loss in her category is lower; if humans misestimate machine quality, prefer to stay in control, or are biased against algorithms, the characterization of the optimal delegate does not carry over.

Editorial extensions

If this is right

  • Team performance, not standalone accuracy, is the right objective when an AI is used as a delegate: the paper's Fact 1 shows the oblivious machine can be arbitrarily worse than the optimal delegate.
  • Designing an optimal delegate reduces to selecting a subset of human categories to retain; the machine should average the optimal action only over states in those categories within each machine category (Propositions 1 and 2).
  • For separable delegation settings, where the optimal action is additive in human and machine features and the distribution is independent with polynomial-precision probabilities, an optimal delegate can be found in polynomial time (Theorem 4).
  • If the human or the machine observes only a constant number of features beyond the shared ones, an optimal delegate can again be found in polynomial time (Theorem 5).
  • In general the problem is NP-hard (Theorem 7), and a designer who re-trains the machine on the categories where it is adopted converges to a local optimum that experiments show is often near-optimal (Section 5).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The variance decomposition suggests a practical screening rule: leave to the human any category with small within-category variance, and keep for the machine categories that become low-variance once a few high-variance human categories are excluded.
  • If humans delegate noisily, say only when the machine's expected advantage exceeds a threshold, the retained-set characterization becomes approximate, so the exact hardness and tractability boundaries would need re-examination.
  • Since the hardness result holds even for a single isolated decision with no communication, longer-horizon agentic systems are unlikely to admit optimal delegation design without further structural assumptions.
  • A designer with approximately linear $f^*$ and near-independent features could use Theorem 4 as a screening tool to compute the optimal retained set exactly, and fall back on iterative retraining when nonlinearities dominate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper studies the design of an algorithmic delegate when a human decides whether to hand off a decision to the machine, where both agents observe only subsets of binary features and therefore operate under human and machine categories. The main results are a reduction of optimal delegate design to a discrete subset-selection problem (Propositions 1 and 2), a complete analysis of the two-feature case, a polynomial-time algorithm for separable delegation settings (Theorem 4), a polynomial-time algorithm when one side observes a constant number of features (Theorem 5), and an NP-hardness result for the general problem (Theorem 7). The paper also reports computational experiments on an iterative design process that converges to locally optimal delegates.

Significance. If the technical claims hold, this is a valuable contribution to the human-AI collaboration literature. The clean reformulation of optimal delegation as a variance-minimizing subset problem is elegant, and the contrast between the oblivious machine and the optimal delegate (Fact 1) gives a sharp justification for designing machines that account for when they will actually be adopted. The paper is self-contained and analytic, with no fitted parameters or empirical constants in the main results. The two-feature phase diagram is a nice concrete illustration of how even the simplest setting yields non-convex and disconnected optimal-design regions. The authors are also appropriately careful about the limits of the rational-delegation assumption, noting in Section 6.2 that behavioral biases are not covered. The main concern is a proof error in a lemma that supports Theorem 4; although the lemma appears to be repairable, the proof as written is not valid.

major comments (1)
  1. [Appendix E.1, Lemma 4] The proof of Lemma 4 contains an algebraic error in the application of O'Neill's variance decomposition result. With R+ = R* ∪ {s_r'} and R- = R* \ {s_r}, the correct identities are (D^2)+ = |R*|(|R*|+1)(μ(R+)-μ(R*))^2 and (D^2)- = |R*|(|R*|-1)(μ(R*)-μ(R-))^2. The paper instead defines (D^2)+ = |R*|/(|R*|+1)(μ(R+)-μ(R*))^2 and (D^2)- = (|R*|-1)/|R*|(μ(R*)-μ(R-))^2, omitting factors of (|R*|+1) and |R*|, respectively. With the corrected formulas the ratio (D^2)+/(D^2)- is (|R*|-1)/(|R*|+1), which is less than 1 and still contradicts the necessary inequality (D^2)- ≤ (D^2)+, so the lemma is likely salvageable. Nevertheless, the proof as written is invalid, and Theorem 4 depends on this lemma. The authors should rewrite the proof and re-verify the reduction from the weighted problem to the expanded uniform multiset.
minor comments (5)
  1. [Appendix E.1, Lemma 4] In the proof of Lemma 4, the definition R- = R* \ {s_r'} is a typo: since s_r' is assumed not to be in R*, removing it leaves R* unchanged. The intended set is R- = R* \ {s_r}.
  2. [Section 1.1] There is a typo in the overview: 'Thich reveals' should be 'This reveals'.
  3. [Appendix E.4] The problem name 'MaxRegularCliqe' is misspelled; it should be 'MaxRegularClique'. This appears both in the proof of Theorem 7 and in the problem definition.
  4. [Section 3.2] The claim that the maximally adopted machine is worse than the oblivious machine alone 'for most settings' is established only for the two-feature family in Appendix A.1; the scope of this claim should be stated more precisely in the main text.
  5. [Theorem 5 proof] The use of arrangements of ellipsoids and the bound O(h^m) on the number of regions would benefit from a brief explanation of why Chazelle et al. applies to this specific family of ellipsoids, since the reader must otherwise take the geometric claim largely on faith.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation is self-contained mathematical theory with no fitted parameters or load-bearing self-citations.

full rationale

The paper's central claims are derived from its own model definitions rather than from fitted data or from prior results that presuppose the target conclusions. The delegation process, team-loss objective, and the reduction to the retained-category subset problem (Propositions 1 and 2) are proved from the squared-loss structure and the human's loss comparison rule, not assumed by construction. Fact 1 is established by an explicit two-feature family with (a,b)=(0,N) and a direct computation of the oblivious and optimal losses, so the 'arbitrarily better teammate' claim does not reduce to its own definition. The tractability results are built on external mathematical facts (O'Neill's variance decomposition, the contiguity of minimum-variance subsets) and on explicit reductions in the hardness proof; none of these inputs includes the paper's optimal-delegate theorem. Citations to prior work, including some with overlapping authors, are used for motivation and context rather than as the load-bearing justification for the new results. The Section 6.2 limitations, which state that behavioral biases and other loss functions may break the results, are acknowledged restrictions rather than hidden assumptions that make the derivation circular. The only substantive concern raised by a skeptical reading is an algebraic issue in the proof of Lemma 4; that is a correctness or completeness risk for Theorem 4, not a circularity, because it does not show that the conclusion was assumed or fitted as an input. Accordingly, the analysis is self-contained against external benchmarks, and the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No free parameters are fitted to data: P and f* are problem inputs, and the two-feature parameters a,b are examples rather than fitted constants. The model rests on rational delegation, known expected losses, and squared loss. No new physical or conceptual entities are postulated.

assumptions (5)
  • domain assumption Team objective is expected squared loss (f_M(K(x)) - f*(x))^2
    Section 2 defines the loss; all optimality results use this loss function.
  • domain assumption Human optimally delegates iff the machine's expected loss is lower within the human category, breaking ties by not delegating
    Section 2 delegation process and the team loss equation; this is central to Proposition 1 and all subsequent results.
  • domain assumption Human knows the expected losses of herself and the machine in each category
    Section 2 states: 'we assume the human knows whether her expected loss is better or worse than the machine's loss in each category'.
  • domain assumption Designer has full knowledge of I_H, I_M, P, and f* for the main results
    Section 2 distinguishes the full-knowledge regime from the iterative regime of Section 5.
  • domain assumption Categories are induced by observable binary features
    Section 2 defines categories via feature restrictions; Appendix C extends the results to arbitrary partitions, so this is not load-bearing for all results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Designing Algorithmic Delegates: The Role of Indistinguishability in Human-AI Handoff." pith.science (2026). https://pith.science/paper/I4GJJR7T

@misc{pith2026250603102,
  author       = {Pith},
  title        = {Pith review of: Designing Algorithmic Delegates: The Role of Indistinguishability in Human-AI Handoff},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I4GJJR7T}},
  note         = {Machine review of arXiv:2506.03102}
}
read the original abstract

As AI technologies improve, people are increasingly willing to delegate tasks to AI agents. In many cases, the human decision-maker chooses whether to delegate to an AI agent based on properties of the specific instance of the decision-making problem they are facing. Since humans typically lack full awareness of all the factors relevant to this choice for a given decision-making instance, they perform a kind of categorization by treating indistinguishable instances -- those that have the same observable features -- as the same. In this paper, we define the problem of designing the optimal algorithmic delegate in the presence of categories. This is an important dimension in the design of algorithms to work with humans, since we show that the optimal delegate can be an arbitrarily better teammate than the optimal standalone algorithmic agent. The solution to this optimal delegation problem is not obvious: we discover that this problem is fundamentally combinatorial, and illustrate the complex relationship between the optimal design and the properties of the decision-making task even in simple settings. Indeed, we show that finding the optimal delegate is computationally hard in general. However, we are able to find efficient algorithms for producing the optimal delegate in several broad cases of the problem, including when the optimal action may be decomposed into functions of features observed by the human and the algorithm. Finally, we run computational experiments to simulate a designer updating an algorithmic delegate over time to be optimized for when it is actually adopted by users, and show that while this process does not recover the optimal delegate in general, the resulting delegate often performs quite well.

Figures

Figures reproduced from arXiv: 2506.03102 by the authors.

Figure 1
Figure 1. Visualization of human and machine categories. A decision-making instance corresponds to a cell [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Categories emerge from limited access to information. Here there are [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Flow of decision-making control in delegation. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Example delegation problem. In Figure 4d we show the team consisting of 𝑓 ∗ 𝐻 from Figure 4b and 𝑓 obliv 𝑀 from Figure 4c. In this case, the human will delegate in categories 𝐶1,𝐶2, and 𝐶4. If the machine can only observe the machine category 𝐾𝑗 , this delegate 𝑓 obliv…
Figure 5
Figure 5. Figure 5: General two-feature setting First, consider the case where 𝑎 = 0, 𝑏 = 1, as shown in Figure 6a. In this case, the optimal action is entirely determined by the machine’s feature 𝑥2: the machine has perfect information. The optimal machine therefore retains all human cat…
Figure 6
Figure 6. Figure 6: Optimal delegation for different settings of [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: For each pair (𝑎, 𝑏), we compute the optimal delegate 𝑓 ∗ 𝑀 for ground truth function 𝑓 ∗ 𝑎,𝑏 . In (a) we plot the expected loss of the human-machine team where the human delegates to 𝑓 ∗ 𝑀 , ℓ(𝑓 ∗ 𝐻 , 𝑓 ∗ 𝑀 ). In (b) we plot the categories R ∗ that are retained by 𝑓 ∗…
Figure 8
Figure 8. Figure 8: A family of delegation settings when the human and machine each observe a single distinct feature, in [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: The grey shaded region of Figure 9a is the set of delegation settings [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Illustration of iterative vs. optimal machine design. In the delegation setting shown, the human [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Relationships between the losses of different possible human and machine teams. [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Relative performance between the iterative solution and the optimal solution. We consider delegation [PITH_FULL_IMAGE:figures/full_fig_p025_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

55 extracted references · 46 canonical work pages

  1. [1]

    is not the oblivious machine, despite the oblivious machine being the machine achieving the best expected performance across both𝐶1 and𝐶2! In order to guarantee adoption in𝐶1, the firm sacrifices performance in𝐶2 rather than treating the two categories equally as in the oblivious delegate (even though the two categories have equal probability). In general...

    work page 2025

  2. [2]

    Thus ℓ({𝐶 1,𝐶 2})= 1 2ℓ𝑀(𝑓R 𝑀,𝐶 1)+ 1 2ℓ𝑀(𝑓R 𝑀,𝐶 2)= 1 8(𝑎2+(𝑏−1) 2)

    Similarly, ℓ𝑀(𝑓R 𝑀,𝐶 2)= 1 2(𝑓∗(x21)−𝑓 R 𝑀(𝐾1)) 2+ 1 2(𝑓∗(x22)−𝑓 R 𝑀(𝐾2)) 2 = 1 2(𝑎/2) 2+ 1 2((𝑏−1)/2) 2 = 1 8(𝑎2+(𝑏−1) 2). Thus ℓ({𝐶 1,𝐶 2})= 1 2ℓ𝑀(𝑓R 𝑀,𝐶 1)+ 1 2ℓ𝑀(𝑓R 𝑀,𝐶 2)= 1 8(𝑎2+(𝑏−1) 2). Next, ifR={𝐶 1}, 𝑓R 𝑀(𝐾1)=0, 𝑓 R 𝑀(𝐾1)=1, and ℓ𝑀(𝑓R 𝑀,𝐶 1)=0. So ℓ({𝐶 1})= 1 2ℓ𝑀(𝑓R 𝑀,𝐶 1)+ 1 2ℓ𝐻(𝑓∗ 𝐻,𝐶 2)= 1 2·(𝑎−𝑏) 2 4 =(𝑎−𝑏) 2/8. EC ’25, July 7–10, 2025, Sta...

    work page 2025

  3. [3]

    Let 𝑅𝑘∈arg min 𝑅⊆[ℎ]:|𝑅|=𝑘 𝜎2(𝑢𝑖|𝑖∈𝑅)

    Fix𝑘, and suppose that there is no contiguous minimum variance subset of size𝑘. Let 𝑅𝑘∈arg min 𝑅⊆[ℎ]:|𝑅|=𝑘 𝜎2(𝑢𝑖|𝑖∈𝑅). Let𝑖=min𝑅 𝑘,𝑖′ =max𝑅 𝑘. We may assume without loss of generality that there is no𝑢𝑗 =𝑢𝑖 for 𝑗>𝑖 and 𝑗∉𝑅 𝑘; otherwise replace𝑅𝑘 with𝑅𝑘∪{𝑗}\{𝑖} , which will also be a minimum variance subset. Similarly we may assume that there is no𝑢 𝑗 =𝑢𝑖′...

    work page 2014

  4. [4]

    Thus in all subsequent results, we will assume that𝐼𝐻∪𝐼𝑀 =[𝑑]

    First, we show that any efficient algorithm for finding an optimal delegate when all features are observed by either the human or the machine can be used to find an optimal delegate in general settings. Thus in all subsequent results, we will assume that𝐼𝐻∪𝐼𝑀 =[𝑑]. Lemma 1.Letx 𝐻∪𝑀 denotexrestricted to the features in 𝐼𝐻∪𝐼 𝑀 . Given a ground truth optimal...

    work page 2025

  5. [5]

    If for some 𝑖, there are two values 𝑠𝑟,𝑠𝑟′ such that 𝑠𝑟 =𝑠𝑟′ =𝑢𝑖 , then𝑠 𝑟∈𝑅∗ =⇒𝑠 𝑟′∈𝑅∗. Proof. Suppose not, that is, there are two values 𝑠𝑟,𝑠𝑟′ such that 𝑠𝑟 =𝑠 𝑟′ =𝑢 𝑖, but 𝑠𝑟 ∈𝑅 ∗, 𝑠𝑟′ ∉𝑅∗. We again use the shorthand𝜇(𝑅)=𝜇(𝑠 𝑟|𝑟∈𝑅),𝜎 2(𝑅)=𝜎 2(𝑠𝑟|𝑟∈𝑅). Let𝑅+ =𝑅∗∪{𝑠𝑟′},𝑅− =𝑅∗\{𝑠𝑟′}. Since𝑅∗ is an optimal solution, it must be the case that 1− |𝑅∗| 𝑇 𝜎2(𝑤)...

    work page 2025

  6. [6]

    We examine 𝑓∗(x)=v 𝑇 xfor1000samples of v∼Normal( 0,𝐼) , and compute the sample proportion of settings𝑓∗ in which the iterative solution is optimal (specifically, if the loss of the iterative solution is within a tolerance of10−8 of the optimal loss.) # of machine-observable features,|𝐼 𝑀| 1 2 3 4 5 6 # of human- observable features,|𝐼 𝐻| 1 0.39 0.57 0.69...

    work page 2025

  7. [7]

    The intrinsic value of choice: The propensity to under-delegate in the face of potential gains and losses.Journal of Risk and Uncertainty54 (2017), 187 –

    work page 2017

  8. [11]

    InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency(Chicago, IL, USA)(FAccT ’23)

    Harms from Increasingly Agentic Algorithmic Systems. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency(Chicago, IL, USA)(FAccT ’23). Association for Computing Machinery, New Designing Algorithmic Delegates EC ’25, July 7–10, 2025, Stanford, CA, USA York, NY, USA, 651–666. https://doi.org/10.1145/3593013.3594033 Bernard...

    arXiv 2023

Show all 55 references
  1. [12]

    https: //doi.org/10.1016/0304-3975(91)90261-Y Giovanni De Toni, Nastaran Okati, Suhas Thejaswi, Eleni Straitouri, and Manuel Gomez-Rodriguez

    A singly exponential stratification scheme for real semi-algebraic varieties and its applications.Theoretical Computer Science84, 1 (1991), 77–105. https: //doi.org/10.1016/0304-3975(91)90261-Y Giovanni De Toni, Nastaran Okati, Suhas Thejaswi, Eleni Straitouri, and Manuel Gome...

    1991 doi

  2. [13]

    Curran Associates, Inc., 31380–31409. https://proceedings.neurips.cc/paper_files/paper/2024/file/37d4d4413b7c7558cc27a6d3d42ea998-Paper-Conference.pdf Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su

    2024

  3. [17]

    2022), 74–88

    Taking Advice from (Dis)Similar Machines: The Impact of Human-Machine Similarity on Machine-Assisted Decision-Making.Proceedings of the AAAI Conference on Human Computation and Crowdsourcing10, 1 (Oct. 2022), 74–88. https://doi.org/10.1609/hcomp.v10i1.21989 Karim Hamade, Reid ...

    2022 doi

  4. [18]

    arXiv:2402.11157 [econ.TH] https://arxiv.org/abs/2402.11157 Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q

    The Value of Context: Human versus Black Box Evaluators. arXiv:2402.11157 [econ.TH] https://arxiv.org/abs/2402.11157 Vivian Lai, Samuel Carton, Rajat Bhatnagar, Q. Vera Liao, Yunfeng Zhang, and Chenhao Tan

    arXiv

  5. [19]

    InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems(New Orleans, LA, USA)(CHI ’22)

    Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems(New Orleans, LA, USA)(CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 54, 18 pages. h...

    2022

  6. [21]

    https://doi.org/10.1016/j.obhdp

    Algorithm appreciation: People prefer algorithmic to human judgment.Organizational Behavior and Human Decision Processes151 (2019), 90–103. https://doi.org/10.1016/j.obhdp. 2018.12.005 Brian Lubars and Chenhao Tan

    2019 doi

  7. [22]

    arXiv:2402.12237 [cs.LG] https://arxiv.org/abs/2402.12237 David Madras, Toniann Pitassi, and Richard Zemel

    Learning to Defer in Content Moderation: The Human-AI Interplay. arXiv:2402.12237 [cs.LG] https://arxiv.org/abs/2402.12237 David Madras, Toniann Pitassi, and Richard Zemel

    arXiv

  8. [23]

    C B Mervis and E Rosch

    Allocation of functions.Handbook of human factors and ergonomics methods(2005), 34–1. C B Mervis and E Rosch

    2005

  9. [26]

    http://www.jstor.org/stable/24591747 David Owens, Zachary Grossman, and Ryan Fackler

    Some Useful Moment Results in Sampling Problems.The American Statistician68, 4 (2014), 282–296. http://www.jstor.org/stable/24591747 David Owens, Zachary Grossman, and Ryan Fackler

    2014

  10. [27]

    American Economic Journal: Microeconomics6, 4 (November 2014), 138–61

    The Control Premium: A Preference for Payoff Autonomy. American Economic Journal: Microeconomics6, 4 (November 2014), 138–61. https://doi.org/10.1257/mic.6.4.138 Samir Passi and Mihaela Vorvoreanu. 2022.Overreliance on AI: Literature Review. Technical Report MSR-TR-2022-12. Mi...

    2014 doi

  11. [29]

    Maithra Raghu, Katy Blumer, Greg Corrado, Jon Kleinberg, Ziad Obermeyer, and Sendhil Mullainathan

    What Is Agentic AI, and How Will It Change Work? (2024). Maithra Raghu, Katy Blumer, Greg Corrado, Jon Kleinberg, Ziad Obermeyer, and Sendhil Mullainathan

    2024

  12. [32]

    https://doi.org/10.3758/s13423-013-0506-0 Aris Spanos

    Prototypes, exemplars, and the natural history of categorization.Psychonomic Bulletin & Review21, 2 (01 Apr 2014), 312–331. https://doi.org/10.3758/s13423-013-0506-0 Aris Spanos. 1999.Probability theory and statistical inference: empirical modelling with observational data. Ca...

    2014 doi

  13. [33]

    https://doi.org/10.17705/1thci.00058 Tesla

    The Buck Stops There: The Impact of Perceived Accountability and Control on the Intention to Delegate to Software Agents.AIS Transactions on Human-Computer Interaction6 (03 2014). https://doi.org/10.17705/1thci.00058 Tesla. 2024.Autopilot and Full Self-Driving (Supervised). ht...

    2014 doi

  14. [35]

    InMachine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I(Würzburg, Germany)

    Novel Dense Subgraph Discovery Primitives: Risk Aversion and Exclusion Queries. InMachine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I(Würzburg, Germany). Springer-Verlag, Berl...

    2019 doi

  15. [36]

    https://proceedings.neurips.cc/ paper_files/paper/2023/file/5950bf290a1570ea401bf98882128160-Paper-Datasets_and_Benchmarks.pdf Berkeley J

    Curran Associates, Inc., 28091–28114. https://proceedings.neurips.cc/ paper_files/paper/2023/file/5950bf290a1570ea401bf98882128160-Paper-Datasets_and_Benchmarks.pdf Berkeley J. Dietvorst, Joseph P. Simmons, and Cade Massey

    2023

  16. [37]

    https://proceedings.neurips.cc/paper_files/paper/2024/file/ f9743a7bfee6592e3ff913ffadd8a857-Paper-Conference.pdf John R

    Curran Associates, Inc., 138088–138129. https://proceedings.neurips.cc/paper_files/paper/2024/file/ f9743a7bfee6592e3ff913ffadd8a857-Paper-Conference.pdf John R. Anderson

    2024

  17. [38]

    InProceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol

    Do Large Language Models Perform the Way People Expect? Measuring the Human Generalization Function. InProceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235), Ruslan Salakhutdinov, Zico Kolter, Katherine Heller...

    2024

  18. [39]

    arXiv:2402.09384 [econ.TH] https://arxiv.org/abs/2402.09384 Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan

    Persuasion, Delegation, and Private Information in Algorithm-Assisted Decisions. arXiv:2402.09384 [econ.TH] https://arxiv.org/abs/2402.09384 Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan

    arXiv

  19. [40]

    https://proceedings

    Curran Associates, Inc., 20744–20757. https://proceedings. neurips.cc/paper_files/paper/2022/file/82ad13ec01f9fe44c01cb91814fd7b8c-Paper-Conference.pdf YXD

    2022

  20. [41]

    Integrating Categorization and Decision-Making.Cognitive Science47, 1 (2023), e13235. https://doi.org/10.1111/cogs.13235 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cogs.13235 Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yon...

    2023 doi

  21. [42]

    WebArena: A Realistic Web Environment for Building Autonomous Agents. arXiv:2307.13854 https://webarena.dev A Computing the loss in the single feature setting Recall the setting of Section 3, where each state has two binary features, the human observes the first feature, and t...

    2025 arXiv

  22. [45]

    Designing Algorithmic Delegates EC ’25, July 7–10, 2025, Stanford, CA, USA This is indeed the case

    Combining this with the analogue of Lemma 1, we may assume as before that there is exactly one statexin𝐶∩𝐾for each𝐶,𝐾. Designing Algorithmic Delegates EC ’25, July 7–10, 2025, Stanford, CA, USA This is indeed the case. We know that Problem3≡min R min 𝑓𝑀 ∑︁ 𝐶∈H\R 𝑃(𝐶)ℓ𝐻(𝑓∗ 𝐻,𝐶)...

    2025

  23. [46]

    This in turn means thatD(𝑓 ∗ 𝐻,𝑓 ∗ 𝑀)is a solution to Problem 2.□ Now, recall Proposition 2, where𝑋(R)= Ð 𝐶∈R 𝐶and𝑓 R 𝑀(𝐾)=E[𝑓 ∗|𝑋(R)∩𝐾]

    On the other hand, if𝑓∗ 𝑀 is a solution to Problem 1, then𝑓∗ 𝑀,D(𝑓 ∗ 𝐻,𝑓 ∗ 𝑀) is a solution to Problem 1′ and thus to Problem 2′. This in turn means thatD(𝑓 ∗ 𝐻,𝑓 ∗ 𝑀)is a solution to Problem 2.□ Now, recall Proposition 2, where𝑋(R)= Ð 𝐶∈R 𝐶and𝑓 R 𝑀(𝐾)=E[𝑓 ∗|𝑋(R)∩𝐾]. EC ’25, J...

    2025

  24. [48]

    =min 𝑅 1 ℎ ∑︁ 𝑖∉𝑅 𝜎2(𝑤𝑗|𝑗∈[𝑚],𝑖)+ |𝑅| 𝑛 ∑︁ 𝑗 𝜎2(𝑢𝑖|𝑖∈𝑅,𝑗) since variance is does not depend on additive constants. In turn, this is equivalent to min 𝑅 ℎ−|𝑅| ℎ 𝜎2(𝑤𝑗|𝑗∈[𝑚])+ |𝑅| 𝑛 𝑚𝜎2(𝑢𝑖|𝑖∈𝑅) =min 𝑅 1− |𝑅| ℎ 𝜎2(𝑤)+ |𝑅| 𝑛 𝑛 ℎ𝜎2(𝑢𝑖|𝑖∈𝑅)(𝜎 2(𝑤):=𝜎 2(𝑤𝑗|𝑗∈[𝑚])) =min 𝑘 min 𝑅:|𝑅|=𝑘 ...

    2013

  25. [49]

    □ To prove Lemmas 3 and 4, we will rely heavily on a result relating the variance of a set to that of its subsets, shown by O’Neill [2014].7 Result 1(Corollary of Result 1 of O’Neill [2014]).For 𝑆⊂R , define𝜇(𝑆)= 1 |𝑆| Í 𝑥∈𝑆 𝑥, and𝜎2(𝑆)= 1 |𝑆| Í 𝑥∈𝑆(𝑥−𝜇(𝑆))

    2014

  26. [50]

    population

    shows this result in the case of sample variance, but it is easy to extend to true (“population”) variance as we show here. Designing Algorithmic Delegates EC ’25, July 7–10, 2025, Stanford, CA, USA Then |𝑆|𝜇(𝑆)=|𝑆 1|𝜇(𝑆 1)+|𝑆 2|𝜇(𝑆

    2025

  27. [53]

    arrangements

    subsets ofH. Moreover, we may compute𝑓R 𝑀 and thenℓ(𝑓 ∗ 𝐻,𝑓 R 𝑀) in𝑂(𝑛) for eachR. Thus, we may compare the loss of𝑓 R 𝑀 for eachR⊆Hin time𝑂(𝑛 2)and selectRwith the minimum loss. Part 2.Suppose that |𝐼𝑀|=𝑂( 1). As before, we assume𝑃(𝐶)> 0for all 𝐶; these categories do not affe...

    1991

  28. [54]

    This has time complexity𝑂(𝑛)

    We can computeR 0 by making a linear pass through all the states. This has time complexity𝑂(𝑛). Designing Algorithmic Delegates EC ’25, July 7–10, 2025, Stanford, CA, USA For a human category𝐶∈H, ℓ𝑀(𝑓R 𝑀,𝐶)=0⇐⇒ ∑︁ x∈𝐶 𝑃(x) 𝑃(𝐶) 𝑓R 𝑀(𝐾(x))−𝑓 ∗(x) 2 =0 ⇐⇒𝑓 R 𝑀(𝐾(x))=𝑓 ∗(x)for al...

    2025

  29. [55]

    Theorem 7.Unless P=NP , there is no algorithm to find an optimal delegate 𝑓∗ 𝑀 in time polynomial in𝑛for all delegation settings. Proof. In Proposition 2, we showed that when the human and machine features partition the set of all features it is necessary to solve the problemV...

    2016

  30. [202]

    2024.How AI Travel Can Curate Your Perfect Wine Tour in Italy

    https://doi.org/10.1007/s11166-017-9259-x Booked AI. 2024.How AI Travel Can Curate Your Perfect Wine Tour in Italy. https://www.booked.ai/blogs/how-ai-travel- can-curate-your-perfect-wine-tour-in-italy Ulrik Brandes, Eugenia Holm, and Andreas Karrenbauer

    2024 doi

  31. [1981]

    https://doi.org/10.1146/annurev.ps.32.020181.000513 Allen E

    Categorization of Natural Objects.Annual Review of Psychology32, Volume 32, 1981 (1981), 89–115. https://doi.org/10.1146/annurev.ps.32.020181.000513 Allen E. Milewski and Steven H. Lewis

    1981

  32. [1985]

    https://doi.org/10.1177/ 001872088502700104 Mark Purdy

    The Allocation of Functions in Systems.Human Factors27, 1 (1985), 33–45. https://doi.org/10.1177/ 001872088502700104 Mark Purdy

    1985

  33. [1986]

    http://www.jstor.org/stable/255943 Jennifer M

    Predictors and Consequences of Delegation.The Academy of Management Journal29, 4 (1986), 754–774. http://www.jstor.org/stable/255943 Jennifer M. Logg, Julia A. Minson, and Don A. Moore

    1986

  34. [1991]

    https://doi.org/10.1037/0033-295x.98.3.409 F

    The Adaptive Nature of Human Categorization.Psychological Review98, 3 (1991), 409–429. https://doi.org/10.1037/0033-295x.98.3.409 F. Ashby and W. Maddox

    1991 doi

  35. [1997]

    https://doi.org/10.1006/ijhc.1996.0100 Sendhil Mullainathan

    Delegating to software agents.International Journal of Human-Computer Studies46, 4 (1997), 485–500. https://doi.org/10.1006/ijhc.1996.0100 Sendhil Mullainathan

    1997

  36. [2000]

    https://doi.org/10.1075/pc.8.1.05tow Charalampos E

    Exploring the relations between categorization and decision making with regard to realistic face stimuli.Pragmatics & Cognition8, 1 (2000), 83–105. https://doi.org/10.1075/pc.8.1.05tow Charalampos E. Tsourakakis, Tianyi Chen, Naonori Kakimura, and Jakub Pachocki

    2000 doi

  37. [2005]

    https: //doi.org/10.1146/annurev.psych.56.091103.070217 Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, and Daniel S

    Human Category Learning.Annual review of psychology56 (02 2005), 149–78. https: //doi.org/10.1146/annurev.psych.56.091103.070217 Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, and Daniel S. Weld

    2005

  38. [2013]

    https: //stackoverflow.com/a/20143904 Rong Zheng, Jerome R

    Computing the subset giving the minimum standard deviation in an array.Stack Overflow(2013). https: //stackoverflow.com/a/20143904 Rong Zheng, Jerome R. Busemeyer, and Robert M. Nosofsky

    2013

  39. [2014]

    https://doi.org/10.1037/xge0000033 Alexandre Drouin, Maxime Gasse, Massimo Caccia, Issam H

    Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err.CSN: Business (Topic)(2014). https://doi.org/10.1037/xge0000033 Alexandre Drouin, Maxime Gasse, Massimo Caccia, Issam H. Laradji, Manuel Del Verme, Tom Marty, David Vazquez, Nicolas Chapados, and Ale...

    2014 doi

  40. [2016]

    https://doi.org/10

    Supervised autonomous robotic soft tissue surgery.Science Translational Medicine8, 337 (2016), 337ra64–337ra64. https://doi.org/10. 1126/scitranslmed.aad9398 arXiv:https://www.science.org/doi/pdf/10.1126/scitranslmed.aad9398 J. David Smith

    2016 doi

  41. [2017]

    https: //psycnet.apa.org/record/2018-44226-011 Rohan Alur, Manish Raghavan, and Devavrat Shah

    To Delegate or Not to Delegate: Gender Differences in Affective Associations and Behavioral Responses to Delegation.Academy of Management Journal(2017). https: //psycnet.apa.org/record/2018-44226-011 Rohan Alur, Manish Raghavan, and Devavrat Shah

    2017

  42. [2018]

    The Algorithmic Automation Problem: Prediction, Triage, and Human Effort. (2018). Azad Shademan, Ryan S. Decker, Justin D. Opfermann, Simon Leonard, Axel Krieger, and Peter C. W. Kim

    2018

  43. [2019]

    Machine Learning Interpretability: A Survey on Methods and Metrics.Electronics8, 8 (2019). https://doi.org/10.3390/electronics8080832 Alan Chan, Rebecca Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, M...

    2019 doi

  44. [2021]

    https://doi.org/10.1609/aaai.v35i13.17359 Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S

    Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork.Proceedings of the AAAI Conference on Artificial Intelligence35, 13 (May 2021), 11405–11414. https://doi.org/10.1609/aaai.v35i13.17359 Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S. Lasecki, Daniel S. Wel...

    2021 doi

  45. [2022]

    https://doi.org/10.1016/j.chb.2022.107308 Diogo V

    Rise of the machines: Delegating decisions to autonomous AI.Computers in Human Behavior134 (2022), 107308. https://doi.org/10.1016/j.chb.2022.107308 Diogo V. Carvalho, Eduardo M. Pereira, and Jaime S. Cardoso

    2022

  46. [2023]

    Practices for Governing Agentic AI Systems. (2023). Modupe Akinola, Ashley E. Martin, and Katherine W. Phillips

    2023

  47. [2024]

    InProceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol

    WorkArena: How Capable are Web Agents at Solving Common Knowledge Work Tasks?. InProceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235), Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.