{"id":"ceda7867-223f-4094-9f53-b1ca242c0ad6","arxiv_id":"2607.00420","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A randomized field experiment finds that limiting downward overrides to two per machine reduces inventory by 1.28% without sales loss by prompting workers to select higher-value SKUs for intervention.","lead":"The study ran a randomized field experiment with 553 workers at a Chinese vending machine company, testing limits on how often humans can override an AI's inventory decisions. A two-override cap per machine cut excess stock by 1.28% while preserving sales, unlike unlimited overrides which also reduced sales.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"LATE confirmation of selective SKU overrides does not isolate mechanism from other behavioral or compliance differences induced by the constraint itself.","rationale":"The reader's weakest assumption directly matches the load-bearing identification issue for the mechanism. The randomized design supports the headline reduced-form effects, but moving from those effects to the stated causal story about selective filtering requires the additional exclusion and selection evidence that the LATE step is meant to provide. Full-text details on the exact IV specification and robustness checks would be needed to raise confidence above low.","tokens_in":1776,"tokens_out":312,"duration_ms":14929,"concrete_test":"Re-estimate the LATE of number of overrides on sales separately for the free-override and constrained arms (or interact treatment with override count); if the marginal effect of an additional override differs significantly between arms after controlling for SKU characteristics, the selective-quality interpretation is supported; otherwise the mechanism claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim attributes sales preservation specifically to workers choosing higher-value SKUs under the two-override limit. Randomization identifies the reduced-form effect on inventory and sales, but the LATE step (instrumenting overrides with treatment assignment) requires the exclusion restriction that the constraint affects outcomes only through the count and selection of overrides. The constraint could instead alter worker attention, effort allocation across machines, or compliance patterns in ways that preserve sales independently of SKU quality. The abstract provides no detail on how LATE isolates the selection channel versus these alternatives.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript reports results from a randomized field experiment with 553 workers at a Chinese smart vending retailer managing over 59,000 machines. It compares no-override, free-override, and constrained (two downward overrides per machine) policies. Free overrides reduce inventory by 1.95% and sales by 1.19%; constrained overrides reduce inventory by 1.28% with no sales loss. The authors interpret the sales preservation as evidence of selective filtering of higher-value SKUs by workers, supported by LATE analysis. Effects are larger for experienced workers, high-incentive SKUs, and growth-stage SKUs; a simulated personalized policy raises sales probability by 9.1%.","tokens_in":1881,"tokens_out":361,"duration_ms":19798,"significance":"If the causal identification is robust, the work supplies field-experiment evidence on structuring limited human discretion to improve AI-supervised operational decisions, with direct implications for retail inventory, logistics, and resource allocation. The scale of the experiment (thousands of machines, thousands of SKUs) and the low-cost nature of the intervention are strengths that could inform practice without requiring model retraining or information redesign.","major_comments":[{"comment":"Abstract: the LATE step is presented as confirming that sales preservation occurs because workers select better SKUs under the two-override limit. However, the exclusion restriction—that the constraint affects outcomes solely through override count and selection—is not shown to hold against plausible alternatives (e.g., changes in worker attention, effort reallocation across machines, or compliance patterns). This assumption is load-bearing for the selective-filtering mechanism claim.","section":"Abstract"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comment on the identification strategy. We address the concern regarding the LATE exclusion restriction below and propose a targeted revision to strengthen the presentation without altering the core claims.","responses":[{"response":"We appreciate the referee raising this identification issue. The LATE in the manuscript instruments the realized number of downward overrides with the randomized two-override constraint to isolate the effect of selective filtering on sales. While direct tests of the exclusion restriction are not feasible, the field-experiment design randomizes the policy at the worker-machine level with no changes to information, incentives, or task structure, making it unlikely that the constraint operates through attention shifts or effort reallocation (workers manage the same machines under all arms). Compliance is high by design, as the limit is mechanically enforced. Nevertheless, to address the concern transparently, we will revise the abstract to state the identifying assumption more explicitly and add a short robustness subsection discussing why alternative channels are implausible given the low-cost, rule-based intervention. This constitutes a partial revision focused on clarity rather than new empirical work.","revision_made":"partial","referee_comment":"[Abstract] Abstract: the LATE step is presented as confirming that sales preservation occurs because workers select better SKUs under the two-override limit. However, the exclusion restriction—that the constraint affects outcomes solely through override count and selection—is not shown to hold against plausible alternatives (e.g., changes in worker attention, effort reallocation across machines, or compliance patterns). This assumption is load-bearing for the selective-filtering mechanism claim."}],"tokens_in":1430,"tokens_out":340,"duration_ms":14864,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core result is that limiting downward overrides to two per machine in this smart vending setup reduces inventory by 1.28 percent without cutting sales, whereas unrestricted overrides reduce inventory more but also reduce sales by 1.19 percent. The randomized assignment to 553 workers and the LATE estimates give a clean reduced-form comparison across the three arms.\n\nThe experiment is the main strength. It is a real field test with actual operational decisions, many machines, and a large SKU set, and the authors report heterogeneous effects by worker experience and SKU type. That is useful evidence on how to structure human discretion around an autonomous replenishment algorithm.\n\nThe soft spot is the mechanism claim. The paper attributes the sales preservation to workers making higher-quality selective overrides under the cap. Randomization identifies the overall effect, but the LATE step needs the constraint to affect outcomes only through the number and choice of overrides. It could instead change how much attention workers pay to each machine or how they allocate effort across the fleet. The abstract gives no detail on checks that would separate those channels, so the selective-filtering story is plausible but not yet isolated.\n\nThis paper is for people working on human-AI oversight in operations or retail who want a low-cost policy lever rather than new algorithms. A reader running field experiments on discretion would find the design and the LATE framing worth looking at.\n\nIt should go to peer review. The experiment itself is worth referee time even if the mechanism section needs tightening.","headline":"The vending RCT shows a two-override limit cuts inventory 1.28% with no sales drop while free overrides hurt sales, but the selective-SKU mechanism rests on an untested exclusion restriction.","tokens_in":2353,"tokens_out":388,"would_cite":false,"duration_ms":13607,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Limiting overrides to two per machine lets workers reduce inventory by 1.28% without losing sales.","keywords":["human supervision of AI","override policy design","field experiment","inventory management","smart vending machines","discretion limits","selective filtering"],"falsifier":"Finding that the SKUs overridden under the constrained policy are no higher value than those overridden without the limit, yet sales still hold steady, would challenge the claim that selective filtering drives the result.","tokens_in":2675,"feed_emoji":"📦","tokens_out":575,"duration_ms":21107,"temperature":0.7,"pith_summary":"The paper tests whether limiting how often workers can override an AI's inventory decisions improves outcomes. It finds that capping downward overrides at two per vending machine reduces stock levels by 1.28 percent while keeping sales steady, because workers focus overrides on the most valuable items. Free overrides, by contrast, cut both inventory and sales. This matters for any setting where humans supervise autonomous systems, as it shows a low-cost way to capture useful private information without letting bias degrade performance.","feed_headline":"Limit on overrides trims vending inventory 1.28% with no sales drop","feed_subtitle":"Workers under a two-change cap select better products to adjust than those given free rein","key_machinery":"The constrained override policy that limits the number of overrides per decision episode to enable selective filtering of high-value cases.","core_discovery":"In a randomized experiment with 553 workers managing smart vending machines, a policy limiting downward overrides to two per machine produced a 1.28 percent inventory reduction with no sales decline. Workers under this constraint selected higher-value SKUs to override compared to those with unlimited overrides, which reduced inventory by 1.95 percent but also cut sales by 1.19 percent. The effect was confirmed through local average treatment effects and was strongest among experienced workers, high-incentive products, and growth-stage SKUs.","pith_inferences":["The same limit-based approach might improve human oversight in other operational AI systems such as logistics scheduling.","Managers could experiment with adjusting the override cap based on worker tenure or product type to amplify the benefits."],"forward_implications":["Gains are largest for experienced workers.","High-incentive SKUs and growth-stage SKUs benefit most.","A simulated personalized policy further increases sales probability by 9.1 percent."],"fun_headline_variants":["Override cap cuts vending inventory 1.28% sales steady","Two override limit trims stock 1.28% no sales drop","Constrained overrides lower inventory 1.28% sales hold","Vending trial override cap reduces stock 1.28% sales flat"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"That the lack of sales loss under the two-override limit results specifically from workers choosing better SKUs to override, rather than from unrelated differences between the worker groups or machines.","fun_headline_variants_meta":{"raw":{"variants":["Override cap cuts vending inventory 1.28% sales steady","Two override limit trims stock 1.28% no sales drop","Constrained overrides lower inventory 1.28% sales hold","Vending trial override cap reduces stock 1.28% sales flat"]},"model":"grok-4.3","cost_usd":0.00641,"raw_usage":{"total_tokens":2970,"prompt_tokens":758,"num_sources_used":0,"completion_tokens":73,"cost_in_usd_ticks":64103000,"prompt_tokens_details":{"text_tokens":758,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2139,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":758,"tokens_out":73,"duration_ms":15927,"temperature":1.0,"reasoning_tokens":2139,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-02T06:54:24.167716+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Finding that the SKUs overridden under the constrained policy are no higher value than those overridden without the limit, yet sales still hold steady, would challenge the claim that selective filtering drives the result.","supporting_citations":[],"review_version":1}