{"id":"df3b86ba-13e0-4547-a012-7f2768f9b6d4","arxiv_id":"2611.05275","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Routing each input to the best-suited model beats one fixed model by 5.4 points in principle, but learned routers capture only 0.3 points of that, while cost-aware routing multiplies efficiency by about 8.8x.","lead":"This paper measures how much is gained by routing each AI task to the best-suited model instead of always using one fixed model, across 25 benchmarks and 452 candidate configurations, from text and images to code and robots. The robust payoff is cost: roughly 8.8x better quality-per-dollar at near-zero quality loss, while the potential quality gain of perfect routing is 5.4 points but learned routers capture only about 0.3.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"The abstract's headline 8.82x average is computed from fitted routes, not the certified ones: only the code and text rows of Table 5 carry Proposition 5 guarantees, and the fitted intervals and image row do not establish the claimed near-zero quality loss.","rationale":"The paper is unusually careful: it discloses that baseline construction can flip the sign of a router's gain (Section 4.4), that fitted floors fail in only 12.5% of refits, and that part of the oracle gap is outcome noise. The theory in Appendix A appears coherent, and the certified code/text episode is well executed. The concern is not a formal flaw in Theorem 2 or Proposition 5; it is a mismatch between what is certified and what the abstract advertises. The central claim would be true if every route contributing to the 8.82x average had a certified quality floor and if the quoted efficiencies were the certified ones. Neither holds. Only CodeRouterBench and RouterBench are certified, and only under matched i.i.d. replay (b511, b560); image generation and robots are fitted evaluations whose intervals do not meet the quality tolerance. The reader's weakest assumption about matched i.i.d. deployment sampling is the precondition for those certified rows, so my proposed check overlaps with it but focuses on the headline's use of fitted rows. Because the authors distinguish fitted and certified rows in Table 5 and state the i.i.d. limitation in the limitations section, the correct fix is to report the certified average and explicitly qualify the abstract's claim, which is consistent with the reader's CONDITIONAL verdict. I would not change that verdict.","tokens_in":40129,"tokens_out":11044,"duration_ms":84907,"concrete_test":"Recompute Table 5's average using the certified median code/text rows (19.4x and 6.7x) instead of the fitted rows (25.5x and 7.1x), keeping the image and robot rows, and report the average with a paired bootstrap over the four testbeds. If the certified-route average is about 7.2x rather than 8.82x, or if the bootstrap interval's lower bound falls below the near-order-of-magnitude claim, the headline should be revised to the certified average; this also settles whether the abstract's 'near-zero loss' is supported, since the image interval's lower bound is -0.87, below the stated 0.5-point tolerance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is the relation between the headline efficiency claim and the certified floor. The abstract presents 'routing raises efficiency by 8.82x on average' with '25.5x on coding tasks and 7.1x on text QA' as the main result. Table 5 shows that these are the fitted, zero-loss rows, not the certified rows. The certified code and text routes are 19.4x and 6.7x, with the floor guaranteed only under Proposition 5's matched i.i.d. deployment-sample protocol, a condition the authors state in Appendix C.2 (b511) and Limitations (b560). Replacing fitted with certified code/text values changes the average to roughly (19.4 + 6.7 + 1.35 + 1.33)/4 = 7.2x, and even that number is not fully certified because image generation and robots have no Proposition 5 guarantee. The quality side of the fitted claim is also not established by the reported intervals: code is -0.44 [-1.75, +0.97] and text is -0.40 [-0.84, +0.08], both crossing zero; the image-generation route included in the average has interval -0.47 [-0.87, -0.06], whose lower endpoint is below the 0.5-point tolerance; and the robot interval includes zero. Thus the '8.82x at near-zero loss' statement attributes certified reliability to numbers that are fitted evaluations. If the central claim is that per-input routing with a certified floor reliably cuts cost by almost an order of magnitude, the evidence supports a smaller certified average on two testbeds under an idealized deployment assumption; the unqualified headline is not supported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"This paper develops a general framework and empirical testbed for per-input routing among heterogeneous AI candidates. The theory characterizes efficiency-optimal routers as maximizers of v - lambda c (Theorem 2), proves an information ladder (Proposition 3), contrasts outcome regression with winner-label training (Proposition 4), and provides a fixed-sequence p-value certification of quality floors (Proposition 5). The empirical contribution is a trajectory matrix of 374,401 scored runs over 25 benchmarks and 452 configurations; the main findings are a 5.4-point oracle gap over the best fixed candidate, a learned capability-tree router 0.3 points above the fixed candidate, a robot router matching per-condition best policies with 24% less policy time, and an efficiency claim of 8.82x average with 25.5x on code and 7.1x on text QA. The paper's central efficiency claim is the one requiring scrutiny: the headline numbers come from fitted routes, while only the code and text rows carry a certified floor under matched i.i.d. replay.","tokens_in":40550,"tokens_out":11268,"duration_ms":80726,"significance":"The manuscript makes a substantial empirical and theoretical contribution if its claims are properly qualified. The appendix proofs are coherent and use standard machinery (Dinkelbach, LP duality, Hoeffding and empirical-Bernstein bounds), and the experimental practice is careful: matched inputs, held-out source-group splits, disclosed exclusions such as SciCode, and paired bootstrap intervals. The negative results on winner-label training and the matched-baseline artifact are valuable and transferable. The main weakness is not the derivation but the mapping from Table 5 to the abstract: the 8.82x average and the 'below 0.5 points' loss are fitted evaluations; the certified guarantees apply only to code and text under an idealized deployment protocol, and the efficiency multipliers themselves are not certified. With a revised presentation of fitted versus certified numbers, this would be a significant contribution to model routing.","major_comments":[{"comment":"The headline '8.82x on average' and the 'quality loss below 0.5 points' are computed from the fitted rows of Table 5, not from the certified rows. The code and text fitted rows are 25.5x and 7.1x; the certified rows are 19.4x and 6.7x, and image generation and robots carry no Proposition 5 guarantee. Replacing the code and text fitted values with the certified ones changes the four-testbed average to approximately 7.2x, and even that value is not fully certified. The quality intervals in Table 5 also do not establish 'near-zero loss' as stated: code -0.44 [-1.75,+0.97] and text -0.40 [-0.84,+0.08] both include zero, and image -0.47 [-0.87,-0.06] has a lower endpoint below the 0.5-point tolerance. The abstract, Figure 1 caption, and conclusion should present fitted and certified numbers separately and should not present the 8.82x figure as certified.","section":"Abstract, Fig. 1, §5.1, Table 5","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The certified floor is guaranteed only under the protocol stated in Appendix C.2: n inputs drawn i.i.d. from the deployment law, independent of the fitting data, with matched replay of every candidate on the price path and the reference. The authors acknowledge this in Appendix D, but the main-text and abstract framing of the certified efficiency result does not carry the condition into the headline. This matters because the paper itself reports that fitted floors held on test in only 12.5% of 200 refits; if deployment inputs drift or replay is infeasible, the certified claim does not bind, and the unqualified 'reliably cuts cost' reading is unsupported. The certification paragraph needs an explicit statement that the guarantee is conditional on the matched i.i.d. replay protocol and does not extend to the image and robot testbeds.","section":"§5.1, Appendix C.2 (b511), Appendix D (b560)","authors_acknowledge":"Certifying the floor.Theorem 2 is applied to fitted values and costs, so a fitted floor need not hold on test inputs.","authors_acknowledge_p":0.71},{"comment":"Proposition 5 certifies a quality floor, not an efficiency multiplier. The '19.4x' and '6.7x' entries are median test-set efficiency ratios of routers selected by the certification procedure; their efficiency is not covered by the Proposition 5 guarantee. The text should state this explicitly wherever 'certified' appears next to an efficiency number, including the Finding 6 paragraph and the Figure 1 caption, so that readers do not infer that the certification procedure validates the efficiency gain itself.","section":"Table 5, Appendix A.5","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The word 'anlyasitical' in the introduction should be 'analytical'.","section":"§1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The leaf counts differ between the rule-based capability tree (107 leaves) and the learned capability-tree router (157 leaves); the text should clarify that these are different trees rather than leaving an apparent inconsistency.","section":"§3, §4.1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The headline +0.30 gain for the one-router study is reported without its paired bootstrap interval, although intervals are promised in the Section 4 setup; adding the interval would help calibrate this claim.","section":"Table 3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The abstract says 25 benchmarks while Table 1 reports 24 benchmarks for the potential-gain analysis because of the SciCode exclusion; a footnote or an explicit sentence would prevent an apparent inconsistency.","section":"Abstract, Table 1","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"You should know two things. First, this is one of the most honest large-scale routing papers I have read: it shows its own baseline construction flipping a router's gain sign, reports that fitted quality floors hold on test in only 12.5% of 200 refits, and demonstrates that a headline \"oracle gap\" partly vanishes under repeated runs. Second, the abstract overstates the efficiency result. The 8.82x average comes from fitted routes, not certified ones; only the code and text rows of Table 5 carry the Proposition 5 guarantee, and averaging the certified numbers with the uncertified image and robot rows gives roughly 7.2x, not 8.82x. The quality intervals for code and text cross zero, and the image interval's lower endpoint sits below the tolerance.\n\nWhat is genuinely new: the omni-modal trajectory matrix (374,401 scored runs across 25 benchmarks, 452 candidates) is reusable infrastructure; the information ladder (Proposition 3) cleanly separates what coarse states can capture; Proposition 4's \"learn outcomes, not winners\" result is a real negative finding with a convincing robot demonstration; and Proposition 5's certified quality floor, applied with empirical-Bernstein p-values and fixed-sequence testing, held in 200 of 200 resplits at a median 19.4x on code. The appendix proofs check out: the Hoeffding p-value algebra is right, the Dinkelbach/LP duality step is standard, and the price-path construction is coherent. The empirical practice is careful: matched inputs, a stated split rule, disclosed exclusions, and paired bootstrap intervals on most panels.\n\nSoft spots, in proportion. The main one is the abstract's conflation of fitted and certified numbers; that should be fixed before publication, ideally by reporting intervals on the main router panel and moving the certified efficiency figures into the abstract. The learned-router headline (+0.30 points) is a single-seed result with no interval and is contradicted by the matched 11-benchmark study (-0.40); the robot gain includes zero. None of this sinks the paper, but it means the \"near-zero loss\" claim is not uniformly established. The certification protocol's i.i.d. deployment-sample assumption is real, but the authors state it plainly in Appendix C.2 and the Limitations section, so I would not count it as a hidden flaw.\n\nWho this is for: anyone working on LLM cascades, model routing, or cost-quality trade-offs in multi-model systems. It deserves a serious referee and likely acceptance after the headline is reined in. I would bring it to reading group and would cite the certified-floor result and the trajectory matrix.","headline":"A serious, unusually honest routing study whose headline efficiency number is fitted rather than certified; the certified core is smaller but real.","tokens_in":41242,"tokens_out":1118,"duration_ms":11623},"pith_extraction":{"msc":["68T05","68T07","90C32"],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"The paper argues that per-input routing across a pool of specialist AI models achieves 8.82x the quality-per-dollar of one reference model, with a certified quality floor that held in 200 of 200 resplits.","keywords":["model routing","language model routing","quality-cost trade-off","Dinkelbach fractional programming","certified quality floor","omni-modal trajectory matrix","robot policy routing","inference efficiency"],"falsifier":"Run the certified zero-loss route on a stream that is not i.i.d. — coding tasks grouped by source or difficulty, or a stream where the reference model's outcome is missing on a fraction of inputs — and count how often the realized quality change drops below $-\\varepsilon$; the guarantee $\\mathbb{P}(Q<-\\varepsilon)\\le\\alpha$ would fail if that frequency exceeds $\\alpha$. Even in-distribution the paper notes the guarantee is aggregate: the certified router loses quality in a median 8 of 21 test cells, so a per-cell floor is already not certified.","tokens_in":41478,"feed_emoji":"🧭","tokens_out":14737,"duration_ms":99062,"temperature":0.7,"pith_summary":"This paper sets out to show that routing — choosing, for each individual input, which of several specialist AI models, tools, or policies should handle it — is both theoretically principled and practically profitable across every output type, not just text. It proves that the efficiency-optimal router plays, in every input state, the candidate maximizing expected value minus $\\lambda$ times expected cost at one equilibrium price, and that naming a reference model plus a quality tolerance fixes that price instead of a hand-tuned budget (Theorem 2). It also proves a finite-sample certification: testing candidate routers in fixed sequence on an independent sample controls the probability that a cheaper route's quality falls below the tolerance (Proposition 5). Empirically, on a matrix of 374,401 scored runs over 25 benchmarks spanning language, image, video, speech, code, agents, and robots, the best candidate per input beats the best fixed model by 5.4 points, a single learned router beats it by 0.3 overall, and price routing raises quality per dollar $8.82\\times$ on average — $25.5\\times$ on coding — while the certified zero-loss floor held in all 200 resplits at a median $19.4\\times$. If correct, per-input orchestration of a heterogeneous model ecosystem becomes a dependable way to approach reference-level quality at a fraction of the inference cost.","feed_headline":"Routing cuts AI inference cost 8.82x with a certified floor","feed_subtitle":"Pick the best model per input: 25.5x cheaper coding and 7.1x cheaper text, with zero quality loss certified.","key_machinery":"The load-bearing object is the net-value rule $v(s,u)-\\lambda c(s,u)$ paired with the Dinkelbach root $\\eta^\\star$: the unique price at which the maximum attainable net value $\\Phi(\\lambda)=\\sum_s\\pi(s)\\max_u[v(s,u)-\\lambda c(s,u)]$ vanishes. Theorem 2 shows the efficiency-optimal router is precisely a maximizer of this net value at $\\eta^\\star$, and its constrained version shows that a quality floor merely selects a different single price, fixed by the reference and the tolerance. The second piece of machinery is the price path — the sequence of routers that maximize fitted net value as $\\lambda$ rises from $0$ to the fitted $\\eta^\\star$ — together with the certification of Proposition 5, which computes valid $p$-values for the null that a route loses more than $\\varepsilon$ and tests them in fixed sequence on an independent matched-replay sample, so the deployed route violates the floor with probability at most $\\alpha$. The third piece is the information ladder of Proposition 3, which makes the best fixed candidate and the per-input oracle the measurable outer rungs of every router's attainable performance, and Proposition 4, which fixes the training target: regress each candidate's outcome rather than classify the winner.","core_discovery":"The central claim, stated on the paper's own terms, is that routing a heterogeneous pool of candidates is a single optimization rather than a collection of heuristics. A router observes a state and should play, in every state, a candidate maximizing expected value minus $\\lambda$ times expected cost; the highest attainable efficiency $\\eta^\\star$ is the unique root of the convex function $\\Phi(\\lambda)=\\sum_s \\pi(s)\\max_u[v(s,u)-\\lambda c(s,u)]$, so an efficiency-optimal router is exactly a maximizer of expected net value at that one price (Theorem 2). When a reference model and a quality tolerance $\\varepsilon$ are named, the tolerance replaces the budget: floor-optimal routing is routing at a price fixed by the reference and the tolerance rather than by hand. The same framework yields an information ladder (Proposition 3) — attainable performance is sandwiched between the best fixed candidate and the per-input oracle, with the gap closable only by per-input information — and a finite-sample guarantee (Proposition 5): fixed-sequence testing of the price-path routers on an independent, matched-replay sample certifies at level $\\alpha$ that the deployed route's quality loss does not exceed $\\varepsilon$. On top of this theory, the paper reports that per-input oracle selection beats the best fixed model by 5.4 points across 24 test sets, that a single capability-tree router recovers part of that gap (0.3 points overall, 2.1 on video generation, 1.1 on robot manipulation with 24% less policy time), and that price routes raise efficiency $8.82\\times$ on average over the reference — $25.5\\times$ on code and $7.1\\times$ on text — with a certified zero-loss floor that held in all 200 of 200 resplits at a median $19.4\\times$.","pith_inferences":["Editorial inference: the reference-plus-tolerance framing turns the router's operating point into a deployment contract — anyone who can name a default model and a quality tolerance inherits a price and a guarantee — which also means the guarantee is only as meaningful as the named reference and the stated tolerance.","Editorial inference: in production, matched replay is rarely free and traffic rarely i.i.d., so the certified floor should be treated as a ceiling on the guarantee; the realistic fallback is the fitted floor, which on the code testbed held in only 25 of 200 refits (12.5%).","Editorial inference: the outcomes-not-winners result generalizes beyond routing to any training signal derived from choices — preference or selection datasets whose ties are broken by latency, cost, or ordering train models to optimize the tie-break, so outcome regression is the safer target wherever outcomes are measurable.","Editorial inference: the $8.82\\times$ headline is the simple mean of four testbed ratios ($25.5\\times$, $7.1\\times$, $1.35\\times$, $1.33\\times$), so a cost-weighted average would be far smaller; the headline gain is carried by the coding testbed, and the same route logic would need re-validation on workloads dominated by image, video, or robot inference."],"forward_implications":["A cost-cutting route can be deployed with a statistical guarantee: pick the reference and the tolerance, certify the floor on an independent sample, and the efficiency gain — a median $19.4\\times$ on code in 200 of 200 resplits — is reproducible rather than a fitted artifact.","Efficiency-optimal routing needs no separate budget-tuning stage: one price, set by the reference and the tolerance, replaces hand-picked operating points, which the paper shows land far lower ($2.16\\times$–$10.85\\times$) than the zero-loss route's $25.5\\times$ on the code testbed.","Router training should regress each candidate's outcome per input: on the same robot data, per-policy outcome regression reaches 71.6% success while winner-label classification reaches 42.8%, because a latency tie-break decided 94% of the winner labels.","Task-type labels alone recover only about 8% of the oracle's 5.4-point advantage, so routing gains must come from per-input prediction inside task types rather than from coarser classification.","Comparing candidates on the same inputs is a precondition for measuring any routing gain: a fixed baseline that required a score on every training input flipped a 0.4-point deficit into a 2.4-point gain for the learned router."],"supporting_citations":[{"why":"Supplies the nonlinear fractional-programming method whose unique root is the maximum achievable efficiency; Theorem 2's proof of optimal routing at a price is built on it.","marker":"[6]"},{"why":"The CodeRouterBench testbed: 9,999 coding tasks with per-model outcomes and token prices on shared prompts; it carries the headline code results (25.5x fitted, 19.4x certified median) and the verifier, portfolio, and memory studies.","marker":"[4]"},{"why":"The RouterBench testbed: 36,497 prompts with eleven models; it carries the text-QA efficiency results (7.1x fitted, 6.7x certified median).","marker":"[5]"},{"why":"The prior unified Lagrangian formulation of LLM routing and cascading that Theorem 2 extends; the paper contrasts its reference-and-tolerance price with a budget-tuned price.","marker":"[3]"},{"why":"Fixed-sequence multiple-testing methods that Proposition 5 adapts so the price-path certification needs no multiplicity correction.","marker":"[7]"},{"why":"The learn-then-test risk-control framework that grounds the finite-sample quality-floor guarantee used in Proposition 5.","marker":"[8]"},{"why":"Hoeffding's inequality supplies the p-value that makes the certified floor valid at level alpha on the independent sample.","marker":"[67]"},{"why":"Empirical-Bernstein bounds sharpen the certified p-values on the mixture-refined path, raising the certified median efficiency from 18.83x to 19.38x.","marker":"[68]"}],"fun_headline_variants":["Routing as one optimization: expected value minus cost","Route Anything certifies zero-loss routing at 8.82x efficiency","Oracle routing beats best fixed model by 5.4 points","Learned routers cut policy time 24% and beat fixed model","Per-input routing lifts efficiency with certified quality floor"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The certified efficiency numbers assume that certification inputs are drawn i.i.d. from the deployment distribution and that every candidate on the price path plus the reference model is scored on every one of those inputs (matched replay); the paper states that its theory takes the law of input states as given and that certification needs the reference's outcome on every certification input, so the guarantee does not bind under drift, non-i.i.d. arrivals, or missing reference outcomes.","fun_headline_variants_meta":{"raw":{"variants":["Routing as one optimization: expected value minus cost","Route Anything certifies zero-loss routing at 8.82x efficiency","Oracle routing beats best fixed model by 5.4 points","Learned routers cut policy time 24% and beat fixed model","Per-input routing lifts efficiency with certified quality floor"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000172,"raw_usage":{"cost":0.0001722084,"is_byok":false,"cost_details":{"upstream_inference_cost":0.0001722084,"upstream_inference_prompt_cost":3.88024e-05,"upstream_inference_completions_cost":0.000133406},"total_tokens":2065,"prompt_tokens":1332,"completion_tokens":733,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":1132,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":663}},"tokens_in":1332,"tokens_out":733,"cost_source":"openrouter_usage","duration_ms":8021,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":663,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-11-04T07:39:03.085470+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Run the certified zero-loss route on a stream that is not i.i.d. — coding tasks grouped by source or difficulty, or a stream where the reference model's outcome is missing on a fraction of inputs — and count how often the realized quality change drops below $-\\varepsilon$; the guarantee $\\mathbb{P}(Q<-\\varepsilon)\\le\\alpha$ would fail if that frequency exceeds $\\alpha$. Even in-distribution the paper notes the guarantee is aggregate: the certified router loses quality in a median 8 of 21 test cells, so a per-cell floor is already not certified.","supporting_citations":[{"cited_title":"On nonlinear fractional programming.Management science, 13(7):492–498, 1967","cited_arxiv_id":null,"evidence_quote":"Supplies the nonlinear fractional-programming method whose unique root is the maximum achievable efficiency; Theorem 2's proof of optimal routing at a price is built on it."},{"cited_title":"Agent-as-a-Router: Agentic Model Routing for Coding Tasks","cited_arxiv_id":"2606.22902","evidence_quote":"The CodeRouterBench testbed: 9,999 coding tasks with per-model outcomes and token prices on shared prompts; it carries the headline code results (25.5x fitted, 19.4x certified median) and the verifier, portfolio, and memory studies."},{"cited_title":"RouterBench: A Benchmark for Multi-LLM Routing System","cited_arxiv_id":"2403.12031","evidence_quote":"The RouterBench testbed: 36,497 prompts with eleven models; it carries the text-QA efficiency results (7.1x fitted, 6.7x certified median)."},{"cited_title":"A unified approach to routing and cascading for LLMs","cited_arxiv_id":null,"evidence_quote":"The prior unified Lagrangian formulation of LLM routing and cascading that Theorem 2 extends; the paper contrasts its reference-and-tolerance price with a budget-tuned price."},{"cited_title":"Westfall and Alok Krishen","cited_arxiv_id":null,"evidence_quote":"Fixed-sequence multiple-testing methods that Proposition 5 adapts so the price-path certification needs no multiplicity correction."},{"cited_title":"Angelopoulos, Stephen Bates, Emmanuel J","cited_arxiv_id":null,"evidence_quote":"The learn-then-test risk-control framework that grounds the finite-sample quality-floor guarantee used in Proposition 5."},{"cited_title":"Journal of the American Statistical Association , author =","cited_arxiv_id":null,"evidence_quote":"Hoeffding's inequality supplies the p-value that makes the certified floor valid at level alpha on the independent sample."},{"cited_title":"Empirical Bernstein bounds and sample variance penalization","cited_arxiv_id":null,"evidence_quote":"Empirical-Bernstein bounds sharpen the certified p-values on the mixture-refined path, raising the certified median efficiency from 18.83x to 19.38x."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}