{"id":"319c3b05-b94a-4a2a-9c53-8133b5258ec8","arxiv_id":"2609.30123","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"HEXIS compiles agent skills into extended finite state machines whose explicit transitions enforce the skill's required steps, improving task success by 16.1 percentage points on average over ReAct-based skill execution across four executors.","lead":"This paper presents HEXIS, a tool that converts a written agent skill into a small state machine so the model no longer has to guess which step comes next at each moment. Tested across four benchmark suites and four AI models, it beat plain \"skill plus ReAct\" execution by about 16 percentage points on average and cut token use on several models by a third to nearly ninety percent.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Table 1 reports no uncertainty or significance measures, yet the central claim is quantitative: the 16.1-pp average and '15 of 16 settings' rest on test sets of 51-121 tasks, with DABench and LongSeal margins of 0-5.9 pp (0-3 flipped tasks), so the headline magnitude is not supported at the stated…","rationale":"The reader's CONDITIONAL verdict and weakest assumption (judge-label routing fidelity) capture a real, author-conceded limitation: the machine's guards route on model-written verdicts whose correctness the compiler cannot check (b197, b300, Appendix F), and Appendix G's Prompt-only control plus cross-backbone token reductions partially defend the mechanism claim. My stress-test pass elevates a different, unaddressed issue to primary status: the headline is a precise quantitative superiority claim (16.1 pp; 15 of 16 settings) supported only by single-run point estimates on 51-121-task test sets with no variance or significance reporting. Recomputing per-cell margins shows DABench gains of 1.9-4.0 pp (1-2 successes on n=51) and a LongSeal tie, while the same Table exhibits baseline swings of up to 64.7 pp (AFlow on DABench) from swapping only the executor, demonstrating that the noise floor is several points wide. The 16.1-pp average therefore overstates what the data can bear; the contribution is better characterized as positive-but-noisy, with the magnitude concentrated in LiveMath and SpreadsheetBench. This is an addressable omission rather than a fatal flaw, so the verdict remains CONDITIONAL (UNCHANGED), but the conditions the reader should impose must now include uncertainty reporting for every Table 1 cell and for the aggregate average, not only code release and amortized compilation cost. I agree with the reader's identification of judge-label fidelity as a genuine and load-bearing mechanism concern; I disagree only that it is the single most load-bearing one, since it is explicitly conceded by the authors and partially controlled, whereas the statistical fragility of the quantified central claim is neither conceded nor controlled.","tokens_in":25965,"tokens_out":8429,"duration_ms":64436,"concrete_test":"Regenerate Table 1 from per-task paired outcomes and report, for every cell, a McNemar test (or bootstrap 95% CI over tasks) comparing HEXIS vs Skill + ReAct, plus a bootstrap CI for the aggregate 16.1-pp average across the 16 cells (test sets: 51, 51, 57, 121). If DABench (margins 1.9-4.0 pp) and LongSeal (0-13.8 pp) gains fail to exclude zero and the 95% CI of the average does not exclude a ~6-8 pp floor, then the headline must be re-framed as 'consistently non-worse with significant gains on LiveMath and SpreadsheetBench,' and the '15 of 16' count should be replaced by a significance-filtered count.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (b9, b166-b170) is quantitative: HEXIS improves success over Skill + ReAct by 16.1 pp on average and in 15 of 16 settings. No confidence interval, variance, or significance test appears anywhere in Table 1, and the test sets are small: SpreadsheetBench 57, DABench 51, LongSeal 51, LiveMath 121. Reconstructing the cells: DABench HEXIS-vs-S+R margins are 4.0, 4.0, 3.9, and 1.9 pp, i.e., 1-2 extra task successes each on n=51; LongSeal margins are 13.8, 11.8, 0.0 (exact tie on Qwen3.5-9B), and 5.9 pp, the last being 3 of 51 tasks. With binary per-task outcomes and n≈51-57, standard errors are roughly 5-7 pp, so several 'improvement' cells are within 1-2 flipped tasks of the baseline. The 16.1-pp average is dominated by LiveMath (~34 pp) and SpreadsheetBench (~19 pp); DABench contributes only ~3.5 pp. Baseline instability reveals the noise floor: AFlow on DABench is 80.4% with qwen3.6-flash but 15.7% with Qwen3.8-27B (a 64.7-pp swing from swapping only the executor), and ReasoningBank on LiveMath ranges 8.3-41.3% across executors. A handful of outcome flips would shrink '15 of 16' and cut the headline average by several points. The mechanism direction is better defended than the reader's weakest assumption suggests: Appendix G's Prompt-only ablation (identical machine content, no programmatic enforcement) isolates control-flow from content, and the large gains on GLM/Qwen3.8-27B with 28.4-94.5% fewer tokens argue against a compute-budget confound; the judge-label fidelity limit is explicitly conceded (b197, b300, Appendix F: 'verified' means only that the file read-back matches the selected letter). What is unaddressed is magnitude robustness: the claim is stated to one decimal place on data whose noise floor, demonstrated by the baselines' own swings, is several points wider than the margins being asserted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper introduces HEXIS, a method that compiles natural-language agent skills into extended finite state machines. Skill knowledge is placed in state-local prompts that guide in-state model reasoning, while operation order, data dependencies, branching, repetition, and termination are encoded as guarded transitions with typed variables. Compilation proceeds in two phases: an initial machine is drafted from the skill document by a construction model under static checks, and then development trajectories are aligned with the machine to add or reuse states, with updates accepted only when static checks and replay of all previously accepted traces pass (Eq. 9). The authors evaluate on four benchmarks (SpreadsheetBench, LiveMath, DABench, LongSeal) with four executors, claiming a 16.1-percentage-point average success improvement over Skill + ReAct, success in 15 of 16 settings, and token reductions of 38.4--88.9% for Qwen3.8-27B. Machines are compiled once from qwen3.6-flash traces and transferred unchanged to the other backbones. A prompt-only ablation (Appendix G) and a compiler-sensitivity experiment (Appendix D) support the claim that programmatic control-flow enforcement, rather than improved text alone, drives the gains.","tokens_in":26498,"tokens_out":4251,"duration_ms":31111,"significance":"If the central empirical claim holds, the contribution is real and useful: HEXIS provides a mechanism for turning free-text skills into executable structure that is reusable across backbones without target-model recompilation, with evidence of reduced inference cost on some executors. The design has genuine strengths that I want to credit explicitly. The evaluation uses held-out test tasks with machines compiled only on development traces, so the reported numbers are not a fitted reduction on the test set. Cross-model transfer (11 of 12 target-model settings) is independent grounding for the representation's generality, because the machines were never adapted to the target executors. The prompt-only ablation in Appendix G directly isolates programmatic enforcement from content changes, and the compiler-sensitivity check in Appendix D shows the gains are not an artifact of a specific construction model. The replay-based validation of archived traces is a concrete, machine-checkable property rather than a purely statistical claim.","major_comments":[{"comment":"Every number in Table 1 is a single run with no confidence intervals, standard errors, or significance tests, yet the central claim (b9, b166) is quantitative. With binary success outcomes and test sizes of n=51 (DABench, LongSeal), n=57 (SpreadsheetBench), and n=121 (LiveMath), the standard error per cell is roughly 5-7 pp. Several reported HEXIS-vs-Skill+ReAct margins are within 1-3 task flips of zero: DABench margins of 4.0, 4.0, 3.9, and 1.9 pp correspond to 1-2 extra successes on n=51, and the LongSeal margin of 5.9 pp on Qwen3.8-27B is 3 of 51 tasks, with an exact tie on Qwen3.5-9B. The 16.1-pp average is dominated by LiveMath (~34 pp) and SpreadsheetBench (~19 pp), with DABench contributing only ~3.5 pp. Baseline instability (AFlow on DABench: 80.4% with qwen3.6-flash vs 15.7% with Qwen3.8-27B; ReasoningBank on LiveMath: 8.3-41.3% across executors) shows the noise floor. I request per-cell binomial confidence intervals or significance tests and, where feasible, multiple seeds; the '15 of 16 settings' and 16.1-pp claims should be re-stated with uncertainty.","section":"Table 1, §5.3","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The relationship between Table 1 'success' and the Eq. (10) metric is not clear, and Appendix F concedes a crack in what the machine actually certifies. The metric section states that checks cover 'file delivery, output format, and tool-use requirements independently of answer correctness,' while Table 1 reports 'Success rates.' Appendix F states that the terminal category 'verified' only certifies that the delivered file agrees with the machine's selected answer letter, 'not that the mathematical answer is correct.' For LiveMath in particular, the headline 31.4-38.0 pp gains may thus measure procedural compliance (producing a file containing the model's selected letter) rather than selecting the correct answer. The paper should state, per benchmark, what Table 1 measures and how it relates to answer correctness, and should temper the interpretation of the gains if they are compliance gains rather than correctness gains.","section":"Eq. (10), §5.2; Appendix F","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The convergence analysis depends on assumptions that presuppose the conclusion. The assumption that 'updates preserve semantic distinctions already represented' via a deterministic mapping g_n with R_n = g_n(R_{n+1}) (b435, b436), and the assumption that each surviving incorrect interpretation is eliminated with probability at least gamma > 0 each round (b444), are exactly the properties the appendix aims to derive. The authors honestly label these as assumptions (b440: 'Semantic preservation is an assumption of this analysis'), and the implementation only guarantees replayability of accepted event paths (b409), not inter-round semantic preservation or a fixed elimination probability. As a result, Eq. (C.4)'s claim that expected representation loss tends to zero is conditional on unverified idealization. I recommend reframing Appendix C explicitly as a formalization of ideal conditions under which accumulation would converge, with the assumptions stated as open conditions for the algorithm to satisfy, rather than presenting convergence as an established property of HEXIS.","section":"Appendix C, §C.2-C.3","authors_acknowledge":"Semantic preservation is an assumption of this analysis; the implemented checks and replay directly maintain the replayability of accepted event paths.","authors_acknowledge_p":0.77},{"comment":"The paper's value proposition includes cost, but the compilation and development cost is never reported. The machines are compiled by Claude Fable 5.1 (a frontier commercial model) over all qwen3.6-flash development trajectories, including 487 LiveMath tasks across 49 groups (Appendix H), with a retry limit on candidates and acceptance only after replay of the full archive. The reported token figures (Fig. 5) are execution tokens only, and the paper's efficiency claims (e.g., 38.4-88.9% reduction for Qwen3.8-27B; Table 2's 69k tokens) omit the compile-time inference cost, which can be substantial given the trace volume and the replay validation. Since 'lower inference cost' is part of the stated motivation, I request a reported compile cost (model calls, tokens, or wall-clock per skill) and a statement of whether the cost amortizes over the reusability claim.","section":"§5.2, Appendix H, §5.3; Fig. 5","authors_acknowledge":null,"authors_acknowledge_p":null}],"minor_comments":[{"comment":"The DABench split is described as 'stratified by difficulty' and LiveMath as 'stratified by month,' but the stratification procedure and the number of strata are not specified; a sentence describing how stratification was done would aid reproducibility.","section":"§5.1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The baseline list spells the method 'A WM' (agent workflow memory) while Table 1 and Figure 4 use 'AWM'; the spelling should be made consistent.","section":"§5.2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"The representational objective in Eq. (5) is presented as a minimization over M, but no optimization over M is actually performed; the compilation procedure is heuristic (draft, align, replay, commit). The paper notes this in passing, but a sentence stating that Eq. (5) is motivational and not optimized explicitly would prevent a misreading.","section":"Eq. (5), §3.2","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Table 1 does not report the number of runs per cell, and the paper nowhere states that all experiments were single-run; a note in the metric section confirming single-run reporting and the absence of variance would make the statistics transparent.","section":"§5.2; Table 1","authors_acknowledge":null,"authors_acknowledge_p":null},{"comment":"Figure 4 and Figure 5 each use different axis scales across panels, and the token figures are reported in thousands (k); the captions should state the units explicitly in the text, not only in the axis labels.","section":"Fig. 4, Fig. 5","authors_acknowledge":null,"authors_acknowledge_p":null}],"recommendation":"major_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this is a genuinely useful engineering paper, and the central mechanism claim holds up better than the headline-number weakness suggests. The trace-guided incremental compiler with regression-style acceptance — an update commits only if static checks pass and the new trace plus every previously accepted trace replays — is a real contribution I don't see in the cited prior work. StateFlow uses predefined task-domain machines, TraceCompiler derives programs without skill-text initialization, Formal Skill needs hand-written programs. The evaluation is also more honestly grounded than much of this subfield: machines are compiled on qwen3.6-flash development traces and transferred unchanged to three other executors, which gives the cross-backbone claim independent grounding. And the Prompt-only ablation in Appendix G is the right kind of control — identical machine content rendered as a document, no programmatic enforcement — so the gains on GLM and Qwen3.8-27B with 28–94% fewer tokens aren't a compute-budget artifact.\n\nThe paper also states its limits plainly. b197 concedes that static checks and trace replay validate recorded paths but not in-state reasoning; Appendix F says \"verified\" only means the file read-back matches the selected letter, not that the math is right. Acknowledged limits, not hidden flaws.\n\nNow the soft spots, in proportion. The stress-test concern lands: the 16.1-point average is stated to one decimal with no variance or significance anywhere, on test sets of 51–121 tasks. Binary outcomes on n≈51 give standard errors of 5–7 points. DABench margins are 1.9–4.0 points — one or two flipped tasks each — and LongSeal has an exact tie on Qwen3.5-9B. The baselines' own swings (AFlow on DABench goes from 80.4% to 15.7% just by swapping executors) show a noise floor wider than several claimed margins. The headline magnitude isn't supported at the stated precision; the direction is.\n\nTwo more omissions worth naming: StateFlow and Formal Skill — the closest related methods — are cited but never run, and compilation cost isn't amortized into the token-savings claims. Both are addressable.\n\nThe Appendix C convergence analysis I'd treat as decoration. The assumptions (semantic preservation, every distinguishing trace eliminated with fixed probability γ, traces always consistent with the original skill) guarantee the result, and b440 concedes the implementation only maintains replayability. Honest, but not evidence for the empirical claim.\n\nBottom line: solid, well-written engineering work with a real mechanism and an honestly held-out evaluation, weak on statistical reporting and with one theory appendix that overreaches. Send it to review with conditions: error bars, the two missing baselines if feasible, amortized compile cost, and a rewrite of Appendix C's claims to match what replay actually guarantees.","headline":"A solid engineering contribution with a genuinely new regression-style skill compiler, held back mainly by the absence of any uncertainty reporting on its headline numbers.","tokens_in":27148,"tokens_out":2122,"duration_ms":15677},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"HEXIS compiles agent skills into extended finite state machines that separate skill knowledge from control flow, and reports that this raises task success over native skill execution by 16.1 percentage points on average across four…","keywords":["agent skills","extended finite state machine","skill compilation","LLM agents","procedural compliance","trace replay","control flow","workflow learning"],"falsifier":"Take a held-out task outside the development-traced branches and run the compiled machine with a judge model known to mislabel a specific check; if the wrong label routes execution to a branch ending in 'verified' with an incorrect outcome, while a native run of the same skill surfaces the failure, the claim that control is truly separated from reasoning quality holds only when in-state judgment is sound. Concretely, instrument the LiveMath machine from Appendix F so the consistency judge returns 'pass' for a file that contradicts the selected letter, and check whether the runtime still reaches END_VERIFIED and reports a wrong answer — that isolates the region where the compiler's guarantee stops.","tokens_in":29382,"feed_emoji":"🤖","tokens_out":2045,"duration_ms":48959,"temperature":0.7,"pith_summary":"The paper argues that agent skills fail less from missing knowledge than from coupling reasoning with control decisions: the model must repeatedly infer the next step from the skill document, so prescribed operations get skipped or misapplied. HEXIS compiles a skill document into an extended finite state machine that splits these apart — skill knowledge stays in per-state prompts, while explicit transitions over typed variables decide what runs next. A trace-guided compiler builds and refines the machine, accepting an update only when it passes static checks and replays every previously accepted trace. Across four benchmarks and four executors the method beats native Skill + ReAct by 16.1 percentage points on average, improves 15 of 16 settings, and cuts execution tokens by 38.4–88.9% on a mid-size local model. If correct, this gives a reusable, cross-backbone way to enforce prescribed skill steps at lower inference cost.","feed_headline":"Compiling skills into state machines lifts agent success by 16 points","feed_subtitle":"HEXIS turns skill text into explicit transitions, cutting a local model's tokens by up to 89 percent.","key_machinery":"The load-bearing object is the extended finite state machine $M = (Q,q_0,V,a,E,F,\\tau)$ : states carry local prompts holding skill knowledge, operations with declared read and write variable sets, and tool argument templates; ordered outgoing edges carry guard conditions evaluated on variable values, with unconditional edges last; counters bound loop traversals; terminal states return an outcome category (verified, unverified, fallback). The selection rule — pick the first enabled edge $j^* = \\min\\{j : g_j(\\nu^+) = \\text{true}\\}$ — is what removes next-step control from model inference. The compiler's acceptance condition — static checks of structure, variable dependencies, terminal evidence, and declared constraints, plus replay of the current and every archived trace — is what lets the machine grow without breaking previously accepted behaviour. Judge states are the bridge where model judgment (a label) meets machine routing (the guard), and the paper concedes that bridge is where reasoning quality the compiler cannot check still decides the outcome.","core_discovery":"The central claim is that prescribed skill steps are followed more reliably when the skill's control requirements are turned into explicit guarded transitions of a state machine, leaving the model only the in-state reasoning it is good at. The machine records the current stage, typed variables hold task inputs and intermediate results, and after each operation the runtime picks the first enabled edge to route execution, so choosing the next step no longer requires another model decision. Judge states emit a fixed label that guards use to branch to revision or verification. A compiler first maps skill clauses and tool interfaces to states, local prompts, data bindings, and transitions, then aligns development traces to add missing operations and dependencies, committing an update only when static checks and replay of the current and all archived traces pass. The claim is that this division — program-enforced control, model-enforced content — reduces the per-step deviation events the paper models as a product over a chain rule, and that the resulting machines transfer unchanged to other backbones without target-model recompilation.","pith_inferences":["The weakest link is the judge label: if an in-state judge mislabels a failed check, the guard dutifully routes execution down the wrong branch and the machine's 'verified' terminal still reports success — compilation shifts failure from step-selection into verdict quality, so a test that isolates mislabel rates would separate those two sources of error.","Because replay uses recorded tool outputs and placeholder model results, it certifies recorded paths, not behaviour on variations the traces never exercised; the natural stress test is sweeping novel branches and adversarial task inputs, where compiled and native execution should diverge.","Since 'verified' only certifies that the delivered file agrees with the model's own selected answer letter rather than that the answer is correct, measuring congruence with ground truth would reveal how much of the compliance gain is procedural versus substantive.","Cross-backbone transfer suggests the compiled machine encodes model-agnostic structure; if so, the same machines might compress prompting further or feed cheap verifiers, though the paper does not test these extensions."],"forward_implications":["Machines compiled once from one executor's traces transfer to other backbones unchanged and still improve success in 11 of 12 target-model settings, so compilation cost need not be paid per model.","On Qwen3.8-27B execution tokens fall 38.4–88.9% across benchmarks, indicating that enforcing control in transitions can cut the context overhead of carrying the whole skill and interaction history.","Combining HEXIS with skill-text optimization reaches 84.2% success on SpreadsheetBench at 69k tokens per task, suggesting content quality and control structure are complementary levers.","The deviation model, which frames cumulative compliance as a product over per-step deviation probabilities, predicts that removing repeated next-step inference compounds, matching the pattern of gains on LiveMath.","The information-theoretic framing with mutual-information accumulation bounds says the machine's representation converges to the skill's semantics given enough distinguishing traces, under assumptions the authors state explicitly."],"supporting_citations":[{"why":"Defines the ReAct baseline that HEXIS replaces and the native-execution frame it is measured against.","marker":"(Yao et al., 2023)"},{"why":"StateFlow organizes execution with predefined state machines; HEXIS extends this from hand-defined to automatically compiled skills.","marker":"(Wu et al., 2024)"},{"why":"Formal Skill tracks progress and restricts actions with a program, the closest prior control-enforcement approach.","marker":"(Zhang et al., 2026)"},{"why":"SkillOpt revision of the skill text is the optimization combined with HEXIS in the Table 2 result.","marker":"(Yang et al., 2026)"},{"why":"SkillsBench provides the evidence that curated skills raise pass rates, motivating the compilation problem.","marker":"(Li et al., 2026)"},{"why":"SOPBench documents poor procedural compliance, the failure mode the paper targets.","marker":"(Li et al., 2025)"},{"why":"SpreadsheetBench is the spreadsheet benchmark with the 50/57 development/test split used for one evaluation.","marker":"(Ma et al., 2024)"},{"why":"LiveMathematicianBench is the benchmark with the largest reported gains (31.4–38.0 points) and the worked machine example in Appendix F.","marker":"(He et al., 2026)"},{"why":"Supplies the chain rule for mutual information used in the representation-accumulation analysis.","marker":"(Polyanskiy & Wu, 2016)"}],"fun_headline_variants":["HEXIS compiles skills to state machines, +16.1 points","State machines enforce skill steps, lifting success by 16 points","Explicit transitions cut token use up to 89% for local LLM","HEXIS: control flow in state machines, reasoning stays local"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The load-bearing premise is that the compiled machine's transition guards and judge-label routing faithfully encode the skill's required control flow, and that reaching the 'verified' terminal means acceptable task completion; the paper itself concedes that static checks and trace replay validate recorded paths, not in-state reasoning or unseen situations, and that 'verified' means the delivered file agrees with the model's chosen answer letter, not that the mathematical answer is correct.","fun_headline_variants_meta":{"raw":{"variants":["HEXIS compiles skills to state machines, +16.1 points","State machines enforce skill steps, lifting success by 16 points","Explicit transitions cut token use up to 89% for local LLM","HEXIS: control flow in state machines, reasoning stays local"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000854,"raw_usage":{"cost":0.000853888,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000853888,"upstream_inference_prompt_cost":0.000102368,"upstream_inference_completions_cost":0.00075152},"total_tokens":3620,"prompt_tokens":936,"completion_tokens":2684,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":2605}},"tokens_in":936,"tokens_out":2684,"cost_source":"openrouter_usage","duration_ms":16839,"image_pages":[],"temperature":1.0,"billed_attempts":1,"reasoning_tokens":2605,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-09-25T03:20:44.652834+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-daily-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Take a held-out task outside the development-traced branches and run the compiled machine with a judge model known to mislabel a specific check; if the wrong label routes execution to a branch ending in 'verified' with an incorrect outcome, while a native run of the same skill surfaces the failure, the claim that control is truly separated from reasoning quality holds only when in-state judgment is sound. Concretely, instrument the LiveMath machine from Appendix F so the consistency judge returns 'pass' for a file that contradicts the selected letter, and check whether the runtime still reaches END_VERIFIED and reports a wrong answer — that isolates the region where the compiler's guarantee stops.","supporting_citations":[{"cited_title":"Gopinath, Karthik Narasimhan, and Shunyu Yao","cited_arxiv_id":null,"evidence_quote":"Defines the ReAct baseline that HEXIS replaces and the native-execution frame it is measured against."},{"cited_title":"2016 , howpublished =","cited_arxiv_id":null,"evidence_quote":"Supplies the chain rule for mutual information used in the representation-accumulation analysis."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}