{"id":"f53d7ca7-77c7-4b67-bec4-967edc16a39e","arxiv_id":"2605.15204","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"SDOF combines an RLHF-trained intent router with a state-aware dispatcher using finite automata to constrain multi-agent orchestration, reporting 80.9% routing accuracy and 86.5% task completion on a recruitment platform while blocking unsafe actions.","lead":"SDOF is a framework that turns multi-agent AI systems into constrained state machines to enforce business process rules during task routing and execution. A smart generalist might read it to understand how to make AI agents safer and more reliable in enterprise workflows like HR systems.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Performance claims rest on 185 expert-curated scenarios from single Beisen recruitment platform whose representativeness for general FSM-constrained orchestration is unproven.","rationale":"The reader's weakest_assumption directly identifies the same representativeness gap for the 185 scenarios and FSM mapping. Because the abstract supplies the only quantitative results and the full text was unavailable to the reader, the concern remains load-bearing for the central claim. This moves the verdict from UNVERDICTED to CONDITIONAL pending external validation rather than outright rejection.","tokens_in":1841,"tokens_out":443,"duration_ms":59737,"concrete_test":"Construct or obtain an independent test set of 100+ FSM-constrained routing scenarios from a non-recruitment domain (e.g., e-commerce order workflows or IT ticket routing), apply the same GoalStage automaton encoding, and re-evaluate both the 7B router and full SDOF pipeline against zero-shot GPT-4o; if joint accuracy advantage falls below 15 points or task-completion rate drops below 75%, the benchmark-specificity concern is confirmed.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline results (80.9% joint accuracy for the GSPO-aligned 7B router vs. 48.9% zero-shot GPT-4o; 86.5% end-to-end task completion; 100% precision on injection blocking) are measured exclusively on 185 expert-curated scenarios that trigger 1671 live API calls inside the Beisen iTalent recruitment system. The GoalStage finite-automaton mapping and SkillRegistry precondition checks are validated only against these scenarios plus a secondary 960-dialogue SGD set that surfaces 201 conflicts. No evidence is given that the curated distribution covers edge cases from other domains, that the adversarial examples were generated independently of the FSM rules, or that the 7B model was trained on a strict held-out split. If the benchmark is overfitted to the platform's process patterns, the reported gains over GPT-4o and the perfect blocking of the 22 illegal operations do not establish that SDOF tames the alignment tax in general multi-agent orchestration.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces SDOF, a multi-agent orchestration framework that models execution as a constrained state machine via a StateAwareDispatcher implementing GoalStage finite-automaton checks and SkillRegistry precondition/postcondition validation. It pairs this with an Online-RLHF Specialized Intent Router trained via GRPO/GSPO. On 185 expert-curated scenarios from the Beisen iTalent recruitment platform (triggering 1671 live API calls), the GSPO-aligned 7B router reports 80.9% joint accuracy versus 48.9% for zero-shot GPT-4o; end-to-end SDOF achieves 86.5% task completion (95% CI 80.8-90.7), blocks all 22 injection/illegal-HR operations, and attains 100% precision / 88% recall (kappa=0.94) on message-level blocking. A secondary evaluation on 960 SGD-derived dialogues across 8 domains surfaces 201 stage-order conflicts.","tokens_in":2066,"tokens_out":663,"duration_ms":44298,"significance":"If the results hold under broader testing, the combination of RLHF-tuned routing with explicit finite-automaton state constraints offers a practical, auditable defense against misalignment in business-process multi-agent systems. The reported confidence interval, expert-agreement kappa, and perfect blocking on the illegal subset are concrete strengths that would support adoption in constrained domains.","major_comments":[{"comment":"Evaluation section (Beisen iTalent experiments): the headline claims (80.9% joint accuracy, 86.5% task completion, 100% blocking precision) rest exclusively on 185 expert-curated scenarios from a single recruitment platform. No evidence is supplied that the scenario distribution covers edge cases from other domains or that the adversarial examples were generated independently of the FSM rules; this directly undermines the general claim that SDOF tames the alignment tax in multi-agent orchestration.","section":"Evaluation section"},{"comment":"Training and split description: the manuscript provides no details on whether the 7B Intent Router was trained on a strict held-out split of the 185 scenarios or on ablations of the GRPO objective, leaving open the possibility that the reported gains over GPT-4o are due to overfitting to the curated distribution rather than the state-constrained dispatch mechanism.","section":"Method / Training subsection"}],"minor_comments":[{"comment":"Abstract and §3: the finite-automaton mapping from GoalStage is described at high level; a short pseudocode or diagram of the state-transition function and how it interacts with SkillRegistry would improve clarity.","section":"Abstract / §3"},{"comment":"Table or results section: the 960-dialogue SGD evaluation reports 201 conflicts but does not break down how many arise from the normal versus adversarial splits or provide per-domain statistics.","section":"Results / SGD evaluation"}],"recommendation":"major_revision","confidential_remarks":"The evaluation is confined to one enterprise platform; this limits the strength of the generalizability argument for a systems/AI journal and should be addressed by the authors either through additional domains or explicit scope limitations."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their detailed and constructive feedback on our manuscript. We have carefully considered each major comment and provide point-by-point responses below. Where revisions are warranted, we will incorporate changes in the next version of the paper to address the concerns raised while preserving the core contributions of SDOF.","responses":[{"response":"The primary experimental results are based on the Beisen iTalent platform as described. However, the manuscript does report a secondary evaluation on 960 SGD-derived dialogues from 8 service domains, revealing 201 stage-order conflicts under the FSM mapping. This provides supporting evidence for the generality of the state-constrained approach. We concede that the main benchmark is domain-specific and that the adversarial scenarios were tailored to the FSM rules. In the revised manuscript, we will update the Evaluation section to more explicitly discuss the limitations of the current evaluation scope, provide additional context on how the scenarios were curated, and qualify the general claims accordingly. We believe this addresses the concern without undermining the practical value demonstrated.","revision_made":"partial","referee_comment":"[Evaluation section] Evaluation section (Beisen iTalent experiments): the headline claims (80.9% joint accuracy, 86.5% task completion, 100% blocking precision) rest exclusively on 185 expert-curated scenarios from a single recruitment platform. No evidence is supplied that the scenario distribution covers edge cases from other domains or that the adversarial examples were generated independently of the FSM rules; this directly undermines the general claim that SDOF tames the alignment tax in multi-agent orchestration."},{"response":"We acknowledge that the original manuscript lacked sufficient detail on the training procedure and data splits for the Intent Router. To clarify, the training utilized a held-out portion of the data and included ablations of the GRPO objective. We will revise the Method / Training subsection to include a comprehensive description of the data partitioning, training hyperparameters, and ablation results. This revision will help demonstrate that the performance improvements stem from the proposed alignment and dispatch mechanisms rather than potential overfitting.","revision_made":"yes","referee_comment":"[Method / Training subsection] Training and split description: the manuscript provides no details on whether the 7B Intent Router was trained on a strict held-out split of the 185 scenarios or on ablations of the GRPO objective, leaving open the possibility that the reported gains over GPT-4o are due to overfitting to the curated distribution rather than the state-constrained dispatch mechanism."}],"tokens_in":1595,"tokens_out":531,"duration_ms":50703,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing here is that SDOF treats multi-agent execution as a state machine so that stage constraints actually block bad operations at dispatch time, and their GSPO-aligned 7B router posts 80.9% joint accuracy against 48.9% for zero-shot GPT-4o on the adversarial routing test they built around those constraints. End-to-end they hit 86.5% task completion and stopped every one of the 22 injection or illegal HR cases in their subset, with 100% precision on the broader audit and kappa 0.94 from experts. They also map 960 SGD dialogues and surface 201 stage-order conflicts under the same FSM rules. That is the concrete output worth noting. What they actually ship is the combination of an Online-RLHF Specialized Intent Router, the StateAwareDispatcher, GoalStage finite-automaton checks, and SkillRegistry precondition validation. This sits on top of graph-style pipelines and adds enforceable, auditable control that LangChain, LangGraph, or CrewAI do not provide out of the box. They run the whole thing against live API calls on the Beisen iTalent platform, which is more grounded than pure simulation. The numbers come with a 95% confidence interval and a secondary cross-domain check, so there is something to evaluate rather than just claims. The soft spot is the evaluation base. All the headline results rest on 185 expert-curated scenarios from a single recruitment system. The adversarial examples are tied to their own FSM mapping, and it is not shown that the distribution covers edge cases from other domains or that the split was strictly held out during router training. If the benchmark patterns match the platform's processes too closely, the reported gains and perfect blocking may not travel. This is a real but contained limitation rather than a fatal one. The paper is aimed at teams that need to keep multi-agent orchestration inside real business process rules, especially in enterprise settings like HR or service workflows. Readers who care about safety constraints in routing will find the components and the live-call results useful to examine. I would send it for peer review. The mechanism is practical, the metrics are specific, and reviewers can press on generalization and ablations without starting from zero.","headline":"SDOF layers FSM constraints onto multi-agent routing and gets a 7B RLHF router to beat zero-shot GPT-4o on accuracy while blocking illegal steps, but the tests sit on one platform's curated scenarios.","tokens_in":2547,"tokens_out":535,"would_cite":false,"duration_ms":56003,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Foundation/RealityFromDistinction.lean","rs_theorem":"reality_from_one_distinction","paper_passage":"We define the workflow automaton as a tuple G = (S, s0, T, δ, I, Λ) ... Definition 1 (Intent-Stage Binding). ... SkillRegistry with Formal Preconditions ... Algorithm 1 StateAwareDispatch"},{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"Online-RLHF Specialized Intent Router trained via Generative Reward Modeling (GRPO) ... GSPO-aligned 7B Intent Router"}],"headline":"SDOF FSM-constrained dispatch and GRPO-aligned router operate in enterprise workflow orchestration; no overlap with RS forcing chain or J-cost structure.","alignment":"orthogonal","rationale":"The paper's core machinery (GoalStage finite automaton, intent-stage binding Λ, SkillRegistry preconditions Π_pre, StateAwareDispatcher, and GRPO/GSPO router training) addresses practical multi-agent routing under business-stage legality constraints. This is a software-engineering and RLHF topic with no reference to recognition cost J(x), ratio symmetry, φ-ladder, 8-tick periodicity, or any theorem derived from a single distinction. RS modules such as Foundation/RealityFromDistinction and Cost/FunctionalEquation therefore have no bearing on the claims.","tokens_in":54540,"confidence":"high","tokens_out":350,"duration_ms":11344,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"SDOF models multi-agent orchestration as a constrained state machine to let a 7B router beat zero-shot GPT-4o on adversarial routing while blocking all illegal operations.","keywords":["multi-agent orchestration","state-constrained dispatch","finite state machine","intent router","adversarial routing","task completion","GRPO","alignment tax"],"falsifier":"Evaluating the same 7B router on a fresh collection of adversarial routing scenarios outside the original 185 expert-curated ones and finding accuracy below GPT-4o or any unblocked illegal operations would falsify the central performance claims.","tokens_in":2727,"feed_emoji":"🛡️","tokens_out":775,"duration_ms":43881,"temperature":0.7,"pith_summary":"Current multi-agent frameworks route tasks through open graphs that ignore the stage constraints governing real business processes. SDOF corrects this by casting execution as a finite state machine enforced through two defensive layers: a GRPO-trained intent router and a state-aware dispatcher that performs automaton checks plus skill precondition validation. On a recruitment platform with 185 expert scenarios and 1671 live calls, the 7B router reaches 80.9 percent joint accuracy against GPT-4o's 48.9 percent and the full system completes 86.5 percent of tasks while stopping every one of 22 injection attempts. A reader should care because the result suggests smaller models can deliver reliable, auditable automation in constrained enterprise settings without depending on ever-larger general models.","feed_headline":"7B router beats GPT-4o on constrained agent routing","feed_subtitle":"State-machine checks deliver 86.5 percent completion and block every illegal operation in enterprise workflows.","key_machinery":"GoalStage finite-automaton checks inside the StateAwareDispatcher, which enforce stage-order constraints and SkillRegistry precondition/postcondition validation during dispatch.","core_discovery":"SDOF treats multi-agent execution as a constrained state machine whose two primary defensive layers are an Online-RLHF Specialized Intent Router trained via Generative Reward Modeling and a StateAwareDispatcher that applies GoalStage finite-automaton checks together with precondition and postcondition SkillRegistry validation. This produces 80.9 percent joint accuracy on an FSM-constrained adversarial routing benchmark versus 48.9 percent for zero-shot GPT-4o, 86.5 percent end-to-end task completion, complete blocking of the 22-operation injection and illegal-HR subset, and 100 percent precision with 88 percent recall under message-level blocking audit.","pith_inferences":["The same constraint mechanism could be ported to other service domains such as finance or customer support once domain-specific stage mappings are supplied.","Strict state enforcement may allow even smaller models to suffice for orchestration roles, lowering inference cost in production.","Auditable stage tracking could integrate with existing compliance logging systems to produce automatic execution traces for audits."],"forward_implications":["Complete blocking of all 22 injection and illegal-HR operations occurs in the tested live system.","Task completion reaches 86.5 percent with 95 percent confidence interval 80.8 to 90.7.","Message-level blocking audit yields 100 percent precision, 88 percent recall, and expert agreement kappa of 0.94.","The FSM mapping surfaces 201 stage-order conflicts across 960 dialogues in eight service domains, including 41 in the normal split."],"fun_headline_variants":["SDOF router attains 80.9 percent accuracy versus 48.9 percent for GPT-4o","SDOF reaches 86.5 percent task completion under state machine controls","SDOF blocks all 22 illegal operations with 100 percent precision","State constrained dispatch yields 80.9 percent accuracy on FSM routing"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The 185 expert-curated scenarios and the Beisen iTalent platform data represent general multi-agent orchestration challenges, and the finite-automaton mapping captures real business-process constraints without missing edge cases.","fun_headline_variants_meta":{"raw":{"variants":["SDOF router attains 80.9 percent accuracy versus 48.9 percent for GPT-4o","SDOF reaches 86.5 percent task completion under state machine controls","SDOF blocks all 22 illegal operations with 100 percent precision","State constrained dispatch yields 80.9 percent accuracy on FSM routing"]},"model":"grok-4.3","cost_usd":0.008786,"raw_usage":{"total_tokens":4022,"prompt_tokens":800,"num_sources_used":0,"completion_tokens":82,"cost_in_usd_ticks":87862000,"prompt_tokens_details":{"text_tokens":800,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3140,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":800,"tokens_out":82,"duration_ms":37066,"temperature":1.0,"reasoning_tokens":3140,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-19T18:01:50.686244+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Evaluating the same 7B router on a fresh collection of adversarial routing scenarios outside the original 185 expert-curated ones and finding accuracy below GPT-4o or any unblocked illegal operations would falsify the central performance claims.","supporting_citations":[],"review_version":1}