{"id":"e40dc6ef-20b1-4614-a5a4-792b3be5d493","arxiv_id":"2607.23263","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A compact multi-agent judge with a shared 9B backbone matches or beats rule-based reward signals in online RL for computer-use agents, per the authors' held-out success-rate measurements.","lead":"This paper presents SeekJudge, a four-agent model that judges whether a computer-use agent completed an instruction by splitting 'find the right screen' from 'read the screen closely', and it reports that the resulting 9B reward model can replace hand-written rule checkers during reinforcement learning. It also releases a new human-labeled benchmark with step-level judgments (CUAStepBench) and a lower-cost reward server for RL.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2's 'match or surpass' rests on average-of-training-checkpoint test success; final-checkpoint reporting could reverse it.","rationale":"The paper's central claim is entirely empirical, and Table 2 is the only direct evidence. Appendix D specifies a nonstandard reporting protocol. This is not a stylistic issue: because the training reward for the rule arm is the same verifier used for test evaluation, that arm is expected to have a different training-time trajectory than a model-reward arm. The paper itself asserts that the rule arm's high training reward reflects overfitting (Section 4.3), so test curves are likely non-monotonic. Under such conditions, the mean over checkpoints is not a consistent estimator of final performance and can favor whichever arm peaks early. The ablation in Table 5 shows that reward granularity has large effects, making trajectory-shape differences between arms plausible. The concern is addressable by releasing learning curves or at least final/best checkpoint numbers. I agree with the reader's identification of this as the weakest assumption; because it is empirically checkable and the verdict is already conditional, no verdict change is needed.","tokens_in":17189,"tokens_out":5991,"duration_ms":60733,"concrete_test":"Recompute Table 2 for every completed arm using only the step-75 test evaluation (three passes averaged), and additionally report the best-checkpoint mean and the full test-success learning curve over GRPO steps. If the step-75 or best-checkpoint SeekJudge margin over Rule-based is not positive across completed domains—or reverses on UI-TARS—the 'match or surpass' claim should be scaled back to 'during training-time averaging.' Report run-to-run 95% CIs (reported σ ≈ 2.0%) alongside these numbers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SeekJudge's model-based reward 'matches or surpasses' native rule-based supervision as measured by downstream test success (Abstract; Table 2). The reported test success is not a final or best checkpoint but the mean over all test evaluations taken during training (steps 15/30/45/60/75, three passes each; Section 4.2 and Appendix D). This protocol is load-bearing because the paper itself argues that rule-based training reward overfits (Section 4.3), implying that test-success curves are plausibly non-monotonic. If SeekJudge's test success peaks early and then decays while the rule-based policy keeps improving, the mean can favor SeekJudge even though no deployed final checkpoint is better. Conversely, if the rule-based curve peaks early, the mean could obscure a genuine SeekJudge advantage. Table 2 also omits learning curves, so the reader cannot tell which regime holds. The claim is specifically about downstream success rate, which convention and deployment attach to a final model; averaging over training-time checkpoints is nonstandard and could reverse the headline under standard reporting. The missing Qwen-OS run and the Qwen-Impress deficit (48.41 vs 49.28) make this sensitivity material.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SeekJudge, a multi-agent reward framework for computer-use agents that decomposes trajectory judging into localization (Condense/Ground/Seek agents) and extraction (Analyze agent), all sharing a single 9B open-source backbone. It introduces CUAStepBench, a human-annotated benchmark with dense step-level labels across 177 applications, and reports offline F1 scores on three reward benchmarks and online RL results (GRPO) on Chrome, Impress, and OS domains under two actor backbones. The central claim is that SeekJudge is the first practical model-based reward to match or surpass native rule-based supervision in online RL, measured by downstream test success on held-out goals, while being cheaper and supporting longer trajectories.","tokens_in":17526,"tokens_out":3819,"duration_ms":36082,"significance":"If the central claim survives scrutiny, this is a meaningful step toward replacing brittle rule verifiers with practical model-based judges in CUA reinforcement learning. The paper ships an open-source 9B model, a new human-annotated benchmark with dense step labels, a cost/latency analysis, and a reward-server architecture. The decoupled localization/extraction design and the step-level output are plausible and well-motivated. However, the headline comparison is sensitive to the reporting protocol (mean over training checkpoints) and to incomplete/negative cells in the main table. The offline F1 evaluation also involves calibration on the evaluation benchmarks and a benchmark built with the authors' own judge, which limits the strength of the offline claims. The online RL comparison against external rule verifiers is not intrinsically circular, which is a credit to the design, but the load-bearing RL claim needs more standard and complete evidence.","major_comments":[{"comment":"The reported test success is the mean over all test evaluations during training (steps 15/30/45/60/75, three passes each), not the final or best checkpoint. This is nonstandard and load-bearing: Section 4.3 argues rule-based training reward overfits, implying test curves are plausibly non-monotonic. Averaging over checkpoints can invert a final-checkpoint comparison in either direction. Since the claim is about downstream success rate, which convention attaches to a deployed final model, please report final and best checkpoint numbers with learning curves, plus run-to-run variance for every Table 2 cell. Without this, the headline 'match or surpass' claim is not established.","section":"Section 4.2, Appendix D, Table 2"},{"comment":"The abstract claims SeekJudge 'matches or surpasses' native rule-based supervision, but the completed cells show one deficit and one missing run: Qwen3VL-8B on Impress (48.41 vs 49.28) and the Qwen-OS cell is pending. Of the five completed cells, four surpass the rule baseline and one does not. The claim should be qualified to the completed settings, or the missing cell should be filled and the deficit explained. This is not a cosmetic issue because the paper's own Section 4.3 uses this table to support the broad claim.","section":"Table 2"},{"comment":"The offline F1 results are reported using a score regression fitted jointly on all three evaluation benchmarks (Table 7, row 1). The leakage probe in Table 7 shows the diagonal advantage is small, but it does not eliminate the fact that the calibration has seen the test distributions; the probe is a sensitivity check, not a leakage-control method. In addition, CUAStepBench was constructed by filtering trajectories with the authors' own judge's uncertainty (Section 4.1), so offline F1 on it is not a fully independent test of judge quality. These issues do not affect the online RL comparison against rule verifiers, but they weaken the offline claims and the statement about 'first to clear the Rule baseline.' Please report an independent calibration split or reframe the offline results as indicative rather than definitive.","section":"Section 3.2, Appendix B, Table 7"},{"comment":"The paper states that 'the whole paper uses only two parameters' for calibration, but the reward pipeline also depends on the per-step nine-class rubric constants, the w_step weight, and the seed-calibrated criteria and seek prompts. These are tunable and no sensitivity analysis is provided for w_step or the rubric constants. Since the RL result depends on the scalar reward formed by the outcome score plus w_step times the per-step scores, a sensitivity analysis of these choices is needed to show that the headline comparison is not the result of a particular calibration.","section":"Section 4.2, Appendix D"}],"minor_comments":[{"comment":"The caption says '(dashed = test set)' and implies learning curves, but no learning curves are shown in the paper. Either add the curves or revise the caption to match Table 2.","section":"Figure 1(a) caption"},{"comment":"For baselines without native step-level judgments, step labels are obtained via SeekJudge's step extraction procedure. This may bias the step-level comparison; please clarify whether the step prompt was applied identically and whether any baseline-specific adaptation was made.","section":"Table 3"},{"comment":"The pending Qwen-OS cell should be flagged in the abstract or conclusions as well as in the table caption, since readers may overlook the incompleteness.","section":"Section 4.2 / Table 2"},{"comment":"The cost model charges cache hits at zero cost for repeated prefixes. While this matches the deployment logs, it is a potentially favorable assumption for methods with long reused contexts; a sensitivity table under a no-cache or partial-cache assumption would strengthen the cost comparison.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is well-engineered and the framework itself is plausible, but the headline claim currently rests on a nonstandard evaluation protocol and an incomplete main table. The offline calibration and benchmark-construction choices also need more careful framing. These issues are fixable within the manuscript's scope, so I do not recommend rejection, but the archival version should not present the 'match or surpass' claim without the requested reporting changes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one if you work on computer-use agents or model-based rewards. The framework is real engineering: splitting judging into localization and extraction, the Seek-Analyze loop, a single 9B backbone for all four roles, and a small per-call context that scales to long trajectories. CUAStepBench is a genuine contribution: dense human step-level labels plus trajectory verdicts on 177 applications, including iOS coverage that other reward benchmarks lack. The multi-image dilution experiment (decisive image always present, trajectory vs noise padding) is clean and supports their design story. The ablations show Condense and Ground earn their keep, and the leakage-control appendix is a good-faith attempt to address the obvious circularity worries with the calibration.\n\nNow the soft spots, and they matter. Table 2 reports test success as the mean over all test evaluations taken during training (every 15 steps, three passes each), not the final or best checkpoint. The paper itself argues that rule-based training reward overfits, so test curves are plausibly non-monotonic; under that assumption, averaging over checkpoints can favor whichever method peaks early. The authors are explicit about the protocol, so it is not hidden, but the headline claim is specifically about downstream success rate, and convention attaches that to a final model. Without learning curves in the paper, the reader cannot tell which regime holds. One main cell is still pending (Qwen-OS), and one completed cell is below the rule baseline (Impress 48.41 vs 49.28). Those two facts make the sensitivity material, and the stress-test note is right that final-checkpoint reporting could reverse the \"match or surpass\" claim. Offline F1 also has no error bars, and step-level F1 of 38.1, while better than baselines, is low in absolute terms; that tempers the \"step-level reward\" selling point.\n\nStill, the core architecture and the benchmark are solid, and the offline comparisons on matched backbones support the framework's value independent of the RL headline. The paper deserves a serious referee. The fixes are standard: report final or best-checkpoint success, show learning curves, fill in the pending cell, and give run-to-run error bars on the offline numbers. If those hold, the claim is credible. If not, the contribution narrows to the framework plus benchmark, which is still more than most papers deliver.\n\nSend it to review, with a request for those specific revisions.","headline":"A serious, well-engineered reward-framework paper whose headline RL claim depends on a nonstandard mean-over-checkpoints protocol; referee it, but push for final-checkpoint reporting.","tokens_in":716,"tokens_out":918,"would_cite":true,"duration_ms":28021,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims a small, learned reward model can replace hand-written rule checks when training computer-use agents, matching or exceeding rule-based supervision in online RL while costing far less.","keywords":["computer-use agents","reward model","reinforcement learning","GUI trajectory evaluation","model-based judging","step-level rewards","long-horizon tasks","distillation"],"falsifier":"Re-run the RL comparison under standard final-checkpoint reporting with longer episode caps and fill the missing backbone/environment cell; if SeekJudge's test success does not match or exceed the rule-based reward under that protocol, the central claim collapses.","tokens_in":17096,"feed_emoji":"🤖","tokens_out":4737,"duration_ms":46987,"temperature":0.7,"pith_summary":"The paper tries to establish that judging whether a GUI-agent trajectory fulfills its instruction can be done by a learned judge rather than by hand-coded rules, and that this learned judge works as an RL reward signal. It argues the key is to decompose judging into localization, finding the frame with the decisive evidence, and extraction, reading that frame accurately, instead of pouring all screenshots into one forward pass. With that decomposition, a single open 9B model matches or surpasses native rule-based supervision on downstream test success, at a fraction of the cost and with a context small enough for long trajectories.","feed_headline":"A 9B model judge matches hand-written rules in agent training","feed_subtitle":"Splitting judging into localization and extraction makes learned rewards practical and cheap.","key_machinery":"The core object is the Seek-Analyze loop over a condensed timeline. A Condense agent and a Ground agent compress the trajectory into compact text entries; a stateful Seek agent decides whether evidence is sufficient and, when it is not, names one step and one image to inspect; a stateless Analyze agent reads that single screenshot and returns the decisive detail as text. All four roles share one 9B backbone trained through a seed-calibrated distillation pipeline, and the dense outputs (seven trajectory-level dimension scores, nine step-level classes) are reduced by a fitted regressor to a scalar reward for group-relative policy optimization. A rollout-overlapped, zero-client-state reward ser","core_discovery":"SeekJudge's central claim is that model-based trajectory judging is now a practical drop-in for rule-based evaluation in computer-use RL: used as a reward, it matches or surpasses the environment's native rule verifier on held-out test goals, where previous model judges left a gap. The discovery behind it is that multi-image judging dilutes the decisive evidence, so the task should be split into locating the decisive image and then reading that image alone, one at a time. This decomposition enables a small 9B model to beat closed-source large-model judges and to offer step-level judgments, cheap inference, and a context footprint that scales to long trajectories.","pith_inferences":["If the localization/extraction decomposition is the true source of gain, the same two-stage design could transfer to other long-context vision-language judging tasks, such as document QA or video understanding, where a single decisive frame is buried in a long observation stream.","The paper's component ablation is run on the base model but not on the trained specialist, so attributing the full pipeline gain to the Seek-Analyze loop is an extrapolation; a direct ablation on the distilled model would test this.","A standard-protocol re-run, using final-checkpoint test success and longer episode caps, would be a stringent external check of the claim that model-based reward matches rule-based supervision.","The exclusion of goal-level dimensions from the scalar reward follows from the group-relative structure of the RL objective, suggesting that other multi-turn RL settings should similarly drop objective-fixed dimensions to avoid adding only offsets."],"forward_implications":["If SeekJudge is right, RL for computer-use agents no longer depends on version-frozen parsers and hard-coded answers; a learned reward can track human intention and survive app updates and drifting online content.","Step-level judgments supply a denser training signal than a single pass/fail bit, and the granularity ablation shows continuous plus per-step rewards improve test success over a Boolean reward.","The small per-call context lets judging scale to trajectories of hundreds of steps that would overflow a single-pass judge's context window.","The asynchronous reward server cuts the stall the reward model adds to RL rollouts, making model-based reward practical inside the training loop.","A single 9B open model makes judging orders of magnitude cheaper than closed-source large-model judges, lowering the cost barrier for RL experiments."],"fun_headline_variants":["Localize, then read: 9B judge bests rule rewards in agent RL","Split judging into find-and-read: 9B model matches rules","9B judge matches rule-based rewards via evidence localization","Cheap 9B judge beats big models for GUI agent training","Decompose judging: 9B model matches rule verifiers in RL"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline comparison depends on a nonstandard RL protocol — test success averaged over all evaluations during only 75 training steps with episodes capped at 15 training and 25 test steps, plus one backbone/environment cell left pending in Table 2 — so if standard final-checkpoint or longer-horizon reporting changes the ranking, the 'match or surpass' claim may not hold.","fun_headline_variants_meta":{"raw":{"variants":["Localize, then read: 9B judge bests rule rewards in agent RL","Split judging into find-and-read: 9B model matches rules","9B judge matches rule-based rewards via evidence localization","Cheap 9B judge beats big models for GUI agent training","Decompose judging: 9B model matches rule verifiers in RL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1416,"prompt_tokens":756,"completion_tokens":660,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":500,"completion_tokens_details":{"reasoning_tokens":566}},"tokens_in":500,"tokens_out":660,"duration_ms":6917,"temperature":1.0,"reasoning_tokens":566,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T23:54:03.479483+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the RL comparison under standard final-checkpoint reporting with longer episode caps and fill the missing backbone/environment cell; if SeekJudge's test success does not match or exceed the rule-based reward under that protocol, the central claim collapses.","supporting_citations":[],"review_version":1}