{"id":"4fb1872a-1bbf-47e8-a853-c9ddc0962de6","arxiv_id":"2605.29801","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"AgentDoG 1.5 trains lightweight models on a taxonomy-guided dataset to achieve comparable safety performance to GPT-5.4 in agentic scenarios and deploys them as training-free guardrails.","lead":"The paper presents AgentDoG 1.5, small models (0.8B-8B) trained on about 1k samples via an updated agent safety taxonomy and influence-function data purification to match large closed models on safety tasks and serve as guardrails. A generalist reader might examine it for practical ways to reduce compute costs when aligning autonomous AI agents for real-world use.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"The reader's UNVERDICTED / LOW verdict already flags the absence of the full text and the resulting high uncertainty. No additional load-bearing concern can be formulated until the methodological and results sections are available for inspection.","tokens_in":1774,"tokens_out":218,"duration_ms":17514,"concrete_test":"Retrieve and read the full manuscript from the paper_source_context tool; if the experimental section contains explicit benchmark definitions, baseline scores, and ablation results on the purification step, re-evaluate the SOTA claim against those numbers.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The provided source material consists only of the abstract; the full manuscript text referenced in the tool description is not present in the query. The abstract states performance claims and describes the taxonomy update plus influence-function purification but supplies no equations, benchmark tables, ablation results, or evaluation protocols. Without access to those sections, no concrete internal inconsistency, hidden assumption, or unsupported derivation can be isolated in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes AgentDoG 1.5, a lightweight and scalable alignment framework for AI agent safety and security. It updates the agent safety taxonomy to address emergent risks from Codex and OpenClaw execution scenarios, introduces a taxonomy-guided data engine with influence-function purification to train 0.8B–8B parameter models on approximately 1k samples, claims performance comparable to GPT-5.4, constructs an efficient agentic safety SFT/RL environment that reduces Docker-level deployment overhead by two orders of magnitude, and deploys the model as a training-free online guardrail, reporting state-of-the-art results in diverse interactive agentic scenarios with open release of all models and datasets.","tokens_in":1810,"tokens_out":345,"duration_ms":21888,"significance":"If the empirical claims hold, the work would offer a practical route to data-efficient safety alignment for open-world agents using small open models, with substantial reductions in training and deployment cost and an open release that could serve as a community baseline. The combination of taxonomy update, purification step, and guardrail deployment addresses a timely gap between frontier agent capabilities and existing alignment methods.","major_comments":[{"comment":"Abstract: the central claims of SOTA performance and comparability to GPT-5.4 rest on empirical results, yet the provided text supplies no metrics, baselines, evaluation protocols, ablation studies, or error analysis, rendering the claims impossible to assess for support.","section":null}],"minor_comments":[],"recommendation":"uncertain","confidential_remarks":"The query supplies only the abstract; the referenced full manuscript text is absent, which prevents any section-specific technical review or verification of the reported experiments."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address the major comment point by point below.","responses":[{"response":"We agree that the abstract, as written, does not include specific metrics, baselines, protocols, ablations, or error analysis, which limits the ability to assess the central empirical claims from the abstract alone. The full manuscript contains these details in the Experiments and Evaluation sections. To address the concern directly, we will revise the abstract to incorporate key quantitative results (e.g., safety scores vs. GPT-5.4 and other baselines), a brief reference to the evaluation protocol, and mention of ablations. This revision will make the claims more transparent and assessable.","revision_made":"yes","referee_comment":"[—] Abstract: the central claims of SOTA performance and comparability to GPT-5.4 rest on empirical results, yet the provided text supplies no metrics, baselines, evaluation protocols, ablation studies, or error analysis, rendering the claims impossible to assess for support."}],"tokens_in":1345,"tokens_out":231,"duration_ms":16478,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The core pitch is that an updated agent safety taxonomy plus influence-function data purification lets them train tiny models (0.8B to 8B) on roughly 1k samples that perform like GPT-5.4, then drop the model in as a training-free guardrail while cutting Docker deployment costs by two orders of magnitude. They also release the models and data.\n\nWhat registers as new is the specific pipeline: extending the taxonomy to cover execution risks from agents like OpenClaw, applying influence functions to clean the safety data, and packaging the result as an online moderator rather than another fine-tuned policy. The emphasis on low-overhead deployment and open release is a practical step that could matter for people who actually ship agents.\n\nThe soft spots are straightforward. The abstract asserts SOTA results and parity with closed models but gives no numbers, baselines, evaluation protocols, or error bars. Without those, the claim that the purified data genuinely captures emergent risks without new blind spots stays uncheckable. There is also no comparison to earlier work on influence functions or agent alignment data cleaning, so it is unclear how much of the pipeline is incremental versus genuinely distinct.\n\nThis is the sort of paper that would interest engineers building lightweight safety layers for interactive agents who care about compute budgets and open artifacts. A reader looking for reproducible evidence on small-model guardrails would need the full experiments and tables before treating the results as reliable.\n\nI would send it to peer review. The topic is timely and the deployment angle is worth checking, even if the current version needs substantial additional evidence to hold up.","headline":"The abstract claims 0.8B-8B models match GPT-5.4 on agent safety with 1k samples via taxonomy update and influence-function cleaning, but supplies zero metrics or comparisons to support it.","tokens_in":2507,"tokens_out":414,"would_cite":false,"duration_ms":19909,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"AgentDoG 1.5 aligns small AI models for agent safety to match closed-source leaders using only about 1,000 samples.","keywords":["agent safety","AI alignment","lightweight models","influence function purification","safety taxonomy","online guardrail","agentic scenarios","small language models"],"falsifier":"Demonstrating that a new agentic risk scenario not addressed by the updated taxonomy causes AgentDoG 1.5 to underperform compared to baselines or that performance does not hold in untested real-world interactive environments.","tokens_in":2659,"feed_emoji":"🛡️","tokens_out":754,"duration_ms":24711,"temperature":0.7,"pith_summary":"The paper establishes a lightweight framework for aligning AI agents against new safety risks from powerful open-world agents like OpenClaw and advanced models that lower attack barriers. It updates the agent safety taxonomy to cover Codex and OpenClaw execution scenarios and uses a taxonomy-guided data engine with influence-function purification to create training data. This allows training variants from 0.8B to 8B parameters that perform comparably to GPT-5.4 in complex interactive scenarios. A sympathetic reader would care because current alignment methods are inadequate for real-world deployment, and this approach reduces the resources needed for safety while enabling efficient training and real-time moderation.","feed_headline":"Small models match GPT-5.4 on agent safety with 1k samples","feed_subtitle":"Updated taxonomy and data purification let 0.8B-8B models achieve SOTA while cutting training costs and deployment overhead by 100x.","key_machinery":"The taxonomy-guided data engine with influence-function purification that generates and refines the training data for agent safety alignment.","core_discovery":"AgentDoG 1.5 updates the agent safety taxonomy for emergent risks from Codex and OpenClaw, builds a taxonomy-guided data engine with influence-function purification to train 0.8B-8B models with around 1k samples achieving performance comparable to GPT-5.4, constructs a highly efficient agentic safety SFT and RL training environment reducing Docker-level deployment overhead by two orders of magnitude, and deploys as a training-free online guardrail, with extensive results showing state-of-the-art performance in diverse and complex interactive agentic scenarios.","pith_inferences":["Similar purification techniques could be adapted for safety alignment in other AI domains such as multimodal or embodied agents.","The framework's efficiency might encourage wider adoption of safety measures in resource-constrained environments.","If the taxonomy covers future risks, it could serve as a foundation for evolving agent safety standards.","Open release allows independent verification and extension by the community."],"forward_implications":["Small models from 0.8B to 8B parameters can match leading closed-source models like GPT-5.4 in agent safety tasks using only around 1k samples.","The efficient SFT and RL training environment reduces deployment overhead in Docker-level environments by two orders of magnitude.","Deployment as a training-free online guardrail enables real-time safety moderation without additional training.","All models and datasets are openly released to support further development.","AgentDoG 1.5 achieves state-of-the-art performance in diverse and complex interactive agentic scenarios."],"fun_headline_variants":["0.8B-8B models match GPT-5.4 on agent safety with 1k samples","AgentDoG 1.5 uses 1k samples to align models with GPT-5.4 safety levels","Taxonomy-guided data engine trains AgentDoG 1.5 with 1k samples","Efficient SFT RL environment reduces Docker overhead 100x in safety tasks"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The updated agent safety taxonomy and influence-function purification produce training data that genuinely captures and mitigates emergent risks without introducing new blind spots or overfitting to the process.","fun_headline_variants_meta":{"raw":{"variants":["0.8B-8B models match GPT-5.4 on agent safety with 1k samples","AgentDoG 1.5 uses 1k samples to align models with GPT-5.4 safety levels","Taxonomy-guided data engine trains AgentDoG 1.5 with 1k samples","Efficient SFT RL environment reduces Docker overhead 100x in safety tasks"]},"model":"grok-4.3","cost_usd":0.01101,"raw_usage":{"total_tokens":4877,"prompt_tokens":731,"num_sources_used":0,"completion_tokens":98,"cost_in_usd_ticks":110099500,"prompt_tokens_details":{"text_tokens":731,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":4048,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":731,"tokens_out":98,"duration_ms":29202,"temperature":1.0,"reasoning_tokens":4048,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T07:16:54.756344+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Demonstrating that a new agentic risk scenario not addressed by the updated taxonomy causes AgentDoG 1.5 to underperform compared to baselines or that performance does not hold in untested real-world interactive environments.","supporting_citations":[],"review_version":1}