{"id":"b84e2fd6-5ee1-4067-8850-c01ec09ed224","arxiv_id":"2607.11106","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"Self-regulated implicit visual tools let MLLMs match or beat external-tool and latent-reasoning methods on high-resolution perception while cutting inference latency by up to 86%.","lead":"BEE internalizes visual tools (zoom, crop, rotate) as discrete tokens inside an MLLM and trains a self-regulated policy that calls them only when internal knowledge is insufficient. The result is SOTA fine-grained perception with up to 86% lower latency than external-tool baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The p/τ capability-boundary proxy is the softest link in the self-regulation story, but the empirical Pareto claim does not critically depend on it being a perfect metacognitive signal.","rationale":"The reader correctly isolates the p/τ construction (Sec. 3.3.1 and the reward of Eq. 10) as the weakest technical assumption underlying the self-regulation narrative. That assumption is real: the paper supplies only a limited τ ablation (Table 17) and no multi-seed variance or coefficient sensitivity. Nevertheless the strongest claim is the empirical efficiency–accuracy Pareto improvement (Abstract, Fig. 1, Tables 1–2, 4). Those numbers are supported by multiple orthogonal ablations (tool-slot variants Table 5, NTG Table 7, token length Fig. 6, MC vs Static Fig. 9/Table 8) and by the public code. Even if the metacognitive story is only approximate, the measured gains over both agentic TwI baselines and latent baselines remain large. No evaluation leakage, broken math, or unfair baseline comparison was found that would overturn the numbers. Hence the ACCEPT verdict stands; a fuller multi-τ / multi-seed check would strengthen the paper but is not required to accept the central empirical claim.","tokens_in":41262,"tokens_out":707,"duration_ms":40866,"concrete_test":"Re-train Stage-2 of BEE-7B for the same number of steps while sweeping τ ∈ {1/3, 1/2, 2/3, 3/4, 5/6} (or replace p by a single held-out greedy accuracy estimate) with all other reward terms and coefficients frozen; recompute NTG, easy/hard tool-call rates on MME-Real-Lite, and the four perception averages of Table 2. If NTG falls below ~40 % or easy-tool rate remains >10 % for any τ near 2/3, the proxy is load-bearing and fragile; if all metrics stay within 5 % of the reported values, the concern is minor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of adaptive, low-latency reasoning rests on MC-Reward teaching the model to insert implicit tool slots only when internal knowledge is insufficient (Sec. 3.3). That signal is defined by comparing the online group-average accuracy p (G rollouts) against a fixed threshold τ = 2/3, then applying the hand-tuned continuous reward of Eq. (10). If p is a noisy or biased estimate of true capability—because early-policy rollouts are high-variance, because the same p both shapes the reward and moves with the policy, or because the cosine buffer and linear coefficients are over-fit—the learned policy may simply acquire a global tool-frequency prior rather than genuine knowledge routing. NTG rising from 6.2 % to 66 % and easy-problem tool rates falling to ~0.2 % (Fig. 8, Table 7) are encouraging post-hoc observations on the training distribution, yet they do not independently validate that p is a reliable online metacognitive proxy. A misspecified proxy would weaken the “self-regulated” interpretation even if the raw accuracy and latency numbers remain strong.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes BEE, a two-stage training paradigm that internalizes visual tool operations (zoom, crop, rotate, etc.) as discrete implicit tool-slot tokens rather than external API calls with image re-encoding. Stage 1 uses formalized CoT SFT on a 320K mixture of tool and no-tool trajectories to activate tool-slot generation and basic adaptive switching. Stage 2 applies DAPO with a composite reward that includes MC-Reward, which shapes tool propensity from the online group-average accuracy p relative to a fixed threshold τ (default 2/3), so tools are encouraged only when internal knowledge appears insufficient. The authors introduce Net Tool Gain (NTG) to diagnose redundant tool use after SFT+simple RL, then show that MC-Reward raises NTG from 6.2% to 66% while cutting easy-problem tool rates to ~0.2%. Empirically, BEE-4B/7B/8B improve fine-grained perception (V*, HRBench, MME-RealWorld) over strong open-source and several closed-source baselines, stay competitive on general/OOD reasoning, and report large inference-speed gains (up to ~86.2% vs DeepEyes at 8K) with lower deployment cost.","tokens_in":41594,"tokens_out":1043,"duration_ms":20092,"significance":"If the results hold under broader scrutiny, BEE is a practically important step for multimodal reasoning: it shows that much of the accuracy benefit of “Thinking with Images” can be retained without the I/O and re-encoding tax of explicit tool agents, while adding an explicit self-regulation objective rather than a fixed latent budget. The combination of formalized tool-slot CoT, the NTG diagnostic, system-level latency/throughput/cost tables, multi-backbone scaling (4B/7B/8B), and controlled ablations on slot format, token length, and MC-Reward is stronger engineering evidence than many concurrent latent-reasoning papers. The work is therefore significant for efficient MLLM deployment and for research on capability-aware tool routing, even if the metacognitive interpretation of p is only partially validated.","major_comments":[{"comment":"Section 3.3.1 and the definition of p: the central “self-regulated knowledge routing” claim rests on treating group-average accuracy over G rollouts as an online capability boundary, then gating tool advantage via fixed τ=2/3 and the hand-specified continuous reward in Eq. (10). The paper shows that easy-problem tool rates fall and NTG rises (Fig. 8, Table 7), but does not independently test whether p is a reliable metacognitive signal versus a noisy co-evolving statistic that mainly induces a global tool-frequency prior. A load-bearing addition would be (i) held-out calibration of p against true difficulty (e.g., frozen-base accuracy or human difficulty bins), (ii) an ablation that replaces p with a non-capability signal (random or length-based) while keeping the same reward shape, and/or (iii) reporting tool-use conditional accuracy stratified by p at evaluation time. Without this, the","section":null},{"comment":"Eqs. (9)–(10) and Appendix A.1: MC-Reward coefficients (α1,β1,α2,β2,γ,μ) and the cosine buffer are hand-chosen after a theoretical admissible-region argument. Table 17 varies τ but does not report a comparable sensitivity study for the reward coefficients or for G. Because Stage-2 policy updates are driven by this shaped advantage, the manuscript should either (a) show that nearby coefficient choices yield the same qualitative routing and NTG gains, or (b) describe a selection protocol (grid search on a validation split, automatic calibration) so the result is not tied to one opaque reward surface. This is especially important given the claim that continuous shaping is necessary to avoid collapse of tool use.","section":null},{"comment":"Tables 1–2 and the efficiency protocol (Sec. 4.1, Table 4, Fig. 5): the Pareto claim versus agentic TwI methods is central, yet several comparisons mix different codebases, tool sandboxes, and (for Thyme) a non-vLLM stack. The paper already standardizes hardware and BF16 batch-1 settings, which is good, but it should also report (i) matched maximum generation length / decoding hyperparameters for all agentic baselines, (ii) tool-call count and re-encode count per sample, and (iii) at least mean±std over multiple seeds or evaluation shuffles for the main BEE numbers. Without variance and fully matched agentic budgets, the ~86.2% acceleration figure and some accuracy margins could be partly protocol-dependent.","section":null}],"minor_comments":[],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean empirical methods paper that actually delivers on the efficiency–accuracy trade-off for high-resolution multimodal perception. The core move is simple and useful: instead of external tool calls and re-encoding (DeepEyes/Thyme style), they fold concrete visual operations (zoom, crop, rotate, etc.) into discrete semantic tool-slot tokens that the model generates autoregressively inside its CoT. Stage 1 SFT on mixed tool/no-tool formalized trajectories activates the slots and basic switching; Stage 2 uses the NTG diagnostic (net gain after subtracting base capability) plus a continuous MC-Reward that shapes tool use according to online group accuracy p versus a fixed τ=2/3. Result: BEE-4B/7B/8B hit SOTA or near-SOTA on V*, HRBench-4K/8K, MME-RealWorld while staying competitive on LogicVista/POPE, and the system tables show real throughput/latency/cost wins (1.61 samples/s vs 0.07 for DeepEyes, cost down to $0.72/1k). Ablations on slot format, token length (even K=4 works), τ, static vs MC, and perception/reasoning decoupling are thorough; code and construction details are public.\n\nWhat is new is the combination of discrete operation-specific tokens (not pure continuous latent like Monet/EVA) with an explicit capability-boundary reward. NTG rising from 6% to 66% and easy-problem tool rates falling to ~0.2% are concrete. The softest link is exactly the one the stress-test flags: p is a noisy online proxy and the reward coefficients are free parameters, so the “self-regulated metacognition” story is a bit stronger than the mechanism strictly warrants—it may just learn a useful frequency prior. That does not sink the Pareto claim; the accuracy and latency numbers stand even if the interpretation is slightly oversold. No multi-seed variance is a minor reporting gap, not a red flag. Citations cover the TwI and latent literature fairly; data is public benchmarks plus carefully filtered Thyme/Visual-CoT trajectories.\n\nThis is for anyone building low-latency high-res VLMs or agentic perception stacks. The math is standard DAPO + continuous shaping, the data pipeline is reproducible, and the gains are large enough that a serious editor should send it to referees. I’d bring it to reading group, cite the efficiency numbers and NTG idea, and engage with the work.","headline":"Practical Pareto win: discrete implicit tool tokens + NTG/MC-Reward cut TwI latency ~86% at 8K while beating larger models on high-res perception; self-reg proxy is hand-tuned but numbers hold.","tokens_in":42232,"tokens_out":630,"would_cite":true,"duration_ms":17787,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Multimodal models can internalize visual tools and self-regulate when to use them, matching external-tool methods on fine-grained perception while cutting inference latency by up to ~86%.","keywords":["multimodal large language models","Thinking with Images","implicit visual tools","self-regulated tool use","chain-of-thought","reinforcement learning","inference efficiency","fine-grained visual perception"],"falsifier":"On a held-out high-resolution perception set, disable the self-regulated reward (or fix tool use always-on / always-off) and check whether Net Tool Gain and end-to-end accuracy-plus-latency still improve together relative to the SFT-only model and to an external-tool baseline; if accuracy rises only when tools fire on problems the base model already solves, the boundary-calibration claim fails.","tokens_in":42166,"feed_emoji":"👁️","tokens_out":1016,"duration_ms":13347,"temperature":0.7,"pith_summary":"Thinking with Images has improved fine-grained visual perception by letting models zoom, crop, and otherwise manipulate images, but each external tool call and re-encoding costs latency and compute. This paper argues that the same behaviors can be folded into the model as implicit tool tokens, and that the model can learn a cognitive boundary so it only invokes those tools when internal knowledge is not enough. Training proceeds in two stages: supervised fine-tuning on formalized chains of thought that mix tool and no-tool trajectories, then a self-regulated reward that penalizes redundant tool use. The result is a model that routes between parametric knowledge and internalized visual operations, reporting state-of-the-art perception accuracy with single-pass inference speed. A sympathetic reader cares because it offers a practical path to strong visual reasoning without the I/O tax of agentic tool stacks.","feed_headline":"Internalized visual tools cut MLLM latency by ~86%","feed_subtitle":"Models learn when to use zoom-like slots themselves, matching agentic perception without external calls.","key_machinery":"Self-regulated implicit visual tools (IvT) plus MC-Reward: discrete tool-slot tokens stand in for operations such as zoom or crop; Net Tool Gain measures effective versus redundant tool use; MC-Reward uses online group-average accuracy p against a threshold τ to encourage tools on hard queries and penalize them on easy ones.","core_discovery":"BEE shows that visual tool invocation can be trained as an implicit, self-regulated behavior inside an MLLM: formalized CoT supervision activates tool slots and adaptive switching, and a self-regulated reward (guided by Net Tool Gain) teaches the model to invoke those slots only when group rollout accuracy indicates the task exceeds its current capability, yielding high fine-grained perception performance with far lower inference latency than external-tool Thinking-with-Images systems.","pith_inferences":["If group-rollout accuracy is a good boundary signal, similar self-regulated gating could apply to other latent or tool-augmented modalities (search, code execution) without hard-coding budgets.","The largest reported gains on monitoring and driving-style subtasks suggest the paradigm is most valuable where base models under-perceive dense or distant objects rather than where OCR is already strong.","Failure modes on multi-object scenes and sparse logical CoT imply that data coverage, not only the reward shape, still bounds when self-regulation helps versus when the model skips needed tools or steps.","A natural next test is whether NTG-style rewards remain stable as base models get stronger and the easy/hard mix of a fixed benchmark drifts."],"forward_implications":["Fine-grained perception at high resolution can be served in a single forward pass without sandbox APIs or repeated image re-encoding.","Redundant tool calls can be measured (Net Tool Gain) and trained down, lowering cost per sample while preserving or improving accuracy.","Implicit tool slots of modest length (even a handful of discrete tokens) can substitute for explicit zoom/crop/rotation pipelines on the evaluated benchmarks.","The same self-regulated routing can keep general reasoning competitive rather than trading it away for perception gains.","Deployment simplifies to a standard instruct MLLM stack, with reported throughput and cost advantages over multi-turn agentic tool systems."],"fun_headline_variants":["BEE internalizes visual tools cutting MLLM latency ~86%","Self-regulated implicit slots replace external TwI calls","NTG rewards teach MLLMs when to invoke internal zoom tools","Formalized CoT activates adaptive implicit visual tools","MLLMs learn to route knowledge vs tools for faster perception"],"cache_read_input_tokens":32896,"weakest_assumption_plain":"That the average accuracy of a small group of sampled rollouts is a trustworthy live signal of whether a problem is inside or outside the model’s capability, so a fixed threshold cleanly tells the model when tools help and when they waste effort.","fun_headline_variants_meta":{"raw":{"variants":["BEE internalizes visual tools cutting MLLM latency ~86%","Self-regulated implicit slots replace external TwI calls","NTG rewards teach MLLMs when to invoke internal zoom tools","Formalized CoT activates adaptive implicit visual tools","MLLMs learn to route knowledge vs tools for faster perception"]},"model":"grok-4.5","effort":"low","cost_usd":0.008394,"raw_usage":{"total_tokens":1975,"prompt_tokens":864,"num_sources_used":0,"completion_tokens":84,"cost_in_usd_ticks":83940000,"prompt_tokens_details":{"text_tokens":864,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1027,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":864,"tokens_out":84,"duration_ms":9821,"temperature":1.0,"reasoning_tokens":1027,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T07:00:48.322153+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On a held-out high-resolution perception set, disable the self-regulated reward (or fix tool use always-on / always-off) and check whether Net Tool Gain and end-to-end accuracy-plus-latency still improve together relative to the SFT-only model and to an external-tool baseline; if accuracy rises only when tools fire on problems the base model already solves, the boundary-calibration claim fails.","supporting_citations":[],"review_version":1}