Pith. sign in

REVIEW 15 cited by

ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2505.21457 v2 pith:VNAKPWUD submitted 2025-05-27 cs.CV cs.AI

Muzhi Zhu , Hao Zhong , Canyu Zhao , Zongze Du , Mingyu Liu , Zheng Huang , Anzhou Li , Hao Chen
show 4 more authors
Cheng Zou Jingdong Chen Ming Yang Chunhua Shen
This is my paper
classification cs.CVcs.AI
keywords perceptionactiveactive-o3mllmscapabilitiesefficientframeworkfurther
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Active vision, also known as active perception, refers to actively selecting where and how to look in order to gather task-relevant information. It is a critical component of efficient perception and decision-making in humans and advanced embodied agents. With the rise of Multimodal Large Language Models (MLLMs) as central planners in robotic systems, the lack of methods for equipping MLLMs with active perception has become a key gap. We first provide a systematic definition of MLLM-based active perception tasks and show that GPT-o3's zoom-in strategy can be viewed as a special case, though it suffers from low efficiency and inaccurate region selection. To address these issues, we propose ACTIVE-o3, a reinforcement learning framework built on GRPO that equips MLLMs with active perception capabilities. Leveraging a modular sensing-action design and a dual-form reward, ACTIVE-o3 autonomously learns efficient and stable region selection strategies without explicit region-selection supervision. We further establish a comprehensive benchmark covering both open-world tasks, including small- and dense-object grounding, and domain-specific scenarios, including remote sensing, autonomous driving, and interactive segmentation. Experimental results demonstrate that ACTIVE-o3 significantly enhances active perception capabilities compared to baselines. Moreover, we show that our framework not only preserves the model's general understanding ability but can also serve as a proxy task for leveraging perception data, further improving performance on benchmarks such as RealWorldQA and MME.

Discussion (0). Sign in to comment.

Forward citations

Cited by 15 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Where to Look: Can Foundation Models Reach a Target Viewpoint Through Active Exploration?

    cs.CV 2026-05 accept novelty 8.0 of 10

    Introduces the TVR active viewpoint-matching task and TVRBench indoor simulation benchmark, where foundation models start at low single-digit success rates but reach 51.4% after visual-action SFT and multi-turn GRPO p...

  2. Eliciting Complex Spatial Reasoning in MLLMs through Wide-Baseline Matching

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    Authors create ReasonMatch-Bench and DCRL training to boost MLLM performance on wide-baseline matching, reporting gains over baselines while preserving general capabilities.

  3. PInVerify: An Offline Embodied Benchmark for Active Instance Verification

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    PInVerify is a new offline embodied benchmark for active instance verification that supplies multi-view captures and 6-sector navigation topology, with MLLM baselines reaching 85.6% after fine-tuning but showing no re...

  4. Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    Visual CoT agents exhibit tool-use collapse where tool usage declines but task accuracy rises, and adding entropy regularization for rollout diversity produces the strongest performance.

  5. BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

    cs.CV 2026-07 conditional novelty 6.0 of 10

    BVS combines early-stop attention rollout priors with a scale-aware non-stationary kernel and GP-UCB to locate tiny objects in UHR images more accurately and with fewer MLLM queries than prior visual-search methods.

  6. EAGLE-360: Embodied Active Global-to-Local Exploration in 360$^\circ$

    cs.CV 2026-07 unverdicted novelty 6.0 of 10

    EAGLE-360 introduces a global-to-local exploration framework for 360° visual search, adapting RoPE Rolling, creating a new VQA dataset, and using SFT+GRPO training to claim SOTA performance with 8x accuracy gain.

  7. Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization

    cs.CV 2026-04 unverdicted novelty 6.0 of 10

    MAPO improves multimodal chain-of-thought reasoning by requiring explicit textual descriptions of visual tool results and using a novel advantage estimator that combines semantic alignment with task rewards.

  8. Boosting Reasoning in Large Multimodal Models via Activation Replay

    cs.CV 2025-11 unverdicted novelty 6.0 of 10

    Activation Replay boosts multimodal reasoning in post-trained LMMs by replaying low-entropy activations from base models to RLVR counterparts at test time via visual token manipulation.

  9. Learning Active Perception via Self-Evolving Preference Optimization for GUI Grounding

    cs.CV 2025-09 conditional novelty 6.0 of 10

    LASER uses self-evolving preference optimization to teach GUI models to crop and zoom before clicking, reaching 55.7% on ScreenSpot-Pro with a 7B model.

  10. Perception-Aware Policy Optimization for Multimodal Reasoning

    cs.CL 2025-07 unverdicted novelty 6.0 of 10

    PAPO integrates perception-aware supervision via a KL-based loss into RLVR methods like GRPO, yielding 4.4-17.5% gains on multimodal benchmarks and 30.5% fewer perception errors, with larger gains on vision-heavy tasks.

  11. From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models

    cs.CL 2026-06 unverdicted novelty 5.0 of 10

    The survey formalizes MLLM perception as a unified vision-language capability and traces its evolution via a new five-stage taxonomy while outlining future challenges.

  12. Latent Visual States for Efficient Multimodal Reasoning

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    EVA generates adaptive Latent_slot tokens as internal visual thoughts, trained end-to-end with text tokens via D-GSPO on the EVA-230K dataset, claiming performance gains and better inference efficiency.

  13. CaptchaMind: Training CAPTCHA Solvers via Reinforcement Learning with Explicit Reasoning Supervision

    cs.CV 2026-05 unverdicted novelty 5.0 of 10

    Presents CaptchaBench benchmark and CaptchaMind RL solver achieving 82.9% success on benchmark tasks and 71% on real-world CAPTCHAs via explicit reasoning process supervision.

  14. DRS-GUI: Dynamic Region Search for Training-Free GUI Grounding

    cs.AI 2026-05 unverdicted novelty 5.0 of 10

    DRS-GUI introduces a dynamic region search method with Focus/Shift/Scatter actions and MCTS-based planning that improves GUI grounding accuracy by 14% on ScreenSpot-Pro for both general and GUI-specific MLLMs without ...

  15. Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search

    cs.CV 2025-09 unverdicted novelty 5.0 of 10

    Mini-o3 scales visual search reasoning to tens of interaction turns via a new probe dataset, iterative trajectory collection, and over-turn masking in RL, claiming SOTA performance while training only up to six turns.

Pith tools