SemanticXR introduces the first device-cloud system for real-time open-vocabulary semantic mapping and querying that organizes work around semantically identifiable objects to meet XR power, bandwidth, and memory limits.
hub Mixed citations
Faster Segment Anything: Towards Lightweight SAM for Mobile Applications
Mixed citation behavior. Most common role is background (40%).
40
Pith papers citing it
Background
40% of classified citations
abstract
Segment Anything Model (SAM) has attracted significant attention due to its impressive zero-shot transfer performance and high versatility for numerous vision applications (like image editing with fine-grained control). Many of such applications need to be run on resource-constraint edge devices, like mobile phones. In this work, we aim to make SAM mobile-friendly by replacing the heavyweight image encoder with a lightweight one. A naive way to train such a new SAM as in the original SAM paper leads to unsatisfactory performance, especially when limited training sources are available. We find that this is mainly caused by the coupled optimization of the image encoder and mask decoder, motivated by which we propose decoupled distillation. Concretely, we distill the knowledge from the heavy image encoder (ViT-H in the original SAM) to a lightweight image encoder, which can be automatically compatible with the mask decoder in the original SAM. The training can be completed on a single GPU within less than one day, and the resulting lightweight SAM is termed MobileSAM which is more than 60 times smaller yet performs on par with the original SAM. For inference speed, With a single GPU, MobileSAM runs around 10ms per image: 8ms on the image encoder and 4ms on the mask decoder. With superior performance, our MobileSAM is around 5 times faster than the concurrent FastSAM and 7 times smaller, making it more suitable for mobile applications. Moreover, we show that MobileSAM can run relatively smoothly on CPU. The code for our project is provided at \href{https://github.com/ChaoningZhang/MobileSAM}{\textcolor{red}{MobileSAM}}), with a demo showing that MobileSAM can run relatively smoothly on CPU.
hub tools
citation-role summary
background 2
baseline 2
method 1
citation-polarity summary
representative citing papers
PR-MaGIC refines prompts in in-context segmentation via test-time gradient flow from the mask decoder plus top-1 selection, yielding better masks across benchmarks without training.
Boxes2Pixels distills noisy SAM pseudo-masks into a compact DINOv2-based student with auxiliary localization and one-sided self-correction, delivering +6.97 anomaly mIoU and +9.71 binary IoU gains over baselines on wind turbine data with 80% fewer parameters.
OmniOVCD uses SAM 3's decoupled outputs and an SFID strategy to achieve state-of-the-art IoU scores of 67.2, 66.5, 24.5, and 27.1 on four OVCD benchmarks, surpassing prior methods.
ReA-OVCD is a training-free reliability-aware method for open-vocabulary change detection that uses semantic change reasoning and boundary-aware refinement to reduce artifacts and improve accuracy on remote sensing datasets.
DB-Nav/SER-Nav improves zero-shot object navigation by reranking frontier goals using activation from object co-occurrence and inhibition from similar distractors and failed visits.
Presents MMIO benchmark and RTVP method achieving state-of-the-art 42.2% AP in zero-shot industrial defect detection.
Presents MMIOC-1M benchmark with 1M+ samples across 14 super-categories and RTVPNet with domain projection, sparse sampling, and bidirectional interaction, claiming SOTA on MMIOC-1M, LVIS, and COCO.
SLAD uses shared LoRA adapters in joint training to align teacher-student features, boosting both models' performance and halving training time versus fine-tuning in distillation.
Proposes an Uncertainty-Aware Gaussian Map for VLN that builds a Semantic Gaussian Map from panoramas and extends it with three uncertainty types into a 3D Value Map for more reliable navigation.
InstructSAM uses learnable queries in a VLM to condition SAM3 for single-pass multi-instance segmentation from arbitrary instructions, with a new Inst2Seg benchmark.
RepSAM applies CKA-guided rank allocation in PEFT plus multi-modal fusion to adapt SAM, reaching 97.9% of full fine-tuning mIoU with 158x fewer parameters on robotic benchmarks.
SparseSAM achieves 2x faster inference and 2.8x memory reduction in SAM with only 0.004 mIoU loss at 0.4 density via Stripe-Sort Attention and Residual-Consistency MLP.
DarkLLM trains an LLM to generate language-driven adversarial perturbations that unify targeted, untargeted, segmentation, and multi-model attacks on foundation models.
ProCompNav builds a candidate pool from ambiguous queries then uses pool-splitting binary questions for disambiguation, improving success rate and shortening responses on CoIN-Bench and TextNav.
GLMap combines explicit 3D Gaussians with multi-scale language semantics in a dual-modality structure and uses an analytical Gaussian Estimator for incremental map building, improving zero-shot performance on navigation and reasoning tasks.
HTNav combines imitation and reinforcement learning in a staged, tiered structure with map learning to reach state-of-the-art performance on the CityNav benchmark for urban aerial navigation.
Uni-Hand forecasts 2D/3D hand waypoints, head motion, and contact states in egocentric views using vision-language fusion and dual-branch diffusion, with new benchmarks for downstream robotics and action tasks.
FiLM-Nav fine-tunes VLMs on a mixture of simulated navigation tasks to reach state-of-the-art SPL and success on HM3D ObjectNav and OVON benchmarks with generalization to unseen categories.
MapNav uses annotated semantic maps as memory for VLN agents, claiming SOTA results in simulation and real-world tests while promising code and data release.
Grounded SAM integrates Grounding DINO and SAM to support text-prompted open-world detection and segmentation, achieving 48.7 mean AP on SegInW zero-shot with the base detector and huge segmenter.
PGE-SAM adds a Prompt Guidance Generator, multi-scale feature interaction, and foreground reconstruction loss to SAM for better interactive segmentation on degraded images, plus a new DM-Seg benchmark.
RS4D distills ViT knowledge into SSM backbones for remote sensing instance segmentation, delivering 8x fewer parameters and 9x fewer FLOPs than ViT methods while matching or exceeding accuracy on SSDD, WHU, and NWPU datasets.
GASE automates high-fidelity simulation scene reconstruction from multi-view panoramic videos via Gaussian splatting, object extraction, and inpainting, yielding robot policies with under 10% performance gap versus real-world training.
citing papers explorer
-
SemanticXR: Low Power and Real-time Queryable Semantic Mapping with an Object-Level Device-Cloud Architecture
SemanticXR introduces the first device-cloud system for real-time open-vocabulary semantic mapping and querying that organizes work around semantically identifiable objects to meet XR power, bandwidth, and memory limits.
-
PR-MaGIC: Prompt Refinement Via Mask Decoder Gradient Flow For In-Context Segmentation
PR-MaGIC refines prompts in in-context segmentation via test-time gradient flow from the mask decoder plus top-1 selection, yielding better masks across benchmarks without training.
-
Boxes2Pixels: Learning Defect Segmentation from Noisy SAM Masks
Boxes2Pixels distills noisy SAM pseudo-masks into a compact DINOv2-based student with auxiliary localization and one-sided self-correction, delivering +6.97 anomaly mIoU and +9.71 binary IoU gains over baselines on wind turbine data with 80% fewer parameters.
-
OmniOVCD: Streamlining Open-Vocabulary Change Detection with SAM 3
OmniOVCD uses SAM 3's decoupled outputs and an SFID strategy to achieve state-of-the-art IoU scores of 67.2, 66.5, 24.5, and 27.1 on four OVCD benchmarks, surpassing prior methods.
-
ReA-OVCD: Reliability-Aware Open-Vocabulary Change Detection via Semantic and Spatial Refinement
ReA-OVCD is a training-free reliability-aware method for open-vocabulary change detection that uses semantic change reasoning and boundary-aware refinement to reduce artifacts and improve accuracy on remote sensing datasets.
-
Semantic Evidence Regulation via Relational Bias for Zero-Shot Object Navigation
DB-Nav/SER-Nav improves zero-shot object navigation by reranking frontier goals using activation from object co-occurrence and inhibition from similar distractors and failed visits.
-
Zero-Shot Learning in Industrial Scenarios: New Large-Scale Benchmark, Challenges and Baseline
Presents MMIO benchmark and RTVP method achieving state-of-the-art 42.2% AP in zero-shot industrial defect detection.
-
Unification of Closed-Open Industrial Detection Scenarios: New Large-Scale Benchmarks,Challenges and Baselines
Presents MMIOC-1M benchmark with 1M+ samples across 14 super-categories and RTVPNet with domain projection, sparse sampling, and bidirectional interaction, claiming SOTA on MMIOC-1M, LVIS, and COCO.
-
SLAD : Shared LoRA Adapters for Task Specific Distillation
SLAD uses shared LoRA adapters in joint training to align teacher-student features, boosting both models' performance and halving training time versus fine-tuning in distillation.
-
Uncertainty-Aware Gaussian Map for Vision-Language Navigation
Proposes an Uncertainty-Aware Gaussian Map for VLN that builds a Semantic Gaussian Map from panoramas and extends it with three uncertainty types into a 3D Value Map for more reliable navigation.
-
InstructSAM: Segment Any Instance with Any Instructions
InstructSAM uses learnable queries in a VLM to condition SAM3 for single-pass multi-instance segmentation from arbitrary instructions, with a new Inst2Seg benchmark.
-
RepSAM: Bridging Foundation Models to Robotic Vision via Representation-Guided Adaptation
RepSAM applies CKA-guided rank allocation in PEFT plus multi-modal fusion to adapt SAM, reaching 97.9% of full fine-tuning mIoU with 158x fewer parameters on robotic benchmarks.
-
SparseSAM: Structured Sparsification of Activations in Segment Anything Models
SparseSAM achieves 2x faster inference and 2.8x memory reduction in SAM with only 0.004 mIoU loss at 0.4 density via Stripe-Sort Attention and Residual-Consistency MLP.
-
DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Models
DarkLLM trains an LLM to generate language-driven adversarial perturbations that unify targeted, untargeted, segmentation, and multi-model attacks on foundation models.
-
ProCompNav: Proactive Instance Navigation with Comparative Judgment for Ambiguous User Queries
ProCompNav builds a candidate pool from ambiguous queries then uses pool-splitting binary questions for disambiguation, improving success rate and shortening responses on CoIN-Bench and TextNav.
-
Multi-Scale Gaussian-Language Map for Zero-shot Embodied Navigation and Reasoning
GLMap combines explicit 3D Gaussians with multi-scale language semantics in a dual-modality structure and uses an analytical Gaussian Estimator for incremental map building, improving zero-shot performance on navigation and reasoning tasks.
-
HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigation
HTNav combines imitation and reinforcement learning in a staged, tiered structure with map learning to reach state-of-the-art performance on the CityNav benchmark for urban aerial navigation.
-
Uni-Hand: Universal Hand Motion Forecasting in Egocentric Views
Uni-Hand forecasts 2D/3D hand waypoints, head motion, and contact states in egocentric views using vision-language fusion and dual-branch diffusion, with new benchmarks for downstream robotics and action tasks.
-
FiLM-Nav: Efficient and Generalizable Navigation via VLM Fine-tuning
FiLM-Nav fine-tunes VLMs on a mixture of simulated navigation tasks to reach state-of-the-art SPL and success on HM3D ObjectNav and OVON benchmarks with generalization to unseen categories.
-
MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigation
MapNav uses annotated semantic maps as memory for VLN agents, claiming SOTA results in simulation and real-world tests while promising code and data release.
-
Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks
Grounded SAM integrates Grounding DINO and SAM to support text-prompted open-world detection and segmentation, achieving 48.7 mean AP on SegInW zero-shot with the base detector and huge segmenter.
-
PGE-SAM: Prompt-Guided Feature Enhancement for Interactive Segmentation under Degradation
PGE-SAM adds a Prompt Guidance Generator, multi-scale feature interaction, and foreground reconstruction loss to SAM for better interactive segmentation on degraded images, plus a new DM-Seg benchmark.
-
Efficient Remote Sensing Instance Segmentation with Linear-Time State Space Distilled Visual Foundation Models
RS4D distills ViT knowledge into SSM backbones for remote sensing instance segmentation, delivering 8x fewer parameters and 9x fewer FLOPs than ViT methods while matching or exceeding accuracy on SSDD, WHU, and NWPU datasets.
-
GASE: Gaussian Splatting-Based Automated System for Reconstructing Embodied-Simulation Environments
GASE automates high-fidelity simulation scene reconstruction from multi-view panoramic videos via Gaussian splatting, object extraction, and inpainting, yielding robot policies with under 10% performance gap versus real-world training.
-
CLUE: Adaptively Prioritized Contextual Cues by Leveraging a Unified Semantic Map for Effective Zero-Shot Object-Goal Navigation
CLUE adaptively weights room-type and object-co-location cues from an LLM to construct a unified semantic value map that improves success rate and efficiency in zero-shot object-goal navigation.
-
TinySAM 2: Extreme Memory Compression for Efficient Track Anything Model
TinySAM 2 reaches 90% of SAM 2.1 performance on DAVIS and SA-V using 7% of the memory tokens and 3% of the training data via frame selection, spatial average pooling, temporal similarity-based token pruning, and a RepViT image encoder.
-
CAR-SAM: Cross-Attention Reconstruction for Post-Training Quantization of the Segment Anything Model
CAR-SAM introduces MatMul-Aware Compensation and Joint Cross-Attention Reconstruction to enable stable 4-bit post-training quantization of SAM, outperforming prior PTQ methods by 14.6% mAP on SAM-B and 6.6% on SAM-L.
-
The Midas Touch for Metric Depth
MTD turns relative depth into metric depth via segment-wise sparse graph optimization and discontinuity-aware geodesic pixel refinement, claiming better accuracy and generalization than prior depth methods.
-
Deep Reprogramming Distillation for Medical Foundation Models
DRD introduces a reprogramming module and CKA-based distillation to enable efficient, robust adaptation of medical foundation models to downstream 2D/3D classification and segmentation tasks, outperforming prior PEFT and KD methods on 18 tasks.
-
TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object Navigation
TrajRAG uses a topological-polar trajectory representation and hierarchical retrieval to accumulate and reuse geometric-semantic navigation experiences, improving zero-shot ObjectNav on MP3D and HM3D benchmarks.
-
Lightweight Distillation of SAM 3 and DINOv3 for Edge-Deployable Individual-Level Livestock Monitoring and Longitudinal Visual Analytics
Distilled SAM 3 and DINOv3 models deliver near-teacher accuracy in pig tracking (92.29% MOTA, 96.15% IDF1) and behavior classification while achieving 7.77x parameter reduction and fitting on Jetson Orin NX with headroom.
-
Unposed-to-3D: Learning Simulation-Ready Vehicles from Real-World Images
Unposed-to-3D learns simulation-ready 3D vehicle models from unposed real images by predicting camera parameters for photometric self-supervision, then adding scale prediction and harmonization.
-
SocialMirror: Reconstructing 3D Human Interaction Behaviors from Monocular Videos with Semantic and Geometric Guidance
SocialMirror reconstructs 3D meshes of closely interacting humans from monocular videos using semantic guidance from vision-language models and geometric constraints in a diffusion model to handle occlusions and maintain temporal and spatial consistency.
-
IGV-RRT: Prior-Real-Time Observation Fusion for Active Object Search in Changing Environments
IGV-RRT improves object goal navigation in dynamic indoor environments by combining uncertainty-aware priors from 3D scene graphs with online VLM observations in a real-time tree planner.
-
Towards Any-Quality Image Segmentation via Generative and Adaptive Latent Space Enhancement
GleSAM++ improves SAM robustness on degraded images by using generative enhancement, feature alignment, and adaptive degradation prediction while adding few parameters.
-
APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms
Presents APRIL-MedSeg, a modular YAML-configurable toolbox for 2D medical image segmentation integrating semi-supervised, domain adaptation, distillation, weakly supervised, text-guided, and foundation model paradigms with unified dataset and deployment interfaces.
-
EffiNav: Fusing Depth and Vision-Language for Efficient Object Goal Navigation
EffiNav combines depth and vision-language inputs for efficient object goal navigation, matching or exceeding baselines on success rate and path-length-weighted success across simulation benchmarks and real-robot tests.
-
A Deployable Embodied Vision-Language Navigation System with Hierarchical Cognition and Context-Aware Exploration
Introduces a hierarchical VLN architecture with asynchronous layers, incremental memory graph, and WTRP-based exploration that improves success and efficiency on resource-constrained robots.
- RoboAtlas: Contextual Active SLAM
- On Efficient Variants of Segment Anything Model: A Survey