IQA-Spider unifies reasoning, grounding, and referring for multi-granularity image quality assessment via a four-task paradigm and two-stage LMM training with training-free text-to-point mapping.
LaSagnA: Language-based segmentation assistant for complex queries
11 Pith papers cite this work. Polarity classification is still indexing.
11
Pith papers citing it
abstract
Recent advancements have empowered Large Language Models for Vision (vLLMs) to generate detailed perceptual outcomes, including bounding boxes and masks. Nonetheless, there are two constraints that restrict the further application of these vLLMs: the incapability of handling multiple targets per query and the failure to identify the absence of query objects in the image. In this study, we acknowledge that the main cause of these problems is the insufficient complexity of training queries. Consequently, we define the general sequence format for complex queries. Then we incorporate a semantic segmentation task in the current pipeline to fulfill the requirements of training data. Furthermore, we present three novel strategies to effectively handle the challenges arising from the direct integration of the proposed format. The effectiveness of our model in processing complex queries is validated by the comparable results with conventional methods on both close-set and open-set semantic segmentation datasets. Additionally, we outperform a series of vLLMs in reasoning and referring segmentation, showcasing our model's remarkable capabilities. We release the code at https://github.com/congvvc/LaSagnA.
citation-role summary
background 1
baseline 1
citation-polarity summary
representative citing papers
VISTAQA is a new benchmark for joint visual question answering correctness and pixel-level grounding, evaluated with the GROVE metric that uses per-sample geometric mean to require both dimensions to succeed.
SegAnswer trains an MLLM to generate segmentation masks instead of bounding boxes when zooming into image regions during visual reasoning, yielding consistent improvements across perception and hallucination benchmarks.
InstanceControl uses VLMs to auto-generate instance masks from text and visual conditions, with adaptive refinement, to enable controllable multi-object image generation without manual labeling.
A plug-in Q-Synth Module plus Attention-to-Point Decoder converts text-conditioned attention in frozen MLLMs into point heatmaps, improving part-level grounding accuracy on multiple datasets.
MedSIGHT unifies medical image comprehension and segmentation in Med-LVLMs via a Region Perceiver module and region codebook, trained progressively on 72K pairs to reach SOTA on both tasks across modalities.
X2SAM unifies any-segmentation across images and videos in one MLLM by adding a Mask Memory module for temporal consistency and joint training on mixed datasets.
The paper defines UAV reasoning segmentation, releases DRSeg—10,000 aerial images with reasoning QA and masks—and shows its PixDLM baseline beats prior reasoning-segmentation models under fine-tuning.
GTPBD-MM is the first multimodal benchmark for global terraced parcel extraction, integrating image, text, and DEM data with experiments showing that textual and terrain cues improve delineation accuracy over image-only approaches.
Sa2VA unifies SAM-2 segmentation with MLLM reasoning into a single model for referring segmentation and conversation on images and videos, supported by a new 72k-expression Ref-SAV dataset.
The survey formalizes MLLM perception as a unified vision-language capability and traces its evolution via a new five-stage taxonomy while outlining future challenges.
citing papers explorer
-
IQA-Spider: Unifying Multi-Granularity Image Quality Assessment with Reasoning, Grounding and Referring
IQA-Spider unifies reasoning, grounding, and referring for multi-granularity image quality assessment via a four-task paradigm and two-stage LMM training with training-free text-to-point mapping.
-
VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence
VISTAQA is a new benchmark for joint visual question answering correctness and pixel-level grounding, evaluated with the GROVE metric that uses per-sample geometric mean to require both dimensions to succeed.
-
Segmentation before Answering: Pixel Grounding for MLLM Visual Reasoning
SegAnswer trains an MLLM to generate segmentation masks instead of bounding boxes when zooming into image regions during visual reasoning, yielding consistent improvements across perception and hallucination benchmarks.
-
InstanceControl: Controllable Complex Image Generation without Instance Labeling
InstanceControl uses VLMs to auto-generate instance masks from text and visual conditions, with adaptive refinement, to enable controllable multi-object image generation without manual labeling.
-
Enhancing Part-Level Point Grounding for Any Open-Source MLLMs
A plug-in Q-Synth Module plus Attention-to-Point Decoder converts text-conditioned attention in frozen MLLMs into point heatmaps, improving part-level grounding accuracy on multiple datasets.
-
MedSIGHT: Towards Grounded Visual Comprehension in Medical Large Vision-Language Models
MedSIGHT unifies medical image comprehension and segmentation in Med-LVLMs via a Region Perceiver module and region codebook, trained progressively on 72K pairs to reach SOTA on both tasks across modalities.
-
X2SAM: Any Segmentation in Images and Videos
X2SAM unifies any-segmentation across images and videos in one MLLM by adding a Mask Memory module for temporal consistency and joint training on mixed datasets.
-
PixDLM: A Dual-Path Multimodal Language Model for UAV Reasoning Segmentation
The paper defines UAV reasoning segmentation, releases DRSeg—10,000 aerial images with reasoning QA and masks—and shows its PixDLM baseline beats prior reasoning-segmentation models under fine-tuning.
-
GTPBD-MM: A Global Terraced Parcel and Boundary Dataset with Multi-Modality
GTPBD-MM is the first multimodal benchmark for global terraced parcel extraction, integrating image, text, and DEM data with experiments showing that textual and terrain cues improve delineation accuracy over image-only approaches.
-
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos
Sa2VA unifies SAM-2 segmentation with MLLM reasoning into a single model for referring segmentation and conversation on images and videos, supported by a new 72k-expression Ref-SAV dataset.
-
From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
The survey formalizes MLLM perception as a unified vision-language capability and traces its evolution via a new five-stage taxonomy while outlining future challenges.