Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

A standard multimodal LLM can deliver dense predictions—segmentation, depth, referring masks—straight from its vision tokens, with no task-specific decoder.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 23:17 UTC pith:YBOV7DRY

load-bearing objection DenseMLLM has a genuinely new multi-label vision-token loss and a convincing ablation, but the RefCOCO results rely on a two-stage pipeline that contradicts the core zero-extra-inference claim; otherwise worth reviewing. the 3 major comments →

arxiv 2602.14134 v2 pith:YBOV7DRY submitted 2026-02-15 cs.CV cs.AIcs.LG

DenseMLLM: Standard Multimodal LLMs for Dense Prediction

Yi Li , Hongze Shen , Lexiang Tang , Xin Li , Xinpeng Ding , Yinsong Liu , Deqiang Jiang , Xing Sun
show 1 more author
Xiaomeng Li
This is my paper
classification cs.CV cs.AIcs.LG
keywords multimodal large language modelsdense predictionsemantic segmentationdepth estimationreferring expression segmentationvision token supervisionmulti-label lossnext-token prediction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

DenseMLLM claims that a standard multimodal large language model—the usual vision-encoder, projector, and LLM stack—can produce dense predictions such as semantic segmentation, depth maps, and referring-expression masks without any task-specific decoder. The key step is to treat the vision tokens the model already generates as spatial feature maps: index their logits with the vocabulary IDs of the predicted categories, then upsample and take an argmax. To make those vision tokens carry the needed detail, the paper extends next-token prediction from text to vision tokens, supervising each vision token with all the labels present in its patch (multi-label), and handles the large vocabulary with a relevant-negative sampling loss. If correct, dense perception becomes a free capability of a general-purpose MLLM rather than a separate module.

Core claim

The paper's central discovery is that a standard MLLM, after training with a multi-label version of next-token prediction applied to its vision tokens, is itself an intrinsic dense predictor. At inference, the model first predicts the text labels of the categories present in the image; for each category it looks up the corresponding token IDs in the shared vocabulary, averages the raw vision-token logits across those IDs, reshapes them into a spatial grid, upscales with bilinear interpolation, and takes an argmax over categories to obtain the final mask or depth map. No external decoder, task token, or extra inference pass is involved. The paper reports that this minimalist recipe reaches co

What carries the argument

The load-bearing component is NTP-M, a multi-label next-token prediction loss for vision tokens. Where standard NTP assigns each token a single vocabulary ID, NTP-M builds a multi-hot target vector per vision token, modelling each vocabulary entry as an independent Bernoulli trial with sigmoid activations. Because the vocabulary is enormous and most entries are irrelevant, the loss averages positives and, for negatives, keeps only the top-k most probable (most confusing) entries per token; this relevant-negative sampling is what makes training stable. On the inference side, the companion mechanism is the 'index-the-logits' extraction: predicted category names supply vocabulary IDs, and the m

Load-bearing premise

The 1/32-resolution vision-token grid, after simple bilinear upsampling, retains enough spatial detail to produce competitive pixel-level masks and depth maps; if the grid is too coarse at native resolution, the method's high scores would depend on test-time upscaling rather than the model itself.

What would settle it

Feed the model a synthetic image of alternating 1-pixel vertical stripes on a plain background and ask for segmentation at the model's native input resolution; if the output map cannot reproduce the stripes even after upsampling — while a human or a full-resolution model can — the token grid is the binding resolution limit, and the 'intrinsic dense predictor' claim would need to be retracted to 'upscaled dense predictor.'

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A single forward pass of a standard MLLM can replace segmentation heads, depth heads, and mask-retrieval modules for three common dense tasks.
  • The same training objective transfers to other standard MLLMs: the paper's transfer experiment shows the recipe also improves dense-prediction accuracy when applied to a different off-the-shelf vision-language backbone.
  • Dense prediction performance scales with input resolution at test time: larger images yield steadily better maps, giving a simple compute-accuracy trade-off without retraining.
  • Because no task-specific components are added, dense perception can be integrated into general-purpose models without hurting their VQA, OCR, and reasoning scores.
  • Task-specific fine-tuning on top of the trained model gives further gains, so the model serves as a reusable dense-prediction backbone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the vision tokens really carry this level of spatial detail, similar multi-label supervision could plausibly extend to video (treating frame tokens as vision tokens) — a direction the paper leaves untested.
  • The success of simple bilinear upsampling suggests a practical ceiling tied to the 1/32 token grid; a light learned-upsampling layer would likely improve fine details further, at the cost of breaking the 'no decoder' purity.
  • The paper frames the predictor as 'intrinsic,' but its own ablations show the capability is learned from the multi-label supervision, not present in a plain MLLM; a fairer reading is that standard architectures have the capacity, and the loss unlocks it.
  • Instance segmentation and panoptic segmentation remain out of reach without an explicit instance-discrimination mechanism, as the paper notes; the multi-label token supervision may need an auxiliary grouping cue for those tasks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces DenseMLLM, a 4B-parameter multimodal LLM built from a standard ViT, projector, and LLM backbone. Its main contribution is a multi-label vision-token next-token-prediction loss (NTP-M), which supervises the logits of every vision token against a unified vocabulary containing semantic classes, depth bins, and referring-segmentation labels. At inference, dense predictions are read out directly from those vision-token logits by indexing with text-token IDs, applying argmax, and bilinearly upsampling to image resolution. The authors report strong results on semantic segmentation (54.2 mIoU on ADE20k), depth estimation (87.6 δ1 on DDAD), and referring segmentation (80.7 cIoU on RefCOCO-val), while maintaining competitive general VQA/OCR performance against other 4B MLLMs. The paper claims that this demonstrates a standard MLLM can act as an intrinsic dense predictor without task-specific decoders and without additional inference steps.

Significance. The potential significance is substantial. If the claims hold, the paper shows that a standard MLLM architecture, trained with a simple modification of next-token prediction, can produce dense maps without any specialized decoder, contradicting a widespread assumption in the field. The NTP-M objective is clearly motivated by the multi-label nature of vision tokens, and the internal ablations (Table 3) provide strong evidence that the relevant-negative-sampling component is crucial: performance on ADE20k jumps from 16.7 to 32.7 to 51.2 mIoU across the ablations. The preservation of general VQA and OCR capabilities (Table 2) is also a useful result. The paper provides a detailed four-stage training recipe and identifies a concrete, reproducible mechanism. The main weaknesses are in the evaluation protocol: the RefCOCO numbers come from a two-stage grounding-then-segmentation pipeline, and the depth evaluation uses benchmark-specific prompts and per-dataset quantization ranges. These issues directly affect the central 'zero extra inference' and 'general-purpose' claims, so they must be resolved before the paper can be accepted.

major comments (3)
  1. [§4.1 Evaluation; Appendix D.2; Table 8] The RefCOCO cIoU numbers in Table 1 are not produced by the single-pass readout claimed in Sec. 1 and Fig. 2c. Appendix D.2 describes the actual protocol: a first MLLM inference answers 'Please provide the bounding box coordinate...', the image is cropped with 1.2× padding and resized to short edge 1280, a second MLLM inference on the crop ('Segment the core target.') yields the mask, and the mask is reinserted into the original image. This is two full model forwards plus image transformations. Table 8 therefore reports 'Ours 0' extra inference only by not counting the RefCOCO procedure, and the Abstract/Sec. 1 statement about 'no additional inference steps' is false for one of the three headline tasks. The single-pass polygon variant is described as 'basic' and 'not evaluated.' Please provide a single-pass RefCOCO result, or explicitly re-scope the claim to a two-stage standard-MLLM pro
  2. [§4.1 Evaluation; Appendix D.3] Depth evaluation conditions on dataset identity and per-dataset depth ranges. The prompt says 'from the B dataset' with B = NYUv2/Cityscapes/DDAD, and the text describes different linear quantization ranges per benchmark (0–10 m, 0–80 m, 0.05–120 m). The reported δ1 numbers therefore measure the model with access to the benchmark name and its known depth range, not a prompt-agnostic generalist depth predictor. Please state whether these prompts and quantizations were seen during training, and include at least one cross-dataset evaluation with an open-style prompt (e.g., 'Estimate the depth') to support the claim that a standard MLLM performs general-purpose depth estimation without task-specific customization.
  3. [§3.1 Eq. (2); Table 5; Appendix D.1] The 'zero overhead' framing should account for test-time scaling and post-processing. Eq. (2) up-samples the 1/32-resolution vision-token logits with bilinear interpolation, and Table 5 shows that results continue to improve up to the largest tested scale (NTP-M 45.0 at 1/32 vs. 51.2 at 4/32). The evaluation protocols use 2×–4× input upscaling on several benchmarks and, for ADE20k, a softmax temperature of 0.2 plus DenseCRF. These are standard engineering choices, but they are part of the reported dense-prediction pipeline and should be disclosed in the main-text cost comparison (Table 8), since they add non-negligible compute and are not common to all compared baselines.
minor comments (6)
  1. [Table 1] Typos: 'Qwne2.5-VL' should be 'Qwen2.5-VL'; UniPixel is cited as (Rasheed et al., 2024) but the correct reference is (Liu et al., 2025).
  2. [Sec. 3.1] The phrase 'ensuring the indirectness of the framework' is unclear; presumably 'standardness' or 'simplicity' is intended.
  3. [Sec. 4.1] The sentence 'ing our proprietary internal data sources cannot be disclosed' is garbled; please rephrase.
  4. [Tables 3 and 4] The baseline naming is inconsistent: Table 3 calls the starting point 'Base (BCE)' while Table 4 calls the equivalent configuration 'NTP-M raw'. Clarify that these denote the same setup.
  5. [Appendix D.1] The ADE20k evaluation uses randomly sampled prompts, but no experiment quantifies prompt sensitivity. Reporting mean±std over prompt sets would strengthen the robustness claim.
  6. [Fig. 2c] The label 'No Additions' is in tension with the two-stage RefCOCO protocol described in Appendix D.2. Please align the figure's claim with the actual evaluation.

Circularity Check

0 steps flagged

No circular derivation; the RefCOCO two-stage protocol is a consistency/correctness issue, not a circularity.

full rationale

The paper's derivation chain is not circular. The central mechanism is direct supervision: the NTP-M loss in Eqs. (3)-(5) trains the very vision-token logits that Eq. (2) reads out with argmax, and the dense labels come from external benchmarks and datasets. That is ordinary supervised training, not a prediction that reduces to its inputs by construction. The ablation study (Table 3) and comparisons against external baselines (VisionLLM-v2, DepthLM, UFO, Qwen3-VL) provide independent empirical content. Self-citations to Youtu-LLM (Lu et al., 2026) for the backbone architecture and to the authors' earlier token-activation work are present but are not load-bearing: the dense-prediction claim is validated against external benchmarks, and the backbone is a standard LLM architecture. The one notable issue is the RefCOCO evaluation: Sec. 4.1 and Appendix D.2 describe a grounding-then-crop-then-second-inference protocol, while Table 8 claims 'Ours 0' extra inference. This is an internal inconsistency and an overclaim about the evaluation protocol, not a circular reduction: the final mask is still predicted from the cropped image, not read off from training labels or from a fitted parameter. Therefore the correct finding is no significant circularity; the RefCOCO pipeline claim should be corrected but does not make the derivation circular.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No new physical entities or mediators are introduced. The method rests on empirical assumptions about token spatial resolution, multi-hot label semantics, benchmark-conditioned prompts, and inheritance from an unpublished base model, along with several hand-chosen inference hyperparameters.

free parameters (5)
  • Relevant negative sampling top-k = 32
    Hyperparameter chosen by hand; Table 7 shows robustness across 8-128, but the headline ablation uses k=32.
  • ADE20k softmax temperature and DenseCRF = temperature 0.2 + DenseCRF
    Evaluation post-processing for semantic segmentation; applied to all segmentation benchmarks and needed for the reported headline numbers.
  • Per-benchmark depth quantization range = NYU 0-10m; DDAD 0.05-120m; Cityscapes 0-80m; open-world 0.5-100m
    Depth discretization range is benchmark-specific and supplied by the prompt; directly determines all depth metric values.
  • Background/sigmoid thresholds for open-set segmentation = background constant 0.5, logits scaled by 0.25 or divided by 4
    Post-processing for background classes in Context59 and open-set segmentation; varies by protocol.
  • Test-time upscaling factors = ADE 4x, Context59 3x, COCOStuff 3x, Cityscapes 2x, RefCOCO short side 1280
    Per-benchmark resolution scaling used during inference; materially affects mIoU and is not part of the model.
axioms (4)
  • domain assumption 32x-downsampled vision tokens preserve enough spatial detail for dense prediction after bilinear upsampling
    Central design choice in Section 3.1, Eq. (2); Table 5 varies downsample ratio only from 1/32 to 4/32 and never tests full native resolution. If token features lose fine detail, the method's readout cannot be accurate.
  • domain assumption Vision tokens can be assigned multiple vocabulary labels across tasks without ambiguity, and sigmoid Bernoulli supervision aligns with LLM logits
    Foundation of NTP-M in Eqs. (3-5); if token semantics cannot be decomposed into multi-hot targets, the supervision targets are noisy.
  • domain assumption Prompting the model with the test dataset name is an acceptable input for a generalist and does not invalidate comparisons
    Used throughout depth evaluation ('Please estimate the depth of this image from the B dataset') and some segmentation protocols; in a truly open-world deployment this benchmark identity is unavailable.
  • domain assumption The underlying Youtu-LLM 4B language model and SigLIP-2 encoder carry the general VL capabilities reported in Table 2
    DenseMLLM's general benchmark results largely inherit an unpublished, partially internal base model; the contributions of the dense-prediction method versus the base model are not isolated.

pith-pipeline@v1.3.0-alltime-deepseek · 24048 in / 13740 out tokens · 144023 ms · 2026-08-02T23:17:40.045645+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of DenseMLLM: Standard Multimodal LLMs for Dense Prediction." pith.science (2026). https://pith.science/paper/YBOV7DRY

@misc{pith2026260214134,
  author       = {Pith},
  title        = {Pith review of: DenseMLLM: Standard Multimodal LLMs for Dense Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YBOV7DRY}},
  note         = {Machine review of arXiv:2602.14134}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Multimodal Large Language Models (MLLMs) have demonstrated exceptional capabilities in high-level visual understanding. However, extending these models to fine-grained dense prediction tasks, such as semantic segmentation and depth estimation, typically necessitates the incorporation of complex, task-specific decoders and other customizations. This architectural fragmentation increases model complexity and deviates from the generalist design of MLLMs, ultimately limiting their practicality. In this work, we challenge this paradigm by accommodating standard MLLMs to perform dense predictions without requiring additional task-specific decoders. The proposed model is called DenseMLLM, grounded in the standard architecture with a novel vision token supervision strategy for multiple labels and tasks. Despite its minimalist design, our model achieves highly competitive performance across a wide range of dense prediction and vision-language benchmarks, demonstrating that a standard, general-purpose MLLM can effectively support dense perception without architectural specialization. This project is available at github.com/Eli-YiLi/DenseMLLM.

Figures

Figures reproduced from arXiv: 2602.14134 by Deqiang Jiang, Hongze Shen, Lexiang Tang, Xiaomeng Li, Xing Sun, Xin Li, Xinpeng Ding, Yi Li, Yinsong Liu.

Figure 1
Figure 1. Figure 1: Motivation. (a): PCA visualization of the hidden states reveals that the vision tokens of the proposed DenseMLLM intrin￾sically encode fine-grained details. Thus, DenseMLLM achieves high-quality dense predictions (segmentation and depth) directly from vision tokens without task-specific decoders. (b): A single vision token typically represents multiple vocabulary IDs (labels), especially in multi-task scen… view at source ↗
Figure 2
Figure 2. Figure 2: The framework of DenseMLLM. (a) Overview: Employing a standard MLLM architecture (Vision Encoder, Projector, LLM), our model outputs both text responses and dense predictions without specialized heads. (b) Training with NTP-M: To handle vision tokens containing multiple semantics (e.g., objects and depth), we propose NTP-M. This multi-label strategy supervises vision tokens against a vocabulary with the pr… view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative Results. Visualization of three dense prediction tasks (semantic segmentation from ADE20k, depth estimation from the NYUv2, and referring expression segmentation from the RefCOCO) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visualization of Vision Token Representations. This figure compares the Principal Component Analysis (PCA) visualiza￾tions (Oquab et al., 2023) of the last-layer hidden states of vision tokens. We contrast DenseMLLM-4B (with vision token supervision) against three models without such supervision: DenseMLLM-4B w/o, Qwen2.5-VL (Yang et al., 2024a), and Qwen3-VL (Bai et al., 2025a). Leveraging vision token su… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative example of semantic segmentation the ADE20k dataset. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative example of semantic segmentation from the ADE20k dataset. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative example of referring expression segmentation from the RefCOCO dataset. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative example of depth estimation from the NYUv2 dataset. Depth Estimation User Instruction: Please estimate the depth of this image from the NYUv2 dataset Model Response This is the <depth>.<mask>RLE str</mask> [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative example of depth estimation from the NYUv2 dataset. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Vision as Unified Multimodal Generation

    cs.CV 2026-07 conditional novelty 7.0

    A single unified multimodal model matches leading task-specialized vision systems across detection, segmentation, dense geometry, and multi-view 3D by casting all outputs as native text or image generation.

Reference graph

Works this paper leans on

23 extracted references · 9 linked inside Pith · cited by 1 Pith paper

  1. [3]

    RLE string

    The results is further improved after the task-specific fine-tuning. This demonstrates that our model can serve effectively as a foundational backbone for downstream tasks, allowing fine-tuning with minimal training overhead. 13 DenseMLLM: Standard Multimodal LLMs are Intrinsic Dense Predictors C. Qualitative Examples Semantic Segmentation User Instructio...

    2017

  2. [4]

    To reduce computation, we performed a 2x2 non-overlapping crop of the images for testing. The specific prompts are as follows: 18 DenseMLLM: Standard Multimodal LLMs are Intrinsic Dense Predictors <image> Question: Segment: train, motorcycle, vegetation, person, wall, terrain, pole, sky, traffic light, fence, bicycle, road, traffic sign, rider, building, ...

    2014

  3. [9]

    S., and Lin, M

    Qi, P., Liu, Z., Zhou, X., Pang, T., Du, C., Lee, W. S., and Lin, M. Defeating the training-inference mismatch via fp16.arXiv preprint arXiv:2510.26788,

    arXiv

  4. [10]

    Swinmtl: A shared architecture for simultaneous depth estimation and se- mantic segmentation from monocular camera images

    Taghavi, P., Langari, R., and Pandey, G. Swinmtl: A shared architecture for simultaneous depth estimation and se- mantic segmentation from monocular camera images. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4957–4964. IEEE,

    2024

  5. [11]

    Ufo: A unified approach to fine-grained visual perception via open-ended language interface.arXiv preprint arXiv:2503.01342, 2025a

    Tang, H., Xie, C., Wang, H., Bao, X., Weng, T., Li, P., Zheng, Y ., and Wang, L. Ufo: A unified approach to fine-grained visual perception via open-ended language interface.arXiv preprint arXiv:2503.01342, 2025a. Tang, J., Wang, Y ., Cao, H., and Xu, L. Basic: Boosting visual alignment with intrinsic refined embeddings in multimodal large language models....

    arXiv

  6. [12]

    org/abs/2502.14786

    URL https://arxiv. org/abs/2502.14786. Wang, H., Tang, H., Jiang, L., Shi, S., Naeem, M. F., Li, H., Schiele, B., and Wang, L. Git: Towards generalist vision transformer through universal language interface. InEuropean Conference on Computer Vision, pp. 55–73. Springer,

    Pith/arXiv arXiv

  7. [13]

    K., Singhal, S., Som, S., et al

    10 DenseMLLM: Standard Multimodal LLMs are Intrinsic Dense Predictors Wang, W., Bao, H., Dong, L., Bjorck, J., Peng, Z., Liu, Q., Aggarwal, K., Mohammed, O. K., Singhal, S., Som, S., et al. Image as a foreign language: Beit pretraining for vision and vision-language tasks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition...

    Pith/arXiv arXiv

  8. [14]

    Wu, J., Zhong, M., Xing, S., Lai, Z., Liu, Z., Chen, Z., Wang, W., Zhu, X., Lu, L., Lu, T., et al

    URLhttps://arxiv.org/abs/2508.18265. Wu, J., Zhong, M., Xing, S., Lai, Z., Liu, Z., Chen, Z., Wang, W., Zhu, X., Lu, L., Lu, T., et al. Visionllm v2: An end- to-end generalist multimodal large language model for hundreds of vision-language tasks.Advances in Neural Information Processing Systems, 37:69925–69975,

    Pith/arXiv arXiv

  9. [15]

    Qwen2.5 technical report

    Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., et al. Qwen2.5 technical report. arXiv:2412.15115, 2024a. Yang, L., Kang, B., Huang, Z., Zhao, Z., Xu, X., Feng, J., and Zhao, H. Depth anything v2.Advances in Neu- ral Information Processing Systems, 37:21875–21911, 2024b. Yin, Y ., Zhao, Y ., Zhang, Y ., Zhang, Y ., L...

    Pith/arXiv arXiv 2025

  10. [16]

    Visual representation alignment for multimodal large language models.arXiv preprint arXiv:2509.07979,

    Yoon, H., Jung, J., Kim, J., Choi, H., Shin, H., Lim, S., An, H., Kim, C., Han, J., Kim, D., et al. Visual representation alignment for multimodal large language models.arXiv preprint arXiv:2509.07979,

    arXiv

  11. [19]

    Semantic Segmentation for the Open World

    is applied to the logits to refine the pixel-level predictions. This operation is applied to all tasks related to semantic segmentation during evaluation, which is a common practice in segmentation methodsds. Here’s a specific example: <image> Question: Segment: tree, towel, tank, armchair, refrigerator, countertop, blanket, railing, hood, bathtub, radiat...

    2016

  12. [21]

    RLE string

    (grounding-then-segmentation).Due to the precision limitations from the number of points, we recommend first using grounding and then semantic segmentation. Specifically, after grounding outputs bounding boxes on the original image, we draw boxes on the image with random colors and crop the image using a padding ratio of 1.2 (an extra 0.2). We then resize...

    2012

  13. [23]

    During testing, the predicted values need to be de-quantized to obtain the real depth; otherwise, only relative depth is obtained

    Values outside this range are set to 0 (IGNORE). During testing, the predicted values need to be de-quantized to obtain the real depth; otherwise, only relative depth is obtained. Additionally, we require inputs to be resized to a focal length of 2000 pixels to simulate a camera with a default focal length. Discrepancies in actual focal length may lead to...

    2000

  14. [1000]

    During testing, we dequantize to the actual depths and exclude invalid depths

    Invalid depths are set to 0 and ignored during training. During testing, we dequantize to the actual depths and exclude invalid depths. (4) The prompt must precede the image so that the model can learn the potential quantization methods. The output format is a string of depth names corresponding to the pixel sizes after argmax and resizing. For NYUv2, sin...

    2016

  15. [2011]

    Token activation map to visually explain multimodal llms

    Li, Y ., Wang, H., Ding, X., Wang, H., and Li, X. Token activation map to visually explain multimodal llms. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 48–58, October 2025a. Li, Y ., Wang, H., Duan, Y ., Zhang, J., and Li, X. A closer look at the explainability of contrastive language-image pre-training.Pattern R...

    arXiv 2025

  16. [2014]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193,

    Oquab, M., Darcet, T., Moutakanni, T., V o, H., Szafraniec, M., Khalidov, V ., Fernandez, P., Haziza, D., Massa, F., El- Nouby, A., et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193,

    Pith/arXiv arXiv

  17. [2016]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    Yu, Q., Zhang, Z., Zhu, R., Yuan, Y ., Zuo, X., Yue, Y ., Dai, W., Fan, T., Liu, G., Liu, L., et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476,

    Pith/arXiv arXiv

  18. [2017]

    9 DenseMLLM: Standard Multimodal LLMs are Intrinsic Dense Predictors Liu, Y ., Ma, Z., Pu, J., Qi, Z., Wu, Y ., Shan, Y ., and Chen, C. W. Unipixel: Unified object referring and segmen- tation for pixel-level visual reasoning.arXiv preprint arXiv:2509.18094,

    arXiv

  19. [2018]

    Depthlm: Metric depth from vision language models.arXiv preprint arXiv:2509.25413,

    Cai, Z., Yeh, C.-F., Xu, H., Liu, Z., Meyer, G., Lei, X., Zhao, C., Li, S.-W., Chandra, V ., and Shi, Y . Depthlm: Metric depth from vision language models.arXiv preprint arXiv:2509.25413,

    arXiv

  20. [2023]

    URL https: //arxiv.org/abs/2308.12966. Bai, S., Cai, Y ., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y ., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Re...

    Pith/arXiv arXiv

  21. [2024]

    Unidepthv2: Universal monocular metric depth estimation made simpler.arXiv preprint arXiv:2502.20110,

    Piccinelli, L., Sakaridis, C., Yang, Y .-H., Segu, M., Li, S., Abbeloos, W., and Van Gool, L. Unidepthv2: Universal monocular metric depth estimation made simpler.arXiv preprint arXiv:2502.20110,

    Pith/arXiv arXiv

  22. [2025]

    V ., Khedr, H., Huang, A., et al

    Carion, N., Gustafson, L., Hu, Y .-T., Debnath, S., Hu, R., Suris, D., Ryali, C., Alwala, K. V ., Khedr, H., Huang, A., et al. Sam 3: Segment anything with concepts.arXiv preprint arXiv:2511.16719,

    Pith/arXiv arXiv

  23. [2026]

    Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., Cheng, H., Chang, K., Galley, M., and Gao, J

    URLhttps://arxiv.org/abs/2512.24618. Lu, P., Bansal, H., Xia, T., Liu, J., Li, C., Hajishirzi, H., Cheng, H., Chang, K., Galley, M., and Gao, J. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InICLR,

    arXiv