Pith. sign in

REVIEW 4 major objections 5 minor 81 references

The paper claims that explicitly supervising an embodied vision-language model to output contact points and 3D boxes, on top of its usual text and 2D outputs, makes it more capable at embodied perception and localization and a better initia

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 01:52 UTC pith:P5D6KUIQ

load-bearing objection A serious, large-scale technical report with a clean real-robot comparison and an interesting scaling result, but the headline generalization numbers rest on self-built benchmarks that share names with the training mixture—until the splits are disclosed, treat those numbers as unverified. the 4 major comments →

arxiv 2607.17977 v2 pith:P5D6KUIQ submitted 2026-07-20 cs.RO

RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Model

Kehan Li , Bohan Hou , Minghao Zhu , Tianyi Zhang , Zesen Cheng , Zhikai Wang , Sicong Leng , Xin Li
show 23 more authors
Xiao Lin Biying Yao Minghua Zeng Jiangpin Liu Ronghao Dang Jiayan Guo Siteng Huang Haoyu Zhao Heng Ping Yaxi Zhao Tong Zhao Kexiang Wang Tong Lu Shengke Xue Jiahao Tang Yulei Wang Zejing Wang Jianwei Gao Shijian Lu Chengju Liu Jianfei Yang Mingxiu Chen Deli Zhao
This is my paper
classification cs.RO
keywords embodied foundation modelvision-language-action3D groundingcontact point predictioncross-embodiment action spacespatial understandingrobot manipulationflow matching
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that moving an embodied foundation model's pretraining closer to robot action—adding grasp contact-point prediction and native 3D grounding alongside standard language and 2D spatial outputs—improves both perception and downstream control. At three scales, the largest mixture-of-experts model is reported to outperform all evaluated proprietary and open-source competitors on three embodied benchmarks. The same model, used as the base for a vision-language-action policy, is said to beat a same-recipe policy built from a general vision-language model on real-robot tasks, and joint training across multiple embodiments and tasks is reported to beat per-task fine-tuning. The sympathetic reading is that explicit physical supervision, not scale alone, is what buys embodied generalization.

Core claim

RynnBrain 1.1 claims that a single autoregressive vision-language model can serve as an embodied foundation when its output vocabulary includes spatial tokens—2D boxes, points, trajectories, 3D boxes, and grasp contact points—trained jointly with text. The 122B-A10B variant is reported to surpass all compared proprietary and open-source models on VSI-Bench, MMSI, and RefSpatial-Bench, with a clear upward trend from 2B to 9B and 122B. For the 2B and 9B models, explicit 3D supervision yields native language-conditioned 3D grounding from a single image in camera coordinates. When post-trained as a flow-matching VLA in a unified action space with embodiment-specific masks, the resulting policies

What carries the argument

The load-bearing mechanism is a unified physically grounded output space: continuous 2D and 3D coordinates and angles are quantized into integer tokens and predicted autoregressively like text, so perception, localization, and reasoning share one interface. New for 1.1 are contact-point prediction, which replaces grasp rectangles with a compact center-plus-angle representation, and native 3D grounding, which predicts a 9-dimensional oriented 3D box in camera coordinates. For action, the VLA uses an 81-dimensional unified action space partitioned into body-part groups (arms, grippers, hands, torso, head) with embodiment-specific masks, so heterogeneous robots can be trained jointly without al

Load-bearing premise

The claimed generalization rests on the unstated assumption that the RynnBrain-* evaluation benchmarks and WildDet3D-Bench are held out from the training mixtures with no image-level overlap; the report never specifies how the splits were constructed.

What would settle it

Compute image-level near-duplicate overlap between the WildDet3D-Data training subsets and WildDet3D-Bench evaluation set, and between each RynnBrain-* training dataset and its same-named benchmark; finding eval images in training data would show the reported gains are in-distribution rather than evidence of embodied generalization.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the scaling analysis holds, reasoning-intensive embodied cognition is not emergent from general VLM scaling; it requires explicit spatiotemporal supervision, and larger models without it can regress.
  • If RynnBrain-initialized policies really beat same-recipe general-VLM policies, then embodied pretraining transfers to real manipulation and a stronger base model is a better VLA start.
  • If joint multi-task and multi-embodiment training improves over per-task fine-tuning, robot data from different morphologies can be pooled into one policy rather than split by robot.
  • If native 3D grounding works at 2B and 9B scale, compact models can produce metric 3D estimates from a single image, reducing reliance on depth sensors for grounding tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit is whether contact-point and 3D supervision help on genuinely novel object categories or only on near-training distribution; a per-category breakdown would separate these cases.
  • The same unified action space could likely support new embodiments by only writing an embodiment layer and collecting demonstrations, but the paper does not demonstrate zero-shot or few-shot embodiment transfer.
  • An external check would be to evaluate the released 2B and 9B weights on a fresh held-out set of indoor scenes with independently annotated 3D boxes, outside the reported benchmark mixtures.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents RynnBrain 1.1, a family of embodied foundation models at 2B, 9B, and 122B-A10B scales, initialized from Qwen3.5. It adds two pretraining capabilities to RynnBrain 1.0: contact-point prediction and native 3D grounding (for the 2B and 9B models). It also introduces RynnBrain-VLA, trained in a unified 81-dimensional cross-embodiment action space with embodiment-specific masking, and deploys it on Unitree G1, Astribot-S1, and Tianji-Wuji. The headline claims are: the 122B-A10B model beats all evaluated proprietary and open models on VSI-Bench, MMSI, and RefSpatial-Bench; RynnBrain-initialized VLA policies outperform Qwen-based and generalist VLA baselines; and joint multi-task/multi-embodiment training improves process scores and success rates over per-task training.

Significance. If the reported results withstand scrutiny, the paper is a substantial technical contribution: the RynnBrain-VLA controlled comparison with a Qwen-based policy under the same recipe is a valuable ablation, the unified action space with masking is a practical design for multi-embodiment training, and the scaling analysis across 2B/9B/122B-A10B under one recipe addresses an important question. Releasing model weights and code URLs is a strength. However, the current evidence for the central generalization claims is weakened by the heavy reliance on self-built benchmarks whose relationship to the training mixture is not disclosed. The external benchmarks and the real-robot Qwen-controlled comparison provide a credible core, but the manuscript needs major strengthening before the broad claims can be accepted.

major comments (4)
  1. [§3.2, Table 2; §5.1–5.2, Tables 3–5] The evaluation sets RynnBrain-Object, RynnBrain-Spatial, RynnBrain-Grounding, RynnBrain-Area, RynnBrain-Affordance, and RynnBrain-Trajectory appear verbatim as components of the pretraining mixture in Table 2, and RefSpatial appears both in the training mixture (§3.2 Area Localization) and in the evaluation list (§5.2 RefSpatial-Bench). No split construction, image-level deduplication, or overlap analysis is provided. The headline generalization claims therefore rest on self-built benchmarks whose held-out status is unverified. Please disclose how each RynnBrain-* eval set was partitioned from the corresponding training source, report exact and near-duplicate overlap statistics, and either replace or re-label the RynnBrain-* rows. The relationship between VSI-590K (training) and VSI-Bench (evaluation) should also be clarified.
  2. [§3.2 vs §5.4, Figure 5] The 3D grounding models are trained on WildDet3D Essential (102,979 images) and WildDet3D Synthetic (896,004 images) from WildDet3D-Data, and evaluated on WildDet3D-Bench, which appears to belong to the same dataset family. The paper does not state whether WildDet3D-Bench is a disjoint official held-out split, nor does it report deduplication between the filtered WildDet3D training subsets and the benchmark. Without this, the reported 23.44 vs 22.6 AP3D advantage over the specialized WildDet3D detector may be an in-distribution result. Please provide the split metadata and an overlap analysis, or move the 3D-grounding comparison to SUN RGB-D and other independent benchmarks.
  3. [§5.5, Figure 7] Contact-point prediction is presented as a new capability of RynnBrain 1.1 in the abstract, introduction, and contribution list, but Section 5.5 provides only qualitative visualizations and explicitly states that no standardized metric exists. A contribution claimed at this level needs at least one quantitative measure: e.g., success rate on a held-out grasp dataset, agreement with human annotations under a tolerance-based metric, or simulated grasping success. Otherwise the claim should be softened to a qualitative demonstration rather than a validated capability.
  4. [§5.6.1–5.6.3, Table 6] The claim that RynnBrain-VLA outperforms 'representative generalist VLAs' (GR00T N1.7 and π0.5) is not controlled: these baselines are not described as fine-tuned on the evaluated task demonstrations, and the action-chunk lengths differ (40 for GR00T N1.7, 50 for π0.5, 32 for RynnBrain-VLA). If these are zero-shot comparisons, this should be stated and the comparison framed accordingly. The Qwen-Based-VLA comparison, which uses the identical recipe and chunk length, is the sound evidence for the base-model advantage; the generalist comparison is weaker and should not be presented on equal footing.
minor comments (5)
  1. [Tables 3–5] The asterisk for 'our reproduction' should be accompanied by a detailed protocol (prompt template, sampling settings, decoding parameters, version/access date for proprietary APIs) so that the reproduced numbers can be verified.
  2. [Figure 4] The scaling-decomposition averages aggregate small sets of benchmarks (e.g., the reasoning-intensive group contains only MMSI and MindCube). Add per-group standard errors or per-benchmark detail, and note the small number of tasks when interpreting the 'negative scaling' of Qwen3.5.
  3. [§4.4] The RTC guidance strength β=10.0 is reported without sensitivity analysis. Since this parameter directly controls chunk-boundary continuity, a short ablation would help establish that the chosen value is not a critical tuning artifact.
  4. [§4.2, Figure 3] For the Unitree G1, the policy predicts 14D hand actions plus a 64D SONIC token outside the 81D unified action space. Clarify whether these 64D tokens are masked or shared in any way during joint multi-embodiment training, and how they are treated by the embodiment-specific masking mechanism.
  5. [§5.6.1, Eq. (2), Table 6] Reporting success over 20 trials without confidence intervals is understandable for real-robot work, but a binomial confidence interval or per-sub-task failure counts would help assess the significance of the 91.28→94.14 process-score improvement in the generalist setting. Also state whether sub-tasks are equally weighted and list the sub-task decomposition for the three long-horizon tasks.

Circularity Check

2 steps flagged

Self-built RynnBrain-* benchmarks and WildDet3D-Bench share names and data sources with the pretraining mixture, so several localization/3D-grounding results are in-distribution scores; external benchmarks keep the central claim partially independent.

specific steps
  1. fitted input called prediction [Sec. 3.2 (Table 2) and Secs. 5.1-5.2 (Tables 3-5)]
    "Table 2 training mixture lists: 'Object Understanding ... RynnBrain-Object, RefCOCO ... Spatial Understanding ... RynnBrain-Spatial ... Object Localization ... RynnBrain-Grounding ... Area Localization ... RynnBrain-Area ... Affordance Localization RynnBrain-Affordance ... Trajectory Prediction RynnBrain-Trajectory'. Sec. 5.1: 'The evaluation includes ... RynnBrain-Object, and RynnBrain-Spatial.' Sec. 5.2: 'We evaluate embodied localization across six spatial grounding tasks: ... RynnBrain-Grounding, RynnBrain-Area, RynnBrain-Affordance, and RynnBrain-Trajectory.'"

    The same names appear on both sides of the pipeline: Table 2 places these RynnBrain-* datasets in the pretraining mixture, while Secs. 5.1-5.2 report them as evaluation benchmarks. The paper gives no split, deduplication, or temporal separation between the training and evaluation uses. Absent a disclosed held-out split, the Tables 3-5 scores for these benchmarks are not predictions about unseen data; they are training-distribution fit presented as generalization.

  2. fitted input called prediction [Sec. 3.2 (3D-grounded Data) and Sec. 5.4]
    "Sec. 3.2: 'we adopt WildDet3D-Data [31] ... We use its two curated subsets after additional filtering: Essential (102,979 images, 374K manually verified annotations ...) and Synthetic (896,004 images, 888K automatically selected annotations ...).' Sec. 5.4: 'A similar scaling trend is observed on WildDet3D-Bench.'"

    The model is pretrained on WildDet3D Essential/Synthetic subsets and then evaluated on WildDet3D-Bench from the same dataset family. No overlap or split analysis is provided. The paper's contrast with a detector 'trained with additional in-domain data' is inapposite because RynnBrain's 3D supervision is also in-domain WildDet3D data; the AP3D improvement is therefore partly a measure of fit to the training distribution, not held-out generalization.

full rationale

The strongest abstract claim—outperforming proprietary and open models on VSI-Bench, MMSI, and RefSpatial-Bench—relies on external benchmarks and is not circular. However, the embodied-cognition and localization tables include RynnBrain-Object, RynnBrain-Spatial, RynnBrain-Grounding, RynnBrain-Area, RynnBrain-Affordance, and RynnBrain-Trajectory, all of which appear verbatim in the Table 2 training mixture, and the 3D-grounding evaluation uses WildDet3D-Bench after training on WildDet3D-Data. No split construction, deduplication, or overlap analysis is disclosed for any of these self-built benchmarks. The controlled VLA comparison and the external benchmark results provide independent support, so the paper is not wholly circular, but several reported 'generalization' results reduce to in-distribution scores. Score 6 reflects this partial circularity.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 2 invented entities

The central claims rest on the accuracy of lifted 3D annotations, the validity of self-built benchmarks, and the hand-designed action-space/masking choices; none are independently verified in the paper.

free parameters (7)
  • RTC guidance strength beta = 10.0
    Hand-set parameter in Real-Time Chunking (Sec 4.4) controlling how strongly the previous chunk guides the new chunk; no sensitivity analysis shown.
  • Action chunk length = 32 for RynnBrain-VLA (40 for GR00T N1.7, 50 for pi0.5)
    Chosen per-method in Sec 5.6.1; differences may affect comparability of real-robot results.
  • 2D spatial tokenization scale = coordinates normalized to [0,1000]
    Discretization of continuous coordinates in Sec 3.1; affects precision of all spatial outputs and 3D outputs.
  • 3D bounding box parameterization = 9D: center (cx,cy,cz), size (w,l,h), orientation (pitch,yaw,roll)
    Output format chosen in Sec 3.2, following prior works; no ablation of alternative parameterizations.
  • Unified action space partitioning = 81D = Arm-Joint 14 + Arm-EEF 18 + Gripper 2 + Hand 40 + Torso 4 + Head 3
    Hand-designed semantic grouping in Sec 4.2; central to cross-embodiment training, no ablation.
  • WildDet3D Synthetic filter retention threshold = >10 (Molmo2 perceptual score)
    Data filtering threshold for the Synthetic subset in Sec 3.2; impacts 3D supervision quality and evaluated AP3D.
  • Training hyperparameters = LR 5e-6/2e-6, batch 512/1024, warmup 0.03
    Scale-specific choices from Table 1; standard but hand-tuned, no full search reported.
axioms (5)
  • domain assumption WildDet3D monocular lifted 3D annotations are accurate enough for training and evaluation.
    Sec 3.2 uses WildDet3D Essential/Synthetic subsets as 3D supervision; Sec 5.4 uses WildDet3D-Bench as ground truth. If the lifted boxes carry systematic error, both training and reported AP3D are biased.
  • domain assumption The RynnBrain-* datasets used in evaluation are held out from pretraining.
    Tables 3-5 report scores on RynnBrain-Object/Spatial/Grounding/Area/Affordance/Trajectory, while Table 2 lists the same names as training sources. No split or contamination analysis is given.
  • domain assumption The unified action space with embodiment-specific masking preserves action statistics and supports joint training.
    Sec 4.2; the claim that joint training helps depends on this premise; no ablation with alternative action spaces is provided.
  • domain assumption Flow-matching DiT trained with action chunks produces executable robot actions at deployment.
    Sec 4.1; inherited from the pi0 literature, not re-derived here; real-robot results depend on it.
  • domain assumption 20 trials per task is a sufficient sample to rank policies.
    Sec 5.6.1; no confidence intervals, so reported differences such as 85% vs 80% could be noise.
invented entities (2)
  • Unified 81D cross-embodiment action space no independent evidence
    purpose: Maps heterogeneous robot action vectors into shared body-part groups (arms, gripper, hand, torso, head) with embodiment-specific masking so one policy can train across robots.
    Introduced in Sec 4.2; central to the claimed cross-embodiment benefit. No external validation; the paper's own real-robot trials are the only evidence.
  • Contact-point representation (p, theta) no independent evidence
    purpose: Replaces grasp rectangles with a contact center and in-plane orientation for action grounding.
    Sec 3.2; evaluated only qualitatively in Sec 5.5, so no falsifiable handle outside the paper yet.

pith-pipeline@v1.3.0-alltime-deepseek · 3998 in / 3992 out tokens · 186426 ms · 2026-08-03T01:52:26.469139+00:00 · methodology

0 comments
read the original abstract

We present RynnBrain 1.1, a family of embodied foundation models spanning 2B, 9B, and 122B-A10B scales. Trained with a unified spatio-temporal and physically grounded framework, RynnBrain 1.1 supports embodied perception, spatial reasoning, localization, and planning. Compared with RynnBrain 1.0, it further introduces contact-point prediction across the model family and native 3D grounding for the 2B and 9B models, yielding representations and outputs that are more directly aligned with robot manipulation. We also develop RynnBrain-VLA with a unified cross-embodiment action space and embodiment-specific masking, and deploy it on Unitree G1, Astribot-S1, and Tianji-Wuji. RynnBrain 1.1 achieves strong results on embodied cognition, localization, and 3D grounding, with the 122B-A10B model outperforming all evaluated proprietary and open-source models on VSI-Bench, MMSI, and RefSpatial-Bench. Real-robot experiments show that RynnBrain-initialized policies outperform Qwen-based and representative generalist VLAs, while joint multi-task and multi-embodiment training improves process scores and success rates over per-task training.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

81 extracted references · 33 linked inside Pith

  1. [1]

    Introducing claude sonnet 4.6, February 2025.https://www.anthropic.com/news/ claude-sonnet-4-6

    Anthropic. Introducing claude sonnet 4.6, February 2025.https://www.anthropic.com/news/ claude-sonnet-4-6

    2025

  2. [2]

    Cosmos-reason1: From physical common sense to embodied reasoning

    Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay, Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, et al. Cosmos-reason1: From physical common sense to embodied reasoning. arXiv preprint arXiv:2503.15558, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Graspclutter6d: A large-scale real-world dataset for robust perception and grasping in cluttered scenes.IEEE Robotics and Automation Letters, 2025

    Seunghyeok Back, Joosoon Lee, Kangmin Kim, Heeseon Rho, Geonhyup Lee, Raeyoung Kang, Sangbeom Lee, Sangjun Noh, Youngjin Lee, Taeyeop Lee, et al. Graspclutter6d: A large-scale real-world dataset for robust perception and grasping in cluttered scenes.IEEE Robotics and Automation Letters, 2025

    2025

  4. [4]

    Qwen3-vl technical report, 2025.https: //arxiv.org/abs/2511.21631

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    Pith/arXiv arXiv 2025

  5. [5]

    arXiv preprint arXiv:2410.24164, 2024

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al.π0: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024

    Pith/arXiv arXiv 2024

  6. [6]

    In9th Annual Conference on Robot Learning, 2025

    KevinBlack, NoahBrown, JamesDarpinian, KaranDhabalia, DannyDriess, AdnanEsmail, MichaelRobertEqui, Chelsea Finn, Niccolo Fusai, Manuel Y Galliker, et al.π0.5: A Vision-Language-Action Model with Open-World Generalization. In9th Annual Conference on Robot Learning, 2025

    2025

  7. [7]

    Real-time execution of action chunking flow policies.Advances in Neural Information Processing Systems, 38:33383–33407, 2026

    Kevin Black, Manuel Galliker, and Sergey Levine. Real-time execution of action chunking flow policies.Advances in Neural Information Processing Systems, 38:33383–33407, 2026

    2026

  8. [8]

    Activitynet: A large-scale video benchmark for human activity understanding

    Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. InProceedings of the ieee conference on computer vision and pattern recognition, pages 961–970, 2015

    2015

  9. [9]

    Coco-stuff: Thing and stuff classes in context

    Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. Coco-stuff: Thing and stuff classes in context. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1209–1218, 2018

    2018

  10. [10]

    Scaling spatial intelligence with multimodal foundation models.arXiv preprint arXiv:2511.13719, 2025

    Zhongang Cai, Ruisi Wang, Chenyang Gu, Fanyi Pu, Junxiang Xu, Yubo Wang, Wanqi Yin, Zhitao Yang, Chen Wei, Qingping Sun, et al. Scaling spatial intelligence with multimodal foundation models.arXiv preprint arXiv:2511.13719, 2025

    arXiv 2025

  11. [11]

    Scaling spatial intelligence with multimodal foundation models

    Zhongang Cai, Ruisi Wang, Chenyang Gu, Fanyi Pu, Junxiang Xu, Yubo Wang, Wanqi Yin, Zhitao Yang, Chen Wei, Qingping Sun, Tongxi Zhou, Jiaqi Li, Hui En Pang, Oscar Qian, Yukun Wei, Zhiqian Lin, Xuanke Shi, Kewang Deng, Xiaoyang Han, Zukai Chen, Xiangyu Fan, Hanming Deng, Lewei Lu, Liang Pan, Bo Li, Ziwei Liu, Quan Wang, Dahua Lin, and Lei Yang. Scaling spa...

    2026

  12. [12]

    Robo2vlm: Visual question answering from large-scale in-the-wild robot manipulation datasets.arXiv preprint arXiv:2505.15517, 2025

    Kaiyuan Chen, Shuangyu Xie, Zehan Ma, Pannag R Sanketi, and Ken Goldberg. Robo2vlm: Visual question answering from large-scale in-the-wild robot manipulation datasets.arXiv preprint arXiv:2505.15517, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    Sharegpt4video: Improving video understanding and generation with better captions

    Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, et al. Sharegpt4video: Improving video understanding and generation with better captions. Advances in Neural Information Processing Systems, 37:19472–19495, 2024

    2024

  14. [14]

    Detect what you can: Detecting and representing objects using holistic models and body parts

    Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu, Sanja Fidler, Raquel Urtasun, and Alan Yuille. Detect what you can: Detecting and representing objects using holistic models and body parts. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1971–1978, 2014

    1971

  15. [15]

    Molmo2: Open weights and data for vision-language models with video understanding and grounding.arXiv preprint arXiv:2601.10611, 2026

    Christopher Clark, Jieyu Zhang, Zixian Ma, Jae Sung Park, Mohammadreza Salehi, Rohun Tripathi, Sangho Lee, Zhongzheng Ren, Chris Dongjoo Kim, Yinuo Yang, et al. Molmo2: Open weights and data for vision-language models with video understanding and grounding.arXiv preprint arXiv:2601.10611, 2026

    Pith/arXiv arXiv 2026

  16. [16]

    Open X-Embodiment: Robotic learning datasets and RT-X models.https://arxiv.org/abs/2310.08864, 2023

    Open X-Embodiment Collaboration, Abby O’Neill, and et.al. Open X-Embodiment: Robotic learning datasets and RT-X models.https://arxiv.org/abs/2310.08864, 2023

    Pith/arXiv arXiv 2023

  17. [17]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Agibot world colosseum.https://github.com/OpenDriveLab/ AgiBot-World, 2024

    AgiBot World Colosseum contributors. Agibot world colosseum.https://github.com/OpenDriveLab/ AgiBot-World, 2024

    2024

  19. [19]

    Ecbench: Can multi-modal foundation models understand the egocentric world? a holistic embodied cognition benchmark

    Ronghao Dang, Yuqian Yuan, Wenqi Zhang, Yifei Xin, Boqiang Zhang, Long Li, Liuyi Wang, Qinyang Zeng, Xin Li, and Lidong Bing. Ecbench: Can multi-modal foundation models understand the egocentric world? a holistic embodied cognition benchmark. InProceedings of the Computer Vision and Pattern Recognition Conference, 2025

    2025

  20. [20]

    Rynnbrain: Open embodied foundation models.arXiv preprint arXiv:2602.14979, 2026.https: //arxiv.org/abs/2602.14979

    Ronghao Dang, Jiayan Guo, Bohan Hou, Sicong Leng, Kehan Li, Xin Li, Jiangpin Liu, Yunxuan Mao, Zhikai Wang, Yuqian Yuan, Minghao Zhu, Xiao Lin, Yang Bai, Qian Jiang, Yaxi Zhao, Minghua Zeng, Junlong Gao, Yuming Jiang, Jun Cen, Siteng Huang, Liuyi Wang, Wenqiao Zhang, Chengju Liu, Jianfei Yang, Shijian Lu, and Deli Zhao. Rynnbrain: Open embodied foundation...

    arXiv 2026

  21. [21]

    Embspatial-bench: Benchmarking spatial understanding for embodied tasks with large vision-language models

    Mengfei Du, Binhao Wu, Zejun Li, Xuan-Jing Huang, and Zhongyu Wei. Embspatial-bench: Benchmarking spatial understanding for embodied tasks with large vision-language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 346–355, 2024

    2024

  22. [22]

    Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction

    ZhiwenFan, JianZhang, RenjieLi, JungeZhang, RunjinChen, HezhenHu, KevinWang, HuaizhiQu, DilinWang, Zhicheng Yan, et al. Vlm-3r: Vision-language models augmented with instruction-aligned 3d reconstruction. arXiv preprint arXiv:2505.20279, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Graspnet-1billion: A large-scale benchmark for general object grasping

    Hao-Shu Fang, Chenxi Wang, Minghao Gou, and Cewu Lu. Graspnet-1billion: A large-scale benchmark for general object grasping. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11444–11453, 2020

    2020

  24. [24]

    Finevideo.https:// huggingface.co/datasets/HuggingFaceFV/finevideo, 2024

    Miquel Farré, Andi Marafioti, Lewis Tunstall, Leandro Von Werra, and Thomas Wolf. Finevideo.https:// huggingface.co/datasets/HuggingFaceFV/finevideo, 2024

    2024

  25. [25]

    Env-qa: A video question answering benchmark for compre- hensive understanding of dynamic environments

    Difei Gao, Ruiping Wang, Ziyi Bai, and Xilin Chen. Env-qa: A video question answering benchmark for compre- hensive understanding of dynamic environments. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 1675–1685, October 2021

    2021

  26. [26]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18995–19012, 2022

    2022

  27. [27]

    Roboafford++: A generative ai-enhanced dataset for multimodal affordance learning in robotic manipulation and navigation.arXiv preprint arXiv:2511.12436, 2025

    Xiaoshuai Hao, Yingbo Tang, Lingfeng Zhang, Yanbiao Ma, Yunfeng Diao, Ziyu Jia, Wenbo Ding, Hangjun Ye, and Long Chen. Roboafford++: A generative ai-enhanced dataset for multimodal affordance learning in robotic manipulation and navigation.arXiv preprint arXiv:2511.12436, 2025

    arXiv 2025

  28. [28]

    Mimo-embodied: X-embodied foundation model technical report.arXiv preprint arXiv:2511.16518, 2025

    Xiaoshuai Hao, Lei Zhou, Zhijian Huang, Zhiwen Hou, Yingbo Tang, Lingfeng Zhang, Guang Li, Zheng Lu, Shuhuai Ren, Xianhui Meng, et al. Mimo-embodied: X-embodied foundation model technical report.arXiv preprint arXiv:2511.16518, 2025

    Pith/arXiv arXiv 2025

  29. [29]

    Interlv-search: Benchmarking interleaved multimodal agentic search.arXiv preprint arXiv:2605.07510, 2026

    Bohan Hou, Jiuning Gu, Jiayan Guo, Ronghao Dang, Sicong Leng, Xin Li, Xuemeng Song, and Jianfei Yang. Interlv-search: Benchmarking interleaved multimodal agentic search.arXiv preprint arXiv:2605.07510, 2026

    Pith/arXiv arXiv 2026

  30. [30]

    Revisiting multi- modal positional encoding in vision-language models.arXiv preprint arXiv:2510.23095, 2025

    Jie Huang, Xuejing Liu, Sibo Song, Ruibing Hou, Hong Chang, Junyang Lin, and Shuai Bai. Revisiting multi- modal positional encoding in vision-language models.arXiv preprint arXiv:2510.23095, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    Wilddet3d: Scaling promptable 3d detection in the wild

    Weikai Huang, Jieyu Zhang, Sijun Li, Taoyang Jia, Jiafei Duan, Yunqian Cheng, Jaemin Cho, Matthew Walling- ford, Rustin Soraki, Chris Dongjoo Kim, Shuo Liu, Donovan Clay, Taira Anderson, Winson Han, Ali Farhadi, Bharath Hariharan, Jason Ren, and Ranjay Krishna. Wilddet3d: Scaling promptable 3d detection in the wild. arXiv preprint arXiv:2604.08626, 2026

    Pith/arXiv arXiv 2026

  32. [32]

    Robobrain: A unified brain model for robotic manipulation from abstract to concrete

    Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, et al. Robobrain: A unified brain model for robotic manipulation from abstract to concrete. arXiv preprint arXiv:2502.21257, 2025

    Pith/arXiv arXiv 2025

  33. [33]

    Egotaskqa: Understandinghumantasksinegocentric videos.Advances in Neural Information Processing Systems, 35:3343–3360, 2022

    BaoxiongJia, TingLei, Song-ChunZhu, andSiyuanHuang. Egotaskqa: Understandinghumantasksinegocentric videos.Advances in Neural Information Processing Systems, 35:3343–3360, 2022

    2022

  34. [34]

    Galaxea open-world dataset and g0 dual-system vla model.arXiv preprint arXiv:2509.00576, 2025

    Tao Jiang, Tianyuan Yuan, Yicheng Liu, Chenhao Lu, Jianning Cui, Xiao Liu, Shuiqi Cheng, Jiyang Gao, Huazhe Xu, and Hang Zhao. Galaxea open-world dataset and g0 dual-system vla model.arXiv preprint arXiv:2509.00576, 2025

    Pith/arXiv arXiv 2025

  35. [35]

    Visualgenome: Connectinglanguageandvisionusingcrowdsourced dense image annotations.International journal of computer vision, 123(1):32–73, 2017

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-JiaLi, DavidAShamma, etal. Visualgenome: Connectinglanguageandvisionusingcrowdsourced dense image annotations.International journal of computer vision, 123(1):32–73, 2017

    2017

  36. [36]

    Molmoact: Action reasoning models that can reason in space

    Jason Lee, Jiafei Duan, Haoquan Fang, Yuquan Deng, Shuo Liu, Boyang Li, Bohan Fang, Jieyu Zhang, Yi Ru Wang, Sangho Lee, Winson Han, Wilbert Pumacay, Angelica Wu, Rose Hendrix, Karen Farley, Eli VanderBilt, Ali Farhadi, Dieter Fox, and Ranjay Krishna. Molmoact: Action reasoning models that can reason in space. arXiv preprint arXiv:2508.07917, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Jacquard v2: Refining datasets using the human in the loop data correction method

    Qiuhao Li and Shenghai Yuan. Jacquard v2: Refining datasets using the human in the loop data correction method. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 7932–7938. IEEE, 2024

    2024

  39. [39]

    Describe anything: Detailed localized image and video captioning.arXiv preprint arXiv:2504.16072, 2025

    Long Lian, Yifan Ding, Yunhao Ge, Sifei Liu, Hanzi Mao, Boyi Li, Marco Pavone, Ming-Yu Liu, Trevor Dar- rell, Adam Yala, et al. Describe anything: Detailed localized image and video captioning.arXiv preprint arXiv:2504.16072, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  41. [41]

    Rdt-1b: a diffusion foundation model for bimanual manipulation

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation. InInternational Conference on Learning Representations, pages 29982–30009, 2025

    2025

  42. [42]

    Sonic: Supersizing motion tracking for natural humanoid whole-body control

    Zhengyi Luo, Ye Yuan, Tingwu Wang, Chenran Li, Fernando Castañeda, Sirui Chen, Zi-Ang Cao, Jiefeng Li, David Minor, Qingwei Ben, et al. Sonic: Supersizing motion tracking for natural humanoid whole-body control. arXiv preprint arXiv:2511.07820, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    Videogpt+: Integrating image and video encoders for enhanced video understanding.arXiv preprint arXiv:2406.09418, 2024

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Videogpt+: Integrating image and video encoders for enhanced video understanding.arXiv preprint arXiv:2406.09418, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    Generation and comprehension of unambiguous object descriptions

    Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 11–20, 2016

    2016

  45. [45]

    Deepstack: Deeply stacking visual tokens is surprisingly simple and effective for lmms.Advances in Neural Information Processing Systems, 37:23464–23487, 2024

    Lingchen Meng, Jianwei Yang, Rui Tian, Xiyang Dai, Zuxuan Wu, Jianfeng Gao, and Yu-Gang Jiang. Deepstack: Deeply stacking visual tokens is surprisingly simple and effective for lmms.Advances in Neural Information Processing Systems, 37:23464–23487, 2024

    2024

  46. [46]

    The mapillary vistas dataset for semantic understanding of street scenes

    Gerhard Neuhold, Tobias Ollmann, Samuel Rota Bulo, and Peter Kontschieder. The mapillary vistas dataset for semantic understanding of street scenes. InProceedings of the IEEE international conference on computer vision, pages 4990–4999, 2017

    2017

  47. [47]

    Llama-nemotron-vlm-dataset-v1.https://huggingface.co/datasets/nvidia/ Llama-Nemotron-VLM-Dataset-v1, 2025

    NVIDIA. Llama-nemotron-vlm-dataset-v1.https://huggingface.co/datasets/nvidia/ Llama-Nemotron-VLM-Dataset-v1, 2025

    2025

  48. [48]

    GR00T N1: An open foundation model for generalist humanoid robots

    NVIDIA, Johan Bjorck, Nikita Cherniadev Fernando Castañeda, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You L...

    2025

  49. [49]

    Cosmos 3: Omnimodal world models for physical ai.arXiv preprint arXiv:2606.02800, 2026

    NVIDIA et al. Cosmos 3: Omnimodal world models for physical ai.arXiv preprint arXiv:2606.02800, 2026

    Pith/arXiv arXiv 2026

  50. [50]

    Introducing GPT-5.4, March 2025.https://openai.com/index/introducing-gpt-5-4/

    OpenAI. Introducing GPT-5.4, March 2025.https://openai.com/index/introducing-gpt-5-4/

    2025

  51. [51]

    Thinker: A vision-language foundation model for embodied intelligence.https://arxiv.org/abs/2601.21199, 2025

    Baiyu Pan, Daqin Luo, Junpeng Yang, Jiyuan Wang, Yixuan Zhang, Hailin Shi, and Jichao Jiao. Thinker: A vision-language foundation model for embodied intelligence.https://arxiv.org/abs/2601.21199, 2025

    arXiv 2025

  52. [52]

    Egothinker: Unveiling egocentric reasoning with spatio-temporal cot.arXiv preprint arXiv:2510.23569, 2025

    Baoqi Pei, Yifei Huang, Jilan Xu, Yuping He, Guo Chen, Fei Wu, Yu Qiao, and Jiangmiao Pang. Egothinker: Unveiling egocentric reasoning with spatio-temporal cot.arXiv preprint arXiv:2510.23569, 2025

    arXiv 2025

  53. [53]

    Qwen3.5-omni technical report.arXiv preprint arXiv:2604.15804, 2026

    Qwen Team. Qwen3.5-omni technical report.arXiv preprint arXiv:2604.15804, 2026

    Pith/arXiv arXiv 2026

  54. [54]

    Paco: Parts and attributes of common objects

    Vignesh Ramanathan, Anmol Kalia, Vladan Petrovic, Yi Wen, Baixue Zheng, Baishan Guo, Rui Wang, Aaron Marquez, Rama Kovvuri, Abhishek Kadian, et al. Paco: Parts and attributes of common objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7141–7151, 2023

    2023

  55. [55]

    Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos.arXiv preprint arXiv:2408.00714, 2024

    Pith/arXiv arXiv 2024

  56. [56]

    Cinepile: A long video question answering dataset and benchmark.arXiv preprint arXiv:2405.08813, 2024

    Ruchit Rawal, Khalid Saifullah, Miquel Farré, Ronen Basri, David Jacobs, Gowthami Somepalli, and Tom Goldstein. Cinepile: A long video question answering dataset and benchmark.arXiv preprint arXiv:2405.08813, 2024

    Pith/arXiv arXiv 2024

  57. [57]

    Grounding dino 1.5: Advance the" edge" of open-set object detection.arXiv preprint arXiv:2405.10300, 2024

    Tianhe Ren, Qing Jiang, Shilong Liu, Zhaoyang Zeng, Wenlong Liu, Han Gao, Hongjie Huang, Zhengyu Ma, Xiaoke Jiang, Yihao Chen, et al. Grounding dino 1.5: Advance the" edge" of open-set object detection.arXiv preprint arXiv:2405.10300, 2024

    Pith/arXiv arXiv 2024

  58. [58]

    Seed1.8 model card: Towards generalized real-world agency, 2026.https://arxiv.org/abs/ 2603.20633

    Bytedance Seed. Seed1.8 model card: Towards generalized real-world agency, 2026.https://arxiv.org/abs/ 2603.20633

    Pith/arXiv arXiv 2026

  59. [59]

    Robovqa: Multimodal long-horizon reasoning for robotics

    Pierre Sermanet, Tianli Ding, Jeffrey Zhao, Fei Xia, Debidatta Dwibedi, Keerthana Gopalakrishnan, Christine Chan, Gabriel Dulac-Arnold, Sharath Maddineni, Nikhil J Joshi, et al. Robovqa: Multimodal long-horizon reasoning for robotics. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 645–652. IEEE, 2024

    2024

  60. [60]

    Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics

    Chan Hee Song, Valts Blukis, Jonathan Tremblay, Stephen Tyree, Yu Su, and Stan Birchfield. Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 15768–15780, 2025

    2025

  61. [61]

    Graspfactory: A large object-centric grasping dataset.arXiv preprint arXiv:2509.20550, 2025

    Srinidhi Kalgundi Srinivas, Yash Shukla, Adam Arnold, and Sachin Chitta. Graspfactory: A large object-centric grasping dataset.arXiv preprint arXiv:2509.20550, 2025

    arXiv 2025

  62. [62]

    Robobrain 2.5: Depth in sight, time in mind.arXiv preprint arXiv:2601.14352, 2026

    Huajie Tan, Enshen Zhou, Zhiyu Li, Yijie Xu, Yuheng Ji, Xiansheng Chen, Cheng Chi, Pengwei Wang, Huizhu Jia, Yulong Ao, et al. Robobrain 2.5: Depth in sight, time in mind.arXiv preprint arXiv:2601.14352, 2026

    arXiv 2026

  63. [63]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalk- wyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    Pith/arXiv arXiv 2023

  64. [64]

    Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Are- nas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Pith/arXiv arXiv 2025

  65. [65]

    Hy-embodied-0.5: Embodied foundation models for real-world agents.arXiv preprint arXiv:2604.07430, 2026

    Tencent Robotics X, HY Vision Team, Xumin Yu, Zuyan Liu, Ziyi Wang, He Zhang, Yongming Rao, Fangfu Liu, Yani Zhang, Ruowen Zhao, Oran Wang, Yves Liang, Haitao Lin, Minghui Wang, Yubo Dong, Kevin Cheng, Bolin Ni, Rui Huang, Han Hu, Zhengyou Zhang, Linus, and Shunyu Yao. Hy-embodied-0.5: Embodied foundation models for real-world agents.arXiv preprint arXiv:...

    Pith/arXiv arXiv 2026

  66. [66]

    Grasp-anything: Large-scale grasp dataset from foundation models

    An Dinh Vuong, Minh Nhat Vu, Hieu Le, Baoru Huang, Huynh Thi Thanh Binh, Thieu Vo, Andreas Kugi, and Anh Nguyen. Grasp-anything: Large-scale grasp dataset from foundation models. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14030–14037. IEEE, 2024

    2024

  67. [67]

    Foundationpose: Unified 6d pose estimation and tracking of novel objects

    Bowen Wen, Wei Yang, Jan Kautz, and Stan Birchfield. Foundationpose: Unified 6d pose estimation and tracking of novel objects. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  68. [68]

    Thinking in space: How multimodal large language models see, remember, and recall spaces

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 10632–10643, 2025

    2025

  69. [69]

    Cambrian-s: Towards spatial supersensing in video.arXiv preprint arXiv:2511.04670, 2025

    Shusheng Yang, Jihan Yang, Pinzhi Huang, Ellis Brown, Zihao Yang, Yue Yu, Shengbang Tong, Zihan Zheng, Yi- fan Xu, Muhan Wang, et al. Cambrian-s: Towards spatial supersensing in video.arXiv preprint arXiv:2511.04670, 2025

    Pith/arXiv arXiv 2025

  70. [70]

    Mmsi-bench: A benchmark for multi-image spatial intelligence.arXiv preprint arXiv:2505.23764, 2025

    Sihan Yang, Runsen Xu, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, et al. Mmsi-bench: A benchmark for multi-image spatial intelligence.arXiv preprint arXiv:2505.23764, 2025

    Pith/arXiv arXiv 2025

  71. [71]

    Spatial mental modeling from limited views

    Baiqiao Yin, Qineng Wang, Pingyue Zhang, Jianshu Zhang, Kangrui Wang, Zihan Wang, Jieyu Zhang, Keshigeyan Chandrasegaran, Han Liu, Ranjay Krishna, et al. Spatial mental modeling from limited views. InStructural Priors for Vision Workshop at ICCV’25, 2025

    2025

  72. [72]

    Modeling context in referring expressions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expressions. InEuropean conference on computer vision, pages 69–85. Springer, 2016

    2016

  73. [73]

    From seeing to doing: Bridging reasoning and decision for robotic manipulation, 2025.https: //arxiv.org/abs/2505.08548

    Yifu Yuan, Haiqin Cui, Yibin Chen, Zibin Dong, Fei Ni, Longxin Kou, Jinyi Liu, Pengyi Li, Yan Zheng, and Jianye Hao. From seeing to doing: Bridging reasoning and decision for robotic manipulation, 2025.https: //arxiv.org/abs/2505.08548

    Pith/arXiv arXiv 2025

  74. [74]

    Osprey: Pixel understanding with visual instruction tuning

    Yuqian Yuan, Wentong Li, Jian Liu, Dongqi Tang, Xinjie Luo, Chi Qin, Lei Zhang, and Jianke Zhu. Osprey: Pixel understanding with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 28202–28211, 2024

    2024

  75. [75]

    Videorefer suite: Advancing spatial-temporal object understanding with video llm

    Yuqian Yuan, Hang Zhang, Wentong Li, Zesen Cheng, Boqiang Zhang, Long Li, Xin Li, Deli Zhao, Wenqiao Zhang, Yueting Zhuang, et al. Videorefer suite: Advancing spatial-temporal object understanding with video llm. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 18970–18980, 2025

    2025

  76. [76]

    Videollama 3: Frontier multimodal foundation models for image and video understanding.arXiv preprint arXiv:2501.13106, 2025

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, et al. Videollama 3: Frontier multimodal foundation models for image and video understanding.arXiv preprint arXiv:2501.13106, 2025

    Pith/arXiv arXiv 2025

  77. [77]

    Pelican-vl 1.0: A foundation brain model for embodied intelligence.arXiv preprint arXiv:2511.00108, 2025

    Yi Zhang, Che Liu, Xiancong Ren, Hanchu Ni, Shuai Zhang, Zeyuan Ding, Jiayu Hu, Hanzhe Shan, Zhenwei Niu, Zhaoyang Liu, et al. Pelican-vl 1.0: A foundation brain model for embodied intelligence.arXiv preprint arXiv:2511.00108, 2025

    arXiv 2025

  78. [78]

    Video instruction tuning with synthetic data.arXiv preprint arXiv:2410.02713, 2024

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruction tuning with synthetic data.arXiv preprint arXiv:2410.02713, 2024

    Pith/arXiv arXiv 2024

  79. [79]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 633–641, 2017

    2017

  80. [80]

    Roborefer: Towards spatial referring with reasoning in vision-language models for robotics.arXiv preprint arXiv:2506.04308, 2025

    Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, et al. Roborefer: Towards spatial referring with reasoning in vision-language models for robotics.arXiv preprint arXiv:2506.04308, 2025

    arXiv 2025

Showing first 80 references.