HUGIN: Enhancing Vision-Language Planning for Autonomous Logistics Sorting

Cangtian Zhou; Haotian Wang; Kebin Liu; Ke Ma; Li Liu; Xikai Sun; Xu Wang; Yunhao Liu; Zaishu Chen

REVIEW 3 major objections 6 minor 19 references

HUGIN: Enhancing Vision-Language Planning for Autonomous Logistics Sorting

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training framework called HUGIN lifts vision-language sorting-planning accuracy from 63.6% to 78.8% on a new benchmark.

desk verdict A genuinely useful new task formulation and benchmark, plus a deployable training recipe—but the headline gain is inflated by a data-budget mismatch; the real same-data gain is about 1.6 points, within plausible noise. read the letter →

arxiv 2608.11692 v1 pith:IINZ4ZTT submitted 2026-08-12 cs.AI

Xikai Sun , Cangtian Zhou , Kebin Liu , Ke Ma , Xu Wang , Zaishu Chen , Haotian Wang , Li Liu

show 1 more author

Yunhao Liu

This is my paper · ORCID

classification cs.AI

keywords vision-languagemodelsjointmulti-sceneunderstandingautonomouslogisticssortingembodiedAIplanningdataaugmentationrepresentationrankingBenchmulti-imagereasoning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Autonomous logistics sorting needs a planner that combines several spatially disjoint camera views into one action plan, a setting the paper formalizes as Joint Multi-Scene Understanding (JMSU). The paper argues that generic vision-language models struggle here because annotated multi-scene data is scarce and attention disperses over long visual contexts. To address both, it proposes HUGIN, whose Endogenous Data Augmentation (EDA) recombines verified atomic facts under operating constraints and whose Global Context Ranking (GCR) pushes the instruction representation toward the complete visual context instead of a partial one. On a new industrial benchmark, SortingBench, HUGIN lifts Qwen3-VL-8B accuracy from 63.6% to 78.8% and improves all five open VLMs tested. Because HUGIN adds no inference-time modules, the paper's claim, if correct, means strong logistics planners could be obtained from open VLMs with only a couple of thousand real annotated samples.

What carries the argument

Two mechanisms carry the argument. Endogenous Data Augmentation decomposes each annotated sorting sample into verifiable atomic facts (package counts, the package closest to a compartment exit, cage occupancy, spatial grounding) and a scripted synthesizer recombines those facts under the operating rules into new full plans whose labels are computed, not hallucinated. Global Context Ranking is a margin-based auxiliary loss on hidden states from the last transformer layer: the local visual context $e_n$ at an intermediate image-boundary token, the global visual context $e_N$ at the final image-boundary token such as `<|vision_end|>`, and the instruction intent $q$ before answer generation. The loss enforces $\cos(e_N, q) > \cos(e_n, q) + \alpha$ with a stop-gradient on $e_n$, so the model is pushed to integrate all views without sacrificing local perception; the objective is removed at inference, leaving the architecture unchanged.

What would settle it

Train Qwen3-VL-8B with GCR's margin term reversed so the loss prefers partial over global alignment; if SortingBench accuracy stays near 78.8%, the claimed ranking direction is not what drives the gain.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that HUGIN turns VLMs into reliable joint multi-scene planners by treating data scarcity and attention dispersion as separate, addressable problems. EDA converts 2,000 real annotated samples into about 22,000 constraint-consistent training records, while GCR adds a training-only margin loss that compares the cosine similarity between the instruction hidden state and the hidden state at the final image boundary against the similarity with an intermediate image boundary. On SortingBench, the result is a consistent gain over matched supervised fine-tuning baselines across five open vision-language models, with the largest reported jump being Qwen3-VL-8B from 63.6% to 78.8%. Held-out layouts and distractor-heavy inputs show the gain comes from using distributed evidence rather than positional shortcuts, and a real system deployment sorted more than 15,000 packages with 73.1% prediction accuracy.

Load-bearing premise

The method's central premise is that a single internal vector at the end of the image stream faithfully summarizes all camera views, so that ranking its similarity to the instruction can steer the model toward complete-context planning.

Editorial extensions

If this is right

JMSU's formal definition of spatial disjointness and decision-level interdependency gives other multi-camera planning tasks a shared evaluation condition.
HUGIN's gains on layouts 3 and 4, which were absent from training, indicate the method transfers to unseen workstation geometry rather than memorizing fixed arrangements.
GCR alone raises SortingBench accuracy by 5.2 to 6.1 percentage points across three VLMs, so the ranking objective is a reusable training signal independent of EDA.
Spillover gains on BLINK visual similarity and MUIRBench scene understanding suggest JMSU-oriented training sharpens cross-image comparison in other embodied benchmarks.
The reported deployment over 15,000 packages with 73.1% accuracy supports the paper's claim that VLM-based planning can operate in a real sorting loop.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of GCR's anchor assumption: applying the same ranking loss at arbitrary mid-context token positions, or swapping the global anchor for a random late visual token, would reveal whether the final image-boundary token is essential or just convenient.
EDA's decompose-and-recombine pattern should transfer to other constrained planning domains, such as multi-dock routing or inventory picking, whenever a plan can be expressed as a composition of auditable facts and hard rules.
If GCR works by enforcing complete-context alignment, it should also reduce position bias and order sensitivity on general multi-image benchmarks; the current evidence is suggestive, but the paper only tests a few such tasks.
The gap between the 78.8% benchmark number and the 73.1% deployment number marks a clear target for future work on lighting variation, occlusion, and controller noise.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know up front. The paper contributes a real new task formulation (JMSU), a new industrial benchmark (SortingBench), a training recipe (EDA + GCR), and a deployment test on 15,000 packages. But the headline +15.2% gain is not a matched comparison, and the stress-test note is right: SFT baselines train on only 2,000 samples, while HUGIN trains on roughly 22,000. That is architecture-matched but data-mismatched. What is genuinely good. The JMSU formalization in Section 3.2 is clean: spatially disjoint views with decision-level interdependency, formalized via a minimal sufficient set and a proper-subset uncertainty condition. SortingBench is a real asset—four layouts, held-out layouts and lighting, and a strict success criterion that requires the full action sequence plus an 85% IoU box. EDA is a sensible adaptation of compositional augmentation: decomposing samples into atomic facts and recombining them under operating constraints keeps labels auditable, and the synthetic scaling curve (Figure 5b) shows real data-efficiency gains. GCR is an interesting training-only ranking objective with no inference overhead; the fact that GCR alone gives 5.2–6.1% gains on the original 2k dataset suggests it does something useful, even if the theoretical anchor story is not fully validated. The necessary-image intervention and distractor stress tests are nice evidence that HUGIN uses complete context rather than positional shortcuts. The deployment result, while modest (73.1%), is real-world evidence that the approach works outside the benchmark. Where the soft spots are. The main flaw is the baseline mismatch. The abstract and Section 5.2 compare HUGIN against an SFT baseline trained on 2,000 samples, so the 63.6% to 78.8% number is misleading. The same-data comparison in Table 3 is 77.2% (CE-only, all data) vs 78.8% (HUGIN)—a 1.6-point gap with no error bars or repeated seeds. For the other four VLMs, no data-matched CE baseline is reported, so the claim of consistently outperforming matched baselines is unverified. The GCR hidden-state-anchor assumption is a secondary concern: the paper cites information-aggregation properties of causal attention but gives no direct validation that the final vision_end hidden state actually summarizes all N images for these specific models. The verified atomic facts language should be softened, too—only random inspection is reported. No code or data release limits reproducibility, though the authors may plan to release. Bottom line: this paper is worth engaging. The benchmark and task formulation are valuable to the multi-image planning community, and the EDA/GCR combination is a solid recipe even if the magnitude of the GCR-specific gain is uncertain. A serious referee should ask for data-matched CE baselines with variance estimates, validation of the anchor assumption, and code/data release. I would bring this to a reading group to discuss both the method and the comparison pitfalls.

Referee Report

3 major / 6 minor

Summary. The paper proposes HUGIN, a training framework for vision-language planning in autonomous logistics sorting systems (ALSS). It formalizes Joint Multi-Scene Understanding (JMSU), constructs a real-world benchmark called SortingBench from four workstation layouts, and introduces two training components: Endogenous Data Augmentation (EDA), which decouples samples into atomic facts and recombines them under operating constraints, and Global Context Ranking (GCR), a training-only ranking loss that aligns the instruction representation with the global visual context rather than a partial one. Experiments compare five open VLMs fine-tuned with HUGIN against SFT baselines, reporting gains such as Qwen3-VL-8B improving from 63.6% to 78.8% on SortingBench, along with stress tests, ablations, and a deployment test over more than 15,000 packages.

Significance. If the results hold, the paper makes a useful contribution: JMSU is a well-motivated industrial multi-image reasoning problem; SortingBench addresses a real data gap with held-out layouts; EDA is a constraint-preserving augmentation strategy with auditable fact-level labels; and the deployment test provides rare evidence of practical viability for VLM-based logistics planning. The GCR idea, using hidden states already present during prefill as anchors, is interesting and incurs no inference-time cost. However, the central empirical claim as stated is stronger than the reported evidence supports.

major comments (3)

[Abstract; §5.1–5.2, Tables 1 and 3] The claim that HUGIN 'consistently outperforms matched baselines' across five open VLMs is not supported by the reported comparisons. In §5.1, SFT baselines use only D_origin (2,000 samples), while HUGIN trains on roughly 22,000 samples (D_origin plus D_atomic, D_syn, D_aux, and D_general). The only data-matched comparison in the paper is Table 3's CE-only row (Demb ✓, Dgen ✓, CE ✓, GCR ✗), which reaches 77.2% for Qwen3-VL-8B versus 78.8% for HUGIN. With no error bars or repeated seeds, a 1.6-point gap is within plausible run-to-run variation, and no data-matched CE baseline is reported for the other four VLMs. The +15.2% headline therefore largely reflects the additional EDA-generated training data rather than the HUGIN framework as a whole. Please re-run all five models against a same-data CE baseline and report variance.
[§5.2–5.3, Tables 1–3] No experiment reports variance across seeds, confidence intervals, or significance tests. Every conclusion, especially the 1.6-point GCR gain in Table 3 and the +5.1% improvement for Qwen3-VL-4B in Table 1, is based on single point estimates. Given that fine-tuning is sensitive to seed and data order, the central claim requires at least three seeds with mean and standard deviation for the main SortingBench comparisons, and ideally a paired significance test for the GCR ablation.
[§4.2, 'Theoretical basis' and 'Anchor indexing'] The GCR objective presupposes that the hidden state at the final <|vision_end|> token (e_N) is a faithful semantic summary of all N images and that the token immediately before answer generation (q) captures instruction intent. This is asserted from the causal-mask information-aggregation property and general citations, but no direct validation is provided for the specific VLMs and layers used. Since GCR is one of the two core components and its measured isolated gain is small (1.6 points in Table 3), the paper should either validate the anchor assumption (e.g., probe or replace the anchor positions) or present the GCR result as empirical rather than mechanism-based.

minor comments (6)

[§5.1, 'Baselines and metrics'] The phrase 'matched SFT baselines' is used in the contributions and §5.2, but the baselines are matched only in architecture and schedule, not in data budget; please rephrase throughout to avoid implying a fully matched comparison.
[Table 1] Table 1 contains formatting errors such as '81.568.3' in the Ovis2.5-2B row and 'RoboBrain2.5-8B-NV77.2' lacking a space; these make the results difficult to read and should be corrected.
[§4.1, 'Combination: constraint-based task synthesis'] The sentence 'We additionally inspect randomly sampled synthetic records and verify their structure and labels before training' should report the number or fraction of records inspected and the verification protocol, since the preceding claim that labels 'remain auditable' is otherwise stronger than the stated evidence.
[§5.4] The deployment result of 73.1% prediction accuracy needs a precise definition: is it the same whole-sequence-plus-IoU criterion as SortingBench, and is it computed per package or per sorting cycle?
[Figure 5(a)] The text says GCR improves all three models by 5.2%–6.1% on D_origin only, but no variance or number of runs is reported; please clarify whether each point is a single run.
[General] References to the supplementary material for the complete EDA implementation are not verifiable in the submitted manuscript; please include the promised material or clearly indicate where it will be available.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning found; the HUGIN gains come from training-time regularizers and recomputed labels, not from fitted inputs or self-citation chains.

full rationale

I walked the paper's derivation chain and found no load-bearing circularity. EDA decomposes D_origin into atomic facts and then recomputes complete plans from structured facts under operating constraints, so the synthetic labels are not borrowed from the benchmark or fitted to the test set. GCR is a training-only ranking loss defined on the model's own hidden states (e_N, e_n, q) and is removed at inference; it does not use test labels, benchmark statistics, or fitted parameters that are later renamed as predictions. The evaluation on SortingBench is a held-out set, and the paper reports gains that are at least partly attributable to the additional EDA-derived training data and the GCR objective. A caveat exists that the headline '+15.2%' compares HUGIN (trained on ~22k samples) with an SFT baseline trained only on D_origin (2k samples), so the phrase 'matched baselines' is imprecise; Table 3 shows a same-data CE-only baseline at 77.2% versus 78.8% for HUGIN, narrowing the gain to 1.6 points. However, this is an experimental comparability and statistical-evidence concern, not a circularity in the derivation: no equation, label, hidden-state anchor, or benchmark number is defined in terms of the quantity it is claimed to predict. Self-citations in the reference list (e.g., MME and VEGA, where one author overlaps) appear only as general benchmarks or related-work context and are not load-bearing justifications for HUGIN's central claims. No step reduces by construction to its own inputs, so a score of 0 is appropriate.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces a task formulation (JMSU), a benchmark (SortingBench), and a training framework (HUGIN), but no new physical entity, force, or conserved quantity. The free parameters are training hyperparameters and success thresholds chosen by hand. The main load-bearing assumptions are the semantic-anchor interpretation of hidden states, the correctness of the LLM decoupler, and the completeness of the encoded business rules.

free parameters (5)

GCR margin alpha = 0.1
Chosen in Section 4.2 to favor complete-context alignment without forcing the model to discard local evidence; no sensitivity analysis is reported.
GCR weight lambda = 0.01
Set in Section 5.1 to balance cross-entropy and ranking loss; no sweep is reported.
Synthetic JMSU sample count |D_syn| = 2,000
Selected in Section 5.3 as a trade-off between accuracy and training time; larger subsets continue to improve accuracy.
Bounding box IoU threshold = 0.85
Set in Section 5.1 as the success criterion for package localization; directly affects reported accuracy.
Fine-tuning epochs = 2
Section 5.1 reports 2 epochs on 4 L20 GPUs with learning rate 1e-5; all variants share the same schedule.

assumptions (4)

domain assumption Hidden states at token positions serve as semantic anchors that summarize the entire preceding visual and textual context.
Section 4.2 'Theoretical basis' asserts this from causal attention and cites prior work; GCR's objective is meaningful only if this holds for the specific VLMs and layers used.
domain assumption The LLM-based decoupler extracts complete and correct atomic facts from raw JMSU samples.
Section 4.1 'Decoupling' relies on an LLM parser; only randomly sampled synthetic records are inspected, so decoupler errors can propagate into EDA-generated labels.
domain assumption The business rules (fullest compartment, FIFO by closest package, largest remaining cage capacity) fully determine the correct sorting plan.
Section 3.1 and Section 4.1 'Combination' encode these rules; if deployment has additional unmodeled constraints, the computed labels may differ from valid plans.
domain assumption The collected real-world samples and SortingBench labels are accurately annotated and representative of ALSS operations.
Section 5.1 describes collection over three months but reports no annotation agreement or label audit.

how reviews work

0 comments

Cite this review

Pith. "Pith review of HUGIN: Enhancing Vision-Language Planning for Autonomous Logistics Sorting." pith.science (2026). https://pith.science/paper/IINZ4ZTT

@misc{pith2026260811692,
  author       = {Pith},
  title        = {Pith review of: HUGIN: Enhancing Vision-Language Planning for Autonomous Logistics Sorting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IINZ4ZTT}},
  note         = {Machine review of arXiv:2608.11692}
}

read the original abstract

Autonomous logistics sorting systems (ALSS) are an important industrial application of embodied AI, which requires joint planning over spatially disjoint camera views. We formulate this setting as Joint Multi-Scene Understanding (JMSU). With open-world visual understanding and task-planning capabilities, vision-language models (VLMs) are promising candidates for JMSU. However, directly applying existing VLMs to JMSU is non-trivial due to scarce cross-scene supervision and attention dispersion caused by long visual context in JMSU. To address these challenges, we propose HUGIN, a training framework with two complementary components. Endogenous Data Augmentation recombines verified atomic facts under operating constraints, while Global Context Ranking aligns the instruction representation more strongly with the complete visual context than with a partial visual context. To support ongoing research, we construct a high-quality industrial sorting dataset and benchmark named SortingBench from four layouts of autonomous logistics sorting systems. Across five open VLMs, HUGIN consistently outperforms matched baselines; for example, the accuracy on SortingBench of Qwen3-VL-8B increases from 63.6% to 78.8%. Additional experiments verify the effectiveness of each component and JMSU's spillover benefits in embodied tasks. Deployment tests involving more than 15,000 packages support the practical viability of VLM-based planning for autonomous logistics sorting.

Figures

Figures reproduced from arXiv: 2608.11692 by the authors.

**Figure 2.** A JMSU sample. The VLM receives the synchro [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** A unified framework to enhance VLMs on JMSU. Hugin includes two stages: EDA to mitigate the scarcity of JMSU FL [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: The data flow process during VLM training. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: (a) GCR gains on Dorigin only. (b) Synthetic JMSU scaling on Qwen3-VL-8B. substantial improvements in specific embodied tasks while preserving its general multimodal performance. 5.3 Diagnostic Tests and Ablations Order and distractor stress tests.To verify the effecti…

**Figure 6.** Figure 6: ALSS execution: reset, move to the chosen compartment, grasp the target package, move to the chosen cage, and place. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 3 canonical work pages

[3]

InProceedings of the 32nd ACM international conference on multimedia, 11198–11201

Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. InProceedings of the 32nd ACM international conference on multimedia, 11198–11201. Fu,C.;Chen,P.;Shen,Y.;Qin,Y.;Zhang,M.;Lin,X.;Yang, J.;Zheng,X.;Li,K.;Sun,X.;etal.2025a. Mme:Acompre- hensiveevaluationbenchmarkformultimodallargelanguage models. InThe Thirty-ninth Annual Confere...

arXiv 2024
[6]

InFindings of the Association for Computational Linguistics: EMNLP 2023, 9318–9333

In-context learning creates task vectors. InFindings of the Association for Computational Linguistics: EMNLP 2023, 9318–9333. Huang,W.;Jia,B.;Zhai,Z.;Cao,S.;Ye,Z.;Zhao,F.;Xu,Z.; Hu, Y.; and Lin, S

work page 2023
[7]

Hudson,D.A.;andManning,C.D.2019.Gqa:Anewdataset for real-world visual reasoning and compositional question answering

Vision-r1: Incentivizing reason- ing capability in multimodal large language models.arXiv preprint arXiv:2503.06749. Hudson,D.A.;andManning,C.D.2019.Gqa:Anewdataset for real-world visual reasoning and compositional question answering. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6700–6709. Hussain,M.2023. YOLO-v1toY...

arXiv 2019
[8]

Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A

Openvla: An open-source vision-language- action model.arXiv preprint arXiv:2406.09246. Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.-Y.; et al

arXiv
[9]

InProceedings of the IEEE/CVF international conference on computer vision, 4015–4026

Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, 4015–4026. Kong, L.; Wang, H.; Mu, W.; Du, Y.; Zhuang, Y.; Zhou, Y.; Song,Y.;Zhang,R.;Wang,K.;andZhang,C.2024.Aligning largelanguagemodelswithrepresentationediting:Acontrol perspective.Advances in Neural Information Processing Systems, 37: 37356–37384. Lai,Z.;Zhan...

work page 2024
[10]

InEu- ropean Conference on Computer Vision,111–127.Springer

Veclip: Improving clip training via visual-enriched captions. InEu- ropean Conference on Computer Vision,111–127.Springer. Liang,J.;Huang,W.;Xia,F.;Xu,P.;Hausman,K.;Ichter,B.; Florence,P.;andZeng,A.2023. Codeaspolicies:Language model programs for embodied control. In2023 IEEE In- ternational conference on robotics and automation (ICRA), 9493–9500. IEEE. L...

arXiv 2023
[12]

Vaswani,A.;Shazeer,N.;Parmar,N.;Uszkoreit,J.;Jones,L.; Gomez,A.N.;Kaiser,Ł.;andPolosukhin,I.2017

The geometry of hidden repre- sentations of large transformer models.Advances in Neural Information Processing Systems, 36: 51234–51252. Vaswani,A.;Shazeer,N.;Parmar,N.;Uszkoreit,J.;Jones,L.; Gomez,A.N.;Kaiser,Ł.;andPolosukhin,I.2017. Attention is all you need.Advances in neural information processing systems,

work page 2017
[14]

Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding.arXiv preprint arXiv:2412.10302. Xu,Y.;Bao,R.;Zhang,L.;Wang,J.;andWang,S.2025.Em- bodiedintelligenceinRO/ROlogisticterminal:Autonomous intelligent transportation robot architecture.Science China Information Sciences, 68(5): 1–17. Ye,A.;Zhang,Z.;Wang,B.;Wang,X.;Zh...

arXiv 2025

Show all 19 references

[15]

Yu, T.; Wang, Z.; Wang, C.; Huang, F.; Ma, W.; He, Z.; Cai, T.; Chen, W.; Huang, Y.; Zhao, Y.; et al

Vla-r1: Enhancing reasoning in vision-language- action models.arXiv preprint arXiv:2510.01623. Yu, T.; Wang, Z.; Wang, C.; Huang, F.; Ma, W.; He, Z.; Cai, T.; Chen, W.; Huang, Y.; Zhao, Y.; et al
[16]

Zhang, Z.; Zhang, A.; Li, M.; Zhao, H.; Karypis, G.; and Smola, A

Minicpm- v 4.5: Cooking efficient mllms via architecture, data, and training recipe.arXiv preprint arXiv:2509.18154. Zhang, Z.; Zhang, A.; Li, M.; Zhao, H.; Karypis, G.; and Smola, A

arXiv
[17]

Zhao, B.; Wu, B.; He, M.; and Huang, T

Multimodal chain-of-thought reasoning in language models.arXiv preprint arXiv:2302.00923. Zhao, B.; Wu, B.; He, M.; and Huang, T

arXiv
[18]

Zhou, C.; Zhang, M.; Chen, P.; Fu, C.; Shen, Y.; Zheng, X.; Sun, X.; and Ji, R

Svit: Scaling up visual instruction tuning.arXiv preprint arXiv:2307.04087. Zhou, C.; Zhang, M.; Chen, P.; Fu, C.; Shen, Y.; Zheng, X.; Sun, X.; and Ji, R

arXiv
[19]

Vega: Learning interleaved image- text comprehension in vision-language large models.arXiv preprint arXiv:2406.10228

arXiv
[30]

Y.; Li, Z.; Liu, Q.; Liu, X.; Ma, M

Wang, F.; Fu, X.; Huang, J. Y.; Li, Z.; Liu, Q.; Liu, X.; Ma, M. D.; Xu, N.; Zhou, W.; Zhang, K.; et al. 2024a. Muir- bench: A comprehensive benchmark for robust multi-image understanding.arXiv preprint arXiv:2406.09411. Wang,W.;Gao,Z.;Gu,L.;Pu,H.;Cui,L.;Wei,X.;Liu,Z.; Jing,L....

2025 arXiv
[2021]

Gentile,C.;andWarmuth,M.K.1998

Simcse: Simple con- trastive learning of sentence embeddings.arXiv preprint arXiv:2104.08821. Gentile,C.;andWarmuth,M.K.1998. Linearhingelossand average margin.Advances in neural information processing systems,

1998 arXiv
[2023]

Duan, H.; Yang, J.; Qiao, Y.; Fang, X.; Chen, L.; Liu, Y.; Dong, X.; Zang, Y.; Zhang, P.; Wang, J.; et al

Rephrase and respond: Let large language models ask better questions for themselves.arXiv preprint arXiv:2311.04205. Duan, H.; Yang, J.; Qiao, Y.; Fang, X.; Chen, L.; Liu, Y.; Dong, X.; Zang, Y.; Zhang, P.; Wang, J.; et al

arXiv
[2024]

Ahn, M.; Brohan, A.; Brown, N.; Chebotar, Y.; Cortes, O.; David, B.; Finn, C.; Fu, C.; Gopalakrishnan, K.; Hausman, K.;etal.2022

Pixtral 12B.arXiv preprint arXiv:2410.07073. Ahn, M.; Brohan, A.; Brown, N.; Chebotar, Y.; Cortes, O.; David, B.; Finn, C.; Fu, C.; Gopalakrishnan, K.; Hausman, K.;etal.2022. Doasican,notasisay:Groundinglanguage in robotic affordances.arXiv preprint arXiv:2204.01691. Bai, S.; ...

2022 arXiv
[2025]

arXiv:2509.06266

Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes. arXiv:2509.06266. Hendel, R.; Geva, M.; and Globerson, A
[2026]

Tian, X.; Zou, S.; Yang, Z.; and Zhang, J

RoboBrain 2.5: Depth in Sight, Time in Mind.arXiv preprint arXiv:2601.14352. Tian, X.; Zou, S.; Yang, Z.; and Zhang, J

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [3]

InProceedings of the 32nd ACM international conference on multimedia, 11198–11201

Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. InProceedings of the 32nd ACM international conference on multimedia, 11198–11201. Fu,C.;Chen,P.;Shen,Y.;Qin,Y.;Zhang,M.;Lin,X.;Yang, J.;Zheng,X.;Li,K.;Sun,X.;etal.2025a. Mme:Acompre- hensiveevaluationbenchmarkformultimodallargelanguage models. InThe Thirty-ninth Annual Confere...

arXiv 2024

[2] [6]

InFindings of the Association for Computational Linguistics: EMNLP 2023, 9318–9333

In-context learning creates task vectors. InFindings of the Association for Computational Linguistics: EMNLP 2023, 9318–9333. Huang,W.;Jia,B.;Zhai,Z.;Cao,S.;Ye,Z.;Zhao,F.;Xu,Z.; Hu, Y.; and Lin, S

work page 2023

[3] [7]

Hudson,D.A.;andManning,C.D.2019.Gqa:Anewdataset for real-world visual reasoning and compositional question answering

Vision-r1: Incentivizing reason- ing capability in multimodal large language models.arXiv preprint arXiv:2503.06749. Hudson,D.A.;andManning,C.D.2019.Gqa:Anewdataset for real-world visual reasoning and compositional question answering. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6700–6709. Hussain,M.2023. YOLO-v1toY...

arXiv 2019

[4] [8]

Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A

Openvla: An open-source vision-language- action model.arXiv preprint arXiv:2406.09246. Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.-Y.; et al

arXiv

[5] [9]

InProceedings of the IEEE/CVF international conference on computer vision, 4015–4026

Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, 4015–4026. Kong, L.; Wang, H.; Mu, W.; Du, Y.; Zhuang, Y.; Zhou, Y.; Song,Y.;Zhang,R.;Wang,K.;andZhang,C.2024.Aligning largelanguagemodelswithrepresentationediting:Acontrol perspective.Advances in Neural Information Processing Systems, 37: 37356–37384. Lai,Z.;Zhan...

work page 2024

[6] [10]

InEu- ropean Conference on Computer Vision,111–127.Springer

Veclip: Improving clip training via visual-enriched captions. InEu- ropean Conference on Computer Vision,111–127.Springer. Liang,J.;Huang,W.;Xia,F.;Xu,P.;Hausman,K.;Ichter,B.; Florence,P.;andZeng,A.2023. Codeaspolicies:Language model programs for embodied control. In2023 IEEE In- ternational conference on robotics and automation (ICRA), 9493–9500. IEEE. L...

arXiv 2023

[7] [12]

Vaswani,A.;Shazeer,N.;Parmar,N.;Uszkoreit,J.;Jones,L.; Gomez,A.N.;Kaiser,Ł.;andPolosukhin,I.2017

The geometry of hidden repre- sentations of large transformer models.Advances in Neural Information Processing Systems, 36: 51234–51252. Vaswani,A.;Shazeer,N.;Parmar,N.;Uszkoreit,J.;Jones,L.; Gomez,A.N.;Kaiser,Ł.;andPolosukhin,I.2017. Attention is all you need.Advances in neural information processing systems,

work page 2017

[8] [14]

Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding.arXiv preprint arXiv:2412.10302. Xu,Y.;Bao,R.;Zhang,L.;Wang,J.;andWang,S.2025.Em- bodiedintelligenceinRO/ROlogisticterminal:Autonomous intelligent transportation robot architecture.Science China Information Sciences, 68(5): 1–17. Ye,A.;Zhang,Z.;Wang,B.;Wang,X.;Zh...

arXiv 2025