Pith. sign in

REVIEW 5 major objections 6 minor 48 references

This paper claims that a robot can fold cloth from spoken, multi-step commands by pairing an LLM that decomposes the instruction into atomic pick-and-place steps with a vision-language module that points to the exact fold locations, and tha

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A robot folds cloth from spoken language by decomposing instructions with GPT-4o and grounding each step with a SigLIP2-based pick-and-place perception module.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Reasonable integration of known pieces, but the headline 'unseen task' result sits on an undefined protocol, so the strongest claim isn't verifiable as written. the 5 major comments →

arxiv 2509.02324 v1 pith:UN4RBFFI submitted 2025-09-02 cs.RO

Language-Guided Long Horizon Manipulation with LLM-based Planning and Visual Perception

Changshi Zhou , Haichuan Xu , Ningquan Gu , Zhipeng Wang , Bin Cheng , Pengpeng Zhang , Yanchao Dong , Mitsuhiro Hayashibe
show 2 more authors
Yanmin Zhou Bin He
This is my paper
classification cs.RO
keywords deformable object manipulationlong-horizon manipulationlanguage-guided planningcloth foldingvision-language modelLLM-based task planningcross-attention fusionSigLIP2
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a robot can fold cloth through multi-step natural-language commands by splitting the job between an LLM that decomposes the instruction into atomic pick-and-place steps and a vision-language module that grounds each step in pixel-level heatmaps. The framework is tested in simulation and on a real robot, and the paper reports that it beats two prior language-guided cloth-folding methods on seen instructions, unseen phrasings, and entirely unseen tasks. The largest reported margin is in unseen tasks: a success rate of 77.9 percent versus 44.6 percent for the strongest baseline. If true, the result is evidence that composing symbolic planning with fine-grained visual grounding is a workable path for deformable-object manipulation, not just rigid pick-and-place.

Core claim

The central claim is that the proposed closed loop of speech transcription, LLM-based task decomposition, VLM-based pick/place grounding, and a robot executor can perform multi-step cloth folding from high-level commands and generalize beyond the exact instructions and tasks it trained on. The perception core is a frozen SigLIP2 encoder paired with a bidirectional cross-attention fusion module: the instruction is split at 'and' into pick and place segments, each segment attends to the shared visual features and vice versa, and two convolutional decoders emit heatmaps for the pick and place points. DoRA fine-tuning adapts the frozen encoder to cloth without full retraining. The paper reports

What carries the argument

The load-bearing object is the bidirectional cross-attention fusion module. It splits each sub-task instruction at the word 'and' so the pick decoder gets its own language segment and the place decoder gets the other, then runs image-to-language and language-to-image attention in parallel, concatenates the two outputs, and adds a residual connection from the image features. This forces the pick and place predictions to be conditioned on distinct parts of the instruction while sharing the same visual observation. A frozen SigLIP2 encoder provides the visual and textual tokens, DoRA supplies the parameter-efficient adaptation, and the final pick and place locations are the argmax of sigmoid he

Load-bearing premise

The paper assumes its 'seen vs unseen' split of instructions and tasks is a clean held-out protocol and that converting the bimanual baseline to a single-arm setup does not handicap it; if either assumption is wrong, the headline 33.3-point unseen-task improvement is not established.

What would settle it

Take the list of training demonstrations and the list of unseen-task test cases: if any test cloth type or any high-level instruction template appears in the training demonstrations, the unseen-task numbers are invalid. Separately, rerun the baseline in its original bimanual form on the same unseen tasks; if its success rate rises substantially, the single-arm conversion, not the framework, drove the reported margin.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A single natural-language command can drive a full multi-step fold, because the LLM planner converts it into a sequence of primitive actions the robot already knows how to execute.
  • Generalization to new phrasings is handled linguistically by the planner, so users can reword commands without retraining the perception model.
  • The pick/place heatmap design transfers from simulation to a real robot: the same SigLIP2 and cross-attention model trained in SoftGym works on a UR5 with a depth camera.
  • Unseen garment categories and folds are within reach: the reported unseen-task gains are largest exactly where the baselines collapse, on trouser and T-shirt folds.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the architecture separates semantic decomposition from visual grounding, the LLM planner could likely be swapped for a smaller local model without retraining the perception module; the coupling is through text, not weights, though the paper only tests GPT-4o.
  • The perception module is trained entirely on heatmap supervision in SoftGym, so the same data collection pipeline likely extends to other deformable objects such as towels, shorts, and blankets, and to symmetric two-handed folds if the 'and' split is generalized beyond two segments.
  • The 33.3-point unseen-task margin may partly reflect baseline weakness on multi-step folds rather than a property of the framework; a cleaner test would give the baselines the same LLM planner and compare only the grounding modules, isolating the perception contribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper presents Talk2Fold, a framework for language-guided multi-step cloth folding that couples an LLM-based task planner (GPT-4o) with a frozen SigLIP2 vision-language encoder adapted via DoRA, a bidirectional cross-attention fusion mechanism, and two convolutional decoders that output pick and place heatmaps. The planner decomposes high-level instructions into atomic pick-and-place sub-tasks; the perception module grounds each sub-task in RGB-D observations. The method is evaluated in SoftGym simulation on five folding tasks and on a UR5 real robot, under seen instructions, unseen instructions, and unseen tasks, reporting average SR gains over Deng et al. and Barbany et al. baselines, plus ablations of backbones, fusion, fine-tuning, and LLM choices.

Significance. If the reported results are correct, the paper would provide a useful blueprint for combining LLM planning with VLM grounding for deformable-object manipulation, and the simulation/real-robot studies cover a broad task set. The ablation study is fairly extensive (three backbones, two fusion designs, three PEFT methods, four LLMs), and the perception module's explicit pick/place branch segmentation is a reasonable design. However, the main generalization claim is currently not verifiable because the 'unseen task' condition is undefined, and there are internal numerical inconsistencies in the results table. The real-world evaluation is too underpowered to support the strength of the claims. As such, the significance is conditional on the authors providing the missing experimental protocol and corrections.

major comments (5)
  1. [Section IV-B and Table I] The 'unseen task' (UT) condition is never defined. Section IV-B lists three instruction conditions but gives no protocol for splitting tasks/instructions into SI/UI/UT; Table I reports UT rows for all five tasks, so UT is not a whole-task holdout. Section III-E gives only aggregate demonstration counts (15,000/750), no per-task or per-instruction breakdown. The headline 33.3-point UT gain (abstract) therefore cannot be verified as a held-out generalization result. Please specify the split rule and report training/test composition, and ensure the same protocol is used in the real-world UT condition.
  2. [Section IV-B and Table I] In the UT row for Deng et al. under DTF, SR=0 while MPD=0.0100 m, which is below the success threshold of 0.0125 m defined in Section IV-B. This is internally inconsistent: either the threshold is not applied as defined or the table entry is wrong. The same inconsistency may affect other cells; please audit Table I against the stated metric.
  3. [Section IV-C] The Barbany et al. baseline is described only as 'adapted to a single-arm variant' with no details. Since BIFOLD is a bimanual method, this adaptation may handicap it; no hyperparameters, retraining procedure, or validation of the adapted baseline are given. As the main SOTA comparison relies on this baseline, please document the adaptation and, if possible, compare with the original bimanual method or a second single-arm baseline.
  4. [Section IV-D and Table I] The text claims 'the highest performance across all three metrics... consistently surpassing existing baselines.' Table I contains counterexamples, e.g., FCIF under UI: Deng 98.0 SR/0.0046 MPD/95.06 MIoU and Barbany 100.0/0.0031/96.20 vs. ours 97.0/0.0062/92.96. The averages support only a limited claim; please report per-condition averages and significance tests, and qualify the superiority claim accordingly.
  5. [Section V-B and Table V] Real-world results use six trials per task, so SR is a multiple of 16.7%; no confidence intervals, error bars, or per-trial data are given. The success criterion (MIoU > 0.8 against a human-demonstrated reference) is not operationalized (e.g., who labels the reference, how many references per task). The real-world generalization claim (SR 88/84/68 vs 84/68/24) is therefore not statistically supported. Please report more trials or intervals and describe the reference protocol.
minor comments (6)
  1. [Abstract] '2.23, 1.87, and 33.3' are percentage-point differences (Table I: 91.70-89.47=2.23, etc.); state 'percentage points' or give relative improvements to avoid overstating.
  2. [Section IV-B] 'Double Straight Fold (TSF)' and 'T-Shirt Fold (TSF)' use the same abbreviation; rename to DSF and TSF.
  3. [Section V-A] '1, m × 1, m workspace' should read '1 m × 1 m'.
  4. [Figure 4 caption] 'See Instructions' should be 'Seen Instructions'.
  5. [Section V-B and Figure 8] Section V-B references Figure 8 for atomic action prediction, but Figure 8 is captioned 'Fabric Categories Used for Model Training and Testing'; the cross-reference appears to be wrong. Also, the caption says 18 samples per fabric while Section V-A says 12 clothes; please reconcile.
  6. [Section III-C] The split of language features at the conjunction 'and' is not defined for instructions that do not contain 'and' or that contain more than two clauses; please state how the planner standardizes sub-task wording.

Circularity Check

0 steps flagged

No significant circularity: the framework is evaluated against external SoftGym ground-truth states and reproduced baselines, and no prediction reduces by construction to the model's own outputs.

full rationale

The paper's derivation chain is self-contained and externally grounded. The visual perception module is trained with a binary cross-entropy loss (Eqs. 14-15) on expert demonstration heatmaps, and success is measured by geometric metrics (MPD, MIoU, SR) computed against ground-truth cloth configurations in SoftGym, not by any quantity derived from the model's own predictions. The LLM planner is an off-the-shelf GPT-4o prompted to decompose instructions; it is not fine-tuned on the test folds, and Table IV is an external comparison of LLM planning accuracy using expert annotations. Baselines from Deng et al. and Barbany et al. are reproduced and retrained on the same dataset, so the comparison is not a fitted-input artifact. The only self-citation, reference [4] in the introduction, is an example of rigid-object manipulation and is not load-bearing. The undefined SI/UI/UT split in Section IV-B is a legitimate experimental-reporting and verifiability concern, but it is not a circularity: even if the split is poorly defined, the reported success numbers are still measured against external physical targets, not against quantities that are equivalent by construction to the model's inputs. No step in the derivation equates a predicted quantity with a fitted parameter, imports a uniqueness theorem from the authors' prior work, or smuggles an ansatz in via self-citation. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claim rests on standard machine-learning assumptions (expert demonstrations are representative, sim-to-real transfer is valid, frozen backbone is adaptable) and on hand-chosen success thresholds. No new physical entities are introduced. The main free parameters are evaluation thresholds and the 'and' split rule.

free parameters (4)
  • Simulation success threshold (MPD) = 0.0125 m
    Hand-chosen threshold below which a simulation trial counts as a success (Section IV-B); the central SR metric depends on it.
  • Real-world success threshold (MIoU) = 0.8
    Hand-chosen overlap threshold against a human-demonstrated reference; a lenient threshold makes real-world SR easier to attain (Section V-B).
  • Instruction split token = and
    The perception module splits sub-task instructions at 'and' to define pick and place branches (Section III-C); this hand-chosen rule shapes the fusion and affects results.
  • DoRA rank = not reported
    The rank of the low-rank adaptation is a capacity-controlling hyperparameter; the paper does not state its value, which limits replication.
axioms (4)
  • domain assumption SoftGym/FleX cloth physics is a faithful proxy for real cloth dynamics
    Simulation results are used as evidence for effectiveness; the sim-to-real transfer claim depends on this assumption (Section IV-A).
  • domain assumption GPT-4o decomposes instructions into robot-executable primitives reliably
    The planning module is not fine-tuned; the whole pipeline assumes the closed-source LLM's decomposition is correct (Section III-B).
  • domain assumption Grounding DINO + SAM correctly segment the garment before grounding
    The perception pipeline crops the image based on detector masks; failures here would propagate to pick/place prediction (Section III-C).
  • domain assumption SigLIP2 frozen features retain fine-grained cloth localization after DoRA tuning
    The method relies on frozen SigLIP2 embeddings being adaptable to dense cloth pick/place prediction (Section III-C).

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Language-Guided Long Horizon Manipulation with LLM-based Planning and Visual Perception." pith.science (2026). https://pith.science/paper/UN4RBFFI

@misc{pith2026250902324,
  author       = {Pith},
  title        = {Pith review of: Language-Guided Long Horizon Manipulation with LLM-based Planning and Visual Perception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UN4RBFFI}},
  note         = {Machine review of arXiv:2509.02324}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Language-guided long-horizon manipulation of deformable objects presents significant challenges due to high degrees of freedom, complex dynamics, and the need for accurate vision-language grounding. In this work, we focus on multi-step cloth folding, a representative deformable-object manipulation task that requires both structured long-horizon planning and fine-grained visual perception. To this end, we propose a unified framework that integrates a Large Language Model (LLM)-based planner, a Vision-Language Model (VLM)-based perception system, and a task execution module. Specifically, the LLM-based planner decomposes high-level language instructions into low-level action primitives, bridging the semantic-execution gap, aligning perception with action, and enhancing generalization. The VLM-based perception module employs a SigLIP2-driven architecture with a bidirectional cross-attention fusion mechanism and weight-decomposed low-rank adaptation (DoRA) fine-tuning to achieve language-conditioned fine-grained visual grounding. Experiments in both simulation and real-world settings demonstrate the method's effectiveness. In simulation, it outperforms state-of-the-art baselines by 2.23, 1.87, and 33.3 on seen instructions, unseen instructions, and unseen tasks, respectively. On a real robot, it robustly executes multi-step folding sequences from language instructions across diverse cloth materials and configurations, demonstrating strong generalization in practical scenarios. Project page: https://language-guided.netlify.app/

Figures

Figures reproduced from arXiv: 2509.02324 by Bin Cheng, Bin He, Changshi Zhou, Haichuan Xu, Mitsuhiro Hayashibe, Ningquan Gu, Pengpeng Zhang, Yanchao Dong, Yanmin Zhou, Zhipeng Wang.

Figure 1
Figure 1. Figure 1: Method Overview. An illustration of the robotic-arm embodied LLM system in the real world, showcasing the integrated workflow of Automatic Speech Recognition, Task Planning, Visual Perception, and Action Execution in a cloth manipulation task. action, demonstrating robustness across diverse cloth configu￾rations. In summary, our contributions are as follows: • We propose a unified framework for language-gu… view at source ↗
Figure 2
Figure 2. Figure 2: Task Decomposition. Example of task decomposition from a high￾level instruction using LLM-based planner. central challenge lies in grounding sequential natural language instructions into accurate and reliable robotic actions, despite the highly deformable nature of cloth. Algorithm 1 outlines the core procedure of our proposed framework. B. Task Planning Module The Task Planning Module MT addresses the cha… view at source ↗
Figure 3
Figure 3. Figure 3: Architecture of the Visual Perception Module. The Visual Perception module uses a frozen SigLIP2 model to extract tokens from an RGB-D image and a natural language instruction. The instruction is split at the conjunction “and” into pick and place segments. Each segment is fused with visual tokens via bidirectional cross-attention, where textual and visual features are jointly aligned. To adapt the frozen S… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Simulation Results on Five Folding Tasks with See Instructions. Each task is represented in two rows: the top row shows the top-view action sequence captured by the overhead camera, and the bottom row displays the network-predicted pick and place heatmaps. fair comparison, we reproduce both baselines and train them on same dataset as ours, following their respective fine-tuning and model updati… view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results on unseen tasks in simulation. Each row shows a multi-step folding sequence for a different cloth type, performed under the unseen task setting [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Experimental Setup. Real-world environment and cloth categories used for model testing [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Fabric Categories Used for Model Training and Testing. The dataset includes 18 samples from each of three fabric types: towels, T-shirts, and shorts. Gripper through hand–eye calibration, inverse kinematics, and URScript-based motion primitives. The robot operates on a 1, m × 1, m workspace covered with a rigid polypropylene (PP) board, which prevents collisions that could damage the arm. An Intel RealSens… view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative Real-World Results on Five Folding Tasks. Each task is visualized in three rows: the top row displays the user-provided voice instruction, the middle row shows the subtask breakdown generated by the LLM-based planner, and the bottom row illustrates the robot performing actions according to the decomposed commands. Tasks. We report Success Rate (SR), based on the proportion of successful trials,… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

48 extracted references · 28 canonical work pages · 1 internal anchor

  1. [1]

    Embodied large language models enable robots to complete complex tasks in unpredictable environments,

    R. Mon-Williams, G. Li, R. Long, W. Du, and C. G. Lucas, “Embodied large language models enable robots to complete complex tasks in unpredictable environments,” Nature Machine Intelligence , pp. 1–10, 2025

    work page 2025

  2. [2]

    A grasping-centered analysis for cloth manipulation,

    J. Borr `as, G. Aleny `a, and C. Torras, “A grasping-centered analysis for cloth manipulation,” IEEE Transactions on Robotics , vol. 36, no. 3, pp. 924–936, 2020

    work page 2020

  3. [3]

    Equivact: Sim(3)-equivariant visuomotor policies beyond rigid object manipulation,

    J. Yang, C. Deng, J. Wu, R. Antonova, L. Guibas, and J. Bohg, “Equivact: Sim(3)-equivariant visuomotor policies beyond rigid object manipulation,” in 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024, pp. 9249–9255

    work page 2024

  4. [4]

    Tactilealoha: Learning bimanual manipulation with tactile sensing,

    N. Gu, K. Kosuge, and M. Hayashibe, “Tactilealoha: Learning bimanual manipulation with tactile sensing,” IEEE Robotics and Automation Letters, 2025

    work page 2025

  5. [5]

    Unigarmentmanip: A unified framework for category-level garment manipulation via dense visual correspondence,

    R. Wu, H. Lu, Y . Wang, Y . Wang, and H. Dong, “Unigarmentmanip: A unified framework for category-level garment manipulation via dense visual correspondence,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 16 340–16 350

    work page 2024

  6. [6]

    Soundmind: Rl-incentivized logic reasoning for audio-language models,

    X. Diao, C. Zhang, K. Kong, W. Wu, C. Ma, Z. Ouyang, P. Qing, S. V osoughi, and J. Gui, “Soundmind: Rl-incentivized logic reasoning for audio-language models,” arXiv preprint arXiv:2506.12935 , 2025

    arXiv 2025

  7. [7]

    Encoder: Entity mining and modification relation binding for composed image retrieval,

    Z. Li, Z. Chen, H. Wen, Z. Fu, Y . Hu, and W. Guan, “Encoder: Entity mining and modification relation binding for composed image retrieval,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 5, 2025, pp. 5101–5109. 10

    work page 2025

  8. [8]

    Cliport: What and where pathways for robotic manipulation,

    M. Shridhar, L. Manuelli, and D. Fox, “Cliport: What and where pathways for robotic manipulation,” in Conference on robot learning . PMLR, 2022, pp. 894–906

    work page 2022

  9. [9]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control,

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid et al. , “Rt-2: Vision-language-action models transfer web knowledge to robotic control,” in Conference on Robot Learning. PMLR, 2023, pp. 2165–2183

    2023

  10. [10]

    Vision-language foundation models as effective robot imitators,

    X. Li, M. Liu, H. Zhang, C. Yu, J. Xu, H. Wu, C. Cheang, Y . Jing, W. Zhang, H. Liu et al., “Vision-language foundation models as effective robot imitators,” in ICLR, 2024

    work page 2024

  11. [11]

    Generalizable clothes manipulation with large language model,

    Y . Deng and D. Hsu, “Generalizable clothes manipulation with large language model,” in 2024 ICRA Workshop on Representing and Manip- ulating Deformable Objects , 2024

    work page 2024

  12. [12]

    Bifold: Bimanual cloth folding with language guidance,

    O. Barbany, A. Colom ´e, and C. Torras, “Bifold: Bimanual cloth folding with language guidance,” arXiv preprint arXiv:2501.16458 , 2025

    Pith/arXiv arXiv 2025

  13. [13]

    Lip-llm: Inte- grating linear programming and dependency graph with large language models for multi-robot task planning,

    K. Obata, T. Aoki, T. Horii, T. Taniguchi, and T. Nagai, “Lip-llm: Inte- grating linear programming and dependency graph with large language models for multi-robot task planning,” IEEE Robotics and Automation Letters, 2024

    work page 2024

  14. [14]

    Hiddendetect: Detecting jailbreak attacks against large vision-language models via monitoring hidden states,

    Y . Jiang, X. Gao, T. Peng, Y . Tan, X. Zhu, B. Zheng, and X. Yue, “Hiddendetect: Detecting jailbreak attacks against large vision-language models via monitoring hidden states,” arXiv preprint arXiv:2502.14744, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Llava steering: Visual instruction tuning with 500x fewer parame- ters through modality linear representation-steering,

    J. Bi, Y . Wang, H. Chen, X. Xiao, A. Hecker, V . Tresp, and Y . Ma, “Llava steering: Visual instruction tuning with 500x fewer parame- ters through modality linear representation-steering,” arXiv preprint arXiv:2412.12359, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Temporal working memory: Query-guided segment re- finement for enhanced multimodal understanding,

    X. Diao, C. Zhang, W. Wu, Z. Ouyang, P. Qing, M. Cheng, S. V osoughi, and J. Gui, “Temporal working memory: Query-guided segment re- finement for enhanced multimodal understanding,” arXiv preprint arXiv:2502.06020, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Finecir: Explicit parsing of fine-grained modification semantics for composed image retrieval,

    Z. Li, Z. Fu, Y . Hu, Z. Chen, H. Wen, and L. Nie, “Finecir: Explicit parsing of fine-grained modification semantics for composed image retrieval,” https://arxiv.org/abs/2503.21309, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features,

    M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdul- mohsin, N. Parthasarathy, T. Evans, L. Beyer, Y . Xia, B. Mustafa et al. , “Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features,” arXiv preprint arXiv:2502.14786, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Dora: Weight-decomposed low-rank adap- tation,

    S.-Y . Liu, C.-Y . Wang, H. Yin, P. Molchanov, Y .-C. F. Wang, K.-T. Cheng, and M.-H. Chen, “Dora: Weight-decomposed low-rank adap- tation,” in Forty-first International Conference on Machine Learning , 2024

    work page 2024

  20. [20]

    Mmap: Multi-modal alignment prompt for cross-domain multi-task learning,

    Y . Xin, J. Du, Q. Wang, K. Yan, and S. Ding, “Mmap: Multi-modal alignment prompt for cross-domain multi-task learning,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 14, 2024, pp. 16 076–16 084

    work page 2024

  21. [21]

    Gaussiangrasper: 3d language gaussian splatting for open-vocabulary robotic grasping,

    Y . Zheng, X. Chen, Y . Zheng, S. Gu, R. Yang, B. Jin, P. Li, C. Zhong, Z. Wang, L. Liu et al., “Gaussiangrasper: 3d language gaussian splatting for open-vocabulary robotic grasping,” IEEE Robotics and Automation Letters, 2024

    work page 2024

  22. [22]

    Tinyvla: Towards fast, data-efficient vision-language- action models for robotic manipulation,

    J. Wen, Y . Zhu, J. Li, M. Zhu, Z. Tang, K. Wu, Z. Xu, N. Liu, R. Cheng, C. Shen et al. , “Tinyvla: Towards fast, data-efficient vision-language- action models for robotic manipulation,” IEEE Robotics and Automation Letters, 2025

    work page 2025

  23. [23]

    Naturalvlm: Leveraging fine-grained natural language for affordance-guided visual manipula- tion,

    R. Xu, Y . Shen, X. Li, R. Wu, and H. Dong, “Naturalvlm: Leveraging fine-grained natural language for affordance-guided visual manipula- tion,” IEEE Robotics and Automation Letters , 2024

    work page 2024

  24. [24]

    Screencoder: Advancing visual-to-code generation for front- end automation via modular multimodal agents,

    Y . Jiang, Y . Zheng, Y . Wan, J. Han, Q. Wang, M. R. Lyu, and X. Yue, “Screencoder: Advancing visual-to-code generation for front- end automation via modular multimodal agents,” arXiv preprint arXiv:2507.22827, 2025

    arXiv 2025

  25. [25]

    Prism: Self-pruning intrinsic selection method for training-free multimodal data selection,

    J. Bi, Y . Wang, D. Yan, X. Xiao, A. Hecker, V . Tresp, and Y . Ma, “Prism: Self-pruning intrinsic selection method for training-free multimodal data selection,” arXiv preprint arXiv:2502.12119 , 2025

    Pith/arXiv arXiv 2025

  26. [26]

    Learning Generalizable Language-Conditioned Cloth Manipulation from Long Demonstrations

    H. Zhao, J. Zhu, Z. Yan, Y . Li, Y . Deng, and X. Wang, “Learn- ing generalizable language-conditioned cloth manipulation from long demonstrations,” arXiv preprint arXiv:2503.04557 , 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  27. [27]

    Language-guided dexterous functional grasping by llm generated grasp functionality and synergy for humanoid manipulation,

    Z. Li, J. Liu, Z. Li, Z. Dong, T. Teng, Y . Ou, D. Caldwell, and F. Chen, “Language-guided dexterous functional grasping by llm generated grasp functionality and synergy for humanoid manipulation,” IEEE Transac- tions on Automation Science and Engineering , 2025

    work page 2025

  28. [28]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24 824–24 837, 2022

    2022

  29. [29]

    Text2motion: From natural language instructions to feasible plans,

    K. Lin, C. Agia, T. Migimatsu, M. Pavone, and J. Bohg, “Text2motion: From natural language instructions to feasible plans,” Autonomous Robots, vol. 47, no. 8, pp. 1345–1365, 2023

    2023

  30. [30]

    Learning to follow and generate instructions for language-capable navigation,

    X. Wang, W. Wang, J. Shao, and Y . Yang, “Learning to follow and generate instructions for language-capable navigation,” IEEE Transac- tions on Pattern Analysis and Machine Intelligence , vol. 46, no. 5, pp. 3334–3350, 2023

    work page 2023

  31. [31]

    Mesh-based dynamics with occlusion reasoning for cloth manipulation,

    Z. Huang, X. Lin, and D. Held, “Mesh-based dynamics with occlusion reasoning for cloth manipulation,” in Robotics: Science and Systems (RSS), 2022

    work page 2022

  32. [32]

    Differentiable cloth parameter identification and state estimation in manipulation,

    D. Zheng, S. Yao, W. Xu, and C. Lu, “Differentiable cloth parameter identification and state estimation in manipulation,” IEEE Robotics and Automation Letters, vol. 9, no. 3, pp. 2519–2526, 2024

    work page 2024

  33. [33]

    Imitating tool-based garment folding from a single visual observation using hand-object graph dynamics,

    P. Zhou, J. Qi, A. Duan, S. Huo, Z. Wu, and D. Navarro-Alarcon, “Imitating tool-based garment folding from a single visual observation using hand-object graph dynamics,” IEEE Transactions on Industrial Informatics, vol. 20, no. 4, pp. 6245–6256, 2024

    work page 2024

  34. [34]

    Folding clothes autonomously: A complete pipeline,

    A. Doumanoglou, J. Stria, G. Peleka, I. Mariolis, V . Petrik, A. Kargakos, L. Wagner, V . Hlav´aˇc, T.-K. Kim, and S. Malassiotis, “Folding clothes autonomously: A complete pipeline,” IEEE Transactions on Robotics , vol. 32, no. 6, pp. 1461–1478, 2016

    work page 2016

  35. [35]

    Rl-vlm-f: reinforcement learning from vision language foundation model feedback,

    Y . Wang, Z. Sun, J. Zhang, Z. Xian, E. Biyik, D. Held, and Z. Erickson, “Rl-vlm-f: reinforcement learning from vision language foundation model feedback,” in Proceedings of the 41st International Conference on Machine Learning , 2024, pp. 51 484–51 501

    work page 2024

  36. [36]

    Diffusion dynamics models with generative state estimation for cloth manipulation,

    T. Tian, H. Li, B. Ai, X. Yuan, Z. Huang, and H. Su, “Diffusion dynamics models with generative state estimation for cloth manipulation,” arXiv preprint arXiv:2503.11999, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Metafold: Language-guided multi-category garment fold- ing framework via trajectory generation and foundation model,

    H. Chen, J. Li, R. Wu, Y . Liu, Y . Hou, Z. Xu, J. Guo, C. Gao, Z. Wei, S. Xu et al., “Metafold: Language-guided multi-category garment fold- ing framework via trajectory generation and foundation model,” arXiv preprint arXiv:2503.08372, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection,

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su et al. , “Grounding dino: Marrying dino with grounded pre-training for open-set object detection,” in European conference on computer vision. Springer, 2024, pp. 38–55

    work page 2024

  39. [39]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Loet al., “Segment anything,” in Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 4015–4026

    2023

  40. [40]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” in International conference on machine learning . PMLR, 2023, pp. 19 730–19 742

    2023

  41. [41]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PmLR, 2021, pp. 8748–8763

    2021

  42. [42]

    Softgym: Benchmarking deep reinforcement learning for deformable object manipulation,

    X. Lin, Y . Wang, J. Olkin, and D. Held, “Softgym: Benchmarking deep reinforcement learning for deformable object manipulation,” in Conference on Robot Learning . PMLR, 2021, pp. 432–448

    work page 2021

  43. [43]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations , 2022

    work page 2022

  44. [44]

    Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,

    H. Liu, D. Tam, M. Muqeeth, J. Mohta, T. Huang, M. Bansal, and C. A. Raffel, “Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 1950–1965, 2022

    1950

  45. [45]

    Doubao-1.5-pro,

    D. Team, “Doubao-1.5-pro,” 2025

    work page 2025

  46. [46]

    Grok 3: Advancing real-time reasoning in AI,

    xAI, “Grok 3: Advancing real-time reasoning in AI,” Tech. Rep., 2025

    work page 2025

  47. [47]

    Deepseek-v3 technical report,

    A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan et al., “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437, 2024

    Pith/arXiv arXiv 2024

  48. [48]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford et al., “Gpt-4o system card,” arXiv preprint arXiv:2410.21276 , 2024

    Pith/arXiv arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.