Pith. sign in

REVIEW 4 major objections 6 minor 34 references

A deformable-attention RGB-D fusion Transformer sets a new state-of-the-art in unseen object instance segmentation across tabletop, indoor, and bin-picking scenes.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:04 UTC pith:XAUNEAQQ

load-bearing objection A plausible deformable cross-modal attention idea plus a potentially useful new OCBD benchmark, but the SOTA claim rests on comparison tables that contain demonstrable copy-paste corruption and are therefore currently unsupported. the 4 major comments →

arxiv 2607.17754 v1 pith:XAUNEAQQ submitted 2026-07-20 cs.CV cs.AI

DA-Fusion: Deformable Attention-Based RGB-D Fusion Transformer for Unseen Object Instance Segmentation

Yesol Park , Hye-Jung Yoon , Juno Kim , Byoung-Tak Zhang This is my paper
classification cs.CV cs.AI
keywords unseen object instance segmentationRGB-D fusiondeformable attentionTransformerbin pickingrobotic manipulationOCBD datasetinstance segmentation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

DA-Fusion is built around a simple failure asymmetry: RGB-only segmentation over-splits objects because of texture, while depth-only under-splits by geometry. The paper's fix is bidirectional deformable cross-attention between the two modalities at each of four encoder stages, where each modality's queries learn where to sample the other's features instead of attending everywhere. Because the fusion happens at multiple scales and is driven by learned offsets, the model can align texture evidence from RGB with geometric boundaries from depth in cluttered, multi-layer bins. The authors claim this beats prior unseen-object instance segmentation methods on tabletop, indoor, and bin-picking benchmarks, and their new OCBD dataset supplies a harder real-world bin test. If true, robotic picking gets a perception front-end that needs no second-stage refinement.

Core claim

DA-Fusion is a Transformer architecture for class-agnostic instance segmentation of objects never seen during training. It runs RGB and depth through parallel Swin backbones, refines each with deformable self-attention, then fuses with deformable cross-attention at every encoder stage: the RGB query samples depth features at learned offsets, and the depth query samples RGB features. The four fused feature maps are concatenated and passed to a Mask Transformer decoder to produce instance masks. The paper reports that this design outperforms prior UOIS methods on OSD, OCID, and the new OCBD benchmark, with the largest gains in cluttered top-down bin scenes, and that it works on real robots for

What carries the argument

Deformable Self-Attention (DS) and Deformable Cross-Attention (DC). DS refines each modality's feature map by letting queries sample from a small set of learned deformable reference points, reducing cost while concentrating on informative regions. DC then performs two directed cross-attention operations: RGB queries sample depth keys and values at offsets predicted by the RGB query, and depth queries sample RGB keys and values at offsets predicted by the depth query. This happens at all four backbone stages, and the fused outputs are concatenated and fed to a Mask Transformer decoder that produces per-query foreground masks. The mechanism aligns texture-rich RGB evidence with geometry-rich d

Load-bearing premise

The strongest claim hinges on the accuracy and comparability of the baseline numbers in Tables I and II; if those tables were mis-transcribed, stale, or computed under different protocols, the reported margin over prior work is not established.

What would settle it

Check Table I's OSD block: the MSMFormer RGB-D row duplicates the UOAIS RGB-D row transposed, and one entry reads '88.888.188.4'. Recompute all baselines from their released checkpoints under the paper's training/evaluation protocol, and rerun DA-Fusion with several seeds. If the corrected tables still show DA-Fusion ahead on OCID, OSD, and OCBD, the central claim holds; if the margins vanish or reverse, it does not.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the reported results hold, a single-stage network without zoom-in refinement can outperform two-stage UOIS pipelines, reducing inference complexity for real-time picking.
  • OCBD's top-down, multi-layer bin scenes (13.2 instances per image) give the UOIS community a benchmark that stresses occlusion and lower-layer objects more than existing datasets.
  • Accurate masks for occluded lower-layer objects would improve grasp and suction estimation in bin picking, since grasp planners rely on clean instance boundaries rather than texture clusters.
  • The deformable cross-attention fusion pattern is not tied to RGB and depth specifically; the same querying of sampled features could apply to any paired sensor streams where one provides appearance and the other geometry.
  • Because fusion happens at all four encoder stages, the model can exploit both local geometry from early layers and global layout from late layers when separating stacked objects.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit is ablating the number of deformable sampling points per layer; if a fraction of the points retains most of the gain, the compute budget in robotics could drop further.
  • A testable asymmetry: make fusion one-directional (depth querying RGB only) to see whether the geometry-to-appearance direction or the appearance-to-geometry direction carries the benefit; the paper's bidirectional design currently bundles both.
  • Independently rerunning the baselines under the paper's training and evaluation protocol is the quickest way to determine how much of the reported margin is architectural and how much comes from evaluation setup; this is standard practice but not reported here.
  • If OCBD is adopted, future work might report occlusion-stratified metrics, since the 13.2-instance images make lower-layer instances a distinct failure mode that current aggregate F-measure numbers can hide.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DA-Fusion, a Swin-backboned Transformer for unseen object instance segmentation from RGB-D input. Two parallel branches apply deformable self-attention to RGB and depth features; a deformable cross-attention module fuses the two modalities at each of the four backbone stages, and a Mask2Former-style decoder produces class-agnostic masks. The authors also introduce OCBD, a new 1,000-image real-world top-down bin-picking dataset, and report experiments on OSD, OCID, and OCBD, claiming state-of-the-art results in tabletop, indoor, and bin environments, together with qualitative real-robot shelf- and bin-picking demonstrations.

Significance. If the empirical results were reliable, the contribution would be a useful, incremental architectural improvement for RGB-D UOIS: deformable cross-attention is a sensible mechanism for keeping fusion cost manageable while preserving geometric correspondences, and a real cluttered-bin benchmark such as OCBD would be valuable to the community. The multi-scale fusion design and the new dataset are the strongest assets. However, the manuscript ships no code, no model weights, no dataset release link, and no error bars; the headline SOTA claim rests entirely on Tables I and II, whose baseline rows are demonstrably corrupted. As printed, the central quantitative claim cannot be credited, although the underlying approach is plausible and the issues are in principle fixable with corrected experiments and full protocol transparency.

major comments (4)
  1. [Table I (Section V-C)] The MSMFormer RGB-D row is identical to the UOAIS† RGB-D row with the OCID and OSD blocks interchanged: the OCID entries 70.7, 86.7, 71.9, 68.2, 78.5, 68.8, 78.7 from UOAIS† appear verbatim as the MSMFormer OSD block, and the OSD entries 85.3, 85.4, 85.2, 72.7, 74.3, 73.1, 79.1 appear as the MSMFormer OCID block. This is not a plausible independent result and cannot serve as a baseline. Since the paper's SOTA claim is derived by comparing the Ours rows with these entries, the central empirical comparison is invalid as printed. The authors must replace this row with genuine MSMFormer numbers or rerun under a documented protocol.
  2. [Table II / Section V-B] Table II is introduced as an evaluation on OCBD with RGB-D input, and Section V-C states the comparison is with 'the same set of methods.' In fact, Table II lists only UCN+, UOAIS, and MSMFormer+ as baselines, omitting Mask2Former, MSMFormer, UCN, and other methods from Table I; no justification is given. In addition, OCBD is introduced by the authors and no release URL or access procedure is provided, so the 'bin environment' half of the SOTA claim is measured on a non-independent, non-public benchmark. Please provide complete baseline coverage and a dataset availability statement.
  3. [Section V-A and V-C] No error bars, standard deviations, or number of seeds are reported for any quantitative result. The protocol also does not state whether the baselines were retrained on UOAIS-SIM with the same augmentation/schedule or evaluated from official checkpoints; different training/evaluation protocols can change P/R/F by several points. Because Tables I and II are the sole quantitative support for the headline claim, the authors should report multi-seed statistics and a precise baseline protocol.
  4. [Table I; Section III-A] The central results table contains unreadable numeric strings: in the Ours* RGB row, '88.888.188.4' appears in the OSD Boundary F column; in the Ours (DS) and Ours (DS+DC) rows, '85.488.2' and '91.784.6' appear. These cannot be parsed as metric values. Also, the number of attention heads M and sampled keys K are introduced in Section III-A but never specified in Section V-A, and the offset network θ_offset is not described. Please correct the table and provide all hyperparameters/network details.
minor comments (6)
  1. [Section III-C, Eq. (6)] Define softmax_k and clarify how the relative position bias φ(B;R) is broadcast; the notation R is reused for reference points and recall.
  2. [Section V-A] The Swin backbone variant (Tiny/Small/Base) and the channel widths C_i are not stated, making the architecture description incomplete.
  3. [Section V-E] The real-robot experiments are qualitative only; no success rates or number of trials are given despite the claim that the method improved success rate.
  4. [Section IV] OCBD is described as 1000 top-down RGB-D images, but no train/test split or annotation protocol is specified. Please include split details and annotation statistics.
  5. [Section II-B] Reference [18] is cited for Mask2Former, but the cited paper is 'Per-pixel classification is not all you need'; the relevant Mask2Former paper is [19], which is used later. Please align citations.
  6. [Section V-C] The sentence 'previous methods typically relied on two-stage clustering models [20], [21]' is not supported by Table I and should be softened or substantiated.

Circularity Check

0 steps flagged

No circular derivation: benchmark comparisons are external and no fitted parameter is renamed as prediction; the only self-citation is non-load-bearing, and the self-introduced OCBD affects verifiability, not definitional circularity.

full rationale

DA-Fusion's derivation chain is an architecture plus an empirical comparison. The model is defined by deformable self/cross-attention equations and is trained on UOAIS-SIM with the Mask2Former loss; its reported numbers on OSD/OCID are measurements on external benchmarks, not quantities derived from its own inputs. There is no step in which a target quantity is defined in terms of the prediction, no fitted parameter is relabeled as a prediction, and no uniqueness or ansatz is imported from a self-citation. Reference [12] is an author self-citation used only as an example of UOIS applications in Related Work; it is not load-bearing for the architecture or the SOTA claim. The OCBD dataset is introduced by the authors and is not yet public, so the 'bin environment' component of the SOTA claim is less independently checkable, but this is an external-validation limitation rather than a circular reduction. The Table I anomaly in which the MSMFormer RGB-D row reproduces the UOAIS† RGB-D row with the OCID/OSD blocks interchanged is a serious data-integrity/correctness problem for the comparison, not a circularity pattern under the taxonomy. Accordingly the score is 2 for the minor non-load-bearing self-citation and self-introduced benchmark, not for a circular derivation.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 1 invented entities

The central claim is empirical; no first-principles derivation is involved. It rests on imported deformable-attention and Mask2Former machinery, on sim-to-real transfer from UOAIS-SIM, and on comparable baseline evaluations. No new theoretical entities are introduced; OCBD is a dataset contribution rather than a theoretical construct.

free parameters (6)
  • Number of attention heads M = not reported
    All attention equations (5)-(14) depend on M, but the paper never gives its value.
  • Sampled keys per head K = not reported
    K defines how many deformed points are sampled per query (Eqs. 5-6, 9-14); not specified.
  • Offset network θ_offset = not reported
    The network that predicts offsets in Eqs. (1), (9), (10) is not described (layers, hidden size).
  • Swin backbone variant and channel widths C_i = not reported
    The paper only says '4-layer Swin Transformer'; no tiny/small/base choice or per-layer C_i.
  • Loss weights λ_ce, λ_dice, λ_cls = 5.0, 5.0, 2.0/0.1
    Hand-set in Section V-A; the final segmentation result depends on them.
  • Optimizer/training schedule = AdamW lr 1e-4, 20 epochs, batch 2
    Chosen without sensitivity analysis; standard but still free hyperparameters.
axioms (5)
  • domain assumption Deformable attention from Deformable DETR remains effective when adapted to cross-modal fusion
    Equations (1)-(14) import Deformable DETR's attention and bilinear sampling; no ablation or proof isolates this design choice.
  • domain assumption Training on UOAIS-SIM transfers to OCID, OSD, and OCBD
    All models are trained on synthetic UOAIS-SIM and evaluated on real benchmarks; no domain-gap analysis is provided.
  • domain assumption OSD/OCID ground truth (including auto-labeled OCID) is accurate enough for comparison
    Figure 1's caption notes GT is auto-labeled [4]; annotation noise could affect methods unevenly.
  • domain assumption Mask2Former decoder and losses work in a class-agnostic UOIS setting
    The decoder and loss are reused from [19] with only class-agnostic changes; no validation of this transfer is given.
  • standard math Bilinear interpolation is differentiable and adequate for offset-based sampling
    Eq. (4) is standard bilinear sampling, assumed differentiable for end-to-end training.
invented entities (1)
  • OCBD dataset no independent evidence
    purpose: Benchmark for bin-picking UOIS with 1000 top-down RGB-D images and human annotations
    The dataset is introduced by the paper and used to claim bin-picking SOTA, but no release/URL is provided, so it cannot be independently accessed or validated.

pith-pipeline@v1.3.0-alltime-deepseek · 10151 in / 21105 out tokens · 215775 ms · 2026-08-01T17:04:21.858283+00:00 · methodology

0 comments
read the original abstract

In logistics automation, precise segmentation of unseen objects is crucial for efficient robotic manipulation in cluttered environments. Tasks such as bin-picking and shelf-picking require robust perception to handle occlusions, varying object shapes, and complex spatial arrangements. Traditional RGB-based methods tend to over-segment objects due to their reliance on texture, while depth-based methods often under-segment by focusing primarily on geometric features. To address these limitations, we propose DA-Fusion, a deformable attention-based RGB-D fusion Transformer designed for unseen object instance segmentation. DA-Fusion effectively combines the strengths of both RGB and depth data, enhancing segmentation accuracy in cluttered and multi-layered object environments. We also introduce the Object Clutter Bin Dataset (OCBD), a benchmark dataset specifically tailored for evaluating bin-picking scenarios in top-down views. Extensive evaluations demonstrate that DA-Fusion outperforms state-of-the-art methods across diverse environments, making it particularly suited for real-world logistics tasks.

Figures

Figures reproduced from arXiv: 2607.17754 by Byoung-Tak Zhang, Hye-Jung Yoon, Juno Kim, Yesol Park.

Figure 1
Figure 1. Figure 1: Unseen Object Segmentation results on [4]. RGB-based segmentation tends to over-segment (top left), while depth-based under￾segments (bottom right). Our fusion strategy improves accuracy (top right). Note that GT (bottom left) is auto-labeled [4]. dynamically fuse RGB and depth data at multiple levels within the model. This approach enables more precise and fine-grained integration of texture and geometric… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the DA-Fusion Architecture. Two parallel branches extract features from RGB and depth inputs, which are fused at multiple layers using deformable attention. The fused features are processed by the Mask Transformer decoder [19] to generate segmentation masks. C. Deformable Self-Attention (DS) In Deformable Self-Attention (DS), the feature maps frgb and fd are refined through attention mechanisms… view at source ↗
Figure 3
Figure 3. Figure 3: OCBD Dataset Examples. Example images from OCBD with their segmentation GTs, illustrating the dataset’s diversity and complexity. E. Hierarchical Mask Transformer Decoder The fused features from all layers (f1, f2, f3, f4) are passed to the Mask Transformer decoder [19], which includes a pixel decoder, Transformer decoder, and mask prediction module. The pixel decoder refines the multi-scale features, whil… view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative Evaluation. Comparison of instance segmentation results across OSD (top row), OCID (middle row), and OCBD (bottom row) datasets. From left to right: UCN, UOAIS, MSMFormer, our DA-Fusion method, and GT. DA-Fusion demonstrates superior segmentation of accumulated objects in OSD, better object delineation in feature-rich backgrounds in OCID, and more accurate detection of occluded objects in lower… view at source ↗
Figure 5
Figure 5. Figure 5: Examples from Real-Robot Experiments. Shelf picking (left column) and bin picking (right column) tasks with corresponding segmen￾tation results demonstrating effectiveness of DA-Fusion in diverse real-world manipulation environments. E. Real-robot Experiments We validated the effectiveness of DA-Fusion through real￾robot experiments focused on two logistics tasks: shelf￾picking from a front-view setup and … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 5 linked inside Pith

  1. [1]

    Depth-aware cnn for rgb-d segmenta- tion,

    W. Wang and U. Neumann, “Depth-aware cnn for rgb-d segmenta- tion,” inProceedings of the European conference on computer vision (ECCV), pp. 135–150, 2018

    2018

  2. [2]

    A brief survey on rgb- d semantic segmentation using deep learning,

    C. Wang, C. Wang, W. Li, and H. Wang, “A brief survey on rgb- d semantic segmentation using deep learning,”Displays, vol. 70, p. 102080, 2021

    2021

  3. [3]

    Deformable detr: Deformable transformers for end-to-end object detection,

    X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,”arXiv preprint arXiv:2010.04159, 2020

    Pith/arXiv arXiv 2010

  4. [4]

    Easylabel: A semi-automatic pixel-wise object annotation tool for creating robotic rgb-d datasets,

    M. Suchi, T. Patten, D. Fischinger, and M. Vincze, “Easylabel: A semi-automatic pixel-wise object annotation tool for creating robotic rgb-d datasets,” in2019 International Conference on Robotics and Automation (ICRA), pp. 6678–6684, IEEE, 2019

    2019

  5. [5]

    Fpcc: Fast point cloud clustering-based instance segmentation for industrial bin-picking,

    Y . Xu, S. Arai, D. Liu, F. Lin, and K. Kosuge, “Fpcc: Fast point cloud clustering-based instance segmentation for industrial bin-picking,” Neurocomputing, vol. 494, pp. 255–268, 2022

    2022

  6. [6]

    A convolutional neural net- work for point cloud instance segmentation in cluttered scene trained by synthetic data without color,

    Y . Xu, S. Arai, F. Tokuda, and K. Kosuge, “A convolutional neural net- work for point cloud instance segmentation in cluttered scene trained by synthetic data without color,”IEEE Access, vol. 8, pp. 70262– 70269, 2020

    2020

  7. [7]

    Metagraspnet v0: A large-scale benchmark dataset for vision-driven robotic grasping via physics-based metaverse synthesis,

    Y . Chen, E. Z. Zeng, M. Gilles, and A. Wong, “Metagraspnet v0: A large-scale benchmark dataset for vision-driven robotic grasping via physics-based metaverse synthesis,”arXiv preprint arXiv:2112.14663, 2021

    Pith/arXiv arXiv 2021

  8. [8]

    Metagraspnet: A large-scale benchmark dataset for scene-aware am- bidextrous bin picking via physics-based metaverse synthesis,

    M. Gilles, Y . Chen, T. R. Winter, E. Z. Zeng, and A. Wong, “Metagraspnet: A large-scale benchmark dataset for scene-aware am- bidextrous bin picking via physics-based metaverse synthesis,” in 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), pp. 220–227, IEEE, 2022

    2022

  9. [9]

    Segmentation of unknown objects in indoor environments,

    A. Richtsfeld, T. M ¨orwald, J. Prankl, M. Zillich, and M. Vincze, “Segmentation of unknown objects in indoor environments,” in2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4791–4796, IEEE, 2012

    2012

  10. [10]

    Dorapicker: An autonomous picking system for general objects,

    H. Zhang, P. Long, D. Zhou, Z. Qian, Z. Wang, W. Wan, D. Manocha, C. Park, T. Hu, C. Cao,et al., “Dorapicker: An autonomous picking system for general objects,” in2016 IEEE International Conference on Automation Science and Engineering (CASE), pp. 721–726, IEEE, 2016

    2016

  11. [11]

    A robust robot design for item picking,

    A. Causo, Z.-H. Chong, R. Luxman, Y . Y . Kok, Z. Yi, W.-C. Pang, R. Meixuan, Y . S. Teoh, W. Jing, H. S. Tju,et al., “A robust robot design for item picking,” in2018 IEEE international conference on robotics and automation (ICRA), pp. 7421–7426, IEEE, 2018

    2018

  12. [12]

    Seg2grasp: A robust modular suction grasping in bin picking,

    H.-J. Yoon, J. Kim, Y . Park, J.-K. Lee, and B.-T. Zhang, “Seg2grasp: A robust modular suction grasping in bin picking,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2921–2927, IEEE, 2024

    2024

  13. [13]

    A sim-to-real object recognition and localization framework for industrial robotic bin picking,

    X. Li, R. Cao, Y . Feng, K. Chen, B. Yang, C.-W. Fu, Y . Li, Q. Dou, Y .-H. Liu, and P.-A. Heng, “A sim-to-real object recognition and localization framework for industrial robotic bin picking,”IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 3961–3968, 2022

    2022

  14. [14]

    Mask r-cnn,

    K. He, G. Gkioxari, P. Doll ´ar, and R. Girshick, “Mask r-cnn,” in Proceedings of the IEEE international conference on computer vision, pp. 2961–2969, 2017

    2017

  15. [15]

    The best of both modes: Separately leveraging rgb and depth for unseen object instance segmentation,

    C. Xie, Y . Xiang, A. Mousavian, and D. Fox, “The best of both modes: Separately leveraging rgb and depth for unseen object instance segmentation,” inConference on robot learning, pp. 1369–1378, PMLR, 2020

    2020

  16. [16]

    Unseen object in- stance segmentation for robotic environments,

    C. Xie, Y . Xiang, A. Mousavian, and D. Fox, “Unseen object in- stance segmentation for robotic environments,”IEEE Transactions on Robotics, vol. 37, no. 5, pp. 1343–1359, 2021

    2021

  17. [17]

    Unseen object amodal instance segmentation via hierarchical occlusion mod- eling,

    S. Back, J. Lee, T. Kim, S. Noh, R. Kang, S. Bak, and K. Lee, “Unseen object amodal instance segmentation via hierarchical occlusion mod- eling,” in2022 International Conference on Robotics and Automation (ICRA), pp. 5085–5092, IEEE, 2022

    2022

  18. [18]

    Per-pixel classification is not all you need for semantic segmentation,

    B. Cheng, A. G. Schwing, and A. Kirillov, “Per-pixel classification is not all you need for semantic segmentation,” inNeurIPS, 2021

    2021

  19. [19]

    Masked-attention mask transformer for universal image segmenta- tion,

    B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar, “Masked-attention mask transformer for universal image segmenta- tion,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1290–1299, 2022

    2022

  20. [20]

    Learning rgb-d feature embeddings for unseen object instance segmentation,

    Y . Xiang, C. Xie, A. Mousavian, and D. Fox, “Learning rgb-d feature embeddings for unseen object instance segmentation,” inConference on Robot Learning, pp. 461–470, PMLR, 2021

    2021

  21. [21]

    Mean shift mask transformer for unseen object instance segmentation,

    Y . Lu, Y . Chen, N. Ruozzi, and Y . Xiang, “Mean shift mask transformer for unseen object instance segmentation,”arXiv preprint arXiv:2211.11679, 2022

    Pith/arXiv arXiv 2022

  22. [22]

    Fuseseg: Semantic segmentation of urban scenes based on rgb and thermal data fusion,

    Y . Sun, W. Zuo, P. Yun, H. Wang, and M. Liu, “Fuseseg: Semantic segmentation of urban scenes based on rgb and thermal data fusion,” IEEE Transactions on Automation Science and Engineering, vol. 18, no. 3, pp. 1000–1011, 2020

    2020

  23. [23]

    Cmx: Cross-modal fusion for rgb-x semantic segmentation with transform- ers,

    J. Zhang, H. Liu, K. Yang, X. Hu, R. Liu, and R. Stiefelhagen, “Cmx: Cross-modal fusion for rgb-x semantic segmentation with transform- ers,”IEEE Transactions on intelligent transportation systems, vol. 24, no. 12, pp. 14679–14694, 2023

    2023

  24. [24]

    Early or late fusion matters: Efficient rgb- d fusion in vision transformers for 3d object recognition,

    G. Tziafas and H. Kasaei, “Early or late fusion matters: Efficient rgb- d fusion in vision transformers for 3d object recognition,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 9558–9565, IEEE, 2023

    2023

  25. [25]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020

    Pith/arXiv arXiv 2010

  26. [26]

    Cross-modal attention fusion network for rgb-d semantic segmentation,

    Q. Zhao, Y . Wan, J. Xu, and L. Fang, “Cross-modal attention fusion network for rgb-d semantic segmentation,”Neurocomputing, vol. 548, p. 126389, 2023

    2023

  27. [27]

    Learning rich features from rgb-d images for object detection and segmentation,

    S. Gupta, R. Girshick, P. Arbel ´aez, and J. Malik, “Learning rich features from rgb-d images for object detection and segmentation,” inComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VII 13, pp. 345– 360, Springer, 2014

    2014

  28. [28]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF international conference on computer vision, pp. 10012–10022, 2021

    2021

  29. [29]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo,et al., “Segment anything,” inProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4015–4026, 2023

    2023

  30. [30]

    Cgnet: A light- weight context guided network for semantic segmentation,

    T. Wu, S. Tang, R. Zhang, J. Cao, and Y . Zhang, “Cgnet: A light- weight context guided network for semantic segmentation,”IEEE Transactions on Image Processing, vol. 30, pp. 1169–1179, 2020

    2020

  31. [31]

    Yale-cmu-berkeley dataset for robotic manipulation research,

    B. Calli, A. Singh, J. Bruce, A. Walsman, K. Konolige, S. Srini- vasa, P. Abbeel, and A. M. Dollar, “Yale-cmu-berkeley dataset for robotic manipulation research,”The International Journal of Robotics Research, vol. 36, no. 3, pp. 261–268, 2017

    2017

  32. [32]

    6-dof pose estimation of household objects for robotic manipulation: An accessible dataset and benchmark,

    S. Tyree, J. Tremblay, T. To, J. Cheng, T. Mosier, J. Smith, and S. Birchfield, “6-dof pose estimation of household objects for robotic manipulation: An accessible dataset and benchmark,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 13081–13088, IEEE, 2022

    2022

  33. [33]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

    Pith/arXiv arXiv 2017

  34. [34]

    V-net: Fully convolutional neural networks for volumetric medical image segmentation,

    F. Milletari, N. Navab, and S.-A. Ahmadi, “V-net: Fully convolutional neural networks for volumetric medical image segmentation,” in2016 fourth international conference on 3D vision (3DV), pp. 565–571, Ieee, 2016

    2016