YCB-LUMA: YCB Object Dataset with Luminance Keying for Object Localization

Thomas P\"ollabauer

REVIEW 3 major objections 5 minor 15 references

YCB-LUMA: YCB Object Dataset with Luminance Keying for Object Localization

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By recording the remaining YCB household objects against a 99.99% light-absorbing screen and extracting masks by brightness thresholding, this paper extends luminance-keying training data to the full YCB object set, including transparent…

desk verdict Useful dataset extension, but the paper's own recording log undercuts the 'high quality' claim for the new object classes; needs mask-quality validation. read the letter →

arxiv 2411.13149 v1 pith:7TFZHMT4 submitted 2024-11-20 cs.CV cs.AI

Thomas Pöllabauer This is my paper · ORCID

classification cs.CVcs.AI

keywords YCBdatasetluminancekeyingobjectdetectionsegmentationsyntheticdatagenerationtraininglocalizationextension

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper extends the YCB-LUMA training-data suite from the YCB-V subset to the complete YCB object set. Following the luminance keying method, the author records the previously missing objects — transparent plastic and glass, reflective metal, dark and multi-color variations, and deformable items such as a chain and a cord — against a 99.99% light-absorbing black screen. Brightness thresholding turns these recordings into foreground masks, and released scripts generate 2D object detection and segmentation training data with annotations. The paper's central claim is that the full YCB set can now be covered by luminance keying, increasing the variety of appearances available for testing detection and segmentation algorithms without manual labeling.

What carries the argument

Luminance keying: recording objects in front of a 99.99% light-absorbing black screen so the background has near-zero brightness, making foreground extraction a brightness threshold rather than color-based chroma keying. In this paper the mechanism both produces the automatic masks and annotations and is the method under test, because the new objects add transparency, reflection, dark surfaces, and deformation. The released processing scripts are the operational part of the machinery, converting raw recordings into 2D detector and segmentation training data without manual labeling.

What would settle it

Run the provided auto-masking scripts on the recordings the paper flags for retake or noise, such as the dark-shiny large marble (object 61), the plastic bolt (object 74), and the plastic nut (object 75), and compare the resulting masks pixel-by-pixel with manually drawn masks; a large drop in intersection-over-union for these objects would show that the automatic annotation claim does not hold uniformly.

Watch

Extended reading notes

Core claim

On its own terms, the paper's contribution is a dataset rather than a new algorithm: it records every YCB object not already present in the YCB-V subset under the same luminance keying setup, and it supplies code that automatically generates detection and segmentation training data from the recordings. The newly recorded objects intentionally include appearance classes that stress the keying approach: transparency, specular metal, dark parts, multiple color variants of the same object, and deformable shapes. Color variants are stored in separate subfolders, which enables training on one variant and testing on another or combining all variants in one set. The stated result is that the YCB-LUMA set now covers the full YCB object set with high quality recordings and automatic annotation, extending the usefulness of the previous YCB-V luminance keying data.

Load-bearing premise

The whole automatic-annotation pipeline depends on a simple brightness threshold cleanly separating each object from the black background, and the paper's own recording table shows this already fails for several dark, shiny, and reflective objects.

Editorial extensions

If this is right

Every YCB object class now has luminance-keying recordings, so 2D object detectors and segmentation models can be trained on the full YCB set without manual annotation.
The added transparent, metallic, and deformable objects provide a stress test for whether the luminance keying approach generalizes beyond the original YCB-V subset.
Separate subfolders for each color variant enable controlled experiments on generalization, such as training on one variant and evaluating on another.
The released processing scripts mean other objects recorded with the same black-screen setup can be converted into detection and segmentation training data automatically.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's own metadata table marks several objects as noisy, undetected, or needing retake (for example objects 21, 26, 36, 61, 74, 75), so the pipeline likely requires per-object threshold tuning or post-processing rather than being fully automatic for every appearance class.
The paper reports no quantitative comparison between automatic masks and manual masks, leaving mask quality on the newly added objects as the main open question.
If the auto-masks are validated, the same black-screen setup could record arbitrary new objects, making luminance keying a practical alternative to 3D-model-based rendering for niche detection tasks.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read the YCB-LUMA paper. It does what it says: records the remaining YCB objects beyond YCB-V using the luminance keying setup from prior work, and ships the data and processing code. That's a real artifact. The added object types—transparent plastic/glass, reflective metal, dark plastic, deformable chain—are genuinely harder for brightness thresholding, and including them is the right stress test for the method.

What's good: the release is complete by the authors' own accounting, the metadata table is unusually honest, and the scripts for automatic masking are provided. The paper also gives a clear path for differentiating object variations, which is useful for generalization tests. I believe the dataset will be a resource for 2D detection/segmentation training, especially for people already using YCB-V-LUMA.

The soft spot is exactly where the stress-test lands. The central claim says 'high quality recordings' and 'automatic generation of training data and annotations.' But the paper provides no quantitative evaluation of mask quality, no IoU against manual annotations, and no downstream detector/segmentation training to show the annotations are usable. Worse, the recording table documents repeated masking failures on the very classes the extension adds: object 9 misses the dark moustache; objects 26-28 have noisy reflective metal; object 30 has noisy results on the wine glass; objects 36, 61, 74-75 are explicitly marked for retake; object 79 can't detect the smallest washers; object 68 is a deformable chain where thin segments are prone to errors. A single global brightness threshold against a 99.99% absorbing background will not cleanly separate dark or transparent foreground from background. The authors acknowledge some of this in the comments but still assert high quality without validation.

This is not a fatal flaw in the dataset—recordings can still be useful, and the authors may have manually filtered or adjusted thresholds. But the paper as written overclaims. The fix is straightforward: report mask quality metrics on a held-out set of manually labeled frames, or at minimum train a YOLO/Mask R-CNN on the auto-generated data and report AP/mAP. That would turn a conditional release into a defensible one.

The citation pattern is fine; the method is from the authors' own prior work and they cite it. The paper is an incremental data extension, not a new technique, but it is a legitimate extension. I'd send it to peer review—dataset papers deserve referee time when the artifact is real—but I'd ask for the quality validation before acceptance. The reader's conditional verdict is right.

Referee Report

3 major / 5 minor

Summary. The paper presents YCB-LUMA, a dataset that extends the existing YCB-V luminance-keying recordings to the remaining objects in the YCB superset. The newly recorded objects include transparent plastics and glass, reflective metal, dark parts, and deformable objects such as a chain and a cord. The authors provide the recordings and scripts that automatically generate masks for 2D object detection and segmentation by brightness thresholding against a 99.99% light-absorbing background. The central claims are that the new release provides high-quality recordings and code for automatic generation of training data and annotations, and that the added object variety demonstrates the usefulness of luminance keying.

Significance. If the claims are substantiated, the contribution is practically useful: it extends a widely used benchmark object set, removes the need for manual labeling in many cases, and includes challenging object categories that are often missing from such datasets. The authors should be credited for publishing URLs to the data and code, and for including a detailed recording log in Figure 2 that transparently reports known issues. However, the usefulness claim is currently unvalidated: the paper's own log documents masking failures on many of the very objects that motivate the extension, and no quantitative mask-quality evaluation or downstream training experiment is provided. The significance is therefore conditional on the authors either fixing the problematic recordings or clearly delimiting the failure modes and measuring the quality of the generated masks.

major comments (3)

[Section 3, Figure 2] The recording log in Figure 2 documents masking failures for many of the newly recorded objects: dark parts not detected (objects 9, 32, 42-43, 61), noisy edges from reflections (objects 26-28, 58-60, 76), and objects explicitly marked for retake (objects 36, 61, 74-75). These are precisely the transparent, reflective, dark, and deformable categories that the paper presents as the added variety of the extension. The abstract's claim of "high quality data" and Section 4's claim of "high quality recordings" are therefore not supported by the evidence in the paper. The authors should either retake or exclude these objects, or provide per-object mask-quality metrics and clearly state the failure rate in the paper.
[Section 3, 'scripts to automatically extract training data'] The central claim that the provided code generates usable training data is not validated. No experiments are reported that use the generated masks for detector or segmentation training, and no mask-quality metric is computed. A small demonstration (for example, training a standard detector on LUMA-generated masks and reporting AP on a hold-out set, or measuring IoU between generated and manually corrected masks on a sample of frames) would provide the missing evidence. Without it, the statement that the code produces usable annotations remains an assertion.
[Section 3, Figure 1 and object 68] For the deformable objects (chain, cord), the paper states that multiple deformation states are recorded, but the automatic masking of thin, self-occluded chain links is prone to errors, as the log suggests for object 68. The release should include a per-sequence statement of whether the automatic masks were verified or hand-corrected for these objects, and the paper should indicate how much manual intervention is needed for the deformable category.

minor comments (5)

[Abstract] The phrase "quality insurance" should be "quality assurance".
[Figure 2] The table is extremely dense and the column headers are ambiguous: "ObjectAvailable?", "Different?", and "Difference?" need definition in the caption, and the meaning of the "Count" and "Retake?" columns should be explicitly explained so that readers can interpret the log.
[Section 3, first paragraph] The sentence "These new recordings complement the original ones depicting all of the objects to be found in the YCB-V subset" is confusing, because YCB-V is itself a subset of YCB. Reword to say that the new recordings cover the remaining YCB objects not already present in YCB-V.
[Section 4] The paper should state the total number of videos, frames, and per-object image counts, as well as the license for the dataset and the code, since these facts are essential for a dataset release.
[References] Reference [8] (Pöllabauer et al., "Advanced post-processing for object detection dataset generation") is listed without a venue or year; please provide the full publication details.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper presents a dataset extension and processing code; its claims are not derived from their own inputs.

full rationale

This is a dataset-contribution paper with no equations, fitted parameters, or predictive derivation chain. The central claim is that the authors record the remaining YCB objects and provide code for automatic generation of training data and annotations. The luminance keying method is inherited from the author's own prior work [10], but that cited method is not used to define the new dataset's content by construction; it is a recording and masking technique that the authors apply to new objects. The new recordings and the code are independent artifacts whose quality can be assessed externally. The paper's own Figure 2 candidly documents masking difficulties (dark parts, reflections, noisy edges, retakes) for several objects, and the paper provides no quantitative mask-quality evaluation; these are correctness or validity concerns about the 'high quality recordings' claim, not circularity. The self-citations [8,10] are not load-bearing in the sense of making a derived result equivalent to its inputs. No step in this paper reduces by definition, by fitted-input renaming, or by self-citation chain to its own assumptions, so the circularity score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the physical assumption of a 99.99% light-absorbing background and on the sufficiency of brightness thresholding for automatic masking. The recording threshold is a hand-tuned parameter, and the paper's own comments show it struggles with dark, shiny, and transparent objects. No new entities are postulated.

free parameters (1)

brightness threshold for foreground/background masking = Not reported in paper; tuned per object
The automatic masking pipeline segments objects by brightness, so the threshold directly controls mask quality. Figure 2 notes 'increase the brightness threshold?' for object 36, showing the threshold was adjusted by hand, and dark or reflective objects produced noisy results.

assumptions (3)

domain assumption A 99.99% light-absorbing black screen makes the background nearly black, so foreground objects can be separated by brightness.
Invoked in Section 3; it is the physical basis of luminance keying and is carried over from the authors' prior work [10].
domain assumption Brightness thresholding yields masks accurate enough for training 2D detectors and segmenters on the new object types.
This is the load-bearing assumption behind the 'high quality' claim; Figure 2's comments on noisy edges and missing dark parts show it is not uniformly satisfied.
domain assumption The YCB object set is a suitable benchmark for object localization.
Used throughout; the paper extends YCB-V, which is described as a 'classic core dataset' in the BOP benchmark.

how reviews work

0 comments

Cite this review

Pith. "Pith review of YCB-LUMA: YCB Object Dataset with Luminance Keying for Object Localization." pith.science (2026). https://pith.science/paper/7TFZHMT4

@misc{pith2026241113149,
  author       = {Pith},
  title        = {Pith review of: YCB-LUMA: YCB Object Dataset with Luminance Keying for Object Localization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7TFZHMT4}},
  note         = {Machine review of arXiv:2411.13149}
}

read the original abstract

Localizing target objects in images is an important task in computer vision. Often it is the first step towards solving a variety of applications in autonomous driving, maintenance, quality insurance, robotics, and augmented reality. Best in class solutions for this task rely on deep neural networks, which require a set of representative training data for best performance. Creating sets of sufficient quality, variety, and size is often difficult, error prone, and expensive. This is where the method of luminance keying can help: it provides a simple yet effective solution to record high quality data for training object detection and segmentation. We extend previous work that presented luminance keying on the common YCB-V set of household objects by recording the remaining objects of the YCB superset. The additional variety of objects - addition of transparency, multiple color variations, non-rigid objects - further demonstrates the usefulness of luminance keying and might be used to test the applicability of the approach on new 2D object detection and segmentation algorithms.

Figures

Figures reproduced from arXiv: 2411.13149 by the authors.

**Figure 1.** Additional objects to complement the YCB-V set. For each object, we took multiple recordings to capture it from all sides. For deformable objects, such as the yellow chain, we record multiple different states of deformation. bleeding and overlap. Instead, luminance keying, utilizing a black screen with high light absorption to capture videos of target objects, circumvents the typical challenges of chroma keying, al… view at source ↗

**Figure 2.** In contrast to the YCB-V subset, the new objects contain a wider range of appearances: there are now multiple transparent objects, such as objects made of transparent plastics and others made of glass, more metallic surfaces, such as on the fork, knife, and spoon, multiple alterations of the same objects, such as the same lego pieces, but of different colors, as well as deformable objects, such as the yellow chain a… view at source ↗

**Figure 2.** Description of our recordings. Each object is attributed to a category and given a name according to the original publication where possible. We also report whether the object is part of this recording session or not (the YCB-V objects have been recorded previously), whether the recorded objects differ in some way compared to the original YCB data, and some additional meaningful meta data [PITH_FULL_IMAGE:figures/f… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 12 canonical work pages

[1]

IEICE TRANSACTIONS on Information and Systems 90(1), 242–249 (2007)

Agata, H., Yamashita, A., Kaneko, T.: Chroma key using a checker pattern back- ground. IEICE TRANSACTIONS on Information and Systems 90(1), 242–249 (2007)

work page 2007
[2]

In: 2020 IEEE International Con- ference on Robotics and Automation (ICRA)

Grenzdörffer, T., Günther, M., Hertzberg, J.: Ycb-m: A multi-camera rgb-d dataset for object recognition and 6dof pose estimation. In: 2020 IEEE International Con- ference on Robotics and Automation (ICRA). pp. 3650–3656. IEEE (2020)

work page 2020
[3]

The Visual Com- puter 26, 1167–1176 (2010)

Grundhöfer, A., Kurz, D., Thiele, S., Bimber, O.: Color invariant chroma keying and color spill neutralization for dynamic scenes and cameras. The Visual Com- puter 26, 1167–1176 (2010)

work page 2010
[4]

European Conference on Computer Vision (ECCV) (2018)

Hodaň, T., Michel, F., Brachmann, E., Kehl, W., Glent Buch, A., Kraft, D., Drost, B., Vidal, J., Ihrke, S., Zabulis, X., Sahin, C., Manhardt, F., Tombari, F., Kim, T.K., Matas, J., Rother, C.: BOP: Benchmark for 6D object pose estimation. European Conference on Computer Vision (ECCV) (2018)

work page 2018
[5]

In: Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16

Hodaň, T., Sundermeyer, M., Drost, B., Labbé, Y., Brachmann, E., Michel, F., Rother, C., Matas, J.: Bop challenge 2020 on 6d object localization. In: Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. pp. 577–594. Springer (2020)

work page 2020
[6]

arXiv preprint arXiv:2403.09799 (2024)

Hodan, T., Sundermeyer, M., Labbe, Y., Nguyen, V.N., Wang, G., Brachmann, E., Drost, B., Lepetit, V., Rother, C., Matas, J.: Bop challenge 2023 on detection, segmentation and pose estimation of seen and unseen rigid objects. arXiv preprint arXiv:2403.09799 (2024)

arXiv 2024
[7]

In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004

LeCun, Y., Huang, F.J., Bottou, L.: Learning methods for generic object recog- nition with invariance to pose and lighting. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. vol. 2, pp. II–104. IEEE (2004)

2004
[8]

Pöllabauer, T., Berkei, S., Knauthe, V., Kuijper, A.: Advanced post-processing for object detection dataset generation (2024)

work page 2024

Show all 15 references

[9]

In: Pattern Recognition and Artificial Intelligence

Pöllabauer, T., Emrich, J., Knauthe, V., Kuijper, A.: Extending 6d object pose estimators for stereo vision. In: Pattern Recognition and Artificial Intelligence. Lecture Notes in Computer Science, vol. 14893 (2024)

2024
[10]

Pöllabauer, T., Knauthe, V., Boller, A., Kuijper, A., Fellner, D.: Fast training data acquisition for object detection and segmentation using black screen luminance keying 32(1-2), 101–110 (2024)

2024
[11]

In: 9th International Workshop on Recovering 6D Object Pose (R6D), at ECCV2024 (2024) 6 Thomas Pöllabauer

Rojtberg, P., Pöllabauer, T.: Ycb-ev 1.1: Event-vision dataset for 6dof object pose estimation. In: 9th International Workshop on Recovering 6D Object Pose (R6D), at ECCV2024 (2024) 6 Thomas Pöllabauer

2024
[12]

In: Proceedings of the 4th International Conference on Pattern Recog- nition and Artificial Intelligence (ICPRAI) (2024)

Rojtberg, P., Pöllabauer, T.: Ycb-ev: Event-vision dataset for 6dof object pose estimation. In: Proceedings of the 4th International Conference on Pattern Recog- nition and Artificial Intelligence (ICPRAI) (2024)

2024
[13]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

Sundermeyer, M., Hodaň, T., Labbe, Y., Wang, G., Brachmann, E., Drost, B., Rother, C., Matas, J.: Bop challenge 2022 on detection, segmentation and pose estimation of specific rigid objects. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition....

2023
[14]

arXiv preprint arXiv:1711.00199 (2017)

Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: Posecnn: A convolutional neu- ral network for 6d object pose estimation in cluttered scenes. arXiv preprint arXiv:1711.00199 (2017)

2017 arXiv
[15]

In: 2008 19th In- ternational Conference on Pattern Recognition

Yamashita, A., Agata, H., Kaneko, T.: Every color chromakey. In: 2008 19th In- ternational Conference on Pattern Recognition. pp. 1–4. IEEE (2008)

2008

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

IEICE TRANSACTIONS on Information and Systems 90(1), 242–249 (2007)

Agata, H., Yamashita, A., Kaneko, T.: Chroma key using a checker pattern back- ground. IEICE TRANSACTIONS on Information and Systems 90(1), 242–249 (2007)

work page 2007

[2] [2]

In: 2020 IEEE International Con- ference on Robotics and Automation (ICRA)

Grenzdörffer, T., Günther, M., Hertzberg, J.: Ycb-m: A multi-camera rgb-d dataset for object recognition and 6dof pose estimation. In: 2020 IEEE International Con- ference on Robotics and Automation (ICRA). pp. 3650–3656. IEEE (2020)

work page 2020

[3] [3]

The Visual Com- puter 26, 1167–1176 (2010)

Grundhöfer, A., Kurz, D., Thiele, S., Bimber, O.: Color invariant chroma keying and color spill neutralization for dynamic scenes and cameras. The Visual Com- puter 26, 1167–1176 (2010)

work page 2010

[4] [4]

European Conference on Computer Vision (ECCV) (2018)

Hodaň, T., Michel, F., Brachmann, E., Kehl, W., Glent Buch, A., Kraft, D., Drost, B., Vidal, J., Ihrke, S., Zabulis, X., Sahin, C., Manhardt, F., Tombari, F., Kim, T.K., Matas, J., Rother, C.: BOP: Benchmark for 6D object pose estimation. European Conference on Computer Vision (ECCV) (2018)

work page 2018

[5] [5]

In: Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16

Hodaň, T., Sundermeyer, M., Drost, B., Labbé, Y., Brachmann, E., Michel, F., Rother, C., Matas, J.: Bop challenge 2020 on 6d object localization. In: Computer Vision–ECCV 2020 Workshops: Glasgow, UK, August 23–28, 2020, Proceedings, Part II 16. pp. 577–594. Springer (2020)

work page 2020

[6] [6]

arXiv preprint arXiv:2403.09799 (2024)

Hodan, T., Sundermeyer, M., Labbe, Y., Nguyen, V.N., Wang, G., Brachmann, E., Drost, B., Lepetit, V., Rother, C., Matas, J.: Bop challenge 2023 on detection, segmentation and pose estimation of seen and unseen rigid objects. arXiv preprint arXiv:2403.09799 (2024)

arXiv 2024

[7] [7]

In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004

LeCun, Y., Huang, F.J., Bottou, L.: Learning methods for generic object recog- nition with invariance to pose and lighting. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. vol. 2, pp. II–104. IEEE (2004)

2004

[8] [8]

Pöllabauer, T., Berkei, S., Knauthe, V., Kuijper, A.: Advanced post-processing for object detection dataset generation (2024)

work page 2024