Pith. sign in

REVIEW 3 major objections 5 minor 27 references

Multiple Light Source Dataset for Colour Research

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper presents a collection of 24 scenes, each recorded under 18 multi-light illumination scenarios, with spectral measurements of camera, lights, and object surfaces plus pixel-by-pixel annotation masks.

desk verdict Genuinely useful multi-illuminant dataset, but its spectral ground truth is only weakly validated; worth publishing after the authors tighten the accuracy story. read the letter →

arxiv 1908.06126 v4 pith:HT3BYK3V submitted 2019-08-16 cs.CV

Anna Smagina , Egor Ershov , Anton Grigoryev This is my paper · ORCID
classification cs.CV
keywords colourconstancymultipleilluminantsspectralgroundtruthreflectancespectradatasetpixel-wiseannotationcamerasensitivitybenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper presents a laboratory dataset built for colour research: 24 scenes, each photographed under 18 lighting configurations that combine halogen, desk-lamp, and RGB-LED illumination. Alongside every image the authors publish the emission spectra of the sources, reflectance spectra of 122 surfaces, the camera's spectral sensitivities, and pixel-by-pixel masks of uniformly coloured regions. The aim is to give computational colour constancy and colour-based segmentation algorithms a benchmark with simultaneous spectral ground truth, which existing laboratory and uncontrolled datasets lack. If the spectral measurements are accurate, algorithm outputs can be compared against known per-scene illuminant and surface colours rather than against a single global white-point estimate.

What carries the argument

The carrying object is the dataset's calibration-and-annotation pipeline: a softbox stage with fixed halogen sources, a dimmable desk lamp, and a computer-controlled RGB LED strip; a camera recording linear RAW images; a miniature spectrometer with a reflectance standard for measuring source and surface spectra; and masks produced by an automatic linear-segmentation algorithm, manually merged into uniform-colour regions. The link between spectra and pixels is the Lambertian image-formation model, and the paper's consistency check uses the colour-homography relation to align observed and predicted chromaticities. The dataset itself is the mechanism: it is what makes the claimed ground truth available to other algorithms.

What would settle it

Take the same camera model, measure its spectral response directly with a monochromator, and recompute the chromaticities of the colour-chart patches under the two-halogen illumination from the supplied spectra; if the remeasured response moves predicted chromaticities outside the residual scatter shown in the paper's Figure 5, then the digitized camera response is the weak link in the ground-truth claim.

Watch

Extended reading notes

Core claim

The central claim is that this dataset supplies simultaneous spectral ground truth for a multi-light colour constancy benchmark. For each of 24 object scenes, the same scene is recorded under 18 illumination scenarios, and for each scenario the authors provide linear raw and colour images, the measured emission spectra of all active sources, the measured reflectance spectra of the objects, and manually refined pixel-wise masks that tie image regions to those reflectance spectra. The paper argues that this combination is the missing piece among published laboratory datasets, which offer ground truth for one or two illuminants but not the complete spectral description needed to evaluate multi-source illumination estimation, spectral colour models, and segmentation at once.

Load-bearing premise

The whole ground-truth chain rests on the camera's spectral sensitivity curves, which the authors took from an earlier paper's figure rather than measuring themselves; if those curves are inaccurate, the predicted colours will not match the images.

Editorial extensions

If this is right

  • Algorithms that estimate multiple illuminants can be scored against the measured spectra of the active sources in each of the 18 configurations, instead of a single scene-level white point.
  • Colour-based segmentation can be evaluated with the supplied pixel-wise masks, each region tied to a measured reflectance spectrum.
  • Low-parametric spectral models of colour transformation and multiple-reflection effects can be tested on linear images because camera sensitivities, source spectra, and surface reflectances are all provided.
  • Holding scene geometry fixed while varying LED colour and intensity isolates the effect of illumination on observed colours, a controlled comparison the paper positions as missing from earlier laboratory sets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fixed geometry across the 18 illumination states creates a natural self-supervised setup: train a network to predict reflectance from several images of the same scene under different lights, using the measured spectra as weak supervision.
  • If the digitized camera response is the main source of residual error, users can still use the dataset to estimate a colour homography between spectral predictions and images, turning the benchmark into a calibration procedure.
  • The varying distance and position of the light sources, not just their colour, could support estimating light-source geometry from a single image, a task the paper lists as motivation but does not evaluate.
  • Adding synthetic renders with the same measured spectra would give dense per-pixel ground truth, complementing the sparse surface masks of the real data.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces the Multiple Light Source (MLS) dataset: 24 multi-object scenes, each recorded under 18 illumination configurations involving halogen lamps, a desk lamp, and controlled red/green/blue LED contributions. The release includes full-resolution linear raw images and demosaiced color images, quarter-resolution previews, pixel-wise annotation masks of uniformly coloured surfaces, measured illuminant emission spectra, measured object reflectance spectra, and camera spectral sensitivities digitized from a published figure. The authors position the dataset as a complete benchmark for multiple-illuminant colour constancy, spectral colour modelling, and colour-based segmentation, and they report in Section 6 a consistency check between spectral predictions and observed chromaticities for one colour-chart image under 2HAL illumination.

Significance. If the spectral ground truth is accurate at the implied level, the dataset fills a genuine gap: most existing laboratory datasets do not simultaneously provide multiple light sources, measured illuminant spectra, object reflectance spectra, and linear raw camera images. The public release of 24 scenes with three image formats, pixel-wise masks, and a detailed spectral measurement protocol is a valuable resource for the colour-constancy community. The authors are also transparent about their main limitation, noting that they did not measure the camera responses themselves. However, the consistency demonstration in Section 6 does not currently establish the accuracy needed to call the spectral data 'ground truth' for benchmarking; the confidence in the central claim would be materially improved by a stronger independent validation.

major comments (3)
  1. [Section 6, Fig. 5a] The consistency check is performed on a single colour-chart image under the 2HAL illumination condition only. The initial agreement between measured and observed chromaticities is visibly poor for several patches, and no quantitative error metric is reported. With 18 illumination configurations in the dataset, a one-image, one-illuminant demonstration does not support the benchmark accuracy claimed for the full dataset.
  2. [Section 6, Fig. 5b] The improved agreement shown in Fig. 5b is obtained by fitting a planar colour homography to the same patch chromaticities that are subsequently compared. Because a homography can absorb linear discrepancies in camera sensitivity, illuminant spectra, or reflectance spectra, the post-fit visual correspondence is not an independent validation of the measured spectral data. A validation using held-out patches or, better, predicting chromaticities for a different illumination scenario from the separately measured LED spectra would be needed to support the ground-truth claim.
  3. [Section 2, camera spectral responses] The camera spectral sensitivities are digitized from a figure in Ref. [24] rather than measured by the authors, and Section 6 attributes the remaining mismatch to 'linear miscalculations in camera spectral sensitivity and lighting setup'. This makes the absolute accuracy of the predicted chromaticities unquantified. Please provide an uncertainty estimate for the digitized camera curves, or explicitly qualify the claim that the spectral data constitute camera-specific ground truth, since the predictions depend on this input.
minor comments (5)
  1. [Section 1] The word 'usefull' should be 'useful'.
  2. [Section 2] The lens model 'Canon EF 2470 mm' is likely meant to be 'Canon EF 24-70 mm'.
  3. [Section 6] The sentence 'the colour chart was glance, not matte' should read 'glossy, not matte'.
  4. [Figure 5] The marker colours are difficult to distinguish in the figure; enlarging the plots or adding text labels would improve readability.
  5. [Sections 5 and 6] The word 'chomaticities' is misspelled in several places; it should be 'chromaticities'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: spectral ground truth is directly measured; the homography-based consistency check is a transparent post-hoc fit, not a derivation loop.

full rationale

The paper's central claim is the release of a dataset with measured spectral characteristics of illuminants, surfaces, and the camera. These data were obtained by direct spectrometer measurements (with a radiometrically calibrated source and a reflectance standard), not by fitting to the image data. The camera spectral sensitivities are taken from an external publication (Baek et al. [24]) and digitized, which is an external source rather than a self-citation. The annotation masks are produced by an automatic pre-segmentation from the authors' own prior work [26] followed by manual merging; this is a procedural tool use and is not load-bearing for the spectral ground truth. The only potentially concerning step is Section 6, where a planar homography is fitted to all observed patch chromaticities and the post-fit correspondence is shown as a consistency demonstration. The paper is transparent about this ('To estimate planar homography matrix we conduct fitting above all of the patches chromaticity coordinates'), and it does not present the post-fit residual as an independent prediction or as a derivation of the spectral data. This is a validation weakness, not circularity: the spectral measurements themselves do not reduce to the image data or to the fitted homography. No self-citation carries the central argument, and no prediction is forced by construction. Therefore the paper shows no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central resource rests on the accuracy of measured spectra, digitized camera sensitivities, the Lambertian linear imaging model, and the segmentation pre-processing. No new physical entities are introduced; free parameters appear only in the small validation exercise.

free parameters (1)
  • Planar homography H = not reported
    Section 6: a 3x3 homography is fitted to all colour chart patch chromaticities to map observed to measured chromaticities; the parameters are not provided with the paper.
assumptions (4)
  • domain assumption Camera spectral response curves for Canon 5D Mark III, digitized from Baek et al. [24], are accurate representations of the true device sensitivities.
    Section 2. If this fails, spectral predictions of image chromaticities carry systematic error; the paper's own validation shows remaining mismatches.
  • domain assumption The scenes follow a Lambertian image formation model with linear camera response.
    Section 6. Predictions of patch chromaticities use this model, and the paper allows for linear miscalibration via homography.
  • domain assumption Spectral measurements made with OceanOptics FLAME-S and WS-1 standard are radiometrically accurate.
    Section 5. No error estimates accompany the spectra, so the reader must accept the calibration as ground truth.
  • domain assumption Automatic segmentation algorithm [26] yields regions with guaranteed colour constancy before manual merging.
    Section 4. Annotation relies on this guarantee; no quantitative validation is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multiple Light Source Dataset for Colour Research." pith.science (2026). https://pith.science/paper/HT3BYK3V

@misc{pith2026190806126,
  author       = {Pith},
  title        = {Pith review of: Multiple Light Source Dataset for Colour Research},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HT3BYK3V}},
  note         = {Machine review of arXiv:1908.06126}
}
read the original abstract

We present a collection of 24 multiple object scenes each recorded under 18 multiple light source illumination scenarios. The illuminants are varying in dominant spectral colours, intensity and distance from the scene. We mainly address the realistic scenarios for evaluation of computational colour constancy algorithms, but also have aimed to make the data as general as possible for computational colour science and computer vision. Along with the images of the scenes, we provide spectral characteristics of the camera, light sources and the objects and include pixel-by-pixel ground truth annotation of uniformly coloured object surfaces thus making this useful for benchmarking colour-based image segmentation algorithms. The dataset is freely available at https://github.com/visillect/mls-dataset.

Figures

Figures reproduced from arXiv: 1908.06126 by the authors.

Figure 1
Figure 1. Schematic view of the experimental setup for image collection. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Example scene viewed under 18 different illumination scenarios. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of all recorded MLS scenes. Here the white balancing is performed for demonstration purpose. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Chromaticity distributions of illuminants and surfaces in camera-specific colour coordinates. The star marker denotes the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Chromaticities of colour chart patches in camera-specific colour coordinates calculated from measured spectra (marked with [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 26 canonical work pages

  1. [24]

    Compact single-shot hyperspectral imaging using a prism,

    S.-H. Baek, I. Kim, D. Gutierrez, and M. H. Kim, “Compact single-shot hyperspectral imaging using a prism,” ACM Transactions on Graphics (TOG) 36(6), 217 (2017)

    work page 2017

  2. [1]

    Computational color constancy: Survey and experiments,

    A. Gijsenij, T. Gevers, and J. Van De Weijer, “Computational color constancy: Survey and experiments,” IEEE Transactions on Image Processing 20(9), 2475–2489 (2011)

    work page 2011

  3. [2]

    About new estimation methods for primary illuminant chromaticity in colour constancy algorithms,

    P. Nikolayev, “About new estimation methods for primary illuminant chromaticity in colour constancy algorithms,” Sensory systems 21(1), 29–44 (2007). In Russian

    work page 2007

  4. [3]

    A spatial processor model for object colour perception,

    G. Buchsbaum, “A spatial processor model for object colour perception,” Journal of the Franklin institute 310(1), 1–26 (1980)

    work page 1980

  5. [4]

    Sfsnet: Learning shape, reflectance and illuminance of faces in the wild’,

    S. Sengupta, A. Kanazawa, C. D. Castillo, and D. W. Jacobs, “Sfsnet: Learning shape, reflectance and illuminance of faces in the wild’,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6296–6305 (2018)

    work page 2018

  6. [5]

    An overview of color constancy algorithms,

    V . Agarwal, B. R. Abidi, A. Koschan, and M. A. Abidi, “An overview of color constancy algorithms,” Journal of Pattern Recognition Research 1(1), 42–54 (2006)

    work page 2006

  7. [6]

    Moving cast shadow resistant for foreground segmentation based on shadow properties analysis,

    H. Zhou, Y . Gao, G. Yuan, and R. Ji, “Moving cast shadow resistant for foreground segmentation based on shadow properties analysis,” in Eighth International Conference on Machine Vision (ICMV 2015), 9875, 98750B, Interna- tional Society for Optics and Photonics (2015)

    work page 2015

  8. [7]

    Single and multiple illuminant estimation using convolutional neural net- works,

    S. Bianco, C. Cusano, and R. Schettini, “Single and multiple illuminant estimation using convolutional neural net- works,” IEEE Transactions on Image Processing26(9), 4347–4362 (2017)

    work page 2017

Show all 27 references
  1. [8]

    Verification of applicability two multiplicative closed spectral models for multiple reflection effect description,

    N. Gusamutdinova, E. Ershov, S. Gladilin, and D. Nikolaev, “Verification of applicability two multiplicative closed spectral models for multiple reflection effect description,” in 2016 International Conference on Robotics and Machine Vision, 10253, 1025305, International Society...

    2017

  2. [9]

    Rehabilitating the colorchecker dataset for illuminant estimation,

    G. Hemrit, G. D. Finlayson, A. Gijsenij, P. Gehler, S. Bianco, B. Funt, M. Drew, and L. Shi, “Rehabilitating the colorchecker dataset for illuminant estimation,” in Color and Imaging Conference, 2018(1), 350–353, Society for Imaging Science and Technology (2018)

    2018

  3. [10]

    Unsupervised learning for color constancy,

    N. Bani ´c and S. Lonˇcari´c, “Unsupervised learning for color constancy,” arXiv preprint arXiv:1712.00436 (2017)

    2017 arXiv

  4. [11]

    A large image database for color constancy research,

    F. Ciurea and B. Funt, “A large image database for color constancy research,” in Color and Imaging Conference, 2003(1), 160–164, Society for Imaging Science and Technology (2003)

    2003

  5. [12]

    Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution,

    D. Cheng, D. K. Prasad, and M. S. Brown, “Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution,” JOSA A31(5), 1049–1058 (2014)

    2014

  6. [13]

    Statistics of spatial cone-excitation ratios in natural scenes,

    S. M. Nascimento, F. P. Ferreira, and D. H. Foster, “Statistics of spatial cone-excitation ratios in natural scenes,” JOSA A 19(8), 1484–1490 (2002)

    2002

  7. [14]

    Frequency of metamerism in natural scenes,

    D. H. Foster, K. Amano, S. M. Nascimento, and M. J. Foster, “Frequency of metamerism in natural scenes,” Josa a 23(10), 2359–2372 (2006)

    2006

  8. [15]

    Color and luminance information in natural scenes,

    C. P ´arraga, G. Brelstaff, T. Troscianko, and I. Moorehead, “Color and luminance information in natural scenes,” JOSA A 15(3), 563–569 (1998)

    1998

  9. [16]

    Color constancy for multiple light sources,

    A. Gijsenij, R. Lu, and T. Gevers, “Color constancy for multiple light sources,” IEEE Transactions on Image Process- ing 21(2), 697–707 (2011)

    2011

  10. [17]

    The rehabilitation of maxrgb,

    B. Funt and L. Shi, “The rehabilitation of maxrgb,” in Color and imaging conference, 2010(1), 256–259, Society for Imaging Science and Technology (2010)

    2010

  11. [18]

    Solving for colour constancy using a constrained dichromatic reflection model,

    G. D. Finlayson and G. Schaefer, “Solving for colour constancy using a constrained dichromatic reflection model,” International Journal of Computer Vision 42(3), 127–144 (2001)

    2001

  12. [19]

    Image segmentation by object color: a unifying framework and connection to color constancy,

    M. H. Brill, “Image segmentation by object color: a unifying framework and connection to color constancy,” JOSA A 7(10), 2041–2047 (1990)

    1990

  13. [20]

    A comparison of computational color constancy algorithms-part ii: Experiments with image data,

    K. Barnard, L. Martin, A. Coath, and B. Funt, “A comparison of computational color constancy algorithms-part ii: Experiments with image data,” IEEE transactions on Image Processing 11(9), 985–996 (2002)

    2002

  14. [21]

    The Amsterdam library of object images,

    J.-M. Geusebroek, G. J. Burghouts, and A. W. Smeulders, “The Amsterdam library of object images,” International Journal of Computer Vision 61(1), 103–112 (2005)

    2005

  15. [22]

    Color constancy and non-uniform illumination: Can existing algorithms work?,

    M. Bleier, C. Riess, S. Beigpour, E. Eibenberger, E. Angelopoulou, T. Tr ¨oger, and A. Kaup, “Color constancy and non-uniform illumination: Can existing algorithms work?,” in 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 774–781, IEEE (2011)

    2011

  16. [23]

    Yaccd2: yet another color constancy database updated,

    A. Rizzi, C. Bonanomi, D. Gadia, and G. Riopi, “Yaccd2: yet another color constancy database updated,” in Color Imaging XVIII: Displaying, Processing, Hardcopy, and Applications, 8652, 86520A, International Society for Optics and Photonics (2013)

    2013

  17. [25]

    Webplotdigitizer

    A. Rohatgi, “Webplotdigitizer.” https://automeris.io/WebPlotDigitizer

  18. [26]

    Linear colour segmentation revisited,

    A. Smagina, V . Bozhkova, S. Gladilin, and D. Nikolaev, “Linear colour segmentation revisited,” in Eleventh Interna- tional Conference on Machine Vision (ICMV 2018), 11041, 110410F, International Society for Optics and Photonics (2019)

    2019

  19. [27]

    Color homography: theory and applications,

    G. Finlayson, H. Gong, and R. B. Fisher, “Color homography: theory and applications,” IEEE transactions on pattern analysis and machine intelligence 41(1), 20–33 (2017)

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.