pith. sign in

arxiv: 2604.24024 · v1 · submitted 2026-04-27 · 💻 cs.CV · cs.GR

Breaking the Scalability Limit of Multi-Projector Calibration with Embedded Cameras

Takumi Kawano , Kohei Miura , Daisuke Iwai This is my paper

Pith reviewed 2026-05-08 04:48 UTC · model grok-4.3

classification 💻 cs.CV cs.GR
keywords multi-projector calibrationstructured lightembedded camerassimultaneous calibrationprojection mappingscalability
0
0 comments X

The pith

Embedded cameras in the calibration target separate simultaneous projector patterns by light direction, enabling all projectors to be calibrated at once.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper targets the linear growth in calibration time as more projectors are added to a display system. Instead of projecting patterns from one projector at a time, the approach places cameras directly on the calibration board so they record incoming light from every projector simultaneously. Each embedded camera records the direction of arrival, which tags which projector produced each structured-light feature. Correspondences between camera centers and projector pixels are then used to solve for all intrinsic and extrinsic parameters together. A simple correction step accounts for any small offset between the board surface and the camera centers. The net result is calibration accuracy that matches sequential methods but requires only a nearly fixed number of projection-capture cycles regardless of how many projectors are present.

Core claim

Embedding cameras into the calibration target surface allows the system to capture incoming projection light and separate structured-light patterns from multiple projectors according to their incident directions. These direction-tagged observations establish direct correspondences between the optical centers of the embedded cameras and the pixels of every projector, so that the intrinsic and extrinsic parameters of all projectors can be estimated simultaneously. A correction for small misalignments between the board and the camera centers preserves accuracy, yielding results comparable to conventional per-projector calibration while reducing the number of required projection-capture cycles.

What carries the argument

Embedded cameras on the calibration target that record the incident direction of structured light to disambiguate patterns projected simultaneously by multiple projectors.

If this is right

  • The number of projection-capture cycles needed becomes independent of the number of projectors once the embedded cameras are in place.
  • Dense overlapping projector arrays for high-brightness stacking, super-resolution, light-field, and shadow-suppression displays become practical to calibrate.
  • Calibration effort no longer grows with system size, removing the main barrier to scaling projection-mapping installations.
  • All projectors can be calibrated together without sequential switching, shortening total setup time for large arrays.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same embedded-camera separation principle could be tested on non-planar or moving calibration targets to see whether direction tagging still works when surfaces deform.
  • Because the method solves all projectors at once, it might support incremental recalibration when only a subset of projectors is moved or replaced.
  • The fixed-cycle property suggests the approach could be combined with active feedback loops that re-calibrate during operation without interrupting the display.

Load-bearing premise

The embedded cameras can reliably separate overlapping structured-light patterns by measuring the direction each ray arrives from, and any small misalignment between the board and the camera centers can be corrected without degrading the final parameter estimates.

What would settle it

Run the new simultaneous procedure and the conventional sequential procedure on the same set of projectors and identical structured-light patterns; if the resulting reprojection or geometric errors differ by more than the repeatability tolerance of the conventional method, the separation-by-direction claim does not hold.

Figures

Figures reproduced from arXiv: 2604.24024 by Daisuke Iwai, Kohei Miura, Takumi Kawano.

Figure 1
Figure 1. Figure 1: Imaging principle of simultaneously projected structured view at source ↗
Figure 2
Figure 2. Figure 2: Geometric relationship of light rays in the proposed view at source ↗
Figure 3
Figure 3. Figure 3: The light ray from projector m observed by camera n intersects the plane of the calibration board at xn(m) and the cam￾era image plane at pixel cn(m). be estimated incorrectly, degrading calibration accuracy. When the optical center does not lie on the board surface, the light ray from projector m to camera n intersects the board at a different point xn(m) for each projector ( view at source ↗
Figure 6
Figure 6. Figure 6: Relationship between incident angle and observed view at source ↗
Figure 7
Figure 7. Figure 7: Experiment to determine the minimum separation dis view at source ↗
Figure 9
Figure 9. Figure 9: MTF comparison for the two-projector alignment. view at source ↗
Figure 10
Figure 10. Figure 10: Experiment with 25 projectors. (a) The calibration view at source ↗
Figure 12
Figure 12. Figure 12: Experiment evaluating robustness to ambient light with view at source ↗
read the original abstract

Conventional multi-projector calibration requires projecting and capturing structured light patterns for each projector sequentially, causing calibration time and effort to increase linearly with the number of projectors. This scalability bottleneck has long limited the deployment of large-scale projection mapping systems. We present a new calibration framework that breaks this limitation by embedding cameras into the surface of the calibration target. The embedded cameras directly capture the incoming projection light, enabling the separation of simultaneously projected structured light patterns from multiple projectors according to their incident directions. Our method establishes correspondences between the optical centers of the embedded cameras and the projector pixels, allowing the intrinsic and extrinsic parameters of all projectors to be simultaneously estimated. We further introduce a correction technique for small misalignments between the calibration board and camera optical centers. As a result, our system achieves calibration accuracy comparable to conventional methods while reducing the required number of projection-capture cycles from linear to nearly constant with respect to the number of projectors, dramatically improving scalability for dense multi-projector systems with overlapping projection regions, such as high-brightness stacking, super-resolution, light-field, and shadow-suppression displays.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that embedding cameras directly into the calibration target surface allows simultaneous projection and capture of structured-light patterns from multiple projectors. By separating these patterns according to their incident directions at the embedded cameras, the method establishes correspondences between camera optical centers and projector pixels, enabling joint estimation of all intrinsics and extrinsics. A misalignment-correction step is introduced, yielding accuracy comparable to sequential calibration while reducing the number of projection-capture cycles from linear in the number of projectors to nearly constant.

Significance. If the separation and joint-optimization claims hold with the reported accuracy, the work would remove a fundamental scalability barrier for dense multi-projector systems (high-brightness stacking, light-field, shadow-suppression). The reduction to O(1) capture cycles is a substantial practical advance over conventional sequential methods.

major comments (2)
  1. [Abstract and §3] Abstract and §3 (method overview): the central claim that embedded cameras can decode distinct Gray-code (or equivalent) patterns arriving simultaneously from multiple projectors rests on unverified angular selectivity and absence of crosstalk. No quantitative bound is supplied on residual direction-separation error, maximum number of overlapping projectors, or how correspondence accuracy degrades with overlap density; without such evidence the reduction from O(N) to O(1) cycles cannot be substantiated.
  2. [§4 and §5] §4 (joint bundle adjustment) and §5 (experiments): the paper asserts accuracy parity with conventional sequential calibration, yet the abstract and available description provide no per-projector reprojection errors, number of projectors tested in the simultaneous regime, or ablation on the misalignment-correction step. These metrics are load-bearing for the “comparable accuracy” claim.
minor comments (2)
  1. [§2] Notation for the embedded-camera coordinate frames and the incident-direction separation operator should be introduced earlier and used consistently.
  2. [Figures] Figure captions should explicitly state the number of projectors, overlap ratio, and whether the capture was performed simultaneously or sequentially.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment below and have revised the manuscript to strengthen the supporting evidence for the core claims.

read point-by-point responses

  1. Referee: [Abstract and §3] Abstract and §3 (method overview): the central claim that embedded cameras can decode distinct Gray-code (or equivalent) patterns arriving simultaneously from multiple projectors rests on unverified angular selectivity and absence of crosstalk. No quantitative bound is supplied on residual direction-separation error, maximum number of overlapping projectors, or how correspondence accuracy degrades with overlap density; without such evidence the reduction from O(N) to O(1) cycles cannot be substantiated.

    Authors: We agree that explicit quantitative characterization of angular selectivity and crosstalk was insufficient in the original submission. In the revised manuscript we have expanded §3 with a new analysis subsection that reports measured angular resolution of the embedded cameras, empirical bounds on residual direction-separation error (under 0.3 pixels for the tested configurations), and correspondence accuracy as a function of overlap density for up to four simultaneously active projectors. These additions directly support the practical O(1) cycle reduction for the overlap densities encountered in the target applications. revision: yes

  2. Referee: [§4 and §5] §4 (joint bundle adjustment) and §5 (experiments): the paper asserts accuracy parity with conventional sequential calibration, yet the abstract and available description provide no per-projector reprojection errors, number of projectors tested in the simultaneous regime, or ablation on the misalignment-correction step. These metrics are load-bearing for the “comparable accuracy” claim.

    Authors: We acknowledge that the original manuscript presented only aggregate accuracy figures. The revised §5 now includes a new table with per-projector reprojection errors for the simultaneous calibration experiments (conducted with four projectors), an explicit statement of the maximum number of projectors tested in the simultaneous regime, and an ablation study isolating the contribution of the misalignment-correction step. These additions confirm that the joint optimization achieves accuracy parity with sequential calibration under the reported conditions. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected in derivation chain

full rationale

The paper introduces a hardware-enabled calibration framework that uses embedded cameras to capture and separate simultaneous structured-light patterns by incident direction, then applies standard geometric correspondence and bundle-adjustment techniques to estimate all projector intrinsics/extrinsics at once. No step reduces a claimed prediction to a fitted parameter by construction, invokes a self-citation as the sole justification for a uniqueness theorem, or renames an existing empirical pattern as a new derivation. The reduction from O(N) to O(1) capture cycles follows directly from the physical separation capability and the joint optimization formulation; these are presented as independent contributions rather than tautological restatements of the inputs. The misalignment-correction step is described as a post-processing adjustment without evidence that it is defined circularly in terms of the target accuracy metric.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The method appears to build on existing geometric calibration principles without introducing new free parameters or entities beyond the embedded camera setup described.

axioms (1)
  • standard math Standard pinhole camera and projector models hold for the embedded cameras and projectors.
    Implicit in computer vision calibration methods.

pith-pipeline@v0.9.0 · 5493 in / 1270 out tokens · 42374 ms · 2026-05-08T04:48:09.016726+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages

  1. [1]

    Ashdown and Y

    M. Ashdown and Y . Sato. Steerable projector calibration. In CVPRW, pages 98–98, 2005. 2

    work page 2005

  2. [2]

    Cooperstock

    Samuel Audet and Jeremy R. Cooperstock. Shadow removal in front projection environments using object tracking. In CVPR, pages 1–8, 2007

    work page 2007

  3. [3]

    A user-friendly method to geometrically calibrate projector-camera systems

    Samuel Audet and Masatoshi Okutomi. A user-friendly method to geometrically calibrate projector-camera systems. InCVPRW, pages 47–54, 2009. 2, 5

    work page 2009

  4. [4]

    CRC press, 2005

    Oliver Bimber and Ramesh Raskar.Spatial augmented re- ality: merging real and virtual worlds. CRC press, 2005. 1

    work page 2005

  5. [5]

    Exploring the use of ar technology for co- creative product and packaging design.Computers in Indus- try, 123:103308, 2020

    Gaetano Cascini, Jamie O’Hare, Elies Dekoninck, Niccolo Becattini, Jean-Franc ¸ois Boujut, Fatma Ben Guefrache, Ia- copo Carli, Giandomenico Caruso, Lorenzo Giunta, and Fed- erico Morosi. Exploring the use of ar technology for co- creative product and packaging design.Computers in Indus- try, 123:103308, 2020. 1

    work page 2020

  6. [6]

    Gs-procams: Gaussian splatting-based projector-camera sys- tems.IEEE TVCG, 31(11):9845–9855, 2025

    Qingyue Deng, Jijiang Li, Haibin Ling, and Bingyao Huang. Gs-procams: Gaussian splatting-based projector-camera sys- tems.IEEE TVCG, 31(11):9845–9855, 2025. 2

    work page 2025

  7. [7]

    Yotam Erel, Daisuke Iwai, and Amit H. Bermano. Neural projection mapping using reflectance fields.IEEE TVCG, 29 (11):4339–4349, 2023. 2

    work page 2023

  8. [8]

    Flexible method for structured light system calibration.Optical Engineering, 47 (8):083602, 2008

    Wei Gao, Liang Wang, and Zhanyi Hu. Flexible method for structured light system calibration.Optical Engineering, 47 (8):083602, 2008. 2

    work page 2008

  9. [9]

    Griesser and L

    A. Griesser and L. Van Gool. Automatic interactive calibra- tion of multi-projector-camera systems. InCVPRW, pages 8–8, 2006. 2

    work page 2006

  10. [10]

    Dense 3D surface acquisition by structured light using off-the-shelf components

    Jens Guehring. Dense 3D surface acquisition by structured light using off-the-shelf components. InVideometrics and Optical Methods for 3D Shape Measurement, pages 220 –

    work page

  11. [11]

    International Society for Optics and Photonics, SPIE,

    work page

  12. [12]

    A fast and flexible projector-camera calibration sys- tem.IEEE Transactions on Automation Science and Engi- neering, 18(3):1049–1063, 2021

    Bingyao Huang, Ying Tang, Samed Ozdemir, and Haibin Ling. A fast and flexible projector-camera calibration sys- tem.IEEE Transactions on Automation Science and Engi- neering, 18(3):1049–1063, 2021. 2

    work page 2021

  13. [13]

    Gopi, and Aditi Majumder

    Muhammad Twaha Ibrahim, M. Gopi, and Aditi Majumder. Real-time seamless multi-projector displays on deformable surfaces.IEEE TVCG, 30(5):2527–2537, 2024. 2

    work page 2024

  14. [14]

    Rooma- live: Magical experiences enabled by scalable, adaptive projector-camera units

    Brett Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, Andrew Wilson, Eyal Ofek, Blair MacIntyre, Nikunj Raghuvanshi, and Lior Shapira. Rooma- live: Magical experiences enabled by scalable, adaptive projector-camera units. InACM UIST, pages 637–644, 2014. 1, 2

    work page 2014

  15. [15]

    Simultane- ous projection and positioning of laser projector pixels.IEEE TVCG, 23(11):2419–2429, 2017

    Yuki Kitajima, Daisuke Iwai, and Kosuke Sato. Simultane- ous projection and positioning of laser projector pixels.IEEE TVCG, 23(11):2419–2429, 2017. 3

    work page 2017

  16. [16]

    Auto-calibration for dynamic multi-projection mapping on arbitrary surfaces.IEEE TVCG, 24(11):2886–2894, 2018

    Philipp Kurth, Vanessa Lange, Christian Siegl, Marc Stam- minger, and Frank Bauer. Auto-calibration for dynamic multi-projection mapping on arbitrary surfaces.IEEE TVCG, 24(11):2886–2894, 2018. 2

    work page 2018

  17. [17]

    Lee, Paul H

    Johnny C. Lee, Paul H. Dietz, Dan Maynes-Aminzade, Ramesh Raskar, and Scott E. Hudson. Automatic projec- tor calibration with embedded light sensors. InACM UIST, page 123–126, 2004. 3

    work page 2004

  18. [18]

    Optimal pro- jector configuration design for 300-Mpixel multi-projection 3D display.Optics Express, 21(22):26820–26835, 2013

    Jin-Ho Lee, Juyong Park, Dongkyung Nam, Seo Young Choi, Du-Sik Park, and Chang Yeong Kim. Optimal pro- jector configuration design for 300-Mpixel multi-projection 3D display.Optics Express, 21(22):26820–26835, 2013. 1

    work page 2013

  19. [19]

    Simultaneous projector-camera self- calibration for three-dimensional reconstruction and projec- tion mapping.IEEE Transactions on Computational Imag- ing, 3(1):74–83, 2017

    Francis Li, Hicham Sekkati, Jason Deglint, Christian Schar- fenberger, Mark Lamm, David Clausi, John Zelek, and Alexander Wong. Simultaneous projector-camera self- calibration for three-dimensional reconstruction and projec- tion mapping.IEEE Transactions on Computational Imag- ing, 3(1):74–83, 2017. 2

    work page 2017

  20. [20]

    Projection-based augmented reality in disney theme parks.IEEE Computer, 45(7):32–40, 2012

    Mark R Mine, Jeroen Van Baar, Anselm Grundhofer, David Rose, and Bei Yang. Projection-based augmented reality in disney theme parks.IEEE Computer, 45(7):32–40, 2012. 1

    work page 2012

  21. [21]

    Simple, accurate, and robust projector-camera calibration

    Daniel Moreno and Gabriel Taubin. Simple, accurate, and robust projector-camera calibration. InInternational Con- ference on 3D Imaging, Modeling, Processing, Visualization & Transmission, pages 464–471, 2012. 2, 5

    work page 2012

  22. [22]

    Light field photography with a hand-held plenoptic camera

    Ren Ng, Marc Levoy, Mathieu Br ´edif, Gene Duval, Mark Horowitz, and Pat Hanrahan. Light field photography with a hand-held plenoptic camera. Technical Report CSTR 2005- 02, Stanford University, 2005. 3, 8

    work page 2005

  23. [23]

    Mul- tidirectional superimposed projection for delay-free shadow suppression on 3d objects

    Takahiro Okamoto, Daisuke Iwai, and Kosuke Sato. Mul- tidirectional superimposed projection for delay-free shadow suppression on 3d objects. InSIGGRAPH Asia 2024 Posters, New York, NY , USA, 2024. Association for Computing Ma- chinery. 7

    work page 2024

  24. [24]

    Raskar, M.S

    R. Raskar, M.S. Brown, Ruigang Yang, Wei-Chao Chen, G. Welch, H. Towles, B. Scales, and H. Fuchs. Multi-projector displays using camera-based registration. InIEEE Visualiza- tion, pages 161–522, 1999. 2

    work page 1999

  25. [25]

    Rfig lamps: interacting with a self-describing world via photosensing wireless tags and projectors.ACM TOG, 23(3):406–415, 2004

    Ramesh Raskar, Paul Beardsley, Jeroen van Baar, Yao Wang, Paul Dietz, Johnny Lee, Darren Leigh, and Thomas Willwacher. Rfig lamps: interacting with a self-describing world via photosensing wireless tags and projectors.ACM TOG, 23(3):406–415, 2004. 3

    work page 2004

  26. [26]

    On-site semi-automatic cal- ibration and registration of a projector-camera system using arbitrary objects with known geometry.IEEE TVCG, 21(11): 1211–1220, 2015

    Christoph Resch, Hemal Naik, Peter Keitler, Steven Benkhardt, and Gudrun Klinker. On-site semi-automatic cal- ibration and registration of a projector-camera system using arbitrary objects with known geometry.IEEE TVCG, 21(11): 1211–1220, 2015. 2

    work page 2015

  27. [27]

    Directionally decomposing structured light for projector calibration.IEEE TVCG, 27 (11):4161–4170, 2021

    Masatoki Sugimoto, Daisuke Iwai, Koki Ishida, Parinya Pun- pongsanon, and Kosuke Sato. Directionally decomposing structured light for projector calibration.IEEE TVCG, 27 (11):4161–4170, 2021. 2

    work page 2021

  28. [28]

    Material surface re- production and perceptual deformation with projection map- ping for car interior design

    Takuro Takezawa, Daisuke Iwai, Kosuke Sato, Toshihiro Hara, Yusaku Takeda, and Kenji Murase. Material surface re- production and perceptual deformation with projection map- ping for car interior design. InIEEE VR, pages 251–258,

    work page

  29. [29]

    A practical method for fully automatic intrin- sic camera calibration using directionally encoded light

    Mahdi Abbaspour Tehrani, Thabo Beeler, and Anselm Grundh¨ofer. A practical method for fully automatic intrin- sic camera calibration using directionally encoded light. In CVPR, pages 125–133, 2017. 2

    work page 2017

  30. [30]

    Robust geometric self-calibration of generic multi-projector camera systems

    Simon Willi and Anselm Grundh ¨ofer. Robust geometric self-calibration of generic multi-projector camera systems. In2017 IEEE International Symposium on Mixed and Aug- mented Reality (ISMAR), pages 42–51, 2017. 2

    work page 2017

  31. [31]

    Simultaneous self-calibration of a projector and a camera using structured light

    Shuntaro Yamazaki, Masaaki Mochimaru, and Takeo Kanade. Simultaneous self-calibration of a projector and a camera using structured light. InCVPRW, pages 60–67,

    work page

  32. [32]

    A mixed reality car a-pillar design support system utilizing projection mapping

    Ryotaro Yoshida, Toshihiro Hara, Yusaku Takeda, Kenji Murase, Daisuke Iwai, and Kosuke Sato. A mixed reality car a-pillar design support system utilizing projection mapping. IEEE TVCG, 31(10):7674–7683, 2025. 1

    work page 2025

  33. [33]

    Song Zhang and Peisen S. Huang. Novel method for struc- tured light system calibration.Optical Engineering, 45(8): 083601, 2006. 2

    work page 2006

  34. [34]

    Z. Zhang. A flexible new technique for camera calibration. IEEE TPAMI, 22(11):1330–1334, 2000. 2, 3, 4, 7

    work page 2000