Setup-Independent Full Projector Compensation
Pith reviewed 2026-05-13 21:28 UTC · model grok-4.3
The pith
SIComp enables full projector compensation that generalizes to unseen setups without retraining or fine-tuning.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
SIComp is the first Setup-Independent framework for full projector Compensation, capable of generalizing to unseen setups without fine-tuning or retraining. It achieves this through a large-scale real-world dataset spanning 277 distinct projector-camera setups and a co-adaptive design that decouples geometry and photometry: an optical flow module for online geometric correction and a photometric network for compensation enhanced by intensity-varying surface priors.
What carries the argument
The co-adaptive optical flow module for geometry and photometric network with intensity-varying surface priors that together enable setup-independent compensation.
If this is right
- SIComp consistently produces high-quality compensation across diverse unseen setups.
- It substantially outperforms existing methods in generalization ability.
- It establishes the first generalizable solution to projector compensation.
- The approach decouples geometry and photometry to improve robustness under varying conditions.
Where Pith is reading between the lines
- Such methods could enable dynamic projection mapping in environments where setups change frequently.
- Similar dataset-driven generalization strategies may benefit other computer vision tasks involving physical hardware configurations.
- Future extensions could incorporate real-time adaptation for moving projectors or surfaces.
Load-bearing premise
The co-adaptive optical-flow geometry module and photometric network trained on the 277-setup dataset will generalize to arbitrary new setups without retraining.
What would settle it
A significant drop in compensation quality on a projector-camera setup with novel geometry or illumination not covered in the training dataset would falsify the generalization claim.
Figures
read the original abstract
Projector compensation seeks to correct geometric and photometric distortions that occur when images are projected onto nonplanar or textured surfaces. However, most existing methods are highly setup-dependent, requiring fine-tuning or retraining whenever the surface, lighting, or projector-camera pose changes. Progress has been limited by two key challenges: (1) the absence of large, diverse training datasets and (2) existing geometric correction models are typically constrained by specific spatial setups; without further retraining or fine-tuning, they often fail to generalize directly to novel geometric configurations. We introduce SIComp, the first Setup-Independent framework for full projector Compensation, capable of generalizing to unseen setups without fine-tuning or retraining. To enable this, we construct a large-scale real-world dataset spanning 277 distinct projector-camera setups. SIComp adopts a co-adaptive design that decouples geometry and photometry: A carefully tailored optical flow module performs online geometric correction, while a novel photometric network handles photometric compensation. To further enhance robustness under varying illumination, we integrate intensity-varying surface priors into the network design. Extensive experiments demonstrate that SIComp consistently produces high-quality compensation across diverse unseen setups, substantially outperforming existing methods in terms of generalization ability and establishing the first generalizable solution to projector compensation. The code and dataset are available on our project page: https://hai-bo-li.github.io/SIComp/
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces SIComp as the first setup-independent framework for full projector compensation. It constructs a real-world dataset spanning 277 distinct projector-camera setups and employs a co-adaptive design that decouples geometry (via a tailored optical-flow module for online correction) from photometry (via a neural network incorporating intensity-varying surface priors). The central claim is that this architecture generalizes to arbitrary unseen setups without fine-tuning or retraining, as evidenced by extensive experiments showing consistent high-quality compensation and outperformance over prior methods.
Significance. If the generalization results hold under rigorous verification, the work would constitute a notable advance in projector compensation by addressing the long-standing setup-dependency limitation. The release of code and the 277-setup dataset provides a concrete resource for the community and supports reproducibility, which strengthens the potential impact beyond the immediate claims.
major comments (2)
- [Abstract] Abstract: the load-bearing claim of generalization to arbitrary unseen setups without retraining is not supported by any explicit quantification of the 277-setup training distribution (e.g., ranges of projector tilt angles, surface curvature radii, texture frequencies, or illumination spectra). Absent these statistics it remains possible that reported test cases lie inside the convex hull of training variations, undermining the setup-independence assertion.
- [Experiments] Experiments section: the reported outperformance lacks error bars, explicit data-exclusion criteria, and full ablation details on the co-adaptive optical-flow and photometric modules. These omissions prevent direct verification that the observed gains stem from the claimed generalization rather than dataset-specific fitting.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback and positive assessment of the potential impact of SIComp. We address each major comment below and will incorporate revisions to strengthen the manuscript's rigor and clarity regarding generalization claims.
read point-by-point responses
-
Referee: [Abstract] Abstract: the load-bearing claim of generalization to arbitrary unseen setups without retraining is not supported by any explicit quantification of the 277-setup training distribution (e.g., ranges of projector tilt angles, surface curvature radii, texture frequencies, or illumination spectra). Absent these statistics it remains possible that reported test cases lie inside the convex hull of training variations, undermining the setup-independence assertion.
Authors: We appreciate this observation. While the experiments section describes the 277 setups, we agree that explicit quantification of the training distribution is needed to robustly support the generalization claim. In the revised manuscript, we will add a dedicated subsection and table summarizing the statistical ranges for projector tilt angles, surface curvature radii, texture frequencies, and illumination spectra across the 277 setups. This will explicitly demonstrate that the held-out test configurations lie outside the training distribution, thereby reinforcing the setup-independence assertion. revision: yes
-
Referee: [Experiments] Experiments section: the reported outperformance lacks error bars, explicit data-exclusion criteria, and full ablation details on the co-adaptive optical-flow and photometric modules. These omissions prevent direct verification that the observed gains stem from the claimed generalization rather than dataset-specific fitting.
Authors: We agree that these reporting elements are essential for rigorous verification. In the revision, we will augment the experiments section with error bars (standard deviations) on all quantitative results to reflect variability across setups and runs. We will also detail the data-exclusion criteria applied during dataset construction. Additionally, we will expand the ablation studies to fully isolate the contributions of the optical-flow geometry module and the photometric network (including the intensity-varying surface priors), with corresponding quantitative comparisons. These changes will clarify that performance improvements arise from the co-adaptive design rather than dataset-specific fitting. revision: yes
Circularity Check
No significant circularity; claims rest on new external dataset and standard components
full rationale
The paper constructs a new 277-setup real-world dataset and applies co-adaptive optical-flow geometry plus photometric networks with intensity-varying priors. No equations reduce the generalization claim to fitted parameters by construction, no self-citations are load-bearing for the core premise, and no ansatz or uniqueness result is imported from prior author work. The derivation chain is self-contained: dataset collection is independent, and performance on unseen setups is evaluated externally rather than forced by input definitions.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Optical flow accurately models geometric distortions between projector and camera in real-world setups
Reference graph
Works this paper leans on
-
[1]
H. Aoki, T. Tochimoto, Y . Hiroi, and Y . Itoh. Towards co-operative beaming displays: Dual steering projectors for extended projection volume and head orientation range.IEEE TVCG, 30(5):2309–2318, 2024. 1
work page 2024
-
[2]
S. Audet and M. Okutomi. A user-friendly method to geometrically calibrate projector-camera systems. InCVPRW, pp. 47–54, 2009. 2
work page 2009
-
[3]
A. H. Bermano, M. Billeter, D. Iwai, and A. Grundhöfer. Makeup lamps: Live augmentation of human faces via projection.Comp. Graph. Forum, 36(2):311–323, 2017. 1 9
work page 2017
- [4]
-
[5]
J. Bluteau, I. Kitahara, Y . Kameda, H. Noma, K. Kogure, and Y . Ohta. Visual support for medical communication by using projector-based augmented reality and thermal markers. InIn ICAT, pp. 98–105, 2005. 1
work page 2005
-
[6]
D. J. Butler, J. Wulff, G. B. Stanley, and M. J. Black. A naturalistic open source movie for optical flow evaluation. InECCV, pp. 611–625, 2012. 4, 5
work page 2012
-
[7]
J.-C. Chien, J.-D. Lee, C.-W. Chang, and C.-T. Wu. A projection- based augmented reality system for medical applications.Appl. Sci., 12(23):12027, 2022. 1
work page 2022
-
[8]
C. Cruz-Neira, D. J. Sandin, and T. A. DeFanti. Surround-screen projection-based virtual reality: the design and implementation of the CA VE. InSIGGRAPH, pp. 135–142, 1993. 1
work page 1993
-
[9]
Q. Deng, J. Li, H. Ling, and B. Huang. GS-ProCams: Gaussian splatting- based projector-camera systems.IEEE TVCG, 2025. 2, 3
work page 2025
-
[10]
P. Edgcumbe, R. Singla, P. Pratt, C. Schneider, C. Nguan, and R. Rohling. Follow the light: projector-based augmented reality in- tracorporeal system for laparoscopic surgery.J. Med. Imag., 5(2):021216,
-
[11]
Y . Erel, D. Iwai, and A. H. Bermano. Neural projection mapping using reflectance fields.IEEE TVCG, 29(11):4339–4349, 2023. 2, 3
work page 2023
-
[12]
Y . Erel, O. Kozlovsky-Mordenfeld, D. Iwai, K. Sato, and A. H. Bermano. Casper DPM: Cascaded perceptual dynamic projection mapping onto hands. InSIGGRAPH Asia, pp. 1–10, 2024. 1
work page 2024
- [13]
-
[14]
A. Grundhöfer. Practical non-linear photometric projector compensation. InCVPRW, pp. 924–929, 2013. 2
work page 2013
-
[15]
A. Grundhöfer and D. Iwai. Robust, error-tolerant photometric projector compensation.IEEE TIP, 24(12):5086–5099, 2015. 1, 2
work page 2015
-
[16]
A. Grundhöfer and D. Iwai. Recent advances in projection mapping algorithms, hardware and applications.Comput. Graph. Forum, 37:653– 675, 2018. 1
work page 2018
-
[17]
M. Harville, B. Culbertson, I. Sobel, D. Gelb, A. Fitzhugh, and D. Tan- guay. Practical methods for geometric and photometric correction of tiled projector. InCVPR, pp. 5–5, 2006. 1
work page 2006
- [18]
-
[19]
B. Huang and H. Ling. CompenNet++: End-to-end full projector compensation. InICCV, pp. 7165–7174, 2019. 1, 2
work page 2019
-
[20]
B. Huang and H. Ling. End-to-end projector photometric compensation. InCVPR, pp. 6810–6819, 2019. 1, 2, 4, 6, 7
work page 2019
-
[21]
B. Huang and H. Ling. DeProCams: Simultaneous relighting, compensa- tion and shape reconstruction for projector-camera systems.IEEE TVCG, 27(5):2725–2735, 2021. 1, 3, 6, 8, 2
work page 2021
- [22]
- [23]
- [24]
-
[25]
M. T. Ibrahim, M. Gopi, and A. Majumder. Projector-camera calibration on dynamic, deformable surfaces. InIEEE VRW, pp. 905–906, 2023. 2
work page 2023
-
[26]
M. T. Ibrahim, M. Gopi, and A. Majumder. Self-calibrating dynamic projection mapping system for dynamic, deformable surfaces with jitter correction and occlusion handling. InISMAR, pp. 293–302, 2023. 2
work page 2023
-
[27]
M. T. Ibrahim, M. Gopi, and A. Majumder. Real-time seamless multi- projector displays on deformable surfaces.IEEE TVCG, 30(5):2527– 2537, 2024. 2
work page 2024
-
[28]
M. T. Ibrahim and A. Majumder. Multi-projector dynamic spatially augmented reality on deformable surfaces. InIEEE VRW, pp. 1156–1157,
-
[29]
M. T. Ibrahim, G. Meenakshisundaram, and A. Majumder. Dynamic projection mapping of deformable stretchable materials. InVRST, pp. 1–5, 2020. 2
work page 2020
-
[30]
D. Iwai. Projection mapping technologies: A review of current trends and future directions.Proc. Jpn. Acad., Ser. B, 100(3):234–251, 2024. 1
work page 2024
-
[31]
S. Kagami and K. Hashimoto. Animated stickies: Fast video projection mapping onto a markerless plane through a direct closed-loop alignment. IEEE TVCG, 25(11):3094–3104, 2019. 2
work page 2019
-
[32]
S. Kagami and K. Hashimoto. Interactive stickies: Low-latency pro- jection mapping for dynamic interaction with projected images on a movable surface. InSIGGRAPH Emerging Technol., pp. 1–2, 2020. 2
work page 2020
-
[33]
Y . Kageyama, D. Iwai, and K. Sato. Online projector deblurring using a convolutional neural network.IEEE TVCG, 28(5):2223–2233, 2022. 1
work page 2022
-
[34]
Y . Kageyama, D. Iwai, and K. Sato. Efficient distortion-free neural projector deblurring in dynamic projection mapping.IEEE TVCG, 30(12):7544–7557, 2024. 1
work page 2024
-
[35]
O. V . Kanivets, I. M. Kanivets, N. V . Kononets, T. M. Gorda, and E. O. Shmeltser. Development of mobile applications of augmented reality for projects with projection drawings. InIn AREdu, pp. 262–273, 2020. 1
work page 2020
-
[36]
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR, 2015. 6
work page 2015
- [37]
- [38]
-
[39]
J. Li, Q. Deng, H. Ling, and B. Huang. DPCS: Path tracing-based differentiable projector-camera systems.IEEE TVCG, 2025. 2, 3, 6, 8
work page 2025
-
[40]
Y . Li, W. Yin, J. Li, and X. Xie. Physics-based efficient full projector compensation using only natural images.IEEE TVCG, 30(8):4968–4982,
-
[41]
S. Liu, Q. Ruan, and X. Li. The color calibration across multi-projector display.J. Signal Inf. Process., 2(2):53, 2011. 1, 2
work page 2011
-
[42]
Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pp. 10012–10022, 2021. 2, 3, 6
work page 2021
-
[43]
B. D. Lucas and T. Kanade. An iterative image registration technique with an application to stereo vision. InIJCAI, vol. 2, pp. 674–679, 1981. 1
work page 1981
-
[44]
D. Moreno and G. Taubin. Simple, accurate, and robust projector-camera calibration. In3DIMPVT, pp. 464–471, 2012. 2
work page 2012
- [45]
-
[46]
S. K. Nayar, H. Peri, M. D. Grossberg, and P. N. Belhumeur. A projection system with radiometric compensation for screen imperfections. InICCV,
-
[47]
J. Park, D. Jung, and M. Bochang. Projector compensation framework using differentiable rendering.IEEE Access, 10:44461–44470, 2022. 2, 3
work page 2022
-
[48]
H.-L. Peng, K. Sato, S. Nakagawa, and Y . Watanabe. Perceptually- aligned dynamic facial projection mapping by high-speed face-tracking method and lens-shift co-axial setup.IEEE TVCG, 2025. 1
work page 2025
- [49]
- [50]
-
[51]
Y . Sato, D. Iwai, and K. Sato. Responsive-extendedhand: Adaptive visuo-haptic feedback recognizing object property with RGB-D camera for projected extended hand.IEEE Access, 12:38247–38257, 2024. 1
work page 2024
-
[52]
M. Shahpaski, L. R. Sapaico, G. Chevassus, and S. Susstrunk. Simulta- neous geometric and radiometric calibration of a projector-camera pair. InCVPR, pp. 4885–4893, 2017. 3
work page 2017
- [53]
- [54]
-
[55]
T. Takahashi, T. Kawano, K. Ito, T. Aoki, and S. Kondo. Performance evaluation of a geometric correction method for multi-projector display using SIFT and phase-only correlation. InICIP, pp. 1189–1192, 2010. 2
work page 2010
-
[56]
M. Takeuchi, H. Kusuyama, D. Iwai, and K. Sato. Projection mapping un- der environmental lighting by replacing room lights with heterogeneous projectors.IEEE TVCG, 30(5):2151–2161, 2024. 1
work page 2024
- [57]
-
[58]
Z. Teed and J. Deng. RAFT: Recurrent all-pairs field transforms for optical flow. InECCV, pp. 402–419, 2020. 3
work page 2020
-
[59]
M. A. Tehrani, M. Gopi, and A. Majumder. Automated geometric registration for multi-projector displays on arbitrary 3D shapes using uncalibrated devices.IEEE TVCG, 27(4):2265–2279, 2019. 2
work page 2019
-
[60]
M. A. Tehrani, M. T. Ibrahim, A. Majumder, and M. Gopi. 3D gamut morphing for non-rectangular multi-projector displays.IEEE TVCG, 30(8):4724–4738, 2023. 2
work page 2023
-
[61]
T. Ueda, D. Iwai, T. Hiraki, and K. Sato. Illuminated focus: Vision augmentation using spatial defocusing via focal sweep eyeglasses and high-speed projector.IEEE TVCG, 26(5):2051–2061, 2020. 1
work page 2051
-
[62]
D. Wang, I. Sato, T. Okabe, and Y . Sato. Radiometric compensation in a projector-camera system based properties of human vision system. In CVPRW, pp. 100–100, 2005. 2
work page 2005
-
[63]
J. Wang, Z. Zhang, W. Lu, and X. J. Jiang. High-accuracy calibration of high-speed fringe projection profilometry using a checkerboard.T-Mech, 27(5):4199–4204, 2022. 2
work page 2022
-
[64]
Y . Wang, H. Ling, and B. Huang. CompenHR: Efficient full compensa- tion for high-resolution projector. InIEEE VR, pp. 135–145, 2023. 1, 2, 3, 4, 5, 6, 7, 8
work page 2023
-
[65]
Y . Wang, H. Ling, and B. Huang. ViComp: Video compensation for projector-camera systems.IEEE TVCG, 2024. 3, 4
work page 2024
-
[66]
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: From error visibility to structural similarity.IEEE TIP, 13(4):600–612, 2004. 7
work page 2004
-
[67]
G. Wetzstein and O. Bimber. Radiometric compensation through inverse light transport. InPacific Graphics, pp. 391–399, 2007. 2, 3
work page 2007
-
[68]
S. Woo, J. Park, J.-Y . Lee, and I. S. Kweon. Cbam: Convolutional block attention module. InECCV, pp. 3–19, 2018. 2, 3, 6
work page 2018
-
[69]
K. Yamamoto, D. Iwai, I. Tani, and K. Sato. A monocular projector- camera system using modular architecture.IEEE TVCG, 29(12):5586– 5592, 2022. 1
work page 2022
- [70]
-
[71]
T. Yoshida, C. Horii, and K. Sato. A virtual color reconstruction system for real heritage with light projection. InVSMM, pp. 1–7, 2003. 1, 2
work page 2003
-
[72]
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, pp. 586–595, 2018. 7 11 Accepted by IEEE Transactions on Visualization and Computer Graphics (TVCG) Setup-Independent Full Projector Compensation — Supplementary Materials — A SURROGATECOMPENSATIONEXPERIMENT A.1 Qual...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.