Pith. sign in

REVIEW 3 major objections 8 minor 53 references

MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generation

T0 review · 3 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MV-CoLight claims that compositing an object with scene-matched lighting and shadows can be done feed-forward in 0.07 seconds per frame, without diffusion sampling.

desk verdict A solid, fast feed-forward compositing pipeline with a genuinely useful dataset, but the headline generalization claim depends on clean background and depth inputs that casual capture rarely provides. read the letter →

arxiv 2505.21483 v1 pith:ZBN2DT5S submitted 2025-05-27 cs.CV

Kerui Ren , Jiayang Bai , Linning Xu , Lihan Jiang , Jiangmiao Pang , Mulin Yu , Bo Dai This is my paper
classification cs.CV
keywords objectcompositingmulti-viewconsistencylightingharmonizationshadowsynthesis3DGaussiansplattingHilbertcurvefeed-forwardnetworksyntheticdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MV-CoLight tries to make inserting a new object into a photograph or a multi-view 3D scene look physically believable: the inserted object must take on the scene's lighting and cast the shadows that the scene's light sources would produce. The paper argues this can be done without slow per-scene optimization or stochastic diffusion sampling, using a feed-forward network that directly predicts harmonized colors. A Hilbert curve orders 3D Gaussian scene primitives into a compact 2D grid, allowing a single transformer to reason about lighting and shadows across views at once. The paper reports roughly 0.07 seconds per image for 2D compositing and better PSNR, SSIM, and LPIPS than the compared 2D and inverse-rendering baselines on its own multi-light dataset and on public benchmarks. It also introduces a synthetic dataset of over 480,000 composite scenes for training and evaluation.

What carries the argument

The load-bearing mechanism is the Hilbert curve ordering, which converts sparse 3D Gaussian primitives into a dense, locality-preserving 2D grid (the paper calls this mapping Φ for feature maps and Ψ for Gaussian colors). This lets an image-based transformer operate directly on 3D scene colors: nearby Gaussians stay nearby in the 2D grid, preserving spatial coherence and enabling multi-view-consistent shadow and highlight prediction. The other essential pieces are the per-view 2D Swin-transformer compositing network and the 3D refinement stage, which freezes all Gaussian geometry and updates only color attributes.

What would settle it

Render a synthetic scene with a known single light source, insert an object, run MV-CoLight, and compare the predicted cast shadow's direction and penumbra against a ray-traced ground-truth composite. If the network's shadows diverge systematically under complex lighting, or if removing the background-only reference degrades the result substantially, the claimed lighting consistency would be falsified for those settings.

Watch

Extended reading notes

Core claim

The paper's central claim is that illumination-consistent object compositing in both single images and multi-view scenes reduces to a learned per-pixel color transformation, not a generative reconstruction problem. Its two-stage framework first runs a per-view 2D network on the composite image, a background-only reference, and a depth map to produce a harmonized image and dense features; then a 3D stage projects those features onto a 3D Gaussian scene model, orders the Gaussians along a Hilbert curve, and predicts new Gaussian colors that keep lighting and shadows consistent across viewpoints. The authors assert that this design exceeds the harmonization quality of diffusion-based inserters and Gaussian-based inverse-rendering baselines while being orders of magnitude faster at inference, and that it generalizes to casually captured real-world scenes.

Load-bearing premise

The load-bearing premise is that every input view comes with a clean background-only image and an accurate depth map, along with known camera poses; the paper's own unposed-input experiments show that quality drops sharply when poses and depths are estimated instead of captured.

Editorial extensions

If this is right

  • If correct, AR and robotics applications could re-render an inserted object with consistent lighting in real time, since the per-frame cost is about 0.07 seconds without diffusion sampling.
  • Repositioning or replacing inserted objects would not require re-optimizing the Gaussian scene, because geometry stays frozen and only colors are predicted, so repeated edits remain cheap.
  • The 2D feature stage is not optional: ablations show that removing the 2D compositing model sharply degrades shadows and highlights, so the 3D color-only stage alone cannot enforce illumination consistency.
  • The introduced multi-light dataset provides a common physically rendered benchmark with paired views, depth, and ground-truth composites for training and comparing future multi-view compositing methods.
  • Because the pipeline predicts Gaussian colors rather than material intrinsics, it can also insert emissive objects and let their light illuminate the surrounding scene, a capability the paper demonstrates for luminous objects.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The strongest practical constraint is hidden in the input requirements: at inference the method expects a background-only image and depth map for every view, but real-world compositing usually starts from footage where the object is already present, so the deployed setting may often lack the clean reference the method was trained on.
  • The paper's own unposed-input experiments show that when camera poses and depths are estimated rather than captured, harmonization quality drops sharply; this suggests the advertised robustness to casual captures applies mainly when geometry and poses are reliable.
  • The same Hilbert-grid trick could be extended to predict other per-Gaussian attributes, such as normals, albedo, or specular coefficients, potentially moving the framework toward material-aware relighting without changing its architecture.
  • A sharper test of the lighting claim would be to measure predicted shadow directions and penumbrae against ray-traced ground truth under known light positions, rather than relying only on pixel-similarity metrics that can favor color-matched but physically wrong shadows.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. MV-CoLight proposes a two-stage feed-forward pipeline for object compositing with lighting and shadow consistency. In the 2D stage, a Swin-transformer network maps the composite image, a background-only image, and a depth map to a harmonized image. In the 3D stage, per-view 2D features and original Gaussian colors are reordered along a Hilbert curve and processed by a second transformer to predict updated Gaussian colors while geometry is frozen; the updated Gaussians are rendered for multi-view consistency. The paper also introduces DTC-MultiLight, a 480k-scene Blender dataset, and evaluates on FOSCom, OWL, the new dataset, and two real captured scenes. Reported results show large PSNR/SSIM/LPIPS margins over diffusion-based and inverse-rendering baselines, with 0.07 seconds per-frame 2D inference.

Significance. If the reported results are representative, the paper makes a useful practical contribution: a fast feed-forward alternative to per-scene inverse rendering that handles multi-view consistency and shadow generation, backed by a large new dataset and extensive comparisons. The paper is honest enough to include VGGT-based unposed experiments (Sec. A.7) and ablation variants without depth and background inputs (Ours-dagger), both of which reveal the sensitivity of the method to its auxiliary inputs. The main significance is therefore conditional: the method is strong in the setup where clean background images and accurate poses and depth are available, but the abstract's 'casually captured real-world' and 'wide generalization' claims exceed the evidence. The central color-only Gaussian-update idea is simple and appears sound. I do not see evidence of circularity: the losses are supervised against ground-truth renderings and external benchmarks are included.

major comments (3)
  1. [Sec. 3.2, Eq. (1); Sec. 3.1; Sec. A.7; Sec. A.8] The headline generalization claim is conditioned on inputs that are not available in the advertised casual-capture regime. Eq. (1) requires a background-only image G and a depth map D for every view, and Sec. 3.1 requires camera poses and depth to build the 3D Gaussian model. In a real scene where the object is already present, no clean background-only image exists, and when poses and depths are estimated automatically the authors' own VGGT experiments (Sec. A.7) show blurred harmonization and the need for Gaussian position optimization to recover sharpness. The limitation list in Sec. A.8 does not mention the background-only dependency. The claims of robustness and wide generalization in the abstract and Sec. 4.2 should be re-scoped to settings with available background references and poses, or supplemented with quantitative experiments where G and D are estimated, for example via inpainting-based background removal and a pose and depth estimator on real scenes.
  2. [Sec. 3.3, Fig. 3] The Hilbert-curve mapping is under-specified and is a stated contribution. The text says that a 3D Hilbert curve is constructed through the Gaussian centers and each primitive is assigned to its nearest curve point, but it does not specify the grid resolution or quantization used to evaluate the curve, how the fixed number M of sampled points is ordered when coordinates are unevenly distributed, or how the 1D sequence is folded into a 2D Hilbert-ordered grid with a well-defined inverse mapping Psi. Without this procedure, the 3D model's input representation and the bijectivity of the projection cannot be reproduced. Please provide the exact mapping algorithm, including normalization, padding, and inverse mapping, in the supplement.
  3. [Tables 1-2; Sec. A.4] All quantitative results are point estimates. The OWL test set has 72 scenes, the own 2D test set has 57 scenes, the 3D test sets have 50 and 7 scenes, and the real-capture evaluation uses two scenes; Tables 1 and 2 report single PSNR/SSIM/LPIPS numbers without error bars, per-scene standard deviations, or significance tests. The user study in Fig. 10 reports percentages without stating the number of participants, questions, or statistical comparisons. Since the central claim is state-of-the-art performance, please add confidence intervals or per-scene distributions and report the user-study protocol.
minor comments (8)
  1. [Abstract; Sec. 4.3] The abstract's '0.07 seconds per frame' refers only to the 2D stage; the 3D stage requires about 1.08 seconds of inference after about 1.08 minutes of scene-specific Gaussian representation learning. Please state this 2D/3D distinction explicitly in the abstract and conclusion.
  2. [Sec. 3.4, Eq. (4)] The notation in Eq. (4) mixes the predicted harmonized Gaussian color map I_hat_C with the rendered images H_hat_i; please define the rendering function that links them and clarify the dimensions and indexing.
  3. [Sec. A.2] The paragraph describing Poly Haven material diversification is duplicated verbatim; please remove the repeated text.
  4. [Sec. A.4] The sentence 'Our method surpasses all baseline methods under objective criteria' should say 'subjective user study', and the sample size should be reported.
  5. [Fig. 16] The LPIPS values in Fig. 16 are difficult to parse; please use a table or clearly annotated labels for each variant.
  6. [Tables 1-2; Fig. 11] Minor typos: 'purposed dataset' should be 'proposed dataset', and 'Foscom' in Fig. 11 should be 'FOSCom'.
  7. [Sec. 4.1] The phrase 'For trainig time' is a typo; also state how many of the 480k generated scenes are used for training and confirm that the 57-scene test set is disjoint from the training split.
  8. [Sec. 4.1, OWL evaluation] The OWL evaluation should clarify how composite inputs are generated, since OWL is a relighting dataset and the compositing protocol affects comparability with methods trained for standard image compositing.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: outputs are trained against ground-truth renderings and evaluated on external benchmarks; the background/depth dependence is a scoping limitation, not a circular reduction.

full rationale

MV-CoLight's derivation is self-contained against external evidence. The 2D network (Eq. 1) maps {I, G, D} to a harmonized image, and the 3D network maps 2D features plus inharmonious Gaussian colors to harmonized colors; both are optimized with MSE and perceptual losses against ground-truth renderings (Eqs. 3-4), so no reported quantity is fitted and then renamed as a prediction. The method is benchmarked on external FOSCom and OWL datasets (Tables 1-2), and the ablations in Table 3 quantify the contribution of the background and depth inputs rather than defining the output in terms of them. The Hilbert-curve ordering and Gaussian color mapping are architectural devices, not assertions that a target result follows from a premise that already contains it. The paper's dependence on background-only images and depth at inference, and the degraded VGGT-based results reported in A.7, are generalization and scoping limitations for real-world use; they do not make the central claim equivalent to its inputs by construction. No load-bearing self-citation or imported uniqueness theorem is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rely on learned mapping assumptions and training hyperparameters rather than fitted physical constants. The main assumptions are the availability of background-depth inputs, the sufficiency of color-only Gaussian editing, and synthetic-to-real transfer. No new physics-like entities are introduced.

free parameters (3)
  • perceptual loss weight lambda = 0.05
    Hand-set in Eq 3 and Eq 4; the reported results depend on this choice, and no sensitivity analysis is provided.
  • 3D loss mixing weight beta = 0.5
    Hand-set in Eq 4 to balance the 3D Gaussian color loss against per-view rendering loss; results in Table 2 depend on this choice.
  • Gaussian point sampling count M = not specified
    Sec 3.1 says a fixed number M of points is randomly sampled to initialize the 3D Gaussian model, but M is never given, and it directly affects representation quality and runtime.
assumptions (4)
  • standard math Hilbert curve ordering preserves spatial locality so that image transformers can process Gaussian colors coherently.
    Invoked in Sec 3.1 and A.1 as the basis for folding 3D Gaussian centers into a 2D grid; standard property of Hilbert curves, but the practical benefit is asserted rather than derived.
  • domain assumption Accurate depth maps, camera poses, and background-only images are available at inference for every view.
    Eq 1 uses background image G and depth D as inputs, and Sec 3.1 requires poses and depth to build point maps; this is a strong input dependency for real-world insertion tasks.
  • ad hoc to paper A color-only update of frozen Gaussian geometry is sufficient to produce harmonized, view-consistent compositing results.
    Sec 3.3 explicitly freezes positions, scales, rotations, and opacities while updating only colors; this efficiently sidesteps geometry refinement but is not derived from physical constraints, and A.8 notes that Gaussian parameterization errors can propagate.
  • domain assumption A model trained on synthetic Blender tabletop scenes with Poly Haven environment maps generalizes to casually captured real scenes.
    Sec 4.2 claims robustness on real captures, but training data (A.2) is synthetic tabletop data and only two real scenes are evaluated, so the transfer assumption is load-bearing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generation." pith.science (2026). https://pith.science/paper/ZBN2DT5S

@misc{pith2026250521483,
  author       = {Pith},
  title        = {Pith review of: MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZBN2DT5S}},
  note         = {Machine review of arXiv:2505.21483}
}
read the original abstract

Object compositing offers significant promise for augmented reality (AR) and embodied intelligence applications. Existing approaches predominantly focus on single-image scenarios or intrinsic decomposition techniques, facing challenges with multi-view consistency, complex scenes, and diverse lighting conditions. Recent inverse rendering advancements, such as 3D Gaussian and diffusion-based methods, have enhanced consistency but are limited by scalability, heavy data requirements, or prolonged reconstruction time per scene. To broaden its applicability, we introduce MV-CoLight, a two-stage framework for illumination-consistent object compositing in both 2D images and 3D scenes. Our novel feed-forward architecture models lighting and shadows directly, avoiding the iterative biases of diffusion-based methods. We employ a Hilbert curve-based mapping to align 2D image inputs with 3D Gaussian scene representations seamlessly. To facilitate training and evaluation, we further introduce a large-scale 3D compositing dataset. Experiments demonstrate state-of-the-art harmonized results across standard benchmarks and our dataset, as well as casually captured real-world scenes demonstrate the framework's robustness and wide generalization.

Figures

Figures reproduced from arXiv: 2505.21483 by the authors.

Figure 1
Figure 1. Illustration of our object compositing pipeline with harmonization and relighting using [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of MV-CoLight. In (a), we insert a white puppy as the composite object onto the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Mapping multi-view observations into a 2D Hilbert-ordered Gaussian color map. Starting [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Single-view qualitative comparison with SOTA methods [ [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Multi-view qualitative comparison with SOTA methods [ [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: We evaluate our method on real-world scenes captured under varying illumination with six [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Visual results of inserting luminous objects. Our method successfully simulates the [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Visualization of the DTC-MultiLight dataset. We showcase rendered results of diverse [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Detailed model architecture of our compositing models. We introduce two compositing [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: User Study Results. We respectively compare our method with baseline methods for both [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Single-view qualitative comparison on the Foscom dataset [ [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Single-view qualitative comparison on the Object with Lighting dataset [ [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Multi-view qualitative comparison on our rendered scenes. In the first case, we remove [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Multi-view visualization after light source insertion. Our method meticulously simulates [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Qualitative comparison under sequential training iterations. We demonstrate the perfor [PITH_FULL_IMAGE:figures/full_fig_p021_15.png]
Figure 16
Figure 16. Figure 16: Object insertion results based on VGGT [ [PITH_FULL_IMAGE:figures/full_fig_p022_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 36 canonical work pages

  1. [1]

    Recovering intrinsic scene characteris- tics

    Harry Barrow, J Tenenbaum, A Hanson, and E Riseman. Recovering intrinsic scene characteris- tics. Comput. vis. syst, 2(3-26):2, 1978

    work page 1978

  2. [2]

    Gi-gs: Global illumination decomposition on gaussian splatting for inverse rendering

    Hongze Chen, Zehong Lin, and Jun Zhang. Gi-gs: Global illumination decomposition on gaussian splatting for inverse rendering. arXiv preprint arXiv:2410.02619, 2024

    arXiv 2024

  3. [3]

    Mureobjectstitch: Multi-reference image composition

    Jiaxuan Chen, Bo Zhang, and Li Niu. Mureobjectstitch: Multi-reference image composition. arXiv preprint arXiv:2411.07462, 2024

    arXiv 2024

  4. [4]

    An empirical study of gpt-4o image generation capabilities

    Sixiang Chen, Jinbin Bai, Zhuoran Zhao, Tian Ye, Qingyu Shi, Donghao Zhou, Wenhao Chai, Xin Lin, Jianzong Wu, Chao Tang, et al. An empirical study of gpt-4o image generation capabilities. arXiv preprint arXiv:2504.05979, 2025

    arXiv 2025

  5. [5]

    High-resolution image harmonization via collaborative dual transformations

    Wenyan Cong, Xinhao Tao, Li Niu, Jing Liang, Xuesong Gao, Qihao Sun, and Liqing Zhang. High-resolution image harmonization via collaborative dual transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18470–18479, 2022

    work page 2022

  6. [6]

    Dovenet: Deep image harmonization via domain verification

    Wenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, and Liqing Zhang. Dovenet: Deep image harmonization via domain verification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8394–8403, 2020

    work page 2020

  7. [7]

    Objaverse-xl: A universe of 10m+ 3d objects

    Matt Deitke, Ruoshi Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram V oleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Information Processing Systems , 36:35799–35813, 2023

    2023

  8. [8]

    Digital twin catalog: A large-scale photorealistic 3d object digital twin dataset

    Zhao Dong, Ka Chen, Zhaoyang Lv, Hong-Xing Yu, Yunzhi Zhang, Cheng Zhang, Yufeng Zhu, Stephen Tian, Zhengqin Li, Geordie Moffatt, et al. Digital twin catalog: A large-scale photorealistic 3d object digital twin dataset. arXiv preprint arXiv:2504.08541, 2025

    arXiv 2025

Show all 53 references
  1. [9]

    Relightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing

    Jian Gao, Chun Gu, Youtian Lin, Zhihao Li, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing. In European Conference on Computer Vision, pages 73–89. Springer, 2024

    2024

  2. [10]

    Outcast: Outdoor single-image relighting with cast shadows

    David Griffiths, Tobias Ritschel, and Julien Philip. Outcast: Outdoor single-image relighting with cast shadows. In Computer Graphics F orum, volume 41, pages 179–193. Wiley Online Library, 2022

    2022

  3. [11]

    Irgs: Inter-reflective gaussian splatting with 2d gaussian ray tracing

    Chun Gu, Xiaofei Wei, Zixuan Zeng, Yuxuan Yao, and Li Zhang. Irgs: Inter-reflective gaussian splatting with 2d gaussian ray tracing. arXiv preprint arXiv:2412.15867, 2024

    2024 arXiv

  4. [12]

    Pct-net: Full resolution image harmonization using pixel-wise color transformations

    Julian Jorge Andrade Guerreiro, Mitsuru Nakazawa, and Björn Stenger. Pct-net: Full resolution image harmonization using pixel-wise color transformations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5917–5926, 2023. 10

    2023

  5. [13]

    Im- age harmonization with transformer

    Zonghui Guo, Dongsheng Guo, Haiyong Zheng, Zhaorui Gu, Bing Zheng, and Junyu Dong. Im- age harmonization with transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 14870–14879, 2021

    2021

  6. [14]

    Intrinsic image harmonization

    Zonghui Guo, Haiyong Zheng, Yufeng Jiang, Zhaorui Gu, and Bing Zheng. Intrinsic image harmonization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16367–16376, June 2021

    2021

  7. [15]

    Dritter Band: Analysis · Grundlagen der Mathematik · Physik V erschiedenes: Nebst Einer Lebensgeschichte

    David Hilbert. Dritter Band: Analysis · Grundlagen der Mathematik · Physik V erschiedenes: Nebst Einer Lebensgeschichte. Springer-Verlag, 2013

    2013

  8. [16]

    Geometry-aware single-image full-body human relighting

    Chaonan Ji, Tao Yu, Kaiwen Guo, Jingxin Liu, and Yebin Liu. Geometry-aware single-image full-body human relighting. In European Conference on Computer Vision , pages 388–405. Springer, 2022

    2022

  9. [17]

    Neural gaffer: Relighting any object via diffusion

    Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely. Neural gaffer: Relighting any object via diffusion. Advances in Neural Information Processing Systems, 37:141129–141152, 2025

    2025

  10. [18]

    Tensoir: Tensorial inverse rendering

    Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Songfang Han, Sai Bi, Xiaowei Zhou, Zexiang Xu, and Hao Su. Tensoir: Tensorial inverse rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 165–174, 2023

    2023

  11. [19]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023

    2023

  12. [20]

    Using color compatibility for assessing image realism

    Jean-Francois Lalonde and Alexei A Efros. Using color compatibility for assessing image realism. In 2007 IEEE 11th International Conference on Computer Vision , pages 1–8. IEEE, 2007

    2007

  13. [21]

    Diffusionrenderer: Neural inverse and forward rendering with video diffusion models

    Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Zhi-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, et al. Diffusionrenderer: Neural inverse and forward rendering with video diffusion models. arXiv preprint arXiv:2501.18590, 2025

    2025 arXiv

  14. [22]

    Photorealistic object insertion with diffusion-guided inverse rendering

    Ruofan Liang, Zan Gojcic, Merlin Nimier-David, David Acuna, Nandita Vijaykumar, Sanja Fidler, and Zian Wang. Photorealistic object insertion with diffusion-guided inverse rendering. In European Conference on Computer Vision, pages 446–465. Springer, 2024

    2024

  15. [23]

    Gs-ir: 3d gaussian splatting for inverse rendering

    Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. Gs-ir: 3d gaussian splatting for inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21644–21653, 2024

    2024

  16. [24]

    Arshadowgan: Shadow generative adversarial network for augmented reality in single light scenes

    Daquan Liu, Chengjiang Long, Hongpan Zhang, Hanning Yu, Xinzhi Dong, and Chunxia Xiao. Arshadowgan: Shadow generative adversarial network for augmented reality in single light scenes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8...

    2020

  17. [25]

    Shadow generation for composite image using diffusion model

    Qingyang Liu, Junqi You, Jianting Wang, Xinhao Tao, Bo Zhang, and Li Niu. Shadow generation for composite image using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8121–8130, 2024

    2024

  18. [26]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10012–10022, 2021

    2021

  19. [27]

    High-resolution image harmonization with adaptive-interval color transformation

    Quanling Meng, Liu Qinglin, Zonglin Li, Xiangyuan Lan, Shengping Zhang, and Liqiang Nie. High-resolution image harmonization with adaptive-interval color transformation. Advances in Neural Information Processing Systems, 37:13769–13793, 2025

    2025

  20. [28]

    Nerf: Representing scenes as neural radiance fields for view synthesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoor- thi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021. 11

    2021

  21. [29]

    Poly haven

    Poly Haven. Poly haven. https://polyhaven.com/

  22. [30]

    Lite2relight: 3d-aware single image portrait relighting

    Pramod Rao, Gereon Fox, Abhimitra Meka, Mallikarjun BR, Fangneng Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, et al. Lite2relight: 3d-aware single image portrait relighting. In ACM SIGGRAPH 2024 Conference Papers, pages 1–12, 2024

    2024

  23. [31]

    Color transfer between images

    Erik Reinhard, Michael Adhikhmin, Bruce Gooch, and Peter Shirley. Color transfer between images. IEEE Computer graphics and applications , 21(5):34–41, 2001

    2001

  24. [32]

    Structure-from-motion revisited

    Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-motion revisited. In Conference on Computer Vision and Pattern Recognition (CVPR) , 2016

    2016

  25. [33]

    Controllable shadow generation using pixel height maps

    Yichen Sheng, Yifan Liu, Jianming Zhang, Wei Yin, A Cengiz Oztireli, He Zhang, Zhe Lin, Eli Shechtman, and Bedrich Benes. Controllable shadow generation using pixel height maps. In European Conference on Computer Vision, pages 240–256. Springer, 2022

    2022

  26. [34]

    Ssn: Soft shadow network for image compositing

    Yichen Sheng, Jianming Zhang, and Bedrich Benes. Ssn: Soft shadow network for image compositing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4380–4390, 2021

    2021

  27. [35]

    Foreground-aware semantic rep- resentations for image harmonization

    Konstantin Sofiiuk, Polina Popenova, and Anton Konushin. Foreground-aware semantic rep- resentations for image harmonization. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 1620–1629, 2021

    2021

  28. [36]

    Objectstitch: Object compositing with diffusion model

    Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, and Daniel Aliaga. Objectstitch: Object compositing with diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18310–18319, 2023

    2023

  29. [37]

    Multi-scale image harmonization

    Kalyan Sunkavalli, Micah K Johnson, Wojciech Matusik, and Hanspeter Pfister. Multi-scale image harmonization. ACM Transactions on Graphics (TOG), 29(4):1–10, 2010

    2010

  30. [38]

    Thinking outside the bbox: Unconstrained generative object compositing

    Gemma Canet Tarrés, Zhe Lin, Zhifei Zhang, Jianming Zhang, Yizhi Song, Dan Ruta, Andrew Gilbert, John Collomosse, and Soo Ye Kim. Thinking outside the bbox: Unconstrained generative object compositing. arXiv preprint arXiv:2409.04559, 2024

    2024 arXiv

  31. [39]

    Objects with lighting: A real-world dataset for evaluating reconstruction and rendering for object relighting

    Benjamin Ummenhofer, Sanskar Agrawal, Rene Sepulveda, Yixing Lao, Kai Zhang, Tianhang Cheng, Stephan Richter, Shenlong Wang, and German Ros. Objects with lighting: A real-world dataset for evaluating reconstruction and rendering for object relighting. In 2024 International Con...

    2024

  32. [40]

    Shadow harmonization for realistic compositing

    Lucas Valença, Jinsong Zhang, Michaël Gharbi, Yannick Hold-Geoffroy, and Jean-François Lalonde. Shadow harmonization for realistic compositing. In SIGGRAPH Asia 2023 Conference Papers, pages 1–12, 2023

    2023

  33. [41]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025

    2025

  34. [42]

    Semi-supervised parametric real-world image harmonization

    Ke Wang, Michaël Gharbi, He Zhang, Zhihao Xia, and Eli Shechtman. Semi-supervised parametric real-world image harmonization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5927–5936, 2023

    2023

  35. [43]

    Metashadow: Object-centered shadow detection, removal, and synthesis, 2024

    Tianyu Wang, Jianming Zhang, Haitian Zheng, Zhihong Ding, Scott Cohen, Zhe Lin, Wei Xiong, Chi-Wing Fu, Luis Figueroa, and Soo Ye Kim. Metashadow: Object-centered shadow detection, removal, and synthesis, 2024

    2024

  36. [44]

    Image quality assessment: from error visibility to structural similarity

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing , 13(4):600– 612, 2004

    2004

  37. [45]

    Lu- minet: Latent intrinsics meets diffusion models for indoor scene relighting

    Xiaoyan Xing, Konrad Groh, Sezer Karaoglu, Theo Gevers, and Anand Bhattad. Lu- minet: Latent intrinsics meets diffusion models for indoor scene relighting. arXiv preprint arXiv:2412.00177, 2024. 12

    2024 arXiv

  38. [46]

    Dccf: Deep comprehensible color filter learning framework for high-resolution image harmonization

    Ben Xue, Shenghui Ran, Quan Chen, Rongfei Jia, Binqiang Zhao, and Xing Tang. Dccf: Deep comprehensible color filter learning framework for high-resolution image harmonization. In European conference on computer vision, pages 300–316. Springer, 2022

    2022

  39. [47]

    Relighting humans: occlusion-aware inverse rendering for full-body human images

    Kanamori Yoshihiro. Relighting humans: occlusion-aware inverse rendering for full-body human images. ACM Trans. Graph., 37:270–1, 2018

    2018

  40. [48]

    Self-supervised outdoor scene relighting

    Ye Yu, Abhimitra Meka, Mohamed Elgharib, Hans-Peter Seidel, Christian Theobalt, and William AP Smith. Self-supervised outdoor scene relighting. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16 , pages 84–101. Sp...

    2020

  41. [49]

    Rgb-x: Image decomposition and synthesis using material- and lighting-aware diffusion models

    Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yannick Hold-Geoffroy, Yiwei Hu, Fujun Luan, Ling-Qi Yan, and Miloš Hašan. Rgb-x: Image decomposition and synthesis using material- and lighting-aware diffusion models. In Special Interest Group on Computer Graphics and Intera...

    2024

  42. [50]

    Control- com: Controllable image composition using diffusion model

    Bo Zhang, Yuxuan Duan, Jun Lan, Yan Hong, Huijia Zhu, Weiqiang Wang, and Li Niu. Control- com: Controllable image composition using diffusion model. arXiv preprint arXiv:2308.10040, 2023

    2023 arXiv

  43. [51]

    Scaling in-the-wild training for diffusion- based illumination harmonization and editing by imposing consistent light transport

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Scaling in-the-wild training for diffusion- based illumination harmonization and editing by imposing consistent light transport. In The Thirteenth International Conference on Learning Representations

  44. [52]

    The unrea- sonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 586–595, 2018

    2018

  45. [53]

    Shadowgan: Shadow synthesis for virtual objects with conditional adversarial networks

    Shuyang Zhang, Runze Liang, and Miao Wang. Shadowgan: Shadow synthesis for virtual objects with conditional adversarial networks. Computational Visual Media, 5:105–115, 2019. 13 A Supplementary Material In the supplementary material, we first present a brief overview of the co...

    2019

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.