REVIEW 3 major objections 26 references
Geometry-aware RGB–depth pretraining plus a 1M-task dataset yields one model that edits and completes 360° panoramas with better spherical fidelity.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-14 15:28 UTC pith:2IG4NREC
load-bearing objection Solid systems paper: velocity circular padding + RGB–depth pretraining + a real 1M multi-task 360 dataset give a unified model that actually respects ERP geometry; the transfer story after dropping depth is the softest link, not a collapse. the 3 major comments →
Enhancing In-context Panoramic Generation via Geometric-aware Pretraining
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Geometry-aware pretraining—parallel RGB–depth generation regularized by positional offsets, similarity loss, and velocity circular padding—supplies spherical spatial priors that, once transferred by unified in-context fine-tuning on Canvas360Dataset, produce a single token-concatenation model whose panoramic fidelity (especially FAED) and geometric consistency exceed prior specialized and general baselines across text-to-panorama, style transfer, inpainting, outpainting, and editing.
What carries the argument
Parallel RGB–depth generation inside a Flow Transformer: post-VAE depth tokens are sequence-concatenated with RGB tokens under a constant 3D-RoPE offset, trained with independent flow-matching losses, velocity circular padding that synchronizes ghost longitude columns, and a squared-correlation similarity penalty that keeps the two velocity fields from collapsing.
Load-bearing premise
That pseudo-depth maps estimated by an off-the-shelf panoramic depth model, plus synthetic edit pairs produced by non-panorama-native generators and then filtered, still carry accurate enough spherical geometry for the learned priors to survive after depth is removed at fine-tuning time.
What would settle it
Retrain the identical architecture without any depth branch or velocity circular padding on the same 100 K panoramas, then re-evaluate FAED, left–right boundary error, and human preference for seam continuity and object distortion on the same held-out set; if the geometry-aware model no longer wins on those three measures, the central claim fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. Canvas360 is a two-stage framework for in-context panoramic generation. Stage 1 pretrains a Flow Transformer (FLUX.1-dev + LoRA) on ~100K RGB–depth ERP panoramas via parallel RGB–depth generation (token-wise concatenation with RoPE offset Td), velocity circular padding for 0°/360° continuity, and a squared-correlation similarity loss L_sim that discourages RGB–depth collapse. Stage 2 discards depth and fine-tunes a single model on a new 1M-scale Canvas360Dataset (style transfer, inpainting, outpainting, editing) using the same token-concatenation recipe with context offset Tc. The paper reports leading FAED and competitive FID-family / perceptual metrics on text-to-panorama (Table 1), strong LPIPS/FAED/PSNR on inpainting–outpainting–editing (Tables 9–10), LRCE-RGB boundary gains (Table 6), ablations (Tables 3/7, Figs. 9–11), and a 71-person preference study favoring boundary continuity, panorama awareness, and overall quality.
Significance. If the geometry-transfer story holds, the work is a useful systems contribution: a large multi-task panoramic in-context dataset, a practical recipe for injecting spherical continuity and depth cues into a modern flow transformer, and a unified token-concatenation model that covers four editing-style tasks without task-specific heads. Strengths that should be credited include the scale and multi-task design of Canvas360Dataset, the explicit LRCE-RGB boundary metric, the user study, and the relatively thorough ablations of depth, positional offset, L_sim, and velocity circular padding. Even if residual gains partly come from data scale rather than pure geometry, the engineering package and public dataset would still be of interest to panoramic generation and immersive-content communities.
major comments (3)
- Secs. 3.2–3.3 and App. B: the central claim is that parallel RGB–depth pretraining installs spherical geometric priors that survive depth removal and drive FAED / seam / distortion-aware editing gains. After fine-tuning, supervision is appearance-only token concatenation. Fig. 7 (cubemap-side depth re-estimation) and Fig. 9 (backbone transfer) are suggestive but indirect. A load-bearing control is missing: same RGB pretraining data and schedule without the depth branch (and without L_sim / depth RoPE offset), then identical Stage-2 fine-tuning. Without that isolation, it remains possible that gains largely reflect better panoramic RGB statistics and the 900K pairs rather than retained spherical geometry. Please add this control (or an equivalent geometry-vs-appearance probe) and temper causal language in the abstract and conclusion accordingly.
- Sec. 3.4 and App. B (pseudo-depth + synthetic pairs): DAP depths are clipped (100 m outdoor / 10 m indoor) and known to be noisy under ERP; style/edit pairs are largely produced by non-panorama-native models (FLUX.2-dev, NanoBanana, etc.) then filtered (50K manual check; Tab. 5 only compares residual noise on 20K edits, not geometric fidelity of teachers). Because Stage 2 discards depth, any teacher-induced ERP-inconsistent structure can be baked into the unified model. Please quantify geometric error of the synthetic edit/style pairs (e.g., line-drawing / seam / polar-distortion checks beyond the brief style filter) and discuss how much of the qualitative “correct panoramic distortion” on edits could be inherited from filtered teachers versus from Stage-1 priors.
- Table 1 vs. Tables 3/7: Canvas360 leads FAED (2.33) and several perceptual metrics but trails DiT360 on FID / FID_pole / FID_equ and shows non-monotonic ablation behavior (depth alone can darken outputs; VCP helps BRISQUE/NIQE more than FID-family). The paper asserts “particularly strong” panorama-specific fidelity. Please reconcile when FAED should be preferred over FID-family for ERP evaluation, report confidence intervals or multi-seed variance for the main Table 1 numbers, and state more carefully which claims are supported by FAED/user study versus which remain mixed.
Circularity Check
No circular derivation; geometry-aware pretraining claims and FAED gains rest on independent ablations, external baselines, and held-out metrics rather than self-definition or fitted-as-prediction.
full rationale
The paper's load-bearing chain is empirical, not definitional. Parallel RGB–depth generation (Eqs. 3–6), velocity circular padding, and L_sim are introduced as training regularizers; their benefit is measured by ablations (Tab. 3/7, Figs. 9–11) that change FAED/BRISQUE/NIQE and qualitative seam/distortion behavior, not by re-labeling a fitted free parameter as a prediction. Downstream transfer after depth removal is tested on held-out Matterport splits and constructed Canvas360Dataset test sets against external methods (DiT360, PAR, FLUX variants, SE360, Omni2, etc.) plus a user study; FAED, LPIPS, PSNR, LRCE-RGB and preference scores are not forced by construction from the DAP clipping thresholds or the synthetic-pair pipeline. Self-citation to DiT360 (overlapping authors) appears only as a related-work baseline and comparison point, not as a uniqueness theorem or load-bearing premise that forbids alternatives. No equation equates a reported metric to its own training objective, no uniqueness result is imported from the authors to force the architecture, and no known empirical pattern is merely renamed. The skeptic concern about pseudo-depth fidelity and non-panorama-native teachers is a correctness/transfer risk, not circularity under the stated criteria. The derivation is therefore self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
free parameters (5)
- similarity-loss weight λ
- positional offsets Td, Tc
- LoRA rank r=64, α=64, dropout 0.1
- depth clip thresholds (100 outdoor / 10 indoor)
- learning rate 1e-5, batch 24, 25 epochs, guidance 3.0 / 28 steps
axioms (4)
- standard math Rectified-flow matching with a DiT backbone learns a usable velocity field for high-resolution image latents.
- domain assumption DAP pseudo-depth (after clipping) is a sufficiently accurate radial geometric prior for equirectangular panoramas.
- ad hoc to paper Token-wise concatenation with a constant RoPE offset adequately disambiguates modalities/contexts without channel-wise fusion.
- domain assumption Synthetic pairs produced by non-panorama-native models (FLUX.2-dev, NanoBanana, etc.) after line-drawing / manual filters remain valid supervision for spherical geometry.
invented entities (3)
-
Canvas360Dataset (1M paired panoramic samples)
no independent evidence
-
velocity circular padding
no independent evidence
-
Canvas360 two-stage geometry-aware + unified in-context framework
no independent evidence
read the original abstract
In this work, we present Canvas360, a two-stage framework for in-context panoramic generation that combines geometry-aware pretraining with downstream task-specific fine-tuning. To address the lack of large-scale, high-quality training data tailored to in-context panoramic tasks, we propose Canvas360Dataset, a collection of 1M high-quality paired panoramic samples for style transfer, inpainting, outpainting, and editing, enabling effective supervision across diverse in-context generation scenarios. On the modeling side, Canvas360 enhances text-to-panorama generation through parallel depth generation, velocity circular padding, and similarity loss regularization, enabling the model to learn geometry-aware representations, capture object distortion details, and improve geometric consistency and global coherence. Furthermore, empowered by strong panoramic priors, Canvas360 enables a unified in-context panoramic generation framework that supports diverse downstream tasks via token-level concatenation, surpassing prior methods in both task coverage and modeling flexibility. Extensive experiments show that Canvas360 improves panoramic image fidelity, achieving particularly strong performance on the panorama-specific FAED metric and competitive or leading results across the reported quantitative evaluations. More information can be found on our project page: https://zry000.github.io/Canvas360/
Figures
Reference graph
Works this paper leans on
-
[1]
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...
Pith/arXiv arXiv 2023
-
[2]
Loosecontrol: Lifting controlnet for generalized depth conditioning
Shariq Farooq Bhat, Niloy Mitra, and Peter Wonka. Loosecontrol: Lifting controlnet for generalized depth conditioning. InACM SIGGRAPH 2024 Conference Papers, pages 1–11,
2024
-
[3]
Black Forest Labs
Accessed: 2024-09-23. Black Forest Labs. Flux.1-fill. https://huggingface.co/black-forest-labs/FLUX. 1-Fill-dev, 2026a. Accessed: 2026-01-21. Black Forest Labs. Flux2. https://github.com/black-forest-labs/flux2, 2026b. Accessed: 2026-01-21. Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions....
2024
-
[4]
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang
Ac- cessed: 2026-01-21. Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. In3DV,
2026
-
[5]
Hao Chen, Yuqi Hou, Chenyuan Qu, Irene Testini, Xiaohan Hong, and Jianbo Jiao
doi: 10.1109/TCSVT.2022.3231041. Hao Chen, Yuqi Hou, Chenyuan Qu, Irene Testini, Xiaohan Hong, and Jianbo Jiao. 360+ x: A panoptic multi-modal scene understanding dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19373–19382,
-
[6]
Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683,
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683,
-
[7]
Haoran Feng, Dizhe Zhang, Xiangtai Li, Bo Du, and Lu Qi. Dit360: High-fidelity panoramic image generation via hybrid training.arXiv preprint arXiv:2510.11712,
-
[8]
Mean flows for one-step generative modeling.arXiv preprint arXiv:2505.13447,
Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling.arXiv preprint arXiv:2505.13447,
-
[9]
Nano banana
Google. Nano banana. https://gemini.google/overview/image-generation/, 2026a. Ac- cessed: 2026-01-21. Google. Veo3.https://deepmind.google/models/veo/, 2026b. Accessed: 2026-01-21. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InarXiv,
2026
-
[10]
Jiehui Huang, Yuechen Zhang, Xu He, Yuan Gao, Zhi Cen, Bin Xia, Yan Zhou, Xin Tao, Pengfei Wan, and Jiaya Jia. Unityvideo: Unified multi-modal multi-task learning for enhancing world-aware video generation.arXiv preprint arXiv:2512.07831, 2025a. Yukun Huang, Yanning Zhou, Jianan Wang, Kaiyi Huang, and Xihui Liu. Dreamcube: 3d panorama generation via multi...
-
[11]
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603,
-
[12]
Kang Liao, Size Wu, Zhonghua Wu, Linyi Jin, Chao Wang, Yikai Wang, Fei Wang, Wei Li, and Chen Change Loy. Thinking with camera: A unified multimodal model for camera-centric understanding and generation.arXiv preprint arXiv:2510.08673,
-
[13]
Depth any panoramas: A foundation model for panoramic depth estimation
Xin Lin, Meixi Song, Dizhe Zhang, Wenxuan Lu, Haodong Li, Bo Du, Ming-Hsuan Yang, Truong Nguyen, and Lu Qi. Depth any panoramas: A foundation model for panoramic depth estimation. arXiv preprint arXiv:2512.16913,
-
[14]
Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,
-
[15]
Step1x-edit: A practical framework for general image editing
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al. Step1x-edit: A practical framework for general image editing. arXiv preprint arXiv:2504.17761,
-
[16]
Minho Park, Taewoong Kang, Jooyeol Yun, Sungwon Hwang, and Jaegul Choo
Accessed: 2026-01-21. Minho Park, Taewoong Kang, Jooyeol Yun, Sungwon Hwang, and Jaegul Choo. Spherediff: Tuning- free omnidirectional panoramic image and video generation via spherical latent representation. In arXiv,
2026
-
[17]
Masked depth modeling for spatial perception.arXiv preprint arXiv:2601.17895,
13 Bin Tan, Changjiang Sun, Xiage Qin, Hanat Adai, Zelin Fu, Tianxiang Zhou, Han Zhang, Yinghao Xu, Xing Zhu, Yujun Shen, and Nan Xue. Masked depth modeling for spatial perception.arXiv preprint arXiv:2601.17895,
-
[18]
Wan: Open and advanced large-scale video generative models
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314,
-
[19]
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun...
-
[20]
Jingyang Zhang, Shiwei Li, Yuanxun Lu, Tian Fang, David McKinnon, Yanghai Tsin, Long Quan, and Yao Yao. Jointnet: Extending text-to-image diffusion for dense distribution modeling.arXiv preprint arXiv:2310.06347, 2023a. Qinsheng Zhang, Jiaming Song, Xun Huang, Yongxin Chen, and Ming-Yu Liu. Diffcollage: Parallel generation of large content with diffusion ...
-
[21]
Inversion-based style transfer with diffusion models
Yuxin Zhang, Nisha Huang, Fan Tang, Haibin Huang, Chongyang Ma, Weiming Dong, and Chang- sheng Xu. Inversion-based style transfer with diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10146–10156, 2023c. Haoyi Zhong, Fang-Lue Zhang, Andrew Chalmers, and Taehyun Rhee. Se360: Semantic edit in 360 p...
-
[22]
In contrast, the model trained with depth supervision produces more coherent depth structures and better cross-view consistency
The model trained without depth supervision exhibits clear depth inconsistencies, especially near stitching seams and boundary regions. In contrast, the model trained with depth supervision produces more coherent depth structures and better cross-view consistency. These results indicate that depth-supervised training effectively injects a geometry prior i...
2025
-
[23]
Training runs for 25 epochs with a per-GPU batch size of 1 and gradient accumulation of 3 (effective batch size = 24). We use a constant learning-rate schedule with a 10% warmup (by steps) and set the training guidance scale to 1.0. The main experiments are conducted at a resolution of 1024×2048, while the ablation studies are performed at 512×1024. For i...
Pith/arXiv arXiv 2048
-
[24]
andCanvas360, respectively, under the same completion setting. FLUX.1-dev tends to introduce blur and artifacts in inpainting, and its outpainting results are less consistent with the conditioning signal, often exhibiting perspective-biased patterns. In contrast,Canvas360produces cleaner and more panorama-consistent completions, indicating that depth-augm...
2025
-
[25]
We report CP (Content Preservation), SR (Style Resemblance), and OV (Overall Vision)
21 Table 8: Quantitative comparison for style transfer. We report CP (Content Preservation), SR (Style Resemblance), and OV (Overall Vision). Higher values indicate better performance. Method CP↑SR↑OV↑ FLUX.1-Kontext-dev 0.457 0.482 0.467 FLUX.2-dev 0.4900.5030.495 Qwen-Image-Edit 0.464 0.483 0.471 Ours0.5020.4910.497 The qualitative results show that our...
arXiv 2018
-
[26]
Across both inpainting and outpainting, our method achieves the best results on all three metrics, yielding lower LPIPS and FAED as well as higher PSNR than all baselines
The results show that our method consistently outperforms all baselines on both inpainting and outpainting tasks. Across both inpainting and outpainting, our method achieves the best results on all three metrics, yielding lower LPIPS and FAED as well as higher PSNR than all baselines. These consistent improvements indicate that our method can better prese...
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.