Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A four-channel VAE fitted to a pretrained RGB VAE beats the 1M-image LayerDiffuse on RGBA reconstruction and generation using about 8K images; a new benchmark scores transparency by compositing onto nine canonical backgrounds.

desk verdict A useful RGBA VAE recipe and benchmark, but the headline 8K-vs-1M claim is not supported because the best numbers use FLUX while LayerDiffuse uses SDXL. read the letter →

arxiv 2507.09308 v1 pith:FL2AKGAK submitted 2025-07-12 cs.CV cs.AI

Zile Wang , Hao Yu , Jiabo Zhan , Chun Yuan This is my paper
classification cs.CVcs.AI
keywords RGBAimagereconstructiontransparentgenerationlatentdiffusionmodelsvariationalautoencoderalphablendingchannelmattingdatasetsqualitybenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most image generators produce flat RGB pictures; genuinely transparent, layered content — the cut-out objects that designers composite onto backgrounds — has been left out because there is no large-scale transparency dataset and no standard way to measure four-channel quality. The paper tries to close both gaps at once. It introduces the ALPHA benchmark, which alpha-blends every RGBA image onto nine solid-color backgrounds and runs ordinary RGB metrics (PSNR, SSIM, FID, LPIPS) on the composites, and it builds AlphaVAE, a four-channel autoencoder obtained by widening the encoder's first convolution and the decoder's last convolution of a pretrained RGB VAE, zero-initializing the new alpha slices, and supervising reconstruction with a blend-aware loss that reduces to a closed form in background moments. The headline claim is that this model, trained on about 8,000 matting-derived images, reconstructs color plus transparency more accurately than LayerDiffuse, a purpose-built RGBA pipeline trained on a million images, and that a latent diffusion model fine-tuned on top produces better transparent images. A careful reader should care because, if the claim holds, transparency becomes an affordable retrofit to ordinary latent diffusion instead of a data-hungry special case.

What carries the argument

The load-bearing identity is the ABMSE reconstruction loss. With $\alpha$ blending written as $A(x,b) = x_{\mathrm{rgb}} \odot x_\alpha + b \odot (1 - x_\alpha)$, the loss $L_{\mathrm{rec}}(E,D) = \mathbb{E}_b \lVert A(\hat{x},b) - A(x,b)\rVert_2^2$ expands — treating the background channels as i.i.d. — to $\lVert P\rVert_2^2 - 2\,\Delta_\alpha \langle \mathbb{E}[b], P\rangle - \Delta_\alpha^2\lVert \mathbb{E}[b^2]\rVert_1$, where $P = \hat{x}_{\mathrm{rgb}}\hat{x}_\alpha - x_{\mathrm{rgb}}x_\alpha$ is the premultiplied-color difference and $\Delta_\alpha = \hat{x}_\alpha - x_\alpha$ is the $\alpha$ difference. Because the expectation reduces to the first and second moments of the background distribution (estimated once from ImageNet), the blending-aware supervision adds no sampling cost during training. Around this loss sit the channel-extension initialization — zeroed $\alpha$ slices in the outer convolutions, decoder $\alpha$ bias set to 1 — and the dual-KL regularizer to the frozen reference encoder, which the ablation study identifies as the component that preserves downstream generation quality.

What would settle it

Render a set of transparent objects in 3D software so the true straight-alpha foreground color is known exactly, composite them onto the nine canonical backgrounds, and run the released AlphaVAE and LayerDiffuse checkpoints on the result; if the PSNR gap observed on the matting-derived ALPHA test set shrinks or reverses on these construction-known images, the advantage is an artifact of the training targets rather than of the architecture or loss.

Watch

Extended reading notes

Core claim

AlphaVAE asserts that RGBA reconstruction and generation do not require a dedicated transparency architecture or a web-scale RGBA corpus: a pretrained three-channel VAE can be converted in place. The conversion widens the first encoder convolution and the last decoder convolution to four channels, copies the pretrained weights into the RGB slices, zero-initializes the $\alpha$ slices, and sets the decoder $\alpha$ bias to 1 so the model starts fully opaque and learns transparency gradually. The training objective supervises appearance where the user sees it: the main reconstruction term, ABMSE, is the expected squared error between prediction and ground truth after both are $\alpha$-blended onto backgrounds, an expectation that collapses to a polynomial in the premultiplied-color difference $P$ and the $\alpha$ difference $\Delta_\alpha$ with coefficients fixed by precomputed background moments. The perceptual term also runs on blended inputs, restricted to white and black backgrounds because a perceptual network cannot be reduced to moments the way the pixel loss can, and two KL terms — one toward the standard normal, one toward the frozen reference encoder's latent on blended inputs with a fourth channel of ones — keep the fine-tuned latent compatible with the pretrained diffusion model. On the ALPHA test split the FLUX-based model reports PSNR 38.20 dB versus 32.45 dB for LayerDiffuse, and the paper's summary claim is that about 8,000 training images suffice where the baseline used about 1M.

Load-bearing premise

The load-bearing premise, stated in Section 3.2, is that concatenating an extracted matting foreground with its alpha matte yields true RGBA ground truth: the stored RGB is assumed to be the clean, unmultiplied object color, and any background contamination or fringing baked into the extracted foregrounds would be baked into both the training targets and the benchmark scores.

Editorial extensions

If this is right

  • A standard RGB diffusion stack (SDXL or FLUX) can be converted into a transparent-image generator by fine-tuning the VAE with the alpha-channel extension plus a low-rank LoRA on the diffusion module, with no new transparency corpus beyond existing matting datasets.
  • RGBA results become comparable across papers: the nine-background compositing protocol turns any RGB metric into a transparency-aware one, so future reconstruction and generation numbers can be checked against LayerDiffuse and AlphaVAE on the same scale.
  • The reference-KL constraint is the part of the recipe that protects generation, not reconstruction: dropping it leaves reconstruction essentially unchanged but raises the generation FID by about 5.8 points on AIM-500, so latent compatibility with the frozen diffusion prior is a deliberate, load-bearing design choice.
  • The ablations show the perceptual LPIPS term and the standard-normal KL term are the largest contributors to reconstruction quality, meaning the gains come from the joint objective rather than from any single loss.
  • Because transparency lives in the VAE latent, downstream uses — the paper names video and interactive layered editing — inherit RGBA capability without redesigning the diffusion backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 8K-versus-1M comparison evaluates a released checkpoint of LayerDiffuse, trained on a different corpus, under metrics defined in this paper; retraining LayerDiffuse on the same 8K ALPHA split under the same protocol would be a sharper test of whether the data-efficiency claim belongs to the architecture or to the new training targets and scoring rule.
  • All reported scores are computed after compositing, so alpha fidelity is never isolated; a matte-only metric (sum of absolute alpha differences, or error restricted to fractional-coverage pixels) would reveal whether the transparency channel itself drives the gap.
  • The paper's training-set construction implies that the matting literature as a whole is a renewable source of RGBA training data, so the 8K number is a property of curation, not a capacity limit; if the method scales with future matting datasets, the data-efficiency headline should be read as "the right 8K images", not "any 8K images".
  • The benchmark scores only solid-color composites, while the qualitative figures show checkerboard rendering; a textured or natural-background variant of the protocol would test whether the ranking of methods survives more realistic compositing.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces ALPHA, a benchmark for evaluating RGBA image reconstruction and generation by compositing images over fixed backgrounds and applying standard RGB metrics, and AlphaVAE, a method that extends a pretrained RGB VAE to four channels by adding an alpha channel with zero-initialized weights and a composite loss (alpha-blended reconstruction, perceptual, dual KL, and GAN terms). The authors train on about 8K images derived from matting datasets and report large PSNR/SSIM improvements over LayerDiffuse, plus improved FID for diffusion-based transparent image generation when the VAE is fine-tuned with LoRA. The paper also provides ablations and qualitative examples.

Significance. If the central claims are validated, the work has practical value: the benchmark protocol is a reasonable way to adapt RGB metrics to RGBA data, the zero-initialization alpha-channel extension is simple and easy to adopt, and the reported data efficiency (8K images versus 1M) would be notable. The paper ships code, data, and models, which supports reproducibility. The evaluation against held-out ground-truth RGBA images is not circular, and aligning the training loss with the alpha-blended evaluation metric is sensible. However, as presented, the headline comparison is confounded by the choice of base VAE, and the effective-zero reference-KL weight undermines the stated role of that term; these issues need to be addressed before the main claims can be accepted.

major comments (4)
  1. [Abstract; Table 2, §5.2] The headline numbers (+4.9 dB PSNR, +3.2% SSIM) compare AlphaVAE+FLUX against LayerDiffuse+SDXL, but LayerDiffuse is built on SDXL and the SDXL-based AlphaVAE is worse than LayerDiffuse on LPIPS (AIM-500: 0.0495 vs 0.0418; ALPHA: 0.0402 vs 0.0324). Because the base VAE differs and LayerDiffuse was not retrained on the same data, the claim that AlphaVAE 'outperforms LayerDiffuse across multiple quantitative metrics' is not established. Please provide a controlled comparison with the same base VAE, the same training data, and the same evaluation protocol, and state which of the two AlphaVAE variants is used for each claim.
  2. [Appendix B, §4.2.3, Table 3] The reference-KL term is weighted by 1e-16, which is effectively zero. The paper describes this term as 'specifically introduced' to preserve latent fidelity and attributes generation gains to it, but a weight of 1e-16 cannot influence training. The 'w/o Ref KL' differences in Table 3 are therefore likely within training noise. Please either use a meaningful weight or remove the claim that the reference-KL term contributes to the reported generation improvements.
  3. [Appendix A, Eq. (18); §4.2.1, Eq. (8)] There is a sign error in the derivation of the reconstruction loss. Expanding E[||P - Δα b||^2] gives +Δα^2 E[||b||^2] (equivalently +Δα^2 ||E[b^2]||_1), not the negative term shown in Eq. (18). The main-text Eq. (8) has the correct plus sign, so the appendix contradicts the main text. If the implemented loss follows the appendix sign, the loss would encourage larger alpha errors, and the reported reconstruction results would need to be re-examined.
  4. [Table 3, §5.2] The generation claim rests on FID improvements of 4.61 points on AIM-500 and 6.15 points on the ALPHA test split, yet no error bars, confidence intervals, or multiple-seed results are reported. With test sets of roughly 500 and 402 images, FID estimates are noisy. Please report variance or repeated runs, or temper the generation claim accordingly.
minor comments (5)
  1. [§3.2, last paragraph] The overview paragraph refers to 'AlphaGen' where the paper otherwise describes AlphaVAE; please correct this typo.
  2. [§4.2.3, Eq. (13)] The notation for the reference-KL term is hard to parse; please spell out the concatenation [A(x,b);1] and the exact arguments of the two encoders more explicitly.
  3. [§3.2] Please clarify whether the foreground RGB used as the RGBA target is the unmultiplied foreground color or an extracted foreground that may contain background contamination; the validity of the training targets depends on this distinction.
  4. [Appendix B] The background statistics E[b] and E[b^2] are estimated from the ImageNet training split; please justify why this distribution is used for the ABMSE expectation rather than the distribution of backgrounds used at evaluation.
  5. [Figure 3 caption] The caption does not define 'GT*' or 'Ours*' consistently; please add explicit definitions so that the qualitative comparison is unambiguous.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reconstruction and generation claims are tested against held-out and external data, and no fitted parameter is renamed as a prediction.

full rationale

The paper's central claims do not reduce to their inputs by construction. AlphaVAE's reconstruction quality is measured on held-out test splits (ALPHA test split and the external AIM-500 dataset) against ground-truth RGBA images, and no fitted parameter or training statistic is relabeled as a prediction. The ABMSE training loss (Eq. 6) is aligned with the alpha-blended evaluation metric (Eq. 2), but this is objective-metric alignment, not a by-construction identity: the model must still generalize to unseen images, and the background statistics E[b] and E[b^2] are pre-computed from ImageNet rather than fit to the test set. The data-efficiency claim is an empirical comparison against LayerDiffuse; while the confounded base model in Table 2 (AlphaVAE+FLUX versus LayerDiffuse+SDXL) raises a correctness concern, it is not circularity. The paper contains no load-bearing self-citations, no imported uniqueness theorem, and no ansatz smuggled in via citation. The limitations in Appendix C concern experimental scope, not circular reasoning.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several hand-chosen loss weights and a data assumption that matting datasets yield valid RGBA ground truth. The most notable ledger item is λ_ref=1e-16, which makes the reference-KL term practically absent; this weakens the paper's claim that dual KL constraints preserve latent fidelity. No new entities are introduced.

free parameters (5)
  • Perceptual loss weight (λ_perc) = 0.5
    Hand-chosen in Appendix B; balances the LPIPS term against reconstruction.
  • Normal KL weight (λ_norm) = 1e-6
    Hand-chosen in Appendix B; controls the standard latent prior regularizer.
  • Reference KL weight (λ_ref) = 1e-16
    Hand-chosen in Appendix B; effectively zero, so the reference-KL constraint is almost inert despite being presented as a key mechanism.
  • GAN loss weight (λ_GAN) = 1.0 after 4000 steps
    Hand-chosen in Appendix B; combined with adaptive weight λ_adapt from Eq. (15).
  • Background augmentation probability = 0.3
    Random solid-color background applied with probability 0.3 during VAE training (Section 5.1).
assumptions (5)
  • standard math Background pixels b in the ABMSE loss are treated as i.i.d.
    Used to derive the closed-form expected loss in Eq. (8); the assumption is harmless for expectation over solid colors since the same b applies per image, but it is an assumption.
  • domain assumption Image matting datasets' foreground and alpha pairs constitute ground-truth RGBA images.
    If the foregrounds are contaminated by the background in the source composite, the RGBA targets are biased.
  • domain assumption Alpha blending over nine solid colors is a sufficient evaluation and optimization proxy for RGBA quality.
    The benchmark and the loss both use this blending scheme; real-world transparency may involve textured backgrounds and other compositing operations.
  • ad hoc to paper The reference KL term is weighted 1e-16, effectively zero.
    This makes the 'dual KL' regularizer nearly inert, so the claim that it preserves latent fidelity is not actually tested.
  • domain assumption Zero-initializing the alpha channel preserves the pretrained RGB VAE's utility; the latent space remains compatible with the pretrained diffusion model.
    If the latent distribution shifted, LoRA fine-tuning on 8K images might not recover generation quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning." pith.science (2026). https://pith.science/paper/FL2AKGAK

@misc{pith2026250709308,
  author       = {Pith},
  title        = {Pith review of: AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FL2AKGAK}},
  note         = {Machine review of arXiv:2507.09308}
}
read the original abstract

Recent advances in latent diffusion models have achieved remarkable results in high-fidelity RGB image synthesis by leveraging pretrained VAEs to compress and reconstruct pixel data at low computational cost. However, the generation of transparent or layered content (RGBA image) remains largely unexplored, due to the lack of large-scale benchmarks. In this work, we propose ALPHA, the first comprehensive RGBA benchmark that adapts standard RGB metrics to four-channel images via alpha blending over canonical backgrounds. We further introduce ALPHAVAE, a unified end-to-end RGBA VAE that extends a pretrained RGB VAE by incorporating a dedicated alpha channel. The model is trained with a composite objective that combines alpha-blended pixel reconstruction, patch-level fidelity, perceptual consistency, and dual KL divergence constraints to ensure latent fidelity across both RGB and alpha representations. Our RGBA VAE, trained on only 8K images in contrast to 1M used by prior methods, achieves a +4.9 dB improvement in PSNR and a +3.2% increase in SSIM over LayerDiffuse in reconstruction. It also enables superior transparent image generation when fine-tuned within a latent diffusion framework. Our code, data, and models are released on https://github.com/o0o0o00o0/AlphaVAE for reproducibility.

Figures

Figures reproduced from arXiv: 2507.09308 by the authors.

Figure 1
Figure 1. Qualitative results of image generation. Texts shown in each column correspond to category [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Training pipeline of ALPHAVAE. 4 ALPHAVAE: Teaching VAE to Reconstruct RGBA Images 4.1 Model Architecture and Initialization Variational Autoencoders (VAEs) encode high-dimensional inputs into a structured latent space and then decode this representation to reconstruct the original data. The encoder E : R C×H×W −→ R D×h×w (3) maps an image x ∈ R C×H×W to a distribution in latent space, where the dimensionality reduc… view at source ↗
Figure 3
Figure 3. Qualitative Results of image reconstruction on transparent images. Each row shows [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (22 more)
Figure 4
Figure 4. Figure 4: Qualitative results of image reconstruction on transparent images blended over solid color [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Color distribution of pixels in ImageNet train split. The pixel values are scaled to [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: A zebra stands with its head turned to the left, showcasing its distinctive black and white [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: A woman sits cross-legged, reading a book titled "Your Soul is a River." [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: A white cat with orange and black markings sits calmly, gazing forward. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: A wallaby is captured mid-hop, showcasing its agile movement and distinctive features. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: A person with long hair and hoop earrings looks upward, bathed in sunlight. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: A person wearing a red sweater is looking to the side with their hand on their chin. [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: A necklace with a black chain and a white, teardrop-shaped pendant hangs against a white [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: A modern table lamp with a concrete base and a round, frosted glass bulb. [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: A hand holds a plastic bag containing an orange. [PITH_FULL_IMAGE:figures/full_fig_p024_14.png]
Figure 15
Figure 15. Figure 15: A glowing jellyfish with translucent blue and yellow hues is shown. [PITH_FULL_IMAGE:figures/full_fig_p025_15.png]
Figure 16
Figure 16. Figure 16: A giraffe with its head turned to the side, showcasing its long neck and distinctive coat [PITH_FULL_IMAGE:figures/full_fig_p026_16.png]
Figure 17
Figure 17. Figure 17: A detailed, intricate crocheted pattern with floral motifs and openwork design. [PITH_FULL_IMAGE:figures/full_fig_p027_17.png]
Figure 18
Figure 18. Figure 18: A construction worker wearing a yellow hard hat, gloves, and tool belt holds a hammer [PITH_FULL_IMAGE:figures/full_fig_p028_18.png]
Figure 19
Figure 19. Figure 19: A close-up of fluffy, yellowish-white catkins with red buds. [PITH_FULL_IMAGE:figures/full_fig_p029_19.png]
Figure 20
Figure 20. Figure 20: A close-up of a leopard’s face with its mouth slightly open, showcasing its sharp teeth and [PITH_FULL_IMAGE:figures/full_fig_p030_20.png]
Figure 21
Figure 21. Figure 21: A clear, faceted crystal with smooth edges and a pointed tip is shown. [PITH_FULL_IMAGE:figures/full_fig_p031_21.png]
Figure 22
Figure 22. Figure 22: A child is joyfully playing with a pink bubble wand, wearing a white sweater with heart [PITH_FULL_IMAGE:figures/full_fig_p032_22.png]
Figure 23
Figure 23. Figure 23: A brown horse with white markings on its face and legs is captured mid-stride. [PITH_FULL_IMAGE:figures/full_fig_p033_23.png]
Figure 24
Figure 24. Figure 24: A bottle with a cork and a glass with ice cubes are placed together. [PITH_FULL_IMAGE:figures/full_fig_p034_24.png]
Figure 25
Figure 25. Figure 25: An elderly man in a suit covers his eyes with his hand, smiling. [PITH_FULL_IMAGE:figures/full_fig_p035_25.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. UniWorld-Design: From Pixel Generation to Layer-Native Design

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A two-model framework generates images as transparent layers and decomposes finished designs into ordered, complete semantic layers, outperforming prior decomposition models on per-layer fidelity and editability.

Reference graph

Works this paper leans on

41 extracted references · 33 canonical work pages · cited by 1 Pith paper

  1. [1]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems , NIPS ’20, Red Hook, NY , USA, 2020. Curran Associates Inc

    work page 2020

  2. [2]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Marina Meila and Tong Zhang, editors, ICML, volume 139 of Proceedings of Machine Learning Research, pages 8162–8171. PMLR, 2021

    work page 2021

  3. [3]

    High-Resolution Image Synthesis with Latent Diffusion Models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjorn Ommer. High-Resolution Image Synthesis with Latent Diffusion Models . In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674–10685, Los Alamitos, CA, USA, June 2022. IEEE Computer Society

    work page 2022

  4. [4]

    Auto-encoding variational bayes, 2022

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2022

    2022

  5. [5]

    Burgess, Xavier Glorot, Matthew M

    Irina Higgins, Loïc Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations , 2016

    work page 2016

  6. [6]

    Neural discrete representation learning

    Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In Proceedings of the 31st International Conference on Neural Information Processing Systems , NIPS’17, page 6309–6318, Red Hook, NY , USA, 2017. Curran Associates Inc

    work page 2017

  7. [7]

    Text2layer: Layered image generation using latent diffusion model, 2023

    Xinyang Zhang, Wentian Zhao, Xin Lu, and Jeff Chien. Text2layer: Layered image generation using latent diffusion model, 2023

    work page 2023

  8. [8]

    Transparent image layer diffusion using latent transparency

    Lvmin Zhang and Maneesh Agrawala. Transparent image layer diffusion using latent transparency. ACM Trans. Graph., 43(4), July 2024

    work page 2024

Show all 41 references
  1. [9]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004

    2004

  2. [10]

    Efros, Eli Shechtman, and Oliver Wang

    Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric . In2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 586–595, Los Alamitos, CA, USA, June 2018...

    2018

  3. [11]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Han...

    2025 arXiv

  4. [12]

    Fast soft color segmentation

    Naofumi Akimoto, Huachun Zhu, Yanghua Jin, and Yoshimitsu Aoki. Fast soft color segmentation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2020

    2020

  5. [13]

    Image vectorization and editing via linear gradient layer decomposition

    Zheng-Jun Du, Liang-Fu Kang, Jianchao Tan, Yotam Gingold, and Kun Xu. Image vectorization and editing via linear gradient layer decomposition. ACM Trans. Graph., 42(4), July 2023

    2023

  6. [14]

    Generative image layer decomposition with visual effects

    Jinrui Yang, Qing Liu, Yijun Li, Soo Ye Kim, Daniil Pakhomov, Mengwei Ren, Jianming Zhang, Zhe Lin, Cihang Xie, and Yuyin Zhou. Generative image layer decomposition with visual effects. arXiv preprint arXiv:2411.17864, 2024

    2024 arXiv

  7. [15]

    Decomposing images into layers with advanced color blending

    Yuki Koyama and Masataka Goto. Decomposing images into layers with advanced color blending. Computer Graphics F orum, 37(7):397–407, 2018

    2018

  8. [16]

    Deep image matting

    Ning Xu, Brian Price, Scott Cohen, and Thomas Huang. Deep image matting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , July 2017

    2017

  9. [17]

    Towards natural image matting in the wild via real-scenario prior

    Ruihao Xia, Yu Liang, Peng-Tao Jiang, Hao Zhang, Qianru Sun, Yang Tang, Bo Li, and Pan Zhou. Towards natural image matting in the wild via real-scenario prior. arXiv:2410.06593, 2024

    2024 arXiv

  10. [18]

    Zim: Zero-shot image matting for anything

    Beomyoung Kim, Chanyong Shin, Joonhyun Jeong, Hyungsik Jung, Se-Yun Lee, Sewhan Chun, Dong-Hyun Hwang, and Joonsang Yu. Zim: Zero-shot image matting for anything. arXiv preprint arXiv:2411.00626, 2024

    2024 arXiv

  11. [19]

    Rethinking portrait matting with pirvacy preserving

    Sihan Ma, Jizhizi Li, Jing Zhang, He Zhang, and Dacheng Tao. Rethinking portrait matting with pirvacy preserving. International Journal of Computer Vision, 2023. 11

    2023

  12. [20]

    Referring image matting

    Jizhizi Li, Jing Zhang, and Dacheng Tao. Referring image matting. In Proceedings of the IEEE Computer Vision and Pattern Recognition, 2023

    2023

  13. [21]

    Semantic soft segmen- tation

    Ya˘gız Aksoy, Tae-Hyun Oh, Sylvain Paris, Marc Pollefeys, and Wojciech Matusik. Semantic soft segmen- tation. ACM Transactions on Graphics (Proc. SIGGRAPH), 37(4):72:1–72:13, 2018

    2018

  14. [22]

    Dreamlayer: Simultaneous multi-layer generation via diffusion model, 2025

    Junjia Huang, Pengxiang Yan, Jinhang Cai, Jiyang Liu, Zhao Wang, Yitong Wang, Xinglong Wu, and Guanbin Li. Dreamlayer: Simultaneous multi-layer generation via diffusion model, 2025

    2025

  15. [23]

    Semantic image matting

    Yanan Sun, Chi-Keung Tang, and Yu-Wing Tai. Semantic image matting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021

    2021

  16. [24]

    Bridging composite and real: towards end-to-end deep image matting

    Jizhizi Li, Jing Zhang, Stephen J Maybank, and Dacheng Tao. Bridging composite and real: towards end-to-end deep image matting. International Journal of Computer Vision, 130(2):246–266, 2022

    2022

  17. [25]

    Generating compositional scenes via text-to-image RGBA instance generation

    Alessandro Fontanella, Petru-Daniel Tudosiu, Yongxin Yang, Shifeng Zhang, and Sarah Parisot. Generating compositional scenes via text-to-image RGBA instance generation. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024

    2024

  18. [26]

    Alfie: Democratising RGBA Image Generation With No $$$

    Fabio Quattrini, Vittorio Pippi, Silvia Cascianelli, and Rita Cucchiara. Alfie: Democratising RGBA Image Generation With No $$$. In Proceedings of the European Conference on Computer Vision Workshops . Springer, 2024

    2024

  19. [27]

    Art: Anonymous region transformer for variable multi-layer transparent image generation, 2025

    Yifan Pu, Yiming Zhao, Zhicong Tang, Ruihong Yin, Haoxing Ye, Yuhui Yuan, Dong Chen, Jianmin Bao, Sirui Zhang, Yanbin Wang, Lin Liang, Lijuan Wang, Ji Li, Xiu Li, Zhouhui Lian, Gao Huang, and Baining Guo. Art: Anonymous region transformer for variable multi-layer transparent i...

    2025

  20. [28]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings - 2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023 , Proceedings of the IEEE Interna- tional Conference on Computer Vision, pages 4172–4182. Institute of Electrical...

    2023

  21. [29]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems , NIPS’17,...

    2017

  22. [30]

    Laion-5b: an open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kun- durthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Lai...

    2022

  23. [31]

    Attention-guided hierarchical structure aggregation for image matting

    Yu Qiao, Yuhao Liu, Xin Yang, Dongsheng Zhou, Mingliang Xu, Qiang Zhang, and Xiaopeng Wei. Attention-guided hierarchical structure aggregation for image matting. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  24. [32]

    Ultrahigh resolution image/video matting with spatio- temporal sparsity

    Yanan Sun, Chi-Keung Tang, and Yu-Wing Tai. Ultrahigh resolution image/video matting with spatio- temporal sparsity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 14112–14121, June 2023

    2023

  25. [33]

    Tripartite information mining and integration for image matting

    Yuhao Liu, Jiake Xie, Xiao Shi, Yu Qiao, Yujie Huang, Yong Tang, and Xin Yang. Tripartite information mining and integration for image matting. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7555–7564, October 2021

    2021

  26. [34]

    Privacy-preserving portrait matting

    Jizhizi Li, Sihan Ma, Jing Zhang, and Dacheng Tao. Privacy-preserving portrait matting. In Proceedings of the 29th ACM International Conference on Multimedia , MM ’21, page 3501–3509, New York, NY , USA,

  27. [35]

    Curless, Steven M

    Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sengupta, Brian L. Curless, Steven M. Seitz, and Ira Kemelmacher-Shlizerman. Real-time high-resolution background matting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8762–8771, June 2021

    2021

  28. [36]

    Mask guided matting via progressive refinement network

    Qihang Yu, Jianming Zhang, He Zhang, Yilin Wang, Zhe Lin, Ning Xu, Yutong Bai, and Alan Yuille. Mask guided matting via progressive refinement network. arXiv preprint arXiv:2012.06722, 2020. 12

    2012 arXiv

  29. [37]

    Transmatting: Enhancing transparent objects matting with transformers

    Huanqia Cai, Fanglei Xue, Lele Xu, and Lili Guo. Transmatting: Enhancing transparent objects matting with transformers. In European Conference on Computer Vision (ECCV), 2022

    2022

  30. [38]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 12873–12883, 2021

    2021

  31. [39]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  32. [40]

    Your Soul is a River

    Black Forest Labs. Flux. https://github.com/black-forest-labs/flux , 2024. 13 Appendix A Formula Derivation In this section, we derive Equation (8). Assuming the three channels of the background color b ∈ R3 are independent and identically distributed, we can work on a single ...

    2024

  33. [2021]

    Association for Computing Machinery

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.