{"id":"498996da-be41-4362-9f84-a520d446adbe","arxiv_id":"2504.18989","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"REED-VAE fine-tunes only the decoder of a Stable Diffusion VAE with iterative encode-decode training, dynamic k incrementation, and a first-step loss, reducing artifact accumulation over 5 to 25 editing iterations.","lead":"This paper trains a better image autoencoder for diffusion models by repeatedly encoding and decoding images during training, so images survive many edits without accumulating artifacts. The result is a drop-in replacement VAE that lets users chain text, mask, example, and traditional edits in one session.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Metric protocol compares REED outputs to the first edited image x1, not to the source x0; this can reward smoothing or reduced editing strength rather than true image-quality preservation, leaving the central claim not fully established.","rationale":"The reader's weakest assumption was that REED training on unedited encode-decode cycles transfers to diffusion editing, citing Sec. 5.5's evidence that NTI adds independent noise. I agree this transfer premise is under-validated, but the more concrete and immediate threat is the evaluation protocol itself: the supplementary material explicitly computes editing metrics against the first edited output x1 rather than the source, and the paper even justifies tolerating non-sensical edits if they are consistent. This protocol cannot distinguish true quality preservation from a decoder that systematically reduces edit strength or smooths away both artifacts and content. The first-step loss (Sec. 4.3, Eq. 2) also means the model is not trained to improve first-step reconstruction, so there is no guarantee that x1 is an equally faithful edit baseline. A single concrete test, comparing against x0 and measuring first-edit fidelity, would settle whether the central claim is supported. I do not think the paper should be rejected: the encode-decode ablation Table 2 is measured against x0 and shows real improvement, and the qualitative figures are suggestive. But the headline editing claim is conditional on closing this metric gap. Hence CONDITIONAL, matching the reader's verdict, with the specific concern sharpened from transfer to evaluation reference.","tokens_in":18815,"tokens_out":2405,"duration_ms":30609,"concrete_test":"Re-run the Table 1 evaluation protocol with two changes: (1) compute MSE, LPIPS, SSIM, PSNR, and FID at 5/15/25 iterations against the original source image x0 instead of the first edited output x1; (2) measure first-edit fidelity with CLIP directional similarity and LPIPS(x1_REED, x1_vanilla) against the intended edit prompt. If REED-VAE still improves metrics against x0 and maintains comparable first-edit fidelity to the vanilla VAE, the central claim holds. If the improvement vanishes against x0, or if x1_REED is substantially closer to the unedited source or otherwise diverges from x1_vanilla without a corresponding edit-fidelity gain, the reported advantage is an evaluation artifact rather than demonstrated quality preservation.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that REED-VAE preserves image quality across iterative editing. The main evidence for this is Table 1, but the supplementary 'Experiments and comparisons' section states that metrics are computed between the image at iteration 5/15/25 and x1, the first edited output, not the source image x0. This is load-bearing because x1 is itself produced by the swapped decoder being evaluated: if REED-VAE's decoder yields a more conservative or smoothed first edit, or one that is semantically closer to the unedited source, then all later outputs being close to x1 could reflect reduced edit magnitude rather than improved quality retention. The paper explicitly accepts 'non-sensical' edits as long as they are consistent, which means the metric rewards self-consistency against a potentially degraded baseline. The first-step loss in Eq. (2) reinforces this concern: training deliberately optimizes toward x1 rather than x0, so first-step reconstruction fidelity is not protected. The paper's own Section 5.5 also shows that inversion methods introduce independent noise, yet the central experiments largely use non-inversion editing methods and compare against self-produced first edits. Consequently, the demonstrated improvement may be an artifact of the evaluation protocol rather than a general improvement in iterative editing quality, and the claimed transfer from unedited encode-decode training to editing is not separately validated against an absolute reference.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces REED-VAE, a fine-tuning scheme for the decoder of a Stable Diffusion VAE, intended to reduce the accumulation of artifacts and noise when images are repeatedly encoded and decoded during multi-step, multi-method iterative image editing. The training objective combines iterative encode-decode rollouts, a dynamic incrementation curriculum on the number of iterations k, and a first-step loss comparing the final rollout output xk to the first reconstruction x1. The authors evaluate the approach by swapping the vanilla SD2.1 decoder for their fine-tuned decoder across InstructPix2Pix, MagicBrush, DiffEdit, Paint-by-Example, and SD Inpainting, reporting qualitative and quantitative improvements in MSE, LPIPS, SSIM, FID, and PSNR at 5, 15, and 25 iterations. They also present ablations on the iterative training, first-step loss, and dynamic incrementation components, and qualitative comparisons with Null-Text Inversion. The paper frames multi-method iterative image editing as a new problem setting and proposes the REED-trained decoder as a drop-in solution.","tokens_in":19074,"tokens_out":5692,"duration_ms":62953,"significance":"If the central claim holds, the contribution is practically useful: it provides a simple, architecture-compatible way to extend diffusion-based editing pipelines to long or interleaved editing sessions without changing the diffusion model. The problem framing is timely, and the qualitative evidence (Figures 1, 2, and the supplementary sequences) is visually compelling. The paper ships an apparently reproducible recipe (public code and models), and the supplementary comparison with SDXL, SD3, and Flux helps contextualize the relevance of the SD2.1-focused experiments. The main uncertainties are empirical: the editing metric protocol compares against a self-produced first edit rather than the source, and the training implementation only backpropagates through the final iteration, which weakens the mechanistic story told in Algorithm 1.","major_comments":[{"comment":"The central claim is that REED-VAE preserves image quality across iterative editing, but Table 1 computes editing metrics between the image at iterations 5/15/25 and x1, the first edited output, not against the source image x0 as the supplement states. This is load-bearing: x1 is itself produced by the decoder being evaluated, so a smoother, more conservative, or semantically weaker first edit will make later outputs closer to x1 even if they drift arbitrarily far from the original image. The main text (Sec. 5, 'Evaluation metrics') does not disclose this reference choice, and Sec. 4.3 says test metrics are 'still computed against x0,' which appears inconsistent with the supplement. I request that all editing metrics be recomputed against x0 (and, where applicable, against the intended target image for each edit), with the protocol stated explicitly in the main text.","section":"Supplementary Material, 'Experiments and comparisons'"},{"comment":"Equation (2) trains against x1, and the supplement states that gradients are only computed for the final iteration. Under this scheme, gradient information does not flow through the earlier encoder-decoder steps that produce x_{k-1}, so the loss does not actually optimize the full iterative chain described by Eq. (1). As a consequence, the paper's explanation of why REED reduces cumulative artifacts is incomplete, and Algorithm 1's unconditional 'Take gradient descent step on ∇Ltrain(x1,xk,zk)' is misleading. Please clarify the exact optimization objective and provide evidence that final-iteration-only gradients suffice, for example by comparing with full backpropagation for a small k.","section":"Algorithm 1 and Supplementary Material, 'Backpropagation strategy'"},{"comment":"All quantitative results are single-run numbers on a 179-image subset of ImagenHub, reported without error bars, confidence intervals, or significance tests. Since the headline comparisons involve many small metric differences at 5, 15, and 25 iterations, the robustness of the 'consistent improvements' claims cannot be assessed without variance estimates. I request that the authors report means and standard deviations over multiple seeds (or equivalent statistical support) for the main editing and ablation tables.","section":"Tables 1 and 2"}],"minor_comments":[{"comment":"The row label 'IT + FSL + DI(k=5)' is confusing: DI stands for dynamic incrementation, so the effective k is not a fixed value of 5; please clarify the reported configuration and the final k used for the full model.","section":"Table 2"},{"comment":"The NTI comparison is presented only qualitatively; adding a quantitative table (e.g., LPIPS and PSNR versus iteration for vanilla NTI and NTI+REED) would strengthen the claim that inversion methods do not inherently solve the degradation.","section":"Section 5.5, Figure 5"},{"comment":"There are several typographical and formatting issues, including 'limtiations' and 'reconstrctions' in the Related Work, inconsistent spacing in 'V AE', and a stray semicolon in Sec. 4.2 ('however; past k = 6').","section":"Throughout"},{"comment":"The choice of reference image (x0 vs x1) is critical to interpreting the results and should be described in the main text rather than only in the supplement.","section":"Section 5, 'Evaluation metrics'"}],"recommendation":"major_revision","confidential_remarks":"The evaluation protocol issue is the main risk: if the improvements in Table 1 largely reflect self-consistency against a REED-produced first edit rather than fidelity to the source, the paper's central claim is not established. The authors should be given the opportunity to recompute metrics against x0 and to clarify the backpropagation scheme, since these are fixable within the scope of a revision. The novelty is moderate but the problem framing is useful and the qualitative evidence is strong; with the above fixes the paper could be suitable for publication in a graphics or vision venue."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about REED-VAE.\n\nFirst, the method is real and novel: a decoder-only fine-tune of the SD2 VAE, trained on repeated encode-decode cycles with a curriculum over the number of cycles (dynamic k) and a first-step loss against x1. Prior iterative-editing work stays in latent space or accepts the artifacts; nobody retrains the VAE to be cycle-stable. The encode-decode ablations against the true source x0 (Table 2) are unconfounded and show a large, consistent improvement at 15 and 25 cycles. The NTI experiment is a nice addition—it shows inversion doesn't dodge the problem. The multi-method iterative-editing framing is a fair task definition.\n\nSecond, the editing evaluation has a real confound. The supplement says all Table 1 metrics compare the output at iteration 5/15/25 to x1, the first edited output, not to x0. Since x1 is itself produced by the decoder being swapped, a decoder that smooths or weakens the first edit will look good by this protocol even if it isn't actually preserving quality through the edit chain. That is load-bearing for the quantitative editing claims. I don't think it sinks the paper—the x0-based encode-decode results, the qualitative sequences, and the inversion experiments all point the same way—but the size of the Table 1 improvements is likely inflated, and the transfer to editing is not as cleanly shown as the abstract implies.\n\nMinor but worth noting: no error bars or significance tests; code/models promised but not released; and the abstract's 'editability' claim has no direct edit-success metric behind it. The first-step loss trades first-step fidelity to x0 for iterative stability, which is a defensible but under-discussed choice.\n\nWho this is for: anyone doing iterative editing, VAE-based pipelines, or NeRF editing with repeated encodings. It's a practical subfield-level improvement, not a paradigm shift.\n\nRecommendation: send it to peer review. The core mechanism is well demonstrated, the novelty is real, and the evaluation flaws are fixable in revision (report vs x0 in addition to x1, add error bars, release code). A serious referee will home in on the x1 protocol.","headline":"Novel decoder-only VAE fine-tuning that clearly fixes encode-decode artifact accumulation, but the headline editing metrics compare to x1 instead of x0, which likely inflates the quantitative gains.","tokens_in":19608,"tokens_out":5828,"would_cite":true,"duration_ms":56846,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that training a VAE decoder on repeated encode-decode cycles prevents the artifact buildup that blocks iterative, multi-method image editing with diffusion models.","keywords":["iterative image editing","latent diffusion models","variational autoencoder","VAE decoder fine-tuning","image quality preservation","multi-method editing","encode-decode cycles","REED training"],"falsifier":"Run 25 edit iterations with a pipeline in which each edit changes the image independently of VAE error—for example, applying heavy JPEG recompression or a strong conventional filter between encode-decode cycles. If REED-VAE's advantage over the vanilla decoder vanishes or reverses under these conditions, then its robustness is specific to pure reconstruction error and does not transfer to arbitrary multi-method edits, which is the paper's stated goal.","tokens_in":18591,"feed_emoji":"🎨","tokens_out":5789,"duration_ms":53547,"temperature":0.7,"pith_summary":"This paper identifies the variational autoencoder (VAE) as the bottleneck that prevents diffusion models from editing the same image repeatedly: every round trip between pixel space and latent space loses fidelity, and after five to twenty-five operations the accumulated noise destroys the image. It proposes REED (re-encode decode) training, which fine-tunes only the VAE decoder on many unrolled encode-decode cycles so that reconstruction error no longer compounds. The training uses three components: iterative unrolling with a horizon k, dynamic incrementation that raises k as validation loss plateaus, and a first-step loss that measures error against the one-step reconstruction rather than the original image. If the claim holds, any Stable-Diffusion-based editing model can swap in the REED decoder and support long chains of mixed edits—text-guided, mask-guided, example-guided, and conventional pixel-space operations—without staying trapped in latent space. The paper validates the claim across five editing frameworks and reports consistent metric improvements that grow with the number of iterations.","feed_headline":"Retrained VAE decoder keeps images clean across 25 edit rounds","feed_subtitle":"Swap it into any Stable Diffusion editor to cut accumulated noise and artifacts in iterative edits.","key_machinery":"The load-bearing object is the REED training objective, Eq. (2): $L_{\\text{train}} = L_{\\text{MSE}}(x_1, x_k) + \\alpha L_{\\text{LPIPS}}(x_1, x_k) + \\beta D_{\\text{KL}}(z_k, \\mathcal{N}(0, I))$, evaluated after k unrolled encode-decode steps. The scheme starts at k=4 and increments k when the validation loss plateaus (up to 20), a curriculum that prevents divergence at high horizons. The first-step loss anchors the target to x_1 rather than x_0, so the model is asked only to match the vanilla VAE's one-step reconstruction while improving stability over longer chains. Only the decoder is fine-tuned; the encoder stays frozen so latent embeddings remain compatible with existing diffusion models. This combination is what lets a single swapped decoder improve artifact accumulation across many editing models.","core_discovery":"The paper's central claim is that the image degradation seen when editing the same image repeatedly is caused mainly by the VAE's decoder drifting under repeated use, and that this drift can be removed by training the decoder to reconstruct after k full encode-decode cycles rather than after one. REED training unrolls the loop D(E(x_i)) = x_{i+1} for i = 0,...,k-1, computes a perceptual plus MSE loss between the first reconstruction x_1 and the final output x_k, and adds a KL term on the final latent. The scheme starts at k=4 and increments k when validation loss plateaus, up to 20, a curriculum that prevents divergence at high horizons. Only the decoder is fine-tuned; the encoder stays frozen so the latent embeddings remain consistent with the diffusion model's training distribution. Across InstructPix2Pix, MagicBrush, DiffEdit, Paint by Example, SD Inpainting, and NTI-based inversion, swapping in the REED decoder consistently reduces quality degradation at 5, 15, and 25 iterations, with the largest gains at the highest iteration counts.","pith_inferences":["If the decoder-only design truly keeps the encoder's latent distribution intact, the same REED recipe should transfer to other latent diffusion models, including 16-channel VAE models, whose decoders could be trained with the same objective; the paper's supplementary experiments suggest those models also accumulate degradation.","The frequency-domain evidence points to a more specific failure mode: repeated VAE cycles both blur high frequencies and inject new high-frequency artifacts. A REED-style objective could be paired with an explicit spectral regularizer to control which frequencies drift, a testable extension the paper does not pursue.","The first-step-loss principle—measure iterative training against the one-step reconstruction rather than the ground truth—may generalize beyond diffusion to any lossy compression loop where the goal is stability of repeated round trips rather than absolute fidelity."],"forward_implications":["Replacing the vanilla SD2.1 decoder with the REED decoder reduces MSE, LPIPS, SSIM, FID, and PSNR degradation at 5, 15, and 25 iterations across InstructPix2Pix, MagicBrush, DiffEdit, Paint by Example, and SD Inpainting, with gains largest at high iteration counts.","Users can interleave diffusion-based edits with conventional pixel-space operations such as saturation or sharpening in the same session, because each operation starts from a cleanly reconstructed image instead of a progressively corrupted one.","Editing models that rely on inversion, such as NTI-based Prompt-to-Prompt, still improve when paired with REED, although inversion itself contributes noise that the decoder alone cannot fully remove.","REED improves editability in the sense that the edited image stays close to the intended target over more iterations, increasing the likelihood that later edit operations succeed.","The ablation results imply each component matters: iterative training needs k larger than 2, the first-step loss helps convergence, and dynamic incrementation gives the largest gains at 15 to 25 iterations."],"supporting_citations":[{"why":"Supplies the vanilla SD2.1 VAE and latent diffusion pipeline that REED fine-tunes and evaluates against.","marker":"[RBL*22]"},{"why":"Defines the VAE formulation and KL-regularized objective that REED's loss is built on.","marker":"[KW13]"},{"why":"InstructPix2Pix is a primary text-guided editing model used to demonstrate REED's improvement.","marker":"[BHE23]"},{"why":"Blended Latent Diffusion is the mask-guided editing framework shown in the multi-method editing sessions.","marker":"[AFL23]"},{"why":"Paint by Example provides the exemplar-driven editing task where REED halves LPIPS at 15 and 25 iterations.","marker":"[YGZ*23]"},{"why":"The prior iterative editing work that stays in latent space; REED positions itself against its restrictions.","marker":"[JUS*24]"},{"why":"Null-Text Inversion is the inversion baseline whose residual noise shows why decoder training is still needed.","marker":"[MHA*23]"},{"why":"ImagenHub supplies the evaluation images and task structure adapted for iterative editing.","marker":"[KLZ*24]"},{"why":"LPIPS is used both as a loss term in REED training and as a perceptual evaluation metric.","marker":"[ZIE*18]"}],"fun_headline_variants":["New decoder training stops image drift over 25 edits","Fix VAE drift so edits stack without artifacts","Curriculum-trained VAE decoder survives 20+ edit rounds","Decoder retraining makes iterative image editing clean","Train VAE decoder on edit loops to cut noise buildup"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that training on repeated encode-decode cycles of unedited images transfers to real iterative editing, where edits and inversion routines inject their own errors that decoder-only training cannot remove.","fun_headline_variants_meta":{"raw":{"variants":["New decoder training stops image drift over 25 edits","Fix VAE drift so edits stack without artifacts","Curriculum-trained VAE decoder survives 20+ edit rounds","Decoder retraining makes iterative image editing clean","Train VAE decoder on edit loops to cut noise buildup"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1420,"prompt_tokens":999,"completion_tokens":421,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":346}},"tokens_in":615,"tokens_out":421,"duration_ms":4009,"temperature":1.0,"reasoning_tokens":346,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:04:44.055610+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run 25 edit iterations with a pipeline in which each edit changes the image independently of VAE error—for example, applying heavy JPEG recompression or a strong conventional filter between encode-decode cycles. If REED-VAE's advantage over the vanilla decoder vanishes or reverses under these conditions, then its robustness is specific to pure reconstruction error and does not transfer to arbitrary multi-method edits, which is the paper's stated goal.","supporting_citations":[],"review_version":1}