{"id":"a3a9856c-9b49-4196-a8f4-35354dea2b0b","arxiv_id":"1908.05861","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A trained noise-prior predictor converts iterative latent search in GAN inpainting into a single feed-forward pass, and a recurrent grouped-prior model enables unsupervised sequence inpainting.","lead":"The paper replaces a slow, per-image optimization step in unsupervised GAN inpainting with a trained network that predicts the latent code directly, and it extends the idea to video frames with a recurrent model. The result is a reported 1500x speedup and better temporal consistency than the iterative baseline, which matters if GAN inpainting is to run in real time.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported quality gain over Yeh et al. is confounded because the learned-prior comparison changes the generator and the loss function at the same time; the central claim needs a fixed-G, fixed-loss ablation.","rationale":"The paper's bottleneck framing is legitimate, and the idea of amortizing Yeh et al.'s latent search is a plausible and useful direction; a feed-forward predictor should in principle be far faster than 1500 gradient-based iterations. The concern here is not whether the method could work but whether the paper demonstrates that the learned prior, rather than the accompanying generator upgrade and extra loss, is responsible for the improved quality. This matters because the abstract's central claim is explicitly 'simultaneous improvement in reconstruction quality' from the parametric prior. The reader's weakest-assumption point about GAN latent-space coverage is related but downstream: even when the GAN covers the data, the reported advantage has not been isolated. I therefore agree with the reader's conditional verdict while suggesting a sharper condition: reproduce the comparison on one frozen G/D/loss set and report wall-clock time.","tokens_in":11558,"tokens_out":9627,"duration_ms":104532,"concrete_test":"On CelebA-HQ with a single frozen G/D, take the same masked test set and compare: (a) z found by iterative optimization of Eq. 2 using exactly the same loss set as M1, including the gradient-difference term and same hyperparameters, capped at 1500 iterations; (b) z = P_theta_z(Id) from the trained predictor. Report PSNR, temporal consistency where applicable, and measured wall-clock time per image, not iteration counts. If the iterative search on the same G and loss reaches or exceeds the feed-forward quality, the paper's 'simultaneous improvement' claim is not due to the learned prior; if not, the claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that a learned noise-prior network gives 'simultaneous improvement in reconstruction quality' over iterative latent search (Abstract, Section 4.1, Table 1) is not supported by a controlled comparison. The proposed M1 differs from the Yeh et al. baseline in at least three coupled ways: (1) a different pretrained generator, which Section 5.1.2 argues is more realistic and Section 5.1.3 says is trained with progressive growing at 256x256; (2) the extra gradient-difference loss in Eq. 5 that is absent from the baseline; and (3) the learned feed-forward predictor itself. Consequently, the large PSNR gains such as 15.7 dB to 20.1 dB on CelebA-HQ in Table 1 cannot be attributed to replacing iterative optimization with P_theta_z. No experiment holds G, D, and the loss fixed while varying only the inference procedure.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to accelerate unsupervised GAN-based inpainting by replacing the iterative latent-code optimization of Yeh et al. [29] with a learned feed-forward network P_theta_z that predicts a noise prior z from a masked image. Training of P_theta_z uses a contextual loss, a realism loss from a frozen discriminator, and a gradient-difference loss, with optional structural priors (facial landmarks) and an LSTM-based grouped prior for sequences. The authors claim a roughly 1500x speedup over [29] with improved reconstruction quality, and present the first unsupervised GAN-based sequence inpainting method, adding a subsequence consistency loss for temporal smoothness. Experiments on SVHN, Cars, CelebA, CelebA-HQ, synthetic sequences, and VidTIMIT compare against [29] and hybrid baselines using PSNR, a proposed temporal-consistency metric, and the MOVIE video-quality metric.","tokens_in":11726,"tokens_out":3175,"duration_ms":29683,"significance":"If the central claims hold, the paper makes a useful contribution: it converts an iterative, slow unsupervised inpainting procedure into a one-shot feed-forward inference, and it extends unsupervised GAN inpainting to sequences. The idea of learning a noise-prior predictor is simple and potentially impactful, and the paper is honest about the current limitation that fully unsupervised GAN inpainting does not scale to ImageNet/Places2 (Section 6). The synthetic-sequence temporal-consistency evaluation and the use of the MOVIE metric are sensible evaluation choices. However, the empirical evidence as presented is not yet convincing because the main comparison against [29] changes multiple factors at once, the reported gains for the subsequence consistency loss are internally inconsistent, the temporal-consistency metric is defined incorrectly in the text, and crucial implementation details are missing. The core concept is defensible, but the load-bearing experimental support needs substantial revision.","major_comments":[{"comment":"The central claim of simultaneous quality improvement over [29] is not supported by a controlled comparison. The proposed M1 differs from the Yeh et al. baseline in at least three coupled ways: (1) a different pretrained generator, which Section 5.1.2 argues is more realistic and Section 5.1.3 trains with progressive growing at 256x256; (2) the additional gradient-difference loss in Eq. (5), which is absent from the baseline's objective in Eq. (2); and (3) the learned feed-forward predictor itself. Consequently, the large PSNR gains such as 15.7 dB to 20.1 dB on CelebA-HQ in Table 1 cannot be attributed solely to replacing iterative optimization with P_theta_z. No experiment holds G, D, and the loss function fixed while varying only the inference procedure. This is the key ablation needed to support the paper's main contribution.","section":"Section 4.1, Eq. (6), Table 1"},{"comment":"The claimed benefit of the subsequence consistency loss is contradicted by the reported numbers. In Table 1 on CelebA@128, M5 (with consistency loss) improves over M4 (without it) on RC (27.6 vs. 26.3) but regresses on RF (28.0 vs. 29.8) and RCh (26.9 vs. 29.4). The paper in Section 5.2.2 asserts that L_ss improves temporal consistency, but the data show mixed results, and M5 is only reported for one dataset and mask set. Without error bars or a clear explanation for the regression, this claim is not established.","section":"Table 1, Section 5.2.2"},{"comment":"The temporal consistency metric is defined inconsistently. The text states 'Ideally, we want eta_temp = 0,' but Eq. (12) defines eta_temp as the mean pairwise PSNR, for which higher values mean more consistent (identical frames give infinite PSNR). The Table 1 caption says 'Higher values of consistency are better,' which contradicts the 'eta_temp = 0' statement. This needs to be corrected and the metric re-stated unambiguously, since it is a primary evaluation tool for the sequence contributions.","section":"Eq. (12), Section 5.2.1, Table 1 caption"},{"comment":"The experimental sections lack the details needed to reproduce or verify the central speed and quality claims. The exact architectures of P_theta_z, the LSTM-CNN module, and the pretrained GANs are not specified; the loss weights lambda_1 through lambda_4, the group size W, and the noise dimension d are not given; the gradient-difference term in Eq. (5) contains an incomplete expression 'G()' with no argument; and no wall-clock timings or error bars are reported for the 1500x speedup claim or for any PSNR/MOVIE numbers. The paper would be substantially strengthened by releasing code, specifying all hyperparameters, and reporting standard deviations over multiple runs.","section":"Section 4, Equations (3)-(6), Section 5, Tables 1-2"}],"minor_comments":[{"comment":"There are several typos and grammatical slips, including 'accelarate' (Abstract), 'simultaneousness' (Section 1), and 'Standford' for Stanford (used throughout, e.g., Section 5.1). These should be corrected.","section":"Abstract and Section 1"},{"comment":"The conditional GAN objective in Eq. (8) writes the second expectation as E_z[1 - D(G(z))], but when conditioning on c, the discriminator should also receive c in the fake term, i.e., E_{z,c}[1 - D(G(z), c)] or an equivalent formulation. Please clarify.","section":"Eq. (8)"},{"comment":"The human evaluation is described as 'the classification accuracy is 5.8% higher for DIP(p = 10^-3)' but it is unclear what 'DIP' refers to here and what statistical test yields the reported p-values. Please specify the hypothesis and the test used.","section":"Section 5.1.2"},{"comment":"There is a stray word 'ames' in the paragraph before Figure 3, and the figure caption states 'Unlock sign means parameters to update' but the figure itself is not visible in the text. Please ensure the figure and caption are consistent.","section":"Figure 3 and Section 4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and has a plausible core idea, but the refereeing process should insist on a controlled ablation that isolates the learned-prior effect from the change of generator and loss, and on complete reproducibility details. The current version's main quantitative claims are not yet reliable. I would also encourage the editor to consider whether the missing code and hyperparameters, combined with the confounded comparison, warrant a higher bar before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's real contribution is amortizing the iterative latent search of Yeh et al. into a learned feed-forward predictor, plus an LSTM-based grouped prior for unsupervised sequence inpainting. The sequence part is the newest: as far as I can tell this is the first unsupervised GAN method that inpaints videos with temporal consistency, and the subsequence consistency loss is a simple but sensible addition. The structural prior (facial landmarks) is also a nice touch, and the synthetic-sequence consistency metric is a reasonable way to evaluate temporal stability. I believe the authors here; the speedup is real in the sense that 1500 optimization iterations are replaced by one network pass, even if the actual wall-clock gain is not measured and the \"1500x\" is an iteration-count comparison.\n\nThe soft spot is the quality claim. The main tables show large PSNR gains over Yeh et al., but the comparison is not controlled. The proposed model uses a different generator (progressive GAN at 256x256 versus whatever Yeh used), adds a gradient-difference loss, and then swaps iterative optimization for a learned predictor. You cannot attribute the 15.7-to-20.1 dB jump on CelebA-HQ to the learned prior alone. The stress-test note has this right. A fixed-G, fixed-loss ablation would be needed to support the \"simultaneous improvement in reconstruction quality\" claim. The paper also has the usual reproducibility gaps: no code, no hyperparameters, no error bars, and at least one or two equations are written sloppily (Eq. 3 and Eq. 5 look malformed). Table 1 has an internal inconsistency: M5 improves over M4 on RC but regresses on RF and RCh, which undercuts the claim that the subsequence consistency loss helps across masks. These are addressable.\n\nThe authors do concede in Section 6 that current GANs cannot handle natural images like ImageNet, so the method is bounded by GAN coverage. That is honest. The paper shows clear thinking and the literature coverage looks adequate, though the \"first sequence inpainting\" claim deserves a prior-art check before publication.\n\nWho is this for? People working on GAN-based inpainting, especially anyone who wants to avoid iterative test-time optimization. It deserves a serious referee: the core idea is useful, the sequence extension is novel, and the flaws are fixable with better evaluation rather than being fatal. I would send it out, but with a strong request for controlled ablations and reproducibility details.","headline":"The speedup story is real and the sequence part is genuinely new, but the quality gain over Yeh et al. is confounded by changing the generator and loss at the same time as the inference procedure.","tokens_in":684,"tokens_out":2110,"would_cite":true,"duration_ms":32165,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Unsupervised GAN inpainting can run as a single feed-forward pass by learning to predict the noise prior, giving roughly 1500x speedup and stronger temporal consistency.","keywords":["GAN inpainting","unsupervised image completion","noise prior prediction","structural priors","facial landmarks","sequence inpainting","recurrent grouped prior","temporal consistency"],"falsifier":"Train the noise-prior predictor on random central masks only, then evaluate on freehand and checkerboard masks; if the reconstruction PSNR drops to the iterative baseline's level, the learned prior has not generalized and the 'one feed-forward pass' claim is conditional on matching mask statistics.","tokens_in":11315,"feed_emoji":"⚡","tokens_out":7633,"duration_ms":68684,"temperature":0.7,"pith_summary":"The paper aims to show that the slow, iterative search for a latent noise vector in unsupervised GAN inpainting can be replaced by a learned predictor. A small network is trained, with the generator and discriminator of a pretrained GAN frozen, to map a masked image directly to a matching noise prior. At test time this yields the inpainted image in one feed-forward pass, about 1500 times faster than iterative latent optimization, with usually better reconstruction fidelity. The same predictor is extended to video with a recurrent grouped-prior network and a subsequence consistency loss, giving what the authors describe as the first unsupervised GAN-based sequence inpainting with improved temporal consistency.","feed_headline":"GAN inpainting gets 1500x faster with a learned noise prior","feed_subtitle":"Unsupervised image and video inpainting now runs in a single forward pass and beats several supervised benchmarks.","key_machinery":"The central object is a learned noise-prior predictor $P_{\\theta_z}$, a CNN that maps a masked image to the $d$-dimensional latent vector $z$ expected by a frozen pretrained GAN generator. It is trained with the composite loss $L_{\\mathrm{com}}^z = \\lambda_1 L_c + \\lambda_2 L_r + \\lambda_3 L_g$, where $L_c$ is the contextual (unmasked-region) fidelity, $L_r$ is the realism score from the frozen discriminator, and $L_g$ is a gradient-difference loss; this converts the iterative argmin of Eq. (2) into the direct evaluation of Eq. (7). For sequences, a CNN descriptor network feeds an LSTM that emits grouped priors for windows of $W$ frames, optimized by the grouped loss $L_{\\mathrm{gr}}^z$ plus a subsequence consistency term $L_{\\mathrm{ss}}^z$ that penalizes pairwise latent differences within a window. Structural priors enter as conditional variables $c$, here detected facial landmarks, so the generator learns to respect pose and expression while filling the mask.","core_discovery":"The central claim is that priors are the deciding factor in GAN inpainting: a data-driven parametric network $P_{\\theta_z}$ can predict the latent code a frozen generator needs, so the per-image iterative optimization used by the unsupervised baseline becomes unnecessary. The predictor is trained only on unmasked pixels, using a contextual loss, a realism loss from the frozen discriminator, and a gradient-difference loss; no pixel under the mask is ever supervised. Adding a structural prior, exemplified by facial landmarks used as a conditioning variable $c$, biases completion toward samples that preserve the pose and expression of the reference, and also improves the generator's samples. For sequences, an LSTM-CNN model predicts grouped noise priors over windows of frames, and a subsequence consistency loss penalizes appearance drift between adjacent reconstructions. Across SVHN, Stanford Cars, CelebA, CelebA-HQ, synthetic sequences, and VidTIMIT, the paper reports consistent gains over the unsupervised baseline in reconstruction quality and temporal consistency, with inference speedups around 1500x and performance comparable or better than hybrid supervised+adversarial methods.","pith_inferences":["If the learned-prior trick transfers, other inverse problems that currently rely on GAN latent optimization—such as super-resolution, denoising, or style transfer under masks—could adopt the same single-pass formulation whenever a suitable pretrained generator exists.","The speed and quality gains likely depend on the mask distribution seen during training; a predictor trained on random central masks may not generalize to freehand or checkerboard masks without fine-tuning, so the reported speedup should be read as tied to the experiment's mask protocol.","The failure boundary the authors concede for ImageNet-like scenes suggests the method could serve as a diagnostic for GAN coverage: the gap between single-pass and iterative reconstructions measures how well the generator's latent space covers the data.","Measured end-to-end, including structural-prior extraction and final mask composition, the 1500x speedup may shrink because those constant overheads are excluded from the headline ratio."],"forward_implications":["Unsupervised GAN inpainting becomes a single feed-forward network evaluation at test time, replacing roughly 1500 gradient-descent iterations per image with one forward pass and a small overhead for structural-prior extraction.","Because the masked pixels are never used in training, the method stays fully unsupervised; better future generators should slot into the same pipeline without retraining the inpainter.","The grouped LSTM prior and subsequence consistency loss improve temporal coherence of reconstructed videos, as measured by higher synthetic-sequence consistency and lower MOVIE scores than the iterative baseline.","Conditioning on structural priors gives the generator a degree of control over pose and expression, so completion respects the reference identity and expression rather than sampling an arbitrary plausible face.","Using progressive GAN training lets the same method operate at 256x256 resolution, where the iterative latent-optimization baseline degrades sharply."],"supporting_citations":[{"why":"The iterative latent-optimization baseline that the paper replaces with a single feed-forward prior prediction.","marker":"[29]"},{"why":"Supplies the GAN formulation whose generator and discriminator are frozen and reused.","marker":"[8]"},{"why":"Provides the real-time facial landmark detector used as the structural prior.","marker":"[13]"},{"why":"Long short-term memory network architecture used for grouped noise prior learning in sequences.","marker":"[10]"},{"why":"Progressive growing training strategy used to scale the GAN to 256x256 resolution.","marker":"[12]"},{"why":"Hybrid supervised+adversarial baseline that the paper compares against to show unsupervised methods can be competitive.","marker":"[23]"},{"why":"Hybrid supervised+adversarial baseline used for image and temporal consistency comparisons.","marker":"[11]"},{"why":"Contextual-attention inpainting benchmark that the paper's unsupervised method matches or improves in several settings.","marker":"[30]"},{"why":"Face-completion benchmark that also uses structural priors, strengthening the paper's argument that object structure helps inpainting.","marker":"[19]"},{"why":"MOVIE metric used to evaluate temporal quality of reconstructed video.","marker":"[25]"}],"fun_headline_variants":["Learned priors make GAN inpainting 1500x faster","Unsupervised GAN inpainting hits 1500x speedup","Noise priors: the hidden key to fast GAN inpainting","Predictive priors turn GAN inpainting into one forward pass","Structural priors boost unsupervised GAN inpainting quality"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the generative model used to synthesize images already covers the kind of images being inpainted; the authors concede this fails for broad natural scenes like ImageNet, so the result is bounded by how good the generator is.","fun_headline_variants_meta":{"raw":{"variants":["Learned priors make GAN inpainting 1500x faster","Unsupervised GAN inpainting hits 1500x speedup","Noise priors: the hidden key to fast GAN inpainting","Predictive priors turn GAN inpainting into one forward pass","Structural priors boost unsupervised GAN inpainting quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000223,"raw_usage":{"total_tokens":1503,"prompt_tokens":1036,"completion_tokens":467,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":652,"completion_tokens_details":{"reasoning_tokens":371}},"tokens_in":652,"tokens_out":467,"duration_ms":3941,"temperature":1.0,"reasoning_tokens":371,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:03:22.779628+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the noise-prior predictor on random central masks only, then evaluate on freehand and checkerboard masks; if the reconstruction PSNR drops to the iterative baseline's level, the learned prior has not generalized and the 'one feed-forward pass' claim is conditional on matching mask statistics.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The iterative latent-optimization baseline that the paper replaces with a single feed-forward prior prediction."},{"cited_title":"Goodfellow, J","cited_arxiv_id":null,"evidence_quote":"Supplies the GAN formulation whose generator and discriminator are frozen and reused."},{"cited_title":"Kazemi and J","cited_arxiv_id":null,"evidence_quote":"Provides the real-time facial landmark detector used as the structural prior."},{"cited_title":"Karras, T","cited_arxiv_id":null,"evidence_quote":"Progressive growing training strategy used to scale the GAN to 256x256 resolution."},{"cited_title":"Pathak, P","cited_arxiv_id":null,"evidence_quote":"Hybrid supervised+adversarial baseline that the paper compares against to show unsupervised methods can be competitive."},{"cited_title":"Iizuka, E","cited_arxiv_id":null,"evidence_quote":"Hybrid supervised+adversarial baseline used for image and temporal consistency comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contextual-attention inpainting benchmark that the paper's unsupervised method matches or improves in several settings."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Face-completion benchmark that also uses structural priors, strengthening the paper's argument that object structure helps inpainting."},{"cited_title":"Seshadrinathan and A","cited_arxiv_id":null,"evidence_quote":"MOVIE metric used to evaluate temporal quality of reconstructed video."}],"review_version":1}