{"id":"060d31ff-ff07-454b-ad09-55c029ef449b","arxiv_id":"2412.01787","paper_version":6,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Reversing a pretrained flow or diffusion generator and fine-tuning it with a classification head yields strong image classifiers, reaching 78.1% top-1 on ImageNet-64.","lead":"This paper proposes PRG, a way to turn a pretrained image-generation model into a classifier by running its generation process backwards and fine-tuning it with labels. It reports state-of-the-art accuracy among generative-model classifiers, including 78% top-1 on ImageNet at 64x64 resolution.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim rests on unretrained baselines at different resolutions and single-run PRG numbers; a controlled 64×64 comparison is needed.","rationale":"The reader's weakest assumption points to the same load-bearing risk: the headline 'state-of-the-art among generative model based methods' is only as strong as the comparability of the baselines. The paper explicitly says 'All results are reported from their original papers' (Tables 3–5), which means different architectures, resolutions, and training protocols. The ImageNet comparison is especially fragile because the only listed generative method that uses fine-tuning is iGPT-L, at a different resolution and with far more parameters, while DMSZC is zero-shot and HybViT uses a small 32² input; DDAE, the strongest fine-tuned generative baseline on CIFAR/Tiny, is not reported on ImageNet. Additionally, the papers shows that label smoothing alone improves CIFAR-10 accuracy by 1.41 points, suggesting that training details materially affect the numbers. Without multiple seeds or a matched retraining of the baselines, the claimed SOTA margin (78.13% vs. 77.5% for DMSZC) could easily be noise or a consequence of training choices. This concern does not invalidate the method as a promising technique, but it directly affects the strongest claim in the abstract and conclusion, so the reader's CONDITIONAL verdict is appropriate and no change is needed.","tokens_in":18522,"tokens_out":11640,"duration_ms":102641,"concrete_test":"Retrain DDAE-DiT-XL2 and DMSZC-DiT-XL2 on ImageNet-64 using PRG's fine-tuning recipe (AdamW, 200 epochs, cosine schedule, label smoothing, no mixup/cutmix, tspan=64, classifier MLP) and run PRG-OTCFM-XL with 5 seeds reporting mean±std. If a baseline reaches or exceeds PRG's mean, or if PRG's 5-seed interval overlaps DMSZC/iGPT-L, the SOTA claim is not established. Also re-evaluate PRG-ICFM-S on CIFAR-10 without label smoothing to isolate the contribution of the training trick.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that PRG is state-of-the-art among generative-model-based methods is not supported by a controlled comparison. Baselines are quoted from other papers with different image resolutions (32², 48², 64², 256²), parameter counts, and fine-tuning protocols; on ImageNet the strongest fine-tuned generative baseline, DDAE, is absent, and the leading listed competitor (DMSZC) is a zero-shot classifier that is not fine-tuned with labels. PRG's own numbers are single-run without error bars, and its fine-tuning recipe includes label smoothing, which alone adds +1.41 on CIFAR-10 (Appendix C.1), so part of the margin over DDAE's 97.2 may reflect training tricks rather than the reversed-generation representation. If DDAE or DMSZC were retrained under PRG's exact 64×64 fine-tuning protocol, or if PRG were run with multiple seeds, the claimed advantage could narrow or reverse.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces PRG (Pretrained Reversible Generation), a two-stage representation learning method. In the first stage, a continuous-time generative model (diffusion or flow matching variants GVP, ICFM, OTCFM) is pretrained on unlabeled images. In the second stage, the model is fine-tuned for downstream classification by running the generative ODE in the reverse direction (from data x0 to latent x1) and attaching a small MLP classifier to the terminal or intermediate states, with a combined cross-entropy and flow-matching loss. The paper reports accuracies of 97.59% on CIFAR-10, 71.12% on Tiny-ImageNet, and 78.1% on ImageNet at 64x64, claiming state-of-the-art among generative-model-based methods, plus OOD robustness on CIFAR-10-C and Tiny-ImageNet-C and a transfer experiment with SiT-XL. The paper also provides a theoretical analysis intended to explain why flow-matching pretraining benefits downstream tasks, and a set of ablations on model type, loss weight, scaling, ODE solvers, and trajectory length.","tokens_in":18760,"tokens_out":13287,"duration_ms":105596,"significance":"The core idea is simple and potentially broadly applicable: the reversibility of continuous-time flow/diffusion models means a pretrained generator can be reused as a feature extractor by reversing its trajectory, avoiding the need to access internal activations. The paper is strong on the empirical side in several respects: the method is model-agnostic; results are robust to ODE solver choice and inference step count (Tables 1, 2 and Appendix C.2); ablations cover model scaling, path type, and loss weight; and the transfer experiment with SiT-XL (Table 7) demonstrates compatibility with large pretrained generative models. The code is publicly available. The main weaknesses are the uncontrolled baseline comparison underlying the SOTA claim (especially on ImageNet), the absence of error bars, and a theoretical section whose information-theoretic argument is not valid for the stated deterministic invertible setting. If the controlled comparison and theory are fixed, the method would be a useful contribution.","major_comments":[{"comment":"The mutual-information justification is not valid for the stated setting. The paper assumes 'deterministic mappings for both the encoder and decoder, which are invertible' (Sec. 3.2). For an invertible deterministic encoder z = F_theta(x), the conditional distribution p(x|z) is a Dirac delta, so the differential mutual information I(X,Z) is ill-defined (or infinite), and Eqs. (4)-(7) cannot be interpreted as a bound on I(X,Z). The argument then replaces the conditional likelihood E_{p(z,x)}[log p_theta'(x|z)] with a bound on the marginal likelihood D_KL(p(x0) || p_theta'(x0)) in Eq. (8); these are different quantities. The empirical proxy in Sec. 4.3.1 computed via the change-of-variables formula (10) is log p(x0) - log p(x1), i.e., the log-density ratio of the generative model's marginal, not a conditional density, so the identity 'E_p[log p(x0|x1)] = I(X0,X1) - H(X0)' does not hold as stated. I recommend either providing a correct information-theoretic statement (e.g., in terms of the ELBO of the generative model) or clearly labeling this as an analogy rather than a derivation.","section":"Section 3.2, Eqs. (4)-(9)"},{"comment":"The assertion that flow matching 'also can be used to increase the likelihood, as shown in Eq. (2)' is unsupported. Eq. (2) is the flow-matching MSE objective; the references cited for likelihood bounds ([42,59,76]) address score-based diffusion ODEs, not the general flow-matching objective used for ICFM/OTCFM. No derivation or reference establishes a likelihood bound for these losses. This claim should be either proved, replaced by a precise citation, or explicitly labeled as a conjecture.","section":"Section 3.2, after Eq. (9); Eq. (2)"},{"comment":"The headline claim of state-of-the-art performance among generative-model-based methods is not supported by a controlled comparison on ImageNet. Table 5 compares PRG at 64x64 (122M) with HybViT at 32x32 (43M), iGPT-L at 48x48 (1362M), and DMSZC at 256x256 (338M, zero-shot); DDAE, the strongest fine-tuned generative baseline on CIFAR/Tiny, is missing from the ImageNet table. All PRG numbers are from single runs without error bars. To substantiate the claim, the authors should provide a matched comparison at a common resolution (e.g., 64x64) using the same fine-tuning loss and augmentation protocol, and report the mean and standard deviation over at least three seeds.","section":"Sec. 4.4.1, Table 5"},{"comment":"Part of the margin over baselines may be due to fine-tuning protocol rather than the proposed representation. Appendix C.1 shows that label smoothing alone improves CIFAR-10 by +1.41 (from 96.18 to 97.59). Since the baseline numbers (e.g., DDAE's 97.2) are quoted from other papers without the same loss and augmentation configuration, the comparison is not like-for-like. The authors should either compare with baselines retrained under the same protocol or quantify the contribution of each training component (loss, augmentation, trajectory length) to the final accuracy.","section":"Appendix C.1, Table 6; Sec. 4.2"}],"minor_comments":[{"comment":"The notation 't ∈ [1, 0]' is non-standard and confusing; please use a standard interval and specify the integration direction and sign for the ODE used to compute F_theta(x0).","section":"Section 3.1"},{"comment":"The summation notation '- NX i=1' appears to be a rendering error; it should be '-∑_{i=1}^N'. Also, the classifier p_phi(y|z) is defined for classification, but Sec. 3.1 also mentions regression; please specify the head and loss for regression.","section":"Eq. (3)"},{"comment":"The baseline row 'MAE (ViT-B/16) [41]' is cited to Ref. [41] (Liu et al., 'Good helper is around you'), which is not the MAE paper; the correct reference for MAE is Ref. [24]. Please fix the citation.","section":"Table 7"},{"comment":"Sec. 4.1 states pre-training uses a fixed learning rate of 1e-4 for 1,200 epochs, while Appendix B.1 Table 2 lists 1000/1000/2000 epochs for CIFAR-10/Tiny-ImageNet/ImageNet; please make these consistent.","section":"Sec. 4.1 vs Appendix B.1"},{"comment":"The abstract and Sec. 3.3 claim adaptation of 'community-developed text-to-image models' and refer to Sec. 4.4.3, but Sec. 4.4.3 only transfers SiT-XL, which is class-conditional rather than text-to-image; either add the claimed text-to-image experiment or correct the claim.","section":"Abstract and Sec. 3.3"},{"comment":"Appendix A.1.1 Table 1 and Table 3 both use labels such as 'PRG-onlyPretrain' and 'PRG w/o pre-training', but they appear to describe different settings (no fine-tuning vs. no pre-training); please clarify the terminology.","section":"Appendix A.1.1 and Table 3"},{"comment":"The accuracy differences between β values are within 0.001 on CIFAR-10, and no error bars or seeds are reported; please report mean ± std over multiple runs or state that the differences are not statistically significant.","section":"Table 9"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the journal's scope. The empirical method appears sound in its internal logic (pretraining helps, fine-tuning is required, ODE discretization is robust), but the SOTA claim and the theoretical framing both need substantial revision before publication. The authors should be encouraged to provide a fully matched comparison with DDAE and a density/likelihood-based justification rather than the current mutual-information argument."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one thing to know: this paper's central move—run a pretrained continuous-time flow/diffusion model backward and fine-tune the whole thing plus a small head with cross-entropy plus flow-matching loss—is clean, testable, and appears to work. On its own terms the empirical evidence is rather strong: three flow-path variants plus a Transformer (SiT), consistent gains over DDAE, HybViT, and the rest, and unusually small drops under CIFAR-10-C/Tiny-ImageNet-C corruption. The ablations are the right ones: frozen features fail, ODE solver choice doesn't matter, step count is robust, pretraining helps monotonically. That is real work and it deserves referee time.\n\nWhat's actually new: using the reverse process of a pretrained continuous-time generator as a feature extractor, without touching internal activations, is distinct from DDAE, iGPT, and likelihood-based generative classifiers. The paper is honest about the failure of frozen features and includes a no-pretraining baseline.\n\nNow the soft spots, in proportion.\n\nThe headline \"state-of-the-art among generative-model-based methods\" is fragile. The ImageNet comparison mixes resolutions and protocols: PRG runs at 64x64; DMSZC is a zero-shot classifier at 256x256; iGPT-L at 48x48; HybViT at 32x32; DDAE is absent from the ImageNet table entirely. On CIFAR-10 the margin over DDAE is 0.39 to 0.45 points, and Appendix C.1 shows label smoothing alone adds 1.41 points on that dataset. So a chunk of the claimed advantage may be a training trick, not the reversed-generation representation. A controlled comparison—DDAE retrained under PRG's exact protocol, or PRG without label smoothing—would settle it. The authors should have done that before claiming SOTA.\n\nSecond, there are no error bars; everything is single-run. For a claim of this specificity, three seeds would be enough.\n\nThird, the theory section over-reaches. The score-matching-to-likelihood bound (Eq. 8) is asserted from prior work without derivation, and the flow-matching extension is stated rather than shown. The mutual-information proxy in Fig 4 is explicitly not a true I(x0;x1) during fine-tuning (the paper concedes H(x1) changes), so the \"compression\" story is hand-wavy. None of this touches the core empirical claim; it's just decoration that should be clearly labeled as such.\n\nMy take: the central claim likely holds, but its precise magnitude is unestablished. A solid referee would ask for the controlled comparison and seeds, not for a new method.\n\nI'd say accept-revise, not desk reject. Worth reading for anyone doing diffusion/flow-based classification.","headline":"A simple and plausible recipe for turning pretrained flow/diffusion models into strong classifiers, but the SOTA claim is only as solid as its unretrained baselines and single-run numbers.","tokens_in":19256,"tokens_out":4678,"would_cite":true,"duration_ms":38517,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Turning a pretrained continuous-time flow/diffusion model backward yields an unsupervised feature extractor that, with a light fine-tuned head, reaches 78% top-1 accuracy on ImageNet at 64×64 and beats prior generative-model-based…","keywords":["representation learning","diffusion models","flow matching","reversible generation","image classification","out-of-distribution robustness","mutual information","unsupervised pretraining"],"falsifier":"Retrain or reuse the identical pretrained diffusion model and compare PRG against a DDAE-style internal-feature extractor at the same 64×64 resolution, same fine-tuning budget, and same evaluation protocol; if PRG does not beat the internal-feature baseline consistently across repeated runs, the claim that reversal is the operative ingredient is falsified. As a second check, track the paper's mutual-information indicator $I(x_0,x_1)+H(x_1)-H(x_0)$ during fine-tuning on a dataset with a known generative process: the explanation predicts it should decline while accuracy rises.","tokens_in":18342,"feed_emoji":"🔄","tokens_out":12492,"duration_ms":97407,"temperature":0.7,"pith_summary":"This paper proposes Pretrained Reversible Generation (PRG), a recipe for turning a pretrained continuous-time generative model into an unsupervised visual feature extractor: instead of reading internal layers, run the model's ODE backward and use the reverse trajectory as the representation. The authors argue that score-matching or flow-matching pretraining maximizes a lower bound on the mutual information between an image and its reversed latent, so the reverse process carries what the model learned about the data, and a small classifier fine-tuned jointly with the generative model outperforms prior generative-model-based approaches. The headline results are 97.65% accuracy on CIFAR-10, 71.33% on Tiny-ImageNet, and 78% top-1 on ImageNet at 64×64 resolution, with stronger out-of-distribution performance under corruptions. The contribution is a simple, model-agnostic way to convert the growing library of pretrained diffusion and flow checkpoints into classifiers without hand-designing which layers to read.","feed_headline":"Run diffusion backward: 78% top-1 on ImageNet at 64×64","feed_subtitle":"A pretrained flow/diffusion model, run in reverse and lightly fine-tuned, becomes a state-of-the-art generative classifier.","key_machinery":"The load-bearing object is the probability-flow ODE (Eq. 1), which makes the generative process of a diffusion or flow model a deterministic, exactly invertible map between data $x_0$ and latent $x_1$; PRG treats the reverse of this map, $F_\\theta(x_0)=x_1$, as the encoder. The training objective is $L_{\\mathrm{total}}=-\\sum_i \\log p_\\phi(y_i|F_\\theta(x_i))+\\beta L_{\\mathrm{FM}}(x)$ (Eq. 3), combining a classification head with a flow-matching term that preserves the generative structure. The theoretical spine is the mutual-information lower bound: pretraining by flow matching maximizes a lower bound on $I(X_0,X_1)$ through the score/flow-matching likelihood bound (Eq. 8), so the reversed latent is information-preserving; fine-tuning then prunes reconstruction-irrelevant detail. Since the ODE can be integrated with adaptive solvers, the network acts as an infinite-depth continuous feature extractor whose features are stable across step counts and solver choices.","core_discovery":"The central claim is that the reverse of a pretrained continuous-time generative process is itself an unsupervised encoder: because the probability-flow ODE is exactly invertible, mapping an image $x_0$ through the same model backward to $x_1$ produces a latent that retains a large lower bound on the mutual information $I(X_0, X_1)$, and after light fine-tuning this latent supports strong downstream classification. The paper establishes the mechanism in two parts. Theoretically, score-matching and flow-matching pretraining are shown to increase data likelihood and therefore the mutual-information lower bound (Eqs. 4–8). Empirically, end-to-end fine-tuning with a two-layer MLP head and a small flow-matching regularizer (Eq. 3) is necessary—freezing the generative model drops accuracy to 47.10% on CIFAR-10 and 58.04% on Tiny-ImageNet, against roughly 97% and 71% when it is fine-tuned—and features at any time $t$ along the reverse trajectory remain usable, making the extractor continuous. On this basis PRG reports state-of-the-art accuracy among generative-model-based classifiers on three benchmarks and shows the approach transfers to a community text-to-image backbone.","pith_inferences":["The paper demonstrates that features at different trajectory times $t$ are usable but does not prescribe how to choose $t$; a natural extension is to concatenate features from several $t$ values, pooling low-level detail near $x_0$ with semantic structure near $x_1$, which could beat any single cut point.","The straightness results (Tab. 8: OTCFM reaches 97% in 128 epochs versus 162 for the curved GVP path) suggest a selection principle the paper leaves implicit: among equally strong generative models, prefer the flow with straighter trajectories because it needs fewer pretraining steps to reach comparable representations; consistency and rectified-flow models should plug into PRG with even faster ad","Because the framework reads only the ODE trajectory, a pure-MLP flow model that satisfies the same invertibility should yield comparable representations, which would isolate whether the ODE structure rather than the image-backbone inductive bias is doing the work.","The same reversal could be applied to prompt-conditioned text-to-image models for zero-shot classification by comparing reconstruction losses of reverse runs under different text labels, bridging PRG's fine-tuned classifiers with the zero-shot generative classifier line the paper surveys."],"forward_implications":["Any pretrained continuous-time diffusion or flow model can be converted into a downstream classifier by running it in reverse and fine-tuning a small head, with no access to internal features.","Stronger generative pretraining directly improves downstream classification: accuracy rises monotonically with pretraining epochs and with the mutual-information indicator the paper tracks.","Fine-tuning must update the generative parameters; freezing the flow model drops accuracy to 47.10% on CIFAR-10 and 58.04% on Tiny-ImageNet, far below the fine-tuned numbers.","The extracted features are continuous and stable to discretization: a checkpoint fine-tuned at the trajectory midpoint works across 20 to 1000 inference steps and ±20% offsets with negligible loss.","The same recipe transfers to community text-to-image backbones and improves out-of-distribution accuracy under common image corruptions."],"supporting_citations":[{"why":"Supplies the probability-flow ODE whose exact invertibility makes the reverse generation an encoder.","marker":"[58]"},{"why":"Provides the neural-ODE adjoint method used to backpropagate through the reversed trajectory during fine-tuning.","marker":"[11]"},{"why":"Defines the flow-matching paths (ICFM/OTCFM) that PRG builds on as generative backbones.","marker":"[60]"},{"why":"Source of the denoising-autoencoder mutual-information argument that PRG extends to continuous-time generative models.","marker":"[65]"},{"why":"DDAE, the main generative representation-learning baseline; PRG adopts its evaluation settings to beat it.","marker":"[68]"},{"why":"HybViT, a hybrid diffusion/classifier baseline that PRG compares against in the generative-classifier line.","marker":"[70]"},{"why":"Score-based generative classifier baseline used for out-of-distribution comparison and as a reference generative-classifier approach.","marker":"[77]"},{"why":"Supplies the likelihood bound for score-based diffusion models used to justify that pretraining maximizes mutual information.","marker":"[59]"}],"fun_headline_variants":["Reverse diffusion: unsupervised features hit 78% on ImageNet","Run generation backwards: SOTA generative classifier at 64×64","Pretrained flow, reversed: a new way to learn visual features","Backward generative models: 78% ImageNet top-1 without labels","Invert diffusion for features: beats prior generative classifiers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline comparisons assume that accuracy numbers reported by other papers at different resolutions and parameter counts are directly comparable to PRG's single-run results at 64×64, and the theory assumes that flow-matching pretraining genuinely raises the mutual-information lower bound the argument leans on.","fun_headline_variants_meta":{"raw":{"variants":["Reverse diffusion: unsupervised features hit 78% on ImageNet","Run generation backwards: SOTA generative classifier at 64×64","Pretrained flow, reversed: a new way to learn visual features","Backward generative models: 78% ImageNet top-1 without labels","Invert diffusion for features: beats prior generative classifiers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1370,"prompt_tokens":991,"completion_tokens":379,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":607,"completion_tokens_details":{"reasoning_tokens":290}},"tokens_in":607,"tokens_out":379,"duration_ms":3611,"temperature":1.0,"reasoning_tokens":290,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:06:22.733244+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain or reuse the identical pretrained diffusion model and compare PRG against a DDAE-style internal-feature extractor at the same 64×64 resolution, same fine-tuning budget, and same evaluation protocol; if PRG does not beat the internal-feature baseline consistently across repeated runs, the claim that reversal is the operative ingredient is falsified. As a second check, track the paper's mutual-information indicator $I(x_0,x_1)+H(x_1)-H(x_0)$ during fine-tuning on a dataset with a known generative process: the explanation predicts it should decline while accuracy rises.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the neural-ODE adjoint method used to backpropagate through the reversed trajectory during fine-tuning."},{"cited_title":"Improving and generalizing flow-based gener- ative models with minibatch optimal transport","cited_arxiv_id":null,"evidence_quote":"Defines the flow-matching paths (ICFM/OTCFM) that PRG builds on as generative backbones."},{"cited_title":"Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion","cited_arxiv_id":null,"evidence_quote":"Source of the denoising-autoencoder mutual-information argument that PRG extends to continuous-time generative models."},{"cited_title":"Denoising diffusion autoencoders are unified self-supervised learners","cited_arxiv_id":null,"evidence_quote":"DDAE, the main generative representation-learning baseline; PRG adopts its evaluation settings to beat it."},{"cited_title":"Maximum likelihood training of score-based diffusion models","cited_arxiv_id":null,"evidence_quote":"Supplies the likelihood bound for score-based diffusion models used to justify that pretraining maximizes mutual information."}],"review_version":1}