{"id":"8ba37d7b-acaf-478c-bf68-6d738b1edd9d","arxiv_id":"2412.00754","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"CtrlNeRF learns a single shared neural radiance field generator that can synthesize controllable, 3D-consistent images of multiple object classes and colors using label-embedded latent codes.","lead":"This paper describes CtrlNeRF, a generative 3D model that produces images of objects with user-chosen class and color labels while keeping 3D consistency. It is a compact extension of the GRAF model that aims to cover many object types with one shared network, reducing storage and training cost.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Shared-MLP capacity degrades with scene count and is never compared against per-scene baselines at matched storage, so the central multi-scene claim lacks quantitative support.","rationale":"I agree with the reader's identification of the load-bearing premise: the single shared-weight MLP must keep distinct scenes separable, and the paper's own figures show degradation as the number of scenes grows. The paper presents no controlled scaling study against per-scene baselines, so the capacity claim is not quantitatively grounded. The ablation further reveals that per-label output heads are essential for convergence, meaning the model is not a fully shared representation. These observations reinforce the conditional verdict rather than overturning it: the idea is plausible, but the evidence is incomplete. The proposed test—comparing FID versus total storage to per-scene GRAF across scene counts—would settle whether the constant-storage advantage is real at matched quality. I do not see an internal inconsistency in the method; the weakness is the lack of a decisive empirical comparison, which is exactly why the reader's conditional acceptance is appropriate.","tokens_in":15568,"tokens_out":7256,"duration_ms":73226,"concrete_test":"Train CtrlNeRF and per-scene GRAF on the same CARs subsets with N=1,2,4,8 classes (fixed M=4 styles), measuring per-class FID and total model storage in megabytes. Plot FID versus total storage for both methods. If per-scene GRAF achieves lower FID than CtrlNeRF at the same total storage for any N, the claimed constant-storage advantage of the single-MLP approach is refuted; if CtrlNeRF is Pareto-better, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that a single shared-weight MLP can represent multiple scenes with high fidelity and explicit control. The authors' own scaling evidence undermines this: Fig.10 shows FID rising steeply as the number of classes N and styles M increases, and Sec.6 concedes that 'image quality diminishes as the quantity and complexity of the scenes increase.' On the 6-scene LLFF dataset, the mean FID is roughly 80 (Fig.9), versus about 46.6 on the 16-combination CARs set. Yet no comparison is made to the natural baseline—per-scene GRAF models on the same data—so it is unknown whether this degradation is an acceptable trade-off for constant storage. The ablation (Table 1) also shows that without the per-label output arrays, training does not converge (Model II, 'no converge'). This indicates the model is a shared backbone with per-class density heads and per-style color heads, not a fully shared representation; the latent-label multiplication is an additional conditioning whose contribution is not isolated. Consequently, the Sec.5.5 assertion that the method can store multiple scenes 'without significantly sacrificing image quality' is unsupported by any matched-quality scaling comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CtrlNeRF, a GRAF-based generative neural radiance field that aims to use a single MLP with shared weights to represent multiple scenes, conditioned on class labels (shape) and color labels (appearance). The generator is described as taking label-embedded latent codes and outputting density and color arrays indexed by labels, with a VGG-based discriminator as an auxiliary classifier. Experiments on a synthetic CARs dataset, NeRF-Synthetic, and LLFF report FID/KID, PSNR/SSIM, qualitative results for label-controlled generation, novel view synthesis, and interpolation, together with an ablation study.","tokens_in":15841,"tokens_out":6370,"duration_ms":54526,"significance":"If the central claim held, representing multiple scenes in one MLP with explicit control would be a useful efficiency result for 3D-aware generative models. The paper has strengths: it reports quantitative FID/KID per class and style, includes an ablation, and honestly documents degradation with scene count in Fig. 10, which is a falsifiable observation. However, the evidence does not currently establish the central claim: the scaling results show substantial quality loss, no per-scene baseline at matched storage is provided, and the PSNR/SSIM comparison to CLIP-NeRF is not a matched comparison. The contribution is incremental over GRAF, but the direction is reasonable.","major_comments":[{"comment":"The PSNR/SSIM comparison in Table 2 is not a matched comparison: CLIP-NeRF is a per-scene reconstruction and manipulation method trained separately on posed images of each scene, whereas CtrlNeRF is a generative model with a single MLP across scenes. Reporting that CLIP-NeRF achieves higher PSNR/SSIM and then concluding that CtrlNeRF is 'on par' or 'marginally less impressive' is misleading because the methods solve different tasks. To support the multi-scene claim, the authors should compare against per-scene GRAF models (or a per-scene variant of CtrlNeRF) at matched total storage and report FID per scene, and use PSNR/SSIM only against equally generative baselines.","section":"Table 2, §5.5"},{"comment":"The scaling experiments show FID rising steeply as the number of classes N and styles M increases; for example, the mean FID on LLFF is roughly 80 versus about 46.6 on CARs. The text in §5.5 asserts that the method stores multiple scenes 'without significantly sacrificing image quality,' but no comparison against per-scene baselines at matched storage is given, and §6 concedes that image quality diminishes as scene quantity and complexity increase. The central claim of efficient multi-scene representation therefore lacks quantitative support. Provide a matched-storage comparison (e.g., N per-scene GRAF models versus one CtrlNeRF) reporting FID and per-scene quality, and report error bars across seeds.","section":"§5.1, Fig. 10, §6"},{"comment":"The ablation shows that removing the per-label output arrays (Model II) causes training not to converge, meaning these arrays are load-bearing. This indicates that the model is a shared backbone with per-class density heads and per-style color heads, rather than a fully shared representation. The contribution of the label-embedding multiplication (the input modification) is not isolated: there is no variant that keeps the output arrays but removes the multiplication, for example by using concatenation or no embedding at all. Please add such an ablation and clarify exactly which parameters are shared across scenes.","section":"§5.4, Table 1"},{"comment":"The sentence 'To further improve image quality, we adopted posed images for training and replaced adversarial loss with reconstruction loss' contradicts the rest of the method: Equation 6 includes L_adv, Algorithm 1 updates both D_phi and D_vgg, and the positioning of CtrlNeRF as a GRAF-derived generative model suggests training without pose supervision. If posed images are indeed used, this is a major deviation that must be stated in the setup; if not, the sentence should be removed. Please clarify the exact training objective and data requirements.","section":"§3, Discriminator"}],"minor_comments":[{"comment":"M and N are swapped in the array definitions: the density array is written as [σ(i)]_{i=0}^{N-1} while M denotes the number of classes and N the number of styles, but labels i and j index classes and styles, respectively, in Algorithm 1 and Fig. 4; make the notation consistent.","section":"§3, Output"},{"comment":"Equation 1 has mismatched parentheses, and the FID score is attributed to 'Kanazawa et al.' although the standard reference is Heusel et al.; please correct both.","section":"Eq. (1), §4.3"},{"comment":"Unresolved reference placeholders appear for pixelNeRF and VGG16 in Section 3, and 'GRIFFEE' should be 'GIRAFFE' throughout the text and figure captions.","section":"§3, References"},{"comment":"The statement that the authors 'successfully entangled and disentangled the label codes into and out of the latent code' is contradictory and should be rephrased to describe the intended conditioning mechanism.","section":"§6"},{"comment":"The GRAF description swaps the roles of z_s and z_a, calling z_a the shape code and z_s the appearance code, which conflicts with the notation in the abstract and Figure 2; please unify the notation.","section":"§3, GRAF review"},{"comment":"FID and KID values are reported without standard deviations or the number of seeds; for a GAN evaluation, results from at least three training runs should be reported as mean ± std.","section":"Table 1, Figs. 8–10"},{"comment":"The paper does not specify the resolution, training iterations, or data used to train GRAF and GIRAFFE for the Table 2 comparison, so it is unclear whether the comparison is on the same scenes and settings; please document the protocol.","section":"§5.5, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The contribution over GRAF is incremental and the evaluation does not yet support the headline multi-scene claim. The paper may be suitable for a workshop or a more applied venue after substantial revision; the editor may also wish to consider the unresolved citation placeholders and the apparent contradiction about posed images."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: CtrlNeRF is exactly what it looks like—GRAF with label-embedded latent codes, per-class density / per-style color output arrays, and an auxiliary VGG classifier. That combination is not in the cited literature, and it is a legitimate, modest engineering extension. The paper is honest about its main weakness: quality degrades as scenes are added, and the shared MLP entangles them. Credit where due: the ablation is useful, especially the finding that without the output arrays training does not converge. That tells you the output modification is load-bearing, not cosmetic.\n\nNow the soft spots, in descending order.\n\nThe central claim is that a single MLP can store multiple scenes 'without significantly sacrificing image quality' (Sec 5.5). The evidence does not support that. Fig.10 shows FID climbing steeply with N and M, and Fig.9 puts mean FID around 80 on LLFF versus ~46 on the car set. There is no comparison to the natural baseline: per-scene GRAF models with matched total storage. Without that, we don't know whether the quality/storage trade-off is acceptable. The stress-test note is right: the model is a shared backbone with per-label output heads, not a fully shared representation, and the label-embedding multiplication's contribution is never isolated.\n\nThe quantitative evaluation is thin. No error bars, no multiple seeds, and the PSNR/SSIM comparison to CLIP-NeRF is apples-to-oranges: CLIP-NeRF reconstructs a single scene from posed images, while CtrlNeRF generates across scenes. Lower PSNR against a per-scene reconstruction method tells us very little.\n\nThe manuscript is unfinished. Citation placeholders ('[?]'), the FID reference is misattributed to Kanazawa et al., and typos throughout. No code or data is released, which makes the empirical claims hard to check, especially since the CARs dataset is synthetic and self-built.\n\nWho is this for? People working on conditional generative radiance fields and multi-scene NeRF compression. They will find the architecture idea useful, but should not cite the quantitative claims without re-running the experiments.\n\nMy recommendation: send it to peer review—the idea is credible and the failure modes are instructive—but require a matched-storage comparison against per-scene GRAF, error bars, a fairer baseline setup, and a cleaned-up manuscript with code/data. As is, conditional accept at best.\n\nBest.","headline":"A plausible GRAF extension for label-controlled multi-scene 3D generation whose central storage-quality claim outruns its evidence; worth a serious referee but needs a real baseline comparison.","tokens_in":16282,"tokens_out":3421,"would_cite":false,"duration_ms":29760,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CtrlNeRF shows that a single MLP with shared weights can represent multiple 3D scenes and control shape and color through label-embedded latent codes, keeping storage flat as scenes grow.","keywords":["Implicit Representation","Novel View Synthesis","Generative Adversarial Network","Neural Radiance Field","Controllable Image Generation","3D-Aware Images","Label Embedding","Multi-scene Representation"],"falsifier":"Render the density slot for one class while feeding the color embedding of another class into the same trained model; if the output geometry changes, the claimed independence of shape and appearance fails. Equivalently, if the FID-versus-class-count curve on the CARs dataset is flat rather than rising, the reported capacity degradation is not the real mechanism.","tokens_in":1376,"feed_emoji":"🚗","tokens_out":3122,"duration_ms":67044,"temperature":0.7,"pith_summary":"CtrlNeRF sets out to show that a single MLP with shared weights can represent multiple 3D scenes and generate images whose shape follows a class label and whose appearance follows a color label. The method takes the generative radiance field (GRAF) and changes the input and output of its MLP: shape and appearance latent codes are multiplied by learned label embeddings, and the network outputs an array of densities and an array of colors indexed by class and style. A VGG-based discriminator is added to classify class and color, and the adversarial loss is augmented with class and style losses. If the claims hold, multi-scene 3D generation no longer needs one network per scene, and storage and training cost stay flat as scenes grow. The paper also reports that image quality degrades as the number and complexity of scenes grows, which it attributes to entanglement in the shared MLP.","feed_headline":"One MLP now controls car shape and color in 3D","feed_subtitle":"CtrlNeRF adds class and color labels to a radiance-field GAN, keeping storage flat as scenes grow.","key_machinery":"The load-bearing object is the label-embedded conditional radiance field: a GRAF-style MLP whose input is $\\gamma(x)+z'_s$ and $\\gamma(d)+z'_a$, where $z'_s$ and $z'_a$ are the latent codes multiplied by learned embeddings of the class and color labels, and whose output is a density array and a color array instead of a single density and color. The output arrays let one shared network expose separate slots for classes and styles, and the VGG-based discriminator supplies the classification signal that pushes those slots apart. Volume rendering of the arrays yields the 3D-consistent images, and linear interpolation between array entries yields the unseen features.","core_discovery":"CtrlNeRF's central claim is that a generative radiance field can be made controllable across multiple scenes without adding networks. By multiplying the shape code $z_s$ and appearance code $z_a$ by label-embedding vectors, and by changing the MLP output into a density array and a color array, a single weight-shared MLP learns to separate scenes by class and style; volume rendering then produces 3D-consistent images, and feature interpolation creates colors and shapes never seen in training. The paper reports that the output modification is essential because training does not converge without it, and that both the label embedding and the VGG discriminator improve FID scores. In quantitative comparisons, the method is said to perform on par with CLIP-NeRF and to exceed GRAF and GIRAFFE in PSNR and SSIM, while keeping model storage and training time constant as the number of scenes increases.","pith_inferences":["The same label-multiplication mechanism could plausibly extend to other attributes the paper lists as future features, such as texture, material, and illumination, by adding more embedding vectors and output slots, at the cost of the same entanglement-induced degradation.","Because the output modification is what makes training converge, a useful stress test would be to compare multiplicative label embeddings against additive conditioning or feature-wise modulation within this architecture; the paper does not run that comparison.","The constant-storage advantage over CLIP-NeRF becomes more attractive as the number of scenes grows, but the declining image quality suggests an upper bound on scene count before per-scene models become preferable; that crossover point is not measured in the paper."],"forward_implications":["Storage and training cost for a generative 3D model no longer scale with the number of scenes; the same single MLP is reused for all classes and styles.","A user can explicitly choose object class and color at inference time, replacing the unpredictable latent-code sampling of GRAF.","Novel views can be rendered by changing camera pose, radius, or translation without retraining.","Linear interpolation between density or color array entries synthesizes shapes and colors absent from the training set.","The reported quality drop with more classes and styles marks a capacity limit of the shared MLP, so the approach is most effective when the number of scenes is moderate."],"supporting_citations":[{"why":"Supplies the NeRF representation and volume rendering that CtrlNeRF's generator is built on.","marker":"[3]"},{"why":"Supplies the GRAF generative radiance field backbone whose input and output are modified, including the disentangled shape and appearance codes.","marker":"[2]"},{"why":"Provides the compositional multi-MLP GIRAFFE baseline that CtrlNeRF compares against on memory cost.","marker":"[40]"},{"why":"Provides the CLIP-NeRF baseline that CtrlNeRF matches in image quality while beating on storage scaling.","marker":"[52]"},{"why":"Provides the GAN objective that is retained as the adversarial loss in the training algorithm.","marker":"[4]"},{"why":"Supplies the auxiliary-classifier conditioning idea that the added VGG discriminator extends.","marker":"[8]"}],"fun_headline_variants":["One MLP, many scenes: CtrlNeRF","Controllable 3D images from a single MLP","3D GAN: new views via feature interpolation","CtrlNeRF: shape and color control in 3D","Radiance fields with a twist: scene control"],"cache_read_input_tokens":18560,"weakest_assumption_plain":"The argument rests on the assumption that multiplying the shape and appearance latent codes by learned label embeddings gives one shared MLP enough capacity and separation to keep many distinct scenes from interfering with one another; the paper itself reports that FID worsens as classes and styles grow, so this capacity assumption is the fragile load-bearing premise.","fun_headline_variants_meta":{"raw":{"variants":["One MLP, many scenes: CtrlNeRF","Controllable 3D images from a single MLP","3D GAN: new views via feature interpolation","CtrlNeRF: shape and color control in 3D","Radiance fields with a twist: scene control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1377,"prompt_tokens":936,"completion_tokens":441,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":362}},"tokens_in":552,"tokens_out":441,"duration_ms":4730,"temperature":1.0,"reasoning_tokens":362,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:01:53.725860+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render the density slot for one class while feeding the color embedding of another class into the same trained model; if the output geometry changes, the claimed independence of shape and appearance fails. Equivalently, if the FID-versus-class-count curve on the CARs dataset is flat rather than rising, the reported capacity degradation is not the real mechanism.","supporting_citations":[{"cited_title":"GIRAFFE: Representing ScenesasCompositionalGenerativeNeuralFeatureFields,IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),","cited_arxiv_id":null,"evidence_quote":"Provides the compositional multi-MLP GIRAFFE baseline that CtrlNeRF compares against on memory cost."},{"cited_title":"GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis","cited_arxiv_id":"2007.02442","evidence_quote":"Supplies the GRAF generative radiance field backbone whose input and output are modified, including the disentangled shape and appearance codes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the GAN objective that is retained as the adversarial loss in the training algorithm."},{"cited_title":"Condi- tionalImageSynthesisWithAuxiliaryClassifierGANs,arXive-prints,","cited_arxiv_id":null,"evidence_quote":"Supplies the auxiliary-classifier conditioning idea that the added VGG discriminator extends."}],"review_version":1}