{"id":"b34f019e-24db-4a84-9506-5f5a63339e00","arxiv_id":"2412.16213","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"AdvIRL uses PPO to adjust Instant-NGP parameters so that CLIP misclassifies rendered 3D objects, with results on banana, truck, horse, and lighthouse scenes.","lead":"This paper introduces AdvIRL, a reinforcement-learning framework that perturbs 3D scene parameters in Neural Radiance Fields to make a classifier see wrong objects. It claims to be the first black-box attack on NeRF models, but the supporting evidence is limited to a few anecdotal scenes without baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No reward-to-transferability path is demonstrated: AdvIRL optimizes against CLIP scores alone, so the black-box threat claim over other vision models and 3D transforms is not established.","rationale":"The reader's weakest_assumption is that optimizing against CLIP outputs transfers to other black-box vision models and remains robust under rotations/scaling. This is exactly the same load-bearing gap I identify: the entire evaluation uses CLIP as both the optimization oracle and the test model, with no transfer or transformation ablation. The reader already notes the reward underspecification, absence of baselines, and conflict between abstract and results; adding to that, the reward definition has an internal wrinkle worth flagging: Eq. 1 uses both theta0 and theta1 but the text says theta1 = -1, so the 'True' term is added, and the dictionary averaging over images means the reward can be maximized by a few high-confidence target views rather than consistent misclassification; this reinforces the transferability concern but is secondary. The paper does have independent value in being fully public with code and a specific pipeline, and no formal math is claimed, but for an empirical threat-model paper the missing generalization evidence is decisive. A REJECT verdict is appropriate with moderate-to-high confidence: the claims as stated outrun the evidence, and the concrete test above would be the minimal experiment that could change that assessment. I agree with the reader's weakest_assumption rather than only partially, because the same single-model evaluation underlies both the 'black-box' and the 'robust under transformations' assertions.","tokens_in":7851,"tokens_out":1512,"duration_ms":13691,"concrete_test":"Run the released AdvIRL checkpoints for the banana, truck, horse, and lighthouse scenes against at least two unseen classifiers (e.g., ImageNet-pretrained ResNet-50 with fixed label set, and ViT-B/16), and against CLIP under held-out rotations and scales at test time. Report per-scene targeted-attack success rate (fraction of views classified as target with confidence above a fixed threshold) plus clean-image baseline; if success against unseen models or unseen transformations is near chance, the robustness and black-box generalization claims are not supported.","verdict_should_be":"REJECT","load_bearing_attack":"The central threat claim is that AdvIRL performs black-box attacks that remain robust under diverse 3D transformations. But in the reported experiments the only target model is CLIP ResNet-50, which is also the model used to compute the reward during RL training (Method, Eq. 1-2; Algorithm 1). There is no held-out classifier, no transfer test to another architecture or to a downstream detector, and no rotation/scale/viewpoint ablation with quantitative success rates. The abstract and conclusion claim robustness and transferability, but the Results section reports only unthresholded CLIP confidences on training-time renders. Without an evaluation on unseen transformations or unseen models, a CLIP-specific fit (including possibly reward hacking the averaged-confidence dictionary, since the reward averages per-class confidences across views rather than requiring consistent misclassification) cannot be distinguished from a genuine black-box attack. The 'black-box' label is also weakened because the reward uses the same CLIP model that defines success, so the attacker has output-level access to the target at training time; calling this a black-box attack on CLIP is fair, but calling it a practical threat to arbitrary vision systems is unsupported. This is internally consistent, not contradicted by other evidence in the paper, and the load-bearing step is the unshown generalization.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes AdvIRL, a reinforcement-learning framework that perturbs the parameters of an Instant-NGP NeRF model so that rendered images are misclassified by a CLIP vision model. The pipeline segments input images, renders multi-view images, uses PPO to modify NeRF parameters, and computes a reward from CLIP classification confidences with an MSE penalty. Experiments on Tanks and Temples scenes and a banana scene report misclassification counts and confidence ranges, and the authors claim that AdvIRL is the first black-box adversarial framework for NeRF and that the generated noise is robust to rotations, scaling, and viewpoint changes. The implementation is publicly available.","tokens_in":8110,"tokens_out":7607,"duration_ms":73543,"significance":"If the claims were supported, AdvIRL would be a useful contribution to 3D adversarial machine learning: a gradient-free, segmentation-localized attack on NeRF models with public code and multi-view evaluation. The paper also points to a plausible downstream use for adversarial training. However, the current evidence is not sufficient to establish these claims. The reward function is underspecified, the evaluation uses the same CLIP model that provides the training reward, no baselines or error bars are reported, and the robustness and transferability statements go beyond the experiments. The central threat model is therefore not validated in the present form.","major_comments":[{"comment":"The reward equation is not defined: 'Target' and 'True' are never introduced in the method text, and the dictionary D built in Algorithm 1 is not connected to them. With the stated values θ0 = 100 and θ1 = −1, the term −True·θ1 adds True to the reward, which would encourage correct predictions rather than penalize them, contradicting the adversarial objective. Since this reward is the optimization signal for the entire pipeline, the method as written is not reproducible.","section":"Method, Eq. (1)-(2) and Algorithm 1"},{"comment":"All reported attack successes are measured on CLIP ResNet-50, which is the same model used to compute the reward during PPO training. No held-out classifier, transfer test to another architecture, or evaluation on a downstream detector is reported. Consequently, the reported misclassification rates are consistent with overfitting to CLIP's outputs and do not support the abstract's claim of 'effective black-box attacks in real-world scenarios' or the conclusion's claim of 'transferable adversarial noise.' A transfer experiment to a different vision backbone, together with a random-perturbation baseline, is essential.","section":"Experimental Setup and Results"},{"comment":"The central claim of robustness under rotations and scaling is not supported by any quantitative ablation. The results show example images from different angles and distances, but there is no success rate as a function of rotation angle, scale factor, or viewpoint, and no comparison across transformation ranges. Without such measurements, the phrase 'robust under diverse 3D transformations, including rotations and scaling' in the abstract is an overclaim.","section":"Abstract and Results"},{"comment":"The experimental section reports counts and confidence ranges without baselines, error bars, or repeated runs. For example, the train scene reports misclassifications with confidences as low as 4%, and the horse scene reports confidences from 3% to 70%; without a random-noise or untrained-PPO baseline, it is unclear whether the RL training is responsible for these outcomes. The lack of repeated runs is especially problematic because PPO is stochastic and the reported numbers appear to come from single runs.","section":"Results"},{"comment":"The paper does not specify the candidate class label set used with CLIP for each scene. CLIP confidence values and 'misclassification' rates are only interpretable relative to the label set; if the set is small or chosen after the fact, the success rates are not meaningful. This omission also prevents replication of the experiments.","section":"Experimental Setup"}],"minor_comments":[{"comment":"The abstract states that a banana is labeled as a slug, but the Results section reports that the banana is misclassified as a flatworm; these statements should be made consistent.","section":"Abstract and Results, Banana Scene"},{"comment":"The paper cites Dosovitskiy et al. 2021 for CLIP ResNet-50, but that reference is for Vision Transformers; the CLIP model should be attributed to Radford et al. 2021, which is already in the reference list.","section":"Experimental Setup"},{"comment":"The caption says the environment computes a reward 'as defined in the accompanying figure,' but the reward equation appears in the text (Eq. 1-2), not in the figure; the caption should refer explicitly to the equation.","section":"Figure 1 caption"},{"comment":"The sentence introducing θ0 and θ1 says they 'balance the confidence of the true predictions of the target class and the true predictions,' which is grammatically ambiguous and should be rewritten to define the roles of Target and True precisely.","section":"Method"},{"comment":"The targeted horse attack used a 'modified reward system' in which the number of correctly classified target images was factored into the reward, but this modification is not defined; without this detail, the experiment cannot be reproduced.","section":"Results, Horse Scene"}],"recommendation":"reject","confidential_remarks":"The paper has an interesting core idea, but the experimental protocol is too weak to support the central claims: the reward is underspecified, the evaluation is circular with respect to the training signal, and there is no baseline or transfer test. The required fixes are not local; they would require a substantially new evaluation, including held-out classifiers and controlled transformation ablations. In my assessment this goes beyond a normal major revision and warrants rejection of the current submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before reading. The method combination — PPO acting on Instant-NGP parameters, CLIP confidence as reward, Detectron2 masks to localize perturbation — is genuinely new in the cited literature, and the code is public. That earns real credit. But the evaluation is too thin to support the paper's central claims of robustness and transferability.\n\nWhat's actually new is the machinery: treating the NeRF parameter vector as the action space, using multi-view CLIP scores as a reward, and restricting edits to segmented objects. That integration is reasonable, and testing on four different scenes rather than a single synthetic object is a strength. Publishing code also helps reproducibility.\n\nThe soft spots are load-bearing. First, Equation 1 leaves Target and True undefined; Algorithm 1 partially clarifies, but the exact mapping from class prediction dictionaries to scalar reward components is never specified. Second, and more seriously, the success metric is CLIP ResNet-50's confidence — the same model that provides the reward during RL training. So the reported misclassification rates mostly demonstrate fitting to the evaluation model, not a black-box threat to arbitrary vision systems. There is no held-out classifier, no transfer test to another architecture or a downstream detector, and no rotation/scale/viewpoint ablation with quantitative success rates. The abstract and conclusion call the noise 'robust and transferable,' but the results only show unthresholded confidences on training-time renders. The abstract also says the banana became a slug while the results say flatworm — a minor inconsistency, but symptomatic.\n\nThe 'first black-box NeRF attack' claim is contestable given Adv3D and other prior work, though if read narrowly as 'first RL-based black-box NeRF attack' it is more defensible. The stress-test note holds up: there is no demonstrated path from optimizing against CLIP scores to transferable attacks on other models or unseen 3D transformations.\n\nThis is a workshop-level contribution with a working demo, not yet a rigorous attack paper. The idea is plausible and the flaws are fixable: add an independent classifier, explicit success metrics with error bars, and at least one baseline. That would make it a solid paper. I would not desk-reject it; I would send it to reviewers with a clear request for additional experiments, or accept it as preliminary work with the transfer claims removed.","headline":"Plausible RL+NeRF attack idea with public code, but the evaluation does not support the transferability and robustness claims — the same CLIP model defines reward and success, and no independent classifier is tested.","tokens_in":8600,"tokens_out":2128,"would_cite":false,"duration_ms":21990,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 3D scene rendered by a NeRF can be made to mislead a vision classifier in a black-box setting by learning perturbations to the renderer's parameters with reinforcement learning.","keywords":["adversarial attacks","Neural Radiance Fields","Instant-NGP","reinforcement learning","black-box attack","CLIP","image segmentation","3D vision"],"falsifier":"Render the adversarial NeRF produced by AdvIRL and classify the same viewpoints with a different model, such as an ImageNet-trained ResNet or ViT, and with rotations and scales outside the training distribution; the central claim is false if misclassification rates return to baseline or if the true labels reappear.","tokens_in":7650,"feed_emoji":"🎯","tokens_out":8787,"duration_ms":75944,"temperature":0.7,"pith_summary":"Neural Radiance Fields (NeRFs) render 3D scenes into images from any viewpoint, and vision systems increasingly rely on such renderings. This paper tries to establish that a NeRF itself can be weaponized: by tweaking the renderer's parameters with reinforcement learning, a scene can be made to consistently fool a classifier, with no access to the classifier's gradients or architecture. The claimed payoff is practical, since attacks that survive rotations and scaling could threaten autonomous driving, surveillance, and VR training systems, where 2D adversarial methods fall short. The paper reports successful targeted misclassifications (a banana as a slug, a truck as a cannon) and untargeted ones, and argues this is the first black-box adversarial framework for NeRF.","feed_headline":"RL tweaks 3D scenes to fool vision models without gradients","feed_subtitle":"A reward loop over CLIP scores makes bananas read as slugs and trucks as cannons.","key_machinery":"The key machinery is a closed-loop parameter-space reward cycle. A PPO agent outputs an action vector $A$; the new Instant-NGP parameters are $P_{\\text{new}} = A + P_{\\text{old}}$, and the modified NeRF renders multi-view images $X_{\\text{adv}}$ that are scored by a CLIP classifier. The reward $R = \\mathrm{Target}\\cdot\\theta_0 - \\mathrm{True}\\cdot\\theta_1 - \\mathrm{MSE}(X, X_{\\text{adv}})\\cdot\\theta_2$ combines average confidence for the target class, a penalty for the true class, and a fidelity term that limits visible change; a Detectron2 segmentation mask restricts the MSE comparison to the object region. This machinery lets the attack run without gradients or model internals, using only label-confidence outputs as feedback.","core_discovery":"AdvIRL's central claim is that the 13 million parameters of an Instant-NGP NeRF model are a sufficient attack surface: an agent learns parameter perturbations that, when rendered from multiple viewpoints, push a target classifier's labels away from the true class. The attack is black-box because the agent only sees the classifier's predicted labels and confidences, never its weights or gradients. Concretely, a PPO agent proposes an action $A$, updates the current NeRF parameters to $P_{\\text{new}} = A + P_{\\text{old}}$, renders new images $X_{\\text{adv}}$, and receives reward $R = \\mathrm{Target}\\cdot\\theta_0 - \\mathrm{True}\\cdot\\theta_1 - \\mathrm{MSE}(X, X_{\\text{adv}})\\cdot\\theta_2$, which rewards high target-class confidence, penalizes the true class, and limits visible distortion. On scenes from Tanks and Temples plus a banana scene, targeted attacks produced high-confidence misclassifications such as banana to slug and truck to cannon; an untargeted train-scene attack misclassified all 20 rendered images. The paper further claims this is the first black-box adversarial framework for NeRF.","pith_inferences":["The paper does not test whether a target model different from CLIP is fooled; since CLIP both supplies the reward and is the target, the black-box claim should be read as query-only, not as proven transfer to arbitrary vision systems.","Because the reward is computed from averaged CLIP confidences over multiple views, an equivalent white-box version using CLIP gradients could show how close the RL search is to an optimal perturbation, and whether the 13-million-parameter action space is necessary.","The segmentation mask limits the MSE penalty to the object region, so a direct extension would be to bake the learned perturbation into the texture map and re-render under new lighting and pose, testing whether the attack survives a full NeRF retraining rather than just parameter tweaks."],"forward_implications":["Separating a scene into an untouched background and a modified object means an attack can be localized to a single object (for example, a banana or truck) without visibly corrupting the rest of the scene.","Because the agent only needs predicted labels and confidences, any vision model that exposes those outputs is a candidate victim, not just CLIP-backed systems.","NeRF parameters become a new attack vector: defending a perception stack means validating 3D scene content, not only filtering 2D images.","The same adversarial NeRF models could be injected into training pipelines as adversarial data to make classifiers more resistant to distorted 3D renderings.","Multi-view rendering during training ties the attack to camera geometry, so the resulting misclassifications persist over angles and distances rather than appearing in one viewpoint."],"supporting_citations":[{"why":"Defines NeRF, the 3D scene representation the attack perturbs.","marker":"Mildenhall et al. 2020"},{"why":"Provides Instant-NGP, whose 13-million parameters are the attack's action space.","marker":"Müller et al. 2022"},{"why":"Supplies the CLIP classifier whose label confidences drive the reward.","marker":"Radford et al. 2021"},{"why":"Detectron2's segmentation masks localize noise to the target object.","marker":"Wu et al. 2019"},{"why":"Supplies the Tanks and Temples scenes used for evaluation.","marker":"Knapitsch et al. 2017"},{"why":"Earlier white-box 3D adversarial work that the black-box claim is contrasted against.","marker":"Zeybey, Ergezer, and Nguyen 2024"}],"fun_headline_variants":["RL attacks 3D NeRFs: banana becomes slug, truck becomes cannon","First black-box attack on NeRF: RL flips labels across viewpoints","RL poisons 3D scenes: banana reads as slug, truck as cannon","Reinforcement learning crafts adversarial 3D scenes with swapped labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central bet is that optimizing against CLIP's outputs through this reward will fool other black-box vision models and survive rotations and scaling, and the paper reports no direct test of either.","fun_headline_variants_meta":{"raw":{"variants":["RL attacks 3D NeRFs: banana becomes slug, truck becomes cannon","First black-box attack on NeRF: RL flips labels across viewpoints","RL poisons 3D scenes: banana reads as slug, truck as cannon","Reinforcement learning crafts adversarial 3D scenes with swapped labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000934,"raw_usage":{"total_tokens":4045,"prompt_tokens":1043,"completion_tokens":3002,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":659,"completion_tokens_details":{"reasoning_tokens":2920}},"tokens_in":659,"tokens_out":3002,"duration_ms":18269,"temperature":1.0,"reasoning_tokens":2920,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:08:48.307693+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render the adversarial NeRF produced by AdvIRL and classify the same viewpoints with a different model, such as an ImageNet-trained ResNet or ViT, and with rotations and scales outside the training distribution; the central claim is false if misclassification rates return to baseline or if the true labels reappear.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Earlier white-box 3D adversarial work that the black-box claim is contrasted against."}],"review_version":1}