{"id":"2934314e-5c14-429b-bd64-f649b395a28c","arxiv_id":"2412.00067","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A scene graph-based framework for object-level machine unlearning that removes a specific object from a generative image model while keeping other objects and samples intact.","lead":"Researchers propose removing a single object, like one person in a photo, from an AI image generator's memory, instead of deleting the whole image or a whole category. They use scene graphs to locate the object, and report that masking the object's region during fine-tuning best preserves the rest of the image.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never tests its own success criterion: Eq. (1) requires the removed object to be absent, but A1 only measures ROI distance between original and unlearned models, so the claim that Obj-MK-PA/Obj-MK-NS 'remove' the object is unverified.","rationale":"The reader's identified weakest assumption is that scene graphs and bounding boxes exactly localize the object. That is a real deployment limitation, but it is not the most load-bearing issue for the paper's central claim: even with perfect scene graphs and boxes, the reported metrics would still not verify Eq. (1). A1 compares original and unlearned model ROIs; it measures change, not absence. The paper's own Section 7 admits that large distances can coexist with recognizable objects, which directly undercuts the interpretation of A1 as unlearning effectiveness. The reader's rationale does mention this ('rather than testing whether the object is actually absent'), but the stated weakest_assumption is the scene-graph noise issue, so I mark partial agreement. My proposed detector or classifier-based presence test would settle the ambiguity. If the object is still detected, the central claim collapses; if it is not, the paper's conclusion survives and the remaining concern is the noisy-scene-graph robustness, which the authors should also address. Since the reader already issued a conditional verdict, my read does not change that verdict, but it sharpens the condition: the required revision is not only robustness tests for scene-graph noise, but an actual absence or presence verification of the unlearned object.","tokens_in":27981,"tokens_out":5228,"duration_ms":52059,"concrete_test":"Run an object-detection or classification probe on the 64x64 generated images: train a ResNet or Faster R-CNN on Visual Genome crops to classify the nine human-related categories used in the paper, then measure the presence or classification rate of the requested object in the ROI and in the whole image for the original model, Obj-MK-PA, Obj-MK-NS, and a retrained-without-object baseline, aggregated over all unlearning requests with confidence intervals. If the unlearned models' detection rate is not near the retrained baseline and clearly below the original model, then Eq. (1) is not satisfied and the central claim fails; a merely large A1 distance is not sufficient.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equation (1) defines unlearning success as the removed object not appearing in the unlearned model's generation: for all o in Delta_O, o is not in f_theta-(x). The evaluation does not test this. A1 (Table 3) is defined as the difference between the ROI generated by the original model and the ROI generated by the unlearned model, measured with SSIM/LPIPS/MAE. A large A1 distance is treated as 'removal,' but a large distance can arise from blurring, recoloring, rescaling, or displacing the object while it remains present and identifiable; Section 7 explicitly concedes that metrics may show large distances while visual features remain highly recognizable. The indirect leakage tests are only qualitative (Figure 7), and no detector, classifier, or human evaluation checks Eq. (1). Therefore the headline result, that Obj-MK-PA and Obj-MK-NS remove the requested object, is not established by the reported evidence. The experiments show the object's representation changed, not that it was forgotten.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a scene graph-based framework for object-level machine unlearning in scene-graph-to-image (SG2I) generators. It formalizes object unlearning as the removal of a specific object instance from a generative model while preserving other objects in the same image and objects of the same category in other images. The framework uses scene graphs to localize the target object via its bounding box and introduces three unlearning methods: negative guidance fine-tuning (Obj-NG), mask-based fine-tuning with patch or noise masks (Obj-MK-PA, Obj-MK-NS), and influence-function-based partial model redaction (Obj-IF). The methods are evaluated against sample- and feature-level unlearning baselines on Visual Genome for image reconstruction and synthesis, using A1/A2/A3 distance metrics, latent-space t-SNE analyses, and indirect leakage attacks. The paper claims that Obj-MK-PA and Obj-MK-NS achieve satisfactory object removal while preserving model utility.","tokens_in":28196,"tokens_out":4483,"duration_ms":43854,"significance":"If the central claim were established, the paper would address a genuine gap: existing unlearning operates on whole samples or global features, whereas privacy requests such as removing one person from a group photo require object-instance-level forgetting. The scene-graph representation is a natural and transparent way to specify the target, and the systematic comparison across sample, feature, and object granularities is informative for the community. The paper also releases source code, which supports reproducibility. However, the significance is currently conditional because the evaluation does not directly test the paper's own success criterion: Eq. (1) requires that the removed object not appear in the generated output, while the reported A1 metrics only measure pixel-level distance between original and unlearned model outputs at the object ROI. This gap weakens the empirical support for the headline claim, so the contribution is not yet demonstrated at the level the paper asserts.","major_comments":[{"comment":"The stated success criterion in Eq. (1) is that the unlearned model does not generate the removed object o in f_theta-(x), but the A1 metrics in Table 3 measure pixel-wise distance (SSIM/LPIPS/MAE) between the ROI produced by the original model and the ROI produced by the unlearned model. A large A1 distance is consistent with the object still being present but blurred, recolored, rescaled, or displaced; Section 7 explicitly concedes that metrics may show large distances while visual features of the unlearned objects remain highly recognizable. Because no object detector, classifier, or human evaluation is used to check whether the object is actually absent, the headline claim that Obj-MK-PA and Obj-MK-NS remove the requested object is not established by the reported evidence. Please add a direct presence/absence test, e.g., a pre-trained object detector or a classifier evaluated on the generated ROI, and report detection rates for removed versus retained objects.","section":"§4 (Unlearning Verification Metrics) and §5 (Metrics)"},{"comment":"The A1 metric is the pixel-wise reconstruction discrepancy between the original and unlearned models' ROIs, and the negative guidance loss L_ng in Eq. (5) is exactly that ROI reconstruction loss, negated and added to the training objective. For Obj-NG, a large A1 distance is therefore partly by construction: the fine-tuning directly maximizes the discrepancy on the same pixels that A1 later measures. The mask-based methods similarly perturb the ROI during fine-tuning and are then evaluated on that same ROI. This circularity means A1 cannot serve as independent evidence of forgetting for these methods. An independent verification protocol (detection, classification, or human judgment) is needed to break the circularity and validate the central claim.","section":"§4.2, Eq. (5)"},{"comment":"The Obj-IF method assumes that zeroing the visual embedding z_v in the fused object embedding z_s = concat(z_v, z_b, z_o) in Eq. (9) removes the object's visual concept from the model, but this is an unvalidated modeling assumption; the influence function in Eq. (8) only estimates the parameter change induced by the loss computed under that zeroing. The ablation in Appendix A shows that redaction on different modules and different scalar values λ lead to very different outcomes, and the choice of λ (e.g., 1e-3 in Figure 11) is made by visually inspecting reconstructed images. This means the reported Obj-IF result is not a parameter-free benchmark, and the selection procedure is not specified as a reproducible rule. Please justify the z_v-nulling assumption with an experiment and define a principled, non-visual criterion for choosing λ.","section":"§4.2, Method 3 and Appendix A"},{"comment":"The framework relies on scene graph generation and bounding boxes to localize the requested object exactly; Section 3 assumes that every image invokes an established algorithm to generate a scene graph, and Section 4.2 states that each object is assigned a unique bounding box. However, no experiment tests robustness to imperfect scene graphs, missed objects, or loose bounding boxes. If the bounding box is imprecise, the mask in Obj-MK-PA/NS will either fail to cover the whole object or will perturb neighboring pixels, directly affecting both A1 and A2. Please include a robustness study with noisy or automatically generated scene graphs, for example by adding Gaussian noise to the bounding boxes or by using an off-the-shelf scene graph generator instead of ground-truth annotations.","section":"§5 (Pre-processing) and §4.2"},{"comment":"The indirect leakage test is qualitative: Figure 7 presents only example images, but the text concludes that none of the three query attacks can make the model leak information about the unlearned object. This conclusion is based on visual inspection without any quantitative measure, such as detector confidence on the unlearned object, retrieval similarity, or a classification score. Please report a quantitative metric for each attack type so the robustness claim can be evaluated and compared across methods.","section":"§6.4 (Indirect Leakage Test)"}],"minor_comments":[{"comment":"Definition 1 defines ∆D but then refers to 'the set ∆O', which is not introduced there; the notation should be made consistent so that the relationship between the unlearning request qunl, ∆O, and ∆D is clear.","section":"§3, Definition 1"},{"comment":"There are several typos, including 'influence influence function' and 'adaptation' written as 'adapation'; a careful proofread is needed throughout.","section":"§5 (Unlearning Baselines)"},{"comment":"The fine-tuning settings are under-specified: the text mentions 2000 epochs for pretraining and 200 epochs for unlearning fine-tuning, but does not report optimizer, learning rate, batch size, or the values of λ and noise σ used for the final results. Please provide these hyperparameters, ideally in an appendix.","section":"§5 (Image Generation Training Settings)"},{"comment":"The notation (fθg ◦ fθl ◦ fθd)(zs+) is confusing because the superscripts and composition order are not defined; please spell out which modules the notation refers to and how the partial redaction selects the GRL.","section":"Appendix A, Eq. (9)"}],"recommendation":"major_revision","confidential_remarks":"The core weakness is the mismatch between the paper's own success criterion, Eq. (1), and the A1 evaluation metric, which is essentially the same quantity that the negative-guidance objective maximizes. This is a load-bearing issue, but it is fixable by adding a direct object-presence evaluation (detector or classifier) and by reporting quantitative leakage-test results. The novelty claim should also be checked against recent concept-erasure work in diffusion models, though the scene-graph-instance granularity appears distinct from class-level erasure. I recommend major revision rather than rejection because the proposed framework and the mask-based methods are plausible and the experiments, if augmented with the missing verification, could support the claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi — quick take on 2412.00067.\n\nThe genuinely new thing is the granularity: defining unlearning at the level of a single object instance inside a sample, using scene graphs to localize the target, and adapting negative guidance, masking, and influence-function redaction to that setting. That is a real step beyond sample unlearning and class-level erasure. The comparison against sample and feature baselines is informative, and the masking methods (Obj-MK-PA, Obj-MK-NS) consistently show large ROI changes while leaving other objects relatively intact. The ablation on which module to redact for Obj-IF is also useful. The conceptual framework is sound.\n\nThe soft spot is the verification of the central claim. Equation (1) says the unlearned model should not generate the removed object. But none of A1, A2, A3 test that. A1 is a distance between the original and unlearned models' generated ROIs, using SSIM/LPIPS/MAE. A large A1 can mean the object is blurred, recolored, or displaced while still being identifiable. The paper itself concedes this in Section 7, saying metrics may show large distances while the visual features remain highly recognizable. The indirect leakage test is qualitative only, and there is no detector, classifier, or human evaluation that checks whether the object is actually absent. So the experiments establish that the object's representation changed, not that it was forgotten. That is the load-bearing gap.\n\nSecond, the reliance on scene graphs for exact localization is assumed rather than tested. The paper says it assumes an established algorithm will generate scene graphs, but no robustness experiment with noisy or loose bounding boxes is provided. If the box is off, the mask or nulling will hit the wrong pixels. That matters for a privacy claim.\n\nAlso, free parameters like lambda for Obj-IF and the noise sigma are tuned by visible inspection, and no error bars are reported. Minor relative to the main verification problem.\n\nNet: this is a serious paper with a real conceptual contribution, but the headline claim is not established by the reported evidence. It deserves a proper peer review, with the expectation of major revision. The revision should add object-detector-based leakage tests, quantitative membership-style probes, and robustness to imperfect scene graphs. I'd cite it for the object-unlearning formalization and scene-graph framing.","headline":"Object-level unlearning via scene graphs is a genuine conceptual step, but the paper verifies forgetting only as model-to-model distance, not actual object absence, so the headline claim needs a detector-based evaluation.","tokens_in":28758,"tokens_out":1649,"would_cite":true,"duration_ms":16541,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Scene-graph masks can delete a single object from a trained image generator.","keywords":["object unlearning","machine unlearning","scene graphs","image generation","privacy","influence functions","generative models","right to be forgotten"],"falsifier":"Take a held-out set of images with known object boxes, deliberately shift each requested box by a fraction of the image width (or run a noisy scene-graph generator), apply the patch- and noise-masking methods, and check whether the object still appears in outputs and whether retained objects degrade; if either happens, the localization assumption fails.","tokens_in":27730,"feed_emoji":"🎯","tokens_out":4282,"duration_ms":37750,"temperature":0.7,"pith_summary":"This paper tries to establish that a user's request to erase one object from a trained image model—for example, Alice's face in a group photo—can be honored without deleting the whole photo or erasing the entire object category. It proposes object unlearning: a fine-grained form of machine unlearning in which the thing removed is a single object instance inside a sample. The route is to use scene graphs as the interface: a scene graph names each object, its attributes, and its relations, and gives it a bounding box, so an unlearning request becomes a precise region to mask or guide. The paper claims that masking-based fine-tuning on that region (patching it to zero or adding noise) removes the target object from the generator's output while preserving other objects in the same image and same-category objects in other images. A sympathetic reading takes the contribution as evidence that granular, instance-level unlearning is achievable in practice, not just retraining at the sample or feature level.","feed_headline":"Scene-graph mask deletes a single object from a trained image generator","feed_subtitle":"A masking fine-tune removes one requested object while keeping the rest of the scene and other samples intact.","key_machinery":"The load-bearing object is the scene graph, a structured representation of an image as nodes (objects with category and attributes) and edges (relationships such as \"has\" or \"standing on\"), with each object node carrying a unique bounding box. During unlearning, that box defines a region of interest: the mask-based method either zeroes the region (patch masking) or injects Gaussian noise into it (noise masking), then fine-tunes the scene-graph-to-image generator to reconstruct the masked image, forcing the model to drop the object's visual features while keeping the unmasked scene. A secondary mechanism is influence-function-based partial model redaction, which estimates a closed-form parameter change and applies it only to the graph representation learner, though the paper's own experiments show this redaction route is less effective than masking.","core_discovery":"On the paper's own terms, the central discovery is that object-level unlearning can work by localizing the target object through the scene graph and then fine-tuning the generator against a masked version of that region. The paper formalizes object unlearning as the removal of a set of objects $\\Delta O$, with two verification objectives: the unlearned model should not reproduce any $o \\in \\Delta O$ (effectiveness), and should preserve every retained object in the same sample and in other samples (utility). Across image reconstruction and image synthesis on an annotated image dataset, the two mask-based methods—patch masking and noise masking on the object's bounding box—achieve unlearning effectiveness comparable to sample unlearning but with markedly less collateral damage, and they outperform negative-guidance fine-tuning and influence-function redaction on both utility and stability. The paper also shows that three label-tampering query attacks cannot coax the unlearned object back out of the model.","pith_inferences":["The paper's localization assumption is untested: an imperfect scene-graph generator that misses an object or draws a loose box would likely make the mask hit the wrong pixels, so the framework's practical robustness depends on external scene-graph quality.","The same masking scheme could transfer to text or multimodal data, since a scene graph is agnostic to output modality; redaction of a named entity in a caption would be the analogous operation.","Distance-based verification (SSIM, LPIPS, MAE) may understate privacy leakage: an object can be statistically different yet still visually recognizable, so a recognition-based audit could be a stricter test.","Stronger adversaries than label-tampering queries, such as exploiting the difference between original and unlearned models, may be able to recover the removed object; the paper names this as a future concern."],"forward_implications":["A service provider can act on a request like \"remove Tom from this photo\" by fine-tuning against a masked copy of Tom's bounding box, rather than deleting every photo that contains Tom.","The unlearned generator should no longer output the object in either reconstruction (when the original image is still supplied) or synthesis (when only a scene graph is supplied).","Objects of the same category in other samples, such as two other men in a group photo, are largely retained, which neither sample unlearning nor feature unlearning achieves.","The three query-based attacks tested—replacing the object's label with a broader label, modifying a neighbor node, or modifying an isolated node—do not resurrect the unlearned object.","Fine-tuning-based unlearning runs in about one-thirtieth of the retraining time, while influence-function redaction runs faster but, in the paper's experiments, fails to balance effectiveness and utility."],"supporting_citations":[{"why":"Supplies the scene-graph-to-image generation paradigm that the framework wraps its unlearning around.","marker":"[13]"},{"why":"Provides the scene-graph-to-image generator backbone, including the graph representation learner and decoder, that the experiments fine-tune and redact.","marker":"[41]"},{"why":"Gives the influence-function closed-form parameter change used for node-level graph unlearning in the Obj-IF method.","marker":"[19]"},{"why":"Provides the annotated image and scene graph dataset used to train and evaluate the unlearned generators.","marker":"[50]"},{"why":"Basis of the feature-unlearning baseline for pre-trained GANs and VAEs that the paper compares against.","marker":"[12]"},{"why":"Basis of the concept-erasure baseline for diffusion models that the paper contrasts with instance-level object unlearning.","marker":"[11]"},{"why":"Supplies the LPIPS perceptual metric used in the A1/A2/A3 verification scores.","marker":"[52]"}],"fun_headline_variants":["Scene graph unlearning erases a single object per image","Mask fine-tuning deletes one object, preserves the scene","Object-level forgetting: remove a person, keep the background","Granular unlearning: forget an object, not the whole sample"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the scene graph's bounding box locates the requested object exactly; if the box is noisy or misses the object, the mask or noise hits the wrong pixels and unlearning either fails or damages neighbors.","fun_headline_variants_meta":{"raw":{"variants":["Scene graph unlearning erases a single object per image","Mask fine-tuning deletes one object, preserves the scene","Object-level forgetting: remove a person, keep the background","Granular unlearning: forget an object, not the whole sample"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000236,"raw_usage":{"total_tokens":1532,"prompt_tokens":1002,"completion_tokens":530,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":461}},"tokens_in":618,"tokens_out":530,"duration_ms":5573,"temperature":1.0,"reasoning_tokens":461,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:39:18.103344+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of images with known object boxes, deliberately shift each requested box by a fraction of the image width (or run a noisy scene-graph generator), apply the patch- and noise-masking methods, and check whether the object still appears in outputs and whether retained objects degrade; if either happens, the localization assumption fails.","supporting_citations":[{"cited_title":"Image generation from scene graphs,","cited_arxiv_id":null,"evidence_quote":"Supplies the scene-graph-to-image generation paradigm that the framework wraps its unlearning around."},{"cited_title":"Semantic image manipulation using scene graphs,","cited_arxiv_id":null,"evidence_quote":"Provides the scene-graph-to-image generator backbone, including the graph representation learner and decoder, that the experiments fine-tune and redact."},{"cited_title":"The unreasonable effectiveness of deep features as a perceptual metric,","cited_arxiv_id":null,"evidence_quote":"Supplies the LPIPS perceptual metric used in the A1/A2/A3 verification scores."}],"review_version":1}