{"id":"d77b6132-fd35-4efc-a210-06fa881076d8","arxiv_id":"2505.23386","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"VModA combines prompt engineering, region zooming, and LLM-based answer aggregation to improve zero-shot NSFW image moderation across multiple categories.","lead":"VModA is a new framework that uses vision-language models and large language models to detect NSFW images without training. It reports large accuracy gains over commercial and academic moderation tools, especially on subtle and context-dependent content.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'up to 54.3% accuracy improvement' is measured on all-positive datasets, so it reflects recall only; no false-positive rate is assessed for blood or school-bully moderation.","rationale":"The reader's weakest assumption correctly identifies that the one-class evaluation invalidates the headline 'accuracy improvement' as a statement about real-world moderation. This is the most load-bearing concern because the abstract and introduction lead with the 54.3% figure, and the paper's stated contributions include 'extensive NSFW evaluation' with state-of-the-art accuracy. Without measuring false positives on these categories, the claim that VMODA significantly outperforms existing methods is not established for blood and school-bully content. The concern is concrete and testable: adding neutrals to the evaluation would settle it. The paper does present meaningful results on balanced datasets (pornographic and meme datasets), so the framework may still be effective; therefore, a conditional verdict that requires the metric issue to be resolved is appropriate rather than outright rejection. The reader's verdict of CONDITIONAL already captures this, so no change to the verdict is needed here. The label-noise analysis and real-world evaluation provide some independent support, but they do not mitigate the one-class accuracy problem in the core comparison.","tokens_in":26131,"tokens_out":3659,"duration_ms":36139,"concrete_test":"Construct a balanced evaluation for the blood and school-bully datasets by adding an equal number of neutral images (e.g., randomly sampled non-NSFW images from the same source or a general web corpus). Re-run VMODA and the commercial baselines on this mixed set and compute accuracy, precision, recall, and F1. If VMODA's precision drops substantially or its balanced accuracy is no longer higher than baselines, the reported 54.3% improvement would be shown to be an artifact of the one-class evaluation setup.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim rests on Table I, where the blood and school-bully datasets contain only NSFW images. Section V-A4 says these datasets 'lack positive samples' (likely a typo for 'lack negative samples') and therefore report only accuracy. With no negatives, accuracy equals recall (TP/total positives), so a trivial all-positive classifier would score 100%. The 54.3-percentage-point gap comes from the school-bully column (Sightengine 0.407 vs. VMODA-GPT4o 0.950): this is a recall difference, not an accuracy difference in any realistic moderation setting. Because no neutral images are included, false positives cannot even be measured, and the real-world usefulness of VMODA on these categories is unquantified. The same issue affects the cross-prompt experiment in Table IV, which uses only violent protest images (score ≥ 0.6) and all school-bully images, both positive-only sets. The paper's central claim of accuracy improvement across NSFW types is therefore not supported by the evidence for these categories; it measures sensitivity alone.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VMODA, a zero-shot framework for NSFW image moderation that combines a base VLM with an LLM aggregator. The pipeline consists of preprocessing (super-resolution, adaptive ROI zoom-in, text extraction), standard image moderation with conditional prompting and iterative output aggregation, and advanced semantic analysis using a CoT-based hierarchical description strategy. The authors evaluate VMODA on six datasets covering pornography, blood/gore, protest violence, school bullying, and hateful/harmful memes, comparing against commercial APIs (Sightengine, AWS Rekognition, Azure, Imagga) and academic baselines (Pro-Cap, ExplainHM). They report up to a 54.3% accuracy improvement over baselines, and also contribute a label-noise discovery and correction effort on public benchmark datasets, plus a real-world meme detection case study.","tokens_in":26496,"tokens_out":3923,"duration_ms":38819,"significance":"If its evaluation claims were fully supported, VMODA would be a valuable training-free alternative to category-specific NSFW detectors, and the paper's design choices (e.g., iterative output aggregation to handle VLM refusals, CoT-based semantics extraction) are generally reasonable and potentially transferable. The label-correction effort submitted to dataset maintainers is a concrete community contribution. However, the central headline improvement rests on a one-class evaluation that conflates accuracy with recall. The two-class experiments (porn and meme datasets) are more convincing and show consistent gains, so the framework's core idea has merit, but the evidence as presented does not support the abstract's broad claim of an accuracy improvement across all NSFW types.","major_comments":[{"comment":"The blood and school-bully datasets contain no negative samples: Section V-A1 states that the blood set uses all 1,305 images depicting bleeding scenes and the school-bully set uses 300 frames that 'are all considered violations.' Consequently, the 'accuracy' reported in Table I for these columns is identical to recall (TP / total positives). The headline '54.3% accuracy improvement' is computed from the school-bully column (Sightengine 0.407 vs. Ours GPT4o 0.950), so it is a recall gain, not an accuracy gain. A trivial classifier that labels every image as NSFW would score 100% under this metric. The abstract's claim of 'up to a 54.3% accuracy improvement across NSFW types' is therefore unsupported for these categories. The manuscript should either rename this metric to recall/sensitivity with an explicit caveat, or add a negative-image set for these categories so that true accuracy and false-positive rates can be measured.","section":"V-A4, Table I, Abstract"},{"comment":"The cross-prompt experiment also evaluates on positive-only sets: the protest subset with violence score ≥ 0.6 (all violent) and the entire school-bully set (all violations). The 'Acc' columns in Table IV are thus recall values, not accuracy. The conclusion in the text that this experiment 'quantifies its specificity and generality' is not justified because specificity (true-negative rate) cannot be estimated without negative examples. This undermines the adaptability claim for scenario-based rules. Please re-frame the reported numbers as detection rates on positive-only samples, or include a balanced set to support claims about specificity and accuracy.","section":"V-C, Table IV"},{"comment":"The sentence 'For datasets that lack positive samples, specifically the blood and school bully datasets, we report only accuracy' appears to contain a typo: the datasets lack negative samples, not positive samples. This misstatement is not merely cosmetic; it reflects the metric issue: when no negatives are present, 'accuracy' is a misnomer. The paper's broader comparative claims in Section V-B ('VMODA consistently achieves strong performance in all NSFW categories') should be revised to separate one-class sensitivity results from two-class accuracy results, and the 54.3% figure should be attributed to sensitivity improvement only.","section":"V-A4, Section V-B"}],"minor_comments":[{"comment":"Typo: 'experiement setting' should be 'experiment setting'.","section":"V-A3"},{"comment":"The system prompt begins 'Your are a professional image reviewer' and should read 'You are a professional image reviewer.'","section":"IV-B1, prompt PS"},{"comment":"The caption states that the bars illustrate 'the proportion detected by each module when identifying negative samples.' Based on the text in Section VI-B, this should read 'positive samples' (i.e., NSFW images), since the figure reports the contribution of each step to detecting violations.","section":"Figure 4 caption"},{"comment":"The robustness analysis reports standard deviations of accuracy across the four VLMs but does not provide any confidence intervals or significance tests; given the stochastic sampling (n=10) and aggregation, a short discussion of variance or a table of per-backbone repeats would improve reliability.","section":"V-D"},{"comment":"Reference [31] is incomplete in the bibliography: the author list is truncated to 'Y. M. X. S. Y. Qu, N. Y. M. Backes, and S. Z. Y. Zhang' instead of listing the full author names.","section":"References"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look, with one important caveat: the headline number is not what it appears to be. The 54.3% improvement comes from the school-bully column in Table I, where every image is a violation. The paper says these datasets \"lack positive samples\" when it means they lack negative samples, so the reported accuracy is just recall. A trivial all-positive classifier would score 100% on that metric. The same problem leaks into the cross-prompt experiment in Table IV, which uses only positive images. That is the main soft spot, and it is real.\n\nNow the good parts. VMODA is a sensible zero-shot pipeline: super-resolution, RAM+GroundingDINO zoom-in, CoT-hierarchical descriptions, and LLM-based aggregation to handle VLM refusals. None of the pieces are new, but the integration for NSFW moderation is, and the two-class evaluations on porn, memes, and protest-violence show consistent gains over commercial tools and trained baselines. The label-noise analysis is the strongest contribution: identifying mislabeled samples in five benchmarks, manually re-annotating them, and getting two maintainers to confirm corrections is concrete and useful. They also showed that their framework finds real harmful memes that existing detectors miss.\n\nOther weak spots are moderate. No error bars, and the code/data are only promised, not shipped. The per-dataset module choices (e.g., skipping ROI zoom-in for memes) are a mild form of tuning on the test sets. The GPT-4o contamination risk is plausible but not damning, since the commercial baselines face the same issue. I do not think these sink the paper.\n\nThe core two-class results hold up, and the label-noise work stands on its own. This paper deserves a serious referee, but the positive-only datasets need to be either re-analyzed as recall/sensitivity or supplemented with negative samples so false positives can be measured. If that happens, the headline claim can be corrected and the real contribution—an adaptable, zero-shot framework plus dataset auditing—comes through clearly.","headline":"Useful zero-shot moderation framework and a genuine label-noise contribution, but the headline 54.3% gain is recall on all-positive datasets, not accuracy.","tokens_in":26891,"tokens_out":1879,"would_cite":false,"duration_ms":20498,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VMODA claims a training-free pipeline of ROI zoom-in, chain-of-thought description, and aggregated sampling lets stock vision-language models beat commercial NSFW detectors by up to 54.3 accuracy points across five content categories.","keywords":["NSFW image moderation","vision-language models","chain-of-thought prompting","adaptive region-of-interest","zero-shot adaptation","hateful meme detection","output aggregation","label noise"],"falsifier":"Take the school-bullying test set and add an equal number of benign classroom and school-corridor images, then run VMODA and the best commercial baseline on the balanced set and compute accuracy, precision, and recall. If a 'flag everything' rule also matches VMODA's recall while VMODA's precision on the benign half falls well below its reported 0.950 accuracy, the headline 54.3-point gain is an artifact of the one-class metric rather than evidence of superior moderation.","tokens_in":2057,"feed_emoji":"🛡️","tokens_out":2378,"duration_ms":109774,"temperature":0.7,"pith_summary":"This paper argues that NSFW-image moderation can be made adaptive and detail-aware without retraining, by wrapping a stock vision-language model (VLM) in a pipeline that stages the review from easy to hard. The framework, VMODA, first improves and dissects the image (super-resolution, text extraction, and a zoom-in on semantically selected regions), then runs a standard violation check under a prompt that names the image type and the applicable rule, and finally, if no violation is found, escalates to a chain-of-thought semantic pass that describes objects, states, and rhetorical devices before a large language model makes the final call. The paper claims this design beats commercial detectors and trained meme classifiers across six benchmarks, by up to 54.3 percentage points of accuracy on a school-bullying set, and that it adapts to different rule sets, scenarios, and VLM backbones with little variance. A sympathetic reader would care because the method is training-free, so it can track shifting platform regulations without data collection or fine-tuning, and because the authors show the framework doubles as a benchmark audit tool, flagging hundreds of mislabeled samples in public datasets.","feed_headline":"Flags subtle NSFW images that commercial moderation tools miss","feed_subtitle":"Rule-adaptive pipeline combines ROI zoom-in with chain-of-thought semantics to cut detection misses across five NSFW categories.","key_machinery":"The load-bearing object is the VMODA pipeline itself, with two mechanisms carrying most of the weight. The first is the adaptive region-of-interest (ROI) zoom-in: an open-set tagging model labels image regions, a grounding detector frames the relevant objects, and the VLM is shown the full image together with a marked, enlarged crop, so its reading of the detail is conditioned on the whole scene (formalized as $\\hat{R}_{p_i} = \\arg\\max( P(R_{p_i} \\mid T^V, I_{p_i}) + P(R_{p_i} \\mid T^V, P(I \\mid I_{p_i})) )$). The second is the chain-of-thought (CoT) hierarchical description, which forces three escalating levels of description (objects, states, rhetorical devices), each consuming the previous level and each separately screened for violations. Around both sits an iterative output-aggregation strategy: at temperature 1 the VLM is sampled ten times per question, and an LLM merges the answers while discarding refusals, so that low-probability but informative responses survive. The whole review is progressive (text, whole image, zoomed regions, object reading, state reading, rhetorical reading, final LLM review), and any stage that flags a violation ends the process.","core_discovery":"The central claim is that a training-free orchestration of a VLM and a large language model can outperform both commercial NSFW APIs and task-specific trained classifiers. VMODA attributes the gain to three mechanisms: conditional prompting that injects the moderation rule and the image type as prior knowledge; an adaptive region-of-interest step that anchors zoomed-in segments to the full image with bounding overlays, so the VLM sees detail without losing context; and a chain-of-thought hierarchy that forces the model to move from object identification to state description to rhetorical-device analysis, with an LLM aggregating ten sampled outputs at each stage to suppress refusals and sanitized answers. On the reported numbers, VMODA reaches 0.985 accuracy on a pornographic-versus-neutral binary set, 0.950 on school-bullying frames versus 0.407 for the best commercial tool, and detection ratios above 0.8 on high-severity protest violence where commercial tools stay near 0.5. The paper further claims the framework exposes label noise in all five public benchmarks, and that re-annotation by human raters confirmed most of the disputed labels.","pith_inferences":["The ROI zoom-in with full-image anchoring is a general remedy for the wide-receptive-field blindness of VLM encoders, and could transfer to fine-grained visual question answering, document layout understanding, or medical image reading where small regions carry the decision.","The iterative aggregation trick (sampling at high temperature and having an LLM merge answers while discarding refusals) is a jailbreak-adjacent technique: the same mechanism that recovers useful moderation signals could lower the cost of extracting disallowed content from safety-aligned models, a dual-use risk the paper does not address.","If the label-noise findings replicate on other benchmarks, the practical takeaway is that nontrivial fractions of public NSFW datasets inherit noise from keyword-based collection or shifting standards, which strengthens the case for training-free moderation over retraining on those sets.","The demonstrated link between rule granularity and the precision-recall trade-off suggests a testable deployment recipe the paper leaves implicit: run several prompts at different granularities in parallel and pick by the platform's own policy threshold."],"forward_implications":["Platforms could update moderation rules by editing a prompt rather than collecting and re-labeling training data, since VMODA is training-free and rule-adaptive.","Fine-grained rules such as the school-bullying definition buy precision on their target scene but sacrifice recall on broader violence; the paper's cross-prompt experiment quantifies that trade-off, so deployment can choose between high-precision and high-coverage prompts.","The framework works across four VLM backbones with small accuracy variance (standard deviation at or below 0.051 on five of six datasets), so the reported gains are not tied to one model's prompting quirks.","Benchmark label noise is large enough to matter: the paper reports 62 of 82 disputed porn test images were normal, 30 of 43 disputed blood images were not bloody scenes, and hundreds of meme labels were inconsistent, with corrections submitted to maintainers and two datasets confirming updates.","Every stage of the progressive pipeline recovers content the earlier stages miss; the paper reports that advanced semantic analysis alone contributes between 27.8% and 88.8% of meme detections, so each design component earns its place."],"supporting_citations":[{"why":"Supplies the pornographic and blood datasets used for the headline accuracy comparisons.","marker":"[32]"},{"why":"Supplies the Facebook Hateful Memes subset used for hateful meme moderation evaluation.","marker":"[10]"},{"why":"Supplies the Harmeme COVID-19 corpus used as the second meme benchmark.","marker":"[65]"},{"why":"Supplies the violent protest dataset used to test scenario-based rule adaptation.","marker":"[66]"},{"why":"Sightengine, the strongest commercial baseline whose 0.407 school-bullying score anchors the 54.3-point improvement claim.","marker":"[14]"},{"why":"Pro-Cap, a training-based meme moderation baseline that VMODA must beat or match without training.","marker":"[12]"},{"why":"ExplainHM, the other training-based meme moderation baseline compared on both meme datasets.","marker":"[35]"},{"why":"Provides the vanilla-VLM moderation protocol and prior evidence that raw VLMs fail NSFW moderation, motivating the framework.","marker":"[26]"},{"why":"GroundingDino, the detector that frames objects selected for the adaptive ROI zoom-in.","marker":"[62]"},{"why":"RAM, the open-set tagging model that labels image regions and seeds the ROI selection step.","marker":"[61]"}],"fun_headline_variants":["Flags subtle NSFW images that commercial tools miss","Adaptive VLM pipeline beats commercial NSFW filters","Chain-of-thought + zoom-in catches tricky NSFW images","Outperforms existing NSFW moderators by up to 54.3%"],"cache_read_input_tokens":29056,"weakest_assumption_plain":"The load-bearing premise is that the reported 'accuracy' on the blood and school-bullying datasets measures real moderation skill; those datasets contain only NSFW images, so the score is a recall rate, and a moderation system that flagged every image as a violation would score 100 percent on them.","fun_headline_variants_meta":{"raw":{"variants":["Flags subtle NSFW images that commercial tools miss","Adaptive VLM pipeline beats commercial NSFW filters","Chain-of-thought + zoom-in catches tricky NSFW images","Outperforms existing NSFW moderators by up to 54.3%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1328,"prompt_tokens":1009,"completion_tokens":319,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":249}},"tokens_in":625,"tokens_out":319,"duration_ms":4160,"temperature":1.0,"reasoning_tokens":249,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:46:22.275370+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the school-bullying test set and add an equal number of benign classroom and school-corridor images, then run VMODA and the best commercial baseline on the balanced set and compute accuracy, precision, and recall. If a 'flag everything' rule also matches VMODA's recall while VMODA's precision on the benign half falls well below its reported 0.950 accuracy, the headline 54.3-point gain is an artifact of the one-class metric rather than evidence of superior moderation.","supporting_citations":[{"cited_title":"Tovilag: Your visual-language generative model is also an evildoer,","cited_arxiv_id":null,"evidence_quote":"Supplies the pornographic and blood datasets used for the headline accuracy comparisons."},{"cited_title":"Detecting harmful memes and their targets,","cited_arxiv_id":null,"evidence_quote":"Supplies the Harmeme COVID-19 corpus used as the second meme benchmark."},{"cited_title":"Protest activity detec- tion and perceived violence estimation from social media images,","cited_arxiv_id":null,"evidence_quote":"Supplies the violent protest dataset used to test scenario-based rule adaptation."},{"cited_title":"Image moderation,","cited_arxiv_id":null,"evidence_quote":"Sightengine, the strongest commercial baseline whose 0.407 school-bullying score anchors the 54.3-point improvement claim."},{"cited_title":"Pro-cap: Leveraging a frozen vision-language model for hateful meme detection,","cited_arxiv_id":null,"evidence_quote":"Pro-Cap, a training-based meme moderation baseline that VMODA must beat or match without training."},{"cited_title":"Towards explainable harmful meme detection through multimodal debate between large language models,","cited_arxiv_id":null,"evidence_quote":"ExplainHM, the other training-based meme moderation baseline compared on both meme datasets."},{"cited_title":"Open-set image tagging with multi-grained text supervision,","cited_arxiv_id":null,"evidence_quote":"RAM, the open-set tagging model that labels image regions and seeds the ROI selection step."}],"review_version":1}