{"id":"bbfa8596-85cc-4902-94d0-5550d51d7e1d","arxiv_id":"2505.11926","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Aligning video multimodal models with a 350K-pair synthetic preference dataset, built via textual video descriptions, lifts GPT-4o-judged safety rates on a new video benchmark by up to 42 percentage points.","lead":"This paper introduces SafeVid, a framework, dataset, and benchmark for improving the safety of video large multimodal models by turning videos into text descriptions and using them to create safety preference data. A smart generalist might read it because it offers a concrete recipe and public data for reducing harmful outputs from video AI models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GPT-4o-as-judge loop may explain much of the reported safety gains; human evaluation is needed to validate the 42.39-point improvement.","rationale":"The reader's weakest assumption identifies the GPT-4o-as-judge loop, and this is indeed the most load-bearing concern. The central claim is empirical: DPO fine-tuning on SafeVid-350K substantially improves video LLM safety. The entire measurement of 'safety' depends on GPT-4o judgments, while the training signal comes from GPT-4-written preferred responses. Because the two GPT models are closely related and may share stylistic preferences, the large measured gains could reflect the aligned model learning to produce responses that GPT-4o labels as safe, rather than responses that humans consider safe. The paper provides no human validation, no alternative judge, no confidence intervals, and no released benchmark, so the concern cannot be dismissed from the manuscript alone. However, this is not a reason to reject the paper outright: the dataset is public, the approach is clearly described, and the OOD generalization results on other benchmarks provide some independent support, even though those benchmarks are also largely evaluated with automated metrics. The appropriate disposition is the reader's CONDITIONAL verdict: accept only if the judge-bias concern is resolved with human evaluation or an independent judge. Since the reader already reached CONDITIONAL, my read does not change the verdict, so I mark it UNCHANGED.","tokens_in":15648,"tokens_out":2480,"duration_ms":27946,"concrete_test":"Stratified human validation: sample 200 responses per condition (LLaVA-NeXT-Video before and after SafeVid-350K) from SafeVidBench-Base and SafeVidBench-Challenge, have three blind annotators rate safety using the paper's seven-category rubric, compute the human-assessed safety delta and GPT-4o-human Cohen's kappa. If the human delta is far below the reported 42.39/39.71 points or kappa is below about 0.6, the headline claim should be downgraded to 'improved safety as judged by GPT-4o'.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim is that DPO on SafeVid-350K raises safety rates by tens of percentage points (e.g., 42.39% on SafeVidBench-Base, Table 2). But both sides of the loop are GPT-family models: the chosen responses used for DPO are generated by GPT-4 (Sec 3.1, Preference Pair Synthesis), and all safety/helpfulness scores are assigned by GPT-4o (Sec 4.1, Evaluation Metrics). DPO directly increases the likelihood of GPT-4-written chosen responses (Eq. 1), so the aligned model is being optimized to imitate GPT-4's refusal style and phrasing. If GPT-4o's safety judgments are correlated with stylistic similarity to GPT-4 responses, then the reported safety-rate deltas would partly reflect alignment-to-the-judge rather than alignment to safety. The paper reports no human agreement study, no inter-annotator reliability, no alternate judge, and no error bars or confidence intervals. Consequently, the magnitude of the claimed safety improvement is not yet established; what is established is that aligned models satisfy GPT-4o's safety rubric more often. Minor inconsistencies (39.17% in the abstract vs. 39.71% in Sec 4.2; VidSafeBench vs. SafeVidBench) further signal that the evaluation pipeline needs tightening, but the judge-loop issue is the load-bearing concern.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SafeVid, a framework for safety alignment of video large multimodal models (VLMMs). The framework has three components: (1) SafeVid-350K, a large-scale preference dataset of 350,000 video-specific query–response pairs, constructed using textual video descriptions as an interpretive bridge and GPT-4-generated chosen responses; (2) Direct Preference Optimization (DPO) fine-tuning of VLMMs on this dataset; and (3) SafeVidBench, a new video-safety benchmark with Base and Challenge subsets. The main empirical claim is that DPO on SafeVid-350K substantially improves safety rates on SafeVidBench, e.g., a 42.39 percentage-point increase for LLaVA-NeXT-Video on SafeVidBench-Base, while producing only a minimal alignment tax on general video understanding (MMBench-Video). The paper also reports out-of-distribution evaluations on image-text and text-only safety benchmarks.","tokens_in":15874,"tokens_out":5895,"duration_ms":54288,"significance":"If the central claim is valid, the paper makes a useful contribution: it releases a large public preference dataset, a benchmark for video safety, and evidence that safety reasoning can be transferred from text to video through DPO with detailed textual descriptions. The breadth of evaluated models and the inclusion of OOD benchmarks are strengths. However, the validity of the quantitative claim currently rests on the assumption that GPT-4o adjudication provides an unbiased measure of safety; because the preferred training responses are also GPT-4-generated, the reported gains may partly reflect imitation of GPT-4 response style rather than improved safety. No human evaluation or alternative judge is provided. The core methodology is sound and the concerns are addressable with additional experiments, but the current evidence does not yet establish the magnitude of the safety improvement.","major_comments":[{"comment":"The evaluation loop is circular: all safety and helpfulness scores in Tables 2 and 3 are adjudicated by GPT-4o (Section 4.1), while the chosen training responses are synthesized by GPT-4 (Section 3.1). DPO (Eq. 1) directly increases the likelihood of these GPT-4-written responses. If GPT-4o's judgments correlate with stylistic resemblance to GPT-4, then the reported deltas (e.g., +42.39 in Table 2) may reflect alignment to the judge rather than to safety. Please provide a human-annotation agreement study (e.g., Cohen's kappa between GPT-4o and human annotators on a random sample of responses) and/or validate the same comparisons with a second, non-GPT judge (e.g., a dedicated safety classifier or Llama-Guard). Without such evidence, the magnitude of the claimed safety improvement is not established.","section":"Section 4.1 (Evaluation Metrics) and Section 3.1 (Preference Pair Synthesis)"},{"comment":"The paper reports only DPO training and does not include a supervised fine-tuning (SFT) control on the same chosen responses. Since all chosen responses are GPT-4-generated, a model fine-tuned with SFT on the chosen responses alone could recover much of the reported improvement by learning GPT-4's refusal style, independent of the preference loss. Please add an SFT-on-chosen-only baseline for both base models (LLaVA-NeXT-Video and Qwen2.5-VL-7B) and compare its SafeVidBench safety rates with the DPO results. This would isolate the contribution of the preference optimization itself.","section":"Section 4.1 and Section 4.2"},{"comment":"There is a numerical inconsistency in the headline Challenge-set improvement: the abstract reports 39.17% for LLaVA-NeXT-Video + SafeVid-350K on SafeVidBench-Challenge, while Section 4.2 and Table 2 report 39.71%. Additionally, Table 2's headers use the name \"VidSafeBench\" while the rest of the paper uses \"SafeVidBench\"; the same inconsistent name appears in Section 2 (Related Work). Please correct the number and harmonize the benchmark name throughout the manuscript.","section":"Abstract, Section 4.2, and Table 2"},{"comment":"All reported safety rates are single-run point estimates without confidence intervals, significance tests, or error bars. This is particularly important for the smaller OOD deltas in Table 3 (e.g., the +7.31 safety-rate gain on miniJailBreakV-28K for LLaVA-NeXT-Video, or the +8.37 gain for Qwen2.5-VL-7B on the same benchmark), where the improvement is much smaller than the headline SafeVidBench numbers. Figure 3, which varies the training-data fraction, also lacks error bars across random subsets. Please report variance across at least three random seeds or, at minimum, provide bootstrap confidence intervals for the key comparisons.","section":"Section 4.1 and Figure 3"}],"minor_comments":[{"comment":"The number of safety subcategories is never stated explicitly. The benchmark construction in Section 3.3 counts 1,380 questions as 30 scene categories × 2 questions × 23 subcategories, but the text only lists 7 aggregate categories in Table 2. The same number of subcategories is needed to reconcile the 350K-pair dataset size with 12,377 curated videos (12,377 × 28 ≈ 350K, which requires 28 queries per video). Please state the exact number of subcategories and the number of preference pairs generated per video so the dataset size is reproducible.","section":"Section 3.1 and Section 3.3"},{"comment":"The sentence \"We adapte LLaMA-Factory training framework\" is missing a word; it should read \"We adapt the LLaMA-Factory training framework\" or similar. Please also verify the reference to LLaMA-Factory [52] is correctly cited.","section":"Section 4.1"},{"comment":"The caption reads \"Impact of SafeVid-350K data scale on alignment effectiveness\" but the figure contains two panels (SafeVidBench and MM-SafetyBench) with different y-axis ranges. Please update the caption to describe both panels and label them explicitly.","section":"Figure 3"},{"comment":"Reference [8] is cited as \"HarmEval\" in Table 1, but the reference title is \"SafeInfer: Context adaptive decoding time safety alignment for large language models.\" Please verify that this is the correct source for the HarmEval benchmark and, if so, clarify the relationship in the text.","section":"References"},{"comment":"The Limitations paragraph acknowledges the reliance on textual video descriptions but does not mention the potential bias from using GPT-family models both as the source of preferred responses and as the judge. Please add a sentence acknowledging this limitation and the need for human-validated evaluation.","section":"Section 5 (Limitations)"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea—using textual descriptions as a bridge to transfer text-based safety reasoning to video—is timely and the public dataset is a useful resource. The main obstacle is the LLM-as-judge circularity, which could substantially inflate the reported improvements. I believe this is fixable within a revision if the authors add a human-validated evaluation subset or a second judge, plus the SFT control. The manuscript also needs careful attention to the numerical inconsistency and naming errors before it can be considered for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SafeVid is a resource paper, and the resource is the point. It contributes a 350K video-specific safety preference dataset, a new benchmark with a base set and a human-red-teamed challenge set, and DPO experiments on two open VLMMs. The dataset construction is genuinely thoughtful: a scene taxonomy over InternVid, multi-model video descriptions refined by GPT-4, Gemini-generated adversarial questions, and GPT-4-written chosen responses. The benchmark has no overlap with the training videos and questions, which is the right design. On its own metric, alignment improves LLaVA-NeXT-Video from roughly 54% to about 96% safe on the base set, with similarly large jumps on several OOD benchmarks. That is enough to make the paper worth reading.\n\nThe soft spot is the evaluation loop. GPT-4 writes the chosen responses, GPT-4o judges every safety and helpfulness score in Tables 2 and 3, and DPO directly increases the likelihood of the GPT-4-written responses. So part of the reported 42-point gain is likely the model learning to imitate the judge's style. The stress-test note is right: without a human agreement study, an alternate judge, or error bars, the magnitude of the safety improvement is not established. I don't think this is fraud or sloppiness; it is a common design choice, but it is the load-bearing weakness, and it is fixable.\n\nMinor issues: the abstract says 39.17% on the Challenge set while Section 4.2 says 39.71%; the benchmark appears as both VidSafeBench and SafeVidBench; and the benchmark/code are not released, only the dataset. The paper's own limitations paragraph concedes that the textual bridge can miss subtle visual-temporal safety nuances, which is honest and should stay in the final version.\n\nBottom line: this is a useful contribution for people working on multimodal safety alignment. It deserves a serious referee. I would ask for human evaluation on a subset and a second judge before accepting the headline numbers, but I would not desk reject it. I'd also bring it to reading group to talk about evaluation methodology.","headline":"Useful resource paper: the 350K video safety dataset is the real contribution, but the headline safety gains are partly an artifact of the GPT-4/GPT-4o judge loop and need human evaluation before being taken at face value.","tokens_in":16471,"tokens_out":2462,"would_cite":true,"duration_ms":23883,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Video-language models can be made substantially safer by preference training on video-grounded synthetic data, the paper argues.","keywords":["video large multimodal models","safety alignment","direct preference optimization","preference dataset","mismatched generalization","safe refusal","video safety benchmark","textual description bridge"],"falsifier":"Have human annotators blind to model identity score a random sample of aligned-model responses from SafeVidBench-Challenge with the same seven-category rubric; if the human-rated safety rate is close to the pre-alignment model's rate rather than the reported 85.94%, the headline improvement is an artifact of the AI judge.","tokens_in":15412,"feed_emoji":"🛡️","tokens_out":12664,"duration_ms":111404,"temperature":0.7,"pith_summary":"Video-language models often accept a harmful request when it is accompanied by a relevant video, even though the same model would refuse the identical request as text; the paper names this 'mismatched generalization' and proposes to fix it by training on safety preferences grounded in video context. It builds SafeVid-350K, 350,000 video-specific question-response preference pairs, by turning each video into a detailed textual description and using that description as a bridge for text-based safety reasoning: a strong LLM writes adversarial questions and both a safe and an unsafe answer for each video. Fine-tuning open video-language models with Direct Preference Optimization on this data raises AI-judged safety rates by tens of percentage points—on the base benchmark, LLaVA-NeXT-Video goes from 53.99% to 96.38% safe—and the paper reports that the gains transfer to image and text safety benchmarks without large capability loss. The result matters because it offers a data-driven route to video-specific safety alignment without waiting for expensive human red-teaming at scale.","feed_headline":"Video AI safety jumps 42 points after 350K-pair alignment","feed_subtitle":"A text-description bridge teaches video models to refuse harmful queries, and the gains reach unseen benchmarks.","key_machinery":"The device that carries the argument is the interpretive bridge: a video is converted into a detailed textual description by several video-language models and condensed by an LLM, so that a text-only safety reasoner can apply rule-based judgment to the dynamic scene. All downstream components—adversarial question generation, chosen-response construction, and benchmark question design—operate on this text rather than on the raw video. The alignment step is Direct Preference Optimization, which trains the policy to increase the likelihood of chosen responses and decrease the likelihood of rejected responses:\n\n$$L_{\\mathrm{DPO}}(\\pi_\\$\\theta$;\\pi_{\\mathrm{ref}}) = -\\mathbb{E}_{(v,q,y_c,y_r)\\sim\\mathcal{D}}\\log\\$\\sigma$\\!\\left(\\$\\beta$\\log\\frac{\\pi_\\$\\theta$(y_c\\mid v,q)}{\\pi_{\\mathrm{ref}}(y_c\\mid v,q)} - \\$\\beta$\\log\\frac{\\pi_\\$\\theta$(y_r\\mid v,q)}{\\pi_{\\mathrm{ref}}(y_r\\mid v,q)}\\right).$$\n\nSafeVidBench fixes the target quantity as Safety Rate over seven harmful categories, in an automated base set and a human-red-teamed challenge set that share no videos or questions with the training data.","core_discovery":"The paper's central claim is that the safety gap between text and video in large multimodal models is not irreducible: textual descriptions can carry enough of the video's meaning for an LLM to reason about harm, generate adversarial queries, and write rule-following responses. The authors operationalize this as SafeVid, a closed loop of data construction, alignment, and evaluation. They curate 12,377 videos into 30 scene categories, describe each video through multiple video-language models refined by an LLM, use those descriptions to prompt an instruction-following generator for candidate adversarial questions per safety subcategory, then produce a rejected response from a vulnerable video-language model and a chosen response from an LLM armed with safety guidelines. DPO alignment on the resulting 350K preference pairs lifts LLaVA-NeXT-Video from 53.99% to 96.38% average safety on SafeVidBench-Base and from 46.23% to 85.94% on SafeVidBench-Challenge, with similarly large gains for Qwen2.5-VL-7B. The authors read this as evidence that video-specific safety principles can be instilled through preference learning on description-grounded data.","pith_inferences":["If the description-bridge mechanism is what carries the effect, the same recipe could generate safety alignment pairs for any captionable scene or language without collecting new videos.","A direct stress test this suggests is to corrupt or temporally truncate the textual descriptions and watch SafeVidBench safety fall; the drop would localize which video information the bridge needs.","The judge-writer alignment loop implies a test the paper does not run: re-train with chosen responses produced by a different family of models and re-score with the same judge; if gains collapse, part of the measured improvement is stylistic conformity.","Because the evaluation reads through the same textual bridge that generated the training labels, probing the description step itself with misleading captions would be a natural next attack surface."],"forward_implications":["On the paper's measurement, LLaVA-NeXT-Video's average safety on SafeVidBench-Base rises from 53.99% to 96.38% after DPO on SafeVid-350K, and on the human-red-teamed Challenge set from 46.23% to 85.94%.","Qwen2.5-VL-7B improves from 77.03% to 95.87% on the Base set and from 59.78% to 89.78% on the Challenge set, so the effect is not architecture-specific.","The gains appear to transfer out of distribution: the aligned LLaVA-NeXT-Video cuts VLBreakBench attack success from 68.00% to 16.48% and raises MM-SafetyBench safety from 42.02% to 74.31%, while Helpful Rate stays high.","The alignment tax is small: on MMBench-Video, perception scores hold roughly steady or improve and hallucination scores improve, while reasoning scores dip only modestly.","Data-scaling results indicate that basic refusal behavior appears with a small fraction of the 350K pairs, but the harder Challenge set and out-of-distribution benchmarks keep improving as more pairs are added, so dataset scale buys robustness rather than just refusal."],"supporting_citations":[{"why":"Supplies the InternVid-10M-FLT corpus from which SafeVid-350K's video set is filtered and selected.","marker":"[41]"},{"why":"Supplies the base video-language model whose outputs become rejected responses and whose aligned version is the main result.","marker":"[51]"},{"why":"Supplies the LLM that condenses the video descriptions and synthesizes the chosen safe responses.","marker":"[2]"},{"why":"Supplies the model that generates and selects the adversarial questions for each scene and safety subcategory.","marker":"[38]"},{"why":"Supplies the DPO objective that the alignment step optimizes.","marker":"[33]"},{"why":"Supplies the second base video-language model that is aligned and evaluated.","marker":"[6]"},{"why":"Supplies the video-text embeddings used to pick representative videos for each scene category.","marker":"[46]"},{"why":"Supplies an out-of-distribution image-text benchmark used to test whether the safety gains generalize.","marker":"[26]"}],"fun_headline_variants":["Video AI safety jumps 42 points via text-bridge training","SafeVid: Text descriptions align video models to safety","350K-pair DPO boosts video safety by 42%","From text to video: SafeVid bridges safety alignment","Video models learn to refuse via SafeVid's 350K pairs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the AI model used as judge for every reported safety score agrees with human safety judgments; if the judge merely rewards the answer style of the same model family that wrote the preferred training responses, the headline gains would be alignment to the judge rather than real safety.","fun_headline_variants_meta":{"raw":{"variants":["Video AI safety jumps 42 points via text-bridge training","SafeVid: Text descriptions align video models to safety","350K-pair DPO boosts video safety by 42%","From text to video: SafeVid bridges safety alignment","Video models learn to refuse via SafeVid's 350K pairs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000244,"raw_usage":{"total_tokens":1577,"prompt_tokens":1038,"completion_tokens":539,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":452}},"tokens_in":654,"tokens_out":539,"duration_ms":5535,"temperature":1.0,"reasoning_tokens":452,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:44:45.390959+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have human annotators blind to model identity score a random sample of aligned-model responses from SafeVidBench-Challenge with the same seven-category rubric; if the human-rated safety rate is close to the pre-alignment model's rate rather than the reported 85.94%, the headline improvement is an artifact of the AI judge.","supporting_citations":[{"cited_title":"Llava-next: A strong zero-shot video understanding model, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the base video-language model whose outputs become rejected responses and whose aligned version is the main result."},{"cited_title":"Videoclip: Contrastive pre-training for zero- shot video-text understanding","cited_arxiv_id":null,"evidence_quote":"Supplies the video-text embeddings used to pick representative videos for each scene category."},{"cited_title":"Mm-safetybench: A benchmark for safety evaluation of multimodal large language models","cited_arxiv_id":null,"evidence_quote":"Supplies an out-of-distribution image-text benchmark used to test whether the safety gains generalize."}],"review_version":1}