{"id":"4456017c-4f11-4778-8234-794b21d04add","arxiv_id":"2608.03096","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new benchmark and a fine-tuning plus random-forest aggregation pipeline let eleven of twelve image-level deepfake detectors beat the best video-level detector on AI-generated video.","lead":"This paper builds a large benchmark of nearly 100,000 real and AI-generated videos to test whether image-based deepfake detectors can be lifted to video. It also introduces a fine-tuning and aggregation pipeline that lets eleven of twelve image detectors beat a state-of-the-art video detector on the benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Head-to-head comparison is not matched: image-level detectors get test-set-chosen aggregation and fine-tuning/MMA training on data video-level detectors never see, so the 80.16% and 93.80% AUC advantages over FTCN are not established.","rationale":"The reader's weakest assumption correctly identifies the post-hoc best-mode selection and the unspecified MMA training set; the rationale also notes the fine-tuned-versus-off-the-shelf asymmetry. These are the same load-bearing points I find. The concern is not that the benchmark is worthless: the dataset construction, detector coverage, and evaluation metrics are described in detail, and the promised repository would make the numbers checkable. The issue is that the headline comparisons are not currently interpretable as performance of usable pipelines. The naive image-level numbers are oracle-selected over six modes, and the IV-Bridge numbers depend on an undisclosed training set plus a data/adaptation advantage over frozen video-level detectors. Each of these is correctable by reporting validation-based mode selection, fully specifying D_vid and confirming disjointness, and either fine-tuning video-level detectors under the same protocol or clearly labeling the comparison as adapted-image-level versus off-the-shelf-video-level. Because the fixes are straightforward and the underlying resource has value, CONDITIONAL remains the right verdict; my stress-test does not move it.","tokens_in":27364,"tokens_out":6867,"duration_ms":62931,"concrete_test":"Re-run the central comparison with a matched protocol: (1) choose each naive image-level aggregation mode on a held-out validation split (or pre-register one mode) and recompute the Table 3 AUCs; (2) require the authors to specify D_vid and verify it is disjoint from all evaluation videos, then retrain each MMA random forest only on that specified set (or on the training splits of FF++/GenVideo); (3) optionally apply VFT fine-tuning to the video-level detectors on the same data. If RINE's 80.16% and RINE-IV's 93.80% drop to or below FTCN's 79.99%, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that image-level detectors, especially IV-Bridge-enhanced ones, outperform the best video-level detector (FTCN, 79.99% AUC). As reported, this claim rests on an unmatched evaluation protocol. In Table 3, naive image-level AUCs are 'the optimal aggregated results by one of the six modes' (caption, Section 5.2). No validation split is described, so the aggregation mode appears to be chosen after observing test-set performance. Taking the maximum over six correlated statistics on the test set inflates the reported number; RINE's 80.16% MIN-mode result is not the performance of a fixed, independently selected pipeline. The IV-Bridge comparison is asymmetric as well: image-level detectors are fine-tuned in VFT on FF++ (c23) plus GenVideo training frames, and MMA trains a random forest on a dataset D_vid that is never specified (Section 6.1), while the video-level detectors are used off-the-shelf without equivalent fine-tuning or learned aggregation. The paper states D_ft has no overlap with the evaluation set, but it never defines D_vid or rules out overlap. If D_vid includes evaluation videos or frames, the 93.80% RINE-IV result is directly contaminated. These are protocol/correctness risks, not internal inconsistencies; the benchmark itself is detailed and potentially valuable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces FakeI2V-Bench, a benchmark of 97,548 videos (CDFV2, DFD, GenVideo, GenVidBench) for evaluating deepfake video detectors, with a focus on systematically assessing image-level detectors. It evaluates eight video-level detectors and twelve naive image-level detectors, reporting frame-level and video-level performance under six aggregation modes. The paper further proposes IV-Bridge, a two-stage enhancement consisting of Video-Frame Fine-Tuning (VFT) and Multi-Mode Aggregation (MMA) via a random forest, claiming that eleven of twelve IV-Bridge-enhanced detectors surpass the best video-level detector (FTCN, 79.99% AUC), with RINE-IV reaching 93.80% AUC. The central claims are that naive image-level detectors can slightly surpass video-level detectors after test-set-optimal aggregation (80.16% vs. 79.99% AUC) and that IV-Bridge-enhanced image-level detectors are both more accurate and cheaper than video-level detectors.","tokens_in":27648,"tokens_out":3394,"duration_ms":32376,"significance":"If the headline results were established under a sound protocol, the benchmark would be a valuable resource: it is the first to systematically compare a broad set of image-level detectors on video-level tasks, it covers recent generation models, and it proposes a lightweight enhancement pathway (IV-Bridge) that could make mature image-level detectors practical for video. The release of code and data is a concrete strength, and the per-detector, per-dataset breakdown in Table 3 is useful for the community. However, the two central quantitative claims rest on protocol choices that currently overstate the case: reporting the post-hoc best aggregation mode for naive image-level detectors and failing to specify the training set for the random forest in MMA. These issues are load-bearing for the paper's main comparisons, though they appear fixable with a validation-based protocol and full disclosure.","major_comments":[{"comment":"The paper reports naive image-level detector results as 'the optimal aggregated results by one of the six modes' (Table 3 caption). Because the aggregation mode is selected after observing test-set performance, the 80.16% AUC for RINE is the maximum over six correlated statistics rather than the performance of a fixed, independently selected pipeline. The 0.17 percentage-point margin over FTCN (79.99%) is exactly the kind of number that this selection process can manufacture. Please fix the aggregation protocol on a validation split (or pre-register a single mode per detector and report all six modes for transparency), then report the test-set numbers under that fixed protocol.","section":"Section 5.2 and Table 3 caption"},{"comment":"The random forest classifier C_RF in MMA is trained on a dataset D_vid that is never specified. The paper explicitly states that the fine-tuning set D_ft has no overlap with the evaluation set, but it is silent about D_vid. If D_vid contains FakeI2V-Bench evaluation videos or frames, the headline 93.80% AUC for RINE-IV is directly contaminated. Please define D_vid, including its source, size, and how it relates to the evaluation sets, and rule out any overlap. Ideally, train the random forest via cross-validation on a disjoint split so the aggregation learner has genuinely unseen data.","section":"Section 6.1 (MMA) and Section 6.2"},{"comment":"The comparison between IV-Bridge-enhanced image-level detectors and video-level detectors is asymmetric: image-level detectors receive VFT fine-tuning on FF++ (c23) plus GenVideo training frames and a learned random-forest aggregator, while all video-level detectors are evaluated with official pretrained weights and no adaptation. The deployment-cost table (Table 5) therefore compares 'off-the-shelf video-level detectors' against 'fine-tuned and learned-aggregated image-level detectors,' which is not a like-for-like comparison. Please either apply an equivalent fine-tuning/adaptation protocol to the video-level detectors, or explicitly restrict the claim to 'image-level detectors augmented by IV-Bridge fine-tuning and aggregation.'","section":"Section 6.2 and Table 5"},{"comment":"No error bars, confidence intervals, or training seeds are reported, and the SMP aggregation mode in Appendix A is explicitly random. Given that the naive-image-level versus video-level margin is only 0.17 AUC points and that the SMP mode is stochastic, a single run is insufficient to support the comparison. Please report means and standard deviations over at least three seeds, or otherwise fix and state the random seed and show that the conclusions are stable.","section":"Tables 3 and 5; Appendix A"}],"minor_comments":[{"comment":"The phrase 'the optimal aggregated results by default' is ambiguous: it is not clear whether the optimal mode is chosen per detector globally or per dataset, and the caption should state this explicitly.","section":"Section 5.2 and Table 3 caption"},{"comment":"The 'Time (ms)' column should specify what workload the timing refers to (e.g., one video of a given length, one frame, or one full evaluation clip), since inference time for frame-based methods depends on the number of frames processed.","section":"Table 5"},{"comment":"The fine-tuning details for VFT are incomplete: learning rate, number of epochs, optimizer, and batch size are not given, although the paper states it follows original hyperparameters. Please provide these in an appendix or supplementary material.","section":"Section 6.2"},{"comment":"In the provided version, the figure labels appear as corrupted glyph sequences (e.g., '/uni00000013'); if these are not rendering artifacts of the PDF, the figures need to be regenerated with correct text labels.","section":"Figures 2, 4, 5, 6, and 7"},{"comment":"The number '97, 548' appears with an extra space in the abstract; please fix the formatting. Also, consider adding the total real/fake video counts in Table 2 for clarity.","section":"Abstract and Table 2"}],"recommendation":"major_revision","confidential_remarks":"The benchmark and code release are potentially valuable, and the idea of adapting image-level detectors with a lightweight aggregator is timely. My concern is that the two headline claims (80.16% vs. 79.99% and 93.80% vs. 79.99%) are not yet established because of test-set-selected aggregation and an undisclosed D_vid. These are fixable within the manuscript's scope by adopting a validation-based protocol and fully disclosing the training data. I am therefore recommending major revision rather than rejection, but I would not accept the paper in its current form without these protocol changes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version. The benchmark is a genuinely useful resource — a 97k-video testbed spanning 2024-2025 generators with both facial and general content, plus a systematic sweep of twelve image-level detectors and eight video-level detectors. That fills a real gap: prior benchmarks either skipped image-level detectors or evaluated them only at frame level. The IV-Bridge pipeline (fine-tune on video frames, then random-forest aggregation of six frame-to-video statistics) is not conceptually deep, but it is a reasonable and practical baseline to ship. If the released code and data match the paper, this will be a citable resource.\n\nThe soft spots are all in the comparison protocol, and the stress-test note accurately identifies them.\n\nFirst, the Table 3 caption says naive image-level results are \"the optimal aggregated results by one of the six modes.\" There is no validation split described, so the aggregation mode is selected after seeing test performance. That makes RINE's 80.16% versus FTCN's 79.99% a comparison between a pipeline tuned on the test set and a fixed video detector. It is a real inflation, and the paper's own Section 5.2 shows how much the mode matters (RINE ranges from 68.60 to 80.16). This needs to be fixed by reporting all six modes or by selecting the mode on a validation split.\n\nSecond, IV-Bridge's MMA random forest is \"trained on a dataset D_vid\" (Section 6.1) that is never defined. The paper explicitly rules out overlap for the fine-tuning set D_ft, but says nothing about D_vid. If D_vid contains evaluation videos, the 93.80% RINE-IV number is contaminated. This is a missing specification, not an internal contradiction, but it is load-bearing.\n\nThird, the comparison is asymmetric: IV-Bridge detectors get task-specific fine-tuning and a learned aggregator, while the video-level detectors are used off-the-shelf. Showing that adapted image detectors beat unadapted video detectors is still interesting, but it is not the same as showing image detectors are state of the art for video. The authors should either adapt the video detectors too or soften the claim.\n\nMinor: no error bars or seeds are reported; for a benchmark paper that is worth stating even if not fatal.\n\nBottom line: the benchmark resource is valuable and the questions are worth asking. The headline numbers, as reported, are not established. This deserves a serious referee, not a desk reject, with required revisions around the aggregation protocol and the D_vid specification. If those are fixed, I'd cite it and use it.","headline":"Useful new benchmark, but the headline comparison is not yet supported: image detectors get test-set-chosen aggregation and extra training data the video detectors don't.","tokens_in":28164,"tokens_out":3415,"would_cite":true,"duration_ms":29981,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FakeI2V-Bench claims that fine-tuned image-level detectors with random-forest aggregation surpass the best video-level deepfake detector on a 97,548-video benchmark.","keywords":["deepfake detection","video deepfakes","AI-generated video","image-level detectors","frame-to-video aggregation","detector benchmarking","AIGC","IV-Bridge"],"falsifier":"Re-run the benchmark with one aggregation mode fixed before seeing test labels and with the random forest trained on an explicitly disjoint, published split; if the best adapted image-level detector no longer exceeds FTCN's 79.99% mean AUC, the claimed superiority is an artifact of the evaluation protocol.","tokens_in":27165,"feed_emoji":"🎥","tokens_out":8609,"duration_ms":71473,"temperature":0.7,"pith_summary":"FakeI2V-Bench argues that image-level deepfake detectors—models that score individual frames—are not obsolete in the video era, but are being evaluated the wrong way. On a new benchmark of 97,548 videos spanning facial forgeries and general AI-generated content, naive image-level detectors drop sharply at frame level, yet one of them reaches 80.16% AUC after simple frame-to-video aggregation, slightly above the best video-level detector's 79.99% AUC. The paper's IV-Bridge pipeline, which fine-tunes image detectors on video frames and then learns to fuse six frame-score statistics with a random forest, lifts eleven of twelve image-level detectors above that video-level state of the art, with the best variant reaching 93.80% AUC. If these comparisons hold, the practical recipe for video deepfake detection becomes cheaper and more general than current video-level models.","feed_headline":"93.8% AUC: image detectors beat video-level deepfake models","feed_subtitle":"IV-Bridge fine-tunes frame detectors and fuses six score statistics, topping the 79.99% video-level state of the art.","key_machinery":"IV-Bridge, a two-stage adaptation pipeline. Stage one, Video-Frame Fine-Tuning (VFT), fine-tunes each image-level detector on a mixed pool of real and fake video frames from FaceForensics++ and the GenVideo training split, using cross-entropy or binary cross-entropy depending on the detector's output head. Stage two, Multi-Mode Aggregation (MMA), computes six frame-score statistics per video—the sampled single frame, mean, max, min, median, and variance—and feeds the six values into a random forest trained on a separate video set to output the final forgery probability. The key work the random forest does is to learn, per detector and per video, which aggregation mode or combination of modes is most trustworthy, replacing a fixed rule such as 'average over frames'.","core_discovery":"The central discovery is that the forgery signal needed for video deepfake detection mostly lives in frames, not in long-range temporal structure, and the bottleneck is the frame-to-video decision rule. The paper shows that ten of twelve image-level detectors lose substantial accuracy when moved from static forged images to video frames (e.g., NPR drops from 92.17% to 55.38% AUC), but that aggregation restores much of the loss: RINE goes from 71.11% frame-level AUC to 80.16% video-level AUC, surpassing the strongest video-level detector evaluated (FTCN at 79.99%). IV-Bridge then closes most of the remaining gap by fine-tuning each image detector on real and fake video frames and feeding a six-statistic summary of its frame scores through a random forest. In the paper's Table 3, eleven of twelve enhanced detectors beat FTCN overall, and the best, RINE-IV, reaches 93.80% AUC and 97.63% AP.","pith_inferences":["A testable extension the authors leave implicit: the six frame-score statistics are detector-agnostic, so one shared random forest could be trained on scores from all twelve detectors at once; the resulting ensemble might exceed any single -IV variant.","Because the aggregation features encode only score spread, not score order, an extension worth testing is adding temporal-order statistics—where in the video the max, min, or highest-variance segment occurs—to address the sparse-tampering regime where all current variants lose ground.","The reported best-of-six protocol means the numbers should be read as an upper bound on what a deployed system would achieve; fixing one aggregation mode on a validation split would give the operational performance estimate."],"forward_implications":["Eleven of the twelve IV-Bridge-enhanced image-level detectors exceed the best video-level detector (FTCN, 79.99% mean AUC), with RINE-IV reaching 93.80% AUC and 97.63% AP.","The practical cost equation flips: three enhanced image-level detectors (NPR-IV, Patch-IV, CoDE-IV) beat FTCN on accuracy, parameter count, and inference time at once, with Patch-IV at 4.34M parameters and 9.69 ms per video.","Comparing image-level detectors at frame level against video-level detectors is misleading; once frame scores are aggregated to a video decision, even a naive detector (RINE, 80.16% AUC) can edge past the strongest video-level model.","Adapted image-level detectors generalize across 14 video generation models, averaging 95.51% AUC versus 91.01% for the best video-level detector, while Sora and image-initialized generators (SVD, MuseV) remain the hardest cases.","Under sparse tampering (only 20% forged frames in Celeb-DF v2), all detectors degrade, but the best enhanced image-level detector (Patch-IV, 77.15% AUC) still clearly beats the best video-level detector (LAA-Net, 67.15% AUC)."],"supporting_citations":[{"why":"FTCN is the strongest video-level detector evaluated; its 79.99% mean AUC is the threshold the paper's enhanced image-level detectors are claimed to surpass.","marker":"[62]"},{"why":"GenVidBench supplies 55,200 fake and 13,800 real general-domain videos from eight recent generators, anchoring the GVB evaluation column.","marker":"[33]"},{"why":"GenVideo supplies the GV evaluation dataset and part of the real/fake video-frame pool used for fine-tuning; DeMamba is also one of the eight video-level baselines.","marker":"[10]"},{"why":"Celeb-DF v2 provides the main facial evaluation videos, one of the two facial datasets in the benchmark.","marker":"[28]"},{"why":"DFD contributes the second facial evaluation set with non-uniform resolutions, testing resolution robustness.","marker":"[17]"},{"why":"FaceForensics++ (c23) provides the 92,157 real and 92,146 fake video frames used to fine-tune image-level detectors in Video-Frame Fine-Tuning.","marker":"[45]"},{"why":"RINE is the naive image-level detector that reaches 80.16% AUC by aggregation and becomes the best IV-Bridge variant at 93.80% AUC.","marker":"[26]"},{"why":"DeepfakeBench is the prior benchmark whose frame-level reporting protocol the paper argues underestimates image-level detectors, motivating the aggregation-based comparison.","marker":"[58]"},{"why":"Random forests are the classifier used in Multi-Mode Aggregation to combine the six frame-to-video mode scores.","marker":"[4]"}],"fun_headline_variants":["Image detectors beat video-level models on deepfake video","New benchmark: frame-level signals dominate video deepfake detection","IV-Bridge boosts image detectors to 93.8% AUC on video deepfakes","FakeI2V-Bench: image-level detectors can rival video-level deepfake tools"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline advantage is measured by allowing each image-level detector to be scored with whichever of six aggregation modes happens to look best on the test set, and by training the random-forest aggregator on data whose separation from the evaluation videos is not spelled out.","fun_headline_variants_meta":{"raw":{"variants":["Image detectors beat video-level models on deepfake video","New benchmark: frame-level signals dominate video deepfake detection","IV-Bridge boosts image detectors to 93.8% AUC on video deepfakes","FakeI2V-Bench: image-level detectors can rival video-level deepfake tools"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000272,"raw_usage":{"total_tokens":1691,"prompt_tokens":1062,"completion_tokens":629,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":678,"completion_tokens_details":{"reasoning_tokens":549}},"tokens_in":678,"tokens_out":629,"duration_ms":5545,"temperature":1.0,"reasoning_tokens":549,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:52:30.808120+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the benchmark with one aggregation mode fixed before seeing test labels and with the random forest trained on an explicitly disjoint, published split; if the best adapted image-level detector no longer exceeds FTCN's 79.99% mean AUC, the claimed superiority is an artifact of the evaluation protocol.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FTCN is the strongest video-level detector evaluated; its 79.99% mean AUC is the threshold the paper's enhanced image-level detectors are claimed to surpass."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Celeb-DF v2 provides the main facial evaluation videos, one of the two facial datasets in the benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DFD contributes the second facial evaluation set with non-uniform resolutions, testing resolution robustness."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"RINE is the naive image-level detector that reaches 80.16% AUC by aggregation and becomes the best IV-Bridge variant at 93.80% AUC."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DeepfakeBench is the prior benchmark whose frame-level reporting protocol the paper argues underestimates image-level detectors, motivating the aggregation-based comparison."}],"review_version":2}