{"id":"f24d7dbf-7f4d-4728-8ac1-e2432e7ff894","arxiv_id":"2506.11549","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A VQA model that combines free-energy-inspired frame restoration with a scan-and-gaze prediction head reports competitive results on five benchmarks, though the claimed state-of-the-art performance is not uniformly supported.","lead":"EyeSim-VQA is a video quality assessment model that restores video frames with two enhancement networks and then predicts quality with a dual 'gaze and scan' attention head. It reports competitive benchmark scores, but the evaluation depends on unpublished data splits and no code is released.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim is not yet supported because Table I compares EyeSim to baseline numbers from other papers under an unspecified 8:2 split; small reported gains (0.003-0.004 SROCC) could be split artifacts.","rationale":"The reader's weakest assumption is exactly the same as the load-bearing issue: the 8:2 split is unspecified, baseline numbers are not measured under that split, and the reported gains are within the range of split-to-split variability. I agree with the CONDITIONAL verdict because these are addressable reproducibility problems: releasing split indices, code, and rerunning the strongest baselines under identical conditions would settle the claim. The paper's own text (Section IV-A) invites this concern by referring to 'the partitioning protocol from our previous work' without naming the work or giving a seed. The abstract's more modest 'competitive or superior' claim is a fair fallback, but the highlighted contribution explicitly claims SOTA, and that claim depends on the suspect comparison. No internal mathematical inconsistency in the architecture was found; the enhancement and prediction modules are coherently described, and the ablations are extensive. The single most decisive fix is to make the comparison protocol exact and reproducible, after which the verdict could move to ACCEPT or REJECT depending on the outcome.","tokens_in":19720,"tokens_out":5734,"duration_ms":54678,"concrete_test":"Ask the authors to release the exact split indices (or the random seed and partition script) used for the 8:2 split in Section IV-A, then rerun KSVQE and DOVER from their official implementations on that split with identical frame sampling and evaluation code. If either baseline matches or exceeds EyeSim on KVQ, KoNViD-1k, or YouTube-UGC, the SOTA claim is not supported; if EyeSim remains ahead across all three datasets, the comparison is credible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is stated in the contributions: 'EyeSim-VQA achieves state-of-the-art (SOTA) performance compared to existing mainstream approaches.' For that claim to hold, every baseline in Table I must be measured under the same training/test partitioning and evaluation protocol as EyeSim. Section IV-A says: 'For the first four datasets, we follow the partitioning protocol from our previous work by randomly splitting each dataset into training and testing sets at an 8:2 ratio.' No previous work is named, no seed or split indices are given, and the paper does not report rerunning any baseline on that split. The baseline entries therefore appear to be quoted from their original papers, which use their own split conventions. The decisive comparisons are tiny: KVQ SROCC 0.870 vs KSVQE 0.867, YouTube-UGC 0.902 vs 0.900, and LSVQ-test 0.888 vs 0.886. Random split variance on datasets of these sizes can easily exceed 0.003-0.004, so the reported advantage may reflect partition choice rather than model quality. This is compounded by component choices such as alpha1=0.3, alpha2=0.01 in Table XII being selected on KoNViD-1k, which is also reported as a test set in Table I. The concern is about the comparison protocol, not the architectural ideas; the abstract's weaker claim of 'competitive or superior performance' remains plausible until the protocol is clarified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EyeSim-VQA, a no-reference video quality assessment framework that combines a dual-branch architecture (aesthetic and technical) with free-energy-inspired visual enhancement modules, a biologically motivated prediction head that models gaze and scanning behavior via DyT-based attention and a Mamba-inspired temporal scan, and a composite enhancement loss with pixel, identity, and CONTRIQUE-based quality terms. The method is evaluated on KVQ, KoNViD-1k, YouTube-UGC, LIVE-VQC, and LSVQ, and the paper claims state-of-the-art or competitive performance, supported by extensive ablation studies on each component.","tokens_in":20075,"tokens_out":3646,"duration_ms":35603,"significance":"If the empirical comparison is sound, the paper makes a meaningful contribution by extending free-energy-guided self-repair from IQA to VQA, a direction that has been largely unexplored. The ablation study is unusually thorough, covering the enhancement modules, the dual-branch head, the embedding strategy, the number of enhanced frames, the DyT placement, and the loss components. The biologically inspired prediction head is a plausible architectural novelty, and the cross-dataset experiments, if properly controlled, would strengthen the generalization claim. However, the central SOTA claim currently depends on an evaluation protocol that is not fully specified, so the significance cannot be fully assessed until the comparison is made reproducible and fair.","major_comments":[{"comment":"The SOTA claim rests on Table I, but the evaluation protocol is not specified in a way that supports head-to-head comparison. Section IV-A says that for KVQ, KoNViD-1k, YouTube-UGC, and LIVE-VQC, the authors 'follow the partitioning protocol from our previous work by randomly splitting each dataset into training and testing sets at an 8:2 ratio,' without naming the previous work, giving a seed, or providing split indices. The baseline numbers in Table I appear to be quoted from the original papers, which use their own split conventions. The reported advantages are small (e.g., KVQ SROCC 0.870 vs 0.867 for KSVQE, YouTube-UGC 0.902 vs 0.900, LSVQ-test 0.888 vs 0.886), and random split variance on datasets of these sizes can easily exceed these margins. Please rerun all baselines on the exact same splits, provide the split indices or seed, and report results across multiple random splits with confidence intervals or variance.","section":"§IV-A, Table I"},{"comment":"The loss hyperparameters alpha_1=0.3 and alpha_2=0.01 are selected using Table XII, which is explicitly 'tested on KoNViD-1k.' The same dataset and split are then reported as a headline result in Table I. This constitutes selection on the test set and can inflate the reported KoNViD-1k performance relative to baselines that did not tune on that test set. The paper should either use a held-out validation split for hyperparameter selection, report results for default or non-tuned hyperparameters, or otherwise demonstrate that the reported improvements are not an artifact of test-set tuning.","section":"§IV-D-11, Table XII"},{"comment":"The cross-dataset generalization claim is supported by Tables II and III, but the experimental protocol for the baseline entries is not described. It is not stated whether SimpleVQA, FastVQA, and KSVQE were retrained by the authors under the same training splits and preprocessing, or whether the numbers are quoted from the KVQ paper. If the latter, the comparison may not be controlled for the same training data, input sampling, or evaluation code. Please specify the protocol for every entry, or rerun the baselines under identical conditions.","section":"§IV-C-2, Tables II and III"}],"minor_comments":[{"comment":"The method name is written inconsistently as 'EyeSimVQA' in the abstract and 'EyeSim-VQA' elsewhere; please use one consistent name.","section":"Abstract and throughout"},{"comment":"The text says the comparison includes '13 representative approaches,' but the list and Table I contain 14 methods; please correct the count.","section":"§IV-C-1"},{"comment":"The column headers 'ET' and 'EA' are not defined; spell them out as technical-branch enhancement and aesthetic-branch enhancement.","section":"Table VII"},{"comment":"The caption says 'Loss Function Hyperparameters,' but the table rows are the loss components (L_pixel, L_identity, L_IQA); retitle the caption to match the content.","section":"Table XI"},{"comment":"Reference [59] is cited for AdamW but is the Adam paper; please cite the AdamW source or correct the reference.","section":"References"},{"comment":"Equation (2) defines K=Q with a pooled query, but the attention computation between the pooled vector and each spatial/temporal position is terse; a short clarifying sentence or diagram would help readers understand the gaze mechanism.","section":"Equation (2)"}],"recommendation":"major_revision","confidential_remarks":"The core architectural idea is interesting and the ablations are extensive, but the SOTA claim is currently under-supported by the comparison protocol. The authors should be asked to either provide exact split indices and rerun baselines, or soften the claim to 'competitive.' No code or supplementary material was provided, which makes verifying the numbers harder. The paper may also benefit from reporting performance over multiple random splits, especially for the small-margin comparisons."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The architecture is a sensible combination of existing pieces and the ablation work is unusually thorough; what is not yet established is the headline SOTA claim, because the comparison protocol in Table I is uncontrolled. The paper's own abstract is more careful (\"competitive or superior\"), and that weaker claim may well be true, but the evidence as reported does not support the stronger one.\n\nWhat is genuinely new: this is a plausible first extension of the free-energy self-repair idea from IQA to VQA, and the dual-branch design (global resized frames plus fragment patches, each with its own enhancement network) is a legitimate extension of the DOVER/FastVQA line. The scan-and-gaze head is lightweight and clearly described; using DyT only in the head, not the backbone, is a sensible and well-tested choice. The ablation suite is a real asset: component-wise ablations, enhancement network selection, fusion strategies, loss design, hyperparameters, plus t-SNE and attention-map visualizations. That level of care is not common.\n\nThe soft spot is the evaluation. The 8:2 split is attributed to \"our previous work\" with no citation, no seed, and no split indices. The baselines in Table I look like numbers quoted from the original papers, which use their own split conventions. The decisive differences are tiny—0.870 vs 0.867 on KVQ, 0.902 vs 0.900 on YouTube-UGC, 0.888 vs 0.886 on LSVQ-Test. Split variance on datasets of these sizes is easily large enough to account for 0.003–0.004 SROCC. On top of that, the loss hyperparameters are tuned on KoNViD-1k, and KoNViD-1k also appears in Table I, so the test set has leaked into hyperparameter selection. That is a real problem for the SOTA claim, though not necessarily for the architecture itself.\n\nThe cross-dataset tables are better controlled and show the model is competitive, but even there the gains over KSVQE are small and inconsistent—they lose on one direction (KVQ→KoNViD SROCC 0.645 vs 0.650) while winning on others. So the claim of consistently outperforming LLM-based methods is overstated.\n\nThis is not a circularity problem: the VQA head is trained on human MOS, and the enhancement network uses CONTRIQUE as a proxy quality signal, which is defensible. The paper is also honest in its limitations section about the added pre-training complexity and the lack of LLM integration.\n\nBottom line: the paper deserves a serious referee, but only if the authors provide the split, code, and a rerun of baselines under a matched protocol. The architecture and ablation depth are worth referee time; the current comparative table is not reproducible as written. I would not cite it as a SOTA method until that is fixed.","headline":"A well-ablated but uncontrolled comparison: the architecture is plausible, the SOTA claim needs a rerun on matched splits.","tokens_in":20596,"tokens_out":2675,"would_cite":false,"duration_ms":25059,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EyeSim-VQA claims that free-energy-style self-repair before assessment lifts no-reference video quality prediction to state-of-the-art on several benchmarks.","keywords":["video quality assessment","no-reference VQA","free-energy principle","visual self-repair","dual-branch architecture","DyT attention","Mamba scanning","user-generated content"],"falsifier":"Re-run EyeSim-VQA and every Table I baseline on identical fixed 8:2 splits across several random seeds; if the reported margins (for example 0.870 vs 0.867 SROCC on KVQ, or 0.919 vs 0.922 on KoNViD-1k) flip sign or vanish under any seed, the claimed state-of-the-art status is an artifact of split choice rather than of the model.","tokens_in":19540,"feed_emoji":"👁️","tokens_out":9490,"duration_ms":84703,"temperature":0.7,"pith_summary":"EyeSim-VQA applies the free-energy principle of visual self-repair—the idea that the visual system actively reconstructs degraded input—to no-reference video quality assessment, a setting where it has not been used before. The model first enhances the input video in two complementary ways: a global branch cleans resized full frames with a lightweight super-resolution network, while a technical branch iteratively restores patch fragments with a residual network. A biologically inspired prediction head then fuses the two representations through a DyT-based gaze module and a Mamba-inspired scan module. The paper reports that the full model achieves state-of-the-art or competitive SROCC/PLCC on KVQ, KoNViD-1k, YouTube-UGC, LIVE-VQC, and LSVQ, in several cases outperforming LLM-based models. The authors note that the enhancement modules must be pre-trained separately before VQA training, that the framework does not yet use LLM knowledge, and that the restoration backbones are limited to two classical video-super-resolution designs.","feed_headline":"Eye-simulation model tops video quality benchmarks","feed_subtitle":"Self-repair enhancement before scoring plus scan-and-gaze fusion lifts accuracy across five public datasets.","key_machinery":"The load-bearing mechanism is the pairing of free-energy-style visual enhancement with a two-branch prediction head. In the aesthetic branch, a lightweight BasicVSR-mini network super-resolves and denoises whole frames; in the technical branch, CleanNet iteratively restores 32-frame patch fragments. Each enhanced output is concatenated with the original input before the backbone, avoiding changes to pre-trained weights. The head then models human perception with a DyT-based gaze branch, using dynamic tanh attention $\\mathrm{DyT}(x)=\\tanh(\\alpha\\cdot x)\\odot w+b$, and a Mamba-inspired scan branch that aggregates temporal neighborhood features before projection. The combination is what the paper credits for improved global-local fusion and for the reported gains in SROCC/PLCC.","core_discovery":"The central claim is that modeling how the human eye repairs degraded scenes improves automated video quality judgment. Concretely, EyeSim-VQA asserts that enhancing a video before feature extraction—restoring full frames globally and patch fragments locally—produces features that correlate more tightly with human mean opinion scores, and that a dual-branch prediction head which scans frames sequentially (Mamba-style temporal convolution) and gazes selectively (DyT-based attention) fuses global and local evidence better than a single regressor. On Table I, the model reports the best or second-best correlation on each of five benchmarks, including 0.870 SROCC on KVQ (above KSVQE's 0.867), 0.886 on LIVE-VQC, and 0.888 SROCC on LSVQ test, with its largest margin on LSVQ 1080p (0.807 vs KSVQE's 0.790). The paper's abstract states the outcome as 'competitive or superior,' while the contributions section describes it as state-of-the-art.","pith_inferences":["Training the enhancement networks on synthetic distortions that match each target benchmark would test whether the reported cross-dataset gains come from the self-repair mechanism or from distribution overlap between synthetic and real distortions.","The scan branch approximates Mamba with a compact temporal convolution; replacing it with an actual selective state-space scan under the same parameter budget would show whether sequential sweeping is the active ingredient or a stand-in.","An even simpler control, swapping the learned enhancement modules for a fixed denoiser or sharpening filter, would isolate how much of the gain requires learned self-repair rather than any pre-processing.","The absence of a named split or seed means the Table I comparison can be made airtight only by re-evaluating all baselines under the authors' exact protocol; this is a reproducibility check, not a claim about model performance."],"forward_implications":["Free-energy self-repair, previously confined to image quality assessment, transfers to video quality assessment without requiring retraining of the pre-trained backbone.","The enhancement step can be added by concatenating restored frames with originals before feature extraction, so existing VQA backbones do not need adapters or modified weights.","A scan-and-gaze prediction head trained from scratch outperforms a single-head regressor, and replacing LayerNorm with DyT in this head improves accuracy.","Cross-dataset tests indicate the enhancement mechanism helps generalization; the paper reports strong transfer in both directions between KVQ and other datasets.","The composite restoration loss, including a CONTRIQUE-based quality penalty, steers restoration toward perceptual quality rather than pixel fidelity alone."],"supporting_citations":[{"why":"Supplies the fragment-based patch sampling strategy used by the technical branch to preserve fine visual details.","marker":"[19]"},{"why":"Provides the dual-branch aesthetic/technical design that EyeSim-VQA extends with enhancement modules and a new prediction head.","marker":"[21]"},{"why":"Contributes the KVQ dataset, the LLM-based KSVQE baseline, and the ranking-pairs protocol used for evaluation.","marker":"[10]"},{"why":"BasicVSR is the super-resolution base for the aesthetic branch's lightweight enhancement module.","marker":"[33]"},{"why":"RealBasicVSR motivates CleanNet's residual-block design for patch-level enhancement.","marker":"[49]"},{"why":"Supplies DyT, the dynamic tanh transformation used in the gaze-branch attention mechanism.","marker":"[37]"},{"why":"Supplies Mamba's selective state-space scanning, which the scan branch approximates with a temporal convolution.","marker":"[38]"},{"why":"CONTRIQUE provides the pre-trained NR-IQA model used in the enhancement training loss.","marker":"[52]"},{"why":"KoNViD-1k is the source of the synthetic distortion training set and the main ablation benchmark.","marker":"[7]"}],"fun_headline_variants":["EyeSim-VQA: Free-energy self-repair powers VQA gains","Eye-inspired dual branches set new VQA standards","Free-energy repair lifts video quality scores on 5 benchmarks","Scan-and-gaze fusion upgrades video quality assessment","EyeSim-VQA tops five video quality benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 8:2 random train/test splits used for KVQ, KoNViD-1k, YouTube-UGC, and LIVE-VQC are comparable to the splits on which the cited baseline results in Table I were originally measured; the paper says it follows 'the partitioning protocol from our previous work' without naming that work or giving a seed.","fun_headline_variants_meta":{"raw":{"variants":["EyeSim-VQA: Free-energy self-repair powers VQA gains","Eye-inspired dual branches set new VQA standards","Free-energy repair lifts video quality scores on 5 benchmarks","Scan-and-gaze fusion upgrades video quality assessment","EyeSim-VQA tops five video quality benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000977,"raw_usage":{"total_tokens":4164,"prompt_tokens":973,"completion_tokens":3191,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":3113}},"tokens_in":589,"tokens_out":3191,"duration_ms":23560,"temperature":1.0,"reasoning_tokens":3113,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:04:39.833005+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run EyeSim-VQA and every Table I baseline on identical fixed 8:2 splits across several random seeds; if the reported margins (for example 0.870 vs 0.867 SROCC on KVQ, or 0.919 vs 0.922 on KoNViD-1k) flip sign or vanish under any seed, the claimed state-of-the-art status is an artifact of split choice rather than of the model.","supporting_citations":[{"cited_title":"Fast-vqa: Efficient end-to-end video quality assessment with fragment sampling,","cited_arxiv_id":null,"evidence_quote":"Supplies the fragment-based patch sampling strategy used by the technical branch to preserve fine visual details."},{"cited_title":"Kvq: Kwai video quality assessment for short-form videos,","cited_arxiv_id":null,"evidence_quote":"Contributes the KVQ dataset, the LLM-based KSVQE baseline, and the ranking-pairs protocol used for evaluation."},{"cited_title":"Basicvsr: The search for essential components in video super-resolution and beyond,","cited_arxiv_id":null,"evidence_quote":"BasicVSR is the super-resolution base for the aesthetic branch's lightweight enhancement module."},{"cited_title":"Investigating tradeoffs in real-world video super-resolution,","cited_arxiv_id":null,"evidence_quote":"RealBasicVSR motivates CleanNet's residual-block design for patch-level enhancement."},{"cited_title":"Image quality assessment using contrastive learning,","cited_arxiv_id":null,"evidence_quote":"CONTRIQUE provides the pre-trained NR-IQA model used in the enhancement training loss."},{"cited_title":"The konstanz natural video database (konvid-1k),","cited_arxiv_id":null,"evidence_quote":"KoNViD-1k is the source of the synthetic distortion training set and the main ablation benchmark."}],"review_version":1}