{"id":"99ff64f8-1920-4abe-82ee-855721aed49b","arxiv_id":"2602.04584","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SalFormer360, a SegFormer-based transformer with a custom decoder and decaying center-bias, reports state-of-the-art CC scores on three 360-degree video saliency benchmarks.","lead":"This paper introduces SalFormer360, a lightweight transformer model that predicts where people look in 360-degree videos using just two frames. It reports higher accuracy than previous methods on three public eye-tracking datasets, which could improve VR video streaming efficiency.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-protocol baseline comparison may explain the PVS-HM and VR-EyeTracking improvements; unified re-evaluation is needed.","rationale":"The reader's weakest assumption precisely identifies the load-bearing fragility: the empirical SOTA claim depends on cross-paper numbers that are not protocol-aligned. The paper's own notation admits that 360Spred used uniform sampling while the proposed method used sinusoidal weighting, and the ground-truth maps for two of the three datasets were regenerated by the authors with a specific Gaussian kernel. These are not minor details: CC is sensitive to spherical weighting, and different GT maps change the reference against which all methods are measured. The PVS-HM improvement is particularly thin (2.5% relative), so even a small protocol shift could eliminate it. The model itself appears well-designed, with ablations supporting the contributions of center bias and the loss components, and the lightweight efficiency claim is credible. However, the central claim of outperforming state-of-the-art is not yet independently verified under a common evaluation protocol. The reader's conditional verdict is appropriate; my stress-test does not uncover a different or more severe issue, so the verdict remains unchanged pending the recommended re-evaluation.","tokens_in":17196,"tokens_out":2570,"duration_ms":26003,"concrete_test":"Obtain the pre-trained 360Spred and 3DSphereNet models (or retrain them on the same train splits) and evaluate on the same test frames of PVS-HM and VR-EyeTracking using the authors' generated ground-truth saliency maps and the same latitudinal sinusoidal weighting used for SalFormer360. Compare the resulting CC, NSS, KL, and AUC-Judd to the SalFormer360 values in Table I. If 360Spred's PVS-HM CC reaches or exceeds 0.807 under this unified protocol, the central SOTA claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim rests on cross-paper comparisons in Table I. The paper itself marks 360Spred (the previous best on PVS-HM) with a dagger (†), meaning uniform spherical sampling, while SalFormer360 uses the latitudinal sinusoidal factor (asterisk, *). Since CC is computed over spherical sampling points, this weighting difference can change scores non-trivially — especially for maps with polar content. More importantly, the ground-truth saliency maps for PVS-HM and VR-EyeTracking were generated by the authors using a hand-chosen 7-degree Gaussian kernel (Sec. IV-A), whereas prior methods may have used different kernels or the original datasets' own saliency maps. If the comparison GT differs, the CC numbers are not directly comparable. The PVS-HM margin is only 0.807 vs. 0.787 (a 2.5% relative improvement). If 360Spred were re-evaluated with the same sinusoidal weighting and the same GT maps, that margin could shrink, vanish, or even reverse. The VR-EyeTracking improvement (18.6%) is larger but still dependent on the GT generation protocol. Without a unified evaluation protocol — identical spherical weighting, identical GT, identical test splits — the headline superiority over prior art is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SalFormer360, a transformer-based saliency estimation model for 360-degree video. The architecture uses a SegFormer MiT-B0 encoder modified to accept two concatenated RGB frames (current and t-5), a custom convolutional decoder, and a center-bias term with a learnable dynamic/static weighting. The model is trained with a combination of Pearson correlation, KL divergence, spherical MSE, and binary cross-entropy losses. Experiments on Sport360, PVS-HM, and VR-EyeTracking report state-of-the-art CC values of 0.722, 0.807, and 0.593, respectively, with 3.70M parameters and about 196 fps inference. The authors also generated ground-truth saliency maps for PVS-HM and VR-EyeTracking from head/gaze data and released code and resources.","tokens_in":17529,"tokens_out":3486,"duration_ms":40692,"significance":"If the claimed results hold, the paper makes a useful contribution: a lightweight, real-time transformer-based model for 360-degree video saliency, with informative ablations showing the benefit of the center-bias components and the multi-term loss. The release of the extended PVS-HM and VR-EyeTracking datasets is also valuable. However, the headline superiority over prior art is not yet established because the comparisons in Table I mix evaluation protocols: spherical weighting differs across methods, and ground-truth maps for two datasets were generated by the authors with a different kernel than those used by prior baseline reports. The ablations and efficiency analysis are solid, and the central architectural idea is plausible, but the SOTA claim needs a unified re-evaluation before it can be accepted.","major_comments":[{"comment":"The central SOTA claim is not yet supported because the comparison mixes spherical sampling protocols. 360Spred, the previous best on PVS-HM, is marked with a dagger (uniform spherical sampling) while SalFormer360 uses the latitudinal sinusoidal factor (asterisk). Since CC is computed over spherical sample points, this weighting difference can change scores non-trivially, especially for maps with polar content. The PVS-HM margin is only 2.5% (0.807 vs 0.787); without re-evaluating 360Spred under the same sinusoidal weighting and the same ground-truth maps, this margin could shrink or reverse. Please provide a unified evaluation of all top baselines using identical spherical weighting, GT maps, and test splits.","section":"Table I, Sec. IV-D and IV-E.1"},{"comment":"Ground-truth saliency maps for PVS-HM and VR-EyeTracking were generated by the authors using a hand-chosen 7-degree Gaussian kernel, while Sport360's GT maps come from the original dataset with a different kernel (3.34 degrees at lower resolution). If prior methods reported scores on different GT maps, then CC and KL numbers are not directly comparable across rows in Table I. This is particularly load-bearing for the small PVS-HM gain. Please either use the original datasets' GT maps or re-evaluate all competing methods on the same GT maps as SalFormer360.","section":"Sec. IV-A, Saliency maps generation"},{"comment":"The center bias is computed from the training-set first-frame ground truth, and alpha/beta are learned per dataset. This is not a test-label leak, and the ablation shows the components contribute. However, this design makes the model dataset-specific: the reported gains on PVS-HM and VR-EyeTracking may reflect fitting to each dataset's initial center-bias tendency rather than a generalizable saliency model. Please report cross-dataset evaluation (e.g., train on one dataset, test on another) or explicitly discuss this limitation, since the abstract's general claim of improved saliency estimation should not be overstated.","section":"Sec. III-A3, Eqs. (2)-(3)"},{"comment":"Several baseline numbers are taken from secondary sources ([40], [42]) rather than original papers, and many table cells are missing ('–'). This makes the 'consistently outperforms' claim hard to verify. Please provide a table footnote listing the exact source for every baseline value and, ideally, run a unified evaluation of the top-5 competitors on all three datasets with the same metric implementation and preprocessing.","section":"Sec. IV-C, Table I"}],"minor_comments":[{"comment":"The loss equation uses L_BCE(1, S), but the text describes binary cross-entropy between predicted and ground-truth maps. This appears to be a typo; it should likely be L_BCE(ˆS, S).","section":"Eq. (4)"},{"comment":"The figure legend contains 'OthersOurs' with unclear alignment. The categories and their correspondence to the listed methods need clarification.","section":"Fig. 1"},{"comment":"Several entries are malformed, e.g., '0.6205.125– 0.937' and '0.2913.768– 0.899'. Please fix spacing and formatting for readability.","section":"Table I"},{"comment":"The statement that a 7-degree kernel is chosen to approximate the Sport360 kernel of '3.34 degrees for 128×256 pixels' is unclear. Are these values directly comparable given different resolutions and projection formats? Clarify the relationship.","section":"Sec. IV-A"},{"comment":"The text says the model 'ranks fourth in terms of FLOPs and model size,' but Table II does not provide complete FLOPs/model-size data for all methods. Please show the ranking explicitly or qualify the statement.","section":"Sec. IV-E.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the journal's scope and the proposed model is plausible, with a clean ablation study and useful dataset extensions. The main blocker is the cross-protocol comparison in Table I, which prevents the SOTA claim from being accepted as-is. The paper should be revisable in a major revision: a unified re-evaluation of key baselines on the same GT and with the same spherical weighting is necessary. I see no indication of bad faith; the issues are methodological rigor in benchmarking."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: SalFormer360 is a genuinely new, lightweight transformer architecture for 360-degree video saliency — 3.7M parameters, ~196 fps — and the ablations make a clean case for each design choice. What is not solid is the headline state-of-the-art claim, particularly on PVS-HM, because the comparison protocol is not apples-to-apples.\n\nWhat's good: The idea of adapting a SegFormer encoder (by duplicating weights for a two-frame, 6-channel input) and adding a learnable center-bias decay is simple and effective. The ablation table shows both the short-term decay δ and the persistent bias β contribute across all three datasets, with sensible per-dataset differences. The model complexity is impressive: 3.7M parameters and 1.21 GFLOPs, with measured 5.1ms per frame. They also release code and generated saliency maps for PVS-HM and VR-EyeTracking, which is a useful resource, even if the generation protocol is non-standard.\n\nThe soft spots are real. First, the PVS-HM and VR-EyeTracking ground-truth maps were generated by the authors with a 7-degree Gaussian kernel, while the Sport360 GT comes from the original dataset. That means their CC numbers on those two datasets are not directly comparable to numbers reported in prior papers unless all baselines were re-evaluated on the same GT. They weren't. Second, the spherical sampling is inconsistent: 360Spred, the previous best on PVS-HM, is marked with uniform sampling (†) while the proposed method uses the sinusoidal factor (*). The margin over 360Spred is only 0.807 vs 0.787 (2.5% relative). Under a unified protocol, that margin could shrink, vanish, or reverse. The VR-EyeTracking margin (18.6% over SST-Sal) is larger and both use *, so it's more robust, but the GT issue still applies. Also, no error bars are reported, and Eq. 4 has a suspicious L_BCE(1,S) term that is almost certainly a typo — the authors should fix that.\n\nBottom line: The engineering is solid and the paper deserves a serious referee, but the current experimental section overstates the SOTA claim. The fix is straightforward: re-run all baselines on the authors' own GT with a single spherical weighting scheme, or at minimum report the comparison under both protocols and clearly qualify the results. For the reader community, this is a useful model to know about, and I'd happily cite it as a lightweight baseline, but not as the definitive SOTA of 360 video saliency.","headline":"Solid lightweight model with clean ablations, but the SOTA claim rests on cross-protocol comparisons and a 2.5% PVS-HM margin that could evaporate under unified evaluation.","tokens_in":17966,"tokens_out":5426,"would_cite":true,"duration_ms":52496,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SalFormer360, a lightweight transformer model, claims state-of-the-art saliency prediction on three 360-degree video benchmarks while running at ~196 fps.","keywords":["saliency estimation","360-degree video","transformer","SegFormer","viewing center bias","viewport prediction","omnidirectional video","lightweight model"],"falsifier":"Re-run all baseline methods under a single protocol: identical ground-truth saliency maps (same Gaussian kernel and fixation-to-map pipeline), identical evaluation sampling (same sinusoidal weighting), and identical train/test splits. If, under that protocol, 360Spred or another prior method achieves a CC above SalFormer360's 0.807 on PVS-HM or above 0.722 on Sport360, then the paper's central claim of state-of-the-art performance is contradicted.","tokens_in":1435,"feed_emoji":"🎯","tokens_out":2341,"duration_ms":55555,"temperature":0.7,"pith_summary":"The paper sets out to show that a transformer encoder originally built for 2D semantic segmentation can be repurposed into a state-of-the-art saliency model for 360-degree video. The authors pair the SegFormer encoder with a custom decoder and an adaptive viewing-center bias, reporting Pearson correlation coefficients of 0.722 on Sport360, 0.807 on PVS-HM, and 0.593 on VR-EyeTracking—improvements of 8.4%, 2.5%, and 18.6% over the best previously reported values. The model is lightweight, with 3.70M parameters and a throughput of about 196 frames per second, which matters because accurate saliency maps are a key input to viewport prediction and thus to efficient VR streaming. The paper also extends two datasets by converting head-orientation data into ground-truth saliency maps, making them usable for training and evaluation.","feed_headline":"Transformer tops 360-video saliency with 18.6% CC gain","feed_subtitle":"At 3.7M parameters and ~196 fps, it could power real-time viewport prediction on headsets.","key_machinery":"The key machinery is the SegFormer MiT-B0 encoder, a hierarchical transformer modified to accept a six-channel input formed by concatenating the current and t-5 frames. A custom decoder of convolutions, batch norm, ReLU, upsampling, and a sigmoid head produces an initial saliency map. A viewing center bias, computed as the average first-frame saliency of training videos, is fused via S = w_t·CB + (1-w_t)·S_init, with w_t = (1-β)·exp(-α(t/C)^2) + β, where α and β are learnable per dataset. Training uses a four-part loss combining Pearson correlation, KL divergence, spherical mean squared error with latitudinal weighting, and binary cross-entropy.","core_discovery":"The central claim is that a transformer encoder trained for segmentation can be transferred to 360-degree video saliency estimation with a small custom decoder and a viewing-bias prior, achieving top results on all three largest public benchmarks. SalFormer360 takes the current frame and a frame five time steps earlier, concatenates them into a six-channel input, and runs them through the fine-tuned SegFormer MiT-B0 encoder. The decoder upsamples the final feature map to produce an initial saliency map, which is blended with a dataset-specific viewing center bias whose weight decays exponentially over time with learnable parameters. Ablations show that both the time-decaying and the persiste","pith_inferences":["If evaluation protocols were fully aligned—same sampling scheme and same ground-truth generation kernel—the reported margins, especially the 2.5% gain on PVS-HM, could shrink; the comparison against 360Spred on PVS-HM is the one to scrutinize.","The architecture implies that a single frame pair (t and t-5) carries enough temporal signal for 360-degree saliency; testing with larger frame gaps or with optical flow as an extra input would show whether more motion information adds anything.","The per-dataset learned bias parameters are a compact diagnostic of viewing behavior: Sport360's beta near zero indicates almost no persistent center bias, while PVS-HM's higher beta reflects stronger sustained center fixation.","A testable extension is to apply the same encoder-decoder and learnable center bias to single 360-degree images, isolating how much of the gain comes from the temporal input versus the bias mechanism."],"forward_implications":["Saliency maps accurate enough to aid viewport prediction can be produced in real time on client devices, given the model's 3.7M parameters and ~196 fps throughput.","The successful transfer from segmentation to saliency suggests that other segmentation backbones may serve as strong starting points for attention prediction, potentially reducing training data requirements.","The learnable center-bias formulation provides a compact way to encode dataset-specific viewing tendencies, which could be injected into other 360-degree models.","The released extended datasets for PVS-HM and VR-EyeTracking give the community standardized splits and ground-truth saliency maps for training and evaluation.","The four-part loss with spherical weighting offers a training recipe that balances correlation, distribution, point accuracy, and geometry."],"fun_headline_variants":["Transformer model boosts 360-video saliency by 18.6%","SegFormer-based net tops 360 saliency with 18.6% gain","360-video saliency: transformer encoder beats SOTA by 18.6%","SalFormer360: transformer saliency model for 360 videos","Viewing bias boosts transformer saliency for 360 videos"],"cache_read_input_tokens":19328,"weakest_assumption_plain":"The reported superiority rests on cross-paper baseline comparisons whose evaluation conditions are not fully aligned: the previous best on PVS-HM was measured with uniform spherical sampling while SalFormer360 uses sinusoidal weighting, and the ground-truth saliency maps for PVS-HM and VR-EyeTracking were generated with a hand-chosen 7-degree Gaussian kernel that may differ from kernels used by prior methods.","fun_headline_variants_meta":{"raw":{"variants":["Transformer model boosts 360-video saliency by 18.6%","SegFormer-based net tops 360 saliency with 18.6% gain","360-video saliency: transformer encoder beats SOTA by 18.6%","SalFormer360: transformer saliency model for 360 videos","Viewing bias boosts transformer saliency for 360 videos"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001195,"raw_usage":{"total_tokens":4759,"prompt_tokens":730,"completion_tokens":4029,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":3932}},"tokens_in":474,"tokens_out":4029,"duration_ms":28665,"temperature":1.0,"reasoning_tokens":3932,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T04:31:18.337696+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run all baseline methods under a single protocol: identical ground-truth saliency maps (same Gaussian kernel and fixation-to-map pipeline), identical evaluation sampling (same sinusoidal weighting), and identical train/test splits. If, under that protocol, 360Spred or another prior method achieves a CC above SalFormer360's 0.807 on PVS-HM or above 0.722 on Sport360, then the paper's central claim of state-of-the-art performance is contradicted.","supporting_citations":[],"review_version":1}