{"id":"591a4648-c253-48d8-a52b-08c1c55fa7d7","arxiv_id":"2603.28759","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":7,"one_line_summary":"FlowIt initializes optical flow with hierarchical-transformer global matching cast as optimal transport, then refines with confidence- and occlusion-guided updates, claiming SOTA Sintel and strong zero-shot generalization.","lead":"FlowIt is a new optical flow network that starts with global matching via hierarchical transformers and optimal transport, then refines using confidence and occlusion maps. It reports top Sintel scores and strong zero-shot transfer to Spring and LayeredFlow, which matters for video, robotics, and autonomous driving systems that need reliable motion under large displacements and hard surfaces.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Ablations leave OT+guidance gains entangled with capacity and TartanAir/S2M2 pretraining; SOTA claims rest on incompletely isolated factors.","rationale":"The reader correctly flags that the weakest assumption is the causal role of Sinkhorn OT maps + guided refinement versus capacity/pretraining confounds, and that Table 4 only partially isolates them. My stress-test confirms the same soft spot is load-bearing for the SOTA claim: the only controlled ablations sit on FlyingChairs with the small model, while the numbers that justify “state-of-the-art” and “new zero-shot SOTA” come from large models that also enjoy TartanAir pretraining and S2M2 weight transfer. Table S1 already shows a non-trivial TartanAir effect; capacity scaling in Table 4 is larger than the OT or guidance deltas. No other internal inconsistency appears (architecture, losses, and evaluation protocol are standard and coherent). Therefore the verdict remains CONDITIONAL—accept-shaped once independent reimplementation or the proposed full-schedule ablations confirm that OT+guidance still drive the gains after capacity and pretraining are controlled. No stronger rejection is warranted; the paper is a solid empirical systems contribution whose main claim simply needs tighter isolation.","tokens_in":20660,"tokens_out":720,"duration_ms":6046,"concrete_test":"Retrain the (L) or (XL) model under the full C+T+S+K+H schedule (and the TartanAir-pretrained path) with three controlled ablations: (i) softmax initialization instead of Sinkhorn OT, (ii) refinement without O/Γ conditioning, (iii) RAFT-style flow-only loss. Evaluate zero-shot on Sintel train Clean/Final and Spring train 1px/EPE. If the OT+guidance configuration no longer leads by a clear margin (or the gap shrinks below the capacity-induced gap), the causal attribution in the strongest claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim attributes Sintel SOTA (Clean 0.85 / Final 1.84) and zero-shot gains primarily to hierarchical-transformer global matching + Sinkhorn OT initialization (with dustbin-derived O0/Γ0) + confidence/occlusion-guided residual updates (§3.2–3.4). Table 4 isolates pieces only on FlyingChairs val with the (S) model: OT vs softmax (0.617 vs 0.631 EPE), guidance vs none (0.617 vs 0.708), and the joint loss. These deltas are small, measured far from the Sintel/KITTI/Spring/LayeredFlow regimes that support the headline numbers, and never re-run after TartanAir pretraining or S2M2 weight transfer (explicitly used for compatible layers, §4.2). Table S1 shows that removing TartanAir already moves XL Clean from 0.85 to 0.90 and Final from 1.84 to 2.12, while larger capacity alone (S\to XL) produces larger EPE drops than OT or guidance. Consequently it remains under-determined whether the OT-derived maps and their guided propagation are the main causal drivers of the reported SOTA, or whether capacity + pretraining dominate.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"FlowIt is a two-frame optical flow architecture that extracts multi-scale CNN+FPN features, processes them with Multi-Resolution Transformer (MRT) blocks for hierarchical global context, builds a 1/4-resolution all-pairs correlation volume, and obtains an initial flow via entropy-regularized optimal transport (Sinkhorn with dustbins). The OT plan also yields initial occlusion and confidence maps that condition a RAFT-style iterative residual refinement (axis-wise u/v heads, logit-space accumulation for O and Γ, convex upsampling). Supervision combines Smooth-L1/L1 flow losses with L1 losses on forward–backward occlusion pseudo-labels and EPE-threshold confidence pseudo-labels. Four capacity variants (S/M/L/XL) are trained with a TartanAir\to C\to T\to C+T+S+K+H schedule (plus optional KITTI fine-tune) and evaluated on Sintel/KITTI benchmarks and zero-shot Sintel-train, KITTI-train, Spring, and LayeredFlow. The XL model reports Sintel test Clean 0.85 / Final 1.84 and strong zero-shot numbers; ablations on FlyingChairs isolate MRT, OT vs softmax, conf/occ guidance, refinement steps, axis-wise heads, and the joint loss.","tokens_in":21081,"tokens_out":1452,"duration_ms":11496,"significance":"If the gains are real, the paper supplies a practical and competitive global-matching alternative to the dominant local iterative family (RAFT/SEA-RAFT/WAFT). The explicit OT-derived occlusion and confidence maps that both initialize and guide residual updates are a clean architectural idea, and the multi-variant design plus public-benchmark and zero-shot tables (including the difficult LayeredFlow non-Lambertian setting) make the empirical contribution usable by the community. The work is incremental rather than paradigm-shifting, but the combination of hierarchical transformers, OT initialization, and guided multi-output refinement is well-motivated and the reported Sintel and cross-dataset numbers would be of clear interest if the causal role of the OT+guidance components is better isolated.","major_comments":[{"comment":"The central causal claim—that Sinkhorn OT initialization plus conf/occ-guided refinement is the main driver of Sintel SOTA and zero-shot gains—is only weakly isolated. Table 4 measures OT vs softmax (0.617 vs 0.631 EPE) and guidance vs none (0.617 vs 0.708) solely on FlyingChairs val with the (S) model; the deltas are small and never re-run after TartanAir pretraining or S2M2 weight transfer (§4.2). Table S1 shows that removing TartanAir alone moves XL Clean from 0.85 to 0.90 and Final from 1.84 to 2.12, while capacity scaling (S\to XL) produces larger EPE drops than OT or guidance. A load-bearing ablation on at least one target regime (Sintel train zero-shot or Spring) that holds capacity and pretraining fixed is required to support the attribution in the abstract and §1/§5.","section":null},{"comment":"SOTA language is uneven across tables. On KITTI test (Table 1) XL is best only on Non-Occ and trails WAFT-DAv2-a2 and multi-frame methods on Fl-all; on KITTI train zero-shot (Table 2) Fl-all is worse than several WAFT and FlowSeek variants. The abstract and conclusion should qualify “competitive on KITTI” more carefully and avoid implying uniform leadership. Likewise, multi-frame methods in block (A) of Table 1 are not a fair peer group for the two-frame claim; the ranking narrative should keep them clearly separated.","section":null},{"comment":"§3.2–3.4 and Eqs. (4)–(5), (9)–(11): the quality of the OT-derived O0/Γ0 maps is never validated against the pseudo-labels used for supervision (forward–backward consistency and EPE < 4 px). Without even a simple correlation or IoU check, it remains possible that the refinement network largely ignores these maps and that the reported gains come from capacity and pretraining. A short quantitative check (or an ablation that freezes/randomizes the maps at inference) would make the guided-propagation story falsifiable.","section":null}],"minor_comments":[{"comment":"Figure 3 caption and surrounding text claim the correlation volume is “notably smoother” than SEA-RAFT; a quantitative smoothness or peakiness statistic would make the visual comparison more convincing.","section":null},{"comment":"Hyper-parameters (r=2, ε=10−6, λ F=1, λ Γ=λ O=0.1, three refinement steps, EPE threshold 4 px, FB threshold 2 px) are stated but never ablated beyond the coarse Table 4; a short sensitivity note would help reproducibility.","section":null},{"comment":"Notation: ΓΓΓ and O are used both for maps and for their residual updates; a clearer distinction (e.g., Γ t vs ΔΓ t) would reduce ambiguity in Eqs. (6)–(8).","section":null},{"comment":"Supplementary Table S1 is valuable; the main text should explicitly point readers to the “without TartanAir” rows when discussing pretraining impact.","section":null},{"comment":"Latency and GMAC numbers in Table 4 are measured on L40S for the (S) model only; reporting the same for XL on a common GPU would better support the efficiency claims made for the smaller variants.","section":null}],"recommendation":"major_revision","confidential_remarks":"The architecture is a competent synthesis of existing pieces (MRT from stereo S2M2, Sinkhorn OT, RAFT-style refinement). The main risk is over-claiming novelty and causality for OT+guidance when capacity and TartanAir/S2M2 transfer appear to do most of the heavy lifting. If the authors supply the requested target-domain ablations and tone down the SOTA language on KITTI, the paper is publishable; without them the contribution is hard to distinguish from a well-engineered capacity/pretraining study."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"FlowIt is a clean, well-executed systems paper in optical flow. The punchline is that the specific assembly works: hierarchical multi-resolution transformers for global context, Sinkhorn OT over the 1/4 all-pairs volume (with dustbins) for a robust initial flow plus explicit occlusion and confidence maps, then RAFT-style residual updates that are conditioned on those maps and supervised jointly. XL hits 0.85/1.84 on Sintel Clean/Final and posts new zero-shot highs on Sintel train, Spring, and LayeredFlow while staying competitive on KITTI. Those numbers are not vapor; they sit on public benchmarks with the usual training schedule plus TartanAir and partial S2M2 weight transfer.\n\nWhat is actually new is the packaging for flow, not the ingredients. MRT-style cross-scale transformers and OT matching with dustbins already exist in stereo (S2M2); iterative refinement and convex upsampling are RAFT lineage. The useful move is treating the OT plan as a source of usable O0/Γ0 that then guide propagation into ambiguous regions, plus the axis-wise heads and the joint loss that actually supervises those maps. The qualitative results on LayeredFlow and fine motion look better than the recent WAFT/SEA-RAFT baselines for a reason.\n\nSoft spots are real but proportionate. Table 4 isolates OT vs softmax and guidance vs none only on FlyingChairs with the small model; the deltas are modest and never re-run after TartanAir or S2M2 init. Table S1 shows removing TartanAir already moves XL Clean from 0.85 to 0.90 and Final from 1.84 to 2.12, and capacity alone (S→XL) produces larger drops than the OT or guidance ablations. So the stress-test concern lands: the abstract attributes the SOTA primarily to OT+guidance, yet those factors remain under-isolated from capacity and pretraining. That does not erase the results; it just means the causal story is softer than claimed. No released code is a practical annoyance, not a scientific flaw.\n\nMath and losses are standard and transparent; citation pattern engages the right recent work without obvious gaps. This is for people who build or use two-frame flow and care about Sintel-level accuracy plus cross-dataset robustness. It deserves a serious referee. I would engage with it, cite the numbers if I am working in the area, and expect the isolation questions to be settled by reimplementation rather than by desk rejection.","headline":"Solid systems paper: hierarchical transformers + OT init + conf/occ-guided refinement deliver real Sintel SOTA and strong zero-shot numbers; ablations leave the OT piece partly entangled with capacity and pretraining.","tokens_in":21668,"tokens_out":627,"would_cite":true,"duration_ms":14636,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"FlowIt turns optical flow into a global matching problem solved by optimal transport, then refines with the confidence and occlusion maps that transport produces, setting new Sintel and zero-shot records.","keywords":["optical flow","global matching","optimal transport","hierarchical transformer","occlusion estimation","confidence-guided refinement","zero-shot generalization"],"falsifier":"Re-train the identical architecture but replace the Sinkhorn step with ordinary softmax matching (keeping every other weight, loss term and pre-training schedule fixed) and check whether the Sintel Clean / Final and zero-shot Spring / LayeredFlow numbers collapse back to the softmax baseline levels reported in the ablation table.","tokens_in":21542,"feed_emoji":"🌊","tokens_out":903,"duration_ms":9712,"temperature":0.7,"pith_summary":"Optical flow is the dense 2-D motion field between consecutive video frames. Most recent networks still search only locally, so they struggle with large displacements, occlusions and textureless regions. FlowIt instead builds a hierarchical transformer that keeps global context at quarter resolution, casts the initial correspondence problem as entropy-regularized optimal transport (with dustbins for unmatched pixels), and obtains from that transport plan not only a flow field but also explicit occlusion and confidence maps. A subsequent refinement stage is conditioned on those maps so that reliable motion is deliberately propagated into ambiguous areas. The paper shows that this combination yields state-of-the-art numbers on the Sintel benchmark and the strongest published zero-shot cross-dataset results on Sintel, Spring and LayeredFlow, while remaining competitive on KITTI. Multiple model sizes let the same idea trade accuracy for compute.","feed_headline":"Global matching plus optimal transport resets optical-flow records","feed_subtitle":"Transport plans supply flow, occlusion and confidence; guided refinement sets new Sintel and zero-shot marks","key_machinery":"Optimal-transport flow initialization: the 4-D correlation volume is turned into a soft mutual-consistency transport plan via entropy-regularized Sinkhorn (with dustbins). Marginal sums of that plan become the occlusion map; windowed probability mass around the peak becomes the confidence map; both maps then guide residual updates.","core_discovery":"A hierarchical transformer that produces multi-scale features, followed by Sinkhorn optimal transport over the all-pairs correlation volume at 1/4 resolution, yields a robust initial flow together with usable occlusion and confidence maps; conditioning a short RAFT-style residual refinement on those maps, and supervising them jointly, produces more accurate and more generalizable optical flow than local-search or pure-transformer baselines.","pith_inferences":["The same transport-plus-guided-refinement pattern should transfer almost unchanged to stereo or multi-view matching, where mutual consistency and occlusion reasoning are equally central.","Because the confidence map is already produced at every iteration, it could be exposed at test time for uncertainty-aware downstream modules (e.g., robust odometry or selective key-frame selection) without any extra training.","The residual gap on KITTI relative to multi-frame methods suggests that adding a lightweight temporal memory on top of the same transport initialization could close the remaining real-world gap."],"forward_implications":["Global matching need not be abandoned for memory reasons; a hierarchical transformer plus optimal transport at 1/4 resolution is already competitive at full Sintel resolution.","Explicit occlusion and confidence maps derived from transport can be used as free auxiliary signals to steer iterative refinement without extra network heads.","The same pipeline produces four scalable variants (S–XL) that trade parameters for accuracy, giving practitioners a concrete accuracy–compute curve.","Zero-shot gains on Spring and LayeredFlow (transparent / reflective surfaces) indicate that mutual-consistency transport helps more than local search when photometric assumptions break."],"fun_headline_variants":["Hierarchical transformers and optimal transport for optical flow","FlowIt: global matching via transformers plus Sinkhorn OT","OT over multi-scale features yields flow occlusion and confidence","Guided residual refinement from OT maps improves optical flow","Global context matching with optimal transport for flow estimation"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That the occlusion and confidence maps extracted from Sinkhorn transport are accurate enough, and informative enough when fed into the residual updates, to be the main reason for the reported gains rather than model size or pre-training alone.","fun_headline_variants_meta":{"raw":{"variants":["Hierarchical transformers and optimal transport for optical flow","FlowIt: global matching via transformers plus Sinkhorn OT","OT over multi-scale features yields flow occlusion and confidence","Guided residual refinement from OT maps improves optical flow","Global context matching with optimal transport for flow estimation"]},"model":"grok-4.5","effort":"low","cost_usd":0.007132,"raw_usage":{"total_tokens":1748,"prompt_tokens":743,"num_sources_used":0,"completion_tokens":77,"cost_in_usd_ticks":71320000,"prompt_tokens_details":{"text_tokens":743,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":928,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":743,"tokens_out":77,"duration_ms":8838,"temperature":1.0,"reasoning_tokens":928,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T16:07:23.226003+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-train the identical architecture but replace the Sinkhorn step with ordinary softmax matching (keeping every other weight, loss term and pre-training schedule fixed) and check whether the Sintel Clean / Final and zero-shot Spring / LayeredFlow numbers collapse back to the softmax baseline levels reported in the ablation table.","supporting_citations":[],"review_version":1}