{"id":"3ec08ba3-e128-4ae0-b7f8-ea5627087501","arxiv_id":"2506.23151","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"MEMFOF achieves state-of-the-art optical flow on Spring, Sintel, and KITTI while using only 2.09 GB of GPU memory at 1080p inference, enabling native high-resolution processing.","lead":"MEMFOF is a new optical flow method that uses three video frames and a redesigned correlation module to cut GPU memory to 2.09 GB at 1080p, about four times less than similar models. A smart generalist might read it because it suggests high-resolution motion estimation could run on ordinary graphics cards without losing accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"State-of-the-art claims rely on curated leaderboard comparisons; official KITTI/Sintel/Spring leaderboard verification is needed before 'best Fl-all' and 'leads Sintel' can be accepted.","rationale":"The reader's weakest_assumption focused on the 2x-upsampling training strategy. That concern is reasonable, but it is partially mitigated by the paper's strong zero-shot result on Spring (1px 3.600, better than several fine-tuned methods), which suggests the upsampled pretraining transfers to real high-resolution data. The more immediately load-bearing issue is that the headline SOTA claims are supported only by curated comparison tables. The memory-efficiency claim (2.09 GB at 1080p inference, 28.5 GB training) is tied to a specific architecture and reproducible from the released code; it is internally consistent and plausible. However, the 'state-of-the-art' claim is an external, factual assertion about leaderboard standings at submission time. Tables 2 and 3 omit multiple well-known high-accuracy methods that are cited in the paper itself (FlowFormer++, DIP, GMFlow), so the reader cannot verify whether MEMFOF truly leads on KITTI and Sintel. This is a correctness risk, not a matter of consensus: if any omitted method has a better metric, the abstract's claims are false. The proposed concrete test (checking official leaderboard snapshots) would settle this definitively. If the leaderboard check passes, the paper's claims stand and the manuscript is largely sound; if it fails, the claims must be narrowed. Therefore the verdict remains conditional: the paper should be accepted only after the official leaderboard standings are verified and the comparison tables are either expanded or the claims qualified. This does not change the reader's conditional verdict, but it identifies a different and more decisive soft spot than the upsampling concern.","tokens_in":14830,"tokens_out":19355,"duration_ms":201349,"concrete_test":"Query the official KITTI-2015, Sintel, and Spring leaderboards as of 29 Jun 2025 (e.g., via the Wayback Machine) and enumerate all submissions with Fl-all < 2.94% (KITTI-2015), EPE < 0.963 (Sintel clean), and 1px < 3.289 (Spring). If any such method exists, the corresponding 'best'/'leads' claims must be revised or qualified. If no such method exists, the SOTA claims stand as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that MEMFOF 'achieves state-of-the-art accuracy' on Spring, Sintel clean, and KITTI-2015 is supported only by Tables 2 and 3, which compare against a curated subset of methods. Several strong optical flow models cited elsewhere in the paper (e.g., FlowFormer++, DIP, GMFlow) are absent from these tables. In particular, the KITTI-2015 Fl-all of 2.94% is asserted as 'the best Fl-all error', yet the official KITTI-2015 leaderboard contains many submissions not listed; if any published method has Fl-all below 2.94%, the abstract's claim is false. Similarly, the Sintel clean EPE of 0.963 is asserted as 'leads', but not all leading submissions are tabulated, so this claim is not verifiable from the manuscript. Because the 'without sacrificing accuracy' part of the central claim rests directly on these benchmark numbers, the incomplete comparison is load-bearing. The memory-efficiency and architectural claims are reproducible from the released code and are not the primary risk.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MEMFOF, a multi-frame optical flow method that extends SEA-RAFT to three frames with bidirectional flow prediction, reduces the correlation volume to 1/16 resolution, increases feature dimensions, adds a GMA module, and trains on 2x-upsampled datasets with full-frame crops. The central claims are a low GPU memory footprint (2.09 GB at 1080p inference, 28.5 GB for training) and state-of-the-art accuracy on Spring, Sintel clean, and KITTI-2015 benchmarks. The paper includes systematic ablations of training resolution, frame count, correlation volume resolution, and inference-time optimizations, and releases code.","tokens_in":15103,"tokens_out":9397,"duration_ms":84500,"significance":"If the reported memory and accuracy numbers hold, MEMFOF addresses a practical bottleneck: multi-frame optical flow at native FullHD resolution with memory consumption within consumer GPU limits. The training strategy of upsampling standard datasets to better match high-resolution motion distributions is well motivated by the histogram analysis and supported by the ablations. The paper's strength is its systematic empirical evaluation: Tables 4, 5, 6, and 9 cover the key design choices, and the memory reduction is corroborated by the architecture change (1/16 correlation volume) plus the ablation comparison. The code release and official leaderboard use add to reproducibility. However, the state-of-the-art claims depend on the completeness of the benchmark comparisons, which is the main risk.","major_comments":[{"comment":"The state-of-the-art claims are not fully supported by the tabulated comparisons. The abstract and Section 1 claim that MEMFOF 'achieves state-of-the-art performance' and 'leads' Sintel clean with EPE 0.963 and achieves the 'best Fl-all error' on KITTI-2015 at 2.94%, but Tables 2 and 3 list only a curated subset of methods. Methods cited elsewhere in the paper, such as FlowFormer++, DIP, and GMFlow, are absent from both tables. The official Spring, Sintel, and KITTI leaderboards contain far more submissions, so the assertions of 'first', 'leads', and 'best' are not verifiable from the manuscript alone. This is load-bearing because the 'without sacrificing accuracy' part of the central contribution rests directly on these benchmark numbers. Please either include all published results for the listed benchmarks (or explicitly state that the rankings refer to the leaderboard at the time of submission and cite the leaderboard entries), or qualify the claims to be relative to the methods actually compared.","section":"4.2, Tables 2 and 3"},{"comment":"The memory and runtime numbers are reported as single measurements without a description of the measurement protocol, variance, or number of runs. For example, the headline '2.09 GB' inference memory and '28.5 GB' training memory are cited in the abstract, but the paper does not state whether these are peak allocated memory, reserved memory, or measured with torch.cuda.max_memory_allocated, nor does it indicate how many repetitions were averaged. Since the memory-efficiency claim is a central contribution, the measurement methodology should be specified so the results are reproducible; reporting at least the protocol and ideally mean+-std across runs would strengthen the paper.","section":"4.1 and Tables 1, 2, 5, 6"},{"comment":"The claim that training is performed 'without the need for cropping or downsampling' is contradicted by Table 1. The table lists explicit crop sizes for every training stage (e.g., [480,960] for TartanAir, [864,1920] for Things and TSKH, [872,1920] for Sintel-ft, [750,1920] for KITTI-ft); only the Spring-ft stage uses [1080,1920], which matches a full 1080p frame. Thus the full-frame, no-crop training applies only to the final fine-tuning stage. Please reword the abstract and Section 3.2 to accurately describe that the method can fine-tune at native 1080p without cropping or downsampling, while the pre-training stages still use crops.","section":"Abstract, 3.2, and Table 1"}],"minor_comments":[{"comment":"Typo: 'Plese refer' should be 'Please refer'.","section":"4.3, Figure 4 caption"},{"comment":"The metric is called 'Fl-score' in the text, while Table 3 uses 'Fl-all' and the KITTI benchmark uses 'Fl-all'; please use consistent terminology and define the relationship.","section":"4.1, evaluation metrics"},{"comment":"Typo: 'sumbissions' should be 'submissions'.","section":"Table 8 caption"},{"comment":"Typo: 'identifer' should be 'identifier'.","section":"6. Acknowledgments"},{"comment":"The strided convolution applied to the 1/8-resolution features is not fully specified; please provide the kernel size, stride, and output channel details, since the feature dimension increase (Df from 256 to 1024) is a key part of the design.","section":"3.2"},{"comment":"The context network is described only by its inputs and outputs; a brief architectural description (or an explicit reference to the SEA-RAFT context network) would help readers understand the three-frame extension.","section":"3.1, context network"},{"comment":"The statement that this is the 'first multi-frame optical flow method designed for memory efficiency at FullHD' is a strong novelty claim; given that StreamFlow also focuses on streamlined multi-frame processing, it would be more accurate to say 'to our knowledge, the first multi-frame method to operate at FullHD with a few GB of memory', with a discussion of how MEMFOF differs.","section":"1 or 4.2"},{"comment":"For Sintel and KITTI, the paper states that inputs are bilinearly upscaled by 2x and outputs downscaled by 2x; please discuss whether this pre-processing could put MEMFOF at an advantage or disadvantage relative to methods operating at native resolution, and whether the reported memory numbers apply to the upscaled inference or native resolution.","section":"4.2, KITTI/Sintel protocol"}],"recommendation":"major_revision","confidential_remarks":"The central technical contribution appears sound and the ablations are thorough. The main risk is the over-claiming of state-of-the-art status based on an incomplete comparison set; verificating the official leaderboard rankings or adding missing baselines should be a condition for acceptance. The code release and detailed ablations are definite strengths. The 'no cropping' claim in the abstract should also be corrected to match Table 1."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front. MEMFOF delivers a genuine engineering result: 2.09 GB inference memory at 1080p with three-frame input, versus 8+ GB for SEA-RAFT and about 19 GB for StreamFlow. The reduction comes from a 1/16-resolution correlation volume with wider features, and the ablations show the trade-offs clearly. Second, the state-of-the-art claims in the abstract are not actually backed by the tables. Tables 2 and 3 compare against a curated subset. FlowFormer++, DIP, GMFlow, and CroCo v2 are cited in the text but absent from the benchmark comparisons, so \"leads Sintel\" and \"best Fl-all on KITTI\" cannot be verified from this manuscript. That gap is load-bearing, not cosmetic.\n\nWhat the paper does well: the memory-efficiency analysis is concrete and reproducible. The 2x upsampled training is a simple, effective fix for the motion-magnitude mismatch between standard datasets and FullHD, and the motion histograms in Figure 4 are a clear way to show the problem. The ablations isolate correlation volume scale, frame count, feature dimension, and the GMA module, and the code is released. That is real evidence.\n\nSoft spots, in proportion. The \"first multi-frame memory-efficient method\" claim is overstated given StreamFlow's stated focus on spatiotemporal efficiency, but that is a minor rhetorical point. The memory and runtime numbers are self-measured on one RTX 3090 without repeated trials, so treat them as approximate. The upsampled-training assumption is tested only on Spring train; it may transfer to other high-res domains, but that is not proven. These are all minor compared with the comparison-table gap.\n\nBottom line: this is a useful paper with a real result, and it deserves a serious referee. The authors should either expand the benchmark tables to include all leading published methods or temper the abstract's SOTA claims. I would cite it for the memory-efficiency design and would bring it to reading group. Recommend: send to peer review, with the comparison issue as the main revision point.","headline":"Solid memory-efficiency engineering with a real 2.09 GB result, but the SOTA claims need a fuller comparison table before they hold.","tokens_in":15592,"tokens_out":4485,"would_cite":true,"duration_ms":41492,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MEMFOF claims a three-frame optical flow model that runs at native 1080p on 2.09 GB of GPU memory and reaches state-of-the-art accuracy on Spring, Sintel clean, and KITTI-2015.","keywords":["optical flow","multi-frame optical flow","memory-efficient inference","high-resolution training","1080p","RAFT-style architectures","correlation volume","benchmark evaluation"],"falsifier":"Evaluate the 2x-upsampled-trained MEMFOF on a ground-truth optical flow video set whose motion is shifted toward very large displacements, such as fast panning FullHD footage, and compare its 1px and EPE against a model trained on native-resolution data; if the gap over native training shrinks or reverses, the motion-distribution-transfer assumption underpinning the training strategy would be falsified.","tokens_in":14660,"feed_emoji":"🎬","tokens_out":5788,"duration_ms":59398,"temperature":0.7,"pith_summary":"Optical flow, the per-pixel motion between video frames, normally becomes memory-hungry at high resolution because RAFT-style correlation volumes grow quadratically with image size, so many methods downsample or crop to fit. MEMFOF claims a better trade-off: by computing correlation volumes at 1/16 resolution instead of 1/8 and processing three frames with bidirectional flows, it cuts inference memory to 2.09 GB at 1080p while improving accuracy. The authors also argue that training on 2x-bilinearly-upsampled versions of standard datasets, on full frames rather than crops, aligns training motion magnitudes with real FullHD video and prevents underfitting on large motions. If correct, this would be the first multi-frame flow method that combines native FullHD training and runtime with state-of-the-art results on Spring, Sintel clean, and KITTI-2015, reporting 1px 3.289 on Spring, EPE 0.963 on Sintel clean, and Fl-all 2.94% on KITTI-2015.","feed_headline":"FullHD optical flow in 2 GB of GPU memory tops three benchmarks","feed_subtitle":"Multi-frame MEMFOF runs at native 1080p without cropping and beats larger models on Spring, Sintel clean, and KITTI-2015.","key_machinery":"The load-bearing mechanism is the reduced-resolution dual correlation volume: two all-pairs correlation volumes at 1/16 input resolution instead of the standard 1/8, which cuts the memory of the two volumes from about 10.4 GB to 0.65 GB at 1080p, with feature dimension raised from 256 to 1024 and update-block dimension raised from 128 to 512 to preserve information. Around this sit three-frame bidirectional flow estimation, in which flows from the central frame to both neighbors are refined jointly, a training protocol that 2x-upsamples datasets and trains on full 1080p frames, and inference-time reuse of feature maps and correlation volumes across consecutive video frames.","core_discovery":"On the paper's own terms, the central discovery is that a RAFT-style multi-frame optical flow architecture can be made memory-efficient enough to run and train at native 1080p without sacrificing accuracy. Reducing the correlation volume resolution from 1/8 to 1/16, widening feature channels to compensate, and extending SEA-RAFT to estimate bidirectional flows from a triplet of frames cuts runtime memory to 2.09 GB and training memory to 28.5 GB. Combined with a training protocol that bilinearly upsamples existing datasets by 2x and trains on full frames, this yields state-of-the-art results: first place on Spring with a 1px outlier rate of 3.289, EPE 0.963 on Sintel clean, and Fl-all 2.94% on KITTI-2015, outperforming more memory-hungry two-frame and multi-frame methods. The paper attributes the gains to aligning training motion statistics with the FullHD domain and to temporal coherence from three-frame bidirectional processing.","pith_inferences":["The memory-saving recipe of 1/16 correlation volumes with wider features is separable from the three-frame idea, so dropping it into any RAFT-style two-frame model would make the trade-off independently testable.","The authors observe that accuracy degrades when moving from 3 to 5 frames and attribute this to context and update network capacity, which suggests a testable extension: scale those modules with temporal width to see whether longer temporal context helps.","The 2x upsampling training trick addresses a distribution mismatch that also affects other high-resolution dense prediction tasks, so it may transfer to stereo matching or video frame interpolation, which face the same low-resolution-to-FullHD gap."],"forward_implications":["At 2.09 GB inference memory, MEMFOF can run at native 1080p on consumer GPUs, removing the need to downsample or tile high-resolution inputs.","Training at native 1080p without cropping lets the model see full-frame global motion context during training, not just local crops.","Videos can be processed more cheaply by caching and reusing feature maps and correlation volumes across overlapping frame pairs.","Memory efficiency in the RAFT family does not have to come at the cost of accuracy, since MEMFOF beats heavier multi-frame methods on the reported benchmarks.","The simple training-only change of upsampling low-resolution data by 2x is a recipe other optical flow methods could adopt to handle large motions at high resolution."],"supporting_citations":[{"why":"Supplies the iterative all-pairs correlation and recurrent refinement paradigm whose memory scaling MEMFOF targets.","marker":"[31]"},{"why":"Provides the two-frame SEA-RAFT baseline, training protocol, and Mixture-of-Laplace loss that MEMFOF extends to three frames.","marker":"[32]"},{"why":"Provides the tri-frame bidirectional flow formulation that MEMFOF adapts.","marker":"[26]"},{"why":"Contributes the global motion aggregation module that MEMFOF reintroduces with a resolution-adapted scale factor.","marker":"[13]"},{"why":"A multi-frame memory-based method used as a comparison and as the source of the attention scale-factor modification.","marker":"[4]"},{"why":"A multi-frame method whose cached feature and correlation reuse motivates MEMFOF's inference-time optimizations and memory comparison.","marker":"[30]"},{"why":"The high-resolution Spring benchmark on which MEMFOF ranks first and on which the training ablations are run.","marker":"[18]"},{"why":"The Sintel benchmark used for EPE evaluation where MEMFOF reports the clean-pass lead.","marker":"[2]"},{"why":"The KITTI-2015 benchmark used for Fl-all evaluation where MEMFOF reports the best result among non-scene-flow methods.","marker":"[19]"}],"fun_headline_variants":["FullHD optical flow in 2 GB GPU memory, first on Spring","Native 1080p training, 2 GB inference: optical flow tops 3 benchmarks","Multi-frame optical flow: 2.09 GB memory for 1080p, best on 3 benchmarks","2.09 GB runs triple-frame flow at 1080p, beats larger models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that bilinearly upsampling low-resolution training videos by 2x makes their motion statistics similar enough to real FullHD video that the model learns to handle FullHD motion; if real high-resolution motion lies outside that upsampled range, the accuracy gains may not carry over to other high-resolution domains.","fun_headline_variants_meta":{"raw":{"variants":["FullHD optical flow in 2 GB GPU memory, first on Spring","Native 1080p training, 2 GB inference: optical flow tops 3 benchmarks","Multi-frame optical flow: 2.09 GB memory for 1080p, best on 3 benchmarks","2.09 GB runs triple-frame flow at 1080p, beats larger models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001611,"raw_usage":{"total_tokens":6448,"prompt_tokens":1011,"completion_tokens":5437,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":627,"completion_tokens_details":{"reasoning_tokens":5343}},"tokens_in":627,"tokens_out":5437,"duration_ms":40210,"temperature":1.0,"reasoning_tokens":5343,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:47:37.812341+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the 2x-upsampled-trained MEMFOF on a ground-truth optical flow video set whose motion is shifted toward very large displacements, such as fast panning FullHD footage, and compare its 1px and EPE against a model trained on native-resolution data; if the gap over native training shrinks or reverses, the motion-distribution-transfer assumption underpinning the training strategy would be falsified.","supporting_citations":[{"cited_title":"Videoflow: Exploiting temporal cues for multi-frame optical flow estimation","cited_arxiv_id":null,"evidence_quote":"Provides the tri-frame bidirectional flow formulation that MEMFOF adapts."},{"cited_title":"Learning to estimate hidden motions with global motion aggregation","cited_arxiv_id":null,"evidence_quote":"Contributes the global motion aggregation module that MEMFOF reintroduces with a resolution-adapted scale factor."},{"cited_title":"Memflow: Optical flow esti- mation and prediction with memory","cited_arxiv_id":null,"evidence_quote":"A multi-frame memory-based method used as a comparison and as the source of the attention scale-factor modification."},{"cited_title":"Streamflow: streamlined multi- frame optical flow estimation for video sequences","cited_arxiv_id":null,"evidence_quote":"A multi-frame method whose cached feature and correlation reuse motivates MEMFOF's inference-time optimizations and memory comparison."},{"cited_title":"Spring: A high-resolution high- detail dataset and benchmark for scene flow, optical flow and stereo","cited_arxiv_id":null,"evidence_quote":"The high-resolution Spring benchmark on which MEMFOF ranks first and on which the training ablations are run."},{"cited_title":"A naturalistic open source movie for op- tical flow evaluation","cited_arxiv_id":null,"evidence_quote":"The Sintel benchmark used for EPE evaluation where MEMFOF reports the clean-pass lead."},{"cited_title":"Object scene flow for autonomous vehicles","cited_arxiv_id":null,"evidence_quote":"The KITTI-2015 benchmark used for Fl-all evaluation where MEMFOF reports the best result among non-scene-flow methods."}],"review_version":1}