{"id":"0780380d-8db6-4865-90f0-ba3304319e21","arxiv_id":"2412.09072","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The cross-attention map of a pretrained cross-view completion model is a strong zero-shot dense correspondence cost volume, outperforming encoder/decoder features and diffusion/DINO baselines on HPatches and ETH3D.","lead":"Cross-view completion models like CroCo-v2 are trained to reconstruct a masked view from another view, not to find matching points. This paper shows that the model's cross-attention map already acts as a dense correspondence cost volume, enabling zero-shot matching and competitive multi-frame depth estimation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing specification of the 'dense zoom-in' component makes the headline zero-shot AEPE (9.41) unverifiable; the paper cannot rule out that the reported result comes from the undisclosed post-processing step rather than from CroCo-v2's cross-attention map.","rationale":"The Reader's weakest_assumption is the appearance-vs-geometry claim about what the cross-attention map learns, and this is genuinely the conceptual foundation of the paper: if attention retrieves appearance-similar patches rather than geometrically corresponding ones, the entire analogy and the zero-shot claim collapse. I agree with that assessment and with the CONDITIONAL verdict. However, the single most load-bearing and immediately actionable weakness is different: the reported headline number depends on an undocumented 'dense zoom-in' step. The paper never defines this step, so the central empirical claim (cross-attention maps are excellent zero-shot correspondence estimators) cannot be verified from the text: a reader cannot tell whether the 9.41 AEPE is due to the cross-attention map or to an unspecified post-processing procedure. This is a presentation/internally-specified gap, not a disagreement with consensus, and it directly affects correctness risk. The reader's weakest_assumption focuses on why the attention map might work; the dense-zoom-in gap concerns whether the reported evidence actually demonstrates that it works. Both support CONDITIONAL: the verdict should remain CONDITIONAL rather than ACCEPT (because of the missing specification) and rather than REJECT (because the core comparison, cross-attention vs. encoder/decoder correlations on the same backbone, is internally consistent and the effect size is large, though an honest non-rejection could also be argued if one treats the missing step as merely an omitted implementation detail). My recommendation is CONDITIONAL with a specific condition: specify or release the dense zoom-in component and confirm that the base pipeline (without it) reproduces the 10.85 baseline. I mark partial agreement with the Reader because I share the CONDITIONAL verdict but identify a different, more concrete load-bearing gap.","tokens_in":32157,"tokens_out":3288,"duration_ms":30869,"concrete_test":"Ask the authors to specify the dense zoom-in algorithm precisely (pseudocode, resolution schedule, token windows, any use of image content, any supervision or filtering rules), or release code for ZeroCo. Then independently reproduce the base pipeline of §3.2 (Eq. 7 with 12 cross-attention layers, register-token minimum replacement, average over heads, soft-argmax with τ=1e-4) on HPatches-240 and check whether it recovers AEPE 10.85 (Tab. 7, row I). If the base pipeline does not reproduce 10.85, the missing component is even more serious. If the base pipeline is reproduced, apply the specified dense zoom-in and check that it yields 9.41 without any correspondence supervision or ground-truth information. If the dense zoom-in requires a different procedure at test time than the one used for the ablation, the reported comparison is not well posed.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline zero-shot result (AEPE 9.41 on HPatches-240, Tab. 1) is produced by a pipeline that includes a component called 'dense zoom-in' (Tab. 7: row VII). The ablation shows this component is necessary for the best number: without it, AEPE is 10.85 (row I); with it, 9.41. Yet the main text only says 'incorporating the dense zoom-in approach ... significantly enhances zero-shot performance' (§4.5), and the supplementary never describes it: no algorithm, no resolution schedule, no definition of which tokens or layers it modifies, no mention in §A.1 (zero-shot implementation details) or Fig. 6. This is a load-bearing gap because the paper's central claim is that the cross-attention map itself is a zero-shot correspondence estimator. If 'dense zoom-in' is a post-processing heuristic (e.g., iterative nearest-neighbor propagation, mutual-nearest-neighbor filtering, or a coarse-to-fine zooming schedule that uses the target image or a learned component), then the 9.41 metric cannot be attributed to the geometric content of the cross-attention map without specifying and ablating that component. It is also the kind of missing detail that prevents independent reproduction and makes the SOTA claim (vs. DIFT and SD-DINO) untestable. The reader's verdict correctly identifies the appearance-vs-geometry assumption as a deep conceptual risk; however, the sharpest concrete threat to the central claim is this unspecified component. Note also the zero-shot claim is further weakened by the lack of code release, but the dense zoom-in gap is the specific blocker: without its specification, the difference between the raw cross-attention cost volume and the reported 9.41 cannot be assessed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the CroCo-v2 cross-view completion (CVC) model and argues that the cross-attention maps in its decoder encode dense geometric correspondence more effectively than the encoder or decoder features that prior work (e.g., DUSt3R, MASt3R, CroCo-Flow) has used. The authors propose ZeroCo, a zero-shot inference technique that averages cross-attention maps over decoder layers and enforces reciprocity by running the model on swapped input pairs; they also introduce learning-based variants (ZeroCo-finetuned, ZeroCo-flow, ZeroCo-depth) with lightweight heads for supervised geometric matching and multi-frame depth estimation. The paper reports large improvements over prior zero-shot matchers (DIFT, SD-DINO, DINOv2) on HPatches and ETH3D, and competitive results for learning-based matching and depth, with ablations supporting the use of cross-attention maps.","tokens_in":32484,"tokens_out":5254,"duration_ms":54250,"significance":"If the central claim holds, the paper identifies a previously underused signal in cross-view completion models: the decoder cross-attention map, rather than the descriptor features, is the carrier of geometric correspondence knowledge. The main strengths are the controlled comparisons on the same CroCo-v2 backbone (Tabs. 1, 2, 8), the genuinely frozen pretrained setting for the zero-shot experiments, and the large reported margins over existing zero-shot baselines (9.41 vs. 26.14 AEPE on HPatches-240). The paper also provides useful layer-wise and pretrained-weight analyses in the supplement. However, the significance is currently limited by two issues: the 'dense zoom-in' component that is necessary for the best zero-shot number is never specified, and the title/abstract claim about 'cross-view completion models' is broader than the evidence, which only supports a specific pretrained CroCo-v2 checkpoint at its native 224x224 resolution.","major_comments":[{"comment":"The 'dense zoom-in' component is load-bearing for the headline zero-shot result but is never defined. Tab. 7 shows that without dense zoom-in the AEPE on HPatches-240 is 10.85 (row I) and with it the reported result is 9.41 (row VII); the main text only says that incorporating it 'significantly enhances zero-shot performance.' The supplementary material, including the zero-shot implementation details in Sec. A.1, does not describe the component: there is no algorithm, no resolution schedule, no statement of which tokens or layers it modifies, and no mention in Fig. 6. Because the paper's central claim is that the cross-attention map itself is a zero-shot correspondence estimator, the headline metric cannot be attributed to the geometric content of the cross-attention map unless this post-processing step is fully specified and ablated. This also prevents independent reproduction.","section":"§4.5, Tab. 7, Supp. §A.1"},{"comment":"The paper's title and abstract claim a property of 'cross-view completion models' in general, but the evidence supports only CroCo-v2 at 224x224 resolution. Supp. Tab. 10 shows that for CroCo-v1, the cross-attention map (39.41 AEPE on HPatches-240) is actually worse than the encoder correlation (30.56), directly contradicting the general claim. Supp. Tab. 9 shows that ZeroCo degrades dramatically at higher input resolutions (35.39 AEPE at 224x224 vs. 186.73 at 448x448 on HPatches-Original). The authors acknowledge these facts in Sec. C.2-C.3, but the central claim is not correspondingly qualified. The paper should either restrict the claim to CroCo-v2 at its training resolution or provide evidence for a broader class of CVC models.","section":"Supp. Tabs. 9 and 10, title/abstract"},{"comment":"The 'zero-shot' claim should be qualified by the amount of benchmark-specific tuning in the inference procedure. The soft-argmax temperature (tau = 1e-4), the register-token replacement, and the dense zoom-in are all selected or validated on the HPatches-240 benchmark (Tab. 7, Supp. Tab. 11). While the model weights are frozen, the inference pipeline is not parameter-free; at minimum, the paper should state whether any components were tuned on the evaluation set and report sensitivity to tau and to the dense zoom-in configuration.","section":"§3.2, Eq. (7), Supp. §A.1"}],"minor_comments":[{"comment":"Equation (1) should specify the axis over which the softmax is computed; based on the text and Eq. (2), it is over source positions j, but this should be stated explicitly.","section":"§3.1, Eq. (1)"},{"comment":"The label 'HPatches-Original' is misleading: Supp. Sec. A.1 states that all inputs are resized to 224x224 for the main experiments, and Tab. 9 confirms that the 'HPatches-Original' numbers at 224x224 match those in Tab. 1. Consider renaming the column to reflect the actual input resolution.","section":"Tabs. 1 and 9"},{"comment":"The row label 'ZeroCo(ZeroCo-finetuned)' is unclear; it should be clarified whether this entry measures the zero-shot inference of the fine-tuned model or the full learning-based pipeline, since the memory/time values are close to the zero-shot 'ZeroCo' row.","section":"Supp. Tab. 12"},{"comment":"The register-token replacement procedure should specify how many register tokens are present in CroCo-v2, how they are identified across layers, and whether the 'minimum attention value' is computed per query or globally.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":"The core finding is interesting and the controlled comparisons on the same backbone are convincing as far as they go, but the missing specification of dense zoom-in is a genuine blocker for the main zero-shot claim. The paper would also benefit from tempering the 'cross-view completion models' generalization in the title and abstract until more model variants are tested. If the authors can fully specify dense zoom-in, provide the ablation/sensitivity analysis, and qualify the scope of the claims, the paper could be a solid contribution. I would also encourage the editor to request code or a detailed algorithm box for the zero-shot pipeline, given how much of the claim rests on an inference-time procedure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline finding holds up: cross-attention maps from CroCo-v2 encode dense correspondence far better than encoder or decoder features. The controlled comparisons on the same backbone in Tabs. 1, 2, and 8 are convincing, and the margin over DIFT and SD-DINO is large enough that this is not noise. The paper also does a real service by showing the effect is specific to cross-attention and by giving a clean zero-shot extraction recipe (reciprocity plus register-token correction). That is a genuine, useful contribution for anyone building on CroCo-v2.\n\nWhat the paper does well beyond the main claim: the layer-wise analysis in the supplement is thorough, the ablation study is honest about which components matter, and the limitation section is straightforward. The learning-based extensions are sensible, and the depth results, while not SOTA as claimed, do show that a cross-attention cost volume can stand in for an epipolar volume.\n\nThe soft spots are real but mostly addressable. The dense zoom-in component is never specified, and the ablation shows it moves the headline HPatches AEPE from 10.85 to 9.41. That is a reproducibility blocker, but it is not fatal to the central claim: even the 10.85 result beats the previous zero-shot methods by a wide margin. The paper should describe the component, or at least release code, before the numbers can be fully trusted. The SOTA language is also loose in a couple of places: Table 4 shows DualRefine beating ZeroCo-depth on KITTI AbsRel, and the supplement shows MASt3R's cross-attention beating ZeroCo zero-shot on HPatches-Original. MASt3R is fine-tuned, so that comparison is not apples-to-apples, but the text should say so. Finally, the appearance-vs-geometry question is not fully settled; the brittleness at higher resolutions and the CroCo-v1 failure suggest the property is tied to CroCo-v2's specific training. That is a limitation worth stating more prominently, not a fatal flaw.\n\nThis paper deserves a serious referee. It reports a reproducible-in-principle empirical finding (with code release pending), the experiments are extensive, and the central observation is likely to influence how people use CVC models. My recommendation: send it to review, but require the authors to specify dense zoom-in, calibrate the SOTA claims, and engage with the MASt3R cross-attention comparison.","headline":"The core observation is real: CroCo-v2's cross-attention maps are surprisingly good zero-shot correspondence estimators, but the paper overreaches on some SOTA claims and leaves a load-bearing component unspecified.","tokens_in":735,"tokens_out":1124,"would_cite":true,"duration_ms":47302,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a cross-view completion model's decoder cross-attention map already encodes dense geometric correspondence, and that reading it as a cost volume yields zero-shot matching that outperforms prior unsupervised baselines.","keywords":["cross-view completion","cross-attention map","zero-shot dense matching","geometric correspondence","cost volume","self-supervised representation learning","multi-frame depth estimation","reciprocity"],"falsifier":"Run the ZeroCo cost volume on image pairs with the same geometry but artificially decorrelated appearance, such as grayscale or strong color jitter on one view: if the cross-attention map's AEPE advantage over encoder and decoder correlations shrinks toward zero, the map is matching appearance rather than geometry, and the central claim fails. The paper's own resolution sweep already bounds the claim, since at 448x448 the advantage collapses to 186.73 AEPE from 35.39 at 224x224.","tokens_in":31953,"feed_emoji":"🔍","tokens_out":8969,"duration_ms":87136,"temperature":0.7,"pith_summary":"Cross-view completion models are trained to reconstruct a masked target image from an unmasked source image, with no correspondence supervision. This paper argues that the decoder's cross-attention map, not the encoder or decoder feature descriptors that prior work has relied on, is where the learned geometric knowledge lives. Reading that map as a cost volume gives zero-shot dense matching with 9.41 average endpoint error on HPatches-240 and 12.72 on ETH3D, well below the best previous zero-shot baselines. The same map, equipped with light learnable heads, also produces competitive learned matching and multi-frame depth estimation. If correct, the result reframes cross-view completion as implicit self-supervised correspondence learning and tells downstream users to extract geometry from the attention map itself.","feed_headline":"One attention map outperforms zero-shot matchers with no training","feed_subtitle":"A network trained only to reconstruct masked views already knows where pixels match across viewpoints.","key_machinery":"The load-bearing object is the cross-attention map $C^l_{\\mathrm{att}}(i,j)=\\mathrm{softmax}(D^{l,Q}_t(i)\\cdot D^{l,K}_s(j)/\\sqrt{d})$ inside each decoder layer of a cross-view completion model: the attention distribution by which the decoder retrieves source patches to reconstruct the masked target. ZeroCo treats the layer-averaged map as a dense cost volume, enforces reciprocity by adding the transposed map computed from the swapped input pair, suppresses register-token artifacts by replacing those attention values with the minimum, and reads out flow by applying $\\mathrm{softargmax}$. This map carries the correspondence signal; correlations of encoder or decoder features are broader, noisier, and match far worse.","core_discovery":"The central claim is that the geometric knowledge learned by cross-view completion lives in the decoder's cross-attention map, not in the encoder or decoder feature descriptors that earlier systems such as DUSt3R, MASt3R, and CroCo-flow have used. The argument rests on an analogy: the cross-view completion objective, which warps source features through a softmax attention distribution to reconstruct a masked target, is the same computation as self-supervised correspondence learning, where a cost volume is softmaxed into a matching distribution and used to warp the source view. The paper verifies this by comparing matching costs from encoder features, decoder features, and the cross-attention map, and the attention map is sharper, less noisy, and far more accurate. The resulting zero-shot procedure, ZeroCo, fuses the forward and swapped cross-attention maps and reaches 9.41 AEPE on HPatches-240 and 12.72 on ETH3D, compared with 26.14 and 25.69 for the best prior zero-shot baselines. With small learnable heads on top of the same map, the paper also reports competitive learned geometric matching and self-supervised multi-frame depth estimation on KITTI and Cityscapes.","pith_inferences":["Because the effect is sensitive to pretraining resolution and data scale, the natural next experiment is to pretrain cross-view completion at higher resolution and with harder viewpoint pairs; the paper's own resolution analysis predicts this would extend zero-shot matching to finer scales.","The reciprocal fusion of forward and swapped attention maps is a generic cycle-consistency mechanism that could be dropped into any cross-attention cost volume, including ones in models already trained with cross-view completion, without retraining.","A direct appearance-shortcut probe, such as grayscaling or color-jittering one view and watching the cross-attention AEPE, would separate geometric retrieval from appearance matching; the paper does not run this experiment.","The same reading of cross-attention as a cost volume may transfer to other two-view objectives, such as stereo or video frame prediction, where reconstruction pressure also forces retrieval of matching source regions."],"forward_implications":["Pretrained cross-view completion models can be used as zero-shot dense matchers by reading the cross-attention map as a cost volume and applying soft-argmax, with no training or fine-tuning.","Previous CVC-based systems that used encoder or decoder descriptors, such as DUSt3R and MASt3R, were built on the weaker signal; the paper's layer analysis implies that routing their matching through cross-attention maps would improve them.","The cross-attention cost volume can replace epipolar cost volumes in multi-frame depth estimation, improving robustness to dynamic objects and image noise while reaching 0.090 AbsRel on KITTI.","With a small learnable aggregation and upsampling head on top of the frozen map, learned matching reaches 13.61 AEPE on HPatches-Original and 2.88 on ETH3D, competitive with heavily pretrained baselines.","The zero-shot property is tied to the exact pretraining setup: it degrades sharply at 448 and 672 resolution and does not appear in CroCo-v1 cross-attention, so the result is specific to CroCo-v2-style training rather than a universal property of cross-attention."],"supporting_citations":[{"why":"Supplies the pretrained CroCo-v2 model whose decoder cross-attention map is analyzed and used for matching and depth.","marker":"[91]"},{"why":"Introduced the cross-view completion pretext task and the cross-attention decoder that the paper reinterprets as a cost volume.","marker":"[90]"},{"why":"Provides the HPatches benchmark and dense geometric matching protocol used for zero-shot and learned evaluation.","marker":"[3]"},{"why":"Provides the ETH3D real-world benchmark used to test generalization beyond homographies.","marker":"[76]"},{"why":"Prior CVC-based 3D reconstruction method that relies on decoder and encoder descriptors, the main comparison point for where geometric knowledge lives.","marker":"[84]"},{"why":"Prior CVC-based matching method compared against, also showing cross-attention emerges in other CVC-trained models.","marker":"[51]"},{"why":"Diffusion-based zero-shot matching baseline whose correlation-map performance ZeroCo outperforms.","marker":"[78]"},{"why":"Stable-diffusion-plus-DINO baseline for zero-shot correspondence, outperformed by the cross-attention cost volume.","marker":"[97]"},{"why":"Self-supervised vision foundation model used as a zero-shot matching baseline with strong geometric awareness.","marker":"[63]"},{"why":"Formulation of self-supervised optical flow with reconstruction loss that motivates the analogy between cross-view completion and correspondence learning.","marker":"[45]"}],"fun_headline_variants":["One attention map yields zero-shot matching with zero training","Cross-attention from completion models is a free zero-shot matcher","No training, just attention: cross-view completion does matching","Reconstruction's attention map is the zero-shot matching key"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that CroCo-v2's reconstruction loss trained the decoder cross-attention to retrieve geometrically corresponding source patches rather than appearance-based lookalikes; if that premise fails, the zero-shot gains collapse on appearance-changing pairs.","fun_headline_variants_meta":{"raw":{"variants":["One attention map yields zero-shot matching with zero training","Cross-attention from completion models is a free zero-shot matcher","No training, just attention: cross-view completion does matching","Reconstruction's attention map is the zero-shot matching key"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00118,"raw_usage":{"total_tokens":4829,"prompt_tokens":856,"completion_tokens":3973,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":3905}},"tokens_in":472,"tokens_out":3973,"duration_ms":28707,"temperature":1.0,"reasoning_tokens":3905,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:21:22.367580+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the ZeroCo cost volume on image pairs with the same geometry but artificially decorrelated appearance, such as grayscale or strong color jitter on one view: if the cross-attention map's AEPE advantage over encoder and decoder correlations shrinks toward zero, the map is matching appearance rather than geometry, and the central claim fails. The paper's own resolution sweep already bounds the claim, since at 448x448 the advantage collapses to 186.73 AEPE from 35.39 at 224x224.","supporting_citations":[{"cited_title":"Croco v2: Improved cross-view completion pre- training for stereo matching and optical flow","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained CroCo-v2 model whose decoder cross-attention map is analyzed and used for matching and depth."},{"cited_title":"Croco: Self-supervised pre-training for 3d vision tasks by cross-view completion","cited_arxiv_id":null,"evidence_quote":"Introduced the cross-view completion pretext task and the cross-attention decoder that the paper reinterprets as a cost volume."},{"cited_title":"A multi-view stereo benchmark with high- 12 resolution images and multi-camera videos","cited_arxiv_id":null,"evidence_quote":"Provides the ETH3D real-world benchmark used to test generalization beyond homographies."},{"cited_title":"Dust3r: Geometric 3d vision made easy","cited_arxiv_id":null,"evidence_quote":"Prior CVC-based 3D reconstruction method that relies on decoder and encoder descriptors, the main comparison point for where geometric knowledge lives."},{"cited_title":"Emergent correspondence from image diffusion","cited_arxiv_id":null,"evidence_quote":"Diffusion-based zero-shot matching baseline whose correlation-map performance ZeroCo outperforms."},{"cited_title":"A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence","cited_arxiv_id":null,"evidence_quote":"Stable-diffusion-plus-DINO baseline for zero-shot correspondence, outperformed by the cross-attention cost volume."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Self-supervised vision foundation model used as a zero-shot matching baseline with strong geometric awareness."},{"cited_title":"Barron, Ariel Gordon, Kurt Konolige, and Anelia Angelova","cited_arxiv_id":null,"evidence_quote":"Formulation of self-supervised optical flow with reconstruction loss that motivates the analogy between cross-view completion and correspondence learning."}],"review_version":1}