{"id":"49b1ce18-5365-432a-92fa-c92caafd0817","arxiv_id":"2606.00706","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":3.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"CR-JEPA extends JEPA-style predictive objectives to cross-modal remote sensing retrieval using shared transformers, sketched regularization, and separate same-modal and cross-modal heads, reporting gains on BEN-14K, CBRSIR_VS, and DSRSID.","lead":"CR-JEPA applies a joint-embedding predictive architecture with modality-specific stems and decoupled heads to improve retrieval of semantically related scenes across different remote sensing modalities such as SAR and optical. A smart generalist might read it to see how predictive self-supervised methods can handle mismatched imaging physics in earth observation data.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Abstract reports cross-modal gains but omits same-modal metrics needed to verify no performance trade-off","rationale":"The reader's weakest assumption already isolates the exact joint-alignment vs. neighborhood-preservation tension; the missing same-modal numbers make that assumption the single most load-bearing point. Full text may contain the missing table rows, but until they are checked the empirical claim remains only partially supported. No other internal inconsistency is visible from the supplied text.","tokens_in":1776,"tokens_out":336,"duration_ms":17877,"concrete_test":"From the results section or Table on BEN-14K, extract the S1→S1 and S2→S2 retrieval scores for both CR-JEPA and X-JEPA; if either same-modal score for CR-JEPA drops more than 3 points relative to X-JEPA while cross-modal improves, the no-trade-off claim does not hold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim asserts that CR-JEPA improves cross-modal retrieval (S1→S2: 61.23%→75.82%; S2→S1: 63.73%→75.40%) while remaining competitive on same-modal retrieval with fewer parameters. The reader's weakest assumption correctly flags the risk that modality-specific stems + shared trunk + JEPA objectives + sketched isotropic Gaussian regularization + decoupled heads may not jointly preserve same-modal neighborhoods without collapse or degradation. Because the abstract supplies only the cross-modal deltas and a qualitative statement on same-modal performance, the no-trade-off half of the claim rests on an unquantified assertion rather than reported numbers.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes CR-JEPA, a Cross-modal Retrieval Joint-Embedding Predictive Architecture for dual-modality remote sensing image retrieval. It employs modality-specific stems, a shared transformer trunk, JEPA-style predictive objectives to estimate masked latent target features within and across modalities, sketched isotropic Gaussian regularization on raw retrieval projections, and a decoupled-head design with a unified retrieval head for same-modal tasks and a cross-modal retrieval head. Evaluations on BEN-14K, CBRSIR_VS, and DSRSID report cross-modal gains on BEN-14K (S1 to S2 retrieval from 61.23% to 75.82%; S2 to S1 from 63.73% to 75.40% over X-JEPA) while claiming competitive same-modal retrieval with fewer parameters.","tokens_in":1901,"tokens_out":521,"duration_ms":31073,"significance":"If the empirical results hold under full experimental scrutiny, the work could provide a practical framework for cross-modal retrieval in remote sensing by combining predictive objectives with regularization to achieve semantic alignment across heterogeneous modalities without collapse. The decoupled-head design and use of JEPA objectives represent architectural strengths for jointly handling alignment and neighborhood preservation. The reported cross-modal improvements suggest potential utility in multi-sensor applications, though significance hinges on verifying the no-trade-off claim for same-modal performance.","major_comments":[{"comment":"Abstract (final sentence): The central claim that CR-JEPA achieves 'competitive same-modal retrieval with fewer parameters' provides no quantitative metrics, tables, or direct comparisons for same-modal performance. This is load-bearing for the contribution, as the abstract and introduction stress that a single objective may fail to jointly support cross-modal alignment and same-modal neighborhood preservation, yet only cross-modal deltas are quantified.","section":"Abstract"},{"comment":"Abstract and likely §4 (Experimental results): Specific percentage gains are stated (e.g., 61.23% to 75.82%) but without any mention of experimental protocol, baseline implementation details beyond X-JEPA, error bars, statistical tests, dataset splits, or number of runs. This prevents assessment of whether the numbers reliably support the claims.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract packs many technical components into one paragraph; a brief clarification of how the sketched isotropic Gaussian regularization interacts with the JEPA objectives would aid readability.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on the abstract. We address the major comments point by point below and will make revisions to strengthen the quantitative support and experimental transparency.","responses":[{"response":"We agree that the abstract would be strengthened by including quantitative support for the same-modal claim. Section 4 of the manuscript contains tables reporting same-modal mAP and parameter counts demonstrating competitive performance with fewer parameters than X-JEPA. We will revise the abstract's final sentence to reference these specific metrics and tables.","revision_made":"yes","referee_comment":"[Abstract] Abstract (final sentence): The central claim that CR-JEPA achieves 'competitive same-modal retrieval with fewer parameters' provides no quantitative metrics, tables, or direct comparisons for same-modal performance. This is load-bearing for the contribution, as the abstract and introduction stress that a single objective may fail to jointly support cross-modal alignment and same-modal neighborhood preservation, yet only cross-modal deltas are quantified."},{"response":"Dataset splits, baseline reproduction details, and evaluation procedures are described in Sections 3.3 and 4.1. However, the abstract omits reference to them, and the current version lacks error bars and statistical tests. We will add a concise protocol note to the abstract, report error bars from multiple runs, and include statistical significance in the revised §4.","revision_made":"partial","referee_comment":"[Abstract] Abstract and likely §4 (Experimental results): Specific percentage gains are stated (e.g., 61.23% to 75.82%) but without any mention of experimental protocol, baseline implementation details beyond X-JEPA, error bars, statistical tests, dataset splits, or number of runs. This prevents assessment of whether the numbers reliably support the claims."}],"tokens_in":1498,"tokens_out":391,"duration_ms":26302,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that this paper takes LeJEPA and X-JEPA predictive learning and adapts them for cross-modal retrieval between Sentinel-1 and Sentinel-2 imagery. It adds modality-specific stems, a shared trunk, within- and cross-modality prediction, sketched isotropic Gaussian regularization on the projections, and decoupled heads for same-modal versus cross-modal retrieval.\n\nThe architecture choices make sense for the stated problem. Different sensors produce very different appearances, so separate stems plus a shared transformer plus separate heads is a reasonable way to try to keep both cross-modal alignment and same-modal neighborhoods intact. The regularization step is a direct response to collapse risk in these setups.\n\nThe soft spot is the experimental section as described in the abstract. It supplies specific numbers for the cross-modal improvements over X-JEPA on BEN-14K, but only calls same-modal retrieval \"competitive\" with fewer parameters and gives no numbers, no other baselines, no splits, and no error bars. That makes it hard to judge whether the design actually avoids the trade-off the authors themselves flag as a risk.\n\nThis is aimed at remote-sensing practitioners who need cross-modal search in earth-observation pipelines. Someone already working on JEPA-style methods or multimodal RS retrieval could pick up the stem-and-head design as a concrete starting point.\n\nI would bring it to a reading group only if the group focuses on applied self-supervised learning in remote sensing. I would not cite it in my own work in the next year because it does not introduce new theoretical machinery. The thinking is clear and it engages the recent literature honestly, so it deserves peer review so referees can check the full tables and protocol.","headline":"CR-JEPA applies existing JEPA ideas to cross-modal RS retrieval and reports cross-modal gains on BEN-14K, but the abstract leaves the no-trade-off claim on same-modal performance unquantified.","tokens_in":2391,"tokens_out":423,"would_cite":false,"duration_ms":17096,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"CR-JEPA uses modality-specific stems, a shared predictive trunk, and decoupled heads to raise cross-modal remote sensing retrieval accuracy by more than 14 points while matching same-modal performance with fewer parameters.","keywords":["cross-modal retrieval","remote sensing image retrieval","joint embedding predictive architecture","modality alignment","sentinel data","image retrieval","predictive learning"],"falsifier":"An experiment on BEN-14K in which cross-modal accuracy falls back to the X-JEPA level or same-modal accuracy drops when the sketched isotropic Gaussian regularization is removed would falsify the central claim.","tokens_in":2685,"feed_emoji":"🛰️","tokens_out":746,"duration_ms":20805,"temperature":0.7,"pith_summary":"The paper aims to show that one architecture can simultaneously align scenes across different remote-sensing sensors and preserve neighborhoods within each sensor. It does so by predicting masked latent features both inside and between modalities, applying sketched isotropic Gaussian regularization to the projections, and routing same-modal and cross-modal tasks through separate heads. A sympathetic reader would care because paired observations from sensors such as Sentinel-1 and Sentinel-2 often differ sharply in physics and appearance, yet many applications need to retrieve semantically related scenes regardless of sensor. The reported gains on BEN-14K, CBRSIR_VS, and DSRSID indicate that the joint predictive objective plus regularization can support both goals without collapse or large parameter overhead.","feed_headline":"Predictive model raises cross-sensor retrieval by 14 points","feed_subtitle":"Shared trunk with within- and across-modality prediction aligns different remote-sensing sensors while keeping single-sensor neighborhoods i","key_machinery":"Modality-specific stems plus shared transformer trunk trained with within- and across-modality JEPA predictive objectives, stabilized by sketched isotropic Gaussian regularization on the retrieval projections and routed through decoupled same-modal and cross-modal heads.","core_discovery":"CR-JEPA employs modality-specific stems feeding a shared transformer trunk that is trained with JEPA-style predictive objectives to reconstruct masked latent targets within and across modalities. Sketched isotropic Gaussian regularization is applied to the raw retrieval projections, and a decoupled-head design supplies one unified head for same-modal retrieval and a separate head for cross-modal search. On BEN-14K this produces S1-to-S2 retrieval of 75.82 percent and S2-to-S1 retrieval of 75.40 percent, against 61.23 percent and 63.73 percent for the prior X-JEPA baseline, while same-modal retrieval stays competitive and total parameter count is lower.","pith_inferences":["The decoupled-head pattern could be tested on other pairs of imaging modalities that differ in physics.","The regularization step may be the main stabilizer when predictive objectives are applied to high-variance sensor data.","If the gains hold on larger or noisier collections, the method could reduce the need for separate retrieval systems per sensor type."],"forward_implications":["Cross-modal retrieval between Sentinel-1 and Sentinel-2 scenes improves by more than 14 percentage points.","Same-modal retrieval accuracy remains competitive without increasing model size.","The architecture jointly optimizes semantic alignment across modalities and local neighborhood preservation within each modality.","The same design yields gains on three separate remote-sensing retrieval benchmarks."],"fun_headline_variants":["CR-JEPA achieves 75.82 percent S1 to S2 retrieval on BEN-14K","Cross modal retrieval reaches 75.40 percent S2 to S1 with CR-JEPA","Shared transformer trunk aligns modalities in CR-JEPA","Decoupled head design separates same and cross modal retrieval"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The predictive objectives, regularization, and decoupled heads together produce cross-modal alignment and same-modal neighborhood preservation at the same time without collapse or accuracy trade-offs on heterogeneous remote-sensing data.","fun_headline_variants_meta":{"raw":{"variants":["CR-JEPA achieves 75.82 percent S1 to S2 retrieval on BEN-14K","Cross modal retrieval reaches 75.40 percent S2 to S1 with CR-JEPA","Shared transformer trunk aligns modalities in CR-JEPA","Decoupled head design separates same and cross modal retrieval"]},"model":"grok-4.3","cost_usd":0.007781,"raw_usage":{"total_tokens":3591,"prompt_tokens":742,"num_sources_used":0,"completion_tokens":78,"cost_in_usd_ticks":77812000,"prompt_tokens_details":{"text_tokens":742,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2771,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":742,"tokens_out":78,"duration_ms":18839,"temperature":1.0,"reasoning_tokens":2771,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T19:26:20.808223+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An experiment on BEN-14K in which cross-modal accuracy falls back to the X-JEPA level or same-modal accuracy drops when the sketched isotropic Gaussian regularization is removed would falsify the central claim.","supporting_citations":[],"review_version":1}