{"id":"3e3c96f1-0e39-4aa0-bf4f-62984d682b81","arxiv_id":"2608.10680","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"JFRDet aligns infrared and visible features with an explicit affine transformation and illumination-guided fusion, reaching 69.7% mAP50 on the new DVMA benchmark.","lead":"JFRDet is a detector that estimates one global affine transform to align thermal features to visible features before fusing them, and it reports state-of-the-art results on a new synthetic severely misaligned drone benchmark, DVMA. It matters because real RGB-thermal camera pairs are frequently misaligned, and most existing detectors only cope with small offsets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim is confounded by privileged affine-correspondence supervision: JFRDet trains on DVMA's recorded warp parameters while the compared baselines do not, so the reported 3-point gain may not reflect the architecture.","rationale":"The central claim is the reported 69.7% mAP50 SOTA on DVMA. The most load-bearing issue is not internal inconsistency but evaluation fairness: DVMA's synthetic construction records the exact affine warp, and the paper uses those records to supervise CMAA's alignment loss, while all compared baselines are trained with detection labels only. The ablations cannot disentangle architecture from this privileged supervision because the baseline has no module that can ingest the correspondence fields. This directly threatens the headline number even on the authors' own benchmark. The reader's weakest assumption about global-affine representativeness is related and valid, but the privileged-supervision confound is more immediate: it affects whether the reported comparison is apples-to-apples at all, independent of real-world transfer. I do not consider the mismatched figure artifacts or missing code/data as the central technical flaw, though they reinforce the need for conditional acceptance. The architecture is plausible and the ablations are directionally consistent, so the paper should not be rejected; it should be conditioned on releasing code/data and on the baseline-with-supervision control experiment described above.","tokens_in":14164,"tokens_out":6852,"duration_ms":75097,"concrete_test":"Retrain the strongest baseline (DMM + S2A-Net) with an auxiliary loss that consumes the same recorded DVMA affine parameters / ground-truth offset fields used by CMAA, under the exact same schedule, optimizer, and hyperparameters, and compare mAP50 on DVMA. If the baseline rises to approximately 69.7% or beyond, the SOTA gap is explained by privileged supervision; if it stays near 66.7%, the architecture contributes independently. A complementary check would be evaluating JFRDet on DVMA variants with non-affine per-object warps or on a real misaligned visible-thermal set, but the baseline-with-supervision test is the one that settles the numeric SOTA claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's 'SOTA' claim is not a like-for-like comparison. DVMA is built by applying recorded affine warps to the IR image (Sec. 3.1), and those recorded parameters are used to generate dense ground-truth correspondence and offset fields at every feature resolution; the paper explicitly says this enables supervised cross-modal registration during training. CMAA's alignment loss L_align (Sec. 3.5) is therefore trained with privileged geometric supervision that none of the compared methods (S2A-Net, DMM, C2Former, COMO) receive. The reported +3.0 mAP50 over DMM+S2A-Net may come from this extra supervision rather than from the CMAA/IGCF/AQCG design. The component ablation in Table 3 does not remove this confound: the baseline row has no alignment module that could consume the correspondence labels, so the gain from CMAA is entangled with the gain from seeing the ground-truth affine warp. A second, related limitation is external validity. Because DVMA contains only global affine distortions of the IR image, the benchmark is generated from exactly the model class CMAA assumes, so the 69.7% number cannot support the broader claim of bridging severe cross-modal misalignment in real scenes with parallax or independent object motion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes JFRDet, an end-to-end visible-infrared oriented object detector targeting severe cross-modal spatial misalignment. The method has three main components: a Cross-Modal Affine Alignment (CMAA) module that estimates an image-level affine transform from visible to infrared and warps multi-level infrared features; an Illumination-Guided Complementary Fusion (IGCF) module that adaptively suppresses unreliable visible cues under poor illumination; and an Alignment Quality-Consistency Gating (AQCG) strategy that reweights detection supervision based on alignment quality and gradient consistency. The authors also construct DVMA, a synthetic benchmark derived from DroneVehicle by applying random global affine transforms to the infrared image and recording the warp parameters to generate dense correspondence labels. On DVMA, JFRDet reports 69.7% mAP50 and 36.1% mAP50:95, outperforming DMM+S2A-Net by 3.0 mAP50, with component ablations from 66.0 to 69.7 mAP50.","tokens_in":14452,"tokens_out":5843,"duration_ms":63809,"significance":"Explicit feature-domain affine registration for severely misaligned visible-infrared detection is a worthwhile and underexplored problem, and the paper contains a clean formulation of coarse-to-fine correspondence, confidence-weighted affine fitting, and differentiable warping. The proposed DVMA benchmark could be a useful testbed if its limitations are clearly stated. However, the central SOTA claim is currently supported only by experiments on a synthetic benchmark whose generative process is exactly the global affine model assumed by CMAA, and the comparison is potentially confounded by privileged geometric supervision that the baselines do not receive. The mAP50:95 margin over the best baseline is only 1.0 point. These issues make the quantitative claims provisional rather than established.","major_comments":[{"comment":"The reported SOTA gain is confounded by privileged affine-correspondence supervision. DVMA records all transformation parameters, and Sec. 3.1 states that exact cross-modal correspondences are generated at each feature resolution and that correspondence supervision is provided in both directions; these labels are used by CMAA's alignment loss L_align. None of the compared baselines (S2A-Net, DMM, C2Former, COMO) receive such geometric supervision. Therefore the +3.0 mAP50 over DMM+S2A-Net in Table 2, and the CMAA ablation in Table 3, do not isolate the contribution of the architecture: the gain could come from seeing ground-truth warp parameters. Please add a control experiment in which a comparable baseline is given the same privileged information (for example, pre-warping infrared features with the recorded affine transform, or training a simple alignment module with the same offset-field supervision), and/or train CMAA without correspondence supervision to measure the architecture-only gain.","section":"Sec. 3.1 / Sec. 3.5 / Tables 2-3"},{"comment":"The external validity of the benchmark is not established. DVMA is constructed by applying a single global 2D affine transform to the infrared image, and CMAA estimates exactly such an image-level affine transform. The test distribution is therefore generated from the same model class the method assumes. This does not support the abstract's broad claim of 'bridging severe cross-modal misalignment' in real scenes, where parallax, independent object motion, and non-affine distortion occur. Please evaluate on a real misaligned dataset or augment DVMA with non-affine and object-level perturbations, and temper the claims to the global-affine setting.","section":"Sec. 3.1 / Sec. 3.3.3 / Abstract"},{"comment":"The central architecture figure is inconsistent with the described method. Figure 3 contains elements such as 'Hyperspectral image', 'SGMCT framework', 'Spectral-Guided Fusion Module (SGFM)', and 'RS2Conv' that are not part of JFRDet and appear to come from an unrelated manuscript on hyperspectral image classification. This must be corrected with an actual diagram of CMAA, IGCF, and AQCG, and the provenance of the current figure must be clarified. As it stands, the visual evidence for the architecture cannot be trusted.","section":"Figure 3"},{"comment":"All results are single-run, and code and dataset are not released. With a 3.0-point mAP50 margin and no variance estimate, the SOTA claim is not statistically grounded; additionally, the mAP50:95 margin over the best baseline is only 1.0 point (36.1 vs 35.1), so the improvement is much smaller on the stricter localization metric. Please report multiple seeds with mean and standard deviation, and release code and dataset or specify a detailed evaluation protocol.","section":"Sec. 4.3 / Tables 2-3"}],"minor_comments":[{"comment":"Please define 'Coarse Matching Matrices' before using the term; the current text jumps from the similarity matrix S to the bidirectional confidence matrices without specifying how the coarse matching matrices are formed.","section":"Sec. 3.3.1"},{"comment":"Figure 2 shows both 'Transformed RGB' and 'Transformed IR', while the text says only the infrared image undergoes a compound transformation and the visible image is center-cropped. Please clarify whether the RGB branch is also resized or transformed.","section":"Figure 2 / Sec. 3.1"},{"comment":"Please state how oriented ground-truth boxes are handled when images are resized to 480x384, and whether the recorded affine parameters are used to transform the annotations during DVMA construction; this affects the interpretation of the reported AP values.","section":"Sec. 4.2"}],"recommendation":"major_revision","confidential_remarks":"The Figure 3 issue is serious and should be investigated editorially: the figure appears to contain material from an unrelated paper on hyperspectral image analysis, which raises provenance concerns independent of the technical content. The privileged-supervision confound and the synthetic-only evaluation also need to be addressed before the paper can be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, the core idea is reasonable: register the IR feature pyramid to the RGB frame with an image-level affine transform before fusion, with illumination gating and a training-time quality gate. The modules are assembled from familiar building blocks, the math is standard, and the ablations are internally consistent. Second, the headline result is not a like-for-like comparison. DVMA is built by applying recorded affine warps to IR images, and those same recorded parameters are used to generate dense correspondence labels and offset fields at every feature resolution. CMAA is trained with this privileged geometric supervision (the L_align term). None of the compared baselines receive anything equivalent, so the +3 mAP over DMM+S2A-Net cannot be attributed to the architecture itself. The stress-test concern is correct, and it is not a minor technicality: the component ablation does not remove the confound because the baseline has no alignment module that could consume those labels. The central claim of SOTA performance on DVMA is therefore not established.\n\nThe paper does some things well. The DVMA benchmark is a new artifact, and building it from DroneVehicle with controlled affine perturbations is a sensible way to create a challenging testbed—provided the limitations are stated. The affine fitting and warping equations are correct, and the AQCG mechanism, while a bit elaborate, is clearly described. The authors are transparent about using the recorded warps for supervision; they do not hide that. What they hide, or at least fail to emphasize, is that this makes the comparison to prior work unfair.\n\nThe soft spots, in order of severity: (1) the privileged supervision confound, which alone prevents accepting the headline; (2) the external validity problem—DVMA only contains global affine distortions, the exact model class CMAA assumes, so the numbers say little about parallax, independent object motion, or other real severe misalignments; (3) no code or data release despite the abstract promising it; (4) single-run results with no error bars; (5) a glaring presentation issue: Figure 3 looks like a collage of figures from unrelated papers (hyperspectral classification blocks, SGMCT, etc.) and does not coherently depict JFRDet. That last one is a desk-reject-level artifact in a normal submission, but since the science is otherwise legible, I'd fix it rather than reject outright.\n\nWho gets value from this? Researchers working on RGB-thermal fusion for drone or surveillance detection, especially those interested in misalignment-robust training. The weight of evidence, though, is provisional. A serious referee should engage with it, but the acceptance decision should hinge on a revised version that either removes the privileged supervision, adds a baseline trained with the same correspondence labels, or clearly frames the result as supervised affine registration rather than comparing against methods that lack that supervision. I would not cite it in my own work until that happens.\n\nFinal recommendation: send it to peer review, but expect heavy revision and insist on code/data release and a fixed Figure 3.","headline":"A coherent system paper whose headline SOTA claim is undermined by privileged geometric supervision that baselines don't receive; the idea is worth scrutiny, but the current evidence only shows the method works when it sees the warp parameters.","tokens_in":15004,"tokens_out":1752,"would_cite":false,"duration_ms":20815,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Explicitly registering infrared features to the visible reference before fusion lifts severe-misalignment visible-infrared detection to 69.7% mAP50 on a new benchmark.","keywords":["visible-infrared object detection","oriented object detection","cross-modal misalignment","affine feature registration","illumination-guided fusion","multimodal benchmark","thermal infrared","drone imagery"],"falsifier":"Run JFRDet on a real dual-sensor rig with rigidly offset cameras that produce measured large misalignments involving parallax, and compare mAP50 against DMM + S2A-Net: the affine-registration claim would be falsified if the gap shrinks to noise or if the estimated transform leaves visible feature displacement.","tokens_in":13954,"feed_emoji":"🚁","tokens_out":8520,"duration_ms":69854,"temperature":0.7,"pith_summary":"The paper addresses visible-infrared object detection when the two images are severely misaligned — offsets of tens to hundreds of pixels, plus rotation and scale differences — rather than the weakly aligned pairs most prior work assumes. It argues that implicit feature adaptation is insufficient at this scale and that explicit, image-level affine registration in the feature domain, done end-to-end with detection, is the missing piece. To make the case, the authors build DVMA, a benchmark derived from DroneVehicle by applying controlled rotation, anisotropic scaling, and translation to infrared images, and report that their JFRDet reaches 69.7% mAP50, outperforming the best compared method (DMM + S2A-Net at 66.7%) by about 3 points. If true, the result indicates that global geometric correction is a practical first-class step for misaligned multimodal detection, not merely a preprocessing nicety.","feed_headline":"Affine registration lifts misaligned RGB-IR detection to 69.7%","feed_subtitle":"End-to-end affine alignment plus illumination-guided fusion tops the strongest baseline by 3 points on a new severe-offset benchmark.","key_machinery":"The load-bearing object is the image-level affine transformation $T_{\\mathrm{ir}\\to\\mathrm{rgb}}$ estimated by Cross-Modal Affine Alignment (CMAA): a $2\\times3$ matrix mapping infrared coordinates to visible coordinates, fitted by confidence-weighted least squares from coarse-to-fine bidirectional correspondence matches and then applied to every level of the infrared feature pyramid through bilinear warping. It converts cross-modal fusion from an operation on misaligned feature maps into one on a common geometric reference, which is what lets the detector exploit complementary cues despite large offsets. The two supporting mechanisms are Illumination-Guided Complementary Fusion (IGCF), which scales a discrepancy-based spatial gate by a darkness factor computed from mean visible intensity, and Alignment Quality-Consistency Gating (AQCG), which gates the detection loss by an exponential moving average of alignment loss and cosine similarity of alignment/detection gradients.","core_discovery":"JFRDet is an end-to-end visible-infrared oriented object detector built around a Cross-Modal Affine Alignment (CMAA) module. CMAA first finds coarse bidirectional correspondences between RGB and infrared features at 1/16 resolution, refines them locally, and then estimates a single affine transformation $T_{\\mathrm{ir}\\to\\mathrm{rgb}}\\in\\mathbb{R}^{2\\times3}$ by confidence-weighted least squares. That transform is rescaled to each feature level and applied by inverse bilinear grid sampling, so every infrared pyramid feature is warped to the visible reference before fusion. Fusion is then gated by an illumination estimate, and an Alignment Quality-Consistency Gating (AQCG) mechanism reweights detection loss according to how reliable and gradient-consistent the alignment is. On the new DVMA benchmark, built from DroneVehicle pairs with rotations of 15–30 degrees, scaling factors of 0.90–1.10, and translations producing overlaps of 0.70–0.80 IoU, JFRDet obtains 69.7% mAP50 and 36.1% mAP50:95, the best numbers reported in the paper, ahead of the strongest baseline DMM + S2A-Net at 66.7% and 35.1%.","pith_inferences":["Editorial inference: If real misalignment includes parallax, independent object motion, or lens distortion, one global affine transform cannot represent it; a natural next test is replacing the image-level fit with piecewise-affine or dense-flow refinement after an affine warm start.","Editorial inference: The DVMA benchmark's synthetic perturbations and known ground truth could serve as a pretraining or calibration stage for real misaligned pairs, where true correspondences are unavailable.","Editorial inference: The illumination gate in IGCF uses only global mean visible intensity; extending it to a local or learned illumination estimate is a direct, testable variant that could matter for scenes with mixed shadows or strong light sources."],"forward_implications":["On DVMA, the full JFRDet reaches 69.7% mAP50, which is 3.0 points above the best compared method (DMM + S2A-Net at 66.7%).","The component ablation attributes the gain to each mechanism: affine alignment alone moves the no-alignment fusion baseline from 66.0% to 67.3%, and adding illumination-guided fusion and gating brings it to 69.7%.","The comparison also shows that infrared-only S2A-Net (61.6% mAP50) beats several RGB-IR fusion methods on DVMA, so unaligned fusion can be worse than a single modality; explicit alignment is what makes the extra modality pay off.","Because DVMA records exact transformation parameters, it supplies per-pixel correspondence supervision at every feature resolution, enabling direct supervised training of cross-modal registration rather than relying on implicit adaptation."],"supporting_citations":[{"why":"Provides the aligned drone visible-infrared pairs and oriented annotations from which DVMA is constructed by applying controlled affine perturbations.","marker":"[24]"},{"why":"Supplies the S2A-Net oriented detection head used in JFRDet and in the strongest compared baseline, making the comparison apples-to-apples.","marker":"[7]"},{"why":"Defines the DMM dual-stream backbone JFRDet follows and the best competing multimodal baseline at 66.7% mAP50.","marker":"[44]"},{"why":"COMO, an offset-guided fusion baseline, is among the multimodal detectors compared on DVMA.","marker":"[17]"},{"why":"C2Former, a calibrated and complementary transformer baseline, is compared to show gains over implicit-adaptation fusion methods.","marker":"[37]"},{"why":"Supplies the motivation that translation, scale, and rotation are the cross-modal misalignment modes that explicit alignment should target.","marker":"[36]"}],"fun_headline_variants":["Affine alignment + illumination gate hits 69.7% on misaligned RGB-IR","Explicit affine registration propels RGB-IR detection to new SOTA","End-to-end affine alignment wins on severe RGB-IR misalignment","JFRDet: affine registration tames large-offset RGB-IR detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method assumes that the spatial mismatch between a visible-infrared image pair can be captured by one global 2D affine transformation, and that the synthetic affine distortions used to build the test benchmark behave like the severe misalignment found in real scenes.","fun_headline_variants_meta":{"raw":{"variants":["Affine alignment + illumination gate hits 69.7% on misaligned RGB-IR","Explicit affine registration propels RGB-IR detection to new SOTA","End-to-end affine alignment wins on severe RGB-IR misalignment","JFRDet: affine registration tames large-offset RGB-IR detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000627,"raw_usage":{"total_tokens":2956,"prompt_tokens":1060,"completion_tokens":1896,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":676,"completion_tokens_details":{"reasoning_tokens":1816}},"tokens_in":676,"tokens_out":1896,"duration_ms":11289,"temperature":1.0,"reasoning_tokens":1816,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:32:15.837492+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run JFRDet on a real dual-sensor rig with rigidly offset cameras that produce measured large misalignments involving parallax, and compare mAP50 against DMM + S2A-Net: the affine-registration claim would be falsified if the gap shrinks to noise or if the estimated transform leaves visible feature displacement.","supporting_citations":[{"cited_title":"Align deep features for oriented object detection.IEEE transac- tions on geoscience and remote sensing, 60:1–11, 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the S2A-Net oriented detection head used in JFRDet and in the strongest compared baseline, making the comparison apples-to-apples."},{"cited_title":"Dmm: Disparity-guided multispectral mamba for oriented object detection in remote sensing.IEEE Transactions on Geoscience and Remote Sensing, 63:1–13,","cited_arxiv_id":null,"evidence_quote":"Defines the DMM dual-stream backbone JFRDet follows and the best competing multimodal baseline at 66.7% mAP50."},{"cited_title":"Como: Cross-mamba interaction and offset- guided fusion for multimodal object detection.Information Fusion, 125:103414, 2026","cited_arxiv_id":null,"evidence_quote":"COMO, an offset-guided fusion baseline, is among the multimodal detectors compared on DVMA."},{"cited_title":"C 2former: Calibrated and complementary transformer for rgb-infrared object de- tection.IEEE Transactions on Geoscience and Remote Sens- ing, 62:1–12, 2024","cited_arxiv_id":null,"evidence_quote":"C2Former, a calibrated and complementary transformer baseline, is compared to show gains over implicit-adaptation fusion methods."},{"cited_title":"Transla- tion, scale and rotation: cross-modal alignment meets rgb- infrared vehicle detection","cited_arxiv_id":null,"evidence_quote":"Supplies the motivation that translation, scale, and rotation are the cross-modal misalignment modes that explicit alignment should target."}],"review_version":1}