{"id":"140f7d10-5f08-4a53-b765-cb80ab57e033","arxiv_id":"2506.14696","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A unified RGB-thermal detection framework with a P3 fusion point and a ControlNet-style fine-tuning strategy, reporting FLIR mAP up to 47.61%.","lead":"YOLOv11-RGBT fuses visible and infrared images inside single-stage detectors, adding a fine-tuning trick that freezes a pretrained thermal model and injects visible features through zero-initialized convolutions. The authors report detection gains on FLIR and LLVIP, but the headline numbers and comparisons have internal inconsistencies that need fixing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FLIR MCF gain claim rests on non-pretrained IR baselines; against the COCO-pretrained baselines in Table 7 the gains are 0.45-3.18 mAP, not 3.41-5.65.","rationale":"The reader's CONDITIONAL verdict is well-founded, and the stress-test pass isolates the sharpest version of the protocol concern. The MCF strategy is the paper's main claimed contribution, and its headline number is computed against the wrong baseline. This is an internal comparison inconsistency, not a disagreement with field consensus, so it is directly load-bearing: the central consistently-improved claim is exactly what the tables fail to support. I am not alleging anything about intent; the text itself juxtaposes Table 3 and Table 7 numbers, and Section 4.4's from 41.96 to 47.61 makes the baseline explicit. The fair comparison still shows positive but much smaller gains, so the method is not refuted; the claim needs re-scoping and a single-protocol evaluation. Code availability and the breadth of fusion modes are real positives and support keeping a conditional rather than a reject verdict. A fixed-protocol rerun with seeds would settle whether MCF's advantage over a COCO-pretrained IR model is real and consistent across model sizes.","tokens_in":34036,"tokens_out":7272,"duration_ms":63332,"concrete_test":"Recompute the five FLIR MCF deltas using only Table 7: for each YOLOv11 size, subtract the COCO-pretrained IR AP from the YOLOv11-RGBT-MCF AP. If the deltas are +0.88, +0.45, +0.67, +1.06, and +3.18 instead of the abstract's 3.41-5.65%, the headline claim is confirmed to rest on a non-pretrained baseline; then rerun the five MCF variants under one fixed optimizer with three seeds against the same COCO-pretrained IR baseline and report mean and standard deviation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim in the abstract is that MCF consistently improved YOLOv11 models' mAP by 3.41%-5.65% on FLIR. That range is exactly what you get by comparing Table 7 MCF AP values (41.18, 43.89, 44.79, 44.93, 47.61 for n/s/m/l/x) against the non-pretrained IR-only AP values in Table 3 (37.31, 39.74, 41.38, 41.45, 41.96). Section 4.4 confirms this reading when it says MCF boosted mAP from 41.96% to 47.61%, citing the Table 3 YOLOv11x IR value rather than the COCO-pretrained IR value of 44.43 in the same Table 7. Under the controlled comparison that Table 7 actually advertises, with both sides COCO-pretrained, the deltas are only +0.88, +0.45, +0.67, +1.06, and +3.18, and YOLOv11s gains less than half a point. Table 6 compounds the problem by selecting the optimizer and learning rate per model after seeing validation results, and no seed or error-bar information is reported. The framework and code may still be useful engineering, but the headline consistent 3.41-5.65% improvement is not established by the tables as presented; it depends on mixing a non-pretrained baseline with a pretrained method.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript presents YOLOv11-RGBT, a single-stage multispectral object detection framework built on YOLOv11. The framework implements six fusion modes (early, mid, mid-to-late, late, score, and weight-sharing) across YOLOv3-YOLOv12 and RT-DETR, proposes a P3 single-node mid-fusion strategy, and introduces Multispectral Controllable Fine-tuning (MCF), which freezes a pretrained single-modality model and injects the other modality through zero-initialized convolution layers. The authors report experiments on FLIR, LLVIP, and M3FD, with the headline result that MCF consistently improves YOLOv11 models by 3.41%-5.65% mAP on FLIR, reaching 47.61% AP.","tokens_in":34337,"tokens_out":9419,"duration_ms":87470,"significance":"If validated, the framework is a potentially useful engineering contribution: it provides a unified codebase for multispectral detection across many YOLO versions, and the observation that single-node P3 fusion often matches or beats multi-node fusion with fewer parameters is an interesting empirical finding. The MCF strategy, adapting ControlNet-style zero convolutions to modality injection, is a plausible and simple design. The open-sourced code and the breadth of models and datasets are strengths. However, the central quantitative claim is currently undermined by uncontrolled baseline choices and post-hoc hyperparameter selection; once those are corrected, the paper would provide a solid benchmark-style contribution.","major_comments":[{"comment":"The headline improvement range in the abstract is not supported by the controlled comparison in Table 7. The values 41.18/43.89/44.79/44.93/47.61 for YOLOv11n/s/m/l/x-RGBT-MCF are compared against the non-pretrained IR baselines in Table 3 (37.31/39.74/41.38/41.45/41.96) to obtain 3.41%-5.65%, but Table 7 itself reports COCO-pretrained IR baselines of 40.30/43.44/44.12/43.87/44.43. Against those same-protocol baselines the gains are 0.45-3.18 mAP, and YOLOv11s gains only 0.45. The sentence in §4.4 that the algorithm 'boosted mAP from 41.96% to 47.61%' likewise mixes the non-pretrained IR baseline with the pretrained MCF result. Please recompute all reported deltas using baselines trained under the same pretraining protocol, or state explicitly that the headline range is relative to a non-pretrained baseline.","section":"Abstract; §4.4; Tables 3 and 7"},{"comment":"The per-model choice of optimizer and learning rate (Adam for YOLOv11n/l/x, SGD for YOLOv11s/m) is made after observing the test-set results, and no seed or error-bar information is reported. This transforms the 'consistent improvement' claim into a post-hoc selection of the best of three configurations per model size. Please report mean and standard deviation over at least three seeds under a fixed, pre-specified training recipe, or use a separate validation split for hyperparameter selection and state this explicitly.","section":"§4.4, Table 6"},{"comment":"On M3FD, every MCF model underperforms the corresponding Midfusion-P3 model in Table 12 (e.g., YOLOv11s: 60.13 vs 62.20 AP; YOLOv11x: 64.23 vs 66.17 AP). The text in §4.6 discloses this, but the abstract and contribution (3) present MCF as a general mechanism for 'enhancing model adaptability and robustness'; the claims should be scoped to datasets such as FLIR and LLVIP where the controlled comparison is positive, or the method should be made adaptive to dataset characteristics.","section":"§4.6, Table 12"},{"comment":"Comparisons with prior published methods are drawn from literature numbers without reimplementation and without controlling for pretraining, input resolution, evaluation protocol, or training schedule. For example, the text in §4.4 claims superiority over CFT and other 2019-2024 methods based on Table 7, but those rows come from different papers and pipelines. Please either reimplement the most competitive baselines under the same protocol or clearly mark literature numbers as indicative and avoid 'better than' claims without a controlled comparison.","section":"§4.4, Table 7; §4.6, Table 12"}],"minor_comments":[{"comment":"The sentence 'YOLOv11s trained on multispectral data achieves an AP50 of 89.84% and an AP of 53.29%, which is better than the visible-light-only model's AP50 of 89.84% and AP of 53.29%' contains identical numbers for the two conditions and does not match Table 8; the values correspond to the RGB pretrained row in Table 9. Please correct the sentence and re-check similar narrative statements against the tables.","section":"Section 4.5"},{"comment":"The YOLOv4-Tiny row is labeled 'YOLOv4-Tiny-Midfusion' although the table reports Midfusion-P3 results; rename it for consistency with the other rows.","section":"Table 5"},{"comment":"The sentence 'K*K is consistent with formula 4' should refer to Eq. (2), and the DFL summation over p=0..3 should be clarified.","section":"Section 3.5, Eq. (4)"},{"comment":"The column header 'SDG' should read 'SGD'.","section":"Tables 6, 13, 14"},{"comment":"The parameter counts for YOLOv3-Tiny (98.89M) and YOLOv3 (11.57M) appear to be swapped relative to the usual model sizes; please verify and correct.","section":"Tables 2 and 3"},{"comment":"The text says experiments were run on five datasets, but only FLIR, LLVIP, and M3FD receive tabulated results; KAIST and VEDAI are mentioned but not tabulated. Please state clearly which datasets are used for quantitative evaluation.","section":"Section 4.3"},{"comment":"The abstract and contribution (1) mention six fusion modes, while §3.1 says the framework supports five modes plus weight-sharing; unify the count.","section":"Abstract; §3.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the journal's scope and the engineering contribution is real. I recommend major revision: the headline FLIR improvement needs to be recomputed against same-protocol baselines, and the per-model hyperparameter selection needs to be replaced by a controlled protocol. I see no indication of fabrication; the issues look like careless reporting that can be fixed by re-analysis and re-writing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read: this is a practical engineering paper, and the framework part is genuinely useful. The authors build a YOLOv11-RGBT package with six fusion modes, apply it across YOLOv3-YOLOv12 and RT-DETR, and add two ideas: P3 single-node mid-fusion and MCF, a ControlNet-style fine-tuning that freezes a pretrained single-modality detector and injects the other modality through zero-initialized convolutions. I haven't seen that exact trick in multispectral detection before. The ablation across fusion locations and modes is more systematic than most papers in this space. Credit where due: code is public, weights are promised at publication, and comparisons against a broad set of detectors are included.\n\nThe soft spot is real and load-bearing. The abstract's 3.41%-5.65% improvement comes from Table 7 MCF numbers compared with Table 3 IR-only models trained without COCO pretraining. Against the controlled baselines in Table 7, where both sides are COCO-pretrained, the deltas are 0.45 to 3.18 mAP, and YOLOv11s gains only 0.45. That changes the story from \"consistent large gains\" to \"most models gain a little, the large model gains more.\" Table 6 compounds this: the optimizer and learning rate are chosen per model after seeing results, so part of the gain is selection. There are no seeds or error bars. I am not saying the method is worthless: 47.61 on FLIR is still high, and the LLVIP MCF results look respectable. But the headline claim as written is not supported by the tables.\n\nSmaller things: Section 4.5 has an internal copy-paste error where the multispectral and visible-only numbers are identical, which undermines confidence in the surrounding prose. The paper says five datasets but reports detailed results for three; VEDAI and KAIST are mentioned but not evaluated in the main experiments. The M3FD result where MCF underperforms P3 is acknowledged in the text, and I appreciate that they say it plainly.\n\nBottom line: the framework and the MCF idea deserve serious referee time, but the paper needs a single-protocol evaluation, corrected abstract numbers, and error bars before acceptance. If the authors redo the comparison honestly, the core engineering claim — single-node P3 fusion plus zero-conv fine-tuning is a cheap and effective recipe for RGBT detection — will probably survive.","headline":"Useful multispectral detection framework, but the headline FLIR gain is inflated by comparing MCF against non-pretrained IR baselines; the controlled gains are much smaller.","tokens_in":34941,"tokens_out":1917,"would_cite":true,"duration_ms":19719,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that freezing a pretrained infrared detector and adding a visible-light branch through zero-initialized convolutions raises multispectral detection to 47.61% mAP on FLIR.","keywords":["multispectral object detection","YOLOv11-RGBT","multispectral controllable fine-tuning","P3 mid-fusion","RGB-thermal detection","pedestrian detection","single-stage detector","feature fusion strategy"],"falsifier":"Train the five YOLOv11 sizes on FLIR under one fixed protocol chosen before seeing test results: identical COCO initialization, identical epochs, identical optimizer and learning rate, and evaluate infrared-only, Midfusion, Midfusion-P3, and RGBT-MCF variants. If MCF does not beat infrared-only across all five sizes, or P3 does not beat multi-node fusion in most sizes, the paper's central claim of consistent gains fails.","tokens_in":33801,"feed_emoji":"🌡️","tokens_out":7822,"duration_ms":71195,"temperature":0.7,"pith_summary":"This paper sets out to give single-stage object detectors a reusable recipe for combining visible (RGB) and thermal infrared images. Its central proposal is YOLOv11-RGBT, a framework that instantiates several multispectral fusion modes on YOLO-family and transformer-based single-stage detectors, with two specific improvements: fuse the modalities once at the P3 feature layer instead of at many layers, and apply multispectral controllable fine-tuning (MCF), which freezes a pretrained single-modality model while a trainable second-modality branch is connected through zero-initialized convolutions. The authors report that on FLIR this combination consistently improves the infrared-only YOLOv11 baselines by 3.41-5.65 mAP, reaching 47.61% for the largest model, and that the same strategy generalizes to LLVIP and M3FD. The reason a sympathetic reader would care is that the recipe is architecture-agnostic and cheap: it upgrades an already-trained detector rather than training a fused network from scratch. The evidence stands or falls on whether the comparisons are controlled fairly across training protocols.","feed_headline":"YOLOv11-RGBT lifts multispectral detection to 47.61% mAP on FLIR","feed_subtitle":"Frozen infrared backbone plus a visible-light branch yields up to +5.65 mAP and a new single-node P3 fusion mode.","key_machinery":"The two load-bearing mechanisms are P3 mid-fusion and multispectral controllable fine-tuning (MCF). P3 mid-fusion concatenates the RGB and infrared feature maps only at the P3 layer of the neck and passes them through a trainable module, instead of fusing at P3, P4, and P5; the paper's hypothesis is that aligned multispectral features are redundant, so fewer fusion nodes reduce interfering and mismatched information. MCF copies a pretrained YOLOv11, typically the infrared model, into a frozen branch and attaches a trainable visible-light branch whose features are added into the frozen network through Zero Conv2d layers, trainable 2D convolutions initialized to zero, so the visible signal starts at zero and is learned gradually. These mechanisms jointly encode the claim that one modality should be treated as dominant and the other as a corrective stream.","core_discovery":"On the paper's own terms, the central discovery is that a single-stage multispectral detector should not fuse modalities everywhere. Fusing the RGB and infrared streams once at the P3 feature level beats the common P3-to-P5 multi-node fusion in most FLIR comparisons while using fewer parameters and fewer FLOPs. When pretrained weights are available, MCF extends this by freezing a COCO-pretrained infrared model and adding a trainable visible-light branch whose outputs enter through Zero Conv2d layers, 2D convolutions initialized to zero; this lets the visible stream contribute without overwriting the infrared features. The reported outcome is consistent gains over infrared-only models on FLIR (3.41-5.65 mAP, maximum 47.61% AP), a smaller but real gain on LLVIP, and a dataset-dependent outcome on M3FD where full multispectral transfer learning often beats MCF. The paper also reports that one modality usually dominates, infrared on FLIR and LLVIP and visible on VEDAI and M3FD, so the dominant modality should be the frozen branch.","pith_inferences":["Editorial inference: because MCF works by zero-initialized gating, it should extend to any pixel-aligned auxiliary modality such as depth, event cameras, or SAR, and the natural test is whether the same frozen-backbone recipe transfers without architectural changes.","Editorial inference: the redundancy explanation predicts that on misaligned or poorly registered RGB-T data, multi-node fusion should regain its advantage; running the paper's fusion-mode table on a misaligned benchmark would directly test this prediction.","Editorial inference: the paper selects optimizers and learning rates per model after observing results, so the headline 47.61% is an upper envelope; a fixed-protocol rerun is the stress test that would separate method strength from tuning luck.","Editorial inference: when full transfer learning already beats MCF, as on M3FD, MCF is best used as a fallback; a hybrid that unfreezes a few late layers after MCF might combine the stability of frozen features with the flexibility of full fine-tuning."],"forward_implications":["If the claims hold, an existing COCO-pretrained infrared detector can be turned into a multispectral detector by fine-tuning a small visible-light branch while the rest of the network stays frozen, cutting training cost and preserving learned features.","P3 single-node fusion provides a smaller and faster multispectral model than multi-node fusion, which matters for real-time deployment on edge devices.","The same fusion modes are reported to transfer across YOLOv3 through YOLOv12 and RT-DETR, so the recipe is not tied to one architecture.","Practitioners should choose which modality to freeze based on the dataset: the paper reports infrared dominance on FLIR and LLVIP and visible dominance on VEDAI and M3FD."],"supporting_citations":[{"why":"Defines the YOLOv11 architecture that the framework extends and that provides the RGB and IR baseline numbers.","marker":"[16]"},{"why":"ControlNet supplies the zero-initialized convolution conditioning idea that MCF adapts to multispectral detection.","marker":"[59]"},{"why":"FLIR is the primary dataset where the 3.41-5.65 mAP gains and the 47.61% top result are reported.","marker":"[40]"},{"why":"COCO pretrained weights initialize the frozen branch in the transfer-learning and MCF experiments.","marker":"[60]"},{"why":"TFDet provides the literature comparison numbers reproduced in the FLIR and LLVIP tables.","marker":"[61]"},{"why":"CFT is the mid-fusion multispectral baseline that the framework's P3 and MCF results are explicitly compared against.","marker":"[31]"},{"why":"LLVIP supplies the low-light pedestrian benchmark used to show that MCF generalizes beyond FLIR.","marker":"[44]"},{"why":"M3FD is the dataset where full multispectral transfer learning beats MCF, establishing the dataset-dependence claim.","marker":"[4]"}],"fun_headline_variants":["Fusing once at P3 beats multi-node fusion in YOLOv11-RGBT","Frozen IR + trainable RGB: YOLOv11-RGBT hits 47.61% mAP on FLIR","Multispectral fine-tuning adds up to 5.65 mAP in YOLOv11-RGBT","YOLOv11-RGBT: single-stage multispectral with P3 mid-fusion","Zero Conv2d lets RGB add to frozen IR in YOLOv11-RGBT"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported improvements are caused by P3 fusion and MCF rather than by differences in training protocol, because the paper mixes non-pretrained and COCO-pretrained comparisons, chooses optimizers and learning rates per model after seeing results, and compares against literature numbers that were not reimplemented under identical conditions.","fun_headline_variants_meta":{"raw":{"variants":["Fusing once at P3 beats multi-node fusion in YOLOv11-RGBT","Frozen IR + trainable RGB: YOLOv11-RGBT hits 47.61% mAP on FLIR","Multispectral fine-tuning adds up to 5.65 mAP in YOLOv11-RGBT","YOLOv11-RGBT: single-stage multispectral with P3 mid-fusion","Zero Conv2d lets RGB add to frozen IR in YOLOv11-RGBT"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00055,"raw_usage":{"total_tokens":2678,"prompt_tokens":1050,"completion_tokens":1628,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":666,"completion_tokens_details":{"reasoning_tokens":1505}},"tokens_in":666,"tokens_out":1628,"duration_ms":12104,"temperature":1.0,"reasoning_tokens":1505,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:48:01.483021+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the five YOLOv11 sizes on FLIR under one fixed protocol chosen before seeing test results: identical COCO initialization, identical epochs, identical optimizer and learning rate, and evaluate infrared-only, Midfusion, Midfusion-P3, and RGBT-MCF variants. If MCF does not beat infrared-only across all five sizes, or P3 does not beat multi-node fusion in most sizes, the paper's central claim of consistent gains fails.","supporting_citations":[{"cited_title":"YOLOv11: An overview of the key architectural enhancements","cited_arxiv_id":null,"evidence_quote":"Defines the YOLOv11 architecture that the framework extends and that provides the RGB and IR baseline numbers."},{"cited_title":"Adding conditional control to text-to-image diffusion models","cited_arxiv_id":null,"evidence_quote":"ControlNet supplies the zero-initialized convolution conditioning idea that MCF adapts to multispectral detection."},{"cited_title":"Multispectral fusion for object detection with cyclic fuse-and-refine blocks","cited_arxiv_id":null,"evidence_quote":"FLIR is the primary dataset where the 3.41-5.65 mAP gains and the 47.61% top result are reported."},{"cited_title":"TFDet: Target-aware fusion for RGB-t pedestrian detection","cited_arxiv_id":null,"evidence_quote":"TFDet provides the literature comparison numbers reproduced in the FLIR and LLVIP tables."},{"cited_title":"Cross-modality fusion transformer for multispectral object detection","cited_arxiv_id":null,"evidence_quote":"CFT is the mid-fusion multispectral baseline that the framework's P3 and MCF results are explicitly compared against."},{"cited_title":"LLVIP: A visible-infrared paired dataset for low-light vision","cited_arxiv_id":null,"evidence_quote":"LLVIP supplies the low-light pedestrian benchmark used to show that MCF generalizes beyond FLIR."}],"review_version":2}