{"id":"1bbfe956-944b-466a-a0f8-d0c9e5e5a2af","arxiv_id":"2504.15728","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"SAGA, a semantic-aware instance-level grayscale augmentation, consistently improves RGB-to-thermal domain adaptation for object detection, and a new drone dataset IndraEye is introduced.","lead":"Researchers propose SAGA, an augmentation that turns objects in RGB training images gray while keeping backgrounds colored, to help detectors adapt to thermal cameras. They also release IndraEye, a new drone RGB-thermal dataset, and report small but consistent accuracy gains on several benchmarks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 5's CMT baseline on LLVIP (41.6 vs source 82.8) and the contradiction with Table 9 make the main evidence for SAGA's gains unreliable; the central claim is not currently supported.","rationale":"The reader's weakest_assumption identifies the lack of a physical model for grayscale-to-thermal equivalence. That is a valid generalization risk, but it is not the most load-bearing issue: a purely empirical augmentation can be acceptable if the experiments are sound. The load-bearing issue is that the main experimental table, which is the sole support for the central claim, contains internally inconsistent baseline numbers. In particular, CMT at 41.6 on LLVIP versus Source at 82.8 is implausible, and Table 9 contradicts Table 5. If the CMT baseline is actually near source-level, then SAGA's reported gain would be a large decrease. This cannot be resolved by theory; it requires re-running the experiments. I do not accuse the authors of misconduct; the inconsistencies could be typos or mislabeled tables. But as written, the central claim is unsupported, and I cannot verify it. Therefore the appropriate status is UNVERDICTED—the paper should not be accepted until the authors release code and reproduce Table 5 and identify the setting of Table 9. This is a stronger position than the reader's CONDITIONAL because the concern targets the central evidence itself, not peripheral errors. The concrete test would settle the concern: if the CMT numbers reproduce as reported and training has not diverged, then the claim holds; if the baseline shifts, the claim fails. Agreement is partial because the reader did identify internal inconsistencies in the rationale, but their stated weakest_assumption was the heuristic's lack of physical grounding, which is a different concern.","tokens_in":14321,"tokens_out":8905,"duration_ms":72367,"concrete_test":"Run the released CMT code with the paper's hyperparameters (VGG16, 50k iterations, batch 16, burn-up 10k) on LLVIP (RGB→IR) and reproduce Table 5's Source, CMT, and CMT+SAGA rows. Also run CMT on IndraEye with the 20k schedule. If CMT-LLVIP reproduces ≈41.6 and source ≈82.8, verify that CMT has not diverged by inspecting loss and pseudo-label quality; if CMT-LLVIP reproduces ≈82.8, Table 5's CMT+SAGA value is a decrease and the claimed +7.6 gain is invalid.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim—SAGA yields consistent +0.4 to +7.6 mAP gains—rests entirely on Table 5. That table contains an implausible baseline: CMT on LLVIP is reported as 41.6 mAP, while the same table reports Source (RGB-trained, no adaptation) at 82.8 mAP and D3T at 88.1 mAP. An adaptation method that drops 41 points below its own source model is not credible without a documented failure mode; no such explanation is given. The anomaly repeats on IndraEye, where CMT (17.1) is 33 points below Source (50.5). Additionally, Table 9 lists CMT at 65.7 mAP and CMT+SAGA at 65.7 mAP—identical—while Table 5 reports CMT at 63.3 and CMT+SAGA at 65.7 for (presumably) the same FLIR benchmark. Since the dataset and setting for Table 9 are unstated, and its numbers overlap with Table 5's CMT+SAGA row, the tables cannot be reconciled. If the CMT-LLVIP baseline is actually near 82.8, then CMT+SAGA at 49.2 would represent a 33-point drop, directly contradicting the claimed +7.6 gain. Thus the empirical support for SAGA is internally inconsistent and cannot be accepted as evidence without reproduction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes SAGA, an instance-level color augmentation that converts foreground object pixels in source RGB images to grayscale while retaining colored backgrounds, intended to reduce the RGB-to-thermal domain gap in unsupervised domain adaptation (UDA) for object detection. The authors also present IndraEye, a new RGB-IR drone dataset with detection and segmentation annotations over 13 classes, and evaluate SAGA by integrating it with CMT and D3T on FLIR, LLVIP, and IndraEye, reporting consistent mAP improvements of +0.4 to +7.6. Additional experiments benchmark object detectors on IndraEye and examine cross-dataset generalization and compatibility with the Jigsaw pretraining method.","tokens_in":14567,"tokens_out":6896,"duration_ms":54862,"significance":"The contribution is potentially useful: a simple, training-only augmentation that improves RGB-to-IR adaptation would be readily adoptable, and a new public multi-sensor drone dataset with both detection and segmentation labels is a valuable resource. The paper is honest in that SAGA does not fit parameters to target data, and the gains are measured against external baselines. However, the central experimental evidence in Table 5 and Table 9 contains serious internal inconsistencies that must be resolved before the claims can be assessed; the method's defining equation is also imprecise. The dataset contribution is substantial and likely of interest to the community, but the paper as a whole is not yet self-consistent.","major_comments":[{"comment":"The grayscale conversion in Eq. (3) is not a valid grayscale transformation. The standard luminance conversion is Y = 0.2989R + 0.587G + 0.114B, producing a single channel, whereas Eq. (3) multiplies the R, G, and B channels by the respective constants and preserves three channels, which would change color relationships without removing chromatic information. Since SAGA is defined by this equation, the exact pixel operation must be specified unambiguously (e.g., setting R=G=B=Y) and the implementation used in the experiments should be described.","section":"3.2, Eq. (3)"},{"comment":"The CMT baseline is reported as 41.6 mAP on LLVIP and 17.1 on IndraEye, while the Source row (no adaptation, RGB-trained model tested on IR) is 82.8 and 50.5, respectively. An adaptation method performing 41 and 33 points below its own source model is highly implausible and no failure mode is documented. If these baselines are incorrectly reported, the claimed gains of +7.6 and +2.2 for CMT+SAGA are not supported; if they are correct, the adaptation setup itself is broken. This anomaly must be resolved before the central claim can be evaluated.","section":"Table 5, LLVIP and IndraEye CMT rows"},{"comment":"Table 9 is missing a dataset label and its numbers are inconsistent with Table 5 for what appears to be the same setting: CMT is 65.7 in Table 9 but 63.3 in Table 5, CMT+SAGA is 65.7 in both tables (implying zero gain) while Table 5 reports +2.4, and Oracle is 63.3 in Table 9 but 67.5 in Table 5. Since Table 9 is the only evidence for compatibility with Jigen, these unresolved discrepancies leave that claim unsupported.","section":"Table 9"},{"comment":"The text states that SAGA improves CMT by 3.6 mAP on LLVIP, but Table 5 reports an improvement of +7.6 mAP; this text/table mismatch compounds the baseline problem above. Additionally, D3T on LLVIP (88.1) exceeds the IR Oracle (83.0), which is surprising for an unsupervised adaptation method and warrants an explanation or a check of the Oracle definition.","section":"Section 4.2.1, Table 5"}],"minor_comments":[{"comment":"In Eq. (1), \"Ltrt\" appears to be a typo for \"L_tgt\"; please correct.","section":"Equation (1)"},{"comment":"Table 9 lacks a caption identifying the dataset and experimental setting.","section":"Table 9"},{"comment":"Section 4.1 states that AT is evaluated, but no AT results appear in Table 5 or elsewhere.","section":"Section 4.1, Table 5"},{"comment":"The abstract claims 145,666 instances, but summing the scene-wise instances in Table 1 gives approximately 142,991; this discrepancy and the class-list count (14 entries versus the stated 13 classes) should be reconciled.","section":"Abstract, Table 1"},{"comment":"The claim that IndraEye is \"the first multisensor aerial perception dataset\" is contradicted by VEDAI, DroneVehicle, and M3FD, which the paper itself lists in Table 2; the wording should be softened.","section":"Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The IndraEye dataset is arguably the strongest part of the submission and would be of value to the community. Before any further consideration, the authors should be asked to provide exact experimental logs or corrected tables, particularly for CMT on LLVIP and IndraEye, and to reconcile Table 5 with Table 9. If the CMT baselines are actually near the Source numbers, the paper's central claim collapses; if the baselines are correct, the adaptation setup is pathological and requires explanation. I would also recommend independent verification of the grayscale implementation described by Eq. (3)."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the honest take. The paper has one genuinely new idea—instance-level grayscale on object boxes for RGB-to-IR domain adaptation—and one new dataset, IndraEye, that looks like a useful resource. But the main results table is internally contradictory, and that's a load-bearing problem.\n\nWhat's good: SAGA is simple, training-free, and easy to add to existing mean-teacher UDA pipelines. The ablation against full-image grayscale (Table 8) is the right control and shows the instance-level twist matters. IndraEye offers 5,612 RGB/IR images with detection and segmentation labels, day/night splits, and diverse drone viewpoints; if it's released cleanly, it fills a real gap in aerial thermal perception. The dataset construction is described in reasonable detail.\n\nWhere it falls down: Table 5 has numbers that don't survive scrutiny. CMT on LLVIP is reported at 41.6 mAP, while the non-adapted Source is 82.8 and D3T is 88.1. An adaptation method performing 41 points below its own source model would need an explicit failure-mode explanation; none is given. The same pattern appears on IndraEye (CMT 17.1 vs Source 50.5). Then Table 9—whose dataset is not stated—shows CMT and CMT+SAGA both at 65.7, identical, while Table 5 has CMT at 63.3 and CMT+SAGA at 65.7 for FLIR. The two tables can't be reconciled, making the reported +7.6 mAP gain on LLVIP especially suspect. Eq. (3) is also not a grayscale conversion; it's a channel-wise multiplication by the luminance weights, which yields a dark colored image, not grayscale. That's a math/description error, but it casts doubt on how carefully the experiments were run and reported.\n\nThese aren't minor typos. The central claim of consistent gains rests entirely on a table whose baselines look wrong. It's possible the underlying effect is real, but I can't take the current numbers as evidence.\n\nThe paper deserves a serious referee—the method is cheap to test and the dataset is potentially useful—but the referee should request the original logs, corrected tables, and code/data before anything is accepted. I wouldn't cite it in its current form.","headline":"Simple, plausible augmentation and a new aerial dataset, but the main results table is internally contradictory; the empirical claims need verification before they can be trusted.","tokens_in":15145,"tokens_out":2802,"would_cite":false,"duration_ms":25376,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes SAGA, a training-time augmentation that grayscales only object instances, and reports +0.4 to +7.6 mAP gains for RGB-to-thermal adaptation.","keywords":["domain-adaptive thermal object detection","RGB-to-IR unsupervised domain adaptation","grayscale color augmentation","mean teacher pseudo-labels","RGB-IR drone dataset","object detection","thermal imaging","IndraEye benchmark"],"falsifier":"A direct test would compare CMT with and without SAGA on a thermal target containing objects with low or inverted thermal contrast, for example pedestrians on a hot day or vehicles right after engine-off; if the mAP gain disappears or reverses on those classes, the grayscale-proxy assumption is the failing link.","tokens_in":14110,"feed_emoji":"🔥","tokens_out":10379,"duration_ms":85962,"temperature":0.7,"pith_summary":"The paper tries to establish that a source-only augmentation, applied while training a visible-RGB detector for use on unlabeled thermal images, can close much of the modality gap without paired RGB-IR data. SAGA takes each labeled object instance in a source RGB image, converts just those pixels to grayscale, and leaves the background in color. Run inside existing mean-teacher adaptation methods, SAGA yields reported mAP gains of +0.4 to +7.6 across the FLIR, LLVIP, and newly introduced IndraEye datasets. The same paper introduces IndraEye, a 5,612-image RGB-IR drone dataset with 145,666 instances across 13 classes, offered as a benchmark for adaptation, detection, and segmentation. A sympathetic reader would care because the proposal is a drop-in preprocessing step, not a new detector architecture, so improvements would transfer to existing pipelines.","feed_headline":"Gray out objects, not backgrounds, to lift thermal mAP by up to 7.6","feed_subtitle":"It grays out object regions only and improves adaptation accuracy on FLIR, LLVIP, and IndraEye.","key_machinery":"The machinery is the hybrid RGB-grayscale image produced by SAGA. For each source image, the paper extracts every labeled object region, multiplies its red, green, and blue channels by the luminance weights $[0.2989, 0.5870, 0.1140]$, and pastes the gray object back into the color background. This object-level gray image is fed into a student-teacher adaptation loop in which the teacher's weights are an exponential moving average of the student's weights. The augmentation's role is to strip the dominant color cue from objects while keeping scene context intact, so the teacher generates better pseudo-labels on unlabeled IR frames.","core_discovery":"The central discovery is that replacing color inside object bounding boxes with grayscale, while keeping the color background, improves RGB-to-IR adaptation in mean-teacher detectors. In the paper's experiments, the contrastive mean teacher (CMT) rises from 63.3 to 65.7 mAP on FLIR and from 41.6 to 49.2 mAP on LLVIP, while the dual-domain teacher (D3T) rises from 61.8 to 66.1 mAP on FLIR; on IndraEye, CMT rises from 17.1 to 19.3 and D3T from 30.3 to 32.9 mAP. The paper attributes the improvement to reduced color bias, which yields fewer false positives and better pseudo-labels on the unlabeled IR target. It also reports that converting the entire image to grayscale lowers CMT on FLIR to 61.1 mAP, supporting the claim that preserving background context is doing part of the work.","pith_inferences":["Because SAGA is a source-side preprocessing step rather than a change to the detector, the same recipe could plausibly extend to other self-training or adversarial RGB-to-IR pipelines and to semantic segmentation, although the paper reports only detection experiments.","The grayscale weights used are luminance weights tuned to human vision, not a physical model of thermal emission; classes with weak or inverted thermal contrast, such as cool objects in hot scenes, may not follow the reported pattern.","The mechanism likely acts through pseudo-label quality, so ablating the teacher's confidence threshold or the EMA decay rate could reveal whether SAGA's benefit is concentrated in the early adaptation phase or persists throughout training.","A stronger variant would replace neutral gray with a class-specific synthetic thermal appearance, which would test whether the gray proxy is a ceiling of the idea or just its first approximation."],"forward_implications":["Adding SAGA to the contrastive mean teacher raises reported mAP by +2.4 on FLIR, +7.6 on LLVIP, and +2.2 on IndraEye.","Adding SAGA to the dual-domain teacher raises reported mAP by +4.3 on FLIR, +0.4 on LLVIP, and +2.6 on IndraEye.","Converting the entire image to grayscale lowers CMT performance on FLIR to 61.1 mAP, below the 63.3 no-augmentation baseline, so object-level selectivity matters.","SAGA remains beneficial when combined with jigsaw-based pretraining, reaching 68.2 mAP for D3T+SAGA on FLIR versus 66.1 without the pretraining.","Qualitative results in the paper show fewer false positives and better detection of small objects in IR drone imagery when SAGA is used."],"supporting_citations":[{"why":"The contrastive mean-teacher baseline that SAGA augments and is compared against on all three datasets.","marker":"[3]"},{"why":"The dual-domain teacher baseline whose reported adaptation results SAGA improves on FLIR and IndraEye.","marker":"[13]"},{"why":"A state-of-the-art adaptive-teacher baseline included in the paper's method set for testing SAGA.","marker":"[34]"},{"why":"Supplies the mean-teacher exponential-moving-average update mechanism used throughout the adaptation framework.","marker":"[55]"},{"why":"Provides the LLVIP low-light paired RGB-IR dataset used as a target-domain evaluation benchmark.","marker":"[26]"},{"why":"Provides the aligned FLIR RGB-IR benchmark used for the autonomous-driving adaptation experiments.","marker":"[66]"},{"why":"The jigsaw-based pretraining method whose compatibility with SAGA is tested in the paper's Table 9.","marker":"[5]"}],"fun_headline_variants":["Gray objects, keep backgrounds: thermal mAP gains up to 7.6","Gray only objects to close RGB-to-IR gap, mAP +7.6","Semantic graying increases thermal detection by up to 7.6 mAP","Selective graying of objects improves thermal adaptation up to 7.6 mAP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that a grayscale version of an RGB object is a good enough stand-in for its thermal image to improve teacher pseudo-labels, even though grayscale luminance and thermal radiance are not physically tied.","fun_headline_variants_meta":{"raw":{"variants":["Gray objects, keep backgrounds: thermal mAP gains up to 7.6","Gray only objects to close RGB-to-IR gap, mAP +7.6","Semantic graying increases thermal detection by up to 7.6 mAP","Selective graying of objects improves thermal adaptation up to 7.6 mAP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001079,"raw_usage":{"total_tokens":4561,"prompt_tokens":1041,"completion_tokens":3520,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":3432}},"tokens_in":657,"tokens_out":3520,"duration_ms":21527,"temperature":1.0,"reasoning_tokens":3432,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:18:42.653648+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would compare CMT with and without SAGA on a thermal target containing objects with low or inverted thermal contrast, for example pedestrians on a hot day or vehicles right after engine-off; if the mAP gain disappears or reverses on those classes, the grayscale-proxy assumption is the failing link.","supporting_citations":[{"cited_title":"Contrastive mean teacher for domain adaptive ob- ject detectors","cited_arxiv_id":null,"evidence_quote":"The contrastive mean-teacher baseline that SAGA augments and is compared against on all three datasets."},{"cited_title":"D3t: Distinctive dual-domain teacher zigzagging across rgb- thermal gap for domain-adaptive object detection","cited_arxiv_id":null,"evidence_quote":"The dual-domain teacher baseline whose reported adaptation results SAGA improves on FLIR and IndraEye."},{"cited_title":"Cross-domain adaptive teacher for object detection","cited_arxiv_id":null,"evidence_quote":"A state-of-the-art adaptive-teacher baseline included in the paper's method set for testing SAGA."},{"cited_title":"Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results","cited_arxiv_id":null,"evidence_quote":"Supplies the mean-teacher exponential-moving-average update mechanism used throughout the adaptation framework."},{"cited_title":"Llvip: A visible-infrared paired dataset for low-light vision","cited_arxiv_id":null,"evidence_quote":"Provides the LLVIP low-light paired RGB-IR dataset used as a target-domain evaluation benchmark."},{"cited_title":"Multispectral fusion for object detection with cyclic fuse-and-refine blocks","cited_arxiv_id":null,"evidence_quote":"Provides the aligned FLIR RGB-IR benchmark used for the autonomous-driving adaptation experiments."},{"cited_title":"Domain generalization by solving jigsaw puzzles","cited_arxiv_id":null,"evidence_quote":"The jigsaw-based pretraining method whose compatibility with SAGA is tested in the paper's Table 9."}],"review_version":1}