{"id":"24ad830f-ff0f-4da0-bc4b-42473c90e216","arxiv_id":"2506.09429","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A compact encoder-decoder with edge-map fusion reports higher captioning scores on Sydney, UCM, and RSICD, though the gains rely on test-set model selection and possibly mismatched baselines.","lead":"Remote sensing image captioning can be made lighter by shrinking a transformer encoder, using a distilled GPT-2 decoder, and adding edge-detected image channels to preserve object boundaries. The paper reports higher captioning scores on three datasets, but the comparison to prior work is weakened by using corrected captions that earlier baselines did not use.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Uncontrolled benchmark: KD-EBF is evaluated on corrected captions [7] but baselines in Tables XIII-XV come from original-caption papers, so the claimed SOTA margins are not established.","rationale":"I agree with the reader's weakest_assumption: the comparison is not controlled because the proposed model is evaluated on the corrected datasets from [7] while the baseline numbers are copied from papers that used the original captions. This is more load-bearing than the test-set selection of PRF and edge detector, because even an honest internal comparison cannot repair a mismatch of reference captions across rows of the same table. The concrete check of re-running on original captions would settle it by showing whether the margins survive on a common benchmark. If they do not, the paper's central claim fails; if they do, the verdict should be revisited, though the test-set selection and underspecified distillation would remain secondary concerns. Since the reader already reached REJECT and this concern supports that conclusion, no verdict adjustment is needed.","tokens_in":13831,"tokens_out":9952,"duration_ms":104769,"concrete_test":"Re-run the complete KD-EBF pipeline on the original (unmodified) SYDNEY, UCM, and RSICD captions from [1] and [2], using the same train/validation/test splits as in Section IV.B, and recompute all seven metrics for Tables XIII-XV. If KD-EBF does not outperform the listed baselines on a majority of the seven metrics on each dataset, the abstract's claim of significant SOTA improvement is not supported. Report the same generated captions scored against both original and corrected references to quantify the benchmark shift.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that KD-EBF significantly improves caption quality compared to state-of-the-art methods rests on Tables XIII-XV, where baseline scores are taken from earlier papers. Section IV.B, however, states that the authors 'have used the modified version of these datasets [7]' for all experiments, meaning the SYDNEY, UCM, and RSICD reference captions were corrected by Das and Sharma. The baseline numbers in Tables XIII-XV were computed by the original papers on the original, uncorrected captions. Since BLEU, METEOR, ROUGE-L, and CIDEr all measure overlap with reference captions, changing the reference set changes every metric even for identical model outputs. A model trained and scored on corrected captions is therefore not directly comparable to published scores from the original benchmark. The reported margins over baselines such as Conv-GPT [6] do not establish an improvement unless all models are evaluated on the same reference captions. This is the single load-bearing weakness: it breaks the paper's headline comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a lightweight transformer-based remote sensing image captioning model (KD-EBF) that combines a ConvNeXt encoder with reduced channel dimensions through a Parameter Reduction Factor (PRF), a DistilGPT2 decoder, knowledge distillation from a larger teacher model, and an edge-enhanced input fusion strategy that concatenates an edge map (Canny, Sobel, or Laplacian) with the original image as a six-channel input. The paper evaluates on SYDNEY, UCM, and RSICD datasets and reports superior results over several baselines including the authors' previous TextGCN and Conv-GPT models, with extensive ablations over PRF values, edge detectors, and decoder types.","tokens_in":14011,"tokens_out":7833,"duration_ms":76036,"significance":"If the reported results are valid, the paper would make a useful empirical contribution by demonstrating that a substantially smaller transformer encoder with a distilled GPT-2 decoder and edge-augmented inputs can match or exceed the caption quality of much larger models, which is relevant for practical deployment. The systematic ablations of PRF, edge detectors, and decoders are valuable, and the paper includes a human evaluation. However, the main experimental claim of state-of-the-art performance is currently confounded by the use of corrected datasets for the proposed model while citing baseline numbers from papers that used the original datasets, an unspecified knowledge distillation procedure, and test-set-based hyperparameter selection without statistical significance. These issues must be addressed before the contribution can be accepted.","major_comments":[{"comment":"The paper states in Section IV.B that \"we have used the modified version of these datasets [7]\" for all experiments, yet the baseline scores in Tables XIII-XV are taken from published papers that evaluated on the original, uncorrected datasets. Since all seven metrics (BLEU-1 through BLEU-4, METEOR, ROUGE-L, CIDEr) measure n-gram overlap with reference captions, altering the reference text changes the metric values even for identical model outputs. The paper must re-evaluate all baselines on the corrected datasets (or use the original datasets for all methods) to provide a fair comparison; otherwise the claimed SOTA margins in Tables XIII-XV are not established.","section":"Section IV.B and Tables XIII-XV"},{"comment":"The knowledge distillation procedure is never described. The paper says knowledge is transferred from \"the full-size teacher model\" to the lightweight student, but it does not specify the teacher model (presumably PRF=1), the distillation loss function (e.g., soft-target KL divergence, feature matching), the temperature, the loss weights, or the training schedule. The \"Distil\" entries in Tables I-III and X-XII are therefore irreproducible, and one cannot determine whether the improvements are due to distillation or to re-initialization or retraining. A complete specification of the distillation method is required.","section":"Section III.A"},{"comment":"The best PRF and the best edge detector are chosen per dataset based on the test-set results (e.g., PRF=3 and Canny for SYDNEY; PRF=2 and Laplacian for UCM and RSICD). This constitutes test-set hyperparameter selection, which biases the comparison in favor of the proposed method because the baselines use fixed configurations. The authors should either select these hyperparameters on a held-out validation split and report the chosen values, or adopt a nested cross-validation scheme, to ensure an unbiased comparison.","section":"Section III.A (Tables I-III) and Section III.B (Tables IV-VI)"},{"comment":"No error bars, standard deviations, or significance tests are reported. On the SYDNEY dataset with only 58 test images, the differences between KD-EBF and the strongest baselines (e.g., BLEU-4 0.6498 vs 0.5786 for TextGCN in Table XIII) could easily fall within sampling noise. The paper should report mean plus or minus the standard deviation over at least three random seeds for all experiments and include a paired significance test (e.g., bootstrap) for the head-to-head comparisons in Tables XIII-XV.","section":"Throughout experimental section"}],"minor_comments":[{"comment":"The heading \"Peformance Metrices Used\" contains typos; it should be \"Performance Metrics Used\".","section":"Section IV.C"},{"comment":"The abbreviation \"CIDE\" should be \"CIDEr\".","section":"Section IV.C.4"},{"comment":"The method is called \"KD-EBD\" in these places, but \"KD-EBF\" is used throughout the rest of the paper; the name should be consistent.","section":"Section V.E and Figure 4"},{"comment":"The first column header \"Edge\" appears to be a copy-paste error and should be \"Method\" (or \"Model\").","section":"Tables XIII-XV"},{"comment":"The list of PRF modifications is clear, but the paper does not specify which components are excluded from reduction (e.g., the initial patch embedding) or how the interpolation-based weight resizing is implemented in detail; please provide these for reproducibility.","section":"Section III.A"},{"comment":"The exact parameters of the edge detectors (e.g., Canny thresholds, Sobel or Laplacian kernel sizes) are not given; please specify them for reproducibility.","section":"Section III.B"}],"recommendation":"major_revision","confidential_remarks":"The paper leans heavily on the authors' own prior work [6], [7]: the corrected datasets come from [7], and the primary transformer baseline Conv-GPT is from [6]. While this is not inherently improper, it means the comparison is partly self-referential, and the re-evaluation of baselines on the corrected datasets must be done transparently to avoid bias. The technical idea of combining a compressed ConvNeXt encoder with edge-map input fusion is plausible and the ablations are informative, so I recommend major revision rather than rejection. If the authors cannot re-evaluate all baselines, they should substantially temper the claim of \"state-of-the-art\" performance and instead claim parity or improvement when evaluated on the same, corrected benchmark."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's headline SOTA claim is not established because they evaluate on corrected captions from their own prior dataset paper while citing baseline scores from the original-caption benchmarks. That's the load-bearing flaw. The engineering combination itself is reasonable and the internal ablations are informative.\n\nWhat's genuinely new: the specific stack—PRF-based pruning of a ConvNext encoder, knowledge distillation from the full model to the pruned one, DistilGPT2 as decoder, and a six-channel input that fuses the original image with an edge map. Each component is known, but the combination is not in the literature. The paper also shows that the lightweight teacher-student setup actually improves over the bare model on small datasets (Tables I-III), which is a believable and useful result. The ablations (Tables X-XII) isolate the contribution of distillation and edge fusion sensibly.\n\nThe soft spots are real. Section IV.B states they used the modified versions of SYDNEY, UCM, and RSICD from Das & Sharma [7], with corrected spellings/grammar. Tables XIII-XV then compare KD-EBF against baselines whose published scores were computed on the original captions. Since BLEU/METEOR/ROUGE/CIDEr all measure overlap with reference captions, changing the references changes every metric even for identical outputs. The claimed margins over Conv-GPT, TextGCN, etc. are therefore not meaningful without re-running those baselines on the same corrected captions. This is not a minor detail; it breaks the central comparison.\n\nTwo additional problems: the best PRF and the best edge detector are chosen per dataset from the test tables, so the final numbers are selected on the test set without a validation split. No error bars or multiple seeds are reported, so we can't tell whether the gains over, say, PRF=3 vs PRF=2 are noise. And the knowledge distillation procedure is underspecified—no teacher model details, loss, temperature, or training schedule. The paper would need all three fixed.\n\nOn the positive side, the writing is clear, the limitations section is honest (they acknowledge data scarcity and capacity loss), and the internal teacher-student comparisons are self-consistent. This is a competent engineering paper with a fixable evaluation flaw.\n\nWho this is for: people working on lightweight RSIC or edge deployment will find the design worth replicating, as long as they re-evaluate properly. A serious referee should see it, because the core idea is sensible and the flaw is addressable. I'd recommend major revision: re-run all baselines on the same corrected captions, report standard deviations over multiple runs, and spell out the distillation recipe.\n\nVerdict: not reject outright; request revision. But as it stands, the SOTA claim is unsupported.","headline":"A clearly written engineering combination with an uncontrolled benchmark: the claimed SOTA margins rest on comparing corrected-caption results against original-caption baselines.","tokens_in":14545,"tokens_out":2543,"would_cite":false,"duration_ms":25487,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A lightweight transformer that fuses edge maps into its input reports the best captioning scores on three remote-sensing benchmarks.","keywords":["remote sensing image captioning","lightweight transformer","knowledge distillation","edge detection","image fusion","ConvNeXt","DistilGPT2","CIDEr"],"falsifier":"Run the same baseline models on the corrected SYDNEY, UCM, and RSICD splits and captions from [7]. If their BLEU and CIDEr numbers match the values copied into Tables XIII-XV, the claimed gaps hold; if their numbers change substantially, the reported improvement is largely an artifact of dataset revision rather than architecture.","tokens_in":13619,"feed_emoji":"🛰️","tokens_out":3058,"duration_ms":31705,"temperature":0.7,"pith_summary":"The paper tries to show that a deliberately shrunk transformer model can match or beat much larger captioning systems if it is trained with knowledge distillation and fed edge-enhanced images alongside the originals. It claims that this combination, called KD-EBF, improves caption quality on the SYDNEY, UCM, and RSICD datasets across BLEU-1 to BLEU-4, METEOR, ROUGE-L, and CIDEr. The practical payoff is that accurate remote-sensing image captioning could run with far fewer parameters, making it usable in resource-constrained settings.","feed_headline":"Small transformer plus edges beats larger captioning models","feed_subtitle":"KD-EBF sets top scores on SYDNEY, UCM, and RSICD using a distilled decoder and six-channel edge-fused input.","key_machinery":"The core mechanism is a Parameter Reduction Factor (PRF) that proportionally shrinks channel counts, normalization dimensions, and fully connected layers of the ConvNeXt encoder, with weights resized by interpolation. Knowledge distillation transfers the full teacher's behavior into this smaller student. An edge-aware fusion step extracts an edge map via Canny, Sobel, or Laplacian, concatenates it to the original RGB image, and feeds six-channel input through a modified first convolutional layer, so structural boundary cues enter the network without a second processing stream.","core_discovery":"The paper's central claim is that KD-EBF, an encoder-decoder model using a dimension-reduced ConvNeXt transformer encoder, a DistilGPT2 decoder, knowledge distillation from a full teacher, and an edge-detection-based input fusion, outperforms all listed baselines on all seven metrics on all three datasets. The authors further find that distilled lightweight versions with parameter reduction factors of 2 or 3 match or exceed the original model, and that adding edge information consistently improves results, with Canny best on SYDNEY and Laplacian best on UCM and RSICD.","pith_inferences":["A direct implication the paper leaves implicit is that the same edge-fusion trick could be dropped into other lightweight image-to-text models, not just this specific architecture, and likely give them a similar boost.","Because the paper trains and evaluates on the corrected captions of [7] while the baseline tables copy numbers from papers that used the original captions, re-running the baselines on the corrected datasets would reveal how much of the reported gap comes from data revision rather than from the model design.","The PRF-based shrinkage recipe, with interpolation-based weight resizing, could be tested as a general cheap way to slim other CNN-transformer encoders in vision-language tasks.","The human evaluation uses a single annotator; repeating the related/partially-related/unrelated rating with several annotators would test whether the perceived quality advantage is stable."],"forward_implications":["The six-channel input design lets a captioning model exploit edge structure without doubling the encoder, preserving the lightweight goal.","Distillation recovers most or all of the accuracy lost when the encoder is shrunk, so parameter reduction does not have to cost caption quality.","Edge fusion gives consistent metric gains on all three datasets, suggesting boundary information is a broadly useful complement to semantic features in remote-sensing scenes.","KD-EBF's reported numbers on SYDNEY, UCM, and RSICD would be the new best known results for these benchmarks if the comparisons hold.","The resulting model has a much smaller decoder footprint than GPT-2 or RoBERTa, easing deployment on limited hardware."],"supporting_citations":[{"why":"Supplies the SYDNEY and UCM caption datasets and the original encoder-decoder formulation the paper builds on.","marker":"[1]"},{"why":"Provides the RSICD dataset and the R-BOW and L-FV baselines that appear in the comparison tables.","marker":"[2]"},{"why":"Supplies the TrTr-CMR dual-transformer baseline, the strongest transformer comparison in the tables.","marker":"[5]"},{"why":"Gives the Conv-GPT baseline, the direct transformer-based predecessor whose decoder choices the paper compares against.","marker":"[6]"},{"why":"Supplies the corrected and cleaned versions of the three datasets used for training and evaluation, plus the TextGCN baseline.","marker":"[7]"},{"why":"Defines the Canny edge detector, the best fusion choice on the SYDNEY dataset.","marker":"[20]"},{"why":"Defines the Laplacian edge detector, the best fusion choice on the UCM and RSICD datasets.","marker":"[22]"}],"fun_headline_variants":["Edge-aware fusion lifts lightweight remote sensing captioning","Edges plus distilled decoder beat big captioning models","Lightweight transformer with edge cues tops remote captioning","Distilled GPT-2 decoder plus edge fusion improves captions","Small edge-aware model wins remote captioning benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline comparisons assume the published baseline scores, produced on the original captions, are directly comparable to the proposed model trained and evaluated on the corrected datasets of [7].","fun_headline_variants_meta":{"raw":{"variants":["Edge-aware fusion lifts lightweight remote sensing captioning","Edges plus distilled decoder beat big captioning models","Lightweight transformer with edge cues tops remote captioning","Distilled GPT-2 decoder plus edge fusion improves captions","Small edge-aware model wins remote captioning benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000673,"raw_usage":{"total_tokens":3007,"prompt_tokens":831,"completion_tokens":2176,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":447,"completion_tokens_details":{"reasoning_tokens":2100}},"tokens_in":447,"tokens_out":2176,"duration_ms":17955,"temperature":1.0,"reasoning_tokens":2100,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:48:11.860631+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same baseline models on the corrected SYDNEY, UCM, and RSICD splits and captions from [7]. If their BLEU and CIDEr numbers match the values copied into Tables XIII-XV, the claimed gaps hold; if their numbers change substantially, the reported improvement is largely an artifact of dataset revision rather than architecture.","supporting_citations":[{"cited_title":"Deep semantic understanding of high resolution remote sensing image,","cited_arxiv_id":null,"evidence_quote":"Supplies the SYDNEY and UCM caption datasets and the original encoder-decoder formulation the paper builds on."}],"review_version":1}