{"id":"386b1c87-b41b-4a38-ba27-da1a01e77d24","arxiv_id":"2504.16761","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Tri-FusionNet is a new ViT+RoBERTa+CLIP captioning model claiming state-of-the-art scores that do not survive internal consistency checks.","lead":"This paper proposes Tri-FusionNet, an image captioning model that combines a Vision Transformer encoder with dual attention, a RoBERTa-style decoder, and CLIP-based fusion. It claims record results on three standard benchmarks, but those scores are internally contradictory and far beyond any published captioning system.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central SOTA claim is not supported because the reported evaluation numbers are internally inconsistent (abstract vs. Table VIII, prose vs. tables) and no evaluation protocol or code is provided; the claim therefore has no verifiable basis.","rationale":"The reader's REJECT is well-founded. My independent reading found the same load-bearing weakness: the only evidence for the central SOTA claim is a set of metric tables whose origin and protocol cannot be verified. The internal contradictions are directly in the text: the abstract's Flickr8k numbers (1.483/0.358/0.789) disagree with Table VIII (1.381/0.389/0.654), and Section IV.B assigns these same numbers to MS-COCO; Section IV.A mislabels the columns of Table III in a systematic way (calling BLEU-1 'METEOR', BLEU-2 'ROUGE-L', METEOR 'BLEU-2', etc.). These are not stylistic issues: they make it impossible to know which run produced which number. The claimed MS-COCO values are also far outside the published range in the paper's own comparison table, and the qualitative examples are short and generic. The manuscript consistently refers to 'supplementary.tex'/'supplementary.txt' for algorithms and implementation details, but no supplementary file is included. A machine-checked proof or release of code would be independent support; none exists. Because the central claim is unsupported and internally inconsistent, the reader's REJECT verdict should stand unchanged.","tokens_in":16926,"tokens_out":7537,"duration_ms":60346,"concrete_test":"Obtain the authors' exact evaluation script and model outputs (or, failing that, request the missing supplementary files and reimplement the described architecture), and evaluate on the standard MS-COCO Karpathy test split with pycocoevalcap and the same reference files used by the paper. If the reproduced BLEU-1/BLEU-4/METEOR/CIDEr/ROUGE-L values do not match Table VIII (0.893/0.725/0.780/1.880/0.689) within the 95% bootstrap confidence interval of the metric, the state-of-the-art claim is unsupported. This test also requires the authors to state which of the two Flickr8k triplets (abstract or Table VIII) came from the reported run.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—state-of-the-art image captioning on MS-COCO, Flickr30k, and Flickr8k—rests entirely on the correctness and mutual consistency of the reported metric tables. That foundation fails internal checks. The abstract gives Flickr8k as CIDEr 1.483, METEOR 0.358, ROUGE-L 0.789, whereas Table VIII gives Flickr8k as CIDEr 1.381, METEOR 0.389, ROUGE-L 0.654; Section IV.B then assigns the first triplet to MS-COCO. Section IV.A's prose misattributes every metric in the ViT+RoBERTa+CLIP row of Table III (e.g., the 0.741 BLEU-1 is called METEOR, the 0.621 BLEU-2 is called ROUGE-L, the 0.389 METEOR is called BLEU-2). The headline MS-COCO METEOR 0.780 and CIDEr 1.880 are far outside the range reported by any comparable system in Table X (best METEOR 0.392, best CIDEr 1.415), and no evaluation protocol, split definition, beam size, or code is provided. The paper repeatedly defers implementation details to 'supplementary.tex'/'supplementary.txt', but no supplementary file is included. If the numbers cannot be reproduced or even consistently attributed, the 'breaks all records' conclusion has no evidentiary basis.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Tri-FusionNet, an image-captioning model that combines a Vision Transformer (ViT) encoder with a dual-attention mechanism, a RoBERTa decoder, and a CLIP integration module. The authors report state-of-the-art results on MS-COCO, Flickr30k, and Flickr8k across BLEU, METEOR, CIDEr, and ROUGE-L, and present ablations, qualitative examples, and comparisons with prior work. The central claim is that Tri-FusionNet surpasses all existing models on the three benchmarks, with headline numbers of BLEU 0.893, METEOR 0.780, CIDEr 1.880, and ROUGE-L 0.689 on MS-COCO.","tokens_in":17260,"tokens_out":2602,"duration_ms":20635,"significance":"If the reported results were reliable, the paper would be a substantial contribution: a single architecture outperforming all prior image-captioning systems on three standard benchmarks by a wide margin would be noteworthy, and the fusion of ViT, RoBERTa, and CLIP with dual attention is a plausible design direction worth exploring. The paper's qualitative examples and heatmap analysis do provide some illustrative value. However, the significance hinges entirely on the correctness of the quantitative evaluation, and the manuscript's internal inconsistencies and improbable metric values undermine that foundation. The paper does not ship code, reproducible evaluation scripts, or the promised supplementary files, so there is no independent verification path either.","major_comments":[{"comment":"The reported numbers are internally inconsistent. The abstract states Flickr8k results as CIDEr 1.483, METEOR 0.358, ROUGE-L 0.789, while Table VIII reports the same dataset as CIDEr 1.381, METEOR 0.389, ROUGE-L 0.654. Moreover, Section IV.B's prose assigns the abstract's Flickr8k triplet (1.483, 0.358, 0.789) to MS-COCO instead of the values in Table VIII (1.88, 0.78, 0.689). Since the central state-of-the-art claim rests on these tables, this mutual contradiction means the evaluation evidence is not trustworthy as presented.","section":"Abstract vs. Table VIII and Section IV.B"},{"comment":"The prose misattributes every metric in the ViT+RoBERTa+CLIP rows. For Table III (Flickr30k), the row contains B-1=0.741, B-2=0.621, B-3=0.573, B-4=0.428, C=1.092, M=0.389, R-L=0.432, but the text states '0.741 for METEOR, 0.621 for ROUGE-L, 0.573 for BLEU-4, 0.428 for BLEU-3, 1.092 for CIDEr, 0.389 for BLEU-2, and 0.432 for BLEU-1.' The same misattribution occurs in the Flickr8k discussion. If the prose cannot correctly map reported numbers to their columns, the reader has no way to know which values actually correspond to which metrics.","section":"Section IV.A, Tables III and IV"},{"comment":"The claimed MS-COCO METEOR of 0.780 and CIDEr of 1.880 are far outside the range of all comparable systems in Table X, where the best listed METEOR is 0.392 and the best CIDEr is 1.415. No evaluation protocol is given: the paper does not specify the test split, beam size, caption preprocessing, or whether the online MS-COCO test server was used for the numbers in Table VIII. The sentence in Section IV.C that Tri-FusionNet 'breaks all other records as the new baseline' is therefore unsupported by reproducible evidence.","section":"Section IV.C, Table X"},{"comment":"The architecture description is incomplete in a way that prevents reproducibility. RoBERTa is a masked-language-model encoder, yet the paper uses it as a decoder without specifying the output layer, vocabulary projection, or how captions are generated during inference. The implementation details are repeatedly deferred to 'supplementary.tex' and 'supplementary.txt' (e.g., Sections III.A.1, III.A.4, and IV), but no supplementary file is included with the manuscript. This makes it impossible to verify the proposed method or the reported numbers.","section":"Section III.A.3 and Supplementary Material"}],"minor_comments":[{"comment":"The text before Table III says the table reports Flickr8k results, while the caption and the actual values correspond to Flickr30k; please correct the mismatch between the caption, the prose, and the table content.","section":"Section IV.A"},{"comment":"The 'BLEU' column is ambiguous; it should state whether it refers to BLEU-1, BLEU-4, or an average, because the comparison with prior work is otherwise not meaningful.","section":"Tables X-XIII"},{"comment":"The paragraph after Table VIII repeats the wrong metric-to-dataset assignment: BLEU values for MS-COCO are followed by CIDEr/METEOR/ROUGE-L values 1.483/0.358/0.789, which are neither the MS-COCO values in Table VIII nor the Flickr8k values in Table VIII.","section":"Section IV.B"}],"recommendation":"reject","confidential_remarks":"The paper's empirical claims are contradicted by its own tables and prose, and the reported state-of-the-art numbers are implausibly high without any evaluation protocol or code. This is not a case of a fixable local error; the central evidence is unreliable. I would recommend rejection rather than major revision, because the authors would need to re-run all experiments and report consistent, verifiable numbers to establish any of the claimed conclusions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this one should be rejected. The architecture is a reasonable combination of ViT with dual attention, a RoBERTa decoder, and CLIP-based fusion, and the method section provides clear equations for the attention mechanisms. That part is a legitimate research direction. But the experimental reporting is a mess, and the claimed state-of-the-art numbers don't survive contact with the paper's own tables.\n\nThe internal inconsistencies are not subtle. The abstract gives Flickr8k CIDEr 1.483, METEOR 0.358, ROUGE-L 0.789, while Table VIII reports 1.381, 0.389, 0.654. Section IV.A's prose mislabels the metrics in the ViT+RoBERTa+CLIP row of Table III: BLEU-1 0.741 is called METEOR, BLEU-2 0.621 is called ROUGE-L, and METEOR 0.389 is called BLEU-2. In Section IV.B, after listing MS-COCO BLEU scores, the text assigns CIDEr 1.483, METEOR 0.358, ROUGE-L 0.789 to MS-COCO—these are actually the abstract's Flickr8k numbers. On MS-COCO, the proposed model claims METEOR 0.780 and CIDEr 1.880, while the best comparable system in Table X has METEOR 0.392 and CIDEr 1.415. That is not a step change; it is a red flag that the evaluation protocol or metric implementation is wrong. Table XI, supposedly on the online MS-COCO test server, shows METEOR 0.750 against a previous best of 0.338.\n\nThere is also no code, no hyperparameters, no split details, no beam size, and the paper repeatedly defers to 'supplementary.tex' or 'supplementary.txt' that are not included. The comparison tables use a single 'BLEU' column without specifying which BLEU (1-4), which makes the comparisons hard to interpret.\n\nWhat is genuinely useful here? The idea of combining spatial window attention and channel-group attention in a ViT encoder with a RoBERTa decoder and CLIP contrastive alignment is plausible. If the experiments were redone properly and the numbers came back consistent, the paper could be a modest contribution to image captioning. As it stands, the central claim has no verifiable basis.\n\nI would desk reject this. If you send it to peer review, the reviewers will just spend their time enumerating these inconsistencies. The authors need to re-run the evaluation, release code and settings, and correct the numbers before it deserves a serious referee.\n\nRecommendation: reject.","headline":"Architecture is a plausible mashup of ViT, RoBERTa, and CLIP, but the reported results are internally inconsistent and the SOTA claims do not hold up.","tokens_in":17769,"tokens_out":4466,"would_cite":false,"duration_ms":37089,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Tri-FusionNet claims record image captioning scores by fusing ViT, RoBERTa, and CLIP with dual attention.","keywords":["Image Description Generation","Natural Language Processing","Computer Vision","Transformers","Dual Attention Mechanism","Vision Transformer","RoBERTa","CLIP"],"falsifier":"Re-run the test split for MS-COCO with the published metric implementations and confirm that Tri-FusionNet yields BLEU-1 0.893, BLEU-4 0.725, CIDEr 1.880, METEOR 0.780, and ROUGE-L 0.689; if the reproduced numbers fall well below the best published models (e.g., CIDEr below 1.4), the state-of-the-art claim fails.","tokens_in":16712,"feed_emoji":"🖼️","tokens_out":6492,"duration_ms":51848,"temperature":0.7,"pith_summary":"The paper argues that image captioning can be improved by fusing three transformer-based modules: a Vision Transformer encoder that uses dual attention over spatial windows and channel groups, a RoBERTa-based decoder, and a CLIP integration module that aligns visual and textual embeddings with contrastive learning. On the MS-COCO, Flickr30k, and Flickr8k benchmarks, it reports scores that would surpass every previous model, including a BLEU-1 of 0.893 and CIDEr of 1.880 on MS-COCO. If those numbers are correct, the architecture is a new high-water mark for accessible, contextually rich image description generation.","feed_headline":"Tri-FusionNet claims record caption scores on three benchmarks","feed_subtitle":"A ViT-RoBERTa-CLIP fusion with dual attention reports top BLEU, CIDEr, METEOR, and ROUGE-L scores on MS-COCO, Flickr30k, and Flickr8k.","key_machinery":"The central object is the dual-attention mechanism inside the ViT encoder. It runs spatial window self-attention over local, non-overlapping image regions in parallel with channel-group self-attention across the channel dimension, and concatenates the two outputs; this gives linear complexity in both spatial and channel dimensions while capturing local and global context. The RoBERTa decoder generates the caption text from word embeddings and the ViT output, and the CLIP integration module concatenates the two embeddings and aligns them with a contrastive loss so that corresponding image and text pairs sit close in a shared space. These three modules are the machinery that carries the argument from raw images to the reported metric improvements.","core_discovery":"Tri-FusionNet integrates three pretrained transformer families: a Vision Transformer encoder with a dual attention mechanism that combines spatial window attention and channel-group attention, a RoBERTa decoder that contextualizes text embeddings, and a CLIP integration module that maps visual and textual embeddings into a shared space via contrastive loss. The paper reports that this fusion, optimized with Adam and cross-entropy loss, achieves state-of-the-art results across all tested metrics on MS-COCO (BLEU-1 0.893, BLEU-4 0.725, CIDEr 1.880, METEOR 0.780, ROUGE-L 0.689), Flickr30k, and Flickr8k. The central discovery claimed is that dual-attention ViT encoding combined with contrastive CLIP alignment and RoBERTa decoding produces descriptions that are both more semantically accurate and more fluent than those of earlier transformer captioning models.","pith_inferences":["Editorial inference: if the dual-attention and contrastive components are responsible for the gains, the same recipe could be transferred to adjacent tasks such as visual question answering or text-to-image generation, where aligning visual and textual spaces is also central.","The paper's own tables contain inconsistencies (the abstract's Flickr8k values do not match Table VIII), so an independent re-evaluation on a fixed protocol would be needed before treating the exact record numbers as settled.","A testable extension would be ablating CLIP alignment while keeping the dual-attention encoder to isolate how much of the CIDEr gain actually comes from contrastive fusion rather than from the encoder alone.","Because the model concatenates ViT and RoBERTa embeddings and passes them through dense layers, the architecture could likely be trimmed to smaller pretrained backbones for deployment on edge devices, at some cost in accuracy."],"forward_implications":["If the reported scores are reproducible, fusing a dual-attention ViT encoder with a pretrained language decoder and CLIP alignment becomes a viable template for reaching state-of-the-art image captioning.","The dual-attention encoder's linear complexity suggests the architecture can scale to higher-resolution images or longer caption sequences without quadratic cost.","The CLIP contrastive alignment may reduce the semantic gap between visual and textual representations, making generated captions more grounded in image content.","The model's success on three different benchmarks implies the fusion recipe transfers across datasets of different sizes and styles.","The qualitative failures reported (e.g., misidentifying a motorcycle as a car) indicate the remaining bottleneck is fine-grained object recognition, pointing to where future gains could come from."],"supporting_citations":[{"why":"Serves as a primary state-of-the-art baseline on MS-COCO that the proposed model claims to outperform.","marker":"[7]"},{"why":"Provides the contrastive language-image pre-training principle that the CLIP integration module uses.","marker":"[9]"},{"why":"Introduces the double-attention idea that the ViT encoder's dual-attention mechanism builds on.","marker":"[14]"},{"why":"A strong recent baseline whose CIDEr score of 1.414 Tri-FusionNet claims to surpass.","marker":"[15]"},{"why":"A recent baseline on MS-COCO and Flickr30k that Tri-FusionNet claims to beat.","marker":"[16]"},{"why":"The strongest prior Rouge-L baseline on MS-COCO that Tri-FusionNet claims to exceed.","marker":"[28]"},{"why":"Supplies the Vision Transformer architectural background used for the encoder module.","marker":"[3]"}],"fun_headline_variants":["Tri-FusionNet fuses ViT, RoBERTa, CLIP for top caption scores","Dual-attention ViT with CLIP and RoBERTa sets new captioning benchmarks","Tri-FusionNet tops BLEU, CIDEr, METEOR on three benchmarks","ViT-RoBERTa-CLIP fusion with dual attention achieves top captions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported scores were obtained under the standard image-captioning evaluation protocol on the correct test splits and are correctly assigned to the specified model and metric columns.","fun_headline_variants_meta":{"raw":{"variants":["Tri-FusionNet fuses ViT, RoBERTa, CLIP for top caption scores","Dual-attention ViT with CLIP and RoBERTa sets new captioning benchmarks","Tri-FusionNet tops BLEU, CIDEr, METEOR on three benchmarks","ViT-RoBERTa-CLIP fusion with dual attention achieves top captions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001016,"raw_usage":{"total_tokens":4356,"prompt_tokens":1081,"completion_tokens":3275,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":697,"completion_tokens_details":{"reasoning_tokens":3180}},"tokens_in":697,"tokens_out":3275,"duration_ms":20988,"temperature":1.0,"reasoning_tokens":3180,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:56:03.881096+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the test split for MS-COCO with the published metric implementations and confirm that Tri-FusionNet yields BLEU-1 0.893, BLEU-4 0.725, CIDEr 1.880, METEOR 0.780, and ROUGE-L 0.689; if the reproduced numbers fall well below the best published models (e.g., CIDEr below 1.4), the state-of-the-art claim fails.","supporting_citations":[{"cited_title":"Image cap- tioning using transformer-based double attention network,","cited_arxiv_id":null,"evidence_quote":"Introduces the double-attention idea that the ViT encoder's dual-attention mechanism builds on."},{"cited_title":"With a little help from your own past: Prototypical memory networks for image captioning,","cited_arxiv_id":null,"evidence_quote":"A strong recent baseline whose CIDEr score of 1.414 Tri-FusionNet claims to surpass."},{"cited_title":"Haav: Hierarchical aggregation of augmented views for image captioning,","cited_arxiv_id":null,"evidence_quote":"A recent baseline on MS-COCO and Flickr30k that Tri-FusionNet claims to beat."},{"cited_title":"Transformer- based local-global guidance for image captioning,","cited_arxiv_id":null,"evidence_quote":"The strongest prior Rouge-L baseline on MS-COCO that Tri-FusionNet claims to exceed."}],"review_version":1}