{"id":"682a43cb-cf62-47d4-86ad-7cead32d8d54","arxiv_id":"1908.05407","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A self-supervised rewarding framework using fluency and multi-level visual relevance rewards improves unpaired cross-lingual image captioning.","lead":"This paper proposes a reinforcement learning method that adds self-supervised fluency and visual relevance rewards to improve cross-lingual image captioning without paired training data. It reports stronger caption scores than prior pivoting methods for both English and Chinese.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The relevancy reward may entrench pivot-to-target translation errors because ML-VSE is trained on the same noisy pseudo pairs; Table 4's low target-domain retrieval scores make it unclear that the reward actually fixes visual irrelevancy.","rationale":"The reader's weakest assumption matches the most load-bearing point. The paper's contribution is specifically that self-supervised rewards correct translator errors, and the relevancy reward is the novel mechanism for visual errors. Because the reward model is trained on the same noisy pseudo pairs, the correctness of that mechanism is circular: the reward can only be as reliable as the translations it is supposed to supervise. Table 4 gives direct in-paper evidence that the sentence-level model is much weaker on the target test domain, and the concept-level ablation (Chinese CIDEr 18.5 -> 18.3) shows the fine-grained reward is not consistently beneficial. These observations do not invalidate the paper; the method may still work because the contrastive objective can average over correct translations and the fluency reward may carry most of the gain. But the headline claim of fixing visual irrelevancy is not secure until the reward is shown to agree with true visual semantics in the target domain. The proposed oracle test settles this directly. The reader's conditional verdict is therefore appropriate; I recommend no change.","tokens_in":12630,"tokens_out":7688,"duration_ms":75746,"concrete_test":"Run an oracle diagnostic on the English task: keep the SSR pipeline and hyper-parameters unchanged, but train ML-VSE on the clean human English captions of MSCOCO (and, for the Chinese task, on AIC-ICC human Chinese captions) instead of the machine-translated pseudo pairs, then report CIDEr/BLEU-4/METEOR on the same test sets. If the clean-reward version does not at least match the reported SSR numbers, the pseudo-pair-trained reward is not demonstrably corrupted; if it clearly outperforms, the paper's self-supervised relevancy reward is masking a deficiency and the claim that it fixes visual irrelevancy is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SSR significantly improves unpaired cross-lingual captioning depends on the multi-level visual relevancy rewards providing a trustworthy signal. But ML-VSE is trained with the same machine-translated pseudo pairs DT that the captioning model is trained on, using contrastive loss that treats each translated caption as a positive for its image. Fluent-but-visually-wrong translations are therefore learned as correct visual-semantic associations; the reward can then reinforce, rather than correct, the translator's errors. The paper itself supplies evidence that this is a live risk: Table 4 reports sentence-level retrieval on the MSCOCO test set (the target evaluation domain for English) with image-to-text R@1=22.7 and text-to-image R@1=12.8, far below the AIC-ICC val numbers (52.8 and 37.7), and the authors attribute the gap to 'noises in pseudo pairs and image domain mismatch.' This is precisely the regime in which generated captions are judged. The concept-level model is not a safeguard: nouns and verbs are extracted from the same pseudo captions, so they inherit semantic (not just disfluency) errors, and the ablation in Table 3 shows adding r_crlv actually lowers Chinese CIDEr from 18.5 to 18.3. Thus, unless the noise in the reward model is shown to be benign, the claimed visual-relevancy improvement is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses unpaired cross-lingual image captioning, where no image–caption pairs exist in the target language. It uses a pivot language: an image-to-pivot captioning model and a pivot-to-target machine translation model produce pseudo image–target caption pairs, and a captioning model is trained on these noisy pairs. The authors propose a self-supervised rewarding (SSR) framework in reinforcement learning: a fluency reward from a target-language LSTM language model trained on mono-lingual text, and sentence-level and concept-level visual relevancy rewards from a multi-level visual-semantic embedding model (ML-VSE) trained on the same pseudo pairs. Experiments for unpaired English (using Chinese as pivot) and unpaired Chinese (using English as pivot) on MSCOCO and AIC-ICC report large gains over the implemented baselines, with ablations and human evaluation supporting the contribution.","tokens_in":12945,"tokens_out":3622,"duration_ms":37337,"significance":"If the reported gains are robust, the paper offers a practical method for extending image captioning to low-resource languages without paired data, using only mono-lingual corpora and an existing MT system. The reinforcement-learning formulation with self-supervised rewards is a reasonable extension of prior fluency-guided work, and the inclusion of both sentence-level and concept-level relevancy signals is a useful idea. The paper also provides ablations, an out-of-domain corpus study, a comparison to supervised captioning with limited data, and human evaluation, which are strengths. However, the central mechanism—the visual relevancy reward—is trained on the same noisy pseudo pairs that generate the training signal, and the paper's own retrieval evaluation shows weak performance on the target evaluation domain; without direct evidence that the reward is trustworthy, the claimed correction of visual irrelevancy remains uncertain. The absence of error bars/significance tests and the omission of pivot baselines for Chinese further limit confidence in the quantitative claims.","major_comments":[{"comment":"The relevancy reward is produced by ML-VSE trained on the same pseudo image–caption pairs DT that supervise the captioning model, which creates a circularity risk: fluent-but-visually-wrong translations may be learned as correct visual-semantic associations and then be reinforced by the reward. The paper's own Table 4 shows a large drop in retrieval accuracy when moving from the training domain (AIC-ICC val: image-to-text R@1=52.8, text-to-image R@1=37.7) to the target evaluation domain (MSCOCO test: R@1=22.7 and 12.8, respectively), which is precisely the regime in which generated captions are judged. Please provide direct evidence that the reward is reliable on the target domain—for example, ranking accuracy on held-out target-domain captions, or an analysis comparing reward scores for correct versus hallucinated captions.","section":"§3.3, Eq. (6)–(10) and Table 4"},{"comment":"The ablation results are not monotonic for Chinese: adding the concept-level reward r_crlv to r_flc + r_srlv decreases CIDEr from 18.5 to 18.3 (BLEU-4 improves from 12.1 to 12.3). The text states that combining fluency with both sentence- and concept-level rewards achieves additional gains on both languages, but this is not supported for Chinese CIDEr. Please report the variance across runs or significance tests and discuss this inconsistency, especially given the argument that concept-level rewards are needed.","section":"Table 3"},{"comment":"The Chinese unpaired captioning experiments omit the two-stage pivot baselines (2-Stage pivot Google and 2-Stage pivot joint) that are reported for the English task. As a result, the claim that SSR achieves the best performance among all compared methods is not fully demonstrated for Chinese, and the cross-language comparison is asymmetric. Please add these baselines for Chinese or explicitly justify their omission (e.g., incompatibility of the Chinese tokenizer or translation API).","section":"Table 2"},{"comment":"No error bars, standard deviations, or significance tests are reported for any table, yet the abstract claims \"significant performance improvement over state-of-the-art methods.\" The hyperparameters α, β, γ, λ are tuned on the validation set, which can inflate reported test performance. Please report results over multiple random seeds with standard deviations and, where feasible, paired significance tests (e.g., bootstrap or t-test) for the main comparisons.","section":"§4.2–§4.3"}],"minor_comments":[{"comment":"\"descent image captioning models\" should be \"decent image captioning models.\"","section":"Abstract"},{"comment":"\"Lanet al.\" is missing a space and should read \"Lan et al.\"","section":"§2.2"},{"comment":"The sentence \"we call this relevancy reward computed by the visual semantic matching model as 'self-supervised' reward\" begins with a lowercase letter; please capitalize \"We\" and check the phrasing for grammatical completeness.","section":"§3.3"},{"comment":"The concept-level reward r_crlv(w_j) is applied per token without a self-critical baseline, unlike the sentence-level rewards. Please clarify whether this is intended as a token-level reward and how its scale is calibrated relative to the sentence-level rewards in the joint loss of Eq. (2).","section":"Eq. (10)"},{"comment":"The comparison with supervised captioning models would be clearer if it stated whether the supervised models use the same architecture (ResNet-101 encoder, single-layer LSTM decoder) and the same training schedule; currently only the number of pairs is varied.","section":"Table 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and the proposed method is plausible, but the main technical risk is the circularity of the relevancy reward. I would ask the authors to add target-domain reward reliability analysis and statistical significance reporting before publication. The omission of the two-stage pivot baselines in the Chinese experiments is also a substantive gap. These issues are addressable within the scope of the paper, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look for anyone working on cross-lingual captioning or on self-supervised rewards in RL for generation. The core idea is simple and sensible: when you generate target-language captions by pivoting through a language with paired data, the pivot-to-target translation introduces disfluency and visual irrelevancy errors; SSR uses a fluency reward from a language model trained on monolingual target text and a two-level visual relevance reward from a visual-semantic embedding trained on the same pseudo pairs. The integration in a self-critical RL loop is new, and the ablation structure is clear.\n\nWhat the paper does well: the experiments cover two directions (English via Chinese pivot and Chinese via English pivot) and include human evaluation. The ablations show that the fluency reward alone helps, and adding the sentence-level relevance reward gives a large jump on English (CIDEr from 16.1 to 26.7). That pattern supports the story that visual relevance is the key missing signal. The gains over the two-stage pivot baselines on the English task are large.\n\nThe soft spots are real but not fatal. The biggest is that the ML-VSE model is trained on exactly the noisy pseudo pairs that may contain the errors the rewards are meant to correct. The paper itself provides a red flag: Table 4 shows image-to-text R@1 of 22.7 and text-to-image R@1 of 12.8 on the target MSCOCO test set, far below the AIC-ICC validation numbers. The authors attribute the gap to noise and domain shift, which is plausible, but this is the domain where the captioning model is judged. If the relevance reward is unreliable there, the claimed fix for visual irrelevancy is not fully established. The concept-level reward, which should help, actually slightly hurts Chinese CIDEr (18.5 to 18.3) in the ablation. That is a small effect, but it undercuts the multi-level selling point.\n\nAlso, the Chinese task omits the two-stage pivot baselines that are reported for English, and there are no error bars or significance tests anywhere. In this subfield that is common, but the abstract's 'significant performance improvement' should be read with that caveat.\n\nOverall, the central idea is sound and the method is a genuine advance over re-weighting pseudo pairs by fluency alone. The paper deserves a serious referee. I would send it to review, not desk reject. I would not personally cite it in my own work, but I'd bring it to a reading group if the group cares about RL for generation.","headline":"A well-executed integration of fluency and visual-relevance rewards into RL for unpaired cross-lingual captioning, with real gains over baselines but a soft spot in the relevance reward's reliability on the target domain.","tokens_in":13439,"tokens_out":2541,"would_cite":false,"duration_ms":23909,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Unpaired cross-lingual captioning is improved by self-supervised fluency and visual-relevancy rewards, with no paired target-language captions.","keywords":["cross-lingual image captioning","unpaired image captioning","self-supervised reward","reinforcement learning","visual-semantic matching","pivot language","fluency reward"],"falsifier":"Collect a small set of manually verified image–caption pairs in the target language, run the trained ML-VSE model on them, and measure retrieval recall: if true captions rank near chance while fluent translated captions score highly, the relevancy reward is not correcting visual errors but is inheriting the translator’s bias.","tokens_in":12435,"feed_emoji":"🖼️","tokens_out":9650,"duration_ms":82692,"temperature":0.7,"pith_summary":"This paper tackles unpaired cross-lingual image captioning, where captions must be written in a target language without any human-annotated image–caption pairs in that language. It starts from the standard pivot approach—generate a caption in a pivot language, then machine-translate it into the target language—and argues that the remaining bottleneck is translation error, specifically disfluency and visual irrelevancy. The proposed self-supervised rewarding (SSR) model keeps the no-paired-data setup but trains the captioning model with reinforcement learning, using a fluency reward from target-language text and sentence- plus concept-level visual-relevancy rewards. If the paper is right, a language-pivoted pipeline can reach caption quality close to a supervised model trained on about four thousand paired captions, on both English and Chinese tasks, without collecting any target-language image–caption pairs.","feed_headline":"Self-supervised rewards fix translated caption errors","feed_subtitle":"Self-supervised fluency and relevancy rewards beat pivot baselines without paired target captions.","key_machinery":"The load-bearing mechanism is the self-critical reinforcement learning loop, with three self-supervised reward functions replacing external caption-quality metrics. A fixed LSTM language model, pre-trained on mono-lingual target-language sentences, provides the fluency reward. A multi-level visual-semantic matching model (ML-VSE) provides the relevancy rewards: a sentence-level image–sentence scorer trained with a contrastive ranking loss and hard negative mining, and a concept-level image–concept scorer whose scores are normalized by concept prior probability to prevent frequent concepts from dominating. Both relevance scorers are trained on the same pseudo-translated image–caption pairs used to train the caption generator, which is why the rewards are called self-supervised.","core_discovery":"The central claim is that translation errors, not model architecture, are the main weakness of pivot-based unpaired cross-lingual captioning, and that self-supervised rewards can repair them. Concretely, the paper shows that a captioning model trained on machine-translated pseudo pairs, then fine-tuned with reinforcement learning guided by three reward signals—an LSTM fluency score, an image–sentence relevance score, and an image–concept relevance score—outperforms the pseudo-pair baseline, the same baseline trained with CIDEr reward, and two two-stage pivot baselines. The experiments cover unpaired English captioning (Chinese as pivot) on MSCOCO and unpaired Chinese captioning (English as pivot) on AIC-ICC, and the paper reports the best BLEU, METEOR, and CIDEr scores among the compared methods, with human ratings on an English test set also favoring the SSR captions on fluency and visual relevancy.","pith_inferences":["The same recipe could be tested on low-resource languages beyond English and Chinese, or on video captioning, whenever a pivot language and mono-lingual target text exist.","Because the relevance reward is learned from the same noisy pseudo pairs that train the captioner, a translator with a systematic visual bias (for example, always dropping people) could teach the reward model to penalize correct captions; filtering pseudo pairs or adding robust training before reinforcement learning would be a natural safeguard.","Swapping the LSTM language model for a larger pretrained language model is an obvious extension; whether fluency rewards then capture longer-range coherence, and whether that closes more of the gap to fully supervised captioning, remains an open empirical question."],"forward_implications":["A useful captioning model for a new language can be built from mono-lingual text plus an existing pivot-language captioning system, without any target-language image–caption pairs.","Unpaired training with the proposed rewards reaches CIDEr comparable to a supervised model trained on roughly four thousand paired captions, indicating the gap to supervised captioning is not prohibitive.","The same reward scheme can be applied on top of any pivot-to-target translation service, because the rewards correct the translator’s errors instead of requiring the translator to be retrained or coupled to the captioning model.","Out-of-domain mono-lingual text still produces gains, so the fluency reward does not depend on having caption-style text in the target language."],"supporting_citations":[{"why":"defines the language-pivoting unpaired captioning task and provides the two two-stage pivot baselines that this paper compares against","marker":"[11]"},{"why":"prior fluency-guided approach that only handles disfluency, setting up the visual-relevancy gap this paper addresses","marker":"[20]"},{"why":"supplies the self-critical sequence training algorithm used to optimize the caption generator with rewards","marker":"[25]"},{"why":"provides the contrastive ranking loss with hard negative mining used to train the sentence-level and concept-level visual-semantic matching models","marker":"[8]"},{"why":"provides the base encoder-decoder image captioning architecture that serves as the caption generator","marker":"[28]"},{"why":"supplies the English image-caption data used for the unpaired English task and for training the English fluency language model","marker":"[21]"},{"why":"supplies the Chinese image-caption data used for the unpaired Chinese task and for training the Chinese fluency language model","marker":"[29]"}],"fun_headline_variants":["Self-taught rewards beat pivot translation errors in captions","RL with self-supervised rewards cleans up translated captions","Fixing cross-lingual captions without paired data via rewards","Reward learning compensates for poor machine translation in captions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The relevancy reward model is trained on the same machine-translated captions that the captioning model learns from, so the whole improvement rests on the assumption that those pseudo pairs are accurate enough about which words belong with which image for the reward signal to correct errors rather than reinforce the translator’s mistakes.","fun_headline_variants_meta":{"raw":{"variants":["Self-taught rewards beat pivot translation errors in captions","RL with self-supervised rewards cleans up translated captions","Fixing cross-lingual captions without paired data via rewards","Reward learning compensates for poor machine translation in captions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1343,"prompt_tokens":941,"completion_tokens":402,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":332}},"tokens_in":557,"tokens_out":402,"duration_ms":4903,"temperature":1.0,"reasoning_tokens":332,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:13:58.411951+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a small set of manually verified image–caption pairs in the target language, run the trained ML-VSE model on them, and measure retrieval recall: if true captions rank near chance while fluent translated captions score highly, the relevancy reward is not correcting visual errors but is inheriting the translator’s bias.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the language-pivoting unpaired captioning task and provides the two two-stage pivot baselines that this paper compares against"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"prior fluency-guided approach that only handles disfluency, setting up the visual-relevancy gap this paper addresses"},{"cited_title":"Rennie, Etienne Marcheret, and Youssef Mroueh et al","cited_arxiv_id":null,"evidence_quote":"supplies the self-critical sequence training algorithm used to optimize the caption generator with rewards"},{"cited_title":"Lawrence Zitnick, and Piotr Dol- lÃąr","cited_arxiv_id":null,"evidence_quote":"supplies the English image-caption data used for the unpaired English task and for training the English fluency language model"}],"review_version":1}