Ensuring Consistency for In-Image Translation

Baohang Li; Bing Qin; Chengpeng Fu; Dandan Tu; Duyu Tang; Hui Wang; Ting Liu; Wenshuai Huo; Xiaocheng Feng; Yichong Huang

REVIEW 4 major objections 6 minor 21 references

Ensuring Consistency for In-Image Translation

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A two-stage cascade translates in-image text with scene context and matching text style.

desk verdict Sensible two-stage design for in-image translation, but the consistency claims rest on in-distribution synthetic tests and a small, potentially biased real-image evaluation. read the letter →

arxiv 2412.18139 v1 pith:DKHK3AXB submitted 2024-12-24 cs.CL

Chengpeng Fu , Xiaocheng Feng , Yichong Huang , Wenshuai Huo , Baohang Li , Zhirui Zhang , Yunfei Lu , Dandan Tu

show 4 more authors

Duyu Tang Hui Wang Bing Qin Ting Liu

This is my paper · ORCID

classification cs.CL

keywords in-imagetranslationmultimodalmachinechain-of-thoughtlearningdiffusionmodelstyleconsistencyimagebackfillingmultilinguallargelanguage

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper addresses in-image translation, the task of translating text that appears inside an image and then rendering the translated text back into that image. It argues that this task demands two kinds of consistency that current systems miss: translation consistency, meaning the translation uses what the image shows to resolve ambiguous words, and image generation consistency, meaning the rendered text matches the original font, color, and placement while the background stays clean. The proposed solution, HCIIT, is a two-stage cascade: a multimodal multilingual large language model translates the text using a four-step chain-of-thought prompt that forces the model to look at the image, and a diffusion model backfills the image with a style latent module that conditions text generation on the source text style and the erased background. The framework is trained on more than 400,000 synthetic style-consistent pseudo pairs and evaluated on an ambiguity-focused translation benchmark, synthetic style tests, and 40 real and generated images. If the framework delivers what it claims, in-image translation can move from literal text replacement to translations that read naturally in the scene and look as though the original designer rendered them.

What carries the argument

The mechanism that carries the argument is the style latent module in the diffusion backfilling stage. It takes the source style image $S$ and the text-erased background $B$, encodes both with a VAE decoder $D$, sums their latent maps, and passes them through a convolutional fusion layer $g$ to produce $Z_s = g(D(S)+D(B))$. This style-background latent is combined with the glyph latent $Z_a = f(G(l_g)+P(l_p)+D(l_m))$ and fed into the text-control diffusion model, which predicts the noise under $\mathcal{L}_d = \mathbb{E}_{T_l,Z_s,Z_a,\mathcal{P},t\sim\mathcal{N}(0,1)}[\lVert\epsilon-\epsilon_\theta(T_t,Z_s,Z_a,\mathcal{P},t)\rVert_2^2]$ plus a text perceptual loss $\mathcal{L}_p$. The module is what allows the model to copy the font, color, and thickness of the source text while keeping the background, and the 400,000 synthetic style-consistent pairs are what train it.

What would settle it

Take a held-out set of 100 authentic film posters and book covers in the paper's language pairs, have a designer produce reference images with the translated text placed in the original style, and compare HCIIT with the previous cascade and commercial systems on SSIM/L1 and human ratings of font match and background coherence. If the style-latent model does not beat the baselines on this authentic set, the claim that the synthetic training generalizes to real in-image translation fails.

Watch

Extended reading notes

Core claim

The central claim is that the two consistency failures in in-image translation come from each stage ignoring information it already has. The translation stage ignores the pixels around the text, so a word like 'bank' can be rendered as the financial institution instead of the riverbank shown in the picture; the rendering stage ignores the source text's visual style, so the result replaces it with a default font. The first stage of HCIIT fixes the translation failure by prompting a multimodal large language model through four steps — recognize, describe, correct recognition, correct translation — so scene context disambiguates the final answer. The second stage fixes the rendering failure with a text-control diffusion model whose style latent module fuses the encoded style image and the encoded text-erased background into the denoising condition, alongside glyph, position, and masked-image conditions, trained with a diffusion loss and a text perceptual loss. In the paper's evaluations this yields higher BLEU/COMET on the CoMMuTE benchmark, higher SSIM and lower L1 on the synthetic style test, and better human and large-model ratings on translation accuracy, background coherence, and font style consistency than the previous cascade and commercial systems.

Load-bearing premise

The synthetic pseudo-parallel pairs used for training and for the style-consistency test sets are produced by the same text-rendering pipeline, and the real test images receive manual text-erasure cleanup, so the style-consistency results could overstate performance on authentic posters and scene photos.

Editorial extensions

If this is right

A cascade of an off-the-shelf multimodal LLM and a fine-tuned diffusion model suffices for style-consistent in-image translation; the translation model itself needs no retraining.
The four-step chain-of-thought prompt is a reusable, training-free recipe for making multimodal LLMs use image context to disambiguate words during translation.
Synthetic style-consistent pseudo pairs can supply training data at scale where real translated image pairs are scarce.
Consistency in in-image translation becomes a measurable axis, with translation accuracy, background coherence, and font style consistency scored separately via image similarity and human or model ratings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the style module conditions on image appearance rather than on language, the same backfilling stage could apply to sign replacement, book-cover localization, or meme translation, where matching the original design matters more than the source language.
The synthetic-pair training strategy implies a stress test: on a real-image benchmark with diverse posters and scene photos, the style-consistency gains could shrink unless the synthetic renderer covers the styles found there.
The four-step chain-of-thought prompts could be adapted to other multimodal generation tasks that require resolving ambiguous content from scene context, such as visual question answering or captioning.
Because the cascade separates detection, translation, and backfilling, errors in upstream text-detection or text-erasure steps should propagate directly to the final image, so improving those tools would likely improve overall consistency.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know. The paper has a genuinely useful framing—translation consistency and image generation consistency as separate requirements—and the two-stage design (MMLLM with CoT, then style-conditioned diffusion) is a reasonable engineering response. The second thing is that the quantitative evidence for the image-generation claim doesn't establish generalization. The SSIM/L1 numbers in Table II are measured on synthetic test sets generated by the same rendering pipeline as the 400k training pairs, so the large margins are expected in-distribution behavior, not evidence about real images. The paper actually says this: 'our model is trained specifically on this type of data, resulting in superior performance.'

Where it does well: the translation stage is evaluated on CoMMuTE, an external benchmark, and the CoT prompting does give small but consistent gains over the few-shot baseline. The design of the style latent module is clean and well explained. The authors compare against commercial systems and AnyTrans, and the case studies illustrate the differences clearly.

Soft spots, in proportion. The circularity is the load-bearing one. Beyond the synthetic test sets, the real-image evaluation is only 40 images, with scores 1-3 on three axes, no error bars or statistical testing, and Appendix C says that when AnyText's text removal was subpar on real images, the authors manually edited the images with external software—applied to their own pipeline but not to the commercial baselines. That bias is real. The translation gains, while external, are modest (BLEU +1-2, COMET +1-3), and the baseline is Qwen-VL-Chat with a few-shot prompt rather than the full AnyTrans system, so the comparison is representative but not necessarily the strongest available. No code or data is released, which matters for a paper whose main claim is about a trained model.

Who is this for? Anyone working on in-image translation or styled text rendering will find the problem decomposition and the style latent module worth reading. But the central claim of 'ensuring consistency' on authentic images is not supported by the current evaluation. The paper deserves a serious referee, not a desk reject, because the task is practical, the translation evidence is external, and the method is plausible. I would send it to review with the expectation of major revision: independent or at least held-out style test sets, a larger real-image evaluation with baselines run under the same preprocessing, and code/data release.

Referee Report

4 major / 6 minor

Summary. The paper defines two consistency requirements for in-image translation (IIT): translation consistency (using image context during translation) and image generation consistency (preserving text style and background integrity when rendering translated text back onto the image). It proposes HCIIT, a two-stage framework: stage 1 uses a multimodal multilingual LLM (Qwen-VL-Chat) with chain-of-thought prompting for text recognition and translation; stage 2 fine-tunes a text-controlled diffusion model, based on AnyText, with added style-latent and glyph-latent modules, trained on 400,000 synthetic pseudo-parallel image pairs. Experiments report improved BLEU/COMET on CoMMuTE over a few-shot baseline, higher SSIM/lower L1 on synthetic test sets, and higher human/GPT-4o ratings on 40 real and fabricated images compared with AnyTrans and commercial systems.

Significance. The task is practically relevant, and the two-consistency framing is a useful organizing principle for in-image translation. The stage-1 CoT approach is a simple and sensible way to inject image context, and the CoMMuTE results, though modest, are on an external benchmark. The stage-2 design is a reasonable extension of AnyText with a style-latent module; if that module is shown to generalize, it would be a genuine contribution. However, the principal evidence for image-generation consistency is in-distribution: Table II uses test sets rendered by the same pipeline as the 400k training pairs, and Section IV.C.b concedes this. The 40-image real evaluation is small, lacks significance testing, and Appendix C describes manual cleanup applied to the authors' pipeline only. These issues leave the central claim of 'ensuring consistency' on authentic images only partially supported.

major comments (4)

[Section IV.A / Section IV.C.b / Table II] The SSIM/L1 results in Table II are obtained on synthetic test sets constructed with the same fonts, colors, sizes, deformations, random background crops, and parallel corpora as the training data described in Section IV.A. Section IV.C.b explicitly states that 'our model is trained specifically on this type of data, resulting in superior performance.' Under these in-distribution conditions, the large margins over AnyTrans are expected to reflect reconstruction of the training distribution rather than generalization to authentic text-image styles. The claim that HCIIT 'ensures consistency' on real images is therefore not supported by Table II. Please add evaluation on held-out style conditions (e.g., fonts or renderers not seen in training) and, if possible, on real images with ground-truth style references, reporting these separately from the in-distribution numbers.
[Appendix C / Figure 5] The real-image evaluation uses only 40 images and no statistical analysis. Appendix C states that when AnyText's text removal is 'subpar' on real images, the authors used image editing software to refine the input; this manual cleanup is described for the authors' pipeline but not for the commercial baselines. If this preprocessing is not applied uniformly, the comparison in Figure 5 is biased. Please clarify whether all competing methods received identical preprocessing, and report confidence intervals or significance tests (e.g., bootstrap or paired test) for the human and GPT-4o ratings, along with inter-annotator agreement.
[Section III.C / Section IV] The central novel component of stage 2 is the style latent module, but no ablation isolates its contribution. The paper does not report results without Zs or without the perceptual loss Lp, so it is unclear whether the SSIM/L1 improvements in Table II come from the style module or from simply training on the synthetic distribution. An ablation on the synthetic test set (and ideally on a held-out style condition) is needed to support the claim that the style-latent mechanism is what ensures style consistency.
[Table I / Section IV.A] The translation-stage comparison in Table I is against AnyTrans's few-shot 'Original' prompting; the gains are small (BLEU +0.8–1.3, COMET +1.0–2.4) and no significance tests are reported. Also, Table I includes En-De although Section IV.A states experiments are on En-Fr and En-Zh, and the text refers to 'four directions' while listing only two language pairs. Please reconcile the dataset description with Table I, and include significance tests or error bars for the CoMMuTE results.

minor comments (6)

[Abstract / Section IV.A] The abstract states 400,000 pairs, and Section IV.A says 100,000 pairs in each of four directions; the text should explicitly list the four directions (En-Fr, Fr-En, En-Zh, Zh-En) and reconcile with the En-De row in Table I.
[Figure 4 caption] The caption contains a grammatical error: 'can be find' should be 'can be found'; also the paper uses 'Anytrans' and 'AnyTrans' inconsistently throughout.
[Section III.C.a] The equation for the style latent Zs = g(D(S) + D(B)) is ambiguous: please specify whether the sum is in latent space before fusion, define the dimensions, and clarify the roles of D and g.
[Footnote 1] The explanation for excluding GoogleTrans is unclear: the footnote says GoogleTrans has no official image translation API, yet it is shown in Figure 1 and discussed in the case study; please clarify the status of GoogleTrans in the evaluation.
[Figure 5] The scores in Figure 5 are presented as bare numbers without error bars or variance; adding them would make the differences across systems easier to assess.
[Appendix D] The evaluation rubric uses 'Point 1/2/3' while the text refers to 'levels'; unify the terminology for clarity.

Circularity Check

1 steps flagged · score 5.0 of 10

Synthetic test sets reuse the training rendering pipeline, so Table II's SSIM/L1 margins are in-distribution reconstruction; the image-generation-consistency claim is only partially supported, while translation results rest on external CoMMuTE.

fitted input called prediction [Section IV.A (Datasets) and Section IV.C.b (Results on Synthesis Test Sets), Table II]
"We generate 100,000 pairs of parallel images for training and 200 pairs for testing in each of the four directions. ... Our model is trained specifically on this type of data, resulting in superior performance."

The Stage-2 style-consistency diffusion model is trained on pseudo-parallel pairs from the text-synthesis pipeline described in IV.A (20+ fonts, random colors/sizes/deformations, random background crops, TIT parallel corpora). The 'curated test set' used for Table II is generated by the same pipeline (200 pairs per direction). Consequently, the SSIM/L1 scores measure the model's ability to reconstruct the exact rendering distribution from which its training targets were drawn, not generalization to authentic image styles.

full rationale

The paper's translation-consistency claim is grounded in CoMMuTE, an external benchmark, and the CoT prompting is evaluated there; that part is not circular. The main circularity concern is the image-generation-consistency evaluation: the same text-synthesis pipeline produces both the 400k training pairs and the 200-pair-per-direction synthetic test sets, so Table II's SSIM/L1 figures largely reflect in-distribution reconstruction. The paper explicitly admits this in IV.C.b ('Our model is trained specifically on this type of data'). The real-image evaluation is only 40 images, mixes real and fabricated images, and Appendix C notes manual image-editing cleanup when AnyText's text removal is subpar, which further weakens the authentic-image evidence. There is no load-bearing self-citation chain or imported uniqueness theorem; reference [17] is a minor related-work citation by overlapping authors and does not carry the argument. Overall, the central claim is partially circular for one of its two components, but not fully reduced to its inputs, so a moderate score of 5 is appropriate.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central claims rest on a set of domain assumptions about the reliability of off-the-shelf components and the transferability of synthetic data. No new physical or mathematical entities are introduced, and the only hand-chosen scalar in the objective is the perceptual loss weight lambda.

free parameters (1)

lambda (perceptual loss weight) = 0.01
Set to 0.01, stated to be consistent with Anytext; weights the text perceptual loss Lp against the diffusion loss Ld in L = Ld + lambda * Lp (Section IV.B).

assumptions (5)

domain assumption Qwen-VL-Chat can accurately recognize and translate text in images when given the described CoT prompts.
Stage 1 relies on an off-the-shelf multimodal LLM; the paper does not fine-tune it and assumes its image understanding is reliable enough to correct recognition and translation (Section III.B).
domain assumption Synthetic text-image pairs generated by the described rendering pipeline are representative of real in-image translation images.
Training and synthetic evaluation both use this pipeline, so transfer to authentic images is assumed, not demonstrated (Section IV.A, Section IV.C.b).
domain assumption Text erasure via AnyText, with manual image-editing corrections, removes the original text without meaningfully altering the background or biasing evaluation.
Stage 2 and the real-image evaluation depend on this preprocessing step (Appendix C).
domain assumption SSIM and L1 distance between generated images and reference images measure text style consistency sufficiently.
Table II uses these full-image metrics to conclude style consistency, but they also capture background similarity (Section IV.C.b).
domain assumption The text position detection model provides accurate bounding boxes for all test images.
The pipeline in Section III.A uses these boxes, but the model is not named or evaluated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Ensuring Consistency for In-Image Translation." pith.science (2026). https://pith.science/paper/DKHK3AXB

@misc{pith2026241218139,
  author       = {Pith},
  title        = {Pith review of: Ensuring Consistency for In-Image Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DKHK3AXB}},
  note         = {Machine review of arXiv:2412.18139}
}

read the original abstract

The in-image machine translation task involves translating text embedded within images, with the translated results presented in image format. While this task has numerous applications in various scenarios such as film poster translation and everyday scene image translation, existing methods frequently neglect the aspect of consistency throughout this process. We propose the need to uphold two types of consistency in this task: translation consistency and image generation consistency. The former entails incorporating image information during translation, while the latter involves maintaining consistency between the style of the text-image and the original image, ensuring background integrity. To address these consistency requirements, we introduce a novel two-stage framework named HCIIT (High-Consistency In-Image Translation) which involves text-image translation using a multimodal multilingual large language model in the first stage and image backfilling with a diffusion model in the second stage. Chain of thought learning is utilized in the first stage to enhance the model's ability to leverage image information during translation. Subsequently, a diffusion model trained for style-consistent text-image generation ensures uniformity in text style within images and preserves background details. A dataset comprising 400,000 style-consistent pseudo text-image pairs is curated for model training. Results obtained on both curated test sets and authentic image test sets validate the effectiveness of our framework in ensuring consistency and producing high-quality translated images.

Figures

Figures reproduced from arXiv: 2412.18139 by the authors.

**Figure 2.** The process of two-stage in-image translation with consistent style. Our framework consists of two stages, comprising an MMLLM-based TIT and a [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** An overview of stage 2. We incorporated a style latent module as a constraint for the Text ControlNet. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Case study on En-Zh in-image translation on (a) constructed [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: Human evaluation and large language model evaluation. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Examples of the constructed pseudo parallel En-Zh image pairs. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Case Study on Stage 1. Therefore, our upcoming research will investigate endto-end methods for IIMT to overcome these challenges and improve efficiency [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 16 canonical work pages

[1]

Improving end-to-end text image translation from the auxiliary text translation task,

Cong Ma, Yaping Zhang, Mei Tu, Xu Han, Linghui Wu, Yang Zhao, and Yu Zhou, “Improving end-to-end text image translation from the auxiliary text translation task,” in 2022 26th International Conference on Pattern Recognition (ICPR) . IEEE, 2022, pp. 1664–1670

work page 2022
[2]

Exploring better text image translation with multimodal codebook,

Zhibin Lan, Jiawei Yu, Xiang Li, Wen Zhang, Jian Luan, Bin Wang, Degen Huang, and Jinsong Su, “Exploring better text image translation with multimodal codebook,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 3479–3491

work page 2023
[3]

AnyTrans: Translate AnyText in the image with large scale models,

Zhipeng Qian, Pei Zhang, Baosong Yang, Kai Fan, Yiwei Ma, Derek F. Wong, Xiaoshuai Sun, and Rongrong Ji, “AnyTrans: Translate AnyText in the image with large scale models,” in Findings of the Association for Computational Linguistics: EMNLP 2024 , Nov. 2024, pp. 2432–2444

work page 2024
[4]

Multimodal atten- tion for neural machine translation,

Ozan Caglayan, Lo ¨ıc Barrault, and Fethi Bougares, “Multimodal atten- tion for neural machine translation,” arXiv preprint arXiv:1609.03976 , 2016

arXiv 2016
[5]

Multimodal transformer for multimodal machine translation,

Shaowei Yao and Xiaojun Wan, “Multimodal transformer for multimodal machine translation,” in Proceedings of the 58th annual meeting of the association for computational linguistics , 2020, pp. 4346–4350

work page 2020
[6]

A novel graph-based multi-modal fusion encoder for neural machine translation,

Yongjing Yin, Fandong Meng, Jinsong Su, Chulun Zhou, Zhengyuan Yang, Jie Zhou, and Jiebo Luo, “A novel graph-based multi-modal fusion encoder for neural machine translation,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , 2020, pp. 3025–3035

work page 2020
[7]

Cross-lingual visual pre-training for multimodal machine translation,

Ozan Caglayan, Menekse Kuyu, Mustafa Sercan Amac, Pranava Swa- roop Madhyastha, Erkut Erdem, Aykut Erdem, and Lucia Specia, “Cross-lingual visual pre-training for multimodal machine translation,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , 2021, pp. 1317–1324

work page 2021
[8]

On vision features in multimodal machine translation,

Bei Li, Chuanhao Lv, Zefan Zhou, Tao Zhou, Tong Xiao, Anxiang Ma, and Jingbo Zhu, “On vision features in multimodal machine translation,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2022, pp. 6327– 6337

work page 2022

Show all 21 references

[9]

Translation camera,

Yasuhiko Watanabe, Yoshihiro Okada, Yeun-Bae Kim, and Tetsuya Takeda, “Translation camera,” in Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No. 98EX170) . IEEE, 1998, vol. 1, pp. 613–617

1998
[10]

Integrating natural language processing with image document analysis: what we learned from two real-world applications,

Jinying Chen, Huaigu Cao, and Premkumar Natarajan, “Integrating natural language processing with image document analysis: what we learned from two real-world applications,” International Journal on Document Analysis and Recognition (IJDAR) , vol. 18, pp. 235–247, 2015

2015
[11]

Translatotron-v (ison): An end-to-end model for in-image machine translation,

Zhibin Lan, Liqiang Niu, Fandong Meng, Jie Zhou, Min Zhang, and Jinsong Su, “Translatotron-v (ison): An end-to-end model for in-image machine translation,” arXiv preprint arXiv:2407.02894 , 2024

2024 arXiv
[12]

Glyphdraw: Learning to draw chinese characters in image synthesis models coherently,

Jian Ma, Mingjun Zhao, Chen Chen, Ruichen Wang, Di Niu, Haonan Lu, and Xiaodong Lin, “Glyphdraw: Learning to draw chinese characters in image synthesis models coherently,” arXiv preprint arXiv:2303.17870 , vol. 2, 2023

2023 arXiv
[13]

Glyphcontrol: Glyph conditional control for visual text generation,

Yukang Yang, Dongnan Gui, Yuhui Yuan, Weicong Liang, Haisong Ding, Han Hu, and Kai Chen, “Glyphcontrol: Glyph conditional control for visual text generation,” Advances in Neural Information Processing Systems, vol. 36, 2024

2024
[14]

Textdiffuser: Diffusion models as text painters,

Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei, “Textdiffuser: Diffusion models as text painters,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024
[15]

Textdiffuser-2: Unleashing the power of language models for text rendering,

Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei, “Textdiffuser-2: Unleashing the power of language models for text rendering,” arXiv preprint arXiv:2311.16465 , 2023

2023 arXiv
[16]

Anytext: Multilingual visual text generation and editing,

Yuxiang Tuo, Wangmeng Xiang, Jun-Yan He, Yifeng Geng, and Xuan- song Xie, “Anytext: Multilingual visual text generation and editing,” in The Twelfth International Conference on Learning Representations , 2023

2023
[17]

Aligning translation-specific understand- ing to general understanding in large language models,

Yichong Huang, Xiaocheng Feng, Baohang Li, Chengpeng Fu, Wenshuai Huo, Ting Liu, and Bing Qin, “Aligning translation-specific understand- ing to general understanding in large language models,” arXiv preprint arXiv:2401.05072, 2024

2024 arXiv
[18]

Tackling ambiguity with images: Improved multi- modal machine translation and contrastive evaluation,

Matthieu Futeral, Cordelia Schmid, Ivan Laptev, Beno ˆıt Sagot, and Rachel Bawden, “Tackling ambiguity with images: Improved multi- modal machine translation and contrastive evaluation,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics...

2023
[19]

Bleu: a method for automatic evaluation of machine translation,

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceed- ings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318

2002
[20]

Comet: A neural framework for mt evaluation,

Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie, “Comet: A neural framework for mt evaluation,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 2685–2702

2020
[21]

Spatial attentive single-image deraining with a high quality real rain dataset,

Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson WH Lau, “Spatial attentive single-image deraining with a high quality real rain dataset,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 12270–12279. VI. A PPE...

2019

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Improving end-to-end text image translation from the auxiliary text translation task,

Cong Ma, Yaping Zhang, Mei Tu, Xu Han, Linghui Wu, Yang Zhao, and Yu Zhou, “Improving end-to-end text image translation from the auxiliary text translation task,” in 2022 26th International Conference on Pattern Recognition (ICPR) . IEEE, 2022, pp. 1664–1670

work page 2022

[2] [2]

Exploring better text image translation with multimodal codebook,

Zhibin Lan, Jiawei Yu, Xiang Li, Wen Zhang, Jian Luan, Bin Wang, Degen Huang, and Jinsong Su, “Exploring better text image translation with multimodal codebook,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 3479–3491

work page 2023

[3] [3]

AnyTrans: Translate AnyText in the image with large scale models,

Zhipeng Qian, Pei Zhang, Baosong Yang, Kai Fan, Yiwei Ma, Derek F. Wong, Xiaoshuai Sun, and Rongrong Ji, “AnyTrans: Translate AnyText in the image with large scale models,” in Findings of the Association for Computational Linguistics: EMNLP 2024 , Nov. 2024, pp. 2432–2444

work page 2024

[4] [4]

Multimodal atten- tion for neural machine translation,

Ozan Caglayan, Lo ¨ıc Barrault, and Fethi Bougares, “Multimodal atten- tion for neural machine translation,” arXiv preprint arXiv:1609.03976 , 2016

arXiv 2016

[5] [5]

Multimodal transformer for multimodal machine translation,

Shaowei Yao and Xiaojun Wan, “Multimodal transformer for multimodal machine translation,” in Proceedings of the 58th annual meeting of the association for computational linguistics , 2020, pp. 4346–4350

work page 2020

[6] [6]

A novel graph-based multi-modal fusion encoder for neural machine translation,

Yongjing Yin, Fandong Meng, Jinsong Su, Chulun Zhou, Zhengyuan Yang, Jie Zhou, and Jiebo Luo, “A novel graph-based multi-modal fusion encoder for neural machine translation,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , 2020, pp. 3025–3035

work page 2020

[7] [7]

Cross-lingual visual pre-training for multimodal machine translation,

Ozan Caglayan, Menekse Kuyu, Mustafa Sercan Amac, Pranava Swa- roop Madhyastha, Erkut Erdem, Aykut Erdem, and Lucia Specia, “Cross-lingual visual pre-training for multimodal machine translation,” in Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume , 2021, pp. 1317–1324

work page 2021

[8] [8]

On vision features in multimodal machine translation,

Bei Li, Chuanhao Lv, Zefan Zhou, Tao Zhou, Tong Xiao, Anxiang Ma, and Jingbo Zhu, “On vision features in multimodal machine translation,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2022, pp. 6327– 6337

work page 2022