REVIEW 1 cited by
Multimodal Pivots for Image Caption Translation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We present an approach to improve statistical machine translation of image descriptions by multimodal pivots defined in visual space. The key idea is to perform image retrieval over a database of images that are captioned in the target language, and use the captions of the most similar images for crosslingual reranking of translation outputs. Our approach does not depend on the availability of large amounts of in-domain parallel data, but only relies on available large datasets of monolingually captioned images, and on state-of-the-art convolutional neural networks to compute image similarities. Our experimental evaluation shows improvements of 1 BLEU point over strong baselines.
Forward citations
Cited by 1 Pith paper
-
TopicVD: A Topic-Based Dataset of Video-Guided Multimodal Machine Translation for Documentaries
This paper builds TopicVD, a topic-based documentary video-subtitle translation dataset, and shows with a cross-modal attention model that visual and contextual information improve BLEU scores.
Discussion (0). Continue with ORCID to comment.