REVIEW 2 cited by
CPTR: Full Transformer Network for Image Captioning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
CPTR: Full Transformer Network for Image Captioning
read the original abstract
In this paper, we consider the image captioning task from a new sequence-to-sequence prediction perspective and propose CaPtion TransformeR (CPTR) which takes the sequentialized raw images as the input to Transformer. Compared to the "CNN+Transformer" design paradigm, our model can model global context at every encoder layer from the beginning and is totally convolution-free. Extensive experiments demonstrate the effectiveness of the proposed model and we surpass the conventional "CNN+Transformer" methods on the MSCOCO dataset. Besides, we provide detailed visualizations of the self-attention between patches in the encoder and the "words-to-patches" attention in the decoder thanks to the full Transformer architecture.
Forward citations
Cited by 2 Pith papers
-
MultiMem: Measuring and Mitigating Memorization in Multi-Modal Contrastive Learning
Introduces MultiMem as the first metric for memorization in multi-modal contrastive learning, identifies cross-modal misalignment (text dominant) as key driver, and shows targeted augmentations reduce it while improvi...
-
A Deep Learning Model of Mental Rotation Informed by Interactive VR Experiments
A deep neural model with equivariant, symbolic, and agentic components reproduces human accuracy and action counts on Shepard-Metzler mental rotation, but not full reaction-time behavior.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.