REVIEW 1 cited by
Attention Mechanism in Neural Networks: Where it Comes and Where it Goes
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
A long time ago in the machine learning literature, the idea of incorporating a mechanism inspired by the human visual system into neural networks was introduced. This idea is named the attention mechanism, and it has gone through a long development period. Today, many works have been devoted to this idea in a variety of tasks. Remarkable performance has recently been demonstrated. The goal of this paper is to provide an overview from the early work on searching for ways to implement attention idea with neural networks until the recent trends. This review emphasizes the important milestones during this progress regarding different tasks. By this way, this study aims to provide a road map for researchers to explore the current development and get inspired for novel approaches beyond the attention.
Forward citations
Cited by 1 Pith paper
-
Ensemble-Based Survival Models with the Self-Attended Beran Estimator Predictions
SurvBESA applies self-attention to predicted survival functions from bagged Beran estimators and reports improved ranking performance on benchmark survival datasets.
Discussion (0). Continue with ORCID to comment.