Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

Colin Raffel; Daniel P. W. Ellis

arxiv: 1512.08756 · v5 · pith:ASI366PQnew · submitted 2015-12-29 · 💻 cs.LG · cs.NE

Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

Colin Raffel , Daniel P. W. Ellis This is my paper

classification 💻 cs.LG cs.NE

keywords attentionfeed-forwardlong-termmemorymodelnetworksproblemssolve

0 comments

read the original abstract

We propose a simplified model of attention which is applicable to feed-forward neural networks and demonstrate that the resulting model can solve the synthetic "addition" and "multiplication" long-term memory problems for sequence lengths which are both longer and more widely varying than the best published results for these tasks.

This paper has not been read by Pith yet.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

An Attention Mechanism for Musical Instrument Recognition
cs.IR 2019-07 unverdicted novelty 5.0

An attention model improves multi-label instrument recognition accuracy on the weakly labeled OpenMIC dataset compared to baseline, RNN, and fully connected networks across 20 instruments.