REVIEW 2 cited by
An Attentive Survey of Attention Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Attention Model has now become an important concept in neural networks that has been researched within diverse application domains. This survey provides a structured and comprehensive overview of the developments in modeling attention. In particular, we propose a taxonomy which groups existing techniques into coherent categories. We review salient neural architectures in which attention has been incorporated, and discuss applications in which modeling attention has shown a significant impact. We also describe how attention has been used to improve the interpretability of neural networks. Finally, we discuss some future research directions in attention. We hope this survey will provide a succinct introduction to attention models and guide practitioners while developing approaches for their applications.
Forward citations
Cited by 2 Pith papers
-
A Rose by Any Other Name Would Smell as Sweet: Categorical Homotopy Theory for Large Language Models
The paper argues that LLM next-token distributions form Markov categories whose paraphrase equivalences can be studied by homotopy theory, but its main theorem is unsupported.
-
Topos Theory for Generative AI and LLMs
The paper claims the category of LLM functions is a topos and uses that to propose new compositional architectures like pullbacks, pushouts, and subobject classifiers, but gives no implementation or complete proofs.
Discussion (0). Continue with ORCID to comment.