Modern Hopfield Networks with Continuous-Time Memories

Andr\'e F.T. Martins; Ant\'onio Farinhas; Daniel C. McNamee; Saul Santos

REVIEW 2 major objections 5 minor 14 references

Modern Hopfield Networks with Continuous-Time Memories

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that modern Hopfield networks can retrieve from a compressed continuous-time memory, replacing the discrete softmax over stored patterns with a Gibbs density over time, and that this matches discrete retrieval quality…

desk verdict Clean derivation of a continuous-memory Hopfield update, but the advertised efficiency gain is unverified and may vanish once quadrature cost is counted. read the letter →

arxiv 2502.10122 v4 pith:DFBU47HH submitted 2025-02-14 cs.LG

Saul Santos , António Farinhas , Daniel C. McNamee , André F.T. Martins This is my paper · ORCID

classification cs.LG

keywords modernHopfieldnetworkscontinuousattentionassociativememorycompressionGibbsdensityridgeregressionvideoretrievalworking

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Modern Hopfield networks retrieve an attractor by iterating an update derived from an energy over a discrete set of stored patterns. This paper replaces that discrete set with a continuous function reconstructed from the observations, compressing L samples into N basis-function coefficients with N much smaller than L. It defines a continuous Hopfield energy, shows through the concave-convex procedure that the update becomes the expectation of the reconstructed signal under a Gibbs density, and reports retrieval quality on synthetic patterns and long videos roughly on par with the discrete network. The point of the claim is that associative memories can store long sequences in a compact, function-valued form without sacrificing recall.

What carries the argument

The load-bearing object is the reconstructed continuous memory $\bar{x}(t)=B^\top\psi(t)$: a linear combination of $N$ basis functions (uniform rectangular bases in the experiments) whose coefficients $B$ are fixed by multivariate ridge regression, $B^\top=X^\top F^\top(FF^\top+\lambda I)^{-1}$. On top of it sits the continuous Hopfield energy (6), whose concave term is the log-partition of the Gibbs density $p(t)\propto\exp(\beta s(t))$. Applying the concave-convex procedure linearizes that concave term and produces the identity $q^{(i+1)}=\mathbb{E}_{p(t)}[\bar{x}(t)]$, which is the direct continuous analogue of the discrete softmax update and is what lets the network retrieve from a compressed function rather than from each stored point.

What would settle it

Run the same retrieval experiment on a dataset where the L observations are temporally shuffled or drawn as independent noise, so no smooth $x(t)$ exists; if the continuous network with $N\ll L$ still matches the discrete network's cosine similarity, the smoothness premise is not what drives the result, whereas a sharp drop would confirm it is load-bearing.

Watch

Extended reading notes

Core claim

The paper's central claim is that Hopfield retrieval can be carried out over a continuous reconstructed memory $\bar{x}(t)=B^\top\psi(t)$ instead of a discrete memory matrix $X$, with the coefficient matrix $B$ fitted by ridge regression so that $\bar{x}(t_\ell)\approx x_\ell$. The continuous Hopfield energy $E(q)=-\frac{1}{\beta}\log\int_0^1 \exp(\beta \bar{x}(t)^\top q)\,dt + \frac{1}{2}\|q\|^2$ replaces the discrete sum over patterns with an integral over time, and the concave-convex procedure yields the Gibbs expectation update $q^{(i+1)}=\int p(t)\bar{x}(t)\,dt = B^\top\int p(t)\psi(t)\,dt$, where $p(t)\propto\exp(\beta s(t))$ is a Gibbs density over the memory interval. The paper presents this as a modern Hopfield network whose memories are functions rather than points, and argues it keeps retrieval competitive while using fewer stored coefficients.

Load-bearing premise

The argument rests on the discrete observations being genuine samples from a smooth, ordered function of time, so that a small number of fixed basis functions can reconstruct the memory faithfully; when the data is not smooth or its order is arbitrary, the continuous memory is a lossy stand-in for the original patterns.

Editorial extensions

If this is right

Retrieval cost can scale with the number of basis functions $N$ instead of the number of observations $L$, so long video or time-series memories can be compressed before the associative update runs.
The discrete softmax-over-patterns update is replaced by a Gibbs-density expectation, giving a concrete bridge between Hopfield retrieval and continuous-attention transformers.
On video embeddings, the continuous network matches or exceeds the discrete full-memory network for large $L$ while using $N$ much smaller than $L$, with the gap widening as $L$ grows.
The formulation gives a recurrent-network implementation of continuous resource-allocation theories of working memory, rather than discrete memory slots.
With fixed uniform rectangular bases, performance degrades as $N$ approaches $L$, so the compression benefit is tied to keeping the basis count well below the number of observations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial: Nothing in the proof depends on the basis being rectangular, so learned or adaptive basis functions could plausibly preserve retrieval at even smaller $N$ and remove the degradation the paper observes as $N$ approaches $L$.
Editorial: Because the discrete Hopfield update maps onto transformer attention, the same continuous-memory construction suggests long-context transformers whose key-value store is a compact set of basis coefficients plus a Gibbs density; the paper does not test this.
Editorial: The capacity question is left open: the experiments retrieve one video memory from a query, not many independent patterns stored in one continuous memory, so a direct many-pattern capacity test would be needed to extend the compressed-memory claim.
Editorial: A direct test of the smoothness premise would permute the temporal ordering of the observations or use non-smooth data; if retrieval does not degrade, the premise is not doing the work claimed.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea here is a genuine extension: replace the discrete memory matrix in a modern Hopfield network with a continuous reconstruction x̄(t)=Bᵀψ(t), define the energy in (6), and derive the Gibbs-density update (7) via CCCP. The derivation is straightforward and correct (modulo a stray β in Appendix A that cancels). The link to continuous attention is nicely made, and the paper is honest about the degradation it sees as N approaches L. The code being public is a real plus.

The soft spot is the one the stress-test note flags: the abstract's claim of 'reducing computational costs' is not backed by any complexity analysis or wall-clock measurement. With trapezoidal quadrature, each iteration costs O(ND+TN), not O(ND). The ablation in Appendix B uses about 500 quadrature points for L=512 and L=2048, which means T≈L for the smaller case and T/D can be non-negligible when D is a few hundred. So the N≪L compression can be fully eaten by the integration cost. This is a load-bearing claim, and it is currently unsupported. The comparison against discrete HN with Lsub=N also muddies the picture; the efficiency comparison should be against the full L discrete model.

The smoothness assumption is a second, lesser concern. The method presupposes that the L observations are ordered samples from a smooth function, and that a fixed uniform rectangular basis plus ridge regression captures that function. On video frame embeddings this is plausible, but no experiment tests what happens when the ordering is arbitrary or the signal is non-smooth. This limits the claimed generality but does not break the method's internal logic.

Who is this for? People working on Hopfield networks, continuous attention, and memory-efficient transformers will find the construction useful and worth discussing. It is not a breakthrough—it is an incremental but principled step, with a correct proof and preliminary evidence. It deserves a serious referee: the derivation is sound, the idea is clearly presented, and the weaknesses are addressable in revision. I would send it to review, but the authors must supply a complexity analysis and runtime benchmarks before the efficiency claim can stand.

For my own work, I would not cite it in the next year unless I were actively working in this subfield, but I'd bring it to a reading group for a discussion of what 'compression' really costs in continuous attention.

Referee Report

2 major / 5 minor

Summary. This paper proposes an extension of modern Hopfield networks in which a discrete memory set X = [x_1, ..., x_L] is replaced by a continuous reconstruction x̄(t) = Bᵀψ(t) built from N basis functions via ridge regression (Eq. 5). The energy of Ramsauer et al. is modified by replacing the sum over stored patterns with an integral over t ∈ [0,1] (Eq. 6), and Proposition 1 derives a CCCP update q^{i+1} = Bᵀ∫p(t)ψ(t)dt, i.e., the expectation under a Gibbs density. Experiments on synthetic energy contours and on MovieChat-1K video frames and embeddings compare cosine-similarity retrieval against discrete Hopfield networks with L or N stored patterns. The paper claims that the continuous memory maintains competitive retrieval performance while reducing computational costs.

Significance. If the efficiency claim were established, this would be a clean and useful bridge between modern Hopfield networks and continuous-attention architectures, with potential value for long-context or streaming-memory settings. The strengths are that Proposition 1 is a correct and simple application of CCCP to the proposed energy, the code is publicly released, and the experimental protocol includes a memory-matched baseline (Lsub = N) rather than comparing only against the full discrete memory. The principal weakness is that the advertised computational advantage is not substantiated by any complexity analysis, wall-clock benchmark, or FLOP measurement, and the quadrature cost may erase the benefit in the low-dimensional embedding regime.

major comments (2)

[§4–§5, Eq. (7), Appendix B] The paper's headline claim of 'reducing computational costs' (Abstract) is not supported by any complexity analysis or runtime measurement. With T-point trapezoidal quadrature for the integral in Eq. (7), each update costs O(ND + TN), versus O(LD) for the discrete update (2); the saving requires N/L + T/D ≪ 1. Appendix B reports that about 500 quadrature points are 'sufficient' for L = 512 and L = 2048; with embedding dimensions D of a few hundred, T/D is not negligible, so the N ≪ L advantage can be erased. The main text never reports T or β for Figures 2–3, and Figure 4 does not vary T jointly with β, even though the Gibbs density sharpens with β and may require more quadrature points. Please add a per-iteration complexity comparison, wall-clock/FLOP measurements, and an ablation of T across β and L, or substantially temper the efficiency claim.
[§4, Eq. (5), §5.2] The reconstruction assumption is load-bearing but unquantified. The method replaces the true memories with x̄(t) = Bᵀψ(t), and the continuous dynamics are faithful to the original discrete memory only insofar as x̄(t) ≈ x(t). The paper assumes smoothness and a meaningful ordering t_1 ≤ ... ≤ t_L, but provides no reconstruction-error measurements on the video or embedding data, no sensitivity analysis with respect to the ordering, and no principled guidance for choosing N. The observed degradation as N approaches L (Section 5.2) shows that the fixed uniform-basis approximation is not always favorable; this deserves quantitative analysis rather than a hypothesis about discrete queries. I request a reconstruction-error analysis and a discussion of the conditions under which the compressed memory is faithful.

minor comments (5)

[Appendix A, proof of Proposition 1] In the displayed equation after the chain-rule step, the paper writes ∇E_concave(q) = -E_{p(t)}[βx̄(t)]; the preceding derivation and the final update use -E_p[x̄(t)]. This stray β should be removed.
[§5.3] The sentence 'even surpassing the discrete HN for the full memory Lsub = L but using N ≪ L, except for L = 512' is ambiguous: clarify whether the exception applies to all N or only some N, and add markers or a table showing the crossover points.
[§5] The paper does not report the embedding dimension D or the β values used in the retrieval experiments; without these numbers, the computational and retrieval comparisons are not reproducible.
[Appendix B, Figure 4] Define 'Number of Points' explicitly as the quadrature size T, and state the values of L, N, D, and β used for each panel.
[§4] The sentence 'ψ(t) form a basis of “discrete memories”' is unclear: ψ(t) are basis functions evaluated at continuous t, not discrete memories. Please rephrase.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the Gibbs-expectation update is a direct CCCP consequence of the proposed energy; self-citations are motivational/implementation choices, not load-bearing.

full rationale

The derivation chain is self-contained. Proposition 1 follows from applying CCCP to the continuous energy (6); Appendix A supplies the differentiation-under-the-integral algebra that yields ∇E_concave(q) = -E_p(t)[x̄(t)] and hence q^{i+1} = B^T∫p(t)ψ(t)dt. This is a mathematical consequence, not an assumption equivalent to the conclusion. B is fit by ridge regression (5), but the evaluations use masked/noisy queries and measure cosine similarity to stored memories, which is the standard autoassociative retrieval task; the reported scores are not fitted predictions. The self-citations (Martins et al. 2020/2022 for continuous attention; Santos et al. 2025 for rectangular basis functions) are motivational or implementation choices and do not carry the proof load; no uniqueness theorem is imported. The N-versus-L comparison in Figures 2-3 is arguably unfair because the continuous model's B is derived from all L frames while the discrete Lsub=N baseline sees only N frames, but this is an experimental-design weakness rather than a circular reduction. The computational-efficiency claim is under-supported because trapezoidal quadrature costs O(ND+TN) and T is not reported in the main figures, but that is a correctness/evidence gap, not circularity. The paper honestly flags degradation as N approaches L. Verdict: no significant circularity; score reflects only minor framing self-citations.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or latent entity; the continuous memory function is a construction from the data. The main free inputs are the ridge penalty, the temperature, and the quadrature resolution.

free parameters (3)

λ (ridge regularization) = not reported
Controls the reconstruction of B in Eq. 5; chosen by hand and not swept or reported in experiments.
β (inverse temperature) = 1 in synthetic experiment; unspecified for video
Controls the sharpness of the Gibbs density; a hyperparameter rather than a fitted constant.
quadrature sample count for trapezoidal rule = 500 per ablation
The integrals in Proposition 1 are approximated numerically; the number of points affects accuracy and cost, and is chosen in the ablation (Appendix B).

assumptions (3)

domain assumption The L observations are samples from a smooth function x(t) on [0,1] with t_1 ≤ ... ≤ t_L.
Section 4 opening paragraph; this is the load-bearing premise that makes low-dimensional basis reconstruction valid.
ad hoc to paper Uniform rectangular basis functions ψ(t) combined with ridge regression (Eq. 5) yield a faithful reconstruction x̄(t) ≈ x(t).
Modeling choice specific to this paper, not externally validated; the paper acknowledges degradation as N approaches L.
standard math CCCP converges for the energy in Eq. 6 as a concave-convex decomposition.
Standard application of Yuille & Rangarajan; the paper invokes it for Proposition 1 without proving convergence conditions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Modern Hopfield Networks with Continuous-Time Memories." pith.science (2026). https://pith.science/paper/DFBU47HH

@misc{pith2026250210122,
  author       = {Pith},
  title        = {Pith review of: Modern Hopfield Networks with Continuous-Time Memories},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DFBU47HH}},
  note         = {Machine review of arXiv:2502.10122}
}

read the original abstract

Recent research has established a connection between modern Hopfield networks (HNs) and transformer attention heads, with guarantees of exponential storage capacity. However, these models still face challenges scaling storage efficiently. Inspired by psychological theories of continuous neural resource allocation in working memory, we propose an approach that compresses large discrete Hopfield memories into smaller, continuous-time memories. Leveraging continuous attention, our new energy function modifies the update rule of HNs, replacing the traditional softmax-based probability mass function with a probability density, over the continuous memory. This formulation aligns with modern perspectives on human executive function, offering a principled link between attractor dynamics in working memory and resource-efficient memory allocation. Our framework maintains competitive performance with HNs while leveraging a compressed memory, reducing computational costs across synthetic and video datasets.

Figures

Figures reproduced from arXiv: 2502.10122 by the authors.

**Figure 2.** Video retrieval performance across different numbers of basis functions. Plotted are the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Video embedding retrieval performance across different numbers of basis functions. Plot [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Performance on video and embedding data across different numbers of sampling points [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 9 canonical work pages

[5]

The results demonstrate that 500 sampling points are sufficient for the approximation, where the Hopfield network with continuous memories performs comparably to the modern Hopfield network. 8 New Frontiers in Associative Memoryworkshop at ICLR 2025 0 200 400 600 800 1000 1200 Number of Points 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Video Continuous HN Discrete HN 0 ...

work page 2025
[9]

Saul Santos, Vlad Niculae, Daniel McNamee, and Andr´e F. T. Martins. Hopfield-fenchel-young net- works: A unified framework for associative memory retrieval. arXiv preprint arXiv:2411.08590, 2024a. URL https://arxiv.org/abs/2411.08590. Saul Santos, Vlad Niculae, Daniel C McNamee, and Andre F.T. Martins. Sparse and structured hopfield networks. In Internat...

work page arXiv
[11]

doi: 10.7554/eLife.91034

ISSN 2050-084X. doi: 10.7554/eLife.91034. URL https://doi.org/10.7554/eLife.91034. Danil Tyulmankov, Ching Fang, Annapurna Vadaparty, and Guangyu Robert Yang. Biological learn- ing in key-value memory networks. InAdvances in Neural Information Processing Systems,

work page doi:10.7554/elife.91034
[512]

For embedding reconstruction, we set L = 2048 and N = 1024 with σ =

work page
[1956]

Hopfield networks is all you need

Hubert Ramsauer, Bernhard Sch ¨afl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlovi ´c, Geir Kjetil Sandve, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008.02217,

arXiv 2008
[1971]

Benjamin Hoover, Duen Horng Chau, Hendrik Strobelt, Parikshit Ram, and Dmitry Krotov

doi: 10.1126/science.173.3997.652. Benjamin Hoover, Duen Horng Chau, Hendrik Strobelt, Parikshit Ram, and Dmitry Krotov. Dense associative memory through the lens of random features. In The Thirty-eighth Annual Confer- ence on Neural Information Processing Systems ,

work page doi:10.1126/science.173.3997.652
[2003]

A P ROOF OF PROPOSITION 1 The CCCP algorithm works as follows: at thetth iteration, it linearizes the concave functionEconcave by using a first-order Taylor approximation around q(t), Econcave(q) ≈ ˜Econcave(q) := Econcave(q(t)) + ∂Econcave(q(t)) ∂q ⊤ (q − q(t)). (9) Then, it computes a new iterate by solving the convex optimization problem q(i+1) := arg ...

work page 2025
[2014]

URL https://doi.org/10

doi: 10.1038/nn.3655. URL https://doi.org/10. 1038/nn.3655. Andr´e Martins, Ant ´onio Farinhas, Marcos Treviso, Vlad Niculae, Pedro Aguiar, and Mario Figueiredo. Sparse and continuous attention mechanisms. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural In- formation Processing Systems , volume 33, pp. 20989–21...

doi:10.1038/nn.3655

Show all 14 references

[2016]

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi

URL https://proceedings.neurips.cc/paper/2016/file/ eaae339c4d89fc102edd9dbdb6a28915-Paper.pdf. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In Proceedings of the ...

2016
[2017]

doi: 10.1007/s10955-017-1806-y

ISSN 1572-9613. doi: 10.1007/s10955-017-1806-y. URL http://dx.doi.org/ 10.1007/s10955-017-1806-y . Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual representation learning at scale

doi
[2020]

6 New Frontiers in Associative Memoryworkshop at ICLR 2025 Andr´e F

URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/f0b76267fbe12b936bd65e203dc675c1-Paper.pdf. 6 New Frontiers in Associative Memoryworkshop at ICLR 2025 Andr´e F. T. Martins, Marcos Treviso, Ant´onio Farinhas, Pedro M. Q. Aguiar, M´ario A. T. Figueiredo, Mathieu ...

2020
[2023]

Jerry Yao-Chieh Hu, Bo-Yu Chen, Dennis Wu, Feng Ruan, and Han Liu

URL https://arxiv.org/abs/2309.12673. Jerry Yao-Chieh Hu, Bo-Yu Chen, Dennis Wu, Feng Ruan, and Han Liu. Nonparametric modern hopfield models,

arXiv
[2024]

Dmitry Krotov and John J

URL https://arxiv.org/abs/2404.03900. Dmitry Krotov and John J. Hopfield. Dense associative memory for pattern recogni- tion. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems , volume

arXiv
[2025]

Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Xun Guo, Tian Ye, Yan Lu, Jenq-Neng Hwang, et al

URL https://arxiv.org/abs/2501.19098. Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Xun Guo, Tian Ye, Yan Lu, Jenq-Neng Hwang, et al. Moviechat: From dense token to sparse memory for long video understanding. arXiv preprint arXiv:2307.16449,

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [5]

The results demonstrate that 500 sampling points are sufficient for the approximation, where the Hopfield network with continuous memories performs comparably to the modern Hopfield network. 8 New Frontiers in Associative Memoryworkshop at ICLR 2025 0 200 400 600 800 1000 1200 Number of Points 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Video Continuous HN Discrete HN 0 ...

work page 2025

[2] [9]

Saul Santos, Vlad Niculae, Daniel McNamee, and Andr´e F. T. Martins. Hopfield-fenchel-young net- works: A unified framework for associative memory retrieval. arXiv preprint arXiv:2411.08590, 2024a. URL https://arxiv.org/abs/2411.08590. Saul Santos, Vlad Niculae, Daniel C McNamee, and Andre F.T. Martins. Sparse and structured hopfield networks. In Internat...

work page arXiv

[3] [11]

doi: 10.7554/eLife.91034

ISSN 2050-084X. doi: 10.7554/eLife.91034. URL https://doi.org/10.7554/eLife.91034. Danil Tyulmankov, Ching Fang, Annapurna Vadaparty, and Guangyu Robert Yang. Biological learn- ing in key-value memory networks. InAdvances in Neural Information Processing Systems,

work page doi:10.7554/elife.91034

[4] [512]

For embedding reconstruction, we set L = 2048 and N = 1024 with σ =

work page

[5] [1956]

Hopfield networks is all you need

Hubert Ramsauer, Bernhard Sch ¨afl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlovi ´c, Geir Kjetil Sandve, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008.02217,

arXiv 2008

[6] [1971]

Benjamin Hoover, Duen Horng Chau, Hendrik Strobelt, Parikshit Ram, and Dmitry Krotov

doi: 10.1126/science.173.3997.652. Benjamin Hoover, Duen Horng Chau, Hendrik Strobelt, Parikshit Ram, and Dmitry Krotov. Dense associative memory through the lens of random features. In The Thirty-eighth Annual Confer- ence on Neural Information Processing Systems ,

work page doi:10.1126/science.173.3997.652

[7] [2003]

A P ROOF OF PROPOSITION 1 The CCCP algorithm works as follows: at thetth iteration, it linearizes the concave functionEconcave by using a first-order Taylor approximation around q(t), Econcave(q) ≈ ˜Econcave(q) := Econcave(q(t)) + ∂Econcave(q(t)) ∂q ⊤ (q − q(t)). (9) Then, it computes a new iterate by solving the convex optimization problem q(i+1) := arg ...

work page 2025

[8] [2014]

URL https://doi.org/10

doi: 10.1038/nn.3655. URL https://doi.org/10. 1038/nn.3655. Andr´e Martins, Ant ´onio Farinhas, Marcos Treviso, Vlad Niculae, Pedro Aguiar, and Mario Figueiredo. Sparse and continuous attention mechanisms. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural In- formation Processing Systems , volume 33, pp. 20989–21...

doi:10.1038/nn.3655