Pith. sign in

REVIEW 4 major objections 4 minor 60 references

A single temporal-convolutional embedder, frozen after pre-training, can support four distinct on-device learning scenarios — few-shot, continual, zero-shot, and in-context — on one chip, with the first silicon demonstrations of the latter

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

ECL unifies few-shot, continual, zero-shot, and in-context learning on a single TCN-based edge chip, with first hardware baselines on several tasks.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection A credible systems unification of FSL/CL/ZSL/ICL on one chip, honestly reported; the ZSL/ICL results are feasibility demos under aggressive quantization, and the paper itself flags the quantizer mismatch. the 4 major comments →

arxiv 2607.29353 v1 pith:2UONNCID submitted 2026-07-31 cs.LG cs.AIeess.AS

Versatile On-device Adaptation at the Edge by Unifying Few-shot, Zero-shot, Continual, and In-context Learning

Douwe den Blanken , Martin Lefebvre , Charlotte Frenkel This is my paper
classification cs.LG cs.AIeess.AS
keywords embedder-centric learningfew-shot learningcontinual learningzero-shot learningin-context learningtemporal convolutional networkson-device learningedge AI accelerator
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that four seemingly distinct ways of adapting a network at the edge — learning from a few examples, accumulating classes over time, learning from semantic descriptions alone, and learning from labeled context within a sequence — are all instances of one pattern: a shared embedder that produces low-dimensional embeddings, plus a small scenario-specific head. The authors argue that this embedder-centric learning pattern lets a single TCN-based accelerator handle images, audio, and token sequences, and they demonstrate it in silicon with a chip running all four scenarios at 9.5 µW to 7.83 mW. Specifically, the paper reports new state-of-the-art few-shot accuracy on Omniglot, the first fully on-chip continual-learning baseline on a 200-class keyword benchmark, and the first hardware demonstrations of zero-shot and in-context learning. If the framework holds, edge devices could personalize without cloud retraining and without backpropagation.

Core claim

The paper's central claim is that every learning scenario can be re-cast in an embedder-centric form, so that the same frozen temporal-convolutional embedder, pre-trained off-chip once per sensory modality, serves all scenarios: few-shot learning averages support embeddings into class prototypes (equivalently, a single FC layer); continual learning appends new prototype rows rather than overwriting old ones; zero-shot learning replaces labeled supports with semantic embeddings fed through a relation network; and in-context learning concatenates query and support tokens before embedding and passes the joint embedding through an MLP. On the authors' chip, this yields 96.8% 5-way 1-shot and 83.

What carries the argument

A temporal convolutional network (TCN) serves as a universal embedder, with the prototype-to-FC-weight equivalence as the load-bearing trick. The TCN's causal dilated convolutions give it logarithmic memory scaling with sequence length, letting one embedder handle flattened images, MFCC audio frames, and tokens; the paper rewrites the L2-nearest-prototype rule as a matrix-vector multiplication by expressing prototypes as weights and biases of a single FC layer, so all scenario-specific heads run on the same inference datapath. A small learning controller switches between averaging support embeddings, appending FC parameters, and passing joint embeddings through an MLP.

Load-bearing premise

A single temporal-convolutional embedder, pre-trained off-chip and then frozen and quantized to 4-bit log2 weights, yields embeddings that separate entirely new classes and modalities, so that simple heads (prototype averaging, a relation network, or an MLP) can classify with no update to the embedder.

What would settle it

Take a class of audio or image data structurally different from the pre-training domain (e.g., new keyword phrases from an unseen language or Omniglot alphabets excluded from training), quantize the embedder to 4-bit log2 weights, and measure nearest-prototype accuracy against chance; if the frozen quantized embeddings collapse for classes outside the training distribution, the framework's central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A device that only needs to run inference can also learn: ECL requires no gradient computation, no activation storage for backpropagation, and adds only 0.5% area overhead for the learning logic.
  • Embedding memory stays tiny (6.4 kB holds prototypes for 200 classes), so class accumulation over time does not require off-chip memory.
  • The same hardware supports multiple modalities: flattened images, audio MFCCs, and token sequences all flow through the same TCN embedder, so one chip can cover use cases previously needing separate specialized systems.
  • The reported numbers give the first in-silico baselines for semantic-data zero-shot learning and in-context learning, so future edge-learning chips can be compared directly against them.
  • A TCN can approach transformer-level in-context accuracy (within one point in software) at 31× less activation memory, suggesting that in-context learning at this scale does not hinge on the attention mechanism.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same frozen-embedder recipe could be tested on larger and non-temporal tasks; the paper openly notes its results are on small datasets. A natural next check is whether a pre-trained TCN embedder stays separable for classes far outside its training distribution — if not, the framework's versatility is bounded by embedder generality.
  • Combining ECL with head-only gradient descent or forward-learning updates could recover part of the quantization gap (for instance, ZSL's drop from 80.8% to 60.6%) without full backpropagation, and would be a cheap experiment on the existing codebase.
  • If TCN-based in-context learning transfers to harder formal languages or small language-modeling tasks, it would suggest that attention is not necessary for ICL at scale, opening a low-power alternative to transformer-based edge language models.
  • The 4-bit log2 quantization and shared embedding-quantizer limitations of the current chip explain most of the ZSL accuracy loss; a chip with per-channel scaling or 6-bit weights would likely narrow the FP32 gap, making ZSL the least hardware-ready scenario in this framework.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes embedder-centric learning (ECL), a framework that unifies few-shot learning (FSL), continual learning (CL), zero-shot learning (ZSL), and in-context learning (ICL) by decomposing each scenario into a shared temporal-convolutional-network (TCN) embedder and a scenario-specific fully-connected head. The authors deploy ECL on the Chameleon SoC and report: Omniglot FSL accuracies of 96.8% (5-way 1-shot) and 83.3% (32-way 1-shot); a NeuroBench keyword FSCIL accuracy of 71.8% (200-way 5-shot); a 5-way FSC ZSL accuracy of 60.6%; and a RegBench ICL accuracy of 46.2% at token 500. They claim new state-of-the-art FSL among silicon accelerators, the first fully on-chip CL baseline on NeuroBench keyword FSCIL, and the first hardware demonstrations of semantic ZSL and ICL at micro-to-milliwatt power. The paper also contributes open-source code and a detailed mapping of the four scenarios onto the Chameleon learning controller.

Significance. If the results hold, ECL is a useful systems contribution: it shows that one embedder-centric architecture can support four adaptation paradigms across image, audio, and token modalities with only 0.5% area overhead for learning logic and with power figures in the microwatt-to-milliwatt range. The direct silicon measurements, the open-source release, and the comparisons against existing silicon FSL and software CL baselines are concrete strengths. The main significance is therefore engineering versatility and hardware efficiency rather than algorithmic novelty. However, the ZSL and ICL demonstrations are less complete: they lack external baselines, use a newly introduced split for ZSL, and are strongly affected by the 4-bit log2 quantization on Chameleon. The paper's broader claim of versatile on-device adaptation is defensible but is currently conditional on the robustness of the quantized frozen embedder, which is tested on only one small dataset per scenario.

major comments (4)
  1. [Abstract and Section IV-B, Fig. 5] The abstract's 'new state-of-the-art performance for FSL character recognition' is not qualified. Fig. 5 compares only against silicon FSL accelerators (Kim et al., SAPIENS, FSL-HDnn), and Table I shows that the TCN in software (97.9% 5-way 1-shot) is below the commonly used CNN baseline (98.8%). The claim is thus 'state of the art among silicon FSL accelerators', not 'state of the art for FSL'. Please qualify this throughout the abstract and conclusions.
  2. [Section IV-D, Table IV and Fig. 7] The hardware ZSL result is quantizer-dominated. Table IV reports 60.6% on-chip accuracy versus 80.8% in FP32, a 20.2-point drop. Fig. 7 attributes 19.6 points to 4-bit log2 weights and 4.7 points to the shared embedding quantizer, but the Fig. 7 caption states that the 4-bit log2 bar uses a larger bias bit width than Chameleon supports. This means the deployed accuracy is lower than the ablation bar shown, so the on-chip result is not a clean test of the frozen-embedder generalisation claim. Please report the actual bias bit width used on-chip and the corresponding accuracy, and state explicitly that the hardware ZSL result is a feasibility demonstration under aggressive quantization rather than robust validation of ECL's ZSL capability.
  3. [Sections IV-D and IV-E] The ZSL and ICL evaluations lack external baselines. For ZSL, Section IV-D.2 states that no direct comparisons exist because the paper introduces a new 5-way split of FSC; the comparison to Relation Network is only in terms of model size. For ICL, Section IV-E.2 compares to a parameter-matched transformer baseline generated in this work, but there is no comparison to established RegBench results or to prior on-chip learning systems. To support the 'first hardware demonstration' and 'versatile support' claims, either add external baselines on the same splits/tasks or soften the claims to 'proof-of-concept demonstrations'.
  4. [Section III and Section V (Future Work)] The central assumption of ECL is that one frozen, 4-bit-log2-quantized TCN embedder produces separable embeddings for classes and modalities not seen during pre-training. This assumption is tested on only one dataset per scenario, and the paper's own ablation shows a 20.2-point ZSL drop from quantization. The manuscript acknowledges the small datasets as a limitation, but this is load-bearing: if the quantized embeddings are not reliably separable on a second dataset or split, the 'versatile on-device adaptation' claim is overstated. I recommend adding at least one additional ZSL/ICL experiment or an embedding-separability analysis under the deployed quantization, or explicitly conditioning the contribution on this assumption.
minor comments (4)
  1. [Section IV-A] Typo: 'repeats a a number of times' should read 'repeats a number of times'.
  2. [General] The paper uses both 'SotA' and 'SOTA'; please standardize the abbreviation.
  3. [Fig. 7 caption] The caption's note that the 4-bit log2 ablation uses a larger bias bit width than Chameleon supports is important and should also appear in the main text when Table IV is discussed, not only in the figure caption.
  4. [Section IV-D.1] For the new FSC ZSL split, please report the exact number of training/validation/test classes and any effect of class imbalance, since this split is introduced in this paper.

Circularity Check

0 steps flagged

No significant circularity: the paper's claims rest on silicon measurements, external benchmark comparisons, and standard algorithms; minor self-citations are not load-bearing.

full rationale

The derivation chain in ECL is an architectural decomposition, not a predictive derivation whose output is encoded in its inputs. Each learning scenario is implemented with established, externally cited methods (prototypical networks [28], relation networks [29], TCNs [46]), and the four scenario results are independent measurements on Chameleon compared against different external baselines (FSL-HDnn, Clo-HDnn, NeuroBench M5/SNN, parameter-matched transformer). No fitted parameter is renamed as a prediction: the embedder is pre-trained off-chip and heads are computed from support/semantic embeddings, with test accuracies reported directly. The only noteworthy self-citations are (i) the FSL numbers, which the paper explicitly says "were previously reported in [25] and are included here to show the performance of ECL on a representative use case for FSL" — this imports a silicon measurement from the same group, but it is a measurement, not an unverified theorem; and (ii) the NeuroBench reference [38], co-authored by one of the present authors, which provides the CL benchmark and baselines. Neither forces the paper's conclusions by definition. The paper also discloses the main limitation of the ZSL hardware number: the Fig. 7 caption states "4-bit log2 accuracy here is higher than in Table IV because this ablation uses a larger bias bit width than Chameleon supports," and the authors acknowledge "we demonstrated ECL on relatively small datasets and NNs." These weaken external-validity or quantization-robustness claims, but they are correctness/robustness concerns, not circularity: the accuracy drops are measured rather than assumed. Overall, the central claim is self-contained against external benchmarks and independent in content; the score of 2 reflects only the existence of minor, non-load-bearing self-citations.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

Everything the empirical claims rest on that is not measured in this paper: the frozen-embedder generalization premise, the equivalence of PN to an FC layer, the validity of RegBench/FSC as benchmarks, and the representative nature of Chameleon measurements. There are no invented physical entities.

free parameters (3)
  • Model architecture choices (TCN depth/channels, embedding dim, MLP sizes) = Not fully reported; examples: 64-dim CL embeddings, 120k semantic embedder, 96.7k TCN, 28.6k relation head, 133k ICL mod
    These choices determine whether the central claims hold; the paper does not report a search/selection procedure, only final sizes for some components.
  • 4-bit log2 / 4-bit activation quantization scheme and shared embedding quantizer = 4-bit log2 weights, 4-bit activations; single shared embedding quantizer for ZSL
    Hardware accuracy results are directly tied to these fixed formats; the ZSL ablation shows format choice changes accuracy by ~20 points.
  • Per-scenario clock frequencies/voltages = 14.4 kHz/0.73 V for CL; 5.8 kHz/0.73 V for ZSL; 100 MHz for ICL/FSL
    Power/energy numbers are measured at these operating points; changing them changes headline energy results.
axioms (6)
  • standard math Prototypical-network classification is exactly equivalent to a single FC layer with W = 2 * prototype embedding and b = -||prototype||^2.
    Used for FSL/CL heads on the PE array; taken from [28] and invoked in Section III-1.
  • domain assumption A single frozen TCN embedder (possibly quantized) produces embeddings for images, audio, and token streams that are good enough for all four learning scenarios.
    Section III: 'ECL requires an NN that can embed samples containing long temporal dependencies'; this is the load-bearing premise for all hardware results.
  • domain assumption In-context learning can be performed in a forward pass by concatenating support and query before embedding, without modifying parameters.
    Section III-4 and Section IV-E; the RegBench result depends on this definition of ICL.
  • domain assumption RegBench is a valid measure of in-context learning ability for small models.
    Section IV-E1, citing [51]; the ICL hardware claim is benchmarked against this dataset.
  • domain assumption The FSC split with no class overlap is a valid zero-shot learning benchmark.
    Section IV-D1; the paper introduces this split and has no external baseline.
  • domain assumption Power/latency/area results on Chameleon are representative of real-time edge operation.
    Section IV-A; all hardware efficiency claims are measured on this one SoC.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Versatile On-device Adaptation at the Edge by Unifying Few-shot, Zero-shot, Continual, and In-context Learning." pith.science (2026). https://pith.science/paper/2UONNCID

@misc{pith2026260729353,
  author       = {Pith},
  title        = {Pith review of: Versatile On-device Adaptation at the Edge by Unifying Few-shot, Zero-shot, Continual, and In-context Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2UONNCID}},
  note         = {Machine review of arXiv:2607.29353}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

With the ever-increasing pervasiveness of smart edge devices, the demand is growing for applications that can be tailored to users (e.g., custom keyword spotting) or patients (e.g., adaptive health monitoring). Yet, most edge devices rely on fixed inference algorithms and thus cannot learn on-device to personalize predictions. When they can, devices typically support only a specific learning scenario, such as few-shot learning (FSL): going beyond this requires resorting either to another specialized device or to cloud-based retraining, which implies significant energy and latency overheads, a lack of real-time capabilities, and privacy concerns. In this work, we introduce embedder-centric learning (ECL), a framework that unifies four different online learning scenarios: FSL for on-the-fly customization, continual learning (CL) for knowledge accumulation, zero-shot learning (ZSL) for leveraging semantic data, and in-context learning (ICL) for adapting beyond classification. We demonstrate in silicon that ECL can be deployed on resource-constrained devices across four real-world use cases representative of the aforementioned learning scenarios. Our approach establishes a new state-of-the-art performance for FSL character recognition (Omniglot: 96.8% for 5-way 1-shot, 83.3% for 32-way 1-shot), and the first hardware baseline for CL in keyword spotting (NeuroBench keyword FSCIL: 71.8% for 200-way 5-shot). Moreover, we present the first hardware demonstrations of ZSL with semantic data (60.6% for 5-way spoken sentence classification) and ICL (46.2% at the 500th token of RegBench) operating at micro-to-milliwatt power budgets. Therefore, by unifying multiple learning scenarios, we pave the way for smart and versatile devices that can adapt right at the edge, without reliance on the cloud.

Figures

Figures reproduced from arXiv: 2607.29353 by Charlotte Frenkel, Douwe den Blanken, Martin Lefebvre.

Figure 1
Figure 1. Figure 1: (a) Overview of current approaches to learning at the edge and their limitations. (b) Learning at the edge requires supporting different learning scenarios, such as few-shot learning (FSL), continual learning (CL), zero￾shot learning (ZSL), and in-context learning (ICL), while accommodating for different sensory modalities, forming a fundamental challenge. (c) Outline of the proposed embedder-centric learn… view at source ↗
Figure 2
Figure 2. Figure 2: (a) Comparison of four different learning use cases, from customizing to new tasks on the fly to adapting beyond classification tasks. (b) To enable each of these use cases, a learning scenario can be adopted: FSL, CL, ZSL, or ICL, each corresponding to a use case. Finally, we offer concluding remarks in Section V. To promote reproducibility, reuse, and improvement, all code for this paper is open-source, … view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the ECL framework. (a) ECL unifies four online learning scenarios through an embedder-centric formulation: support and query samples are embedded before a scenario-specific head, consisting of one or more FC layers, produces a prediction on the query sample. (b) FSL in ECL uses prototypical networks (PNs) [28]: support embeddings are averaged class-wise into prototypes, and the query sample is … view at source ↗
Figure 4
Figure 4. Figure 4: (a) Simplified architecture overview of Chameleon. The SoC includes a matrix-vector processing element (PE) array which uses 4-bit log2 weights for multiplier-free PEs and 4-bit activations. Together with the inference control logic, it forms the SoC’s NN inference data path. (b) Comparison of Chameleon’s configurations per learning scenario, which are implemented by the SoC’s learning controller. (c)-(f) … view at source ↗
Figure 5
Figure 5. Figure 5: FSL test accuracy comparison on the Omniglot dataset between FSL [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: (a) CL accuracy comparison between ECL in software (SW) and in hardware (HW) on Chameleon and the NeuroBench baselines on the keyword FSCIL dataset [38]. While ECL in FP outperforms both baselines after learning 20 new classes, our framework incurs a 4.4 accuracy-point drop compared to the ∼ 96× larger M5 baseline after continually learning the full dataset under tight quantization. Shaded areas indicate 9… view at source ↗
Figure 8
Figure 8. Figure 8: Cumulative ICL test accuracy comparison between ECL in software [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

60 extracted references · 8 linked inside Pith

  1. [1]

    A survey on the convergence of edge computing and ai for uavs: Opportunities and challenges,

    P. McEnroe, S. Wang, and M. Liyanage, “A survey on the convergence of edge computing and ai for uavs: Opportunities and challenges,”IEEE Internet Things J., vol. 9, no. 17, pp. 15 435–15 459, 2022

    2022

  2. [2]

    V ocell: A 65-nm speech-triggered wake-up soc for 10- µ w keyword spotting and speaker verification,

    J. S. P. Giraldo, S. Lauwereins, K. Badami, and M. Verhelst, “V ocell: A 65-nm speech-triggered wake-up soc for 10- µ w keyword spotting and speaker verification,”IEEE J. Solid-State Circuits, vol. 55, no. 4, pp. 868–878, 2020

    2020

  3. [3]

    A 23-uw keyword spotting ic with ring-oscillator-based time-domain feature extraction,

    K. Kim, C. Gao, R. Gra c ¸a, I. Kiselev, H. -J. Yoo, T. Delbruck, and S.-C. Liu, “A 23-uw keyword spotting ic with ring-oscillator-based time-domain feature extraction,”IEEE J. Solid-State Circuits, vol. 57, no. 11, pp. 3298–3311, 2022

    2022

  4. [4]

    Tan, W.-H

    F. Tan, W.-H. Yu, J. Lin, K.-F. Un, R. P. Martins, and P.-I. Mak, “A 1.8% far, 2 ms decision latency, 1.73 nj/decision keywords-spotting (kws) chip incorporating transfer-computing speaker verification, hybrid-if-domain computing and scalable 5t-sram,”IEEE J. Solid-State Circuits, vol. 60, no. 3, pp. 1103–1112, 2025

    2025

  5. [5]

    A 5.6µw 10-keyword end-to-end keyword spotting system using passive-averaging sar adc and sign-exponent-only layer fusion with 92.7% accuracy,

    S. Park, K. Shin, D. Lee, M. Kang, S. Lee, Y . Park, M. Seok, and D. Jeon, “A 5.6µw 10-keyword end-to-end keyword spotting system using passive-averaging sar adc and sign-exponent-only layer fusion with 92.7% accuracy,” in2024 IEEE Symp. VLSI Technol. Circuits (VLSI Technol. Circuits), 2024, pp. 1–2

    2024

  6. [6]

    An ultra-low power reconfigurable biomedical ai processor with adaptive learning for versatile wearable intelligent health monitoring,

    J. Liu, J. Fan, Z. Zhong, H. Qiu, J. Xiao, Y . Zhou, Z. Zhu, G. Dai, N. Wang, Q. Liu, et al., “An ultra-low power reconfigurable biomedical ai processor with adaptive learning for versatile wearable intelligent health monitoring,”IEEE Trans. Biomed. circuits Syst., vol. 17, no. 5, pp. 952–967, 2023

    2023

  7. [7]

    A 3.9 mw 25- electrode reconfigured sensor for wearable cardiac monitoring system,

    L. Yan, J. Bae, S. Lee, T. Roh, K. Song, and H. -J. Yoo, “A 3.9 mw 25- electrode reconfigured sensor for wearable cardiac monitoring system,” IEEE J. Solid-State Circuits, vol. 46, no. 1, pp. 353–364, 2011

    2011

  8. [8]

    A 1.06- µ w smart ecg processor in 65-nm cmos for real-time biometric authentication and personal cardiac monitoring,

    S. Yin, M. Kim, D. Kadetotad, Y . Liu, C. Bae, S. J. Kim, Y . Cao, and J.-S. Seo, “A 1.06- µ w smart ecg processor in 65-nm cmos for real-time biometric authentication and personal cardiac monitoring,” IEEE J. Solid-State Circuits, vol. 54, no. 8, pp. 2316–2326, 2019

    2019

  9. [9]

    A 2.89 µ w dry-electrode enabled clockless wireless ecg soc for wearable applications,

    X. Zhang, Z. Zhang, Y . Li, C. Liu, Y . X. Guo, and Y . Lian, “A 2.89 µ w dry-electrode enabled clockless wireless ecg soc for wearable applications,”IEEE J. Solid-State Circuits, vol. 51, no. 10, pp. 2287– 2298, 2016

    2016

  10. [10]

    An Ultra- Low Power Reconfigurable Biomedical AI Processor With Adaptive Learning for Versatile Wearable Intelligent Health Monitoring,

    J. Liu, J. Fan, Z. Zhong, H. Qiu, J. Xiao, Y . Zhou, Z. Zhu, G. Dai, N. Wang, Q. Liu, Y . Xie, H. Liu, L. Chang, and J. Zhou, “An Ultra- Low Power Reconfigurable Biomedical AI Processor With Adaptive Learning for Versatile Wearable Intelligent Health Monitoring,”IEEE Trans. Biomed. Circuits Syst., vol. 17, no. 5, pp. 952–967, Oct. 2023, Conference Name: I...

    2023

  11. [11]

    Reckon: A 28nm sub-mm2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales,

    C. Frenkel and G. Indiveri, “Reckon: A 28nm sub-mm2 task-agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales,” in2022 IEEE Int. Solid-State Circuits Conf. (ISSCC), vol. 65, 2022, pp. 1–3

    2022

  12. [12]

    Tinyvers: A tiny versatile system-on-chip with state-retentive emram for ml inference at the extreme edge,

    V . Jain, S. Giraldo, J. De Roose, L. Mei, B. Boons, and M. Verhelst, “Tinyvers: A tiny versatile system-on-chip with state-retentive emram for ml inference at the extreme edge,”IEEE J. Solid-State Circuits, vol. 58, no. 8, pp. 2360–2371, 2023

    2023

  13. [13]

    Embedded deep neural network processing: Algorithmic and processor techniques bring deep learning to iot and edge devices,

    M. Verhelst and B. Moons, “Embedded deep neural network processing: Algorithmic and processor techniques bring deep learning to iot and edge devices,”IEEE Solid-State Circuits Mag., vol. 9, no. 4, pp. 55–65, 2017

    2017

  14. [14]

    Efficient processing of deep neural networks: A tutorial and survey,

    V . Sze, Y .-H. Chen, T.-J. Yang, and J. S. Emer, “Efficient processing of deep neural networks: A tutorial and survey,”Proc. IEEE, vol. 105, no. 12, pp. 2295–2329, 2017

    2017

  15. [15]

    A 640m pixel/s 3.65 mw sparse event-driven neuromorphic object recognition processor with on-chip learning,

    J. K. Kim, P. Knag, T. Chen, and Z. Zhang, “A 640m pixel/s 3.65 mw sparse event-driven neuromorphic object recognition processor with on-chip learning,” in2015 Symp. VLSI Circuits (VLSI Circuits), IEEE, 2015, pp. C50–C51

    2015

  16. [16]

    A 42pj/decision 3.12 tops/w robust in-memory machine learning classifier with on-chip training,

    S. K. Gonugondla, M. Kang, and N. Shanbhag, “A 42pj/decision 3.12 tops/w robust in-memory machine learning classifier with on-chip training,” in2018 IEEE Int. Solid-State Circuits Conference-(ISSCC), IEEE, 2018, pp. 490–492

    2018

  17. [17]

    A 55nm time-domain mixed-signal neuromorphic accelerator with stochastic synapses and embedded reinforcement learning for autonomous micro-robots,

    A. Amravati, S. B. Nasir, S. Thangadurai, I. Yoon, and A. Raychowd- hury, “A 55nm time-domain mixed-signal neuromorphic accelerator with stochastic synapses and embedded reinforcement learning for autonomous micro-robots,” in2018 IEEE Int. Solid-State Circuits Conference-(ISSCC), IEEE, 2018, pp. 124–126

    2018

  18. [18]

    Fsl-hdnn: A 5.7 tops/w end-to- end few-shot learning classifier accelerator with feature extraction and hyperdimensional computing,

    H. Yang, C. E. Song, W. Xu, B. Khaleghi, U. Mallappa, M. Shah, K. Fan, M. Kang, and T. Rosing, “Fsl-hdnn: A 5.7 tops/w end-to- end few-shot learning classifier accelerator with feature extraction and hyperdimensional computing,” in2024 IEEE Eur. Solid-State Electron. Res. Conf. (ESSERC), 2024, pp. 33–36

    2024

  19. [19]

    Clo-hdnn: A 4.66 tflops/w and 3.78 tops/w continual on-device learning accelerator with energy-efficient hyperdimensional computing via progressive search,

    C. E. Song, W. Xu, K. Fan, S. Jain, G. Hota, H. Yang, L. Liu, K. Akarvardar, M.-F. Chang, C. H. Diaz, G. Cauwenberghs, T. Rosing, and M. Kang, “Clo-hdnn: A 4.66 tflops/w and 3.78 tops/w continual on-device learning accelerator with energy-efficient hyperdimensional computing via progressive search,” in2025 Symp. VLSI Technol. Circuits (VLSI Technol. Circu...

    2025

  20. [20]

    One-shot learning with memory- augmented neural networks using a 64-kbit, 118 gops/w rram-based non-volatile associative memory,

    H. Li, W. -C. Chen, A. Levy, C. -H. Wang, H. Wang, P. -H. Chen, W. Wan, H.-S. P. Wong, and P. Raina, “One-shot learning with memory- augmented neural networks using a 64-kbit, 118 gops/w rram-based non-volatile associative memory,” in2021 Symp. VLSI Technol., 2021, pp. 1–2

    2021

  21. [21]

    An in-memory computing sram macro for memory-augmented neural network,

    S. Kim, W. Lee, S. Kim, S. Park, and D. Jeon, “An in-memory computing sram macro for memory-augmented neural network,”TCAS-II, vol. 69, no. 3, 2022

    2022

  22. [22]

    A 4096-neuron 1m-synapse 3.8-pj/sop spiking neural network with on-chip stdp learning and sparse weights in 10-nm finfet cmos,

    G. K. Chen, R. Kumar, H. E. Sumbul, P. C. Knag, and R. K. Krishnamurthy, “A 4096-neuron 1m-synapse 3.8-pj/sop spiking neural network with on-chip stdp learning and sparse weights in 10-nm finfet cmos,”IEEE J. Solid-State Circuits, vol. 54, no. 4, pp. 992–1002, 2018

    2018

  23. [23]

    Tess: A scalable temporally and spatially local learning rule for spiking neural networks,

    M. P. E. Apolinario, K. Roy, and C. Frenkel, “Tess: A scalable temporally and spatially local learning rule for spiking neural networks,” in2025 Int. Joint Conf. Neural Netw. (IJCNN), 2025, pp. 1–9

    2025

  24. [24]

    Online spatio-temporal learning in deep neural networks,

    T. Bohnstingl, S. Wo ´zniak, A. Pantazi, and E. Eleftheriou, “Online spatio-temporal learning in deep neural networks,”IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 11, pp. 8894–8908, 2023

    2023

  25. [25]

    Chameleon: A multiplier-free temporal convolutional network accelerator for end-to-end few-shot and continual learning from sequential data,

    D. d. Blanken and C. Frenkel, “Chameleon: A multiplier-free temporal convolutional network accelerator for end-to-end few-shot and continual learning from sequential data,”IEEE J. Solid-State Circuits, pp. 1–16, 2026

    2026

  26. [26]

    Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,

    S. Davis and P. Mermelstein, “Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,” IEEE Trans. Acoust., speech, signal Process., vol. 28, no. 4, pp. 357–366, 1980

    1980

  27. [27]

    Human-level concept learning through probabilistic program induction,

    B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, “Human-level concept learning through probabilistic program induction,”Science, vol. 350, no. 6266, 2015

    2015

  28. [28]

    Prototypical networks for few-shot learning,

    J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” inAdvances Neural Inf. Process. Syst., vol. 30, 2017

    2017

  29. [29]

    Learning to compare: Relation network for few-shot learning,

    F. Sung, Y . Yang, L. Zhang, T. Xiang, P. H. Torr, and T. M. Hospedales, “Learning to compare: Relation network for few-shot learning,” inProc. IEEE Conf. Comput. Vis. pattern Recognit., 2018, pp. 1199–1208

    2018

  30. [30]

    Few-shot keyword spotting in any language,

    M. Mazumder, C. Banbury, J. Meyer, P. Warden, and V . J. Reddi, “Few-shot keyword spotting in any language,”arXiv preprint arXiv:2104.01454, 2021

    Pith/arXiv arXiv 2021

  31. [31]

    Matching networks for one shot learning,

    O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al., “Matching networks for one shot learning,”Advances neural Inf. Process. Syst., vol. 29, 2016

    2016

  32. [32]

    Domain-adaptive discriminative one-shot learning of gestures,

    T. Pfister, J. Charles, and A. Zisserman, “Domain-adaptive discriminative one-shot learning of gestures,” inEur. Conf. Comput. Vis., Springer, 2014, pp. 814–829

    2014

  33. [33]

    Model-agnostic meta-learning for fast adaptation of deep networks,

    C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” inInt. Conf. Mach. Learn., PMLR, 2017, pp. 1126–1135. 11

    2017

  34. [34]

    One shot learning of simple visual concepts,

    B. Lake, R. Salakhutdinov, J. Gross, and J. Tenenbaum, “One shot learning of simple visual concepts,” inProc. Annu. meeting Cogn. Sci. Soc., vol. 33, 2011

    2011

  35. [35]

    Siamese neural networks for one-shot image recognition,

    G. Koch, R. Zemel, R. Salakhutdinov, et al., “Siamese neural networks for one-shot image recognition,” inICML deep Learn. workshop, Lille, vol. 2, 2015, pp. 1–30

    2015

  36. [36]

    One-shot learning of object categories,

    L. Fei-Fei, R. Fergus, and P. Perona, “One-shot learning of object categories,”IEEE Trans. pattern analysis Mach. Intell., vol. 28, no. 4, pp. 594–611, 2006

    2006

  37. [37]

    Few- shot class-incremental learning,

    X. Tao, X. Hong, X. Chang, S. Dong, X. Wei, and Y . Gong, “Few- shot class-incremental learning,” inProc. IEEE/CVF Conf. Comput. Vis. pattern Recognit., 2020, pp. 12 183–12 192

    2020

  38. [38]

    The neurobench framework for benchmarking neuromorphic computing algorithms and systems,

    J. Yik, K. Van den Berghe, D. den Blanken, Y . Bouhadjar, M. Fabre, P. Hueber, W. Ke, M. A. Khoei, D. Kleyko, N. Pacik-Nelson, et al., “The neurobench framework for benchmarking neuromorphic computing algorithms and systems,”Nature Commun., vol. 16, no. 1, p. 1545, 2025

    2025

  39. [39]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., “Overcoming catastrophic forgetting in neural networks,”Proc. Nat. Acad. Sci., vol. 114, no. 13, pp. 3521–3526, 2017

    2017

  40. [40]

    Attribute-based classification for zero-shot visual object categorization,

    C. H. Lampert, H. Nickisch, and S. Harmeling, “Attribute-based classification for zero-shot visual object categorization,”IEEE Trans. pattern analysis Mach. Intell., vol. 36, no. 3, pp. 453–465, 2013

    2013

  41. [41]

    Trained transformers learn linear models in-context,

    R. Zhang, S. Frei, and P. L. Bartlett, “Trained transformers learn linear models in-context,”J. Mach. Learn. Res., vol. 25, no. 49, pp. 1–55, 2024

    2024

  42. [42]

    What can transformers learn in-context? a case study of simple function classes,

    S. Garg, D. Tsipras, P. S. Liang, and G. Valiant, “What can transformers learn in-context? a case study of simple function classes,”Advances neural Inf. Process. Syst., vol. 35, pp. 30 583–30 598, 2022

    2022

  43. [43]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., “Language models are few-shot learners,”Advances neural Inf. Process. Syst., vol. 33, pp. 1877–1901, 2020

    1901

  44. [44]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances neural Inf. Process. Syst., vol. 30, 2017

    2017

  45. [45]

    Mlps learn in-context on regression and classification tasks,

    W. L. Tong and C. Pehlevan, “Mlps learn in-context on regression and classification tasks,”arXiv preprint arXiv:2405.15618, 2024

    Pith/arXiv arXiv 2024

  46. [46]

    An empirical evaluation of generic convolutional and recurrent networks for sequence modeling

    S. Bai, J. Z. Kolter, and V . Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling”, 2018. arXiv:1803.01271

    Pith/arXiv arXiv 2018

  47. [47]

    Hyperdimensional computing: An introduction to comput- ing in distributed representation with high-dimensional random vectors,

    P. Kanerva, “Hyperdimensional computing: An introduction to comput- ing in distributed representation with high-dimensional random vectors,” Cogn. computation, vol. 1, no. 2, pp. 139–159, 2009

    2009

  48. [48]

    Very deep convolutional neural networks for raw waveforms,

    W. Dai, C. Dai, S. Qu, J. Li, and S. Das, “Very deep convolutional neural networks for raw waveforms,” in2017 IEEE Int. Conf. Acoust., speech signal Process. (ICASSP), IEEE, 2017, pp. 421–425

    2017

  49. [49]

    Speech Model Pre-training for End-to-End Spoken Language Understanding

    L. Lugosch, M. Ravanelli, P. Ignoto, V . S. Tomar, and Y . Bengio, “Speech Model Pre-training for End-to-End Spoken Language Understanding”, arXiv:1904.03670 [eess], Jul. 2019. Accessed: Nov. 10, 2025

    Pith/arXiv arXiv 1904

  50. [50]

    A high accuracy and ultra-energy-efficient zero- shot-retraining seizure detection processor,

    J. Liu, X. Liu, X. Wang, Z. Xie, C. Guo, Z. Zhong, J. Fan, H. Qiu, Y . Xu, H. Qin, et al., “A high accuracy and ultra-energy-efficient zero- shot-retraining seizure detection processor,”IEEE J. Solid-State Circuits, vol. 59, no. 11, pp. 3549–3565, 2024

    2024

  51. [51]

    In-context language learning: Architectures and algorithms,

    E. Aky ¨urek, B. Wang, Y . Kim, and J. Andreas, “In-context language learning: Architectures and algorithms,”arXiv preprint arXiv:2401.12973, 2024

    Pith/arXiv arXiv 2024

  52. [52]

    Efficient lifelong learning with a-gem,

    A. Chaudhry, M. Ranzato, M. Rohrbach, and M. Elhoseiny, “Efficient lifelong learning with a-gem,”arXiv preprint arXiv:1812.00420, 2018

    Pith/arXiv arXiv 2018

  53. [53]

    Normalization matters in zero-shot learning,

    I. Skorokhodov and M. Elhoseiny, “Normalization matters in zero-shot learning,” 2020

    2020

  54. [54]

    Tf- gczsl: Task-free generalized continual zero-shot learning,

    C. Gautam, S. Parameswaran, A. Mishra, and S. Sundaram, “Tf- gczsl: Task-free generalized continual zero-shot learning,”Neural Netw., vol. 155, pp. 487–497, 2022

    2022

  55. [55]

    Lifelong zero-shot learning.,

    K. Wei, C. Deng, X. Yang, et al., “Lifelong zero-shot learning.,” in IJCAI, 2020, pp. 551–557

    2020

  56. [56]

    Tinytrain: Resource-aware task-adaptive sparse training of dnns at the data-scarce edge,

    Y . D. Kwon, R. Li, S. I. Venieris, J. Chauhan, N. D. Lane, and C. Mascolo, “Tinytrain: Resource-aware task-adaptive sparse training of dnns at the data-scarce edge,”arXiv preprint arXiv:2307.09988, 2023

    Pith/arXiv arXiv 2023

  57. [57]

    Decision transformer: Reinforcement learning via sequence modeling,

    L. Chen, K. Lu, A. Rajeswaran, K. Lee, A. Grover, M. Laskin, P. Abbeel, A. Srinivas, and I. Mordatch, “Decision transformer: Reinforcement learning via sequence modeling,”Advances neural Inf. Process. Syst., vol. 34, pp. 15 084–15 097, 2021

    2021

  58. [58]

    Offline reinforcement learning as one big sequence modeling problem,

    M. Janner, Q. Li, and S. Levine, “Offline reinforcement learning as one big sequence modeling problem,”Advances neural Inf. Process. Syst., vol. 34, pp. 1273–1286, 2021

    2021

  59. [59]

    Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks,

    C. Frenkel, M. Lefebvre, and D. Bol, “Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks,”Frontiers Neuroscience, vol. V olume 15 - 2021, 2021,ISSN: 1662-453X

    2021

  60. [60]

    The forward-forward algorithm: Some preliminary investi- gations

    G. Hinton, “The forward-forward algorithm: Some preliminary investi- gations”, 2022.arXiv:2212.13345. Douwe den Blanken(Graduate Student Member, IEEE) received the M.Sc. degree (with honors) in embedded systems from Delft University of Tech- nology (TU Delft), Delft, The Netherlands, in 2023, where he is currently pursuing the Ph.D. degree, under the supe...

    Pith/arXiv arXiv 2022

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.