Pith. sign in

REVIEW 5 major objections 7 minor 1 cited by

TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computers

T0 review · 5 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that carefully pre-trained and LoRA fine-tuned language models of 30-124M parameters can match or beat billion-parameter models on embedded sensor classification, and can run quickly on single-board computers.

desk verdict A useful framework recipe for small task-specific LMs at the edge, with a plausible central claim that its own small, error-bar-free evaluation doesn't yet fully support. read the letter →

arxiv 2412.15304 v1 pith:RZIJCDIE submitted 2024-12-19 cs.LG cs.DCcs.ETcs.NI

Savitha Viswanadh Kandala , Pramuka Medaranga , Ambuj Varshney This is my paper · ORCID
classification cs.LGcs.DCcs.ETcs.NI
keywords smalllanguagemodelsedgeinferencesensordataclassificationpre-trainingcurationLoRAfine-tuningGPT-2single-boardcomputersembeddedsensing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TinyLLM asks whether the scaling trend in language models can be reversed for embedded sensing: instead of relying on billion-parameter models served over the network, the authors pre-train GPT-2-style models of 30-124M parameters on a curated mix of web text and sensor data, then LoRA fine-tune them on task-specific prompts. They report that these small models match or exceed the accuracy of much larger models (Phi-2/3, Llama-2/3) on hand-gesture, indoor-localisation, and swimming-style classification, while generating tokens far faster on single-board computers. The practical stake is that sensor data can be classified locally on devices with a few gigabytes of RAM, avoiding the latency, connectivity, and privacy problems of remote inference. The paper is an extension claim: it shows a recipe, not a law, proposing that careful data curation, rather than scale, carries accuracy on these tasks.

What carries the argument

The load-bearing mechanism is a GPT-2-style decoder-only transformer whose size is set by depth $l$, with hidden size $C=64l$ and parameters roughly $N=0.05l^3+3.2l$ million, spanning 30M to 124M parameters. It is pre-trained on a corpus that mixes FineWeb web text with SHL and ExtraSensory sensor data in user-chosen proportions (40:60 by default), then LoRA fine-tuned on Alpaca-style instruction/input/response prompts with sensor readings serialized to integers in a compact range so they fit the 1024-token window. The task reduces to next-token prediction: the correct label must appear within the first few generated tokens, and accuracy is scored exactly that way.

What would settle it

Run the same fine-tuning recipe on several held-out sensor datasets collected from different users, devices, lighting conditions, and locations than the ones used here (for example, public human-activity benchmarks with more classes and varied sensor placements), and compare 30-124M models against the largest fine-tuned baselines under identical prompt templates. If the small models' label-in-first-tokens accuracy falls well below the larger models' on datasets outside the original three, or if a small model again scores zero on a task with short sensor windows, the claim that small curated models are broadly sufficient for embedded sensing fails.

Watch

Extended reading notes

Core claim

The paper claims that by controlling the pre-training data and then fine-tuning, models with only tens of millions of parameters can be sufficient for embedded sensing. Concretely, GPT-2-style models of 30M to 124M parameters, pre-trained on a mixture of general web text and curated sensor data, then fine-tuned with LoRA on Alpaca-style prompt/response pairs, match or surpass the classification accuracy of much larger fine-tuned models on gesture, localisation, and swimming-style tasks. The authors also show these small models run locally on single-board computers with higher token rates than the larger models, and that a balanced sensor/web pre-training mix matters: pure web or pure sensor data degrades accuracy. The paper is candid about limits: only GPT-2-style architectures can be pre-trained, the context window is 1024 tokens, and on the one external dataset the top F1 of 0.78 trails the 0.97 of the original swimming-style study, in part because only 3 of 11 data streams fit the context window.

Load-bearing premise

The entire comparison rests on whether the three evaluation datasets (630 gesture samples, 350 localisation samples, and 3,730 swimming samples, only one from outside the authors' lab) represent real embedded sensing workloads; if those tasks are not representative, the conclusion that small models are broadly sufficient for edge sensing collapses.

Editorial extensions

If this is right

  • Models of 30-124M parameters can be trained on a single H100 GPU in about nine hours, so sensor applications no longer necessarily require cloud-hosted inference.
  • On constrained single-board computers, the small custom models still generate several tokens per second even where off-the-shelf models cannot be deployed, and multiple specialised models can run concurrently on a mid-range board.
  • A balanced mix of sensor data and general web text yields better fine-tuned accuracy than either extreme, suggesting that domain tokens should be a substantial minority of pre-training data.
  • LoRA fine-tuning requires relatively few examples, e.g. 440 gesture samples, which makes it practical to adapt a custom model to a new sensing task.
  • Larger models are not uniformly better on these tasks: custom 124M and 101M models match or beat several Phi and Llama variants on gesture and localisation accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the data-mix finding generalizes, the optimal recipe for other domains may be a majority of in-domain tokens rather than pure domain data; the paper only tests sensor/web splits on two tasks, so testing whether the 40:60 optimum shifts with model size or task would be a natural next step.
  • The 30M model's zero accuracy on gestures, alongside its highest token rate, suggests a practical Pareto frontier between accuracy and speed; a deployment tool could expose that trade-off instead of prescribing a single model size.
  • Because the framework converts sensor readings to compact text and uses next-token prediction, the same pipeline could be applied to other serialized time-series modalities such as audio features, IMU streams, or ECG, which the paper does not test.
  • The 'label in first tokens' scoring rule means the reported accuracy is tied to the chosen prompt template; a user changing the phrasing or adding distractor text could shift results, so the framework's real-world robustness depends on the template staying close to the training distribution.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes TinyLLM, a framework for pre-training small GPT-2-style language models (30-124M parameters) on mixtures of general web text and sensor data, fine-tuning them with LoRA on Alpaca-style prompt templates, and deploying them on single-board computers. The evaluation covers three sensing tasks: two in-house datasets (hand-gesture detection with 630 samples and indoor localization with 350 samples) and one external swimming-style dataset (3,730 samples), comparing accuracy, token generation rate, and inference time against fine-tuned Phi-2/3 and Llama-2/3 models. The central claim is that careful data curation allows these small custom models to match or surpass much larger off-the-shelf models on specific embedded sensing tasks while enabling faster local inference.

Significance. If fully supported, the central comparison would be a useful contribution to edge ML: the paper gives a concrete recipe for training and deploying sub-125M models locally, includes a systematic study of the web-to-sensor data mix (Fig. 11), and provides deployment measurements across three SBCs, multiple concurrent instances, and varied background loads (Figs. 13-17). These deployment results are the strongest part of the paper. The accuracy evidence for the headline 'surpass larger models' claim is currently weakened by the first-3-4-token accuracy rule, the absence of error bars and significance tests on the accuracy plots, the post hoc selection of the preferred data split, and the restricted swimming benchmark that uses only 3 of 11 data streams. The paper also ships a framework website and states training costs explicitly, which aids reproducibility. On balance, the central claim is defensible but needs additional evidence and a more careful comparison before it can be accepted at the level stated in the abstract and conclusion.

major comments (5)
  1. [§4, accuracy definition] The accuracy rule defined in §4 - checking for the expected label within the first 3-4 generated tokens, ignoring line breaks - creates a systematic bias against models that produce verbose but correct answers. Fine-tuned off-the-shelf models can emit a correct label after a preamble such as 'Based on the sensor data, the gesture is...', which would be counted as incorrect, while custom models trained on the exact Alpaca template are more likely to emit the bare label first. Because the headline claim rests on Fig. 10, please re-evaluate all models by extracting the label from the full generated output, report both the token-window metric and a full-output metric, or show explicitly that no correct labels occur outside the first 3-4 tokens.
  2. [§4, Figures 10-12] The experiment setup states that ten trials are run per configuration and that averages and standard deviations are plotted unless specified otherwise, but Figures 10-12 show no error bars or confidence intervals, and no statistical tests are reported. Several custom-vs-large differences are small (for example, the custom 124M model versus Phi-3 in Fig. 10b), so without variance information or a paired test the superiority claim is not established. Please add per-model error bars or confidence intervals on the accuracy figures and run a paired comparison (for example, a McNemar test on test predictions) for the key custom-versus-large contrasts.
  3. [§4.2, Figure 11] The 'Preferred Split' in Figure 11 is highlighted after inspecting the accuracy curve, and the text itself notes that the 0.9 split on the localization dataset reaches 33% accuracy only because the model consistently returns a single label for all prompts. This makes the claim that a balanced mix is optimal a post hoc selection rather than a tested hypothesis. Please select the split by a pre-specified rule (for example, lowest validation loss) and then evaluate on a held-out test split, or report results across multiple fine-tuning seeds for each split so that the reader can assess variability.
  4. [§4.3, Table 2 and swimming dataset] The swimming-style evaluation uses only 3 of the 11 available data streams and achieves a maximum F1-score of 0.78, well below the 0.97 reported by Brunner et al. [5], as the paper acknowledges. This weakens the conclusion that small models match or surpass larger and dedicated systems 'across various applications.' Please either evaluate on the full set of streams with an appropriate context-window representation, or clearly present the swimming result as a reduced-stream ablation and compare against a dedicated baseline trained on the same 3 streams.
  5. [§3.4 and §3.5, fine-tuning sensitivity] Because LoRA fine-tuning uses only 630 and 350 samples for the two in-house datasets, adherence to the instruction 'Give your answer only as Tap, Double, or Hold' is itself a variable that affects the reported accuracy. The paper does not report how often each model violates this output format, nor how sensitive the results are to the broad LoRA hyperparameter ranges listed in §3.5 (rank 16-256, dropout 0.1-0.3, learning rate 4e-4 to 6e-4). Please add an instruction-following breakdown per model (bare label, label with preamble, gibberish or repeated prompt) and a small hyperparameter sensitivity study for at least the 124M custom model.
minor comments (7)
  1. [§1, §3.4] The acronym is spelled 'LoRa' in one place in §1 and 'LoRA' elsewhere; please make the spelling consistent throughout.
  2. [§3.3] The text says the framework builds on 'llm.c' but no citation or repository link is given for that codebase; please add one.
  3. [§4.1, Figure 10] The text explains that Phi 3_q2 achieved zero accuracy, but Figure 10 omits this point entirely; please state it in the caption or show it as a 0% point so the reader can see the full comparison.
  4. [§4.2, Figure 11] The artificial 33% accuracy spike at split 0.9 for the localization dataset is discussed only in the text; please mark it in Figure 11 or note it in the caption.
  5. [§4.3, Figure 12] The text notes that the 30M custom model achieves zero accuracy on the gesture dataset; this is an important result and should be visible in Figure 12 or its caption rather than only in the body text.
  6. [Abstract] The abstract contains the typo 'invocating'; it should be 'invoking'.
  7. [§4.4-§4.6] The experiments in Figures 15-17 use different prompts and different values of n (number of generated tokens); please state the exact prompt and n in each caption so the token-rate comparisons are reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the accuracy claims rest on held-out test-set measurements and external architectures/toolkits, not on self-referential derivation.

full rationale

TinyLLM is an empirical systems paper. Its central claim—that custom 30–124M parameter GPT-2-style models can match or exceed larger models on selected embedded-sensing benchmarks—is supported by accuracy measurements on held-out test splits (Section 4, Figures 10–12), not by a derivation or by fitting a parameter and then predicting a quantity that is equivalent to the fit by construction. The pre-training data-mix ratio (Figure 11) is selected empirically by validation accuracy, and the LoRA hyperparameters are chosen by evaluation loss; these are standard model-selection practices, not circular predictions. The paper's accuracy metric (checking for the expected label within the first 3–4 tokens, ignoring line breaks) could disadvantage verbose off-the-shelf models, but this is a measurement-validity concern, not a definitional equivalence between input and output. The architectural choices (GPT-2, Alpaca template, LoRA, llama.cpp, FineWeb, SHL, Extrasensory) are all drawn from external prior work with no self-citation chain carrying the argument. The paper itself flags limitations (only GPT-2 pretraining, 1024-token context, reduced datastreams for swimming), which further indicates that no hidden self-referential mechanism is being used to force the results. No circular step can be quoted from the paper because none exists; the claims are empirical and independently falsifiable on the described test splits.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on hand-picked architecture and data-mix parameters, plus domain assumptions about text serialization and evaluation; no free-standing theoretical derivation is present.

free parameters (4)
  • sensor:web pre-training data mix ratio = 40:60 default; preferred split 0.6 in Figure 11
    The split is treated as a tunable hyperparameter and the preferred value is chosen after observing accuracy; no independent justification is given.
  • architecture scale constants (l, C=64l) = l=6,8,10,11,12; C=384,512,640,704,768
    Chosen by hand to hit the 30-124M parameter range; the capacity versus accuracy trade-off depends on these values.
  • LoRA fine-tuning hyperparameters (rank, dropout, LR, steps) = rank 16-256, dropout 0.1-0.3, LR 4e-4 to 6e-4, steps 100-300
    Final values are not reported; models are selected by validation loss from these ranges, so the reported accuracy depends on this tuning.
  • generation and evaluation parameters = T=0.7, repeat penalty=1.1, check first 3-4 tokens
    Fixed for consistency, but accuracy values are sensitive to these choices and no sensitivity analysis is provided.
assumptions (5)
  • domain assumption The curated 9B-token corpus (Fineweb plus SHL and ExtraSensory serialized readings) is a sufficient pre-training distribution for small sensor-capable models.
    Invoked in Section 3.5; the paper does not test alternative corpora or ablations beyond the 40:60 mix.
  • domain assumption Serializing sensor values into text with min-max normalization to integers in 0-100 preserves task-relevant information.
    Used in Section 3.2 and Figure 3; no comparison to alternative encodings is made.
  • domain assumption Alpaca-style instruction, input, response templates are sufficient for small models to map sensor readings to labels.
    Used in Section 3.4 and Figure 5; no template ablation is performed.
  • domain assumption Checking for the expected label within the first 3-4 generated tokens is a valid measure of task accuracy.
    Stated in Section 4; this metric could miss partially correct responses and is sensitive to tokenizer behavior.
  • standard math The GPT-2 architecture and the llm.c implementation are correct and behave as standard transformers.
    Background in Section 2 and Section 3.3 rely on prior implementation without formal verification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computers." pith.science (2026). https://pith.science/paper/RZIJCDIE

@misc{pith2026241215304,
  author       = {Pith},
  title        = {Pith review of: TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RZIJCDIE}},
  note         = {Machine review of arXiv:2412.15304}
}
read the original abstract

Language models have gained significant interest due to their general-purpose capabilities, which appear to emerge as models are scaled to increasingly larger parameter sizes. However, these large models impose stringent requirements on computing systems, necessitating significant memory and processing requirements for inference. This makes performing inference on mobile and edge devices challenging, often requiring invocating remotely-hosted models via network calls. Remote inference, in turn, introduces issues like latency, unreliable network connectivity, and privacy concerns. To address these challenges, we explored the possibility of deviating from the trend of increasing model size. Instead, we hypothesize that much smaller models (~30-120M parameters) can outperform their larger counterparts for specific tasks by carefully curating the data used for pre-training and fine-tuning. We investigate this within the context of deploying edge-device models to support sensing applications. We trained several foundational models through a systematic study and found that small models can run locally on edge devices, achieving high token rates and accuracy. Based on these findings, we developed a framework that allows users to train foundational models tailored to their specific applications and deploy them at the edge.

Figures

Figures reproduced from arXiv: 2412.15304 by the authors.

Figure 1
Figure 1. An embedded application often involves sensors that collect environmental data, which is then communicated to an edge device. TinyLLM provides a framework for training foundational models tailored for edge deployment, enabling these models to support a variety of tasks. This work explores training custom foundational models to enhance sensor data analysis. Our approach demonstrates a significantly smaller parameter-… view at source ↗
Figure 2
Figure 2. TinyLLM trains a custom foundational model for deployment at the edge device following a series of steps. It begins by appending a curated dataset with general conver￾sational data. After pre-processing, the dataset is tokenized to pre-train a small model (30-120M parameter). The pre-trained model undergoes fine-tuning with the custom dataset before deployment on the edge device to support embedded applica￾tions. Co… view at source ↗
Figure 3
Figure 3. Processing the dataset is essential for effective pre-training. This step addresses the challenges posed by the dataset’s diversity, ensures alignment of the dataset with the model’s context window size limitations, and formats the data appropriately for its usage with the subsequent training process. Transformation. The dataset structure often varies based on the application, with much of it consisting of numerical… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: The high-level representation of the architecture for the model used in this work is based on the GPT-2. The model architecture consists of l transformer blocks Transformer Block (Repeated 𝑙 Times): Each transformer block consists of multiple sub-blocks which are as fo…
Figure 5
Figure 5. Figure 5: We borrow a template from Alpaca for prompts and dataset entries required for fine-tuning a pre-trained model. Fine-tuning is an important step to ensure accurate responses to user queries for the specific application scenario. parameter size between 30M and 124M, as s…
Figure 6
Figure 6. Figure 6: Pre-trained foundational models, despite careful dataset curation, often show lower accuracy. Fine-tuning these models with a small, curated dataset from the target application scenario significantly improves their accuracy. Our TinyLLM framework supports fine-tuning f…
Figure 7
Figure 7. Figure 7: (a) A user performing a hand gesture, and (b) ob￾served light intensity values for different hand gestures. 6.5 cm 3.0 c m 14.6 cm 1 0.2 c m Orange Pi Zero 2W 10.0 cm 6.2 c m Orange Pi 5B LattePanda Sigma Computational Power [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Embedded platforms for running TinyLLM -trained models vary in processing and memory capabilities, ranging from a few hundred megabytes to several gigabytes of RAM. levels: low (100–200 lux), medium (600–750 lux), and high (1500–1600 lux), as well as across two distanc…
Figure 9
Figure 9. Figure 9: The localisation dataset was created based on data collected from sensors deployed on a moving robot in an indoor workspace. (a) shows sensors deployed on an indoor robot for sensor-based location detection, and (b)The physical location and sensor data collected. data …
Figure 3
Figure 3. Figure 3: This sensor dataset was combined with the Fineweb [PITH_FULL_IMAGE:figures/full_fig_p009_3.png]
Figure 10
Figure 10. Figure 10: Compares the accuracy of fine-tuned off-the-shelf models (Phi and Llama) and custom models across (a) gesture and (b) localisation datasets, as a function of model size. The plots show that larger models do not always achieve high accuracy. In several cases, a smaller…
Figure 11
Figure 11. Figure 11: Compares the accuracy of fine-tuned custom models (124M) on the (a) gesture and (b) localization datasets. The models are pre-trained on varying splits of sensor data and general web data, with a split of 0 indicating training solely on web data and a split of 1 indic…
Figure 13
Figure 13. Figure 13: Shows the variation of evaluation tokens per second when prompted with a gesture recognition prompt. The lowest token generation rate is comparable to the average human typing speed, demonstrating that custom models achieve rea￾sonable performance, even on resource-co…
Figure 14
Figure 14. Figure 14: Smaller models enable rapid inference. TinyLLM -trained models significantly improve inference time while maintaining high accuracy for the sensor data analysis. Insights. Smaller models achieve higher token generation rates and reduced inference times, presenting a t…
Figure 16
Figure 16. Figure 16: Displays the total time taken by different models for inferring location under various background load conditions on the LattePanda Sigma. Custom models significantly outperform the other models by completing the inference task within a second. results of the previous…
Figure 17
Figure 17. Figure 17: The smaller size of these models enables concurrent loading of multiple specialized models. Their token generation rates show they maintain efficient inference speeds even when running multiple instances simultaneously. We evaluated the performance of the custom model…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Communication-Efficient Hybrid Language Model via Uncertainty-Aware Opportunistic and Compressed Transmission

    cs.DC 2025-05 conditional novelty 6.0 of 10

    An uncertainty-aware hybrid language model skips and compresses uplink token transmissions, achieving up to 206 times higher token throughput with 97.4% accuracy in simulation.

Reference graph

Works this paper leans on

65 extracted references · 53 canonical work pages · cited by 1 Pith paper

  1. [5]

    Swimming style recognition and lap counting using a smartwatch and deep learning

    Gino Brunner, Darya Melnyk, Birkir Sigfússon, and Roger Wattenhofer. Swimming style recognition and lap counting using a smartwatch and deep learning. In Proceedings of the 2019 ACM International Symposium on Wearable Computers, ISWC ’19, page 23–31, New York, NY, USA,

    work page 2019

  2. [1]

    Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone, 2024

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, et al. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone, 2024

    work page 2024

  3. [2]

    The Claude 3 Model Family: Opus, Sonnet, Haiku, 2023

    Anthropic AI. The Claude 3 Model Family: Opus, Sonnet, Haiku, 2023. Accessed: 2024-06-24

    work page 2023

  4. [3]

    Beelink EQ13 N100, 2024

    Beelink. Beelink EQ13 N100, 2024. Accessed: 2024-10-23

    work page 2024

  5. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    work page 2020

  6. [6]

    Sparks of Artificial General Intelligence: Early experiments with GPT-4, 2023

    Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. Sparks of Artificial General Intelligence: Early experiments with GPT-4, 2023

    work page 2023

  7. [7]

    Deepseek- coder-v2: Breaking the barrier of closed-source models in code intelli- gence

    DeepSeek-AI, Qihao Zhu, Daya Guo, Zhihong Shao, et al. Deepseek- coder-v2: Breaking the barrier of closed-source models in code intelli- gence. arXiv preprint arXiv:2406.11931, 2024

    arXiv 2024

  8. [8]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

Show all 65 references
  1. [9]

    Speech-transformer: A no- recurrence sequence-to-sequence model for speech recognition

    Linhao Dong, Shuang Xu, and Bo Xu. Speech-transformer: A no- recurrence sequence-to-sequence model for speech recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5884–5888, 2018

    2018

  2. [10]

    An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weis- senborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  3. [11]

    The llama 3 herd of models, 2024

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Ka- dian, et al. The llama 3 herd of models, 2024

    2024

  4. [12]

    AdamW Optimizer

    Hugging Face. AdamW Optimizer. https://huggingface.co/docs/ bitsandbytes/main/en/reference/optim/adamw, 2024. Accessed: 2024- 05-20

    2024

  5. [13]

    Codebert: A pre-trained model for programming and natural languages, 2020

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. Codebert: A pre-trained model for programming and natural languages, 2020

    2020

  6. [14]

    Model Inversion Attacks that Exploit Confidence Information and Basic Countermea- sures

    Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model Inversion Attacks that Exploit Confidence Information and Basic Countermea- sures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security , CCS ’15, page 1322–1333, New York, NY, USA, 2...

    2015

  7. [15]

    Description of Quantization Types in llama.cpp

    Georgi Gerganov. Description of Quantization Types in llama.cpp. https://github.com/ggerganov/llama.cpp/pull/1684, 2024. Accessed: 2024-05-20

    2024

  8. [16]

    Llama.cpp.https://github.com/ggerganov/llama.cpp,

    Georgi Gerganov. Llama.cpp.https://github.com/ggerganov/llama.cpp,

  9. [17]

    Mahoney, and Kurt Keutzer

    Amir Gholami, Zhewei Yao, Sehoon Kim, Coleman Hooper, Michael W. Mahoney, and Kurt Keutzer. AI and Memory Wall . IEEE Micro , 44(03):33–39, May 2024

    2024

  10. [18]

    The university of sussex-huawei locomotion and transportation dataset for multimodal analytics with mobile devices

    Hristijan Gjoreski, Mathias Ciliberto, Lin Wang, Francisco Javier Or- donez Morales, Sami Mekki, Stefan Valentin, and Daniel Roggen. The university of sussex-huawei locomotion and transportation dataset for multimodal analytics with mobile devices. IEEE Access, 6:42592–42604, 2018

    2018

  11. [19]

    Rae, Oriol Vinyals, and Laurent Sifre

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osin...

    2022

  12. [20]

    Eval- uating large language models as virtual annotators for time-series physical sensing data, 2024

    Aritra Hota, Soumyajit Chatterjee, and Sandip Chakraborty. Eval- uating large language models as virtual annotators for time-series physical sensing data, 2024

    2024

  13. [21]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adapta- tion of large language models. In International Conference on Learning Representations, 2022

    2022

  14. [22]

    Dai, Matthew D

    Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, and Douglas Eck. Music Transformer, 2018

    2018

  15. [23]

    Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bam- ford, and other

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bam- ford, and other. Mistral 7B, 2023

    2023

  16. [24]

    Brown, Ben- jamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Ben- jamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling Laws for Neural Language Models, 2020

    2020

  17. [25]

    Mahoney, and Kurt Keutzer

    Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W. Mahoney, and Kurt Keutzer. SqueezeLLM: Dense-and-Sparse Quantization, 2024

    2024

  18. [26]

    Lattepanda sigma, 2024

    LattePanda Team. Lattepanda sigma, 2024. Accessed: 2024-10-23

    2024

  19. [27]

    Starcoder: may the source be with you!, 2023

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, et al. Starcoder: may the source be with you!, 2023

    2023

  20. [28]

    Large language models are few-shot health learners, 13 Conference’17, July 2017, Washington, DC, USA Viswanadh and Ambuj 2023

    Xin Liu, Daniel McDuff, Geza Kovacs, Isaac Galatzer-Levy, Jacob Sun- shine, Jiening Zhan, Ming-Zher Poh, Shun Liao, Paolo Di Achille, and Shwetak Patel. Large language models are few-shot health learners, 13 Conference’17, July 2017, Washington, DC, USA Viswanadh and Ambuj 2023

    2017

  21. [29]

    The era of 1-bit llms: All large language models are in 1.58 bits, 2024

    Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei. The era of 1-bit llms: All large language models are in 1.58 bits, 2024

    2024

  22. [30]

    Mankowitz et al

    Daniel J. Mankowitz et al. Faster sorting algorithms discovered using deep reinforcement learning. Nature, 618(7964):257–263, 2023

    2023

  23. [31]

    On faithfulness and factuality in abstractive summarization

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDon- ald. On faithfulness and factuality in abstractive summarization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computa- tio...

    1906

  24. [32]

    Meta large language model compiler: Foundation models of compiler optimization, 2024

    Meta AI. Meta large language model compiler: Foundation models of compiler optimization, 2024. Accessed: 2024-06-30

    2024

  25. [33]

    Codestral: Hello, world!, 2024

    Mistral AI. Codestral: Hello, world!, 2024. Accessed: 2024-06-30

    2024

  26. [34]

    Iot-lm: Large multisensory language models for the internet of things, 2024

    Shentong Mo, Russ Salakhutdinov, Louis-Philippe Morency, and Paul Pu Liang. Iot-lm: Large multisensory language models for the internet of things, 2024

    2024

  27. [35]

    ChatGPT (June 2024 version)

    OpenAI. ChatGPT (June 2024 version). https://www.openai.com/ chatgpt, 2024. Accessed: 2024-06-24

    2024

  28. [36]

    OpenAI Model Pricing, 2024

    OpenAI. OpenAI Model Pricing, 2024. Accessed: 2024-10-02

    2024

  29. [37]

    Gpt-4 technical report, 2024

    OpenAI, Josh Achiam, et al. Gpt-4 technical report, 2024

    2024

  30. [38]

    Carbon Emissions and Large Neural Network Training, 2021

    David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon Emissions and Large Neural Network Training, 2021

    2021

  31. [39]

    FineWeb, 04 2024

    Guilherme Penedo et al. FineWeb, 04 2024

    2024

  32. [40]

    Puccinelli and M

    D. Puccinelli and M. Haenggi. Wireless sensor networks: applica- tions and challenges of ubiquitous sensing. IEEE Circuits and Systems Magazine, 5(3):19–31, 2005

    2005

  33. [41]

    Language Models are Unsupervised Multitask Learners

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language Models are Unsupervised Multitask Learners. 2019

    2019

  34. [42]

    Raspberry Pi, 2024

    Raspberry Pi Foundation. Raspberry Pi, 2024. Accessed: 2024-06-20

    2024

  35. [43]

    Phi-2: The Surprising Power of Small Language Models

    Microsoft Research. Phi-2: The Surprising Power of Small Language Models. https://www.microsoft.com/en-us/research/blog/phi-2-the- surprising-power-of-small-language-models/ , 2023. Accessed: 2024- 06-24

    2023

  36. [44]

    High-resolution image synthesis with latent diffu- sion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffu- sion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684–10695, 2022

    2022

  37. [45]

    Code llama: Open foundation models for code

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023

    2023 arXiv

  38. [46]

    Grandmaster-level chess without search, 2024

    Anian Ruoss, Grégoire Delétang, Sourabh Medapati, Jordi Grau- Moya, Li Kevin Wenliang, Elliot Catt, John Reid, and Tim Genewein. Grandmaster-level chess without search, 2024

    2024

  39. [47]

    From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference

    Siddharth Samsi, Dan Zhao, Joseph McDonald, Baolin Li, Adam Michaleas, Michael Jones, William Bergeron, Jeremy Kepner, Devesh Tiwari, and Vijay Gadepally. From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference. In 2023 IEEE High Performance Extrem...

    2023

  40. [48]

    Orange Pi 5, 2022

    Shenzhen Xunlong Software CO., Limited. Orange Pi 5, 2022. Accessed: 2024-06-12

    2022

  41. [49]

    Orange Pi Zero 2W, 2022

    Shenzhen Xunlong Software CO., Limited. Orange Pi Zero 2W, 2022. Accessed: 2024-07-01

    2022

  42. [50]

    Knowledge boosting during low-latency inference, 2024

    Vidya Srinivas, Malek Itani, Tuochao Chen, Sefik Emre Eskimez, Takuya Yoshioka, and Shyamnath Gollakota. Knowledge boosting during low-latency inference, 2024

    2024

  43. [51]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford Alpaca: An Instruction-following LLaMA model. https://github.com/ tatsu-lab/stanford_alpaca, 2023

    2023

  44. [52]

    Gemini: A Family of Highly Capable Multimodal Models, 2024

    Gemini Team, Rohan Anil, et al. Gemini: A Family of Highly Capable Multimodal Models, 2024

    2024

  45. [53]

    Gemma: Open Models Based on Gemini Research and Technology, 2024

    Gemma Team, Thomas Mesnard, et al. Gemma: Open Models Based on Gemini Research and Technology, 2024

    2024

  46. [54]

    LLaMA 2: Open Foundation and Fine-Tuned Chat Models

    Hugo Touvron et al. LLaMA 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  47. [55]

    Recognizing de- tailed human context in the wild from smartphones and smartwatches

    Yonatan Vaizman, Katherine Ellis, and Gert Lanckriet. Recognizing de- tailed human context in the wild from smartphones and smartwatches. IEEE Pervasive Computing, 16(4):62–74, 2017

    2017

  48. [56]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention Is All You Need, 2023

    2023

  49. [57]

    Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M

    Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners, 2022

    2022

  50. [58]

    Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebas- tian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent Abilities of Large Language Models, 2022

    2022

  51. [59]

    SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models. In Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceed- ings of Mach...

    2023

  52. [60]

    Pene- trative ai: Making llms comprehend the physical world

    Huatao Xu, Liying Han, Qirui Yang, Mo Li, and Mani Srivastava. Pene- trative ai: Making llms comprehend the physical world. In Proceedings of the 25th International Workshop on Mobile Computing Systems and Applications, HOTMOBILE ’24, page 1–7, New York, NY, USA, 2024. Associa...

    2024

  53. [61]

    Hallucination is Inevitable: An Innate Limitation of Large Language Models, 2024

    Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. Hallucination is Inevitable: An Innate Limitation of Large Language Models, 2024

    2024

  54. [62]

    Mobile foundation model as firmware

    Jinliang Yuan, Chen Yang, et al. Mobile foundation model as firmware. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking, ACM MobiCom ’24, page 279–295, New York, NY, USA, 2024. Association for Computing Machinery

    2024

  55. [63]

    A Survey on Efficient Inference for Large Language Models, 2024

    Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, Shengen Yan, Guohao Dai, Xiao-Ping Zhang, Yuhan Dong, and Yu Wang. A Survey on Efficient Inference for Large Language Models, 2024. 14

    2024

  56. [2019]

    Association for Computing Machinery

  57. [2024]

    Accessed: 2024-05-20

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.