Pith. sign in

REVIEW 4 major objections 6 minor 45 references

Tiny-Align: Bridging Automatic Speech Recognition and Large Language Model on the Edge

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Tiny-Align claims that ASR-LLM alignment can be done on an 8GB edge device by training only a transformer projector to match the LLM's text embeddings, cutting training time 50x and raising ROUGE scores by over 50%.

desk verdict Plausible idea, overstated results; missing the one baseline that would validate the embedding-surrogate premise. read the letter →

arxiv 2411.13766 v4 pith:MYAH3ZAG submitted 2024-11-21 cs.SD cs.AIeess.AS

Ruiyang Qin , Dancheng Liu , Gelei Xu , Zheyu Yan , Chenhui Xu , Yuting Hu , Shaocong Wang , X. Sharon Hu
show 2 more authors
Jinjun Xiong Yiyu Shi
This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords edgeASR-LLMcross-modalalignmentBridgeFormerEmbedLinkprojectortrainingpersonalizedspeechrecognitionROUGEevaluationon-devicepersonalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Tiny-Align claims that ASR and a large language model can be aligned on a resource-constrained edge device by training only a single transformer-based projector, not the ASR or the LLM. The projector is trained to turn audio features into embeddings that match the LLM's own text embeddings, avoiding the expensive LLM generation step used by existing alignment methods. On five speech datasets from speakers with dementia, aphasia, and language impairments, the paper reports roughly 50x faster convergence and ROUGE-1/ROUGE-L gains of more than 50% over three baselines, and an extra ~50% gain from injecting instructions at inference time. If this holds, personalized voice assistants could adapt to a specific user's way of speaking entirely on the device, without sending audio to a server.

What carries the argument

The load-bearing mechanism is EmbedLink, a training pipeline that replaces LLM generation with embedding matching, together with BridgeFormer, a transformer-encoder projector without positional encoding. Given a paired audio-transcript sample, the ASR encoder (wav2vec2 in the main experiments) extracts audio features; BridgeFormer maps them to shape $\mathbb{R}^{1\times T\times D_l}$; the LLM's embedding layer maps the transcript, padded or truncated to T tokens, to the same shape; and training minimizes $\alpha\cdot\text{MSE}+\beta\cdot(1-\cos)$. BridgeFormer uses an input MLP, four encoder layers, adaptive pooling to T=30 tokens, and an output MLP, omitting positional encoding because temporal order is already present in ASR features. During inference, instruction embeddings are concatenated ahead of the audio embeddings before the frozen LLM generates text.

What would settle it

Hold out a set of audio from users or conditions not represented in the five datasets, train Tiny-Align on the provided data, and check whether ROUGE-1/ROUGE-L scores against the transcript-only LLM outputs stay above the baselines while the EmbedLink loss remains low; a collapse to baseline quality would show that embedding matching alone is not carrying the alignment.

Watch

Extended reading notes

Core claim

The central discovery is that full end-to-end audio-language fine-tuning is unnecessary: an LLM's frozen embedding layer can serve as the training target. Tiny-Align trains BridgeFormer, a transformer encoder without positional encoding, to minimize a combined MSE and cosine-similarity loss between BridgeFormer's audio-derived embeddings and the LLM embedding layer's embeddings of the corresponding transcript, both cast to a fixed token size T. Because only the projector's parameters are updated and no LLM generation happens during training, the alignment converges in minutes on an 8GB device instead of hours, and the resulting audio embeddings feed the frozen LLM for generation. The paper reports that this projector-only alignment beats methods that fine-tune the LLM or the ASR encoder, across five LLMs and five datasets of atypical speech.

Load-bearing premise

The method assumes that making audio embeddings closely match the LLM's transcript embeddings is enough for the frozen LLM to generate good answers; if users' speech or instructions carry meaning that those text embeddings do not capture, the quality gains would shrink even though training stays fast.

Editorial extensions

If this is right

  • Personalized ASR-LLM alignment can happen on the user's own device: one 8GB edge board suffices for training, not just inference, so speech data need not leave the device.
  • The projector-only design cuts convergence time by about 50x compared with the NExT-GPT, X-VILA, and LanguageBind baselines, with higher ROUGE-1 and ROUGE-L scores on the reported datasets.
  • Because instructions are injected at inference time, the same trained projector can serve different tasks without retraining, and instruction injection alone improves output quality by roughly 50%.
  • The method transfers across LLM families (Llama-3.2, Gemma-2, Phi-3.5, StableLM-2), suggesting the embedding-matching target is not tied to one model's embedding space.
  • Feature-based ASR encoders such as wav2vec2 are the best fit for edge alignment, balancing memory use and convergence speed against transformer-based and generative ASR alternatives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if embedding matching is the real driver, the same EmbedLink recipe could align other modalities, such as images or video frames, to a frozen LLM provided the modality encoder's output can be pooled to a fixed token count.
  • Editorial inference: the casted token size caps the amount of audio context the LLM sees, so for open-ended dialogue the alignment may need a longer or hierarchical pooling scheme, a natural extension the paper leaves implicit.
  • Editorial inference: the reported quality metric rewards faithfulness to the ground-truth transcript, so an open question is whether the assistant's answers to a user's underlying intent improve as much as the ROUGE scores suggest.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Tiny-Align, a resource-efficient framework for cross-modal alignment between ASR encoders and frozen small LLMs on edge devices. The only trainable component is BridgeFormer, a transformer-encoder projector without positional encoding, trained with EmbedLink to map wav2vec2 audio features into the LLM's embedding space by minimizing a weighted MSE plus cosine loss against the LLM embeddings of the corresponding (truncated or padded) text. At inference time, instruction embeddings can be concatenated to the audio-derived embeddings. The authors evaluate on five TalkBank datasets and five small LLMs, claiming 50x faster training convergence and more than 50% improvement in ROUGE-1 and ROUGE-L over three baselines (NExT-GPT, X-VILA, LanguageBind), with a throughput and RAM validation on a Raspberry Pi 5.

Significance. If the claims were substantiated, the paper would address a real deployment problem: replacing expensive end-to-end ASR-LLM alignment with a projector-only objective that avoids LLM generation during training could make on-device personalization feasible. The paper uses appropriate clinical speech datasets, reports convergence curves, and the EmbedLink idea of regressing audio embeddings to frozen LLM text embeddings is plausible and worth testing. However, the current evidence does not establish the central claims. The 'more than 50% improvement' is contradicted by entries in Table I, the evaluation metric does not measure response quality relative to the actual task, the training was not run on the claimed edge device, and the baselines are not sufficiently controlled. The central embedding-matching premise is motivated but not directly validated.

major comments (4)
  1. [IV-B.2, Table I] The abstract and contributions claim 'more than 50%' improvement in alignment quality, but Table I contains cells where Tiny-Align is below the best baseline. For example, on ENNI with Gemma-2-2B, Ours has ROUGE-1 0.244 versus LanguageBind (A3) 0.256; on ENNI with Llama-3.2-3B, Ours has ROUGE-1 0.143 versus X-VILA (A2) 0.157. The paper does not specify the aggregation rule used to compute the 'more than 50%' improvement, so the claim is not evaluable and, under a best-baseline comparison, is false. Please report per-cell comparisons and a precise summary statistic.
  2. [IV-A.4 and III-C] The evaluation computes ROUGE between Output A (LLM generation from audio-derived embeddings) and Output L (LLM generation from the ground-truth transcript). This is a faithfulness measure, not a quality measure, and it has a ceiling of 1.0 if the embedding surrogate were exact. The reported ROUGE-1 values are mostly 0.10-0.27, far below that ceiling. The paper must distinguish whether BridgeFormer fails to reach the embedding target or whether the embedding-layer surrogate is insufficient for generation. A direct oracle test that feeds Etext, the embedding of the ground-truth text, to the LLM and measures the resulting ROUGE would provide this separation. Without such a test, the central premise in Section III-C remains an assumption.
  3. [IV-A.3 and IV-B.1] The 'on edge' claim is not supported. All comprehensive training and convergence results, including Table I and Figures 7-9, were obtained on a single Nvidia P100 GPU, as stated in Section IV-A.3. Section IV-B.1 on Raspberry Pi 5 only measures throughput and RAM for BridgeFormer configurations, not training time or output quality. Convergence time measured on a datacenter GPU cannot be directly transferred to a Raspberry Pi 5, so the 50x training speedup claim for edge devices is not demonstrated. Please run the training and evaluation protocol on the target edge hardware, or substantially qualify the claims.
  4. [IV-A.5 and IV-B.2] The baseline comparisons are not controlled. The text states that 'we must limit their trainable parameters so that they can run in resource-limited environments' but does not report which parameters were frozen, the number of trainable parameters, or the hyperparameter tuning for each baseline under these constraints. In addition, there is no baseline that feeds the wav2vec2-recognized transcript directly to the LLM; this natural baseline would isolate the value of the projector. Without these controls, the reported improvements over A1/A2/A3 may reflect weakened baselines rather than the proposed method.
minor comments (6)
  1. [Abstract] The abstract in the review materials refers to 'NVIDIA Jetson Orin (8GB RAM)' while the full-text abstract and experiments refer to 'Raspberry Pi 5 (8GB RAM)'; these must be reconciled.
  2. [IV-A.1] The dataset description uses placeholders 'approximately X training pairs and Y validation pairs' instead of actual dataset sizes; please provide the exact numbers for reproducibility.
  3. [IV-A.4] Section IV-A.4 defines convergence via 'delta loss < epsilon' but never reports epsilon or the criterion used; without this, the convergence-time comparisons in Table I and Figures 7-8 are not reproducible.
  4. [Figures 4 and 6] Figure 4's caption contains 'BridageFormer' and Figure 6's caption contains 'Raspbarry Pi 5'; please correct these typos.
  5. [III-C, Eq. (6)] Equation (6) writes 'BridgeFormer(Faudio) in R^{1xN x Da} -> R^{1xT x Dl}', which mixes the input and output spaces in a single set-membership expression; it should be written as a function type, for example BridgeFormer: R^{1xN x Da} -> R^{1xT x Dl}.
  6. [References] The reference list is incomplete: entries [8], [18], [21], [22], and [42] use 'et al.' placeholders without full author lists; please complete them.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Tiny-Align trains a projector against a frozen external target (LLM text embeddings) and evaluates with an independent downstream ROUGE comparison.

full rationale

The derivation is self-contained. Tiny-Align's training target (Eq. 7) is a fixed, frozen function of the ground-truth transcript and the LLM embedding layer: Etext = LLMembed(X), Eaudio = BridgeFormer(Faudio), with loss L = α·MSE + β·(1−cos). The fitted parameters are only in BridgeFormer; the target does not depend on those parameters, so the objective is not self-definitional. The evaluation metric (ROUGE between Output A and Output L) is an external, downstream measure of whether the embedding-space equivalence transfers to generation; although it is related to the training objective, it is not the training loss and is not forced to be high by construction, as the absolute ROUGE values (≈0.1–0.3) show. The claimed speedups and quality gains are comparisons against three separately defined baselines (NExT-GPT, X-VILA, LanguageBind) under stated resource limits, not against the method's own fitted values. Self-citations appear only in background and related work and are not load-bearing; no uniqueness theorem is imported from the authors' prior work. The central premise—that embedding matching suffices for cross-modal alignment—is an empirical assumption that could fail, but a potentially false assumption is not circularity under the standard requiring an exhibited equation-to-equation reduction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim rests on a handful of hand-set hyperparameters (casted token size, loss weights, convergence threshold) and on the domain assumption that LLM text embeddings are a good alignment target. No external benchmarks are used to calibrate these choices, so the quantitative results are sensitive to them.

free parameters (4)
  • casted token size T = 30
    Empirically set in Section IV-A-3 to balance sentence length coverage and compute; affects the output token count of BridgeFormer and the target text embeddings.
  • loss weights alpha and beta = not specified
    Equation (7) defines the loss as alpha*MSE + beta*(1-cos), but the paper never reports the chosen values; the balance between the two terms is a free choice that affects training.
  • convergence target loss = 0.05
    Table I measures convergence time to a target loss of 0.05; this threshold is chosen by hand and directly determines the reported speedups.
  • training data size for comprehensive evaluation = 100 samples
    Section IV-B-2 states 'we use a dataset of 100 samples for this comprehensive evaluation'; this size is a design choice and may affect ROUGE comparisons.
assumptions (4)
  • domain assumption LLM embedding layer is a sufficient training target for cross-modal alignment
    Stated in Section III-C: matching audio embeddings to LLM text embeddings is claimed to achieve effective alignment without LLM fine-tuning or inference during training.
  • domain assumption Temporal position is already encoded in ASR features, so positional encoding can be omitted from BridgeFormer
    Section III-B removes positional encoding because 'temporal positioning is already encoded' in ASR-extracted audio features; if this assumption fails, the transformer projector loses ordering information.
  • domain assumption ROUGE similarity between ASR-driven output and text-driven output is a valid proxy for alignment quality
    Section IV-A-4 defines the dual-path comparison using ROUGE-1 and ROUGE-L; the entire quality claim rests on this metric's validity for this task.
  • ad hoc to paper The selected baselines remain representative after their trainable parameters are limited
    Section IV-B-2 notes that baseline parameters had to be limited to run in resource-limited environments, but no details are given, which may handicap the baselines.
invented entities (1)
  • BridgeFormer
    purpose: Transformer-based projector that maps ASR audio features into LLM-compatible embeddings (Section III-B).
    A new architectural component introduced by the paper; its effectiveness is only evidenced by the paper's own experiments, with no external independent validation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tiny-Align: Bridging Automatic Speech Recognition and Large Language Model on the Edge." pith.science (2026). https://pith.science/paper/MYAH3ZAG

@misc{pith2026241113766,
  author       = {Pith},
  title        = {Pith review of: Tiny-Align: Bridging Automatic Speech Recognition and Large Language Model on the Edge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MYAH3ZAG}},
  note         = {Machine review of arXiv:2411.13766}
}
read the original abstract

The combination of Large Language Models (LLM) and Automatic Speech Recognition (ASR), when deployed on edge devices (called edge ASR-LLM), can serve as a powerful personalized assistant to enable audio-based interaction for users. Compared to text-based interaction, edge ASR-LLM allows accessible and natural audio interactions. Unfortunately, existing ASR-LLM models are mainly trained in high-performance computing environments and produce substantial model weights, making them difficult to deploy on edge devices. More importantly, to better serve users' personalized needs, the ASR-LLM must be able to learn from each distinct user, given that audio input often contains highly personalized characteristics that necessitate personalized on-device training. Since individually fine-tuning the ASR or LLM often leads to suboptimal results due to modality-specific limitations, end-to-end training ensures seamless integration of audio features and language understanding (cross-modal alignment), ultimately enabling a more personalized and efficient adaptation on edge devices. However, due to the complex training requirements and substantial computational demands of existing approaches, cross-modal alignment between ASR audio and LLM can be challenging on edge devices. In this work, we propose a resource-efficient cross-modal alignment framework that bridges ASR and LLMs on edge devices to handle personalized audio input. Our framework enables efficient ASR-LLM alignment on resource-constrained devices like NVIDIA Jetson Orin (8GB RAM), achieving 50x training time speedup while improving the alignment quality by more than 50\%. To the best of our knowledge, this is the first work to study efficient ASR-LLM alignment on resource-constrained edge devices.

Figures

Figures reproduced from arXiv: 2411.13766 by the authors.

Figure 1
Figure 1. Evaluation of ASR-LLM alignment using Whisper-base [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Tiny-Align Framework. It unifies the trainable parameters into the designed transformer-based projector (BridgeFormer), [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Architecture of BridgeFormer. sole trainable module in Tiny-Align, our projector aims to leverage the computational resources saved from avoiding LLM generation and training, allowing for a moderately larger architecture while remaining within edge device constraints. To meet these design criteria, we introduce a transformer encoder architecture into the projector, named BridgeFormer to reflect its role in bridging … view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Resource efficiency comparison of three ASR models. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 8
Figure 8. Figure 8: Evaluation of our method and baselines on differ [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 7
Figure 7. Figure 7: Demonstration of the loss decrease when training the [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 9
Figure 9. Figure 9: Performance comparison of instruction injection in [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 24 canonical work pages

  1. [1]

    Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesis

    Ruiyang Qin, Jun Xia, Zhenge Jia, Meng Jiang, Ahmed Abbasi, Peipei Zhou, Jingtong Hu, and Yiyu Shi. Enabling on-device large language model personalization with self-supervised data selection and synthesis. arXiv preprint arXiv:2311.12275 , 2023

    work page Pith review arXiv 2023

  2. [2]

    An llm-based framework for fingerprinting internet-connected devices

    Armin Sarabi, Tongxin Yin, and Mingyan Liu. An llm-based framework for fingerprinting internet-connected devices. In Proceedings of the 2023 ACM on Internet Measurement Conference , pages 478–484, 2023

    work page 2023

  3. [3]

    Language models for online depression detection: A review and benchmark analysis on remote interviews

    Ruiyang Qin, Ryan Cook, Kai Yang, Ahmed Abbasi, David Dobolyi, Salman Seyedi, Emily Griner, Hyeokhyen Kwon, Robert Cotes, Zifan Jiang, et al. Language models for online depression detection: A review and benchmark analysis on remote interviews. ACM Transactions on Management Information Systems , 2024

    work page 2024

  4. [4]

    Combating partial perception deficit in autonomous driving with multimodal llm commonsense

    Yuting Hu, Chenhui Xu, Ruiyang Qin, Dancheng Liu, Amir Nasserel- dine, Yiyu Shi, and Jinjun Xiong. Combating partial perception deficit in autonomous driving with multimodal llm commonsense. arXiv preprint arXiv:2503.07020, 2025

    work page arXiv 2025

  5. [5]

    PI-Whisper: Designing an Adaptive and Incremental Automatic Speech Recognition System for Edge Devices

    Amir Nassereldine, Dancheng Liu, Chenhui Xu, and Jinjun Xiong. Pi- whisper: An adaptive and incremental asr framework for diverse and evolving speaker characteristics. arXiv preprint arXiv:2406.15668, 2024

    work page Pith review arXiv 2024

  6. [6]

    Automatic speech recognition , volume 1

    Dong Yu and Lin Deng. Automatic speech recognition , volume 1. Springer, 2016

    2016

  7. [7]

    Mala-asr: Multimedia-assisted llm-based asr

    Guanrou Yang, Ziyang Ma, Fan Yu, Zhifu Gao, Shiliang Zhang, and Xie Chen. Mala-asr: Multimedia-assisted llm-based asr. arXiv preprint arXiv:2406.05839, 2024

    arXiv 2024

  8. [8]

    Seed-asr: Understanding diverse speech and contexts with llm-based speech recognition

    Bai et al. Seed-asr: Understanding diverse speech and contexts with llm-based speech recognition. arXiv preprint arXiv:2407.04675 , 2024

    arXiv 2024

Show all 45 references
  1. [9]

    Salsa: Speedy asr-llm synchronous aggregation

    Ashish Mittal, Darshan Prabhu, Sunita Sarawagi, and Preethi Jyothi. Salsa: Speedy asr-llm synchronous aggregation. arXiv preprint arXiv:2408.16542, 2024

    2024 arXiv

  2. [10]

    Next-gpt: Any-to-any multimodal llm

    Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-to-any multimodal llm. arXiv preprint arXiv:2309.05519, 2023

    2023 arXiv

  3. [11]

    X-vila: Cross-modality alignment for large language model

    Hanrong Ye, De-An Huang, Yao Lu, Zhiding Yu, Wei Ping, Andrew Tao, Jan Kautz, Song Han, Dan Xu, Pavlo Molchanov, et al. X-vila: Cross-modality alignment for large language model. arXiv preprint arXiv:2405.19335, 2024

    2024 arXiv

  4. [12]

    Language- bind: Extending video-language pretraining to n-modality by language- based semantic alignment

    Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, HongFa Wang, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zongwei Li, et al. Language- bind: Extending video-language pretraining to n-modality by language- based semantic alignment. arXiv preprint arXiv:2310.01852 , 2023

    arXiv

  5. [13]

    Robust speech recognition via large-scale weak supervision

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning , pages 28492–28518. PMLR, 2023

    2023

  6. [14]

    Gemma 2: Improving open language models at a practical size

    Team Gemma. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118 , 2024

    2024 arXiv

  7. [15]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323 , 2022

    2022 arXiv

  8. [16]

    wav2vec 2.0: A framework for self-supervised learning of speech representations

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems , 33:12449–12460, 2020

    2020

  9. [17]

    A comparative analysis between conformer-transducer, whisper, and wav2vec2 for im- proving the child speech recognition

    Andrei Barcovschi, Rishabh Jain, and Peter Corcoran. A comparative analysis between conformer-transducer, whisper, and wav2vec2 for im- proving the child speech recognition. In 2023 International Conference on Speech Technology and Human-Computer Dialogue (SpeD) , pages 42–47....

    2023

  10. [18]

    Conformer: Convolution-augmented transformer for speech recognition

    Gulati et al. Conformer: Convolution-augmented transformer for speech recognition. arXiv preprint arXiv:2005.08100 , 2020

    2005 arXiv

  11. [19]

    Automatic screening for children with speech disorder using automatic speech recognition: Opportunities and challenges

    Dancheng Liu, Jason Yang, Ishan Albrecht-Buehler, Helen Qin, Sophie Li, Yuting Hu, Amir Nassereldine, and Jinjun Xiong. Automatic screening for children with speech disorder using automatic speech recognition: Opportunities and challenges. In Proceedings of the AAAI Symposium ...

    2024

  12. [20]

    Diffsound: Discrete diffusion model for text-to-sound generation

    Dongchao Yang, Jianwei Yu, Helin Wang, Wen Wang, Chao Weng, Yuexian Zou, and Dong Yu. Diffsound: Discrete diffusion model for text-to-sound generation. IEEE/ACM Transactions on Audio, Speech, and Language Processing , 31:1720–1733, 2023

    2023

  13. [21]

    Audiolm: a language modeling approach to audio generation

    Borsos et al. Audiolm: a language modeling approach to audio generation. IEEE/ACM transactions on audio, speech, and language processing, 31:2523–2533, 2023

    2023

  14. [22]

    Tango 2: Aligning diffusion-based text-to-audio genera- tive models through direct preference optimization

    Majumder et al. Tango 2: Aligning diffusion-based text-to-audio genera- tive models through direct preference optimization. In ACM Multimedia 2024, 2024

    2024

  15. [23]

    Open-ended multi-modal relational reasoning for video question answer- ing

    Haozheng Luo, Ruiyang Qin, Chenwei Xu, Guo Ye, and Zening Luo. Open-ended multi-modal relational reasoning for video question answer- ing. In 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) , pages 363–369. IEEE, 2023

    2023

  16. [24]

    Empirical guidelines for deploying llms onto resource-constrained edge devices

    Ruiyang Qin, Dancheng Liu, Chenhui Xu, Zheyu Yan, Zhaoxuan Tan, Zhenge Jia, Amir Nassereldine, Jiajie Li, Meng Jiang, Ahmed Abbasi, et al. Empirical guidelines for deploying llms onto resource-constrained edge devices. arXiv preprint arXiv:2406.03777 , 2024

    2024 arXiv

  17. [25]

    Nvcim- pt: An nvcim-assisted prompt tuning framework for edge llms

    Ruiyang Qin, Pengyu Ren, Zheyu Yan, Liu Liu, Dancheng Liu, Amir Nassereldine, Jinjun Xiong, Kai Ni, Sharon Hu, and Yiyu Shi. Nvcim- pt: An nvcim-assisted prompt tuning framework for edge llms. arXiv preprint arXiv:2411.08244, 2024

    2024 arXiv

  18. [26]

    Robust implementation of retrieval-augmented generation on edge-based computing-in-memory architectures

    Ruiyang Qin, Zheyu Yan, Dewen Zeng, Zhenge Jia, Dancheng Liu, Jianbo Liu, Zhi Zheng, Ningyuan Cao, Kai Ni, Jinjun Xiong, et al. Robust implementation of retrieval-augmented generation on edge-based computing-in-memory architectures. arXiv preprint arXiv:2405.04700 , 2024

    2024 arXiv

  19. [27]

    From linguistic giants to sensory maestros: A survey on cross-modal reasoning with large language models

    Shengsheng Qian, Zuyi Zhou, Dizhan Xue, Bing Wang, and Changsheng Xu. From linguistic giants to sensory maestros: A survey on cross-modal reasoning with large language models. arXiv preprint arXiv:2409.18996, 2024

    2024 arXiv

  20. [28]

    Enabling edge devices that learn from each other: Cross modal training for activity recognition

    Xing et al. Enabling edge devices that learn from each other: Cross modal training for activity recognition. In Proceedings of the 1st International Workshop on Edge Systems, Analytics and Networking , pages 37–42, 2018

    2018

  21. [29]

    Fl-nas: Towards fairness of nas for resource constrained devices via large language models

    Ruiyang Qin, Yuting Hu, Zheyu Yan, Jinjun Xiong, Ahmed Abbasi, and Yiyu Shi. Fl-nas: Towards fairness of nas for resource constrained devices via large language models. In 2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC) , pages 429–434. IEEE, 2024

    2024

  22. [30]

    Dementia mortality among adults age 65 and older: United states, 2018-2022

    Ellen A Kramarow and Betzaida Tejada-Vera. Dementia mortality among adults age 65 and older: United states, 2018-2022. 2024

    2018

  23. [31]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  24. [32]

    Ibert: Idiom cloze-style reading comprehension with attention

    Ruiyang Qin, Haozheng Luo, Zheheng Fan, and Ziang Ren. Ibert: Idiom cloze-style reading comprehension with attention. arXiv preprint arXiv:2112.02994, 2021

    2021 arXiv

  25. [33]

    Instruction tuning with gpt-4

    Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277 , 2023

    2023 arXiv

  26. [34]

    The talkbank project

    Brian MacWhinney. The talkbank project. In Creating and digitizing language corpora: Volume 1: Synchronic databases , pages 163–180. Springer, 2007

    2007

  27. [35]

    Tackling the adress challenge: A mul- timodal approach to the automated recognition of alzheimer’s dementia

    Matej Martinc and Senja Pollak. Tackling the adress challenge: A mul- timodal approach to the automated recognition of alzheimer’s dementia. In Interspeech, pages 2157–2161, 2020

    2020

  28. [36]

    Baycrest pwa corpus

    Jed Meltzer. Baycrest pwa corpus

  29. [37]

    Fast mapping in late-talking toddlers

    Susan Ellis Weismer, Courtney E Venker, Julia L Evans, and Maura Jones Moyle. Fast mapping in late-talking toddlers. Applied Psycholinguistics, 34(1):69–89, 2013

    2013

  30. [38]

    Dis- criminating children with language impairment among english-language learners from diverse first-language backgrounds

    Johanne Paradis, Phyllis Schneider, and Tamara Sorenson Duncan. Dis- criminating children with language impairment among english-language learners from diverse first-language backgrounds. 2013

    2013

  31. [39]

    Aphasia talkbank neural dataset

    TalkBank. Aphasia talkbank neural dataset. https://aphasia.talkbank.org/ access/English/Aphasia/NEURAL.html, 2024. Accessed: 2024-11-18

    2024

  32. [40]

    Institutional review boards: Purpose and challenges

    Christine Grady. Institutional review boards: Purpose and challenges. Chest, 148(5):1148–1155, 2015

    2015

  33. [41]

    Llama 3.2: Revolutionizing edge ai and vision with open, customizable models

    Meta AI. Llama 3.2: Revolutionizing edge ai and vision with open, customizable models. arXiv, 2024

    2024

  34. [42]

    Phi-3 technical report: A highly capable language model locally on your phone

    Abdin et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219 , 2024

    2024 arXiv

  35. [43]

    Stable lm 2 1.6 b technical report

    Marco Bellagente, Jonathan Tow, Dakota Mahan, Duy Phung, Maksym Zhuravinskyi, Reshinth Adithyan, James Baicoianu, Ben Brooks, Nathan Cooper, Ashish Datta, et al. Stable lm 2 1.6 b technical report. arXiv preprint arXiv:2402.17834, 2024

    2024 arXiv

  36. [44]

    Rouge: A package for automatic evaluation of sum- maries

    Chin-Yew Lin. Rouge: A package for automatic evaluation of sum- maries. In Text summarization branches out , pages 74–81, 2004

    2004

  37. [45]

    Autofl: Enabling heterogeneity- aware energy efficient federated learning

    Young Geun Kim and Carole-Jean Wu. Autofl: Enabling heterogeneity- aware energy efficient federated learning. In MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture , pages 183– 198, 2021

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.