Pith. sign in

REVIEW 3 cited by

On-Device Qwen2.5: Efficient LLM Inference with Model Compression and Hardware Acceleration

Not yet reviewed by Pith; the record is open.

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2504.17376 v1 pith:K4AU6ZIW submitted 2025-04-24 cs.AR cs.LG

classification cs.ARcs.LG
keywords modelcompressionratechallengescomputationaledgeefficientexecution
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Transformer-based Large Language Models (LLMs) have significantly advanced AI capabilities but pose considerable challenges for deployment on edge devices due to high computational demands, memory bandwidth constraints, and energy consumption. This paper addresses these challenges by presenting an efficient framework for deploying the Qwen2.5-0.5B model on the Xilinx Kria KV260 edge platform, a heterogeneous system integrating an ARM Cortex-A53 CPU with reconfigurable FPGA logic. Leveraging Activation-aware Weight Quantization (AWQ) with FPGA-accelerated execution pipelines, the proposed approach enhances both model compression rate and system throughput. Additionally, we propose a hybrid execution strategy that intelligently offloads compute-intensive operations to the FPGA while utilizing the CPU for lighter tasks, effectively balancing the computational workload and maximizing overall performance. Our framework achieves a model compression rate of 55.08% compared to the original model and produces output at a rate of 5.1 tokens per second, outperforming the baseline performance of 2.8 tokens per second.

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Military AI Cyber Agents (MAICAs) Constitute a Global Threat to Critical Infrastructure

    cs.CY 2025-06 conditional novelty 6.0 of 10

    Autonomous AI cyber agents could credibly cause catastrophic damage to critical infrastructure by self-replicating and operating across global networks, according to this risk analysis.

  2. Efficient Online RFT with Plug-and-Play LLM Judges: Unlocking State-of-the-Art Performance

    cs.LG 2025-06 reject novelty 4.0 of 10

    A frozen 7B-8B LLM with a JSON rubric and a small LoRA adapter is claimed to outperform 27B-70B reward models and enable 92% GSM-8K exact match under online PPO.

  3. Standardization of Neuromuscular Reflex Analysis -- Role of Fine-Tuned Vision-Language Model Consortium and OpenAI gpt-oss Reasoning LLM Enabled Decision Support System

    cs.CV 2025-08 reject novelty 3.0 of 10

    A consortium of fine-tuned VLMs plus a reasoning LLM is proposed for H-reflex image analysis, but the claimed high accuracy is backed only by anecdotal examples.

Pith tools