Pith. sign in

REVIEW 4 major objections 7 minor 75 references

FBLayout claims a single tile-based memory layout can resolve the forward-backward layout conflict that slows on-device LLM fine-tuning, speeding training 2.2–5.7×.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 08:17 UTC pith:WIMQG37T

load-bearing objection A solid, well-motivated mobile training systems paper with a plausible new layout technique; the speedups are probably real, but the paper lacks code, error bars, and a rigorous proof of its core tile-invariance assumption. the 4 major comments →

arxiv 2607.21624 v1 pith:WIMQG37T submitted 2026-07-07 cs.AI cs.DCcs.LG

FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUs

Kahou Tam , Wei Niu , Yu Bao , Xiaomin Ouyang , Chengzhong Xu , Li Li This is my paper
classification cs.AI cs.DCcs.LG
keywords on-device trainingLLM fine-tuningmobile GPUtensor layouttexture memorycache efficiencylayout transformation elimination
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

FBLayout tries to establish that the main bottleneck in on-device LLM fine-tuning is not compute but the way tensors are stored in memory, and that a single tile-based layout can serve both training passes. The claim is that a unified R-Tile layout, designed around the 2D spatial locality of mobile GPUs' texture caches, keeps both forward and backward reductions cache-efficient because reductions along different axes load the same tile elements. If correct, this makes on-device fine-tuning 2.2–5.7× faster than existing mobile frameworks, with fewer cache misses and lower energy, without changing model accuracy. A reader should care because it offers a concrete path to private, personalized LLMs that update locally on phones.

Core claim

Tile-invariance: forward and backward matrix multiplications on a shared weight load exactly the same elements — only aggregation order changes. Since mobile GPU texture caches fetch rectangular, horizontally-wide blocks, a tile layout with cache-line-aligned intra-tile packing and the traversal-heavy dimension placed horizontally keeps both passes cache-resident. Transformer reshape/transpose chains usually preserve tile atomicity, so physical data movement can be replaced by precomputed tile-index remapping; an operator taxonomy anchors R-Tile at reduction-heavy ops and propagates it globally. This yields 2.2–5.7× faster fine-tuning on seven transformer models across two GPU families than

What carries the argument

R-Tile (Reduction-aware Tiling) — a tile-based memory layout that maps tensor elements to a mobile GPU's 2.5D texture memory in two levels: intra-tile packing (row-major strips for small tiles, benchmark-selected 2D blocks for large tiles) and inter-tile arrangement (row-major with the most-traversed dimension placed horizontally, matching the cache block's wider horizontal extent). It works because reductions along any axis within a tile fetch the same elements, so one layout serves both passes. Supporting machinery: tile-based index transformation — replacing reshape/transpose with offline-composed tile-index maps after legality checks (tile dimension dependency, tile boundary preservation

Load-bearing premise

The argument rests on tile-invariance — that every forward and backward reduction on a shared tensor fetches exactly the same set of tile elements (only the aggregation order differs) — and on the profiled texture-cache block shape matching the real phone; if any reduction touches only part of a tile, or the cache geometry differs from the benchmark, the unified layout loses its edge over per-pass layouts.

What would settle it

Record the exact cache lines touched by a forward MatMul and its backward gradient MatMul on the same weight matrix (e.g., M=512, K=2048, N=8192 vs. M=512, K=8192, N=2048). Tile-invariance predicts identical cache-line sets up to tile padding; if the backward pass touches lines the forward pass never uses, the unified R-Tile layout is not truly serving both passes and the claimed speedup mechanism is falsified.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • On-device fine-tuning of decoder-only LLMs (Llama3.2-1B, Qwen2.5-1.5B, Gemma2-2B) becomes roughly 4–5.7× faster than existing mobile frameworks, making private, personalized language models on phones practical.
  • Memory overhead falls: the framework issues ~3.5× fewer global-memory accesses and ~4.2× fewer cache misses than the baselines, cutting energy use by 3.5–6.3× over a training session.
  • Layout compilation for a 1B-parameter training graph takes about 6 seconds (versus hundreds or thousands for baselines), and adapting to a new GPU is a one-time ~20–26 second profiling step, so per-task optimization is feasible on-device.
  • The same approach speeds up CNN fine-tuning (MobileNetV2, VGG-19, ResNet34) by 2.2–4.2×, showing the forward-backward reduction conflict and its tile-based fix are not specific to transformers.
  • Because the method only rearranges how tensors are stored, it composes with LoRA, QLoRA, and FlashAttention-style algorithmic optimizations, attacking memory traffic on an orthogonal axis.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A consequence the paper leaves implicit: the R-Tile principle should transfer to any accelerator whose cache fetches data in 2D blocks with unequal width and height — the heuristic 'align the heavy reduction dimension with the wider cache axis' is a general rule that could be stated and tested on desktop GPUs and NPUs.
  • If tile-invariance is exact, the optimal tile shape on a new GPU should be predictable from two hardware numbers (cache block width and height) rather than from empirical search; the reported 6-second tuning time hints at such a law, but the paper does not state it.
  • For federated or continual on-device learning, R-Tile could serve as a canonical tensor exchange format between clients, preventing layout conversions during model aggregation — an extension the paper does not discuss.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. FBLayout is a layout-optimization framework for on-device LLM fine-tuning on mobile GPUs. The paper identifies a forward-backward layout conflict in training graphs, classifies operators into four categories based on forward-tensor dependency and access-pattern consistency, and proposes a unified R-Tile layout intended to serve both passes without explicit transformation. It augments this with tile-based index transformation (replacing physical transpose/reshape with tile-level coordinate remapping) and activation-guided global layout selection. Evaluations on seven transformer models across three phones (Adreno 830/730 and Mali Immortalis-G925) report 2.2–5.7× end-to-end speedups over MNN, TFLite, and TVM, along with reduced cache misses, lower energy, and short compilation time.

Significance. The paper targets an important and under-served problem: efficient on-device fine-tuning, not just inference. If the tile-invariance premise holds, R-Tile is an elegant synthesis of texture-memory locality with the bidirectional dataflow of training. The strengths include a broad model/device evaluation, incremental ablations that isolate R-Tile, index elimination, and global selection, compilation-time comparisons (6.1 s vs. 723 s for MNN), energy measurement, and portability tests on two GPU families. However, the central invariance claim is supported by only a single MatMul illustration, the manual-construction fairness of the TVM/TFLite baselines is weakly argued, and no code, data, or error bars are provided for the headline results. The memory-footprint claim in the abstract is not directly measured. These gaps prevent the current manuscript from substantiating the strongest claims, but the core idea is promising and the end-to-end evidence is qualitatively consistent with it.

major comments (4)
  1. [§4.2, Figs. 8–10, Eq. (1)] The central premise — that a single R-Tile layout is near-optimal for both forward and backward reductions — is not established for multi-tile tensors. §4.2 asserts tile-invariance from the observation that reduction over a tile loads the same elements regardless of direction, but the only concrete illustration is a single 4×4 MatMul tile (Fig. 9). For a tensor that is more than one tile tall in the vertical dimension, one of the two passes must traverse between tile rows. The inter-tile heuristic in §4.2 places the dimension with more tiles horizontally; the other dimension is necessarily vertical, and Fig. 4 shows column-major/vertical access is the lowest-throughput pattern (normalized baseline 1.0×, versus row-major 1.68×). The paper never measures the cost of this vertical component, never tests a tensor with more than one vertical tile, and never compares R-Tile against the rejecte
  2. [§5.2, Fig. 13] Baseline fairness and reproducibility are insufficiently supported. For TFLite and TVM, which have limited native training support, the paper says the authors 'manually construct the joint forward-and-backward computational graphs' and claims this does not affect runtime because the frameworks' optimizations are 'architecture-agnostic and operate independently of the graph-generation process.' This is a strong assertion with no evidence; manually assembled graphs may miss framework-specific fusion, layout, or memory-planning opportunities, and the claim that graph construction is irrelevant to runtime is not self-evident. Moreover, the end-to-end latency results in Fig. 13 are reported with no error bars, standard deviations, or number of repetitions; only the supplementary microbenchmark for Fig. 4 reports repetitions (10 runs). No code, model graphs, or measurement scripts are provided
  3. [Abstract, §5.4] The abstract claims FBLayout 'significantly improv[es] cache efficiency and reduc[es] memory footprint.' §5.4 and Fig. 15 report global-memory-access counts and cache-miss counts (3.5× and 4.2× reductions, respectively), but no peak-memory or memory-footprint measurement appears anywhere in the evaluation. Cache-miss counts do not directly establish a memory-footprint reduction. Either add a peak-memory comparison (e.g., against MNN's BFC allocator) or soften the claim to 'reduces global memory traffic and cache misses.'
  4. [Table 1, §5.6] Table 1 classifies MatMul, Conv2D, BatchNorm, and LayerNorm as FTD-D operators, and the system overview (Fig. 7) suggests R-Tile is applied to all reduction-type FTD-D operators. However, R-Tile's tile-invariance is demonstrated only for MatMul (Fig. 9). The CNN portability results in §5.6 report end-to-end speedups for MobileNetV2, VGG-19, and ResNet34, but there is no microbenchmark or layout analysis showing how R-Tile handles the reduction conflicts in Conv2D, BatchNorm, or LayerNorm. Without this, the generalization from MatMul to the full FTD-D set is an inference, not a demonstrated property. Adding a per-operator breakdown for at least one non-MatMul FTD-D op would materially strengthen the paper.
minor comments (7)
  1. [§4.2] Typo: 'Figute 4' should be 'Figure 4'.
  2. [Fig. 4] The axes of Fig. 4 are not described in the main text; the x-axis (number of active threads) and normalization to column-major appear only in the supplementary material. Define these in the caption.
  3. [Table 3 / Ref. [67]] Table 3 lists Qwen2.5-1.5B, but reference [67] is 'Qwen3 Technical Report.' Verify the correct checkpoint and citation.
  4. [Table 4] The column 'R-Tile w/o pruning' is not defined in the text. State which components are enabled in this configuration.
  5. [Table 2] The table entries 'FTD-D+Elim' and 'Greedy Search' are not explained in the table or caption. Define the notation.
  6. [Supplementary Material] The paragraph 'Each thread reads different data sections in a streaming manner. The entire data footprint is larger than the L2 cache to rule out the impact of temporal data locality.' is repeated verbatim twice.
  7. [§5.2] The 'Metric' paragraph says three metrics are measured but then lists two (training throughput and cache efficiency). Energy is measured later; clarify the list.

Circularity Check

0 steps flagged

No significant circularity: the central speedup claims are measured against external frameworks, and R-Tile is presented as a design objective rather than a fitted prediction.

full rationale

FBLayout's headline results (2.2–5.7x over MNN, TFLite, TVM) are empirical end-to-end measurements against independent external baselines, not quantities derived from the paper's own fitting procedure. Eq. (1) defines N_cache from benchmarked cache geometry, and Eq. (2) is an explicit design objective (argmax over R-Tile candidates of GFLOPS x MemEfficiency); neither equation is used to 'predict' the measured speedups, so there is no self-definitional reduction. The tile-invariance premise in §4.2 is supported by a concrete MatMul illustration (Fig. 9) and then generalized to FTD-D operators; this is an evidentiary generalization that may be too strong, but it is not circular: the paper does not define tile-invariance in terms of the speedup it later claims. The hardware constants (warp size, cache geometry, access-pattern throughput) are obtained from microbenchmarks and from cited prior work [15], not from the target performance numbers. The only author-overlap citations ([36], [45]–[47], etc.) are used for background context or prior layout-transform overhead motivation, not as a load-bearing uniqueness theorem or as the source of R-Tile's correctness. The skeptical concern that vertical inter-tile traversal is unmeasured and that per-pass layout reuse is not compared is a threat to external validity and completeness, not a circularity of derivation. Overall, the paper's claimed gains stand or fall on the reported experimental comparisons, which are independent of the paper's own definitions.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central claim rests on domain-specific assumptions about mobile texture-cache behavior, the tile-invariance of forward/backward reductions, and two heuristic optimizations (greedy global search and manual baseline-graph construction). The tile sizes and block dimensions are tuned via hardware benchmarks and Eq. 2, which is a design objective, not a circular prediction. No physical invented entities are introduced.

free parameters (3)
  • R-Tile candidate tile dimensions (tile_h, tile_w) = per-tensor/operator selected by arg max in Eq. 2 over benchmarked GFLOPS × MemEfficiency
    The actual tile sizes are not derived from cache geometry alone; they are chosen by evaluating candidates on hardware (§4.2, Eq. 2), so the central layout is tuned rather than predicted.
  • Intra-tile 2D block dimensions for large tiles = selected by 'pre-offline benchmarking' on the target GPU
    When tile_w > C_w, block dimensions are found by profiling, not by a formula (§4.2 Physical Mapping).
  • Heuristic pruning thresholds C1–C3 = warp multiple 32/64; cache-line 64B; padding ≤10%
    These thresholds in Algorithm 1 and §4.2 constrain the tile candidate set; they are hand-set and architecture-specific, so they influence all downstream performance numbers.
axioms (5)
  • domain assumption Mobile GPU texture cache consists of 2D rectangular blocks (wider horizontally), and horizontal/blocked access is faster than strided vertical access.
    Invoked in Fig. 4 and §4.2 intra/inter-tile mapping; based on cited hardware references plus an in-paper microbenchmark. R-Tile's entire benefit depends on this.
  • domain assumption Tile-invariance: forward and backward reductions of an FTD-D tensor touch exactly the same tile elements, differing only in traversal order.
    Stated in §4.2 and Fig. 9 for MatMul and generalized to all reduction ops. If a reduction reads only a subset of a tile, a single R-Tile cannot be optimal for both passes.
  • ad hoc to paper Consecutive FTD-D operators in Transformers have largely overlapping R-Tile candidates; greedy search by input tensor size is near-optimal.
    Assumed in §4.4.2 to justify greedy global layout selection; no exhaustive search or optimality bound is provided.
  • ad hoc to paper Manually constructing TVM/TFLite forward-backward graphs does not affect their runtime.
    Fairness assumption in §5.2 Baselines; the authors argue optimizations are architecture-agnostic, but graph construction can affect fusion and scheduling opportunities.
  • ad hoc to paper Resource contention affects all R-Tile candidates equally, preserving their ranking.
    Claimed in §5.4 R-Tile Efficiency without supporting experiment; if contention changes cache behavior non-uniformly, the selected tile may no longer be optimal.

pith-pipeline@v1.3.0-alltime-deepseek · 24665 in / 11583 out tokens · 104360 ms · 2026-08-02T08:17:52.529920+00:00 · methodology

0 comments
read the original abstract

Transformer-based models have enabled unprecedented capabilities across language, vision, and multimodal tasks. On-device fine-tuning of transformer models offers a privacy-preserving path to personalized AI, yet remains inefficient on mobile GPUs due to severe memory constraints and frequent layout transformations in attention mechanism during training. Existing mobile training frameworks either use unified layouts for forward and backward passes -- leading to fragmented memory access and poor GPU utilization during backpropagation -- or rely on explicit layout conversions, which introduce significant transformation overhead. To overcome this, we propose FBLayout, a layout-aware framework that co-designs tensor organization with mobile GPU platforms. FBLayout introduces: (1) a unified R-Tile layout for multi-dimensional reductions across forward/backward passes; (2) tile-based index transformation to eliminate physical data movement; and (3) activation-guided layout selection to propagate efficient layouts globally. Evaluations on seven transformer models across different mobile phones (including ARM Mali and Qualcomm Adreno GPUs) show that FBLayout achieves 2.2-5.7x speedup over MNN, TFLite, and TVM, while significantly improving cache efficiency and reducing memory footprint, enabling practical on-device large model fine-tuning.

Figures

Figures reproduced from arXiv: 2607.21624 by Chengzhong Xu, Kahou Tam, Li Li, Wei Niu, Xiaomin Ouyang, Yu Bao.

Figure 1
Figure 1. Figure 1: The latency comparison of on-device LLM inference [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Example of FB-layout conflict for a weight matrix. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Texture memory spatial locality. (a) Different 4 [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Performance overhead of reusing forward layouts [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Overhead analysis of explicit layout transforma [PITH_FULL_IMAGE:figures/full_fig_p004_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of conventional memory layout and [PITH_FULL_IMAGE:figures/full_fig_p005_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: R-Tile layout organization: (a) intra-tile memory [PITH_FULL_IMAGE:figures/full_fig_p006_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Comparison between elimination-based layout [PITH_FULL_IMAGE:figures/full_fig_p007_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Workflow of the Tile-based Index Transformation. [PITH_FULL_IMAGE:figures/full_fig_p008_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Training speed comparison between other frameworks and [PITH_FULL_IMAGE:figures/full_fig_p010_13.png] view at source ↗
Figure 15
Figure 15. Figure 15: Analysis of Cache Efficiency. All results are nor [PITH_FULL_IMAGE:figures/full_fig_p010_15.png] view at source ↗
Figure 17
Figure 17. Figure 17: Optimization breakdown with tile-based layout [PITH_FULL_IMAGE:figures/full_fig_p011_17.png] view at source ↗
Figure 20
Figure 20. Figure 20: Portability evalua￾tion on other network archi￾tectures. Device: Snapdragon 8Elite. (a) Adreno 730 (Snapdragon 8Gen1) (b) Mali Immortalis-G925 (MediaTek Dimensity 9400+) [PITH_FULL_IMAGE:figures/full_fig_p012_20.png] view at source ↗
Figure 18
Figure 18. Figure 18: System overhead with different components under [PITH_FULL_IMAGE:figures/full_fig_p012_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 18 linked inside Pith

  1. [1]

    Adreno. [n. d.]. Adreno wikipedia. https://en.wikipedia.org/wiki/Adreno

  2. [2]

    Joshua Ainslie, James Lee-Thorp, Michiel De Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. 2023. Gqa: Training generalized multi-query trans- former models from multi-head checkpoints.arXiv preprint arXiv:2305.13245 (2023)

    Pith/arXiv arXiv 2023

  3. [3]

    Kyle Aubrey and Nick Stam. 2025. HInside NVIDIA Blackwell Ultra: The Chip Powering the AI Factory Era. https://developer.nvidia.com/blog/inside-nvidia- blackwell-ultra-the-chip-powering-the-ai-factory-era/. Accessed: 2026.03

    2025

  4. [4]

    Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. 2021. Crossvit: Cross- attention multi-scale vision transformer for image classification. InProceedings of the IEEE/CVF international conference on computer vision. 357–366

    2021

  5. [5]

    2018.{TVM}: An automated{End-to-End} optimizing compiler for deep learning

    Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. 2018.{TVM}: An automated{End-to-End} optimizing compiler for deep learning. In13th MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom Kahou Tam, Wei Niu, Yu Bao, Xiaomin Ouyang, ChengZhong Xu, and Li Li USENIX Sym...

    2018

  6. [6]

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. 2022. Vision transformer adapter for dense predictions.arXiv preprint arXiv:2205.08534(2022)

    Pith/arXiv arXiv 2022

  7. [7]

    Tri Dao. 2023. Flashattention-2: Faster attention with better parallelism and work partitioning.arXiv preprint arXiv:2307.08691(2023)

    Pith/arXiv arXiv 2023

  8. [8]

    Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 2021. 8-bit optimizers via block-wise quantization.arXiv preprint arXiv:2110.02861(2021)

    Pith/arXiv arXiv 2021

  9. [9]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms.Advances in neural information processing systems36 (2023), 10088–10115

    2023

  10. [10]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). 4171–4186

    2019

  11. [11]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929(2020)

    Pith/arXiv arXiv 2020

  12. [12]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models.arXiv e-prints(2024), arXiv–2407

    2024

  13. [13]

    In Gim and JeongGil Ko. 2022. Memory-efficient DNN training on mobile devices. InProceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services. 464–476

    2022

  14. [14]

    Google. 2025. TensorFlow Lite. https://www.tensorflow.org/lite. Accessed: 2025.9

    2025

  15. [15]

    Jiexiong Guan, Zhenqing Hu, Christos D Antonopoulos, Nikolaos Bellas, Spyros Lalis, Evgenia Smirni, Gang Zhou, Gagan Agrawal, and Bin Ren. 2025. TM- Model: Modeling Texture Memory and Mobile GPU Performance to Accelerate DNN Computations. InProceedings of the 39th ACM International Conference on Supercomputing. 205–220

    2025

  16. [16]

    Ziyad S Hakura and Anoop Gupta. 1997. The design and analysis of a cache architecture for texture mapping. InProceedings of the 24th annual international symposium on Computer architecture. 108–120

    1997

  17. [17]

    Zixu Hao, Jianyu Wei, Tuowei Wang, Minxing Huang, Huiqiang Jiang, Shiqi Jiang, Ting Cao, and Ju Ren. 2025. Scaling llm test-time compute with mobile npu on smartphones.arXiv preprint arXiv:2509.23324(2025)

    arXiv 2025

  18. [18]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition. 770–778

    2016

  19. [19]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. InInternational conference on machine learning. PMLR, 2790–2799

    2019

  20. [20]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models.ICLR1, 2 (2022), 3

    2022

  21. [21]

    Jinwu Hu, Zhitian Zhang, Guohao Chen, Xutao Wen, Chao Shuai, Wei Luo, Bin Xiao, Yuanqing Li, and Mingkui Tan. 2025. Test-time learning for large language models.arXiv preprint arXiv:2505.20633(2025)

    Pith/arXiv arXiv 2025

  22. [22]

    Kai Huang, Boyuan Yang, and Wei Gao. 2023. Elastictrainer: Speeding up on- device training with runtime elastic tensor selection. InProceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services. 56–69

    2023

  23. [23]

    Xiaotang Jiang, Huan Wang, Yiliu Chen, Ziqi Wu, Lichuan Wang, Bin Zou, Yafeng Yang, Zongyang Cui, Yu Cai, Tianhang Yu, et al. 2020. MNN: A universal and efficient inference engine.Proceedings of Machine Learning and Systems2 (2020), 1–13

    2020

  24. [24]

    Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El Saddik, and Eric Xing. 2024. Efficient test-time adaptation of vision-language models. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14162–14171

    2024

  25. [25]

    Liang Li, Xingke Yang, Wen Wu, Hao Wang, Tomoaki Ohtsuki, Xin Fu, Miao Pan, and Xuemin Shen. 2025. MobiLLM: Enabling LLM Fine-Tuning on the Mobile Device via Server Assisted Side Tuning.arXiv preprint arXiv:2502.20421(2025)

    Pith/arXiv arXiv 2025

  26. [26]

    Xiang Li, Zhenyan Lu, Dongqi Cai, Xiao Ma, and Mengwei Xu. 2024. Large language models on mobile devices: Measurements, analysis, and insights. In Proceedings of the Workshop on Edge and Mobile Foundation Models. 1–6

    2024

  27. [27]

    Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation.arXiv preprint arXiv:2101.00190(2021)

    Pith/arXiv arXiv 2021

  28. [28]

    Rendong Liang, Ting Cao, Jicheng Wen, Manni Wang, Yang Wang, Jianhua Zou, and Yunxin Liu. 2022. Romou: Rapidly generate high-performance tensor kernels for mobile gpus. InProceedings of the 28th Annual International Conference on Mobile Computing And Networking. 487–500

    2022

  29. [29]

    Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Hao Zhang, Yong Liu, Chuhan Wu, Xiangyang Li, Chenxu Zhu, et al . 2025. How can recommender systems benefit from large language models: A survey.ACM Transactions on Information Systems43, 2 (2025), 1–47

    2025

  30. [30]

    Renyuan Liu, Yuyang Leng, Kaiyan Liu, Shaohan Hu, Peijun Zhao, Heechul Yun, Shuochao Yao, et al. 2025. DAF: An Efficient End-to-End Dynamic Activation Framework for on-Device DNN Training.arXiv preprint arXiv:2507.07149(2025)

    Pith/arXiv arXiv 2025

  31. [31]

    LLVM. [n. d.]. LoopStrengthReduce. https://llvm.org/doxygen/ LoopStrengthReduce_8cpp_source.html

  32. [32]

    Eric Mahurin. 2023. Qualocmm®Hexagon™NPU . In2023 IEEE Hot Chips 35 Symposium (HCS). IEEE Computer Society, Los Alamitos, CA, USA, 1–19. doi:10.1109/HCS59251.2023.10254715

    arXiv 2023

  33. [33]

    MNN. [n. d.]. MNN GitHub Repository: BFC Allocator. https://github.com/ alibaba/MNN/blob/2.3.0/source/core/BufferAllocator.cpp

  34. [34]

    Inc Monsoon Solutions. 2023. High voltage power moniter. https://www.msoon. com/. Accessed: 2026.03

    2023

  35. [35]

    Wei Niu, Jiexiong Guan, Yanzhi Wang, Gagan Agrawal, and Bin Ren. 2021. Dnn- fusion: accelerating deep neural networks execution with advanced operator fusion. InProceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation. 883–898

    2021

  36. [36]

    Wei Niu, Md Musfiqur Rahman Sanim, Zhihao Shu, Jiexiong Guan, Xipeng Shen, Miao Yin, Gagan Agrawal, and Bin Ren. 2024. Smartmem: Layout transformation elimination and adaptation for efficient dnn execution on mobile. InProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3. 916–931

    2024

  37. [37]

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology. 1–22

    2023

  38. [38]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library.Advances in neural information processing systems32 (2019)

    2019

  39. [39]

    Qualcomm. [n. d.]. Snapdragon profiler. https://developer.qualcomm.com/ software/snapdragon-profiler

  40. [40]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. InInternational conference on machine learning. PMLR, 28492–28518

    2023

  41. [41]

    Josh Render. [n. d.]. Android phones are now selling with a crazy 24GB of RAM — buy or skip? https://www.tomsguide.com/phones/android-phones/android- phones-are-now-selling-with-a-crazy-24gb-of-ram-buy-or-skip. Accessed: 2025.9

    2025

  42. [42]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis With Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10684–10695

    2022

  43. [43]

    Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang- Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4510–4520

    2018

  44. [44]

    Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556(2014)

    Pith/arXiv arXiv 2014

  45. [45]

    Kahou Tam, Li Li, Bo Han, Chengzhong Xu, and Huazhu Fu. 2023. Federated noisy client learning.IEEE transactions on neural networks and learning systems 36, 1 (2023), 1799–1812

    2023

  46. [46]

    Kahou Tam, Li Li, Yan Zhao, and Chengzhong Xu. 2023. Fedcoop: Cooperative federated learning for noisy labels. InECAI 2023. IOS Press, 2298–2306

    2023

  47. [47]

    Kahou Tam, Chunlin Tian, Li Li, Haikai Zhao, and ChengZhong Xu. 2024. Fed- Hybrid: Breaking the Memory Wall of Federated Learning via Hybrid Tensor Management. InProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems. 394–408

    2024

  48. [48]

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. 2024. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118(2024)

    Pith/arXiv arXiv 2024

  49. [49]

    Chunlin Tian, Li Li, Zhan Shi, Jun Wang, and ChengZhong Xu. 2022. Harmony: Heterogeneity-aware hierarchical management for federated learning system. In2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 631–645

    2022

  50. [50]

    Chunlin Tian, Xinpeng Qin, Kahou Tam, Li Li, Zijian Wang, Yuanzhe Zhao, Minglei Zhang, and Chengzhong Xu. 2025. {CLONE}: Customizing{LLMs} for Efficient{Latency-Aware} Inference at the Edge. In2025 USENIX Annual Technical Conference (USENIX ATC 25). 563–585

    2025

  51. [51]

    Chunlin Tian, Zhan Shi, Zhijiang Guo, Li Li, and Chengzhong Xu. 2024. Hydralora: An asymmetric lora architecture for efficient fine-tuning.Advances in Neural Information Processing Systems37 (2024), 9565–9584. FBLayout: Optimizing Memory Layout for Efficient LLM Finetuning on Mobile GPUs MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom

    2024

  52. [52]

    Chunlin Tian, Kahou Tam, Yebo Wu, Shuaihang Zhong, Li Li, Nicholas D Lane, and ChengZhong Xu. 2026. Floe: Federated Specialization for Real-Time LLM– SLM Inference.IEEE Transactions on Parallel and Distributed Systems(2026)

    2026

  53. [53]

    Alicia Tsai, Adam Kraft, Long Jin, Chenwei Cai, Anahita Hosseini, Taibai Xu, Zemin Zhang, Lichan Hong, Ed H Chi, and Xinyang Yi. 2024. Leveraging llm reasoning enhances personalized recommender systems. InFindings of the Asso- ciation for Computational Linguistics: ACL 2024. 13176–13188

    2024

  54. [54]

    Yu-Min Tseng, Yu-Chao Huang, Teng-Yun Hsiao, Wei-Lin Chen, Chao-Wei Huang, Yu Meng, and Yun-Nung Chen. 2024. Two tales of persona in llms: A survey of role-playing and personalization.arXiv preprint arXiv:2406.01171(2024)

    Pith/arXiv arXiv 2024

  55. [55]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.Advances in neural information processing systems30 (2017)

    2017

  56. [56]

    Qipeng Wang, Mengwei Xu, Chao Jin, Xinran Dong, Jinliang Yuan, Xin Jin, Gang Huang, Yunxin Liu, and Xuanzhe Liu. 2022. Melon: Breaking the memory wall for resource-efficient on-device machine learning. InProceedings of ACM MobiSys. 450–463

    2022

  57. [57]

    wiki. 2025. Texture mapping unit. https://en.wikipedia.org/wiki/Texture_ mapping_unit. Accessed: 2025.9

    2025

  58. [58]

    Yebo Wu, Jingguang Li, Zhijiang Guo, and Li Li. 2025. Elastic Mixture of Rank- Wise Experts for Knowledge Reuse in Federated Fine-Tuning.arXiv preprint arXiv:2512.00902(2025)

    arXiv 2025

  59. [59]

    Yebo Wu, Jingguang Li, Chunlin Tian, Zhijiang Guo, and Li Li. 2025. Memory- Efficient Federated Fine-Tuning of Large Language Models via Layer Pruning. arXiv preprint arXiv:2508.17209(2025)

    Pith/arXiv arXiv 2025

  60. [60]

    Yebo Wu, Jingguang Li, Chunlin Tian, Kahou Tam, Zhijiang Guo, and Li Li. 2026. Beyond End-to-End: Dynamic Chain Optimization for Private LLM Adaptation on the Edge.arXiv preprint arXiv:2604.06819(2026)

    Pith/arXiv arXiv 2026

  61. [61]

    Yebo Wu, Jingguang Li, Chunlin Tian, Kahou Tam, Li Li, and Chengzhong Xu

  62. [62]

    Yebo Wu, Li Li, and Cheng-zhong Xu. 2025. Breaking the memory wall for heterogeneous federated learning via progressive training. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1. 1623–1632

    2025

  63. [63]

    Yebo Wu, Chunlin Tian, Jingguang Li, He Sun, Kahou Tam, Zhanting Zhou, Haicheng Liao, Jing Xiong, Zhijiang Guo, Li Li, et al. 2025. A survey on federated fine-tuning of large language models.arXiv preprint arXiv:2503.12016(2025)

    arXiv 2025

  64. [64]

    Daliang Xu, Mengwei Xu, Qipeng Wang, Shangguang Wang, Yun Ma, Kang Huang, Gang Huang, Xin Jin, and Xuanzhe Liu. 2022. Mandheling: Mixed- precision on-device dnn training with dsp offloading. InProceedings of the 28th Annual International Conference on Mobile Computing And Networking. 214–227

    2022

  65. [65]

    Jiajun Xu, Zhiyuan Li, Wei Chen, Qun Wang, Xin Gao, Qi Cai, and Ziyuan Ling

  66. [66]

    Mengwei Xu, Wangsong Yin, Dongqi Cai, Rongjie Yi, Daliang Xu, Qipeng Wang, Bingyang Wu, Yihao Zhao, Chen Yang, Shihe Wang, et al . 2024. A survey of resource-efficient llm and multimodal foundation models.arXiv preprint arXiv:2401.08092(2024)

    Pith/arXiv arXiv 2024

  67. [67]

    On-device language models: A comprehensive review.arXiv preprint arXiv:2409.00088(2024)

    Pith/arXiv arXiv 2024

  68. [68]

    Zhehao Zhang, Ryan A Rossi, Branislav Kveton, Yijia Shao, Diyi Yang, Hamed Zamani, Franck Dernoncourt, Joe Barrow, Tong Yu, Sungchul Kim, et al. 2024. Per- sonalization of large language models: A survey.arXiv preprint arXiv:2411.00027 (2024)

    Pith/arXiv arXiv 2024

  69. [69]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

    Pith/arXiv arXiv 2025

  70. [70]

    Keren Zhou, Mario Lezcano-Casado, Adam P Goucher, Akhmed Rakhmati, Jeff Niu, Justin Lebar, Pawel Szczerbuk, Peter Bell, Phil Tillet, Thomas Raoux, et al

  71. [71]

    Zihuai Zhao, Wenqi Fan, Jiatong Li, Yunqing Liu, Xiaowei Mei, Yiqi Wang, Zhen Wen, Fei Wang, Xiangyu Zhao, Jiliang Tang, et al. 2024. Recommender systems in the era of large language models (llms).IEEE Transactions on Knowledge and Data Engineering36, 11 (2024), 6889–6907

    2024

  72. [74]

    Mobile GPU texture caches are organized as rectangu- lar blocks spanning wider horizontally than vertically

    Regarding 2D spatial locality alignment: Block access pat- terns (Block-4, Block-8) achieve more consistent performance across thread counts because they align with the texture cache’s 2D block structure. Mobile GPU texture caches are organized as rectangu- lar blocks spanning wider horizontally than vertically. Block pat- terns access data in compact 2D ...

  73. [75]

    This occurs because row-major traversal crosses cache line boundaries when threads access data exceeding the cache line width

    Regarding cache line boundary effects: Row-major perfor- mance degrades sharply as thread count increases (1 .68× at 8 threads and1.21× at 64 threads, a 28% drop), while block patterns remain more stable (Block-8:1.54× and 1.36%, only 12% drop). This occurs because row-major traversal crosses cache line boundaries when threads access data exceeding the ca...

    2026

  74. [2024]

    Bridging Memory Gaps: Scaling Federated Learning for Heterogeneous Clients.arXiv preprint arXiv:2408.10826(2024)

    arXiv 2024

  75. [2026]

    InProceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1

    Linear Layouts: Robust Code Generation of Efficient Tensor Computation Using F_2. InProceedings of the 31st ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1. 132–146. Supplementary Material Texture spatial data locality We investigate the spatial data locality characteristics of mobile GPU tex...