Pith. sign in

REVIEW 2 cited by

Enhancing Training Efficiency Using Packing with Flash Attention

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.09105 v6 pith:FVEV65XE submitted 2024-07-12 cs.LG cs.AI

Achintya Kundu , Rhui Dih Lee , Laura Wynter , Raghu Kiran Ganti , Mayank Mishra This is my paper · ORCID
classification cs.LGcs.AI
keywords trainingpackingbatchexamplesfacehuggingpaddingresources
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Padding is often used in tuning LLM models by adding special tokens to shorter training examples to match the length of the longest sequence in each batch. While this ensures uniformity for batch processing, it introduces inefficiencies by including irrelevant padding tokens in the computation and wastes GPU resources. Hugging Face SFT trainer has always offered the option to use packing to combine multiple training examples, allowing for maximal utilization of GPU resources. However, up till now, it did not offer proper masking of each packed training example. This capability has been added to Hugging Face Transformers 4.44. We analyse this new feature and show the benefits across different variations of packing.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation

    cs.CL 2026-06 unverdicted novelty 7.0 of 10

    RLSR trains source rewriters via RL with translation-quality improvement as the reward, outperforming prompt baselines at 4B scale while matching larger models.

  2. LobRA: Multi-tenant Fine-tuning over Heterogeneous Data

    cs.DC 2025-09 accept novelty 6.0 of 10

    LobRA reduces GPU seconds for multi-tenant LoRA fine-tuning by 45.03%-60.67% through heterogeneous FT replicas and per-step workload-balanced dispatching.

Pith tools