Pith. sign in

REVIEW 5 cited by

InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2303.04947 v2 pith:ZYGU6ZW6 submitted 2023-03-08 cs.CV

Ziheng Qin , Kai Wang , Zangwei Zheng , Jianyang Gu , Xiangyu Peng , Zhaopan Xu , Daquan Zhou , Lei Shang
show 3 more authors
Baigui Sun Xuansong Xie Yang You
This is my paper
classification cs.CV
keywords infobatchcostdatalosslesstraininglesspruningsamples
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Data pruning aims to obtain lossless performances with less overall cost. A common approach is to filter out samples that make less contribution to the training. This could lead to gradient expectation bias compared to the original data. To solve this problem, we propose \textbf{InfoBatch}, a novel framework aiming to achieve lossless training acceleration by unbiased dynamic data pruning. Specifically, InfoBatch randomly prunes a portion of less informative samples based on the loss distribution and rescales the gradients of the remaining samples to approximate the original gradient. As a plug-and-play and architecture-agnostic framework, InfoBatch consistently obtains lossless training results on classification, semantic segmentation, vision pertaining, and instruction fine-tuning tasks. On CIFAR10/100, ImageNet-1K, and ADE20K, InfoBatch losslessly saves 40\% overall cost. For pertaining MAE and diffusion model, InfoBatch can respectively save 24.8\% and 27\% cost. For LLaMA instruction fine-tuning, InfoBatch is also able to save 20\% cost and is compatible with coreset selection methods. The code is publicly available at \href{https://github.com/henryqin1997/InfoBatch}{github.com/NUS-HPC-AI-Lab/InfoBatch}.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OPERA: Online Data Pruning for Efficient Retrieval Model Adaptation

    cs.IR 2026-03 conditional novelty 6.0 of 10

    Dynamic hierarchical data pruning improves NDCG@10 and Recall@20 for dense retrievers while reaching full performance in half the iterations.

  2. Differential-informed Sample Selection Accelerates Multimodal Contrastive Learning

    cs.CV 2025-07 conditional novelty 6.0 of 10

    DISSect selects training samples for multimodal contrastive learning by ranking the difference between historical and current model similarity scores, matching full-data performance with 70% fewer samples.

  3. Learning Faster without Deeper Networks: A*-Inspired Batch Selection for Efficient CNN Training

    cs.LG 2026-07 conditional novelty 5.0 of 10

    A*-inspired mini-batch selection with a loss-based difficulty score and reuse penalty improves CNN accuracy on all twelve MedMNIST-2D tasks and outperforms ResNet baselines on six.

  4. RL-Selector: Reinforcement Learning-Guided Data Selection via Redundancy Assessment

    cs.LG 2025-06 conditional novelty 5.0 of 10

    RL-Selector uses an A2C reinforcement learning agent, rewarded by an epsilon-sample cover score, to pick training subsets that improve accuracy and cut training cost.

  5. Multimodal-Guided Dynamic Dataset Pruning for Robust and Efficient Data-Centric Learning

    cs.LG 2025-07 conditional novelty 4.0 of 10

    A dynamic pruning method scores each sample by combining task loss with CLIP image-text similarity and selects samples near the median score each epoch.

Pith tools