DivIn samples initial noise from a guidance potential posterior via Langevin dynamics to improve diversity in class-to-image and text-to-image generation.
hub
Accelerating deep learning by focusing on the biggest losers
11 Pith papers cite this work. Polarity classification is still indexing.
11
Pith papers citing it
abstract
This paper introduces Selective-Backprop, a technique that accelerates the training of deep neural networks (DNNs) by prioritizing examples with high loss at each iteration. Selective-Backprop uses the output of a training example's forward pass to decide whether to use that example to compute gradients and update parameters, or to skip immediately to the next example. By reducing the number of computationally-expensive backpropagation steps performed, Selective-Backprop accelerates training. Evaluation on CIFAR10, CIFAR100, and SVHN, across a variety of modern image models, shows that Selective-Backprop converges to target error rates up to 3.5x faster than with standard SGD and between 1.02--1.8x faster than a state-of-the-art importance sampling approach. Further acceleration of 26% can be achieved by using stale forward pass results for selection, thus also skipping forward passes of low priority examples.
hub tools
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
DR-IS selects low-contamination subsets via bounded rank-disagreement in proxy ensembles under an ε-contamination model, with O(√(log(N/δ)/K)) concentration rates that certify separation when the expectation gap Δ' is positive.
Online SFT data selection acts as an implicit preference model, shifting refusal rates, verbosity, and sycophancy in directions predictable from the selected data's attribute mixture.
K-ABENA combines threshold-based sample exclusion with Horvitz-Thompson inverse-probability reweighting to achieve unbiased gradient estimation while saving 28-54% of per-epoch backward-pass compute.
Dataset pruning is unified as a maximum-weight clique problem on a sample graph, solved greedily with an approximation guarantee and reported >40% ImageNet training-time cut without accuracy loss.
A task-aware policy learned via reinforcement learning allocates high-resolution pixels on dual-stream sensors in real time, outperforming fixed or non-predictive baselines under tight pixel budgets in both simulation and 200 MP hardware tests.
Short-term data selectors in multi-stage LLM fine-tuning can slow future learning and increase forgetting, formalized as myopic selection with a proposed LHAS objective to address it.
Selective-pivot-token training plus layer-averaging-with-recurrence reportedly gives 2.5x parameter compression on a small Korean LLM, but the efficiency claim lacks its decisive controls and the abstract advertises results absent from the body.
RCAP introduces class-aware probabilistic pruning that uses closed-form per-class fractions updated by loss and high-loss sampling to preserve worst-group accuracy at high pruning rates.
A curriculum sampling questions with high variance in success rate improves reinforcement learning performance for LLM reasoning tasks.
Step-Video-T2V describes a 30B-parameter text-to-video model with custom Video-VAE, 3D DiT, flow matching, and Video-DPO that claims state-of-the-art results on a new internal benchmark.
citing papers explorer
-
Initialization is Half the Battle: Generating Diverse Images from a Guidance Potential Posterior
DivIn samples initial noise from a guidance potential posterior via Langevin dynamics to improve diversity in class-to-image and text-to-image generation.
-
Disagreement-Regularized Importance Sampling for Adversarial Label Corruption
DR-IS selects low-contamination subsets via bounded rank-disagreement in proxy ensembles under an ε-contamination model, with O(√(log(N/δ)/K)) concentration rates that certify separation when the expectation gap Δ' is positive.
-
Online Data Selection Is Implicit Alignment
Online SFT data selection acts as an implicit preference model, shifting refusal rates, verbosity, and sycophancy in directions predictable from the selected data's attribute mixture.
-
K-ABENA: K-Adaptive Backpropagation with Error-based N-exclusion Algorithm : (Compensated Loss-Based Sample Exclusion with Unbiased Gradient Estimation)
K-ABENA combines threshold-based sample exclusion with Horvitz-Thompson inverse-probability reweighting to achieve unbiased gradient estimation while saving 28-54% of per-epoch backward-pass compute.
-
Selecting Samples on Graphs: A Unified Dataset Pruning Framework for Lossless Training Acceleration
Dataset pruning is unified as a maximum-weight clique problem on a sample graph, solved greedily with an approximation guarantee and reported >40% ImageNet training-time cut without accuracy loss.
-
Policy-based Foveated Imaging and Perception
A task-aware policy learned via reinforcement learning allocates high-resolution pixels on dual-stream sensors in real time, outperforming fixed or non-predictive baselines under tight pixel budgets in both simulation and 200 MP hardware tests.
-
The Long-Term Effects of Data Selection in LLM Fine-Tuning
Short-term data selectors in multi-stage LLM fine-tuning can slow future learning and increase forgetting, formalized as myopic selection with a proposed LHAS objective to address it.
-
CHERRY: Compressed Hierarchical Experts with Recurrent Representational Yield
Selective-pivot-token training plus layer-averaging-with-recurrence reportedly gives 2.5x parameter compression on a small Korean LLM, but the efficiency claim lacks its decisive controls and the abstract advertises results absent from the body.
-
RCAP: Robust, Class-Aware, Probabilistic Dynamic Dataset Pruning
RCAP introduces class-aware probabilistic pruning that uses closed-form per-class fractions updated by loss and high-loss sampling to preserve worst-group accuracy at high pruning rates.
-
Learning to Reason at the Frontier of Learnability
A curriculum sampling questions with high variance in success rate improves reinforcement learning performance for LLM reasoning tasks.
-
Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model
Step-Video-T2V describes a 30B-parameter text-to-video model with custom Video-VAE, 3D DiT, flow matching, and Video-DPO that claims state-of-the-art results on a new internal benchmark.