Dataset distillation creates a tiny synthetic training set that, when used with a fixed network initialization, produces models whose performance approximates that of models trained on the full original dataset.
hub
Proceedings of the IEEE , volume=
22 Pith papers cite this work. Polarity classification is still indexing.
22
Pith papers citing it
hub tools
citation-role summary
background 2
citation-polarity summary
roles
background 2polarities
background 2representative citing papers
LOSCAR-SGD combines local updates, sparse model averaging, and communication-computation overlap with a delay-corrected merge rule, providing convergence rates for smooth non-convex objectives under worker heterogeneity.
Proposes pointwise Riemannian Dimension from feature eigenvalues to derive tighter, representation-aware generalization bounds for deep networks in the nonlinear regime.
Ringmaster LMO extends delay-thresholding from ASGD to LMO-based momentum updates, providing convergence guarantees under (L0, L1)-smoothness and time-complexity bounds that recover optimal rates in the Euclidean case.
DRATS derives a minimax objective from a feasibility formulation of MTRL to adaptively sample tasks with the largest return gaps, leading to better worst-task performance on MetaWorld benchmarks.
FiBeR adds a closed-form filter-aware correction A(ω)σ_w² to the second-moment term for temporally filtered DP gradients, improving adaptive optimization performance.
For any distribution of pure n-qubit states, a QAE with k encoder ancillas and n decoder ancillas achieves the optimal average fidelity among all CPTP encoder-decoder pairs, with the encoder threshold proven sharp.
Temperature scaling of density-matrix eigenvalues from LLM semantic embeddings optimizes proper-score calibration and corrects systematic overconfidence so entropy equals risk.
A Graph Tsetlin Machine with hypervectorized macro multigraphs and message-passing clauses anticipates four USD/JPY regimes, reaching 70.7% overall OOS accuracy and beating reduced-graph and CoTM baselines on stagnant and choppy classes.
SGD is reformulated via a master equation from discrete updates, producing a discrete Fokker-Planck equation that predicts non-stationary variance growth proportional to learning rate in flat Hessian directions.
Provides sufficient conditions for successful distillation of combinatorial optimization tasks into DP-aligned graph neural networks under the linear representation hypothesis for the source model.
A new LLM-based system generates and validates runnable modular MCMC samplers directly from natural-language Bayesian model descriptions, reporting success on 120 of 132 benchmark models.
CoCD converts stale gradients into stable descent directions through coherent coordinate updates equivalent to BCCD with warm starts, enabling O(1) queries per step plus implicit landscape smoothing from larger finite-difference steps.
LIVEditor-14B applies a new sparse attention method (ISA) that prunes context and uses query-sharpness routing to cut attention latency ~60% with no loss in editing quality on standard benchmarks.
SPIN lets weak LLMs become strong by self-generating training data from previous model versions and training to prefer human-annotated responses over its own outputs, outperforming DPO even with extra GPT-4 data on benchmarks.
A gradient-similarity complexity measure that generalizes polynomial degree, kernel length scale, neighbor count, tree splits, and forest size while offering insights into double descent.
CMRU restores gradient flow in BMRU via cumulative state updates with skip-connections through time, yielding better convergence and benchmark performance while retaining quantized persistent memory.
Rennala MVR improves time complexity over Rennala SGD for smooth nonconvex stochastic optimization in heterogeneous parallel systems under a mean-squared smoothness assumption.
FedAvg DeepSurv across Lifelines (n=148k, self-report) and Rotterdam Study (n=10k, linked outcomes) raised C-statistics from 0.728 to 0.739 and 0.783 to 0.787 versus local training.
Random feature selection outperforms many state-of-the-art unsupervised feature selection methods on standard performance and efficiency metrics.
Step-Video-T2V describes a 30B-parameter text-to-video model with custom Video-VAE, 3D DiT, flow matching, and Video-DPO that claims state-of-the-art results on a new internal benchmark.
citing papers explorer
-
Dataset Distillation
Dataset distillation creates a tiny synthetic training set that, when used with a fixed network initialization, produces models whose performance approximates that of models trained on the full original dataset.
-
LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
LOSCAR-SGD combines local updates, sparse model averaging, and communication-computation overlap with a delay-corrected merge rule, providing convergence rates for smooth non-convex objectives under worker heterogeneity.
-
Pointwise Generalization in Deep Neural Networks
Proposes pointwise Riemannian Dimension from feature eigenvalues to derive tighter, representation-aware generalization bounds for deep networks in the nonlinear regime.
-
Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method
Ringmaster LMO extends delay-thresholding from ASGD to LMO-based momentum updates, providing convergence guarantees under (L0, L1)-smoothness and time-complexity bounds that recover optimal rates in the Euclidean case.
-
Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Sampling
DRATS derives a minimax objective from a feasibility formulation of MTRL to adaptively sample tasks with the largest return gaps, leading to better worst-task performance on MetaWorld benchmarks.
-
FIBER: A Differentially Private Optimizer with Filter-Aware Innovation Bias Correction
FiBeR adds a closed-form filter-aware correction A(ω)σ_w² to the second-moment term for temporally filtered DP gradients, improving adaptive optimization performance.
-
Toward the Goldilocks blind compression of quantum states
For any distribution of pure n-qubit states, a QAE with k encoder ancillas and n decoder ancillas achieves the optimal average fidelity among all CPTP encoder-decoder pairs, with the encoder threshold proven sharp.
-
Eigenvalue Calibration for Semantic Embeddings of Large Language Models
Temperature scaling of density-matrix eigenvalues from LLM semantic embeddings optimizes proper-score calibration and corrects systematic overconfidence so entropy equals risk.
-
Macroeconomic Message Passing for Anticipating Foreign Exchange Regime Changes: A Deep Logical Learning Approach using Graph Tsetlin Machines
A Graph Tsetlin Machine with hypervectorized macro multigraphs and message-passing clauses anticipates four USD/JPY regimes, reaching 70.7% overall OOS accuracy and beating reduced-graph and CoTM baselines on stagnant and choppy classes.
-
Why SGD is not Brownian Motion: A New Perspective on Stochastic Dynamics
SGD is reformulated via a master equation from discrete updates, producing a discrete Fokker-Planck equation that predicts non-stationary variance growth proportional to learning rate in flat Hessian directions.
-
Towards Distillation Guarantees under Algorithmic Alignment for Combinatorial Optimization
Provides sufficient conditions for successful distillation of combinatorial optimization tasks into DP-aligned graph neural networks under the linear representation hypothesis for the source model.
-
AI4BayesCode: From Natural Language Descriptions to Validated Modular Stateful Bayesian Samplers
A new LLM-based system generates and validates runnable modular MCMC samplers directly from natural-language Bayesian model descriptions, reporting success on 120 of 132 benchmark models.
-
Turning Stale Gradients into Stable Gradients: Coherent Coordinate Descent with Implicit Landscape Smoothing for Lightweight Zeroth-Order Optimization
CoCD converts stale gradients into stable descent directions through coherent coordinate updates equivalent to BCCD with warm starts, enabling O(1) queries per step plus implicit landscape smoothing from larger finite-difference steps.
-
LIVEditor-14B: Lightning Unified Video Editing via In-Context Sparse Attention
LIVEditor-14B applies a new sparse attention method (ISA) that prunes context and uses query-sharpness routing to cut attention latency ~60% with no loss in editing quality on standard benchmarks.
-
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
SPIN lets weak LLMs become strong by self-generating training data from previous model versions and training to prefer human-annotated responses over its own outputs, outperforming DPO even with extra GPT-4 data on benchmarks.
-
A Rigorous, Tractable Measure of Model Complexity
A gradient-similarity complexity measure that generalizes polynomial degree, kernel length scale, neighbor count, tree splits, and forest size while offering insights into double descent.
-
Improving the Performance and Learning Stability of Parallelizable RNNs Designed for Ultra-Low Power Applications
CMRU restores gradient flow in BMRU via cumulative state updates with skip-connections through time, yielding better convergence and benchmark performance while retaining quantized persistent memory.
-
Rennala MVR: Improved Time Complexity for Parallel Stochastic Optimization via Momentum-Based Variance Reduction
Rennala MVR improves time complexity over Rennala SGD for smooth nonconvex stochastic optimization in heterogeneous parallel systems under a mean-squared smoothness assumption.
-
Federated Deep Learning for Privacy-Preserving Cardiovascular Disease Risk Prediction
FedAvg DeepSurv across Lifelines (n=148k, self-report) and Rotterdam Study (n=10k, linked outcomes) raised C-statistics from 0.728 to 0.739 and 0.783 to 0.787 versus local training.
-
Worse than Random: The Importance of a Baseline for Unsupervised Feature Selection
Random feature selection outperforms many state-of-the-art unsupervised feature selection methods on standard performance and efficiency metrics.
-
Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model
Step-Video-T2V describes a 30B-parameter text-to-video model with custom Video-VAE, 3D DiT, flow matching, and Video-DPO that claims state-of-the-art results on a new internal benchmark.
- Convergence of difference inclusions: a diameter criterion and step-size conditions