Pith. sign in

REVIEW 2 major objections 2 minor 63 references

LaPrune: Controllable Differentiable Sparsity at Million Scale

T0 review · 2 major / 2 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LaPrune introduces a differentiable top-k layer in which a single normalized hardness parameter controls mask softness while the selection budget stays exactly fixed.

desk verdict Useful exact-budget soft top-k layer with a clean hardness knob, but the central theorem overclaims: ties at the selection boundary break the every-γ guarantee. read the letter →

arxiv 2608.04057 v1 pith:HP54DCX6 submitted 2026-08-04 cs.LG cs.AI

Jakub Antczak , Joanna Wojciechowicz , {L}ukasz Struski , Jacek Tabor This is my paper · ORCID
classification cs.LGcs.AI
keywords top-kselectiondifferentiablesparsityexactbudgetnormalizedsecondmomentLaplaceCDFmaskimplicitdifferentiationscaleinvariancesparseautoencoders
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LaPrune is a differentiable layer for top-k selection whose central claim is that the budget and the softness of a mask can be controlled independently and exactly. It adds a normalized second-moment constraint to the exact-budget Laplace-CDF mask, and proves that every hardness value $\gamma\in[0,1]$ determines a unique temperature, so the mask interpolates monotonically from an equal-mass allocation to the hard top-k indicator without ever changing the selected mass $k$. If the construction is right, annealed schedules that gradually harden a mask during training become well-defined objects, the hardness parameter keeps its meaning when scores are rescaled, and deployment can use an exact hard top-k mask. The paper supports this with a population model of mask saturation, a near-binary limiting law, a distribution-free lower bound on near-zero entries, and a million-scale implementation that maintains the budget to numerical tolerance.

What carries the argument

The load-bearing object is the normalized second moment of the mask, $\gamma=(\sum_i p_i^2-ak)/(k-ak)$, used as a scale-free hardness coordinate that is constrained alongside the exact budget $\sum_i p_i=k$, where $a=k/n$. The mask is the Laplace-CDF map $p_i=\sigma((r_i-b)/t)$, with the barrier $b$ enforcing the budget and the temperature $t$ solved for rather than chosen. Strict monotonicity of $M_2(t)$—established from the positivity of a weighted covariance between where a score sits and what mask value it receives—makes $\gamma$ a well-defined coordinate, and the 2x2 constraint system $F_1=\sum_i p_i-k$, $F_2=\sum_i p_i^2-\beta k$ with its Jacobian in $(b,\log t)$ is the mechanism that realizes it and yields implicit gradients. The same Jacobian appears in the backward pass, so the solver trajectory never needs to be stored.

What would settle it

On a score vector with all entries distinct, sweep $t$ over a fine grid and plot $M_2(t)=\sum_i\sigma((r_i-b(t))/t)^2$ together with the target levels for many $\gamma$; the theorem predicts a strictly decreasing curve crossing each level exactly once, so a non-monotone segment, a repeated level, or a converged solve at $\gamma\in(0,1)$ that cannot reach both constraints to tolerance would refute the exact-budget uniqueness claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that mask hardness can be made a constraint rather than a heuristic dial. For the Laplace-CDF mask $p_i(t)=\sigma((r_i-b(t))/t)$, where $b(t)$ is chosen so that $\sum_i p_i=k$, the second moment $M_2(t)=\sum_i p_i(t)^2$ is shown to be continuous and strictly decreasing in $t$ for every nonconstant score vector. With pairwise distinct scores it runs from $ak$ at $t\to\infty$ to $k$ as $t\downarrow0$, so the normalized hardness $\gamma=(M_2-ak)/(k-ak)$ has a unique temperature $t_\gamma$ in the interior, with $\gamma=0$ and $\gamma=1$ defined directly as the uniform and hard top-k masks. LaPrune realizes this by solving the 2x2 system $\sum_i p_i=k$, $\sum_i p_i^2=\beta k$ with $\beta=a+(1-a)\gamma$ jointly for $(b,\log t)$ and differentiating the converged solution implicitly, which keeps the backward pass independent of the solver path. The accompanying theory predicts the saturated fraction under a population score model, gives the near-binary law that the non-saturated fraction falls as $\frac{8}{3}|\ln(2\varepsilon)|a(1-a)(1-\gamma)$, and proves the tight worst-case floor $\mathrm{meas}\{p_i<\varepsilon\}\ge(1-a)\max(0,1-a(1-\gamma)/\varepsilon)$ for every score distribution.

Load-bearing premise

The whole construction depends on the scores being all different and on the two requirements—keep the total selected mass fixed and keep the second moment fixed—remaining genuinely independent at the solved temperature; that independence is only checked numerically and is hardest to maintain for the most binary masks, so tied scores or a degenerating solver would break the guarantee.

Editorial extensions

If this is right

  • Annealing becomes a formal schedule: raising $\gamma$ from 0 to 1 traces the unique budget-exact soft-to-hard path for each fixed $k$, so no hand-tuned temperature is needed to harden a mask during training.
  • Score-scale invariance holds: rescaling all scores by a positive constant leaves the mask and $\gamma$ unchanged because the solved $b$ and $t$ rescale correspondingly, while a fixed temperature changes hardness under rescaling.
  • Near the binary endpoint, the fraction of non-saturated mask entries is asymptotically $\frac{8}{3}|\ln(2\varepsilon)|a(1-a)(1-\gamma)$, which depends on budget fraction and saturation threshold but not on the local score density.
  • Every mask satisfying the two moment constraints must contain at least $(1-a)\max(0,1-a(1-\gamma)/\varepsilon)$ near-zero entries, a distribution-free floor that tends to the hard top-k zero fraction as $\gamma\to1$.
  • The implementation is practical at scale: sort-free Newton iterations with an implicit backward reach $n=10^7$ in about 10.75 ms on an A100 while keeping budget and second-moment residuals near $10^{-7}$.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the uniqueness theorem only uses strict monotonicity and the endpoint limits of the mask family, so the same two-moment construction should transfer to other smooth cumulative-distribution masks with symmetric densities; the Laplace form is a convenient choice, not a necessary one.
  • Editorial inference: the density-independent coefficient in the near-binary law suggests that annealed hardness curves will look similar across very different score distributions; a direct test is to measure the saturated-fraction slope against $1-\gamma$ on heavy-tailed or bimodal scores.
  • Editorial inference: because gradients come from implicit differentiation, the forward solver need not itself be differentiable; a faster approximate root-finder could replace the Newton-bisection fallback as long as the converged point meets tolerance, potentially extending the method beyond $10^7$ dimensions.
  • Editorial inference: exact per-mask budgets make the layer a natural relaxation for per-token expert routing, where the current practice is to train with a soft gate and deploy a hard top-k route; LaPrune would make that hand-tuned transition a two-constraint interpolation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. LaPrune is a differentiable top-k mask layer that augments the LapSum soft top-k operator with a normalized hardness parameter γ. For a fixed budget k, the layer solves for a temperature t and barrier b so that the mask has both the desired total mass and a target normalized second moment βk = (a+(1−a)γ)k. The paper claims that for every nonconstant score vector and every γ∈(0,1) there is a unique finite temperature realizing the target, that γ=0 and γ=1 correspond to the equal-mass and hard top-k masks, and that the layer is differentiable through the implicit solution. The authors also derive a mean-field population prediction for the saturated fraction, an asymptotic law near the binary limit, and a distribution-free lower bound on the fraction of near-zero mask entries. The empirical sections test scale invariance, feature recovery, sparse-autoencoder training, worst-case floor satisfaction, and scalability to n=10^7.

Significance. If the correctness gap in Theorem 1 is repaired, this is a strong and useful contribution. The normalized-hardness parameter is a genuinely useful interface for differentiable sparsity, and the paper's supporting analysis is substantial: the proof of monotonicity of M2(t) is sound for nonconstant scores, the proof of the worst-case sparsity floor (Theorem 2) is rigorous and tight, and the mean-field prediction is verified numerically to within 0.0022. The scalability experiments with an analytical implicit backward pass up to n=10^7 are also a concrete practical strength. The paper is careful in several places to state the scope of its density-dependent claims, and the Limitations section is candid about numerical conditioning near the binary endpoint. However, the main existence theorem is currently stated more broadly than the proof supports, and the advertised 'controllable at every γ' guarantee fails in a simple tied-score case, so the manuscript requires substantive revision before the central claims can be accepted.

major comments (2)
  1. [§3, Theorem 1] Theorem 1 as stated is false for nonconstant score vectors with a tie at the selection boundary, and the advertised exact-budget hardness control therefore does not hold for every γ∈(0,1). Consider n=4, r=(1,1,0,0), k=3, a=3/4. As t↓0 the budget equation forces b(t)/t→0, giving p→(1,1,1/2,1/2) and M2→5/2; the target second moment for γ=0.9 is βk=(a+(1−a)γ)k=2.925>2.5, so no finite t solves both constraints, and in fact every γ>1/3 is infeasible on this input. The supplementary proof establishes the endpoint limit M2(t)→k only under the pairwise-distinct assumption, so the theorem's 'Consequently' overreaches. The claim should be restricted to pairwise distinct scores or amended with a data-dependent feasibility interval [0,γ_max] for the tied case, and the abstract and Section 1 should be revised accordingly.
  2. [§3, Eqs. (5)–(7)] The backward pass in Eq. (7) invokes the implicit function theorem for the 2×2 system, but the paper never states the required nonsingularity condition on J_{(b,τ)}F. The condition does hold for nonconstant scores: det J=(2/t)(S0 U1−S1 U0)=(2/t)S0^2 Cov_w(z,σ(z))>0 because σ is strictly increasing, but this should be stated and proved as part of the differentiability claim. The solver-convergence guarantee is also missing; the supplementary material reports that the bisection fallback is required for 97% of rows at γ=0.99, so the 'controllable at each budget' claim needs a precise condition under which the Newton-bisection solver reaches tolerance or a qualification near the binary endpoint.
minor comments (2)
  1. [Abstract and §3] The abstract and introduction describe the layer as having an 'exact budget' while the numerical sections report residuals around 10^-7; adding a sentence clarifying that 'exact' refers to the mathematical constraint satisfied to solver tolerance would prevent confusion.
  2. [Supplementary Materials, Table 3] Table 3 in the supplementary presents a CIFAR-100 ablation showing a sharp collapse for fixed γ=0.9, but the main text refers to this table only indirectly; please add a cross-reference and a brief interpretive sentence so that the reader understands the collapse as an optimization-dynamics phenomenon rather than a property of the mask constraint.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found; LaPrune's guarantees follow from its defining moment equations rather than from fitted values or self-citations that smuggle in the target result.

full rationale

LaPrune's central construction is the joint constraint system in Eq. (4): F1 enforces the budget sum p_i = k and F2 enforces the target second moment sum p_i^2 = beta k with beta = a + (1-a) gamma. The unique-temperature theorem is derived, not assumed: the supplementary proof differentiates M2(t) and shows it is strictly decreasing because the weighted covariance of z and sigma(z) is positive, and the endpoint limits come from the LapSum operator's previously established behavior for pairwise distinct scores. No parameter is fitted to data and then relabeled as a prediction. The population saturated-fraction formulas in Eqs. (8)-(9) are direct consequences of the population mask m(x) = sigma((x-b)/t) and the same two moment equations, so their agreement with simulations is an honest mathematical consistency check. The near-binary law and the worst-case sparsity floor in Theorem 2 are algebraic and asymptotic consequences of the first and second moments of the mask, with no fitted constants. The paper does rely on the prior LapSum operator from Struski et al. 2025, and two current authors are among that paper's authors, but that cited result is a separate, parameter-free published theorem whose assumptions do not include LaPrune's target claim, so it functions as independent support rather than as a circular self-citation. One mathematical caveat is worth flagging: Theorem 1 states the unique-temperature guarantee for every nonconstant score vector, but the proof's endpoint limits require pairwise distinct scores; with a tie at the selection boundary, high-gamma targets can become infeasible. That is a correctness gap in the theorem statement or proof, not a circular step.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central construction rests on the Laplace-CDF mask family inherited from LapSum and on the smoothness of the two-moment system. The only user-chosen knob is gamma; the saturation-threshold epsilon is an analysis constant. No new physical or model entities are introduced, and no parameter is fitted to downstream data.

free parameters (2)
  • gamma (hardness target) = user-specified in [0,1]
    Target normalized second moment of the mask; chosen by the user, not fitted to data. The paper analyzes its effect but does not estimate it from data.
  • epsilon (saturation threshold) = 1e-3 (also 1e-4 and 1e-2 in sensitivity sweeps)
    Defines when a mask entry counts as numerically zero or one in the saturation analysis and Theorem 2. It is an analysis threshold, not a fitted model parameter.
assumptions (5)
  • domain assumption Scores are nonconstant and pairwise distinct for the endpoint limits of M2(t) and the unique-temperature theorem.
    Theorem 1 states the limits M2 -> k as t->0 require pairwise distinct scores; tied scores can make the hard top-k limit ambiguous.
  • standard math The 2x2 constraint Jacobian J(b,tau) F is nonsingular at the solution, so the implicit function theorem applies for the backward pass.
    Invoked in Eq. (7) for implicit gradients; the paper does not explicitly prove det(J) > 0, though it follows from the weighted-covariance positivity used in the Theorem 1 proof.
  • domain assumption For the mean-field and near-binary results, scores are i.i.d. draws from a density rho that is positive in a neighborhood of the (1-a)-quantile.
    Section 4 states this assumption before Equation (8) and the limiting law; it is needed to approximate empirical sums by integrals and to expand M1-M2 to first order.
  • domain assumption The mask family is restricted to the Laplace CDF sigma((r_i-b)/t).
    The entire construction, including the closed-form gradient and moments, relies on the Laplace-CDF form inherited from LapSum; the paper notes this as a limitation.
  • standard math After sorting, a mask is represented by a nondecreasing rank profile f on [0,1].
    Used in Theorem 2 to translate vector moment constraints into integral constraints on f; sorting makes f nondecreasing by construction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LaPrune: Controllable Differentiable Sparsity at Million Scale." pith.science (2026). https://pith.science/paper/HP54DCX6

@misc{pith2026260804057,
  author       = {Pith},
  title        = {Pith review of: LaPrune: Controllable Differentiable Sparsity at Million Scale},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HP54DCX6}},
  note         = {Machine review of arXiv:2608.04057}
}
abstract

Top-$k$ selection determines which components of a sparse model remain active. Hard selection blocks gradients, while continuous relaxations often couple mask hardness to the selected mass. We introduce LaPrune, a mathematically exact-budget differentiable layer that controls the normalized second moment while preserving the selected mass. A LapSum barrier preserves the selection mass, and a normalized second-moment constraint moves the mask from a dense equal-mass allocation toward hard top-$k$ at each budget. We derive a population prediction of the saturated fraction, a near-binary limiting law, and a tight worst-case guarantee on the near-zero fraction. The normalized hardness parameter is invariant to score scale, while a fixed LapSum temperature is not.

Figures

Figures reproduced from arXiv: 2608.04057 by the authors.

Figure 1
Figure 1. Combined forward-and-backward runtime (top) [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Normalized second-moment control on Gaussian scores with [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Direct numerical check of the worst-case sparsity floor at budget fraction [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 6
Figure 6. Figure 6: Realized soft-mask mass against DFTopK softness [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 5
Figure 5. Figure 5: Population prediction and simulation for three [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 7
Figure 7. Figure 7: Feature-recovery F1 against deployment budget [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Sparse-autoencoder FVU and linear-probe accuracy against deployment budget [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Sparse-autoencoder FVU and linear-probe accuracy against dictionary size [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 28 canonical work pages

  1. [1]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  2. [2]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  3. [3]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

    1980

  4. [4]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  5. [5]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

    doi:10.1016/s0020-7373(84)80003-6 1984

  6. [6]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  7. [7]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  8. [8]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

Show all 63 references
  1. [9]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  2. [10]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  3. [11]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  4. [12]

    International Conference on Learning Representations (ICLR) , year =

    Grover, Aditya and Wang, Eric and Zweig, Aaron and Ermon, Stefano , title =. International Conference on Learning Representations (ICLR) , year =. 1903.08850 , archivePrefix =

    1903 arXiv

  5. [13]

    International Conference on Machine Learning (ICML) , year =

    Blondel, Mathieu and Teboul, Olivier and Berthet, Quentin and Djolonga, Josip , title =. International Conference on Machine Learning (ICML) , year =. 2002.08871 , archivePrefix =

    2002 arXiv

  6. [14]

    International Conference on Machine Learning (ICML) , year =

    Prillo, Sebastian and Eisenschlos, Julian Martin , title =. International Conference on Machine Learning (ICML) , year =. 2006.16038 , archivePrefix =

    2006 arXiv

  7. [15]

    International Conference on Machine Learning (ICML) , year =

    Petersen, Felix and Borgelt, Christian and Kuehne, Hilde and Deussen, Oliver , title =. International Conference on Machine Learning (ICML) , year =. 2105.04019 , archivePrefix =

    arXiv

  8. [16]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Xie, Yujia and Dai, Hanjun and Chen, Minshuo and Dai, Bo and Zhao, Tuo and Zha, Hongyuan and Wei, Wei and Pfister, Tomas , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 2002.06504 , archivePrefix =

    2002 arXiv

  9. [17]

    and Puigcerver, Joan and Djolonga, Josip and Peyr

    Sander, Michael E. and Puigcerver, Joan and Djolonga, Josip and Peyr. Fast, Differentiable and Sparse Top- k : a Convex Analysis Perspective , booktitle =. 2023 , eprint =

    2023

  10. [18]

    Proceedings of the 42nd International Conference on Machine Learning , pages =

    Struski,. Proceedings of the 42nd International Conference on Machine Learning , pages =. 2025 , pdf =

    2025

  11. [19]

    Tabor, Jacek and Struski,

  12. [20]

    2025 , eprint=

    Differentiable Fast Top-K Selection for Large-Scale Recommendation , author=. 2025 , eprint=

    2025

  13. [21]

    , title =

    Sanh, Victor and Wolf, Thomas and Rush, Alexander M. , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 2005.07683 , archivePrefix =

    2005 arXiv

  14. [22]

    and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , title =

    Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , title =. International Conference on Learning Representations (ICLR) , year =. 2106.09685 , archivePrefix =

    arXiv

  15. [23]

    , title =

    Han, Song and Pool, Jeff and Tran, John and Dally, William J. , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 1506.02626 , archivePrefix =

    arXiv

  16. [24]

    International Conference on Learning Representations (ICLR) , year =

    Frankle, Jonathan and Carbin, Michael , title =. International Conference on Learning Representations (ICLR) , year =. 1803.03635 , archivePrefix =

    arXiv

  17. [25]

    International Conference on Learning Representations (ICLR) , year =

    Shazeer, Noam and Mirhoseini, Azalia and Maziarz, Krzysztof and Davis, Andy and Le, Quoc and Hinton, Geoffrey and Dean, Jeff , title =. International Conference on Learning Representations (ICLR) , year =. 1701.06538 , archivePrefix =

    arXiv

  18. [26]

    Journal of Machine Learning Research (JMLR) , year =

    Fedus, William and Zoph, Barret and Shazeer, Noam , title =. Journal of Machine Learning Research (JMLR) , year =. 2101.03961 , archivePrefix =

    arXiv

  19. [27]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Rao, Yongming and Zhao, Wenliang and Liu, Benlin and Lu, Jiwen and Zhou, Jie and Hsieh, Cho-Jui , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 2106.02034 , archivePrefix =

    arXiv

  20. [28]

    International Conference on Learning Representations (ICLR) , year =

    Makhzani, Alireza and Frey, Brendan , title =. International Conference on Learning Representations (ICLR) , year =. 1312.5663 , archivePrefix =

    arXiv

  21. [29]

    Scaling and Evaluating Sparse Autoencoders , booktitle =

    Gao, Leo and Dupr. Scaling and Evaluating Sparse Autoencoders , booktitle =. 2025 , eprint =

    2025

  22. [30]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Berthet, Quentin and Blondel, Mathieu and Teboul, Olivier and Cuturi, Marco and Vert, Jean-Philippe and Bach, Francis , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 2002.08676 , archivePrefix =

    2002 arXiv

  23. [31]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Cuturi, Marco , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 1306.0895 , archivePrefix =

    arXiv

  24. [32]

    , title =

    Blelloch, Guy E. , title =

  25. [33]

    , title =

    Harris, Mark and Sengupta, Shubhabrata and Owens, John D. , title =. GPU Gems 3 , year =

  26. [34]

    Merrill, Duane and Garland, Michael , title =

  27. [35]

    Berkay and Swami, Ananthram , title =

    Papernot, Nicolas and McDaniel, Patrick and Jha, Somesh and Fredrikson, Matt and Celik, Z. Berkay and Swami, Ananthram , title =. IEEE European Symposium on Security and Privacy (EuroS&P) , year =. 1511.07528 , archivePrefix =

    arXiv

  28. [36]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    Modas, Apostolos and Moosavi-Dezfooli, Seyed-Mohsen and Frossard, Pascal , title =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =. 1811.02248 , archivePrefix =

    arXiv

  29. [37]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Dong, Xiaoyi and Chen, Dongdong and Bao, Jianmin and Qin, Chuan and Yuan, Lu and Zhang, Weiming and Yu, Nenghai and Chen, Dong , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 2010.13773 , archivePrefix =

    2010 arXiv

  30. [38]

    Natural Evolution Strategies , journal =

    Wierstra, Daan and Schaul, Tom and Glasmachers, Tobias and Sun, Yi and Peters, Jan and Schmidhuber, J. Natural Evolution Strategies , journal =. 2014 , url =

    2014

  31. [39]

    International Conference on Machine Learning (ICML) , year =

    Ilyas, Andrew and Engstrom, Logan and Athalye, Anish and Lin, Jessy , title =. International Conference on Machine Learning (ICML) , year =. 1804.08598 , archivePrefix =

    arXiv

  32. [40]

    and Flammarion, Nicolas and Hein, Matthias , title =

    Croce, Francesco and Andriushchenko, Maksym and Singh, Naman D. and Flammarion, Nicolas and Hein, Matthias , title =. AAAI Conference on Artificial Intelligence , year =. 2006.12834 , archivePrefix =

    2006 arXiv

  33. [41]

    arXiv preprint arXiv:2212.07495 , year =

    Imtiaz, Tooba and Kohler, Morgan and Miller, Jared and Wang, Zifeng and Eskandar, Masih and Sznaier, Mario and Camps, Octavia and Dy, Jennifer , title =. arXiv preprint arXiv:2212.07495 , year =. 2212.07495 , archivePrefix =

  34. [42]

    International Conference on Learning Representations (ICLR) , year =

    Xu, Kaidi and Liu, Sijia and Zhao, Pu and Chen, Pin-Yu and Zhang, Huan and Fan, Quanfu and Erdogmus, Deniz and Wang, Yanzhi and Lin, Xue , title =. International Conference on Learning Representations (ICLR) , year =. 1808.01664 , archivePrefix =

    arXiv

  35. [43]

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    Woo, Sanghyun and Debnath, Shoubhik and Hu, Ronghang and Chen, Xinlei and Liu, Zhuang and Kweon, In So and Xie, Saining , title =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =. 2301.00808 , archivePrefix =

    arXiv

  36. [44]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , year =

    Zhong, Xuyang and Liu, Chen , title =. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) , year =. doi:10.1109/TPAMI.2025.3630185 , note =

    2025

  37. [45]

    IEEE/CVF International Conference on Computer Vision (ICCV) , year =

    Croce, Francesco and Hein, Matthias , title =. IEEE/CVF International Conference on Computer Vision (ICCV) , year =. 1909.05040 , archivePrefix =

    1909 arXiv

  38. [46]

    -zero: Gradient-based Optimization of _0 -norm Adversarial Examples , booktitle =

    Cin. -zero: Gradient-based Optimization of _0 -norm Adversarial Examples , booktitle =. 2025 , eprint =

    2025

  39. [47]

    European Conference on Computer Vision (ECCV) , year =

    Fan, Yanbo and Wu, Baoyuan and Li, Tuanhui and Zhang, Yong and Li, Mingyang and Li, Zhifeng and Yang, Yujiu , title =. European Conference on Computer Vision (ECCV) , year =. doi:10.1007/978-3-030-58542-6_3 , url =

    doi

  40. [48]

    , title =

    Roberts, Lawrence G. , title =. IRE Transactions on Information Theory , volume =

  41. [49]

    Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation , journal =

    Bengio, Yoshua and L. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation , journal =. 2013 , eprint =

    2013

  42. [50]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    van den Oord, Aaron and Vinyals, Oriol and Kavukcuoglu, Koray , title =. Advances in Neural Information Processing Systems (NeurIPS) , year =. 1711.00937 , archivePrefix =

    arXiv

  43. [51]

    International Conference on Learning Representations (ICLR) , year =

    Jang, Eric and Gu, Shixiang and Poole, Ben , title =. International Conference on Learning Representations (ICLR) , year =. 1611.01144 , archivePrefix =

    arXiv

  44. [52]

    and Mnih, Andriy and Teh, Yee Whye , title =

    Maddison, Chris J. and Mnih, Andriy and Teh, Yee Whye , title =. International Conference on Learning Representations (ICLR) , year =. 1611.00712 , archivePrefix =

    arXiv

  45. [53]

    , title =

    Williams, Ronald J. , title =. Machine Learning , volume =. 1992 , doi =

    1992

  46. [54]

    arXiv preprint arXiv:1703.03864 , year =

    Salimans, Tim and Ho, Jonathan and Chen, Xi and Sidor, Szymon and Sutskever, Ilya , title =. arXiv preprint arXiv:1703.03864 , year =. 1703.03864 , archivePrefix =

    arXiv

  47. [55]

    and Ba, Jimmy , title =

    Kingma, Diederik P. and Ba, Jimmy , title =. International Conference on Learning Representations (ICLR) , year =. 1412.6980 , archivePrefix =

    arXiv

  48. [56]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    Sparse-PGD: A Unified Framework for Sparse Adversarial Perturbations Generation , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  49. [57]

    arXiv preprint arXiv:1901.10517 , year=

    Reparameterizable subset sampling via continuous relaxations , author=. arXiv preprint arXiv:1901.10517 , year=

    1901 arXiv

  50. [58]

    International Conference on Machine Learning , pages=

    Differentiable top-k classification learning , author=. International Conference on Machine Learning , pages=. 2022 , organization=

    2022

  51. [59]

    International conference on machine learning , pages=

    From softmax to sparsemax: A sparse model of attention and multi-label classification , author=. International conference on machine learning , pages=. 2016 , organization=

    2016

  52. [60]

    Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

    Sparse sequence-to-sequence models , author=. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages=

  53. [61]

    Advances in Neural Information Processing Systems , volume=

    Dselect-k: Differentiable selection in the mixture of experts with applications to multi-task learning , author=. Advances in Neural Information Processing Systems , volume=

  54. [62]

    arXiv preprint arXiv:2412.06410 , year=

    Batchtopk sparse autoencoders , author=. arXiv preprint arXiv:2412.06410 , year=

    arXiv

  55. [63]

    2018 , eprint=

    Smooth Loss Functions for Deep Top-k Classification , author=. 2018 , eprint=

    2018

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.