REVIEW 4 cited by
Reparameterizable Subset Sampling via Continuous Relaxations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Many machine learning tasks require sampling a subset of items from a collection based on a parameterized distribution. The Gumbel-softmax trick can be used to sample a single item, and allows for low-variance reparameterized gradients with respect to the parameters of the underlying distribution. However, stochastic optimization involving subset sampling is typically not reparameterizable. To overcome this limitation, we define a continuous relaxation of subset sampling that provides reparameterization gradients by generalizing the Gumbel-max trick. We use this approach to sample subsets of features in an instance-wise feature selection task for model interpretability, subsets of neighbors to implement a deep stochastic k-nearest neighbors model, and sub-sequences of neighbors to implement parametric t-SNE by directly comparing the identities of local neighbors. We improve performance in all these tasks by incorporating subset sampling in end-to-end training.
Forward citations
Cited by 4 Pith papers
-
Rank-Conditioned Sample Reuse for the Plackett--Luce Best-of-$K$ Objective
Rank-conditioned Horvitz–Thompson reuses all C(n,K) subsets of one Gumbel-Top-n pool for unbiased Plackett–Luce best-of-K value and score-function gradient, with an exact Max-specific DP collapse to a 1-D integral.
-
LaPrune: Controllable Differentiable Sparsity at Million Scale
A differentiable top-k mask layer that enforces an exact selection budget and uses a normalized hardness parameter to interpolate from equal-weight masks to hard binary masks, with saturation theory and million-scale results.
-
DeeperRadar: End-to-End MIMO Radar Design and Multi-Modal Fusion for Autonomous Vehicle Perception
Learning which MIMO radar receivers to activate jointly with camera–LiDAR fusion lets fewer receivers match or exceed full-array 3D detection on RADIal, with the best budget depending on the sensor stack.
-
Learn from A Rationalist: Distilling Intermediate Interpretable Rationales
Distilling a teacher rationale-extraction model's feature selections and predictions into smaller students improves student accuracy by up to ~14 points on CIFAR-10 while keeping the same rationale sparsity.
Discussion (0). Continue with ORCID to comment.