On Differentiating Parameterized Argmin and Argmax Problems with Application to Bi-level Optimization
read the original abstract
Some recent works in machine learning and computer vision involve the solution of a bi-level optimization problem. Here the solution of a parameterized lower-level problem binds variables that appear in the objective of an upper-level problem. The lower-level problem typically appears as an argmin or argmax optimization problem. Many techniques have been proposed to solve bi-level optimization problems, including gradient descent, which is popular with current end-to-end learning approaches. In this technical report we collect some results on differentiating argmin and argmax optimization problems with and without constraints and provide some insightful motivating examples.
This paper has not been read by Pith yet.
Forward citations
Cited by 9 Pith papers
-
Constraint-Aware Flow Matching: Decision Aligned End-to-End Training for Constrained Sampling
Constraint-Aware Flow Matching integrates constraint projections into the flow matching training objective to align model dynamics with constrained sampling and reduce distributional shift.
-
Decision-Focused Learning via Tangent-Space Projection of Prediction Error
Regret gradients in DFL are the tangent-space projection of prediction error scaled by curvature, enabling efficient direct computation without differentiating through solvers.
-
Learning Latent Trees with Stochastic Perturbations and Differentiable Dynamic Programming
A fully differentiable parser that stochastically samples projective dependency trees using Gumbel perturbations and dynamic programming to boost downstream task performance without direct supervision.
-
Decision-Focused Learning via Tangent-Space Projection of Prediction Error
PEAR computes regret gradients via tangent-space projection of prediction error, delivering top decision quality and efficiency on LP and QP tasks without solver differentiation.
-
Representation-Guided Parameter-Efficient LLM Unlearning
REGLU guides LoRA-based unlearning via representation subspaces and orthogonal regularization to outperform prior methods on forget-retain trade-off in LLM benchmarks.
-
SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
SalUn uses gradient-based weight saliency to achieve effective machine unlearning of data, classes, or concepts in image classification and generation, narrowing the gap to exact retraining.
-
InfoGeo: Information-Theoretic Object-Centric Learning for Cross-View Generalizable UAV Geo-Localization
InfoGeo reformulates cross-view geo-localization as an information bottleneck that aligns object-centric structural relations across views while minimizing view-specific noise.
-
InfoGeo: Information-Theoretic Object-Centric Learning for Cross-View Generalizable UAV Geo-Localization
InfoGeo applies an information bottleneck to object-centric learning for improved cross-view generalization in UAV geo-localization.
-
InfoGeo: Information-Theoretic Object-Centric Learning for Cross-View Generalizable UAV Geo-Localization
InfoGeo reformulates cross-view geo-localization as an information bottleneck that aligns object-centric structural relations while suppressing view-specific noise, outperforming prior methods on benchmarks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.