Learning Set Functions with Implicit Differentiation

Chengzhi Shi; G\"ozde \"Ozcan; Stratis Ioannidis

REVIEW 3 major objections 5 minor 85 references

Learning Set Functions with Implicit Differentiation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that, under a uniform bound on the multilinear relaxation, the fixed-point iterations for learning set functions converge to a unique solution, and shows how to differentiate through that fixed point via implicit…

desk verdict Sound contraction theory plus a genuinely useful implicit-differentiation trick, but the evaluated variants solve a different fixed-point equation than the theorem covers, and the printed pseudocode omits the root-finding loop. read the letter →

arxiv 2412.11239 v2 pith:CO6N42ET submitted 2024-12-15 cs.LG cs.AI

Gözde Özcan , Chengzhi Shi , Stratis Ioannidis This is my paper · ORCID

classification cs.LGcs.AI MSC 68T0765H10

keywords setfunctionsoptimalsubsetoraclemean-fieldvariationalinferencefixed-pointiterationimplicitdifferentiationdeepequilibriummodelsmultilinearrelaxationselection

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles a bottleneck in learning set functions from an optimal subset oracle: the mean-field variational inference of Ou et al. (2022) reduces training to a fixed-point iteration, but those iterations can diverge and backpropagating through them stacks huge Jacobians. The authors prove that if the multilinear relaxation of the utility function is bounded in absolute value by the reciprocal of the ground-set size, the fixed-point map is a contraction, so the iterations converge to a unique solution from any start. They then show how to differentiate through the converged fixed point using the implicit function theorem, replacing memory-hungry loop unrolling with a linear solve. If correct, this makes the method both theoretically grounded and practical on larger problems.

What carries the argument

The load-bearing object is the map $T_\theta(\psi)=\sigma(\nabla_\psi \tilde{F}(\psi,\theta))$, built from the multilinear relaxation $\tilde{F}(\psi,\theta)$ of the set function and the sigmoid. The proof that $T_\theta$ is a contraction uses the multivariate mean value theorem, the fact that the sigmoid derivative is at most $1/4$, and a Hessian bound of $4\sup|\tilde{F}|$, yielding a Lipschitz constant of $|V|\sup|\tilde{F}| < 1$ under Assumption 4.1; Banach's fixed-point theorem then gives uniqueness and convergence. For differentiation, the implicit function theorem applied to $G(\psi,\theta)=\sigma(\nabla_\psi \tilde{F}(\psi,\theta))-\psi=0$ turns the Jacobian into the solution of a linear system $A^{-1}B$, avoiding unrolled backpropagation.

What would settle it

Train a set function on any of the paper's datasets and measure $\sup_{\psi\in[0,1]^{|V|}}|\tilde{F}(\psi,\theta)|$ during training; if it exceeds $1/|V|$ while the fixed-point iterations still converge, or if two different starting points $\psi^{(0)}$ lead to different limits, then the contraction condition is not necessary — and the guarantee as stated would not cover the practical regime. More directly, one could construct a simple set function (e.g., a quadratic multilinear extension) with $\sup|\tilde{F}|$ slightly above $1/|V|$ and check whether the fixed-point map has multiple fixed points or the iteration cycles.

Watch

Extended reading notes

Core claim

The central claim is Theorem 4.2: under the uniform bound $\sup_{\psi\in[0,1]^{|V|}} |\tilde{F}(\psi,\theta)| < 1/|V|$, the map $T_\theta(\psi)=\sigma(\nabla_\psi \tilde{F}(\psi,\theta))$ is a contraction on the unit hypercube, so the fixed-point equation $\psi=\sigma(\nabla_\psi \tilde{F}(\psi,\theta))$ has a unique solution and the iterations $\psi^{(k)}=T_\theta(\psi^{(k-1)})$ converge to it from any starting point. From this, Theorem 4.3 derives the gradient of the loss through the fixed point as $\partial_\theta \psi^*(\theta)=A^{-1}B$, where $A=I-\Sigma'(\nabla_\psi \tilde{F})\nabla^2_\psi \tilde{F}$ and $B=\Sigma'(\nabla_\psi \tilde{F})\partial_\theta \nabla_\psi \tilde{F}$, which requires only the final fixed point rather than all iterates. The paper further supports the theory with experiments on synthetic and real-world subset selection tasks, showing the implicit-differentiation variants match or exceed prior methods while using constant memory.

Load-bearing premise

The whole convergence guarantee rests on Assumption 4.1, the uniform bound $|\tilde{F}(\psi,\theta)| < 1/|V|$, but the paper never verifies this bound for the trained networks, and the implemented variants normalize the gradient or scale by a constant, so the fixed-point equation they actually solve is a different one.

Editorial extensions

If this is right

Under Assumption 4.1, the variational mean-field solver inherits a rigorous convergence guarantee: uniqueness of the fixed point and global convergence from any initialization (Theorem 4.2).
The implicit-differentiation gradient formula $\partial_\theta \psi^* = A^{-1}B$ (Theorem 4.3) holds at the fixed point, so training can use constant memory $O(1)$ instead of $O(K)$ in the number of iterations.
Because the gradient no longer requires unrolling, practitioners can run the fixed-point solver to a tight tolerance (e.g., $10^{-6}$) rather than the $K\le 5$ iterations used by prior work.
The theory applies to any Lipschitz-regularized network with appropriately chosen Boltzmann temperature, since scaling the energy by a constant controls the bound in Assumption 4.1.
In experiments, the iDiffMF variants (with gradient normalization by $\ell^2$ or nuclear norm) reach best-or-second-best Jaccard scores on most datasets, suggesting the practical value of implicit differentiation in this setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the bound in Assumption 4.1 is hard to certify, a testable alternative is to enforce it during training (e.g., by spectral normalization of the network or by adaptive temperature), which would make the convergence guarantee apply to the actual training procedure rather than a scaled variant.
The contraction condition is only sufficient; the empirical success of the normalized variants suggests the true fixed-point map may be contracting in a wider region, or that the relevant quantity is the Lipschitz constant of $T_\theta$ rather than the sup-norm bound on $\tilde{F}$. A direct measurement of the Lipschitz constant during training would settle this.
The $A^{-1}B$ formula could be extended to other energy-based models with different link functions (e.g., softmax over subsets), where a similar implicit-differentiation treatment would avoid unrolling over the combinatorial state space.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the paper proves a real contraction theorem and applies implicit differentiation to a fixed-point problem where unrolling was expensive. That is a legitimate contribution. But the methods that actually get the good numbers solve a different fixed-point equation than the one covered by the theorem, and the printed algorithm has no loop to convergence. Those gaps are fixable, but they matter for the empirical claims.

The new thing is Theorem 4.2: under sup_{ψ}|F~(ψ,θ)| < 1/|V|, the map ψ ↦ σ(∇ψF~(ψ,θ)) is a contraction on the cube, so the DiffMF iterations converge to a unique fixed point from anywhere. I checked App. E; the proof is sound. Applying DEQ-style implicit differentiation to this system is a natural and useful move, and the memory measurements support the claimed O(1) backward memory. The citation pattern is fine, with the novelty over Ou et al. clearly stated. On the experimental side, iDiffMF2 and iDiffMF* are best or second-best on most of the Amazon and BindingDB tasks, which is a genuine improvement over the baselines.

Now the soft spots. First, Assumption 4.1 is never verified for the trained networks. The authors say Lipschitz networks and temperature can make it hold, but they don't actually do it. Second, and more important: the two variants that get the good results, iDiffMF2 and iDiffMF*, replace ∇ψF~ by 2∇ψF~/(|V|Q) with Q the ψ-dependent norm. That is equation (27), not equation (8). The contraction theorem and the implicit-differentiation formula apply to G(ψ,θ)=σ(∇ψF~)−ψ, not to the normalized map. So the theoretical guarantees don't cover the headline methods. For iDiffMFc the fixed point is still of the same form with a constant scaling, so it could be covered, but c is chosen by cross-validation and Assumption 4.1 isn't checked at that c. Third, Algorithm 2 as printed performs a single fixed-point update per element and contains no loop to convergence; it doesn't match the root-finding procedure described in the text or used in the experiments. The supplement says code will be released after review; at the moment there's no public code.

None of this makes me think the method is wrong. The contraction proof is solid, and the normalization heuristics plausibly help. But the paper as submitted overclaims the connection between its theorem and its experiments. A revision should either extend the analysis to the normalized maps, or honestly frame the normalization as a heuristic and verify the original assumption on at least some datasets. The pseudocode needs to match the implementation.

Who should read it: anyone working on learning set functions from optimal subset oracles, and people applying implicit differentiation to variational fixed points. It deserves a serious referee; I'd send it out. With the gaps closed, it would be a solid paper in a niche area.

Referee Report

3 major / 5 minor

Summary. The paper studies the problem of learning a set function from an optimal-subset oracle, in the framework of Ou et al. (2022) where the oracle is modeled by a Boltzmann distribution and mean-field variational inference leads to a fixed-point equation ψ = σ(∇ψ F~(ψ,θ)). The authors' contribution is twofold: (i) Theorem 4.2 gives a contraction guarantee and uniqueness for this fixed point under Assumption 4.1 (uniform bound on the multilinear relaxation), and (ii) Theorem 4.3 uses the implicit function theorem to compute the parameter gradient as A⁻¹B, avoiding backpropagation through the unrolled iterations. The proposed algorithm iDiffMF is evaluated against DiffMF, EquiVSet_ind, EquiVSet_copula on set anomaly detection, product recommendation, and compound selection datasets. The reported iDiffMF2/iDiffMF* variants achieve the best or second-best Jaccard coefficient in most cases, with constant GPU memory in the number of fixed-point iterations.

Significance. The convergence and implicit-differentiation results are cleanly stated, and the contraction proof in Appendix E is mathematically sound under Assumption 4.1. The paper's empirical results are promising, and the O(1) memory benefit is clearly demonstrated in Figure 1. However, the manuscript currently has a load-bearing disconnect between the proved fixed-point equation and the algorithm actually evaluated: the normalized variants iDiffMF2/iDiffMF* solve equation (27), not equation (7), and no convergence/invertibility analysis is given for that map. In addition, Algorithm 2 as printed does not contain the iterative root-finding loop described in the text. Both issues are fixable in revision. If addressed, this would be a solid contribution to the implicit-differentiation and neural-set-function literature.

major comments (3)

[§5.1, App. I.3, Eq. (27)] Theorem 4.2 and Theorem 4.3 are stated for the fixed-point equation ψ = σ(∇ψ F~(ψ,θ)) and the corresponding implicit map G(ψ,θ) = σ(∇ψ F~(ψ,θ)) − ψ. The methods reported as the main empirical contribution, iDiffMF2 and iDiffMF*, instead solve Eq. (27), ψ = σ(2∇ψ F~(ψ,θ)/(|V|Q)), where Q = ||∇ψ F~(ψ,θ)||₂ or Q = ||∇ψ F~(ψ,θ)||★. Since Q depends on ψ, this map is not of the form σ(∇ψ F~) for a fixed function F~, so the contraction proof in App. E and the implicit-differentiation formula A⁻¹B of Theorem 4.3 do not apply to the evaluated variants. The iDiffMFc variant is covered in form (a constant rescaling of F~), but Assumption 4.1 is never verified at the chosen c, and the dynamic normalization in iDiffMF2/iDiffMF* changes the fixed point itself. Please provide an analysis of the normalized map or verify the contraction/invertibility conditions empirically (e.g., by tracking an estimate of sup_ψ |F~(ψ,θ)| during training), and state clearly which variants are covered by Theorem 4.2.
[Algorithm 2, Sec. 5.1] The pseudocode for iDiffMF performs a single fixed-point update: lines 5–8 compute ψ* from ψ(0) without any loop or stopping criterion. However, the text (Sec. 5.1 and Sec. 4.3) and the reported results use a root-finding procedure with tolerance ε = 10⁻⁶, and Sec. 4.4 analyzes the number of iterations to convergence. This inconsistency makes the algorithm as printed not the algorithm evaluated. The pseudocode should either include the iterative loop (e.g., while ||ψ(k) − ψ(k−1)|| > ε) or be explicitly labeled as a simplified single-step illustration with the actual iterative procedure stated in the appendix.
[Assumption 4.1, Sec. 5.1] Assumption 4.1 is the precondition for both convergence and the invertibility of A in Theorem 4.3, yet the paper never checks it in any of the reported experiments. The claim that a Lipschitz-regularized network can satisfy the assumption by adjusting the temperature parameter is plausible but not demonstrated for the architectures and hyperparameters used. Without any empirical estimate of sup_ψ |F~(ψ,θ)| (or, failing that, a discussion of a practical enforcement mechanism), the theoretical guarantee remains disconnected from the experimental validation. Please add such a verification, at least for a representative set of datasets and checkpoints.

minor comments (5)

[Eq. (9)] The constraint "for all i ∈ {i, . . . , n}" should read "for all i ∈ {1, . . . , N}"; the current index set is self-referential.
[App. E] The proof text contains "we are ready to proof Thm. 4.2" (should be "prove"), and Eq. (21) uses "|0,1|" where "[0,1]" is meant.
[Table 1] Several entries lack spacing (e.g., "CelebA202,599100008" and "furniture32"), which makes the table hard to read.
[Sec. 5.1 / App. I.3] The text describes iDiffMF2 and iDiffMF* as using "Frobenius and nuclear norm scaling", but App. I.3 defines Q as the ℓ₂ norm and the nuclear norm of the gradient; for a vector gradient the ℓ₂ norm is not the Frobenius norm, so the terminology should be aligned.
[Sec. 4.4] In the displayed complexity expression, the denominator is written as "log ω" but the same quantity is denoted q in App. H; unify the notation and make clear that ω is the Lipschitz constant of the fixed-point map.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the convergence and implicit-differentiation results are derived from Banach's theorem and the implicit function theorem applied to the inherited fixed-point equation, with no load-bearing reduction to fitted values or self-citation.

full rationale

The paper's derivation chain is self-contained. Equation (7), the fixed-point condition, is derived by differentiating the ELBO in Appendix C.1, and the multilinear relaxation is defined in Eq. (6) following the external reference Calinescu et al. (2011). Theorem 4.2 is a Banach fixed-point argument resting on Assumption 4.1 and the explicit bound in Lemma E.2, not on a fitted value or on an assertion by the authors' prior work. Theorem 4.3 is a direct application of the implicit function theorem to G(psi,theta)=sigma(grad_psi F~(psi,theta))-psi, so the Jacobian formula A^{-1}B is algebraically derived rather than assumed. The citations to Ou et al. (2022) supply the problem formulation and the fixed-point equation, but the convergence and implicit-differentiation results are new arguments applied to that equation, and the authors of Ou et al. are not the present authors. No parameter is fit and then renamed a prediction: the only tuned quantities are the temperature and gradient-normalization constants, and the empirical Jaccard scores are not claimed to be derived from the theory. The skeptical concern that the evaluated iDiffMF2 and iDiffMF* solve the normalized fixed-point equation (27) rather than Equation (8), and that Assumption 4.1 is never verified, is a genuine applicability and completeness gap that could affect the validity of the guarantees for the implemented variants, but it is not circularity: Theorem 4.2 and Theorem 4.3 are not equivalent to the normalization heuristic or to any dataset-dependent fitted input. Similarly, the single-update pseudocode in Algorithm 2 is a mismatch with the root-finding description in the text, but that is an algorithmic correctness issue, not a circular derivation. No circular step can be exhibited by quoting the paper, so the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The theoretical result depends on a smallness condition (Assumption 4.1) that is not measured. The method-specific normalizers c and Q are cross-validated heuristics that alter the fixed-point equation, so the free parameters are central to the empirical behavior.

free parameters (2)

scaling constant c (iDiffMFc) = 100, 1000, or 10000 depending on dataset
Multiplier in 2/(|V|c), chosen by cross-validation (App. I.7), to heuristically satisfy the boundedness assumption.
gradient normalization Q (iDiffMF2, iDiffMF*) = Frobenius norm or nuclear norm of ∇ψF~(ψ,θ)
Used in Eq. (27) to modify the fixed-point equation; a normalization choice, not derived from Assumption 4.1.

assumptions (5)

ad hoc to paper Assumption 4.1: sup |F~(ψ,θ)| < 1/|V| for all θ.
Introduced to make the contraction proof work; not verified experimentally.
domain assumption The ELBO maximization is equivalent to minimizing KL divergence (Eq. 5), inherited from Ou et al. (2022).
The whole variational inference setup rests on this equivalence.
domain assumption The fixed-point condition ψ = σ(∇ψF~) characterizes the ELBO stationary point.
Derived in App. C.1, assumes the mean-field family is adequate.
standard math Implicit function theorem and Banach fixed-point theorem.
Used to prove Thm. 4.2 and Thm. 4.3.
ad hoc to paper Lipschitz-regularized networks can satisfy Assumption 4.1 by adjusting temperature.
Asserted without proof in Sec. 4.1; not used in experiments.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning Set Functions with Implicit Differentiation." pith.science (2026). https://pith.science/paper/CO6N42ET

@misc{pith2026241211239,
  author       = {Pith},
  title        = {Pith review of: Learning Set Functions with Implicit Differentiation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CO6N42ET}},
  note         = {Machine review of arXiv:2412.11239}
}

read the original abstract

Ou et al. (2022) introduce the problem of learning set functions from data generated by a so-called optimal subset oracle. Their approach approximates the underlying utility function with an energy-based model, whose parameters are estimated via mean-field variational inference. Ou et al. (2022) show this reduces to fixed point iterations; however, as the number of iterations increases, automatic differentiation quickly becomes computationally prohibitive due to the size of the Jacobians that are stacked during backpropagation. We address this challenge with implicit differentiation and examine the convergence conditions for the fixed-point iterations. We empirically demonstrate the efficiency of our method on synthetic and real-world subset selection applications including product recommendation, set anomaly detection and compound selection tasks.

Figures

Figures reproduced from arXiv: 2412.11239 by the authors.

**Figure 1.** Effects of the choice of differentiation method on the relationship between the allocated GPU memory and the number [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

85 extracted references · 73 canonical work pages

[1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Anderson, D. G. 1965. Iterative Procedures for Nonlinear Integral Equations. J. ACM, 12(4): 547–560

1965
[4]

Apostol, T. 1974. Mathematical Analysis. Addison-Wesley series in mathematics. Addison-Wesley. ISBN 9780201002881

1974
[5]

Arbel, M.; and Mairal, J. 2022. Amortized implicit differentiation for stochastic bilevel optimization. In ICLR

2022
[6]

Bach, F. 2013. Learning with Submodular Functions: A Convex Optimization Perspective. Foundations and Trends in machine learning , 6(2-3): 145--373

work page 2013
[7]

Z.; and Koltun, V

Bai, S.; Kolter, J. Z.; and Koltun, V. 2019. Deep Equilibrium Models . NeurIPS

work page 2019
[8]

Balcan, M.-F.; and Harvey, N. J. 2018. Submodular functions: Learnability, structure, and optimization. SICOMP

work page 2018

Show all 85 references

[9]

Banach, S. 1922. Sur les op \'e rations dans les ensembles abstraits et leur application aux \'e quations int \'e grales. Fundamenta mathematicae, 3(1): 133--181

1922
[10]

Bertrand, Q.; Klopfenstein, Q.; Blondel, M.; Vaiter, S.; Gramfort, A.; and Salmon, J. 2020. Implicit differentiation of lasso-type models for hyperparameter optimization. In ICML. PMLR

2020
[11]

Bhatt, G.; Das, A.; and Bilmes, J. 2024. Deep Submodular Peripteral Network. NeurIPS

2024
[12]

P.; Hop, C

Bhhatarai, B.; Walters, W. P.; Hop, C. E.; Lanza, G.; and Ekins, S. 2019. Opportunities and challenges using artificial intelligence in ADME/Tox. Nature materials, 18(5): 418--422

2019
[13]

Bilmes, J.; and Bai, W. 2017. Deep submodular functions. arXiv preprint arXiv:1701.08939

2017 arXiv
[14]

M.; Kucukelbir, A.; and McAuliffe, J

Blei, D. M.; Kucukelbir, A.; and McAuliffe, J. D. 2017. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518): 859--877

2017
[15]

Blondel, M.; Berthet, Q.; Cuturi, M.; Frostig, R.; Hoyer, S.; Llinares-L \'o pez, F.; Pedregosa, F.; and Vert, J.-P. 2022. Efficient and Modular Implicit Differentiation. NeurIPS

2022
[16]

Bonab, H.; Aliannejadi, M.; Vardasbi, A.; Kanoulas, E.; and Allan, J. 2021. Cross-market product recommendation. In CIKM

2021
[17]

J.; Leary, C.; Maclaurin, D.; Necula, G.; Paszke, A.; Vander P las, J.; Wanderman- M ilne, S.; and Zhang, Q

Bradbury, J.; Frostig, R.; Hawkins, P.; Johnson, M. J.; Leary, C.; Maclaurin, D.; Necula, G.; Paszke, A.; Vander P las, J.; Wanderman- M ilne, S.; and Zhang, Q. 2018. JAX : composable transformations of P ython+ N um P y programs

2018
[18]

Burke, J. V. 2014. Nonlinear optimization. Lecture Notes, Math, 408: 80

2014
[19]

Calinescu, G.; Chekuri, C.; Pal, M.; and Vondr \'a k, J. 2011. Maximizing a monotone submodular function subject to a matroid constraint. SICOMP

2011
[20]

T.; Rubanova, Y.; Bettencourt, J.; and Duvenaud, D

Chen, R. T.; Rubanova, Y.; Bettencourt, J.; and Duvenaud, D. K. 2018. Neural ordinary differential equations. NeurIPS

2018
[21]

Davies, B. 2018. Exploring chaos: Theory and experiment. CRC Press

2018
[22]

De, A.; and Chakrabarti, S. 2022. Neural estimation of submodular functions with applications to differentiable subset selection. NeurIPS

2022
[23]

Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In North American Chapter of the Association for Computational Linguistics

2019
[24]

Djolonga, J.; and Krause, A. 2017. Differentiable learning of submodular models. NeurIPS

2017
[25]

W.; and Bilmes, J

Dolhansky, B. W.; and Bilmes, J. A. 2016. Deep submodular functions: Definitions and learning. NeurIPS

2016
[26]

Feldman, V.; and Kothari, P. 2014. Learning coverage functions and private release of marginals. In COLT. PMLR

2014
[27]

Feng, Q.; Zhou, Y.; and Lan, R. 2016. Pairwise linear regression classification for image set retrieval. In CVPR

2016
[28]

J.; and Leary, C

Frostig, R.; Johnson, M. J.; and Leary, C. 2018. Compiling machine learning programs via high-level tracing. Systems for Machine Learning, 4(9)

2018
[29]

Giannone, G.; and Winther, O. 2022. Scha-vae: Hierarchical context aggregation for few-shot generation. In ICML

2022
[30]

A.; Kulesza, A.; Fox, E.; and Taskar, B

Gillenwater, J. A.; Kulesza, A.; Fox, E.; and Taskar, B. 2014. Expectation-maximization for learning determinantal point processes. NeurIPS

2014
[31]

Gionis, A.; Gunopulos, D.; and Koudas, N. 2001. Efficient and tumble similar set retrieval. In Proceedings of the 2001 ACM SIGMOD international conference on Management of data, 247--258

2001
[32]

Gomez-Rodriguez, M.; Leskovec, J.; and Krause, A. 2012. Inferring networks of diffusion and influence. ACM Transactions on Knowledge Discovery from Data (TKDD), 5(4): 1--37

2012
[33]

Gouk, H.; Frank, E.; Pfahringer, B.; and Cree, M. J. 2021. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 110: 393--416

2021
[34]

He, X.; Xu, K.; Kempe, D.; and Liu, Y. 2016. Learning influence functions from incomplete observations. NeurIPS

2016
[35]

Heek, J.; Levskaya, A.; Oliver, A.; Ritter, M.; Rondepierre, B.; Steiner, A.; and van Z ee, M. 2023. F lax: A neural network library and ecosystem for JAX

2023
[36]

R.; and Stiefel, E

Hestenes, M. R.; and Stiefel, E. 1952. Methods of conjugate gradients for solving linear systems. NBS Washington, DC

1952
[37]

Hinton, G.; Osindero, S.; Welling, M.; and Teh, Y.-W. 2006. Unsupervised discovery of nonlinear structure using contrastive backpropagation. Cognitive science, 30(4): 725--731

2006
[38]

Huang, Z.; Bai, S.; and Kolter, J. Z. 2021. ( Implicit )^2 : Implicit Layers for Implicit Representations. NeurIPS

2021
[39]

Karalias, N.; Robinson, J.; Loukas, A.; and Jegelka, S. 2022. Neural set function extensions: Learning with discrete functions in high dimensions. NeurIPS

2022
[40]

Kim, J.; Yoo, J.; Lee, J.; and Hong, S. 2021. SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data. In CVPR

2021
[41]

Kimura, M.; Shimizu, R.; Hirakawa, Y.; Goto, R.; and Saito, Y. 2024. On permutation-invariant neural networks. arXiv preprint arXiv:2403.17410

2024 arXiv
[42]

P.; and Ba, J

Kingma, D. P.; and Ba, J. 2015. Adam: A Method for Stochastic Optimization . In ICLR

2015
[43]

P.; and Welling, M

Kingma, D. P.; and Welling, M. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114

2013 arXiv
[44]

Kirsch, A.; Farquhar, S.; Atighehchian, P.; Jesson, A.; Branchaud-Charron, F.; and Gal, Y. 2023. Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learning. TMLR

2023
[45]

Kolter, Z.; Duvenaud, D.; and Johnson, M. 2020. Deep Implicit Layers - Neural ODEs, Deep Equilibirum Models, and Beyond . NeurIPS

2020
[46]

Kothawade, S.; Girdhar, J.; Lavania, C.; and Iyer, R. 2020. Deep submodular networks for extractive data summarization. arXiv preprint arXiv:2010.08593

2020 arXiv
[47]

G.; and Parks, H

Krantz, S. G.; and Parks, H. R. 2002. The implicit function theorem: history, theory, and applications. Springer Science & Business Media

2002
[48]

LeCun, Y.; Chopra, S.; Hadsell, R.; Ranzato, M.; and Huang, F. J. 2006. A tutorial on energy-based learning. Predicting structured data, 1(0)

2006
[49]

Lee, J.; Lee, Y.; Kim, J.; Kosiorek, A.; Choi, S.; and Teh, Y. W. 2019. Set transformer: A framework for attention-based permutation-invariant neural networks. In ICML

2019
[50]

L.; Chen, R

Li, X.; Wong, T.-K. L.; Chen, R. T.; and Duvenaud, D. 2020. Scalable gradients for stochastic differential equations. In AISTATS

2020
[51]

Liu, Z.; Li, Y.; Han, L.; Li, J.; Liu, J.; Zhao, Z.; Nie, W.; Liu, Y.; and Wang, R. 2015 a . PDB-wide collection of binding data: current status of the PDBbind database. Bioinformatics, 31(3): 405--412

2015
[52]

Liu, Z.; Luo, P.; Wang, X.; and Tang, X. 2015 b . Deep Learning Face Attributes in the Wild. In Proceedings of International Conference on Computer Vision (ICCV)

2015
[53]

Lorraine, J.; Vicol, P.; and Duvenaud, D. 2020. Optimizing millions of hyperparameters by implicit differentiation. In AISTATS

2020
[54]

Ma s kov \'a , M.; Zorek, M.; Pevn \`y , T.; and S m \' dl, V. 2024. Deep anomaly detection on set data: Survey and comparison. Pattern Recognition, 110381

2024
[55]

Mnih, A.; and Hinton, G. 2005. Learning nonlinear constraints with contrastive backpropagation. In IJCNN. IEEE

2005
[56]

Murphy, K. P. 2012. Machine learning: a probabilistic perspective. MIT press

2012
[57]

Nelsen, R. B. 2006. An Introduction to Copulas. New York, NY, USA: Springer, second edition

2006
[58]

Nikishin, E.; Abachi, R.; Agarwal, R.; and Bacon, P.-L. 2022. Control-oriented model-based reinforcement learning with implicit differentiation. In AAAI

2022
[59]

Ning, X.; Walters, M.; and Karypis, G. 2011. Improved machine learning models for predicting selective compounds. In Proceedings of the 2nd ACM Conference on Bioinformatics, Computational Biology and Biomedicine, 106--115

2011
[60]

Ou, Z.; Xu, T.; Su, Q.; Li, Y.; Zhao, P.; and Bian, Y. 2022. Learning Neural Set Functions Under the Optimal Subset Oracle . NeurIPS

2022
[61]

Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; and Lerer, A. 2017. Automatic differentiation in pytorch. In NeurIPS Autodiff Workshop

2017
[62]

Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; Vanderplas, J.; Passos, A.; Cournapeau, D.; Brucher, M.; Perrot, M.; and Duchesnay, E. 2011. Scikit-learn: Machine Learning in P ython. Journ...

2011
[63]

Rechnitzer, A. 2003. Fixed Points - Summary [Lecture Notes]. Dynamical Systems and Chaos — 620341

2003
[64]

Rudin, W. 1976. Principles of Mathematical Analysis. International series in pure and applied mathematics. McGraw-Hill. ISBN 9780070856134

1976
[65]

Saad, Y.; and Schultz, M. H. 1986. GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM Journal on scientific and statistical computing, 7(3): 856--869

1986
[66]

Saito, Y.; Nakamura, T.; Hachiya, H.; and Fukumizu, K. 2020. Exchangeable deep neural networks for set-to-set matching and learning. In ECCV

2020
[67]

B.; Konstan, J.; and Riedl, J

Schafer, J. B.; Konstan, J.; and Riedl, J. 1999. Recommender systems in e-commerce. In Proceedings of the 1st ACM conference on Electronic commerce, 158--166

1999
[68]

Singh, S.; Gupta, H.; Sharma, P.; and Sahi, S. 2024. Advances in Artificial Intelligence (AI)-assisted approaches in drug screening. Artificial Intelligence Chemistry, 2(1): 100039

2024
[69]

Sittoni, P.; and Tudisco, F. 2024. Subhomogeneous Deep Equilibrium Models. In ICML

2024
[70]

Sklar, A. 1973. Random variables, joint distribution functions, and copulas. Kybernetika, 9(6): 449--460

1973
[71]

Soelch, M.; Akhundov, A.; van der Smagt, P.; and Bayer, J. 2019. On Deep Set Learning and the Choice of Aggregations. In Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation: 28th International Conference on Artificial Neural Networks, M...

2019
[72]

Szegedy, C.; Zaremba, W.; Sutskever, I.; Bruna, J.; Erhan, D.; Goodfellow, I.; and Fergus, R. 2014. Intriguing properties of neural networks. In ICLR

2014
[73]

Tschiatschek, S.; Sahin, A.; and Krause, A. 2018. Differentiable submodular maximization. In IJCAI

2018
[74]

Virmaux, A.; and Scaman, K. 2018. Lipschitz regularity of deep neural networks: analysis and efficient estimation. NeurIPS

2018
[75]

Wagstaff, E.; Fuchs, F.; Engelcke, M.; Posner, I.; and Osborne, M. A. 2019. On the limitations of representing functions on sets. In ICML

2019
[76]

Wendler, C.; Amrollahi, A.; Seifert, B.; Krause, A.; and P \"u schel, M. 2021. Learning set functions that are sparse in non-orthogonal Fourier bases. In AAAI

2021
[77]

Wendler, C.; P \"u schel, M.; and Alistarh, D. 2019. Powerset convolutional neural networks. NeurIPS

2019
[78]

Winston, E.; and Kolter, J. Z. 2020. Monotone operator equilibrium networks. NeurIPS

2020
[79]

L.; and Gould, S

Xu, M.; Molloy, T. L.; and Gould, S. 2024. Revisiting implicit differentiation for learning problems in optimal control. NeurIPS

2024
[80]

R.; and Smola, A

Zaheer, M.; Kottur, S.; Ravanbakhsh, S.; Poczos, B.; Salakhutdinov, R. R.; and Smola, A. J. 2017. Deep sets. NeurIPS

2017
[81]

Zhang, L.; Tozzo, V.; Higgins, J.; and Ranganath, R. 2022 a . Set norm and equivariant skip connections: Putting the deep in deep sets. In ICML. PMLR

2022
[82]

Zhang, Y.; Hare, J.; and Prugel-Bennett, A. 2019. Deep set prediction networks. NeurIPS

2019
[83]

W.; Lacoste-Julien, S.; Burghouts, G

Zhang, Y.; Zhang, D. W.; Lacoste-Julien, S.; Burghouts, G. J.; and Snoek, C. G. 2022 b . Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation. In ICLR

2022
[84]

Zhao, H.; Jiang, L.; Fu, C.-W.; and Jia, J. 2019. PointWeb: Enhancing Local Neighborhood Features for Point Cloud Processing. In CVPR

2019
[85]

Zucchet, N.; and Sacramento, J. 2022. Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation. Neural Computation, 34(12): 2309--2346

2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Anderson, D. G. 1965. Iterative Procedures for Nonlinear Integral Equations. J. ACM, 12(4): 547–560

1965

[4] [4]

Apostol, T. 1974. Mathematical Analysis. Addison-Wesley series in mathematics. Addison-Wesley. ISBN 9780201002881

1974

[5] [5]

Arbel, M.; and Mairal, J. 2022. Amortized implicit differentiation for stochastic bilevel optimization. In ICLR

2022

[6] [6]

Bach, F. 2013. Learning with Submodular Functions: A Convex Optimization Perspective. Foundations and Trends in machine learning , 6(2-3): 145--373

work page 2013

[7] [7]

Z.; and Koltun, V

Bai, S.; Kolter, J. Z.; and Koltun, V. 2019. Deep Equilibrium Models . NeurIPS

work page 2019

[8] [8]

Balcan, M.-F.; and Harvey, N. J. 2018. Submodular functions: Learnability, structure, and optimization. SICOMP

work page 2018