RECORD
Maximum Entropy Flow Networks
Methodology · Submitted 12 Jan 2017
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Simulation results demonstrate the effectiveness of our method, and applications to finance and computer vision show the flexibility and accuracy of using maximum entropy flow networks.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth and invertible transformation that maps a simple distribution to the desired maximum entropy distribution. Doing so is nontrivial in that the objective being maximized (entropy) is a function of the density itself. By exploiting recent developments in normalizing flow networks, we cast the maximum entropy problem into a finite-dimensional constrained optimization, and solve the problem by combining stochastic optimization with the augmented Lagrangian method. Simulation results demonstrate the effectiveness of our method, and applications to finance and computer vision show the flexibility and accuracy of using maximum entropy flow networks.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth and invertible transformation that maps a simple distribution to the desired maximum entropy distribution. | method 0.61 | 0.93 | 0.11 | asserted 0.99 |
| s2 | Doing so is nontrivial in that the objective being maximized (entropy) is a function of the density itself. | background 0.51 | 0.27 | 0.03 | asserted 1.00 |
| s3 | By exploiting recent developments in normalizing flow networks, we cast the maximum entropy problem into a finite-dimensional constrained optimization, and solve the problem by combining stochastic optimization with the augmented Lagrangian method. | method 0.97 | 0.91 | 0.12 | asserted 1.00 |
| s4 | Simulation results demonstrate the effectiveness of our method, and applications to finance and computer vision show the flexibility and accuracy of using maximum entropy flow networks. | result 0.98 | 0.95 | 0.78 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth and invertible transformation that maps a simple distribution to the desired maximum entropy distribution. Doing so is nontrivial in that the objective being maximized (entropy) is a function of the density itself. By exploiting recent developments in normalizing flow networks, we cast the maximum entropy problem into a finite-dimensional constrained optimization, and solve the problem by combining stochastic optimization with the augmented Lagrangian method. Simulation results demonstrate the effectiveness of our method, and applications to finance and computer vision show the flexibility and accuracy of using maximum entropy flow networks.
Human review
How the literature uses it
5 arXiv papers cite this in 6 sentences
Papers citing it · 3 sentences without a confident label
Builds on it 1 paper
-
Similar to [ref] , we solve this constrained optimization problem but with an implicit distribution.
AR-DAE: Towards Unbiased Neural Entropy Gradient Estimation · Maximum entropy modeling · 0.81
Cites it as background 2 papers
-
This so-called maximum entropy principle [ref] has been used in reinforcement learning [ref] , natural language processing [ref] , normalizing flows [ref] , and computer vision [ref] successfully.
Batch norm with entropic regularization turns deterministic autoencoders into generative models · THE MAXIMUM ENTROPY PRINCIPLE AND REGULARIZER-FREE LATENTS · 0.89
-
[ref] use NFs to solve maximum entropy problems, rather than match a target distribution.
Neural Autoregressive Flows · Related work · 0.93
What it cites · 12 sentences without a confident label
Uses the method or tool of 4 papers
-
For the transformation family $ℱ$ we use the real NVP network structure proposed in [ref] (we adapted <https://github.com/taesung89/real-nvp>).
Density estimation using Real NVP · Modeling images of textures · 1.00
-
We follow [ref] (we adapted <https://github.com/ProofByConstruction/texture-networks>) to create a texture loss measure $T(): [0,1]^d →ℝ$,
Texture Networks: Feed-forward Synthesis of Textures and Stylized Images · Modeling images of textures · 1.00
-
Augmented Lagrangian iteration $k = 1,...,k_max$ SGD iteration $i = 1,...,i_max$ Sample $^(1),...,^(n) ∼p_0$, get transformed variables $^(i)_ϕ= f_ϕ(^(i)), i = 1,...,n$ Update $ϕ$ by descending its stochastic gradient (using e.g. ADADELTA [ref] ): [equation] Sample $^(1),...,^(ñ) ∼p_0$, get transformed variables $^(i)_ϕ= f_ϕ(^(i)), i = 1,...,ñ$ Update $λ_k+1 = λ_k + c_k 1/ñ ∑_i=1^ñ T( ^(i)_ϕ)$ Update $c_k+1≥c_k$ (see text for detail)
ADADELTA: An Adaptive Learning Rate Method · Algorithm · 1.00
Cites as background 3 papers
-
Following [ref] , we define a normalizing flow as the transformation of a probability density through a sequence of invertible mappings.
Variational Inference with Normalizing Flows · Normalizing flows · 0.88
-
Recently [ref] proposed a normalizing flow with convolutional, multiscale structure that is suitable for image modeling and has shown promise in density estimation for natural images.
Density estimation using Real NVP · Normalizing flows · 0.85
-
More recent works exploit more complicated features [ref] .
Texture Networks: Feed-forward Synthesis of Textures and Stylized Images · Modeling images of textures · 0.97
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.