Pith. sign in

REVIEW 4 major objections 5 minor 40 references

Latent Flow Transformer

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single flow-matching layer can replace a block of transformer layers and beat layer-skipping baselines on a 410M-parameter language model.

desk verdict Promising idea for compressing transformer blocks with a flow-matching layer, but the headline gain over the regression baseline is not compute-matched, so the central empirical claim is not yet convincing. read the letter →

arxiv 2505.14513 v1 pith:2QZACGKZ submitted 2025-05-20 cs.LG cs.AI

Yen-Chen Wu , Feng-Ting Liao , Meng-Hsi Chen , Pei-Chen Ho , Farhang Nabiei , Da-shan Shiu This is my paper
classification cs.LGcs.AI
keywords flowmatchingtransformercompressionknowledgedistillationlatentlayerWalkingRecouplingRatiooptimaltransportlanguagemodelefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a transformer's stacked layers can be viewed as discrete samples of a continuous latent flow, and that a contiguous block of layers in a pretrained language model can be replaced by a single transport operator learned with flow matching. On a 410M-parameter model, the authors show that replacing six middle layers with one such operator trained by flow matching yields lower divergence from the teacher's logits than skipping two layers outright. They then introduce Flow Walking, a three-step training scheme that preserves the one-to-one coupling between input and output hidden states, allowing twelve middle layers to be compressed into one operator with better teacher-student agreement than skipping three layers. The paper also introduces the Recoupling Ratio, an optimal-transport-based score that predicts which blocks of layers are safe to compress, and reports that middle layers are the most compressible. If this holds, depth compression of transformers becomes a continuous-transport problem rather than a discrete pruning one.

What carries the argument

The latent flow layer is the central object: a DiT-style transformer block, meaning a transformer layer conditioned on time through scale and shift, whose output minus input serves as an estimated velocity $u_\theta(x_t,t)$ for the hidden state. Flow matching trains it to follow straight-line paths $x_t=(1-t)x_0+t x_1$ between the block's input and output latents, and at inference the layer is unrolled over $k$ discrete time steps, making it structurally equivalent to a stack of transformer layers. Flow Walking is the second piece: it trains the same velocity estimator by integrating three steps and penalizing the distance between the final estimate and the true target, $\mathcal{L}_{\mathrm{FW}(3)} = \mathbb{E}\|\hat{x}_1 - x_1\|^2$, which keeps paired trajectories from crossing in latent space, a failure mode of standard flow matching. The third piece is the Recoupling Ratio $R = 1 - \mathbb{E}[\mathrm{Tr}(M)/O_M]$, where $M$ is the optimal-transport pairing between source and target latents; low $R$ predicts few flow crossings and good compressibility. Together these pieces support the claim that layer selection and training objective, not just capacity, decide whether a block of layers can be replaced by one flow operator.

What would settle it

Distill layers 0-6 of Pythia-410M, where the Recoupling Ratio is high, using the same Flow Walking recipe. If the resulting one-layer LFT does not beat skipping layers 0-6 on held-out KL divergence, then the claim that middle layers are uniquely flow-compressible is falsified.

Watch

Extended reading notes

Core claim

The central claim is that the hidden-state trajectory through the middle layers of a transformer can be learned as a velocity field, so that one flow layer reproduces the effect of many discrete layers. Concretely, the paper reports that on Pythia-410M, an LFT replacing layers 6-12 and trained with standard flow matching reaches a teacher-student KL divergence of 0.407 on held-out language modeling, beating the skip-two-layers baseline of 0.529. With the Flow Walking algorithm, the same architecture compressing layers 6-18, twelve layers into one, reaches a KL of 0.736, below the skip-three-layers baseline of 0.932 and below the single-layer regression baseline of 0.864. The authors interpret this as evidence that flow-based latent transport is a viable compression mechanism for autoregressive language models, not just for image generation.

Load-bearing premise

The result stands on the premise that a single learned velocity field, trained from three-step paths between hidden states, can reproduce what the original twelve transformer layers do to a token's representation at inference time, including at time steps it was never trained on.

Editorial extensions

If this is right

  • A 410M-parameter transformer can lose half of its 24 layers, with the removed block replaced by a single flow layer, while keeping end-to-end perplexity near 20 instead of exceeding 22 under the equivalent skip baseline.
  • Compression decisions can be guided before training: the Recoupling Ratio computed from 256 tokens ranks layer blocks by expected flow-matching difficulty, identifying the middle layers as the easiest to replace.
  • Inference cost becomes adjustable at runtime by choosing the number of discrete time steps $k$; the FW-trained layer performs best at $k=3$ and degrades only mildly when extrapolated to $k=8$.
  • Because an unrolled flow layer over $k$ steps is structurally a stack of transformer layers, the compressed model remains compatible with transformer serving infrastructure.
  • Standard flow matching helps but plateaus; Flow Walking resolves flow-path proximity enough to beat both layer-skipping and regression baselines for distant layer blocks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the flow view is right, the layer-depth versus capability trade-off in transformers becomes a continuous parameter: one could fine-tune or serve with a per-token step count, spending more integration steps on hard tokens, something discrete pruning cannot express.
  • The Recoupling Ratio suggests a cheap screening tool for other residual architectures: run it on hidden states of vision transformers or state-space models before deciding which blocks to distill into flow layers.
  • A testable extension is to replace early and late layers as well, after first optimizing the input and output layers to reduce flow crossings; the authors say this could cut layers by an order of magnitude, but results were not yet available.
  • Flow Walking's reliance on three fixed integration steps during training may become a bottleneck at larger scales; comparing it against a curriculum that gradually increases $k$ would show whether the multi-step extrapolation observed at $k=8$ survives.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes the Latent Flow Transformer (LFT), which replaces a contiguous block of transformer layers with a single learned transport operator trained via flow matching, and a Flow Walking (FW) algorithm intended to preserve input-output coupling when learning such latent transports. On Pythia-410M, the authors report that a flow-matched LFT replacing layers 6-12 achieves a logit KL of 0.407 versus 0.529 for skipping two layers, and that FW with k=3 replacing layers 6-18 achieves 0.736 versus 0.932 for skipping three layers and 0.864 for a single-layer regression baseline. A Recoupling Ratio based on Optimal Transport on 256 tokens is introduced to select which layer blocks to replace.

Significance. If the claims were fully supported, the work would be an interesting step toward structural compression of LLMs and toward connecting continuous transport models with transformer inference. The paper has concrete strengths: the code is released, the Recoupling Ratio is a simple and interpretable predictor, the experiments report both latent NMSE and logit KL at several integration step counts, and the observation that middle layers are more compressible is consistent with prior layer-pruning findings. However, the headline comparison is not capacity- or compute-matched, the results are single-run point estimates with no error bars, and the selected layer blocks are chosen via a small-sample heuristic and then evaluated only on those blocks. These issues currently prevent the empirical claims from being accepted at face value.

major comments (4)
  1. [§5.3, Table 1; §3.1, Algorithm 3] The comparison between LFT-FW (k=3) and the regression baseline is not compute- or capacity-matched. Per Eq. (5) and Algorithm 3, each of the three integration steps in FW uses the midpoint rule, requiring two forward passes of the DiT velocity network, so the k=3 result uses roughly six forward passes of a model that already carries additional scale/shift and MLP parameters, whereas the regression baseline is a single plain transformer layer evaluated once (Section 5.3). The reported KL gap of 0.736 vs. 0.864 could therefore be explained by recurrent depth and extra parameters rather than by the flow/walking formulation. Please add matched controls: the same DiT block unrolled for k steps and trained with endpoint MSE without time conditioning, and a regression model matched in parameters and FLOPs, and report parameter counts and inference FLOPs for every row of Table 1.
  2. [§5.2–5.3, Table 1] The empirical comparisons consist of single point estimates with no error bars, multiple seeds, or significance tests, and the blocks evaluated (6–12 and 6–18) were selected after inspecting the Recoupling Ratio computed on only 256 tokens (§5.2). This creates a selection bias and makes small differences such as FW k=1 (0.838) vs. regression (0.864) uninterpretable. Please report mean and standard deviation over at least three seeds, and either evaluate all candidate layer blocks or provide a validation protocol that does not use the test split in block selection.
  3. [§4, Eq. (8), Algorithms 2–3] The FW objective in Eq. (8) is endpoint supervision: it minimizes the squared error between the k-step unrolled output and x1, and it does not supervise intermediate velocities as in Eq. (3). Consequently the description of FW as learning a velocity field or as producing 'implicit velocity estimates' is not directly supported; the method is a multi-step recurrent regression objective. The paper should state this explicitly and test whether time-conditioning and the flow parameterization contribute at all, e.g., by comparing against a weight-shared recurrent regression block trained with the same Eq. (8) but without time embeddings.
  4. [§1, §5.3] The contribution bullet in §1 and the abstract claim 'significant capability recovery after fine-tuning,' but the experiments section contains no fine-tuning evaluation; Table 1 reports only distillation-time latent and logit KL on a held-out subset. Either add fine-tuned evaluations or remove/qualify the claim.
minor comments (5)
  1. [Abstract and §1] The text says LFT 'compresses 6 of 24 layers' for the layers 6–12 setting, but layers 6 through 12 inclusive constitute 7 layers; please reconcile the counting with the stated 25% parameter reduction.
  2. [§1 and §5.3] §1 says training uses 'only 108 training tokens,' while §5.3 says distillation uses 2.6 billion tokens from The Pile; this discrepancy needs to be resolved.
  3. [§4, Figure 3] The text says 'Panels (b) and (c) show FW with k=1, and k=2,' while the Figure 3 caption assigns FW k=1 and k=2 to panel (b) and k=3 to panel (c); the description and caption should agree.
  4. [§3.2, Eq. (6)] Equation (6) introduces the cost d(h_m^(i), h_n^(j)) without defining d; please define it formally (Euclidean distance is suggested in the prose but not in the equation).
  5. [§5.1, Figure 5] The notation KLx||x and the axis labels in Figure 5 are not fully defined; please spell out the distributions over which each KL is computed and clarify the relationship between KLx||x and KLP||Q in Table 1.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central FW result is supervised endpoint regression with held-out language-modeling evaluation; the only flagged item is a minor non-load-bearing same-group citation.

full rationale

The paper's derivation chain is self-contained rather than circular. The LFT replaces a contiguous teacher block with a learned transport operator, and the Flow Walking objective (Eq. 8) directly minimizes squared distance between the integrated final state and the teacher latent x1. That is supervised regression, not a definition that assumes the conclusion; the headline KLP||Q of 0.736 is computed from LM logits on a held-out subset of The Pile, an evaluation that is not part of the training loss. The Recoupling Ratio is used as a layer-selection heuristic; its claim that low R predicts trainability is definitionally tied to OT optimality of the original pairing, but the paper validates the selection by downstream LM KL rather than by R itself, so the selection is not a fitted parameter masquerading as a prediction. The regression baseline comparison is not capacity/compute-matched (k=3 FW evaluates a time-conditioned DiT block three times with midpoint updates, versus a single plain layer for regression), but this is an experimental-fairness concern, not circularity. The only self-citation is [4] from the same group, used to motivate surrogate transport models and to justify avoiding reflow; it is not load-bearing because the external [35] also supports the motivation and the FW algorithm and experiments stand independently. Hence no step reduces by construction; score 2 reflects only the minor non-load-bearing self-citation.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the flow-matchability of Pythia's hidden states. Several free hyperparameters (k, alpha, OT sample size, chosen layer block) are set from data or intuition, and the method depends on domain assumptions about straight-line interpolation and the representational capacity of the velocity estimator.

free parameters (4)
  • number of integration steps k in Flow Walking = 3
    Chosen empirically in Section 4 ('We empirically choose ... using k=3') because k=2 with random t1 did not generalize to inference-time k>2.
  • hybrid regularization weight alpha (Eq. 9) = 0.001
    Used in the toy demo in Figure 3(d); no sensitivity analysis is provided.
  • OT batch size for Recoupling Ratio = 256 tokens
    Section 5.2: 'Sampling just 256 tokens from the first portion of the Pile' to compute the OT matrix; generalization of this small sample is assumed.
  • selected layer block = layers 6-12 and 6-18
    Chosen based on Recoupling Ratio on the same dataset; no comparison to arbitrary layer blocks is shown in the paper.
assumptions (4)
  • domain assumption Straight-line constant-speed interpolation between teacher hidden states is a learnable target for flow matching (Eq. 3 with x_t=(1-t)x0 + t x1).
    Invoked in Sections 2.2 and 3.3; if the latent trajectories are highly nonlinear or discontinuous, flow matching with straight paths cannot capture them.
  • ad hoc to paper Optimal Transport recoupling on a 256-token sample reflects full-data flow-crossing difficulty.
    Section 5.2 uses this small-sample OT matrix to predict which layers to compress; no convergence or stability analysis is given.
  • standard math The midpoint/solver discretization (Eq. 5) is stable enough for training and inference of the latent flow layer.
    Used in Algorithm 2 and Eq. 5; no error analysis or step-size sensitivity is provided.
  • domain assumption Pythia-410M latent spaces are approximately Euclidean and continuous so that straight-line flow is meaningful.
    This underpins the entire flow matching approach; the paper does not test for deviations.
invented entities (1)
  • Latent Flow Layer
    purpose: Replaces a contiguous block of transformer layers with a single learned transport operator trained by flow matching or Flow Walking.
    Demonstrated only on Pythia-410M in this paper (Table 1); no external replication or independent benchmark yet, so it lacks an independent falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Latent Flow Transformer." pith.science (2026). https://pith.science/paper/2QZACGKZ

@misc{pith2026250514513,
  author       = {Pith},
  title        = {Pith review of: Latent Flow Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2QZACGKZ}},
  note         = {Machine review of arXiv:2505.14513}
}
read the original abstract

Transformers, the standard implementation for large language models (LLMs), typically consist of tens to hundreds of discrete layers. While more layers can lead to better performance, this approach has been challenged as far from efficient, especially given the superiority of continuous layers demonstrated by diffusion and flow-based models for image generation. We propose the Latent Flow Transformer (LFT), which replaces a block of layers with a single learned transport operator trained via flow matching, offering significant compression while maintaining compatibility with the original architecture. Additionally, we address the limitations of existing flow-based methods in \textit{preserving coupling} by introducing the Flow Walking (FW) algorithm. On the Pythia-410M model, LFT trained with flow matching compresses 6 of 24 layers and outperforms directly skipping 2 layers (KL Divergence of LM logits at 0.407 vs. 0.529), demonstrating the feasibility of this design. When trained with FW, LFT further distills 12 layers into one while reducing the KL to 0.736 surpassing that from skipping 3 layers (0.932), significantly narrowing the gap between autoregressive and flow-based generation paradigms.

Figures

Figures reproduced from arXiv: 2505.14513 by the authors.

Figure 1
Figure 1. Velocity field estimator. (a) The DiT block, mapping an input hidden state to an output hidden state conditioned on time t. (b) Velocity field estimator derived from the DiT block, mapping an input state xt and time t to a velocity vt. (c) Velocity field estimator for Pythia [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Static structure of an unrolled LFT. We highlight only the latent flow layer. The simple reconstruction rule of Equation 4 is assumed. (a) LFT based on Pythia with single step reconstruction. (b) LFT based on Pythia with two step reconstruction. Algorithm 1 Learning the velocity field via standard flow matching [24] 1: D is a set of training data, LLMteacher(d, m) is the latent of LLM at layer m over data d, AdamW i… view at source ↗
Figure 3
Figure 3. Toy trajectories for paired data. Faded lines show the ground-truth trajectories of paired points, while solid curves depict predictions from various flow-matching algorithms. (a) Standard flow matching fails to maintain pairwise correspondence. (b) FW results for k = 1 (top) and k = 2 (bottom): using fewer integration steps leads to poor trajectory estimates. (c) FW with k = 3: projected view (top) and full traject… view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Inference perfor￾mance (KLx||xˆ and NMSE) vs number of discrete points k. We report results of LFT-SFM and LFT-FW with latent flow layer replacing layer 6-18 of Pythia-410m. To address the challenge of crossing trajectory in 2.3 while to enable the unrolling of latent …
Figure 6
Figure 6. Figure 6: Optimal Transport Matrices and Recoupling Ratio [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

40 extracted references · 20 canonical work pages

  1. [1]

    Building normalizing flows with stochastic interpolants.arXiv preprint arXiv:2209.15571, 2022

    Michael S Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants.arXiv preprint arXiv:2209.15571, 2022

    arXiv 2022

  2. [2]

    Neural ordinary differential equations.Advances in neural information processing systems, 31, 2018

    Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations.Advances in neural information processing systems, 31, 2018

    2018

  3. [3]

    Contiformer: Continuous-time transformer for irregular time series modeling.Advances in Neural Information Processing Systems, 36:47143–47175, 2023

    Yuqi Chen, Kan Ren, Yansen Wang, Yuchen Fang, Weiwei Sun, and Dongsheng Li. Contiformer: Continuous-time transformer for irregular time series modeling.Advances in Neural Information Processing Systems, 36:47143–47175, 2023

    2023

  4. [4]

    Image generation with shortest path diffusion, 2023

    Ayan Das, Stathi Fotiadis, Anil Batra, Farhang Nabiei, FengTing Liao, Sattar Vakili, Da shan Shiu, and Alberto Bernacchia. Image generation with shortest path diffusion, 2023

    work page 2023

  5. [5]

    Flowing through layers: A continuous dynamical systems perspective on transformers, 2025

    Jacob Fein-Ashley. Flowing through layers: A continuous dynamical systems perspective on transformers, 2025

    work page 2025

  6. [6]

    One Step Diffusion via Shortcut Models, October 2024

    Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One Step Diffusion via Shortcut Models, October 2024. arXiv:2410.12557 [cs]

    arXiv 2024

  7. [7]

    Scaling up test-time compute with latent reasoning: A recurrent depth approach.arXiv preprint arXiv:2502.05171, 2025

    Jonas Geiping, Sean McLeish, Neel Jain, John Kirchenbauer, Siddharth Singh, Brian R Bartold- son, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. Scaling up test-time compute with latent reasoning: A recurrent depth approach.arXiv preprint arXiv:2502.05171, 2025

    arXiv 2025

  8. [8]

    Compressing bert: Studying the effects of weight pruning on transfer learning

    Mitch Gordon, Kevin Duh, and Nicholas Andrews. Compressing bert: Studying the effects of weight pruning on transfer learning. InProceedings of the 5th Workshop on Representation Learning for NLP, 2020

    work page 2020

Show all 40 references
  1. [9]

    Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Daniel A. Roberts. The unreasonable ineffectiveness of the deeper layers, 2025

    2025

  2. [10]

    Calmflow: V olterra flow matching using causal language models, 2024

    Sizhuang He, Daniel Levine, Ivan Vrkic, Marco Francesco Bressana, David Zhang, Syed Asad Rizvi, Yangtian Zhang, Emanuele Zappala, and David van Dijk. Calmflow: V olterra flow matching using causal language models, 2024

    2024

  3. [11]

    Flow matching for conditional text generation in a few sampling steps

    Vincent Hu, Di Wu, Yuki Asano, Pascal Mettes, Basura Fernando, Björn Ommer, and Cees Snoek. Flow matching for conditional text generation in a few sampling steps. In Yvette Graham and Matthew Purver, editors,Proceedings of the 18th Conference of the European Chapter of the Ass...

    2024

  4. [12]

    Editing models with task arithmetic

    Gabriel Ilharco, Mitchell Wortsman, et al. Editing models with task arithmetic. InICML, 2023

    2023

  5. [13]

    Tinybert: Distilling bert for natural language understanding

    Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. Tinybert: Distilling bert for natural language understanding. InFindings of EMNLP, 2019

    2019

  6. [14]

    On neural differential equations.arXiv preprint arXiv:2202.02435, 2022

    Patrick Kidger. On neural differential equations.arXiv preprint arXiv:2202.02435, 2022

    2022 arXiv

  7. [15]

    Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling.arXiv preprint arXiv:2312.15166, 2023

    Dahyun Kim, Chanjun Park, Sanghoon Kim, Wonsung Lee, Wonho Song, Yunsu Kim, Hyeon- woo Kim, Yungi Kim, Hyeonju Lee, Jihoo Kim, et al. Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling.arXiv preprint arXiv:2312.15166, 2023

    arXiv

  8. [16]

    Simulation-Free Training of Neural ODEs on Paired Data, October 2024

    Semin Kim, Jaehoon Yoo, Jinwoo Kim, Yeonwoo Cha, Saehoon Kim, and Seunghoon Hong. Simulation-Free Training of Neural ODEs on Paired Data, October 2024. arXiv:2410.22918 [cs]

    2024 arXiv

  9. [17]

    Simulation-free training of neural odes on paired data.Advances in Neural Information Processing Systems, 37:60212–60236, 2024

    Semin Kim, Jaehoon Yoo, Jinwoo Kim, Yeonwoo Cha, Saehoon Kim, and Seunghoon Hong. Simulation-free training of neural odes on paired data.Advances in Neural Information Processing Systems, 37:60212–60236, 2024

    2024

  10. [18]

    ALBERT: A Lite BERT for Self-supervised Learning of Language Representations, February 2020

    Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations, February 2020. arXiv:1909.11942. 10

    2020 arXiv

  11. [19]

    ODE transformer: An ordinary differential equation-inspired model for sequence generation

    Bei Li, Quan Du, Tao Zhou, Yi Jing, Shuhan Zhou, Xin Zeng, Tong Xiao, JingBo Zhu, Xuebo Liu, and Min Zhang. ODE transformer: An ordinary differential equation-inspired model for sequence generation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedi...

    2022

  12. [20]

    Diffusion via conditional transport.arXiv preprint arXiv:2305.08891, 2023

    Xiangning Li, Shuang Li, Zhengyang Zhou, et al. Diffusion via conditional transport.arXiv preprint arXiv:2305.08891, 2023

    2023 arXiv

  13. [21]

    Yangyang Li, Qin Huang, Xuan Pei, Yanqiao Chen, Licheng Jiao, and Ronghua Shang. Cross- layer attention network for small object detection in remote sensing imagery.IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 14:2148–2161, 2020

    2020

  14. [22]

    Text generation with diffusion language models: A pre-training approach with continuous paragraph denoise

    Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Nan Duan, and Weizhu Chen. Text generation with diffusion language models: A pre-training approach with continuous paragraph denoise. InInternational Conference on Machine Learning, pages 21051–21064. PMLR, 2023

    2023

  15. [23]

    Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky T. Q. Chen, David Lopez-Paz, Heli Ben-Hamu, and Itai Gat. Flow matching guide and code, 2024

    2024

  16. [24]

    Flow matching for generative modeling

    Yotam Lipman, Talia Ringer Cohen, Yujia Zhang, Ehsan Hajiramezanali, et al. Flow matching for generative modeling. InNeurIPS, 2022

    2022

  17. [25]

    Rectified flow: Learning deep generative models using optimal transport.arXiv preprint arXiv:2209.14687, 2022

    Minghao Liu, Belinda Tzen, Maxim Raginsky, et al. Rectified flow: Learning deep generative models using optimal transport.arXiv preprint arXiv:2209.14687, 2022

    2022 arXiv

  18. [27]

    Reassessing layer pruning in llms: New insights and methods.arXiv preprint arXiv:2411.15558, 2024

    Yao Lu, Hao Cheng, Yujie Fang, Zeyu Wang, Jiaheng Wei, Dongwei Xu, Qi Xuan, Xiaoniu Yang, and Zhaowei Zhu. Reassessing layer pruning in llms: New insights and methods.arXiv preprint arXiv:2411.15558, 2024

    2024 arXiv

  19. [28]

    Merging models with fisher-weighted averaging

    Michael Matena and Colin Raffel. Merging models with fisher-weighted averaging. InarXiv preprint arXiv:2206.14841, 2022

    2022 arXiv

  20. [29]

    Normalizing flows for probabilistic modeling and inference.Journal of Machine Learning Research, 22(57):1–64, 2021

    George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference.Journal of Machine Learning Research, 22(57):1–64, 2021

    2021

  21. [30]

    Dogyun Park, Sojin Lee, Sihyeon Kim, Taehoon Lee, Youngjoon Hong, and Hyunwoo J. Kim. Constant acceleration flow, 2024

    2024

  22. [31]

    Scalable diffusion models with transformers, 2023

    William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023

    2023

  23. [32]

    Computational optimal transport: With applications to data science.F oundations and Trends® in Machine Learning, 11(5-6):355–607, 2019

    Gabriel Peyré, Marco Cuturi, et al. Computational optimal transport: With applications to data science.F oundations and Trends® in Machine Learning, 11(5-6):355–607, 2019

    2019

  24. [33]

    On the effect of dropping layers of pre-trained transformer models.Computer Speech & Language, 77:101429, January 2023

    Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. On the effect of dropping layers of pre-trained transformer models.Computer Speech & Language, 77:101429, January 2023

    2023

  25. [34]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. InarXiv preprint arXiv:1910.01108, 2019

    1910 arXiv

  26. [35]

    Consistency models, 2023

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models, 2023

    2023

  27. [36]

    Patient knowledge distillation for bert model compression, 2019

    Siqi Sun, Yu Cheng, Zhe Gan, and Jingjing Liu. Patient knowledge distillation for bert model compression, 2019. 11

    2019

  28. [37]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in neural information processing systems, 2017

    2017

  29. [38]

    Latent space chain-of- embedding enables output-free llm self-evaluation.arXiv preprint arXiv:2410.13640, 2024

    Yiming Wang, Pei Zhang, Baosong Yang, Derek F Wong, and Rui Wang. Latent space chain-of- embedding enables output-free llm self-evaluation.arXiv preprint arXiv:2410.13640, 2024

    2024 arXiv

  30. [39]

    Ar-diffusion: auto-regressive diffusion model for text generation

    Tong Wu, Zhihao Fan, Xiao Liu, Hai-Tao Zheng, Yeyun Gong, Yelong Shen, Jian Jiao, Juntao Li, Zhongyu Wei, Jian Guo, Nan Duan, and Weizhu Chen. Ar-diffusion: auto-regressive diffusion model for text generation. InProceedings of the 37th International Conference on Neural Inform...

    2023

  31. [40]

    A survey on knowledge distillation of large language models, 2024

    Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A survey on knowledge distillation of large language models, 2024

    2024

  32. [41]

    Prune once for all: Sparse pre-trained language models

    Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. Prune once for all: Sparse pre-trained language models. InarXiv preprint arXiv:2102.07436, 2021. 12

    2021 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.