Pith. sign in

REVIEW 4 major objections 7 minor 53 references

On the Effectiveness of Low-Rank Matrix Factorization for LSTM Model Compression

T0 review · 4 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Low-rank matrix factorization compresses LSTM language models by about 60% with a loss of roughly one perplexity point.

desk verdict Useful empirical benchmark for LSTM/ELMo compression, but the nuclear-norm explanation is a post-hoc story contradicted by the paper's own SQuAD results. read the letter →

arxiv 1908.09982 v1 pith:7ZOXSGF2 submitted 2019-08-27 cs.CL

Genta Indra Winata , Andrea Madotto , Jamin Shin , Elham J. Barezi , Pascale Fung This is my paper · ORCID
classification cs.CL
keywords LSTMcompressionlow-rankmatrixfactorizationtruncatedSVDsemi-NMFnuclearnormlanguagemodelingELMomodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether low-rank matrix factorization can compress the recurrence matrices of an LSTM without retraining, and how much damage different kinds of compression do. Working from a 24-million-parameter LSTM language model, the authors show that replacing the hidden-to-hidden matrix $W_h$ with a rank-10 SVD or Semi-NMF factor pair cuts the model to about 9 million parameters (roughly 60 percent reduction) at the cost of about one perplexity point on Penn Treebank, and that fine-tuning the compressed model can recover or slightly beat the original. The paper's central explanatory claim is that the hidden-to-hidden (multiplicative) recurrence is inherently more compressible than the input-to-hidden (additive) recurrence, and that this asymmetry is visible in matrix norms: $W_h$ has a lower nuclear norm, indicating lower intrinsic rank. It also argues that matrix factorization generally beats magnitude pruning except when the target matrix is already sparse. If these claims hold, practitioners can compress deployed LSTMs cheaply and predict which weight matrix to attack first.

What carries the argument

The machinery is low-rank matrix factorization applied separately to the two LSTM recurrence matrices, $W_i$ and $W_h$, each of which stacks the four gates' input and hidden weights. Replacing $W$ with $U V$, where $U$ is $m \times r$ and $V$ is $r \times n$, cuts parameter count from $mn$ to $r(m+n)$; the paper uses truncated SVD, which takes the top $r$ singular values and vectors, and Semi-NMF, which factors $W$ as $U V$ with $V$ nonnegative, at ranks $r=10,100,200,300,400$. The explanatory instrument is the nuclear norm, the sum of singular values, used as a proxy for intrinsic matrix rank: the paper observes $W_h$ has the lower nuclear norm and argues this is why it compresses more gracefully. The factorized cell replaces each $W x$ term with $U(V x)$, so only the factor matrices remain in the computation graph.

What would settle it

Train a new LSTM on a task not in the paper, measure the nuclear norms of $W_i$ and $W_h$, and compress both to the same low rank with truncated SVD; the paper's explanation predicts the matrix with the lower nuclear norm should compress with less perplexity or accuracy loss, and if compressing $W_h$ is worse despite a lower nuclear norm, or $W_i$ is better despite a higher one, the correlation claim fails.

Watch

Extended reading notes

Core claim

The central discovery is that low-rank matrix factorization post-processing compresses LSTM recurrences far below their nominal size while keeping predictive performance close: on PTB, factorizing $W_h$ to rank 10 with SVD or Semi-NMF gives 9M parameters and perplexity in the high 70s compared with 58.3 for the 24M baseline, and with fine-tuning the compressed models reach about 58, slightly better than the baseline, with a 2.13x speedup. The authors find compressing $W_h$, the matrix multiplying the previous hidden state, which they call multiplicative recurrence, works notably better than compressing $W_i$, the input matrix or additive recurrence, and they explain this by nuclear norm: $W_h$'s nuclear norm is consistently lower, indicating that $W_h$ is closer to a low-rank matrix and therefore loses less when projected into a low-dimensional factor space. A second finding is that matrix factorization is generally a better compression strategy than magnitude pruning, with pruning winning only when the original matrix is already sparse. The paper also shows the same method compresses pretrained ELMo biLSTM layers for SNLI and SST-5 with little loss, while SQuAD results reverse the $W_i$/$W_h$ ordering, a caveat the norm analysis does not fully explain.

Load-bearing premise

The load-bearing premise is that a lower nuclear norm reliably marks a matrix as safely compressible, so the hidden-to-hidden recurrence is generally compressible before the input-to-hidden recurrence; this is inferred from the same language-model experiments it is used to explain, and the SQuAD results already go the other way.

Editorial extensions

If this is right

  • An LSTM language model can be cut from 24M to 9M parameters with SVD or Semi-NMF and lose only about one perplexity point, without retraining.
  • Fine-tuning the compressed model at rank 10 can recover the baseline's performance and give a roughly 2x inference speedup.
  • In language modeling, compressing the hidden-to-hidden matrix $W_h$ should be preferred over compressing the input matrix $W_i$, and the nuclear norm of a trained matrix estimates how safely it can be compressed.
  • For dense low-rank matrices, low-rank factorization beats magnitude pruning; for already-sparse matrices, pruning is the better choice.
  • The same factorization transfers to pretrained biLSTM contextualizers: ELMo layers can be compressed by over 10M parameters with near-baseline F1 or accuracy on several downstream tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A pre-compression nuclear-norm scan could serve as a cheap decision rule for which matrix to factorize in any trained recurrent model, since the paper's correlation predicts the lower-nuclear-norm matrix should compress first.
  • The SQuAD reversal suggests the additive-versus-multiplicative ordering is task-dependent; the durable generalization may be 'compress the lower-nuclear-norm matrix' rather than 'compress $W_h$', and that stronger claim still needs testing.
  • The factor-pair replacement transfers to other recurrent cells such as GRUs, whose analogous input and hidden matrices could be ranked by nuclear norm, so the method is not LSTM-specific.
  • Because fine-tuning already reaches or passes the baseline, layering the factorization with quantization or knowledge distillation could push compression further without changing the architecture.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper studies post-hoc compression of LSTM weight matrices Wi and Wh using truncated SVD, Semi-NMF, and magnitude pruning, applied in two settings: AWD-LSTM language models on PTB and WikiText-2, and the ELMo biLSTM layers used in BiDAF/SQuAD, ESIM/SNLI, and BCN/SST-5. The central empirical result is that low-rank factorization can reduce the 24M-parameter AWD-LSTM to 9M parameters with roughly one perplexity point of loss after fine-tuning, and that compressing Wh (the hidden-to-hidden recurrence) degrades performance less than compressing Wi (the input-to-hidden recurrence) on the language-modeling tasks. The authors further propose a norm-based explanation: Wh has a lower nuclear norm than Wi, which they interpret as Wh having lower intrinsic rank. The paper also claims 1.5x and 2x inference-time speedups and introduces an efficiency measure E(r) to compare methods across compression ratios.

Significance. If the empirical findings hold, the paper provides a useful benchmark for low-rank LSTM compression across several NLP tasks, including compression of a widely used pretrained contextual representation model (ELMo). The experimental scope is broad: five datasets, two factorization methods, a pruning baseline, two ranks, and fine-tuning in the PTB setting, with the AWD-LSTM baseline reproduced. The main value is the comparative evidence that matrix factorization is often preferable to magnitude pruning at high compression ratios and that the input and hidden recurrence matrices respond very differently to compression. However, the paper's headline generalization — that additive recurrence is more important than multiplicative recurrence, explained by nuclear norms — is not established by the reported evidence. The norm argument is dimensionally unnormalized, is formed and 'confirmed' on the same experiments, and is contradicted by the SQuAD results. The speedup claims are also not backed by timing measurements. These are load-bearing issues for the paper's central claims, not merely presentational.

major comments (4)
  1. [Section 4.4, Figure 2 panels (e)-(f), and Table 3] The nuclear-norm explanation is not valid as stated and does not support the claim that Wh is inherently lower-rank than Wi. First, Wi and Wh have different shapes (Section 3.1: Wi ∈ R^{4·ninp × 4·ndim}, Wh ∈ R^{4·ndim × 4·ndim}), so raw nuclear norms are not comparable across the two matrices; a lower value may simply reflect different dimensions or overall scale. Second, the nuclear norm is the sum of singular values, not an approximation of rank (rank is the number of nonzero singular values), so 'having a lower nuclear norm is often an indicator of low-rank' is a much weaker and scale-dependent statement than the paper needs. Third, the hypothesis is generated and confirmed on the same PTB/WT2/ELMo experiments, with no held-out test or out-of-sample prediction. Fourth, Table 3 directly contradicts the proposed mechanism: at r=400 on SQuAD, compressing Wi is consistently at least as good as compressing Wh (e.g., Semi-NMF F1 81.78 for Wi vs 81.55 for Wh; SVD 81.78 vs 81.62) even though Section 4.4 reports that Wh has the lower nuclear norm in ELMo. To keep the explanatory claim, the authors should normalize the norm measure (e.g., stable rank ||W||_F²/||W||_2², or nuclear norm divided by dimension/Frobenius norm), test whether the rank/compression-loss correlation holds on held-out matrices or tasks, and reconcile the SQuAD rows. Without this, the norm analysis should be presented only as a post-hoc observation, not as an explanation.
  2. [Abstract, Section 1 contribution bullets, Section 4.2] The paper claims inference-time speedups of 1.5x and 2x (and '2.13x speedup' in Section 4.2), but no wall-clock timing measurements are reported anywhere in the manuscript. Footnote 2 gives the CPU model, and Section 4.1 says the authors 'report ... inference time,' but Tables 1-3 contain only parameter counts, perplexity/accuracy/F1, and E(r). Parameter reduction does not imply wall-clock speedup, especially for factorized recurrent cells, which involve two matrix multiplications and additional operations, and the speedup claim is a central advertised contribution. The authors should either provide measured inference times (with hardware, batch size, sequence length, and implementation details) or rephrase the contribution in terms of parameter and FLOP reduction.
  3. [Section 4, Tables 1-3] The paper states that all experiments are 'averaged over 5 runs,' but no standard deviations, confidence intervals, or significance tests are reported for any table. Many of the comparisons that support the ranking of methods are numerically close (e.g., PTB fine-tuned r=400 perplexities 57.19-58.18; SST-5 r=400 accuracies 53.93-54.66), and the claim that one compression target 'works notably better' than another requires variance information. The parenthetical '(avg)' in Table 3 is not a substitute. Please report means with standard deviations over the 5 runs and, where possible, a paired significance test between the best and runner-up configurations.
  4. [Section 4.3] The experimental protocol for the ELMo downstream tasks is ambiguous. The text says the authors 'train publicly available state-of-the-art models' with ELMo and then 'low-rank factorize the pre-trained ELMo layer only,' but it is not clear whether the downstream model is trained first and the ELMo layer is compressed afterward (post-hoc compression of a deployed model), or whether the ELMo weights are factorized before downstream training and then kept fixed. These are different regimes: post-hoc compression measures the robustness of an already-trained model, whereas pre-training factorization measures the ability to learn with the compressed representation. This distinction matters for comparing the compressed results with the uncompressed baselines and with prior compression work. Please state the exact chronology of factorization, downstream training, and any fine-tuning.
minor comments (7)
  1. [Section 2 (Related Work)] Several citations are incomplete or in an unusual authorless format, such as '(2008; 2013; 2014; 2014)', '(2010)', '(2016a)', and '(2017)'. These should be replaced with proper author-year citations so that the reader can identify the referenced works.
  2. [Equation (1)] The matrix notation in Equation (1) is malformed: '(Wi Wh)' should be presented as a block-row matrix [Wi, Wh] that multiplies (xt; ht−1), with dimensions given for the block matrices. The current rendering obscures the structure of the LSTM update.
  3. [Section 4.1, Equation (10)] The efficiency measure E(r) has a typo ('the ration') and the definition of R(a,b) is ambiguous for perplexity, where the paper reports using R(M_r, M) instead of R(M, M_r). Please define the formula cleanly for both higher-is-better and lower-is-better metrics and verify that the reported E(r) values in the tables are consistent with that definition.
  4. [Section 3.2] The claim that pruning 'requires a special implementation of sparse matrix multiplication' is overstated: magnitude pruning is often implemented with a dense binary mask and does not inherently require sparse kernels. The comparison would be clearer if the paper stated whether pruning was evaluated with dense or sparse multiplication.
  5. [Section 4.4] The statement 'Nuclear norm approximates the matrix rank' is misleading. The nuclear norm is the convex envelope of rank on the unit spectral-norm ball, which is not the same as approximating rank. Please rephrase to avoid conveying that a low nuclear norm directly implies a low rank in the matrix.
  6. [Figures 3 and 4] The heatmaps are discussed as showing a 'salient red line' in Wh, but no color scale or axis labels are provided, and the figures are hard to interpret in grayscale. Please add a color bar and describe the visualization protocol.
  7. [Throughout] The paper contains several typographical errors ('intepretations', 'Non-Matrix Factorization' in Related Work, repeated 'modifed'), and the reference list has entries with only years and no author names. A thorough proofreading pass is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation; the compression benchmarks are self-contained and the norm analysis is post-hoc rather than definitionally forced.

full rationale

The paper's central results in Tables 1-3 are empirical comparisons against reproduced external baselines (AWD-LSTM, TT-LSTM, BiDAF/ESIM/BCN with ELMo) using standard low-rank factorization and pruning methods. No parameter is fitted to a subset of the data and then reported as a prediction of a closely related quantity; the perplexity and F1 outcomes are directly measured after compression. The observation that compressing Wh works better than compressing Wi is an empirical finding, not a consequence of the definition of the factorization. Section 4.4 is a post-hoc explanatory narrative: the authors compute L1 and nuclear norms and interpret Wh's lower nuclear norm as evidence of lower intrinsic rank, then link this to the observed compression results. This explanation is formed and assessed on the same experiments, and the SQuAD rows in Table 3 are not fully reconciled with the generalization, so the explanatory claim is evidentially weak. However, that is a correctness or generalizability concern, not circularity. The norms do not by construction determine the measured performance, and no self-citation, imported uniqueness theorem, or ansatz citation is load-bearing. Accordingly, no circular step is exhibited, and the score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities or physical postulates. The central claims rest on standard linear algebra (SVD, Semi-NMF) plus domain assumptions about LSTM behavior and the meaning of matrix norms. The only tuned hyperparameter explicitly varied is the rank r; fine-tuning details are underspecified.

free parameters (2)
  • Factorization rank r = 10 and 400
    The rank r is a chosen hyperparameter controlling the compression level; the paper shows results only for r=10 and r=400 and performance depends strongly on this choice.
  • Fine-tuning epochs = unspecified
    Section 4.2 says the compressed PTB model is 'fine-tuned for several epochs' but the exact number, learning rate, and schedule are not provided, making the fine-tuning results difficult to reproduce.
assumptions (4)
  • domain assumption Replacing LSTM weight matrices W_i and W_h with low-rank products UV preserves the functional behavior of the gates well enough for compression.
    The entire method assumes that a low-rank approximation of the weight matrices does not destroy the recurrent computation; this is the standard premise of all factorization-based compression, introduced in Section 3.2.
  • domain assumption Nuclear norm is a reliable proxy for intrinsic low-rankness and thus for compressibility.
    Section 4.4 states 'having a lower nuclear norm is often an indicator of low-rank in a matrix; hence, we hypothesize that Wh is inherently low-rank than Wi', but the paper does not validate this relationship against reconstruction error or performance on held-out data.
  • domain assumption W_i corresponds to 'additive recurrence' and W_h to 'multiplicative recurrence' following the terminology of Levy et al. 2018.
    Section 3.1 assigns these labels to the input and hidden matrices; the naming is taken from the cited literature and is not itself established in this paper.
  • domain assumption The E(r) efficiency measure fairly compares methods with different parameter reductions.
    Section 4.1 defines E(r) as the ratio of relative performance loss to relative parameter loss, implicitly assuming these two ratios can be meaningfully combined on a single scale across methods and datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Effectiveness of Low-Rank Matrix Factorization for LSTM Model Compression." pith.science (2026). https://pith.science/paper/7ZOXSGF2

@misc{pith2026190809982,
  author       = {Pith},
  title        = {Pith review of: On the Effectiveness of Low-Rank Matrix Factorization for LSTM Model Compression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7ZOXSGF2}},
  note         = {Machine review of arXiv:1908.09982}
}
read the original abstract

Despite their ubiquity in NLP tasks, Long Short-Term Memory (LSTM) networks suffer from computational inefficiencies caused by inherent unparallelizable recurrences, which further aggravates as LSTMs require more parameters for larger memory capacity. In this paper, we propose to apply low-rank matrix factorization (MF) algorithms to different recurrences in LSTMs, and explore the effectiveness on different NLP tasks and model components. We discover that additive recurrence is more important than multiplicative recurrence, and explain this by identifying meaningful correlations between matrix norms and compression performance. We compare our approach across two settings: 1) compressing core LSTM recurrences in language models, 2) compressing biLSTM layers of ELMo evaluated in three downstream NLP tasks.

Figures

Figures reproduced from arXiv: 1908.09982 by the authors.

Figure 1
Figure 1. Factorized LSTM Cell Sn×n, and Vn×n are obtained by taking the top r singular values from the diagonal matrix S and the corresponding singular vectors from U and V. 3.4 Semi-NMF Semi-NMF generalizes Non-negative Matrix Factor￾ization (NMF) by relaxing some of the sign con￾straints on negative values for U and W (V has to be kept positive). Semi-NMF is more prefer￾able in application to Neural Networks because of thi… view at source ↗
Figure 2
Figure 2. Norm analysis comparisons between MF and Pruning in Language Modeling (PTB) and ELMo. Rank versus [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Heatmap LSTM weights on PTB [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Heatmap of ELMo forward weights. deviation, in both PTB and ELMo. The first notable pattern from [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 45 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  3. [3]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  4. [4]

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

    arXiv 2014

  5. [5]

    Francois Belletti, Alex Beutel, Sagar Jain, and Ed Chi. 2018. Factorized recurrent neural architectures for longer range dependence. In International Conference on Artificial Intelligence and Statistics , pages 1522--1530

    work page 2018

  6. [6]

    Bowman, Gabor Angeli, Christopher Potts, and Christopher D

    Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Association for Computational Linguistics

    work page 2015

  7. [7]

    Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced lstm for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , volume 1, pages 1657--1668

    work page 2017

  8. [8]

    Misha Denil, Babak Shakibi, Laurent Dinh, Nando De Freitas, et al. 2013. Predicting parameters in deep learning. In Advances in neural information processing systems , pages 2148--2156

    work page 2013

Show all 53 references
  1. [9]

    Chris HQ Ding, Tao Li, and Michael I Jordan. 2010. Convex and semi-nonnegative matrix factorizations. IEEE transactions on pattern analysis and machine intelligence , 32(1):45--55

    2010

  2. [10]

    Hao-Teng Fan, Jeih-weih Hung, Xugang Lu, Syu-Siang Wang, and Yu Tsao. 2014. Speech enhancement using segmental nonnegative matrix factorization. In Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on , pages 4483--4487. IEEE

    2014

  3. [11]

    Maryam Fazel. 2002. Matrix rank minimization with applications . Ph.D. thesis, PhD thesis, Stanford University

    2002

  4. [12]

    u rgen T Geiger, Jort F Gemmeke, Bj \

    J \"u rgen T Geiger, Jort F Gemmeke, Bj \"o rn Schuller, and Gerhard Rigoll. 2014. Investigating nmf speech enhancement for neural network based acoustic models. In Proc. INTERSPEECH 2014, ISCA, Singapore, Singapore

    2014

  5. [13]

    Gers, J\" u rgen A

    Felix A. Gers, J\" u rgen A. Schmidhuber, and Fred A. Cummins. 2000. Learning to forget: Continual prediction with lstm. Neural Comput. , 12(10):2451--2471, October

    2000

  6. [14]

    Yunchao Gong, Liu Liu, Ming Yang, and Lubomir Bourdev. 2014. Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115

    2014 arXiv

  7. [15]

    Artem M Grachev, Dmitry I Ignatov, and Andrey V Savchenko. 2017. Neural networks compression for language modeling. In International Conference on Pattern Recognition and Machine Intelligence , pages 351--357. Springer

    2017

  8. [16]

    Song Han, Jeff Pool, John Tran, and William Dally. 2015. Learning both weights and connections for efficient neural network. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28 , pages 1135--1143. ...

    2015

  9. [17]

    Song Han, Huizi Mao, and William J Dally. 2016. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. ICLR

    2016

  10. [18]

    Luheng He, Kenton Lee, Mike Lewis, and Luke Zettlemoyer. 2017. Deep semantic role labeling: What works and what’s next. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , volume 1, pages 473--483

    2017

  11. [19]

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. stat , 1050:9

    2015

  12. [20]

    Sepp Hochreiter and J \"u rgen Schmidhuber. 1997. Long short-term memory. Neural computation , 9(8):1735--1780

    1997

  13. [21]

    Seunghoon Hong, Jonghyun Choi, Jan Feyereisl, Bohyung Han, and Larry S Davis. 2016. Joint image clustering and labeling by matrix factorization. IEEE transactions on pattern analysis and machine intelligence , 38(7):1411--1424

    2016

  14. [22]

    Qiangui Huang, Kevin Zhou, Suya You, and Ulrich Neumann. 2018. Learning to prune filters in convolutional neural networks. arXiv preprint arXiv:1801.07365

    2018 arXiv

  15. [23]

    Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. 2014. Speeding up convolutional neural networks with low rank expansions. In Proceedings of the British Machine Vision Conference. BMVA Press

    2014

  16. [24]

    Caiming Xiong James Bradbury, Stephen Merity and Richard Socher. 2017. Quasi-recurrent neural networks. In International Conference on Learning Representations

    2017

  17. [25]

    Oleksii Kuchaiev and Boris Ginsburg. 2017. Factorization tricks for lstm networks. ICLR Workshop

    2017

  18. [26]

    Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017. End-to-end neural coreference resolution. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , pages 188--197

    2017

  19. [27]

    Omer Levy, Kenton Lee, Nicholas FitzGerald, and Luke Zettlemoyer. 2018. Long short-term memory as a dynamically computed element-wise weighted sum. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages 732--...

    2018

  20. [28]

    Xuelong Li, Guosheng Cui, and Yongsheng Dong. 2017. Graph regularized non-negative low-rank matrix factorization for image clustering. IEEE transactions on cybernetics , 47(11):3840--3853

    2017

  21. [29]

    Xuan Liu, Di Cao, and Kai Yu. 2018. Binarized lstm language model. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) , pages 2113--2121. Association for Comp...

    2018

  22. [30]

    Zhiyun Lu, Vikas Sindhwani, and Tara N Sainath. 2016. Learning compact recurrent neural networks. In Acoustics, Speech and Signal Processing (ICASSP), 2016 IEEE International Conference on , pages 5960--5964. IEEE

    2016

  23. [31]

    Zhi-Quan Luo and Paul Tseng. 1992. On the convergence of the coordinate descent method for convex differentiable minimization. Journal of Optimization Theory and Applications , 72(1):7--35

    1992

  24. [32]

    Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini

    Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: The penn treebank. Comput. Linguist. , 19(2):313--330, June

    1993

  25. [33]

    Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in translation: Contextualized word vectors. In Advances in Neural Information Processing Systems , pages 6294--6305

    2017

  26. [34]

    Gábor Melis, Chris Dyer, and Phil Blunsom. 2018. On the state of the art of evaluation in neural language models. In International Conference on Learning Representations

    2018

  27. [35]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. Pointer sentinel mixture models. ICLR

    2017

  28. [36]

    Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018. Regularizing and optimizing LSTM language models. In International Conference on Learning Representations

    2018

  29. [37]

    Antonio Valerio Miceli Barone. 2018. Low-rank passthrough neural networks. In Proceedings of the Workshop on Deep Learning Approaches for Low-Resource NLP , pages 77--86. Association for Computational Linguistics

    2018

  30. [38]

    Tom \'a s Mikolov. 2012. Statistical language models based on neural networks. Presentation at Google, Mountain View, 2nd April

    2012

  31. [39]

    Nasser Mohammadiha, Paris Smaragdis, and Arne Leijon. 2013. Supervised and unsupervised speech enhancement using nonnegative matrix factorization. IEEE Transactions on Audio, Speech, and Language Processing , 21(10):2140--2151

    2013

  32. [40]

    Ivan V Oseledets. 2011. Tensor-train decomposition. SIAM Journal on Scientific Computing , 33(5):2295--2317

    2011

  33. [41]

    Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: H...

    2018

  34. [42]

    Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pages 2383--2392. Association for Computational Linguistics

    2016

  35. [43]

    Abigail See, Minh-Thang Luong, and Christopher D Manning. 2016a. Compression of neural machine translation models via pruning. CoNLL 2016 , page 291

    2016

  36. [44]

    Abigail See, Minh-Thang Luong, and Christopher D. Manning. 2016b. Compression of neural machine translation models via pruning. In Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning , pages 291--301. Association for Computational Linguistics

  37. [45]

    Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. ICLR 2017

    2016

  38. [46]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language ...

    2013

  39. [47]

    Martin Sundermeyer, Ralf Schl \"u ter, and Hermann Ney. 2012. Lstm neural networks for language modeling. In Thirteenth Annual Conference of the International Speech Communication Association

    2012

  40. [48]

    Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems , pages 3104--3112

    2014

  41. [49]

    Andros Tjandra, Sakriani Sakti, and Satoshi Nakamura. 2017. Compressing recurrent neural network with tensor train. In Neural Networks (IJCNN), 2017 International Joint Conference on , pages 4451--4458. IEEE

    2017

  42. [50]

    George Trigeorgis, Konstantinos Bousmalis, Stefanos Zafeiriou, and Bjoern Schuller. 2014. A deep semi-nmf model for learning hidden representations. In International Conference on Machine Learning , pages 1692--1700

    2014

  43. [51]

    Wei Wen, Yuxiong He, Samyam Rajbhandari, Minjia Zhang, Wenhan Wang, Fang Liu, Bin Hu, Yiran Chen, and Hai Li. 2018. Learning intrinsic sparse structures within long short-term memory. In International Conference on Learning Representations

    2018

  44. [52]

    Kevin W Wilson, Bhiksha Raj, Paris Smaragdis, and Ajay Divakaran. 2008. Speech denoising using nonnegative matrix factorization with priors. In Acoustics, Speech and Signal Processing, 2008. ICASSP 2008. IEEE International Conference on , pages 4029--4032. IEEE

    2008

  45. [53]

    Yinchong Yang, Denis Krompass, and Volker Tresp. 2017. Tensor-train recurrent neural networks for video classification. In International Conference on Machine Learning , pages 3891--3900

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.