Pith. sign in

REVIEW 3 major objections 4 minor 62 references

Image Captioning with Sparse Recurrent Neural Network

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An end-to-end pruning method for image-captioning decoders reaches 97.5% sparsity (40x compression) with only about 2% performance loss after fine-tuning.

desk verdict A useful but imperfect pruning method for captioning decoders; the headline 40x compression claim is contradicted by the paper's own Table 5. read the letter →

arxiv 1908.10797 v2 pith:E3MNHY4Y submitted 2019-08-28 cs.CV cs.CLcs.LG

Jia Huei Tan , Chee Seng Chan , Joon Huang Chuah This is my paper · ORCID
classification cs.CVcs.CLcs.LG
keywords imagecaptioningmodelpruningrecurrentneuralnetworkssparsitylearnablegatingvisualattentioncompressionMS-COCO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Image-captioning models are too large for many phones and embedded devices, mostly because the recurrent decoder must read its full weight matrices at every time step. This paper tries to show that a decoder can be pruned almost to the bone in one training pass: up to 97.5% of its weights can be removed, a 40x reduction in non-zero parameters, while losing only about 2% on standard captioning metrics after a short fine-tuning stage. The mechanism is a set of learnable binary gates, one per weight, trained with the captioning loss plus a cosine-annealed penalty that pulls the average gate toward a user-chosen sparsity target. Across sparsity levels from 80% to 97.5%, the authors report this method holds caption quality better than gradual magnitude pruning or hard magnitude pruning, for both LSTM and GRU decoders, with only one or two hyperparameters. If the claim holds, the practical upshot is that sparse captioning decoders can be produced without iterative prune-and-retrain loops, making on-device captioning substantially cheaper.

What carries the argument

The load-bearing mechanism is a learnable gating matrix $G_l$, one entry per weight, whose sigmoid-activated values are sampled as Bernoulli masks during training and rounded to fixed binary masks at the end. The captioning loss teaches which connections must survive, while the cosine-annealed sparsity penalty $\lambda_s L_s$ pulls the average gate below 0.5 so most weights are dropped. Gating variables are trained at a high constant learning rate with a straight-through estimator and frozen during the end-to-end fine-tuning stage.

What would settle it

A controlled rerun in which gradual and hard pruning receive validation-based hyperparameter tuning (or their original authors' recommended configurations) would settle the claim: if tuned baselines match the proposed method's BLEU-4 and CIDEr at 90-97.5% sparsity on the same MS-COCO split, the reported advantage is an artifact of weak comparisons.

Watch

Extended reading notes

Core claim

The paper's central claim is that a recurrent image-captioning decoder can be made up to 97.5% sparse during ordinary training, rather than through a separate prune-and-retrain loop, and that the resulting sparse model stays close to the dense baseline in quality. For every weight matrix $W_l$, the method adds a gating matrix $G_l$ of the same shape; effective weights are $W_l \odot \mathrm{Bern}(\sigma(G_l))$, where $\sigma$ is the logistic sigmoid and $\mathrm{Bern}(\cdot)$ samples a binary mask, while a straight-through estimator allows gradients to flow through the sampling. A sparsity regularizer $L_s$ measures the absolute difference between the achieved sparsity, computed with $\mathrm{Round}(\sigma(G_l))$, and a target $s_{\mathrm{target}}$; the penalty is multiplied by a cosine annealing curve so the model learns freely early and is pushed toward the target later. After decoder training, the whole model including the CNN encoder is fine-tuned with the gates frozen, and the final weights are $W_l \odot \mathrm{Round}(\sigma(G_l))$; the gates are then discarded. On the MS-COCO split used, the authors report relative BLEU-4 and CIDEr drops of roughly 1% to 3% at 80-95% sparsity and 4.8% to 6.4% at 97.5% sparsity before fine-tuning, shrinking to roughly 1% to 2.5% after fine-tuning, with the sparse models beating gradual and hard pruning at the highest sparsities.

Load-bearing premise

The claimed advantage over the two comparison methods assumes they are run at their best settings; the paper uses fixed heuristics for them while choosing its own starting values and penalty strength on the test set.

Editorial extensions

If this is right

  • A 40x reduction in non-zero decoder parameters with roughly 2% metric loss would make it practical to fit an image-captioning model into on-chip SRAM or otherwise reduce memory-bandwidth cost on mobile and embedded devices.
  • Because the method is one-shot and needs only a target sparsity plus one weight for the sparsity penalty, it removes the iterative train-and-prune loops that make pruning pipelines slow to adopt.
  • The reported large-sparse versus small-dense comparison implies that for captioning, starting from an overparameterized network and pruning it beats training a small dense network directly at the same parameter budget.
  • Caption uniqueness does not fall at high sparsity; in the reported runs the fraction of unseen captions rises slightly, indicating the pruned model is not simply memorizing training captions.
  • At 97.5% sparsity the proposed method retains clearly higher BLEU-4 and CIDEr than gradual or hard pruning, suggesting the quality cliff that appears around 90% sparsity in magnitude-based methods can be pushed higher.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's own experiments, the layer-wise asymmetry suggests that a per-layer sparsity budget could improve the trade-off further: in the reported runs the attention MLP and Value projections are pruned least while Key and Query projections are pruned most, so a testable extension is to allocate sparsity targets according to each layer's surviving importance.
  • A testable extension the paper only gestures toward is applying the same gating and annealing schedule to Transformer decoders; the outcome would show whether the method's benefit is tied to the per-step weight reads of RNNs or transfers to attention-only architectures.
  • Because the gating starting value and sparsity-penalty weight were selected using MS-COCO test-set scores, a conservative expectation is that the roughly 2% loss figure is dataset-specific; applying the method to another captioning domain would likely require re-tuning those two values on validation data.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes an end-to-end pruning method for image captioning decoders based on learnable binary gating variables. A gating matrix is attached to every weight matrix in the RNN decoder (including the attention module), trained jointly with the captioning loss and an annealed sparsity-regularization term that drives the fraction of nonzero gates toward a user-specified target. At inference, gates are thresholded and the sparsified weights are kept. The authors evaluate the method on MS-COCO using LSTM and GRU decoders with visual attention, comparing against dense baselines and against in-house implementations of gradual pruning [1] and hard pruning [25]. The central claim is that the method reaches up to 97.5% sparsity (40× compression) with only about 2% performance loss after fine-tuning, and that it outperforms the competing pruning methods at high sparsity levels. The paper also includes ablations on gating initialization and regularization weight, a fine-tuning study, a large-sparse versus small-dense comparison, and analyses of caption uniqueness, caption length, and layer-wise sparsity allocation.

Significance. If the central claims hold, the paper provides a practical and simple method for compressing attention-based captioning decoders by one to two orders of magnitude with modest quality loss, which is directly relevant to on-device deployment. The single-shot end-to-end training procedure, the automatic layer-wise allocation of sparsity, and the evaluation across two RNN types and several sparsity levels are assets. The paper also gives a useful comparison against gradual and hard pruning baselines and an interesting large-sparse versus small-dense experiment. However, the significance of the empirical contribution is weakened by an internal inconsistency in the reported compression arithmetic, by hyperparameter selection on the test set, and by the absence of error bars despite the paper itself noting run-to-run CIDEr variability. These issues need to be resolved before the quantitative claims can be accepted as stated.

major comments (3)
  1. [§5.3, Table 5] The headline claim of 40× compression at 97.5% sparsity is not supported by the parameter counts in Table 5. The dense LSTM baseline is listed as 2.62M parameters and the 40× row as 101K nonzero parameters, which gives a compression ratio of 2.62M/101K ≈ 25.9, not 40; the dense GRU baseline is 1.97M and the 40× row is 89K, a ratio of ≈ 22.1. If the baseline counts refer to the full decoder, then the actual achieved sparsity for these rows is about 96.1% (LSTM) and 95.5% (GRU), not 97.5%; if the baseline counts refer only to the RNN cell while the compression ratio is computed against the full decoder, then the 'NNZ parameters' column mixes parameter scopes and the reader cannot verify the central claim. Please clarify the parameter scope, recalculate the sparsity/compression ratios consistently, and reconcile the abstract and Section 2.2 statements with the corrected numbers.
  2. [§5.1, Tables 1 and 2; §5.2] The gating initialization m and the sparsity-regularization weight λs are selected based on MS-COCO test set scores (Tables 1 and 2), and the final results in Tables 3–5 are reported on the same test split. This introduces optimistic selection bias in the reported performance. In addition, no error bars are given, although the paper notes in Section 5.2 that CIDEr scores have a standard deviation of 0.3–0.9 across runs. Please report results with the hyperparameters chosen on the validation split (or use nested evaluation) and provide run-to-run variability for at least the main comparisons.
  3. [§4.1; §5.2] The comparison against gradual pruning [1] and hard pruning [25] relies on in-house implementations whose fidelity to the original methods is not demonstrated. The paper states heuristic choices such as gradual pruning starting after epoch 1 and ending at epoch 15 with frequency 1000, and hard pruning retraining for 10 epochs, but provides no evidence that these match the original papers' optimal configurations. If these baselines are undertuned, the reported advantage of the proposed method could be an artifact of weak comparisons. Please justify the baseline configurations, or compare with results reported in the original papers, or provide sensitivity analysis for the baseline hyperparameters.
minor comments (4)
  1. [§3.2, Eqs. (11) and (14)] The sparsity level is enforced by the loss term Ls rather than discovered by the network, so the statement that the method 'achieves' 97.5% sparsity should be phrased as 'achieves the user-specified target'; the meaningful empirical result is the performance at that enforced sparsity. The current wording is easy to misread as a predictive finding.
  2. [§5.2, Tables 3 and 4] The text states that 'all the models have been verified to have achieved the targeted sparsity levels,' but the achieved sparsity is not reported for each row. Please include the measured sparsity (or NNZ counts) for the competing methods so that the comparison is complete.
  3. [Figure 2b] The legend distinguishes 'Weighted annealed loss' from 'Loss,' which is confusing because both are functions of the same sparsity objective. Please label the curves more explicitly, for example 'Ls before weighting' and 'λs Ls after weighting.'
  4. [§6] The sentence 'we can find sparse models that performs better than dense counterparts' contains a subject–verb agreement error and slightly overstates the results: in Tables 3 and 4 the sparse models do not consistently outperform the dense baselines on all metrics. Please rephrase to 'can perform comparably to, and occasionally slightly better than, the dense baseline on some metrics.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: sparsity is an explicit optimization target and the performance claims rest on external benchmarks.

full rationale

The paper's central claim is an empirical method result: an end-to-end gating-based pruning scheme that reaches a user-specified sparsity target while retaining captioning quality. Sparsity is not derived as a prediction from the model; it is explicitly set via starget in Eq. 11 and optimized through Eq. 14. Reaching the target is not tautological, since Table 2 shows that with starget = 0.9 and lambda_s = 1.0 the achieved sparsity is only 0.662, meaning the loss term does not guarantee the target without sufficient weighting. The performance claims are measured against dense baselines and external pruning baselines [1, 25] on MS-COCO using standard captioning metrics, so those results are externally falsifiable and are not loaded into the loss. The only self-citation, [40] in Sec. 5.6, is a supporting observation about word-embedding redundancy and is not load-bearing. The apparent inconsistency in Table 5 between NNZ parameter counts and the stated 40x compression ratio is an internal arithmetic or scope-mixing issue, but it is not a circularity of derivation: the sparsity and compression labels are defined by the chosen target, not by the table's parameter counts. No circular step is exhibited.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities beyond standard gating variables used in pruning. The free parameters are all hyperparameters of the training procedure, several of which were selected using the MS-COCO test set. The axioms are standard assumptions in deep learning and transfer learning; the annealing schedule is the most paper-specific choice.

free parameters (6)
  • Gating initialization m = 5.0
    Chosen based on MS-COCO test set scores (Table 1); directly affects which weights are retained initially and final performance.
  • Sparsity regularizer weight lambda_s = max(5, 0.5/(1-starget))
    Heuristic tuned to reach the target sparsity on MS-COCO test set (Tables 2, Sec 4.1).
  • Gating learning rate = 100 (no annealing)
    Chosen by hand; lower LR causes gates to train too slowly (Sec 4.1).
  • Sparse network dropout rates = 0.11 (RNN), 0.03 (attention)
    Lowered from 0.35/0.1 to account for reduced capacity of sparse networks (Sec 4.1).
  • Target sparsity starget = 0.8 to 0.975
    User-specified; defines the compression ratio. The method is designed to hit this target via the loss.
  • Beam size = 3 (2 for one comparison)
    Inference hyperparameter, standard for captioning; not part of the pruning method.
assumptions (5)
  • domain assumption Straight-through estimator: derivative of Bernoulli sampling is treated as identity
    Used in Eq. 10 to backpropagate through z(sigma(G)). Standard practice but not proven; if STE gradients are poor, the learned masks may not reflect true weight saliency.
  • ad hoc to paper Cosine annealing of sparsity loss with weight (1-alpha) starting at zero is beneficial
    The loss weight ramps from 0 to 1 over training (Eq. 11-12). The authors say this is consistent with observations in [1,32], but no ablation compares different schedules.
  • domain assumption Pretrained GoogLeNet features are suitable for the captioning task and can be fine-tuned
    Standard transfer learning assumption; all methods in the comparison share the same encoder, so this is not specific to the proposed method.
  • domain assumption MS-COCO Karpathy split and evaluation metrics are valid measures of caption quality
    Standard benchmark; the paper relies on BLEU, METEOR, ROUGE-L, CIDEr, SPICE without questioning their validity.
  • domain assumption Bernoulli sampling during training closely approximates the final hard mask at inference
    Training uses Bern(sigma(G)) (Eq. 10) while inference uses Round(sigma(G)) (Eq. 15). The paper assumes the stochastic masks during training transfer to deterministic binary masks without significant performance degradation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Image Captioning with Sparse Recurrent Neural Network." pith.science (2026). https://pith.science/paper/E3MNHY4Y

@misc{pith2026190810797,
  author       = {Pith},
  title        = {Pith review of: Image Captioning with Sparse Recurrent Neural Network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E3MNHY4Y}},
  note         = {Machine review of arXiv:1908.10797}
}
read the original abstract

Recurrent Neural Network (RNN) has been widely used to tackle a wide variety of language generation problems and are capable of attaining state-of-the-art (SOTA) performance. However despite its impressive results, the large number of parameters in the RNN model makes deployment to mobile and embedded devices infeasible. Driven by this problem, many works have proposed a number of pruning methods to reduce the sizes of the RNN model. In this work, we propose an end-to-end pruning method for image captioning models equipped with visual attention. Our proposed method is able to achieve sparsity levels up to 97.5% without significant performance loss relative to the baseline (~ 2% loss at 40x compression after fine-tuning). Our method is also simple to use and tune, facilitating faster development times for neural network practitioners. We perform extensive experiments on the popular MS-COCO dataset in order to empirically validate the efficacy of our proposed method.

Figures

Figures reproduced from arXiv: 1908.10797 by the authors.

Figure 1
Figure 1. An overview of our proposed end-to-end pruning method. “XE” denotes cross-entropy. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Training progression of our proposed end-to-end pruning method. Best viewed in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Layer-wise comparison of final sparsity levels. Best viewed in colour. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 53 canonical work pages

  1. [1]

    To prune, or not to prune: exploring the efficacy of pruning for model compression,

    M. Zhu and S. Gupta, “To prune, or not to prune: exploring the efficacy of pruning for model compression,” in 6th International Conference on Learning Representations (ICLR), Workshop Track Proceedings, 2018, pp. 1–10

    work page 2018

  2. [25]

    Compression of neural machine translation models via pruning,

    A. See, M.-T. Luong, and C. D. Manning, “Compression of neural machine translation models via pruning,” inProceedings of The 20th SIGNLL Conference on Computational Natural Language Learning. Association for Computational Linguistics, 2016, pp. 291– 301

    work page 2016

  3. [2]

    Exploring sparsity in recurrent neural networks,

    S. Narang, E. Elsen, G. Diamos, and S. Sengupta, “Exploring sparsity in recurrent neural networks,” in International Conference on Learning Representations (ICLR) , 2017, pp. 1–10

    work page 2017

  4. [3]

    Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding,

    S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding,” inInternational Conference on Learning Representations (ICLR), 2016, pp. 1–10

    work page 2016

  5. [4]

    Learning both weights and connections for efficient neural network,

    S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” inAdvances in Neural Information Processing Systems, 2015, pp. 1135– 1143

    work page 2015

  6. [5]

    Persistent RNNs: Stashing recurrent weights on-chip,

    G. Diamos, S. Sengupta, B. Catanzaro, M. Chrzanowski, A. Coates, E. Elsen, J. Engel, A. Hannun, and S. Satheesh, “Persistent RNNs: Stashing recurrent weights on-chip,” in International Conference on Machine Learning, 2016, pp. 2024–2033

    work page 2016

  7. [6]

    Convex neural networks,

    Y . Bengio, N. L. Roux, P. Vincent, O. Delalleau, and P. Marcotte, “Convex neural networks,” in Advances in neural information processing systems, 2006, pp. 123–130

    work page 2006

  8. [7]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015

    arXiv 2015

Show all 62 references
  1. [8]

    Understanding deep learning requires rethinking generalization,

    C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, “Understanding deep learning requires rethinking generalization,”arXiv preprint arXiv:1611.03530, 2016

    2016 arXiv

  2. [9]

    BinaryConnect: Training deep neural networks with binary weights during propagations,

    M. Courbariaux, Y . Bengio, and J.-P. David, “BinaryConnect: Training deep neural networks with binary weights during propagations,” inAdvances in neural information processing systems, 2015, pp. 3123–3131

    2015

  3. [10]

    Quantized neural networks: Training neural networks with low precision weights and activations,

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Bengio, “Quantized neural networks: Training neural networks with low precision weights and activations,” The Journal of Machine Learning Research, vol. 18, no. 1, pp. 6869–6898, 2017

    2017

  4. [11]

    XNOR-Net: ImageNet classification using binary convolutional neural networks,

    M. Rastegari, V . Ordonez, J. Redmon, and A. Farhadi, “XNOR-Net: ImageNet classification using binary convolutional neural networks,” inEuropean Conference on Computer Vision (ECCV), 2016, pp. 525–542

    2016

  5. [12]

    Optimal brain damage,

    Y . LeCun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” inAdvances in neural information processing systems, 1990, pp. 598–605

    1990

  6. [13]

    Optimal brain surgeon and general network pruning,

    B. Hassibi, D. G. Stork, and G. J. Wolff, “Optimal brain surgeon and general network pruning,” inIEEE international conference on neural networks. IEEE, 1993, pp. 293–299

    1993

  7. [14]

    Skeletonization: A technique for trimming the fat from a network via relevance assessment,

    M. C. Mozer and P. Smolensky, “Skeletonization: A technique for trimming the fat from a network via relevance assessment,” inAdvances in neural information processing systems, 1989, pp. 107–115

    1989

  8. [15]

    A simple procedure for pruning back-propagation trained neural networks,

    E. D. Karnin, “A simple procedure for pruning back-propagation trained neural networks,” IEEE transactions on neural networks, vol. 1, no. 2, pp. 239–242, 1990

    1990

  9. [16]

    A back-propagation algorithm with optimal use of hidden units,

    Y . Chauvin, “A back-propagation algorithm with optimal use of hidden units,” inAdvances in neural information processing systems, 1989, pp. 519–526

    1989

  10. [17]

    Structural learning with forgetting,

    M. Ishikawa, “Structural learning with forgetting,” Neural networks, vol. 9, no. 3, pp. 509–521, 1996

    1996

  11. [18]

    Dynamic network surgery for efficient DNNs,

    Y . Guo, A. Yao, and Y . Chen, “Dynamic network surgery for efficient DNNs,” inAdvances In Neural Information Processing Systems, 2016, pp. 1379–1387. 12

    2016

  12. [19]

    Variational dropout and the local reparameteri- zation trick,

    D. P. Kingma, T. Salimans, and M. Welling, “Variational dropout and the local reparameteri- zation trick,” in Advances in Neural Information Processing Systems, 2015, pp. 2575–2583

    2015

  13. [20]

    Variational dropout sparsifies deep neural networks,

    D. Molchanov, A. Ashukha, and D. Vetrov, “Variational dropout sparsifies deep neural networks,” in Proceedings of the 34th International Conference on Machine Learning- Volume 70. JMLR. org, 2017, pp. 2498–2507

    2017

  14. [21]

    Compressing neural networks using the variational information bottleneck,

    B. Dai, C. Zhu, B. Guo, and D. Wipf, “Compressing neural networks using the variational information bottleneck,” in International Conference on Machine Learning , 2018, pp. 1143–1152

    2018

  15. [22]

    Pruning filters for efficient ConvNets,

    H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient ConvNets,” in International Conference on Learning Representations (ICLR), 2017, pp. 1–10

    2017

  16. [23]

    ThiNet: A filter level pruning method for deep neural network compression,

    J.-H. Luo, J. Wu, and W. Lin, “ThiNet: A filter level pruning method for deep neural network compression,” inProceedings of the IEEE international conference on computer vision, 2017, pp. 5058–5066

    2017

  17. [24]

    NISP: Pruning networks using neuron importance score propagation,

    R. Yu, A. Li, C.-F. Chen, J.-H. Lai, V . I. Morariu, X. Han, M. Gao, C.-Y . Lin, and L. S. Davis, “NISP: Pruning networks using neuron importance score propagation,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 9194–9203

    2018

  18. [26]

    SNIP: Single-shot network pruning based on connec- tion sensitivity,

    N. Lee, T. Ajanthan, and P. H. Torr, “SNIP: Single-shot network pruning based on connec- tion sensitivity,” inInternational Conference on Learning Representations (ICLR), 2019, pp. 1–10

    2019

  19. [27]

    Long short-term memory,

    S. Hochreiter and J. Schmidhuber, “Long short-term memory,”Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997

    1997

  20. [28]

    AMC: AutoML for model compres- sion and acceleration on mobile devices,

    Y . He, J. Lin, Z. Liu, H. Wang, L.-J. Li, and S. Han, “AMC: AutoML for model compres- sion and acceleration on mobile devices,” inProceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 784–800

    2018

  21. [29]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks,

    J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks,” in International Conference on Learning Representations (ICLR) , 2019, pp. 1–10

    2019

  22. [30]

    Grow and prune compact, fast, and accurate LSTMs,

    X. Dai, H. Yin, and N. K. Jha, “Grow and prune compact, fast, and accurate LSTMs,”arXiv preprint arXiv:1805.11797, 2018

    2018 arXiv

  23. [31]

    NeST: A neural network synthesis tool based on a grow-and- prune paradigm,

    X. Dai, H. Yin, and N. Jha, “NeST: A neural network synthesis tool based on a grow-and- prune paradigm,”IEEE Transactions on Computers, 2019

    2019

  24. [32]

    Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP,

    H. Yu, S. Edunov, Y . Tian, and A. S. Morcos, “Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLP,” arXiv preprint arXiv:1906.02768, 2019

    1906 arXiv

  25. [33]

    Training sparse neural networks,

    S. Srinivas, A. Subramanya, and R. Venkatesh Babu, “Training sparse neural networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 138–145

    2017

  26. [34]

    Learning sparse neural networks throughl_0 regularization,

    C. Louizos, M. Welling, and D. P. Kingma, “Learning sparse neural networks throughl_0 regularization,” inInternational Conference on Learning Representations (ICLR), 2018, pp. 1–10

    2018

  27. [35]

    Learning compact recurrent neural networks,

    Z. Lu, V . Sindhwani, and T. N. Sainath, “Learning compact recurrent neural networks,” in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2016, pp. 5960–5964

    2016

  28. [36]

    FastGRNN: A fast, accurate, stable and tiny kilobyte sized gated recurrent neural network,

    A. Kusupati, M. Singh, K. Bhatia, A. Kumar, P. Jain, and M. Varma, “FastGRNN: A fast, accurate, stable and tiny kilobyte sized gated recurrent neural network,” in Advances in Neural Information Processing Systems, 2018, pp. 9017–9028. 13

    2018

  29. [37]

    Structured word embedding for low memory neural network language model,

    K. Shi and K. Yu, “Structured word embedding for low memory neural network language model,” inInterspeech, 2018, pp. 1254–1258

    2018

  30. [38]

    LightRNN: Memory and computation-efficient recurrent neural networks,

    X. Li, T. Qin, J. Yang, and T.-Y . Liu, “LightRNN: Memory and computation-efficient recurrent neural networks,” in Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc., 2016, pp. 4385–4393

    2016

  31. [39]

    Exploring memory and time efficient neural networks for image captioning,

    S. N. Parameswaran, “Exploring memory and time efficient neural networks for image captioning,” in National Conference on Computer Vision, Pattern Recognition, Image Processing, and Graphics. Springer, 2017, pp. 338–347

    2017

  32. [40]

    COMIC: Towards a compact image captioning model with attention,

    J. H. Tan, C. S. Chan, and J. H. Chuah, “COMIC: Towards a compact image captioning model with attention,”IEEE Transactions on Multimedia, 2019

    2019

  33. [41]

    Efficient sequence learning with group recurrent networks,

    F. Gao, L. Wu, L. Zhao, T. Qin, X. Cheng, and T.-Y . Liu, “Efficient sequence learning with group recurrent networks,” inProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Lon...

    2018

  34. [42]

    Show, attend and tell: Neural image caption generation with visual attention,

    K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, and Y . Ben- gio, “Show, attend and tell: Neural image caption generation with visual attention,” in International Conference on Machine Learning (ICML), 2015, pp. 2048–2057

    2015

  35. [43]

    Aligning Where to See and What to Tell: Image Captioning with Region-based Attention and Scene-specific Contexts,

    K. Fu, J. Jin, R. Cui, F. Sha, and C. Zhang, “Aligning Where to See and What to Tell: Image Captioning with Region-based Attention and Scene-specific Contexts,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 12, pp. 2321–2334, 2017

    2017

  36. [44]

    Bottom-up and top-down attention for image captioning and visual question answering,

    P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould, and L. Zhang, “Bottom-up and top-down attention for image captioning and visual question answering,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6077–6086

    2018

  37. [45]

    Learn- ing phrase representations using RNN encoder-decoder for statistical machine translation,

    K. Cho, B. van Merrienboer, C. Gulcehre, F. Bougares, H. Schwenk, and Y . Bengio, “Learn- ing phrase representations using RNN encoder-decoder for statistical machine translation,” in Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014

    2014

  38. [46]

    Neural machine translation by jointly learning to align and translate,

    D. Bahdanau, K. Cho, and Y . Bengio, “Neural machine translation by jointly learning to align and translate,” in Proceedings of the International Conference on Learning Represen- tations (ICLR), 2015

    2015

  39. [47]

    Deconstructing lottery tickets: Zeros, signs, and the Supermask,

    H. Zhou, J. Lan, R. Liu, and J. Yosinski, “Deconstructing lottery tickets: Zeros, signs, and the Supermask,”arXiv preprint arXiv:1905.01067, 2019

    1905 arXiv

  40. [48]

    Estimating or propagating gradients through stochastic neurons for conditional computation,

    Y . Bengio, N. Léonard, and A. Courville, “Estimating or propagating gradients through stochastic neurons for conditional computation,”arXiv preprint arXiv:1308.3432, 2013

    2013 arXiv

  41. [49]

    Understanding the difficulty of training deep feedforward neural networks,

    X. Glorot and Y . Bengio, “Understanding the difficulty of training deep feedforward neural networks,” inProceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010, pp. 249–256

    2010

  42. [50]

    Going deeper with convolutions,

    C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V . Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1–9

    2015

  43. [51]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift,

    S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” inInternational Conference on Machine Learning (ICML), 2015, pp. 448–456

    2015

  44. [52]

    ImageNet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition. Ieee, 2009, pp. 248–255

    2009

  45. [53]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” inProceedings of the 3rd International Conference on Learning Representations (ICLR), 2014

    2014

  46. [54]

    Microsoft COCO: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in European Conference on Computer Vision (ECCV), 2014, pp. 740–755. 14

    2014

  47. [55]

    Deep visual-semantic alignments for generating image descrip- tions,

    A. Karpathy and L. Fei-Fei, “Deep visual-semantic alignments for generating image descrip- tions,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 3128–3137

    2015

  48. [56]

    BLEU: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting on association for computational linguistics, 2002, pp. 311–318

    2002

  49. [57]

    METEOR: An automatic metric for MT evaluation with im- proved correlation with human judgments,

    S. Banerjee and A. Lavie, “METEOR: An automatic metric for MT evaluation with im- proved correlation with human judgments,” in Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, vol. 29, 2005, pp. 65–72

    2005

  50. [58]

    ROUGE: A package for automatic evaluation of summaries,

    C.-Y . Lin, “ROUGE: A package for automatic evaluation of summaries,” inText summa- rization branches out: Proceedings of the ACL-04 workshop, vol. 8, 2004, pp. 1–8

    2004

  51. [59]

    CIDEr: Consensus-based image description evaluation,

    R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “CIDEr: Consensus-based image description evaluation,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 4566–4575

    2015

  52. [60]

    SPICE: Semantic propositional image caption evaluation,

    P. Anderson, B. Fernando, M. Johnson, and S. Gould, “SPICE: Semantic propositional image caption evaluation,” inEuropean Conference on Computer Vision (ECCV), 2016, pp. 382–398

    2016

  53. [61]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in neural information processing systems, 2017, pp. 5998–6008

    2017

  54. [62]

    Compressing word embeddings via deep compositional code learning,

    R. Shu and H. Nakayama, “Compressing word embeddings via deep compositional code learning,” inInternational Conference on Learning Representations (ICLR), 2018, pp. 1–10. 15

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.