Towards Universal & Efficient Model Compression via Exponential Torque Pruning

Sarthak Ketanbhai Modi; Shang-Wei Lin; Shourya Kuchhal; Yon Shin Teo; Yupeng Cheng; Yushi Cao; Zhiming Li; Zi Pong Lim

REVIEW 5 major objections 6 minor 15 references

Towards Universal & Efficient Model Compression via Exponential Torque Pruning

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pruning penalty that grows exponentially with module distance compresses networks with less accuracy loss than linear torque pruning.

desk verdict A simple, useful twist on Torque pruning with consistent gains, but the paper overclaims speed-ups and skips pivot-sensitivity analysis. read the letter →

arxiv 2506.22015 v3 pith:2HWDBLGA submitted 2025-06-27 cs.CV

Sarthak Ketanbhai Modi , Zi Pong Lim , Shourya Kuchhal , Yushi Cao , Yupeng Cheng , Yon Shin Teo , Shang-Wei Lin , Zhiming Li This is my paper

classification cs.CV

keywords structuredpruningmodelcompressionexponentialtorqueregularizationneuralnetworktransformercomputervisionnaturallanguageunderstandingtime-seriesforecasting

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Exponential Torque Pruning (ETP), a structured-pruning regularizer that changes how hard the training loss pushes a neural module's weights toward zero. Earlier torque-based pruning applies a force that grows linearly with a module's distance from a per-layer pivot; the authors argue this leaves distant modules too dense and penalizes nearby modules too much. ETP replaces that with an exponential penalty, so redundant modules far from the pivot are driven to zero while pivot-adjacent modules are preserved. On vision, language, graph, and time-series benchmarks, the paper reports higher speed-up at smaller accuracy drop than Torque and DepGraph, for example 9x speed-up with a 2.2% accuracy drop on VGG-19/CIFAR-100 versus Torque's 7.16% drop. If the claim holds, ETP is a simple, architecture-agnostic way to compress models for deployment on resource-constrained devices.

What carries the argument

The central mechanism is the exponential torque regularization term, an exponential approximation of a Heaviside step in the force-versus-distance relationship. With one module in each layer chosen as pivot, the penalty on module $i$ is $\|w^l_i\|_2 \lambda^{\|\rho^l_i-\rho^l_p\|_2}$, where $\lambda$ controls how quickly force rises with distance and $\rho$ indexes modules. This single term replaces Torque's linear penalty; during training, it pushes distant modules' weight norms to zero while leaving pivot-near modules relatively unpenalized, producing the sparse structure that makes later pruning cheap and accurate.

What would settle it

Prune only the farthest modules from each layer's pivot in a trained VGG-19 or BERT and check whether any single distant module's removal causes a large accuracy drop; if one does, the monotonic importance assumption fails and ETP's exponential penalty would be crushing a necessary module.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that the default linear force application in torque-based pruning is miscalibrated: it under-penalizes distant modules, leaving the network dense after pruning, and over-penalizes modules close to the pivot that the network needs. ETP instead regularizes module weights with a penalty proportional to $\|w^l_i\|_2 \lambda^{\|\rho^l_i-\rho^l_p\|_2}$, making the force to zero grow exponentially with distance from the layer's pivot. During training this drives the L2 norms of distant modules to zero while sparing pivot-near modules, and after structured pruning the accuracy drop is smaller than baselines at comparable or higher speed-up. In the reported tables, ETP achieves 9x speed-up with a 2.2% drop on VGG-19/CIFAR-100, a 2.93x speed-up with a small accuracy gain on ResNet-56/CIFAR-10, and 11x (BERT) and 13.5x (RoBERTa) speed-ups with 1.4% and 2.4% drops on SST-2.

Load-bearing premise

The load-bearing premise is that a module's importance to the task decreases monotonically with its distance from the chosen pivot point, so the farthest modules are always the safest to prune.

Editorial extensions

If this is right

Because ETP adds only a regularizer to the training loss and requires no architectural change, the same recipe can be ported across CNNs, transformers, graph networks, and time-series forecasters.
At aggressive speed-up ratios, the paper's aggressive-pruning analysis shows ETP retains accuracy much better than baselines, so it is suited to edge deployment with tight compute budgets.
The paper's BLIP experiment suggests the benefit carries to large pretrained models, so compressing large vision-language models under a fixed parameter budget may preserve more generation quality than L1 or Torque pruning.
If the exponential scheme is the reason for the gain, then models with more redundancy should see larger compression benefits, matching the paper's observation that ETP's margin over baselines is largest on VGG-19.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial note: the introduction's advertised 42x speed-up with 2.4% drop for BERT/SST-2 is not what the tables show; Table 3 reports 11x for BERT and 13.5x for RoBERTa at those drops, so the central mechanism should be assessed against the table numbers.
A natural next experiment is to move the pivot from the first module to the middle or to a randomly chosen module per layer; if the accuracy after pruning changes sharply, ETP's universality depends on the pivot convention rather than on the exponential scheme alone.
The base $\lambda$ and coefficient $\beta$ are coupled, so an automatic per-layer schedule for $\lambda$ could push compression further in very deep networks where module distances span a wide range.
The exponential penalty is a pure distance prior; combining it with data-dependent importance scores could prune close-but-redundant modules that ETP currently preserves.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague — the short version: ETP replaces the linear distance weighting in Torque pruning with an exponential one, and that single change buys consistent, modest improvements across vision, language, graph, and time-series benchmarks. The idea is simple, code is provided, and the L2-norm analysis supports the mechanism. It deserves a serious look, but the paper has a few rough edges that need work before the main claims are taken at face value.

What's actually new: the exponential term in Eq. (10) is a genuine departure from the cited Torque work (Eq. (2)), which uses a linear factor. The authors also show that ETP drives distant modules' L2 norms to zero more effectively than Torque. That's useful evidence. The breadth of evaluation—four domains, plus a large BLIP model—is a real strength, and the citation pattern is fine: Torque and DepGraph are properly credited.

Where it wobbles: the introduction claims 42× and 23× speed-ups for BERT/SST-2 and VGG-19/CIFAR-100. Those numbers don't appear in the tables; the tables report 11× and 9×. That's an overclaim, not a trivial typo, because the same sentences compare accuracy drops against prior work. Also, the hyperparameters λ and β are never reported in the main text, there are no error bars, and the pivot sensitivity is untouched. The stress-test's concern about index distance as a proxy for importance is legitimate: module indices are arbitrary labels, and ETP's exponential penalty assumes importance decays with distance from the first module in each layer. The paper fixes that pivot and never checks whether results change if you reorder modules. The effect could be a favorable ordering rather than a universal mechanism. However, the reported gains are consistent across many settings, so this is a gap in robustness, not a clear refutation. They also honestly note the low-speed-up limitation in the conclusion.

Net: this is a solid workshop-to-conference-level paper with a reproducible heuristic and broad experiments. A serious referee will want the intro claims corrected, hyperparameters specified, error bars added, and a pivot-sensitivity experiment. I'd send it to review.

For the reading group: maybe—it's not a breakthrough, but it's a clean example of a regularization tweak that transfers across domains.

Referee Report

5 major / 6 minor

Summary. This manuscript proposes Exponential Torque Pruning (ETP), a structured-pruning regularizer that replaces the linear distance-weighted penalty of Torque pruning with an exponential term: each module's L2 norm is penalized by λ^{d_i}, where d_i is the index distance from a fixed pivot (the first module of each layer). The authors argue that this enforces stronger shrinkage on distant modules while preserving nearby modules, and they report experiments on CIFAR-10/100, ImageNet, GLUE (SST-2, MRPC), PPI, ETTh1, and Flickr8k/BLIP, comparing with Torque, DepGraph, and other baselines. The central empirical claim is that ETP achieves higher speed-up and lower accuracy drop across domains.

Significance. The proposed modification is simple, easy to implement, and, if the reported gains are reproducible, it would be a useful contribution to structured pruning: the paper covers four domains and multiple architectures, and the main tables consistently favor ETP over Torque. The authors also provide an implementation link and analyze L2-norm trajectories, which is helpful. However, the current evidence is weakened by unsupported headline numbers, absence of hyperparameters and error bars, and an unexamined pivot-dependence assumption, so the significance claim of universality is not yet established.

major comments (5)

[Section 1, Tables 1 and 3] The Introduction states that ETP achieves a 42× speed-up with 2.4% accuracy drop on BERT (SST-2) and a 23× speed-up with 4.5% drop on VGG-19 (CIFAR-100), but Table 3 reports 11× for BERT/SST-2 and Table 1 reports 9.03× for VGG-19/CIFAR-100. If these numbers come from the aggressive-pruning experiments in Figure 4, the figure and its axis labels must be included and cited explicitly; as written, the main claims of the paper are not supported by the data presented in the tables.
[Section 3, Eq. (8)] Equation (8) defines the proposed force as ||w_i||_2 · λ^{||ρ_i−ρ_p||_2}, but this is not a differentiable approximation of the Heaviside step function used in Eq. (6): a step function is bounded and saturates, whereas λ^d with λ>1 grows without bound and with 0<λ<1 decays. The paper needs either to justify why an unbounded exponential is the intended approximation or to replace this motivation; otherwise the derivation in Eqs. (5)-(8) is internally inconsistent.
[Section 4.1, Appendix] The experimental setup is incomplete: the text refers to an appendix for detailed setup, but no appendix is provided; in particular, the values of λ and β, the pruning schedule, training epochs, and hardware are missing. Without these, the reported results cannot be reproduced and the sensitivity of the method to its two free parameters cannot be assessed.
[Section 3, footnote 2 and Eq. (10)] The regularizer assumes that module importance decays monotonically with index distance from the first module of each layer. Because filter, head, and output indices are arbitrary orderings, this prior needs justification; the paper does not report pivot-sensitivity experiments, and a different pivot could invert the ranking of penalties. The universality claim depends on this assumption, so the authors should either provide a theoretical reason or experiments with multiple pivot choices.
[Tables 1-4] All results are single runs without standard deviations or multiple seeds. Several headline differences are small (e.g., +0.22 accuracy on CIFAR-10 ResNet-56, −1.4% vs −2.6% on SST-2) and could be within run-to-run noise; error bars or repeated-seed results are needed to support the claim of consistent improvement.

minor comments (6)

[Section 2.2, Eq. (2)] The torque analogy is used loosely; please define the meaning and units of the position vector and distance, or state explicitly that the analogy is only heuristic.
[Equations (1), (8), (10)] The symbol λ is used for the strength of the pruning regularizer in Eq. (1) and for the base of the exponential in Eq. (8); using β for the ETP coefficient in Eq. (10) avoids the collision, but the text should explicitly distinguish the two roles.
[Table 1] The table distinguishes 'Torque (p)' and 'Torque (r)' without defining the abbreviations; please define them in the caption or in the experimental setup.
[Figure 4] Figure 4 is referenced for the aggressive-pruning analysis, but the figure is not legible in the submitted version; please provide axes, legends, and results for all six tasks.
[References] Some references are duplicated (Gupta et al. 2024a and 2024b are the same publication, and Fang et al. 2023a and 2023b are also the same); please consolidate.
[Section 5] The first sentence of the structured-pruning paragraph is incomplete: 'Structured pruning focuses on removing higher-level structures... This yields...' is followed by a bare reference list; please rewrite as a complete sentence.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ETP's exponential penalty is a directly proposed regularizer ansatz, and its reported improvements are tested on held-out benchmarks rather than derived from its own definition.

full rationale

The paper's derivation chain begins with the Torque regularizer from externally cited prior work and proposes replacing the linear distance weighting with an exponential one in Eq. (10): L_task + beta * sum_l sum_i ||w_i^l||_2 * lambda^{||rho_i^l - rho_p^l||_2}. This term is defined directly; no parameter is fitted to the accuracy-drop or speed-up numbers that are subsequently reported, and the accuracy-drop measurements come from held-out test sets. The claim that distant modules are redundant is an empirical assumption, not a tautology, and its failure would show up as accuracy loss, so the evaluation is not forced by construction. The paper contains no load-bearing self-citation: the Torque baseline is externally cited prior work, and no uniqueness theorem is imported. The unsubstantiated 42x BERT/SST-2 claim in the introduction conflicts with the 11x in Table 3, and the fixed pivot choice is undertested, but these are correctness and robustness concerns, not circularity. Consequently no circular step is exhibited.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities. Its reliance is on hyperparameters (lambda, beta) that are not reported, and on assumptions about the pivot and the exponential approximation that are not empirically justified in the main text.

free parameters (2)

lambda (base of exponential)
Hyperparameter in Eq. (8) that controls the threshold distance for pruning. Not reported in the main text; likely tuned per dataset.
beta (regulatory coefficient)
Coefficient of the ETP loss in Eq. (9). Not reported in the main text; needs tuning.

assumptions (3)

domain assumption The first module in each layer is a suitable pivot; module importance decreases with distance from it.
Invoked in Section 3, Eq. (10), where the penalty grows exponentially with distance from the pivot. No evidence or ablation supports this ordering.
ad hoc to paper The exponential function is a valid differentiable approximation of the Heaviside step function for this purpose.
Section 3 states the exponential approximates the Heaviside, but no analysis justifies why this specific approximation is better than others (e.g., sigmoid).
domain assumption After regularization, modules with zero or negligible weight can be pruned without further retraining.
The pruning step after training is not described in the main text; the paper assumes the standard magnitude thresholding works.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Universal & Efficient Model Compression via Exponential Torque Pruning." pith.science (2026). https://pith.science/paper/2HWDBLGA

@misc{pith2026250622015,
  author       = {Pith},
  title        = {Pith review of: Towards Universal & Efficient Model Compression via Exponential Torque Pruning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2HWDBLGA}},
  note         = {Machine review of arXiv:2506.22015}
}

read the original abstract

The rapid growth in complexity and size of modern deep neural networks (DNNs) has increased challenges related to computational costs and memory usage, spurring a growing interest in efficient model compression techniques. Previous state-of-the-art approach proposes using a Torque-inspired regularization which forces the weights of neural modules around a selected pivot point. Whereas, we observe that the pruning effect of this approach is far from perfect, as the post-trained network is still dense and also suffers from high accuracy drop. In this work, we attribute such ineffectiveness to the default linear force application scheme, which imposes inappropriate force on neural module of different distances. To efficiently prune the redundant and distant modules while retaining those that are close and necessary for effective inference, in this work, we propose Exponential Torque Pruning (ETP), which adopts an exponential force application scheme for regularization. Experimental results on a broad range of domains demonstrate that, though being extremely simple, ETP manages to achieve significantly higher compression rate than the previous state-of-the-art pruning strategies with negligible accuracy drop.

Figures

Figures reproduced from arXiv: 2506.22015 by the authors.

**Figure 2.** (a) Visualization of vanilla Torque-prune regularization. The circles below the coordinate [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Comparison of the L2-norm curves during the training process. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Results of aggressive pruning analysis for six distinct tasks. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

15 extracted references · 4 canonical work pages

[5]

Amc: Automl for model compression and acceleration on mobile devices

Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. Amc: Automl for model compression and acceleration on mobile devices. In Proceedings of the European conference on computer vision (ECCV), pages 784–800, 2018b. Geoffrey Hinton. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531,

arXiv
[8]

Oscar: Object-semantics aligned pre-training for vision-language tasks

Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16, pages 121–137. Springer,

work page 2020
[9]

Roberta: A robustly optimized bert pretraining approach

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692,

arXiv 1907
[1989]

A signal propagation perspective for pruning neural networks at initialization

Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, and Philip HS Torr. A signal propagation perspective for pruning neural networks at initialization. arXiv preprint arXiv:1906.06307, 2019a. Namhoon Lee, Thalaiyasingam Ajanthan, and Philip H.S. Torr. Snip: Single-shot network pruning based on connection sensitivity. In ICLR, 2019b. Hao Li, Asim Kadav, I...

arXiv 1906
[1996]

Fine-tuning vision transformer using lora for image classification

Anupam Tiwary, Shek Diya Sarkar, Aditya Pratap Singh, Pankaj Kumar Agarwal, Subham Burman, and Rishab Poddar. Fine-tuning vision transformer using lora for image classification. In 2025 8th International Conference on Electronics, Materials Engineering & Nano-Technology (IEMENTech), pages 1–4. IEEE,

work page 2025
[2002]

To compress, or not to compress: Characterizing deep learning model compression for embedded inference

Qing Qin, Jie Ren, Jialong Yu, Hai Wang, Ling Gao, Jie Zheng, Yansong Feng, Jianbin Fang, and Zheng Wang. To compress, or not to compress: Characterizing deep learning model compression for embedded inference. In 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, S...

work page 2018
[2009]

Bert: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186,

2019
[2015]

Glue: A multi-task benchmark and analysis platform for natural language understanding

Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018a. Chaoqi Wang, Roger Grosse, Sanja Fidler, and Guodong Zhang. Eigendamage: Structured pruning in the kronecker-factored eigenbasis. In Internati...

arXiv

Show all 15 references

[2016]

Structured pruning for deep convolutional neural networks: A survey

Yang He and Lingao Xiao. Structured pruning for deep convolutional neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 46(5):2900–2919, 2023a. Yang He and Lingao Xiao. Structured pruning for deep convolutional neural networks: A survey. I...

arXiv
[2017]

Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding

Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015a. Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections ...

arXiv
[2018]

Very deep convolutional networks for large-scale image recognition

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

arXiv
[2019]

Picking winning tickets before training by preserving gradient flow

Chaoqi Wang, Guodong Zhang, and Roger Grosse. Picking winning tickets before training by preserving gradient flow. In ICLR, 2020a. Huan Wang, Can Qin, Yulun Zhang, and Yun Fu. Neural pruning via growing regularization. arXiv preprint arXiv:2012.09243, 2020b. Xin Wang, Fisher Y...

2012 arXiv
[2020]

Reducing transformer depth on demand with structured dropout

10 Angela Fan, Edouard Grave, and Armand Joulin. Reducing transformer depth on demand with structured dropout. arXiv preprint arXiv:1909.11556,

1909 arXiv
[2022]

Learning multiple layers of features from tiny images.(2009),

11 Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images.(2009),

2009
[2025]

Graph attention networks

12 Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903,

arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [5]

Amc: Automl for model compression and acceleration on mobile devices

Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, and Song Han. Amc: Automl for model compression and acceleration on mobile devices. In Proceedings of the European conference on computer vision (ECCV), pages 784–800, 2018b. Geoffrey Hinton. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531,

arXiv

[2] [8]

Oscar: Object-semantics aligned pre-training for vision-language tasks

Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16, pages 121–137. Springer,

work page 2020

[3] [9]

Roberta: A robustly optimized bert pretraining approach

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692,

arXiv 1907

[4] [1989]

A signal propagation perspective for pruning neural networks at initialization

Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, and Philip HS Torr. A signal propagation perspective for pruning neural networks at initialization. arXiv preprint arXiv:1906.06307, 2019a. Namhoon Lee, Thalaiyasingam Ajanthan, and Philip H.S. Torr. Snip: Single-shot network pruning based on connection sensitivity. In ICLR, 2019b. Hao Li, Asim Kadav, I...

arXiv 1906

[5] [1996]

Fine-tuning vision transformer using lora for image classification

Anupam Tiwary, Shek Diya Sarkar, Aditya Pratap Singh, Pankaj Kumar Agarwal, Subham Burman, and Rishab Poddar. Fine-tuning vision transformer using lora for image classification. In 2025 8th International Conference on Electronics, Materials Engineering & Nano-Technology (IEMENTech), pages 1–4. IEEE,

work page 2025

[6] [2002]

To compress, or not to compress: Characterizing deep learning model compression for embedded inference

Qing Qin, Jie Ren, Jialong Yu, Hai Wang, Ling Gao, Jie Zheng, Yansong Feng, Jianbin Fang, and Zheng Wang. To compress, or not to compress: Characterizing deep learning model compression for embedded inference. In 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, S...

work page 2018

[7] [2009]

Bert: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186,

2019

[8] [2015]

Glue: A multi-task benchmark and analysis platform for natural language understanding

Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018a. Chaoqi Wang, Roger Grosse, Sanja Fidler, and Guodong Zhang. Eigendamage: Structured pruning in the kronecker-factored eigenbasis. In Internati...

arXiv