Pith. sign in

REVIEW 19 cited by

DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2004.12993 v1 pith:VSGBFURZ submitted 2020-04-27 cs.CL cs.LG

DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

Ji Xin , Raphael Tang , Jaejun Lee , Yaoliang Yu , Jimmy Lin This is my paper
classification cs.CL cs.LG
keywords bertdeebertinferenceapplicationsmodelmodelsableaccelerate
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large-scale pre-trained language models such as BERT have brought significant improvements to NLP applications. However, they are also notorious for being slow in inference, which makes them difficult to deploy in real-time applications. We propose a simple but effective method, DeeBERT, to accelerate BERT inference. Our approach allows samples to exit earlier without passing through the entire model. Experiments show that DeeBERT is able to save up to ~40% inference time with minimal degradation in model quality. Further analyses show different behaviors in the BERT transformer layers and also reveal their redundancy. Our work provides new ideas to efficiently apply deep transformer-based models to downstream tasks. Code is available at https://github.com/castorini/DeeBERT.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 19 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Skip a Layer or Loop It? Learning Program-of-Layers in LLMs

    cs.LG 2026-06 unverdicted novelty 7.0

    Dynamic skipping and looping of LLM layers via a learned prediction network improves math reasoning accuracy while using fewer layers than standard or prior dynamic-depth inference.

  2. Two-dimensional early exit optimisation of LLM inference

    cs.CL 2026-03 unverdicted novelty 7.0

    Coordinating layer-wise and sentence-wise early exits in LLMs produces multiplicative speedups of 1.4-2.3x over single-dimension early exit on sentiment classification tasks.

  3. Eliciting Latent Predictions from Transformers with the Tuned Lens

    cs.LG 2023-03 accept novelty 7.0

    Training per-layer affine probes on frozen transformers yields more reliable latent predictions than the logit lens and enables detection of malicious inputs from prediction trajectories.

  4. Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readouts

    cs.LG 2026-07 conditional novelty 6.0

    In looped transformers, halting-gate failures come mainly from how gate training reshapes the trajectory; fixed-prior depth supervision plus simple confidence readouts yields better accuracy per unit of compute.

  5. CascadeFormer: Depth-Tapered Transformers Motivated by Gradient Fan-in Asymmetry

    cs.LG 2026-06 unverdicted novelty 6.0

    CascadeFormer tapers Transformer width with depth based on gradient fan-in asymmetry to match uniform baselines in perplexity while cutting latency 8.6%.

  6. RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

    cs.LG 2026-06 unverdicted novelty 6.0

    RKSC delivers 3.008x mean speedup over baseline and 1.66x over vLLM prefix caching for multi-branch LLM reasoning via similarity-based KV sharing and confidence-gated early exit, with 0.37% error rate.

  7. Compute Where it Counts: Self Optimizing Language Models

    cs.LG 2026-05 unverdicted novelty 6.0

    SOL trains a policy to dynamically control multiple efficiency mechanisms per token via group-relative policy optimization on teacher-forced episodes, yielding better quality at matched average budget than static or r...

  8. Sparse Layers are Critical to Scaling Looped Language Models

    cs.LG 2026-05 unverdicted novelty 6.0

    Looped MoE models scale better than standard transformers because different experts activate on each loop pass, recovering expressivity without extra parameters, and support superior early exits.

  9. A Comparative Analysis on the Performance of Upper Confidence Bound Algorithms in Adaptive Deep Neural Networks

    cs.LG 2026-04 conditional novelty 6.0

    UCB-V and UCB-Tuned dominate accuracy-energy and accuracy-latency trade-offs while all tested UCB strategies achieve sub-linear regret in adaptive DNN early-exit experiments on CIFAR datasets.

  10. Dr.LLM: Dynamic Layer Routing in LLMs

    cs.CL 2025-10 unverdicted novelty 6.0

    Dr. LLM retrofits frozen LLMs with MCTS-supervised per-layer routers for skip/execute/repeat decisions, delivering up to +3.4% accuracy and 5-layer savings on reasoning tasks with strong out-of-domain generalization.

  11. Visual Programmability: A Guide for Code-as-Thought in Chart Understanding

    cs.CV 2025-09 conditional novelty 6.0

    A vision-language model learns to dynamically switch between code-based and visual reasoning for chart questions, improving average accuracy by about one point over fixed strategies.

  12. Sparse Layers are Critical to Scaling Looped Language Models

    cs.LG 2026-05 unverdicted novelty 5.0

    Looped-MoE models scale better than dense looped or standard transformers because routing changes across loops, and they enable stronger compute-quality trade-offs via early exits at loop boundaries.

  13. HyperLens: Quantifying Cognitive Effort in LLMs with Fine-grained Confidence Trajectory

    cs.AI 2026-05 unverdicted novelty 5.0

    HyperLens reveals that deeper transformer layers magnify small confidence changes into fine-grained trajectories, allowing quantification of cognitive effort where complex tasks demand more and standard SFT can reduce it.

  14. Complexity Horizons of Compressed Models in Analog Circuit Analysis

    cs.AI 2026-05 unverdicted novelty 5.0

    Prerequisite graphs map compressed LLM performance boundaries in analog circuit analysis to allow selecting the smallest viable model for a given task complexity.

  15. A Comparative Analysis on the Performance of Upper Confidence Bound Algorithms in Adaptive Deep Neural Networks

    cs.LG 2026-04 unverdicted novelty 5.0

    Comparative study applies UCB-V, UCB-Tuned, UCB-Bayes and UCB-BwK to ADNN early-exit selection on ResNet and MobileViT using CIFAR-10/100, reporting sub-linear regret with UCB-Bayes fastest and UCB-V/UCB-Tuned best on...

  16. Harnessing Input-Adaptive Inference for Efficient VLN

    cs.CV 2025-08 conditional novelty 5.0

    A three-part input-adaptive inference method (view masking, adaptive early exit, view caching) cuts VLN computation roughly in half on seven benchmarks with moderate success-rate loss.

  17. TOAST: Transformer Optimization using Adaptive and Simple Transformations

    cs.LG 2024-10 unverdicted novelty 5.0

    TOAST approximates full transformer blocks in pretrained models via lightweight closed-form mappings to cut parameters and FLOPs without retraining or finetuning.

  18. Network Edge Inference for Large Language Models: Principles, Techniques, and Opportunities

    cs.DC 2026-04 unverdicted novelty 3.0

    A survey synthesizing challenges, system architectures, model optimizations, deployment methods, and resource management techniques for large language model inference at the network edge.

  19. T-TAMER: Provably Taming Trade-offs in ML Serving

    cs.LG 2025-09 reject novelty 3.0

    T-TAMER claims recall is necessary and sufficient for provably optimal early-exit and cascade serving policies, but the main extensions are under-derived and partly reduce to known Gittins-index results.