REVIEW 6 cited by
A Thorough Examination of Decoding Methods in the Era of LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Decoding methods play an indispensable role in converting language models from next-token predictors into practical task solvers. Prior research on decoding methods, primarily focusing on task-specific models, may not extend to the current era of general-purpose large language models (LLMs). Moreover, the recent influx of decoding strategies has further complicated this landscape. This paper provides a comprehensive and multifaceted analysis of various decoding methods within the context of LLMs, evaluating their performance, robustness to hyperparameter changes, and decoding speeds across a wide range of tasks, models, and deployment environments. Our findings reveal that decoding method performance is notably task-dependent and influenced by factors such as alignment, model size, and quantization. Intriguingly, sensitivity analysis exposes that certain methods achieve superior performance at the cost of extensive hyperparameter tuning, highlighting the trade-off between attaining optimal results and the practicality of implementation in varying contexts.
Forward citations
Cited by 6 Pith papers
-
A First-Principles Theory of Slow Thinking and Active Perception
Active lifting of data distributions via latent-sequence sampling and max-rate uncertainty reduction formally derives slow-thinking LLMs and places them on representation and sampler hierarchies that can be climbed.
-
Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures
Xent Games formalize a large family of LLM evaluation tasks as games whose rewards and constraints are signed cross-entropy sums, and propose using them to build general capability measures.
-
Attention to Detail: Evaluating Energy, Performance, and Accuracy Trade-offs Across vLLM Configurations
vLLM attention kernel and prefix caching drive model- and task-dependent energy and latency effects, with no universal best config, and can unexpectedly shift measured accuracy.
-
Rare Event Analysis of Large Language Models
Using annealed transition path sampling plus MBAR reweighting, the authors estimate TinyStories-8M completion probabilities for extreme ARI and log-probability values that are unobservable by direct sampling.
-
A Comparative Study of Decoding Strategies in Medical Text Generation
Across five medical text tasks, deterministic decoding methods generally score higher than stochastic sampling, while medical-specific models do not outperform general models and are more sensitive to decoding choice.
-
Bridging On-Device and Cloud LLMs for Collaborative Reasoning: A Unified Methodology for Local Routing and Post-Training
Reinforcement learning post-training gives on-device LLMs an internal 'call for help' action, eliminating external routers and achieving strong math accuracy under cloud-use budgets.
Discussion (0). Sign in to comment.