Pith. sign in

REVIEW 4 cited by

Learning Harmonized Representations for Speculative Sampling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2408.15766 v3 pith:HRQX7JBA submitted 2024-08-28 cs.LG cs.CL

Lefan Zhang , Xiaodan Wang , Yanhua Huang , Ruiwen Xu This is my paper
classification cs.LGcs.CL
keywords harmonizeddecodinghasssamplingspeculativecontextmodelsrepresentations
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Speculative sampling is a promising approach to accelerate the decoding stage for Large Language Models (LLMs). Recent advancements that leverage target LLM's contextual information, such as hidden states and KV cache, have shown significant practical improvements. However, these approaches suffer from inconsistent context between training and decoding. We also observe another discrepancy between the training and decoding objectives in existing speculative sampling methods. In this work, we propose a solution named HArmonized Speculative Sampling (HASS) that learns harmonized representations to address these issues. HASS accelerates the decoding stage without adding inference overhead through harmonized objective distillation and harmonized context alignment. Experiments on four LLaMA models demonstrate that HASS achieves 2.81x-4.05x wall-clock time speedup ratio averaging across three datasets, surpassing EAGLE-2 by 8%-20%. The code is available at https://github.com/HArmonizedSS/HASS.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. SpecRoll: Fast-Slow Verifier-Feedback Adaptation for Speculative Reinforcement Learning Rollouts

    cs.LG 2026-08 conditional novelty 6.0 of 10

    SpecRoll accelerates GRPO rollouts with future-token heads plus a gradient-free Reflex memory, achieving 1.21x to 2.04x end-to-end speedup over vanilla GRPO while preserving the target sampling distribution.

  2. DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

    cs.AI 2026-07 accept novelty 6.0 of 10

    DSpark's semi-autoregressive drafter plus load-aware confidence scheduling raises accepted draft length and shifts the production serving Pareto frontier by 60-85% higher per-user speed at matched throughput versus MTP-1.

  3. POSS: Position Specialist Generates Better Draft for Speculative Decoding

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Using position-specialized draft layers instead of one single draft model improves later-token acceptance in speculative decoding, yielding modest speedups on Llama-3-8B and Llama-2-13B.

  4. ODIA: Oriented Distillation for Inline Acceleration of LLM-based Function Calling

    cs.LG 2025-07 conditional novelty 3.0 of 10

    ODIA routes 60% of function-calling traffic in a music app to a small 1.3B model, reducing expected latency by 45% and median latency by 78%.

Pith tools