REVIEW 11 cited by
Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
This work aims at decreasing the end-to-end generation latency of large language models (LLMs). One of the major causes of the high generation latency is the sequential decoding approach adopted by almost all state-of-the-art LLMs. In this work, motivated by the thinking and writing process of humans, we propose Skeleton-of-Thought (SoT), which first guides LLMs to generate the skeleton of the answer, and then conducts parallel API calls or batched decoding to complete the contents of each skeleton point in parallel. Not only does SoT provide considerable speed-ups across 12 LLMs, but it can also potentially improve the answer quality on several question categories. SoT is an initial attempt at data-centric optimization for inference efficiency, and showcases the potential of eliciting high-quality answers by explicitly planning the answer structure in language.
Forward citations
Cited by 11 Pith papers
-
VLM-R$^3$: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought
VLM-R3 uses reinforcement learning to teach a multimodal model when and where to crop an image during its chain-of-thought, improving scores on several visual reasoning benchmarks.
-
Kinetics: Rethinking Test-Time Scaling Laws
A memory-aware test-time scaling law shows small models are overestimated and sparse attention is needed for efficient scaling.
-
VeriThinker: Learning to Verify Makes Reasoning Model Efficient
VeriThinker shows that fine-tuning a reasoning model only on a solution-verification task reduces chain-of-thought length on MATH500 and AIME by 20-45% while preserving or slightly improving accuracy.
-
MASTER: A Multi-Agent System with LLM Specialized MCTS
A multi-agent framework whose tree search is guided by LLM self-evaluation instead of simulations, reporting 76% on HotpotQA, 80% on WebShop, and 91% on MBPP.
-
Efficient Reasoning on the Edge
LoRA adapters, budget-forced GRPO, dynamic switching, parallel verification and FPTQuant enable practical chain-of-thought reasoning on quantized Qwen2.5-7B for edge devices.
-
What makes prompts a graph: necessary and sufficient conditions for prompt graph engineering
The paper defines prompt graph engineering via four necessary and sufficient conditions (explicit structure, structure/content separation, executable semantics, first-class artifact) and an inclusion/exclusion test th...
-
Parallel Decoder Transformer: Planner-Conditioned Latent Coordination for Model-Intrinsic Parallel Generation
A Parallel Decoder Transformer architecture for multi-lane parallel text generation is proposed, but the paper's own evidence is incomplete and internally contradictory.
-
Pipelined Decoder for Efficient Context-Aware Text Generation
The pipelined decoder trains a T5 model to generate multiple subsequences in parallel with partial token dependencies, achieving 1.7x to 7x faster inference on context-aware generation tasks with roughly unchanged quality.
-
Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs
ASTS extends locally typical sampling with semantic scoring and dynamic thresholds, reporting improved perplexity, MAUVE, and diversity on story and summarization tasks.
-
S$^2$-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiency
S2-MAD's decision mechanism filters redundant viewpoints and conditionally skips participation, cutting token costs by up to 94.5% versus standard multi-agent debate while keeping accuracy within about 2 points in the...
-
Agent-Centric Projection of Prompting Techniques and Implications for Synthetic Training Data for Large Language Models
The paper defines linear and non-linear interaction contexts for LLMs and conjectures that prompting techniques and multi-agent systems are equivalent projections of the same patterns.
Discussion (0). Continue with ORCID to comment.