CPP improves LLM QA by generating TP/TN/FP/FN propositions then answering with CoT, outperforming many prompting baselines on medical and commonsense benchmarks.
System 2 atten- tion (is something you might need too)
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
abstract
Soft attention in Transformer-based Large Language Models (LLMs) is susceptible to incorporating irrelevant information from the context into its latent representations, which adversely affects next token generations. To help rectify these issues, we introduce System 2 Attention (S2A), which leverages the ability of LLMs to reason in natural language and follow instructions in order to decide what to attend to. S2A regenerates the input context to only include the relevant portions, before attending to the regenerated context to elicit the final response. In experiments, S2A outperforms standard attention-based LLMs on three tasks containing opinion or irrelevant information, QA, math word problems and longform generation, where S2A increases factuality and objectivity, and decreases sycophancy.
representative citing papers
ThinkSwitch uses iterative self-distillation with QLoRA and spherical weight interpolation to raise both instruct and thinking checkpoint accuracy on small AIME and PubMedQA sets using only 15 human prompts per domain.
Metacognition supplies a three-level framework (computational, algorithmic, ecological) for bounded self-governance in generative AI systems.
Pangu-ACE improves educational response quality on EduBench from 0.457 to 0.538 and format validity from 0.707 to 0.866 by routing 19.7% of samples to a 1B model while escalating the rest to 7B.
A systematic survey categorizes prompt engineering methods for LLMs and VLMs by application area, summarizing methodologies, applications, models, datasets, strengths, and limitations for each technique along with a taxonomy and summary table.
citing papers explorer
-
Concretized Proposition Prompting Resolves Composition-Knowledge Dichotomy in Large Language Models
CPP improves LLM QA by generating TP/TN/FP/FN propositions then answering with CoT, outperforming many prompting baselines on medical and commonsense benchmarks.
-
ThinkSwitch: Context Distillation with LoRA and Weight Interpolation for Specific-Purpose Reasoning Tasks
ThinkSwitch uses iterative self-distillation with QLoRA and spherical weight interpolation to raise both instruct and thinking checkpoint accuracy on small AIME and PubMedQA sets using only 15 human prompts per domain.
-
Metacognition Should Be the Scientific Framework for Bounded and Effective Self-Governance in Generative AI
Metacognition supplies a three-level framework (computational, algorithmic, ecological) for bounded self-governance in generative AI systems.
-
Pangu-ACE: Adaptive Cascaded Experts for Educational Response Generation on EduBench
Pangu-ACE improves educational response quality on EduBench from 0.457 to 0.538 and format validity from 0.707 to 0.866 by routing 19.7% of samples to a 1B model while escalating the rest to 7B.
-
A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications
A systematic survey categorizes prompt engineering methods for LLMs and VLMs by application area, summarizing methodologies, applications, models, datasets, strengths, and limitations for each technique along with a taxonomy and summary table.