REVIEW 6 cited by
Empowering LLMs with Logical Reasoning: A Comprehensive Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have achieved remarkable successes on various tasks. However, recent studies have found that there are still significant challenges to the logical reasoning abilities of LLMs, which can be categorized into the following two aspects: (1) Logical question answering: LLMs often fail to generate the correct answer within a complex logical problem which requires sophisticated deductive, inductive or abductive reasoning given a collection of premises. (2) Logical consistency: LLMs are prone to producing responses contradicting themselves across different questions. For example, a state-of-the-art question-answering LLM Macaw, answers Yes to both questions Is a magpie a bird? and Does a bird have wings? but answers No to Does a magpie have wings?. To facilitate this research direction, we comprehensively investigate the most cutting-edge methods and propose a detailed taxonomy. Specifically, to accurately answer complex logic questions, previous methods can be categorized based on reliance on external solvers, prompts, and fine-tuning. To avoid logical contradictions, we discuss concepts and solutions of various logical consistencies, including implication, negation, transitivity, factuality consistencies, and their composites. In addition, we review commonly used benchmark datasets and evaluation metrics, and discuss promising research directions, such as extending to modal logic to account for uncertainty and developing efficient algorithms that simultaneously satisfy multiple logical consistencies.
Forward citations
Cited by 6 Pith papers
-
Symbolic Neural Generation with Applications to Lead Discovery in Drug Design
A hybrid system that learns symbolic interval constraints from a few examples and uses an LLM plus rejection filtering to generate new candidate drug molecules.
-
Rethinking Reasoning Quality in Large Language Models through Enhanced Chain-of-Thought via RL
DRER rewards CoT trajectories that increase the model's likelihood of the correct answer, plus a length penalty, and the new LogicTree benchmark reportedly lifts a 7B model's average accuracy from 0.13 to 0.60.
-
Uncertainty-aware Reward Design Process
URDP couples LLM-based reward component design with uncertainty-weighted Bayesian optimization, reporting better reward quality and efficiency than Eureka and Text2Reward on three benchmarks.
-
Debunk and Infer: Multimodal Fake News Detection via Diffusion-Generated Evidence and LLM Reasoning
A framework called DIFND generates debunking evidence via conditional diffusion and uses multi-agent MLLM reasoning to detect fake news videos, outperforming baselines on FakeSV and FVC.
-
Unveiling Effective In-Context Configurations for Image Captioning: An External & Internal Analysis
For Flamingo-style models, increasing the number of in-context examples improves language coherence but degrades visual-text alignment, and similarity-based image retrieval inflates CIDEr scores by encouraging caption...
-
Method-Based Reasoning for Large Language Models: Extraction, Reuse, and Continuous Improvement
An external store of problem-solution prompt pairs is retrieved and injected into LLM queries, and toy experiments claim this improves factual checking and generalizes across made-up software names.
Discussion (0). Sign in to comment.