REVIEW 6 cited by
Chain of Logic: Rule-Based Reasoning with Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Rule-based reasoning, a fundamental type of legal reasoning, enables us to draw conclusions by accurately applying a rule to a set of facts. We explore causal language models as rule-based reasoners, specifically with respect to compositional rules - rules consisting of multiple elements which form a complex logical expression. Reasoning about compositional rules is challenging because it requires multiple reasoning steps, and attending to the logical relationships between elements. We introduce a new prompting method, Chain of Logic, which elicits rule-based reasoning through decomposition (solving elements as independent threads of logic), and recomposition (recombining these sub-answers to resolve the underlying logical expression). This method was inspired by the IRAC (Issue, Rule, Application, Conclusion) framework, a sequential reasoning approach used by lawyers. We evaluate chain of logic across eight rule-based reasoning tasks involving three distinct compositional rules from the LegalBench benchmark and demonstrate it consistently outperforms other prompting methods, including chain of thought and self-ask, using open-source and commercial language models.
Forward citations
Cited by 6 Pith papers
-
Animating Petascale Time-varying Data on Commodity Hardware with LLM-assisted Scripting
A framework with Generalized Animation Descriptor, cloud data access, tailored rendering, and LLM scripting enables 3D animations of petascale datasets on commodity workstations in minutes to hours.
-
Rule2Text: A Framework for Generating and Evaluating Natural Language Explanations of Knowledge Graph Rules
Rule2Text generates and evaluates natural language explanations of knowledge graph rules, finding that chain-of-thought prompting with entity types works best and that fine-tuning Zephyr on LLM-built ground truth shar...
-
Rule2Text: Natural Language Explanation of Logical Rules in Knowledge Graphs
LLMs generate mostly correct and clear explanations of knowledge-graph logical rules, and combining chain-of-thought prompting with entity type hints improves quality.
-
Thinking Economically: A Hierarchical Framework for Adaptive-Complexity Reasoning in LLMs
HAB applies coarse-to-fine budgeting to LLM reasoning, predicting per-problem depth and learning intra-step token budgets via PPL comparisons and adaptive Pareto optimization, yielding higher accuracy and lower token ...
-
Animating Petascale Time-varying Data on Commodity Hardware with LLM-assisted Scripting
An LLM-assisted, keyframe-based animation framework streams cloud-hosted petascale datasets to commodity hardware and generates 3D scientific animations from natural-language requests.
-
Do Biased Models Have Biased Thoughts?
The manuscript is internally inconsistent: the abstract describes an LLM fairness experiment while the body is a different paper on pilot-wave quantum mechanics, so no coherent result can be assessed.
Discussion (0). Sign in to comment.