REVIEW 7 cited by
Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
Reasoning encompasses two typical types: deductive reasoning and inductive reasoning. Despite extensive research into the reasoning capabilities of Large Language Models (LLMs), most studies have failed to rigorously differentiate between inductive and deductive reasoning, leading to a blending of the two. This raises an essential question: In LLM reasoning, which poses a greater challenge - deductive or inductive reasoning? While the deductive reasoning capabilities of LLMs, (i.e. their capacity to follow instructions in reasoning tasks), have received considerable attention, their abilities in true inductive reasoning remain largely unexplored. To investigate into the true inductive reasoning capabilities of LLMs, we propose a novel framework, SolverLearner. This framework enables LLMs to learn the underlying function (i.e., $y = f_w(x)$), that maps input data points $(x)$ to their corresponding output values $(y)$, using only in-context examples. By focusing on inductive reasoning and separating it from LLM-based deductive reasoning, we can isolate and investigate inductive reasoning of LLMs in its pure form via SolverLearner. Our observations reveal that LLMs demonstrate remarkable inductive reasoning capabilities through SolverLearner, achieving near-perfect performance with ACC of 1 in most cases. Surprisingly, despite their strong inductive reasoning abilities, LLMs tend to relatively lack deductive reasoning capabilities, particularly in tasks involving ``counterfactual'' reasoning.
Forward citations
Cited by 7 Pith papers
-
Fostering Video Reasoning via Next-Event Prediction
Next-event prediction, training video language models to caption unseen future frames, improves their scores on several temporal benchmarks while roughly preserving general video understanding.
-
One Example Shown, Many Concepts Known! Counterexample-Driven Conceptual Reasoning in Mathematical LLMs
LLMs score poorly on CounterMATH, a new counterexample-based university math benchmark, and a 1,025-sample counterexample fine-tune yields small and partly inconsistent gains.
-
Does Few-Shot Learning Help LLM Performance in Code Synthesis?
Few-shot example choice measurably affects LLM code output, and two proposed selectors (a perplexity ranker and a trained MLP ranker) each improve CodeLlama's Pass@1 on HumanEval+ by about five points.
-
Large Reasoning Models are not thinking straight: on the unreliability of thinking trajectories
Large reasoning models frequently ignore explicitly injected ground-truth solutions mid-trajectory and continue to overthink, often ending on incorrect answers.
-
Improve Rule Retrieval and Reasoning with Self-Induction and Relevance ReEstimate
Using an LLM to induce an abstract rule from a query and then re-ranking retrieved rules with an LLM prompt improves rule retrieval and downstream reasoning in most tested configurations.
-
Generating Symbolic World Models via Test-time Scaling of Large Language Models
Best-of-N sampling plus iterative self-critique, called iVML, lets a 7B open LLM generate PDDL planning domains with over 85% and 71% success on two benchmarks, outperforming o1-mini.
-
On the Role of Model Prior in Real-World Inductive Reasoning
LLMs' hypotheses for real-world classification tasks are driven mostly by task priors, and in-context demonstrations, even with flipped labels, do little to change them.
Discussion (0). Continue with ORCID to comment.