Pith. sign in

REVIEW 3 major objections 5 minor 72 references

Structural ensembling of reasoning graphs — not just final answers — improves LLM accuracy and returns an auditable consensus chain.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 01:26 UTC pith:VJ56QUDD

load-bearing objection Novel structural ensembling, but the 'beats majority vote' claim is confounded and mostly within noise. the 3 major comments →

arxiv 2607.27783 v1 pith:VJ56QUDD submitted 2026-07-30 cs.CL cs.AIcs.LG

Reasoning Consensus: Structural Ensembling of LLM Reasoning via Weighted DAG Aggregation

Amruta Parulekar , Jinu Lee , Dilek Hakkani-T\"ur , Hari Sundaram This is my paper
classification cs.CL cs.AIcs.LG
keywords LLM reasoningchain-of-thoughtensemble methodsreasoning graphsDAG aggregationself-consistencyinterpretabilitymulti-model voting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that when multiple language models reason aloud, the reasoning steps themselves carry a consensus signal that final answers discard. It extracts a directed acyclic graph from each chain of thought, merges graphs across models while preserving logical 'bundles' of jointly supporting premises, weights each merged node by how many traces independently attest to it, and returns the highest-weighted conclusion together with its strongest supporting subgraph. Across six benchmarks spanning statutory interpretation, graduate science, narrative multi-hop reasoning, and first-order logic, this structural ensemble beats majority voting over the same traces at the same budget (up to +3.1% accuracy), and matches or exceeds self-consistency on a single model. Its stated value beyond accuracy is inspectability: users see which steps are well-supported, which alternatives were considered, and how much support each conclusion received.

Core claim

The central claim is that aggregating the internal reasoning structure of LLM traces—rather than only their final answers—produces both accuracy gains and an auditable artifact. Specifically, the paper claims that a weighted merge of DAGs extracted from multiple traces, with weights derived purely from cross-trace attestation, outperforms a matched-budget majority-vote baseline on every dataset tested (maximum +3.1% on a narrative multi-hop benchmark) and matches or exceeds self-consistency on single-model trace pools. The resulting consensus subgraph is preferred by human and LLM judges over random alternative justifications leading to the same answer, with win rates 54.4–65.4% in five of s

What carries the argument

The load-bearing mechanism is bundle-preserving weighted DAG aggregation. Each reasoning trace is converted into an AND/OR DAG whose nodes carry typed labels (Planning, Fact, Reasoning, Conclusion) and whose edges are grouped into bundles: a support bundle is a conjunctive set of premises that jointly justify a node, and multiple bundles are disjunctive alternatives. A merge operator maps nodes across traces to clusters (pruned by embedding similarity, verified by an LLM judge) and fuses identical bundles, so each merged node accumulates an attestation count. Node scores are computed topologically: a bundle's strength is the weakest-link minimum over its members, OR gates take the max over a

Load-bearing premise

The whole pipeline assumes a small extractor model faithfully converts free-form chains of thought into typed, bundle-structured DAGs; if extraction introduces systematic errors, the attestation counts and consensus graphs inherit them.

What would settle it

Take a dataset where majority voting clearly outperforms any single model, run the framework with a deliberately corrupted extractor (e.g., one that randomly swaps or drops nodes), and check whether ensemble accuracy collapses to or below the majority-vote baseline while node weights still look plausible. Alternatively, on a logic benchmark with gold proofs, compare the extracted consensus graph's inferred AND/OR structure against the ground-truth proof: if high-accuracy answers frequently ride on incorrectly reconstructed structure, the claimed mechanism is not doing the work.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • At a fixed trace budget, structural ensembling outperforms answer-level majority voting on all six benchmarks, with the largest gain on narrative multi-hop reasoning.
  • On a single model, the framework matches or exceeds self-consistency while additionally providing an inspectable reasoning graph.
  • Ensemble weights track LLM-judge rankings of reasoning quality (Spearman rho = 0.30–0.51), so the weight signal is not just repetition.
  • The framework preserves competing conclusions and alternative justifications, useful for audit and for open-ended tasks like argument generation.
  • Because step-level confidence is derived from cross-trace agreement, no process reward model or step-level labels are needed; it scales to low-resource domains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The DAG representation invites a natural user-intervention interface: if a specific step is wrong, a human could edit or delete that node and re-propagate weights; the paper does not test this, but the structure makes it possible.
  • Attestation weighting is a training-free proxy for process supervision; if extraction noise is controlled, it might transfer to single-model settings that use diverse prompts or temperatures to simulate multiple traces.
  • The AND/OR bundle semantics could be linked to probabilistic logic or argumentation theory to yield calibrated confidence intervals on each step, a direction the paper only touches qualitatively.
  • One testable extension: on a logic benchmark with gold proofs, compare the extracted consensus graph's logical form against the ground-truth proof to see whether accuracy gains actually come from correct structure reconstruction rather than from answer-frequency artifacts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a framework for ensembling LLM reasoning at the structural level. It extracts a typed, bundle-structured DAG from each chain-of-thought trace, merges nodes across traces via an embedding filter plus an LLM judge, weights merged nodes by cross-trace attestation (simple or accuracy-weighted), and selects the highest-weighted conclusion together with its maximally supported proof subgraph. The framework is evaluated on six reasoning benchmarks (SARA, GPQA-D, three MuSR domains, FOLIO) using a four-model pool with ≤12B parameters, comparing against majority voting and self-consistency at equal trace budgets. The paper reports accuracy gains over the majority-vote baseline (max +3.1% on MuSR-MM), matches or exceeds self-consistency on single-model pools, and provides evidence via LLM-judge and human evaluation that the consensus subgraphs are preferred as justifications.

Significance. If the accuracy claims were established, the framework would be a useful contribution to auditable multi-model reasoning: it generalizes self-consistency from answers to reasoning structure, requires no trained process reward model, and returns an inspectable weighted graph with alternative justifications. The method is clearly specified and reproducible: open-source models, fixed prompts in the appendix, and a stated random seed. The qualitative/explainability results—consensus graphs preferred over length-matched alternatives in 54.4–65.4% of comparisons and moderate Spearman correlations with an independent judge—support the qualitative value of the approach. However, the headline quantitative claim—'outperforms matched-budget majority-vote baseline on every dataset'—is not currently supported by the reported experiments, for the reasons detailed below.

major comments (3)
  1. [§4, §5.1, Table 1(a)] The claimed 'matched-budget' comparison is confounded for the accuracy-weighted rows. Section 4 states that a 50-example held-out split is used to estimate per-model accuracy priors W_acc^m, and Section 3.4 uses these priors for the accuracy-based weights. The baseline is an unweighted 4-model majority vote over the same 20 traces, so the comparison varies two factors simultaneously: (i) DAG aggregation vs. voting, and (ii) access to 50 labeled examples per dataset. The paper's assertion that the difference 'reflects how traces are combined' (§4) is therefore valid only for the simple-weighting rows, which beat majority vote on only 3/6 datasets and lose/tie on SARA, MuSR-OP, and FOLIO. The missing control is an accuracy-weighted majority vote using the same W_acc^m; without it, the 6/6 accuracy gain cannot be attributed to structural aggregation.
  2. [§5.1, Table 1(a)] The claim that the accuracy-weighted ensemble 'outperforms' the majority-vote baseline on 'every dataset' is not statistically supported. For SARA (0.6387±0.0080 vs 0.6279±0.0158), GPQA-D (0.2784±0.0495 vs 0.2662±0.0421), MuSR-OP (0.4922±0.0174 vs 0.4874±0.0184), and MuSR-TA (0.2890±0.0167 vs 0.2830±0.0057), the differences are within one standard deviation of the baseline. FOLIO ties the baseline exactly at the point estimate (0.6405 vs 0.6405), contradicting the 'outperforms' wording in the abstract and Section 1. Only MuSR-MM shows a difference (59.6 vs 56.5) that exceeds the reported standard deviations. The abstract and conclusion should be revised to reflect the specific datasets where gains are statistically meaningful, or additional evidence (e.g., paired tests across seeds) should be provided.
  3. [§3.4, §8] The 'training-free' characterization of the accuracy-weighting scheme is potentially misleading. The method uses 50 ground-truth labels per dataset to set per-model priors (Section 4), which is not 'training' in the model-parameters sense, but it does consume labeled data. This is a minor terminology issue, but it matters for the comparison to majority vote, which sees no labels. The distinction should be stated clearly: the simple-weighting variant is label-free; the accuracy-weighted variant is not.
minor comments (5)
  1. [§5.1, Table 1 caption] Table 1(a) is difficult to parse because the baseline row is placed after two ensemble rows; consider listing the baseline first or adding a separator. Also, the 'Bold/underline' note says 'best/second-best per column in each block' but the table does not bold any values; either apply the formatting or remove the note.
  2. [§5.3, Table 2] The Spearman correlations and win rates are computed on a per-question basis and then averaged; the paper should state whether these averages are over questions or over seeds, and how many questions are included. The 'n=5 subgraphs per question' is mentioned, but the total number of questions per dataset is not.
  3. [§3.3, Eq. (2)] The definition of the merged support function is clear, but the notation π^{-1}(c) is ambiguous when clusters are built from a clustering π: V → C; consider using [c] or a subscripted class notation to avoid confusion with an inverse function.
  4. [§4] The temperature of the trace generators is reported as 0.6, but the number of decoding samples per trace is not; the 'temperature' is a free parameter and the reader cannot reproduce the exact sampling distribution without the seed/sample count. Please report the number of independent generations per model per question.
  5. [§5.2, Table 1(c)] The strong-model homogeneous ensemble uses Qwen 2.5 32B as trace generator and the same model as DAG extractor (Section 4). This shared identity could introduce a systematic bias in the extracted structure. At minimum, a sentence acknowledging this and arguing why it is not circular would be helpful.

Circularity Check

0 steps flagged

No significant circularity: the weighted-DAG consensus is evaluated on held-out external benchmarks, and the accuracy-prior confound is a control issue, not a circular reduction.

full rationale

The derivation chain is self-contained. The consensus answer (Eq. 8) and consensus subgraph (Section 3.5) are defined by the scoring equations (Eqs. 3–7), but the central accuracy claims are evaluated on held-out splits of six external benchmarks (Section 4), so the reported predictions are not defined in terms of the fitted per-model weights. The only self-citation (Lee et al., 2025, motivating the four DAG node labels in Section 3.2) is a design inspiration, not a load-bearing premise: the empirical comparisons between weighted-DAG aggregation and majority vote / self-consistency do not rely on that prior paper's validity. The 'matched-budget' accuracy-weighted comparison is weakened by the fact that the majority-vote baseline does not use the 50-example held-out priors, and the paper itself concedes 'the prior delivers the additional boost for consistent wins'; however, this is an experimental-control concern, not a circular reduction of output to input. Similarly, the acknowledged limitation that 'Extraction errors would propagate through aggregation' is a robustness caveat, not a circular step. No fitted parameter is renamed as a prediction, and no uniqueness theorem or ansatz is imported from the authors' prior work to force the result.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The framework introduces no new physical entities; its free parameters are a merge threshold, held-out model-accuracy priors, and decoding temperatures. The main load-bearing assumptions are about the faithfulness of LLM-based extraction/merging and about attestation as a proxy for correctness.

free parameters (3)
  • Embedding merge threshold tau' = 0.55
    Hand-chosen permissive threshold that prunes node pairs before LLM verification; directly controls which nodes merge and therefore all attestation counts.
  • Per-model accuracy priors W_acc^m = held-out accuracies from 50 examples per dataset
    The accuracy weighting scheme sets each trace's vote to its source model's held-out accuracy; these priors are empirical inputs, not derived from the framework.
  • Temperatures = 0.6 (trace sampling), 0.3 (DAG extraction)
    Decoding temperatures chosen by hand; they affect trace diversity and extraction determinism but are not fitted to the target benchmarks.
axioms (4)
  • domain assumption Free-form CoT traces are faithfully convertible to typed AND/OR DAGs by an LLM extractor.
    Invoked in Section 3.2; the whole aggregation operates on extracted DAGs. Errors propagate, as the Limitations acknowledge.
  • domain assumption The LLM merge judge correctly decides node equivalence.
    Invoked in Section 3.3; false merges change attestation structure and all downstream weights.
  • domain assumption Attestation frequency is a valid proxy for reasoning correctness.
    Invoked in Section 3.4 and Section 6; steps attested by many traces are weighted higher. This could fail if models share correlated biases.
  • ad hoc to paper The scoring equations (3)-(7) define the 'right' consensus.
    W(v)=max(0, alpha+ * Phi+ - alpha- * Phi-) and the weakest-link bundle strength are design choices, not consequences of a derived optimality principle.

pith-pipeline@v1.3.0-daily-deepseek · 16327 in / 12477 out tokens · 114997 ms · 2026-08-01T01:26:48.756819+00:00 · methodology

0 comments
read the original abstract

Large Language Models (LLMs) explore problems through chain-of-thought, but this exploration is buried in unstructured prose. On high-stakes tasks, users cannot tell which steps are well-supported, which alternatives were seriously considered, or how the final conclusion compares to those the model discarded. We propose a framework that ensembles the reasoning structure, not just the answers, of multiple LLMs by weighted merging of Directed Acyclic Graphs (DAGs) extracted from reasoning chains. We weight each step by how many traces independently attest to it, to return "Consensus Reasoning". Across six benchmarks spanning statutory interpretation, graduate-level science, narrative multi-hop reasoning, and first-order logic, our ensemble outperforms a matched-budget majority-vote baseline, with a maximum accuracy gain of 3.1% on MuSR-MM (narrative multi-hop reasoning). On a single model, the framework matches or exceeds self-consistency at the same trace budget while additionally exposing an inspectable consensus reasoning graph. Ensemble weights correlate with LLM-judge rankings of reasoning quality at Spearman $\rho = 0.30$-$0.51$, and consensus subgraphs are preferred over alternatives leading to the majority-vote answer in 54.4-65.4% of head-to-head comparisons across five of six datasets. We observe that our framework can also be used to analyze diverse reasoning perspectives for a problem.

Figures

Figures reproduced from arXiv: 2607.27783 by Amruta Parulekar, Dilek Hakkani-T\"ur, Hari Sundaram, Jinu Lee.

Figure 1
Figure 1. Figure 1: Detailed pipeline. Step 1: Multiple LLMs sample reasoning traces for a query. Step 2: Each trace is decomposed into a DAG with nodes labeled Planning (P), Fact (F), Reasoning (Re), and Conclusion (C). Typed node labels let independently generated DAGs be aligned and merged step-by-step rather than compared as opaque prose. Step 3: Nodes are merged based on semantic similarity. Our hybrid ensembling strateg… view at source ↗
Figure 2
Figure 2. Figure 2: Three reasoning graphs reaching the same conclusion (Answer A) for a MuSR Murder Mysteries question. [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Three reasoning graphs reaching the same conclusion (Answer A) for a MuSR Murder Mysteries question. [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Three reasoning graphs reaching the same [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

72 extracted references · 1 canonical work pages

  1. [1]

    Causal Reasoning and Large Language Models: Opening a New Frontier for Causality , year =

    Emre Kıcıman and Robert Ness and Amit Sharma and Chenhao Tan , date-added =. Causal Reasoning and Large Language Models: Opening a New Frontier for Causality , year =

  2. [2]

    Cladder: Assessing causal reasoning in language models , volume =

    Jin, Zhijing and Chen, Yuen and Leeb, Felix and Gresele, Luigi and Kamal, Ojasv and Lyu, Zhiheng and Blin, Kevin and Gonzalez Adauto, Fernando and Kleiman-Weiner, Max and Sachan, Mrinmaya and others , date-added =. Cladder: Assessing causal reasoning in language models , volume =. Advances in Neural Information Processing Systems , pages =. 2023 , bdsk-file-1 =

    2023

  3. [3]

    Distinguishing cause from effect using observational data: methods and benchmarks , volume =

    Mooij, Joris M and Peters, Jonas and Janzing, Dominik and Zscheischler, Jakob and Sch. Distinguishing cause from effect using observational data: methods and benchmarks , volume =. The Journal of Machine Learning Research , number =. 2016 , bdsk-file-1 =

    2016

  4. [4]

    The origin of bursts and heavy tails in human dynamics , volume =

    Barabasi, Albert-Laszlo , date-added =. The origin of bursts and heavy tails in human dynamics , volume =. Nature , number =. 2005 , bdsk-file-1 =

    2005

  5. [5]

    Attention is all you need , year =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser,. Attention is all you need , year =. Advances in Neural Information Processing Systems , pages =

  6. [6]

    Insights from the Long-Tail: Learning Latent Representations of Online User Behavior in the Presence of Skew and Sparsity , year =

    Krishnan, Adit and Sharma, Ashish and Sundaram, Hari , booktitle =. Insights from the Long-Tail: Learning Latent Representations of Online User Behavior in the Presence of Skew and Sparsity , year =

  7. [7]

    An Adversarial Approach to Improve Long-Tail Performance in Neural Collaborative Filtering , year =

    Krishnan, Adit and Sharma, Ashish and Sankar, Aravind and Sundaram, Hari , booktitle =. An Adversarial Approach to Improve Long-Tail Performance in Neural Collaborative Filtering , year =

  8. [8]

    Improving Latent User Models in Online Social Media , year =

    Krishnan, Adit and Sharma, Ashish and Sundaram, Hari , journal =. Improving Latent User Models in Online Social Media , year =

  9. [9]

    Variational Autoencoders for Collaborative Filtering , year =

    Liang, Dawen and Krishnan, Rahul G and Hoffman, Matthew D and Jebara, Tony , journal =. Variational Autoencoders for Collaborative Filtering , year =

  10. [10]

    Collaborative filtering with temporal dynamics , year =

    Koren, Yehuda , booktitle =. Collaborative filtering with temporal dynamics , year =

  11. [11]

    Neural collaborative filtering , year =

    He, Xiangnan and Liao, Lizi and Zhang, Hanwang and Nie, Liqiang and Hu, Xia and Chua, Tat-Seng , booktitle =. Neural collaborative filtering , year =

  12. [12]

    Fema: flexible evolutionary multi-faceted analysis for dynamic behavioral pattern discovery , year =

    Jiang, Meng and Cui, Peng and Wang, Fei and Xu, Xinran and Zhu, Wenwu and Yang, Shiqiang , booktitle =. Fema: flexible evolutionary multi-faceted analysis for dynamic behavioral pattern discovery , year =

  13. [13]

    Recurrent recommender networks , year =

    Wu, Chao-Yuan and Ahmed, Amr and Beutel, Alex and Smola, Alexander J and Jing, How , booktitle =. Recurrent recommender networks , year =

  14. [14]

    Disan: Directional self-attention network for rnn/cnn-free language understanding , year =

    Shen, Tao and Zhou, Tianyi and Long, Guodong and Jiang, Jing and Pan, Shirui and Zhang, Chengqi , journal =. Disan: Directional self-attention network for rnn/cnn-free language understanding , year =

  15. [15]

    Challenging the long tail recommendation , volume =

    Yin, Hongzhi and Cui, Bin and Li, Jing and Yao, Junjie and Chen, Chen , journal =. Challenging the long tail recommendation , volume =

  16. [16]

    2025 , bdsk-url-1 =

    Fine-tuning Large Language Models for Improving Factuality in Legal Question Answering , url =. 2025 , bdsk-url-1 =. arXiv , author =:2501.06521 , primaryclass =

    Pith/arXiv arXiv 2025

  17. [17]

    Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models , url =

    Dahl, Matthew and Magesh, Varun and Suzgun, Mirac and Ho, Daniel E , doi =. Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models , url =. Journal of Legal Analysis , month = jan, number =. 2024 , bdsk-url-1 =

    2024

  18. [18]

    2025 , bdsk-url-1 =

    MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMs , url =. 2025 , bdsk-url-1 =. arXiv , author =:2507.17476 , primaryclass =

    Pith/arXiv arXiv 2025

  19. [19]

    2024 , bdsk-url-1 =

    Large language models in healthcare and medical domain: A review , url =. 2024 , bdsk-url-1 =. arXiv , author =:2401.06775 , primaryclass =

    Pith/arXiv arXiv 2024

  20. [20]

    2025 , bdsk-url-1 =

    A Survey of Multilingual Reasoning in Language Models , url =. 2025 , bdsk-url-1 =. arXiv , author =:2502.09457 , primaryclass =

    arXiv 2025

  21. [21]

    2025 , bdsk-url-1 =

    Evaluating the Limits of Large Language Models in Multilingual Legal Reasoning , url =. 2025 , bdsk-url-1 =. arXiv , author =:2509.22472 , primaryclass =

    arXiv 2025

  22. [22]

    2024 , bdsk-url-1 =

    Vishvaksenan Rasiah and Ronja Stern and Veton Matoshi and Matthias St. 2024 , bdsk-url-1 =

    2024

  23. [23]

    2019 , bdsk-url-1 =

    What do Language Representations Really Represent? , url =. 2019 , bdsk-url-1 =. arXiv , author =:1901.02646 , primaryclass =

    Pith/arXiv arXiv 2019

  24. [24]

    2024 , bdsk-url-1 =

    PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning , url =. 2024 , bdsk-url-1 =. arXiv , author =:2311.08711 , primaryclass =

    Pith/arXiv arXiv 2024

  25. [25]

    2019 , bdsk-url-1 =

    Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks , url =. 2019 , bdsk-url-1 =. arXiv , author =:1908.10084 , primaryclass =

    Pith/arXiv arXiv 2019

  26. [26]

    2020 , bdsk-url-1 =

    BERTScore: Evaluating Text Generation with BERT , url =. 2020 , bdsk-url-1 =. arXiv , author =:1904.09675 , primaryclass =

    Pith/arXiv arXiv 2020

  27. [27]

    2024 , bdsk-url-1 =

    The Llama 3 Herd of Models , url =. 2024 , bdsk-url-1 =. arXiv , author =:2407.21783 , primaryclass =

    Pith/arXiv arXiv 2024

  28. [28]

    2024 , bdsk-url-1 =

    Identification of Entailment and Contradiction Relations between Natural Language Sentences: A Neurosymbolic Approach , url =. 2024 , bdsk-url-1 =. arXiv , author =:2405.01259 , primaryclass =

    Pith/arXiv arXiv 2024

  29. [29]

    2025 , bdsk-url-1 =

    Qwen3 Technical Report , url =. 2025 , bdsk-url-1 =. arXiv , author =:2505.09388 , primaryclass =

    Pith/arXiv arXiv 2025

  30. [30]

    2025 , bdsk-url-1 =

    Gemini: A Family of Highly Capable Multimodal Models , url =. 2025 , bdsk-url-1 =. arXiv , author =:2312.11805 , primaryclass =

    Pith/arXiv arXiv 2025

  31. [31]

    2020 , bdsk-url-1 =

    Language Models are Few-Shot Learners , url =. 2020 , bdsk-url-1 =. arXiv , author =:2005.14165 , primaryclass =

    Pith/arXiv arXiv 2020

  32. [32]

    2025 , bdsk-url-1 =

    Mitigating Hallucinations in Large Language Models via Causal Reasoning , url =. 2025 , bdsk-url-1 =. arXiv , author =:2508.12495 , primaryclass =

    arXiv 2025

  33. [33]

    2024 , bdsk-url-1 =

    CLadder: Assessing Causal Reasoning in Language Models , url =. 2024 , bdsk-url-1 =. arXiv , author =:2312.04350 , primaryclass =

    Pith/arXiv arXiv 2024

  34. [34]

    2021 , bdsk-url-1 =

    Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis , url =. 2021 , bdsk-url-1 =. arXiv , author =:2104.09420 , primaryclass =

    Pith/arXiv arXiv 2021

  35. [35]

    2025 , bdsk-url-1 =

    Causal Inference with Large Language Model: A Survey , url =. 2025 , bdsk-url-1 =. arXiv , author =:2409.09822 , primaryclass =

    Pith/arXiv arXiv 2025

  36. [36]

    2024 , bdsk-url-1 =

    Wang, Zeyu , booktitle =. 2024 , bdsk-url-1 =

    2024

  37. [37]

    2019 , bdsk-url-1 =

    Choosing Transfer Languages for Cross-Lingual Learning , url =. 2019 , bdsk-url-1 =. arXiv , author =:1905.12688 , primaryclass =

    Pith/arXiv arXiv 2019

  38. [38]

    and Henderson, Peter and Ho, Daniel E

    Zheng, Lucia and Guha, Neel and Arifov, Javokhir and Zhang, Sarah and Skreta, Michal and Manning, Christopher D. and Henderson, Peter and Ho, Daniel E. , booktitle =. A Reasoning-Focused Legal Retrieval Benchmark , url =. 2025 , bdsk-url-1 =. doi:10.1145/3709025.3712219 , month = mar, pages =

    arXiv 2025

  39. [39]

    2020 , bdsk-url-1 =

    A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering , url =. 2020 , bdsk-url-1 =. arXiv , author =:2005.05257 , primaryclass =

    Pith/arXiv arXiv 2020

  40. [40]

    2023 , bdsk-url-1 =

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , url =. 2023 , bdsk-url-1 =. arXiv , author =:2201.11903 , primaryclass =

    Pith/arXiv arXiv 2023

  41. [41]

    2025 , bdsk-url-1 =

    Efficient Dynamic Ensembling for Multiple LLM Experts , url =. 2025 , bdsk-url-1 =. arXiv , author =:2412.07448 , primaryclass =

    arXiv 2025

  42. [42]

    Collaborative Beam Search: Enhancing

    Xu, Yangyifan and Ren, Shuo and Zhang, Jiajun , booktitle =. Collaborative Beam Search: Enhancing. 2025 , bdsk-url-1 =. doi:10.18653/v1/2025.emnlp-main.574 , editor =

    work page doi:10.18653/v1/2025.emnlp-main.574 2025

  43. [43]

    2023 , bdsk-url-1 =

    LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models , url =. 2023 , bdsk-url-1 =. arXiv , author =:2308.11462 , primaryclass =

    Pith/arXiv arXiv 2023

  44. [44]

    2024 , bdsk-url-1 =

    WikiCausal: Corpus and Evaluation Framework for Causal Knowledge Graph Construction , url =. 2024 , bdsk-url-1 =. arXiv , author =:2409.00331 , primaryclass =

    Pith/arXiv arXiv 2024

  45. [45]

    2024 , eprint=

    Causal Reasoning and Large Language Models: Opening a New Frontier for Causality , author=. 2024 , eprint=

    2024

  46. [46]

    2023 , eprint=

    Causal Parrots: Large Language Models May Talk Causality But Are Not Causal , author=. 2023 , eprint=

    2023

  47. [47]

    2026 , eprint=

    From Chains to DAGs: Probing the Graph Structure of Reasoning in LLMs , author=. 2026 , eprint=

    2026

  48. [48]

    Guo, Daya and Yang, Dejian and Zhang, Haowei and Song, Junxiao and Wang, Peiyi and Zhu, Qihao and Xu, Runxin and Zhang, Ruoyu and Ma, Shirong and Bi, Xiao and Zhang, Xiaokang and Yu, Xingkai and Wu, Yu and Wu, Z. F. and Gou, Zhibin and Shao, Zhihong and Li, Zhuoshu and Gao, Ziyi and Liu, Aixin and Xue, Bing and Wang, Bingxuan and Wu, Bochao and Feng, Bei ...

  49. [49]

    2024 , eprint=

    OpenAI o1 System Card , author=. 2024 , eprint=

    2024

  50. [50]

    Graph of Thoughts: Solving Elaborate Problems with Large Language Models , volume=

    Besta, Maciej and Blach, Nils and Kubicek, Ales and Gerstenberger, Robert and Podstawski, Michal and Gianinazzi, Lukas and Gajda, Joanna and Lehmann, Tomasz and Niewiadomski, Hubert and Nyczyk, Piotr and Hoefler, Torsten , year=. Graph of Thoughts: Solving Elaborate Problems with Large Language Models , volume=. Proceedings of the AAAI Conference on Artif...

    doi:10.1609/aaai.v38i16.29720

  51. [51]

    2023 , eprint=

    Self-Consistency Improves Chain of Thought Reasoning in Language Models , author=. 2023 , eprint=

    2023

  52. [52]

    2025 , eprint=

    The Power of Network Pluralism: Multi-Perspective Modeling of Heterogeneous Legal Document Networks , author=. 2025 , eprint=

    2025

  53. [53]

    The Fourteenth International Conference on Learning Representations , year=

    Causal Discovery in the Wild: A Voting-Theoretic Ensemble Approach , author=. The Fourteenth International Conference on Learning Representations , year=

  54. [54]

    2025 , eprint=

    ReasoningFlow: Semantic Structure of Complex Reasoning Traces , author=. 2025 , eprint=

    2025

  55. [55]

    From Arguments to Key Points: T owards Automatic Argument Summarization

    Bar-Haim, Roy and Eden, Lilach and Friedman, Roni and Kantor, Yoav and Lahav, Dan and Slonim, Noam. From Arguments to Key Points: T owards Automatic Argument Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. doi:10.18653/v1/2020.acl-main.371

    doi:10.18653/v1/2020.acl-main.371 2020

  56. [56]

    2023 , eprint=

    GPQA: A Graduate-Level Google-Proof Q&A Benchmark , author=. 2023 , eprint=

    2023

  57. [57]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  58. [58]

    2025 , eprint=

    Gemma 3 Technical Report , author=. 2025 , eprint=

    2025

  59. [59]

    2023 , eprint=

    Mistral 7B , author=. 2023 , eprint=

    2023

  60. [60]

    Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

    Reimers, Nils and Gurevych, Iryna. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. 2019

    2019

  61. [61]

    2024 , eprint=

    Mixture-of-Agents Enhances Large Language Model Capabilities , author=. 2024 , eprint=

    2024

  62. [62]

    2024 , eprint=

    ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs , author=. 2024 , eprint=

    2024

  63. [63]

    2024 , eprint=

    Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning , author=. 2024 , eprint=

    2024

  64. [64]

    2025 , eprint=

    CMCTS: A Constrained Monte Carlo Tree Search Framework for Mathematical Reasoning in Large Language Model , author=. 2025 , eprint=

    2025

  65. [65]

    2025 , eprint=

    Multi-LLM Collaborative Search for Complex Problem Solving , author=. 2025 , eprint=

    2025

  66. [66]

    2024 , eprint=

    Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learning , author=. 2024 , eprint=

    2024

  67. [67]

    2023 , eprint=

    Tree of Thoughts: Deliberate Problem Solving with Large Language Models , author=. 2023 , eprint=

    2023

  68. [68]

    2026 , eprint=

    ARGORA: Orchestrated Argumentation for Causally Grounded LLM Reasoning and Decision Making , author=. 2026 , eprint=

    2026

  69. [69]

    2026 , eprint=

    Beyond Accuracy: Evaluating Strategy Diversity in LLM Mathematical Reasoning , author=. 2026 , eprint=

    2026

  70. [70]

    2026 , eprint=

    HalluGuard: Demystifying Data-Driven and Reasoning-Driven Hallucinations in LLMs , author=. 2026 , eprint=

    2026

  71. [71]

    2024 , eprint=

    FOLIO: Natural Language Reasoning with First-Order Logic , author=. 2024 , eprint=

    2024

  72. [72]

    2024 , eprint=

    MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning , author=. 2024 , eprint=

    2024