Pith. sign in

REVIEW 5 major objections 6 minor 31 references

LLMSR@XLLM25: An Empirical Study of LLM for Structural Reasoning

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Few-shot prompting on an untuned 8B model reaches 5th place on a structured reasoning task.

desk verdict Solid shared-task report with a real test-set selection bias; the 5th place is credible but the 'parity with heavier pipelines' claim is oversold. read the letter →

arxiv 2505.12328 v1 pith:5MBPAFLM submitted 2025-05-18 cs.CL

Xinye Li , Mingqi Wan , Dianbo Sui This is my paper · ORCID
classification cs.CL
keywords structuralreasoningchain-of-thoughtpromptingfew-shotin-contextlearningquestionparsingstatement-evidenceverificationlogicalentailmentMeta-Llama-3-8B-Instructprocessrewardmodeling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a deliberately minimal system for the LLMSR@XLLM25 shared task, where a model must extract all conditions from a problem, split a chain-of-thought into statement–evidence pairs, and judge whether each evidence span entails its statement. The authors claim that an untuned Meta-Llama-3-8B-Instruct, driven by a concise few-shot multi-turn prompt and a regex-based post-processor, ranks 5th overall and achieves macro-F1 scores comparable to substantially more complex and resource-consuming pipelines. The system uses two demonstrations for question parsing and three for CoT parsing and verification, and it follows a three-turn template: system rules, user problem, assistant JSON answer. A reader should care because the result suggests that prompt design and output hygiene, rather than fine-tuning or retrieval, can carry an 8B model to the top of a structured-reasoning leaderboard, while also exposing logical adjudication as the real bottleneck.

What carries the argument

The load-bearing mechanism is a two-call, three-turn prompted pipeline with a hand-picked few-shot budget: the first call extracts every explicit condition as an ordered list (Question Parsing), and a second call consumes the question, its chain-of-thought, and that condition list to emit statement–evidence pairs together with a boolean entailment verdict (CoT Parsing & Verification). Each call is a SYSTEM–USER–ASSISTANT conversation, demonstrations are fenced in ```json``` blocks, and a deterministic regex post-processor checks the schema, normalizes spans, merges duplicate conditions, and aligns statement and evidence counts. The key empirical lever is that the two stages receive different demonstration counts, k=2 for QP and k=3 for CP, chosen by separate ablations, and that separating the stages beats a single fixed prompt size.

What would settle it

Rerun the pipeline with shot counts chosen on a held-out validation split instead of the official test sets, then report macro-F1 on the untouched test sets. If the final numbers fall materially below the reported 75.26 / 33.04 / 13.85 / 7.82 (Test A) or the 5th-place rank does not reproduce, the headline result was inflated by selecting the configuration on the very data used for scoring.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that a fixed, off-the-shelf instruction-tuned 8B model can handle structured reasoning competitively if reasoning is decomposed into two prompted stages and the output is fenced as JSON. Question Parsing with k=2 demonstrations reaches a macro-F1 of 0.7526, and CoT Parsing & Verification with k=3 demonstrations yields statement, statement-and-evidence, and reasoning macro-F1 scores of 0.3304, 0.1385, and 0.0782 on the official Test A set (0.4726, 0.2017, and 0.1164 on Test B). The submission ranks 5th overall. The authors argue this shows that careful prompt design plus minimal hygiene techniques, including fenced ```json``` blocks that cut the unparsable rate from 16% to 2%, can rival far more elaborate pipelines, and that the remaining gap is not in condition extraction but in logical adjudication, where the model hallucinates evidence, paraphrases conditions, and mishandles negation.

Load-bearing premise

The report assumes that choosing the demonstration counts from the same test scores that are then reported as the result does not inflate those scores, and this assumption is load-bearing because Tables 1 and 2 are used both to select the configuration and to present its performance.

Editorial extensions

If this is right

  • A top-5 overall result can be reached without fine-tuning, retrieval, or ensembling, so strong structured-reasoning performance is not exclusive to heavy pipelines.
  • Question parsing and CoT parsing reward different demonstration counts, so stage-specific shot tuning is a practical lever when the total demonstration budget is small.
  • Fenced JSON output reduces the unparsable rate from about 16% to 2%, meaning output-format enforcement is a cheap and effective component of reasoning pipelines.
  • Because reasoning F1 stays low even after format fixes, the paper's own conclusion is that the bottleneck is the model's logic, not the interface: better verifiers or process-level training are the needed next step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A fairer estimate of the method's true strength would require choosing shot counts on a separate validation split; the paper's reported numbers are selected optima, since the same official test sets were used both to pick k and to report final scores.
  • The same two-stage split suggests a transferable recipe for other structured-reasoning benchmarks: spend prompt budget on decomposition and format, then invest model capacity in the verification step.
  • The generated statement–evidence–verdict triples are exactly the supervision Process Reward Models want; using them to train a small critic, rather than treating them as final output, is a natural extension the paper mentions but does not pursue.
  • The observation that adding a fourth demonstration sometimes hurts performance points to prompt-length or attention dilution as a possible constraint for 8B models; testing the same prompts on a longer-context model would separate exemplar-diversity effects from context-window effects.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This manuscript is a shared-task system description for LLMSR@XLLM25. The authors use the off-the-shelf Meta-Llama-3-8B-Instruct model with a few-shot, multi-turn prompt and a lightweight regular-expression post-processor to perform three subtasks: question parsing, chain-of-thought parsing into statement-evidence pairs, and verification of logical entailment. In Sections 4.1 and 4.2 they run k-shot ablations over the number of demonstrations for question parsing and for CoT parsing/verification, then fix the best settings and report final macro-F1 scores on Test A and Test B in Table 3. The paper claims that this simple system ranks 5th overall and achieves macro-F1 scores on par with substantially more complex and resource-consuming pipelines.

Significance. If the empirical claims are sound, this would be a useful data point for the LLMSR shared-task community and for process-reward-modeling research, because it shows that a minimal few-shot pipeline with an 8B model can reach a competitive leaderboard position. The paper is concise, the code is publicly available, and the authors are explicit about the model's weaknesses in logical verification, which is a genuine strength. However, the main quantitative claim is currently undermined by the fact that the model-selection ablations and the final reported scores are computed on the same official test sets, so the reported macro-F1 values are selected optima rather than independent estimates. The manuscript also does not provide a leaderboard comparison table, statistical significance tests, or an explicit development/validation split, which makes the 'on par with substantially more complex pipelines' claim unverifiable from the text alone.

major comments (5)
  1. [§4.1–4.2, Tables 1–3] The shot counts are selected by ablating on the same official test sets whose scores are later reported as the system's final performance. Concretely, the Test A ReasoningF1 of 7.82 in Table 3 is exactly the maximum of the four values 3.91, 2.50, 7.82, and 5.18 in Table 2, so the reported score is a selected optimum, not an independent measurement. This is load-bearing because the abstract's parity claim rests on the magnitude of these numbers. The authors should either hold out a validation split for the k-shot sweeps or report the final test scores for every k with confidence intervals, and they should rephrase the empirical claims accordingly.
  2. [§3.3 and §4] The manuscript is internally ambiguous about whether any development split exists. Section 3.3 reports that JSON guards reduced the unparsable rate 'on the dev set' from 16% to 2%, while Section 4 states that all experiments are conducted on the official test sets. If the same instances were used for prompt development and final evaluation, this is part of the selection problem; if a separate dev split exists, it should be described explicitly with its size and source. Without this information, the reader cannot determine whether the final configuration was chosen independently of the test data.
  3. [Abstract, §5.1, Table 3] The claims of 'ranks 5th overall' and 'achieving macro-F1 scores on par with substantially more complex and resource-consuming pipelines' are not substantiated in the manuscript. The official rank may be an external leaderboard fact, but no leaderboard table comparing the submitted system with the other systems is provided, and Section 5.1 explicitly calls the comparison 'informal.' The authors should include the official leaderboard results, state which phase (Test A or Test B) the ranking refers to, and report whether the differences are statistically meaningful.
  4. [Tables 1 and 2] No variance information or repeated runs are reported, despite the fact that Llama-3-8B-Instruct is a stochastic decoder. In Table 2, the increase in Stmt F1 from k=1 (0.3066) to k=3 (0.3304) and in ReasoningF1 from 0.0391 to 0.0782 could plausibly be within decoding noise, so the claim that 3-shot 'strikes the best trade-off' is not yet supported. The authors should run each configuration multiple times with different seeds and report means and standard deviations, or at least report the fixed random seed used.
  5. [§4, final configuration] The paper does not state whether the k-shot sweeps in Sections 4.1 and 4.2 were performed on Test A or Test B. Since Table 3 reports both phases, this choice matters for interpreting the sentence in Section 4.3 that the hybrid setup 'achieves the highest overall macro-F1 on the public leaderboard.' Please specify the phase used for model selection and justify why that phase is appropriate.
minor comments (6)
  1. [§2.1 and Figure 1] There is a typo in 'multi-steasoning' in Section 2.1, and 'Input Qusetion' in Figure 1 should be 'Input Question.'
  2. [§4] The sentence 'official LLMSR@XLLM25 test sets1' has a missing space before the footnote marker, and the footnote should either be a proper URL citation or be integrated into the text.
  3. [§3.3] The reported drop in unparsable rate from 16% to 2% on the dev set would be more informative if the authors defined what counts as 'unparsable' and reported the number of dev instances used.
  4. [§5.1] The phrase 'Informal leaderboard comparisons indicate...' should be replaced with a formal comparison table, because the current wording does not allow the reader to verify the central competitiveness claim.
  5. [References] The reference for the Meta-Llama-3-8B-Instruct model card lacks a URL or version identifier, and the capitalization of 'MANN and THOMPSON' is inconsistent with the other references.
  6. [General] For reproducibility, the exact prompt templates and the two QP and three CP demonstration exemplars should be included in an appendix, rather than only pointing to the GitHub repository, since repositories can change after publication.

Circularity Check

1 steps flagged · score 4.0 of 10

Test-set shot-count selection inflates reported macro-F1; rank may survive but score comparison is not unbiased.

  1. fitted input called prediction [Section 4, Sections 4.1-4.3, Tables 1-3]
    "We conduct all experiments on the official LLMSR@XLLM25 test sets. ... Macro-F1 peaks at 0.7526 with 2-shot. ... Given its clear advantage, we fix k=2 for all subsequent QP calls. ... Table 2 shows that 3-shot strikes the best trade-off ... The combination of 2-shot QP and 3-shot CP constitutes our submission. This hybrid setup achieves the highest overall macro-F1 on the public leaderboard."

    The same official test sets are used both to select the hyperparameters (k=2 for QP, k=3 for CP) and to report the final macro-F1 values in Table 3. The reported numbers are therefore selected maxima of the swept configurations, not unbiased estimates of a pre-specified system. For example, the final Reasoning F1 of 7.82 in Table 3 is exactly the 0.0782 value shown in Table 2 for the chosen 3-shot CP configuration, which was selected because it was the highest; the alternatives would have given 3.91, 2.50, or 5.18. The abstract's claim of being 'on par with substantially more complex pipelines' rests on these selected optima. The official 5th-place rank is externally defined and may survive, but the reported score comparison is partly fitted to the target data.

full rationale

The paper contains no self-citation chain, uniqueness theorem, or ansatz-smuggling; the core method is a straightforward prompting pipeline. The only circular element is the evaluation protocol in Section 4, where the official test sets are used both for k-shot selection and for the final scores. This makes the final macro-F1 values selected optima rather than independent measurements, so the headline 'on par' comparison is partially a product of fitting the two hyperparameters to the target data. Because the leaderboard rank is an external benchmark fact, the issue is moderate rather than total; the score is set to 4 rather than higher.

Assumptions & free parameters 3 free parameters · 2 assumptions · 0 invented entities

The paper is an empirical system description, so there are no mathematical axioms. The central claim rests primarily on the domain assumption that the official annotations and metrics are valid, and on the free choices of demonstration counts and exemplars, which were fitted on the test set.

free parameters (3)
  • QP shot count k = 2
    Selected by macro-F1 ablation on the test set (Table 1); this selection uses the target metric on the target data, making the final QP score a fitted value.
  • CP shot count k = 3
    Selected by macro-F1 ablation on the test set (Table 2); the final CP and reasoning scores therefore depend on a choice tuned on the reported test sets.
  • Hand-picked demonstration exemplars = 2 QP + 3 CP examples
    The paper states these were hand-picked to cover major logical patterns and were not validated on a held-out split, so the choice is subjective and could affect the outcome.
assumptions (2)
  • domain assumption The official LLMSR@XLLM25 annotations are correct and complete ground truth for condition extraction and entailment verification.
    All F1 scores in the paper rely on these labels without any audit of annotation quality or inter-annotator agreement.
  • domain assumption Macro-F1 is an appropriate evaluation metric for this task despite potential class imbalance in verification labels.
    The paper optimizes and reports macro-F1 without discussing whether the label distribution (e.g., true vs false entailments) affects the metric's informativeness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLMSR@XLLM25: An Empirical Study of LLM for Structural Reasoning." pith.science (2026). https://pith.science/paper/5MBPAFLM

@misc{pith2026250512328,
  author       = {Pith},
  title        = {Pith review of: LLMSR@XLLM25: An Empirical Study of LLM for Structural Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5MBPAFLM}},
  note         = {Machine review of arXiv:2505.12328}
}
read the original abstract

We present Team asdfo123's submission to the LLMSR@XLLM25 shared task, which evaluates large language models on producing fine-grained, controllable, and interpretable reasoning processes. Systems must extract all problem conditions, decompose a chain of thought into statement-evidence pairs, and verify the logical validity of each pair. Leveraging only the off-the-shelf Meta-Llama-3-8B-Instruct, we craft a concise few-shot, multi-turn prompt that first enumerates all conditions and then guides the model to label, cite, and adjudicate every reasoning step. A lightweight post-processor based on regular expressions normalises spans and enforces the official JSON schema. Without fine-tuning, external retrieval, or ensembling, our method ranks 5th overall, achieving macro F1 scores on par with substantially more complex and resource-consuming pipelines. We conclude by analysing the strengths and limitations of our approach and outlining directions for future research in structural reasoning with LLMs. Our code is available at https://github.com/asdfo123/LLMSR-asdfo123.

Figures

Figures reproduced from arXiv: 2505.12328 by the authors.

Figure 1
Figure 1. Illustration of the three-stage LLM-SR Task. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 7 canonical work pages

  1. [1]

    Shayan Ali Akbar, Md Mosharaf Hossain, Tess Wood, Si-Chi Chin, Erica M Salinas, Victor Alvarez, and Erwin Cornejo. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.837 H allu M easure: Fine-grained hallucination measurement using chain-of-thought reasoning . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages ...

    doi:10.18653/v1/2024.emnlp-main.837 2024

  2. [2]

    Guizhen Chen, Weiwen Xu, Hao Zhang, Hou Pong Chan, Chaoqun Liu, Lidong Bing, Deli Zhao, Anh Tuan Luu, and Yu Rong. 2025. Finereason: Evaluating and improving llms’ deliberate reasoning through reflective puzzle solving. arXiv preprint arXiv:2502.20238

    arXiv 2025

  3. [3]

    Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.585 Explaining answers with entailment trees . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7358--7370, Online and Punta Cana, Dominican Re...

    doi:10.18653/v1/2021.emnlp-main.585 2021

  4. [4]

    Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. 2023. https://arxiv.org/abs/2212.07919 Roscoe: A suite of metrics for scoring step-by-step reasoning . Preprint, arXiv:2212.07919

    arXiv 2023

  5. [5]

    Jiashu He, Mingyu Derek Ma, Jinxuan Fan, Dan Roth, Wei Wang, and Alejandro Ribeiro. 2025. Give: Structured reasoning of large language models with knowledge graph inspired veracity extrapolation. arXiv preprint arXiv:2410.08475

    arXiv 2025

  6. [6]

    Mingqian He, Yongliang Shen, Wenqi Zhang, Zeqi Tan, and Weiming Lu. 2024. https://arxiv.org/abs/2407.00390 Advancing process verification for large language models via tree-based preference learning . Preprint, arXiv:2407.00390

    arXiv 2024

  7. [7]

    Yulan Hu, Ge Chen, Jinman Zhao, Sheng Ouyang, and Yong Liu. 2025. https://arxiv.org/abs/2501.13622 Coarse-to-fine process reward modeling for mathematical reasoning . Preprint, arXiv:2501.13622

    work page Pith review arXiv 2025

  8. [8]

    Jinhao Jiang, Kun Zhou, Xin Zhao, Yaliang Li, and Ji-Rong Wen. 2023. Reasoninglm: Enabling structural subgraph reasoning in pre-trained language models for question answering over knowledge graph. In EMNLP, pages 3721--3735

    work page 2023

Show all 31 references
  1. [9]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA. Curran Associates Inc

    2022

  2. [10]

    Cheryl Li, Tianyuan Xu, and Yiwen Guo. 2025. Reasoning-as-logic-units: Scaling test-time reasoning in large language models through logic unit alignment. arXiv preprint arXiv:2502.07803

    2025 arXiv

  3. [11]

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. https://arxiv.org/abs/2305.20050 Let's verify step by step . Preprint, arXiv:2305.20050

    2023 arXiv

  4. [12]

    Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2021. Logiqa: a challenge dataset for machine reading comprehension with logical reasoning. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI'20

    2021

  5. [13]

    MANN and SANDRA A

    WILLIAM C. MANN and SANDRA A. THOMPSON. 1988. https://doi.org/doi:10.1515/text.1.1988.8.3.243 Rhetorical structure theory: Toward a functional theory of text organization . Text - Interdisciplinary Journal for the Study of Discourse, 8(3):243--281

    1988 doi

  6. [14]

    Daniel C. Marcu. 1998. The rhetorical parsing, summarization, and generation of natural language texts. Ph.D. thesis, CAN. AAINQ35238

    1998

  7. [15]

    Meta AI . 2024. https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct Meta-llama-3-8b-instruct model card

    2024

  8. [16]

    Mihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura, Man Luo, Santosh Mashetty, Arindam Mitra, and Chitta Baral. 2024. https://doi.org/10.18653/v1/2024.acl-long.739 L ogic B ench: Towards systematic evaluation of logical reasoning ability of large language models . I...

    2024 doi

  9. [17]

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of EMNLP, pages 5687--5711

    2023

  10. [18]

    Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. 2022. Progprompt: Generating situated robot task plans using large language models. arXiv preprint arXiv:2209.11302

    2022 arXiv

  11. [19]

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. https://arxiv.org/abs/2211.14275 Solving math word problems with process- and outcome-based feedback . Preprint, arXiv:2211.14275

    2022 arXiv

  12. [20]

    Peiyi Wang, Lei Li, Zhihong Shao, R. X. Xu, Damai Dai, Yifei Li, Deli Chen, Y. Wu, and Zhifang Sui. 2024. https://arxiv.org/abs/2312.08935 Math-shepherd: Verify and reinforce llms step-by-step without human annotations . Preprint, arXiv:2312.08935

    2024 arXiv

  13. [21]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://arxiv.org/abs/2203.11171 Self-consistency improves chain of thought reasoning in language models . Preprint, arXiv:2203.11171

    2023 arXiv

  14. [22]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. https://arxiv.org/abs/2201.11903 Chain-of-thought prompting elicits reasoning in large language models . Preprint, arXiv:2201.11903

    2023 arXiv

  15. [23]

    Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu. 2025. Evaluating mathematical reasoning beyond accuracy. arXiv preprint arXiv:2404.05692

    2025 arXiv

  16. [24]

    Zhun Yang, Adam Ishay, and Joohyung Lee. 2023. Coupling large language models with logic programming for robust and general reasoning from text. arXiv preprint arXiv:2307.07696

    2023 arXiv

  17. [25]

    Griffiths, Yuan Cao, and Karthik Narasimhan

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. https://arxiv.org/abs/2305.10601 Tree of thoughts: Deliberate problem solving with large language models . Preprint, arXiv:2305.10601

    2023 arXiv

  18. [26]

    Amir Zeldes, Tatsuya Aoyama, Yang Janet Liu, Siyao Peng, Debopam Das, and Luke Gessler. 2024. https://arxiv.org/abs/2403.13560 erst: A signaled graph theory of discourse relations and organization . Preprint, arXiv:2403.13560

    2024 arXiv

  19. [27]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. Star: Bootstrapping reasoning with self-consistency. In Advances in Neural Information Processing Systems, volume 35, pages 15476--15488

    2022

  20. [28]

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. https://arxiv.org/abs/2210.03493 Automatic chain of thought prompting in large language models . Preprint, arXiv:2210.03493

    2022 arXiv

  21. [29]

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. 2023. https://arxiv.org/abs/2205.10625 Least-to-most prompting enables complex reasoning in large language models . Preprint, arXiv...

    2023 arXiv

  22. [30]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  23. [31]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.