Pith. sign in

REVIEW 5 major objections 4 minor 34 references

A two-tool agent loop that separates fact extraction from graph-based reasoning lifts small language models' kinship accuracy 1.5-2x over story-only baselines—but only when the extracted facts are clean.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 06:18 UTC pith:46SGLAQS

load-bearing objection A useful, modest empirical study of RGCN-grounded hints for SLMs on CLUTRR; the qualitative findings hold, but the headline 1.5–2x gain only holds for the smallest model and there are no error bars. the 5 major comments →

arxiv 2607.14149 v1 pith:46SGLAQS submitted 2026-07-14 cs.AI cs.LG

Enhancing Small Language Models Reasoning through Knowledge Graph Grounding

Dimitrios Kelesis , Konstantinos Bougiatiotis , Georgios Paliouras This is my paper
classification cs.AI cs.LG
keywords small language modelsneuro-symbolic reasoningknowledge graph groundingrelational reasoningRGCNCLUTRRtool-augmented agentsmulti-hop inference
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Small language models fail at multi-hop relational reasoning because they cannot keep a consistent symbolic state, not because they cannot understand the text. This paper tests a two-tool agentic loop on the CLUTRR kinship benchmark: the SLM extracts kinship triplets from a story, an expert graph neural network (an RGCN) reasons over the resulting knowledge graph and returns a predicted relation as a 'hint,' and the SLM answers with the hint in its prompt. In realistic settings where the SLM extracts its own facts, RGCN hints give a 1.5-2x gain over story-only baselines (e.g., Gemma 1B from about 7.5% to 12.6%). The gains are capped by an extraction bottleneck: the RGCN scores 60.69% on ground-truth facts but only 24.88% on self-extracted facts, and noisy facts can actively mislead some models—a 'distraction effect' the paper documents. The paper's central point is that decoupling text understanding from relational logic helps, but reliable fact extraction is the weakest link.

Core claim

The paper's central claim is that small language models' poor multi-hop kinship reasoning is largely a symbolic-state problem, not a linguistic-comprehension problem. On the CLUTRR benchmark, giving the model a perfect oracle hint raises accuracy sharply (Llama 3.2 3B from 16.13% to 62.79%), while giving oracle facts alone does almost nothing. In the realistic loop, where the SLM extracts its own facts, the RGCN's accuracy collapses from 60.69% on ground-truth graphs to 24.88% on self-extracted graphs, and the best realistic configurations reach only about 20%. The authors interpret this as sequential deductive fragility: an early missing or hallucinated relation makes the graph logically un

What carries the argument

The load-bearing machinery is a two-tool agentic loop. extract_facts turns story text into kinship triplets in a target-centric reversed-edge format (e.g., an edge from father to daughter is labeled 'daughter'), and get_hint feeds the resulting per-story knowledge graph into an RGCN—a graph neural network that propagates messages along typed relation edges—returning a predicted relation plus a confidence score. The RGCN is trained only on 2-4-hop chains but is asked to reason on graphs of up to 10 hops; inverse triplets are added post-extraction to guarantee bidirectional message flow. The hint is then injected into the SLM's prompt, giving the model a structured symbolic signal to combine w

Load-bearing premise

The framework assumes the small model will accept the RGCN's hint and use it despite noisy facts in the prompt; even with a perfect hint, Gemma 1B reaches only 34% accuracy, so hint-following itself is not guaranteed.

What would settle it

A direct test: on two-hop CLUTRR stories, give the SLM the exact correct relation as the hint along with the story, and measure accuracy. If a model fails to answer correctly in a substantial share of trials, hint-following rather than extraction becomes the binding constraint. Conversely, corrupt exactly one early triplet in an otherwise clean graph and feed it to the RGCN: if multi-hop accuracy does not collapse, the sequential-fragility explanation would require revision.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Small models can act as controllers that offload relational logic to a specialized graph reasoner, achieving 1.5-2x gains without any fine-tuning or parameter changes.
  • An RGCN trained on short reasoning chains can generalize to much longer chains (up to 10 hops) when its input graph is accurate, so reasoning depth need not be learned explicitly for each length.
  • Extraction quality, not the SLM's reasoning capacity, is the binding constraint: even with expert hints, realistic configurations plateau near 20% accuracy.
  • Adding noisy self-extracted facts to the prompt can hurt more than help, so agentic systems should be selective about which symbolic facts they present to the model.
  • Architectures differ in how they handle this noise—Llama 3.2 3B resists the distraction effect while Gemma variants do not—suggesting attention mechanisms play a role in filtering symbolic context.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If extraction quality were repaired—e.g., with a verification pass over the triplets—the same hint pipeline could plausibly reach the oracle-hint regime, because the RGCN already shows strong long-chain generalization when its input is clean.
  • The distraction effect makes prompt composition a separate design axis: labeling extracted facts as uncertain, or rank-ordering them by confidence, could reduce the anchoring on hallucinated triplets.
  • The RGCN's short-to-long generalization implies relational composition rules can be learned independently of chain length; a cheaper follow-up would train the expert on 2-hop chains only and test whether 4- and 6-hop accuracy survives.
  • Because the framework decouples extractor from reasoner, one can swap or distill the extractor independently and freeze the RGCN, then map accuracy against extraction quality—directly quantifying the bottleneck the paper identifies.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper presents an agentic neuro-symbolic framework for improving small language models (Gemma 3 1B/4B, Llama 3.2 3B) on the CLUTRR kinship reasoning benchmark. The SLM acts as a controller that calls extract_facts to populate a knowledge graph and get_hint, an RGCN trained on CLUTRR training chains, to obtain a predicted relation. The authors evaluate oracle (ground-truth triplets/hints) and realistic (SLM-extracted facts) configurations. They report that RGCN-derived hints yield 1.5-2x gains over story-only baselines, identify an extraction bottleneck, and characterize a "distraction effect" where noisy extracted facts degrade performance despite expert hints.

Significance. If the quantitative claims held, the framework would offer a lightweight, sustainable way to improve SLM multi-hop reasoning by coupling explicit symbolic extraction with graph-based inference. The oracle-vs-realistic contrast is a useful experimental design, and the distraction effect is an interesting observation. However, the current evidence is weaker than the headline claim: the gain is below 1.5x for two of the three models, there are no error bars or repeated-seed numbers, and the "oracle hint" condition supplies the ground-truth answer, so it does not measure reasoning ability. The extraction-prompt details are also incomplete. The paper's main value is as a preliminary characterization, not an established performance claim.

major comments (5)
  1. [Abstract; Section 1; Table 1] The abstract and the contributions list ('our method yields a 1.5 - 2x performance gain') are not supported by Table 1. Using the realistic full pipeline (Story + SLM Facts + GNN Hint) relative to Story, the gains are: Gemma 1B 12.40/7.54=1.64x; Llama 3B 20.52/16.13=1.27x; Gemma 4B 14.98/13.45=1.11x. Even the more favorable Story+GNN Hint row gives Llama 1.26x and Gemma 4B 1.47x. Only Gemma 1B crosses 1.5x. Section 4.3 qualifies 'for smaller models like Gemma 1B', but the abstract and contributions do not. This is a concrete numerical inconsistency in the central claim; please correct the headline to per-model magnitudes.
  2. [Section 4.2; Table 1] The Oracle Hint is defined as the ground-truth relation ('i.e., the actual relation to be predicted'). The improvement from Story to Story+Oracle Hint therefore measures how often the SLM copies a provided label, not how well it combines heuristic guidance with the story. It cannot support the conclusion that 'the primary obstacle for SLMs is not a lack of linguistic comprehension, but rather an inability to maintain and query a consistent symbolic state,' nor the claim of 'latent capacity for complex relational reasoning when provided with high-quality symbolic guidance' (Section 5). A control with a wrong but fluent hint is needed. Note also that Gemma 1B reaches only 34.16% even with the correct answer, so the hint-following mechanism itself is unreliable; this should be acknowledged.
  3. [Section 3.1; Appendix A] The methodology promises that the exact extraction prompt is in Appendix A, but only the Gemma 4B extract_facts prompt is shown. The prompts for Gemma 1B and Llama 3B are not provided, nor the exact 'Logical Facts' formatting used in the prompt table. Since the realistic scenario and the cross-model comparison depend on the extraction prompt, the main experiment is not reproducible as reported.
  4. [Section 4.1; Table 1] All accuracy values are single numbers. There is no indication of the number of seeds, repeated runs, or variance. In zero-shot LLM evaluation, accuracy can vary by several points across runs, so Gemma 4B's 1.11x gain and the distraction-effect gap (19.75 vs 14.98) may not be significant. Please report mean and standard deviation over at least 3-5 runs, or otherwise justify that the differences are beyond run-to-run noise. This is necessary for the quantitative headline claim.
  5. [Section 3.2; Section 4.3; Table 1] The average recall of ground-truth facts is reported as 'approximately 71%' without specifying the model, the subset of stories, or the number of samples used. Similarly, the RGCN expert's collapse on SLM facts is reported only as a single 24.88% value attributed to 'SLMs (i.e., Gemma 4B)', while the realistic pipeline is evaluated with three different extractors. Extraction quality is the central explanatory variable in the paper, so per-model recall (and a breakdown of missing vs hallucinated edges, ideally by hop count) should be reported.
minor comments (4)
  1. [Section 4.2] Typo: 'the efficiency of the SML' should be 'SLM'.
  2. [Section 3.1] Typo: 'adhearing' should be 'adhering'. Also, the target-centric/ascendant directionality description is confusing; please provide a concrete extraction example.
  3. [Table 1] Formatting: the row 'Story + Oracle Hint34.1662.79 59.16' lacks spaces; fix layout.
  4. [Section 2.2] Reference [24] (Catastrophic Failure of LLM Unlearning via Quantization) does not appear to support the 'quantization trap' statement about small models' multi-hop reasoning; verify the citation or replace it with a relevant work.

Circularity Check

0 steps flagged

No circular derivation: RGCN is supervised on CLUTRR train split, SLMs are zero-shot, and the Oracle-Hint rows are explicitly diagnostic rather than predictions.

full rationale

The paper's pipeline is an empirical evaluation rather than a derivation. The RGCN expert is trained on the CLUTRR training partition (k∈{2,3,4}) and evaluated on held-out test stories (Sections 3.3 and 4.1), so its hints are not fitted to the test outputs they are judged against. The SLM configurations are zero-shot and are compared against story-only baselines on the same test set. The 'Oracle Hint' rows in Table 1 inject the true answer and are labeled as an oracle/diagnostic condition, not presented as a prediction; they are used to upper-bound the SLM's capacity to consume correct symbolic guidance. No equation in the paper reduces a reported quantity to a fitted parameter, and no load-bearing claim rests on a self-citation: the cited RGCN, CLUTRR, Gemma, and Llama works are external and provide tools/benchmarks rather than the paper's conclusions. The 'distraction effect' and 'extraction bottleneck' are empirical observations from Table 1 and Figures 2-3, not analytic consequences. The abstract's '1.5-2x gain' is not fully supported by Table 1 for two of three models (computed gains are about 1.26x and 1.47x/1.11x), but that is a quantitative-claim/correctness issue, not circularity, and does not belong in the circularity score per the review rules.

Axiom & Free-Parameter Ledger

1 free parameters · 3 axioms · 0 invented entities

The central claim rests on two unverified empirical premises: RGCN generalization beyond its training hops, and the SLM's ability to follow injected hints. The only free numeric parameter is the RGCN configuration. No new entities are invented.

free parameters (1)
  • RGCN hyperparameters = layers=4, epochs=20, lr=1e-3
    Chosen by hand in Section 4.1 without ablation; the reported hint quality depends on this configuration.
axioms (3)
  • domain assumption RGCN trained on 2-4 hop chains can generalize to 10-hop queries
    Stated in Sections 3.3 and 4.1; if false, hints at high hop counts would be no better than chance and the central gain disappears.
  • domain assumption SLM uses the injected hint rather than ignoring or being confused by it
    The mechanism depends on the SLM combining Story and Hint; Table 1's Story+Oracle Hint row (Gemma 1B 34.16%, well below 100%) shows this is not guaranteed.
  • ad hoc to paper Target-centric reversed edge convention matches CLUTRR's logical schema
    Section 3.1 introduces reversed directionality (object, rel, subject) to fit the dataset schema; RGCN training and hint generation depend on this convention being correct.

pith-pipeline@v1.3.0-alltime-deepseek · 9510 in / 10731 out tokens · 111075 ms · 2026-08-02T06:18:30.193831+00:00 · methodology

0 comments
read the original abstract

Although large language models (LLMs) have set benchmarks for zero-shot reasoning, their deployment remains cost-prohibitive and environmentally taxing. Small Language Models (SLMs) offer a sustainable alternative, but prone to errors, on tasks requiring complex, multi-hop logical grounding. We investigate a neuro-symbolic agentic framework to enhance the reasoning capabilities of SLMs, specifically Gemma 3 (1B, 4B) and Llama 3.2 (3B), using the CLUTRR kinship benchmark. Our approach transforms the SLM into a minimalist agent utilizing two specialized tool calls: extract_facts for symbolic triplet extraction and get_hint for expert reasoning via a Relational Graph Convolutional Network (RGCN). We evaluate these models across two configurations, both in an Oracle scenario with ground-truth triplets and a Realistic scenario relying on self-extracted knowledge. Our results reveal that while RGCN-derived hints provide a 1.5 - 2x performance gain over story-only baselines, the system is constrained by the extraction bottleneck and sequential deductive fragility, where early extraction errors compound over multi-hop chains. Furthermore, we identify a "distraction effect" in specific architectures where noisy, self-generated facts degrade performance despite the presence of expert hints. This work characterizes the challenges of symbolic grounding in low-resource agentic systems and provides a roadmap for iterative verification in neuro-symbolic agentic pipelines.

Figures

Figures reproduced from arXiv: 2607.14149 by Dimitrios Kelesis, Georgios Paliouras, Konstantinos Bougiatiotis.

Figure 1
Figure 1. Figure 1: The proposed agentic loop. The SLM invokes extract_facts to populate a per-story Knowledge Graph, which is then processed by the get_hint tool to provide assistance for the final inference. 3.1. Agentic Tool Calling Mechanism We define two primary tools that the SLM can invoke to enhance its zero-shot reasoning: • extract_facts(story): The SLM parses the story to extract kinship triplets (𝑢, 𝑟𝑒𝑙, 𝑣). In ou… view at source ↗
Figure 2
Figure 2. Figure 2: Model accuracy across queries with different hop-counts using Oracle-provided facts/hint. Performance tends to peak at hop counts up to 4, except mainly for the Llama-based models. attention mechanism may be more efficient at filtering out symbolic noise in favor of high-confidence expert signals. In [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Model accuracy across queries with different hop-counts using SLM-generated facts/hint. Note the overall reduction in accuracy compared to Oracle-based configurations. reliable, autonomous systems capable of causal grounding in open-domain environments. Acknowledgments GCP resources were provided by the National Infrastructures for Research and Technology GRNET and funded by the EU Recovery and Resiliency … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 1 canonical work pages

  1. [1]

    Crispino, K

    N. Crispino, K. Montgomery, F. Zeng, D. Song, C. Wang, Agent instructs large language models to be general zero-shot reasoners, in: R. Salakhutdinov, Z. Kolter, K. A. Heller, A. Weller, N. Oliver, J. Scarlett, F. Berkenkamp (Eds.), Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, volume 235 ofProceedi...

    2024

  2. [2]

    Kojima, S

    T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, Y. Iwasawa, Large language models are zero- shot reasoners, in: S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh (Eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neu- ral Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, Novem- ber 28 - December ...

    2022

  3. [3]

    V. Sanh, A. Webson, C. Raffel, S. H. Bach, L. Sutawika, Z. Alyafeai, A. Chaffin, A. Stiegler, A. Raja, M. Dey, M. S. Bari, C. Xu, U. Thakker, S. S. Sharma, E. Szczechla, T. Kim, G. Chhablani, N. V. Nayak, D. Datta, J. Chang, M. T. Jiang, H. Wang, M. Manica, S. Shen, Z. X. Yong, H. Pandey, R. Bawden, T. Wang, T. Neeraj, J. Rozen, A. Sharma, A. Santilli, T....

    2022

  4. [5]

    Strubell, A

    E. Strubell, A. Ganesh, A. McCallum, Energy and policy considerations for modern deep learning research, in: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2...

    doi:10.1609/aaai 2020

  5. [6]

    Hoffmann, S

    J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, J. W. Rae, O. Vinyals, L. Sifre, Training compute-optimal large language models, CoRR abs/2203.15556 (2022). URL: ...

    Pith/arXiv arXiv doi:10.48550/arxiv.2203.15556 2022

  6. [7]

    C. V. Nguyen, X. Shen, R. Aponte, Y. Xia, S. Basu, Z. Hu, J. Chen, M. Parmar, S. Kunapuli, J. Barrow, J. Wu, A. Singh, Y. Wang, J. Gu, N. K. Ahmed, N. Lipka, R. Zhang, X. Chen, T. Yu, S. Kim, H. Deilamsalehy, N. Park, M. Rimer, Z. Zhang, H. Yang, P. Mathur, G. Wu, F. Dernoncourt, R. A. Rossi, T. H. Nguyen, A survey on small language models, in: G. Angelov...

    2025

  7. [8]

    Z. Lu, X. Li, D. Cai, R. Yi, F. Liu, X. Zhang, N. D. Lane, M. Xu, Small language models: Survey, measurements, and insights, CoRR abs/2409.15790 (2024). URL: https://doi.org/10.48550/arXiv. 2409.15790. doi:10.48550/ARXIV.2409.15790.arXiv:2409.15790

    Pith/arXiv arXiv doi:10.48550/arxiv.2409.15790 2024

  8. [9]

    Belcak, G

    P. Belcak, G. Heinrich, S. Diao, Y. Fu, X. Dong, S. Muralidharan, Y. C. Lin, P. Molchanov, Small language models are the future of agentic ai, 2025. URL: https://arxiv.org/abs/2506.02153. arXiv:2506.02153

    Pith/arXiv arXiv 2025

  9. [11]

    J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, E. H. Chi, T. Hashimoto, O. Vinyals, P. Liang, J. Dean, W. Fedus, Emergent abilities of large language models, Trans. Mach. Learn. Res. 2022 (2022). URL: https://openreview.net/forum? id=yzkSU5zdwD

    2022

  10. [12]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, D. Zhou, Chain- of-thought prompting elicits reasoning in large language models, in: S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh (Eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, N...

    2022

  11. [13]

    Sinha, S

    K. Sinha, S. Sodhani, J. Dong, J. Pineau, W. L. Hamilton, Clutrr: A diagnostic benchmark for inductive reasoning from text, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 4506–4515

    2019

  12. [14]

    Kaplan, S

    J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, D. Amodei, Scaling laws for neural language models, CoRR abs/2001.08361 (2020). URL: https://arxiv.org/abs/2001.08361.arXiv:2001.08361

    Pith/arXiv arXiv 2001

  13. [15]

    Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, P. Fung, Survey of hallucination in natural language generation, ACM Computing Surveys 55 (2023) 1–38. URL: http://dx.doi.org/10.1145/3571730. doi:10.1145/3571730

    doi:10.1145/3571730 2023

  14. [16]

    E. M. Bender, T. Gebru, A. McMillan-Major, S. Shmitchell, On the dangers of stochastic parrots: Can language models be too big?, in: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, Association for Computing Machinery, New York, NY, USA, 2021, p. 610–623. URL: https://doi.org/10.1145/3442188.3445922. doi:10....

    arXiv 2021

  15. [17]

    M. S. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, M. Welling, Modeling relational data with graph convolutional networks, in: A. Gangemi, R. Navigli, M. Vidal, P. Hitzler, R. Troncy, L. Hollink, A. Tordai, M. Alam (Eds.), The Semantic Web - 15th International Conference, ESWC 2018, Heraklion, Crete, Greece, June 3-7, 2018, Proceedings,...

    doi:10.1007/978-3-319-93417-4_38 2018

  16. [18]

    G. T. et al., Gemma 3 technical report, 2025. URL: https://arxiv.org/abs/2503.19786. arXiv:2503.19786

    Pith/arXiv arXiv 2025

  17. [19]

    A. G. et al., The llama 3 herd of models, 2024. URL: https://arxiv.org/abs/2407.21783. arXiv:2407.21783

    Pith/arXiv arXiv 2024

  18. [20]

    T. N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, in: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, OpenReview.net, 2017. URL: https://openreview.net/forum? id=SJU4ayYgl

    2017

  19. [21]

    Lepagnol, T

    P. Lepagnol, T. Gerald, S. Ghannay, C. Servan, S. Rosset, Small language models are good too: An empirical study of zero-shot classification, in: N. Calzolari, M.-Y. Kan, V. Hoste, A. Lenci, S. Sakti, N. Xue (Eds.), Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), EL...

    2024

  20. [22]

    Steele, M

    B. Steele, M. Katz, Scaling trends for multi-hop contextual reasoning in mid-scale language models,

  21. [23]

    N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, P. Liang, Lost in the middle: How language models use long contexts, Trans. Assoc. Comput. Linguistics 12 (2024) 157–173. URL: https://doi.org/10.1162/tacl_a_00638. doi:10.1162/TACL\_A\_00638

    doi:10.1162/tacl_a_00638 2024

  22. [24]

    Zhang, F

    Z. Zhang, F. Wang, X. Li, Z. Wu, X. Tang, H. Liu, Q. He, W. Yin, S. Wang, Catastrophic failure of LLM unlearning via quantization, in: The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, OpenReview.net, 2025. URL: https: //openreview.net/forum?id=lHSeDYamnz

    2025

  23. [25]

    Sadeddine, F

    Z. Sadeddine, F. M. Suchanek, Verifying the steps of deductive reasoning chains, in: W. Che, J. Nabende, E. Shutova, M. T. Pilehvar (Eds.), Findings of the Association for Computational Linguis- tics: ACL 2025, Association for Computational Linguistics, Vienna, Austria, 2025, pp. 456–475. URL: https://aclanthology.org/2025.findings-acl.25/. doi:10.18653/v...

    work page doi:10.18653/v1/2025.findings-acl.25 2025

  24. [26]

    Z. Gou, Z. Shao, Y. Gong, Y. Shen, Y. Yang, M. Huang, N. Duan, W. Chen, Tora: A tool-integrated reasoning agent for mathematical problem solving, CoRR abs/2309.17452 (2023). URL: https: //doi.org/10.48550/arXiv.2309.17452. doi:10.48550/ARXIV.2309.17452. arXiv:2309.17452

    Pith/arXiv arXiv doi:10.48550/arxiv.2309.17452 2023

  25. [27]

    D. Das, D. Banerjee, S. Aditya, A. Kulkarni, MATHSENSEI: A tool-augmented large language model for mathematical reasoning, in: K. Duh, H. Gómez-Adorno, S. Bethard (Eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), NAACL 2024, Mexico...

    doi:10.18653/v1/2024.naacl-long 2024

  26. [28]

    Huguet Cabot, R

    P.-L. Huguet Cabot, R. Navigli, REBEL: Relation extraction by end-to-end language generation, in: M.-F. Moens, X. Huang, L. Specia, S. W.-t. Yih (Eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, Association for Computational Linguistics, Punta Cana, Dominican Republic, 2021, pp. 2370–2381. URL: https://aclanthology.org/2021.fi...

    2021

  27. [30]

    Identify the path: B -> ... -> A

  28. [31]

    Calculate the composition of relationships along that path

  29. [32]

    answer":

    Output the result as: "answer": "term"

  30. [33]

    "" You are an expert Information Extraction system. Task: Extract all explicit kinship relationships from the short story below. Story:

    Logical facts may contain false information. Story Preamble +Story: <story_text>+ Constraints + Instructions Story + Facts Preamble + Story: <story_text> + Logical Facts: <logical_facts> + Constraints + Instructions Story + Hint Preamble + Story: <story_text> + Constraints + Hint: <expert_hint> + Instructions Facts + Hint Preamble + Logical Facts: <logica...

  31. [34]

    Extract them WITHOUT brackets

    Identify people names enclosed in brackets (e.g., [Alice]). Extract them WITHOUT brackets

  32. [35]

    Identify the relationship between them

  33. [36]

    E.g .: - Use grandson/granddaughter instead of grandfather/grandmother

    IMPORTANT: Extract relations in an ASCENDANT fashion (Child -> Parent). E.g .: - Use grandson/granddaughter instead of grandfather/grandmother. - Use son/daughter instead of father/mother. etc. """

  34. [2026]

    URL: https://arxiv.org/abs/2601.04254.arXiv:2601.04254

    arXiv