Pith. sign in

REVIEW 3 major objections 5 minor 35 references

LLM-generated SQL fails on real Oracle databases mostly because the model never sees the real column names; a live-catalog grounding layer recovers most of that gap without retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 12:22 UTC pith:SHQWCJDY

load-bearing objection The paper's real finding — schema grounding moves Oracle NL2SQL from 2.2% to 62.6% EGT — is plausible and well-measured, but the headline 'no manual schema curation' is contradicted by the paper's own methods. the 3 major comments →

arxiv 2607.22572 v1 pith:SHQWCJDY submitted 2026-06-03 cs.AI

Schema-Aware Localisation (SAL): Live Schema Grounding and Hallucination Validation for Oracle NL2SQL

Sanjay Mishra , Divya Chukkapalli , Ganesh R. Naik This is my paper
classification cs.AI
keywords NL2SQLschema groundinghallucination validationexecution-grounded truthOracle SQLschema injectionTPC-Hretrieval-augmented generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that most failures of LLM-generated SQL on real Oracle databases are schema hallucinations—references to tables and columns that do not exist—and that a lightweight runtime layer can fix most of them without retraining. The system reads the live column catalog, picks a question-specific subset of tables, injects the exact column names into the prompt, and then validates every alias.column reference in the generated SQL, auto-correcting prefix omissions and retrying with itemised corrections. On 500 questions over a standard supply-chain schema on a live Oracle database, this lifts execution-grounded truth (executes and matches the reference result) from 2.2% with no schema context to 62.6%, equal to a hand-written static schema hint, while cutting execution failures from 97.6% to 2.6%. The transferable value is the claim that grounding plus validation, not model retraining, is the main unlock for enterprise text-to-SQL.

Core claim

The central claim is that schema grounding can be automated to match a hand-curated hint. SAL's table detector scores each table by keyword hits, expands along join edges, and falls back to the full schema for multi-table questions, so the model always sees real column names. The Hallucination Index then parses the generated SQL, maps aliases to tables, and checks each alias.column against the live catalog; predictable prefix errors like O.ORDERDATE for O.O_ORDERDATE are rewritten without an LLM call, and anything else gets a structured retry with itemised corrections. The reported result is 62.6% execution-grounded truth (96% simple, 95% medium, 40.7% complex), with 37 of the 313 correct an

What carries the argument

Three components carry the argument: (1) a live schema cache populated from Oracle's USER_TAB_COLUMNS catalog, so the prompt contains exact, current column names; (2) complexity-gated table selection that scores tables with keyword vocabularies, expands matches along a JOIN-chain adjacency map, and falls back to the full schema when multi-table complexity signals fire; and (3) the Hallucination Index validator, which statically checks alias.column references against the cache, auto-corrects prefix omissions, and drives at most two feedback retries. The combination converts most of the schema-hallucination failure class into a deterministic validation problem.

Load-bearing premise

The load-bearing premise is that the keyword vocabularies, JOIN-chain adjacency map, complexity phrase set, and gate thresholds—all constructed with knowledge of the benchmark schema—transfer to new Oracle schemas without manual re-curation; if they do not, the 'no manual schema curation' claim collapses and the 62.6% figure is a fitted benchmark score.

What would settle it

Run the same pipeline on a fresh Oracle schema outside the benchmark domain, building keyword maps and the JOIN adjacency without ever reading the evaluation questions; if execution-grounded truth falls well below the hand-written static hint, the generalisability claim fails. A cheaper internal check is to rerun SAL v2 without the Hallucination Index—the paper's own decomposition predicts EGT drops to 55.2%.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Schema grounding, not model reasoning, is the binding constraint in this Oracle setting: removing it drops execution-grounded truth from 62.6% to 2.2%.
  • A dynamic, question-adaptive hint can match a full static schema hint at lower mean token cost (roughly 208 fewer tokens per query) while staying current with schema changes.
  • Schema hallucinations are not random: the paper's sampled no-hint failures are mostly phantom identifiers, and a token-precise validator can catch most of them before execution.
  • The residual ceiling is a reasoning boundary: 174 of 187 failures execute but return semantically wrong results, so further gains need better query reasoning, not more schema context.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The honest generalisation test is on a schema the designers never saw: if the keyword vocabularies, JOIN adjacency map, and complexity phrase set need re-authoring for each enterprise schema, the 'no manual schema curation' claim would not transfer, even if the validator itself does.
  • A natural extension is to generate candidate alias-prefix corrections algorithmically from the catalog itself—e.g., detect common table-initial column prefixes—instead of relying on a hand-tuned prefix pattern.
  • Swapping in a stronger base model should shift the complex-tier 40.7% ceiling upward; comparing that shift would separate grounding effects from reasoning effects.
  • Using Hidx corrections as few-shot demonstrations or as a training signal for a smaller local model could reduce the remaining LLM retries and make the pipeline cheaper.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents Schema-Aware Localisation (SAL), a zero-retraining middleware layer for Oracle NL2SQL. SAL loads live schema metadata from USER_TAB_COLUMNS, selects a question-relevant subset of tables via scored keyword matching, JOIN-chain expansion, and a complexity gate, injects that schema context into the LLM prompt, and then validates the generated SQL with a Hallucination Index (Hidx) that checks alias.column references against the live schema, applies deterministic prefix corrections, and triggers structured LLM retries. Evaluation on 500 TPC-H questions executed against a live Oracle Autonomous Database 23c instance with GPT-4o-mini reports 2.2% EGT with no hint, 62.0% with a hand-written static hint, 58.4% for SAL v1, and 62.6% for SAL v2, with execution failures reduced from 97.6% to 2.6%. The paper claims that SAL achieves this 'with no manual schema curation.'

Significance. If the transferability concern is resolved, the result is significant for enterprise NL2SQL: it demonstrates that live catalog grounding plus deterministic identifier validation can recover most of the benefit of a hand-written static schema hint, at lower prompt cost, without retraining, and with a large reduction in Oracle execution errors. The live-database evaluation, the four-condition ablation, the explicit threat-to-validity analysis, and the plan to release implementation and harness as open source are genuine strengths. However, the central 'no manual schema curation' claim is not established by the evidence in this version. The paper's own design shows that the keyword vocabularies K_t, the JOIN-chain adjacency map J, the complexity phrase set P, and the gate thresholds were hand-built with TPC-H knowledge and tuned on the same 500-question benchmark used for evaluation. The 62.6% EGT figure is therefore best read as a fitted benchmark score until replication on a held-out schema with independently constructed detection components is provided.

major comments (3)
  1. [Abstract; Sections 4.2.1–4.2.3, 6.2; T-I4; Section 12(1)] The abstract's claim that SAL works 'with no manual schema curation' is contradicted by the paper's own system description. Section 4.2.1 defines per-table keyword vocabularies K_t that are 'curated' domain terms; Section 4.2.2 builds the JOIN adjacency map J 'from the known TPC-H foreign-key graph'; Section 4.2.3 says the complexity phrase set P and thresholds (|T_direct|<3, cplx(q)>=2) were 'chosen to trade off prompt length against missed JOIN context on our 500-question benchmark.' Section 6.2 states the 500 questions were authored by the same team that designed these maps. Section 10.1 (T-I4) labels this dataset leakage the 'primary internal validity threat,' and Section 12(1) concedes the maps 'were designed with knowledge of TPC-H table names.' Consequently, the 62.6% EGT is a fitted score for TPC-H, not evidence for the headline general claim. The revision should either remove th
  2. [Section 4.3.1; Definition 5] The abstract and Definition 5 state that Hidx 'validates every alias.column reference,' but the implementation does not. The alias-map parser is the regex (?:FROM|JOIN)\s+([A-Z_]+)\s+([A-Z_]+), and the limitation paragraph in Section 4.3.1 says subqueries, CTEs, quoted identifiers, and aliases introduced by subqueries/CTEs are 'out of scope and are skipped in subsequent validation.' Thus the numerator and denominator in Eq. (2) cover only references bound by simple FROM/JOIN clauses. Given that the complex tier includes correlated subqueries and CTEs, Hidx can silently skip exactly the references where hallucinated columns often occur. The claim 'validates every alias.column reference' should be scoped, or the parser/validator should be extended to handle nested query aliases.
  3. [Sections 5.4–5.5] The paper presents Online Execution-Grounded Verification (OEGV) as a general post-execution verification component, but the deployment-mode description is explicitly TPC-H-specific. Section 5.4 says expected row-count ranges are 'computed from lightweight schema statistics ... for the specific benchmark schema (TPC-H SF=1),' and the sample feedback in Section 5.5 hard-codes the dataset window 1992-01-01 to 1998-12-31 and the anonymised naming convention ('Region#2', not 'ASIA'). No deployment-mode precision or recall is measured. In the evaluation, OEGV relies on cached reference results and Section 5.7 reports that its semantic retry recovered zero additional queries. The manuscript should not imply that OEGV is a validated deployment guard; it is currently either a benchmark-only reference-comparison mechanism or an unvalidated heuristic.
minor comments (5)
  1. [Section 7.4, Table 9] The missing C2 tier-level logs are acknowledged, but the table's 'C2: Static hint' row could be marked 'not retained' more prominently to avoid readers inferring tier-level parity. The text already handles this correctly.
  2. [Section 8.5] The 98.7% schema-hallucination suppression rate is extrapolated from a 50-query manual sample of the 488 C1 failures to an assumed 469 schema-class failures. This should be labeled an estimate with a confidence interval or a conservative range, not a precise rate.
  3. [Section 10.1, T-I2] The mitigation for the harness-as-confounder threat says 'running a baseline condition whose EGT is consistent with prior work.' Which prior work? The no-hint baseline of 2.2% is hard to compare because there is no prior Oracle ADB benchmark. Please specify the comparison or remove the phrase.
  4. [Section 4.2.2] The JOIN-chain map J is said to be 'equivalently derived from Oracle constraint metadata,' but the implementation uses the hand-built TPC-H adjacency graph. If constraint metadata derivation is feasible, the authors should implement it or at least describe how it would eliminate the manual step; as written, this sentence undercuts the 'no manual curation' claim.
  5. [Section 2.6 / Table 1] The sentence 'academic systems address (iv) but not (i)–(iii)' is ambiguous because the related-work systems in Table 1 are academic and do address the 'no retrain' dimension. Rephrase to clarify that SAL is the first to combine all four dimensions.

Circularity Check

1 steps flagged

SAL's 'no manual schema curation' claim is undercut by the paper's own design: detection maps and thresholds are hand-built with TPC-H knowledge, making 62.6% EGT an in-sample fitted score.

specific steps
  1. fitted input called prediction [Abstract 'no manual schema curation'; Sections 4.2.1–4.2.3 (Eqs. 3–6), 6.2, 10.1 (T-I4), 12]
    "Each table t∈T is associated with a domain vocabulary Kt — a curated set of keywords drawn from column names, common synonyms, and domain terms. ... The adjacency map encodes the natural JOIN relationships in the TPC-H schema ... J is constructed once from the known TPC-H foreign-key graph ... The thresholds used in this gate (|T_direct| < 3 and cplx(q) ≥ 2) were chosen to trade off prompt length against missed JOIN context on our 500-question benchmark and were kept fixed across all reported experiments."

    The load-bearing premise is that SAL's table selection is automatic, so that 62.6% EGT demonstrates 'no manual schema curation.' But the selection mechanism is itself manually curated for TPC-H: K_t, J, P, and the gate thresholds are hand-built and tuned on the same 500-question benchmark used to report EGT. The reported accuracy is therefore a fitted, in-sample score for the benchmark, not an out-of-sample prediction that the system transfers to new schemas without curation. T-I4 acknowledges 'This is a form of dataset leakage' and Section 12 states the keyword maps and JOIN adjacency were 'designed with knowledge of TPC-H table names.' The execution-grounded reference results are external, so the fit does not mathematically force the EGT value, but the central generalization claim reduce

full rationale

The derivation chain is not mathematically circular: EGT is computed by executing generated SQL against a live Oracle instance and comparing to reference result sets, and no self-citation is load-bearing. The external ground truth prevents a score above 6. However, the paper's central generalization claim—'no manual schema curation'—is contradicted by its own inputs. SAL's table detector is implemented by curated keyword vocabularies K_t, a TPC-H foreign-key adjacency map J, a manually chosen complexity phrase set P, and thresholds explicitly tuned 'on our 500-question benchmark.' Section 6.2 confirms the questions were authored by the same team that built these maps, and T-I4 labels this 'the primary internal validity threat.' Thus the reported 62.6% is an in-sample fitted score for TPC-H, and the system's transferability to schemas without manual curation is asserted, not derived. The Hidx validator and live catalog grounding are independent and externally verifiable, which is why this is partial circularity rather than full equivalence.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

No new physical or mathematical entities are postulated; the new constructs are software components (SAL Detect, Hidx) whose behavior is specified. The central empirical claim depends on several hand-fitted detection parameters and schema-specific assumptions, which is the main burden.

free parameters (5)
  • Per-table keyword vocabulary K_t = Not fully specified; examples given for Lineitem/Orders
    Manually curated sets of keywords drawn from TPC-H column names and domain terms (Section 4.2.1); developed with knowledge of TPC-H table names (T-I4).
  • Complexity-gate thresholds = |T_direct| < 3; cplx(q) >= 2
    Chosen to trade off prompt length against missed JOIN context on the 500-question benchmark (Section 4.2.3); no cross-validation (T-V3).
  • Complexity indicator phrase set P = Examples: 'who', 'revenue', 'for each', 'across', 'total', 'average'
    Hand-selected multi-table signal phrases in Equation (5); not fixed by any external principle.
  • JOIN-chain adjacency map J = J(Lineitem) = {Orders, Part, Supplier, Partsupp}, etc.
    Static adjacency list constructed from the TPC-H foreign-key graph (Section 4.2.2); schema-specific and hand-coded, though derivable from constraints.
  • Retry budgets and OEGV heuristic thresholds = 2 Hidx retries; 1 OEGV retry; 50% NULL density flag; row-count ranges
    Chosen by the authors; OEGV recovered zero queries, so these do not affect the headline EGT but are still hand-set.
axioms (5)
  • domain assumption Oracle identifier scoping: a column reference alias.column must exactly match USER_TAB_COLUMNS.COLUMN_NAME, otherwise ORA-00904 is raised.
    Definition 2; the whole Hidx validation depends on this exact-match semantics with no fuzzy resolution.
  • domain assumption The 500 reference SQL statements are correct ground truth for semantic equivalence.
    Section 6.2: reference SQL written by domain experts and validated by execution, but not independently audited; a wrong reference would corrupt EGT.
  • ad hoc to paper Generated SQL is single-block and matches Hidx's regex grammar for FROM/JOIN aliases; subqueries, CTEs, and quoted identifiers are out of scope.
    Section 4.3.1 Limitation: the parser skips aliases introduced by subqueries/CTEs, so validation coverage is limited to the prompt template's style.
  • domain assumption Execution on the live Oracle instance plus canonicalized multiset comparison is a valid correctness measure.
    Definition 8 and Section 6.5; construct validity is discussed in T-C1/T-C2 but not proven.
  • domain assumption TPC-H at SF=1 with anonymised region/nation names is representative of enterprise Oracle analytical workloads.
    Section 3.4 and T-E1; the paper itself rates single-schema generalizability as high-severity.

pith-pipeline@v1.3.0-alltime-deepseek · 21103 in / 11638 out tokens · 111278 ms · 2026-08-02T12:22:56.531771+00:00 · methodology

0 comments
read the original abstract

Large language models can generate fluent SQL from natural language, but on real enterprise Oracle databases they frequently fail at execution time: columns and aliases are hallucinated and dialect-specific syntax is missed, leading to ORA-00904 invalid-identifier errors. In this setting, failures are primarily due to missing schema grounding: the model cannot know which tables and columns actually exist. This paper introduces Schema-Aware Localisation (SAL), a lightweight middleware layer for Oracle NL2SQL that requires no model retraining. SAL queries Oracle's USER_TAB_COLUMNS catalog to build a live schema map, selects a relevant table subset for each question (falling back to the full schema for multi-table queries), and injects this ground-truth context into the LLM prompt. Generated SQL is then checked by the Hallucination Index (Hidx), which validates every alias.column reference against the live catalog, automatically rewrites predictable prefix errors, and otherwise triggers a structured retry with itemised corrections. We evaluate SAL on 500 TPC-H natural language questions executed against a live Oracle Autonomous Database 23c instance using GPT-4o-mini. Without any schema grounding, execution-grounded truth (EGT; executes and matches the reference result set) is 2.2% (12/500). A hand-written static schema hint brings EGT to 62.0%. SAL, with no manual schema curation, achieves 62.6% EGT (96% simple, 95% medium, 40.7% complex) while reducing execution failures from 97.6% to 2.6%.

Figures

Figures reproduced from arXiv: 2607.22572 by Divya Chukkapalli, Ganesh R. Naik, Sanjay Mishra.

Figure 1
Figure 1. Figure 1: SAL end-to-end architecture. The server selects relevant tables, injects live [PITH_FULL_IMAGE:figures/full_fig_p012_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Online execution-grounded verification (OEGV) pipeline. [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Offline benchmark harness control flow. Phase 1 precomputes ground-truth result [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 4 linked inside Pith

  1. [1]

    T. Yu, R. Zhang, K. Yang, M. Yasunaga, D. Wang, Z. Li, J. Ma, I. Li, Q. Yao, S. Roman, et al., Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task, in: Proc. EMNLP, 2018, pp. 3911–3921

    2018

  2. [2]

    Anthropic, Model context protocol specification, https: //modelcontextprotocol.io, 2024

    2024

  3. [3]

    W. A. Woods, Progress in natural language understanding: An applica- tion to lunar geology, in: Proc. AFIPS Natl. Comput. Conf., 1973, pp. 441–450

    1973

  4. [4]

    G. G. Hendrix, E. D. Sacerdoti, D. Sagalowicz, J. Slocum, Developing a natural language interface to complex data, ACM Transactions on Database Systems (1978)

    1978

  5. [5]

    Zhong, C

    V. Zhong, C. Xiong, R. Socher, Seq2sql: Generating structured queries from natural language using reinforcement learning, arXiv preprint arXiv:1709.00103, 2017

    Pith/arXiv arXiv 2017

  6. [6]

    Li, et al., BIRD: A big bench for large-scale database grounded text-to-sql evaluation, in: Proc

    J. Li, et al., BIRD: A big bench for large-scale database grounded text-to-sql evaluation, in: Proc. NeurIPS, 2023

    2023

  7. [7]

    L. Dong, M. Lapata, Language to logical form with neural attention, in: Proc. ACL, 2016

    2016

  8. [8]

    X. Xu, C. Liu, D. Song, Sqlnet: Generating structured queries from natural language without reinforcement learning, arXiv preprint arXiv:1711.04436, 2017

    Pith/arXiv arXiv 2017

  9. [9]

    T. Yu, Z. Li, Z. Zhang, R. Zhang, D. Radev, Typesql: Knowledge-based type-aware neural text-to-sql generation, in: Proc. NAACL, 2018

    2018

  10. [10]

    Chen, et al., Shadowgnn: Graph neural networks for multi-turn text-to-sql parsing, in: Proc

    B. Chen, et al., Shadowgnn: Graph neural networks for multi-turn text-to-sql parsing, in: Proc. AAAI, 2021

    2021

  11. [11]

    Cai, et al., Igsql: Database schema interaction graph for conversational text-to-sql, in: Proc

    Y. Cai, et al., Igsql: Database schema interaction graph for conversational text-to-sql, in: Proc. ACL, 2020. 46

    2020

  12. [12]

    Devlin, M.-W

    J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in: Proc. NAACL, 2019

    2019

  13. [13]

    Raffel, et al., Exploring the limits of transfer learning with a unified text-to-text transformer, Journal of Machine Learning Research (2020)

    C. Raffel, et al., Exploring the limits of transfer learning with a unified text-to-text transformer, Journal of Machine Learning Research (2020)

    2020

  14. [14]

    X. V. Lin, et al., Bridge: Bridging the gap between neural text-to-sql and sql execution, in: Proc. ACL, 2020

    2020

  15. [15]

    Yu, et al., Grappa: Grammar-augmented pre-training for table semantic parsing, in: Proc

    T. Yu, et al., Grappa: Grammar-augmented pre-training for table semantic parsing, in: Proc. ICLR, 2021

    2021

  16. [16]

    Gao, et al., Text-to-sql empowered by large language models: A benchmark evaluation, arXiv preprint arXiv:2308.15363, 2023

    D. Gao, et al., Text-to-sql empowered by large language models: A benchmark evaluation, arXiv preprint arXiv:2308.15363, 2023

    Pith/arXiv arXiv 2023

  17. [17]

    Pourreza, D

    M. Pourreza, D. Rafiei, Din-sql: Decomposed in-context learning of text-to-sql with self-correction, in: Proc. NeurIPS, 2023

    2023

  18. [18]

    Dong, et al., C3: Zero-shot text-to-sql with chatgpt, arXiv preprint arXiv:2307.07306, 2023

    X. Dong, et al., C3: Zero-shot text-to-sql with chatgpt, arXiv preprint arXiv:2307.07306, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    Wang, et al., Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers, in: Proc

    B. Wang, et al., Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers, in: Proc. ACL, 2020

    2020

  20. [20]

    Lei, et al., Semql: A meaning representation for natural language to sql, in: Proc

    F. Lei, et al., Semql: A meaning representation for natural language to sql, in: Proc. ACL, 2020

    2020

  21. [21]

    Cao, et al., Lgesql: Line graph enhanced text-to-sql model with mixed local and non-local relations, in: Proc

    R. Cao, et al., Lgesql: Line graph enhanced text-to-sql model with mixed local and non-local relations, in: Proc. ACL, 2021

    2021

  22. [22]

    Guo, et al., Irnet: A general framework for complex text-to-sql parsing, in: Proc

    J. Guo, et al., Irnet: A general framework for complex text-to-sql parsing, in: Proc. ACL, 2019

    2019

  23. [23]

    Maynez, S

    J. Maynez, S. Narayan, B. Bohnet, R. McDonald, On faithfulness and factuality in abstractive summarization, in: Proc. ACL, 2020

    2020

  24. [24]

    Ji, et al., Survey of hallucination in natural language generation, ACM Comput

    Z. Ji, et al., Survey of hallucination in natural language generation, ACM Comput. Surv. 55 (2023) 1–38

    2023

  25. [25]

    Liu, et al., Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation, in: Proc

    J. Liu, et al., Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation, in: Proc. NeurIPS, 2023. 47

    2023

  26. [26]

    Ni, et al., Sql generation with execution-guided decoding, in: Proc

    A. Ni, et al., Sql generation with execution-guided decoding, in: Proc. ACL, 2023

    2023

  27. [27]

    Oracle Corporation, Oracle database SQL language reference 23c, https://docs.oracle.com/en/database/oracle/oracle-database/ 23/sqlrf/, 2024

    2024

  28. [28]

    com/en/platform/features/ai/, 2024

    Oracle Corporation, Oracle apex ai assistant,https://apex.oracle. com/en/platform/features/ai/, 2024. Accessed May 12, 2026

    2024

  29. [29]

    Schick, et al., Toolformer: Language models can teach themselves to use tools, in: Proc

    T. Schick, et al., Toolformer: Language models can teach themselves to use tools, in: Proc. NeurIPS, 2023

    2023

  30. [30]

    Yao, et al., ReAct: Synergizing reasoning and acting in language models, in: Proc

    S. Yao, et al., ReAct: Synergizing reasoning and acting in language models, in: Proc. ICLR, 2023

    2023

  31. [31]

    Lewis, et al., Retrieval-augmented generation for knowledge-intensive nlp tasks, in: Proc

    P. Lewis, et al., Retrieval-augmented generation for knowledge-intensive nlp tasks, in: Proc. NeurIPS, 2020

    2020

  32. [32]

    URL:http://www.tpc

    Transaction Processing Performance Council (TPC), TPC Benchmark H Standard Specification, Revision 2.18.0, 2022. URL:http://www.tpc. org/tpch/, accessed 2026-05-12

    2022

  33. [33]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, Chain-of-thought prompting elicits reasoning in large language models, in: Advances in Neural Information Processing Systems, 2022

    2022

  34. [34]

    N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, P. Liang, Lost in the middle: How language models use long contexts, in: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), 2023

    2023

  35. [35]

    Wohlin, P

    C. Wohlin, P. Runeson, M. H"ost, M. C. Ohlsson, B. Regnell, A. Wesslén, Experimentation in Software Engineering, Springer, 2012. 48 Table 14: Threat Summary: Severity and Mitigation Status ID Category Threat Severity T-I1 Internal LLM non- determinism Low T-I2 Internal Harness as con- founder Low T-I3 Internal Network latency vari- ability Medium T-I4 Int...

    2012