Pith. sign in

REVIEW 4 major objections 5 minor 39 references

Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims schema lineage can be extracted automatically from real multilingual pipeline scripts, and that a 32B open-weight model with one chain-of-thought example matches GPT-4o and GPT-4.1 on the proposed SLiCE metric.

desk verdict Useful benchmark and a plausible formalism, but the headline 32B-result is not yet substantiated because the CoT examples may leak the gold answer, and the metric is unvalidated. read the letter →

arxiv 2508.07179 v1 pith:CB46LKQN submitted 2025-08-10 cs.CL cs.AIcs.DB

Jiaqi Yin , Yi-Wei Chen , Meng-Lung Lee , Xiya Liu This is my paper
classification cs.CLcs.AIcs.DB
keywords schemalineagedatasemanticdriftevaluationmetricSLiCEchain-of-thoughtpromptingsmalllanguagemodelsmultilingualpipelines
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper takes on semantic drift in enterprise data pipelines—the way derived datasets lose the meaning of their original columns after SQL, Python, and C# transformations. It defines schema lineage as a four-part structured record (source schemas, source tables, transformation logic, aggregation) and proposes the SLiCE metric to score model predictions against 1,700 human-annotated lineages from 50 real production scripts. The central experimental finding is that performance scales with model size and prompting richness, and that a 32B open-weight model (Qwen2.5-Coder-32B) given a single chain-of-thought example scores about as well as GPT-4o and GPT-4.1. If the claim holds, organizations can deploy schema-aware agents in-house at 32B scale rather than paying for or hosting much larger proprietary APIs.

What carries the argument

The Schema Lineage Composite Evaluation (SLiCE) metric is the central object: $\text{SLiCE}(\hat L, L^\star) = M_{fmt}(\hat L)\cdot M_{src}(\hat L,L^\star)\cdot[\omega_{tbl}M_{tbl}+\omega_{trf}M_{trf}+\omega_{agg}M_{agg}]$. It multiplies a binary format-correctness term by a binary source-schema exact-match term and a weighted sum of table similarity (exact-match F1 plus Levenshtein-based fuzzy matching), transformation similarity, and aggregation similarity. Transformation and aggregation similarity combine BLEU, weighted BLEU, and a multilingual AST similarity that adapts CodeBLEU to partial, multi-language code fragments. The metric exposes component-level diagnostics while producing a si

What would settle it

Re-score the 1,700 lineages with alternative weight choices—for instance, equal weights, or weights fit to human ratings of lineage usefulness on a held-out sample—and check whether Qwen2.5-Coder-32B with CoT-1 still falls within a few points of GPT-4.1. If the ranking or the gap changes materially, the comparable-performance claim is an artifact of the chosen weights.

Watch

Extended reading notes

Core claim

The paper's central claim is that schema lineage—a structured record of source columns, source tables, transformation logic, and aggregation—can be extracted automatically from raw multi-language pipeline scripts by language models, and the extraction quality can be measured by the proposed Schema Lineage Composite Evaluation (SLiCE) metric. On a new benchmark of 1,700 manually annotated lineages from 50 real-world SQL/Python/C# scripts, 12 models were scored under base, few-shot, and chain-of-thought prompting. The headline result is that Qwen2.5-Coder-32B, prompted with a single human reasoning trace, reaches a corpus-level SLiCE score of 0.734, compared with 0.767 for GPT-4.1 and 0.759 fo

Load-bearing premise

Every model ranking and the headline 32B-versus-GPT comparison rests on hand-chosen SLiCE weights (for example, source tables 0.4, transformation 0.4, aggregation 0.2) that are not validated against any external measure of lineage quality; if those weights are off, the scores and the comparison move.

Editorial extensions

If this is right

  • Schema lineage extraction succeeds with in-context prompting: no task-specific fine-tuning is needed, just base, few-shot, or chain-of-thought prompts.
  • One high-quality human reasoning trace (CoT-1) yields a large jump over one-shot and is enough for a 32B open model to rival GPT-4.1.
  • Within a model family, larger models score higher; sub-3B models lose accuracy when CoT is added, likely due to emergent-reasoning limits and context-length effects.
  • SLiCE provides a component-level diagnostic (format, source, table, transformation, aggregation) that can be used as a reward signal for future fine-tuning or reinforcement learning.
  • The finding supports practical in-house deployment of schema-aware agents with open-weight 32B models, avoiding per-token API costs for scripts of hundreds of thousands of tokens.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The metric's weights are unvalidated design choices; a re-weighting could reshuffle model rankings, so the 32B-vs-GPT comparison should be rechecked under alternative weightings.
  • The same CoT-with-one-trace recipe could transfer to other structured provenance tasks, such as notebook cell lineage or ETL job documentation, as a direct testable extension.
  • The paper's observation that additional reasoning traces give diminishing returns suggests annotation effort is best spent on one trace per script type, not on maximizing trace counts.
  • If SLiCE were validated against human judgment of lineage quality, it could become a shared standard benchmark for lineage extraction; without such validation, it remains a project-specific score.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a formal definition of schema lineage for multilingual enterprise pipeline scripts, a new composite evaluation metric (SLiCE), a manually annotated benchmark of 1,700 schema lineages from 50 real-world SQL/Python/C# scripts, and an empirical comparison of 12 language models under base, few-shot, and chain-of-thought prompting. The headline claim is that Qwen2.5-Coder-32B with a single human-written reasoning trace reaches SLiCE scores (0.734) comparable to GPT-4.1 (0.767) and GPT-4o (0.759), suggesting that open-weight 32B models can substitute for proprietary LLMs in this task.

Significance. If the benchmark and metric are valid, this is a useful contribution: schema lineage extraction is important for data governance and RAG/text-to-SQL pipelines, and the 1,700-example multilingual dataset is larger than existing open lineage benchmarks. The manuscript also ships a reasonably broad model sweep with six random seeds and clear prompting ablations. However, the strength of the central claim depends on three currently unresolved points: (i) whether the few-shot/CoT examples are held out from the evaluated scripts, (ii) whether the hand-set SLiCE weights are robust, and (iii) whether the gold annotations are reliable enough to serve as ground truth. Each of these is fixable, but without fixes the headline comparison is not interpretable.

major comments (4)
  1. [§5.1, §3.3, §A.3] The construction of few-shot/CoT examples is not described with a train/test split. The paper states: "Data experts crafted human reasoning traces ... one reasoning trace per easy script, two per medium script, and three per hard script" (§5.1), and §3.3 scales example counts by script complexity. It is not stated whether the example for a given script S is derived from S itself or from a disjoint script. If the reasoning trace for S is the gold trace for S's schemas, then the CoT-1 prompt contains the answer and the headline CoT-1 scores (Table 2: Qwen2.5-Coder-32B 0.734 vs GPT series 0.759–0.767) are not valid measurements of extraction capability. The authors must either explicitly state and demonstrate disjointness (e.g., leave-one-script-out example selection) or release the exact prompts/examples so readers can verify this.
  2. [§3.2, Eq. (8), §5.1] The SLiCE composite score uses hand-set weights (ωtbl=0.4, ωtrf=0.4, ωagg=0.2; wtbl1=0.7, wtbl2=0.3; wtrf1=wagg1=0.5, wtrf2=wagg2=0.3, wtrf3=wagg3=0.2) with no validation, no sensitivity analysis, and no comparison against human judgments. All model rankings and the central "comparable to GPT" claim are computed under this single arbitrary weighting. Since Mfmt and Msrc are binary gates, small perturbations in the component weights can shift the score by more than the observed gap between the 32B model and GPT-4.1 (0.033) or GPT-4o (0.025). The authors should either calibrate the weights against human pairwise judgments or report a robustness analysis (e.g., rank reversals over the weight simplex).
  3. [§2.2, §2.1] The benchmark is described as "manually annotated by human experts" with "strict consistency guidelines," but no inter-annotator agreement is reported, no double-annotation sample is described, and no annotation guidelines are provided beyond a prose description. Since every model score is computed against this gold standard, undisclosed annotation variability directly threatens the validity of all reported scores and the benchmark's usefulness. The authors should report IAA on a random subset (e.g., Cohen's kappa or agreement on each of the four lineage components) and release the annotation guidelines.
  4. [§5.1, §5.2, Related Work] The experimental comparison includes only LLM/SLM variants. The related work (Section 4) cites conventional code-analysis methods for lineage extraction (AST parsing, metadata mapping, runtime analysis, refs. [14–17]), but no such deterministic baseline is run on the new benchmark. Without a non-LLM baseline, it is unclear whether the LLM results reflect intrinsic difficulty of schema lineage extraction or merely prompt-following ability. Adding a simple rule-based/AST baseline, even with expected low recall on multilingual scripts, would substantially strengthen the claim that the proposed framework improves on prior automated lineage extraction.
minor comments (5)
  1. [§5.1 / Table 2] The paper reports mean ± std across six seeds but no statistical significance tests. The headline 32B-vs-GPT gaps are small; paired tests (e.g., per-script paired bootstrap or Wilcoxon) would make the "comparable" claim more precise.
  2. [§3.2, Eq. (4)] The fuzzy table similarity in Eq. (4) is computed on raw table strings, which in Table A.2 are full Azure blob URIs. The shared prefix of such URIs can dominate Levenshtein similarity, making the fuzzy score near 1 even for different tables. Please clarify whether table names are normalized (e.g., stripping file paths) before matching.
  3. [§3.2, Eq. (5)] The ASTmulti language weights w_l are defined as the normalized proportion of language-specific keywords. If a transformation or aggregation snippet contains no recognizable keywords, this weight assignment is undefined. Specify a fallback (e.g., uniform weights or syntax-based detection) and how partial/non-parseable code snippets are handled.
  4. [Throughout] Numerous typos and minor formatting issues: "repectively" (after Eq. 6), "Furthermose" (Related Work), "mportantly" (Conclusion), "exmaple" (Figure B.1 caption), "YOUR PINELINE SCRIPT" (Appendix A.3 base prompt), and inconsistent "CoT" vs "Cot" in Figure B.3 and text. Also, Tables B.3 Part 1/2/3 are all numbered the same. A careful proofread is needed.
  5. [§5.1 / Table 2] The exclusion of six SLMs due to "excessive inference time or consistently poor performance" is a post hoc selection rule. Please report the excluded model names and their preliminary scores, or at least state that the exclusion is based on the same protocol for all models, to avoid selection bias in the scaling trends.

Circularity Check

1 steps flagged · score 6.0 of 10

CoT-1 'prediction' likely contains the gold answer in the prompt, so the 32B-vs-GPT claim reduces to reproduction rather than extraction.

  1. fitted input called prediction [Section 5.1 (Experimental Setup) with Section 3.3 (Chain-of-Thought) and Appendix A.3 (Prompts)]
    "Data experts crafted human reasoning traces to support the CoT prompting strategy: one reasoning trace per easy script, two per medium script, and three per hard script. ... Chain-of-Thought (CoT): Building upon few-shot prompting, this advanced strategy incorporates detailed human-generated reasoning traces that illustrate step-by-step derivations of schema lineage from pipeline code."

    The reasoning traces are authored per script, and the CoT prompt template places 'Data Pipeline Script: YOUR PIPELINE SCRIPT' and 'Examples: YOUR OUTPUT EXAMPLE(S)' in the same prompt. If the example is the trace for that same script, as the per-script wording implies, the step-by-step derivation of the gold lineage is inserted into the input. The model is then scored against that same gold lineage using SLiCE. Thus the CoT-1 results, including the headline Qwen2.5-Coder-32B (0.734) vs GPT-4.1 (0.767) comparison, may measure copying or paraphrasing of the supplied reasoning trace rather than extraction from unseen code. The paper never states that examples or traces are drawn from different scripts or held out, so the central claim is not established as an independent prediction.

full rationale

The paper's central empirical claim rests on CoT-1 prompting, where human reasoning traces are provided as in-prompt examples. Section 5.1 says one trace is created 'per easy script,' and the prompt template includes the pipeline script and examples together. If the trace for a script is the trace of that same script—the natural reading of 'per script'—then the gold answer is part of the input, and the model's output is evaluated against that same gold answer. This is a reduction-by-construction of the 'prediction' to the input, and the paper provides no explicit held-out or cross-script statement to rule it out. The SLiCE weights are indeed arbitrary and unvalidated, but arbitrary weighting is a measurement-validity concern, not circularity; it does not by itself make a prediction reduce to its inputs. There are no load-bearing self-citations: the references are to external systems and unrelated prior work. If the authors were to confirm that every reasoning trace used in a CoT prompt comes from a different script, the circularity would be resolved and the score would drop to 0-2. As written, the ambiguity directly affects the paper's main '32B comparable to GPT' conclusion, warranting a score of 6.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claims rest on hand-chosen evaluation weights and unpublished proprietary data. The four-part lineage definition is a modeling choice, and representativeness of the 50 scripts is asserted. No new physical or formal entities are introduced; SLiCE is a metric, not an invented entity.

free parameters (6)
  • SLiCE composite weights ω_tbl, ω_trf, ω_agg = 0.4, 0.4, 0.2
    Set by hand in Section 5.1; no justification or sensitivity analysis; these weights directly determine every reported SLiCE score.
  • Table score weights w_tbl1, w_tbl2 = 0.7, 0.3
    Hand-chosen weights in Eq. 3 for exact and fuzzy table matching.
  • Transformation weights w_trf1, w_trf2, w_trf3 = 0.5, 0.3, 0.2
    Hand-chosen weights in Eq. 6 for BLEU, weighted BLEU, and AST similarity.
  • Aggregation weights w_agg1, w_agg2, w_agg3 = 0.5, 0.3, 0.2
    Hand-chosen weights in Eq. 7 for aggregation scoring.
  • Number of few-shot/CoT examples per difficulty = 1 easy, 2 medium, 3 hard
    Section 3.3 chooses these counts as a design heuristic; no evidence they are optimal or fair across models.
  • Script difficulty thresholds = 0-1 easy, 2 medium, 3 hard
    Appendix A.1 defines an ad hoc scoring rule with points for sources, transformations, and aggregations.
assumptions (5)
  • domain assumption Schema lineage can be fully represented by exactly four components: source_schema, source_table, transformation, aggregation.
    Section 2.2 defines this representation; the entire benchmark, metric, and annotations inherit this restriction.
  • domain assumption Human-expert annotations of the 1,700 lineages are correct and consistent.
    Section 2.2 mentions 'strict consistency guidelines' but no inter-annotator agreement is reported.
  • domain assumption The 50 Microsoft pipeline scripts are representative of general enterprise pipelines.
    Section 2.1 asserts representativeness without evidence from other organizations or domains.
  • domain assumption BLEU, weighted BLEU, Levenshtein fuzzy matching, and language-aware AST similarity adequately measure semantic equivalence of transformation and aggregation snippets.
    Section 3.2 builds SLiCE from these components; no validation against human similarity judgments is provided.
  • ad hoc to paper Excluding six SLMs due to 'excessive inference time or consistently poor performance' does not bias the model comparison.
    Section 5.1 reports the exclusion; performance-based exclusion could inflate the average capability of the remaining SLM cohort.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks." pith.science (2026). https://pith.science/paper/CB46LKQN

@misc{pith2026250807179,
  author       = {Pith},
  title        = {Pith review of: Schema Lineage Extraction at Scale: Multilingual Pipelines, Composite Evaluation, and Language-Model Benchmarks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CB46LKQN}},
  note         = {Machine review of arXiv:2508.07179}
}
read the original abstract

Enterprise data pipelines, characterized by complex transformations across multiple programming languages, often cause a semantic disconnect between original metadata and downstream data. This "semantic drift" compromises data reproducibility and governance, and impairs the utility of services like retrieval-augmented generation (RAG) and text-to-SQL systems. To address this, a novel framework is proposed for the automated extraction of fine-grained schema lineage from multilingual enterprise pipeline scripts. This method identifies four key components: source schemas, source tables, transformation logic, and aggregation operations, creating a standardized representation of data transformations. For the rigorous evaluation of lineage quality, this paper introduces the Schema Lineage Composite Evaluation (SLiCE), a metric that assesses both structural correctness and semantic fidelity. A new benchmark is also presented, comprising 1,700 manually annotated lineages from real-world industrial scripts. Experiments were conducted with 12 language models, from 1.3B to 32B small language models (SLMs) to large language models (LLMs) like GPT-4o and GPT-4.1. The results demonstrate that the performance of schema lineage extraction scales with model size and the sophistication of prompting techniques. Specially, a 32B open-source model, using a single reasoning trace, can achieve performance comparable to the GPT series under standard prompting. This finding suggests a scalable and economical approach for deploying schema-aware agents in practical applications.

Figures

Figures reproduced from arXiv: 2508.07179 by the authors.

Figure 1
Figure 1. A visual illustration of schema lineage definition and annotation, based on the formal [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Schema lineage extraction performance comparison across prompting strategies and script [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 21 canonical work pages

  1. [1]

    J. P. Müller and T. Stein. A framework for measuring semantic drift in ontologies. InJoint Proceedings of the Workshops on the Semantic Web: Semantics, Analytics, and Visualisation, SAW, and Trends in the Semantic Web, SemAW, volume 1695, pages 31–38. CEUR-WS, 2016. URL https://ceur-ws.org/Vol-1695/paper42.pdf

    work page 2016

  2. [2]

    Cui and J

    Y. Cui and J. Widom. Lineage tracing for general data warehouse transformations.VLDB, 12 (1):41–58, May 2003. doi: 10.1007/s00778-002-0083-8. URLhttps://doi.org/10.1007/ s00778-002-0083-8

    work page doi:10.1007/s00778-002-0083-8 2003

  3. [3]

    Evaluating large language models trained on code, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, et al. Evaluating large language models trained on code, 2021. URLhttps: //arxiv.org/abs/2107.03374

    arXiv 2021

  4. [4]

    Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, et al

    Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, et al. On the opportunities and risks of foundation models, 2022. URLhttps: //arxiv.org/abs/2108.07258

    arXiv 2022

  5. [5]

    Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021. URL https://arxiv.org/abs/2005.11401. 11 REFERENCES REFERENCES

    arXiv 2021

  6. [6]

    Seven failure points when engineering a retrieval augmented generation system, 2024

    Scott Barnett, Stefanus Kurniawan, Srikanth Thudumu, Zach Brannelly, and Mohamed Abdel- razek. Seven failure points when engineering a retrieval augmented generation system, 2024. URL https://arxiv.org/abs/2401.05856

    arXiv 2024

  7. [7]

    SCOPE: parallel databases meet MapReduce.VLDB Journal, 21(5):611–636, 2012

    Jingren Zhou, Nicolas Bruno, Ming-Chuan Wu, Per-Åke Larson, Ronnie Chaiken, and Darren Shakib. SCOPE: parallel databases meet MapReduce.VLDB Journal, 21(5):611–636, 2012. doi: 10.1007/s00778-011-0231-0

    work page doi:10.1007/s00778-011-0231-0 2012

  8. [8]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948

    arXiv 2025

Show all 39 references
  1. [9]

    URLhttps://arxiv.org/abs/2009.10297

    ShuoRen,DayaGuo,ShuaiLu,LongZhou,ShujieLiu,DuyuTang,etal.Codebleu: amethodfor automatic evaluation of code synthesis, 2020. URLhttps://arxiv.org/abs/2009.10297

    2020 arXiv

  2. [10]

    V. I. Levenshtein. Binary codes capable of correcting deletions, insertions, and reversals.Soviet Physics Doklady, 10(8):707–710, 1966

    1966

  3. [11]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProc. Association for Computational Linguistics (ACL), pages 311–318, Philadelphia, Pennsylvania, USA, July 2002. doi: 10.3115/1073083.1073135. URL ht...

    2002

  4. [12]

    Parshin Shojaee, Aneesh Jain, Sindhu Tipirneni, and Chandan K. Reddy. Execution-based code generation using deep reinforcement learning, 2023. URLhttps://arxiv.org/abs/2301. 13816

    2023

  5. [13]

    Efficient memory management for large language model serving with pagedattention, 2023

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, et al. Efficient memory management for large language model serving with pagedattention, 2023. URL https://arxiv.org/abs/2309.06180

    2023 arXiv

  6. [14]

    sqlparse: a non-validating sql parser for python

    AndiAlbrecht, VictorUriarte, JesúsLeganés-Combarro, JonDufresne, AdamGreenhall, Simon Heisterkamp, et al. sqlparse: a non-validating sql parser for python. Python package (PyPI, Read the Docs), 2025. URLhttps://pypi.org/project/sqlparse/. Online; accessed 2025-07-30

    2025

  7. [15]

    Open source project (sqlfluff.com, GitHub), 2025

    Sqlfluff: The sql linter for humans. Open source project (sqlfluff.com, GitHub), 2025. URL https://sqlfluff.com/. Online; accessed 2025-07-30

    2025

  8. [16]

    Data lineage in classic data catalog

    Microsoft. Data lineage in classic data catalog. Microsoft Learn, Jul 2025. URL https://learn.microsoft.com/en-us/purview/data-gov-classic-lineage. On- line; accessed 2025-07-30

    2025

  9. [17]

    Online; accessed 2025-07-30

    Foundational,Inc.Automateddatalineagetool.Productdescription(foundational.io),2025.URL https://www.foundational.io/product/data-lineage. Online; accessed 2025-07-30

    2025

  10. [18]

    A large language model-based approach for data lineage parsing

    Zhangti Li, Wenbin Guo, Yabing Gao, Di Yang, and Lin Kang. A large language model-based approach for data lineage parsing. Electronics, 14(9):1762, April 2025. doi: 10.3390/ electronics14091762. URL https://www.mdpi.com/2079-9292/14/9/1762

    2025

  11. [19]

    Introducing the Large Lineage Model (LLiM): Our Path to Securing the FutureofData

    Volodymyr Kuznetsov. Introducing the Large Lineage Model (LLiM): Our Path to Securing the FutureofData. CyberhavenEngineeringBlog,March2025. URL https://www.cyberhaven. com/engineering-blog/large-lineage-model-llim-our-path-securing-data/ . Online; accessed 2025-07-30

    2025

  12. [20]

    Tpc-h benchmark – apache doris documentation.https://doris

    The Apache Doris Project. Tpc-h benchmark – apache doris documentation.https://doris. apache.org/docs/benchmark/tpch/. Accessed: 2025-07-30

    2025

  13. [21]

    An open dataset of data lineage graphs for data governance research.Visual Informatics, 8(1):1–5,

    Yunpeng Chen, Ying Zhao, Xuanjing Li, Jiang Zhang, Jiang Long, and Fangfang Zhou. An open dataset of data lineage graphs for data governance research.Visual Informatics, 8(1):1–5,

  14. [22]

    LlamaIndex, 11 2022

    Jerry Liu. LlamaIndex, 11 2022. URLhttps://github.com/jerryjliu/llama_index. 12 REFERENCES REFERENCES

    2022

  15. [23]

    Spider: A large-scalehuman-labeleddatasetforcomplexandcross-domainsemanticparsingandtext-to-sql task, 2019

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, et al. Spider: A large-scalehuman-labeleddatasetforcomplexandcross-domainsemanticparsingandtext-to-sql task, 2019. URLhttps://arxiv.org/abs/1809.08887

    2019 arXiv

  16. [24]

    Gpt-4.1, 2025

    OpenAI. Gpt-4.1, 2025. URLhttps://openai.com/index/gpt-4-1/. Accessed: 2025- 07-30

    2025

  17. [25]

    Hello gpt-4o, May 2024

    OpenAI. Hello gpt-4o, May 2024. URLhttps://openai.com/index/hello-gpt-4o/

    2024

  18. [26]

    Qwen2.5-coder technical report, 2024

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, et al. Qwen2.5-coder technical report, 2024. URLhttps://arxiv.org/abs/2409.12186

    2024 arXiv

  19. [27]

    Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, et al

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, et al. Mistral 7b, 2023. URL https: //arxiv.org/abs/2310.06825

    2023 arXiv

  20. [28]

    Codestral, 2024

    Mistral AI. Codestral, 2024. URLhttps://mistral.ai/news/codestral/

    2024

  21. [29]

    Code llama: Open foundation models for code, 2024

    Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, et al. Code llama: Open foundation models for code, 2024. URLhttps://arxiv.org/abs/ 2308.12950

    2024 arXiv

  22. [30]

    Deepseek- coder-v2: Breaking the barrier of closed-source models in code intelligence, 2024

    DeepSeek-AI, Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, et al. Deepseek- coder-v2: Breaking the barrier of closed-source models in code intelligence, 2024. URL https://arxiv.org/abs/2406.11931

    2024 arXiv

  23. [31]

    Phi-4-mini-reasoning: Exploring the limits of small reasoning language models in math, 2025

    Haoran Xu, Baolin Peng, Hany Awadalla, Dongdong Chen, Yen-Chun Chen, Mei Gao, et al. Phi-4-mini-reasoning: Exploring the limits of small reasoning language models in math, 2025. URL https://arxiv.org/abs/2504.21233

    2025 arXiv

  24. [32]

    Phi-4 technical report, 2024

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, et al. Phi-4 technical report, 2024. URLhttps://arxiv.org/abs/2412. 08905

    2024

  25. [33]

    Phi-4-reasoningtechnicalreport, 2025

    Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, LingjiaoChen, etal. Phi-4-reasoningtechnicalreport, 2025. URL https://arxiv.org/abs/ 2504.21318

    2025 arXiv

  26. [34]

    Asurveyonlargelanguage models for code generation, 2024

    JuyongJiang,FanWang,JiasiShen,SungjuKim,andSunghunKim. Asurveyonlargelanguage models for code generation, 2024. URLhttps://arxiv.org/abs/2406.00515

    2024 arXiv

  27. [35]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, et al. Chain-of-thought prompting elicits reasoning in large language models, 2023. URLhttps: //arxiv.org/abs/2201.11903

    2023 arXiv

  28. [36]

    email_address

    Nelson F. Liu, Kevin Dabrol, John Bradshaw, Bryan McMahan, William Fedus, Noam Shazeer, Kuang-Huei Li, and Adams Wei Yu. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024. doi: 10.1162/tac...

    2024 doi

  29. [38]

    </think>

    <think> ... </think>

  30. [39]

    source_schema

    <answer> {{ 19 A.3 Prompts A DATA GALLERY "source_schema": "...", "source_table": "...", "transformation": "...", "aggregation": "..." }} </answer>. ... (additional instructions omitted for brevity) ... Data Pipeline Script: YOUR PIPELINE SCRIPT Examples: YOUR OUTPUT EXAMPLE(S...

  31. [2024]

    URL http://dblp.uni-trier.de/db/journals/vi/vi8.html#ChenZLZLZ24

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.