Pith. sign in

REVIEW 3 major objections 4 minor 47 references

Generating a Low-code Complete Workflow via Task Decomposition and RAG

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that Task Decomposition and Retrieval-Augmented Generation, formalized as GenAI design patterns, produced a modular, secure, testable workflow-generation system in an industry case study.

desk verdict A solid, readable experience report that formalizes Task Decomposition and RAG as design patterns, but its strongest security claim doesn't follow from the described architecture. read the letter →

arxiv 2412.00239 v1 pith:EPLGSKR2 submitted 2024-11-29 cs.SE cs.AI

Orlando Marquez Ayala , Patrice Béchard This is my paper · ORCID
classification cs.SEcs.AI
keywords generativeAIdesignpatternstaskdecompositionretrieval-augmentedgenerationworkflowlow-codesoftwarequalityattributescasestudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that two existing AI techniques, Task Decomposition and Retrieval-Augmented Generation, deserve to be treated as reusable software design patterns for generative-AI systems, and that teams should choose them with software quality attributes in mind. It grounds this in an industry case study: building Workflow Generation, a low-code system that turns a natural-language requirement into a structured workflow plan using environment artifacts such as database tables and columns. The authors report that these patterns produced a modular, flexible, secure, and testable system, at the price of added complexity in data labeling and model training. The broader claim is that documenting such patterns helps bridge the gap between AI model building and software engineering practice.

What carries the argument

The load-bearing machinery is the pair of formalized patterns. Task Decomposition is divide-and-conquer for ML: an orchestrator, here plain code, splits generation into sub-tasks, each handled by the foundation model, so that output size, latency, labeling effort, and evaluation granularity shrink. RAG adds a retriever that indexes environment artifacts and, at generation time, suggests candidate step names, table names, columns, and values to the FM, using a special choices: token that halts generation to call the retriever when the model needs suggestions. The architecture that carries the argument is the three-layer UI, AI, and data structure, plus the Flow Similarity metric that represents workflows as trees and scores generation by tree edit distance, allowing outline and input-population quality to be evaluated separately.

What would settle it

A comparable team building the same workflow generator without task decomposition or without retrieval, or with only one pattern, and measuring hallucination rate, latency, and maintainability would test the causal claim; if the monolithic version matches the decomposed version on these attributes, the paper's central conclusion loses its empirical support.

Watch

Extended reading notes

Core claim

The core discovery is that Task Decomposition and RAG, formalized as design patterns, are sufficient scaffolding for a production GenAI application that generates structured, code-like output with low hallucination and acceptable engineering quality. Workflow Generation splits the task into createFlow, which produces the ordered outline of steps, and populateInputs, which fills in each step's inputs from a user annotation and prior steps; a separate retriever supplies environment data as candidate choices. The same 7-billion-parameter fine-tuned model handles both sub-tasks, and a deterministic tree-edit-distance metric called Flow Similarity evaluates output. The authors claim the result is a modular, flexible, secure, and testable system whose main cost is extra complexity in model training, and they recommend the two patterns for other GenAI applications.

Load-bearing premise

The paper's conclusion that the observed engineering benefits came from the two patterns rests on the assumption that those benefits were not instead caused by the team's skill, the model choice, or the supportive enterprise setting, a limitation the authors themselves acknowledge.

Editorial extensions

If this is right

  • Teams building GenAI systems with complex structured output can ship a first version by decomposing the task and enabling sub-features incrementally, as the outline-only release did before full input population.
  • Using RAG with retrievable environment artifacts lets the FM be fine-tuned without memorizing customer-specific data, supporting security and per-installation customization.
  • Separate sub-tasks and separate retrieval make evaluation granular: outline correctness, whole-workflow correctness, per-step input quality, and retriever recall can be measured independently.
  • Deployment becomes more flexible: complex sub-tasks can be assigned to larger models or cloud APIs while simpler sub-tasks run on smaller local models, and the same retriever can be reused by other capabilities.
  • The cost of these patterns is real: extra datasets, multi-task training complexity, and more components to deploy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the paper's skeleton-then-leaves decomposition generalizes to any generative task whose output has a stable structure and variable, environment-dependent fields, such as database queries, API calls, or configuration files.
  • If the causal story holds, the same pattern pair should transfer to other low-code or code-generation settings where output must reference a large, changing environment, and it could be tested there without waiting for a second enterprise case study.
  • The deterministic Flow Similarity metric suggests a broader evaluation strategy for structured generative output that avoids the non-determinism of LLM-as-a-judge; extending it to other tree-structured artifacts would be a natural next step.
  • The security argument implies a testable design rule: an FM that never sees environment data in its weights cannot leak it, so deployments could be audited by checking that fine-tuning corpora contain no per-tenant data.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper formalizes two techniques, Task Decomposition and Retrieval-Augmented Generation (RAG), as reusable design patterns for generative-AI-based systems, discussing their impact on software quality attributes such as modularity, maintainability, safety, and security. The second contribution is an industrial case study of a workflow-generation application built at a large enterprise platform company, describing how these two patterns shaped data labeling, model training, evaluation, and deployment. The authors claim that applying Task Decomposition and RAG resulted in a modular, flexible, secure, and testable system, while acknowledging added complexity in model training.

Significance. If the claims are established, the paper would provide a useful bridge between AI techniques and software engineering practice, giving practitioners a structured vocabulary for reasoning about GenAI system design. The pattern formalization using the Gamma et al. template is a reasonable and potentially reusable contribution, and the case study contains honest self-reflection about limitations, including the self-reporting nature of the evidence. The proposed Flow Similarity metric for deterministic evaluation of structured outputs is an interesting idea, though no results are reported. The paper also explicitly connects RAG to security and safety concerns, which is a valuable direction. However, the central empirical claims rest on qualitative assertions and contain a load-bearing internal inconsistency regarding the security guarantee, so the current version overstates what is demonstrated.

major comments (3)
  1. [Section IV.C and Section III.B, Consequence 3] The claim in Section IV.C that 'the FM can only use data that the retriever suggests (integrity)' is not supported and is internally inconsistent with the paper's own pattern description. Section III.B, Consequence 3, states that 'there is no guarantee that... the FM will accept them during generation' and only suggests soft mitigations such as offering many suggestions and training the FM. The described training uses teacher forcing with a 'choices:' token but no constrained decoding, output allow-listing, or serve-time validation is described. Without such an enforcement mechanism, a fine-tuned 7B model can generate an artifact not present among the retrieved choices (e.g., a hallucinated table name or value from another environment), breaking the stated integrity guarantee. Likewise, the assertion that 'the FM does not contain any customer-specific knowledge in its model weights' would require a data-provenance or memorization-leakage audit; fine-tuning on thousands of internal workflows does not by itself rule out memorization. Since 'secure' is one of the four quality attributes in the paper's central conclusion, this section must either describe a concrete enforcement mechanism or substantially weaken the security claims.
  2. [Section IV.D and Section IV.C] The case study gives no quantitative evaluation results, despite invoking several metrics. Section IV.C asserts 'low rates of hallucination' and 'good retrieval quality' after fine-tuning, and Section IV.D introduces Flow Similarity and retrieval recall, but no numbers, confidence intervals, or error rates are reported for any of these measures. There is also no baseline comparison against generating the workflow in one pass or against prompting an off-the-shelf FM. Consequently, the reader cannot assess whether Task Decomposition and RAG actually improved functional correctness or whether the 'acceptable quality' thresholds were met. To substantiate the case-study claims, at least summary statistics for the three evaluation groups (outline, step inputs, full workflow) and retriever recall should be provided, or the paper should explicitly frame these outcomes as qualitative impressions rather than measured results.
  3. [Section V] The concluding claim that 'Task Decomposition and RAG resulted in a modular, flexible, secure, and testable system' is a causal attribution that a descriptive case study cannot support. The paper acknowledges its self-reporting nature in Section V, but it does not address the main rival explanations for the observed engineering benefits, such as the team's expertise, the fine-tuning effort, the small 7B model choice, or the favorable enterprise context. The case study is structured around two research questions, but RQ1 and RQ2 are answered only through the authors' narrative, with no comparison to alternative development trajectories. To make the pattern recommendations credible, the conclusions should be reframed as 'these techniques were compatible with the quality attributes in this context' or backed by a more rigorous research design that isolates the effect of the patterns.
minor comments (4)
  1. [Section IV.B and Section IV.C] Figures 4, 5, and 6 refer to colored text ('in red', 'in light green', 'in blue'), which is ineffective if the paper is read in grayscale or by color-blind readers; consider adding symbols or textual distinctions.
  2. [Section IV.A] The notation uses w for both the workflow and its individual steps (Sw = (w1, w2, ..., wn)); renaming the steps or the workflow would improve readability.
  3. [Section IV.D] The Flow Similarity metric is described only as a tree edit distance; the specific normalization or mapping from edit distance to a similarity score is not defined, making the metric non-reproducible.
  4. [Section III.B, Implementation] The implementation section notes that generation should continue even if retrieval fails, but does not discuss how the FM is trained or prompted to behave in that failure mode; an explicit mechanism or reference would strengthen the pattern description.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper's conclusions rest on a self-reported case study and cited literature, not on a derivation that reduces to its inputs.

full rationale

This paper is a descriptive design-pattern formalization plus an industry case study; it contains no fitted parameter, no predictive equation, and no formal derivation whose output could equal its input by construction. The pattern discussions in Section III are grounded in the cited literature, with the authors' prior RAG paper [22] cited only as one supporting reference among several independent ones. The case-study conclusions in Sections IV and V are explicitly self-reported, and the authors acknowledge this: 'our case study is limited by its self-reporting nature.' That is a validity limitation, not circularity. Similarly, the security/integrity claim in Section IV.C ('the FM can only use data that the retriever suggests') conflicts with the paper's own caveat in Section III.B that 'there is no guarantee that... the FM will accept them [retriever suggestions] during generation'; however, this is an internal-consistency/correctness problem, not a circular-reasoning problem, because the claim is not derived from the caveat. No equation, fitted quantity, or self-citation chain makes the central claim equivalent to its own evidence, so no circular step is identified.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The case study's qualitative conclusions rest on several domain assumptions about causal attribution, evaluation validity, and claimed security properties. No fitted numeric parameters appear in the paper.

assumptions (4)
  • ad hoc to paper The observed engineering benefits are caused by Task Decomposition and RAG, not by team expertise or context.
    Section V admits the case study is 'limited by its self-reporting nature' and no control group or baseline is provided.
  • domain assumption Tree edit distance on parsed workflows is a valid measure of workflow similarity.
    Section IV-D proposes Flow Similarity based on tree edit distance but gives no correlation with human judgment.
  • domain assumption The fine-tuned 7B FM and 100M retriever reach 'acceptable' and 'good' quality.
    Section IV-C asserts these results without reporting scores, intervals, or evaluation set sizes.
  • ad hoc to paper The RAG design guarantees that the FM contains no customer-specific knowledge in its weights.
    Section IV-C claims this security property, but it depends on data separation and permission filtering that are not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generating a Low-code Complete Workflow via Task Decomposition and RAG." pith.science (2026). https://pith.science/paper/EPLGSKR2

@misc{pith2026241200239,
  author       = {Pith},
  title        = {Pith review of: Generating a Low-code Complete Workflow via Task Decomposition and RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EPLGSKR2}},
  note         = {Machine review of arXiv:2412.00239}
}
read the original abstract

AI technologies are moving rapidly from research to production. With the popularity of Foundation Models (FMs) that generate text, images, and video, AI-based systems are increasing their complexity. Compared to traditional AI-based software, systems employing FMs, or GenAI-based systems, are more difficult to design due to their scale and versatility. This makes it necessary to document best practices, known as design patterns in software engineering, that can be used across GenAI applications. Our first contribution is to formalize two techniques, Task Decomposition and Retrieval-Augmented Generation (RAG), as design patterns for GenAI-based systems. We discuss their trade-offs in terms of software quality attributes and comment on alternative approaches. We recommend to AI practitioners to consider these techniques not only from a scientific perspective but also from the standpoint of desired engineering properties such as flexibility, maintainability, safety, and security. As a second contribution, we describe our industry experience applying Task Decomposition and RAG to build a complex real-world GenAI application for enterprise users: Workflow Generation. The task of generating workflows entails generating a specific plan using data from the system environment, taking as input a user requirement. As these two patterns affect the entire AI development cycle, we explain how they impacted the dataset creation, model training, model evaluation, and deployment phases.

Figures

Figures reproduced from arXiv: 2412.00239 by the authors.

Figure 1
Figure 1. Sample structures of Task Decomposition. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Sample structures of Retrieval-Augmented Generation. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Sample workflow mapping to the user requirement [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: shows the more complex labeling task for popu￾lateInputs. In this case, the labeler had to write a label for each step, as an annotation, which is normally text that is part of the requirement. This was more time-consuming because the labeler needed to understand the v…
Figure 4
Figure 4. Figure 4: Labeling createFlow samples involved writing the requirement (in red) given the basic workflow outline [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: Example of how RAG was used in the training process. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: System architecture with UI, AI, and data layers. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 26 canonical work pages

  1. [1]

    On the opportunities and risks of foundation models,

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, E. Brynjolfsson, S. Buch, D. Card, R. Castellon, N. S. Chatterji, A. S. Chen, K. A. Creel, J. Davis, D. Demszky, C. Donahue, M. Doumbouya, E. Durmus, S. Ermon, J. Etchemendy, K. Ethayarajh, L. Fei-Fei, C. Finn, T. Gale, L. E. Gillesp...

    work page

  2. [2]

    Towards uncovering how large language model works: An explainability perspective,

    H. Zhao, F. Yang, B. Shen, H. Lakkaraju, and M. Du, “Towards uncovering how large language model works: An explainability perspective,” 2024. [Online]. Available: https://arxiv.org/abs/2402.10688

    arXiv 2024

  3. [3]

    How does machine learning change software development practices?

    Z. Wan, X. Xia, D. Lo, and G. C. Murphy, “How does machine learning change software development practices?” IEEE Transactions on Software Engineering , vol. 47, no. 9, pp. 1857–1871, 2021

    work page 2021

  4. [4]

    Design patterns for ai-based systems: A multivocal literature review and pattern repository,

    L. Heiland, M. Hauser, and J. Bogner, “Design patterns for ai-based systems: A multivocal literature review and pattern repository,” in 2023 IEEE/ACM 2nd International Conference on AI Engineering – Software Engineering for AI (CAIN) , 2023, pp. 184–196

    work page 2023

  5. [5]

    Architectural design decisions for the machine learning workflow,

    S. J. Warnett and U. Zdun, “Architectural design decisions for the machine learning workflow,” Computer, vol. 55, no. 3, pp. 40–51, 2022

    work page 2022

  6. [6]

    A taxonomy of software engineering challenges for machine learning systems: An empirical investigation,

    L. E. Lwakatare, A. Raj, J. Bosch, H. H. Olsson, and I. Crnkovic, “A taxonomy of software engineering challenges for machine learning systems: An empirical investigation,” in Agile Processes in Software Engineering and Extreme Programming , P. Kruchten, S. Fraser, and F. Coallier, Eds. Cham: Springer International Publishing, 2019, pp. 227–243

    work page 2019

  7. [7]

    Software engineering for machine learning: A case study,

    S. Amershi, A. Begel, C. Bird, R. DeLine, H. Gall, E. Kamar, N. Na- gappan, B. Nushi, and T. Zimmermann, “Software engineering for machine learning: A case study,” in 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 2019, pp. 291–300

    2019

  8. [8]

    A survey on large language models for code generation,

    J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim, “A survey on large language models for code generation,” 2024. [Online]. Available: https://arxiv.org/abs/2406.00515

    arXiv 2024

Show all 47 references
  1. [9]

    Studying software engineering patterns for designing machine learning systems,

    H. Washizaki, H. Uchida, F. Khomh, and Y .-G. Gu ´eh´eneuc, “Studying software engineering patterns for designing machine learning systems,” in 2019 10th International Workshop on Empirical Software Engineering in Practice (IWESEP) , 2019, pp. 49–495

    2019

  2. [10]

    Software engineering for ai-based systems: A survey,

    S. Mart ´ınez-Fern´andez, J. Bogner, X. Franch, M. Oriol, J. Siebert, A. Trendowicz, A. M. V ollmer, and S. Wagner, “Software engineering for ai-based systems: A survey,” ACM Trans. Softw. Eng. Methodol. , vol. 31, no. 2, Apr. 2022. [Online]. Available: https://doi.org/10.1145...

    2022

  3. [11]

    Archi- tectural decisions in ai-based systems: An ontological view,

    X. Franch, S. Mart ´ınez-Fern´andez, C. P. Ayala, and C. G ´omez, “Archi- tectural decisions in ai-based systems: An ontological view,” in Quality of Information and Communications Technology, A. Vallecillo, J. Visser, and R. P ´erez-Castillo, Eds. Cham: Springer International...

    2022

  4. [12]

    Adapting Software Architectures to Machine Learning Challenges ,

    A. Serban and J. Visser, “ Adapting Software Architectures to Machine Learning Challenges ,” in 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) . Los Alamitos, CA, USA: IEEE Computer Society, Mar. 2022, pp. 152–

    2022

  5. [13]

    Architecture decisions in ai-based systems development: An empirical study,

    B. Zhang, T. Liu, P. Liang, C. Wang, M. Shahin, and J. Yu, “Architecture decisions in ai-based systems development: An empirical study,” in 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), 2023, pp. 616–626

    2023

  6. [14]

    Rethinking software engineering in the era of foundation models: A curated catalogue of challenges in the development of trustworthy fmware,

    A. E. Hassan, D. Lin, G. K. Rajbahadur, K. Gallaba, F. R. Cogo, B. Chen, H. Zhang, K. Thangarajah, G. Oliva, J. J. Lin, W. M. Abdullah, and Z. M. J. Jiang, “Rethinking software engineering in the era of foundation models: A curated catalogue of challenges in the development of...

    2024

  7. [16]

    Requirements and reference architecture for mlops:insights from industry,

    I. Kumara, R. Arts, D. D. Nucci, W. J. V . D. Heuvel, and D. A. Tamburri, “Requirements and reference architecture for mlops:insights from industry,” 2022

    2022

  8. [17]

    Iso/iec 25010:2023 systems and software engineering — systems and software quality requirements and evaluation (square) — product quality model,

    “Iso/iec 25010:2023 systems and software engineering — systems and software quality requirements and evaluation (square) — product quality model,” 2023”. [Online]. Available: https://www.iso.org/standard/78176. html

    2023

  9. [18]

    Speech- copilot: Leveraging large language models for speech processing via task decomposition, modularization, and program generation,

    C.-Y . Kuan, C.-K. Yang, W.-P. Huang, K.-H. Lu, and H. yi Lee, “Speech- copilot: Leveraging large language models for speech processing via task decomposition, modularization, and program generation,” 2024. [Online]. Available: https://arxiv.org/abs/2407.09886

    2024 arXiv

  10. [19]

    Film trailer generation via task decomposition,

    P. Papalampidi, F. Keller, and M. Lapata, “Film trailer generation via task decomposition,” 2021. [Online]. Available: https://arxiv.org/abs/ 2111.08774

    2021 arXiv

  11. [20]

    Learning abstract visual reasoning via task decomposition: A case study in raven progressive matrices,

    J. Kwiatkowski and K. Krawiec, “Learning abstract visual reasoning via task decomposition: A case study in raven progressive matrices,” Int. J. Appl. Math. Comput. Sci. , vol. 34, no. 2, p. 309–321, Jun. 2024. [Online]. Available: https://doi.org/10.61822/amcs-2024-0022

    2024 doi

  12. [21]

    Retrieval-augmented generation for large language models: A survey,

    Y . Gao, Y . Xiong, X. Gao, K. Jia, J. Pan, Y . Bi, Y . Dai, J. Sun, M. Wang, and H. Wang, “Retrieval-augmented generation for large language models: A survey,” 2024. [Online]. Available: https://arxiv.org/abs/2312.10997

    2024 arXiv

  13. [22]

    Reducing hallucination in structured outputs via retrieval-augmented generation,

    O. Ayala and P. Bechard, “Reducing hallucination in structured outputs via retrieval-augmented generation,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Tra...

    2024

  14. [23]

    Knowledgpt: Enhancing large language models with retrieval and storage access on knowledge bases,

    X. Wang, Q. Yang, Y . Qiu, J. Liang, Q. He, Z. Gu, Y . Xiao, and W. Wang, “Knowledgpt: Enhancing large language models with retrieval and storage access on knowledge bases,” 2023. [Online]. Available: https://arxiv.org/abs/2308.11761

    2023 arXiv

  15. [24]

    Search- in-the-chain: Interactively enhancing large language models with search for knowledge-intensive tasks,

    S. Xu, L. Pang, H. Shen, X. Cheng, and T.-S. Chua, “Search- in-the-chain: Interactively enhancing large language models with search for knowledge-intensive tasks,” 2024. [Online]. Available: https://arxiv.org/abs/2304.14732

    2024 arXiv

  16. [25]

    Guidelines for conducting and reporting case study research in software engineering,

    P. Runeson and M. H ¨ost, “Guidelines for conducting and reporting case study research in software engineering,” Empirical Softw. Engg., vol. 14, no. 2, p. 131–164, Apr. 2009. [Online]. Available: https://doi.org/10.1007/s10664-008-9102-8

    2009 doi

  17. [26]

    A taxon- omy of foundation model based systems through the lens of software architecture,

    Q. Lu, L. Zhu, X. Xu, Y . Liu, Z. Xing, and J. Whittle, “A taxon- omy of foundation model based systems through the lens of software architecture,” in 2024 IEEE/ACM 3rd International Conference on AI Engineering – Software Engineering for AI (CAIN) , 2024, pp. 1–6

    2024

  18. [27]

    Toward responsible ai in the era of generative ai: A reference architecture for designing foundation model-based systems,

    Q. Lu, L. Zhu, X. Xu, Z. Xing, and J. Whittle, “Toward responsible ai in the era of generative ai: A reference architecture for designing foundation model-based systems,” IEEE Softw. , vol. 41, no. 6, p. 91–100, Jun. 2024. [Online]. Available: https://doi.org/10.1109/MS. 2024.3406333

    2024

  19. [28]

    RAFT: Adapting language model to domain specific RAG,

    T. Zhang, S. G. Patil, N. Jain, S. Shen, M. Zaharia, I. Stoica, and J. E. Gonzalez, “RAFT: Adapting language model to domain specific RAG,” in First Conference on Language Modeling , 2024. [Online]. Available: https://openreview.net/forum?id=rzQGHXNReU

    2024

  20. [29]

    Low-code LLM: Graphical user interface over large language models,

    Y . Cai, S. Mao, W. Wu, Z. Wang, Y . Liang, T. Ge, C. Wu, W. WangYou, T. Song, Y . Xia, N. Duan, and F. Wei, “Low-code LLM: Graphical user interface over large language models,” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computat...

    2024

  21. [30]

    Gamma, R

    E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design patterns: elements of reusable object-oriented software . USA: Addison-Wesley Longman Publishing Co., Inc., 1995

    1995

  22. [31]

    Taskgen: A task-based, memory-infused agentic framework using strictjson,

    J. C. M. Tan, P. Saroj, B. Runwal, H. Maheshwari, B. L. Y . Sheng, R. Cottrill, A. Chona, A. Kumar, and M. Motani, “Taskgen: A task-based, memory-infused agentic framework using strictjson,” 2024. [Online]. Available: https://arxiv.org/abs/2407.15734

    2024 arXiv

  23. [32]

    A jailbroken genai model can cause substantial harm: Genai-powered applications are vulnerable to promptwares,

    S. Cohen, R. Bitton, and B. Nassi, “A jailbroken genai model can cause substantial harm: Genai-powered applications are vulnerable to promptwares,” 2024. [Online]. Available: https://arxiv.org/abs/2408. 05061

    2024

  24. [33]

    The dark side of function calling: Pathways to jailbreaking large language models,

    Z. Wu, H. Gao, J. He, and P. Wang, “The dark side of function calling: Pathways to jailbreaking large language models,” 2024. [Online]. Available: https://arxiv.org/abs/2407.17915

    2024 arXiv

  25. [34]

    Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval,

    S. E. Robertson and S. Walker, “Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval,” in Pro- ceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , ser. SIGIR ’94. Berl...

    1994

  26. [35]

    Seven failure points when engineering a retrieval augmented generation system,

    S. Barnett, S. Kurniawan, S. Thudumu, Z. Brannelly, and M. Abdelrazek, “Seven failure points when engineering a retrieval augmented generation system,” in Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI , ser. CAIN ’24. ...

    2024

  27. [36]

    Jmlr: Joint medical llm and retrieval training for enhancing reasoning and professional question answering capability,

    J. Wang, Z. Yang, Z. Yao, and H. Yu, “Jmlr: Joint medical llm and retrieval training for enhancing reasoning and professional question answering capability,” 2024. [Online]. Available: https: //arxiv.org/abs/2402.17887

    2024 arXiv

  28. [37]

    Toolformer: language models can teach themselves to use tools,

    T. Schick, J. Dwivedi-Yu, R. Dess ´ı, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom, “Toolformer: language models can teach themselves to use tools,” in Proceedings of the 37th International Conference on Neural Information Processing Systems, ...

    2024

  29. [38]

    Large dual encoders are generalizable retrievers,

    J. Ni, C. Qu, J. Lu, Z. Dai, G. Hernandez Abrego, J. Ma, V . Zhao, Y . Luan, K. Hall, M.-W. Chang, and Y . Yang, “Large dual encoders are generalizable retrievers,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , Y . Goldberg, Z. Koz...

    2022

  30. [39]

    Dimensionality reduction by learning an invariant mapping,

    R. Hadsell, S. Chopra, and Y . LeCun, “Dimensionality reduction by learning an invariant mapping,” in 2006 IEEE Computer Society Con- ference on Computer Vision and Pattern Recognition (CVPR’06), vol. 2, 2006, pp. 1735–1742

    2006

  31. [40]

    SimCSE: Simple contrastive learning of sentence embeddings,

    T. Gao, X. Yao, and D. Chen, “SimCSE: Simple contrastive learning of sentence embeddings,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, Eds. Online and Punta Cana, Dominican Repu...

    2021

  32. [41]

    A learning algorithm for continually running fully recurrent neural networks,

    R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,” Neural Computation , vol. 1, no. 2, pp. 270–280, 1989

    1989

  33. [42]

    The challenges of evaluating llm applications: An analysis of automated, human, and llm-based approaches,

    B. Abeysinghe and R. Circi, “The challenges of evaluating llm applications: An analysis of automated, human, and llm-based approaches,” 2024. [Online]. Available: https://arxiv.org/abs/2406.03339

    2024 arXiv

  34. [43]

    G-eval: NLG evaluation using gpt-4 with better human alignment,

    Y . Liu, D. Iter, Y . Xu, S. Wang, R. Xu, and C. Zhu, “G-eval: NLG evaluation using gpt-4 with better human alignment,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for ...

    2023

  35. [44]

    Simple fast algorithms for the editing distance between trees and related problems,

    K. Zhang and D. Shasha, “Simple fast algorithms for the editing distance between trees and related problems,” SIAM Journal on Computing, vol. 18, no. 6, pp. 1245–1262, 1989. [Online]. Available: https://doi.org/10.1137/0218082

    1989 doi

  36. [45]

    Quantifying the capabilities of llms across scale and precision,

    S. Badshah and H. Sajjad, “Quantifying the capabilities of llms across scale and precision,” 2024. [Online]. Available: https://arxiv.org/abs/ 2405.03146

    2024 arXiv

  37. [46]

    Data labeling: An empirical investigation into industrial challenges and mit- igation strategies,

    T. Fredriksson, D. I. Mattos, J. Bosch, and H. H. Olsson, “Data labeling: An empirical investigation into industrial challenges and mit- igation strategies,” in Product-Focused Software Process Improvement , M. Morisio, M. Torchiano, and A. Jedlitschka, Eds. Cham: Springer Int...

    2020

  38. [163]

    Available: https://doi.ieeecomputersociety.org/10.1109/ SANER53432.2022.00029

    [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/ SANER53432.2022.00029

    2022

  39. [2021]

    Available: https://crfm.stanford.edu/assets/report.pdf

    [Online]. Available: https://crfm.stanford.edu/assets/report.pdf

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.