Pith. sign in

REVIEW 3 major objections 5 minor 25 references

LLMShot: Reducing snapshot testing maintenance via LLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A vision-language model can classify snapshot test failure root causes, with 84.21% recall on a 17-case dataset, while prompting it to ignore known changes fails.

desk verdict A useful proof-of-concept for VLM-based snapshot triage, but the headline numbers are in-sample estimates: same 17 cases used for prompt design and evaluation. read the letter →

arxiv 2507.10062 v2 pith:AGQXREUA submitted 2025-07-14 cs.SE

Ergün Batuhan Kaynak , Mayasah Lami , Sahand Moslemi , Anil Koyuncu This is my paper · ORCID
classification cs.SE
keywords snapshottestingvision-languagemodelsUItestmaintenancefailuretriagesemanticclassificationGemma3iOSautomatedanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LLMShot claims that a vision-language model shown the reference image, the failing image, and the pixel-level diff can classify the root cause of a snapshot test failure into semantic categories such as color, padding, layout, text, or animation change, and can produce a natural-language explanation of what changed. The paper reports that the 12B Gemma3 model achieves 84.21% recall and 66.67% precision on its 17-case dataset built from a SwiftUI app with feature flags, while the 4B model trails slightly, and presents this as the first automated semantic analysis of snapshot test failures. It also finds that prompting the model to ignore a known category makes classification collapse, with compliance around 31–35%, so selective filtering remains unsolved. If the result generalizes, developers could automatically triage snapshot failures and spend manual effort only on genuine regressions.

What carries the argument

The load-bearing mechanism is a structured three-image JSON prompt: it feeds the reference snapshot, the failing snapshot, and the pixel diff to the VLM along with the category list and definitions, and requires a JSON response with a list of root-cause labels (including a dynamically invented UNKNOWN-<reason> fallback), numeric pixel and semantic difference scores, affected UI element names, and a short explanation. This prompt, combined with the taxonomy of change types in Table I, is what converts raw pixel differences into semantic classifications, and it defines the evaluation metrics (hit rate, recall, precision, unknown rate, and pixel-difference error) that the paper reports.

What would settle it

Take the LLMShot prompt unchanged and run it on a held-out set of snapshot failures gathered from several production Android, web, or iOS apps, with ground-truth labels assigned by developers who were not involved in designing the prompt; if recall on that set falls well below 84% or the model's unknown rate rises sharply, the generalization claim would be falsified.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that semantic triage of snapshot test failures is achievable by prompting a VLM with the before image, the after image, and the difference image simultaneously, asking it to label the change according to an eight-category taxonomy and to return structured JSON. The 12B model reaches 84.21% recall and 66.67% precision, a hit rate of 82.35% (at least one correct label per failing test), an unknown rate of 5.88%, and a mean pixel-difference estimation error of 0.059. The 4B model reaches 78.95% recall and 57.69% precision. The same experiments show that appending an ignore instruction for an acceptable category degrades the hit rate to 18.75% (IFA) or 41.18% (IFGT) and leaves only about a third of tests compliant with the instruction, which the paper reads as evidence that prompt-based controllable visual reasoning is not yet reliable.

Load-bearing premise

The evaluation rests on the assumption that the authors' 17-case dataset, generated from a single SwiftUI app with feature flags and labeled by the authors on the same cases used to iteratively refine the prompt, is representative enough of real snapshot-testing failures that the reported recall and precision will carry over to other applications and teams.

Editorial extensions

If this is right

  • CI/CD pipelines could automatically attach a suggested root cause, affected elements, and explanation to every failing snapshot test, so a developer only opens cases that the model flags as genuine regressions.
  • The structured JSON output is machine-readable, so failure analyses could be routed into issue trackers or dashboards without manual transcription.
  • The 4B model's smaller footprint gives a plausible low-cost deployment for continuous integration, accepting lower precision to avoid large GPU requirements.
  • The UNKNOWN-<reason> mechanism means the classifier can signal a previously unseen change type rather than forcing a label, which is important as UI change patterns evolve.
  • Because the prompt-based ignore mechanism fails, the paper's own results imply that useful selective filtering will require fine-tuning or another approach beyond instruction-following.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 17-case, single-app dataset likely under-represents the long-tail distribution of real UI changes; a multi-app, multi-platform benchmark would probably lower the measured precision and reveal taxonomy gaps, so the reported numbers should be read as an upper bound on current VLM capability.
  • Pairing LLMShot's semantic labels with deterministic pixel-diff thresholds could offset its precision weakness, since the model's probability of adding a spurious label may correlate with small visual changes that a threshold rule can suppress.
  • The ignore-category failure suggests a concrete next experiment: fine-tune a VLM on paired before/after images with explicit 'ignore X' instructions and measure whether compliance and F1 recover, which would isolate whether the bottleneck is prompting or the models' control mechanism.
  • Because the dataset is generated by toggling feature flags in one app, a natural extension is to mine real project histories for failing snapshot pairs and rerun the same evaluation, giving an out-of-distribution test of both the framework and the taxonomy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces LLMShot, a prompt-based framework that uses Gemma3 vision-language models (4B and 12B variants) to classify snapshot test failures into a predefined semantic taxonomy (e.g., COLOR_CHANGE, PADDING_CHANGE) and to output structured JSON containing pixel and semantic difference estimates, affected UI elements, and natural-language explanations. The evaluation is performed on a new publicly available dataset of 17 failures generated from a purpose-built SwiftUI iOS application with runtime feature flags, with ground-truth labels assigned by two of the authors. The paper reports that Gemma3-12B achieves 84.21% recall and 66.67% precision (Finding 1), and it reports a negative result for a prompt-based ignore mechanism that shows severe performance degradation (Finding 2). The authors claim that LLMShot is the first automated approach to semantic snapshot test analysis.

Significance. If the reported classification performance transfers beyond the authors' 17-case dataset, LLMShot would be a useful step toward automated triage of snapshot test failures, and the released dataset with dynamic artifacts (baseline, failure, diff images, metadata, labels) is a valuable community resource. The paper's strengths include the public dataset, the structured JSON output that is amenable to CI/CD integration, the use of open-weight models with a fixed low temperature, and the honest reporting of a negative result for the ignore mechanism. However, the central quantitative claim is currently supported only by an in-sample evaluation: the prompt and taxonomy were iteratively designed on the same 17 cases used to compute the headline metrics, with no held-out split, no frozen-prompt evaluation, and no baseline comparison. The significance of the paper therefore depends on whether the authors can provide evidence of generalization beyond the exact dataset used to develop the method.

major comments (3)
  1. [Section V-A, Table II; Section III-A; Section IV-A] The headline performance (84.21% recall, 66.67% precision for Gemma3-12B) is computed on the same 17 cases that were used to iteratively design the prompt (Section III-A) and to construct and label the dataset (Section IV-A). This is an in-sample estimate: it measures how well the prompt fits the 17 known cases, not how well it classifies unseen snapshot failures. The paper does not describe freezing the prompt before evaluation, a held-out split, leave-one-out evaluation, or an independent dataset. This is load-bearing because the abstract, Finding 1, and the conclusion all present 'strong classification performance' as a general capability. I ask the authors to either add an evaluation protocol that separates prompt development from testing (e.g., freeze the prompt after design and evaluate on a held-out set, or use leave-one-out cross-validation with a fixed prompt) or substantially soften the general claim to a proof-of-concept result.
  2. [Section VII, Comparison to Baselines; Section V-A] The claim that LLMShot achieves 'strong' classification performance is not supported by any baseline comparison. With 17 cases and a skewed category distribution (COLOR_CHANGE appears in 5 of 17 tests, ANIMATION_CHANGE in 1), a trivial heuristic such as always predicting the most frequent category, or predicting all categories, could achieve nontrivial recall or hit rate. Without a baseline (e.g., a majority-class predictor, a keyword-based image-diff heuristic, or a prior visual-oracle tool such as DiffDroid), the reported recall and precision values are not interpretable as evidence that LLMShot exceeds a simple alternative. This concern is partly acknowledged in Section VII, but the abstract and conclusion still state the performance as 'strong' without qualification.
  3. [Section IV-A, Ground Truth Labeling; Section VII] The ground truth is labeled by two of the authors, who also built the application, designed the taxonomy, and iteratively tuned the prompt. The paper does not report inter-rater agreement, and the labeling process is described as resolving disagreements through joint discussion, which can mask systematic labeling biases. With only 17 cases from a single purpose-built iOS application, the external validity threat acknowledged in Section VII is not merely a minor limitation; it directly affects whether the quantitative findings can be generalized to real snapshot testing workflows. I recommend reporting inter-rater agreement statistics and, if possible, augmenting the dataset with cases from an independently developed application or from real project histories.
minor comments (5)
  1. [Table II and Table III headings] The table headings read 'PERFORMANCE OF LLMS HOT' and 'PERFORMANCE OF LLMS HOT GEMMA 3:4B'; these appear to be typos for 'LLMShot' and should be corrected.
  2. [Table II, Semantic Diff row] The Semantic Diff value for Gemma3-4B is reported as 0.921 ± 0.046, while Gemma3-12B is 0.079 ± 0.079. Given the text states that the 12B model provides 'more accurate pixel difference estimation,' the 4B semantic difference of 0.921 seems inconsistent with a near-identical 5.3% mean pixel difference and may be a typo (perhaps 0.092). Please verify and correct this value.
  3. [Table III, IFA column] The IFA Hit Rate is reported as 18.75%, but the baseline and IFGT columns use 17 tests as the denominator; it is unclear whether IFA is computed on 16 cases because one case was excluded. Please clarify the denominator and the exact procedure for the 'Ignore From Analysis' strategy.
  4. [Section III-A, Listing 1] The prompt template uses placeholders such as {reference}, {failure}, and {difference}, but the paper does not specify how the diff image is computed (e.g., pixel-level absolute difference, highlighted diff, or a tool-generated image). For reproducibility, please describe the diff-image generation process.
  5. [Section IV-B] The paper states that a temperature of 0.1 was used but does not report the number of runs per test or any variance across runs. Since the authors acknowledge LLM randomness in Section VII, reporting the number of independent runs and the stability of the classification metrics would strengthen reproducibility.

Circularity Check

2 steps flagged · score 6.0 of 10

LLMShot's headline recall/precision is computed on the same 17 self-authored, self-labeled cases used to iteratively tune the prompt, with no held-out split or baseline; the central performance claim is an in-sample estimate.

  1. fitted input called prediction [Section III-A (Methodology, 'LLMShot') and Section V-A (Table II, Finding 1)]
    "The core of the framework is a structured prompt, shown in Listing 1, which was developed through an iterative design process."

    The prompt is the component that makes LLMShot work, and it was iteratively edited using knowledge of the same 17-case dataset whose labels define the ground truth. Section V-A then reports Finding 1 (84.21% recall / 66.67% precision for Gemma3-12B) on exactly those cases, with no reported frozen-prompt holdout, leave-one-out split, or independent dataset. The reported 'prediction' is therefore an in-sample estimate of how well the iteratively fitted prompt reproduces the authors' labels, not a measured ability to classify unseen snapshot-test failures.

  2. other [Section III-C ('Evaluation Dataset'), Section IV-A ('Ground Truth Labeling'), and Section V-A ('Finding 1')]
    "We generate this dataset from a purpose-built iOS application, using its integrated feature flags to produce authentic UI changes that mimic common development scenarios. Each resulting failure instance is then manually classified by two authors and labeled according to our detailed taxonomy of UI changes (Table I)."

    The same authors built the app, selected the feature flags, defined the taxonomy, and labeled the 17 failures, and then used those same cases both to iteratively shape the prompt (Section III-A) and to compute Table II, which is cited as Finding 1. The evaluation is therefore self-referential: the ground truth labels and the prompt design come from the same small closed set. The paper's own Section VII concedes the single-application scope and the absence of a head-to-head baseline, but the abstract and conclusion nevertheless present the resulting numbers as evidence of 'strong classification performance' for the framework.

full rationale

The circularity is not definitional in the strongest sense, because the Gemma3 VLM weights are pretrained and frozen and the model could in principle have failed even on the tuning cases. However, the paper's central quantitative claim is a fitted-input-called-prediction pattern: the prompt was iteratively developed on the exact 17-case dataset, the taxonomy and labels were authored by the same team, and no independent validation split or external benchmark is used. Section VII itself acknowledges the single-app limitation and the lack of comparison with prior tools, which confirms that the reported 84.21% recall / 66.67% precision is an in-sample measure of prompt fit rather than an out-of-sample prediction. No load-bearing self-citation chain or imported uniqueness theorem is present; the supporting citations (Gemma3, Ollama, swift-snapshot-testing) are external and non-circular. Because the headline performance claim, which is the paper's main evidence for LLMShot's contribution, reduces to an evaluation on the data used to tune the prompt, the score is a partial-circularity 6 rather than a lower score.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No numeric parameters were fitted to data. The load-bearing design choices are categorical: the nine-category taxonomy, the author-assigned ground truth, the pixel-difference ground truth metric, and the single-app dataset. The prompt itself was iteratively tuned on the evaluation set, which is a form of categorical overfitting and is captured in the circularity score.

free parameters (1)
  • temperature = 0.1
    Gemma3 inference temperature via Ollama, chosen by hand to reduce randomness; standard hyperparameter, not fitted to the dataset.
assumptions (4)
  • domain assumption The taxonomy in Table I is a complete and meaningful set of root-cause categories for snapshot test failures.
    Every precision/recall calculation depends on the taxonomy being the correct target; the authors acknowledge it may not cover all possible changes and add an UNKNOWN catch-all (Section III-A, Table I).
  • domain assumption The two-author manual labels are accurate ground truth for the 17 test instances.
    Labels were assigned by the authors themselves with a joint discussion, without external expert validation (Section IV-A, Section VII Construct Validity).
  • domain assumption The pixel-wise difference metric (Eq. 1) is an appropriate ground-truth measure of visual change.
    Used to evaluate LLMShot's quantitative estimates; it is a simple mean absolute pixel difference and may not match perceptual similarity (Section IV-A).
  • domain assumption Performance on a single custom iOS app predicts performance on other apps and platforms.
    The authors list this as an external validity threat: only one SwiftUI iOS app is used (Section VII).

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLMShot: Reducing snapshot testing maintenance via LLMs." pith.science (2026). https://pith.science/paper/AGQXREUA

@misc{pith2026250710062,
  author       = {Pith},
  title        = {Pith review of: LLMShot: Reducing snapshot testing maintenance via LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AGQXREUA}},
  note         = {Machine review of arXiv:2507.10062}
}
read the original abstract

Snapshot testing has emerged as a critical technique for UI validation in modern software development, yet it suffers from substantial maintenance overhead due to frequent UI changes causing test failures that require manual inspection to distinguish between genuine regressions and intentional design changes. This manual triage process becomes increasingly burdensome as applications evolve, creating a need for automated analysis solutions. This paper introduces LLMShot, a novel framework that leverages Vision-Language Models (VLMs) to automatically analyze snapshot test failures through semantic classification of UI changes. To evaluate LLMShot's effectiveness, we developed a comprehensive dataset using a feature-rich iOS application with configurable feature flags, creating realistic scenarios that produce authentic snapshot differences representative of real development workflows. Our evaluation using Gemma3 models demonstrates strong classification performance, with the 12B variant achieving over 84% recall in identifying failure root causes while the 4B model offers practical deployment advantages with acceptable performance for continuous integration environments. However, our exploration of selective ignore mechanisms revealed significant limitations in current prompting-based approaches for controllable visual reasoning. LLMShot represents the first automated approach to semantic snapshot test analysis, offering developers structured insights that can substantially reduce manual triage effort and advance toward more intelligent UI testing paradigms.

Figures

Figures reproduced from arXiv: 2507.10062 by the authors.

Figure 1
Figure 1. Two Example Diffs for the PADDING_CHANGE Test Failure Class implementation, allowing developers to write more tests and consequently prevent regressions. Despite its benefits, snapshot testing faces significant main￾tenance challenges. Like other GUI testing techniques [5]– [7], current approaches frequently fail to comprehend se￾mantic information [8], [9]. Snapshot tests focus solely on identifying visual differen… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 21 canonical work pages

  1. [1]

    Applying human values theory to software engineering practice: Lessons and implications,

    M. A. Ferrario and E. Winter, “Applying human values theory to software engineering practice: Lessons and implications,” IEEE Trans- actions on Software Engineering , vol. 49, no. 3, pp. 973–990, 2022

    work page 2022

  2. [2]

    An empirical study on the use of snapshot testing,

    S. Fujita, Y . Kashiwa, B. Lin, and H. Iida, “An empirical study on the use of snapshot testing,” in Proceedings of the IEEE International Conference on Software Maintenance and Evolution , ser. ICSME ’23. IEEE, 2023, pp. 335–340

    work page 2023

  3. [3]

    Snapshot testing in practice: Benefits and drawbacks,

    V . P. G. Cruz, H. Rocha, and M. T. Valente, “Snapshot testing in practice: Benefits and drawbacks,” Journal of Systems and Software , vol. 204, p. 111797, 2023

    work page 2023

  4. [4]

    Meta open source is transferring jest to the openjs foundation,

    R. Hanlon, “Meta open source is transferring jest to the openjs foundation,” May 2022. [Online]. Available: https://engineering.fb.com/ 2022/05/11/open-source/jest-openjs-foundation/

    work page 2022

  5. [5]

    Sapienz: multi-objective automated testing for android applications,

    K. Mao, M. Harman, and Y . Jia, “Sapienz: multi-objective automated testing for android applications,” in Proceedings of the 25th International Symposium on Software Testing and Analysis , ser. ISSTA ’16. ACM, 2016, p. 94–105. [Online]. Available: https: //doi.org/10.1145/2931037.2931054

    arXiv 2016

  6. [6]

    Large-scale analysis of framework-specific exceptions in android apps,

    L. Fan, T. Su, S. Chen, G. Meng, Y . Liu, L. Xu, G. Pu, and Z. Su, “Large-scale analysis of framework-specific exceptions in android apps,” in Proceedings of the 40th International Conference on Software Engi- neering, ser. ICSE ’18. IEEE/ACM, 2018, pp. 408–419

    work page 2018

  7. [7]

    Combodroid: Generating high-quality test inputs for android apps via use case combinations,

    J. Wang, Y . Jiang, C. Xu, C. Cao, X. Ma, and J. Lu, “Combodroid: Generating high-quality test inputs for android apps via use case combinations,” in Proceedings of the 42nd International Conference on Software Engineering, ser. ICSE ’20. IEEE/ACM, 2020, pp. 469–480

    work page 2020

  8. [8]

    Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware Decisions

    Z. Liu, C. Chen, J. Wang, M. Chen, B. Wu, X. Che, D. Wang, and Q. Wang, “Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions,” 2023. [Online]. Available: https://arxiv.org/abs/2310.15780

    work page Pith review arXiv 2023

Show all 25 references
  1. [9]

    Benchmarking automated gui testing for android against real-world bugs,

    T. Su, J. Wang, and Z. Su, “Benchmarking automated gui testing for android against real-world bugs,” in Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE ’21. ACM, 2021,...

    2021

  2. [10]

    Automated cross-platform inconsistency detection for mobile apps,

    M. Fazzini and A. Orso, “Automated cross-platform inconsistency detection for mobile apps,” in Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering , ser. ASE ’17. IEEE, 2017, p. 308–318

    2017

  3. [11]

    Owl eyes: spotting ui display issues via visual understanding,

    Z. Liu, C. Chen, J. Wang, Y . Huang, J. Hu, and Q. Wang, “Owl eyes: spotting ui display issues via visual understanding,” in Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE ’20. ACM, 2021, p. 398–409

    2021

  4. [12]

    Glib: towards automated test oracle for graphically-rich applications,

    K. Chen, Y . Li, Y . Chen, C. Fan, Z. Hu, and W. Yang, “Glib: towards automated test oracle for graphically-rich applications,” in Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, se...

    2021

  5. [13]

    Reducing combinatorics in gui testing of android applications,

    N. Mirzaei, J. Garcia, H. Bagheri, A. Sadeghi, and S. Malek, “Reducing combinatorics in gui testing of android applications,” in Proceedings of the 38th International Conference on Software Engineering , ser. ICSE ’16. IEEE/ACM, 2016, pp. 559–570

    2016

  6. [14]

    Translating video recordings of mobile app usages into replayable scenarios,

    C. Bernal-Cárdenas, N. Cooper, K. Moran, O. Chaparro, A. Marcus, and D. Poshyvanyk, “Translating video recordings of mobile app usages into replayable scenarios,” in Proceedings of the ACM/IEEE 42nd Interna- tional Conference on Software Engineering , ser. ICSE ’20. ACM, Jun. ...

    2020

  7. [15]

    Read it, don’t watch it: Captioning bug recordings automatically,

    S. Feng, M. Xie, Y . Xue, and C. Chen, “Read it, don’t watch it: Captioning bug recordings automatically,” 2023. [Online]. Available: https://arxiv.org/abs/2302.00886

    2023 arXiv

  8. [16]

    Automatic web testing using curiosity-driven reinforcement learning,

    Y . Zheng, Y . Liu, X. Xie, Y . Liu, L. Ma, J. Hao, and Y . Liu, “Automatic web testing using curiosity-driven reinforcement learning,”

  9. [17]

    Qexplore: An exploration strategy for dynamic web applications using guided search,

    S. Sherin, A. Muqeet, M. U. Khan, and M. Z. Iqbal, “Qexplore: An exploration strategy for dynamic web applications using guided search,” Journal of Systems and Software , vol. 195, p. 111512, 2023

    2023

  10. [18]

    Snapshot testing dataset,

    E. Bui and H. Rocha, “Snapshot testing dataset,” in Proceedings of the 20th International Conference on Mining Software Repositories , ser. MSR ’23. IEEE, 2023, pp. 558–562

    2023

  11. [19]

    The metamorphosis: Automatic detection of scaling issues for mobile apps,

    Y . Su, C. Chen, J. Wang, Z. Liu, D. Wang, S. Li, and Q. Wang, “The metamorphosis: Automatic detection of scaling issues for mobile apps,” in Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering , ser. ASE ’22. ACM, Oct. 2022, p. 1–12

    2022

  12. [20]

    Nighthawk: Fully automated localizing ui display issues via visual understanding,

    Z. Liu, C. Chen, J. Wang, Y . Huang, J. Hu, and Q. Wang, “Nighthawk: Fully automated localizing ui display issues via visual understanding,” IEEE Transactions on Software Engineering, vol. 49, no. 1, pp. 403–418, 2022

    2022

  13. [21]

    Seeing is believing: Vision-driven non-crash functional bug detection for mobile apps,

    Z. Liu, C. Li, C. Chen, J. Wang, M. Chen, B. Wu, Y . Wang, J. Hu, and Q. Wang, “Seeing is believing: Vision-driven non-crash functional bug detection for mobile apps,” 2024. [Online]. Available: https://arxiv.org/abs/2407.03037

    2024 arXiv

  14. [22]

    Pointfreeco/swift-snapshot-testing: delightful swift snapshot testing

    Pointfreeco, “Pointfreeco/swift-snapshot-testing: delightful swift snapshot testing.” [Online]. Available: https://github.com/pointfreeco/ swift-snapshot-testing

  15. [23]

    Gemma 3 technical report,

    G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Rivière et al. , “Gemma 3 technical report,” arXiv preprint arXiv:2503.19786 , 2025

    2025 arXiv

  16. [24]

    Ollama: A lightweight, extensible framework for building and running language models on the local machine,

    Ollama Inc., “Ollama: A lightweight, extensible framework for building and running language models on the local machine,” 2025. [Online]. Available: https://github.com/ollama/ollama

    2025

  17. [2021]

    Available: https://arxiv.org/abs/2103.06018

    [Online]. Available: https://arxiv.org/abs/2103.06018

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.