From Hugging Face to GitHub: Tracing License Drift in the Open-Source AI Ecosystem

Ahmed E. Hassan; Bram Adams; Gopi Krishnan Rajbahadur; Hao Li; James Jewitt

REVIEW 4 major objections 4 minor 1 cited by

From Hugging Face to GitHub: Tracing License Drift in the Open-Source AI Ecosystem

T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read More than a third of model-to-application transitions violate the upstream model's license, according to the first end-to-end audit of the open-source AI supply chain.

desk verdict Solid first end-to-end license lineage dataset, but the headline 35.5% violation rate is an artifact of the authors' hand-built compatibility matrix, not a measured ecosystem fact. read the letter →

arxiv 2509.09873 v2 pith:YZHQRTXG submitted 2025-09-11 cs.SE cs.AI

James Jewitt , Hao Li , Bram Adams , Gopi Krishnan Rajbahadur , Ahmed E. Hassan This is my paper · ORCID

classification cs.SEcs.AI

keywords licensedriftopen-sourceAIHuggingFacecompatibilitysupplychaincomplianceRecempiricalaudit

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that license obligations are systematically stripped away as AI artifacts move from datasets to models to software applications, and that at the final integration step 35.5% of the traced model-to-application links violate the upstream model's license. It supports this with the first end-to-end audit spanning 364,917 datasets, 1,627,519 models, and 136,375 GitHub repositories. The authors argue this is not random error but a predictable 'gravitational pull' toward permissive licensing, with a single pattern (ML-license to permissive) accounting for 84.9% of model-to-application violations. They also build LicenseRec, a rule engine encoding nearly 200 SPDX and ML-specific clauses, which they say can resolve 86.4% of detected conflicts. If the paper is right, AI supply chains carry a large, correctable, but partly inherent licensing risk.

What carries the argument

The central mechanism is LicenseRec, a rule engine that groups hundreds of license strings into categories (Permissive, Copyleft, Share-Alike, Non-Commercial, ML License, etc.) and applies a compatibility matrix built from the OSADL matrix, Creative Commons rules, and manually encoded ML-specific clauses. Compatibility is defined through the Permission/Duty/Prohibition model: a conflict occurs when one license imposes a duty that another prohibits. LicenseRec also recommends replacement licenses by intersecting the compatibility sets of upstream licenses, which is how it computes 'fixability' and reports 86.4% resolvable conflicts at the model-to-application stage.

What would settle it

A court ruling or authoritative legal analysis holding that obligations in ML-specific model licenses do not bind downstream application developers in the way the matrix assumes would invalidate the 35.5% headline; more directly, re-running the audit with an independently reviewed compatibility matrix that omits the manual ML clauses would reproduce the paper's own comparison numbers (3.2% or 1.1% violation rates) and show the estimate is not robust.

Watch

Extended reading notes

Core claim

The paper's central discovery is that license drift in open-source AI follows a consistent direction: restrictive obligations are progressively erased as artifacts move downstream. On the model-to-application link, 35.5% of transitions are flagged as violating the upstream model's license, overwhelmingly because a model under an ML-specific license is integrated into an application under a permissive license. The authors trace the full lineage from dataset to model to repository, showing that non-commercial, share-alike, and ML-specific obligations have retention rates below 7% at the final stage, while permissive licenses remain stable. They attribute the pattern to developers treating mode

Load-bearing premise

The entire violation count rests on the hand-built license compatibility matrix — particularly the manually added rules for ML-specific license clauses — being a correct statement of legal compatibility; if those rules are wrong, the 35.5% figure changes, and the paper itself shows standard matrices produce 1.1-3.2% instead.

Editorial extensions

If this is right

Automated license checkers can catch a majority of the violations: LicenseRec resolves 86.4% of model-to-application conflicts and 78.0% of dataset-to-model conflicts.
A small set of high-risk patterns dominates the risk: ML-license-to-permissive alone accounts for 84.9% of model-to-application violations, so fixing this one pattern would address most non-compliance.
Some conflicts are unfixable by re-licensing — 14.2% of dataset-to-model violations are inherited upstream, meaning the downstream developer must change the upstream model, not just the license.
Traditional compatibility matrices undercount AI-specific violations: the EU matrix flags only 3.2% of model-to-application links, versus 35.5% with the ML-aware matrix, so generic licensing tools may mislead developers.
Permissive-to-permissive flows are the most stable, while every restrictive category except copyleft collapses at the final integration step, implying a systemic normalization of permissive licensing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The headline 35.5% rate is a direct consequence of the authors' manually extended compatibility matrix; a legally conservative or different interpretation could yield a rate closer to the 1-3% seen with standard matrices, so the number should be read as a modeling result, not a legal fact.
Since Hugging Face license tags are self-reported and many models carry no license at all, the true exposure could be larger than measured; the dataset released with the paper enables re-analysis as legal interpretations evolve.
The same drift mechanism likely applies to proprietary API-based AI services: the authors note 81.5% of such repositories adopt permissive licenses, and the terms-of-service obligations of those services may be similarly discarded.
A testable extension is to run the LicenseRec pipeline on a version-controlled history to see whether violations are actively introduced at integration time or baked in at model creation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new thing here is the end-to-end license lineage: datasets on Hugging Face, the models trained on them, and the GitHub repos that use those models, all linked at a scale nobody has done before. 364k datasets, 1.6M models, 136k repos, with AST-based filtering to confirm actual model use. That is a real step past PeaTMOSS and Stalnaker et al., who each covered only one link. LicenseRec itself is also useful as a prototype: it encodes ML-specific clauses that ordinary matrices ignore, and the comparison against Jiang et al. and the EU matrix is an honest way to show the difference.

The methodology is mostly transparent. They describe the collection pipeline, the signature set is in the replication package, and they openly report the sensitivity of their violation rates to different compatibility matrices. That last point is where the trouble starts.

The central claim, 35.5% of model-to-repository transitions violate the upstream license, is not a finding about the ecosystem. It is a consequence of their own compatibility matrix. The paper's Table 4 shows the same data gives 1.1% with the Jiang et al. matrix and 3.2% with the EU matrix. The difference is almost entirely the ML license row: 84.9% of all model-to-repo violations are ML→Permissive, and that classification rests on the unvalidated legal position that ML licenses like OpenRAIL or the Llama licenses impose copyleft-like obligations on the downstream application's own code. Many of those licenses impose use restrictions and attribution duties but do not require the application's code to be published under the same license. The authors acknowledge license compatibility is interpretive, but they still lead the abstract with 35.5% as if it were a measured property of the supply chain. That is a load-bearing soft spot, not a minor caveat.

The fixability rate is also self-referential: 86.4% is computed by LicenseRec applying its own rules to conflicts it defined. It is a useful engineering metric but not an independent validation. There is also no ground-truth validation of the license extraction pipeline, and the paper mentions a replication package without giving a working link in the text.

None of this sinks the contribution. The dataset and the lineage graph are valuable regardless of the legal matrix one plugs in. The authors are honest enough to show the sensitivity, which is more than most papers do. But they then over-interpret their own matrix as the truth. A serious referee should ask for legal validation or at least a clear statement that the headline rate is one plausible interpretation, not an audit result.

I would bring this to a reading group to discuss the tension between computational tractability and legal interpretation. I would cite the dataset in my own work, but only with the sensitivity caveat.

Decision: peer review it. The flaws are fixable, the data is real, and the question matters.

Referee Report

4 major / 4 minor

Summary. The paper presents an end-to-end audit of license propagation from Hugging Face datasets (364,917) and models (1,627,519) to downstream GitHub repositories (136,375), tracing license category transitions and quantifying violations with a rule engine called LicenseRec. The authors report that 35.5% of model-to-application transitions violate the upstream model's license, mostly because ML-specific restrictions are replaced by permissive licenses, and claim LicenseRec can resolve 86.4% of these conflicts. They release the dataset and prototype. The central empirical claim is the 35.5% violation rate and the associated claim that this reveals systemic non-compliance in the AI supply chain.

Significance. If the headline results held, this would be a valuable first large-scale, end-to-end measure of license drift in the AI supply chain, with a reusable dataset and a prototype tool that the community could build on. The authors also make a useful methodological contribution by extending traditional compatibility matrices with ML-specific clauses, and by comparing their matrix against two existing ones. However, the significance is currently limited because the headline violation rate is not an externally validated legal fact: it is a direct consequence of the authors' hand-built compatibility matrix, and the paper itself shows that alternative matrices reduce the model-to-repository violation rate to 1.1% or 3.2%. The fixability metric is computed self-referentially with the same rule engine. These issues need to be addressed before the paper's central claims can be accepted as evidence about the ecosystem rather than about a particular formal model.

major comments (4)

[Table 4 and Results] The headline claim that 35.5% of model-to-repository transitions violate the upstream license is not robust. Table 4 shows the same transitions yield violation rates of 1.1% with the Jiang et al. matrix and 3.2% with the EU matrix. Table 3 shows that 84.9% of the M→R violations are ML→Permissive transitions, which depend entirely on the authors' manually added rule that ML licenses impose a copyleft-like duty on downstream applications. This rule is legally contested and no legal expert validation or license-text audit is provided. The paper should either obtain independent legal validation of the ML-specific matrix entries, or clearly present the 35.5% as the outcome of a specific formal model, not as a measured property of the ecosystem. As written, the abstract and introduction overstate the finding.
[LicenseRec / Effectiveness of LicenseRec] The claim that LicenseRec 'solves' 86.4% of license conflicts is circular in a way that is not adequately disclosed. LicenseRec defines conflicts using its own compatibility matrix, and then measures fixability by asking whether LicenseRec can recommend a license under that same matrix. There is no independent legal or manual ground truth. This is a reasonable internal consistency check, but it does not establish that 86.4% of real-world license conflicts are solvable. The language in the abstract and Results should be softened to 'within LicenseRec's formal model' unless an external evaluation is added.
[Methodology / Data Collection] The license detection pipeline is not validated against ground truth. Repository licenses are taken from ScanCode outputs, and dataset/model licenses from Hugging Face tags, but there is no manual audit, no precision/recall, and no error analysis. Since the entire violation analysis depends on correct license categorization, misdetection rates could materially change the results. A sample-based manual validation of at least the ML license category and the repository license detection should be reported, especially because the ML category is the driver of the headline finding.
[Results / Fixability interpretation] The distinction between 'fixable' and 'unresolvable' conflicts is presented as a practical guide for developers, but the 'unresolvable' set is also an artifact of the matrix. For example, NC→P transitions are marked unresolvable because the model itself is non-commercial; this is a reasonable conclusion, but it depends on the matrix treating NC as a hard prohibition. The paper should explicitly state which results are robust across compatibility matrices and which are matrix-specific. Currently, only the overall violation rates in Table 4 are compared across matrices; the fixability rates are not subjected to the same sensitivity analysis.

minor comments (4)

[Results] Typo: 'and and Share-Alike' in the paragraph after Figure 3. Also, the figure label 'T ool' is misspelled in Figure 4.
[LicenseRec / Table 5] The category definitions in Table 5 are high-level and sometimes combine licenses with quite different obligations (e.g., Apache-2.0 and MIT are both 'Permissive' despite different patent grant provisions). The paper should acknowledge that the category-level analysis may obscure within-category incompatibilities, and that the reported violation rates are category-level, not license-level.
[Methodology] The popularity threshold of 20 likes is arbitrary and restricts the traceable set to 9,112 of 1.6M models. Some discussion of how this selection affects generalizability would be useful, even if the threshold is motivated by API availability.
[Abstract] The abstract says the tool 'can solve 86.4% of license conflicts'; this wording overstates the result given the self-referential evaluation. Please rephrase to make the formal-model dependence explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical rates are computed under an explicitly disclosed compatibility model; matrix-dependence is a validity concern, not a self-referential derivation.

full rationale

The paper's derivation chain is: (1) collect real dependency data from Hugging Face and GitHub; (2) categorize licenses using a matrix built from OSADL, Creative Commons, and the authors' manual ML-license analysis; (3) compute violation rates and fixability by applying that matrix. The 35.5% model-to-repository violation rate is a conditional result of this matrix, and the paper transparently compares against the Jiang et al. (1.1%) and EU (3.2%) matrices, showing that the rate is sensitive to the chosen legal interpretation. This is an acknowledged limitation and a matter of external validity, not circularity: the matrix is an input, not an output of the empirical analysis, and the transition frequencies come from independent data. Similarly, the 86.4% fixability metric is defined as the fraction of conflicts for which LicenseRec's own recommendation function returns a license; this is a self-consistency/coverage measure for the tool, not a claim that the tool is validated by external legal ground truth. The paper explicitly states 'Our goal is not to offer a definitive legal interpretation but to adopt a formal, replicable model' and 'LicenseRec's accuracy is fundamentally dependent on the quality of the input data.' These statements show the authors are not conflating their model with an external legal fact. No load-bearing self-citation chain, no fitted parameter renamed as a prediction, and no definition that smuggles in the conclusion are present. The weaknesses are about the correctness of the ML-specific compatibility assumptions and the lack of legal validation, which belong to correctness risk rather than circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The key free parameters are the popularity threshold and the hand-built compatibility rules, which are the main drivers of the reported violation rates. The axioms are all assumptions about data completeness and about the validity of the formal license model.

free parameters (3)

Popularity threshold (likes) = 20
Chosen by hand to select popular models for downstream tracing. Changes the set of models and therefore the link sample.
Maximum recommendations per SPDX category = 5
Configurable cap in LicenseRec's ranking step, chosen by design.
Code signature set size = 11,000
Manually curated list of API call signatures used to filter active model usage; not derived from data.

assumptions (5)

domain assumption License compatibility can be modeled as direct logical contradiction between Permission, Duty, and Prohibition statuses
Adopted from Moreau et al. (2019). The paper acknowledges this differs from risk-averse legal practice, which considers cumulative complexity.
domain assumption Hugging Face license tags accurately represent the license of each dataset and model
Used to build the license lineage. The authors admit in the Discussion that wrong tags would invalidate the analysis.
domain assumption ScanCode accurately detects all licenses in GitHub repositories
Used for downstream application licenses. No validation sample is reported.
domain assumption GitHub Code Search plus AST signatures covers the true set of repositories using each model
Only Python files and text-match code search are used, so non-Python or non-matching usages are missed.
ad hoc to paper The manually extended compatibility matrix (OSADL + CC + ML clauses) correctly encodes legal compatibility
The ML-specific clauses are hand-written. The paper shows the violation rate varies from 1.1% to 35.5% depending on the matrix, so the headline result is tied to this matrix.

how reviews work

0 comments

Cite this review

Pith. "Pith review of From Hugging Face to GitHub: Tracing License Drift in the Open-Source AI Ecosystem." pith.science (2026). https://pith.science/paper/YZHQRTXG

@misc{pith2026250909873,
  author       = {Pith},
  title        = {Pith review of: From Hugging Face to GitHub: Tracing License Drift in the Open-Source AI Ecosystem},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YZHQRTXG}},
  note         = {Machine review of arXiv:2509.09873}
}

read the original abstract

Hidden license conflicts in the open-source AI ecosystem pose serious legal and ethical risks, exposing organizations to potential litigation and users to undisclosed risk. However, the field lacks a data-driven understanding of how frequently these conflicts occur, where they originate, and which communities are most affected. We present the first end-to-end audit of licenses for datasets and models on Hugging Face, as well as their downstream integration into open-source software applications, covering 364 thousand datasets, 1.6 million models, and 140 thousand GitHub projects. Our empirical analysis reveals systemic non-compliance in which 35.5% of model-to-application transitions eliminate restrictive license clauses by relicensing under permissive terms. In addition, we prototype an extensible rule engine that encodes almost 200 SPDX and model-specific clauses for detecting license conflicts, which can solve 86.4% of license conflicts in software applications. To support future research, we release our dataset and the prototype engine. Our study highlights license compliance as a critical governance challenge in open-source AI and provides both the data and tools necessary to enable automated, AI-aware compliance at scale.

Figures

Figures reproduced from arXiv: 2509.09873 by the authors.

**Figure 2.** An overview of our LicenseRec engine. links, primarily using the tags field. Since dataset identifiers in this field are often incomplete (e.g., missing an organization name), we normalize them by searching Hugging Face to resolve each dataset’s fully qualified identifier, ensuring consistent dataset → model links. To trace model usage in downstream applications, we select a subset of 9,112 popular models from Hug… view at source ↗

**Figure 3.** License Category Transitions: A Sankey diagram illustrating the flow of license categories from datasets to models [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Summary of license violation analysis. Top: [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

When Model Release Meets Model Reuse: Producer-Consumer Misalignment in Hugging Face
cs.SE 2026-07 conditional novelty 7.0 of 10

Producers and consumers of pre-trained models rely on the same documentation but systematically disagree on where metadata belongs, why lineage is traced, and which governance mechanisms help.

Reference graph

Works this paper leans on

16 extracted references · 2 linked inside Pith · cited by 1 Pith paper

[1]

Bartz, A.; Graeber, C.; and Johnson, K. W. 2024. Bartz v. Anthropic PBC. U.S. District Court for the Northern District of California. No. 3:24-cv-05417 (N.D. Cal. filed Aug. 19, 2024)

2024
[2]

Creative Commons . 2025. Compatible Licenses . https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses/. Accessed: April 15, 2025

2025
[3]

European Commission . 2025. Licensing Assistant

2025
[4]

Free Software Foundation . 2025. Various Licenses and Comments about Them . https://www.gnu.org/licenses/license-list.html. Accessed: April 15, 2025

2025
[5]

GmbH, T. R. E. C.; and Corporation, W. P. 2020. Thomson Reuters Enterprise Centre GmbH v. ROSS Intelligence Inc. U.S. District Court for the District of Delaware. No. 1:20-cv-00613 (D. Del. filed May 5, 2020)

2020
[6]

K.; and Davis, J

Jiang, W.; Yasmin, J.; Jones, J.; Synovic, N.; Kuo, J.; Bielanski, N.; Tian, Y.; Thiruvathukal, G. K.; and Davis, J. C. 2024. PeaTMOSS : A Dataset and Initial Analysis of Pre-Trained Models in Open-Source Software. In Proceedings of the 21st International Conference on Mining Software Repositories (MSR '24), 431--443. New York, NY, USA: Association for Co...

2024
[7]

R.; Chaturvedula, S

Kadasi, P.; Kondam, S. R.; Chaturvedula, S. V.; Sen, R.; Saha, A.; Sikdar, S.; Sarkar, S.; Mittal, S.; Jindal, R.; and Singh, M. 2025. Model Hubs and Beyond: Analyzing Model Popularity, Performance, and Documentation. arXiv:2503.15222

arXiv 2025
[8]

Moran, L. 2020. ROSS Intelligence will shut down amid lawsuit from Thomson Reuters. ABA Journal. Legal AI startup announces closure due to financial strain from copyright litigation

2020

Show all 16 references

[9]

Moreau, B.; Serrano-Alvarado, P.; Perrin, M.; and Desmontils, E. 2019. Modelling the Compatibility of Licenses. In The Semantic Web -- 16th International Conference, ESWC 2019, volume 11503 of Lecture Notes in Computer Science, 255--269. Cham: Springer

2019
[10]

Open Source Automation Development Lab (OSADL) . 2025. Compatibility Matrix of FOSS Licenses . https://www.osadl.org/html/CompatMatrix.html. Accessed: April 15, 2025

2025
[11]

Stalnaker, T.; Chen, R.; Longpre, S.; and Webson, A. 2025. The ML Supply Chain in the Era of Software 2.0: Lessons Learned from Hugging Face . arXiv:2502.04484

2025
[12]

The Law Doesn't Work Like a Computer

Wintersgill, N.; Stalnaker, T.; Heymann, L. A.; Chaparro, O.; and Poshyvanyk, D. 2024. "The Law Doesn't Work Like a Computer": Exploring Software Licensing Issues Faced by Legal Practitioners. arXiv:2403.14927

2024 arXiv
[13]

Zacchiroli, S. 2022. A large-scale dataset of (open source) license text variants. In Proceedings of the 19th International Conference on Mining Software Repositories, MSR ’22, 757–761. ACM

2022
[14]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
[15]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
[16]

write newline

" write newline "" initialize.prev.this.status FUNCTION begin.bib " write newline preamble empty 'skip preamble write newline if " thebibliography " longest.label * " " * write newline " [1] #1 " write newline " url@samestyle " write newline " " write newline " [2] #2 " write ...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.

[1] [1]

Bartz, A.; Graeber, C.; and Johnson, K. W. 2024. Bartz v. Anthropic PBC. U.S. District Court for the Northern District of California. No. 3:24-cv-05417 (N.D. Cal. filed Aug. 19, 2024)

2024

[2] [2]

Creative Commons . 2025. Compatible Licenses . https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses/. Accessed: April 15, 2025

2025

[3] [3]

European Commission . 2025. Licensing Assistant

2025

[4] [4]

Free Software Foundation . 2025. Various Licenses and Comments about Them . https://www.gnu.org/licenses/license-list.html. Accessed: April 15, 2025

2025

[5] [5]

GmbH, T. R. E. C.; and Corporation, W. P. 2020. Thomson Reuters Enterprise Centre GmbH v. ROSS Intelligence Inc. U.S. District Court for the District of Delaware. No. 1:20-cv-00613 (D. Del. filed May 5, 2020)

2020

[6] [6]

K.; and Davis, J

Jiang, W.; Yasmin, J.; Jones, J.; Synovic, N.; Kuo, J.; Bielanski, N.; Tian, Y.; Thiruvathukal, G. K.; and Davis, J. C. 2024. PeaTMOSS : A Dataset and Initial Analysis of Pre-Trained Models in Open-Source Software. In Proceedings of the 21st International Conference on Mining Software Repositories (MSR '24), 431--443. New York, NY, USA: Association for Co...

2024

[7] [7]

R.; Chaturvedula, S

Kadasi, P.; Kondam, S. R.; Chaturvedula, S. V.; Sen, R.; Saha, A.; Sikdar, S.; Sarkar, S.; Mittal, S.; Jindal, R.; and Singh, M. 2025. Model Hubs and Beyond: Analyzing Model Popularity, Performance, and Documentation. arXiv:2503.15222

arXiv 2025

[8] [8]

Moran, L. 2020. ROSS Intelligence will shut down amid lawsuit from Thomson Reuters. ABA Journal. Legal AI startup announces closure due to financial strain from copyright litigation

2020