Pith. sign in

REVIEW 5 major objections 4 minor 97 references

A composite labeling function built on LLM-extracted concepts turns noisy patent citations into silver-standard SDG labels that beat existing baselines.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A weak-supervision pipeline using LLM-extracted concepts and rank fusion creates silver-standard patent-to-SDG labels that recover known citation-derived associations and show high network modularity.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A plausible weak-supervision pipeline for patent-SDG labels, but the evidence that the new labels are trustworthy rests on a p-dependent, unquantified modularity test, and the internal recall is coupled to the calibration signal. the 5 major comments →

arxiv 2509.09303 v1 pith:EFIRRSZ4 submitted 2025-09-11 cs.CL

From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models

Grazia Sveva Ascione , Nicol\`o Tamagnone This is my paper
classification cs.CL
keywords weak supervisionlabeling functionsSDG classificationpatent classificationlarge language modelssemantic similaritynetwork modularitynon-patent literature
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a weak-supervision labeling function can generate trustworthy patent-to-SDG labels without manual annotation or a gold standard. The function uses a large language model to extract 'functions, solutions, applications' from both patents and SDG-tagged scientific papers, measures semantic similarity with embeddings, fuses rankings with Reciprocal Rank Fusion, and calibrates its thresholds with a positive-only loss that does not penalize discovering new SDG associations. The resulting soft 17-dimensional relevance vectors recover known NPL-derived associations better than BM25, fine-tuned transformers, and zero-shot LLMs, and they create more coherent communities in citation, inventor, and applicant networks than CPC or NPL labels. If right, this gives a scalable way to build training data for SDG classification over the global patent corpus.

Core claim

The central claim is that the composite labeling function λ(p) produces silver-standard soft SDG labels that are high enough quality to train downstream multi-label regression models. Concretely, the silver labels achieve the highest recall against the NPL-derived reference (macro 0.711, micro 0.902) and the highest overlapping modularity across all three patent networks (citation 1.05, inventor 0.98, applicant 0.68), exceeding both CPC and NPL baselines. The method also expands coverage: 150,946 SDG labels across 32,895 patents versus 43,956 NPL labels, raising the average number of SDGs per patent from 1.34 to 4.59. The authors interpret this as evidence that semantically grounded labels c

What carries the argument

The composite labeling function λ(p) is defined by three stages. First, an LLM (gpt-4.1-mini) is prompted to extract functions, solutions, and applications from each patent and SDG-tagged paper. Second, per-category cosine similarities are computed with learned thresholds (τ_func, τ_sol, τ_app), and the filtered rankings are fused via Reciprocal Rank Fusion: RRF(d) = Σ_i 1/(k + rank_i(d)). The top-N aligned documents then yield a normalized 17-dimensional SDG frequency vector. Third, a positive-only MSE loss (Eq. 3) tunes {τ_i} and N against known NPL citations, so that new SDG assignments are not penalized. This loss is central because it operationalizes the weak-supervision assumption of l

Load-bearing premise

The load-bearing premise is that the query-based SDG tags on the scientific corpus are correct; if those tags are wrong or systematically biased, every silver label, the recall evaluation, and the modularity validation inherit that error.

What would settle it

Take a random sample of, say, 500 patents from the silver dataset, have human experts independently annotate SDG relevance, and measure agreement (e.g., rank correlation) with the silver vectors. If agreement is near chance, the labels are not trustworthy. Alternatively, permute the SDG tags of the retrieved scientific papers and rerun the pipeline: if the modularity advantage persists under permutation, the signal comes from something other than the tags.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The released silver-standard dataset (32,895 patents, 150,946 SDG labels) can directly serve as training data for multi-label SDG classifiers without manual annotation.
  • A patent's SDG profile can be predicted from title and abstract alone; the paper shows a multi-output regression model with PAT-SPECTER embeddings reaches MSE 0.0187, indicating the silver signal is learnable.
  • Because the positive-only calibration does not penalize new SDG matches, the method can label patents with no scientific citations at all and still produce coherent assignments.
  • The higher modularity of silver labels in citation, inventor, and applicant networks indicates that SDG alignment captures thematic and organizational structure that CPC technology classes miss.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same weak-supervision recipe could be transferred to other noisy bibliometric links (e.g., patent-to-patent citations, grant acknowledgements) to generate soft labels for any policy-relevant taxonomy, not just SDGs.
  • The SDG queries used to tag the scientific corpus bias toward English-language journal articles, so the silver dataset likely underrepresents SDG-relevant innovation from non-English or non-journal literature; a multilingual extension would test that bias.
  • The modularity advantage at low power-parameter values suggests that low-probability secondary SDG assignments carry real structural signal; a human-subject study could check whether experts also find those weak labels plausible.
  • Sensitivity to the fixed ontology (functions/solutions/applications) could be probed by swapping in an alternative patent ontology and re-running the pipeline to see whether recall and modularity degrade.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes a weak-supervision pipeline for patent-to-SDG classification. It starts from citations that patents make to SDG-tagged scientific publications (NPL citations) and builds a composite labeling function λ(p) that: (i) uses an LLM to extract functions, solutions, and applications from patent and paper texts; (ii) computes category-wise cosine similarities and fuses them with Reciprocal Rank Fusion; and (iii) converts the top-N retrieved papers' SDG tags into a soft 17-dimensional relevance vector (Eq. 2). Hyperparameters are tuned on 40% of the data using a positive-only MSE loss (Eq. 3) against the NPL-derived SDG distribution. The remaining 60% is used to produce a silver-standard dataset of 32,895 patents, which is then used to train multi-output regression models. The authors validate the silver labels internally by recall against NPL-derived SDG labels (Table 4) and externally by overlapping modularity in citation, co-inventor, and co-applicant networks (Table 7, Figure 3). They report that the silver labels outperform baselines in both evaluations and are learnable in the downstream regression setting.

Significance. If the validation concerns are resolved, this would be a useful contribution: it addresses a real annotation bottleneck, proposes a scalable and interpretable LF, and brings network homophily as an external check on labels. The paper is well situated in the weak-supervision and patent-SDG literatures, and it includes a clear ablation study. The authors are candid about several limitations in Section 6. However, the central claim that the silver labels are 'trustworthy' is not established by the current evidence: the reported internal score does not correspond to the final dataset, the internal evaluation is calibrated against the same NPL signal it measures, and the external modularity result is p-dependent and lacks a density-matched null. The paper does not state plans to release the dataset or code, which further limits reproducibility for a claimed public training resource.

major comments (5)
  1. [§4.1, Table 4 and ablation] The row labeled 'Silver' in Table 4 reports the full three-concept LF (macro recall 0.711, micro recall 0.902). However, the ablation in the same section shows that 'Solution & Function' achieves higher recall (0.725/0.910), and the text explicitly states 'we remove this dimension in the final construction of the silver dataset.' Thus the headline internal recall is not for the labeling function used to create the released/trained-on dataset. Please report the actual final LF's numbers and re-run the external validation on that final version.
  2. [§3.2.1, Eq. (3); §4.1] The LF hyperparameters {τ_func, τ_sol, τ_app, top-N} are optimized on 40% of the data to match the NPL-derived SDG distribution, and Table 4 then measures recall against the same NPL-derived SDG labels. Even if the evaluation uses the held-out 60%, the metric and the calibration objective are the same noisy signal, and the top-N=30 used for all baselines is itself the optimized LF hyperparameter. This does not demonstrate that newly added SDG labels are valid; it shows that the method can recover the calibration signal. The paper acknowledges the need for external validation at the start of §4.2, but the abstract and conclusion should not rest on Table 4 as evidence of trustworthiness.
  3. [§4.2, Table 7, Figure 3] Table 7 reports overlapping modularity only at p=0.35, but Figure 3 shows that the ordering is p-dependent: near p≈0.8–1.0, NPL and the transformer baselines match or outperform Silver, and Silver's advantage appears only for low p. This is exactly the regime where weak membership weights are amplified, and the Silver labels are much denser than the NPL labels (4.59 vs 1.34 labels per patent, §5.1). Without a density-matched permutation null that preserves per-node label counts and SDG prevalences, and without a p-sweep with significance estimates, the comparison (e.g., 1.05 vs 1.00 on citations) is confounded by label density/entropy. The paper's own text says that near p=1 'npl and the baselines outperform silver,' so the external-validity conclusion is not established.
  4. [§3.1, Eq. (2), Appendix A.1] The entire pipeline treats the Elsevier SDG query tags as ground truth: the silver label is the frequency of these tags among top-N retrieved papers, the internal recall is measured against citations to these tags, and the external modularity uses them as membership weights. There is no independent check of the SDG tags themselves, and the query-splitting procedure in Appendix A.1 is a possible source of subtle errors. Please provide a sample-based validation against expert annotation or an alternative SDG mapping (e.g., manually labeled papers or a different SDG classifier), and report sensitivity to the query-splitting implementation. Without this, a systematic bias in the Elsevier replication propagates into every label and both validations.
  5. [§3.1, Table 3] The dataset description is internally inconsistent. The text reports 55,008 patents citing 21,517 of the top-cited papers and then says 36,942 unique papers are mapped to at least one SDG; Table 3 reports 41,736 SDG-labeled publications and 73,266 patents and calls them unique, while the table's totals are in fact association counts. Since the silver-set construction and the 40/60 split depend on this base population, please reconcile the numbers and clearly distinguish unique patents/publications from patent-goal and paper-goal counts.
minor comments (4)
  1. [§4.1, Table 4 note] Please specify the binarization rule used to convert soft SDG vectors into the predicted labels counted in recall. It is unclear whether a nonzero score in the top-30 output is counted as a prediction, and whether this rule is identical for the LF and all baselines.
  2. [§4.2, Table 7] State how the NPL and CPC membership vectors for modularity are constructed. If NPL is binarized while Silver is soft, or if CPC membership uses a different normalization, the comparison at a fixed p is not apples-to-apples.
  3. [§5.2, Table 8] The learnability evaluation would benefit from a trivial baseline, such as a constant predictor or an untrained embedding-plus-linear model. The MSE values alone do not show that the silver signal is learnable beyond the model's capacity to memorize labels.
  4. [Throughout] Typos and minor wording issues: 'avergage' (Section 5.1), 'diagolals' (Figure 5 caption), 'SGG9' (Section 3.1), 'indirected network' (Section 4.2), 'The point is corroborated Narin et al.' (Section 2.2, missing 'by'). A careful proofread is needed.

Circularity Check

0 steps flagged

No significant circularity: the internal evaluation uses a held-out split and the external network modularity test is independent of label construction.

full rationale

The derivation chain is not circular. Silver labels are produced by Eq. (2) as the normalized SDG frequency among the top-N retrieved SDG-tagged papers; this is a retrieval output, not a fitted parameter. The only fitted quantities are the three similarity thresholds and the top-N cutoff, which are tuned on a 40% stratified validation split by minimizing the positive-only loss in Eq. (3) against NPL-derived SDG distributions. The internal evaluation (Table 4) is run on the held-out 60%, as stated in the abstract: 'internal validation against held-out NPL-based labels.' The recall figures therefore measure generalization to patents whose NPL labels were not used in calibration; this is a standard model-selection split, not a fitted input renamed as a prediction. Eq. (2) and Eq. (3) are not equal by construction: one is the predicted SDG distribution from retrieved documents, the other is the target SDG distribution from cited papers, and they are connected only through the optimization objective. The external modularity evaluation is also not circular: the three networks (citations, co-inventors, co-applicants) are exogenous to the label-generation pipeline, and the labels are not derived from the network structure. Concerns about the choice p=0.35, label-density confounds, and the absence of a permutation null are threats to the strength of the external conclusion, but they are correctness/validity concerns rather than circularity. The only self-citation (Ascione 2023) appears in the literature review and is not load-bearing for any central claim. Limitations are explicitly acknowledged in Section 6 (e.g., selection bias, heuristic homophily proxies, LLM sensitivity), and the paper does not hide its reliance on the Elsevier SDG query replication. Overall, the central claims are supported by a held-out internal benchmark and an independent, if imperfect, network-based validation.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 0 invented entities

The central contribution (silver SDG labels) rests on the external SDG tags for publications, the citation-based positive anchors, the ontology categories, and the embedding similarity. The four fitted thresholds and the chosen p parameter are the main ad hoc degrees of freedom; none have independent evidence outside the paper's own evaluation.

free parameters (7)
  • Semantic similarity threshold for functions (τ_function) = 0.260
    Optimized on the 40% validation split via Optuna to maximize alignment with NPL-derived SDG distributions (Sec 3.2.1).
  • Semantic similarity threshold for solutions (τ_solution) = 0.165
    Optimized on the validation split via Optuna (Sec 3.2.1).
  • Semantic similarity threshold for applications (τ_application) = 0.873
    Optimized on the validation split via Optuna (Sec 3.2.1).
  • Top-N cutoff for retrieved SDG documents = 30
    Optimized on the validation split; defines how many SDG papers contribute to each patent's SDG frequency vector (Sec 3.2.1).
  • Modularity power parameter p = 0.35
    Hand-chosen in the overlapping modularity computation to emphasize weak overlaps; the paper shows that silver labels overtake baselines specifically in the low-p regime (Sec 4.2, Figure 3).
  • RRF dampening constant k
    Appears in Eq 1 but its value is not reported in the paper; affects the fused ranking and thus the SDG vectors.
  • Top-cited paper cutoff per SDG = 20,000
    Each SDG contributes the 20,000 most-cited papers to the citation pool; this data filter shapes the Patent-SDG Dataset and is a design choice, not fitted.
axioms (6)
  • domain assumption Elsevier SDG Mapping Initiative queries correctly identify scientific publications relevant to each SDG, and the authors' segmentation of these queries preserves logical equivalence.
    All paper-side labels come from re-implementing these queries via the Scopus API (Sec 3.1); any tagging error propagates to the silver labels and all validations.
  • domain assumption NPL citations from patents to SDG-tagged papers are informative but noisy and incomplete anchors for SDG relevance.
    The weak supervision setup uses these citations as positive-only targets for calibration (Eq 3) and as the reference for internal evaluation (Sec 4.1).
  • ad hoc to paper The functions-solutions-applications ontology of Zhai et al. (2022) provides a sufficient shared semantic space for aligning patents and papers.
    The LF is built around these three concept categories without an independent test that they capture all relevant patent-paper semantic overlap.
  • domain assumption Cosine similarity between PAECTER embeddings of the LLM-extracted concepts captures semantic relatedness between patent and paper concepts.
    The retrieval ranking and all derived SDG vectors rely on this embedding similarity (Sec 3.2.1).
  • domain assumption Overlapping modularity (Nicosia et al.) with a power affinity function is a valid proxy for label quality in the absence of ground truth.
    The external validation (Sec 4.2) equates higher modularity of SDG labels across citation, inventor, and applicant edges with higher label quality.
  • ad hoc to paper The positive-only MSE loss (Eq 3) implements the intended property of not penalizing discovery of new SDG associations.
    The paper claims this property, but because the predicted vector is normalized over the top-N retrieved papers (Eq 2), assigning mass to a new SDG necessarily reduces the predicted probability of cited SDGs and increases the squared error on those positives.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models." pith.science (2026). https://pith.science/paper/EFIRRSZ4

@misc{pith2026250909303,
  author       = {Pith},
  title        = {Pith review of: From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EFIRRSZ4}},
  note         = {Machine review of arXiv:2509.09303}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Classifying patents by their relevance to the UN Sustainable Development Goals (SDGs) is crucial for tracking how innovation addresses global challenges. However, the absence of a large, labeled dataset limits the use of supervised learning. Existing methods, such as keyword searches, transfer learning, and citation-based heuristics, lack scalability and generalizability. This paper frames patent-to-SDG classification as a weak supervision problem, using citations from patents to SDG-tagged scientific publications (NPL citations) as a noisy initial signal. To address its sparsity and noise, we develop a composite labeling function (LF) that uses large language models (LLMs) to extract structured concepts, namely functions, solutions, and applications, from patents and SDG papers based on a patent ontology. Cross-domain similarity scores are computed and combined using a rank-based retrieval approach. The LF is calibrated via a custom positive-only loss that aligns with known NPL-SDG links without penalizing discovery of new SDG associations. The result is a silver-standard, soft multi-label dataset mapping patents to SDGs, enabling the training of effective multi-label regression models. We validate our approach through two complementary strategies: (1) internal validation against held-out NPL-based labels, where our method outperforms several baselines including transformer-based models, and zero-shot LLM; and (2) external validation using network modularity in patent citation, co-inventor, and co-applicant graphs, where our labels reveal greater thematic, cognitive, and organizational coherence than traditional technological classifications. These results show that weak supervision and semantic alignment can enhance SDG classification at scale.

Figures

Figures reproduced from arXiv: 2509.09303 by Grazia Sveva Ascione, Nicol\`o Tamagnone.

Figure 1
Figure 1. Figure 1: Data integration pipeline for constructing the [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Labeling function Notes: The Figure illustrates the labeling function pipeline. Patent and SDG documents from the Patent-SDG Dataset are processed using LLMs to extract ontological concepts (functions, solutions, applications). Semantic similarities are computed between patent and SDG concepts, and low-similarity pairs (below τsem) are filtered out. Concepts are ranked and fused via Reciprocal Rank Fusion … view at source ↗
Figure 3
Figure 3. Figure 3: Overlapping modularity Q(p) (directed Nicosia; F(a, b) = (a p )(b p )) across labeling methods. Panels: (a) citation graph, (b) inventor graph, (c) applicant graph. By analyzing modularity across these three networks, we are able to evaluate the extent to which labeling schemes align with different forms of structural homophily: thematic proximity (citations), cogni￾tive proximity (inventors), and organiza… view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of SDG label distributions between the NPL signal (red) and the silver signal [PITH_FULL_IMAGE:figures/full_fig_p026_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Row-wise normalized SDG co-occurrence matrices for the NPL signal (left) and the silver [PITH_FULL_IMAGE:figures/full_fig_p028_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Correlation between CPC3 classes and SDG targets using two SDG labeling signals. [PITH_FULL_IMAGE:figures/full_fig_p028_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: SDG publications trend (2015–2022) Notes: The Figure displays the number of SDG-tagged papers published annually between 2005 and 2022 by the Elsevier project. If a paper is classified under multiple SDGs, it is counted once per SDG. Only English-language journal articles are included. 35 [PITH_FULL_IMAGE:figures/full_fig_p035_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

97 extracted references · 24 linked inside Pith

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    https://www.wipo.int/edocs/pubdocs/en/wipo-pub-rn2024-18-en-mapping-innovations.pdf

    author WIPO , title Mapping Innovations: Patents and the Sustainable Development Goals , type Report number WIPO-PUB-RN2024-18 , World Intellectual Property Organization, year 2024 . https://www.wipo.int/edocs/pubdocs/en/wipo-pub-rn2024-18-en-mapping-innovations.pdf

    2024

  4. [4]

    author F. Mazzi , title Patents, social welfare, and sustainability: considerations on the role of the patent system in incentivising the un sustainable development goals , journal A Research Agenda for Patent Law’, Edward Elgar, Forthcoming ( year 2024 )

    2024

  5. [5]

    author J. W. Van der Waal , author T. Thijssens , author K. Maas , title The innovative contribution of multinational enterprises to the sustainable development goals , journal Journal of Cleaner Production volume 285 ( year 2021 ) pages 125319

    2021

  6. [6]

    Hajikhani , author A

    author A. Hajikhani , author A. Suominen , title Mapping the sustainable development goals (sdgs) in science, technology and innovation: application of machine learning in sdg-oriented artefact detection , journal Scientometrics volume 127 ( year 2022 ) pages 6661--6693

    2022

  7. [7]

    Maehara , author Y

    author Y. Maehara , author Y. Shiozawa , author Y. Osabe , title Multi-stage fine-tuning of patent domain-specific deberta for advanced patent landscape on sdgs/decarbonization , journal World Patent Information volume 81 ( year 2025 ) pages 102343

    2025

  8. [8]

    Ciarli , author A

    author T. Ciarli , author A. AlDoh , author S. Arora , author V. Arza , author J. Asinsten , author J. Assa , author J. Chataway , author A. Colonna , author H. Confraria , author P. N. Kombo , et al., title Changing directions: Steering science, technology and innovation towards the sustainable development goals ( year 2022 )

    2022

  9. [9]

    Callaert , author B

    author J. Callaert , author B. Van Looy , author A. Verbeek , author K. Debackere , author B. Thijs , title Traces of prior art: An analysis of non-patent references found in patent documents , journal Scientometrics volume 69 ( year 2006 ) pages 3--20

    2006

  10. [10]

    He , author M

    author Z.-L. He , author M. Deng , title The evidence of systematic noise in non-patent references: A study of new zealand companies’ patents , journal Scientometrics volume 72 ( year 2007 ) pages 149--166

    2007

  11. [11]

    Nagaoka , author I

    author S. Nagaoka , author I. Yamauchi , title The use of science for inventions and its identification: Patent level evidence matched with survey , journal Research Institute of Economy, Trade and Industry (RIETI) ( year 2015 )

    2015

  12. [12]

    Alstott , author G

    author J. Alstott , author G. Triulzi , author B. Yan , author J. Luo , title Mapping technology space by normalizing patent networks , journal Scientometrics volume 110 ( year 2017 ) pages 443--479

    2017

  13. [13]

    Zhai , author L

    author D. Zhai , author L. Zhai , author M. Li , author X. He , author S. Xu , author F. Wang , title Patent representation learning with a novel design of patent ontology: Case study on pem patents , journal Technological Forecasting and Social Change volume 183 ( year 2022 ) pages 121912

    2022

  14. [14]

    Ghosh , author S

    author M. Ghosh , author S. Erhardt , author M. E. Rose , author E. Buunk , author D. Harhoff , title Paecter: Patent-level representation learning using citation-informed transformers , journal arXiv preprint arXiv:2402.19411 ( year 2024 )

    arXiv 2024

  15. [15]

    Nicosia , author G

    author V. Nicosia , author G. Mangioni , author V. Carchiolo , author M. Malgeri , title Extending the definition of modularity to directed graphs with overlapping communities , journal Journal of Statistical Mechanics: Theory and Experiment volume 2009 ( year 2009 ) pages P03024

    2009

  16. [16]

    Li , author J

    author S. Li , author J. Hu , author Y. Cui , author J. Hu , title Deeppatent: patent classification with convolutional neural networks and word embedding , journal Scientometrics volume 117 ( year 2018 ) pages 721--744

    2018

  17. [17]

    Lee , author J

    author J.-S. Lee , author J. Hsiang , title Patentbert: Patent classification with fine-tuning a pre-trained bert model , journal arXiv preprint arXiv:1906.02124 ( year 2019 )

    Pith/arXiv arXiv 1906

  18. [18]

    Haghighian Roudsari , author J

    author A. Haghighian Roudsari , author J. Afshar , author W. Lee , author S. Lee , title Patentnet: multi-label classification of patent documents using deep learning based language understanding , journal Scientometrics volume 127 ( year 2022 ) pages 207--231

    2022

  19. [19]

    De Cunzo , author D

    author F. De Cunzo , author D. Consoli , author F. Perruchas , author A. Sbardella , title Mapping critical raw materials in green technologies , journal Industry and Innovation ( year 2025 ) pages 1--34

    2025

  20. [20]

    Giordano , author A

    author V. Giordano , author A. Castagnoli , author I. Pecorini , author F. Chiarello , title Identifying technologies in circular economy paradigm through text mining on scientific literature , journal PLoS One volume 19 ( year 2024 ) pages e0312709

    2024

  21. [21]

    Caldarola , author D

    author B. Caldarola , author D. Mazzilli , author L. Napolitano , author A. Patelli , author A. Sbardella , title Economic complexity and the sustainability transition: A review of data, methods, and literature , journal Journal of Physics: Complexity volume 5 ( year 2024 ) pages 022001

    2024

  22. [22]

    author G. S. Ascione , title Technological diversity to address complex challenges: the contribution of american universities to sdgs ( year 2023 )

    2023

  23. [23]

    Bordignon , title Dataset of search queries to map scientific publications to the un sustainable development goals , journal Data in Brief volume 34 ( year 2021 ) pages 106731

    author F. Bordignon , title Dataset of search queries to map scientific publications to the un sustainable development goals , journal Data in Brief volume 34 ( year 2021 ) pages 106731

    2021

  24. [24]

    Kashnitsky , author G

    author Y. Kashnitsky , author G. Roberge , author J. Mu , author K. Kang , author W. Wang , author M. Vanderfeesten , author M. Rivest , author L. Ke ler , author R. Jaworek , author M. Vignes , et al., title Identifying research supporting the united nations sustainable development goals , journal arXiv preprint arXiv:2209.07285 ( year 2022 )

    Pith/arXiv arXiv 2022

  25. [25]

    Lampe , title Strategic citation , journal Review of Economics and Statistics volume 94 ( year 2012 ) pages 320--333

    author R. Lampe , title Strategic citation , journal Review of Economics and Statistics volume 94 ( year 2012 ) pages 320--333

    2012

  26. [26]

    Kuhn , author K

    author J. Kuhn , author K. Younge , author A. Marco , title Patent citations reexamined , journal The RAND Journal of Economics volume 51 ( year 2020 ) pages 109--132

    2020

  27. [27]

    Voskuil , author S

    author K. Voskuil , author S. Verberne , title Improving reference mining in patents with bert , journal arXiv preprint arXiv:2101.01039 ( year 2021 )

    Pith/arXiv arXiv 2021

  28. [28]

    Bekamiri , author D

    author H. Bekamiri , author D. S. Hain , author R. Jurowetzki , title Patentsberta: A deep nlp based hybrid model for patent distance and classification using augmented sbert , journal Technological Forecasting and Social Change volume 206 ( year 2024 ) pages 123536

    2024

  29. [29]

    Chen , author N

    author X. Chen , author N. Deng , title A semi-supervised machine learning method for chinese patent effect annotation , in: booktitle 2015 international conference on cyber-enabled distributed computing and knowledge discovery , organization IEEE , year 2015 , pp. pages 243--250

    2015

  30. [30]

    Huang , author J

    author L. Huang , author J. Yu , author Y. Hu , author H. Chang , title A semi-supervised learning framework for triz-based chinese patent classification , in: booktitle Proceedings of the 2020 6th International Conference on Computing and Artificial Intelligence , year 2020 , pp. pages 46--50

    2020

  31. [31]

    Zhang , title Interactive patent classification based on multi-classifier fusion and active learning , journal Neurocomputing volume 127 ( year 2014 ) pages 200--205

    author X. Zhang , title Interactive patent classification based on multi-classifier fusion and active learning , journal Neurocomputing volume 127 ( year 2014 ) pages 200--205

    2014

  32. [32]

    Xiong , author S

    author S. Xiong , author S. Chen , author J. He , author Y. Liu , author J. Mao , author C. Liu , title Scalable multi-label patent classification via iterative large language model-assisted active learning , journal World Patent Information volume 82 ( year 2025 ) pages 102380

    2025

  33. [33]

    Edwards , author J

    author A. Edwards , author J. Camacho-Collados , title Language models for text classification: Is in-context learning enough? , journal arXiv preprint arXiv:2403.17661 ( year 2024 )

    Pith/arXiv arXiv 2024

  34. [34]

    Zhong , author S

    author M. Zhong , author S. Ouyang , author M. Jiang , author V. Hu , author Y. Jiao , author X. Wang , author J. Han , title Reactie: Enhancing chemical reaction extraction with weak supervision , journal arXiv preprint arXiv:2307.01448 ( year 2023 )

    Pith/arXiv arXiv 2023

  35. [35]

    Halpern , author S

    author Y. Halpern , author S. Horng , author D. Sontag , title Clinical tagging with joint probabilistic models , in: booktitle Machine Learning for Healthcare Conference , organization PMLR , year 2016 , pp. pages 209--225

    2016

  36. [36]

    author J. B. Reitsma , author A. W. Rutjes , author K. S. Khan , author A. Coomarasamy , author P. M. Bossuyt , title A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard , journal Journal of clinical epidemiology volume 62 ( year 2009 ) pages 797--806

    2009

  37. [37]

    Xu , author Y

    author S. Xu , author Y. Lu , author G. Schoenebeck , author Y. Kong , title Benchmarking llms' judgments with no gold standard , journal arXiv preprint arXiv:2411.07127 ( year 2024 )

    Pith/arXiv arXiv 2024

  38. [38]

    Bernhard-Harrer , author R

    author J. Bernhard-Harrer , author R. Ashour , author J.-M. Eberl , author P. Tolochko , author H. Boomgaarden , title Beyond standardization: a comprehensive review of topic modeling validation methods for computational social science research , journal Political Science Research and Methods ( year 2025 ) pages 1--19

    2025

  39. [39]

    Ratner , author S

    author A. Ratner , author S. H. Bach , author H. Ehrenberg , author J. Fries , author S. Wu , author C. Ré , title Snorkel: rapid training data creation with weak supervision , journal Proceedings of the VLDB Endowment volume 11 ( year 2017 ) pages 269–282 . http://dx.doi.org/10.14778/3157794.3157797. :10.14778/3157794.3157797

    arXiv 2017

  40. [40]

    author X. J. Zhu , title Semi-supervised learning literature survey ( year 2005 )

    2005

  41. [41]

    Kim , author S

    author J. Kim , author S. Lee , title Technology opportunity analysis for creating innovative solutions: Applying semi-supervised topic modelling on patent data , in: booktitle 2024 Portland International Conference on Management of Engineering and Technology (PICMET) , organization IEEE , year 2024 , pp. pages 1--9

    2024

  42. [42]

    Grootendorst , title Bertopic: Neural topic modeling with a class-based tf-idf procedure , journal arXiv preprint arXiv:2203.05794 ( year 2022 )

    author M. Grootendorst , title Bertopic: Neural topic modeling with a class-based tf-idf procedure , journal arXiv preprint arXiv:2203.05794 ( year 2022 )

    Pith/arXiv arXiv 2022

  43. [43]

    Settles , title Active learning literature survey ( year 2009 )

    author B. Settles , title Active learning literature survey ( year 2009 )

    2009

  44. [44]

    Narin , author K

    author F. Narin , author K. S. Hamilton , author D. Olivastro , title The increasing linkage between us technology and public science , journal Research policy volume 26 ( year 1997 ) pages 317--330

    1997

  45. [45]

    Marx , author A

    author M. Marx , author A. Fuegi , title Reliance on science: Worldwide front-page patent citations to scientific articles , journal Strategic Management Journal volume 41 ( year 2020 ) pages 1572--1594

    2020

  46. [46]

    Marx , author A

    author M. Marx , author A. Fuegi , title Reliance on science by inventors: Hybrid extraction of in-text patent-to-article citations , journal Journal of Economics & Management Strategy volume 31 ( year 2022 ) pages 369--392

    2022

  47. [47]

    Bedard-Vallee , author C

    author A. Bedard-Vallee , author C. James , author G. Roberge , title Elsevier 2023 sustainable development goals (sdgs) mapping , journal Elsevier Data Repository volume 1 ( year 2023 )

    2023

  48. [48]

    Kashnitsky , author G

    author Y. Kashnitsky , author G. Roberge , author J. Mu , author K. Kang , author W. Wang , author M. Vanderfeesten , author M. Rivest , author S. Chamezopoulos , author R. Jaworek , author M. Vignes , et al., title Evaluating approaches to identifying research supporting the united nations sustainable development goals , journal Quantitative Science Stud...

    2024

  49. [49]

    author K. A. Bryan , author Y. Ozcan , author B. Sampat , title In-text patent citations: A user's guide , journal Research Policy volume 49 ( year 2020 ) pages 103946

    2020

  50. [50]

    Ke , title An analysis of the evolution of science-technology linkage in biomedicine , journal Journal of Informetrics volume 14 ( year 2020 ) pages 101074

    author Q. Ke , title An analysis of the evolution of science-technology linkage in biomedicine , journal Journal of Informetrics volume 14 ( year 2020 ) pages 101074

    2020

  51. [51]

    Magerman , author B

    author T. Magerman , author B. Van Looy , author K. Debackere , title Does involvement in patenting jeopardize one’s academic footprint? an analysis of patent-paper pairs in biotechnology , journal Research Policy volume 44 ( year 2015 ) pages 1702--1713

    2015

  52. [52]

    Motohashi , author H

    author K. Motohashi , author H. Koshiba , author K. Ikeuchi , title Measuring science and innovation linkage using text mining of research papers and patent information , journal Scientometrics volume 129 ( year 2024 ) pages 2159--2179

    2024

  53. [53]

    Wang , author L

    author Y. Wang , author L. Pei , author J. Sun , author L. Kang , title Trace on both sides: a two-step text mining method to identify academic inventors’ patent--paper pairs , journal Scientometrics ( year 2024 ) pages 1--28

    2024

  54. [54]

    Cohen-Wang , author S

    author B. Cohen-Wang , author S. Mussmann , author A. Ratner , author C. R \'e , title Interactive programmatic labeling for weak supervision , in: booktitle Proceedings of the KDD DCCL Workshop, Anchorage, AK, USA , year 2019 , pp. pages 4--8

    2019

  55. [55]

    Denham , author E

    author B. Denham , author E. M. Lai , author R. Sinha , author M. A. Naeem , title Witan: unsupervised labelling function generation for assisted data programming , journal Proceedings of the VLDB Endowment volume 15 ( year 2022 ) pages 2334--2347

    2022

  56. [56]

    Block , author M

    author C. Block , author M. Wustmans , author N. Laibach , author S. Br \"o ring , title Semantic bridging of patents and scientific publications--the case of an emerging sustainability-oriented technology , journal Technological Forecasting and Social Change volume 167 ( year 2021 ) pages 120689

    2021

  57. [57]

    Shibata , author Y

    author N. Shibata , author Y. Kajikawa , author I. Sakata , title Detecting potential technological fronts by comparing scientific papers and patents , journal Foresight volume 13 ( year 2011 ) pages 51--60

    2011

  58. [58]

    Lippert , author K

    author K. Lippert , author K. U. F \"o rstner , title Patent-publication pairs for the detection of knowledge transfer from research to industry: reducing ambiguities with word embeddings and references , journal arXiv preprint arXiv:2412.00978 ( year 2024 )

    Pith/arXiv arXiv 2024

  59. [59]

    Masclans-Armengol , author S

    author R. Masclans-Armengol , author S. Hasan , author W. M. Cohen , title Measuring the Commercial Potential of Science , type Technical Report , Working Paper w32262, National Bureau of Economic Research, year 2024

    2024

  60. [60]

    Beltagy , author K

    author I. Beltagy , author K. Lo , author A. Cohan , title Scibert: A pretrained language model for scientific text , journal arXiv preprint arXiv:1903.10676 ( year 2019 )

    Pith/arXiv arXiv 1903

  61. [61]

    Cohan , author S

    author A. Cohan , author S. Feldman , author I. Beltagy , author D. Downey , author D. S. Weld , title Specter: Document-level representation learning using citation-informed transformers , journal arXiv preprint arXiv:2004.07180 ( year 2020 )

    Pith/arXiv arXiv 2004

  62. [62]

    Singh , author M

    author A. Singh , author M. D'Arcy , author A. Cohan , author D. Downey , author S. Feldman , title Scirepeval: A multi-format benchmark for scientific document representations , journal arXiv preprint arXiv:2211.13308 ( year 2022 )

    Pith/arXiv arXiv 2022

  63. [63]

    Guellec , author J

    author D. Guellec , author J. Liu , author M. Amdaoud , author W. Mescheba , author J. Quemener , author D. Sapinho , author J.-M. Deltorn , title Npl v. nlp: Analysing the links between science and technology with citations and semantics , journal NLP: Analysing the Links Between Science and Technology With Citations and Semantics (November 22, 2024) ( y...

    2024

  64. [64]

    Xu , author D

    author S. Xu , author D. Zhai , author F. Wang , author X. An , author H. Pang , author Y. Sun , title A novel method for topic linkages between scientific publications and patents , journal Journal of the Association for Information Science and Technology volume 70 ( year 2019 ) pages 1026--1042

    2019

  65. [65]

    Dagdelen , author A

    author J. Dagdelen , author A. Dunn , author S. Lee , author N. Walker , author A. S. Rosen , author G. Ceder , author K. A. Persson , author A. Jain , title Structured information extraction from scientific text with large language models , journal Nature Communications volume 15 ( year 2024 ) pages 1418

    2024

  66. [66]

    author I. C. Wiest , author F. Wolf , author M.-E. Le mann , author M. van Treeck , author D. Ferber , author J. Zhu , author H. Boehme , author K. K. Bressem , author H. Ulrich , author M. P. Ebert , et al., title Llm-aix: An open source pipeline for information extraction from unstructured medical text based on privacy preserving large language models ,...

    2024

  67. [67]

    Han , author C

    author R. Han , author C. Yang , author T. Peng , author P. Tiwari , author X. Wan , author L. Liu , author B. Wang , title An empirical study on information extraction using large language models , journal arXiv preprint arXiv:2305.14450 ( year 2023 )

    Pith/arXiv arXiv 2023

  68. [68]

    Yoshikawa , author R

    author N. Yoshikawa , author R. Krestel , title Do large language models understand patents? enhancing patent classification through ai-generated summaries , journal World Patent Information volume 81 ( year 2025 ) pages 102353

    2025

  69. [69]

    Hain , author R

    author D. Hain , author R. Jurowetzki , author M. Squicciarini , title Mapping complex technologies via science-technology linkages; the case of neuroscience--a transformer based keyword extraction approach , journal arXiv preprint arXiv:2205.10153 ( year 2022 )

    Pith/arXiv arXiv 2022

  70. [70]

    Achiam , author S

    author OpenAI , author J. Achiam , author S. Adler , author S. Agarwal , author L. A. et al. , title Gpt-4 technical report , year 2024 . https://arxiv.org/abs/2303.08774. http://arxiv.org/abs/2303.08774 arXiv:2303.08774

    Pith/arXiv arXiv 2024

  71. [71]

    Geng , title Label distribution learning , year 2016

    author X. Geng , title Label distribution learning , year 2016 . https://arxiv.org/abs/1408.6027. http://arxiv.org/abs/1408.6027 arXiv:1408.6027

    Pith/arXiv arXiv 2016

  72. [72]

    Akiba , author S

    author T. Akiba , author S. Sano , author T. Yanase , author T. Ohta , author M. Koyama , title Optuna: A next-generation hyperparameter optimization framework , year 2019 . https://arxiv.org/abs/1907.10902. http://arxiv.org/abs/1907.10902 arXiv:1907.10902

    Pith/arXiv arXiv 2019

  73. [73]

    author A. F. Agarap , title Deep learning using rectified linear units (relu) , journal arXiv preprint arXiv:1803.08375 ( year 2018 )

    Pith/arXiv arXiv 2018

  74. [74]

    author S. E. Robertson , author S. Walker , author S. Jones , author M. M. Hancock-Beaulieu , author M. Gatford , et al., title Okapi at trec-3 , journal Nist Special Publication Sp volume 109 ( year 1995 ) pages 109

    1995

  75. [75]

    Devlin , author M.-W

    author J. Devlin , author M.-W. Chang , author K. Lee , author K. Toutanova , title Bert: Pre-training of deep bidirectional transformers for language understanding , in: booktitle Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers)...

    2019

  76. [76]

    Srebrovic , author J

    author R. Srebrovic , author J. Yonamine , title Leveraging the bert algorithm for patents with tensorflow and bigquery , journal White paper ( year 2020 )

    2020

  77. [77]

    Reimers , author I

    author N. Reimers , author I. Gurevych , title Sentence-bert: Sentence embeddings using siamese bert-networks , journal arXiv preprint arXiv:1908.10084 ( year 2019 )

    Pith/arXiv arXiv 1908

  78. [78]

    Moradi , author K

    author M. Moradi , author K. Blagec , author F. Haberl , author M. Samwald , title Gpt-3 models are poor few-shot learners in the biomedical domain , journal arXiv preprint arXiv:2109.02555 ( year 2021 )

    Pith/arXiv arXiv 2021

  79. [79]

    Mu , author B

    author Y. Mu , author B. P. Wu , author W. Thorne , author A. Robinson , author N. Aletras , author C. Scarton , author K. Bontcheva , author X. Song , title Navigating prompt complexity for zero-shot classification: A study of large language models in computational social science , journal arXiv preprint arXiv:2305.14310 ( year 2023 )

    Pith/arXiv arXiv 2023

  80. [80]

    Dhananasekaran , author R

    author S. Dhananasekaran , author R. Palanivel , author S. Pappu , title Adsorption of methylene blue, bromophenol blue, and coomassie brilliant blue by -chitin nanoparticles , journal Journal of advanced research volume 7 ( year 2016 ) pages 113--124

    2016

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.