Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Patent-publication pairs for the detection of knowledge transfer from research to industry: reducing ambiguities with word embeddings and references

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that combining name matching with MeSH-based BERT embeddings and shared references reduces clearly false patent-publication pairs from about half at one common name to roughly ten percent at two and none at four.

desk verdict A useful, openly implemented patent-publication linking pipeline whose headline false-positive rates are confounded by how the evaluation bins were selected. read the letter →

arxiv 2412.00978 v1 pith:XIPQHHFO submitted 2024-12-01 cs.IR

Klaus Lippert , Konrad U. Förstner This is my paper · ORCID
classification cs.IR
keywords patent-publicationmatchingknowledgetransferwordembeddingsBERTMeSHcosinesimilaritycommonreferencesresearchcommercialization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles a bottleneck in measuring the economic value of medical research: linking a patent to the scientific publication it grew out of. Matching author and inventor names alone drowns in homonyms, so the authors add two automatic filters: content similarity computed from word embeddings of MeSH medical terms, and shared references. On a five-year sample, the filters cut the share of clearly false pairs from about half at one common name to roughly ten percent at two common names, and to zero at four or more. If the approach holds, patent data become a usable, open proxy for research commercialization, and the same recipe can be adapted beyond medicine.

What carries the argument

The load-bearing mechanism is the candidate pair, filtered in three stages. First, names are normalized to 'last name, initials' and joined, with a half-to-one-and-a-half-year window between patent filing and publication to respect novelty. Second, each document is reduced to its MeSH terms: for patents these are extracted from English, German, or French text by dictionary lookup, mapped to English, de-duplicated, and alphabetized; for publications they are the curated MeSH headings. A BERT model (a context-predicting word-embedding model) turns each term into a vector, and the vectors are summed to represent the whole document; cosine similarity between the two sums is the content score. Third, the patent's listed references are enriched with DOIs and checked for overlap with the publication's references; one common reference counts as a match. A final statistical step compares the distribution of patent classification codes across 'sure pairs' against the baseline to decide which classes are admissible.

What would settle it

Replace the MeSH-BERT vectors with random vectors of the same dimension, keep the same name and reference filters, and recompute the cosine threshold: if the final set of valid pairs barely changes, the embedding feature is not what is reducing the false-pair rate.

Watch

Extended reading notes

Core claim

The central discovery is that two cheap, automatic signals resolve most of the ambiguity left by author/inventor name matching. For each candidate pair, the paper computes the cosine similarity between a patent and a publication after representing both as the sum of BERT word vectors over their MeSH (a controlled medical vocabulary) terms, and it checks whether the patent's reference list shares a DOI with the publication. On a five-year cohort of European patent first filings and biomedical publications, the combined workflow leaves about half of single-name pairs clearly false, drops that to about 10% for two-name pairs, and to zero for pairs with four or more common names. The paper also claims that a statistical comparison of patent-class distributions can automatically pick the patent classes that are valid for medicine, without manual mapping.

Load-bearing premise

The content filter assumes that reducing a patent or publication to its medical topic terms, alphabetized and added into one vector, keeps enough meaning for cosine similarity to separate true research-to-patent links from accidental name matches.

Editorial extensions

If this is right

  • With two common author/inventor names, only about 10% of pairs remain clearly false after the filters, and with four or more names none are clearly false; the workflow therefore turns raw name joins into a usable signal without manual resolution.
  • A single shared reference is enough to treat a pair as valid, which makes reference enrichment a high-precision filter even when patent reference data are sparse.
  • The two supporting features yield approximately the same cosine-similarity threshold, so they can be combined into one ranking without one dominating the other.
  • The statistical selection of permissible patent classes can be transferred to other scientific domains, replacing manual assignment of patent classes to disciplines.
  • Because all data and code are open, any institution can reproduce the five-year example and apply the pipeline to other periods or databases.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same two-filter recipe could be applied to other homonym-heavy linking problems, such as matching clinical trial records to their publications or grants to papers, where name matching alone is equally unreliable.
  • The paper treats the 'academic patent' signal as a binary +0.1 boost; a natural extension would be to make it a continuous feature derived from the fraction of academic-looking inventors or applicants, which might sharpen rankings further.
  • A stronger validation would be to run the workflow on a cohort where true identities are known through researcher identifiers, measuring precision and recall directly rather than through a small manual review.
  • Since the paper uses an early BERT model, the same pipeline with a biomedical-domain embedding model would likely separate true and false pairs more cleanly; the authors position their results as a baseline for exactly this comparison.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes an automated workflow for linking EPO patent families to PubMed publications as an indicator of knowledge transfer from research to industry. Raw pairs are formed by normalized author/inventor name matches and then filtered by (i) cosine similarity between BERT-vector sums of MeSH terms extracted from patent descriptions and PubMed MeSH headings, (ii) shared references enriched with DOIs via Crossref, (iii) an academic-patent indicator, and (iv) an IPC class filter learned from a subset of 'sure pairs.' The final validity rule accepts pairs with three or more common names, at least one common reference, or cosine similarity above a fitted threshold; only the best three cosine-above-threshold pairs per publication are kept. The paper reports a manual evaluation with false-positive rates falling from about 50% for one common name to about 10% for two names and 0% for four or more names, with all code and data released.

Significance. If the quantitative claims hold, the paper contributes a reproducible, open-data pipeline and two useful disambiguation features for a task that is usually done manually or limited to single institutions. The release of the SQL/Python code, the use of public EPO and PubMed data, and the automatic IPC-class selection are strengths that make the work a reasonable baseline for future embedding-based matching. The central quantitative claim, however, is not established by the evaluation as reported: the bins compared in Figure 8 have passed different filters, thresholds are fitted to the same data later used for evaluation, and sample sizes are not reported. The underlying idea remains plausible and worth pursuing.

major comments (4)
  1. [Evaluation (Figure 8)] The headline false-positive rates compare pairs that have undergone different filter combinations. The text states that one-name pairs were reviewed 'despite the applied filters,' and under the final validity rule two-name pairs can be valid only if they also have at least one common reference, cosine similarity above the threshold, or an academic-patent boost; four-name pairs, by contrast, are accepted by the name-count rule alone. Thus the drop from about 50% to about 10% to 0% in Figure 8 may reflect the effect of the reference and cosine filters rather than the number of common names. Please report the manual evaluation separately for each combination of the name-count, reference, and cosine filters, or stratify the sample before filtering, so that the effect of name count is not confounded with the effect of the other validity conditions.
  2. [Filter by patent classes; Rank by cosine similarity and disambiguate features] Several load-bearing thresholds are derived from the same data that is later evaluated, and one filter is learned from a subset defined by the other features. The cosine threshold is taken from the whiskers of Figures 4 and 5, the IPC filter is derived from 'sure pairs' defined by four common names with matching countries, four common references, or cosine similarity of at least 0.95, and the final validity rule uses the same name, reference, and cosine features; the 1.5% IPC cutoff and the +0.1 academic boost are chosen without a stated independent basis. This makes it difficult to interpret the post-filter false-positive rates as out-of-sample performance. Please validate the thresholds on a held-out period or a held-out subset, and report the sensitivity of the final pair set to each threshold.
  3. [Evaluation] The manual evaluation is not fully reported. The 'defined quantity of pairs per number of common names' is not specified, so the reader cannot tell whether the 0% rate in the four-name bin is based on a handful of pairs or a large sample; no confidence intervals are given; and the treatment of the 'not determinable' category and the instructions to reviewers are absent. Please provide the exact counts per bin, binomial confidence intervals, and a description of how undecidable cases were handled, and preferably double-code a subset to estimate reviewer agreement.
  4. [Patent-publication similarity scores using embeddings] The paper acknowledges that alphabetically sorting and de-duplicating MeSH terms before summing BERT vectors destroys word order and context, and it justifies this with 'the clear results found (see below).' That justification is weakened by the evaluation issues in Figure 8, because the same downstream results are also the source of the fitted cosine thresholds. Please add an independent sanity check of the embedding representation, for example separation of known related versus unrelated patent-publication pairs or a comparison with a sequence-aware model on a small labeled set, so that the method does not rely solely on the outcome it is used to produce.
minor comments (4)
  1. [Patent-publication similarity scores using embeddings] The sentence 'Figure 3 shows the results of the similarity calculation...' should refer to Figure 4, since Figure 3 is the language distribution of patent descriptions.
  2. [Abstract] 'names of authors and investors' should presumably read 'names of authors and inventors'.
  3. [Patent-publication similarity scores using embeddings] 'which provided is with satisfactory results' contains a typo, presumably 'provided us with satisfactory results'.
  4. [Rank by cosine similarity and disambiguate features] The definition of the cosine threshold should be stated more precisely: the text says a threshold is calculated from the lower bounds of the whiskers and then mentions the median; please specify exactly which statistic is used and how the whisker bounds are obtained.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: manual review is independent; name-count comparison confound is a correctness issue.

full rationale

The derivation chain is data-driven but not circular. Raw pairs are formed by normalized inventor/author names; cosine similarity and common-reference features are computed independently from document text and bibliographies. Thresholds are descriptive statistics (lower whisker bounds) of feature distributions and are used as operating points; they are not used as labels in the manual evaluation. The IPC filter is constructed from 'sure pairs' defined by very strict versions of the same features, but this is a transparent training-set heuristic: the paper does not claim the IPC filter is derived from first principles, and the final precision estimates in Figure 8 come from human review of abstracts and full patents, not from the fitted thresholds. The only self-citation (Lippert & Förstner 2024) points to a companion report and is not load-bearing. The comparison of false-positive rates across name-count bins is confounded by the reference/cosine filters (two-name pairs are a filtered subset), but that is a statistical-correctness limitation, and under the hard rules it is not a circularity. No step reduces by construction to its own input.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

No new physical or conceptual entities are introduced. The free parameters are thresholds and boosts chosen from the data or adopted from prior work. The axioms are domain assumptions about name matching, IPC distribution proxies, and the adequacy of the bag-of-MeSH-terms embedding.

free parameters (5)
  • cosine similarity threshold = not reported exactly; derived as median of lower whiskers of boxplots for 2+ common names
    Used to decide which patent-publication pairs are valid; fitted from the pair distribution in Figure 4.
  • IPC class percentage limit = 1.5%
    Arbitrary cutoff to select patent classes that deviate from the baseline distribution in the Q-Q plot.
  • academic patent similarity boost = +0.1
    Added to the cosine similarity for patents with academic indicators (prof or univ in names).
  • maximum valid pairs per publication = 3
    Only the best three pairs above the cosine threshold are kept.
  • patent-publication time window = 0.5 to 1.5 years
    Adopted from Dornbusch and Neuhäusler; restricts pairing to publications within that interval after filing.
assumptions (3)
  • domain assumption Author/inventor names normalized to last name and initials are sufficient for candidate pairing, with acceptable homonym rates (Milojevic 97%).
    Used in 'Pairing publications and patents using common names'.
  • domain assumption The IPC distribution of the strict 'sure pairs' subset is a valid proxy for the patent classes relevant to medical research, so classes deviating from the full EPO baseline are appropriate as a filter.
    Used in 'Filter by patent classes'.
  • ad hoc to paper A bag-of-MeSH-terms representation with summed BERT vectors retains enough semantic information for cosine similarity to be a useful filter.
    Used in 'Patent-publication similarity scores using embeddings'; the paper acknowledges that ordering is lost.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Patent-publication pairs for the detection of knowledge transfer from research to industry: reducing ambiguities with word embeddings and references." pith.science (2026). https://pith.science/paper/XIPQHHFO

@misc{pith2026241200978,
  author       = {Pith},
  title        = {Pith review of: Patent-publication pairs for the detection of knowledge transfer from research to industry: reducing ambiguities with word embeddings and references},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XIPQHHFO}},
  note         = {Machine review of arXiv:2412.00978}
}
read the original abstract

The performance of medical research can be viewed and evaluated not only from the perspective of publication output, but also from the perspective of economic exploitability. Patents can represent the exploitation of research results and thus the transfer of knowledge from research to industry. In this study, we set out to identify publication-patent pairs in order to use patents as a proxy for the economic impact of research. To identify these pairs, we matched scholarly publications and patents by comparing the names of authors and investors. To resolve the ambiguities that arise in this name-matching process, we expanded our approach with two additional filter features, one used to assess the similarity of text content, the other to identify common references in the two document types. To evaluate text similarity, we extracted and transformed technical terms from a medical ontology (MeSH) into numerical vectors using word embeddings. We then calculated the results of the two supporting features over an example five-year period. Furthermore, we developed a statistical procedure which can be used to determine valid patent classes for the domain of medicine. Our complete data processing pipeline is freely available, from the raw data of the two document types right through to the validated publication-patent pairs.

Figures

Figures reproduced from arXiv: 2412.00978 by the authors.

Figure 1
Figure 1. Number of patent families from EPO and publications from PubMed baseline [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Simplified workflow Patent families Based on the process a patent goes through from the first filing with the patent office to the eventual granting of the patent and assignment of the identification number by the patent office, patent data can be split into two main groups: patent applications (second last digit of patent’s publication number is A) and patent specifications (second last digit is B). Within each gro… view at source ↗
Figure 3
Figure 3. Language of patent descriptions used for MeSH extraction. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Cosine similarities of patent-publication pairs separated by the number of [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Cosine similarities of patent-publication pairs separated by the number of [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Zoom into a Q-Q plot of the distribution of patent classes (IPC) for different [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 8
Figure 8. Figure 8: Starting with the found publication-patent pairs, a defined quantity of pairs [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 7
Figure 7. Figure 7: Distribution of N:N relationships of patent-publication pairs for different [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Result of a manual review of the validity of a random selection of the found [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. From scratch to silver: Creating trustworthy training data for patent-SDG classification using Large Language Models

    cs.CL 2025-09 conditional novelty 6.0 of 10

    A weak-supervision pipeline using LLM-extracted concepts and rank fusion creates silver-standard patent-to-SDG labels that recover known citation-derived associations and show high network modularity.

Reference graph

Works this paper leans on

26 extracted references · 18 canonical work pages · cited by 1 Pith paper

  1. [6]

    Francesco Lissoni, Patrick Llerena, Maureen Mckelvey, and Bulat Sanditov

    URL https://doi.org/10.1007/978-3-658-43683-4_6 . Francesco Lissoni, Patrick Llerena, Maureen Mckelvey, and Bulat Sanditov. Academic patenting in europe: New evidence from the keins database. Research Evaluation, 17, 02

    work page doi:10.1007/978-3-658-43683-4_6

  2. [9]

    [Online; ac- cessed 20-April-2022]

    URL http://mesh.inserm.fr/FrenchMesh/. [Online; ac- cessed 20-April-2022]. 14 Jinseok Kim, Jenna Kim, and Jason Owen-Smith. Ethnicity-based name partitioning for author name disambiguation using supervised machine learning. Journal of the Association for Information Science and Technology, 72, 02

    work page 2022

  3. [10]

    doi: 10.1002/asi. 24459. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240, sep

    doi:10.1002/asi

  4. [11]

    Steffen Lemke, Anne Witthake, and Isabella Peters

    doi: 10.1093/bioinformatics/btz682. Steffen Lemke, Anne Witthake, and Isabella Peters. Altmetrics for german medical re- search: what leads to research articles achieving policy impact? In 26th International Conference on Science, Technology and Innovation Indicators (STI 2022),

    doi:10.1093/bioinformatics/btz682

  5. [16]

    Bernd M¨ uller, Christoph Poley, Jana P¨ ossel, Alexandra Hagelstein, and Thomas G¨ ubitz

    doi: 10.1016/j.joi.2013.06.006. Bernd M¨ uller, Christoph Poley, Jana P¨ ossel, Alexandra Hagelstein, and Thomas G¨ ubitz. Livivo – the vertical search engine for life sciences. Datenbank-Spektrum, 17:29–34,

    work page doi:10.1016/j.joi.2013.06.006 2013

  6. [18]

    nist.gov/div898/software/dataplot/refman2/auxillar/cosdist.htm

    URL https://www.itl. nist.gov/div898/software/dataplot/refman2/auxillar/cosdist.htm. [Online; accessed 20-April-2022]. National Library of Medicine (US) (NLM). Pubmed central,

    work page 2022

  7. [20]

    [Online; accessed 20-April-2022]

    URL https://www.nlm.nih.gov/mesh/meshhome.html. [Online; accessed 20-April-2022]. Petar Ristoski and Heiko Paulheim. RDF2Vec: RDF Graph Embeddings for Data Mining. In The Semantic Web – ISWC

    work page 2022

  8. [22]

    doi: https://doi.org/10.1016/ j.wpi.2014.03.002

    ISSN 0172-2190. doi: https://doi.org/10.1016/ j.wpi.2014.03.002. URL https://www.sciencedirect.com/science/article/pii/ S0172219014000404. Suzan Verberne, Ioannis Chios, and Jian Wang. Extracting and matching patent in-text references to scientific publications. In BIRNDL@SIGIR,

    work page 2014

Show all 26 references
  1. [23]

    University patenting in germany before and after 2002: what role did the professors’ privilege play? Industry and Innovation, 19(1):23–44,

    Sidonia Von Proff, Guido Buenstorf, and Martin Hummel. University patenting in germany before and after 2002: what role did the professors’ privilege play? Industry and Innovation, 19(1):23–44,

    2002

  2. [24]

    doi: 10.1080/13662716.2012.649060. Ken S. Voskuil and Suzan Verberne. Improving reference mining in patents with BERT. CoRR, abs/2101.01039,

    2012

  3. [25]

    Wikipedia

    URL https://arxiv.org/abs/2101.01039. Wikipedia. Hochschullehrerprivileg — Wikipedia, the free encyclopedia, 2022a. URL https://de.wikipedia.org/wiki/Hochschullehrerprivileg. [Online; accessed 20-April-2022]. Wikipedia. Strasbourg agreement — Wikipedia, the free encyclopedia, ...

    2022 arXiv

  4. [26]

    doi: 10.1038/sdata.2016.18. 16

    2016 doi

  5. [2000]

    ncbi.nlm.nih.gov/pmc/

    URL https://www. ncbi.nlm.nih.gov/pmc/. [Online; accessed 20-April-2022]. National Library of Medicine (US) (NLM). Medical Subject Headings MeSH,

    2022

  6. [2002]

    1.44.14279

    doi: 10.1287/mnsc.48. 1.44.14279. Marco Baroni, Georgiana Dinu, and Germ´ an Kruszewski. Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. volume 1, pages 238–247, 06

    doi

  7. [2007]

    2007.355

    doi: 10.1038/nprot. 2007.355. 15 Peter van Dongen, Jos Winnink, and Robert Tijssen. Academic inventions and patents in the netherlands: A case study on business sector exploitation. World Patent Information, 38:27–32,

    2007 doi

  8. [2008]

    Francesco Lissoni, Peter Lotz, Jens Schovsbo, and Adele Treccani

    doi: 10.3152/095820208X287171. Francesco Lissoni, Peter Lotz, Jens Schovsbo, and Adele Treccani. Academic patenting and the professor’s privilege: Evidence on denmark from the keins database. Science and Public Policy - SCI PUBLIC POLICY, 36:595–607, 10

    doi

  9. [2012]

    doi: 10.1145/2350036.2350040. A. Grover and J. Leskovec. node2vec: Scalable feature learning for networks. In Inter- national Conference on Knowledge Discovery and Data Mining (KDD),

  10. [2013]

    Deutsches Institut f¨ ur Medizinische Dokumentation und Information (DIMDI)

    doi: 10.1504/IJTTC.2013.064154. Deutsches Institut f¨ ur Medizinische Dokumentation und Information (DIMDI). Ger- man mesh,

    2013

  11. [2014]

    Dirk Brockmann

    doi: 10.3115/v1/P14-1023. Dirk Brockmann. Global connectivity and the spread of infectious diseases. In Nova Acta Leopoldina, number

    doi

  12. [2016]

    Friedrich, and Juliane Fluck

    Harsha Gurulingappa, Bernd M¨ uller, Roman Klinger, Heinz-Theodor Mevissen, Mar- tin Hofmann-Apitius, Christoph M. Friedrich, and Juliane Fluck. Prior art search in chemistry patents based on semantic concepts and co-citation analysis. In Ellen M. Voorhees and Lori P. Buckland...

    2010

  13. [2017]

    URL https://www.livivo.de

    doi: 10.1007/s13222-016-0245-2. URL https://www.livivo.de. National Institute of Standards and Technology (NIST). Cosine distance, cosine similar- ity, angular cosine distance, angular cosine similarity,

    doi

  14. [2018]

    URL https://arxiv.org/abs/1810.04805

    doi: 10.48550/ ARXIV.1810.04805. URL https://arxiv.org/abs/1810.04805. Friedrich Dornbusch and Peter Neuh¨ ausler. Academic patents in germany. Studien zum deutschen Innovationssystem 6-2015, Berlin,

    2015 doi arXiv

  15. [2019]

    [Online; accessed 1- September-2020]

    URL https://www.dimdi.de/dynamic/de/klassifikationen/ weitere-klassifikationen-und-standards/mesh/ . [Online; accessed 1- September-2020]. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding

    2020

  16. [2021]

    Charlotta Dahlborg, Danielle Lewensohn, and Carl Johan Sundberg

    doi: 10.1007/s10994-021-05997-6. Charlotta Dahlborg, Danielle Lewensohn, and Carl Johan Sundberg. Investigating inventive productivity at sweden’s largest medical university. International Jour- nal of Technology Transfer and Commercialisation , 12:102–120, 01

    doi

  17. [2022]

    Klaus Lippert and Konrad U

    doi: 10.5281/zenodo.6645109. Klaus Lippert and Konrad U. F¨ orstner. Nutzung von Patentdaten zur Erfassung der wirtschaftlichen Verwertung von Forschung, pages 97–107. Springer Fachmedien Wies- baden, Wiesbaden,

    doi

  18. [2024]

    doi: 10.1007/978-3-658-43683-4

    ISBN 978-3-658-43683-4. doi: 10.1007/978-3-658-43683-4

    doi

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.