{"id":"620cece3-196c-40e4-bddc-4914dc4a395e","arxiv_id":"1908.08752","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A Google Scholar search for papers with ResearchGate in the title yields 159 documents, mostly journal articles, and suggests growing publication and citation activity after 2015.","lead":"This paper counts and describes 159 publications about the academic social network ResearchGate from 2008 to 2017, found by searching Google Scholar for the word in their titles. It is a bibliometric snapshot that shows a small and growing literature, but its headline numbers contain inconsistencies.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The growth claim rests on a single non-reproducible Google Scholar title search; omitted titleless documents could change the trend.","rationale":"The central claim is only as strong as the count time series. The query is explicit in the Methods, so the paper is internally coherent, but the external validity of the counts is untested. The reader's weakest assumption identifies the same search-validity issue, and the paper's own admission that the raw Google Scholar results were misleading makes the manual filtering step especially load-bearing. A broader search or a second database could either confirm the post-2015 trend or reveal that it is an artifact of title-only coverage. Because this is precisely the kind of reproducibility and validity check the reader requested, the conditional verdict remains appropriate. If the concrete test shows substantial omitted literature or a different temporal distribution, the verdict should move toward rejection or unverified; otherwise the paper's conclusion can stand.","tokens_in":6464,"tokens_out":7924,"duration_ms":72739,"concrete_test":"Re-run the retrieval on a fixed date with the original query `allintitle:researchgate` and with a broader query, e.g. 'researchgate' in title OR abstract OR keywords, for 2008–2017 in Google Scholar plus an equivalent search in Scopus or Web of Science. Apply the paper's inclusion criteria independently with two coders and publish the full record list and screening decisions. The concern lands if the broader or database query adds a substantial number of relevant documents and shifts the share of documents published in 2015–2017 by more than a few percentage points, or changes the sign of the pre/post-2015 trend.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central conclusion — 'Interest in ResearchGate has grown since 2015' — is supported only by the time series derived from `allintitle:researchgate` in Google Scholar, followed by manual deletion of 227 of 386 records (Methods/Procedure and Results). Everything downstream (159 documents, 46% journal papers, author rankings, citation totals) inherits the completeness and reproducibility of that one query. Because the query is title-only, any relevant article whose title does not contain 'ResearchGate' is invisible; because Google Scholar's index is unstable and the manual filter is not codeable, the reported counts cannot be independently checked. The paper itself notes that the raw GS results were 'misleading' (Results), which makes the manual step decisive. If missing titleless publications or non-GS publications grew faster before 2015 than after, the observed post-2015 rise would be an artifact of the search boundary rather than of research interest. The internal inconsistency in the Conclusions (90% of documents since 2015 vs 81% implied by Table 1) reinforces that the quantitative support is not trustworthy as reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a scoping bibliometric review of documents about the academic social network ResearchGate published between 2008 and 2017. The authors searched Google Scholar using the query `allintitle:researchgate`, manually filtered the 386 retrieved records down to 159 documents (excluding 227 bibliographic citations), and analyzed the documents by publication type, author productivity and gender, keywords, language, country affiliation, and citation counts. The main results are that journal articles are the most frequent document type (46%), the two most productive authors (each with 8 papers) are Spanish scholars Delgado López-Cózar and Orduña-Malea, and that research interest in ResearchGate has grown since 2015 as reflected in the number of publications and citations. The paper concludes that 'interest in ResearchGate has grown since 2015'.","tokens_in":6657,"tokens_out":10761,"duration_ms":87989,"significance":"If the reported figures were accurate, the paper would provide a useful descriptive baseline for the literature on ResearchGate, a topic that is relevant to bibliometrics, scholarly communication, and altmetrics. The paper's strengths include its attempt to cover a decade of publications, its breakdown by document type, and its attention to authorship patterns and keywords. However, the study's validity is compromised by the non-reproducible nature of the Google Scholar search and by internal inconsistencies between the stated percentages in the abstract and conclusions and the data shown in Table 1. Because the main conclusion (growth since 2015) relies on these counts, the paper needs correction and verification before its quantitative claims can be accepted.","major_comments":[{"comment":"The abstract reports that 'more than 90% of the total cites fell in that period' (i.e., since 2014), and the Conclusions state that '90% of the documents – all papers in journals – and 80% of the citations in our data set date from that year' (i.e., 2015). These claims are contradicted by Table 1. From Table 1, the number of documents published in 2015–2017 is 129/159 = 81.1% (not 90%), and the number of bibliographic citation records for the same period is 142/227 = 62.6% (not 80%). The abstract's more-than-90%-since-2014 figure for the 227 citation records is also incorrect (175/227 = 77.1%). If the abstract instead refers to the 1,465 citations received by the documents (as described in the Results), the paper does not provide any data in the text to verify that 90% of these fell after 2014. These errors affect the headline growth claim and must be corrected.","section":"Abstract and Conclusions"},{"comment":"The search strategy is not reproducible and does not guarantee a valid or representative sample of the literature on ResearchGate. The query `allintitle:researchgate` in Google Scholar is non-deterministic, and the manual filtering of 386 records to 159 is described only qualitatively ('eliminated duplicates and corrected some errors', 'data were cleaned'). The paper itself notes that the initial GS results were 'misleading', and the filtering criterion is not specified in a way that allows replication. Moreover, restricting the search to titles containing 'ResearchGate' excludes any relevant document whose title does not contain that exact phrase, and the use of a single database (Google Scholar) means that coverage depends on the database's transient index. The paper should provide a full audit trail (date-stamped search, inclusion/exclusion criteria, a flow diagram, and ideally a list of the 159 included documents) and cross-validate the counts with an additional database such as Scopus or Web of Science.","section":"Methods/Procedure and Results"}],"minor_comments":[{"comment":"The first sentence of the full text contains a typo: 'ReseachGate' should be 'ResearchGate'.","section":"Full text, first sentence"},{"comment":"The column headers in Table 1 are garbled (e.g., 'Y ear Publication type N (%)' reads as two headings). Please separate and label the columns clearly, especially the 'Total records' and 'Citations' columns.","section":"Table 1"},{"comment":"The sentence 'The filtering criterion was \"articles, excluding patents\" to obtain the total number of items, sorted by the national or institutional web domain' is unclear; please explain how sorting by domain contributed to the filtering and what 'articles, excluding patents' means in the Google Scholar interface.","section":"Methods/Procedure"},{"comment":"In the sentence 'The most used keywords were \"ResearchGate\" (33) and \" Altmetrics\" (14)', there is an extra space before 'Altmetrics'. Also, Figure 1 would benefit from a descriptive caption.","section":"Results – keywords"},{"comment":"The phrase '90% of the documents – all papers in journals –' is grammatically confusing; if it is meant to say that the growth is driven by journal papers, rephrase clearly.","section":"Conclusions"},{"comment":"Please specify whether the custom date range for each year was applied as January 1 to December 31, and describe how the 'adding up the total' was done to avoid double-counting year boundaries.","section":"Methods"}],"recommendation":"major_revision","confidential_remarks":"This is a small descriptive scoping review that fits the scope of European Science Editing. The main technical concerns are the non-reproducible search and the internal numerical inconsistencies. I would encourage the editor to send the paper back for revision with a request to correct the percentages, add a flow diagram, and provide a more detailed search protocol. The paper's topic is timely but its current quantitative support is shaky."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a small descriptive bibliometric review, and it is useful for one thing: it gives a defined inventory of 159 documents with 'ResearchGate' in the title, retrieved from Google Scholar, 2008–2017. The table shows the growth plainly—few documents per year until 2015, then 35, 39, 55. So the paper's main qualitative conclusion, that interest in ResearchGate has grown since 2015, is supported by its own data.\n\nWhat is not supported is the arithmetic in the abstract and conclusions. The abstract says more than 90% of total citations fell since 2014. Table 1's citation column sums to 227; 2014–2017 accounts for 175, which is 77%. The conclusions say 90% of documents and 80% of citations date from 2015. The documents from 2015–2017 are 129 of 159, which is 81%; the citation records are 142 of 227, which is 63%. Those are not rounding errors, and the phrase '90% of the documents – all papers in journals' is internally confused.\n\nThere is also a labeling problem: Table 1's 'Citations' column actually counts the number of raw Google Scholar records that were citations rather than documents (227), not the 1,465 citations the text later reports. That conflation makes the table misleading until you reverse-engineer it.\n\nThe method is a single allintitle:researchgate query, hand-filtered, with 227 records removed. That is not reproducible, and the paper itself says the raw results were misleading. I don't think this sinks the growth claim—the trend is too large—but it does mean the exact counts are soft. The paper would be more honest if it presented the corpus as 'documents found by this particular search', not as a definitive count of research on ResearchGate.\n\nThe keyword analysis is self-referential by construction, and the author rankings are based on a small set, but those are minor. The reference list is fine for the topic, and the paper gives useful background on the ResearchGate literature.\n\nWho is this for? People working on academic social networks or bibliometrics of social media. It is not a technical contribution, but it is a reasonable starting point. With corrected arithmetic and a clearer method section, it would be a passable contribution to a niche area. As is, I'd want a referee to catch the inconsistencies before recommending it.\n\nMy recommendation: send it to peer review with the specific request to verify every percentage against the table and to make the filtering procedure reproducible. The topic is worth a short paper; the current numbers are not.","headline":"Useful descriptive inventory of ResearchGate literature, but the headline percentages do not match the paper's own table.","tokens_in":7143,"tokens_out":6481,"would_cite":false,"duration_ms":54432,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This scoping review counts 159 documents about ResearchGate in 2008-2017 and finds that publications and citations grew sharply after 2015.","keywords":["ResearchGate","academic social networks","bibliometrics","scoping review","Google Scholar","scholarly communication","altmetrics"],"falsifier":"Run the same title-only query in Google Scholar for 2008-2017, and alongside it run a broader topic search covering titles, abstracts, and keywords in Google Scholar plus a second database such as Scopus; if the broader search turns up many documents the title search misses, especially before 2015, the conclusion that interest grew only after 2015 would be overturned.","tokens_in":6278,"feed_emoji":"📚","tokens_out":7215,"duration_ms":69340,"temperature":0.7,"pith_summary":"This paper tries to establish, with a scoping bibliometric review, how much published research on the academic social network ResearchGate appeared in its first ten years, what kinds of documents it produced, and who wrote them. It uses a Google Scholar title search and, after removing citations and duplicates, identifies just 159 genuine documents. The core result is that this literature stayed small until 2015 and then grew: 90 percent of the documents and 80 percent of the citations date from 2015 onward. A sympathetic reader would care because the count gives a first quantitative baseline for a network with millions of members, showing that scholarly attention to ResearchGate itself is recent and concentrated.","feed_headline":"Only 159 papers covered ResearchGate in its first decade","feed_subtitle":"A Google Scholar title sweep shows research on the network stayed scarce until 2015, then grew sharply.","key_machinery":"The machinery is the exact-phrase title query allintitle:researchgate in Google Scholar, restricted to 2008-2017, followed by manual removal of non-documents; this produces the 159-item corpus and all subsequent counts. Its work is to define the population of 'research on ResearchGate' as title-bearing, scholar-indexed documents, which makes the growth and citation trends measurable.","core_discovery":"The central claim is that research on ResearchGate, measured by documents with the word 'ResearchGate' in their titles and retrieved through Google Scholar, was limited but genuine and is now growing. Specifically, the paper reports 159 documents after filtering 386 records; journal papers dominate at 46 percent, followed by conference papers at 19.5 percent; the most productive authors are two Spanish scholars who co-authored all eight of their contributions; the most frequent keywords are 'ResearchGate' and 'Altmetrics'; and more than 90 percent of all citations arrived from 2014 onward. The paper concludes that interest in ResearchGate has grown since 2015, as shown by both publication counts and citation counts.","pith_inferences":["A testable extension would search for 'ResearchGate' in titles plus abstracts and keywords across several bibliographic databases; that would show whether the 159-document count is a floor or an artifact of title-only retrieval.","Because the paper measures 'interest' by publications and citations, an altmetrics-based check using ResearchGate views, RG Score, or social-media mentions could reveal a different or earlier growth curve.","The heavy concentration of Spanish-affiliated authors suggests a possible national research agenda around academic social networks; a collaboration-network map of the 267 authors would test whether this is one community or several isolated groups.","If the same method were applied to rival networks such as Academia.edu, the comparison would show whether the post-2015 surge is specific to ResearchGate or common to academic social networking as a subject."],"forward_implications":["Any 2008-2017 literature review on ResearchGate can be near-exhaustive with a small corpus; the 159-document count bounds the field.","The sharp post-2015 rise means studies of ResearchGate's scholarly impact should weight the 2015-2017 window far more heavily than the early years.","Because most authors are library and information science professionals publishing in English with co-authors from their own institutions, the field is best described as a locally clustered, English-language specialty.","Journal papers increasingly attract citations compared with blogs, conference papers, and other outputs, so formal peer-reviewed venues are becoming the main channel through which ResearchGate scholarship is consumed."],"supporting_citations":[{"why":"Sets the broad context of scholarly social-media use that makes the topic worth measuring.","marker":"[1]"},{"why":"Documents that ResearchGate hosts full-text journal articles, grounding the claim that the platform matters to scholarly communication.","marker":"[6]"},{"why":"Establishes that ResearchGate is dominated by recent articles with higher view counts, giving a baseline against which the paper's growth pattern is interpreted.","marker":"[7]"},{"why":"Supports the choice of Google Scholar by showing ResearchGate is a major source of full-text papers found through it.","marker":"[8]"},{"why":"Explains the RG Score metric and its reputational effects, providing the backdrop for analyzing ResearchGate-focused research.","marker":"[12]"},{"why":"Is the most cited item in the corpus and serves as the citation benchmark in the paper's analysis of highly cited works.","marker":"[14]"}],"fun_headline_variants":["Only 159 papers covered ResearchGate's first decade","159 papers: a decade of scarce ResearchGate research","ResearchGate research stayed rare until 2015, then grew","Ten years of ResearchGate studies: just 159 found","Scant research on ResearchGate until the mid-2010s"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that searching for the exact word 'ResearchGate' in document titles via Google Scholar, then manually removing non-documents, captures the population of research about ResearchGate; if relevant studies omit the word from their titles or sit outside Google Scholar, the counts and growth trend would not represent the field.","fun_headline_variants_meta":{"raw":{"variants":["Only 159 papers covered ResearchGate's first decade","159 papers: a decade of scarce ResearchGate research","ResearchGate research stayed rare until 2015, then grew","Ten years of ResearchGate studies: just 159 found","Scant research on ResearchGate until the mid-2010s"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001331,"raw_usage":{"total_tokens":5428,"prompt_tokens":970,"completion_tokens":4458,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":4376}},"tokens_in":586,"tokens_out":4458,"duration_ms":28526,"temperature":1.0,"reasoning_tokens":4376,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:30:21.999416+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same title-only query in Google Scholar for 2008-2017, and alongside it run a broader topic search covering titles, abstracts, and keywords in Google Scholar plus a second database such as Scopus; if the broader search turns up many documents the title search misses, especially before 2015, the conclusion that interest grew only after 2015 would be overturned.","supporting_citations":[{"cited_title":"Scholarly use of social media and altmetrics: a review of the literature","cited_arxiv_id":null,"evidence_quote":"Sets the broad context of scholarly social-media use that makes the topic worth measuring."}],"review_version":1}