{"id":"f13ac905-5747-4606-a38d-3e4563404066","arxiv_id":"2606.18320","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"TopVenues materializes a versioned, reproducible corpus of 9,925 cybersecurity papers from 11 venues (2017-2026) with 99.86% abstract coverage and tooling for auditable literature reviews and preprint analysis.","lead":"TopVenues is an open-source system that builds a versioned SQLite corpus of cybersecurity papers from DBLP, enriches them with abstracts and BibTeX via APIs, and provides CLI and web access for literature reviews. It enables fixed denominators for reviews and repeatable measurements such as preprint-to-venue tracking.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"DBLP spine plus API enrichment may omit or mis-match papers, altering the 29.2% arXiv rate and 16.5x filter gain","rationale":"The identified concern is identical to the reader's weakest assumption. Because the paper is a tooling substrate whose primary contribution is the reproducible artifact (open code, monotonic snapshot, CLI), the coverage risk is real but does not invalidate the descriptive claims or the value of the fixed-denominator approach; the concrete test above would quantify whether the risk materializes for the reported numbers.","tokens_in":1896,"tokens_out":337,"duration_ms":31599,"concrete_test":"Extract the 2024 USENIX Security papers from the May 2026 SQLite snapshot and compare their count and DOIs against the official USENIX proceedings page; if the snapshot misses >5% of entries or contains mismatches, recompute the arXiv-preprint fraction on the corrected set.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline measurements require (1) every paper from the four top conferences in 2024-2025 to be present in the DBLP-derived corpus and (2) correct identification of which of those papers have arXiv preprints (and which preprints later appear in the venue set). The system uses DBLP as the sole metadata spine plus open APIs and custom extractors; the 250-test suite only checks internal invariants (coverage percentages, BibTeX presence). No external cross-check against publisher proceedings or manual sampling is described, so systematic DBLP gaps for recent security venues or title/author matching errors would directly change the reported statistics.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript presents TopVenues, an open-source system that materializes corpus construction for cybersecurity literature reviews as a versioned research artifact. It declares venue/year scopes, uses DBLP as the metadata spine, enriches records with abstracts and BibTeX via open APIs and publisher extractors, and stores results in a monotonic SQLite snapshot accessible via CLI, web interface, and exports. The May 2026 snapshot contains 9,925 papers from 11 sources (2017-2026) with 99.86% abstract coverage and 99.99% BibTeX coverage; a 250-test suite validates internal invariants, keyword search completes in <31 ms, and the fixed corpus enables measurements including a 29.2% arXiv preprint rate (median 5 months prior) for 2024-2025 papers from four top security conferences plus a 16.5x precision gain at 90% recall from a prior-author-track-record filter.","tokens_in":2015,"tokens_out":455,"duration_ms":35291,"significance":"If the DBLP-based construction and enrichment process accurately reflects the declared scope without material omissions, the work supplies a citable, executable, and auditable substrate that directly addresses the reproducibility problem in cybersecurity literature reviews. The open artifact, monotonic snapshots, fast query performance, and concrete empirical measurements on preprints are concrete strengths that could support more rigorous review protocols.","major_comments":[{"comment":"The description of the 250-test suite (which validates internal invariants such as coverage percentages and BibTeX presence) does not include external cross-validation against publisher proceedings or manual sampling of recent papers. This is load-bearing for the headline empirical claims (29.2% arXiv rate and 16.5x filter gain), because systematic DBLP gaps or title/author matching errors for 2024-2025 conference papers would directly alter those statistics.","section":"validation and empirical measurements sections"}],"minor_comments":[{"comment":"The abstract refers to a 'May 2026 snapshot'; clarifying the exact snapshot date, versioning scheme, and how future snapshots remain monotonic would strengthen reproducibility claims.","section":"Abstract"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback and the recommendation of minor revision. We address the single major comment below and will revise the manuscript accordingly.","responses":[{"response":"We agree that the validation described is internal to the DBLP-derived corpus and its invariants. External cross-validation against publisher proceedings or manual sampling of recent papers is not reported in the current manuscript. This is a valid concern for the reliability of the 29.2% arXiv preprint rate and 16.5x filter gain, as any systematic DBLP omissions or matching errors in 2024-2025 would affect those figures. In the revised manuscript we will add a new subsection under validation that reports the results of manual sampling: we will randomly select and manually verify 100 papers from the four top conferences in 2024-2025 against the corresponding ACM/IEEE/Springer proceedings pages, reporting match rate, any discrepancies, and their impact (if any) on the empirical measurements. We will also add an explicit limitations paragraph noting that DBLP coverage, while high for these venues, is not guaranteed to be exhaustive for the most recent year.","revision_made":"yes","referee_comment":"[validation and empirical measurements sections] The description of the 250-test suite (which validates internal invariants such as coverage percentages and BibTeX presence) does not include external cross-validation against publisher proceedings or manual sampling of recent papers. This is load-bearing for the headline empirical claims (29.2% arXiv rate and 16.5x filter gain), because systematic DBLP gaps or title/author matching errors for 2024-2025 conference papers would directly alter those statistics."}],"tokens_in":1552,"tokens_out":363,"duration_ms":22390,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main contribution is a practical system that declares a venue scope, pulls from DBLP, enriches via APIs, and dumps everything into a monotonic snapshot with a 250-test suite. The May 2026 snapshot claims 99.86% abstract coverage and sub-31 ms searches. It also reports 29.2% of recent top-conference papers appearing on arXiv and a 16.5x precision lift from an author-track-record filter.\n\nThat tooling substrate is new in its specific combination of monotonic storage, validation tests, and public GitHub artifact. The reproducibility angle is handled cleanly for anyone who wants a fixed denominator for literature reviews.\n\nThe soft spot is coverage. The stress-test concern lands: the 29.2% and 16.5x figures require that DBLP contains every paper from the four target venues in 2024-2025 and that the arXiv matching is accurate. The paper describes no external cross-check against publisher proceedings or manual sampling, only internal invariants. If DBLP misses recent security papers or matching fails on titles, those numbers move. The abstract does not claim otherwise.\n\nThis is for people running systematic reviews in cybersecurity who need an auditable starting set rather than ad-hoc searches. A reader who already maintains their own corpus might skip it; someone starting fresh or wanting to cite a reproducible baseline could use the snapshot directly.\n\nIt deserves peer review. The artifact is public, the claims are descriptive and falsifiable, and the reproducibility focus is useful even if the coverage question needs tightening.","headline":"TopVenues ships a working open-source pipeline that builds a versioned SQLite corpus from DBLP for cybersecurity papers, plus some concrete preprint measurements, but the headline numbers rest on unverified DBLP coverage.","tokens_in":2535,"tokens_out":406,"would_cite":false,"duration_ms":17078,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"TopVenues turns shifting publisher data into a fixed, versioned SQLite corpus for cybersecurity literature reviews.","keywords":["cybersecurity","literature review","reproducible corpus","arXiv preprints","DBLP","versioned data","bibliographic metadata","conference papers"],"falsifier":"An independent audit that finds more than a small percentage of papers from the declared venues and years missing from or misclassified in the DBLP-derived corpus would falsify reliable coverage.","tokens_in":2785,"feed_emoji":"📚","tokens_out":769,"duration_ms":25165,"temperature":0.7,"pith_summary":"Cybersecurity literature reviews need a stable denominator of papers before screening begins, yet most current denominators are rebuilt from portals and APIs whose coverage changes. TopVenues addresses this by declaring a venue-year scope, anchoring metadata in DBLP, enriching abstracts and BibTeX entries through open APIs and extractors, and storing the result in a monotonic SQLite snapshot. The May 2026 snapshot holds 9,925 papers across 11 sources from 2017 to 2026 at 99.86 percent abstract coverage. This fixed corpus supports repeatable measurements, including the finding that 29.2 percent of 2024-2025 papers from four top conferences first appear on arXiv with a five-month median lead time. An author track-record filter then raises precision 16.5-fold at 90 percent recall when identifying preprints that later reach those venues.","feed_headline":"Fixed corpus makes cybersecurity literature reviews repeatable","feed_subtitle":"TopVenues builds a versioned SQLite artifact from DBLP that yields 99.86 percent abstract coverage and quantifies 29.2 percent arXiv preprin","key_machinery":"The monotonic SQLite snapshot that serves as the fixed denominator, built from DBLP metadata enriched by open scholarly APIs.","core_discovery":"TopVenues declares a venue and year scope, uses DBLP as the metadata spine, enriches records with abstracts and BibTeX via open APIs and publisher extractors, and stores the results in a monotonic SQLite snapshot that functions as an executable, inspectable, and citable corpus. The approach produces 99.86 percent abstract coverage and 99.99 percent BibTeX coverage on 9,925 papers while enabling the reported preprint statistics as direct, repeatable outputs of the same artifact.","pith_inferences":["The same snapshot approach could be applied to other computer science subfields that maintain stable venue lists.","Longitudinal snapshots would allow tracking of how preprint-to-publication patterns evolve over multiple years.","Linking the corpus to screening software could reduce manual steps in the early stages of a review.","Extending coverage to additional venues would require only updates to the scope declaration rather than rebuilding the entire pipeline."],"forward_implications":["Any review protocol can cite and reuse the exact same corpus snapshot, eliminating reconstruction drift across studies.","Preprint appearance rates, lead times, and author-based triage filters become measurable quantities that can be recomputed on later snapshots.","Keyword search, data-integrity validation, and export to review tools all operate against the same frozen data set.","The corpus itself becomes a citable research artifact rather than an ad-hoc reconstruction.","Precision-recall tradeoffs for preprint triage can be reported against the fixed denominator for direct comparison."],"fun_headline_variants":["TopVenues yields reproducible DBLP corpus for cybersecurity reviews","Versioned SQLite artifact materializes security paper corpus","DBLP-based snapshot achieves high abstract and BibTeX coverage","TopVenues enables repeatable arXiv preprint stats in security venues"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"DBLP combined with the chosen open APIs and extractors captures the intended cybersecurity literature without omissions or errors that would change the coverage or preprint numbers.","fun_headline_variants_meta":{"raw":{"variants":["TopVenues yields reproducible DBLP corpus for cybersecurity reviews","Versioned SQLite artifact materializes security paper corpus","DBLP-based snapshot achieves high abstract and BibTeX coverage","TopVenues enables repeatable arXiv preprint stats in security venues"]},"model":"grok-4.3","cost_usd":0.01022,"raw_usage":{"total_tokens":4523,"prompt_tokens":816,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":102203000,"prompt_tokens_details":{"text_tokens":816,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":3642,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":816,"tokens_out":65,"duration_ms":32268,"temperature":1.0,"reasoning_tokens":3642,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T00:16:27.820189+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An independent audit that finds more than a small percentage of papers from the declared venues and years missing from or misclassified in the DBLP-derived corpus would falsify reliable coverage.","supporting_citations":[],"review_version":1}