{"id":"2b09a618-0e27-4b4b-bf3b-5126d2ee7327","arxiv_id":"1908.05290","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"An online, queryable catalog compiling IceCube's astrophysical neutrino candidate events with recommended values from multiple analyses.","lead":"IceCube has built an online catalog that collects publicly available astrophysical neutrino candidate events, along with their times, positions, and energies. The catalog lets the multi-messenger community query, visualize, and download these events.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The catalog's core value depends on unvalidated 'recommended' values and a live service that the paper itself describes as still on a free test server; neither is independently checkable from the text.","rationale":"The reader's weakest assumption identifies the same load-bearing point: the catalog's usefulness depends on the correctness of the GCN-derived data and on the 'recommended' values chosen via the default index. My read agrees and adds that the paper's own text explicitly describes the deployment as a test service with only 25 events and future additions still pending, which strengthens the concern that the advertised resource is not yet the stable, comprehensive catalog the abstract implies. The concern is not that the authors are being deceptive; the paper is honest about these limitations. It is rather that the central claim, as stated, cannot be verified from the manuscript alone. There is no versioned data release, no static snapshot of the catalog contents, and no independent check that the default-index values correspond to the most accurate published values. For a catalog paper, this is the kind of reproducibility gap that makes an unconditional accept inappropriate. The paper otherwise contains a clear description of the schema, the web application, and the rationale for the catalog, and there is no internal inconsistency in the technical design. The missing elements are empirical validation and a stable, versioned artifact. These are exactly the conditions under which a CONDITIONAL verdict is appropriate, and my stress-test does not move the verdict. I therefore recommend UNCHANGED: the reader's conditional assessment is the right one, and the concrete test above would settle whether the concern actually lands.","tokens_in":4943,"tokens_out":4428,"duration_ms":43466,"concrete_test":"Download the full CSV from the live URL (https://neutrino-catalog.icecube.aq), or request a static snapshot, and cross-check every event against the GCN Notices and Circulars cited in its detail page. For each event, compare the recommended RA/Dec and uncertainty with the value from the source designated by the default index; if any recommended value differs from the source listed as recommended, or if the live service is unreachable, the curation claim fails. Additionally, if the default-index source is a lower-resolution alert rather than the later circular, that would indicate the 'best information' selection is not reliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the catalog is a reliable community resource: it records multiple published versions of each event and 'highlights the best information' (abstract, Section 2.2). Two conditions must hold for this to be true: (i) the values designated by the 'default index' must actually be the best available, and (ii) the live web service must remain available and stable. The paper does not establish either. Section 2.2 defines the default index only as the value 'we suggest is used by the public,' with no selection criterion, weighting, or validation against, for example, the most precise reconstruction or consistency checks; Figure 4 says merely that the GCN circular 'is set to be recommended result source.' If an event's preferred coordinates or energy are taken from a less accurate source, users who download the CSV will unknowingly propagate those errors. The paper also states in Section 2.2 that the database is hosted on AWS 'for testing purposes' and that the free tier supports no more than 100 concurrent connections, and Section 4 describes other IceCube publications as still to be added. Thus the current service is explicitly a partial test deployment, and the paper contains no versioned data snapshot or export that would let a reader verify the catalog's contents independently. The described tool may be useful, but the paper's own text undercuts the claim that it is already the maintained, 'best information' resource presented in the abstract.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an online catalog of astrophysical neutrino candidate events from IceCube, built on a MongoDB database behind a Firebase-hosted web application. The catalog is designed to store multiple published reconstructions of the same event (currently from GCN Notices and GCN Circulars), to designate a default 'recommended' value for each attribute, and to let users query, visualize, and download the data. The manuscript reports that the database currently contains 25 events with a total size of 53.73 KB and that additional IceCube publications will be added in the future. No physics analysis or new measurements are presented; the deliverable is a community tool.","tokens_in":5262,"tokens_out":3292,"duration_ms":35891,"significance":"If maintained and validated, the catalog would fill a practical niche by relieving external users of the need to collect and reconcile multiple GCN/ICRC versions of the same neutrino event. The paper is transparent about the current partial deployment and makes no overreaching physics claims. Its main value proposition is curation: the 'recommended' values and the record of multiple published versions. Because that curation is not yet validated and because the live service is explicitly on a test server with no versioned snapshot, the utility claim rests on unverified infrastructure and undocumented judgment. The paper would be strengthened by a clear selection rule for recommended values, a data-validation description, and a machine-readable release of the catalog at the time of publication.","major_comments":[{"comment":"The default index is the load-bearing element of the paper's central claim that the catalog 'highlights the best information' (abstract), but the manuscript gives no criterion for choosing it. The text only says that the default index identifies which source's value 'we suggest is used by the public.' Because the home table and CSV download expose these recommended values, the authors should specify the selection rule (for example, the most recent offline reconstruction, the smallest reported angular uncertainty, or an explicit IceCube preference hierarchy) and validate the recommended values against the original GCN Notices/Circulars for a sample of events. Without this, users cannot assess whether the catalog propagates the best available coordinates and energies.","section":"Section 2.2"},{"comment":"The paper states that the database is hosted on AWS 'for testing purposes,' that the free tier supports no more than 100 concurrent connections, and that other IceCube publications 'will be added' in the future. This contradicts the abstract's present-tense framing that 'All studies by IceCube that produce astrophysical candidates will be included in our catalog' and prevents a reader from independently verifying the claimed contents (25 events, 53.73 KB). The authors should clearly state the production status of the service, provide a versioned data snapshot (for instance, a CSV or JSON file in the proceedings or a persistent DOI), and describe the planned long-term hosting and maintenance for the URL.","section":"Sections 2.2, 2.3, and 4"},{"comment":"There is no description of data-integrity checks for the GCN-extracted values or for the 'recommended result source' assignment illustrated in Figure 4. Since the catalog's value depends on not propagating typos, stale positions, or inconsistent reconstruction choices, the authors should document their ingestion and parsing workflow, any automated or manual validation steps, and how errors discovered after ingestion are corrected and versioned. At a minimum, a comparison of a few catalog entries with the corresponding GCN Notices and Circulars would establish that the extraction and indexing procedures are reliable.","section":"Sections 2.1 and 3"}],"minor_comments":[{"comment":"The abstract promises that 'signalness' will be contained in the catalog, but Section 2.2 explicitly states that the signalness field is not yet present; please clarify whether signalness is currently available to users.","section":"Abstract and Section 2.2"},{"comment":"The caption contains a typo: 'communicates bewteen' should be 'communicates between,' and 'possess' should presumably be 'process.'","section":"Figure 1 caption"},{"comment":"There are several typographical errors, including 'mouns' for 'muons,' 'breif' for 'brief,' and 'the the value' for 'the value.' A careful proofread is needed.","section":"Page 2 and Section 2.2"},{"comment":"The text says that '10 features' are collected from GCN notices but never lists them; please add a table or list of the field names, units, and data types, including how missing values are represented.","section":"Section 2.1"},{"comment":"The current catalog contains only realtime alert events, while the introduction also describes HESE, up-going muon, and EHE samples; please state which of these categories are already represented among the 25 events and which are planned.","section":"Section 4"},{"comment":"The screenshots would be more useful if the captions included the date the page was accessed and the URL; the live URL is given only as a footnote, and a stable link in the text would help readers.","section":"Figures 2 and 4"}],"recommendation":"major_revision","confidential_remarks":"This is a proceedings-style tool paper, appropriate for ICRC/PoS, and the authors are commendably candid about the test deployment and the 25-event partial catalog. The main risk is that the phrase 'best information' in the abstract overstates what is currently delivered: the recommended values have no documented selection rule and the live service cannot be independently checked because there is no data snapshot. I recommend major revision rather than rejection because the needed additions—selection criteria, validation, a snapshot, and a status statement—are feasible within the scope of the manuscript. I do not see a citation-pattern or novelty problem beyond the need to state clearly that this is an ongoing project rather than a completed resource."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: it's a straightforward infrastructure paper that does what it says. The authors built a queryable, downloadable catalog of IceCube astrophysical neutrino candidates from public GCN notices and circulars. No new physics, no new data, but the aggregation is genuinely useful for the multi-messenger community. The paper is also honest about its current limits: 25 events, no signalness field yet, and the database is hosted on a free AWS test tier.\n\nThe best part is the design. Storing multiple published values for each event and designating a default index to mark the recommended source is a sensible way to handle the messiness of alert streams. The example of IceCube-190503A is clear: the GCN circular, produced after offline reconstruction, is preferred over the initial notice. That gives the reader a concrete sense of how curation works in practice. The web interface with queries, skymap, and CSV export is a practical tool, and the paper claims nothing beyond what it describes.\n\nSoft spots: the default-index selection has no explicit rule. The paper says the GCN circular is set as the recommended result source, but does not say whether the criterion is latest publication, most precise reconstruction, or a manual editorial choice. That matters because users downloading the CSV are implicitly trusting those recommended values. There is also no versioned data snapshot, so the live service is the only source, and its current free-tier hosting is explicitly temporary. These aren't fatal flaws given the paper's modest scope, but they are real reproducibility gaps. The stress-test note gets this right: the catalog's reliability depends on curation quality and service stability, and neither is fully established in the text.\n\nOne clarification: the skeptic's circularity concern does not land. There is no modeling, fitting, or prediction here, so the circularity burden is nil. The paper is what it is: a catalog announcement, not an analysis.\n\nThis deserves a serious referee, but as a modest proceedings contribution. The referee should ask for an explicit curation rule and a downloadable data snapshot so the content can be verified independently. I would accept conditionally. Worth a reading group if you care about community infrastructure; otherwise a quick skim is enough.","headline":"A modest but useful infrastructure paper: a curated catalog of public IceCube alert data, honest about its limits, with real open questions about curation rules and reproducibility.","tokens_in":5701,"tokens_out":2392,"would_cite":false,"duration_ms":24425,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper builds an online catalog that records every published version of each IceCube astrophysical neutrino candidate and flags the best values for public use.","keywords":["astrophysical neutrinos","IceCube","multi-messenger astronomy","neutrino alerts","GCN","data catalog","real-time alerts","event reconstruction"],"falsifier":"Look up a catalogued event such as IceCube-170922A and compare the catalog's displayed right ascension and declination with the values in its GCN circular and in the latest published HESE event list; if the displayed recommended position differs from the best final reconstruction, the catalog is not actually highlighting the best information.","tokens_in":4701,"feed_emoji":"🔭","tokens_out":9482,"duration_ms":77842,"temperature":0.7,"pith_summary":"This paper presents a web-based catalog that gathers IceCube's publicly released astrophysical neutrino candidates and records every published version of each event, because a single alert can carry different reconstructed positions and energies in a GCN notice, a GCN circular, and later offline analyses. The catalog marks which version IceCube recommends for public use, lets users filter events by type and declination, shows them on an interactive sky map, and allows a CSV download of the displayed data. The paper argues this saves the multi-messenger community the difficult task of collecting and choosing among scattered public alerts. A sympathetic reader cares because follow-up campaigns, from gamma-ray blazar studies to dissecting alert regions, depend on using the best available neutrino coordinates and energies.","feed_headline":"One catalog tracks every version of each IceCube neutrino alert","feed_subtitle":"Pick events by type and declination, view them on a sky map, and download the data for follow-up studies.","key_machinery":"The catalog's data model is the one-to-many event document: each event is a database document, and every measured attribute is an array in which the same index across all arrays points to the same source publication; a per-attribute default index marks the value IceCube recommends for public use. This schema lets a single event carry multiple reconstructions from GCN notices, circulars, and offline analyses, while the front end displays only the recommended values and the detail pages expose the full history.","core_discovery":"The central claim is that a single web-accessible catalog can serve as the registry for IceCube astrophysical neutrino candidates, preserving every publicly released reconstruction of each event and designating one recommended set of values. The catalog organizes each event as one database document whose attributes (time, type, direction, energy, angular uncertainty) are stored as parallel arrays, so that all values from a given source share the same array index, and a marked default index per attribute selects which source's value IceCube suggests the public use. The paper demonstrates this with the 27 April 2016 alert, which appeared in a GCN notice, a refined GCN circular, and subsequent HESE conference presentations: the catalog keeps all versions while displaying the recommended one. At the time of writing, the database held 25 realtime-alert candidates, and the design is intended to absorb future samples from HESE, up-going muon, and EHE analyses as they are published.","pith_inferences":["Because the recommended-value flag is chosen by catalog maintainers with no stated selection rule, an external user cannot distinguish an expert curation choice from an arbitrary one; publishing the rule or score behind each default index would make the recommendation independently checkable.","If the same schema were applied to public alert streams from other neutrino detectors, the catalog pattern could become a general multi-messenger registry, not just an IceCube tool.","The current 25-event sample is too small to test selection effects, but once HESE, up-going muon, and EHE samples are added, the catalog would allow the first systematic check of how likely a realtime alert is to survive offline reanalysis with a changed position or energy.","One can infer that the maintainers are making an implicit claim that the latest offline reconstruction is generally the best; a useful test would be to compare early alert coordinates with final published coordinates across all catalogued events to quantify how much offline refinement matters."],"forward_implications":["Community users can download a CSV of recommended event properties without manually reconciling GCN notices, circulars, and proceedings, lowering the barrier to multi-messenger follow-up studies.","Because the detail pages preserve every published reconstruction, comparisons among analyses become straightforward and inconsistencies in event positions or energies can be spotted by inspection.","The array-based schema allows the same catalog to absorb new event samples and new physical fields, such as signalness, without a database redesign.","Sky-map queries by event type and declination make it practical to assemble custom samples for stacking analyses and target searches.","If the catalog is kept current, it can become the shared reference table that observatories use to trigger follow-up on new alerts and to audit past ones."],"supporting_citations":[{"why":"Defines the three-year high-energy starting event sample whose 37 candidate events are among the catalog's sources.","marker":"[4]"},{"why":"Supplies the latest 7.5-year HESE study with 103 candidate events that the catalog records.","marker":"[5, 6]"},{"why":"Establishes up-going muon events above 200 TeV as likely astrophysical candidates.","marker":"[7]"},{"why":"Provides the eight-year up-going muon sample of 35 candidate events for the catalog.","marker":"[9]"},{"why":"Documents the extremely-high-energy search that found the first PeV neutrino, a source of candidate events.","marker":"[10]"},{"why":"Gives the 9-year EHE search with 2 distinct events included as candidates.","marker":"[12]"},{"why":"Describes the realtime alert system that sends public GCN alerts, the current data source for the catalog.","marker":"[13, 14]"},{"why":"The initial GCN notice for the 27 April 2016 alert, one of the multiple published versions tracked.","marker":"[16]"},{"why":"The refined GCN circular for the same alert, illustrating why a single event needs multiple recorded values.","marker":"[17]"},{"why":"Presents IceCube-170922A, the example event used on the catalog's sky map detail page.","marker":"[21]"}],"fun_headline_variants":["Every IceCube neutrino alert version, one catalog","One catalog, all versions of each IceCube neutrino alert","Catalog covers every version of each IceCube neutrino alert","All IceCube neutrino alerts, every reconstruction, one site","IceCube neutrino catalog keeps every version of each alert"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The catalog is only as reliable as the unstated choice of which published version of each event is recommended, since the paper gives no rule for making that choice.","fun_headline_variants_meta":{"raw":{"variants":["Every IceCube neutrino alert version, one catalog","One catalog, all versions of each IceCube neutrino alert","Catalog covers every version of each IceCube neutrino alert","All IceCube neutrino alerts, every reconstruction, one site","IceCube neutrino catalog keeps every version of each alert"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001036,"raw_usage":{"total_tokens":4321,"prompt_tokens":866,"completion_tokens":3455,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":482,"completion_tokens_details":{"reasoning_tokens":3392}},"tokens_in":482,"tokens_out":3455,"duration_ms":20999,"temperature":1.0,"reasoning_tokens":3392,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:17:51.770874+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Look up a catalogued event such as IceCube-170922A and compare the catalog's displayed right ascension and declination with the values in its GCN circular and in the latest published HESE event list; if the displayed recommended position differs from the best final reconstruction, the catalog is not actually highlighting the best information.","supporting_citations":[{"cited_title":"Aartsen et al., Physical review letters 113 (2014) 101101","cited_arxiv_id":null,"evidence_quote":"Defines the three-year high-energy starting event sample whose 37 candidate events are among the catalog's sources."},{"cited_title":"Aartsen et al., The Astrophysical Journal 833 (2016) 3","cited_arxiv_id":null,"evidence_quote":"Establishes up-going muon events above 200 TeV as likely astrophysical candidates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the extremely-high-energy search that found the first PeV neutrino, a source of candidate events."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the 9-year EHE search with 2 distinct events included as candidates."},{"cited_title":"GCN Notice 67093193_127853","cited_arxiv_id":null,"evidence_quote":"The initial GCN notice for the 27 April 2016 alert, one of the multiple published versions tracked."},{"cited_title":"GCN Circular 19363","cited_arxiv_id":null,"evidence_quote":"The refined GCN circular for the same alert, illustrating why a single event needs multiple recorded values."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Presents IceCube-170922A, the example event used on the catalog's sky map detail page."}],"review_version":1}