{"id":"7d88c3bc-5053-4da3-a831-7896a161204d","arxiv_id":"2607.28618","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Claim-centered retrieval over 2.4M grounded chemistry claims yields 100% resolvable DOIs and higher citation density than document-only baselines on a 30-question cross-paper bench.","lead":"AskChem turns chemistry papers into 2.4M provenance-linked claims and searches those claims instead of whole documents. It is a live tool that cuts citation hallucination for AI readers and gives chemists claim-level browse, graph, and agent APIs.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Bench proves citation resolvability, not that claim extractions are semantically faithful synthesis units.","rationale":"The reader’s weakest assumption is exactly the load-bearing gap: provenance schema + hybrid retrieval is treated as enough to use claims as synthesis units, despite the paper’s own RQ1 caveat and missing semantic/taxonomy ablations. The engineering facts (live index, APIs, 100% DOI existence vs 88.3% parametric) are solid and support a conditional accept for an infrastructure paper. They do not, however, license reading Table 1 as evidence that claim-centered retrieval improves chemical correctness of cross-paper answers—only that citations resolve and are denser. A closed agent still dominates quantitative specificity. No change to CONDITIONAL is warranted; the concrete expert audit above is the natural gate before treating the store as a drop-in truth layer. Novelty and systems impact remain as the reader scored them.","tokens_in":11233,"tokens_out":614,"duration_ms":23865,"concrete_test":"Take the 30 AskChem-Bench answers under +AskChem. For each answer, sample up to 5 cited claims (or all if fewer) that contribute quantitative outcomes. Have a domain chemist score each claim against its verbatim_quote/source as correct / partial / wrong on the structured fields used in the answer (value, units, conditions, reactants/products). If >10–15% of cited quantitative claims are wrong or partial, the synthesis-improvement claim weakens to “reduces DOI hallucination” only.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim rests on AskChem-Bench (Table 1; Fig. 6): grounding GPT-5.5 in the claim store yields 100% CrossRef-resolvable DOIs and the highest citation density (18.1). That result is real for provenance plumbing, but it does not establish that the retrieved units are reliable for synthesis. RQ1 and the Limitations section state explicitly that schema checks (claim type + DOI + verbatim quote/locator) “do not prove that every extracted claim semantically interprets its source correctly.” Hybrid retrieval can therefore surface and cite many resolvable DOIs whose structured fields (outcomes, Faradaic efficiencies, conditions, reactants) misread the quote or abstract. Fig. 6 only contrasts fabricated vs. resolvable DOIs; it does not audit whether the reported values match the sources. Paperclip also hits 100% DOI existence, so the jump from the no-retrieval baseline is largely “any grounded RAG vs parametric memory,” not a unique demonstration that claim-level segmentation improves answer correctness. Edison’s much higher grounded specificity (29.2 vs 5.9) further shows the bench’s headline metrics under-weight chemical fidelity. Without semantic claim audits tied to the same answers, “improves cross-paper chemistry synthesis” overclaims what DOI existence and density measure.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"AskChem proposes claim-centered infrastructure for chemistry literature search and synthesis: papers are segmented into atomic, typed claims grounded by DOI plus a verbatim quote or evidence locator, then indexed (2.4M claims / 147K papers) under a stabilized faceted taxonomy, an evidence graph (supports/contradicts/extends/etc.), and an exploratory principle-centered Living Taxonomy. The system is deployed with web, REST, SDK, and MCP access. Evaluation covers provenance checks (100% source-grounded claims), a small expert audit of graph edges (97.9% type precision on 146 decidable edges), corpus-scale operation, and AskChem-Bench (30 cross-paper questions) where a GPT-5.5 reader grounded in AskChem achieves 100% CrossRef-resolvable DOIs (vs 88.3% without retrieval) and the highest citation density among five systems (Table 1; Fig. 6).","tokens_in":11622,"tokens_out":1369,"duration_ms":34388,"significance":"If the result holds under tighter semantic validation, the work is a useful systems contribution: it shifts retrieval from ranked papers to reusable, provenance-carrying claims and ships a live, agent-accessible store at meaningful chemistry scale. Strengths that should be credited include the deployed open index and interfaces, explicit provenance fields on every claim, CrossRef-verified DOI metrics, judge calibration against expert labels (κ=0.914), a domain-expert edge audit, and public code/data/benchmark artifacts. The paper is strongest as infrastructure for grounded retrieval and citation hygiene; its significance for end-to-end chemical synthesis quality is currently less established than the DOI-groundedness result.","major_comments":[{"comment":"Abstract, §7 RQ3, Table 1, and Fig. 6: the headline claim that claim-centered retrieval “improves cross-paper chemistry synthesis” is mainly supported by DOI existence (100%) and citation density (18.1). Those metrics establish citation plumbing, not that structured claim fields (outcomes, conditions, reactants) faithfully interpret sources. RQ1 and Limitations already state schema checks “do not prove… semantic… correctness.” Fig. 6 only contrasts fabricated vs resolvable DOIs. Please add a semantic fidelity audit on the claims actually used in bench answers (value/condition match to quote or full text), or narrow the claim to citation groundedness/density.","section":"§7 RQ3; Table 1; Fig. 6"},{"comment":"Table 1: Paperclip also reaches 100% DOI existence, so the jump from the no-retrieval baseline largely shows grounded RAG vs parametric memory, not a unique benefit of claim segmentation. Edison Scientific substantially outperforms AskChem on grounded specificity (29.2 vs 5.9) and slightly on on-topic rate. Without a paper-level RAG control using the same reader/budget, or an ablation that disables claim structure/taxonomy/graph while keeping the corpus, the causal role of the claim unit remains under-identified. Add such a control or revise the comparative claim.","section":"Table 1; §7 RQ3; Appendix A"},{"comment":"§4 and RQ2: the stabilized faceted taxonomy is described as a retrieval signal in hybrid search, yet the paper reports neither isolation of taxonomy-recall gain nor expert validation of path placement. Living Taxonomy (§5) is explicitly exploratory and unvalidated. These structures are load-bearing in the architecture diagram and contributions list; either quantify their contribution (ablation on AskChem-Bench or retrieval metrics) or move them to clearly optional/exploratory status in the abstract and contribution bullets.","section":"§4; §5; §7 RQ2"},{"comment":"§3 evidence-graph audit: 97.9% edge-type precision on 146 decidable edges from a stratified sample of 148 is encouraging but thin for a core navigational layer (171K edges). Report sampling fractions by relation type, confidence calibration, and error modes (especially contradicts), and state whether graph edges were used in the AskChem-Bench grounded reader or only in UI neighborhood views.","section":"§3; Appendix B"}],"minor_comments":[{"comment":"Model names such as GPT-5.5 and Gemini 3.1 Pro should be pinned to provider snapshot/version dates for reproducibility; Appendix A protocol is otherwise helpful.","section":"Appendix A"},{"comment":"Figure 3 mixes corpus statistics with quality claims; clarify that claim-type and subfield histograms are automatic labels, not expert-coded distributions.","section":"Figure 3"},{"comment":"Table 1 “Grounded specificity” definition (Appendix A) is token-level and favors long quantitative dumps; discuss this bias when interpreting Edison’s large lead.","section":"Table 1; Appendix A"},{"comment":"Limitations correctly notes shallow abstract extraction and partial chemistry coverage; surface the 66%/34% full-paper vs abstract split earlier when discussing claim completeness for conditions/limitations.","section":"Figure 3; Limitations"},{"comment":"Minor polish: spacing in “AskChemaddresses” / “CO 2” and consistent capitalization of Living Taxonomy vs faceted taxonomy.","section":"§1; §5"}],"recommendation":"major_revision","confidential_remarks":"Fit is reasonable for a systems/resources track in CL or AI-for-science venues; novelty is primarily integration and deployment rather than new extraction theory. I would not reject on novelty alone if the authors tighten evaluation claims. The skeptic note is largely correct that DOI metrics ≠ semantic synthesis quality; that should be fixed in revision rather than treated as fatal. No integrity red flags from the manuscript text alone; live URL and released artifacts are a plus if they match the described index."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful fact is simple: they actually built and shipped a claim-centered chemistry index—2.4M provenance-carrying claims from 147K papers, live UI plus REST/SDK/MCP, open code and a CC-BY snapshot—and showed that grounding a reader in it kills DOI fabrication (100% CrossRef-resolvable vs 88.3% parametric; highest citation density among the five systems).\n\nWhat is new is not claim extraction, taxonomy induction, or RRF in isolation. It is the integrated retrieval unit plus the complementary structures over one shared claim identity (stabilized facets for browse/search, a typed evidence graph, exploratory living taxonomy), exposed as agent infrastructure rather than another answer chatbot. The engineering is concrete: dual extractors, schema gates, 99.9% DOI-verified sources, hybrid search, and a small expert edge audit at 97.9% type precision on 146 decidable edges. Fig. 6 is a fair contrast of fabricated vs resolvable citations. Artifacts and the live service make this reproducible in the systems sense.\n\nThe soft spot is real but already half-admitted. Schema + quote/locator establishes traceability, not that structured fields (efficiencies, conditions, reactants) correctly interpret the source. The paper says this in RQ1 and Limitations. AskChem-Bench is only 30 questions; Paperclip also hits 100% DOI existence, so part of the jump is “any grounded retrieval vs memory.” Edison still wins hard on grounded specificity (29.2 vs 5.9), which is the chemical-detail axis. They do not isolate taxonomy-recall gain or validate living-taxonomy placement. None of that sinks the infrastructure claim; it means “improves synthesis” should be read as improves citation groundedness and density, not as audited claim semantics.\n\nCitations look appropriate (SciIE, taxonomy induction, RAG/RRF, chemistry DBs, agent baselines). No circular math story.\n\nThis is for people building scientific IR, chemistry informatics, and agent tooling who care about verifiable literature assembly. Experimentalists get a usable browse/search surface today. I would send it to referees: accept-shaped systems paper that should tighten semantic claim audits and ablations, not desk-reject. Worth engaging; I would cite the deployed claim-store framing and the groundedness numbers when discussing citation-faithful chemistry agents.","headline":"Shipped claim-level chemistry IR with real provenance plumbing and a clean DOI-hallucination win; the bench proves resolvability more than semantic fidelity of the extracted claims.","tokens_in":12269,"tokens_out":577,"would_cite":true,"duration_ms":11082,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"AskChem replaces paper-ranked search with provenance-carrying claims so chemists and agents can assemble verifiable cross-paper answers.","keywords":["claim-centered retrieval","chemistry literature synthesis","provenance grounding","faceted taxonomy","evidence graph","scientific RAG","agent tools","citation groundedness"],"falsifier":"Run the same AskChem-Bench cross-paper questions with human expert audit of whether each cited claim’s meaning matches its source quote, and check whether DOI resolvability and answer quality still beat document-only retrieval when semantic claim errors are counted against the system.","tokens_in":12076,"feed_emoji":"⚗️","tokens_out":789,"duration_ms":19907,"temperature":0.7,"pith_summary":"Chemistry answers are often scattered as specific findings across many papers, but search tools still return ranked document lists that force people and AI agents to hunt, check, and stitch evidence by hand. This paper argues that the right retrieval unit is the atomic, typed scientific claim, each tied to a source DOI and a verbatim quote or explicit evidence locator. Over a shared store of such claims, AskChem adds three complementary structures: a stabilized faceted taxonomy for hierarchical search and browse, an evidence graph that links claims by relations such as supports or contradicts, and an exploratory living taxonomy that places papers under governing principles. The live system indexes millions of claims from a large chemistry corpus and exposes the same objects through a web UI and agent APIs. On a cross-paper benchmark, grounding a language-model reader in AskChem raises resolvable citations to 100% and yields the highest citation density among the systems compared.","feed_headline":"Chemistry search that retrieves claims, not just papers","feed_subtitle":"Grounded claim store lifts resolvable citations to 100% and densest verified answers on a cross-paper bench.","key_machinery":"The provenance-carrying claim: an atomic, typed assertion extracted from a paper, always grounded by a source DOI plus a verbatim quote or evidence locator, and shared as the identity across faceted taxonomy paths, evidence-graph edges, and agent-facing APIs.","core_discovery":"Making the provenance-carrying claim the central object of chemistry literature infrastructure—and grounding a reader in that claim store—improves cross-paper synthesis: on AskChem-Bench it produces fully resolvable DOIs and the densest verified citation set among five tested settings, while remaining usable as interactive search and as an agent tool.","pith_inferences":["Claim-level stores may become a standard substrate for scientific RAG, reducing citation hallucination more than paper-rankers alone.","If semantic claim validation lags provenance checks, community flagging and expert audits will matter as much as scale for high-stakes use.","The design generalizes beyond chemistry wherever answers must be assembled from many scattered measurements and conditions."],"forward_implications":["Cross-paper chemistry questions can be answered from claim-level evidence with citations that resolve to real papers rather than fabricated DOIs.","The same claim objects can serve both human browsing and agent tools without rebuilding separate retrieval stacks.","Supports, extends, and contradicts links let users move from one finding to related or conflicting evidence across papers.","Faceted and principle-centered views give operational and exploratory organization over one shared claim store at corpus scale."],"fun_headline_variants":["AskChem makes claims the unit of chemistry search","Provenance-carrying claims replace paper lists in chem synthesis","Claim store grounds GPT reader to 100% resolvable DOIs","Cross-paper chem answers from 2.4M typed, sourced claims","Evidence graph and faceted claims for chemistry literature"],"cache_read_input_tokens":0,"weakest_assumption_plain":"That schema checks and source grounding are enough to treat extracted claims as reliable synthesis units, even though they do not prove every claim correctly interprets its source.","fun_headline_variants_meta":{"raw":{"variants":["AskChem makes claims the unit of chemistry search","Provenance-carrying claims replace paper lists in chem synthesis","Claim store grounds GPT reader to 100% resolvable DOIs","Cross-paper chem answers from 2.4M typed, sourced claims","Evidence graph and faceted claims for chemistry literature"]},"model":"grok-4.5","effort":"low","cost_usd":0.002083,"raw_usage":{"total_tokens":929,"prompt_tokens":764,"num_sources_used":0,"completion_tokens":66,"cost_in_usd_ticks":20828000,"prompt_tokens_details":{"text_tokens":764,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":99,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":764,"tokens_out":66,"duration_ms":3557,"temperature":1.0,"reasoning_tokens":99,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T02:00:04.980770+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the same AskChem-Bench cross-paper questions with human expert audit of whether each cited claim’s meaning matches its source quote, and check whether DOI resolvability and answer quality still beat document-only retrieval when semantic claim errors are counted against the system.","supporting_citations":[],"review_version":1}