pith. sign in

arxiv: 2606.24725 · v1 · pith:WPIY4CH5new · submitted 2026-06-23 · ⚛️ physics.ins-det · physics.comp-ph

A Grounded Evidence-Retrieval Benchmark and Hybrid RAG Framework for Silicon Pixel Detector R&D

Tianqi Gao , Ruobing Jiang , Dawei Fu , Qiang Li , Matthew Kenzie This is my paper

Pith reviewed 2026-06-25 21:55 UTC · model grok-4.3

classification ⚛️ physics.ins-det physics.comp-ph
keywords silicon pixel detectorsevidence retrievalhybrid retrievalgraph-based retrievalretrieval-augmented generationhigh-energy physics instrumentationbenchmark
0
0 comments X

The pith

Hybrid sparse-dense retrieval recovers evidence most reliably from silicon pixel detector literature.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper creates the first benchmark for testing how retrieval systems locate and ground specific evidence in the expanding silicon pixel detector research literature. It evaluates sparse lexical, dense semantic, hybrid, and graph-based methods on two sets of domain queries using manually annotated chunks and source diagnostics. Results indicate that hybrid retrieval excels at precise evidence recovery while graph approaches better support broad literature mapping. This addresses the practical problem that large language models lack access to specialized long-tail technical details and recent experimental results without reliable grounding in primary sources. The work supplies a reproducible framework to build retrieval-augmented tools for detector R&D.

Core claim

A new evidence-grounded retrieval benchmark with chunk-level annotations, source diagnostics, semantic checks, and abstention tests shows that hybrid sparse-dense retrieval delivers the most reliable evidence recovery across detector-domain queries, whereas graph-based methods perform better for literature exploration than for strict evidence ranking.

What carries the argument

The evidence-grounded retrieval benchmark, consisting of manually curated chunk-level evidence annotations together with source-level diagnostics and negative-query tests on two complementary detector query sets.

If this is right

  • Hybrid retrieval should be the default choice for evidence-grounding tasks in silicon pixel detector R&D.
  • Graph-based tools should be reserved for tasks focused on mapping connections across the literature rather than ranking specific evidence.
  • The benchmark supplies a reusable testbed for comparing future retrieval methods in specialized instrumentation domains.
  • Retrieval-augmented systems for high-energy physics can now be built and evaluated against explicit evidence standards.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same annotation and evaluation approach could be adapted to other instrumentation subfields such as calorimeter or tracking detector literature.
  • Combining the hybrid retriever with existing detector simulation codes might allow direct checks of whether retrieved evidence supports new design choices.
  • Extending the benchmark to include time-stamped queries could test how well methods surface the most recent experimental results.

Load-bearing premise

The manually curated chunk-level evidence annotations and source-level diagnostics accurately represent ground truth for the two detector-domain query sets.

What would settle it

A follow-up study that re-annotates the same queries with independent experts and finds that graph-based methods recover more of the key evidence chunks than hybrid retrieval would falsify the central claim.

Figures

Figures reproduced from arXiv: 2606.24725 by Dawei Fu, Matthew Kenzie, Qiang Li, Ruobing Jiang, Tianqi Gao.

Figure 1
Figure 1. Figure 1: Overview of the complete system pipeline. The six stages cover corpus acquisition, document processing, index and knowledge-graph construction, hybrid and graph-guided retrieval, grounded answer generation, and rigorous evaluation and benchmarking. and detector-concept interpretation. The main contributions of this work are: • the first detector-domain benchmark for grounded evidence retrieval in silicon p… view at source ↗
Figure 2
Figure 2. Figure 2: Representative detector-entity subgraph used for graph-guided literature exploration. Nodes are grouped into detector technologies, sensor-physics concepts, readout electronics, performance metrics, and experimental workflows. Edges represent literature-derived co-occurrence and metadata relationships used for contextual expansion and detector-concept discovery. configurations can misrepresent published re… view at source ↗
Figure 3
Figure 3. Figure 3: Strict Hit@5 and MRR for the core and extension benchmarks. Hybrid retrieval provides the strongest strict chunk-level evidence retrieval, while BM25 remains highly competitive because of exact detector terminology. remains highly competitive, reflecting the acronym-rich and terminology-stable nature of detector literature. Dense retrieval performs worse under strict chunk-level evaluation, although its Pa… view at source ↗
Figure 4
Figure 4. Figure 4: Strict Hit@k profiles for the core and extension benchmarks. The hybrid method consistently reaches the highest or joint-highest recall at larger k, while graph-heavy configurations underperform on strict chunk matching across all k [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation of hybrid retrieval components. Removing BM25 or dense retrieval reduces strict Hit@5. The full hybrid configuration provides the strongest performance, with BM25 contributing exact terminology matching and dense retrieval contributing complementary semantic coverage. terminology and well-defined evidence structure of these topics. Lower performance is observed for broader detector-physics and cro… view at source ↗
Figure 6
Figure 6. Figure 6: Retrieval performance as a function of reasoning complexity. Performance generally decreases as queries require increasingly complex evidence synthesis, while hybrid retrieval remains the most stable configuration across all reasoning levels [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Retrieval performance across detector-domain query categories. Different detector topics exhibit different retrieval characteristics, reflecting variations in terminology stability, evidence density, and conceptual complexity. evaluation, their source-level and semantic retrieval scores remain substantially higher. This indicates that graph expansion frequently reaches the correct publication or a semantic… view at source ↗
Figure 8
Figure 8. Figure 8: Effect of graph expansion on strict retrieval performance. While graph traversal increases semantic coverage, excessive expansion dilutes exact evidence ranking and reduces strict chunk-level retrieval accuracy [PITH_FULL_IMAGE:figures/full_fig_p016_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Comparison of strict chunk-level retrieval, source-level retrieval, and semantic soft-gold retrieval. Graph-based retrieval recovers relevant publications and semantically related passages more effectively than exact supporting evidence. from further optimisation of lexical matching. Retrieval-performance trade-off [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Per-query latency decomposition by pipeline stage. Sparse and dense search dominate the retrieval cost, while graph configurations add further graph-expansion and query-decomposition stages. The breakdown shows where the additional latency of graph-based retrieval originates rather than its total cost alone. graph and agentic-graph configurations add distinct graph-expansion and query-decomposition stages… view at source ↗
Figure 11
Figure 11. Figure 11: Trade-off between strict retrieval performance and query latency. Hybrid retrieval achieves the strongest strict Hit@5 while introducing only a modest latency increase relative to BM25. Graph-based retrieval configurations incur additional latency without improving strict evidence-ranking performance. performance claims or unsupported technology comparisons could lead to misleading scientific conclusions … view at source ↗
Figure 12
Figure 12. Figure 12: Graph coverage and abstention behaviour on negative queries. Graph-based retrieval shows higher false-positive rates than lexical and hybrid retrieval because entity expansion can retrieve plausible but unsupported neighbouring evidence. The principal finding is that hybrid sparse–dense retrieval provides the strongest strict evidence￾ranking performance. Hybrid retrieval achieves Hit@5 values of 0.917 on… view at source ↗
read the original abstract

The rapid growth of silicon pixel detector literature has made systematic evidence retrieval a practical bottleneck for detector R&D. Large language models alone are insufficient for this task, as specialised detector knowledge, long-tail technical details, and recent experimental results must be grounded in primary literature. We present the first evidence-grounded retrieval benchmark and a reproducible retrieval framework for silicon pixel detector studies, combining sparse lexical retrieval, dense semantic retrieval, hybrid retrieval, and graph-based literature exploration. The benchmark includes manually curated chunk-level evidence annotations, source-level diagnostics, semantic relevance checks, and negative-query abstention tests across two complementary detector-domain query sets. Systematic evaluation shows that hybrid sparse-dense retrieval provides the most reliable evidence recovery, while graph-based approaches are more effective for literature exploration than strict evidence ranking. These results highlight the importance of evidence-grounded retrieval for accessing long-tail detector knowledge and provide a practical foundation for retrieval-augmented tools supporting silicon detector research and high-energy physics instrumentation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The paper introduces the first evidence-grounded retrieval benchmark for silicon pixel detector R&D literature, including manually curated chunk-level evidence annotations, source-level diagnostics, semantic relevance checks, and negative-query tests across two detector-domain query sets. It evaluates a hybrid RAG framework combining sparse lexical, dense semantic, hybrid, and graph-based retrieval, with systematic evaluation showing hybrid sparse-dense retrieval as most reliable for evidence recovery and graph-based methods better for literature exploration.

Significance. If the benchmark annotations prove robust, the work could establish a practical foundation for retrieval-augmented systems in high-energy physics instrumentation, helping address access to long-tail technical details in a growing specialized literature.

major comments (1)
  1. [Abstract] Abstract: the central claim that hybrid sparse-dense retrieval provides the most reliable evidence recovery depends on the manually curated chunk-level evidence annotations serving as accurate ground truth, yet the abstract supplies no information on curation protocol, annotator count or expertise, inter-annotator agreement, or conflict resolution. This omission is load-bearing because systematic biases in the annotations (e.g., favoring semantic over lexical matches) could artifactually favor the hybrid method over graph-based ranking.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for highlighting the need for greater transparency in the abstract regarding the evidence annotations. We address this point directly below and will incorporate the requested details in the revised manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that hybrid sparse-dense retrieval provides the most reliable evidence recovery depends on the manually curated chunk-level evidence annotations serving as accurate ground truth, yet the abstract supplies no information on curation protocol, annotator count or expertise, inter-annotator agreement, or conflict resolution. This omission is load-bearing because systematic biases in the annotations (e.g., favoring semantic over lexical matches) could artifactually favor the hybrid method over graph-based ranking.

    Authors: We agree that the abstract should provide a concise summary of the annotation process to support the central claim. The full manuscript (Section 3.2) details the curation protocol, which was performed by two domain experts in silicon pixel detector instrumentation. Inter-annotator agreement was quantified and conflicts resolved through discussion. We will revise the abstract to include a brief statement on the curation protocol, annotator expertise, and agreement metric. On the potential for bias, the annotations consist of explicit chunk-level evidence spans tied directly to query requirements rather than retrieval-method preferences; the benchmark further incorporates source-level diagnostics, semantic relevance checks, and negative-query abstention tests to validate robustness across retrieval paradigms. These safeguards make systematic favoritism toward hybrid retrieval unlikely. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical benchmark evaluation with independent annotations

full rationale

The paper contains no equations, derivations, fitted parameters, or self-citations that reduce claims to inputs by construction. The central evaluation compares retrieval methods against manually curated chunk-level annotations presented as ground truth; these annotations are external inputs to the benchmark rather than outputs derived from the tested methods. No self-definitional loops, fitted-input predictions, or load-bearing self-citations appear. This is the standard non-circular structure of an empirical retrieval benchmark paper.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only; no mathematical content, free parameters, or invented entities described. Relies on standard assumptions in information retrieval and domain knowledge of silicon pixel detectors.

pith-pipeline@v0.9.1-grok · 5707 in / 1055 out tokens · 20955 ms · 2026-06-25T21:55:43.918180+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 18 canonical work pages

  1. [2]

    W. Snoeys. CMOS monolithic active pixel sensors for high energy physics.Nuclear Instruments and Methods in Physics Research Section A, 765:167–171, 2014. doi: 10.1016/j. nima.2014.05.070

    work page doi:10.1016/j 2014

  2. [3]

    Pernegger et al

    H. Pernegger et al. First tests of a novel radiation hard CMOS sensor process for depleted monolithic active pixel sensors.Journal of Instrumentation, 12:P06008, 2017. doi: 10.1088/ 1748-0221/12/06/P06008

    2017

  3. [4]

    Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung. Survey of hallucination in natural language generation.ACM computing surveys, 55(12): 1–38, 2023

    2023

  4. [5]

    Lewis, E

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. K¨ uttler, M. Lewis, W.-t. Yih, T. Rockt¨ aschel, S. Riedel, and D. Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. InAdvances in Neural Information Processing Systems, volume 33, pages 9459–9474, 2020

    2020

  5. [6]

    Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang. Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997, 2023

    Pith/arXiv arXiv 2023

  6. [7]

    When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories

    A. Mallen, A. Asai, V. Zhong, R. Das, D. Khashabi, and H. Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 9802–9822, 2023. doi: 10.18653/v1/2023.acl-long.546

    work page doi:10.18653/v1/2023.acl-long.546 2023

  7. [8]

    Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks

    N. Reimers and I. Gurevych. Sentence-BERT: Sentence embeddings using siamese BERT- networks. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3982–3992, 2019. doi: 10.18653/v1/D19-1410

    work page doi:10.18653/v1/d19-1410 2019

  8. [9]

    Dense Passage Retrieval for Open-Domain Question Answering

    V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih. Dense passage retrieval for open-domain question answering. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6769–6781, 2020. doi: 10.18653/v1/2020.emnlp-main.550. 20

    work page doi:10.18653/v1/2020.emnlp-main.550 2020

  9. [10]

    Thakur, N

    N. Thakur, N. Reimers, A. R¨ uckl´ e, A. Srivastava, and I. Gurevych. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. InAdvances in Neural Information Processing Systems: Datasets and Benchmarks, 2021

    2021

  10. [11]

    The Probabilistic Relevance Framework: BM25 and Beyond,

    S. Robertson and H. Zaragoza. The probabilistic relevance framework: BM25 and beyond.Foundations and Trends in Information Retrieval, 3(4):333–389, 2009. doi: 10.1561/1500000019

    work page doi:10.1561/1500000019 2009

  11. [12]

    C. D. Manning, P. Raghavan, and H. Sch¨ utze.Introduction to Information Retrieval. Cambridge University Press, Cambridge, 2008. doi: 10.1017/CBO9780511809071

    work page doi:10.1017/cbo9780511809071 2008

  12. [13]

    G. V. Cormack, C. L. A. Clarke, and S. Buettcher. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. InProceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 758–759, 2009. doi: 10.1145/1571941.1572114

    work page doi:10.1145/1571941.1572114 2009

  13. [14]

    Lin and X

    J. Lin and X. Ma. A few brief notes on DeepImpact, COIL, and a conceptual framework for learned sparse retrieval.arXiv preprint arXiv:2106.14807, 2021

    arXiv 2021

  14. [15]

    D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, and J. Larson. From local to global: A graph RAG approach to query-focused summarization.arXiv preprint arXiv:2404.16130, 2024

    Pith/arXiv arXiv 2024

  15. [16]

    Z. Guo, L. Xia, Y. Yu, T. Ao, and C. Huang. LightRAG: Simple and fast retrieval-augmented generation.arXiv preprint arXiv:2410.05779, 2024

    Pith/arXiv arXiv 2024

  16. [17]

    Yasunaga, H

    M. Yasunaga, H. Ren, A. Bosselut, P. Liang, and J. Leskovec. QA-GNN: Reasoning with language models and knowledge graphs for question answering. InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics, pages 535–546, 2021. doi: 10.18653/v1/2021.naacl-main.45

    work page doi:10.18653/v1/2021.naacl-main.45 2021

  17. [18]

    Xiong, X

    W. Xiong, X. L. Li, S. Iyer, J. Du, P. Lewis, W. Y. Wang, Y. Mehdad, W.-t. Yih, S. Riedel, D. Kiela, and B. Oguz. Answering complex open-domain questions with multi-hop dense retrieval. InInternational Conference on Learning Representations, 2021

    2021

  18. [19]

    Rui Xu, Xintao Wang, Jiangjie Chen, Siyu Yuan, Xinfeng Yuan, Jiaqing Liang, Zulong Chen, Xiaoqing Dong, and Yanghua Xiao

    G. Izacard and E. Grave. Leveraging passage retrieval with generative models for open domain question answering. InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 874–880, 2021. doi: 10.18653/v1/ 2021.eacl-main.74

    work page doi:10.18653/v1/ 2021

  19. [20]

    Jiang, D

    R. Jiang, D. Fu, C. Jiang, T. Yang, Z. Wang, Y. Wu, Y. Ban, Y. Mao, and Q. Li. Agentic Hybrid RAG for Evidence-Grounded Muon Collider Analysis. 6 2026

    2026

  20. [21]

    Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =

    O. Khattab and M. Zaharia. ColBERT: Efficient and effective passage search via contextu- alized late interaction over BERT. InProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 39–48, 2020. doi: 10.1145/3397271.3401075

    work page doi:10.1145/3397271.3401075 2020

  21. [22]

    Conneau, K

    D. Wadden, S. Lin, K. Lo, L. L. Wang, M. van Zuylen, A. Cohan, and H. Hajishirzi. Fact or fiction: Verifying scientific claims. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 7534–7550, 2020. doi: 10.18653/v1/2020. emnlp-main.609

    work page doi:10.18653/v1/2020 2020

  22. [23]

    SPECTER : Document-level Representation Learning using Citation-informed Transformers

    A. Cohan, S. Feldman, I. Beltagy, D. Downey, and D. S. Weld. SPECTER: Document-level representation learning using citation-informed transformers. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2270–2282, 2020. doi: 10.18653/v1/2020.acl-main.207. 21

    work page doi:10.18653/v1/2020.acl-main.207 2020

  23. [24]

    Tsatsaronis et al

    G. Tsatsaronis et al. An overview of the BioASQ large-scale biomedical semantic indexing and question answering competition.BMC Bioinformatics, 16:138, 2015. doi: 10.1186/ s12859-015-0564-6

    2015

  24. [25]

    P. Lopez. GROBID: Combining automatic bibliographic data recognition and term extraction for scholarship publications. InResearch and Advanced Technology for Digital Libraries, pages 473–474, 2009. doi: 10.1007/978-3-642-04346-8 62

    work page doi:10.1007/978-3-642-04346-8 2009

  25. [26]

    Billion-Scale Similarity Search with GPUs ,

    J. Johnson, M. Douze, and H. J´ egou. Billion-scale similarity search with GPUs.IEEE Transactions on Big Data, 7(3):535–547, 2021. doi: 10.1109/TBDATA.2019.2921572

    work page doi:10.1109/tbdata.2019.2921572 2021

  26. [27]

    M. Mager. ALPIDE, the monolithic active pixel sensor for the ALICE ITS upgrade. Nuclear Instruments and Methods in Physics Research Section A, 824:434–438, 2016. doi: 10.1016/j.nima.2015.09.057

    work page doi:10.1016/j.nima.2015.09.057 2016

  27. [28]

    Poikela, J

    T. Poikela, J. Plosila, T. Westerlund, M. Campbell, M. De Gaspari, X. Llopart, V. Gromov, R. Kluit, M. van Beuzekom, F. Zappon, et al. Timepix3: A 65k channel hybrid pixel readout chip with simultaneous ToA/ToT and sparse readout.Journal of Instrumentation, 9:C05013,

  28. [29]

    doi: 10.1088/1748-0221/9/05/C05013

    work page doi:10.1088/1748-0221/9/05/c05013

  29. [30]

    Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment , author =

    S. Spannagel et al. Allpix squared: A modular simulation framework for silicon detectors. Nuclear Instruments and Methods in Physics Research Section A, 901:164–172, 2018. doi: 10.1016/j.nima.2018.06.020

    work page doi:10.1016/j.nima.2018.06.020 2018

  30. [31]

    Nuclear Instruments and Methods in Physics Research A506(3), 250–303 (2003) https://doi.org/10.1016/S0168-9002(03)01368-8

    S. Agostinelli et al. GEANT4: A simulation toolkit.Nuclear Instruments and Methods in Physics Research Section A, 506(3):250–303, 2003. doi: 10.1016/S0168-9002(03)01368-8. Appendix A Grounding and Abstention Prompt The grounded generation component receives the user query together with the retrieved evidence passages and associated source metadata. The mo...

    work page doi:10.1016/s0168-9002(03)01368-8 2003

  31. [32]

    Do not use external knowledge

  32. [33]

    Cite supporting passages when answering

  33. [34]

    If the evidence is insufficient, return ABSTAIN

  34. [35]

    Time resolution of irradiated LGAD sensors

    Do not infer detector performance beyond what is explicitly stated. Question: {query} Retrieved Evidence: 22 {retrieved_chunks} Answer: For negative-query evaluation, the model is expected to return the predefined token ABSTAIN when no retrieved passage directly supports the queried detector-domain claim. B Retrieval Metrics The primary evaluation metrics...