Pith. sign in

REVIEW 4 major objections 6 minor 31 references

A two-part RAG system expands each query into four portfolios and attaches a source citation to every factual sentence, making generated answers traceable to evidence.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

MPR-CiteG combines four hand-designed query portfolios with reranking and sentence-level citation grounding; it ranked second in the ScienceON AI Challenge.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection A clear, honest engineering write-up of a second-place RAG challenge system; the orchestration is genuinely useful, but the headline claim about guaranteed factual consistency outruns the evidence. the 4 major comments →

arxiv 2607.22706 v1 pith:KLEIEJ5X submitted 2026-07-20 cs.AI cs.DLcs.IR

MPR-CiteG: Enhancing RAG with Multi-Portfolio Retrieval and Citation-Grounded Generation

Hyewon Lee , Minkyung Song , Junghyun Oh , Seunghoon Han , Sungsu Lim This is my paper
classification cs.AI cs.DLcs.IR
keywords Retrieval-Augmented GenerationQuery expansionCitation groundingSentence-level citationMulti-portfolio retrievalLarge Language ModelsHallucination mitigationRe-ranking
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that retrieval-augmented generation can be made verifiable at low cost by splitting the work into two coordinated modules: a retriever that expands each user query into four complementary portfolios, and a generator that first drafts an answer from top documents, then attaches a source citation to each sentence. The retriever's diversification—query expansion, synonyms, focus balancing, and translation—improves recall and re-ranking precision, while the generator's sentence-level citation grounding is meant to separate verifiable facts from general statements, so hallucinated claims become visible. On a scientific question-answering benchmark, the full system outperforms its ablations and lighter backbones, and the authors report that citation-grounded outputs generally tracked retrieved evidence. The paper also presents one failure case showing the static pipeline can miss multi-faceted query requirements, pointing toward dynamic, iterative retrieval as future work.

Core claim

The central claim is that combining a Multi-Portfolio Retriever with a Citation-Grounded Generation module makes RAG outputs both more accurate and explicitly traceable to sources without expensive fine-tuning or multi-stage verification. MPR expands each query into four Boolean query portfolios—recall-oriented, synonym-robust, focus-balanced, and multilingual—then merges and re-ranks candidates into a compact high-precision context. CiteG generates a document-conditioned draft, splits it into sentences, and uses the LLM to align each factual sentence with a single supporting document, attaching incremental citation markers and a reference block. The authors contend this separation of genera

What carries the argument

The central mechanism is the two-module pipeline. The Multi-Portfolio Retriever generates two sets of keywords (Primary and Expanded) via an instruction-tuned LLM, combines them with Boolean operators into four portfolios (Q, S, F, G), executes them in parallel against a document API, de-duplicates the merged pool (about 270 items), and uses a cross-encoder reranker to select the top 50 (and later top 10) documents. The Citation-Grounded Generation module then produces a structured answer strictly from those documents, segments it into sentences, and runs a citation prompt that asks the model to find the single source document directly supporting each sentence, outputting markers and a citat

Load-bearing premise

The assumption that the average cosine similarity between generated sentences and the title-abstracts of the documents the same model selected as citations actually measures factual grounding is load-bearing; if that similarity can be high while the claim is unsupported, the paper's main evidence for citation quality weakens.

What would settle it

Take the generated answers from the paper's evaluation set and have a human (or an independent fact-extraction model) check each cited sentence against the full text of its cited document. If a non-trivial fraction of sentences have high cosine similarity to their cited title-abstract yet state facts the document does not contain (e.g., fabricated numbers or relationships), the self-consistency metric is not measuring factual grounding. A simpler probe: replace one cited document with a topically similar wrong document, keep the sentence unchanged, and see whether the citation-aligner still se

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • RAG answers become auditable: each factual sentence carries a link a reader can open, and uncited sentences are flagged as general statements by construction.
  • Query diversification into four portfolios can be done with one LLM and one reranker on a single GPU, so the approach is feasible when compute and API budget are tight.
  • The sentence-level citation step works as post-processing on top of any document-conditioned generation, so it can be layered onto existing pipelines.
  • The ablation results imply each portfolio contributes differently: the query-expansion portfolio matters most, while the synonym portfolio adds little quality but real latency.
  • The authors' own error case implies the fixed top-10 evidence pool and static portfolios limit complex questions, motivating retrieval intertwined with reasoning steps.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 'Retrieval Accuracy' metric likely measures self-consistency between the generator's sentences and the citations the same model chose, not independent verification; high scores could occur even if the cited document does not actually contain the claimed fact, so an external human or gold-label check would be a stronger test.
  • If the self-consistency metric were replaced by an independent claim-verification score, the gap between system variants might shrink, since the measured effect may partly reflect the model's own preferences.
  • The framework suggests a general pattern: cheap verification can be achieved by forcing the generator to cite at sentence level, then using the citation structure to let readers or downstream tools spot unsupported claims; this could extend to legal or medical QA where traceability matters more than raw fluency.
  • A testable extension: intentionally corrupt the retrieved pool with a plausible but wrong document and see whether the citation aligner attaches it; this would probe whether the module truly verifies or merely pattern-matches.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. MPR-CiteG is a two-stage RAG framework proposed for the ScienceON AI Challenge: a Multi-Portfolio Retriever (MPR) that expands a query into four Boolean-query portfolios (Q, S, F, G) and re-ranks the merged candidates with a cross-encoder, and a Citation-Grounded Generation (CiteG) module that first drafts an answer from the top-10 documents and then attaches sentence-level citations using the same LLM to select a supporting document for each sentence. The paper reports experiments on the challenge dataset (50 queries) with Qwen2.5-14B as the best backbone, an ablation of the four query portfolios, and two case studies. The central claim is that MPR-CiteG improves retrieval efficiency and guarantees factually consistent, source-attributed outputs, thereby mitigating hallucination.

Significance. If the claims are accepted, MPR-CiteG would provide a practical, low-cost recipe for making RAG outputs verifiable in a public-API, fixed-GPU setting. The paper's strengths include its second-place finish in the ScienceON AI Challenge, the public code repository, a clear system architecture, and a thoughtful analysis of a concrete failure case that identifies a plausible path toward dynamic retrieval. However, the quantitative evidence is not yet load-bearing: the only citation-quality metric is a self-consistency score computed between sentences and documents both produced/selected by the same model, the sample size is only 50 queries with no error bars, and there is no external baseline. The paper's own error case contradicts the absolute 'every output' claim in the abstract. The contribution is therefore best viewed as a system description with interesting design intuitions, not as a validated claim of guaranteed factual consistency.

major comments (4)
  1. [Section 3.1, Retrieval Metric; Appendix A.2] The 'Retrieval Accuracy' metric is the average cosine similarity between each generated citation sentence and the title–abstract of the document that the same Qwen2.5-14B model selected in the sentence-level citation grounding step. Because the Citation Prompt instructs the model to 'find the SINGLE source document that provides a DIRECT and VERIFIABLE origin for the sentence' after the sentence is generated, the sentence–document pair is optimized for semantic similarity by construction. This is a self-consistency score, not an external measure of factual grounding. A hallucinated but topically coherent sentence can receive a high score. The reported value (0.4943) has no baseline, no confidence interval, and n=50. This metric is used in the overall final score, so the central quantitative evidence for 'factual consistency' is not load-bearing in the direction claimed.
  2. [Abstract; Section 2.2; Section 3.4.2] The abstract states that the CiteG module 'ensures that every generated output remains factually consistent and explicitly attributed to its source,' and Section 2.2.2 says the pipeline 'guarantees that the final output is a fluent, document-grounded narrative in which each factual statement can be traced to a verifiable source.' The paper's own Error Case (Table 5) contradicts this: the model generated a fully cited, coherent answer that failed to address the query's structural requirement (the HMM/NN integration strategy), and the text acknowledges that the pipeline 'failed to address the query’s structural requirement.' A guarantee of factual consistency cannot coexist with a demonstrated failure to answer the question. The claims need to be scoped from absolute guarantees to empirical observations, or the evaluation must be strengthened to support the strong wording.
  3. [Section 3.2, Section 3.3] All quantitative comparisons are internal: LLM backbones within the MPR-CiteG framework and ablations of the four query portfolios. There is no comparison against a baseline RAG system (e.g., a simple BM25 or embedding retriever with the same generation module, or a standard single-query pipeline) under identical conditions. Without an external baseline, the claim that MPR 'enhances' RAG is not directly supported by the experiments. The authors should add at least one standard baseline and, ideally, report the official challenge leaderboard context to calibrate the score 0.5415.
  4. [Section 3.1, Tables 2 and 3] The experiments are based on only 50 queries (Execution Time is reported as '≈0.7 h(50 queries)'), and the tables report point estimates without variance, confidence intervals, or significance tests. Differences such as BLEU 0.2254 vs. 0.2210 (Full vs. –S) are well within random variation at this sample size. The authors should either report per-query variability and statistical significance, or clearly frame the results as a preliminary case study rather than as validated effectiveness evidence.
minor comments (6)
  1. [Section 2.1.1] The notation N=3 and M=2 is introduced in the portfolio descriptions but N and M are not defined before their first use. Please define them explicitly alongside the descriptions of the keyword sets.
  2. [Section 2.2.2] Typo: 'multiple sentences referencing the same document are reuse the same marker' should be 'multiple sentences referencing the same document reuse the same marker.'
  3. [Section 3.1, Final Score formula] The formula uses ̃T_exec, but the Efficiency Metric is defined as 'Execution Time.' Please clarify how the normalized execution time is computed, and whether lower values are better (the table shows an arrow 'Texec (s)↓', suggesting lower is better, but the formula's normalized term should be defined explicitly).
  4. [Table 5] The question text contains a typo: 'hidden Marov model' should be 'hidden Markov model.'
  5. [Section 3.4] The case studies are labeled 'Correct case' and 'Incorrect case,' but the incorrect case is a single example. Please either add more examples or present the case study as anecdotal, not as a systematic failure analysis.
  6. [Appendix A.3] Tables 6 and 7 show example outputs for different backbones on a Korean query, but there is no quantitative comparison for the Korean subset. Consider reporting Korean-only BLEU/METEOR/Retrieval Accuracy, or explicitly state that these are illustrative only.

Circularity Check

1 steps flagged

CiteG's only quantitative citation-quality metric is a self-consistency score: the same model writes the sentence, picks its supporting document, and is then scored on cosine similarity between the two.

specific steps
  1. self definitional [Section 3.1 (Retrieval Metric) with Section 2.2.2 and Appendix A.2]
    "during citation alignment, each sentence is evaluated against the candidate documents, and the model selects supporting sources as appropriate. ... Retrieval Metric : To evaluate the alignment between the cited documents and the model ’ s supporting evidence, we compute the average cosine similarity between each citation sentence and its corresponding document ’ s title–abstract pair using a SentenceTransformer model (all-MiniLM-L6-v2) [17] to measure consistent semantic correlation across all generated outputs."

    The document paired with each sentence is not an independent ground truth: it is selected after the sentence is written by the same Qwen2.5-14B model, using a prompt that instructs it to 'find the SINGLE source document that provides a DIRECT and VERIFIABLE origin for the sentence' and even tells it not to be overly strict ('if the information plausibly comes from a source, cite it'). The Retrieval Accuracy therefore measures cosine similarity between two artifacts of the same model, a pair that is constructed to be semantically close by the selection step. A high score shows intra-model self-consistency, not that the document contains or entails the claim. Interpreting it as evidence of factual grounding makes the central 'grounded generation' claim equivalent to the model grading its own

full rationale

The paper is an empirical RAG challenge submission; there is no fitted-parameter derivation whose output equals its input, and no load-bearing self-citation chain. The retrieval component is ablated and evaluated against challenge references through BLEU/METEOR, which are external. The circular step is localized but central to the paper's stated contribution: the only quantitative evidence that CiteG produces factually consistent, source-attributed outputs is the 'Retrieval Accuracy' metric. That metric is computed after the same Qwen2.5-14B model has generated the sentence and then selected its supporting document via the Citation Prompt, which explicitly asks for 'the SINGLE source document that provides a DIRECT and VERIFIABLE origin' and instructs the model not to be overly strict. Cosine similarity between the sentence and the selected title-abstract pair is thus a measure of intra-model semantic self-consistency, not independent verification. The document is chosen to match the sentence, so the score is inflated by construction and cannot distinguish a grounded citation from a fluent, topically coherent confabulation. The paper's own Error Case (Table 5) shows exactly this situation: an internally consistent answer with a single citation that misses the question. Because this circular metric is the only quantitative support for the headline claim about hallucination mitigation, the overall score is 6 rather than 0-2; nevertheless the retrieval pipeline itself and the non-citation quality metrics retain independent empirical content.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The system has no formal derivation; its claims rest on the challenge benchmark, hand-set portfolio sizes and cutoffs, and the reliability of Qwen's expansions and citation selections. No new physical or conceptual entities are introduced, so the invented-entities ledger is empty.

free parameters (5)
  • N (top primary keywords per portfolio) = 3
    Chosen by hand for Q/S/F portfolios; no sensitivity analysis provided (§2.1.1).
  • M (remaining primary keywords in F portfolio) = 2
    Manual design choice; no ablation for different M (§2.1.1).
  • candidate pool size target = ≈270
    Result of merging portfolios; used as coverage target, not derived (§2.1.2).
  • top-documents after reranking = 50
    Fixed cutoff for final context; no analysis of tradeoff (§2.1.2).
  • documents used in generation = 10
    Fixed due to context limit; the paper's own error case attributes a failure to this fixed pool (§2.2.1, §3.4.2).
axioms (4)
  • domain assumption The ScienceON challenge dataset and official metrics are a valid benchmark for RAG trustworthiness.
    Used as sole evaluation; no external benchmark or human verification (§3.1).
  • domain assumption Cosine similarity between a generated sentence and its cited document's title-abstract measures citation faithfulness.
    Defines 'Retrieval Accuracy'; generator and citation selector are the same model, so this is partly self-consistency (§3.1).
  • domain assumption Qwen2.5-14B reliably follows the query-expansion and citation prompts.
    Both retrieval and citation grounding depend on LLM outputs without independent validation (§2.1.1, §2.2.2).
  • domain assumption The 50-query test set is representative of the challenge distribution.
    All quantitative claims are based on 50 queries; no error bars or resampling (§3.1).

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of MPR-CiteG: Enhancing RAG with Multi-Portfolio Retrieval and Citation-Grounded Generation." pith.science (2026). https://pith.science/paper/KLEIEJ5X

@misc{pith2026260722706,
  author       = {Pith},
  title        = {Pith review of: MPR-CiteG: Enhancing RAG with Multi-Portfolio Retrieval and Citation-Grounded Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KLEIEJ5X}},
  note         = {Machine review of arXiv:2607.22706}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper presents the MPR-CiteG framework, which achieved second place in the ScienceON AI Challenge by addressing two fundamental challenges in generative AI: inefficient retrieval and the absence of source verification. We propose a dual-component system, termed MPR-CiteG, in which the Multi-Portfolio Retriever (MPR) efficiently retrieves diverse and relevant information, while the Citation-Grounded Generation (CiteG) module ensures that every generated output remains factually consistent and explicitly attributed to its source. MPR-CiteG represents a significant step toward building more trustworthy and accurate LLMs that are not only capable of generating information but also of grounding their responses in reliable evidence, thereby mitigating common issues like model hallucination. Extensive experiments on the challenge dataset validate the effectiveness and reliability of our approach. Our code is available at https://github.com/2noweyh/MPR-citeG.

Figures

Figures reproduced from arXiv: 2607.22706 by Hyewon Lee, Junghyun Oh, Minkyung Song, Seunghoon Han, Sungsu Lim.

Figure 1
Figure 1. Figure 1: Overall framework consisting of Multi-Portfolio Retriever and Citation-Grounded Generation. 2. Architecture In this section, we present the overall architecture that combines the Multi-Portfolio Retriever (MPR) with the Citation-Grounded Generation (CiteG)to achieve both efficient retrieval and source-grounded generation. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 8 linked inside Pith

  1. [1]

    Naveed, A

    H. Naveed, A. U. Khan, S. Qiu, M. Saqib, S. Anwar, M. Usman, N. Akhtar, N. Barnes, A. Mian, A comprehensive overview of large language models, ACM Trans. Intell. Syst. Technol. 16 (2025). URL: https://doi.org/10.1145/3744746. doi:10.1145/3744746

    doi:10.1145/3744746 2025

  2. [2]

    Huang, W

    L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, et al., A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, ACM Transactions on Information Systems 43 (2025) 1–55

    2025

  3. [3]

    W. Fan, Y. Ding, L. Ning, S. Wang, H. Li, D. Yin, T.-S. Chua, Q. Li, A survey on rag meeting llms: Towards retrieval-augmented large language models, KDD ’24, Association for Computing Ma- chinery, New York, NY, USA, 2024, p. 6491–6501. URL: https://doi.org/10.1145/3637528.3671470. doi:10.1145/3637528.3671470

    arXiv 2024

  4. [4]

    Yesim2000, Scienceon ai (sai) challenge, https://kaggle.com/competitions/sai-challenge, 2025. Kaggle

    2025

  5. [5]

    C.-M. Chan, C. Xu, R. Yuan, H. Luo, W. Xue, Y. Guo, J. Fu, Rq-rag: Learning to refine queries for retrieval augmented generation, arXiv preprint arXiv:2404.00610 (2024)

    Pith/arXiv arXiv 2024

  6. [6]

    Z. Li, J. Wang, Z. Jiang, H. Mao, Z. Chen, J. Du, Y. Zhang, F. Zhang, D. Zhang, Y. Liu, Dmqr-rag: Diverse multi-query rewriting for rag, arXiv preprint arXiv:2411.13154 (2024)

    Pith/arXiv arXiv 2024

  7. [7]

    H. Qian, Y. Fan, J. Guo, R. Zhang, Q. Chen, D. Yin, X. Cheng, Vericite: Towards reliable citations in retrieval-augmented generation via rigorous verification, in: Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP 2025), ACM, 2025. doi:10.1145/3767695.3769505

    arXiv 2025

  8. [8]

    T. Cai, Z. Tan, X. Song, T. Sun, J. Jiang, Y. Xu, Y. Zhang, J. Gu, Forag: Factuality-optimized retrieval augmented generation for web-enhanced long-form question answering, in: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 199–210

    2024

  9. [9]

    Nogueira, K

    R. Nogueira, K. Cho, Passage re-ranking with bert, arXiv preprint arXiv:1901.04085 (2019)

    Pith/arXiv arXiv 1901

  10. [10]

    S. Yao, P. Huang, Z. Liu, Y. Gu, Y. Yan, S. Yu, G. Yu, Llm-qe: Improving query expansion by aligning large language models with ranking preferences, arXiv preprint arXiv:2502.17057 (2025)

    Pith/arXiv arXiv 2025

  11. [11]

    A. Y. Qwen, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al., Qwen2. 5 technical report, arXiv preprint (2024)

    2024

  12. [12]

    M. Pan, W. Xiong, S. Zhou, M. Gao, J. Chen, Llm-based query expansion with gaussian kernel semantic enhancement for dense retrieval, Electronics 14 (2025) 1744

    2025

  13. [13]

    M. R. Costa-Jussà, J. Cross, O. Çelebi, M. Elbayad, K. Heafield, K. Heffernan, E. Kalbassi, J. Lam, D. Licht, J. Maillard, et al., No language left behind: Scaling human-centered machine translation, arXiv preprint arXiv:2207.04672 (2022)

    Pith/arXiv arXiv 2022

  14. [14]

    J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, Z. Liu, M3-embedding: Multi-linguality, multi- functionality, multi-granularity text embeddings through self-knowledge distillation, in: Find- ings of the Association for Computational Linguistics ACL 2024, 2024, pp. 2318–2335

    2024

  15. [15]

    N. F. Liu, T. Zhang, P. Liang, Evaluating verifiability in generative search engines, arXiv preprint arXiv:2304.09848 (2023)

    Pith/arXiv arXiv 2023

  16. [16]

    Ouyang, J

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., Training language models to follow instructions with human feedback, Advances in neural information processing systems 35 (2022) 27730–27744

    2022

  17. [17]

    Reimers, I

    N. Reimers, I. Gurevych, Sentence-BERT: Sentence embeddings using Siamese BERT-networks, in: K. Inui, J. Jiang, V. Ng, X. Wan (Eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Nat- ural Language Processing (EMNLP-IJCNLP), Association for Computational Linguistics, H...

    doi:10.18653/v1/ 2019

  18. [18]

    W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, I. Stoica, Efficient memory management for large language model serving with pagedattention, in: Proceedings of the 29th symposium on operating systems principles, 2023, pp. 611–626

    2023

  19. [19]

    Grattafiori, A

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al., The llama 3 herd of models, arXiv preprint arXiv:2407.21783 (2024)

    Pith/arXiv arXiv 2024

  20. [20]

    URL: https://huggingface.co/ KISTI-KONI/KONI-Llama3-8B-Instruct-20240729

    KISTI, Kisti-koni/ koni-llama3-8b-instruct-20240729 (2024). URL: https://huggingface.co/ KISTI-KONI/KONI-Llama3-8B-Instruct-20240729

    2024

  21. [21]

    Trivedi, N

    H. Trivedi, N. Balasubramanian, T. Khot, A. Sabharwal, Interleaving retrieval with chain-of- thought reasoning for knowledge-intensive multi-step questions, arXiv preprint arXiv:2212.10509 (2022). A. Appendix A.1. MPR Prompts The Query Expansion Prompt instructs the model to extract and expand key search terms from a scientific question, ensuring that the...

    Pith/arXiv arXiv 2022

  22. [22]

    – Rank them in order of importance (most important first )

    Primary keywords (5): Must be terms that appear in the question or very close synonyms. – Rank them in order of importance (most important first ). – Maintain scientific accuracy, use noun phrases, and avoid duplicates

  23. [23]

    Expanded keywords (3): May not appear in the question but must be relevant to the intent, and broaden the scope with higher-level concepts, related phenomena, standard terms, or synonyms

  24. [24]

    Do not add any explanations

    Output must be JSON only. Do not add any explanations. A.2. CiteG Prompts The Answer Prompt guides the model to generate concise and fact-grounded answers based solely on the retrieved documents, maintaining the same language as the user ’s question for clarity and coherence. The instruction template is presented as follows: Answer Prompt Role: You are a ...

  25. [25]

    Read the user ’s question and the retrieved documents carefully

  26. [26]

    Formulate a clear and concise answer using the information present in the documents

  27. [27]

    Generate the entire output

    Language policy: in the SAME language as the user ’s question (English → English, Korean → Korean). Generate the entire output. The Citation Prompt enforces factual grounding by instructing the model to identify the most rel- evant supporting document for each sentence, thereby ensuring clear source-level verifiability and consistency. The instruction tem...

  28. [28]

    Your main goal is to ground the text

    Prioritize finding a citation. Your main goal is to ground the text. You should cite any sentence that contains specific information (names, numbers, key concepts, factual claims ) that can be rea- sonably traced back to a single source document

  29. [29]

    Do not be overly strict; if the information plausibly comes from a source, cite it

    A citation MUST be added if the sentence contains a key factual claim. Do not be overly strict; if the information plausibly comes from a source, cite it

  30. [30]

    Only respond with 0 for sentences that are purely generic, introductory, or transitional and lack specific, verifiable information from the sources. A.3. Comparative Model Performance Analysis Tables 6 and 7 compare the performance of KONI-Llama3.1-8B [20] and the Qwen-2.5 series [11] on Korean queries. Llama3.1-8B [19] and Qwen-2.5-7B were excluded from ...

  31. [31]

    국내 IT 산업의 역량 : 국내 IT 산업은 기술경쟁력을 바탕으로 산업자체를 고도화 하고 다른 산업의 생산성과 효율성을 강화하기 위해 융합과 복합을 촉진하고 차세대 기술과의 결합을 통한 신산업을 창출해야 하는 막대한 과제가 주어져 있습니다. 2. 국내 IT 산업의 역량분석: 국내 IT 산업의 역량은 다른 국가에 비해 상대적으로 우위에 있지만 부문별로 그 차이가 나타나고 있습니다. 특히 글로벌 유수평가 기관의 국내 IT 산업에 대한 평가보고서는 매년 그 변동이 심하여 일관성이 부족합니다. 3. IT 융합 신산업육성 및 선도방안 : 정부와 기업을 중심으로 IT융합을 촉진하기 위한 정 책방향과 전략적 시사점을 제시하였습니다. 이를 위해 ...

    2011

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.