To Retrieve or Not to Retrieve? Uncertainty Detection for Dynamic Retrieval Augmented Generation

Kaustubh D. Dhole

REVIEW 4 major objections 4 minor 2 cited by

To Retrieve or Not to Retrieve? Uncertainty Detection for Dynamic Retrieval Augmented Generation

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper demonstrates that uncertainty detection on a temporary sentence can cut retrieval calls by nearly half while losing only a small amount of answer accuracy.

desk verdict Plausible but unproven: the uncertainty-triggered retrieval savings look real, but the missing random baseline and 75-example eval keep the headline claim from landing. read the letter →

arxiv 2501.09292 v3 pith:INXENQKM submitted 2025-01-16 cs.CL cs.AIcs.IR

Kaustubh D. Dhole This is my paper · ORCID

classification cs.CLcs.AIcs.IR

keywords dynamicretrievalretrieval-augmentedgenerationuncertaintydetectionlong-formquestionansweringmulti-hopEccentricityDegreeMatrixJaccardconditional

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether a language model can decide on its own when it needs to look something up while writing a long answer. It proposes generating a temporary next sentence, measuring how uncertain the model is about that sentence with black-box uncertainty metrics, and calling a retriever only when uncertainty exceeds a threshold. On the 2WikiMultihopQA benchmark this conditional scheme cut retrieval calls from about 4.6 to about 2.2 per question when driven by Eccentricity, with average F1 dropping from 0.581 to 0.561. A cheaper metric, Degree Matrix Jaccard, used even fewer searches (about 1.8 per question) at a slightly larger accuracy cost. The result matters because retrieval is often the expensive part of retrieval-augmented generation, and knowing when to skip it could make such systems cheaper without much loss.

What carries the argument

The load-bearing mechanism is the trigger rule of Equation (2): generate a temporary sentence without retrieval, compute a black-box uncertainty score U(t_i) for that sentence, and fetch documents only when U(t_i) exceeds a threshold θU; otherwise continue generating from the current context. The uncertainty scores are computed from pairwise similarity structures over several sampled continuations—semantic sets, degree-matrix Jaccard or NLI similarity, eigenvalue-Laplacian spectral clustering, and Eccentricity, a graph-based score of how far a sampled response sits from the rest of the response cluster. The threshold converts a continuous uncertainty estimate into a binary retrieval decision, and the retrieved documents are appended to the context before the next sentence is generated.

What would settle it

On a held-out set of questions with thresholds frozen from the seed set, compute whether sentences the trigger marks as high-uncertainty actually improve in quality when retrieved passages are appended; if high-uncertainty sentences do not benefit from retrieval, or if the search count does not stay near half of always-retrieve on the larger set, the central claim is refuted.

Watch

Extended reading notes

Core claim

The central discovery is that a sentence-level uncertainty trigger can replace a deterministic always-retrieve policy in long-form question answering. The paper reports that Eccentricity with threshold U>2 yields about 2.2 searches per question versus 4.6 for always-retrieve while keeping average F1 at 0.561 versus 0.581 on 75 examples, and that Degree Matrix Jaccard uses even fewer searches (about 1.8) at a larger F1 cost. Across three runs, Eccentricity consistently gave the best balance between retrieval efficiency and answer accuracy among the uncertainty estimators tested.

Load-bearing premise

The argument stands or falls on the assumption that the uncertainty of a temporary next sentence tracks whether retrieval would actually supply missing knowledge; the paper does not directly test that correlation, and its threshold values are chosen from a 25-example seed set from the same benchmark.

Editorial extensions

If this is right

Dynamic retrieval can run on roughly half the retrieval calls of an always-retrieve baseline, which matters whenever retrieval is the expensive component of a retrieval pipeline.
Eccentricity-based triggering offers the best tested balance between accuracy and cost, while Degree Matrix Jaccard minimizes searches; the choice is a cost-accuracy knob.
Semantic Sets and FLARE-Instruct underperformed on this task, so the uncertainty estimator itself determines whether dynamic retrieval helps.
When retrieval is cheap, always-retrieve may still be preferable, since black-box uncertainty estimation itself requires generating multiple sample sentences.
Uncertainty-triggered retrieval becomes more attractive for heavy, composite retrieval systems that involve reformulation, dense retrieval, and reranking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the same trigger could be reused to decide retrieval depth rather than only retrieval presence—skipping retrieval entirely for questions the model handles confidently and doing multiple rounds for hard ones.
Because the generator is GPT-3 davinci-002, the reported 2.2-versus-4.6 search reduction may not transfer to stronger models, whose uncertainty distributions and knowledge boundaries differ.
The thresholds were chosen from a 25-example seed set from the same benchmark, so a held-out threshold sweep is the natural test of whether U>2 and U>0.4 are stable operating points.
The spread between Eccentricity and Degree Matrix Jaccard results suggests a spectrum of cheap-but-lossy versus accurate uncertainty proxies, so the design choice is where to sit on that spectrum given retrieval cost.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: a small, honest workshop paper testing five off-the-shelf uncertainty metrics as triggers for when to retrieve in FLARE-style RAG. The most defensible result — Eccentricity cuts retrieval calls by about half with only a ~2-point F1 drop on 75 examples — is plausible but not established, because the evaluation lacks the one control that would prove the uncertainty signal is doing the work.

What's new and good: the paper runs Semantic Sets, Eigenvalue Laplacian, Degree Matrix (Jaccard and NLI), and Eccentricity under a unified dynamic-retrieval setup, with sentence-level uncertainty on temporary sentences. The subquery generation step is sensible, and the paper is transparent about its own limits — the conclusion openly says always-retrieve may be preferable when BM25 is cheap, which is more honest than most. The comparative table is a useful reference for practitioners choosing an uncertainty estimator.

Where it's soft: the missing cost-matched baseline is the main gap. The dynamic trigger changes both which sentences trigger retrieval and how many retrievals occur. A random trigger firing on the same ~64% of sentence positions would also cut retrieval calls, and we have no idea whether its F1 drop would be similar. Without that control, the efficiency gain is trivial (retrieve less, pay less) and the accuracy cost is uninterpretable. This isn't a minor omission; it's central to the claim that uncertainty detection specifically enables the tradeoff.

The evaluation is also tiny: 75 examples from one dataset, thresholds chosen from a 25-example seed set from the same dataset, no significance testing. The three runs overlap enough that the 0.561 vs 0.581 F1 difference for Eccentricity is within run-to-run noise. No code is released, which doesn't help.

The abstract overstates — 'almost half' retrieval reduction with 'slight' accuracy loss fits Eccentricity, but Degree Matrix Jaccard drops about 5.7 F1 points. The conclusion is more careful, and the self-citations are peripheral.

Bottom line: a reasonable workshop submission, not a strong citable result. The comparative setup is worth refereeing, but the headline claim isn't established. For a full-length venue I'd want the cost-matched baseline, a second dataset, and bootstrapped intervals first.

Referee Report

4 major / 4 minor

Summary. The paper proposes a dynamic retrieval-augmented generation (RAG) approach in which retrieval is triggered only when a sentence-level uncertainty estimate exceeds a threshold (Eq. 2). The authors evaluate five black-box uncertainty measures on the 2WikiMultihopQA dataset using GPT-3 (davinci-002) and BM25, first on a 25-example seed set and then on 75 examples with three runs per method. They report that Eccentricity and Degree Matrix Jaccard reduce the number of retrieval calls while causing only a small F1 decrease, and they conclude that uncertainty-guided dynamic retrieval improves RAG efficiency. The paper also concedes in Section 7 that always retrieving performs better on the larger set and may be preferable when retrieval is cheap.

Significance. If the central claim were established, the paper would provide a useful efficiency/accuracy trade-off for RAG by replacing deterministic retrieval with uncertainty-triggered retrieval. The authors honestly report run-to-run values and explicitly acknowledge the limitation that always retrieving outperforms conditional retrieval on the larger set, which is a credit to their reporting. However, the evidence as presented is not sufficient to support the abstract's claim: the evaluation uses only 75 examples, thresholds are selected from a 25-example seed set from the same dataset, no significance testing is performed, and the efficiency comparison omits the cost of the uncertainty estimation itself. The most economical interpretation of the results is that retrieving less often reduces retrieval count with a small accuracy cost, which does not require an uncertainty signal.

major comments (4)

[Section 6, Table 2, Eq. (2)] The comparison against the Always Retrieve baseline conflates the effect of the uncertainty signal with the effect of retrieving less often. For Eccentricity (U>2), Table 2 reports an average F1 of 0.561 with 2.23 searches per question, while Always Retrieve achieves 0.581 with 4.63 searches. Since the dynamic trigger in Eq. (2) changes both which positions trigger retrieval and how many retrievals occur, a trigger that fires on roughly 64% of positions at random or on a fixed schedule would produce a similar retrieval count and could plausibly produce a similar small F1 drop on 75 examples. The paper reports no cost-matched control (e.g., a random-trigger baseline matched to the same retrieval rate), so the central claim that uncertainty detection specifically enables the efficiency/accuracy trade-off is not established.
[Section 5, Tables 1 and 2] The thresholds for each method (U>0.4, U>2, U>0.5) are hand-picked from a 25-example seed set drawn from the same dataset, and the reported F1 values on the 75-example set are then obtained with these selected thresholds. No held-out validation of threshold choice is reported, and no significance test is applied. The run-to-run variation in Table 2 for Eccentricity (0.525, 0.565, 0.594) overlaps the difference between its average F1 (0.561) and the Always Retrieve average (0.581), so the 'slight reduction in question-answering accuracy' is not statistically supported by the presented numbers.
[Section 4.2 and Table 2] The efficiency claim counts only retrieval searches and ignores the computational cost of computing the uncertainty estimates. Each uncertainty method in Section 4.2 requires generating n sampled responses and computing pairwise similarities or semantic sets, which means the dynamic system makes multiple additional LLM calls per sentence. Since the paper does not report n or account for these generation costs, the claim of 'reducing the number of retrieval calls by almost half' does not establish end-to-end efficiency; the total LLM invocation count may be much higher than that of the Always Retrieve baseline.
[Section 4.2 and Tables 1-2] The method reported as the best performer, Eccentricity, is never defined in the methods section. Section 4.2 introduces Semantic Sets, Eigenvalue Laplacian, Degree Matrix (Jaccard), and Degree Matrix (NLI), but the reader must infer what 'Eccentricity' computes from the table headers. Given that the paper's main positive result is based on this metric, its absence from the method description is a reproducibility gap that should be fixed.

minor comments (4)

[Section 4.1] The notation for the temporary sentence is inconsistent: Equation (1) uses t_i and the surrounding text refers to t_n, making it unclear whether the uncertainty is evaluated on the current sentence or a future sentence.
[Tables 1 and 2] The table captions and headers are difficult to parse; in particular, the meaning of 'ret ratio' and the 'correct'/'incorrect' columns is not defined in the text, and the relationship between 'searches' and 'steps' should be stated explicitly.
[Section 6] The statement that Degree Matrix (Jaccard) performs 'better than an Always Retrieve baseline' is based on the 25-example seed set (Table 1) and should be qualified as such, since Table 2 shows the opposite ordering on the larger set.
[Section 4.2] The paper does not specify the number of sampled responses n used for the uncertainty estimates, the NLI model used for Semantic Sets and Degree Matrix (NLI), or the pairwise similarity function for the Laplacian and Eccentricity methods; adding these details is necessary for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports an empirical comparison; the uncertainty trigger is defined by Equation (2) and evaluated against measured F1 and retrieval counts, not derived from its own outputs.

full rationale

The paper's central claim, that uncertainty metrics such as Eccentricity and Degree Matrix Jaccard reduce retrieval calls with only a slight F1 drop, is an empirical measurement rather than a derivation. Equation (2) defines the dynamic trigger: retrieval occurs when U(t_i) exceeds a threshold, otherwise generation continues without retrieval. Tables 1 and 2 report measured retrieval counts and F1 scores on examples from 2WikiMultihopQA, an external dataset. No fitted parameter is renamed as a prediction: thresholds are selected on a 25-example seed set and then applied to a 75-example set, which raises a selection or overfitting concern but is not circular reasoning. The cited prior work supplies the uncertainty estimators and the FLARE-style active retrieval setup, but none of the cited results are used to define the paper's outcome. Self-citations (Dhole 2023, 2024a, 2024b, Dhole et al. 2024) are peripheral and do not carry the efficiency claim. The absence of a cost-matched random-trigger baseline is a validity gap concerning whether the uncertainty signal, rather than merely retrieving less often, drives the tradeoff; that is an experimental-design limitation, not a circular step, because the reported numbers are measured rather than constructed from the claim itself. No load-bearing step reduces, by the paper's own equations or by self-citation, to its own inputs, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper contributes an empirical comparison. Its central claim depends on hand-chosen thresholds, the assumption that sentence-level uncertainty reflects retrieval benefit, and a small single-dataset evaluation. No invented entities are introduced.

free parameters (3)

Uncertainty threshold θU per method = Jaccard: 0.4; Eccentricity: 2; Semantic Sets: 2; NLI: 0.5
Chosen by hand after a 25-example seed run, then applied to the 75-example evaluation.
Number of sampled responses n for uncertainty estimation = Not reported
The uncertainty methods require n generated responses per sentence, but the paper does not state n.
Number of retrieved passages k = Not reported
The retriever is BM25 via PyTerrier, but the number of passages appended to the context is not specified.

assumptions (4)

domain assumption Sentence-level uncertainty from sampled responses reflects knowledge gaps that retrieval would fill.
Eq. (2) triggers retrieval when U(t_i) > θU; the paper does not validate this correlation directly.
domain assumption BM25 over 2WikiMultihopQA Wikipedia passages provides sufficient evidence for multi-hop questions.
Section 3 and Section 5 use BM25 without reporting index construction or passage coverage.
ad hoc to paper Thresholds tuned on the 25-example seed set generalize to the 75-example set.
Section 5 describes selecting the 3 best metrics from the seed set, then evaluating on the larger set from the same dataset.
domain assumption F1 computed on generated answers is a valid accuracy measure.
Section 6 reports F1 without specifying exact-match or token-level scoring details.

how reviews work

0 comments

Cite this review

Pith. "Pith review of To Retrieve or Not to Retrieve? Uncertainty Detection for Dynamic Retrieval Augmented Generation." pith.science (2026). https://pith.science/paper/INXENQKM

@misc{pith2026250109292,
  author       = {Pith},
  title        = {Pith review of: To Retrieve or Not to Retrieve? Uncertainty Detection for Dynamic Retrieval Augmented Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INXENQKM}},
  note         = {Machine review of arXiv:2501.09292}
}

read the original abstract

Retrieval-Augmented Generation equips large language models with the capability to retrieve external knowledge, thereby mitigating hallucinations by incorporating information beyond the model's intrinsic abilities. However, most prior works have focused on invoking retrieval deterministically, which makes it unsuitable for tasks such as long-form question answering. Instead, dynamically performing retrieval by invoking it only when the underlying LLM lacks the required knowledge can be more efficient. In this context, we delve deeper into the question, "To Retrieve or Not to Retrieve?" by exploring multiple uncertainty detection methods. We evaluate these methods for the task of long-form question answering, employing dynamic retrieval, and present our comparisons. Our findings suggest that uncertainty detection metrics, such as Degree Matrix Jaccard and Eccentricity, can reduce the number of retrieval calls by almost half, with only a slight reduction in question-answering accuracy.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

DeepRAG: Thinking to Retrieve Step by Step for Large Language Models
cs.AI 2025-02 conditional novelty 6.0 of 10

A retrieval-augmented QA system that learns when to retrieve at each reasoning step outperforms adaptive RAG baselines on six benchmarks.
LLM Ensemble for RAG: Role of Context Length in Zero-Shot Question Answering for BioASQ Challenge
cs.CL 2025-09 conditional novelty 4.0 of 10

An ensemble of zero-shot LLMs with BM25 retrieval and semantic reranking ranked first in one BioASQ 13 yes/no batch, with longer contexts observed to hurt answer quality.

Reference graph

Works this paper leans on

14 extracted references · 6 canonical work pages · cited by 2 Pith papers

[1]

Tr aining a helpful and harmless assistant with reinforcement learning from human feedback

Y untao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna C hen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Tr aining a helpful and harmless assistant with reinforcement learning from human feedback . arXiv preprint arXiv:2204.05862 ,

arXiv
[7]

Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI

6 1st workshop of “Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI” at ICLR’25 Association for Computational Linguistics. doi: 10.18653 /v1/2023.emnlp-main.495. URL https://aclanthology.org/2023.emnlp-main.495. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefe...

arXiv 2023
[8]

doi: 10.18653 /v1/2023.acl-long.546

Association for Computational Linguistics. doi: 10.18653 /v1/2023.acl-long.546. URL https://aclanthology.org/2023.acl-long.546. OpenAI. Gpt-4 technical report,

work page 2023
[9]

Colbertv2: Effective and efﬁcient retrieval via lightweig ht late interaction

Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christop her Potts, and Matei Zaharia. Colbertv2: Effective and efﬁcient retrieval via lightweig ht late interaction. arXiv preprint arXiv:2112.01488,

arXiv
[10]

URL https://aclanthology.org/2023.emnlp-demo.41

18653/v1/2023.emnlp-demo.41. URL https://aclanthology.org/2023.emnlp-demo.41. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aiz awa. Constructing a multi-hop QA dataset for comprehensive evaluation of reaso ning steps. In Proceedings of the 28th International Conference on Computational Lingui stics, pp. 6609–6625, Barcelona, Spain (Online), December

work page 2023
[11]

Gemini: a family of highly capable multimodal models

Gemini Team, Rohan Anil, Sebastian Borgeaud, Y onghui Wu, Je an-Baptiste Alayrac, Jiahui Y u, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et a l. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 ,

arXiv
[12]

Self-dc: When to reason and when t o act? self divide-and-conquer for compositional unknown questions

Hongru Wang, Boyang Xue, Baohang Zhou, Tianhua Zhang, Cunxi ang Wang, Huimin Wang, Guan- hua Chen, and Kam-fai Wong. Self-dc: When to reason and when t o act? self divide-and-conquer for compositional unknown questions. arXiv preprint arXiv:2402.13514 ,

arXiv
[13]

Self-knowledg e guided retrieval augmentation for large language models

Yile Wang, Peng Li, Maosong Sun, and Y ang Liu. Self-knowledg e guided retrieval augmentation for large language models. In Houda Bouamor, Juan Pino, and K alika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023 , pp. 10303–10315, Singapore, Decem- ber

work page 2023

Show all 14 references

[14]

Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI

Association for Computational Linguistics. doi: 10.18653/v1/2023.ﬁndings-emnlp.691. URL https://aclanthology.org/2023.findings-emnlp.691. 7 1st workshop of “Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI” at ICLR’25 Jason Wei, Xue...

2023 doi
[80]

Active retrieval aug mented generation

Zhengbao Jiang, Frank Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Ja ne Dwivedi-Y u, Yiming Y ang, Jamie Callan, and Graham Neubig. Active retrieval aug mented generation. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empir- ical Methods i...

2023
[2020]

URL https://www.aclweb.org/anthology/2020.coling-main.5

International Committee on Computational Linguistics. URL https://www.aclweb.org/anthology/2020.coling-main.5

2020
[2022]

Seakr: Self-aware knowledge retrieval for adaptive ret rieval augmented generation

Zijun Y ao, Weijian Qi, Liangming Pan, Shulin Cao, Linmei Hu, Weichuan Liu, Lei Hou, and Juanzi Li. Seakr: Self-aware knowledge retrieval for adaptive ret rieval augmented generation. arXiv preprint arXiv:2406.19215,

arXiv
[2023]

Kaucus-knowledgeable user simulators for training large language models

Kaustubh Dhole. Kaucus-knowledgeable user simulators for training large language models. In Proceedings of the 1st W orkshop on Simulating Conversation al Intelligence in Chat (SCI-CHAT 2024), pp. 53–65, 2024a. Kaustubh D Dhole. Pyterrier-genrank: The pyterrier plugin for rera...

2024 arXiv
[2024]

LM-polygraph: Uncertainty e stimation for language models

Ekaterina Fadeeva, Roman V ashurin, Akim Tsvigun, Artem V az hentsev, Sergey Petrakov, Kirill Fedyanin, Daniil V asilev, Elizaveta Goncharova, Alexander Panchenko, Maxim Panov, Timothy Baldwin, and Artem Shelmanov. LM-polygraph: Uncertainty e stimation for language models. In ...

2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Tr aining a helpful and harmless assistant with reinforcement learning from human feedback

Y untao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna C hen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Tr aining a helpful and harmless assistant with reinforcement learning from human feedback . arXiv preprint arXiv:2204.05862 ,

arXiv

[2] [7]

Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI

6 1st workshop of “Quantify Uncertainty and Hallucination in Foundation Models: The Next Frontier in Reliable AI” at ICLR’25 Association for Computational Linguistics. doi: 10.18653 /v1/2023.emnlp-main.495. URL https://aclanthology.org/2023.emnlp-main.495. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefe...

arXiv 2023

[3] [8]

doi: 10.18653 /v1/2023.acl-long.546

Association for Computational Linguistics. doi: 10.18653 /v1/2023.acl-long.546. URL https://aclanthology.org/2023.acl-long.546. OpenAI. Gpt-4 technical report,

work page 2023

[4] [9]

Colbertv2: Effective and efﬁcient retrieval via lightweig ht late interaction

Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christop her Potts, and Matei Zaharia. Colbertv2: Effective and efﬁcient retrieval via lightweig ht late interaction. arXiv preprint arXiv:2112.01488,

arXiv

[5] [10]

URL https://aclanthology.org/2023.emnlp-demo.41

18653/v1/2023.emnlp-demo.41. URL https://aclanthology.org/2023.emnlp-demo.41. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aiz awa. Constructing a multi-hop QA dataset for comprehensive evaluation of reaso ning steps. In Proceedings of the 28th International Conference on Computational Lingui stics, pp. 6609–6625, Barcelona, Spain (Online), December

work page 2023

[6] [11]

Gemini: a family of highly capable multimodal models

Gemini Team, Rohan Anil, Sebastian Borgeaud, Y onghui Wu, Je an-Baptiste Alayrac, Jiahui Y u, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et a l. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 ,

arXiv

[7] [12]

Self-dc: When to reason and when t o act? self divide-and-conquer for compositional unknown questions

Hongru Wang, Boyang Xue, Baohang Zhou, Tianhua Zhang, Cunxi ang Wang, Huimin Wang, Guan- hua Chen, and Kam-fai Wong. Self-dc: When to reason and when t o act? self divide-and-conquer for compositional unknown questions. arXiv preprint arXiv:2402.13514 ,

arXiv

[8] [13]

Self-knowledg e guided retrieval augmentation for large language models

Yile Wang, Peng Li, Maosong Sun, and Y ang Liu. Self-knowledg e guided retrieval augmentation for large language models. In Houda Bouamor, Juan Pino, and K alika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023 , pp. 10303–10315, Singapore, Decem- ber

work page 2023