REVIEW 3 major objections 6 minor 64 references
Climate Finance Bench
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Retrieval, not model size, is the RAG bottleneck in climate QA.
desk verdict Useful small benchmark, but the headline retrieval-bottleneck claim needs statistical backing before it can carry weight. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the benchmark itself: 330 expert-validated question-answer pairs across 33 full sustainability reports spanning all 11 GICS sectors (a standard industry-classification scheme), with question types classified as pure extraction, numerical reasoning, or logical reasoning. The central mechanism is the stepwise retrieval comparison: minimal dense retrieval (cosine similarity over embedded chunks) versus a hybrid pipeline that combines dense and BM25 lexical scores, fuses ranked lists by inverse ranks, and applies a cross-encoder reranker (a model that re-scores candidate passages jointly with the query) before sending twelve chunks to the generator. Answers are graded by a three-level LLM-as-a-Judge protocol (a language model that labels answers as correct, incomplete, or incorrect against gold answers) that reaches 68.7 percent hard agreement with human labels, and emissions are estimated per query from local energy logs and vendor-side API measurements.
What would settle it
Run a paired bootstrap or significance test on the 330 question-level outcomes for the minimal, BM25, and reranked configurations; if the 95% confidence intervals for the +4.3 and +3.0 percentage point gains include zero, the claim that retrieval quality is the chief bottleneck is not supported by this dataset.
Extended reading notes
Core claim
The central claim, stated directly by the authors, is that the retriever's ability to locate passages that actually contain the answer is the chief performance bottleneck for RAG over climate disclosures. In their experiments, adding BM25 lexical search to dense retrieval improves the correct-answer rate by 4.3 percentage points, and adding cross-encoder reranking improves it by a further 3.0 points, for a best score of 62 percent with Claude 3.5 and hybrid retrieval. By contrast, switching from Claude 3.5 to DeepSeek R1 costs only about two points, and the paper interprets the small cross-model differences under hybrid retrieval as further evidence for the same conclusion. The authors also report that 4-bit quantization of Llama 3.1 8B leaves accuracy nearly unchanged while reducing per-query carbon emissions by about 75 percent.
Load-bearing premise
The paper's central conclusion treats the measured 3 to 4 percentage point differences between retrieval configurations as meaningful, but they come from only 330 questions and are reported without confidence intervals or significance tests; if those gaps are just random variation, the claim that retrieval is the chief bottleneck loses its main evidence.
Editorial extensions
If this is right
- Improving retrieval—through better chunking, table-aware parsing, or domain-specific reranking—should produce larger accuracy gains on climate-disclosure QA than scaling to larger proprietary models.
- At 62% correct plus 10% incomplete, the best system is usable as a first-pass evidence-surfacing tool, but the roughly one-in-four error rate keeps human review mandatory for regulatory or investment decisions.
- 4-bit weight quantization of smaller local models offers a low-carbon deployment path: roughly 75% lower per-query emissions with only 1 to 2 percentage points of accuracy loss.
- Logical-reasoning questions fail mainly because their evidence is scattered across the report, so multi-hop retrieval or knowledge-graph approaches should be the next target rather than larger generators.
- Reporting per-query carbon emissions alongside accuracy makes environmental cost a measurable dimension of model choice for AI-for-climate applications.
Reading between the lines
- Because the paper reports no confidence intervals on its 330-question evaluation, the 3 to 4 percentage point retrieval gains could plausibly sit within random variation; a natural next step is to bootstrap the question set and report error bars around each configuration's accuracy.
- The LLM-as-a-Judge under-rates correct answers about three times as often as it over-credits wrong ones, so the reported accuracies are conservative; recalibrating the grader could shift the numerical gap between configurations and the retrieval-bottleneck conclusion.
- The same stepwise retrieval diagnostics could transfer to other regulated-disclosure domains, such as financial filings or safety reports, where long documents and scattered facts create the same failure mode; a direct test would be to run the identical pipeline comparison on another document corpus.
- The finding that raw HTML conversion hurts accuracy suggests that structure-preserving parsing needs noise filtering before it can help; one extension would compare table-aware chunking with aggressive tag stripping on the same reports.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Climate Finance Bench, a question-answering benchmark built from 33 recent corporate climate/sustainability reports spanning all 11 GICS sectors, with 330 expert-validated question–answer pairs across pure extraction, numerical reasoning, and logical reasoning. It compares RAG configurations (minimal dense retrieval vs. hybrid BM25+reranking) using seven LLM backends, evaluates 4-bit weight quantization for local models, and reports per-query carbon emissions for each configuration. The central claim is that retrieval quality, not model capacity, is the chief performance bottleneck: the paper reports that adding BM25 improves accuracy by +4.3pp, reranking by another +3.0pp, and that a larger proprietary model (Claude 3.5) edges out DeepSeek R1 by only about 2pp under hybrid retrieval. The best system reaches 62% correct on the 330-question benchmark. The paper also argues for quantized local models as a lower-emission alternative.
Significance. If the bottleneck claim holds, the paper provides actionable guidance for building RAG systems in climate finance, a domain where retrieval-augmented QA is increasingly important for regulatory and investor workflows. The dataset itself is a potentially valuable open resource: it covers a wider set of sectors and report types than FinanceBench, stores gold evidence passages and page numbers, and its annotation process is documented in detail. The carbon-footprint reporting is a strength, and the LLM-as-a-Judge is validated against human labels with 68.7% hard agreement and 83.9% soft agreement, which is genuine independent grounding. However, the central claim currently rests on statistically fragile point estimates: the reported differences between retrieval upgrades and between top models are within one standard error of zero for a 330-item test set, and the paper never directly measures retrieval recall even though the dataset contains the necessary gold passages. The significance is therefore conditional on the authors adding uncertainty quantification and direct retrieval-coverage measurements.
major comments (3)
- [Section 4.2.3, Figure 3] The incremental accuracy gains that ground the central bottleneck claim are reported without any uncertainty quantification. With n=330 graded answers, the standard error of a single proportion is roughly 2.7–3.5 percentage points, and the standard error of the difference between two conditions is on the order of 4–5 points, so the reported +4.3pp (BM25) and +3.0pp (reranking) increments, as well as the ≈2pp Claude-vs-DeepSeek gap in Figure 2, are all within one standard error of zero. The paper should report bootstrap confidence intervals or a paired McNemar test across the 330 questions to establish that the ordering of retrieval upgrades is statistically reliable; without this, the abstract's claim that retrieval is 'the chief performance bottleneck' is not supported by the data as presented.
- [Section 4.2.2, Appendix A.5] Claude 3.5 Sonnet is used both as the primary LLM-as-a-Judge and as the top-scoring system, creating a potential grading artifact. Appendix A.5 reports that the judge assigns 9.9 percentage points fewer correct labels than humans and has a 16.4% Type II error rate, so a model whose outputs share stylistic features with Claude could be systematically favored. The paper should demonstrate that the model ranking is robust to the choice of judge, for example by re-scoring all model outputs with a second judge (e.g., the Llama-based judge or human labels on a subset) and reporting the ranking under that alternative, or by quantifying the judge's bias on the specific outputs of each model.
- [Sections 1, 3.1.3, 4.2.3] The paper claims in the introduction that it measures retrieval coverage, and Section 3.1.3 states that the dataset stores gold evidence passages and page numbers, yet no retrieval-recall or coverage metric is ever reported. The bottleneck conclusion is inferred solely from downstream QA accuracy. Since the gold evidence is already in the dataset, the authors can directly measure retrieval recall (e.g., the fraction of questions for which a gold evidence passage appears in the retrieved top-k) and plot answer accuracy against retrieval recall. This direct measurement would either confirm or refute the central claim, and it is currently missing.
minor comments (6)
- [Section 1] The phrase 'by measuring retrieval coverage' promises a measurement that never appears in the paper; either provide the measurement or rephrase the sentence to describe the intended capability.
- [Section 4.4] 'Quantitation pays off' should read 'Quantization pays off'.
- [Appendix B.5] The word 'substantialy' should be spelled 'substantially'.
- [Section 8] The GitHub URL 'github.com/Pladifes/climate finance bench' contains a space and is not a valid URL; it should be a single string such as 'climate-finance-bench'.
- [Section 3.5] The footnote markers attached to model names ('Claude 3.5 Sonnet 2024-06-20 3' and 'GPT-4o4') render awkwardly and should be separated from the model identifiers.
- [Figure 5] The caption contains 'Break-down'; this should be 'Breakdown'.
Circularity Check
No circularity: the paper's claims rest on empirical measurements, not on definitions, fitted parameters, or a self-citation chain.
full rationale
Climate Finance Bench is an empirical evaluation paper rather than a derivation: it constructs a 330-question dataset, runs fixed RAG configurations, and reports measured accuracies and emissions. No parameter is fitted to the test data and then renamed as a prediction; no result is defined in terms of another result. The central bottleneck claim (Sections 4.2.2-4.2.3 and Figure 3) rests on observed accuracy deltas (BM25 +4.3pp, reranking +3.0pp, Claude-vs-DeepSeek roughly 2pp), which are empirical measurements. A legitimate concern is that Claude 3.5 serves both as the LLM judge (Appendix A) and as the top-scoring generator, and that no confidence intervals or paired significance tests are reported; however, this is a validity and statistical robustness limitation, not a circularity, because the judge is calibrated against human labels (Appendix A.3-A.5, e.g., 68.7% hard match and explicit Type I/II error accounting) and the accuracy numbers are not definitionally tied to the judge's identity. The paper's Limitations section openly notes the small report count and subjectivity in gold answers, but these are scope caveats rather than circular reductions. The absence of direct retrieval-recall measurements weakens the bottleneck inference, yet that is a missing-evidence issue, not a circular step. No equation reduces to its own input, no fitted parameter is called a prediction, and no load-bearing self-citation chain is present. The empirical results are self-contained against the benchmark's own constructed data and external baselines, so the circularity score is 0.
Assumptions & free parameters
free parameters (5)
- chunk_size_tokens =
2048
- hybrid_weight_dense_bm25 =
0.75 / 0.25
- top_k_retrieved =
12
- temperature =
0.2
- max_new_tokens =
512
assumptions (4)
- domain assumption Expert annotation and two-step expert review produce reliable gold answers.
- domain assumption The 33 selected reports, mostly large-cap EU/US companies across all 11 GICS sectors, are representative enough to support general conclusions about RAG on climate disclosures.
- ad hoc to paper Claude 3.5 as LLM-as-a-Judge with 68.7% hard agreement and 83.9% soft agreement with humans is a valid proxy for human evaluation across all models.
- domain assumption Carbon footprint estimates from CodeCarbon and EcoLogits, using TDP-based GPU power and national average carbon intensity, are faithful enough for model comparisons.
Cite this review
Pith. "Pith review of Climate Finance Bench." pith.science (2026). https://pith.science/paper/WK23FN76
@misc{pith2026250522752,
author = {Pith},
title = {Pith review of: Climate Finance Bench},
year = {2026},
howpublished = {\url{https://pith.science/paper/WK23FN76}},
note = {Machine review of arXiv:2505.22752}
}
read the original abstract
Climate Finance Bench introduces an open benchmark that targets question-answering over corporate climate disclosures using Large Language Models. We curate 33 recent sustainability reports in English drawn from companies across all 11 GICS sectors and annotate 330 expert-validated question-answer pairs that span pure extraction, numerical reasoning, and logical reasoning. Building on this dataset, we propose a comparison of RAG (retrieval-augmented generation) approaches. We show that the retriever's ability to locate passages that actually contain the answer is the chief performance bottleneck. We further argue for transparent carbon reporting in AI-for-climate applications, highlighting advantages of techniques such as Weight Quantization.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Bridging Data Gaps: Data Availability and Needs for Addressing Climate-Related Financial Risks,
Network for Greening the Financial System, “Bridging Data Gaps: Data Availability and Needs for Addressing Climate-Related Financial Risks,” NGFS Technical Document, 2022
work page 2022
-
[2]
2023 Status Report: Task Force on Climate-related Financial Disclosures,
Financial Stability Board, “2023 Status Report: Task Force on Climate-related Financial Disclosures,” FSB, 2023
work page 2023
-
[3]
Project Gaia - Enabling climate risk analysis using generative AI,
Bank for International Settlements, “Project Gaia - Enabling climate risk analysis using generative AI,” March 2024
work page 2024
-
[4]
Informing climate risk analysis using textual information – A research agenda ,
Deutsche Bundesbank, “Informing climate risk analysis using textual information – A research agenda ,” Technical Report 2024-01, 2024
work page 2024
-
[5]
Language Models are Few-Shot Learners,
T. B. Brown et al., “Language Models are Few-Shot Learners,” in NeurIPS, vol. 33, pp. 1877–1901, 2020
work page 1901
-
[6]
OpenAI, “GPT-4 Technical Report,” arXiv preprint arXiv:2303.08774 , 2023
arXiv 2023
-
[7]
Survey of Hallucination in Natural Language Generation,
Z. Ji et al., “Survey of Hallucination in Natural Language Generation,” ACM Computing Surveys, vol. 55, no. 12, pp. 1-38, 2023
work page 2023
-
[8]
Augmented Language Models: a Survey,
G. Mialon et al., “Augmented Language Models: a Survey,” arXiv preprint arXiv:2302.07842, 2023
arXiv 2023
Show all 64 references
-
[9]
Retrieval-Augmented Generation for Large Language Models: A Survey,
Y. Guo et al., “Retrieval-Augmented Generation for Large Language Models: A Survey,” arXiv preprint arXiv:2305.09675 , 2023
2023 arXiv
-
[10]
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,
P. Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in NeurIPS, vol. 33, pp. 9459–9474, 2020
2020
-
[11]
Leveraging Passage Retrieval with Generative Models for Open-Domain Question Answering,
G. Izacard and E. Grave, “Leveraging Passage Retrieval with Generative Models for Open-Domain Question Answering,” in ICLR, 2021. 20
2021
-
[12]
FinGPT: Open-Source Financial Large Language Model,
H. Yang et al., “FinGPT: Open-Source Financial Large Language Model,” arXiv preprint arXiv:2306.06031, 2023
2023
-
[13]
ESG-BERT: A Pre-trained Model for ESG Corporate Disclosures Classification,
S. Mehta et al., “ESG-BERT: A Pre-trained Model for ESG Corporate Disclosures Classification,” arXiv preprint arXiv:2204.11110 , 2022
2022 arXiv
-
[14]
FINANCEBENCH: A new benchmark for financial question answering,
P. Islam et al., “FINANCEBENCH: A new benchmark for financial question answering,” arXiv preprint arXiv:2311.11944 , 2023
2023 arXiv
-
[15]
Searching for best practices in retrieval-augmented generation,
X. Wang et al., “Searching for best practices in retrieval-augmented generation,” arXiv preprint arXiv:2407.01219, 2024
2024 arXiv
-
[16]
ClimRetrieve: A benchmarking dataset for information retrieval from corporate climate disclosures,
T. Schimanski et al., “ClimRetrieve: A benchmarking dataset for information retrieval from corporate climate disclosures,” arXiv preprint arXiv:2406.09818 , 2024
2024 arXiv
-
[17]
FinQA: A Dataset of Numerical Reasoning over Financial Data,
Z. Chen et al., “FinQA: A Dataset of Numerical Reasoning over Financial Data,” in EMNLP, pp. 3696–3709, 2021
2021
-
[18]
ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversa- tional Finance Question Answering,
Z. Chen et al., “ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversa- tional Finance Question Answering,” in EMNLP, pp. 6869–6884, 2022
2022
-
[19]
TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Text Data in Finance,
Y. Zhu et al., “TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Text Data in Finance,” in EMNLP, pp. 3277–3288, 2021
2021
-
[20]
Climate-FEVER: A Dataset for Verification of Real-World Climate Claims,
T. Diggelmann et al., “Climate-FEVER: A Dataset for Verification of Real-World Climate Claims,” arXiv preprint arXiv:2012.00614 , 2020
2012 arXiv
-
[21]
Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models,
X. Wu et al., “Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models,” arXiv preprint arXiv:2411.06272 , 2024
2024
-
[22]
Atlas: Few-shot Learning with Retrieval-Augmented Language Models,
G. Izacard et al., “Atlas: Few-shot Learning with Retrieval-Augmented Language Models,” in ICLR, 2022
2022
-
[23]
KILT: A Benchmark for Knowledge Intensive Language Tasks,
F. Petroni et al., “KILT: A Benchmark for Knowledge Intensive Language Tasks,” in NAACL, pp. 2523–2544, 2021
2021
-
[24]
BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models,
N. Thakur et al., “BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models,” in CIKM, pp. 2281–2290, 2021
2021
-
[25]
BloombergGPT: A Large Language Model for Finance,
S. Shen et al., “BloombergGPT: A Large Language Model for Finance,” arXiv preprint arXiv:2303.17564, 2023. 21
2023 arXiv
-
[26]
Carbon Emissions and Large Neural Network Training,
D. Patterson et al., “Carbon Emissions and Large Neural Network Training,” Communi- cations of the ACM , vol. 65, no. 7, pp. 86–96, 2022
2022
-
[27]
Green AI,
R. Schwartz et al., “Green AI,” Communications of the ACM , vol. 63, no. 12, pp. 54–63, 2020
2020
-
[28]
Typed-RAG: Type-aware Multi-Aspect Decomposition for Non-Factoid Question Answering,
D. Lee et al., “Typed-RAG: Type-aware Multi-Aspect Decomposition for Non-Factoid Question Answering,” arXiv preprint arXiv:2503.15879 , 2025
2025 arXiv
-
[29]
Task-aware Retrieval with Instructions,
A. Asai et al., “Task-aware Retrieval with Instructions,” in Findings of the Association for Computational Linguistics , pp. 3650–3675, 2023
2023
-
[30]
Graph Retrieval-Augmented Generation: A Survey,
B. Peng et al., “Graph Retrieval-Augmented Generation: A Survey,” arXiv preprint arXiv:2408.08921, 2024
2024 arXiv
-
[31]
From Local to Global: A GraphRAG Approach to Query-Focused Summarization,
D. Edge et al., “From Local to Global: A GraphRAG Approach to Query-Focused Summarization,” arXiv preprint arXiv:2404.16130 , 2024
2024 arXiv
-
[32]
Language Model Cascades: Token-Level Uncertainty and Beyond,
N. Gupta et al., “Language Model Cascades: Token-Level Uncertainty and Beyond,” in International Conference on Learning Representations (ICLR) , 2024
2024
-
[33]
Investigating Energy Efficiency and Performance Trade-offs in LLM Inference Across Tasks and DVFS Settings,
P. J. Maliakel et al., “Investigating Energy Efficiency and Performance Trade-offs in LLM Inference Across Tasks and DVFS Settings,” arXiv preprint arXiv:2501.08219 , 2024
2024
-
[34]
ReAct: Synergizing Reasoning and Acting in Language Models,
S. Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models,” in International Conference on Learning Representations (ICLR) , 2023
2023
-
[35]
Toolformer: Language Models Can Teach Themselves to Use Tools,
T. Schick et al., “Toolformer: Language Models Can Teach Themselves to Use Tools,” arXiv preprint arXiv:2302.04761 , 2023. 22 A Appendix A Automated Grading with a LLM-as-a-Judge A.1 Human vs. Automatic Evaluation Human grading. For a sample of 330 RAG answers, each one was la...
2023 arXiv
-
[36]
Llama3.1 8B Instruct , without a reminder of the question
-
[37]
Llama3.1 8B Instruct , with a reminder of the question
-
[38]
A.3 Agreement with Human Judgements A soft match counts {correct, incomplete} as agreement, whereas a hard match counts only {correct}
Claude 3.5 Sonnet , with a reminder of the question. A.3 Agreement with Human Judgements A soft match counts {correct, incomplete} as agreement, whereas a hard match counts only {correct}. 23 Table 2: Human–LLM agreement on the 330-answer test set Grader setup Soft match Hard ...
-
[39]
AI-for-climate
while the human grader says it is incorrect or merely incomplete, and type II error (false reject) when the LLM judges an answer incorrect (label 0) or incomplete (label 1) while the human grader deems it correct. 24 With a set of 330 questions: Type I errors = 4 + 14 = 18 (5 ...
-
[40]
Local inference runs (vector-store creation, RAG pipelines and LLM generation on Kaggle GPUs/CPUs)
-
[41]
Remote API calls to proprietary LLM providers (OpenAI’s GPT -4o and Anthropic’s Claude 3.5), including the LLM-as-a-Judge evaluation phase. B.3 Remote API calls For each prompt sent to GPT -4o or Claude we queried the EcoLogits9 service, which returns a minimum and maximum est...
-
[42]
Remote inference on GPT-4o and Claude accounts for ∼5 kg CO2eq—roughly 77.00 % of the subtotal for a single RAG configuration
API calls dominate. Remote inference on GPT-4o and Claude accounts for ∼5 kg CO2eq—roughly 77.00 % of the subtotal for a single RAG configuration. It is expected because these models are substantialy bigger in size compared to the models we ran locally and each prompt triggers...
-
[43]
Compressing LLaMA 3.1 from full precision to 4-bit slashes emissions from 0 .92 to 0 .23 kg CO2eq (≈ 75% savings) while retaining answer quality within two percentage points
Quantization pays off. Compressing LLaMA 3.1 from full precision to 4-bit slashes emissions from 0 .92 to 0 .23 kg CO2eq (≈ 75% savings) while retaining answer quality within two percentage points. Similar gains are expected for NeMo, although its figure is an approximation
-
[44]
Vector stores are not the main emission point. End-to-end indexing of all sustainability reports emitted just 0 .10 kg CO2eq, two orders of magnitude below the API footprint, showing that retrieval costs are negligible compared with repeated LLM inference. Doubling the experim...
-
[45]
Rigorous annotation is critical for trustworthy results
Introduction We build a database that links 33 corporate climate reports to • ten analyst-style questions per report, • the reference answers, and • the evidence passages used to derive those answers, so that retrieval-augmented generation (RAG) systems can be evaluated on the...
-
[46]
Reports to Annotate All reports are supplied as PDF files in the shared drive: Company Reports/ Verify that every document you use is a climate or sustainability report
-
[47]
Annotation Table
Filling in the “Annotation Table” Sheet Each annotator completes all columns for their assigned rows, except Validity status of the annotation, which is for the ESG expert only. Column-by-column instructions
-
[48]
Annotator’s name: format: firstname surname 32
-
[49]
Company’s name: exactly the folder name that contains that company’s reports
-
[50]
• If the question spans several years, list them: 2020, 2021
Fiscal year • Use the fiscal year covered, not the publication year. • If the question spans several years, list them: 2020, 2021. • Check that the year matches the report contents
2020
-
[51]
Question ID: one ID for each of the ten questions (e.g. Q1)
-
[52]
• Replace FYXXXX with the concrete year, e.g
Question • Copy exactly from the master question list. • Replace FYXXXX with the concrete year, e.g. FY2020
-
[53]
Type of question : PE (Pure Extraction), NR (Numerical Reasoning) or LR (Logical Reasoning)
-
[54]
• If the information is not in the report, write exactly: Not available in the retrieved information
Answer • Write a concise English answer once found. • If the information is not in the report, write exactly: Not available in the retrieved information. • You may ask internal or external chatbots, but validate the response against the PDF
-
[55]
• Multiple documents: list in chronological order, separated by commas (e.g
Documents • Use the exact PDF filename(s). • Multiple documents: list in chronological order, separated by commas (e.g. doc1.pdf, doc2.pdf)
-
[56]
• One document: doc1{page17, page26}
Pages • Give the PDF page numbers, not the printed page labels. • One document: doc1{page17, page26}. Multiple documents: doc1{page9}, doc2{page1, page27}. • doc1 = first file named in the Documents column, doc2 = second file named, etc. 33
-
[57]
• One extract: doc1{<extract>}
Document extracts • Copy-paste the full paragraph / table / figure caption. • One extract: doc1{<extract>}. Several extracts: doc1{...}, doc1{...}, doc2{...}. • For a table or figure, copy textual content rather than screenshots. • If the PDF is image-only, screenshot, run OCR...
-
[58]
Extract type: choose from text, table, figure, text+table, text+figure
-
[59]
Comments: pick one of these options • Nothing to report (default) • Uncertain • Additional comments
-
[60]
Write a short, clear remark
Additional comments: fill only if the previous field is Additional comments. Write a short, clear remark
-
[61]
Options: Validated by the expert / Modified by the expert
Validity status of the annotation : ESG expert only. Options: Validated by the expert / Modified by the expert. The default entry is To be validated
-
[62]
Annotation Examples Examples are provided in the workbook: • Text answer → sheet Examples / Example 1 • Table answer → sheet Examples / Example 2 • Figure answer → sheet Examples / Example 3
-
[63]
The expert then sets the Validity status of the annotation to 34 • Validated by the expert, or • Modified by the expert (if corrections were required)
Quality-Control Procedure A dedicated ESG expert reviews all annotations, with special attention to rows whose Comments field is Uncertain or Additional comments. The expert then sets the Validity status of the annotation to 34 • Validated by the expert, or • Modified by the e...
-
[64]
• If the report is an image (no selectable text), take a screenshot, run OCR, and check the transcription carefully
Practical Tips • You may upload the PDF to our internal or external chatbots, ask the question, and then verify the answer in the report. • If the report is an image (no selectable text), take a screenshot, run OCR, and check the transcription carefully. 35
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.