Pith. sign in

REVIEW 5 major objections 5 minor 11 references

Reading Copom's Tone: A Weighted LLM Framework for Hawkish-Dovish Sentiment, Forward Guidance, and Uncertainty

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A weighted sentence-level LLM index can separate hawkish-dovish tone from forward guidance and uncertainty in Copom statements.

desk verdict An honest, incremental LLM tone index for Copom that clearly separates tone from guidance, but whose headline 0.719 correlation is within-model and unvalidated against human labels. read the letter →

arxiv 2608.07251 v1 pith:T2Z6HHBM submitted 2026-08-07 econ.GN cs.AIq-fin.EC

Gabriel de Macedo Santos This is my paper · ORCID
classification econ.GNcs.AIq-fin.EC
keywords centralbankcommunicationCopomhawkish-dovishsentimentlargelanguagemodelsforwardguidanceuncertaintymonetarypolicyBrazil
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper documents a transparent, sentence-level method for measuring the hawkish-dovish tone of the Brazilian Monetary Policy Committee's (Copom) policy statements. A large language model labels each sentence as hawkish, dovish, neutral, or out of context, extracts short supporting phrases, and assigns each phrase an intensity weight, and a bounded document score then combines sentence counts with the statement's average phrase intensity. A second full-document layer scores forward-guidance direction, guidance explicitness, uncertainty level, and change in uncertainty, keeping tone distinct from policy intent. Across 80 statements from August 2016 to August 2026, the average score is mildly hawkish at +0.107, the most hawkish reading is +0.570 in August 2021, and the latest statement scores +0.232 with conditional, directionally ambiguous guidance. The contribution is methodological: every score can be traced back to sentence labels and weights, making the index reproducible and auditable even before external validation.

What carries the argument

The carrying mechanism is a two-layer scoring architecture. The tone layer is the weighted ratio $\mathrm{Score}_t=(\bar{w}_{H,t}N_{H,t}-\bar{w}_{D,t}N_{D,t})/(N_{H,t}+N_{D,t}+N_{N,t})$, where out-of-context sentences are excluded from the denominator and the result is clipped to $[-1,1]$; this layer's defining choice is that the intensity averages are computed from all extracted same-sign phrases in the document, not only from sentences whose final class matches the signal. The structural layer is the product $\mathrm{GuidanceScore}_t=\mathrm{Direction}_t\times\mathrm{Explicitness}_t$, accompanied by uncertainty level and change, produced by a separate full-document LLM call. Together the two layers let the framework report, for the same statement, a hawkish tone score, conditional guidance, and central rising uncertainty.

What would settle it

Hand-label a sample of Copom sentences with economist-provided hawkish/dovish/neutral labels and intensity ratings, then compare them with the LLM's labels and weights; if agreement is near chance, or if re-running the same prompt on the same text changes document scores by more than a small tolerance, the framework's claim to reproducible tone measurement fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that separating rhetorical tone from policy guidance and uncertainty requires a sentence-level measurement architecture rather than a single document-level label. For each statement, the framework classifies sentences into hawkish, dovish, neutral, and out-of-context classes, asks a large language model to extract short hawkish and dovish phrases with 0-to-1 intensity weights, and computes a bounded score $\mathrm{Score}_t=(\bar{w}_{H,t}N_{H,t}-\bar{w}_{D,t}N_{D,t})/(N_{H,t}+N_{D,t}+N_{N,t})$, clipped to $[-1,1]$, where the $N$'s count sentences and each $\bar{w}$ is the document-specific mean intensity of the corresponding signal. A separate full-document layer scores guidance direction ($-1$, $0$, $+1$) times explicitness ($0$, $0.5$, $1$) and records uncertainty level and change. The paper shows that tone and guidance are related but distinct: their contemporaneous Pearson correlation is $0.719$, and the August 2026 statement is hawkish in diagnosis while carrying directionally ambiguous forward guidance. These are descriptive measurements, not a validated forecast of rate decisions or market prices.

Load-bearing premise

The load-bearing premise is that the LLM's hawkish/dovish labels and intensity weights are valid for measuring tone even though no human-labeled benchmark set exists; if the model's classifications are biased or unstable, the score inherits that error.

Editorial extensions

If this is right

  • The framework yields an auditable, reproducible tone index for 80 Copom statements from August 2016 to August 2026, with each score traceable to sentence labels, extracted phrases, and weights.
  • Tone and forward guidance are strongly but not perfectly associated: the contemporaneous Pearson correlation between the tone score and the guidance score is 0.719.
  • Era-level averages document the communication cycle: dovish in 2016–2020 (–0.078), sharply hawkish in 2021–2023 (+0.263), and positive but guidance-neutral in 2024–2026 (+0.237).
  • The August 2026 statement illustrates why one label is insufficient: a hawkish tone score of +0.232 coexists with conditional, directionally ambiguous guidance and rising central uncertainty.
  • The paper identifies the next validation steps: a human-labeled benchmark, model-version controls, tokenizer improvements, aggregation ablations, and an out-of-sample Selic and market-price study.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If human validation confirms the labels, the same sentence-and-phrase architecture could be carried over to other central banks that publish short policy statements, since the method needs only statement text and a prompt.
  • Because tone and guidance are scored separately, the divergence between the two could serve as a communication-regime indicator, flagging statements that are hawkish in diagnosis but uncommitted on the next move.
  • The document-specific mean intensity weighting may add sampling noise in short statements; reporting an unweighted count index alongside the weighted score would show how much of the level and turning points depend on this aggregation choice.
  • A natural extension is to test whether the tone-guidance divergence predicts subsequent rate decisions better than either measure alone, using a lagged dataset constructed to avoid look-ahead bias.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper documents an applied NLP framework for measuring the hawkish-dovish tone of Brazilian Central Bank (Copom) statements. The method classifies each sentence via an LLM into hawkish, dovish, neutral, or out-of-context categories, extracts short sign-specific phrases with 0-to-1 intensity weights, and aggregates sentence counts and document-level mean intensities into a bounded score. A second LLM layer scores forward-guidance direction, guidance explicitness, uncertainty level, and uncertainty change. Using 80 Copom statements from August 2016 to August 2026, the paper reports a mean tone score of +0.107, an extreme of +0.570 in August 2021, and a contemporaneous correlation of 0.719 between tone and the guidance-direction score. The authors are explicit that these are descriptive outputs, not validated forecasts, and that the main contribution is a transparent, auditable, incremental measurement system.

Significance. If taken as a descriptive implementation report, the paper has real value: it provides a fully traceable pipeline, and the supplied code and JSON artifacts appear sufficient to recompute every reported count, score, and correlation from the sentence-level annotations. The paper is also unusually candid: Section 6 and Appendix A identify the missing gold set, the segmentation risk, the aggregation choice, and the absence of Selic/DI validation, which is a genuine strength. The significance is limited, however, by the fact that the central instrument is unvalidated: the same LLM supplies both the sentence classes and the intensity weights, so the validity of the tone index and of the claimed tone-guidance separation is not empirically established. The paper is best read as a methodological blueprint and audit, not as a validated measurement result.

major comments (5)
  1. [Section 6 and Table 7] The central measurement-validity gap is load-bearing: Section 6 concedes that 'the same LLM that interprets the text supplies both the class and the signal weight' and that no manually labeled holdout set exists. Every downstream statistic in Section 5 — the mean score, the extreme readings, the regime averages in Table 4, and the 0.719 correlation — is generated from these unvalidated labels and weights. Consequently, the Section 7 claim that the framework 'separates' rhetorical tone from policy guidance is not supported as an empirical statement. The paper should either add a human-labeled gold-set evaluation with precision, recall, F1, and inter-annotator agreement, or explicitly downgrade the contribution to a procedural implementation note whose substantive conclusions are conditional on future validation.
  2. [Section 5.4] The contemporaneous Pearson correlation of 0.719 is computed between two outputs of the same LLM applied to the same documents, so it measures within-model coherence rather than an independent empirical relationship between tone and guidance. The paper does label this correlation descriptive, but the sentence 'It confirms that tone and guidance often move together' overstates what the design can establish. To make this finding informative, the guidance-direction score needs an independent measurement basis (e.g., human-coded guidance direction or market-based guidance proxies), or the interpretive sentence should be removed.
  3. [Section 4.2 and Appendix A, Table A1] The decision-only rule is stated in Section 4.2 as a special instruction that treats sentences reporting only the mechanical rate decision as neutral with no signals, but the audit in Appendix A records that 'some historical outputs violate it.' Because the score formula in Section 4.3 uses the neutral count as a diluting term, unquantified violations of this rule bias document scores in an unknown direction and magnitude. The manuscript should enforce this rule in post-processing, quantify how many segments violate it and their effect on the index, or justify that the violations are immaterial.
  4. [Section 3 and Appendix A, Table A1] The transparency claim is weakened by the absence of the raw statements, model identifier, prompt version, and inference settings; Appendix A explicitly states that this 'blocks exact end-to-end replication.' The supplied annotations allow recomputation of aggregates from the JSON files, but they do not allow independent verification of the sentence classifications themselves. For a paper whose central contribution is auditability, the raw corpus (or a redacted version) and a pinned model configuration should be part of the artifact set, or the reproducibility claim should be narrowed accordingly.
  5. [Section 4.1 and Section 6] The segmentation limitation is acknowledged but not quantified, and it directly affects the tone score: Section 4.1 states that missing spaces after punctuation can merge logically separate sentences, so one merged segment receives one class and understates mixed communication. Since the tone score is computed from segment-level counts and weights, the paper should report the frequency of such merges and provide a sensitivity analysis (e.g., a tokenizer-ablation exercise) before the historical averages in Table 3 can be treated as stable.
minor comments (5)
  1. [Section 4.2 and Table 1] The class labels are inconsistent: the text uses 'Hawkish', 'Dovish', 'Neutral', and 'Out-of-context', while Table 1 uses 'Hawk', 'Dove', 'Neutral', and 'Out'; harmonize the terminology.
  2. [Section 4.3] In the displayed score formula, the subscripts t on N_H, N_D, and N_N are dropped, which makes the notation ambiguous; define all terms with consistent time subscripts.
  3. [Figure 4 note] The note ends with 'All observations before August 2016.' without a verb; it should read something like 'All observations before August 2016 are excluded from the sample.'
  4. [Table 4 note] The caveat from Section 5.2 that the regime boundaries are descriptive groupings rather than estimated breakpoints should be repeated in the Table 4 note, since the table alone presents the eras as if they were natural periods.
  5. [Appendix B] The interpretation guidance in Appendix B is useful and should appear earlier in the paper, perhaps as a subsection of Section 4, so readers encounter the 'score is not a probability' caveat before the results section.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an openly descriptive measurement exercise and explicitly disclaims the validation that circularity would require.

full rationale

The paper makes no first-principles prediction that is secretly an input. Section 4.3 defines Score_t as an operational formula combining LLM sentence counts and document-specific mean phrase intensities; Section 4.4 defines GuidanceScore_t as Direction x Explicitness from a separate prompt call. Neither quantity is claimed to be derived from the other or from an external target by construction. The three historical anchors (Section 4.2) are qualitative prompt examples, not fitted parameters: the paper states they 'are not hard constraints on the final score' and that realized scores differ materially from the anchor suggestions. Section 5.4 reports the 0.719 tone-guidance correlation but immediately describes it as 'descriptive' and says it 'should not be interpreted as independent validation' because both outputs come from LLM prompts applied to the same documents. Section 6 flags the principal limitation: 'The same LLM that interprets the text supplies both the class and the signal weight,' with no manually labeled holdout set, and Appendix A records missing raw corpus/config and the non-operational FAISS path. These are acknowledged measurement-validity gaps, not circular reductions: they do not make any claimed result equal to its own input by definition, and the paper's stated contribution is 'measurement and organization, not prediction.' There are no load-bearing self-citations, no imported uniqueness theorems, and no ansatz smuggled through citation. Under the explicit-evidence standard, the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework introduces no new physical or economic entities. It relies on hand-selected calibration anchors, intensity bands, a default intensity, and a segmentation threshold as free parameters. The main non-standard assumptions are that sentence-level classes capture tone and that the three prompt anchors are appropriately chosen references.

free parameters (4)
  • Historical calibration anchors = March 2020 strongly dovish; May 2021 hawkish; August 2023 dovish
    Hand-selected examples embedded in the LLM prompt (Section 4.2) to guide classification and intensity weights. They are not hard constraints, but they steer the model and are a form of fitted calibration.
  • Intensity weight bands = weak 0.1-0.3, moderate 0.4-0.6, strong 0.7-1.0
    Ad hoc ranges assigned to extracted phrase signals in Section 4.2. They define the intensity scale used in the score numerator.
  • Default signal intensity = 1.0
    Used when no signal of a given sign is extracted for a document (Section 4.3). Affects the numerator whenever a sign is missing.
  • Minimum segment length = 4 whitespace-delimited tokens
    Segments shorter than this are excluded during sentence splitting (Section 4.1). This changes sentence counts and therefore the tone score.
assumptions (4)
  • domain assumption Hawkish and dovish tone can be decomposed into mutually exclusive sentence-level classes and phrase-level intensity weights.
    This is the core measurement assumption in Sections 4.2 and 4.3. The entire index depends on this decomposition being meaningful.
  • ad hoc to paper The three historical anchors (March 2020, May 2021, August 2023) are valid examples of dovish and hawkish extremes.
    These are author-selected calibration dates inserted into the prompt (Section 4.2). Their correctness is assumed, not demonstrated.
  • ad hoc to paper A sentence that only reports the mechanical rate decision should be treated as neutral with no signals.
    This modeling rule is introduced in Section 4.2 to avoid double-counting the decision. It changes the denominator and numerator of the score.
  • domain assumption The deterministic sentence splitter adequately separates sentences in Copom Portuguese text despite known formatting defects.
    Section 4.1 describes the regex splitter and concedes that missing spaces can merge separate sentences. The tone score is sensitive to this segmentation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reading Copom's Tone: A Weighted LLM Framework for Hawkish-Dovish Sentiment, Forward Guidance, and Uncertainty." pith.science (2026). https://pith.science/paper/T2Z6HHBM

@misc{pith2026260807251,
  author       = {Pith},
  title        = {Pith review of: Reading Copom's Tone: A Weighted LLM Framework for Hawkish-Dovish Sentiment, Forward Guidance, and Uncertainty},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T2Z6HHBM}},
  note         = {Machine review of arXiv:2608.07251}
}
read the original abstract

This paper documents an applied natural-language-processing framework for measuring the tone of Brazilian Monetary Policy Committee (Copom) statements. The project is explicitly inspired by iSent, Ita\'u's Central Bank sentiment classifier, particularly its sentence-level division of official communication into hawkish, dovish, neutral, and out-of-context classes. The implementation extends that idea in three directions. First, an LLM identifies short hawkish and dovish expressions and assigns each a 0-to-1 intensity weight. Second, the document index combines sentence counts with document-specific average signal intensities, producing a bounded score from -1 to 1. Third, a separate full-document layer measures forward-guidance direction, guidance explicitness, uncertainty level, and change in uncertainty. The empirical sample is restricted to communications dated August 2016 or later and contains 80 statements and 1,498 classified sentences from August 31, 2016 through August 5, 2026. Across this sample, 33.3% of sentences are hawkish, 18.0% dovish, 42.1% neutral, and 6.5% out of context. The average document score is +0.107, while the most hawkish reading is +0.570 in August 2021. The latest statement, dated August 5, 2026, scores +0.232, with eight hawkish, two dovish, and nine neutral sentences. Its structural overlay is more nuanced: guidance is directionally ambiguous but partly explicit, while uncertainty is classified as central and higher than at the prior meeting. Tone and the guidance-direction score have a contemporaneous Pearson correlation of 0.719. These are descriptive outputs, not a validated forecast of Selic decisions or DI returns. The main contribution is therefore methodological: a transparent, incremental, auditable system that separates rhetorical tone from policy guidance and uncertainty.

Figures

Figures reproduced from arXiv: 2608.07251 by the authors.

Figure 1
Figure 1. Project pipeline from official text to the tone index and structural overlay. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Sentence-level class composition, August 2016-August 2026. Note: Out-of-context sentences are excluded from the document-score denominator. The LLM extracted 1,552 hawkish phrase signals and 787 dovish signals within the retained sample. Their mean weights are 0.654 and 0.586, respectively, with standard deviations close to 0.20. Signal counts exceed sentence counts because one segment can contain multiple relevant … view at source ↗
Figure 3
Figure 3. Weighted hawkish-dovish score for the 80-statement study sample. Note: Positive readings are hawkish; negative readings are dovish. All observations before August 2016 [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Mean document score by broad communication regime. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Tone, forward guidance, and uncertainty across Copom statements. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 10 canonical work pages

  1. [1]

    Banco Central do Brasil. (n.d.). Copom statements: Chronological archive . Retrieved August 6, 2026, from https://www.bcb.gov.br/en/monetarypolicy/copomstatements/cronologicos

    work page 2026

  2. [2]

    S., Ehrmann, M., Fratzscher, M., De Haan, J., & Jansen, D

    Blinder, A. S., Ehrmann, M., Fratzscher, M., De Haan, J., & Jansen, D. -J. (2008). Central bank communication and monetary policy: A survey of theory and evidence. Journal of Economic Literature, 46 (4), 910 –945. https://doi.org/10.1257/jel.46.4.910

    work page doi:10.1257/jel.46.4.910 2008

  3. [3]

    Amodei, D

    Winter, C., . . . Amodei, D. (2020). Language models are few -shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901. https://papers.nips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a -Abstract.html

    work page 2020

  4. [4]

    M., & Mislang, N

    Correa, R., Garud, K., Londono, J. M., & Mislang, N. (2021). Sentiment in central banks’ financi al stability reports. Review of Finance, 25 (1), 85–120. https://doi.org/10.1093/rof/rfaa014

    work page doi:10.1093/rof/rfaa014 2021

  5. [5]

    N., Garzeri, C., Guillen, D., Lima, A., & Monteiro, V

    Ferreira, L. N., Garzeri, C., Guillen, D., Lima, A., & Monteiro, V. (2025). The not so quiet revolution: Signal and noise in central bank communication (Working Paper Series No. 635). Banco Central do Brasil. https://www.bcb.gov.br/content/publicacoes/WorkingPaperSeries/WP635.pdf

    work page 2025

  6. [6]

    Gambacorta, L., Kwon, B., Park, T., Patelli, P., & Zhu, S. (2024). CB-LMs: Language models for central banking (BIS Working Papers No. 1215). Bank for International Settlements. https://www.bis.org/publ/work1215.htm

    work page 2024

  7. [7]

    Hansen, S., & McMahon, M. (2016). Shocking language: Understanding the macroeconomic effects of central bank communication. Journal of International Economics, 99 (S1), S114–S133. https://doi.org/10.1016/j.jinteco.2015.12.008 Itaú Unibanco Macro Research. (2024, July 5). iSent: Itaú’s central bank sentiment classifier [Macro Vision report]. https://macroa...

    work page arXiv 2016

  8. [8]

    Johnson, J., Douze, M., & Jégou, H. (2017). Billion -scale similarity search with GPUs [Preprint]. arXiv. https://doi.org/10.48550/arXiv.1702.08734

    arXiv doi:10.48550/arxiv.1702.08734 2017

Show all 11 references
  1. [9]

    -T., Rocktäschel, T., Riedel, S., & Kiela, D

    Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W. -T., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval -augmented generation for knowledge -intensive NLP tasks. Advances in Neural Information Processing Systems,...

    2020

  2. [10]

    Pfeifer, M., & Marohl, V. P. (2023). CentralBankRoBERTa: A fine -tuned large language model for central bank communications. The Journal of Finance and Data Science, 9 , 100114. https://doi.org/10.1016/j.jfds.2023.100114

    2023

  3. [11]

    C., Moriya, K., & Veyrune, R

    Silva, T. C., Moriya, K., & Veyrune, R. M. (2025). From text to quantified insights: A large -scale LLM analysis of central bank communication (IMF Working Paper No. 2025/109). International Monetary Fund. https://doi.org/10.5089/9798229013802.001 12 Appendix A. Implementation...

    2025 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.