pith. sign in

arxiv: 2606.29251 · v1 · pith:CXSXNTH3new · submitted 2026-06-28 · 💻 cs.AI · q-fin.CP

When Summaries Distort Decisions: Information Fidelity in LLM-Compressed Financial Analysis

Hoyoung Lee , Suhwan Park , Seunghan Lee , Jun Seo , Jaehoon Lee , Sungdong Yoo , Minjae Kim , CheolWon Na
show 10 more authors
Zhangyang Wang Zach Golkhou Minkyu Kim Sotirios Sabanis Alejandro Lopez-Lira Dhagash Mehta Soonyoung Lee Chanyeol Choi Wonbin Ahn Yongjae Lee
This is my paper

Pith reviewed 2026-06-30 07:38 UTC · model grok-4.3

classification 💻 cs.AI q-fin.CP
keywords LLM compressioninformation fidelityfinancial analysisdecision distortiondecontextualizationearnings callsagentic systemscontext summarization
0
0 comments X

The pith

LLM compression of financial filings and transcripts can alter the investment decisions supported by the originals.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Financial decision-makers need compression because they cannot inspect all source material directly. The paper establishes that LLM-based compression loses information fidelity when it changes the decision induced by the source, even if the compressed text remains fluent and factually plausible. This occurs across filings and earnings-call transcripts through patterns such as decontextualization and model dependency. The authors propose auditing multiple candidate compressions against the original source to detect such losses. A sympathetic reader would care because these fidelity losses can recur and amplify in agentic systems that chain multiple compression steps.

Core claim

The paper claims that across financial filings and earnings-call transcripts, LLM-based compression can produce fluent and factually plausible compressed contexts that nevertheless alter downstream decisions. It frames the problem as information fidelity, where compression loses fidelity precisely when it changes the investment judgment supported by the original source. Two diagnostic patterns are identified: decontextualization, in which salient evidence is retained but separated from necessary caveats, and model dependency, in which different compressors produce different views of the same source. The authors then introduce Agentic Context Compression, which generates multiple candidate co

What carries the argument

Information fidelity, defined as the property that compression preserves the decision induced by the original source material rather than merely preserving facts or fluency.

If this is right

  • Financial compression should be evaluated by its effect on decisions, not only by efficiency or factuality.
  • Fidelity losses may recur across intermediate steps and amplify in agentic decision systems.
  • Generating multiple compressions and auditing disagreements against the original can reduce decision alteration.
  • Different compressor models expose different views of the same source, so model choice affects fidelity.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same decision-preservation test could be applied to summarization in legal or medical domains where context qualifiers matter.
  • Longer chains of agentic steps would likely increase the chance that small fidelity losses compound into larger decision shifts.
  • Human verification of compressed contexts might become a required step for high-stakes financial reports.

Load-bearing premise

The investment decision induced by the original source material can be reliably and consistently determined independently of any compression process.

What would settle it

A controlled study in which independent human analysts derive identical investment decisions from the raw filings and transcripts but derive different decisions from the LLM-compressed versions would support the claim; if analysts instead derive the same decision from both original and compressed versions, the claim would be falsified.

Figures

Figures reproduced from arXiv: 2606.29251 by Alejandro Lopez-Lira, Chanyeol Choi, CheolWon Na, Dhagash Mehta, Hoyoung Lee, Jaehoon Lee, Jun Seo, Minjae Kim, Minkyu Kim, Seunghan Lee, Soonyoung Lee, Sotirios Sabanis, Suhwan Park, Sungdong Yoo, Wonbin Ahn, Yongjae Lee, Zach Golkhou, Zhangyang Wang.

Figure 1
Figure 1. Figure 1: Compression-induced decision flip. Context compression can cause a decision-maker to reach a dif￾ferent decision from the one supported by the original source. For example, information that supports a bullish decision in the original source may lead to a bearish decision after compression. Long-context compression is also inherently open-ended. A financial source text has no sin￾gle correct summary because… view at source ↗
Figure 2
Figure 2. Figure 2: Decision change under one-shot compression. Bars show Decision Flip and source-relative TVD for four [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Model-specific decision movement under one-shot compression. Each panel shows one compressor; [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: (A) Context facts are dropped much more un￾der MD&A than earnings-call. (B) Re-adding dropped context recovers more flipped decisions than boilerplate or random facts. The gain is largest where the decontextualization diagnostic is strongest: for 10-Q MD&A, where dropped context produces the clearest add-back recovery, contextualization lowers the flip rate from 33.0% to 21.3%. For earnings calls, where co… view at source ↗
Figure 5
Figure 5. Figure 5: Budget sensitivity under GPT-5.4-MINI one-shot compression. Top: decision flips remain above the analyst noise floor even at larger budgets. Bottom: source-relative TVD declines with budget but plateaus above the floor. B Inter-Model Agreement Aggregate flip rates can hide whether compressors fail on the same filings [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Pairwise top-decision agreement among one-shot compressors on the same source filings. Cells show the [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Decision movement for MD&A disclosures [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Decision movement for earnings-call Q&A disclosures. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
read the original abstract

Financial decision-makers face more information than they can directly inspect, making context compression necessary. Yet when large language models (LLMs) compress financial source material, they can alter the investment judgment supported by the original source. We frame this problem as information fidelity: compression loses fidelity when it changes the decision induced by the source. In agentic systems, such losses may recur across intermediate steps and amplify throughout the decision process. Across financial filings and earnings-call transcripts, we find that LLM-based compression can produce fluent and factually plausible compressed contexts that nevertheless alter downstream decisions. We analyze two diagnostic patterns associated with fidelity loss: decontextualization, where salient evidence is retained but separated from the caveats and contextual qualifiers needed for correct interpretation, and model dependency, where different compressors expose different views of the same source. We then propose Agentic Context Compression, which generates multiple candidate compressions and audits their disagreements against the original source. Our results suggest that financial compression should be evaluated not only by efficiency or factuality, but also by its ability to preserve decision-relevant context.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript defines information fidelity as the degree to which LLM compression of financial source material (filings and earnings-call transcripts) preserves the investment decision induced by the uncompressed original. It reports that LLM compressors can generate fluent, factually plausible summaries that nevertheless change downstream decisions, attributes this to decontextualization (retaining evidence while stripping qualifiers) and model dependency (different compressors yielding divergent views), and proposes Agentic Context Compression, which generates multiple candidate summaries and audits their disagreements against the source to detect fidelity loss.

Significance. If the measurement of decision change is shown to be reliable, the work usefully reframes compression evaluation in agentic financial systems away from isolated factuality or fluency metrics toward preservation of decision-relevant context. The constructive proposal of auditing multiple compressions is a practical step that could be adopted in pipelines where fidelity matters. The empirical focus on real financial documents adds relevance, though the strength of the contribution hinges on the robustness of the decision-elicitation protocol.

major comments (2)
  1. [Experimental Setup] Experimental Setup (likely §3 or §4): The central claim requires a stable baseline decision D_original that can be compared to D_compressed. No inter-evaluator agreement statistics, test-retest consistency, or variance across human/LLM judges on the same original documents are reported. Without these, observed decision shifts cannot be confidently attributed to compression rather than baseline subjectivity in interpreting financial qualifiers and forward-looking statements.
  2. [Results] Results (likely §5): The abstract and main findings state that compression alters decisions, yet the manuscript supplies no error bars, statistical significance tests, or baseline compressors (e.g., extractive or rule-based) against which the magnitude of fidelity loss is compared. This weakens the quantitative support for the claim that LLM compression specifically induces the observed changes.
minor comments (2)
  1. [Introduction] The term 'information fidelity' is introduced without a formal definition or equation; a precise operationalization (e.g., decision divergence rate) would aid reproducibility.
  2. [Figures/Tables] Figure captions and table headers should explicitly state the number of documents, evaluators, and compressor models used so that effect sizes can be interpreted without returning to the text.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback emphasizing the need for greater rigor in validating the stability of our decision-elicitation protocol and in quantifying the effects. We address each major comment below.

read point-by-point responses

  1. Referee: [Experimental Setup] Experimental Setup (likely §3 or §4): The central claim requires a stable baseline decision D_original that can be compared to D_compressed. No inter-evaluator agreement statistics, test-retest consistency, or variance across human/LLM judges on the same original documents are reported. Without these, observed decision shifts cannot be confidently attributed to compression rather than baseline subjectivity in interpreting financial qualifiers and forward-looking statements.

    Authors: We agree that stability metrics for the baseline decision are necessary to attribute changes to compression. Our protocol used a single LLM judge at temperature 0 to promote determinism, but we did not report agreement or variance statistics. We will add a new subsection reporting test-retest consistency (repeated elicitations on originals) and variance across multiple LLM judges, which will allow readers to assess baseline subjectivity. revision: yes

  2. Referee: [Results] Results (likely §5): The abstract and main findings state that compression alters decisions, yet the manuscript supplies no error bars, statistical significance tests, or baseline compressors (e.g., extractive or rule-based) against which the magnitude of fidelity loss is compared. This weakens the quantitative support for the claim that LLM compression specifically induces the observed changes.

    Authors: We concur that error bars, significance testing, and non-LLM baselines would strengthen the quantitative claims. We will revise the results to include bootstrap-derived confidence intervals, paired statistical tests on decision-change rates, and an extractive baseline (TF-IDF sentence selection) to benchmark the magnitude of LLM-induced fidelity loss against simpler methods. revision: yes

Circularity Check

0 steps flagged

Empirical measurement of decision shifts; no derivations or fitted predictions

full rationale

The paper frames fidelity as an observable change in downstream investment decisions after LLM compression of filings/transcripts and reports empirical patterns (decontextualization, model dependency) plus a proposed auditing method. No equations, parameter fitting, or self-citation chains appear in the abstract or described content; the measurement pipeline compares decisions on original vs. compressed text without reducing any result to its own inputs by construction. This is a standard empirical study whose central claim does not collapse into self-definition or renaming.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that decisions can be induced from source text in a measurable way; no free parameters or invented entities are visible in the abstract.

axioms (1)
  • domain assumption Investment decisions can be consistently induced from source financial material and compared across compressed versions to quantify fidelity loss.
    Invoked to define when compression alters the decision induced by the source.

pith-pipeline@v0.9.1-grok · 5790 in / 1206 out tokens · 29465 ms · 2026-06-30T07:38:21.531364+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 25 canonical work pages · 4 internal anchors

  1. [1]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , month = jul, year =

    Mohamed, Amr and Geng, Mingmeng and Vazirgiannis, Michalis and Shang, Guokan , editor =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , month = jul, year =. doi:10.18653/v1/2025.acl-long.371 , pages =

    work page doi:10.18653/v1/2025.acl-long.371 2025

  2. [2]

    The Fourteenth International Conference on Learning Representations , year =

    Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models , author =. The Fourteenth International Conference on Learning Representations , year =

  3. [3]

    LLMs Corrupt Your Documents When You Delegate

    Laban, Philippe and Schnabel, Tobias and Neville, Jennifer , year =. 2604.15597 , archivePrefix =

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , month = dec, year =

    Min, Sewon and Krishna, Kalpesh and Lyu, Xinxi and Lewis, Mike and Yih, Wen-tau and Koh, Pang and Iyyer, Mohit and Zettlemoyer, Luke and Hajishirzi, Hannaneh , editor =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , month = dec, year =. doi:10.18653/v1/2023.emnlp-main.741 , pages =

    work page doi:10.18653/v1/2023.emnlp-main.741 2023

  5. [5]

    , booktitle =

    Wei, Jerry and Yang, Chengrun and Song, Xinying and Lu, Yifeng and Hu, Nathan and Huang, Jie and Tran, Dustin and Peng, Daiyi and Liu, Ruibo and Huang, Da and Du, Chao and Le, Quoc V. , booktitle =. 2024 , url =

    2024

  6. [6]

    Zhang, Yusen and Zhang, Nan and Liu, Yixin and Fabbri, Alexander and Liu, Junru and Kamoi, Ryo and Lu, Xiaoxin and Xiong, Caiming and Zhao, Jieyu and Radev, Dragomir and McKeown, Kathleen and Zhang, Rui , editor =. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies...

    work page doi:10.18653/v1/2024.naacl-long.187 2024

  7. [7]

    Lei, Yuanyuan and Song, Kaiqiang and Cho, Sangwoo and Wang, Xiaoyang and Huang, Ruihong and Yu, Dong , editor =. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , month = jun, year =. doi:10.18653/v1/2024.naacl-long.291 , pages =

    work page doi:10.18653/v1/2024.naacl-long.291 2024

  8. [8]

    2601.04889 , archivePrefix =

    Aghaebe, Favour Yahdii and Apekey, Tanefa and Williams, Elizabeth and Moosavi, Nafise Sadat , year =. 2601.04889 , archivePrefix =

    work page arXiv

  9. [9]

    2309.17322 , archivePrefix =

    Glasserman, Paul and Lin, Caden , year =. 2309.17322 , archivePrefix =

    work page arXiv

  10. [10]

    Performance Comparison of Deep Learning Models for CO2 Pre- diction: Analyzing Carbon Footprint with Advanced Trackers,

    Nakagawa, Kei and Hirano, Masanori and Fujimoto, Yugo , booktitle =. 2024 , archivePrefix =. doi:10.1109/BigData62323.2024.10826008 , url =

    work page doi:10.1109/bigdata62323.2024.10826008 2024

  11. [11]

    2025 , isbn =

    Lee, Hoyoung and Seo, Junhyuk and Park, Suhwan and Lee, Junhyeong and Ahn, Wonbin and Choi, Chanyeol and Lopez-Lira, Alejandro and Lee, Yongjae , booktitle =. 2025 , isbn =. doi:10.1145/3768292.3770375 , url =

    work page doi:10.1145/3768292.3770375 2025

  12. [12]

    2602.14233 , archivePrefix =

    Kong, Yaxuan and Lee, Hoyoung and Hwang, Yoontae and Lopez-Lira, Alejandro and Levy, Bradford and Mehta, Dhagash and Wen, Qingsong and Choi, Chanyeol and Lee, Yongjae and Zohren, Stefan , year =. 2602.14233 , archivePrefix =

    work page arXiv

  13. [13]

    doi:10.1145/3701716.3715289 , url =

    Loukas, Lefteris and Billert, Fabian and Fergadiotis, Manos and Malakasiotis, Prodromos and Androutsopoulos, Ion , booktitle =. doi:10.1145/3701716.3715289 , url =

    work page doi:10.1145/3701716.3715289

  14. [14]

    2026 , eprint =

    Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction , author =. 2026 , eprint =

    2026

  15. [15]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , month = dec, year =

    Jiang, Huiqiang and Wu, Qianhui and Lin, Chin-Yew and Yang, Yuqing and Qiu, Lili , editor =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , month = dec, year =. doi:10.18653/v1/2023.emnlp-main.825 , pages =

    work page doi:10.18653/v1/2023.emnlp-main.825 2023

  16. [16]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , month = aug, year =

    Jiang, Huiqiang and Wu, Qianhui and Luo, Xufang and Li, Dongsheng and Lin, Chin-Yew and Yang, Yuqing and Qiu, Lili , editor =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , month = aug, year =. doi:10.18653/v1/2024.acl-long.91 , pages =

    work page doi:10.18653/v1/2024.acl-long.91 2024

  17. [17]

    2508.13124 , archivePrefix =

    Mayilvaghanan, Kawin and Gupta, Siddhant and Kumar, Ayush , year =. 2508.13124 , archivePrefix =

    work page arXiv

  18. [18]

    ACON: Optimizing Context Compression for Long-horizon LLM Agents

    Kang, Minki and Chen, Wei-Ning and Han, Dongge and Inan, Huseyin A. and Wutschitz, Lukas and Chen, Yanzhi and Sim, Robert and Rajmohan, Saravan , year =. 2510.00615 , archivePrefix =

    work page internal anchor Pith review Pith/arXiv arXiv

  19. [19]

    2021 , eprint =

    Choi, Eunsol and Palomaki, Jennimaria and Lamm, Matthew and Kwiatkowski, Tom and Das, Dipanjan and Collins, Michael , journal =. 2021 , eprint =

    2021

  20. [20]

    2406.20079 , archivePrefix =

    Gunjal, Anisha and Durrett, Greg , year =. 2406.20079 , archivePrefix =

    work page arXiv

  21. [21]

    , year =

    Kuwahara, Bruce and Lin, Chen-Yuan and Huang, Xiao Shi and Leung, Kin Kwan and Yapeter, Jullian Arta and Stanevich, Ilya and Perez, Felipe and Cresswell, Jesse C. , year =. 2509.20461 , archivePrefix =

    work page arXiv

  22. [22]

    2025 , eprint =

    Trienes, Jan and Schl. 2025 , eprint =

    2025

  23. [23]

    2508.02540 , archivePrefix =

    Zhukova, Anastasia and Ruas, Terry and Hamborg, Felix and Donnay, Karsten and Gipp, Bela , year =. 2508.02540 , archivePrefix =

    work page arXiv

  24. [24]

    FinGround: Detecting and Grounding Financial Hallucinations via Atomic Claim Verification

    Guo, Dongxin and Wu, Jikun and Yiu, Siu Ming , year =. 2604.23588 , archivePrefix =

    work page internal anchor Pith review Pith/arXiv arXiv

  25. [25]

    2510.22967 , archivePrefix =

    Ning, Yucheng and Lin, Xixun and Fang, Fang and Cao, Yanan , year =. 2510.22967 , archivePrefix =

    work page arXiv

  26. [26]

    2507.03194 , archivePrefix =

    Alessa, Abeer and Somane, Param and Lakshminarasimhan, Akshaya and Skirzynski, Julian and McAuley, Julian and Echterhoff, Jessica , year =. 2507.03194 , archivePrefix =

    work page arXiv

  27. [27]

    2504.00025 , archivePrefix =

    Peters, Uwe and Chin-Yee, Benjamin , year =. 2504.00025 , archivePrefix =

    work page arXiv

  28. [28]

    Frame In, Frame Out: Measuring Framing Bias in LLM-Generated News Summaries

    Pastorino, Valeria and Moosavi, Nafise Sadat , year =. 2505.05406 , archivePrefix =

    work page internal anchor Pith review Pith/arXiv arXiv

  29. [29]

    2508.15813 , archivePrefix =

    Zhang, Tinghui and Wang, Yifan and Wang, Daisy Zhe , year =. 2508.15813 , archivePrefix =

    work page arXiv

  30. [30]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  31. [31]

    Publications Manual , year = "1983", publisher =

    1983

  32. [32]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    work page doi:10.1145/322234.322243 1981

  33. [33]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  34. [34]

    Dan Gusfield , title =. 1997

    1997

  35. [35]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  36. [36]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =