Pith. sign in

REVIEW 4 major objections 5 minor 20 references

The Sound of Risk: A Multimodal Physics-Informed Acoustic Model for Forecasting Market Volatility and Enhancing Market Interpretability

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Executives' emotional dynamics in earnings calls predict future market volatility, not stock returns, with a 43.8% out-of-sample R2 for 30-day volatility.

desk verdict Earnings-call emotion predicts volatility but not returns — interesting if real, but the bootstrap validation needs a temporal split before the headline R2 is credible. read the letter →

arxiv 2508.18653 v1 pith:ZR7GTTFJ submitted 2025-08-26 cs.LG cs.AIcs.SDeess.AS

Xiaoliang Chen , Xin Yu , Le Chang , Teng Jing , Jiashuai He , Ze Wang , Yangjun Luo , Xingyu Chen
show 3 more authors
Jiayue Liang Yuchen Wang Jiaying Xie
This is my paper · ORCID
classification cs.LGcs.AIcs.SDeess.AS
keywords earningscallsmarketvolatilityforecastingcomputationalparalinguisticsaffectivestatelabelphysics-informedneuralnetworksmultimodalfinancevocalstressrealized
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that how executives sound during earnings calls carries information about future market risk that traditional text analysis and historical prices miss. Using a physics-informed acoustic model that extracts vocal emotion from raw, distorted teleconference audio, plus a large language model that labels the emotional tone of transcripts, the authors map both channels onto a common three-dimensional affect space. The central finding is a split: these multimodal emotional features do not predict future stock returns, but they explain up to 43.8% of out-of-sample variance in 30-day realized volatility. The strongest signals come from the shift between scripted presentation and spontaneous Q&A, especially a drop in the CFO's textual stability, acoustic instability from CFOs, and arousal variability from CEOs. If the result holds, monitoring executive affect in earnings calls could give investors and regulators a direct, hard-to-manipulate gauge of hidden corporate uncertainty.

What carries the argument

PIAM, the Physics-Informed Acoustic Model, is a multi-task neural network that takes raw earnings-call audio and simultaneously produces a transcript, a discrete acoustic emotion label, and acoustic event labels. It is regularized by the Westervelt equation, a nonlinear-acoustics PDE that penalizes latent-space trajectories inconsistent with nonlinear sound propagation, which the authors argue makes it robust to teleconference clipping and compression. The discrete labels, acoustic and textual, are mapped onto a three-dimensional Affective State Label (ASL) space—Tension, Stability, Arousal—anchored by a human annotation study. The predictive features are statistical moments of these ASL val

What would settle it

Run the identical feature pipeline with a strict temporal split (all training calls before a cutoff date, all test calls after) and with all calls from a given firm confined to one fold. If the 30-day volatility R-squared falls to roughly the 0.251 financials-only baseline, the multimodal outperformance is an artifact of leakage rather than emotional signal.

Watch

Extended reading notes

Core claim

The core claim is the predictive divergence between risk and return. On 1,795 earnings calls from 283 NASDAQ firms (2018-2023), the authors build around 150 features per call from executive affect, including moments of tension, stability, and arousal and the change in those states from prepared remarks to Q&A. An XGBoost model using these features achieves out-of-sample R2 of 0.438 for 30-day realized volatility, compared to 0.251 for a financials-only baseline using historical volatility, while out-of-sample R2 for cumulative abnormal returns is statistically indistinguishable from zero. The paper interprets this as evidence that executive emotion is not a directional signal but a barometer

Load-bearing premise

The headline 43.8% out-of-sample R-squared depends on the bootstrap validation being a genuine holdout, but the paper never states a temporal or firm-level split; overlapping calls from the same firm could leak between training and evaluation.

Editorial extensions

If this is right

  • The full multimodal model adds roughly 18 to 19 percentage points of out-of-sample R2 over the financials-only baseline for 30-day volatility, so acoustic and textual emotion carry information beyond historical price swings.
  • Emotional cues do not forecast cumulative abnormal returns, meaning they signal uncertainty about future performance rather than the direction of that performance.
  • The most stable predictors are role-specific and context-specific: a CFO's drop in textual stability and a CEO's arousal variability in Q&A dominate the top features, so who speaks and when matters.
  • PIAM's real-time factor below 1.0 means the acoustic channel could be monitored live as calls happen, not just in hindsight.
  • If markets already price textual disclosure but not vocal affect, regulators and investors could use this channel to detect hidden uncertainty earlier than price-based measures reveal it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A strict temporal or firm-level split is the natural check: if the 43.8% R2 survives, the signal is real; if it collapses to the baseline, the reported out-of-sample gain is leakage.
  • The presentation-to-Q&A delta design could be ported to other scripted-then-spontaneous disclosure settings, such as central-bank briefings or IPO roadshows, as a ready-made stress test.
  • Because returns are unpredictable but volatility is, the acoustic-textual signal may be pricing a volatility risk premium; the paper does not test tradeability, so that remains an open question.
  • The ASL emotion coordinates were validated on North American public-company speech; applying the pipeline to other dialects, languages, or non-corporate speakers would require re-anchoring the mapping.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes a multimodal pipeline for forecasting market volatility from earnings calls. Raw teleconference audio is processed by PIAM, a wav2vec 2.0-based multi-task model regularized by a Westervelt-equation physics residual, to produce transcripts and acoustic emotion labels; an LLM (DeepSeek-R1) produces textual emotion labels. Both modalities are mapped onto a three-dimensional Affective State Label (ASL) space with dimensions Tension, Stability, and Arousal, and aggregate/delta features are fed to an XGBoost regressor to predict 1-, 7-, and 30-day CAR and realized volatility. Using 1,795 calls from 283 NASDAQ firms (2018-2023), the paper reports that multimodal features have no predictive power for CAR but explain 43.8% of out-of-sample variance in 30-day realized volatility, versus 25.1% for a historical-volatility baseline. An ablation study attributes the gain to complementary acoustic and textual signals. The central regression is not circular: the ASL coordinates are not fitted to volatility outcomes. However, the headline result rests on a bootstrap validation procedure that may not be genuinely out-of-sample, and several supporting evaluations are missing or under-reported.

Significance. If the empirical claim holds, it would be a notable contribution: paralinguistic and textual emotion dynamics from earnings calls would carry incremental information about future uncertainty beyond historical volatility, and the asymmetry between return prediction (null) and volatility prediction (strong) would be an interesting and falsifiable result. The paper also contains useful engineering pieces: a multi-task acoustic model with a nonlinear-acoustics regularizer, a human-grounded ASL mapping, an ablation design, and evaluations on the public DNS challenge for noise suppression. The authors provide detailed hyperparameters and a reproducibility promise. The main value is conditional on the out-of-sample claim being credible; currently the validation protocol does not establish that. The paper would be substantially strengthened by block-resampled or strict temporal/firm-holdout validation, confidence intervals for R2, and a direct evaluation of PIAM's emotion-classification accuracy.

major comments (4)
  1. [§4.3, §5.3] The headline R2=0.438 is produced by a bootstrap validation described only as "random resampling of the training data in each iteration." With 2018-2023 quarterly calls per firm and 30-day realized-volatility labels that overlap in calendar time across calls, random resampling can place the same firm or overlapping time windows in both training and evaluation. This can inflate out-of-sample R2 through a latent time factor and is a genuine leakage risk. Please report results under strict temporal splits (e.g., train on calls before date T, test after) and firm-holdout splits, with the exact split boundaries and number of folds/iterations. Block bootstrap or Purged K-Fold CV would address this directly.
  2. [§5.1, §5.3] PIAM is the source of the acoustic ASL features, but its standalone evaluation in Table 3 is limited to DNS noise-suppression metrics (MOS-LQO/PESQ) and Table 4 reports only RTF. There is no evaluation of emotion-classification accuracy on earnings-call audio or on any labeled emotion corpus. If the discrete acoustic emotion labels are inaccurate, the acoustic ASL features are noisy in an unquantified way, which weakens the multimodal claim. Please report emotion classification accuracy/F1 (e.g., on IEMOCAP or a labeled earnings-call subset) and, ideally, transcription WER, so the quality of the acoustic channel can be assessed.
  3. [§5.3, Table 5] The paper states that CAR R2 values are "statistically indistinguishable from zero" and that 43.8% is "remarkable," but no confidence intervals or significance tests are reported for any R2. Given 1,795 observations and roughly 150 features, the difference between Multimodal (0.438) and Factors-Only (0.251) needs uncertainty quantification. Please provide bootstrap or block-bootstrap CIs for all R2 values and a paired comparison (e.g., corrected resampled t-test or Diebold-Mariano) between Multimodal and Factors-Only/Text-Only. Without this, the ablation "decisively quantifies" claim is not supported.
  4. [§3.4, Appendix A] The ASL mapping in Table 1 is a fixed input to the features, but Appendix A gives no participant count, inter-rater reliability, or distribution of annotations for the coordinates. Since the coordinates are explicitly adapted for financial risk (e.g., happiness has negative Tension), the reliability of this mapping matters for the volatility result. Please report the number of annotators, agreement metrics (ICC or Krippendorff's alpha), and sensitivity of the main result to the ASL coordinate values.
minor comments (5)
  1. [§5.6, Table 5] The text says "substantially outperforms" and "decisively quantifies," but the table lacks error bars and sample sizes per cell. Adding CIs would also clarify the surprising pattern that Acoustic-Only and Text-Only underperform Factors-Only while the combination far exceeds it.
  2. [§3.2, Eq. (3)] The physics regularization weight λ=0.01 is stated in Appendix B, but no sensitivity analysis or ablation with λ=0 is provided. A reader cannot tell whether the Westervelt residual, as opposed to the multi-task architecture, contributes to noise robustness.
  3. [§4.2, Feature Engineering] The definition of realized volatility is not given (e.g., sum of squared returns? annualization?), and the CAR computation is not described (market model? estimation window?). This makes the financial targets hard to reproduce.
  4. [§5.2, Figure 2] The claim that low concordance between modalities is "a strength" is plausible, but it would be more convincing with a quantitative concordance statistic (e.g., Cohen's kappa or mutual information) rather than only a histogram.
  5. [Appendix C] The reproducibility statement promises release only "upon acceptance." For an arXiv preprint, providing the emotion-classification evaluation and split definitions in the supplement would greatly help reviewers and readers assess the claims now.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: emotional-signal features were not fitted to volatility, and the 43.8% R^2 is an empirical out-of-sample claim, not a construction.

full rationale

The central derivation chain is: raw sound -> PIAM emotion labels; transcript -> LLM emotion labels; discrete labels mapped to fixed ASL coordinates (Table 1, Appendix A); moments, deltas, and interactions computed; XGBoost regresses these features on realized volatility. None of these steps defines a feature in terms of the target. The ASL coordinates are domain-tailored (e.g., fear = high tension, low stability) but were set by human annotation before any volatility regression, not fitted to the outcome; the predictive result could have failed. The physics-informed regularizer (Eq. 2) is a soft PDE constraint on a learned pressure proxy, and the Westervelt equation is an external acoustics result; it does not encode market volatility. The only self-citation (Chen et al. 2025) appears in Related Work as an analogy and is not load-bearing. The reported 43.8% R^2 depends on a bootstrap validation that may admit same-firm/overlapping-window leakage, but that is a validity/leakage concern, not an equivalence-by-construction; the evaluation still fits on training data and tests on (possibly contaminated) held-out points. No circular step is exhibited.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim depends on the accuracy of emotion labels, the ASL coordinate mapping, and a valid out-of-sample split; all three are only partially supported. The Westervelt prior is decorative unless the residual is actually computed. The paper is primarily a fitted predictive pipeline, not a derivation, so the free-parameter count is dominated by hand-designed mapping and hyperparameters.

free parameters (4)
  • ASL emotion-to-coordinate mapping = Table 1 values: 7 emotions by 3 dimensions, e.g., happiness (-0.5, 1.0, 0.6), fear (1.0, -1.0, 0.8)
    Chosen by a human annotation study and adapted for financial risk; directly determines all ASL moment, delta, and interaction features used in the volatility model.
  • Physics regularization weight lambda = 0.01
    Hand-picked hyperparameter balancing the PDE residual and task loss in Section 3.2; no sensitivity analysis is reported.
  • PIAM fine-tuning hyperparameters = 10 epochs, AdamW, learning rate 1e-5, batch size 16
    Appendix B specifies these values but gives no indication of tuning or robustness; they affect the emotion labels that feed all downstream features.
  • XGBoost hyperparameters = learning rate 0.05, max depth 3, subsample/colsample 0.8, up to 100 estimators
    Manual settings that affect R2 and feature importance; no search or sensitivity analysis is reported.
assumptions (5)
  • domain assumption The Westervelt equation describes emotion-relevant vocal dynamics, and teleconference distortions such as clipping can be modeled by its lossless form.
    Section 3.1-3.2 applies a nonlinear acoustics PDE to emotional speech without evidence that vocal fold stress obeys this continuum equation in the lossless regime or that the regularizer actually computes the stated residual.
  • domain assumption Discrete emotion labels from PIAM and the LLM accurately represent executive affect.
    Section 5.1 validates PIAM on the DNS noise suppression benchmark, not on emotion classification; no accuracy, confusion matrix, or reliability measure for the emotion labels is reported.
  • ad hoc to paper Random resampling of calls in bootstrap validation yields valid out-of-sample estimates despite likely time and firm overlap.
    Section 4.3 describes bootstrap validation without a temporal or firm-level split; this premise is load-bearing for the 43.8% out-of-sample R2.
  • domain assumption Human annotation consensus defines a valid mapping from discrete emotions to ASL coordinates.
    Appendix A describes the annotation study but gives no sample size, inter-rater reliability, or external validation against established affect benchmarks.
  • domain assumption XGBoost Gini importance identifies features that drive volatility predictions.
    Section 5.5 interprets Gini importance ranks as evidence that CFO and CEO emotional dynamics are the operative risk signals, which is an inferential leap for tree-based impurity importance.
invented entities (1)
  • Affective State Label (ASL) space with dimensions Tension, Stability, and Arousal
    purpose: Map discrete acoustic and textual emotion labels onto a continuous 3D space for financial feature engineering.
    The coordinates are fixed by the paper's own human annotation study, with no independent benchmark. The Stability dimension is explicitly constructed for the financial risk context, so it carries no external falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Sound of Risk: A Multimodal Physics-Informed Acoustic Model for Forecasting Market Volatility and Enhancing Market Interpretability." pith.science (2026). https://pith.science/paper/ZR7GTTFJ

@misc{pith2026250818653,
  author       = {Pith},
  title        = {Pith review of: The Sound of Risk: A Multimodal Physics-Informed Acoustic Model for Forecasting Market Volatility and Enhancing Market Interpretability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZR7GTTFJ}},
  note         = {Machine review of arXiv:2508.18653}
}
read the original abstract

Information asymmetry in financial markets, often amplified by strategically crafted corporate narratives, undermines the effectiveness of conventional textual analysis. We propose a novel multimodal framework for financial risk assessment that integrates textual sentiment with paralinguistic cues derived from executive vocal tract dynamics in earnings calls. Central to this framework is the Physics-Informed Acoustic Model (PIAM), which applies nonlinear acoustics to robustly extract emotional signatures from raw teleconference sound subject to distortions such as signal clipping. Both acoustic and textual emotional states are projected onto an interpretable three-dimensional Affective State Label (ASL) space-Tension, Stability, and Arousal. Using a dataset of 1,795 earnings calls (approximately 1,800 hours), we construct features capturing dynamic shifts in executive affect between scripted presentation and spontaneous Q&A exchanges. Our key finding reveals a pronounced divergence in predictive capacity: while multimodal features do not forecast directional stock returns, they explain up to 43.8% of the out-of-sample variance in 30-day realized volatility. Importantly, volatility predictions are strongly driven by emotional dynamics during executive transitions from scripted to spontaneous speech, particularly reduced textual stability and heightened acoustic instability from CFOs, and significant arousal variability from CEOs. An ablation study confirms that our multimodal approach substantially outperforms a financials-only baseline, underscoring the complementary contributions of acoustic and textual modalities. By decoding latent markers of uncertainty from verifiable biometric signals, our methodology provides investors and regulators a powerful tool for enhancing market interpretability and identifying hidden corporate uncertainty.

Figures

Figures reproduced from arXiv: 2508.18653 by the authors.

Figure 1
Figure 1. Comparative analysis of MOS-LQO scores for [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Descriptive analysis of emotion signals. (a) il [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Feature Importance Comparison for Predicting [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Top 15 Stable Features for Predicting 30-Day [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Distribution of Feature Importance for Top Volatil [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 14 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Baevski, A.; Zhou, Y.; Mohamed, A.; and Auli, M. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33: 12449--12460

    work page 2020

  4. [4]

    Benesty, J.; Chen, J.; and Huang, Y. 2008. Microphone array signal processing. Springer

    work page 2008

  5. [5]

    Blackstock, D. T. 2000. Fundamentals of physical acoustics. John Wiley & Sons

    work page 2000

  6. [6]

    Chen, X.; Yu, X.; Chang, L.; Huang, Y.; He, J.; Zhang, S.; Li, J.; Lin, L.; Zeng, Z.; Tu, X.; et al. 2025. A Synergistic Framework of Nonlinear Acoustic Computing and Reinforcement Learning for Real-World Human-Robot Interaction. arXiv preprint arXiv:2505.01998

    work page Pith review arXiv 2025

  7. [7]

    Gulati, A.; Qin, J.; Chiu, C.-C.; Parmar, N.; Zhang, Y.; Yu, J.; Han, W.; Wang, S.; Zhang, Z.; Wu, Y.; et al. 2020. Conformer: Convolution-augmented transformer for speech recognition. arXiv preprint arXiv:2005.08100

    arXiv 2020

  8. [8]

    F.; and Blackstock, D

    Hamilton, M. F.; and Blackstock, D. T. 2024. Nonlinear acoustics. Springer Nature

    work page 2024

Show all 20 references
  1. [9]

    E.; Kevrekidis, I

    Karniadakis, G. E.; Kevrekidis, I. G.; Lu, L.; Perdikaris, P.; Wang, S.; and Yang, L. 2021. Physics-informed machine learning. Nature Reviews Physics, 3(6): 422--440

    2021

  2. [10]

    Li, Y.; Wang, S.; Ding, H.; and Chen, H. 2023. Large language models in finance: A survey. In Proceedings of the fourth ACM international conference on AI in finance, 374--382

    2023

  3. [11]

    Loizou, P. C. 2007. Speech enhancement: theory and practice. CRC press

    2007

  4. [12]

    Loughran, T.; and McDonald, B. 2011. When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks. The Journal of finance, 66(1): 35--65

    2011

  5. [13]

    J.; Parsons, C

    Mayew, W. J.; Parsons, C. A.; and Venkatachalam, M. 2013. Voice pitch and the labor market success of male chief executive officers. Evolution and Human Behavior, 34(4): 243--248

    2013

  6. [14]

    Raissi, M.; Perdikaris, P.; and Karniadakis, G. E. 2019. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378: 686--707

    2019

  7. [15]

    K.; Gopal, V.; Cutler, R.; Beyrami, E.; Cheng, R.; Dubey, H.; Matusevych, S.; Aichner, R.; Aazami, A.; Braun, S.; et al

    Reddy, C. K.; Gopal, V.; Cutler, R.; Beyrami, E.; Cheng, R.; Dubey, H.; Matusevych, S.; Aichner, R.; Aazami, A.; Braun, S.; et al. 2020. The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results. arXiv preprint arXiv:2...

    2020 arXiv

  8. [16]

    Russell, J. A. 1980. A circumplex model of affect. Journal of personality and social psychology, 39(6): 1161

    1980

  9. [17]

    Schuller, B.; and Batliner, A. 2013. Computational paralinguistics: emotion, affect and personality in speech and language processing. John Wiley & Sons

    2013

  10. [18]

    Su, H.; Qi, W.; Chen, J.; Yang, C.; Sandoval, J.; and Laribi, M. A. 2023. Recent advancements in multimodal human--robot interaction. Frontiers in Neurorobotics, 17: 1084000

    2023

  11. [19]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  12. [20]

    H.; Yang, S.-W.; Lin, Y.-F.; and Lee, H.-y

    Yin-Jyun, A.; Ken, L.; Chen, A. H.; Yang, S.-W.; Lin, Y.-F.; and Lee, H.-y. 2024. FunSoundLLM: A Multimodal Language Model for Understanding and Generating General Audio. arXiv:2405.15243

    2024 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.