Pith. sign in

REVIEW 4 major objections 5 minor 62 references

Asset Pricing in Pre-trained Transformer

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that its SERT model, an encoder-only Transformer with autoencoder pretraining and causal masks, achieves the highest out-of-sample $R^2$ in the COVID-19 period, and that its trend-following strategy beats buy-and-hold on…

desk verdict Survivorship bias and test-set selection sink the headline R2 claims, but the paper's negative results on Transformer tweaks are worth a second look. read the letter →

arxiv 2505.01575 v3 pith:NY6DDMV2 submitted 2025-05-02 q-fin.CP econ.EMq-fin.PR

Shanyan Lai This is my paper · ORCID
classification q-fin.CPecon.EMq-fin.PR
keywords StockpricingTransformerAttentionmechanismTemporaldependencySERTPretrainedFactorinvestingOut-of-sampleforecast
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that Transformer models, built for language, can be adapted to predict stock returns in sparse economic data, and that a specifically modified version does this better than the standard architecture exactly when markets are most turbulent. The proposed model, SERT, replaces BERT-style random masking with an MLP autoencoder pretraining step and enforces causal masks so future information cannot leak into the attention computation. Comparing ten Transformer variants and ten encoder-only variants over the pre-COVID, COVID, and one-year-post-COVID periods, the paper reports that the best SERT reaches 11.2% and 10.91% out-of-sample $R^2$ in the two turbulent periods, ahead of pre-trained Transformers at 10.38% and 9.15%. A sign-signal strategy built on the SERT predictions achieves a downside-risk-adjusted return about 47% higher than the equal-weighted buy-and-hold benchmark and 28% higher than the value-weighted benchmark during the pandemic. If these numbers hold, the paper's contribution is practical: Transformer-based return forecasts can hedge downside risk in crises rather than merely fitting calm markets.

What carries the argument

The load-bearing object is the causal, autoencoder-pretrained attention block. SERT modifies the standard Transformer's encoder self-attention by imposing an upper-triangular causal mask, and replaces BERT's random-word masking pretraining with an MLP autoencoder that maps the 182 factor inputs to 420 latent features before the Transformer body sees them. This pretraining step does double duty: it enlarges the factor dimension in the spirit of large-factor models and fills missing values, which the paper argues lets the attention mechanism capture temporal dependencies in sparse, noisy monthly return data. The causal mask is what stops the model from peeking at future months, and the pretrained dimension expansion is what the paper credits for the model's advantage during high-volatility periods.

What would settle it

Re-run the same SERT training and sign-signal backtest on a point-in-time universe: at each month from 2013 to 2022, trade every stock that was in the top market-cap bucket that month, and carry delisted stocks at their final return. If SERT's out-of-sample $R^2$ no longer exceeds the pre-trained Transformer benchmarks, or its downside-risk-adjusted advantage over buy-and-hold shrinks below the reported 47% and 28% margins, the look-ahead universe selection is the cause.

Watch

Extended reading notes

Core claim

SERT is a single-directional encoder-only Transformer: it keeps the encoder block of the standard Transformer, removes the decoder and cross-attention, adds causal masks to the self-attention layer so that each monthly forecast uses only past months, and prepends an MLP autoencoder that pretrains the 182 portfolio-sorted factors, projecting them to the 420-dimensional output space and simultaneously denoising missing values. The paper's central empirical discovery is that this architecture produces the top out-of-sample model fit in the COVID-19 period (11.2% $R^2$) and the year after (10.91% $R^2$), outperforming standard Transformers, standard encoder-only Transformers, and the pre-trained Transformer models, with pairwise error-difference tests marking the gap as significant only when volatility is extreme. On the strategy side, the discovery is that the simplest trading rule, going long when predicted and realized returns share the same sign, turns the model's crisis-period forecasts into downside-risk-adjusted returns above buy-and-hold, with the advantage concentrated in the pandemic window. The paper also reports negative findings about common Transformer enhancements: layer normalization first does not help, adding attention heads helps only marginally, and a softmax signal filter erases differences between models without improving risk-adjusted performance.

Load-bearing premise

The stock universe used for the out-of-sample test is built from stocks that are known today to have complete data through 2022, so the test never sees stocks that were delisted or went bankrupt during the evaluation period; the reported $R^2$ and downside-risk-adjusted returns would not be the ones an investor could actually have earned.

Editorial extensions

If this is right

  • If the reported out-of-sample $R^2$ is real, Transformer-based factor models are most informative when volatility is high, the regime where traditional linear factor models typically degrade.
  • The trend-following sign-signal strategy built from SERT forecasts offers a crisis-period hedge: during the pandemic it beats the equal-weighted and value-weighted buy-and-hold benchmarks on downside-risk-adjusted return by roughly 47% and 28%.
  • The softmax signal filter's failure to improve risk-adjusted returns implies that the model's edge is in the sign of the forecast, not in the ranking confidence; filtering signals only makes different architectures look alike.
  • Since attention head count has negligible effect on fit, practitioners can use low-head-count models for speed without losing forecast accuracy in this data regime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the 420-stock universe is selected using full data through 2022, the crisis-period $R^2$ and downside-risk-adjusted advantage are upper bounds for a real-time strategy; a point-in-time replication that includes delisted stocks is needed before the edge can be traded.
  • Editorial inference: the paper's own finding that performance improves with volatility suggests the architecture should be tested on higher-volatility cross-sections such as small caps or cryptocurrencies; if the mechanism is right, the gap over benchmarks should widen there.
  • Editorial inference: the flat response to head count implies the model's capacity can be shrunk well below current configurations, which would make the approach feasible for larger universes where per-stock computation matters.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a causal encoder-only Transformer variant called SERT, together with MLP-autoencoder-pretrained Transformers, for monthly excess-return prediction on 420 US large-cap stocks using 182 sorted-portfolio factors. It reports out-of-sample R² and trading-strategy performance across three test windows: a pre-COVID period ('1911'), a period containing the pandemic ('2112'), and a period containing one post-pandemic year ('2212'). The headline claim is that the best SERT model achieves the highest OOS R² (11.2% and 10.91%) during the extreme market periods and that its trend-following strategy achieves Sortino ratios about 47% (equal-weighted) and 28% (value-weighted) above buy-and-hold. The paper also examines attention-head counts, layer normalization first, and a softmax signal filter.

Significance. The architecture proposal is nontrivial and the application domain is relevant: adapting pretraining to numerical asset-pricing factors is a reasonable research direction, and the manuscript provides unusually detailed gradient derivations in Appendix B as well as a broad model comparison. If the empirical claims were supported, the paper would add useful evidence on whether simplified, causally masked encoder-only Transformers are competitive in crisis periods. However, the empirical protocol contains flaws that directly affect the central claims: a look-ahead survivorship filter in the stock universe, model selection performed on the test data, ambiguous/conflicting headline numbers, and overlapping test windows. These issues make the reported performance margins and crisis-period conclusions unsupported by the evidence as presented.

major comments (4)
  1. [Section 3, Tables 5, 10, 12] Section 3 states that the 420 stocks 'satisfy the conditions of having full available data in the testing period.' This is an explicit look-ahead survivorship filter: the universe is selected using information from the entire 2013–2022 test period, so stocks that were delisted or experienced data interruptions during that interval are excluded. All out-of-sample R² values and all strategy performance statistics, including the Sortino ratios in Tables 5, 10, and 12, are therefore computed on a survivor-only universe that is not investable in real time. This removes precisely the extreme downside outcomes that the paper claims SERT hedges. The 'too-big-to-fail' and 'going concern' arguments do not remove the information-timing problem: a practitioner forming a portfolio in 2013 cannot know which stocks will have full data through 2022. This flaw is load-bearing because the central crisis-period performance claim rests on these measurements.
  2. [Section 5.2, Section 5.3, Tables 5, 12, 13] The 'best' model is selected using the test data itself. In Section 5.2, SERT7 is identified as the best model for '2112' and SERT5 for '2212' based on the OOS R² column of Table 5. In Section 5.3, strategy models such as SERT5, SERT2, Trans6, and Trans3 are selected as best based on their Sortino ratios in Tables 8–11 and then compared with benchmarks in Tables 12–13. Since the same test periods are used for model selection and for reporting the headline margins, the reported 'best SERT' performance is inflated by test-set selection. The claims that SERT outperforms standard encoder-only Transformers by the reported margins are not supported without a nested validation or pre-specified model-selection rule.
  3. [Abstract] The abstract reports headline R² values that do not match the body of the paper. The supplied abstract text states '11.94% and 11.47%' for SERT and '11.13% and 9.72%' for pre-trained Transformers, while the paper's own abstract in the full text states '11.2% and 10.91%' and '10.38% and 9.15%'. Neither pair matches the body tables: Table 5 shows 0.1120 and 0.1091 as the best SERT values, and Table 3 shows 0.1038 and 0.0915 as the best pre-trained Transformer values. The reader cannot tell which numbers are the actual reported results, and this discrepancy directly affects the paper's principal quantitative claim.
  4. [Table 1, Section 5.1, Table 4] The three testing periods are nested: '2112' contains '1911' plus the pandemic, and '2212' contains '2112' plus the post-pandemic year. The OOS R² and Diebold–Mariano statistics for these periods are therefore not independent, and the paper treats them as three separate sets of evidence for the model's crisis-period advantage. The DM tests should account for the overlapping structure of the longer windows (for example, with heteroskedasticity- and autocorrelation-robust standard errors). Without this, the reported significance levels for '2112' and '2212' are overstated.
minor comments (5)
  1. [Section 5.3] The trading rule says positions are opened 'at the point that both predicted return and actual return have a positive sign.' If the 'actual return' is the realized return of the same month for which the prediction is made, the signal would not be available at the time of trade execution; please clarify the timing and confirm the strategy is implementable in real time.
  2. [Figures 19 and 20] The captions of Figure 19 and Figure 20 say 'Sign equal weighted accumulative return plots,' but the surrounding text and the tables they illustrate refer to value-weighted portfolios; the captions should be corrected.
  3. [Appendix B] The gradient derivations are a useful addition, but several equations contain apparent typesetting or transposition errors (e.g., Eq. B12), and the multi-head attention section is abbreviated; a careful proofread would improve reliability.
  4. [General] The paper does not provide code or a data-availability statement. Given that the empirical claims are extensive and the implementation details (e.g., exact hyperparameters per model) are only partially reported, a repository with code and data would be important for reproducibility.
  5. [Section 3] The text says 'evaluating the Sharp ratio' where 'Sharpe ratio' is intended; there are also several typographical errors (e.g., 'gradatim') and a stray '?' in reference [17].

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's out-of-sample results are computed from held-out data rather than derived from fitted inputs.

full rationale

The paper is an empirical benchmark comparison, not a derivation chain. The out-of-sample R2 in Eq. (54) is evaluated on held-out test months using predictions from models estimated on the in-sample window, with the training-period mean as the denominator; no reported statistic is algebraically identical to an estimated parameter or to the label-generating rule. The proposed SERT and pre-trained Transformer architectures combine standard Transformer components (Vaswani et al.; BERT; MLP autoencoders) with an MLP pre-training module, and each component is justified by external references rather than by the author's own prior results. The only self-reference is a note pointing to an earlier Zenodo version of the paper, which is not load-bearing. I considered whether the headline 'best proposed SERT model' is circular because the best head count is selected from the same OOS R2 values (Section 5.2). That is a genuine test-set selection and reporting issue, and it weakens the inferential force of the headline, but it is not circularity in the derivation sense: the reported R2 values are still computed from held-out returns, and selection among configurations does not make the numerical result equal to an input by construction. The survivorship filter in Section 3 is a separate validity concern, not a circular reduction; nothing in the paper defines a factor, model, or metric in terms of the target excess return that it is supposed to predict, and no load-bearing result is imported from a self-citation. Therefore no significant circularity is present.

Assumptions & free parameters 6 free parameters · 3 assumptions · 0 invented entities

The reported performance rests on several hand-set hyperparameters and on a survivorship-biased stock universe. No parameter-free derivation is given; the central numbers are empirical and depend on these choices.

free parameters (6)
  • MLP autoencoder latent dimension ratio = 0.7 (70% of input dimension)
    Set in Section 4.1.5/4.2; hand-chosen to balance representation and overfitting, not estimated.
  • Pretrain output dimension = 420 (number of stocks)
    Section 4.2; chosen to match the number of output stocks.
  • Learning rate eta = 0.001
    Appendix B.5 Adam settings; standard default, not tuned.
  • Validation window size = 30% of training window size
    Section 3; hand-chosen.
  • Softmax filter threshold = keep top 50% of signals
    Section 5.3; hand-chosen, filters out worst half of trading signals.
  • Attention head count for best model = 7 for SERT in 2112, 4 or 5 in 2212, 2 in 1911
    Section 5.2, Table 5; the 'best' head count is selected from the test set, making the reported headline a post-hoc maximum.
assumptions (3)
  • standard math Transformer attention and backpropagation definitions of Vaswani et al. (2017)
    The model assumes the validity of the attention mechanism and gradient-based training as in the cited paper.
  • domain assumption Monthly sorted-portfolio returns (182 factors) are valid observable risk factors
    Section 3 uses the factors from Zimmermann [52] without justifying that they span the true factor space.
  • ad hoc to paper Stocks with full data in the test period represent the investable large-cap universe
    Section 3 explicitly filters on future data availability; this is a look-ahead assumption that biases the evaluation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Asset Pricing in Pre-trained Transformer." pith.science (2026). https://pith.science/paper/NY6DDMV2

@misc{pith2026250501575,
  author       = {Pith},
  title        = {Pith review of: Asset Pricing in Pre-trained Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NY6DDMV2}},
  note         = {Machine review of arXiv:2505.01575}
}
abstract

This paper proposes an innovative Transformer model, Single-directional representative from Transformer (SERT), for US large capital stock pricing. It also innovatively applies the pre-trained Transformer models under the stock pricing and factor investment context. They are compared with standard Transformer models and encoder-only Transformer models in three periods covering the entire COVID-19 pandemic to examine the model adaptivity and suitability during the extreme market fluctuations. Namely, pre-COVID-19 period, COVID-19 period and 1-year post-COVID-19. The best proposed SERT model achieves the highest out-of-sample $R^2$, 11.94\% and 11.47\% respectively, when extreme market fluctuation takes place, followed by pre-trained Transformer models (11.13\% and 9.72\%). Their Trend-following-based strategy's performance also proves their excellent capability for hedging downside risks during market shocks. The proposed SERT model achieves a Sortino ratio 47\% higher than the buy-and-hold benchmark in the equal-weighted portfolio and 28\% higher in the value-weighted portfolio in the static transaction cost scenario when the pandemic period is considered. It proves that Transformer models have a strong ability to capture patterns of temporal sparsity in asset pricing factor models, especially with high volatility. I also find the softmax signal filter as the common configuration of Transformer models in alternative contexts, which only eliminates differences between models, but does not improve strategy-wise performance, while increasing attention heads improves the model performance insignificantly and applying the 'layer normalization first' method does not boost the model performance in our case.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 30 canonical work pages

  1. [1]

    Review of Financial Studies 33(5), 2223–2273 (2020) https://doi.org/10.1093/ rfs/hhaa009

    Gu, S., Kelly, B., Xiu, D.: Empirical Asset Pricing via Machine Learning. Review of Financial Studies 33(5), 2223–2273 (2020) https://doi.org/10.1093/ rfs/hhaa009

    2020

  2. [2]

    Journal of Econometrics 222(1), 429–450 (2021)

    Gu, S., Kelly, B., Xiu, D.: Autoencoder asset pricing models. Journal of Econometrics 222(1), 429–450 (2021)

    2021

  3. [3]

    Applied Intelligence 53(20), 23812–23833 (2023) https://doi.org/10.1007/s10489-023-04762-7

    Aydogan-Kilic, D., Selcuk-Kestel, A.S.: Modification of hybrid RNN-HMM model in asset pricing: univariate and multivariate cases. Applied Intelligence 53(20), 23812–23833 (2023) https://doi.org/10.1007/s10489-023-04762-7

    work page doi:10.1007/s10489-023-04762-7 2023

  4. [4]

    In: 2024 International Conference on Computer, Control, Informatics and Its Applications (IC3INA), pp

    Rukmana, P.U., Asriana, Fakhrurroja, H.: Application of LSTM, RNN, and Transformer in stock price prediction of information technology companies: A comparative analysis. In: 2024 International Conference on Computer, Control, Informatics and Its Applications (IC3INA), pp. 249–254. Institute of Electrical and Electronics Engineers Inc., Jakarta, Indonesia ...

    work page arXiv 2024

  5. [5]

    Advances in neural information processing systems 30 (2017)

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)

    2017

  6. [6]

    IEEE Access 12, 24164–24178 (2024) https://doi.org/10.1109/ACCESS

    Gezici, A.H.B., Sefer, E.: Deep Transformer-Based Asset Price and Direction Pre- diction. IEEE Access 12, 24164–24178 (2024) https://doi.org/10.1109/ACCESS. 2024.3358452

    arXiv 2024

  7. [7]

    P¨ aiv¨ arinta, M.: Transformer-based deep learning model for stock return forecast- ing: Empirical evidence (2022) 54

    work page 2022

  8. [8]

    Expert Systems with Applications 202, 117239 (2022)

    Zhang, Q., Qin, C., Zhang, Y., Bao, F., Zhang, C., Liu, P.: Transformer- based attention network for stock movement prediction. Expert Systems with Applications 202, 117239 (2022)

    work page 2022

Show all 62 references
  1. [9]

    Quantitative Finance, 1–26 (2024)

    Cheng, Y., Tang, K.: GPT’s idea of stock factors. Quantitative Finance, 1–26 (2024)

    2024

  2. [10]

    Available at SSRN 3554486 (2021)

    Cong, L.W., Tang, K., Wang, J., Zhang, Y.: AlphaPortfolio: Direct construction through deep reinforcement learning and interpretable AI. Available at SSRN 3554486 (2021)

    2021

  3. [11]

    International Review of Financial Analysis 90 (2023) https://doi.org/10.1016/j.irfa.2023.102876

    Ma, T., Wang, W., Chen, Y.: Attention is all you need: An interpretable transformer-based asset allocation approach. International Review of Financial Analysis 90 (2023) https://doi.org/10.1016/j.irfa.2023.102876

    2023

  4. [12]

    arXiv preprint arXiv:2209.12014 (2022)

    Zhang, C.: Asset pricing and deep learning. arXiv preprint arXiv:2209.12014 (2022)

    2022 arXiv

  5. [13]

    National Bureau of Economic Research (2024)

    Didisheim, A., Ke, S.B., Kelly, B.T., Malamud, S.: APT or “AIPT”? the sur- prising dominance of large factor models. National Bureau of Economic Research (2024)

    2024

  6. [14]

    Management Science 70(2), 714–750 (2024) https://doi.org/10.1287/mnsc.2023.4695

    Chen, L., Pelger, M., Zhu, J.: Deep Learning in Asset Pricing. Management Science 70(2), 714–750 (2024) https://doi.org/10.1287/mnsc.2023.4695

    2024

  7. [15]

    Available at SSRN 4999011 (2024)

    ZHOU, X., Wang, Y.: Learning from memory: Asset pricing via recurrent neural network and attention mechanism. Available at SSRN 4999011 (2024)

    2024

  8. [16]

    In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp

    Bahdanau, D., Cho, K.H., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp. 1–15 (2015)

    2015

  9. [17]

    In: Proceedings of the Asian Conference on Machine Learning, pp

    Li, H., Shen, Y., Zhu, Y.: Stock price prediction using attention-based multi-input lstm. In: Proceedings of the Asian Conference on Machine Learning, pp. 454–469. PMLR, Beijing, China (2018)

    2018

  10. [18]

    arXiv preprint arXiv:1508.04025 (2015)

    Luong, M.-T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015)

    2015 arXiv

  11. [19]

    Complexity 2019 (2019) https://doi.org/10.1155/2019/3582516

    Kim, T., Kim, H.Y., Tabak, B.M.: Optimizing the Pairs-Trading Strategy Using Deep Reinforcement Learning with Trading and Stop-Loss Boundaries. Complexity 2019 (2019) https://doi.org/10.1155/2019/3582516

    2019 doi

  12. [20]

    arXiv preprint arXiv:1410.3916 (2014)

    Weston, J., Chopra, S., Bordes, A.: Memory networks. arXiv preprint arXiv:1410.3916 (2014)

    2014 arXiv

  13. [21]

    In: Proceedings of the 2016 Conference of 55 the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp

    Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of 55 the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 14...

    2016

  14. [22]

    arXiv preprint arXiv:1704.02971 (2017)

    Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., Cottrell, G.: A dual-stage attention-based recurrent neural network for time series prediction. arXiv preprint arXiv:1704.02971 (2017)

    2017 arXiv

  15. [23]

    Expert Systems with Applications 143, 113082 (2020) https: //doi.org/10.1016/j.eswa.2019.113082

    Liu, Y., Gong, C., Yang, L., Chen, Y.: DSTP-RNN: A dual-stage two-phase attention-based recurrent neural network for long-term and multivariate time series prediction. Expert Systems with Applications 143, 113082 (2020) https: //doi.org/10.1016/j.eswa.2019.113082

    2020

  16. [24]

    Energy and Buildings 249 (2021) https://doi.org/10.1016/j.enbuild.2021

    Peng, J., Kimmig, A., Wang, J., Liu, X., Niu, Z., Ovtcharova, J.: Dual-stage attention-based long-short-term memory neural networks for energy demand pre- diction. Energy and Buildings 249 (2021) https://doi.org/10.1016/j.enbuild.2021. 111211

    2021 doi

  17. [25]

    Energy 244 (2022) https://doi.org/10.1016/j.energy.2022.123233

    Yang, K., Tang, Y., Zhang, S., Zhang, Z.: A deep learning approach to state of charge estimation of lithium-ion batteries based on dual-stage attention mechanism. Energy 244 (2022) https://doi.org/10.1016/j.energy.2022.123233

    2022

  18. [26]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., Zhang, W.: Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 11106–11115 (2021)

    2021

  19. [27]

    Applied and Computational Engineering 38, 90–95 (2024)

    Zhao, Z.: Application and challenges of informer model in financial time series prediction: A review. Applied and Computational Engineering 38, 90–95 (2024)

    2024

  20. [28]

    Advances in neural information processing systems 34, 22419–22430 (2021)

    Wu, H., Xu, J., Wang, J., Long, M.: Autoformer: Decomposition transform- ers with auto-correlation for long-term series forecasting. Advances in neural information processing systems 34, 22419–22430 (2021)

    2021

  21. [29]

    arXiv 2022

    Qian, T., Xie, A., Bruckmann, C.: Sensitivity Analysis on Transferred Neural Architectures of BERT and GPT-2 for Financial Sentiment Analysis. arXiv 2022. arXiv preprint arXiv:2207.03037

    2022 arXiv

  22. [30]

    In: Proceedings of the International Conference on Computational Intelligence, Data Science and Cloud Computing (IEM-ICDC 2021), pp

    Rehman, T., Bose, R., Chattopadhyay, S., Sanyal, D.K.: Transfer learning and transformer architecture for financial sentiment analysis. In: Proceedings of the International Conference on Computational Intelligence, Data Science and Cloud Computing (IEM-ICDC 2021), pp. 17–27. S...

    2022

  23. [31]

    IEEE Access (2024)

    Rizinski, M., Peshov, H., Mishev, K., Jovanovik, M., Trajanov, D.: Sentiment Analysis in Finance: From Transformers Back to eXplainable Lexicons (XLex). IEEE Access (2024)

    2024

  24. [32]

    Journal of Computer Science and Technology Studies6(1), 189–194 (2024)

    Ferdus, M.Z., Anjum, N., Nguyen, T.N., Jisan, A.H., Raju, M.A.H.: The Influence 56 of Social Media on Stock Market: A Transformer-Based Stock Price Forecasting with External Factors. Journal of Computer Science and Technology Studies6(1), 189–194 (2024)

    2024

  25. [33]

    arXiv preprint arXiv:2407.15788 (2024)

    Dolphin, R., Dursun, J., Chow, J., Blankenship, J., Adams, K., Pike, Q.: Extract- ing structured insights from financial news: An augmented llm driven approach. arXiv preprint arXiv:2407.15788 (2024)

    2024 arXiv

  26. [34]

    International Journal of Data Science and Analytics, 1–25 (2024)

    Miori, D., Petrov, C.: Narratives from GPT-derived networks of news and a link to financial markets dislocations. International Journal of Data Science and Analytics, 1–25 (2024)

    2024

  27. [35]

    arXiv preprint arXiv:2405.12990 (2024)

    Sharkey, E., Treleaven, P.: BERT vs GPT for financial engineering. arXiv preprint arXiv:2405.12990 (2024)

    2024 arXiv

  28. [36]

    Complexity 2022 (2022) https://doi

    Li, Y., Lv, S., Liu, X., Zhang, Q.: Incorporating Transformers and Attention Networks for Stock Movement Prediction. Complexity 2022 (2022) https://doi. org/10.1155/2022/7739087

    2022 doi

  29. [37]

    arXiv preprint arXiv:2305.06140 (2023)

    Kaplan, H., Mundani, R.-P., R¨ olke, H., Weichselbraun, A.: CrudeBERT: applying economic theory towards fine-tuning transformer-based sentiment analysis models to the crude oil market. arXiv preprint arXiv:2305.06140 (2023)

    2023 arXiv

  30. [38]

    arXiv preprint arXiv:1908.10063 (2019)

    Araci, D.: Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063 (2019)

    2019 arXiv

  31. [39]

    Central Banker Speeches, Tail Stock Returns (September 07, 2024) (2024)

    Gado, K.: Transformers, Central Banker Speeches, Tail Stock Returns. Central Banker Speeches, Tail Stock Returns (September 07, 2024) (2024)

    2024

  32. [40]

    arXiv preprint arXiv:2406.11903 (2024)

    Nie, Y., Kong, Y., Dong, X., Mulvey, J.M., Poor, H.V., Wen, Q., Zohren, S.: A survey of large language models for financial applications: Progress, prospects and challenges. arXiv preprint arXiv:2406.11903 (2024)

    2024 arXiv

  33. [41]

    arXiv preprint arXiv:2308.01430 (2023)

    Wang, Z., Li, Y., Wu, J., Soon, J., Zhang, X.: Finvis-gpt: A multimodal large language model for financial chart analysis. arXiv preprint arXiv:2308.01430 (2023)

    2023 arXiv

  34. [42]

    European Journal of Operational Research 308(1), 306–320 (2023) https://doi.org/10.1016/j.ejor.2022.10.032

    Korangi, K., Mues, C., Bravo, C.: A transformer-based model for default predic- tion in mid-cap corporate markets. European Journal of Operational Research 308(1), 306–320 (2023) https://doi.org/10.1016/j.ejor.2022.10.032

    2023 doi

  35. [43]

    In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), vol

    Sun, J., Fu, W., Lin, J., Jiang, Y., Xia, S.: Deep portfolio optimization modeling based on conv-transformers with graph attention mechanism. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), vol. 2022- July. Institute of Electrical and Electron...

    2022

  36. [44]

    In: Proceedings of the International Conference on Circuit Power and Computing Technologies (ICCPCT 2023), pp

    Tevare, V., Revankar, P.S.: Forecasting stock prices with stack transformer. In: Proceedings of the International Conference on Circuit Power and Computing Technologies (ICCPCT 2023), pp. 1262–1269. Institute of Electrical and Electron- ics Engineers Inc., Kollam, India (2023)...

    2023

  37. [45]

    John Wiley and Sons Inc (2024)

    Zhang, C., Sjarif, N.N.A., Ibrahim, R.: Deep learning models for price forecasting of financial time series: A review of recent advancements: 2020–2022. John Wiley and Sons Inc (2024). https://doi.org/10.1002/widm.1519

    2024 doi

  38. [46]

    arXiv preprint arXiv:2411.16585 (2024)

    Wheeler, A., Varner, J.D.: MarketGPT: Developing a Pre-trained transformer (GPT) for Modeling Financial Time Series. arXiv preprint arXiv:2411.16585 (2024)

    2024 arXiv

  39. [47]

    Available at SSRN 4375798 (2023)

    Lezmi, E., Xu, J.: Time series forecasting with transformer models and application to asset management. Available at SSRN 4375798 (2023)

    2023

  40. [48]

    Available at SSRN 4507511 (2024)

    Gabaix, X., Koijen, R.S.J., Richmond, R., Yogo, M.: Asset embeddings. Available at SSRN 4507511 (2024)

    2024

  41. [49]

    arXiv preprint arXiv:2403.06779 (2024)

    Ye, J., Goswami, B., Gu, J., Uddin, A., Wang, G.: From factor models to deep learning: Machine learning in reshaping empirical asset pricing. arXiv preprint arXiv:2403.06779 (2024)

    2024 arXiv

  42. [50]

    Available at SSRN (2024)

    Giesecke, O.: AI at the frontier of economic research. Available at SSRN (2024)

    2024

  43. [51]

    Available at SSRN 4672561 (2023)

    Giesecke, O.: State Government Debt Structure. Available at SSRN 4672561 (2023)

    2023

  44. [52]

    Centre for Financial Research (CFR), working paper (2020)

    Andrew Y.; Zimmermann, T.C.: Open source cross-sectional asset pricing. Centre for Financial Research (CFR), working paper (2020)

    2020

  45. [53]

    nature 323(6088), 533–536 (1986)

    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back- propagating errors. nature 323(6088), 533–536 (1986)

    1986

  46. [54]

    arXiv preprint arXiv:2003.10555 (2020)

    Clark, K.: Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555 (2020)

    2020 arXiv

  47. [55]

    Journal of machine learning research 21(140), 1–67 (2020)

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21(140), 1–67 (2020)

    2020

  48. [56]

    In: Proceedings of the Winter Simulation Conference (WSC), vol

    Liu, Y., Yan, L., Liu, S., Jiang, T., Zhang, F., Wang, Y., Wu, S.: Enhancing input parameter estimation by machine learning for the simulation of large-scale logistics networks. In: Proceedings of the Winter Simulation Conference (WSC), vol. 2020, pp. 608–619. Institute of Ele...

    2020

  49. [57]

    In: Proceedings of naacL-HLT, vol

    Kenton, J.D.M.-W.C., Toutanova, L.K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of naacL-HLT, vol. 1. Minneapolis, Minnesota, USA (2019)

    2019

  50. [58]

    arXiv preprint arXiv:2108.08999 (2021)

    Cong, L.W., Tang, K., Wang, J., Zhang, Y.: Deep sequence modeling: Devel- opment and applications in asset pricing. arXiv preprint arXiv:2108.08999 (2021)

    2021 arXiv

  51. [59]

    Journal of Business and Economic Statistics 13(3), 253–263 (1995)

    Diebold, F.X., Mariano, R.S.: Comparing predictive accuracy. Journal of Business and Economic Statistics 13(3), 253–263 (1995)

    1995

  52. [60]

    The Journal of finance 23(2), 389–416 (1968)

    Jensen, M.C.: The performance of mutual funds in the period 1945-1964. The Journal of finance 23(2), 389–416 (1968)

    1968

  53. [61]

    Economet- rica 47, 278 (1979)

    Kahneman, D.: Prospect theory: An analysis of decisions under risk. Economet- rica 47, 278 (1979)

    1979

  54. [62]

    Liu, L., Liu, X., Gao, J., Chen, W., Han, J.: Understanding the difficulty of training transformers. arXiv preprint arXiv:2004.08249 (2020) 59 Appendix A Distribution charts of SERT3, SERT5 and SERT7 (a) SER T3 1911 (b) SER T5 1911 (c) SER T7 1911 (d) SER T3 2112 (e) SER T5 21...

    2020 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.