Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Your Spending Needs Attention: Modeling Financial Habits with Transformers

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a transformer trained on transaction strings as text, fused with tabular features, improves recommendation AUC by 1.25% over a hand-crafted baseline.

desk verdict Genuinely new tokenization and fusion, but the AUC claim is confounded by user overlap; needs a user-disjoint split before taking the foundation-model claim at face value. read the letter →

arxiv 2507.23267 v1 pith:QM4SXGWZ submitted 2025-07-31 cs.IR

D. T. Braithwaite , Misael Cavalcanti , R. Austin McEver , Hiroto Udagawa , Daniel Silva , Rohan Ramanath , Felipe Meneses , Arissa Yoshida
show 4 more authors
Evan Wingert Matheus Ramos Brian Zanfelice Aman Gupta
This is my paper
classification cs.IR
keywords transactiondatamodelingself-supervisedlearningtransformerfoundationmodelssequentialrecommendationtabulardeepjointfusionfinancialmachinenext-tokenprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that raw bank transactions, read as text, can teach a transformer model to represent customer financial behavior better than hand-engineered tabular features. The authors propose nuFormer, which tokenizes each transaction into a compact set of special tokens for amount and date plus the merchant description, then pre-trains a causal transformer to predict the next token across hundreds of billions of transactions. They then fine-tune the model jointly with a deep network that combines the learned user embedding with existing tabular features. On a large-scale recommendation task predicting whether a customer will adopt a financial product, this joint-fusion model improves test AUC by 1.25% relative to a gradient-boosted baseline that uses only the hand-crafted features. The result matters because the improvement comes entirely from better learned representations, not from adding any new data source.

What carries the argument

The central machinery is a tokenization scheme that turns each transaction into a short string of special tokens (amount sign, amount bucket, month, day, weekday) followed by a byte-pair-encoded description, so that a member's entire history becomes one causal text sequence trained with the standard next-token prediction objective. Pre-training on this sequence yields a user embedding. The load-bearing device for the downstream gain is joint fusion, an end-to-end fine-tuning setup in which the transformer's final-token embedding is concatenated with a DCNv2 tabular-feature network (using periodic linear embeddings for numerical attributes) and the whole model is trained together with LoRA to avoid overfitting.

What would settle it

Run a prospective randomized experiment that assigns eligible customers to either the nuFormer-based recommendation model or the hand-crafted-feature baseline, and compare actual product adoption and churn after six months; if the 1.25% offline AUC advantage does not produce a statistically significant improvement in these business metrics, the paper's central claim is refuted.

Watch

Extended reading notes

Core claim

The central discovery is that a transformer-based foundation model pre-trained on raw transaction strings can produce user embeddings that, when fused end-to-end with tabular features, yield a 1.25% relative AUC improvement on a production recommendation task at a large digital bank, roughly three times the lift the authors associate with a model launch that produces material business impact. The paper attributes this gain to joint fusion: instead of freezing the embedding and training a separate classifier, nuFormer fine-tunes the transformer and a DCNv2-based tabular network together, letting the transformer discover interactions between the tabular features and the transaction sequence. The authors also show that the gain improves with model size (24M vs 330M parameters), context length, and fine-tuning data volume, and that not all transaction sources help when the context window is limited.

Load-bearing premise

The business-value claim depends on the paper's stated assumption that improvements in offline AUC have historically tracked the online business outcome; if that correlation does not hold for this model, the 1.25% AUC gain does not by itself establish the reported 4.4% churn reduction.

Editorial extensions

If this is right

  • If the 1.25% lift holds in production, a bank can improve a recommendation model without acquiring any new data; the gain comes from modeling existing transactions better.
  • Scaling model size from 24M to 330M parameters and increasing context length both improved AUC, so the authors project that larger transaction models will continue to pay off.
  • Because the tokenization is attribute-agnostic, the same pre-trained model can be fine-tuned for other financial tasks such as risk scoring, fraud detection, or churn prediction, though those tasks are not evaluated here.
  • The result that some transaction sources hurt performance in a limited context window implies that deciding which transactions to include is itself a modeling lever, not just a data availability question.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the reported gain is attributed to representation learning rather than new data, a strong test is to see whether the transformer still helps when the tabular feature set already includes the same transaction-derived aggregates; if the lift vanishes, the contribution may be feature engineering rather than learned representation.
  • The paper presents one deployment outcome (churn reduction) without a confidence interval; a full randomized experiment comparing nuFormer to the baseline on the actual business metric would be the natural next step to confirm the business-value claim.
  • The observed negative contribution of some data sources under a fixed context window suggests an optimization problem that the paper does not solve: choosing which transactions to keep in the context per user. A learned or heuristic selector could extract more signal from the same data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces nuFormer, a causal transformer trained with next-token prediction on tokenized bank transactions, where each transaction is flattened into special tokens for amount and date plus a BPE-tokenized description. The authors propose joint fusion, which combines a LoRA-finetuned transformer embedding with a DCNv2 tabular network using periodic embeddings for numerical features. On a Nubank recommendation task with 203M training rows, nuFormer achieves a 1.25% relative test AUC improvement over a LightGBM baseline trained on 291 tabular features. The paper also reports scaling ablations (model size, context length, data volume) and a production deployment claim of 4.4% relative churn reduction.

Significance. If the result is robust, the paper shows a practical path to replacing hand-crafted transaction features with learned representations in a large financial institution, with a substantial relative AUC gain and no new data sources. Strengths include the large-scale evaluation (203M rows), consistent scaling behavior, the use of a strong tabular baseline, and the attempt to verify with an extended out-of-time test. However, the central claim that the gain comes from generalizable representation learning is not fully established because of the evaluation split design and the lack of a pretraining ablation.

major comments (3)
  1. [§4.1, §4.4] The train/test split is disjoint in time but not by user. Because 'the same user might occur multiple times in the dataset, but at different times' (Sec. 4.1), a test row can contain a user whose transaction history was seen in training. The transformer with a 2048-token raw transaction context can then retrieve that user's identity/history, whereas the LightGBM baseline only receives 291 aggregated tabular features. This user-overlap confound could account for part or all of the reported +1.25% relative AUC. Please add a user-disjoint (cold-user) evaluation, e.g., hold out all rows of a random subset of users, and report the AUC gain under that split. The extended out-of-time test in Sec. 4.4 does not remove this confound because users seen in training are not excluded.
  2. [§4.3] The paper does not compare nuFormer to the same transformer architecture trained directly on the supervised recommendation task without self-supervised pretraining (or with randomly initialized weights). All reported ablations vary model size, context length, and data volume, but none isolate the contribution of the next-token-prediction pretraining step. Without this control, the gain could come from the transformer architecture or the tokenization rather than from SSL pretraining. Please include a from-scratch training curve for at least one model size and show that pretraining is necessary for the reported lift.
  3. [§4.4] The business-impact claims are not supported by the evidence presented. The statement that +1.25% relative AUC is '3x a typical model launch' and the churn-reduction figure of 4.4% rest on the assertion that 'the business outcome closely follows improvements in offline AUC', but no data or reference for this correlation is given, and the measurement of the 4.4% churn reduction is not described (e.g., was it a randomized A/B test, an observational comparison, or an offline estimate? over what population and time horizon?). Please specify the evaluation protocol for the churn result and provide evidence for the AUC-to-business relationship, or remove these claims and confine the paper to the offline AUC result.
minor comments (5)
  1. [§4.3.1, Table 3] The units of 'Absolute AUC Change/Gain' are inconsistent. Table 2 values (e.g., -20.52) and Table 3 values (0.3123, 0.5177) appear to be in different scales; please state units explicitly (percentage points vs. AUC units) and make the reporting uniform.
  2. [§4.1] The claim that 'we can remove most of the hand-crafted transaction features without any loss in performance' is not supported by any experiment in the paper; please either add the supporting result or remove the claim.
  3. [§5] The sentence 'These foundation models can be leverage for tasks across Nubank' contains a grammatical error; change 'leverage' to 'leveraged'.
  4. [§3.2] The statement 'finetuning the entire transformer often leads to overfitting and catastrophic forgetting' is presented without supporting experiment or citation; please provide evidence or a reference.
  5. [References] Reference [19] is an old technical report by Jordan (1986); a more standard citation for the computational advantages of transformers over RNNs would be more useful to readers.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the 1.25% relative AUC gain is an honest out-of-sample comparison, and the acknowledged feature redundancy is not a derivation loop.

full rationale

The paper's central claim is an empirical out-of-sample measurement, not a derivation from its own outputs. A transformer is pretrained with next-token prediction on raw tokenized transactions and fine-tuned on a training window, then evaluated on a held-out test set whose time periods are disjoint from training (Section 4.1). The +1.25% relative test AUC reported in Table 4 is compared against a LightGBM baseline trained only on tabular features, so it does not reduce by construction to a fitted parameter or to a self-citation. The paper explicitly acknowledges that the challenger setup contains redundancy because some tabular features are transaction-derived ('such a challenger setup contains redundancy'), and it tests the representation-learning mechanism through controlled ablations of model size, context length, data volume, and data-source combinations (Tables 2-3, Figures 6-7), which would be meaningless if the result were definitionally forced. The architectural priors (text-is-all-you-need, PLR numerical embeddings, DCNv2) are credited to independent prior work rather than to the authors' own unverified claims. The most load-bearing caveat is the unsupported business-impact correlation asserted in Section 4.4 ('we have historically recorded that the business outcome closely follows improvements in offline AUC'); this is a validity risk for the churn claim, not a circular derivation. No circular step can be exhibited via the paper's own equations or citations.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new theoretical entities; all components are standard ML building blocks. The main ledger items are hand-chosen preprocessing choices (amount bins, context length, model sizes) and the domain assumptions that NTP on stringified transactions and text descriptions carry predictive signal. The most consequential assumption is the offline-AUC-to-business-outcome correlation used for the production claim.

free parameters (4)
  • Amount bucket boundaries and count (21 bins) = unspecified (21 bins)
    The tokenization in Eq. 1 maps continuous amounts to 21 discrete buckets; the bin edges are not stated and are presumably chosen from data distributions. The central claims depend on this encoding because the model only sees binned amounts.
  • Context length = 2048 tokens
    Chosen for training on a single A100 80GB GPU; ablations show longer contexts help, so this is a hand-chosen capacity that bounds what the model can see.
  • Transformer hidden size for 24M vs 330M models = 256 vs 1024
    Two model sizes are compared; the choice of these sizes is a scaling study input, not fitted to the target metric.
  • DCNv2/PLR hyperparameters (embedding dims, frequencies, LoRA rank) = unspecified
    The paper says PLR is used with parametrized frequencies and L2 regularization, but exact dimensions and LoRA configuration are not given; these affect the reported AUC.
assumptions (5)
  • domain assumption Next-token prediction on stringified transactions is a useful SSL objective for learning behavioral embeddings.
    Section 3.1: the model is pre-trained with NTP on the tokenized member sequence; it is assumed this learns representations useful for downstream tasks, analogous to NLP.
  • domain assumption Transaction text descriptions contain predictive signal beyond tabular aggregates.
    Section 1: the paper motivates including natural language descriptions, hypothesizing value in text features; no ablation isolates the text contribution.
  • standard math NoPE (no positional embeddings) still yields usable position information through the causal mask for long transaction sequences.
    Section 3.1, citing Kazemnejad et al. [21]; this is a background result the authors rely on.
  • domain assumption Offline AUC improvements historically correspond to business outcomes at Nubank.
    Section 4.4: 'we have historically recorded that the business outcome closely follows improvements in offline AUC'; this underpins the churn-reduction claim and the '3x' framing.
  • domain assumption DCNv2+PLR can match LightGBM on the tabular features so that the observed gain is attributable to the embeddings.
    Section 4.2, Table 1: parity is shown on this one dataset; the paper acknowledges DNN tabular performance varies by dataset (citing [26]).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Your Spending Needs Attention: Modeling Financial Habits with Transformers." pith.science (2026). https://pith.science/paper/QM4SXGWZ

@misc{pith2026250723267,
  author       = {Pith},
  title        = {Pith review of: Your Spending Needs Attention: Modeling Financial Habits with Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QM4SXGWZ}},
  note         = {Machine review of arXiv:2507.23267}
}
read the original abstract

Predictive models play a crucial role in the financial industry, enabling risk prediction, fraud detection, and personalized recommendations, where slight changes in core model performance can result in billions of dollars in revenue or losses. While financial institutions have access to enormous amounts of user data (e.g., bank transactions, in-app events, and customer support logs), leveraging this data effectively remains challenging due to its complexity and scale. Thus, in many financial institutions, most production models follow traditional machine learning (ML) approaches by converting unstructured data into manually engineered tabular features. Conversely, other domains (e.g., natural language processing) have effectively utilized self-supervised learning (SSL) to learn rich representations from raw data, removing the need for manual feature extraction. In this paper, we investigate using transformer-based representation learning models for transaction data, hypothesizing that these models, trained on massive data, can provide a novel and powerful approach to understanding customer behavior. We propose a new method enabling the use of SSL with transaction data by adapting transformer-based models to handle both textual and structured attributes. Our approach, denoted nuFormer, includes an end-to-end fine-tuning method that integrates user embeddings with existing tabular features. Our experiments demonstrate improvements for large-scale recommendation problems at Nubank. Notably, these gains are achieved solely through enhanced representation learning rather than incorporating new data sources.

Figures

Figures reproduced from arXiv: 2507.23267 by the authors.

Figure 1
Figure 1. This figure shows the process of converting a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. This figure shows the process of constructing a [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Modeling member sequences starts with the [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: This figure shows how a pre-trained transaction [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: This figure shows the difference between late fusion [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: nuFormer test AUC for different context lengths (512, 1024, 2048) with 24M and 330M models. with 24 attention layers, each with 16 attention heads. Importantly, the two model configurations differ only in the hidden embedding size where the 330M model has a 1024 length…
Figure 8
Figure 8. Figure 8: Extended test set (out-of-time) stability analysis. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. PRAGMA: Revolut Foundation Model

    cs.LG 2026-04 unverdicted novelty 5.0 of 10

    PRAGMA pre-trains a Transformer on heterogeneous banking events with a tailored self-supervised masked objective, yielding embeddings that support strong downstream performance on credit scoring, fraud detection, and ...

  2. A Foundation Model for Multimodal Event Sequences in Financial Applications

    cs.LG 2026-07 conditional novelty 4.5 of 10

    Early-fusion next-event pretraining on multimodal bank event sequences yields reusable user embeddings that, combined with engineered features, improve multi-task financial predictions and production NPV.

Reference graph

Works this paper leans on

46 extracted references · 9 canonical work pages · cited by 2 Pith papers

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, et al. 2021. Speecht5: Unified-modal encoder-decoder pre-training for spoken language processing. arXiv preprint arXiv:2110.07205 (2021)

    arXiv 2021

  3. [3]

    Muhammad Awais, Muzammal Naseer, Salman Khan, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, and Fahad Shahbaz Khan

  4. [4]

    Dmitrii Babaev, Nikita Ovsov, Ivan Kireev, Maria Ivanova, Gleb Gusev, Ivan Nazarov, and Alexander Tuzhilin. 2022. Coles: Contrastive learning for event se- quences with self-supervision. In Proceedings of the 2022 International Conference on Management of Data . 1190–1199

    work page 2022

  5. [5]

    Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawel- czyk, and Gjergji Kasneci. 2022. Deep neural networks and tabular data: A survey. Your Spending Needs Attention: Modeling Financial Habits with Transformers IEEE transactions on neural networks and learning systems (2022)

    work page 2022

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901

    2020

  7. [7]

    Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining . 785–794

    2016

  8. [8]

    Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems. 191–198

    2016

Show all 46 references
  1. [9]

    Tri Dao. 2023. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691 (2023)

    2023 arXiv

  2. [10]

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. Flashat- tention: Fast and memory-efficient exact attention with io-awareness. Advances in neural information processing systems 35 (2022), 16344–16359

    2022

  3. [11]

    Nilaksh Das, Saket Dingliwal, Srikanth Ronanki, Rohit Paturi, Zhaocheng Huang, Prashant Mathur, Jie Yuan, Dhanush Bekal, Xing Niu, Sai Muralidhar Jayanthi, et al. 2024. Speechverse: A large-scale generalizable audio language model. arXiv preprint arXiv:2405.08295 (2024)

    2024 arXiv

  4. [12]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  5. [13]

    Hao Ding, Yifei Ma, Anoop Deoras, Yuyang Wang, and Hao Wang. 2021. Zero- shot recommender systems. arXiv preprint arXiv:2105.08318 (2021)

    2021 arXiv

  6. [14]

    Philip Gage. 1994. A new algorithm for data compression. The C Users Journal 12, 2 (1994), 23–38

    1994

  7. [15]

    Yury Gorishniy, Ivan Rubachev, and Artem Babenko. 2022. On embeddings for numerical features in tabular deep learning. Advances in Neural Information Processing Systems 35 (2022), 24991–25004

    2022

  8. [16]

    Jesse Harte, Wouter Zorgdrager, Panos Louridas, Asterios Katsifodimos, Diet- mar Jannach, and Marios Fragkoulis. 2023. Leveraging large language models for sequential recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems. 1096–1102

    2023

  9. [17]

    Yupeng Hou, Shanlei Mu, Wayne Xin Zhao, Yaliang Li, Bolin Ding, and Ji-Rong Wen. 2022. Towards universal sequence representation learning for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Dis- covery and Data Mining . 585–593

    2022

  10. [18]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv 2021. arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  11. [19]

    MI Jordan. 1986. Serial order: a parallel distributed processing approach. technical report, june 1985-march 1986 . Technical Report. California Univ., San Diego, La Jolla (USA). Inst. for Cognitive Science

    1986

  12. [20]

    Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recom- mendation. In 2018 IEEE international conference on data mining (ICDM) . IEEE, 197–206

    2018

  13. [21]

    Amirhossein Kazemnejad, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Payel Das, and Siva Reddy. 2023. The impact of positional encoding on length general- ization in transformers. Advances in Neural Information Processing Systems 36 (2023), 24892–24928

    2023

  14. [22]

    Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems 30 (2017)

    2017

  15. [23]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classifi- cation with deep convolutional neural networks. Advances in neural information processing systems 25 (2012)

    2012

  16. [24]

    Chao Li, Zhiyuan Liu, Mengmeng Wu, Yuchi Xu, Huan Zhao, Pipei Huang, Guoliang Kang, Qiwei Chen, Wei Li, and Dik Lun Lee. 2019. Multi-interest network with dynamic routing for recommendation at Tmall. In Proceedings of the 28th ACM international conference on information and kn...

    2019

  17. [25]

    Jiacheng Li, Ming Wang, Jin Li, Jinmiao Fu, Xin Shen, Jingbo Shang, and Julian McAuley. 2023. Text is all you need: Learning language representations for sequential recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 1258–1267

    2023

  18. [26]

    Duncan McElfresh, Sujay Khandagale, Jonathan Valverde, Vishak Prasad C, Ganesh Ramakrishnan, Micah Goldblum, and Colin White. 2023. When do neural nets outperform boosted trees on tabular data? Advances in Neural Information Processing Systems 36 (2023), 76336–76369

    2023

  19. [27]

    Nikil Pancha, Andrew Zhai, Jure Leskovec, and Charles Rosenberg. 2022. Pinner- former: Sequence modeling for user representation at pinterest. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining . 3702–3712

    2022

  20. [28]

    Qi Pi, Weijie Bian, Guorui Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Practice on long sequential user behavior modeling for click-through rate prediction. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining . 2671–2679

    2019

  21. [29]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sand- hini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al

  22. [30]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. In International conference on machine learning . PMLR, 28492–28518

    2023

  23. [31]

    Kaushik Rangadurai, Yiqun Liu, Siddarth Malreddy, Xiaoyi Liu, Piyush Mahesh- wari, Vishwanath Sangale, and Fedor Borisyuk. 2022. Nxtpost: User to post recommendations in facebook groups. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . ...

    2022

  24. [32]

    David E Rumelhart, Geoffrey E Hinton, Ronald J Williams, et al. 1985. Learning internal representations by error propagation

    1985

  25. [33]

    Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015)

    2015 arXiv

  26. [34]

    Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. 2024. Flashattention-3: Fast and accurate attention with asynchrony and low-precision. Advances in Neural Information Processing Systems 37 (2024), 68658–68685

    2024

  27. [35]

    Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional net- works for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

    2014 arXiv

  28. [36]

    Piotr Skalski, David Sutton, Stuart Burrell, Iker Perez, and Jason Wong. 2023. Towards a foundation purchasing model: Pretrained generative autoregression on transaction sequences. In Proceedings of the Fourth ACM International Conference on AI in Finance . 141–149

    2023

  29. [37]

    Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang

  30. [38]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

    2023 arXiv

  31. [39]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)

    2017

  32. [40]

    Ruoxi Wang, Rakesh Shivanna, Derek Cheng, Sagar Jain, Dong Lin, Lichan Hong, and Ed Chi. 2021. Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. In Proceedings of the web conference 2021 . 1785–1797

    2021

  33. [41]

    Dongchao Yang, Jinchuan Tian, Xu Tan, Rongjie Huang, Songxiang Liu, Xuankai Chang, Jiatong Shi, Sheng Zhao, Jiang Bian, Xixin Wu, et al . 2023. Uniaudio: An audio foundation model toward universal audio generation. arXiv preprint arXiv:2310.00704 (2023)

    2023 arXiv

  34. [42]

    Guri Zabërgja, Arlind Kadra, and Josif Grabocka. 2024. Tabular Data: Is Attention All You Need? arXiv preprint arXiv:2402.03970 (2024)

    2024

  35. [43]

    Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. 2023. Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities. arXiv preprint arXiv:2305.11000 (2023)

    2023 arXiv

  36. [2019]

    In Proceedings of the 28th ACM international conference on information and knowledge management

    BERT4Rec: Sequential recommendation with bidirectional encoder rep- resentations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management . 1441–1450

  37. [2021]

    In International conference on machine learning

    Learning transferable visual models from natural language supervision. In International conference on machine learning . PmLR, 8748–8763

  38. [2023]

    arXiv preprint arXiv:2307.13721 (2023)

    Foundational Models Defining a New Era in Vision: A Survey and Outlook. arXiv preprint arXiv:2307.13721 (2023)

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.