RECORD
TriMLP: Revenge of a MLP-like Architecture in Sequential Recommendation
Machine Learning · Submitted 24 May 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Empirical study on 12 datasets of different scales (50K\textasciitilde 10M user-item interactions) from 4 benchmarks (Amazon, MovieLens, Tenrec and LBSN) show that TriMLP consistently attains promising accuracy/efficiency trade-off, where the average performance boost against several state-of-the-art baselines achieves up to 14.88% with 8.65% less inference cost.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
In this paper, we present a MLP-like architecture for sequential recommendation, namely TriMLP, with a novel Triangular Mixer for cross-token communications. In designing Triangular Mixer, we simplify the cross-token operation in MLP as the basic matrix multiplication, and drop the lower-triangle neurons of the weight matrix to block the anti-chronological order connections from future tokens. Accordingly, the information leakage issue can be remedied and the prediction capability of MLP can be fully excavated under the standard auto-regressive mode. Take a step further, the mixer serially alternates two delicate MLPs with triangular shape, tagged as global and local mixing, to separately capture the long range dependencies and local patterns on fine-grained level, i.e., long and short-term preferences. Empirical study on 12 datasets of different scales (50K\textasciitilde 10M user-item interactions) from 4 benchmarks (Amazon, MovieLens, Tenrec and LBSN) show that TriMLP consistently attains promising accuracy/efficiency trade-off, where the average performance boost against several state-of-the-art baselines achieves up to 14.88% with 8.65% less inference cost.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | In this paper, we present a MLP-like architecture for sequential recommendation, namely TriMLP, with a novel Triangular Mixer for cross-token communications. | contribution 0.97 | 0.91 | 0.07 | asserted 0.99 |
| s1 | In designing Triangular Mixer, we simplify the cross-token operation in MLP as the basic matrix multiplication, and drop the lower-triangle neurons of the weight matrix to block the anti-chronological order connections from future tokens. | method 0.99 | 0.93 | 0.05 | asserted 0.99 |
| s2 | Accordingly, the information leakage issue can be remedied and the prediction capability of MLP can be fully excavated under the standard auto-regressive mode. | interpretation 0.88 | 0.49 | 0.17 | asserted 0.85 |
| s3 | Take a step further, the mixer serially alternates two delicate MLPs with triangular shape, tagged as global and local mixing, to separately capture the long range dependencies and local patterns on fine-grained level, i.e., long and short-term preferences. | method 0.96 | 0.88 | 0.06 | asserted 0.99 |
| s4 | Empirical study on 12 datasets of different scales (50K\textasciitilde 10M user-item interactions) from 4 benchmarks (Amazon, MovieLens, Tenrec and LBSN) show that TriMLP consistently attains promising accuracy/efficiency trade-off, where the average performance boost against several state-of-the-art baselines achieves up to 14.88% with 8.65% less inference cost. | result 1.00 | 0.97 | 0.74 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
In this paper, we present a MLP-like architecture for sequential recommendation, namely TriMLP, with a novel Triangular Mixer for cross-token communications. In designing Triangular Mixer, we simplify the cross-token operation in MLP as the basic matrix multiplication, and drop the lower-triangle neurons of the weight matrix to block the anti-chronological order connections from future tokens. Accordingly, the information leakage issue can be remedied and the prediction capability of MLP can be fully excavated under the standard auto-regressive mode. Take a step further, the mixer serially alternates two delicate MLPs with triangular shape, tagged as global and local mixing, to separately capture the long range dependencies and local patterns on fine-grained level, i.e., long and short-term preferences. Empirical study on 12 datasets of different scales (50K\textasciitilde 10M user-item interactions) from 4 benchmarks (Amazon, MovieLens, Tenrec and LBSN) show that TriMLP consistently attains promising accuracy/efficiency trade-off, where the average performance boost against several state-of-the-art baselines achieves up to 14.88% with 8.65% less inference cost.
Human review
How the literature uses it
1 arXiv paper cite this in 1 sentence
What it cites · 11 sentences without a confident label
Uses the method or tool of 3 papers
-
The basic block alternates two types of MLP along with the residual connection [ref] and layer normalization [ref] : (i) Triangular Mixer to establish the contact among tokens per dimension, (ii)Feed-Forward Network to establish the contact among dimensions per token.
Layer Normalization · Basic Block · 0.98
-
During the training stage, TriMLP adopts the strategy where the batch size is 64, dropout rate is 0.5, and the optimizer is Adam [ref] with the learning rate of 0.001, and keeps updating the parameters until the performance no longer increases for 10 epochs.
Adam: A Method for Stochastic Optimization · Implementation Details · 0.99
-
Gaussian Error Linear Units TriMLP uses Gaussian error linear units [ref] as activation function to endow the model with non-linearity.
Gaussian Error Linear Units (GELUs) · Normalization and Activation Function · 1.00
Compares with 1 paper
-
As shown in Figure [ref], albeit these two MLPs inherit the merit of efficiency (average 0.34 seconds faster than GRU4Rec [ref] ), their recommendation performances averagely stand 22.52% behind other neural network architecture-based sequential recommenders on the metric of HR@10.
Session-based Recommendations with Recurrent Neural Networks · Introduction · 0.91
Cites as background 8 papers
-
Recurrent neural network (RNN)-based sequential recommenders, represented by [ref] , transmits the information in tokens step-by-step.
Session-based Recommendations with Recurrent Neural Networks · Introduction · 0.96
-
Since the pioneering MLPMixer [ref] and ResMLP [ref] have been proposed in the 2020s, all-MLP models are staging a comeback in vision community.
ResMLP: Feedforward networks for image classification with data-efficient training · Token-mixing MLP · 0.95
-
According to the dimensions of mapping tokens, these variants can be divided into three categories: (i) employing both the axial direction and the channel dimension [ref] which proposes to orthogonally decompose the cross-token mixing, maintain long-range dependency and encode cross-token relation along axial directions, (ii) considering only the channel dimension [ref] which aligns features at different spatial locations to the same channel by shifting and then interacting with spatial information through channel projection.
S$^2$-MLPv2: Improved Spatial-Shift MLP Architecture for Vision · Token-mixing MLP · 0.91
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.