RECORD
GUIM -- General User and Item Embedding with Mixture of Representation in E-commerce
Artificial Intelligence · Submitted 2 Jul 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Our experimental results in these downstream tasks clearly show the comparative value of embeddings learned from our GUIM model.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Our goal is to build general representation (embedding) for each user and each product item across Alibaba's businesses, including Taobao and Tmall which are among the world's biggest e-commerce websites. The representation of users and items has been playing a critical role in various downstream applications, including recommendation system, search, marketing, demand forecasting and so on. Inspired from the BERT model in natural language processing (NLP) domain, we propose a GUIM (General User Item embedding with Mixture of representation) model to achieve the goal with massive, structured, multi-modal data including the interactions among hundreds of millions of users and items. We utilize mixture of representation (MoR) as a novel representation form to model the diverse interests of each user. In addition, we use the InfoNCE from contrastive learning to avoid intractable computational costs due to the numerous size of item (token) vocabulary. Finally, we propose a set of representative downstream tasks to serve as a standard benchmark to evaluate the quality of the learned user and/or item embeddings, analogous to the GLUE benchmark in NLP domain. Our experimental results in these downstream tasks clearly show the comparative value of embeddings learned from our GUIM model.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Our goal is to build general representation (embedding) for each user and each product item across Alibaba's businesses, including Taobao and Tmall which are among the world's biggest e-commerce websites. | background 0.52 | 0.27 | 0.03 | asserted 0.93 |
| s1 | The representation of users and items has been playing a critical role in various downstream applications, including recommendation system, search, marketing, demand forecasting and so on. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s2 | Inspired from the BERT model in natural language processing (NLP) domain, we propose a GUIM (General User Item embedding with Mixture of representation) model to achieve the goal with massive, structured, multi-modal data including the interactions among hundreds of millions of users and items. | contribution 0.61 | 0.82 | 0.04 | asserted 0.96 |
| s3 | We utilize mixture of representation (MoR) as a novel representation form to model the diverse interests of each user. | method 0.89 | 0.88 | 0.04 | asserted 0.99 |
| s4 | In addition, we use the InfoNCE from contrastive learning to avoid intractable computational costs due to the numerous size of item (token) vocabulary. | method 0.97 | 0.87 | 0.03 | asserted 1.00 |
| s5 | Finally, we propose a set of representative downstream tasks to serve as a standard benchmark to evaluate the quality of the learned user and/or item embeddings, analogous to the GLUE benchmark in NLP domain. | contribution 0.79 | 0.88 | 0.09 | asserted 0.94 |
| s6 | Our experimental results in these downstream tasks clearly show the comparative value of embeddings learned from our GUIM model. | result 0.91 | 0.96 | 0.80 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Our goal is to build general representation (embedding) for each user and each product item across Alibaba's businesses, including Taobao and Tmall which are among the world's biggest e-commerce websites. The representation of users and items has been playing a critical role in various downstream applications, including recommendation system, search, marketing, demand forecasting and so on. Inspired from the BERT model in natural language processing (NLP) domain, we propose a GUIM (General User Item embedding with Mixture of representation) model to achieve the goal with massive, structured, multi-modal data including the interactions among hundreds of millions of users and items. We utilize mixture of representation (MoR) as a novel representation form to model the diverse interests of each user. In addition, we use the InfoNCE from contrastive learning to avoid intractable computational costs due to the numerous size of item (token) vocabulary. Finally, we propose a set of representative downstream tasks to serve as a standard benchmark to evaluate the quality of the learned user and/or item embeddings, analogous to the GLUE benchmark in NLP domain. Our experimental results in these downstream tasks clearly show the comparative value of embeddings learned from our GUIM model.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 6 sentences without a confident label
Uses the method or tool of 2 papers
-
To address this challenge, we use the InfoNCE method [ref] in our model, by essentially sampling only some items from some noise distribution as negative cases.
Representation Learning with Contrastive Predictive Coding · Introduction · 0.97
-
The one-layer FFN consists of a linear transformation followed by a Gaussian Error Linear Unit (GELU) activation function [ref] .
Gaussian Error Linear Units (GELUs) · Model Structure · 0.97
Cites as support 1 paper
-
The used InfoNCE method greatly reduces the computation costs but still keeps good learning performance, by enjoying the good estimation property proven by [ref] .
Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency · Introduction · 0.99
Cites as background 6 papers
-
model proposed by [ref] in the NLP domain.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding · BERT Related Work · 0.96
-
Since the success of the BERT, many of its variants have merged in the NLP domain by adding or varying pre-training tasks [ref] .
Unified Language Model Pre-training for Natural Language Understanding and Generation · BERT Related Work · 0.91
-
Since the success of the BERT, many of its variants have merged in the NLP domain by adding or varying pre-training tasks [ref] .
XLNet: Generalized Autoregressive Pretraining for Language Understanding · BERT Related Work · 0.91
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.