REVIEW 2 cited by
Language Modeling Using Tensor Trains
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We propose a novel tensor network language model based on the simplest tensor network (i.e., tensor trains), called `Tensor Train Language Model' (TTLM). TTLM represents sentences in an exponential space constructed by the tensor product of words, but computing the probabilities of sentences in a low-dimensional fashion. We demonstrate that the architectures of Second-order RNNs, Recurrent Arithmetic Circuits (RACs), and Multiplicative Integration RNNs are, essentially, special cases of TTLM. Experimental evaluations on real language modeling tasks show that the proposed variants of TTLM (i.e., TTLM-Large and TTLM-Tiny) outperform the vanilla Recurrent Neural Networks (RNNs) with low-scale of hidden units. (The code is available at https://github.com/shuishen112/tensortrainlm.)
Forward citations
Cited by 2 Pith papers
-
Advantages of density in tensor network geometries for gradient based training
Densely connected tensor network geometries train to lower infidelity than sparse ones on random quantum states, and a new leaf-contraction trick reduces memory while improving training.
-
No-Free-Lunch Theories for Tensor-Network Machine Learning Models
Tensor-network machine learning models (MPS and PEPS) have average generalization risk lower bounded by explicit functions of training-set size and bond dimension, formalizing no-free-lunch limits for quantum-inspired...
Discussion (0). Continue with ORCID to comment.