REVIEW 6 cited by
Tensor Networks Meet Neural Networks: A Survey and Future Perspectives
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Tensor networks (TNs) and neural networks (NNs) are two fundamental data modeling approaches. TNs were introduced to solve the curse of dimensionality in large-scale tensors by converting an exponential number of dimensions to polynomial complexity. As a result, they have attracted significant attention in the fields of quantum physics and machine learning. Meanwhile, NNs have displayed exceptional performance in various applications, e.g., computer vision, natural language processing, and robotics research. Interestingly, although these two types of networks originate from different observations, they are inherently linked through the typical multilinearity structure underlying both TNs and NNs, thereby motivating a significant number of developments regarding combinations of TNs and NNs. In this paper, we refer to these combinations as tensorial neural networks~(TNNs) and present an introduction to TNNs from both data processing and model architecture perspectives. From the data perspective, we explore the capabilities of TNNs in multi-source fusion, multimodal pooling, data compression, multi-task training, and quantum data processing. From the model perspective, we examine TNNs' integration with various architectures, including Convolutional Neural Networks, Recurrent Neural Networks, Graph Neural Networks, Transformers, Large Language Models, and Quantum Neural Networks. Furthermore, this survey also explores methods for improving TNNs, examines flexible toolboxes for implementing TNNs, and documents TNN development while highlighting potential future directions. To the best of our knowledge, this is the first comprehensive survey that bridges the connections among NNs and TNs. We provide a curated list of TNNs at https://github.com/tnbar/awesome-tensorial-neural-networks.
Forward citations
Cited by 6 Pith papers
-
KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matrices
KromHC uses Kronecker products of small doubly stochastic matrices to make Hyper-Connection residual matrices exactly balanced with O(n^2C) parameters, and matches or beats prior variants on small LLM pretraining runs.
-
No-Free-Lunch Theories for Tensor-Network Machine Learning Models
Tensor-network machine learning models (MPS and PEPS) have average generalization risk lower bounded by explicit functions of training-set size and bond dimension, formalizing no-free-lunch limits for quantum-inspired...
-
Put Teacher in Student's Shoes: Cross-Distillation for Ultra-compact Model Compression Framework
EI-BERT compresses a Chinese NLU model to 1.91 MB with competitive accuracy using attention-based vocabulary pruning, cross-distillation, and module-wise INT8 quantization, and reports deployment at Alipay.
-
Cross Space and Time: A Spatio-Temporal Unitized Model for Traffic Flow Forecasting
A low-rank 'unitized cell' plug-in improves traffic flow forecasting accuracy of several STGNN backbones on four PEMS datasets at small computational overhead.
-
Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks
The paper makes the case that tensorized neural networks offer valuable compression, scaling, and interpretability advantages that the deep learning community has not yet fully exploited.
-
Quantum Computing for Energy Management: A Semi Non-Technical Guide for Practitioners
A review-based guide concludes that quantum speedup for energy management is unproven and presents a practical framework for selecting quantum and quantum-inspired approaches.
Discussion (0). Continue with ORCID to comment.