On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
read the original abstract
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks. The neural machine translation models often consist of an encoder and a decoder. The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates a correct translation from this representation. In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder--Decoder and a newly proposed gated recursive convolutional neural network. We show that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase. Furthermore, we find that the proposed gated recursive convolutional network learns a grammatical structure of a sentence automatically.
This paper has not been read by Pith yet.
Forward citations
Cited by 33 Pith papers
-
Session-based Recommendations with Recurrent Neural Networks
RNNs with ranking loss outperform item-to-item baselines for session-based recommendations on two datasets.
-
Weakly Supervised Cross-Modal Learning for 4D Radar Scene Flow Estimation
A task-specific iterative framework for weakly supervised 4D radar scene flow estimation uses instance-aware self-supervised losses from 2D tracking/segmentation and a rigid static loss from odometry to outperform LiD...
-
Weakly Supervised Cross-Modal Learning for 4D Radar Scene Flow Estimation
Weakly supervised iterative framework for radar scene flow estimation using back-projected 2D instance masks and odometry-based rigid static loss to outperform LiDAR-dependent and fully supervised baselines on the VoD...
-
PluRule: A Benchmark for Moderating Pluralistic Communities on Social Media
PluRule is a new multimodal multilingual benchmark showing that state-of-the-art vision-language models perform only marginally better than a trivial baseline at detecting specific rule violations in pluralistic onlin...
-
TCD-Arena: Assessing Robustness of Time Series Causal Discovery Methods Against Assumption Violations
TCD-Arena is a new customizable testing framework that runs millions of experiments to map how 33 different assumption violations affect time series causal discovery methods and shows ensembles can boost overall robustness.
-
Reduced-Order Surrogates for Forced Flexible Mesh Coastal-Ocean Models
Koopman autoencoders with forcings and temporal unrolling deliver accurate year-long predictions for coastal-ocean models at 300-1400x speedup, outperforming POD in two of three cases.
-
Recurrent Video Masked Autoencoders
RVM uses recurrent computation inside a masked autoencoder to learn video representations that match or exceed prior video and image models on classification, tracking, and dense spatial tasks with up to 30x better pa...
-
Conserved Kinematic Representations enable Zero-Shot Decoding in Handwriting BCIs
A zero-shot machine learning decoder for handwriting BCIs achieves 64% hits@3 retrieval on unseen letters by exploiting conserved kinematic neural representations.
-
Neural architectures for resolving references in program code
New seq2seq architectures for permutation indexing outperform baselines on synthetic reference-resolution tasks and reduce real decompilation error rates by 42%.
-
The illusory simplicity of the feedforward pass: evidence for the dynamical nature of stimulus encoding along the primate ventral stream
Primate ventral stream encodes visual stimuli through evolving neural dynamics that carry category information beyond any fixed spatial pattern during the initial feedforward pass.
-
Leveraging Artist Catalogs for Cold-Start Music Recommendation
ACARec attends over artist catalogs to generate CF embeddings for new tracks, more than doubling recall and NDCG versus content-only baselines in music recommendation.
-
M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling
M²RNN achieves perfect state tracking at unseen lengths and outperforms Gated DeltaNet hybrids by 0.4-0.5 perplexity on 7B models with 3x smaller recurrent states.
-
SpectraLLM: Uncovering the Ability of LLMs for Molecular Structure Elucidation from Multi-Spectral Data
SpectraLLM is an LLM fine-tuned to predict small-molecule structures from single or multiple spectra, reporting state-of-the-art results on four public benchmarks with gains from multi-modal input.
-
DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory
DragNUWA integrates text, image, and trajectory controls into a diffusion video model using a Trajectory Sampler, Multiscale Fusion, and Adaptive Training to enable fine-grained open-domain video generation.
-
Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions
A variational Bayesian framework exploits reciprocity between referents and context plus semantic reproduction to improve referring expression grounding over pairwise methods in supervised and unsupervised settings.
-
Creating A Neural Pedagogical Agent by Jointly Learning to Review and Assess
Bidirectional RNN with attention models real-time user knowledge from question-response sequences to predict correctness, outperforming baselines especially for new users on a large TOEIC mobile app dataset.
-
Recurrent Adversarial Service Times
RNN for arrivals paired with recurrent GAN for service times to model queuing dynamics without assuming specific inter-event distributions.
-
Learning Belief Representations for Imitation Learning in POMDPs
BMIL learns belief modules jointly with policies for GAIL-style imitation learning in POMDPs, outperforming separate training and standard GAIL on continuous control tasks.
-
EHR-RAGp: Retrieval-Augmented Prototype-Guided Foundation Model for Electronic Health Records
EHR-RAGp is a retrieval-augmented EHR foundation model that employs prototype-guided retrieval to dynamically integrate relevant historical patient context, outperforming prior models on clinical prediction tasks.
-
Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balance
Stable-GFlowNet improves training stability and attack diversity in LLM red-teaming by eliminating Z estimation via contrastive trajectory balance while preserving GFN optimality.
-
Delta6: A Low-Cost, 6-DOF Force-Sensing Flexible End-Effector
Delta6 delivers a low-cost 6-DOF force-sensing end-effector with 3.8% FS accuracy using sequence models, validated on robot-arm tasks like buffing and tight assembly.
-
On Safer Reinforcement Learning for Sedation and Analgesia in Intensive Care
Offline RL for ICU sedation shows that adding 30-day mortality to the objective yields policies whose clinician agreement correlates negatively with mortality, unlike pain-only versions.
-
StateX: Enhancing RNN Recall via Post-training State Expansion
StateX post-trains RNNs to expand recurrent state size, improving recall and in-context learning with negligible parameter growth.
-
From Time-series Generation, Model Selection to Transfer Learning: A Comparative Review of Pixel-wise Approaches for Large-scale Crop Mapping
A comparative review with experiments identifying optimal preprocessing, models, and transfer strategies for large-scale pixel-wise crop mapping using Landsat 8 data across five sites.
-
Enriching and Controlling Global Semantics for Text Summarization
A normalizing-flow neural topic model plus control mechanism are added to Transformer summarizers to supply and regulate global semantics, with reported gains over prior models on five benchmarks.
-
Reasoning and Generalization in RL: A Tool Use Perspective
Proposes a tool-use inspired framework with multiple test sets to measure specified types of generalization in RL.
-
Fast-ULCNet: A fast and ultra low complexity network for single-channel speech enhancement
Fast-ULCNet matches original ULCNet speech enhancement quality while cutting model size by more than half and latency by 34% via FastGRNN replacement and a state-drift filter.
-
Neural Cross-Domain Collaborative Filtering with Shared Entities
NeuCDCF is a wide-and-deep neural architecture for cross-domain collaborative filtering that jointly learns matrix factorization and deep representations, reporting better performance than prior CDCF models on four re...
-
Polyphone Disambiguation for Mandarin Chinese Using Conditional Neural Network with Multi-level Embedding Features
A conditional neural network using bidirectional RNN sentence encoding and multi-level word/sentence embeddings reaches 94.69% accuracy on a public Mandarin polyphone dataset.
-
Rare Disease Detection by Sequence Modeling with Generative Adversarial Networks
A GAN-boosted RNN model reaches 0.56 PR-AUC for rare EPI detection on 1.8 million patients and outperforms benchmarks.
-
Alchemy: A Quantum Chemistry Dataset for Benchmarking AI Models
Alchemy releases a new quantum chemistry dataset of 119k molecules and 12 properties plus GNN benchmarks to support AI model development for molecular science.
-
Large Language Models: A Survey
The paper surveys key large language models, their training methods, datasets, evaluation benchmarks, and future research directions in the field.
-
Deep learning applied to computational mechanics: A comprehensive review, state of the art, and the classics
A comprehensive review of deep learning techniques for computational mechanics, including LSTM for constitutive modeling, PINNs for PDE solving, optimizers, and kernel methods.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.