Multiresolution Transformer Networks: Recurrence is Not Essential for Modeling Hierarchical Structure

Hsiang-Fu Yu; Inderjit S. Dhillon; Vikas K. Garg

REVIEW 4 major objections 5 minor 47 references

Multiresolution Transformer Networks: Recurrence is Not Essential for Modeling Hierarchical Structure

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes Multiresolution Transformer Networks, entirely attention-based models that stack masked session encoders over a query-level Transformer, and presents evidence they outperform hierarchical recurrent models on query…

desk verdict A useful hierarchical self-attention architecture for short-session query suggestion, but the title-level claim about recurrence and hierarchy is not supported by the evidence. read the letter →

arxiv 1908.10408 v1 pith:FSRG62LO submitted 2019-08-27 cs.LG cs.IRstat.ML

Vikas K. Garg , Inderjit S. Dhillon , Hsiang-Fu Yu This is my paper · ORCID

classification cs.LGcs.IRstat.ML

keywords multiresolutiontransformernetworksself-attentionhierarchicalstructurequerysuggestionrecurrencemaskedsessionencodersequence-to-sequencedynamics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that recurrence is not essential for modeling hierarchical structure. It introduces Multiresolution Transformer Networks (MTNs), which stack masked session-level encoders over a standard Transformer encoder so that self-attention operates at both the query level and the session level. On two web-search query-suggestion datasets, AOL and OnlineX, the authors report that MTNs beat recurrent and hierarchical recurrent baselines, with at least a 20% improvement on every n-gram precision score and over 25% improvement on BLEU over the best recurrent model on AOL. The paper also argues from layer dynamics why Transformers succeed, citing gradient flow along an ensemble of weakly dependent residual paths, and why adding multiresolution context should help. If the claim holds, attention-only sequence models can replace hierarchical recurrent networks in tasks with nested temporal structure.

What carries the argument

The load-bearing mechanism is the Masked Session Encoder inside a $K$-level MTN. At the bottom level, a standard Transformer encoder represents the tokens of each query; a Query Projection layer turns those token representations into a single vector per query, and positional encodings mark the order of queries in the session. Each higher level then runs self-attention over the query representations under a lower-triangular mask, so query $j$ can attend only to queries $1$ through $j$; after each masked session layer, the session representation for a query is added to its token representations through a skip connection and layer-normalized. This creates a hierarchy in which token-level attention is unrestricted while session-level attention is causal, letting the model capture successive query reformulations. The paper's dynamics propositions (1–4) support the design by expressing an RNN as a masked single-layer network and a Transformer as a layered residual network, and by showing the MTN encoder as a masked composition of these updates.

What would settle it

Run MTN and the hierarchical recurrent baselines on the raw AOL log without the 30-minute timeout, the 3–5 query length filter, duplicate removal, or the 10-token truncation, and compare n-gram precision and BLEU; if the attention-only advantage shrinks or reverses on longer or unfiltered sessions, the title-level claim fails.

Watch

Extended reading notes

Core claim

The central claim is that a fully attention-based architecture, the Multiresolution Transformer Network, can model hierarchical structure that had been thought to require recurrence. MTN encodes a session of queries by first running a standard Transformer encoder on the tokens of each query, projecting each query's token representations into a session-level embedding, and then passing those embeddings through one or more masked session encoders whose causal mask lets a query attend only to itself and earlier queries; a skip connection adds the session representation back into the query's token representations before a standard Transformer decoder produces the next query. The authors report that on the AOL and OnlineX query-suggestion tasks, MTN outperforms the hierarchical recurrent baselines H-LSTM and M-NSRF and also outperforms a similarly sized Transformer, with at least 20% higher n-gram precision on each metric and more than 25% higher BLEU than the best recurrent model on AOL. They conclude that recurrence is not essential for modeling hierarchical structure.

Load-bearing premise

The claim that recurrence is not essential for hierarchical structure rests on the paper's filtered session setup—a 30-minute timeout, sessions trimmed to 3–5 queries, duplicate queries removed, and queries capped at 10 tokens—so if hierarchical structure matters most in longer or unfiltered sessions, the title-level conclusion may not hold.

Editorial extensions

If this is right

Query-suggestion systems can be built without recurrence while improving precision and BLEU scores, since MTN outperforms hierarchical recurrent models on both AOL and OnlineX.
Session-level context can be modeled as causal self-attention over compressed query representations, rather than as a second recurrent state.
The advantage of MTN over a Transformer with the same total number of layers indicates that dedicated session layers do work that extra Transformer encoder layers do not.
If recurrence is not essential for hierarchical structure, hierarchical recurrent encoders in other natural-language tasks may be replaceable by masked multiresolution attention.
The dynamics analysis implies Transformers are relatively robust to vanishing and exploding gradients because their layers do not share weights and residual connections preserve an ensemble of short gradient paths.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Extension: The paper does not ablate the Query Projection or test more than two levels, so one could test whether a learned projection is necessary or whether mean-pooling and additional session levels change the result.
Extension: The same masked-multiresolution design could transfer to other nested sequence tasks, such as document summarization, dialogue state tracking, or code generation, whereas the paper demonstrates only query suggestion.
Extension: Viewing an RNN as a masked single-layer network suggests a continuum between recurrent and attention models, so replacing the fixed lower-triangular mask with learnable or interpolated masks might recover recurrent inductive biases inside an attention stack.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the MTN architecture is a sensible hierarchical self-attention design, and the reported gains on query suggestion are large enough to take seriously. But the paper overstates its central conclusion. 'Recurrence is not essential' is not established by experiments on filtered 3–5-query sessions from two search logs; that's a specific short-session setting, not hierarchy in general. The theoretical propositions are mostly notational restatements.

The genuinely new piece is the encoder: a standard Transformer over tokens per query, a learned projection to a fixed-size query representation, then a masked self-attention session encoder with residual injection of the session representation into token embeddings. That's clean and reproducible from the paper. Showing this beats hierarchical LSTM baselines on AOL and OnlineX by 20–50% on n-gram precision and 25% on BLEU is a real empirical data point, even without error bars. The authors also correctly point out that their session layers add value over simply stacking more transformer layers (Table 3), which strengthens the architecture claim.

Soft spots, in order:

1. Title and abstract overreach. The data is sessions of length 3–5, duplicates removed, queries truncated to 10 tokens. That's a narrow slice of hierarchy. The paper itself lists logical inference [38] as future work, conceding the comparison hasn't been run on the benchmark where the recurrent-versus-attention hierarchy question was originally posed. So 'strong evidence that recurrence is not essential' is not supported.

2. No error bars, no significance tests, no multiple seeds, and no code (or at least code is not mentioned). Combined with the private OnlineX dataset, the headline numbers are hard to verify.

3. The treatment of the Transformer baseline looks like an unfair comparison. The authors say MTN performs best with label smoothing 0.05, while 'Transformer performed well however with little to no smoothing.' That asymmetry means the 20–50% numbers are not apples-to-apples.

4. The 'first architecture that exploits hierarchical structure via self-attention' claim is undercut by Yang et al. hierarchical attention networks and other prior work. Should be softened to 'within this query-suggestion setting' or similar.

5. The dynamics propositions (1–4) are restatements of the architecture definitions with different notation. They don't generate new insight or predictions; the ensemble-of-short-paths story is borrowed from residual networks and not tested here.

The citation pattern is fine; they engage with the relevant challenge [38] and the hierarchical recurrent baselines. The math is correct as far as I can tell, but it is not really doing work.

Bottom line: this is a potentially useful architecture for short-session query prediction, and the empirical claim, if reproduced with proper baselines and variance, would be worth knowing. As written, it needs substantial revision: soften the title, level the baseline, add error bars or release code, and either test on a longer-hierarchy setting or explicitly scope the claim. I would send it to a competent referee, but my own recommendation is reject in current form with an invitation to revise.

Referee Report

4 major / 5 minor

Summary. The paper proposes Multiresolution Transformer Networks (MTNs), a self-attention-only architecture with a masked session-level encoder stacked over a query-level Transformer encoder, for query suggestion. The authors state formal propositions comparing RNN and Transformer output dynamics, then evaluate MTNs against Seq2Seq with attention, Hierarchical LSTM, M-NSRF, and Transformer on AOL and a private OnlineX dataset, reporting large gains in n-gram precision and BLEU, especially on AOL. The abstract concludes that recurrence is not essential for modeling hierarchical structure.

Significance. If the reported gains are reproducible, the work is a useful empirical contribution: it demonstrates that a hierarchical attention model can outperform hierarchical recurrent baselines on query suggestion, and it challenges the claim in [38] that recurrence is needed for hierarchical structure. The paper also provides a clear description of a two-level masked attention encoder that could be adopted for other session-based tasks. However, the significance is currently limited by the narrow evaluation setting, the label-smoothing confound, and the lack of uncertainty quantification; the theoretical propositions are definitional rather than predictive.

major comments (4)

[Section 5.2] The comparison between MTN and Transformer is confounded by label smoothing. The text reports that MTN achieved its best performance with label smoothing 0.05 (or 0.01 after 5 epochs), whereas Transformer performed well with little to no smoothing. Because label smoothing is a known regularizer that can change BLEU substantially, the gains in Table 2 and Figure 2 may be due to this hyperparameter disparity rather than to the MTN architecture. Please report matched-label-smoothing ablations (e.g., 0.0, 0.01, 0.05, and 0.1 for both models) and/or state the exact smoothing values used for each reported model.
[Sections 5.1 and 6] The title-level claim that recurrence is not essential for modeling hierarchical structure is not supported by the evidence. Section 5.1 restricts sessions to lengths 3-5, removes successive duplicate queries, and truncates queries to 10 tokens, while Section 6 defers logical inference [38] to future work. Thus the evaluated hierarchy is at most five shallow timestamp-clustered queries, not the nested or recursive structure discussed in [38]. The conclusion should be restricted to query suggestion under this preprocessing, or additional experiments on longer and more deeply nested hierarchies are needed.
[Section 5.4 and Table 2] No error bars or significance tests are reported in Table 2 or Figure 2. Since the abstract and Section 5.4 use the word 'significantly', please report results over multiple random seeds with standard deviations and, ideally, paired significance tests. This is especially important for the OnlineX 1-gram score, where MTN is not the best model (M-NSRF achieves 36.0 versus MTN's 35.1).
[Section 3 and Supplementary Material] Propositions 1-4 are essentially definitional. In the proof of Proposition 1, the matrix \tilde X is constructed by stacking x_t and h_{t-1}, so equation (2) is an algebraic restatement of the RNN update (1). Likewise, Propositions 2, 3, and 4 introduce \tilde X_l, \tilde D_l, and f that absorb all learned transformations, so the 'dynamics' amount to a notation system rather than a derivation that predicts MTN's advantage. The empirical results stand on their own, but the paper should present this part as a formal description, not as an explanation of why MTN succeeds, or provide falsifiable theoretical predictions.

minor comments (5)

[Abstract and Introduction] The abstract and introduction refer to 'OnlineX' and 'OnlineX3' inconsistently; please unify the terminology.
[Table 2] The caption of Table 2 should state the units of the n-gram precision scores (e.g., percentages or fractions), since the table currently lists numbers without units.
[Section 5.2] Please provide the chosen hyperparameters (learning rate, batch size, number of epochs, and any regularization settings) for all baselines, not only for M-NSRF, to support reproducibility.
[Section 5.3] The n-gram precision metric ignores word order, which is an unusual choice for query suggestion; please justify this choice or also report an order-sensitive metric.
[Section 4 and Figure 2] Figure 1 is not referenced in the text of Section 4, and Figure 2 is referenced only implicitly; please add explicit cross-references.

Circularity Check

1 steps flagged · score 2.0 of 10

Empirical result is self-contained on held-out test data; only the dynamics propositions are definitional rewrites, and they are not the source of the reported gains.

self definitional [Section 3, Proposition 1 (proof in Section 7, Supplementary Material)]
"Our first result describes the dynamics in RNNs. Proposition 1. The evolution of outputs of an RNN on input X can be expressed as Y_t = φ_y(M_t φ_h( \tilde{X} W_1)W_2), where \tilde{X} depends on X, W_1, W_2, and encapsulates the recurrent and the input information."

In the proof, \tilde{X} is defined row-wise as \tilde{x}_t = Concat(x_t, h_{t-1}), i.e., it is assembled from the RNN's own previous hidden states. Substituting this definition into Eq. (2) returns the original RNN update (1), so Proposition 1 is a notational repackaging of the RNN as a masked single layer rather than an independent derivation. The paper later claims to 'leverage the dynamics to design' MTNs, making this definitional step part of the theory narrative, but the central empirical comparison on held-out AOL/OnlineX test sessions is self-contained and does not reduce to this proposition.

full rationale

The paper's headline empirical claim, that MTNs outperform recurrent and hierarchical recurrent models on query suggestion, is evaluated on held-out test splits of AOL and OnlineX: sessions are partitioned into training, validation, and test, model selection is done on validation error, and the reported n-gram precision and BLEU numbers are for the test data. That part of the derivation chain is therefore self-contained and not circular. No fitted parameter is renamed as a prediction, no load-bearing argument reduces to a self-citation, and the cited challenge [38] is external. The main circularity-adjacent feature is confined to the theory section: Propositions 1-4 are close to definitional restatements of the RNN and Transformer/MTN update equations (e.g., Proposition 1 holds only because \tilde{X} is defined to include the RNN's own hidden states). These restatements support the explanatory narrative about gradient flow and the masked-session design, but they are not the source of the reported empirical gains. The title-level generalization to all hierarchical structure is an extrapolation from short filtered sessions (3-5 queries, duplicates removed, 10-token truncation), which is a scope/overreach concern rather than a circularity concern.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim is an empirical architecture claim. Learned network weights are not listed individually; instead we list the hand-selected hyperparameters and modeling assumptions that support the comparison. No new physical or formal entities are introduced.

free parameters (4)

Layer counts (query/session/decoder) = 3/2/3
Chosen by validation on each dataset; MTN with 3 query, 2 session, 3 decoder layers outperformed configurations with different splits (Table 3).
Label smoothing = 0.05 (2 epochs) or 0.01 (5 epochs)
Selected by validation for MTN; Transformer used little to no smoothing. This differential training choice complicates the comparison.
Vocabulary cutoff = 8 occurrences
Words with fewer than 8 occurrences in training replaced by <unk>, changing the difficulty of prediction.
Session length filter = 3 to 5 queries
Sessions outside this range are discarded, following [40]; this limits the scope of the hierarchical claim.

assumptions (4)

domain assumption The standard Transformer architecture can be written as the layer equations in Propositions 2 and 3.
Section 3 assumes this decomposition matches the reference implementation without proving equivalence for all variants.
ad hoc to paper The RNN output can be expressed as Y_t = phi_y(M_t phi_h(X-tilde W1) W2) with X-tilde defined recursively.
The construction of X-tilde is tautological; h_{t-1} itself depends on previous inputs and weights, so equation (2) is an implicit definition rather than a simplification.
domain assumption The ensemble gradient-flow argument from residual networks transfers to Transformers.
Section 3 asserts this 'following the arguments of [27]' without a proof in this paper.
domain assumption Session-level masked attention over projected query summaries captures the hierarchical context needed for query suggestion.
Architecture choice in Section 4; no theoretical guarantee is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multiresolution Transformer Networks: Recurrence is Not Essential for Modeling Hierarchical Structure." pith.science (2026). https://pith.science/paper/FSRG62LO

@misc{pith2026190810408,
  author       = {Pith},
  title        = {Pith review of: Multiresolution Transformer Networks: Recurrence is Not Essential for Modeling Hierarchical Structure},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FSRG62LO}},
  note         = {Machine review of arXiv:1908.10408}
}

read the original abstract

The architecture of Transformer is based entirely on self-attention, and has been shown to outperform models that employ recurrence on sequence transduction tasks such as machine translation. The superior performance of Transformer has been attributed to propagating signals over shorter distances, between positions in the input and the output, compared to the recurrent architectures. We establish connections between the dynamics in Transformer and recurrent networks to argue that several factors including gradient flow along an ensemble of multiple weakly dependent paths play a paramount role in the success of Transformer. We then leverage the dynamics to introduce {\em Multiresolution Transformer Networks} as the first architecture that exploits hierarchical structure in data via self-attention. Our models significantly outperform state-of-the-art recurrent and hierarchical recurrent models on two real-world datasets for query suggestion, namely, \aol and \amazon. In particular, on AOL data, our model registers at least 20\% improvement on each precision score, and over 25\% improvement on the BLEU score with respect to the best performing recurrent model. We thus provide strong evidence that recurrence is not essential for modeling hierarchical structure.

Figures

Figures reproduced from arXiv: 1908.10408 by the authors.

**Figure 1.** (Left) Information flow through a 2-level MTN encoder for query suggestion is shown. The Query Encoder is the standard Transformer encoder, whose output token representations are subject to a projection to maintain the same model dimensionality across the levels. The Masked Session Encoder prevents information flow from subsequent queries. The shadow underneath the encoders conveys that they consist of several layer… view at source ↗

**Figure 2.** BLEU scores on AOL test data Model Enc. Layers Dec. Layers BLEU Transformer (4Q, 0) 4 13.90 Transformer (5Q, 0) 5 13.89 MTN (Ours) (3Q, 2S) 3 14.62 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** The Transformer architecture (source: [13]) [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 42 canonical work pages

[38]

K. Tran, A. Bisazza, and C. Monz. The Importance of Being Recurrent for Modeling Hierarchi- cal Structure, Empirical Methods in Natural Language Processing (EMNLP), pp. 4731–4736, 2018

work page 2018
[1]

Hochreiter and J

S. Hochreiter and J. Schmidhuber. Long short-term memory, Neural computation, 9(8):1735– 1780, 1997

work page 1997
[2]

K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y . Bengio. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation, Empirical Methods in Natural Language Processing (EMNLP), 2014

work page 2014
[3]

Wolter and A

M. Wolter and A. Yao. Complex Gated Recurrent Neural Networks,Neural Information Process- ing Systems (NeurIPS), 2018

work page 2018
[4]

J. Gu, J. Bradbury, C. Xiong, V . O. K. Li, and R. Socher. Non-Autoregressive Neural Machine Translation, International Conference on Learning Representations (ICLR), 2018

work page 2018
[5]

Sutskever, O

I. Sutskever, O. Vinyals, and Q. V . Le. Sequence to sequence learning with neural networks, Neural Information Processing Systems (NIPS), 2014

work page 2014
[6]

Bahdanau, K

D. Bahdanau, K. Cho, and Y . Bengio. Neural Machine Translation by Jointly Learning to Align and Translate, International Conference on Learning Representations (ICLR), 2015

work page 2015
[7]

Luong, H

T. Luong, H. Pham, and C. D. Manning. Effective approaches to attention-based neural machine translation, Empirical Methods in Natural Language Processing (EMNLP), pp. 1412–1421, 2015

work page 2015

Show all 47 references

[8]

Y . Kim, C. Denton, L. Hoang, and A. M. Rush. Structured attention networks, International Conference on Learning Representations (ICLR), 2017

2017
[9]

Zhang, Q

J. Zhang, Q. Lei, and I. S. Dhillon. Stabilizing Gradients for Deep Neural Networks via Efﬁcient SVD Parameterization, International Conference on Machine Learning (ICML), 2018

2018
[10]

Zhang, Y

J. Zhang, Y . Lin, Z. Song, and I. S. Dhillon. Learning Long Term Dependencies via Fourier Recurrent Units, International Conference on Machine Learning (ICML), 2018

2018
[11]

Hochreiter, Y

S. Hochreiter, Y . Bengio, P. Frasconi, and J. Schmidhuber. Gradient ﬂow in recurrent nets: the difﬁculty of learning long-term dependencies, Field Guide to Dynamical Recurrent Networks, 2001

2001
[12]

Gehring, M

J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y . N. Dauphin. Convolutional sequence to sequence learning, International Conference on Machine Learning (ICML), 2017

2017
[13]

Vaswani, N

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need, Neural Information Processing Systems (NIPS), 2017

2017
[14]

Ahmed, N

K. Ahmed, N. S. Keskar, and R. Socher. Weighted Transformer Networks For Machine Translation, arXiv:1711.02132, 2017

2017 arXiv
[15]

Zhang, I

H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena. Self-Attention Generative Adversarial Networks, arXiv:1805.08318, 2018

2018 arXiv
[16]

Jaderberg, K

M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu. Spatial Transformer Networks, Neural Information Processing Systems (NIPS), 2015

2015
[17]

Z. Dai, Z. Yang, Y . Yang, J. Carbonell, Q. V . Le, and R. Salakhutdinov. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context, arXiv:1901.02860v2, 2019

1901 arXiv
[18]

Dehghani, S

M. Dehghani, S. Gouws, O. Vinyals, J. Uszkoreit, and L. Kaiser. Universal Transformers, International Conference on Learning Representations (ICLR), 2019

2019
[19]

P. Shaw, J. Uszkoreit, and A. Vaswani. Self-Attention with Relative Position Representations, Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2018

2018
[20]

Girdhar, J

R. Girdhar, J. Carreira, C. Doersch, and A. Zisserman. Video Action Transformer Network, arXiv:1812.02707, 2018

2018 arXiv
[21]

Parmar, A

N. Parmar, A. Vaswani, J. Uszkoreit, L. Kaiser, N. Shazeer, A. Ku, and D. Tran. Image Transformer, International Conference on Machine Learning (ICML), 2018

2018
[22]

C.-Y . Ma, A. Kadav, I. Melvin, Z. Kira, G. AlRegib, and H. P. Graf. Attend and Interact: Higher- Order Object Interactions for Video Understanding, Computer Vision and Pattern Recognition (CVPR), pp. 6790–6800, 2018

2018
[23]

Devlin, M.-W

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, arXiv: 1810.04805, 2018. 10

2018 arXiv
[24]

K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition,Computer Vision and Pattern Recognition (CVPR), 2016

2016
[25]

R. K. Srivastava, K. Greff, and J. Schmidhuber. Training very deep networks,Neural Information Processing Systems (NIPS), pp. 2368–2376, 2015

2015
[26]

J. G. Zilly, R. K. Srivastava, J. Koutník, and J. Schmidhuber. Recurrent Highway Networks, International Conference on Machine Learning (ICML), 2017

2017
[27]

A. Veit, M. Wilber, and S. Belongie. Residual Networks Behave Like Ensembles of Relatively Shallow Networks, Neural Information Processing Systems (NIPS), pp. 550–558, 2016

2016
[28]

A. Fan, M. Lewis, and Y . Dauphin. Hierarchical Neural Story Generation, Association for Computational Linguistics (ACL), 2018

2018
[29]

Gulordava, P

K. Gulordava, P. Bojanowski, E. Grave, T. Linzen, and M. Baroni. Colorless green recurrent networks dream hierarchically, Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pp. 1195–1205, 2018

2018
[30]

Blevins, O

T. Blevins, O. Levy, and L. Zettlemoyer. Deep RNNs encode soft hierarchical syntax,Associa- tion for Computational Linguistics (ACL), 2018

2018
[31]

Z. Yang, D. Yang, C. Dyer, X. He, A. Smola, and E. Hovy. Hierarchical Attention Networks for Document Classiﬁcation, Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2016

2016
[32]

Chung, S

J. Chung, S. Ahn, and Y . Bengio. Hierarchical Multiscale Recurrent Neural Networks,Interna- tional Conference on Learning Representations (ICLR), 2017

2017
[33]

R. A. Baeza-Yates, C. A. Hurtado, and M. Mendoza. Query recommendation using query logs in search engines, EDBT Conference on Current Trends in Database Technology, 3268: 588–596, 2004

2004
[34]

Q. He, D. Jiang, Z. Liao, S. C. Hoi, K. Chang, E.-P. Lim, and H. Li. Web query recommendation via sequential query prediction, International Conference on Data Engineering (ICDE) , pp. 1443–1454, 2009

2009
[35]

Jiang, Y .-Y

J.-Y . Jiang, Y .-Y . Ke, P.-Yu Chien, and P.-J. Cheng. Learning user reformulation behavior for query auto-completion, ACM conference on Research and development in information retrieval (SIGIR), pp. 445–454. ACM, 2014

2014
[36]

Query auto-completion for rare preﬁxes, Conference on Information and Knowledge Management (CIKM), pp

Bhaskar Mitra and Nick Craswell. Query auto-completion for rare preﬁxes, Conference on Information and Knowledge Management (CIKM), pp. 1755–1758, 2015

2015
[37]

H. Cao, D. Jiang, J. Pei, Q. He, Z. Liao, E. Chen, and H. Li. Context-aware query suggestion by mining click-through and session data, Knowledge discovery and data mining (KDD) , pp. 875–883, 2008

2008
[39]

Sordoni, Y

A. Sordoni, Y . Bengio, H. Vahabi, C. Lioma, J. G. Simonsen, and J.-Y . Nie. A Hierarchical Recurrent Encoder-Decoder for Generative Context-Aware Query Suggestion,Conference on Information and Knowledge Management (CIKM), 2015

2015
[40]

W. U. Ahmad, K.-W. Chang, and Hongning Wang. Multi-Task Learning for Document Ranking and Query Suggestion, International Conference on Learning Representations (ICLR), 2018

2018
[41]

G. Pass, A. Chowdhury, and C. Torgeson. A Picture of Search, The First International Confer- ence on Scalable Information Systems, 2006

2006
[42]

Srivastava, G

N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: a simple way to prevent neural networks from overﬁtting. Journal of Machine Learning Research (JMLR), 15(1):1929–1958, 2014

1929
[43]

Pascanu, T

R. Pascanu, T. Mikolov, and Y . Bengio. On the difﬁculty of training Recurrent Neural Networks, International Conference on Machine Learning (ICML), 2013

2013
[44]

J. J. Bernard, A. Spink, C. Blakely, and S. Koshman. Deﬁning a session on web search engines, Journal of the American Society for Information Science and Technology, 58(6):862-871, 2007. 11

2007
[45]

Papineni, S

K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. BLEU: a method for automatic evaluation of machine translation, Association for computational linguistics (ACL), pp. 311–318, 2002

2002
[46]

Kusupati, M

A. Kusupati, M. Singh, K. Bhatia, A. Kumar, P. Jain, and M. Varma. FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network, Neural Information Processing Systems (NeurIPS), 2018

2018
[47]

Zhang, X

J. Zhang, X. Wang, D. Li, and Y . Wang. Dynamically Hierarchy Revolution: DirNet for Compressing Recurrent Neural Network on Mobile Devices, International Joint Conference on Artiﬁcial Intelligence (IJCAI-18). 12 7 Supplementary Material We now provide proofs for all the resul...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [38]

K. Tran, A. Bisazza, and C. Monz. The Importance of Being Recurrent for Modeling Hierarchi- cal Structure, Empirical Methods in Natural Language Processing (EMNLP), pp. 4731–4736, 2018

work page 2018

[2] [1]

Hochreiter and J

S. Hochreiter and J. Schmidhuber. Long short-term memory, Neural computation, 9(8):1735– 1780, 1997

work page 1997

[3] [2]

K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y . Bengio. Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation, Empirical Methods in Natural Language Processing (EMNLP), 2014

work page 2014

[4] [3]

Wolter and A

M. Wolter and A. Yao. Complex Gated Recurrent Neural Networks,Neural Information Process- ing Systems (NeurIPS), 2018

work page 2018

[5] [4]

J. Gu, J. Bradbury, C. Xiong, V . O. K. Li, and R. Socher. Non-Autoregressive Neural Machine Translation, International Conference on Learning Representations (ICLR), 2018

work page 2018

[6] [5]

Sutskever, O

I. Sutskever, O. Vinyals, and Q. V . Le. Sequence to sequence learning with neural networks, Neural Information Processing Systems (NIPS), 2014

work page 2014

[7] [6]

Bahdanau, K

D. Bahdanau, K. Cho, and Y . Bengio. Neural Machine Translation by Jointly Learning to Align and Translate, International Conference on Learning Representations (ICLR), 2015

work page 2015

[8] [7]

Luong, H

T. Luong, H. Pham, and C. D. Manning. Effective approaches to attention-based neural machine translation, Empirical Methods in Natural Language Processing (EMNLP), pp. 1412–1421, 2015

work page 2015