REVIEW 6 cited by
Word Interdependence Exposes How LSTMs Compose Representations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent work in NLP shows that LSTM language models capture compositional structure in language data. For a closer look at how these representations are composed hierarchically, we present a novel measure of interdependence between word meanings in an LSTM, based on their interactions at the internal gates. To explore how compositional representations arise over training, we conduct simple experiments on synthetic data, which illustrate our measure by showing how high interdependence can hurt generalization. These synthetic experiments also illustrate a specific hypothesis about how hierarchical structures are discovered over the course of training: that parent constituents rely on effective representations of their children, rather than on learning long-range relations independently. We further support this measure with experiments on English language data, where interdependence is higher for more closely syntactically linked word pairs.
Forward citations
Cited by 6 Pith papers
-
POPACheck: A Model Checker for Probabilistic Pushdown Automata
POPACheck is the first model checker for probabilistic pushdown automata, supporting LTL and the POTLfχ fragment of POTL, with a new semi-algorithm based on OVI and expected-runtime certificates.
-
Implicit Rankings for Verifying Liveness Properties in First-Order Logic
Implicit rankings let SMT solvers verify liveness with first-order approximations of ranking functions, and the authors use them on Dijkstra's k-state, 4-state, and 3-state self-stabilizing protocols.
-
Solving First-Order Fixed-Point Logics via a Least-to-Greatest Transformation Based on Game Semantics
A sound and complete μ-to-ν transformation based on parity relations reduces first-order fixed-point validity to constraint solving and is equivalent to constructing winning strategies in the fixpoint parity game.
-
Termination analysis with interpolation-based transition invariant generation
A new interpolation-based method generates disjunctively well-founded transition invariants from terminating traces, enabling one safety-based tool to prove both termination and nontermination.
-
Circuit-Based Program Verification: Sequential Circuits as an Intermediate Representation for Verifying C Programs
Sequential circuits are a viable intermediate representation for verifying C programs, letting mature hardware model checkers handle software tasks competitively.
-
A Primal-Dual Perspective on Program Verification Algorithms (Extended Version)
A generalized Lagrangian duality framework unifies several program verification algorithms and yields a new solver for fixpoint logic over quantified linear arithmetic.
Discussion (0). Continue with ORCID to comment.