{"id":"45128136-f705-46cb-b1fc-a5f0092a5f83","arxiv_id":"1908.00588","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Hidden states in an LSTM language model are visualized by training a separate linear classifier per state to predict the next word, giving compact 'semantic' bar-chart summaries for every state.","lead":"This paper introduces a visualization method that represents an RNN's hidden states as probability distributions over the next word, letting researchers compare what different parts of the network are 'thinking' at each time step. It may interest anyone building or debugging text models who wants an intuitive, human-readable view of how long-term and short-term memory combine.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim is not secured for non-output states: the paper treats the linear PSE's high perplexity for Long-term Memory and Cell states as inherent ambiguity, without testing whether a nonlinear or contextual probe would instead recover semantics.","rationale":"The reader's weakest assumption correctly identifies the context-free linear function G(γ,v) as the fragile point. My stress-test sharpens the concern: the paper's interpretation of high PSE perplexity for Long-term Memory and Cell states as inherent ambiguity conflates two explanations. A probe's failure can be due to the probe class (linear, context-free) rather than to the hidden state's meaning. The paper's strongest quantitative evidence, the Output state's perplexity of 70, validates only the one state whose PSE is closest to the actual classifier F; it does not validate the PSE's meaning for the states whose visualization is most novel. Because the central contribution is precisely the cross-state comparison enabled by PSEs, a missing control that distinguishes probe artifacts from genuine ambiguity leaves the main claim under-supported. This is an addressable gap: adding a nonlinear or context-aware probe would settle it. The paper is clearly written and appropriately preliminary, and it acknowledges its own limitations in Section 5, so a conditional acceptance remains the right verdict; my concern does not move the reader's conditional recommendation, so the verdict should remain UNCHANGED.","tokens_in":7686,"tokens_out":4332,"duration_ms":45713,"concrete_test":"Retrain the Section 4.2 PSE for the 2nd-layer Long-term Memory and Cell states using a one-hidden-layer MLP (or a probe that additionally receives the previous timestep's cell state) on the same 20M training pairs, and report held-out perplexity alongside Figure 3. If perplexity falls from the >400 range toward the RNN's 66.5, the poor linear PSE results reflect the probe's linearity or context-freeness, not the state's ambiguity; if perplexity remains orders of magnitude worse than the Output state, the authors' context-dependence interpretation gains support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The method's load-bearing assertion (Section 6: PSEs express the meaning of hidden states as a whole in relation to task inputs or outputs) is operationalized in Section 4.2 as a context-free function G(γ,v), and Section 5.2 evaluates it only with a simple linear classifier. The only quantitative validation is for the 2nd-layer Output state, whose PSE perplexity is 70 versus the RNN's 66.5. For the Long-term Memory and Cell states, Figure 3 shows PSE perplexities orders of magnitude worse, which the authors interpret as evidence that these states are not reliable indicators of what final prediction will be made and that their semantics are highly ambiguous. This inference is not valid as stated: a linear context-free probe can fail for two distinct reasons: the state genuinely lacks standalone predictive content, or the state's content is not linearly decodable without context. The paper provides no control that distinguishes these. If the second reason is true, the PSE visual encoding for exactly the states whose internal role is most interesting displays artificially flat or white distributions, so the meaning shown is an artifact of the probe class rather than the state. The central claim therefore rests on an unvalidated identification of linear decodability with semantic ambiguity, and Section 5.2's own acknowledgment that these states are highly dependent on other pieces of context is not a test of that explanation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Predictive Semantic Encodings (PSE), a technique that represents each hidden state of an LSTM as a probability distribution over the language model's output vocabulary. A separate context-free classifier G(γ, v) is trained per hidden state kind to map that state's vector to a next-word distribution, and the resulting distributions are displayed with a color rectangle and a mini-bar chart in a proof-of-concept visualization tool. The authors evaluate PSE accuracy on the Penn Treebank language modeling task with a 2-layer LSTM, reporting that the Output state's PSE reaches a test perplexity of 70 versus the RNN's 66.5, while Long-term Memory and Cell states have much higher perplexities, which they interpret as reflecting genuine semantic ambiguity. A case study of the sentence \"we stand in solidarity,\" she emphasized. is used to illustrate how the visualization reveals information flow across LSTM gates.","tokens_in":7943,"tokens_out":4215,"duration_ms":42904,"significance":"The idea of summarizing hidden states as output distributions is simple and potentially useful: it provides a common, task-aligned representation that allows visual comparison across different LSTM components. The paper gives a concrete quantitative evaluation on a standard benchmark and is transparent that the results are preliminary. If the central interpretation is valid, the technique could be a lightweight addition to RNN analysis toolkits. However, the evidence as presented does not yet secure the claim that PSEs express the meaning of hidden states 'as a whole': the quantitative validation rests on a single probe class, and the visualization's utility is supported only by a narrative case study, not by a user study or task-based evaluation.","major_comments":[{"comment":"The inference that Long-term Memory and Cell states are 'highly ambiguous' because their PSE perplexities are orders of magnitude worse than the Output state conflates linear probe failure with absence of semantics. The authors define G(γ,v) as a context-free function and evaluate it only with a simple linear classifier. A high-perplexity result could mean either that the state genuinely lacks standalone predictive content or that the state's content is not linearly decodable without context. No control distinguishes these hypotheses; e.g., an MLP probe or a probe that conditions on the neighboring hidden states (such as the cell input or output gate) would disambiguate them. Because Section 5.1 interprets the white, flat PSEs for c2_6, c2_7, and l2_7 in Figure 1 as meaningful ambiguity, the unvalidated identification of linear decodability with semantic content is load-bearing for the paper's central claim.","section":"§4.2, §5.2, Figure 3"},{"comment":"The claim that the Output state PSE perplexity of 70 is 'on par' with the RNN's 66.5 is supported only by a single point estimate. No error bars, confidence intervals, or significance tests are reported, so the size of the gap (3.5 perplexity points) is not interpretable. The authors should provide repeated training runs or bootstrap intervals, and ideally compare against a trivial baseline (e.g., a frequency-based unigram model) to show that the linear probe is capturing signal beyond chance. Without this, the parity claim, which underpins the statement that PSEs are an 'accurate generalization' of the RNN classifier, is unsubstantiated.","section":"§5.2"},{"comment":"The case study is a post-hoc narrative with no validation against alternative explanations or baseline visualizations. For example, the claim that l2_8 'is still retaining semantics which indicate a noun prediction' and that the Output Gate switches the semantic to a verb is presented as fact, but there is no quantitative measure of how reliable these PSE differences are, nor any comparison to a random or shuffled PSE baseline. Since the paper's contribution is a visualization technique, the absence of any user study or task-based evaluation leaves the core utility claim untested. At minimum, the authors should frame Section 5.1 as an illustrative example and explicitly outline a validation plan (e.g., a controlled experiment with Architects/Trainers) as future work.","section":"§5.1"}],"minor_comments":[{"comment":"The perplexity formula is written as 'T √' which is ambiguous; it should be the T-th root of the product. Also, the text below the equation says 'P(xt|xt ...xt−1)', which is a typo for 'P(xt|x1...xt−1)' or 'P(xt|w1...wt−1)' as in the formula itself.","section":"Equation (1)"},{"comment":"The phrase 'deep nature' to describe RNNs is imprecise; LSTM cells have depth in the unrolled sense, but calling them 'deep' in the first sentence may mislead readers.","section":"Abstract and §1"},{"comment":"The author name 'Y . V . Bengio' appears to have a typo; it should be 'Y. Bengio'.","section":"§2, Reference [1]"},{"comment":"The description 'top-k as 10% of the vocabulary' is unclear because k should be an integer; the authors should specify how the 10% threshold is rounded or state that they show the top 10% of classes.","section":"§4.2"},{"comment":"The text contains 'who's' where 'whose' is meant (the language model, who's outputs).","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a proof of concept and makes that explicit, but the load-bearing inference in Section 5.2—that poor linear PSE perplexity for Long-term Memory and Cell states reflects genuine ambiguity—needs a control experiment before the visualization can be considered trustworthy for non-output states. If the venue expects a full visualization validation, the lack of a user study is also a significant gap; however, for a workshop-style or short paper, the proposed revisions may be sufficient. The authors should also avoid overstating 'on par' without statistical support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a modest but real visualization idea, and its key semantic claim is only supported for the final output state; the memory-state story is an interpretation the paper doesn't actually test.\n\nWhat's new: the PSE formulation maps every LSTM hidden-state kind (embedding, gates, memory cell, etc.) into the same probability distribution over next-word outputs, using a separately trained linear probe per state kind. Previous tools (LSTMVis, RNNVis, Seq2SeqVis) inspect activations, attention, or clusters; none puts all state kinds into one comparable output-probability space. That is a legitimate, if simple, contribution. The positive control is real: the second-layer output state reaches test perplexity 70 against the full RNN's 66.5, close enough to show the probe is capturing the model's classifier. The case study in 5.1 reads as a plausible illustration of information flow through gates. The paper is clearly written and is explicit that the results are preliminary.\n\nWhere it's soft: the stress-test concern lands. Section 5.2 sees that Long-term Memory and Cell states have PSE perplexities orders of magnitude worse, then concludes those states are 'not reliable indicators' of the final prediction and their semantics are 'highly ambiguous.' That is a jump. A linear, context-free probe can fail either because the state genuinely lacks standalone predictive content, or because the state's content is not linearly decodable without context. The paper gives no control for the second reading. If the second is true, then the white/flat visual encoding for exactly the most interesting internal states is an artifact of the probe class, not the state's semantics. That's a load-bearing issue for the 'meaning of hidden states as a whole' claim. In addition, the perplexity numbers are reported as point estimates without error bars or significance tests, and the visualization claims have no user study. No code or data are shipped, so the reproducibility is limited to the description.\n\nWho it's for: people working on RNN interpretability or visualization. They will get a clean idea and a decent positive control for one state type, plus an instructive example of why probe choice matters. It isn't a breakthrough and doesn't change what gets built. With a nonlinear or contextual probe and some error bars, the central claim would be much sturdier. As is, it deserves a serious referee and conditional acceptance; I would send it to review.","headline":"A modest, clearly written PSE visualization idea whose load-bearing 'state semantics as probability distribution' claim is only validated for the output state; the memory-state conclusions need a probe-family control.","tokens_in":8484,"tokens_out":3742,"would_cite":false,"duration_ms":35017,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An RNN state's meaning reduces to its next-word odds","keywords":["recurrent neural networks","LSTM","hidden states","predictive semantic encodings","RNN visualization","model interpretability","language modelling","perplexity"],"falsifier":"Find a held-out sentence and a timestep where the LSTM's own output distribution assigns high probability to the actual next word but the Output-state PSE assigns it near-zero probability; if such discordant cases occur systematically across the test set, the claim that PSEs are accurate generalizations of the classifier $F$ would be refuted.","tokens_in":7452,"feed_emoji":"📊","tokens_out":10148,"duration_ms":92991,"temperature":0.7,"pith_summary":"This paper introduces Predictive Semantic Encodings (PSEs), a way to interpret an RNN hidden state by training a separate function that maps that state to a probability distribution over the task's output labels. The authors argue that this formulation expresses the meaning of a hidden state as a whole, letting any hidden state inside the network be compared with any other state, or with the network's own output, without inspecting raw activation values. They demonstrate the idea on a language model trained on Penn Treebank: the PSE trained on the final output state reaches a test perplexity of 70, close to the RNN's 66.5, which they read as evidence that the encoding is an accurate generalization of the RNN's classifier. States whose meaning depends heavily on other context, such as the Long-term Memory and Cell states, show far higher perplexity, a result the paper interprets as the encoding correctly exposing how context-dependent those states are.","feed_headline":"An RNN state's meaning reduces to its next-word odds","feed_subtitle":"Separate encoders map any LSTM state into a probability distribution, letting internal states be compared directly.","key_machinery":"The central object is the Predictive Semantic Encoding (PSE), defined as the context-free function $G(\\gamma, v)$ that produces a probability distribution over the task's outputs $y$ for a given hidden state kind $\\gamma$ and instance $v$; the RNN's own classifier $F(h)$ is the specialization where $\\gamma$ is the final hidden state. $G$ is trained separately, with its own parameters for every hidden state kind, after the original RNN finishes training. The PSE does two kinds of work: it maps all hidden states into a common output space so they can be compared visually and quantitatively, and its predictive accuracy as measured by perplexity doubles as a measure of how much of a state's meaning survives without surrounding context.","core_discovery":"The central claim is that a hidden state's semantics can be represented by its predicted effect: PSEs express the meaning of hidden states in relation to task inputs or outputs, via a context-free function $G(\\gamma, v)$ that maps a hidden state $v$ of kind $\\gamma$ to a probability distribution over output labels $y$. The RNN's own output classifier $F(h)$ is a special case, $G(\\gamma, h)$ with $\\gamma$ the final hidden state, so the encoding and the task share a visual and mathematical space. The authors demonstrate on a two-layer LSTM language model that the 2nd-layer Output state PSE reaches test perplexity 70 against the RNN's 66.5, establishing parity with $F$, while Long-term Memory and Cell states score orders of magnitude worse. They explain the gap as those states carrying information that must be combined with other context before it becomes a prediction, and they use the encoded distributions to trace how uncertainty at the end of a quotation resolves into a confident verb prediction after the word she.","pith_inferences":["An implication the authors leave implicit is that PSE perplexity could serve as a general diagnostic for gating design: comparing these scores across architectures would quantify which components carry context-dependent information.","The paper mentions predicting outputs at varying timesteps but does not evaluate it; one testable extension is training PSEs to predict words two or three steps ahead and checking whether distributions sharpen at consistent points in a sentence.","The visual case study reads a change in PSE colours as the Short-term Memory switching the model to a verb prediction, but PSEs are correlational summaries; treating those readings as hypotheses and testing them with gate ablations is a natural follow-up the paper does not perform."],"forward_implications":["All hidden state kinds can be placed side by side at any timestep, since each is encoded as the same kind of next-output distribution; the authors use this to show which LSTM components are pointing toward a verb versus a noun.","Because PSE perplexity tracks how self-contained a state's semantics are, the encoding gives a quantitative signal for where information in the LSTM is context-dependent; the paper identifies Long-term Memory and Cell states as exactly such states.","The PSE can be trained after the fact on an already trained RNN, so the technique offers a way to inspect existing language models without retraining or architectural changes.","Since the PSE shares the output representation with the task, visualizations can combine hidden state encodings with model outputs and part-of-speech colourings, foregrounding shifts in grammatical prediction across timesteps."],"supporting_citations":[{"why":"Defines the LSTM architecture and the hidden state kinds that the PSE encoding is applied to.","marker":"[4]"},{"why":"Shows that individual RNN activations can track syntactic features, motivating the search for interpretable hidden-state semantics.","marker":"[6]"},{"why":"Provides the Penn Treebank corpus used to train the language model and to evaluate PSE perplexity.","marker":"[10]"},{"why":"Relates hidden states to model inputs and outputs through co-clustering, the prior approach that PSE extends by drawing a direct parallel to the task's output distribution.","marker":"[12]"},{"why":"Defines user groups for RNN inspection and surveys hidden-state dynamics visualization, the starting point for comparing states across time.","marker":"[17]"}],"fun_headline_variants":["RNN states explained by what they predict","Semantic encoding turns RNN states into probabilities","See RNN states through their output predictions","Predictive encodings reveal RNN state meaning","Compare RNN states via next-word distributions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that a fixed, context-free function trained on one hidden state kind in isolation, $G(\\gamma, v)$, fairly represents that state's meaning; the paper's own results show this fails for Long-term Memory and Cell states, which the authors attribute to those states depending on other context.","fun_headline_variants_meta":{"raw":{"variants":["RNN states explained by what they predict","Semantic encoding turns RNN states into probabilities","See RNN states through their output predictions","Predictive encodings reveal RNN state meaning","Compare RNN states via next-word distributions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00032,"raw_usage":{"total_tokens":1752,"prompt_tokens":841,"completion_tokens":911,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":457,"completion_tokens_details":{"reasoning_tokens":842}},"tokens_in":457,"tokens_out":911,"duration_ms":9604,"temperature":1.0,"reasoning_tokens":842,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:45:19.336885+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find a held-out sentence and a timestep where the LSTM's own output distribution assigns high probability to the actual next word but the Output-state PSE assigns it near-zero probability; if such discordant cases occur systematically across the test set, the claim that PSEs are accurate generalizations of the classifier $F$ would be refuted.","supporting_citations":[{"cited_title":"Marcus, B","cited_arxiv_id":null,"evidence_quote":"Provides the Penn Treebank corpus used to train the language model and to evaluate PSE perplexity."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Relates hidden states to model inputs and outputs through co-clustering, the prior approach that PSE extends by drawing a direct parallel to the task's output distribution."},{"cited_title":"Strobelt, S","cited_arxiv_id":null,"evidence_quote":"Defines user groups for RNN inspection and surveys hidden-state dynamics visualization, the starting point for comparing states across time."}],"review_version":1}