{"id":"a44d29de-814f-4e99-9089-52a5d7bb39f1","arxiv_id":"2505.19275","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Online Shannon information measures that an agent computes while observing a stationary, ergodic environment form stationary, ergodic processes, and the average semantic content of a measurement equals the statistical complexity of the environment's minimal predictive model.","lead":"It treats the online information measures an observing agent computes (surprise, stored information, predictive uncertainty) as stochastic processes, and proves that when the environment is stationary and ergodic these 'information processes' are too.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 12's proof relies on finite-range functions, but causal states and semi-infinite-conditioned information atoms are not finite-range; the ergodicity claim needs a measurable-function argument that is not supplied.","rationale":"The reader's weakest assumption matches the most load-bearing defect: the proofs of Propositions 9–12 explicitly rely on the claim that causal-state and self-information processes are finite-range functions of the environment, but the paper's own definitions involve semi-infinite pasts and futures. This is not a cosmetic issue, because the finite-range propositions (2 and 3) and the citation to Billingsley Thm 36.4 as used there do not cover the constructed objects. The likely fix is standard: a measurable, shift-equivariant function of a stationary ergodic process is itself stationary ergodic, so the theorem is probably true despite the faulty proof. However, the paper does not verify the required measurability of the epsilon-map or of pointwise conditionals on infinite histories, and it does not resolve the ambiguity between the bi-infinite stationary process and the finite-start synchronized agent, which has nonstationary transients. These gaps justify the reader's conditional verdict without moving it: the framework and examples are useful, the conclusions are plausibly correct, but the proof as written does not establish them. No additional concern about Theorem 1 changes this assessment, since that identity is definitional rather than a source of correctness risk. The paper's concrete epsilon-machine examples and explicit computations provide partial independent support for the framework, but they do not test the general ergodicity claim. The proposed factor-map re-derivation would settle whether the gap is merely cosmetic or hides a real failure for some stationary ergodic process.","tokens_in":34766,"tokens_out":7259,"duration_ms":80227,"concrete_test":"Rewrite the proof of Prop. 12 as a factor-map argument: for each temporal atom in Table III, exhibit a Borel-measurable, shift-equivariant function g_A: X^Z → R^Z such that (g_A(X))_t equals the atom at time t, and verify E[|g_A|] < ∞ under the finitary assumption. For r_μ and b_μ, this requires a regular conditional distribution of X_t given the σ-algebras generated by ←X_t and →X_t; if that fails for some stationary ergodic process, the atom is not well-defined. If the derivation succeeds for all atoms, the conclusions of Prop. 12 stand and the paper needs only a corrected proof. If it fails for one atom, compute its time average on a long Even-process realization from the start state and compare with the claimed ensemble value; disagreement would demonstrate the nonstationary-transient exception to the stated guarantee.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Propositions 9–12 do not prove what the central claim needs. Proposition 12 asserts that every self-information process I[A|A](t) is stationary and ergodic if the underlying process is, but its proof says these are 'finite-range functions' and invokes the earlier sliding-window results. This premise is false for the objects actually defined: causal states σ_t = ε(←X_t) in §V.A depend on the semi-infinite past, and the atoms in Table III, e.g. r_μ(t) = I[X_t | ←X_t, →X_t] and b_μ(t) = I[X_t, →X_t | ←X_t], condition on semi-infinite pasts and futures. So Propositions 2–3 do not apply. The claims are probably salvageable by showing each information process is a measurable, shift-equivariant function of the environment and invoking Billingsley Thm 36.4 directly, but the paper does not supply the measurability and well-definedness arguments for pointwise conditional informations on semi-infinite histories, nor does it identify whether the claimed stationarity refers to the bi-infinite stationary ensemble or to the finite-start synchronized agent, which has nonstationary transients in the paper's own examples (§VII.D). Until this is supplied, the ergodicity guarantee underwriting downstream inference is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces \"information processes\"—real-time time series of Shannon information measures that a cognitive agent generates while observing a stochastic environment. It develops a framework in which an agent uses the environment's ε-machine as its internal model, then claims that if the environment is stationary and ergodic, the resulting information processes (entropy rate, ephemeral, bound, and semantic information processes, including causal-state processes) are also stationary and ergodic. It further defines a notion of intrinsic semantics via causal states, proves that the average degree of meaning equals the statistical complexity (Theorem 1), and illustrates the framework on four example processes (biased coin, period-2, golden mean, and even processes) with numerical plots of the information processes.","tokens_in":34944,"tokens_out":3245,"duration_ms":27318,"significance":"If the central ergodicity claim is established rigorously, the paper provides a useful conceptual unification: many time-local informational quantities used in statistical mechanics and complex systems are shown to be statistically well-behaved, justifying downstream inference and decision-making. The paper's integration of computational mechanics, Shannon information measures, and a semantic interpretation through causal states is a strength, as is its concrete treatment of illustrative processes with explicit ε-machines. The examples and the distinction between subjective and intrinsic semantics are clear and pedagogically valuable. However, the proof mechanism for the key ergodicity propositions is currently insufficient, and Theorem 1 is a definitional identity rather than a substantive result as stated. With appropriate revisions, the paper would be a solid contribution to the statistical mechanics of information processing.","major_comments":[{"comment":"The proofs of Propositions 9–12 rely on describing the relevant quantities as finite-range functions of the environment process, but this is not correct for the objects actually defined. Causal states σ_t = ε(←X_t) (§V.A) depend on the entire semi-infinite past, and the information atoms in Table III such as r_μ(t) = I[X_t | ←X_t, →X_t] and b_μ(t) = I[X_t, →X_t | ←X_t] condition on semi-infinite pasts and futures. Propositions 2 and 3 apply only to finite-range sliding-window functions. The ergodicity and stationarity conclusions are likely salvageable by showing each information process is a measurable, shift-equivariant function of the environment and by invoking the ergodic theorem for such factors (e.g., Billingsley, Theorem 36.4), but the manuscript does not supply the needed measurability, well-definedness, or integrability arguments for pointwise conditional informations on semi-infinite histories. This is a load-bearing gap for the paper's central claim.","section":"§VI.F, Propositions 9–12"},{"comment":"Theorem 1 states that the average semantic information equals the statistical complexity C_μ, but this follows directly from Definition 8: Θ(x) is defined as −log₂ Pr(σ) where σ is the causal state selected by x. The proof simply recognizes ⟨Θ(x)⟩ = −Σ_σ Pr(σ) log₂ Pr(σ) = H[S] = C_μ. As written, the 'theorem' is a restatement of the definition and does not establish a substantive connection between semantics and complexity beyond the chosen definition. The authors should either reframe this as a definitional identity or, if a deeper claim is intended, state and prove it from more primitive assumptions.","section":"§VI.E, Theorem 1"},{"comment":"There is an ambiguity about which stochastic process the stationarity claims in Propositions 9–12 refer to. The paper distinguishes the causal-state process (defined for all times) from the recurrent causal-state process (after synchronization), and §VII.D explicitly notes that the Even Process exhibits an infinite-duration transient during which information processes such as h_μ(t) and C_μ(t) are not stationary. The proofs of Propositions 9–12 assert stationarity for the causal-state process without specifying the initial distribution or whether the process is taken from the bi-infinite stationary ensemble or from a finite-start initialization. The latter is nonstationary, as the paper itself concedes. The statement of these propositions must be made precise about the ensemble in question, or the conclusions will be incorrect for the finite-start agent setting that the examples use.","section":"§V.B and §VII.D"}],"minor_comments":[{"comment":"The caption states that \"the forward and reverse entropy rates—h+_μ(t) and h+_μ(t), respectively—are equal for stationary processes,\" but the two displayed symbols appear to be identical; presumably one should be h+_μ(t) and the other h−_μ(t).","section":"Table III caption"},{"comment":"The text says the Biased Coin Process with Pr(x=1)=2/3 has entropy rate h_μ = 1 bit per time step. The entropy rate of an IID binary source with p=2/3 is approximately 0.918 bits, not 1 bit. The figure and table values should be checked for consistency with the stated bias.","section":"§VII.A, Biased Coin example"},{"comment":"The text refers to \"the shift operator τ of Eq. (3)\" when describing the time evolution of information measures, but Eq. (3) is the definition of the push-forward measure μY and not the shift operator. The reference should be to the shift defined in §III.A.","section":"§IV.D and Figure 7"},{"comment":"The displayed expression for μY(y) contains a double integral over ω and z that is notationally ambiguous and likely not what is intended; the measure μY should be defined by a single push-forward integration, and the intermediate sliding-window construction should be separated more cleanly.","section":"Proposition 1 proof, Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The paper is a largely expository development that packages existing computational mechanics and information-measure results into a new process-level framing. The novelty of the ergodicity claims is limited by the fact that the intended proof route (measurable factor maps) is standard, but the current manuscript does not execute it correctly. The definitional status of Theorem 1 should be made explicit to avoid overclaiming. The numerical example issue with the biased coin entropy rate should also be corrected. Overall, the framework is potentially useful, but the technical gaps must be addressed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Janet,\n\nQuick take on Crutchfield and Jurgens. The paper does a useful thing: it packages the time series of Shannon information measures—entropy rate, bound information, ephemeral information, and the rest—as stochastic processes ('information processes') and asks when they inherit stationarity and ergodicity from a stationary, ergodic environment. That framing is genuinely, if modestly, new. The worked examples (Golden Mean, Even, misdirected semantics) are clear, the background is solid, and the authors correctly cite Billingsley's theorem on functions of ergodic processes.\n\nThe soft spot is exactly the one the stress-test note identifies. Propositions 9–12 claim stationarity and ergodicity for the causal-state process, prediction uncertainty, causal-state uncertainty, and general self-information processes by calling them 'finite-range functions.' That premise is false. Causal states depend on the entire semi-infinite past, and the pointwise self-informations in Table III condition on semi-infinite pasts and futures (e.g., r_mu(t) = i[X_t | past, future]). So Propositions 2–3, which handle finite-range functions, do not apply. The conclusions are probably true—these objects are measurable, shift-equivariant functions of the environment, so a factor-map argument plus Billingsley should work—but the paper does not provide that argument. It also leaves ambiguous whether stationarity holds for the bi-infinite stationary measure or for a finite-start agent with nonstationary transients. The paper concedes the transients but does not resolve the ambiguity in the proofs.\n\nI also agree with the reader that Theorem 1 is definitional. The degree of meaning is defined as −log Pr(causal state), so its average is the entropy of the causal-state distribution by construction. Calling that a theorem is a packaging choice, not a substantive result.\n\nThe heavy self-citation is not a flaw here; those references point to real prior work. No code or data shipped, but the examples are specified well enough to reimplement.\n\nBottom line: this deserves a serious referee, but with the expectation of a major revision. The conceptual framework is useful for people monitoring learning agents or information engines, and the examples are worthwhile. The proof gap in Propositions 9–12 is real and needs to be closed before the ergodicity guarantee underwrites downstream inference.","headline":"A readable packaging of existing computational-mechanics results, but the central ergodicity proofs for causal-state and pointwise information processes rest on a finite-range premise the objects don't satisfy.","tokens_in":35566,"tokens_out":3017,"would_cite":false,"duration_ms":27295,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60G10","37A30","94A17"],"pacs":[],"model":"deepseek-v4-flash","headline":"Information signals inherit ergodicity; meaning equals memory","keywords":["information processes","cognitive agents","ϵ-machines","computational mechanics","stationarity","ergodicity","statistical complexity","measurement semantics"],"falsifier":"Run a synchronized ϵ-machine agent on one long realization of the Even Process (a stationary ergodic, infinite-Markov-order generator) and compare the time average of the pointwise bound-information process $b_\\mu(t)$ with the ensemble average computed from the stationary causal-state distribution; if the discrepancy does not vanish with sequence length, the claimed ergodicity of information processes fails.","tokens_in":34407,"feed_emoji":"🤖","tokens_out":7357,"duration_ms":64169,"temperature":0.7,"pith_summary":"This paper aims to show that the real-time informational signals produced by a cognitive agent—its moment-by-moment estimates of surprise, stored information, and predictive uncertainty—are themselves stochastic processes with good statistical properties. For a stationary, ergodic environment, and an agent that uses the environment's minimal optimal predictive model (its ϵ-machine) and is synchronized to it, the paper argues that these information processes are stationary and ergodic. It also proposes an intrinsic semantics: the \"degree of meaning\" of an observation is the information needed to identify the causal state it brings the agent to, and the time-averaged meaning equals the statistical complexity, the information the environment stores in its causal states. If correct, this licenses using such real-time signals for reliable downstream estimation, monitoring, and decision-making, and gives a quantitative answer to what a measurement means.","feed_headline":"Information signals inherit ergodicity; meaning equals memory","feed_subtitle":"Stationary ergodic environments yield stationary ergodic agent signals; average meaning equals stored memory.","key_machinery":"The central objects are information processes: stochastic processes formed by applying a self-information function $i[\\cdot] = -\\log_2 \\Pr(\\cdot)$ to temporal information atoms built from the past $\\overleftarrow{X}_t$, present $X_t$, and future $\\overrightarrow{X}_t$ of the environment process. The load-bearing mechanism is the ϵ-machine—the minimal optimal predictive model whose states, the causal states $\\sigma_t = \\epsilon(\\overleftarrow{X}_t)$, are equivalence classes of pasts with identical future predictions. Because the causal-state process is first-order Markov, and because the paper treats each monitored self-information as a finite-range function of the process, Propositions 7–12 transfer stationarity and ergodicity from the environment to the information process. A second mechanism is \"degree of meaning\" $\\Theta(x) = -\\log_2 \\Pr(\\sigma)$, the information in the causal state an observation selects.","core_discovery":"On the paper's own terms: information processes—time series of Shannon information measures such as the pointwise entropy rate $h_\\mu(t)$, bound information $b_\\mu(t)$, ephemeral information $r_\\mu(t)$, and statistical complexity $C_\\mu(t)$—are functions of the observed environment process. Proposition 12 states that if the environment is stationary and ergodic, then the self-information processes $I[A|A](t)$ are stationary and ergodic; the argument runs through the causal-state process $\\sigma_t = \\epsilon(\\overleftarrow{X}_t)$ being a stationary ergodic first-order Markov process and the monitored quantities being finite-range functions of it. Theorem 1 states that the total average semantic information, $\\langle\\Theta(x)\\rangle$, equals the statistical complexity $C_\\mu = I[S]$, the Shannon entropy of the causal-state distribution. The accompanying examples—biased coin, period-2, Golden Mean, and Even processes—display what these real-time signals look like, including negative pointwise informations, and the misdirected-semantics tables show how an incorrect internal model changes the meaning an agent assigns.","pith_inferences":["Empirically, the ergodicity claim is testable on any finite-state generator: estimate the time average of a pointwise information atom over one long realization and compare to the ensemble average; the convergence rate should track the synchronization time.","The written proof that information processes are finite-range functions does not literally cover causal states, which depend on semi-infinite pasts; a measurable-factor-map argument would be needed to make Propositions 9–12 fully rigorous as stated.","For agents with incorrect internal models, the average degree of meaning may equal the entropy of the model's state distribution rather than the environment's statistical complexity, giving a quantitative measure of how wrong a model is; the paper only sketches this via short-word misdirected-semantics tables.","Because the pre-synchronization epoch is nonstationary, practical monitoring of information processes must either discard an initial burn-in period or model the transient separately, a point the paper itself notes in its online-prediction example."],"forward_implications":["If Proposition 12 holds, an agent can use time-averaged estimates of entropy rate, bound information, and statistical complexity from a single long realization, because time averages converge to ensemble averages.","Theorem 1 gives an operational account of meaning: the average semantic content of an optimal observer's interpretations is exactly the environment's stored information, $C_\\mu$.","The prediction process and causal-state process inherit ergodicity, so downstream inference over these signals is statistically grounded.","The intrinsic self-information grounded in the ϵ-machine resolves the ambiguity in Shannon's self-information of which probability distribution to use.","The example analyses show that pointwise information measures can be negative, so real-time interpretation requires updating intuitions from average information theory."],"supporting_citations":[{"why":"Supplies the explicit, efficient methods for calculating the informational measures from an ϵ-machine that the information-process setup relies on.","marker":"[24]"},{"why":"Establishes causal states, ϵ-machines, their Markovity, and statistical complexity $C_\\mu$, the objects the stationarity and semantic claims are built on.","marker":"[27]"},{"why":"Defines the temporal information atoms (ephemeral, bound, enigmatic, elusive rates) that form the information processes.","marker":"[50]"},{"why":"Introduces the measurement semantics and degree of meaning $\\Theta(x)$ whose time average Theorem 1 equates with $C_\\mu$.","marker":"[59]"},{"why":"Provides the convergence and topological properties of predictive states for infinite pasts, needed to justify the conditional measures in the self-informations.","marker":"[41]"},{"why":"Supplies synchronization results for hidden Markov models, the condition under which the agent's information processes are stationary.","marker":"[61]"},{"why":"Gives the ergodic-theory result used to prove that functions of ergodic processes are ergodic.","marker":"[42]"}],"fun_headline_variants":["Ergodicity of environment passes to agent's information streams","Mean semantic information equals statistical complexity","Agent information inherits ergodicity; meaning equals memory","In ergodic worlds, information signals and their meaning stabilize","Theorem: average meaning is stored memory in agent information"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the monitored quantities—causal states and self-informations conditioned on semi-infinite pasts and futures—are finite-range functions of the environment process, even though they actually depend on the entire infinite history; the ergodicity conclusions likely survive via measurable-function arguments, but the written proof mechanism only covers finite-range statistics.","fun_headline_variants_meta":{"raw":{"variants":["Ergodicity of environment passes to agent's information streams","Mean semantic information equals statistical complexity","Agent information inherits ergodicity; meaning equals memory","In ergodic worlds, information signals and their meaning stabilize","Theorem: average meaning is stored memory in agent information"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000316,"raw_usage":{"total_tokens":1729,"prompt_tokens":826,"completion_tokens":903,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":442,"completion_tokens_details":{"reasoning_tokens":828}},"tokens_in":442,"tokens_out":903,"duration_ms":6044,"temperature":1.0,"reasoning_tokens":828,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:18:26.661608+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a synchronized ϵ-machine agent on one long realization of the Even Process (a stationary ergodic, infinite-Markov-order generator) and compare the time average of the pointwise bound-information process $b_\\mu(t)$ with the ensemble average computed from the stationary causal-state distribution; if the discrepancy does not vanish with sequence length, the claimed ergodicity of information processes fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the explicit, efficient methods for calculating the informational measures from an ϵ-machine that the information-process setup relies on."},{"cited_title":"Gupta, S","cited_arxiv_id":null,"evidence_quote":"Establishes causal states, ϵ-machines, their Markovity, and statistical complexity $C_\\mu$, the objects the stationarity and semantic claims are built on."},{"cited_title":"Schrodinger","cited_arxiv_id":null,"evidence_quote":"Defines the temporal information atoms (ephemeral, bound, enigmatic, elusive rates) that form the information processes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the measurement semantics and degree of meaning $\\Theta(x)$ whose time average Theorem 1 equates with $C_\\mu$."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the convergence and topological properties of predictive states for infinite pasts, needed to justify the conditional measures in the self-informations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies synchronization results for hidden Markov models, the condition under which the agent's information processes are stationary."},{"cited_title":"Vicsek, A","cited_arxiv_id":null,"evidence_quote":"Gives the ergodic-theory result used to prove that functions of ergodic processes are ergodic."}],"review_version":1}