Pith. sign in

REVIEW 4 major objections 5 minor 107 references

Information Theory of Meaningful Communication

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that the semantic information of a clause can be estimated as the difference between total and wording information computed with LLM probabilities, yielding about 20 bits per clause on average.

desk verdict A promising but unvalidated estimator for semantic information in narrative: the headline 20 bits/clause is not supported by the paper's own numbers, but the idea and the data are worth a serious referee. read the letter →

arxiv 2411.12728 v1 pith:UF63P4OB submitted 2024-11-19 cs.CL cs.ITmath.IT

Doron Sivan , Misha Tsodyks This is my paper · ORCID
classification cs.CLcs.ITmath.IT MSC 94A1768T50
keywords semanticinformationtheorylargelanguagemodelsnarrativeclauseentropymeaningLabovnarratives
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the information carried by the meaning of a clause in natural narrative can be quantified in bits, and that large language models can supply the needed probabilities. It defines the semantic information of clause $C_i$ as $I_{M,i} = I_i - I_{W,i}$, where $I_i$ is the total information of the clause under the LLM and $I_{W,i}$ is the wording information estimated after a rephrased version of the story is presented as fixing the meaning. Applied to 20 Labov narratives, the measure yields about 20 bits of semantic information per clause, roughly half of the about 40 bits per clause of total information. If this is right, the paper provides an operational, LLM-based measure of meaning-level information in ordinary spoken stories.

What carries the argument

The load-bearing decomposition is $P(C_i|c_{<i}) = P(M_i|c_{<i}) P(C_i|M_i, c_{<i})$, which rewrites the probability of a clause's wording as the probability of its meaning times the probability of that wording given the meaning. Under the single-meaning approximation this yields $I_{M,i} = I_i - I_{W,i}$. The wording information $I_{W,i}$ is obtained by asking the LLM for conditional probabilities of the original clause tokens in a prompt that presents the rephrased narrative followed by the original and states that they tell the same narrative; the rephrasing is produced by GPT-4 with a clause-preserving prompt.

What would settle it

Construct a narrative and two rephrasings: one that preserves all meanings and one that swaps a single concrete detail, such as changing a swimming race to a running race in the Boy Scout story. If the semantic-information estimate for the altered clause does not decrease substantially relative to the faithful rephrasing, the meaning-fixing assumption fails in a directly observable way. A companion check: among clauses the LLM deems predictable, human guessers should mostly agree; the paper already reports 8 clauses ChatGPT predicts but no human participant could, which is a concrete place to look.

Watch

Extended reading notes

Core claim

The central claim is that, under a 'single meaning approximation' in which each clause has one meaning given its context, the probability of a clause's wording factors into the probability of its meaning times the probability of that wording given the meaning. This yields the identity $I_{M,i} = I_i - I_{W,i}$, so semantic information is the total information of the clause minus the information that is due to the particular wording once the meaning is fixed. The paper estimates both terms with Llama-3.1-70B token probabilities, using a GPT-4-generated rephrasing of the narrative to fix the meaning in the wording-information prompt. The result for 20 Labov narratives is an average semantic information rate near 20 bits per clause, with clauses ranging from roughly 0 to 100 bits and 14 clauses showing negative values that the paper attributes to limitations of the meaning-fixing procedure.

Load-bearing premise

The method assumes that the rephrased narrative placed in the second prompt makes the LLM's conditional probability of the original wording equal to $P(C_i|M_i, c_{<i})$; that is, the paraphrase fixes exactly the meaning of each clause and reveals nothing about its phrasing, style, or register, and the paper's own examples show meaning drift and 14 clauses with negative semantic information indicate the assumption is violated in practice.

Editorial extensions

If this is right

  • The measured rate of about 20 semantic bits per clause, with about 40 total bits per clause, means roughly half of the information in each clause is carried by wording and half by meaning.
  • Clauses whose next meaning can be guessed from preceding context mostly have semantic information near zero, so the measure tracks human predictability.
  • On average the first clause of a narrative is the most informative when the opening question is not supplied; supplying the interviewer's guiding question removes this trend.
  • The estimates are similar when computed with partial rephrasings, with a second independent rephrasing, or with different LLMs (Llama-3.1-8B, Llama-3.1-8B-Instruct, Mistral-Nemo-12B).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the same measure could be applied to dialogues, where shared context should lower semantic information per turn; a drop in $I_M$ across conversational turns would support the paper's claim that context constrains meaning.
  • My inference: if the method is sound, it provides a quantitative handle on 'gist' memory, consistent with the well-known finding that people retain meaning while forgetting exact wording; one could test this by comparing $I_M$ of recalled versus forgotten clauses in memory experiments.
  • My inference: the 14 negative-$I_M$ clauses are a diagnostic resource; analyzing them could reveal exactly which paraphrase failures (lexical, syntactic, or stylistic) push the estimate below zero and suggest improved prompts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an information-theoretic decomposition of clause-level information in narratives into semantic information and wording information, and uses LLM conditional probabilities to estimate both terms. The semantic information of each clause is computed as the difference between the total information of the clause and its wording information, where the latter is estimated by conditioning on a ChatGPT-generated rephrasing of the narrative that is intended to fix the clause's meaning. Applied to 20 Labov personal-experience narratives, the method yields an average semantic information of about 20 bits per clause, roughly half of the total information per clause. The paper reports that clauses humans can predict from context indeed have low semantic information, and that the initial clauses of a narrative carry more information unless the interviewer's guiding question is provided.

Significance. If the proposed identification is valid, the paper offers a novel operational measure of meaning-level information in natural language, bridging Shannon information and semantics, with potential applications to dialogue and summarization. The authors provide open data, a human guessing benchmark, and cross-model consistency checks (SI S.6-S.7), which are valuable assets. However, the central equality between the LLM probability under the rephrasing prompt and P(C_i|M_i,c_<i) is assumed rather than derived, and the paper's own examples and numerical results (negative semantic information, three human-predictable clauses with 10-12 bits) show that the estimator is not faithful to this equality. The headline average of ~20 bits/clause therefore rests on an unvalidated identification.

major comments (4)
  1. [Section 2, Eqs. (5)-(6); SI S.3] The decomposition in Eq. (5) implies that I_M,i = I_i - I_W,i >= 0 for every clause, because I_i = -log P(C_i|c_<i) = -log P(M_i|c_<i) + I_W,i and P(M_i|c_<i) <= 1. Yet SI S.3 reports 14 clauses with negative semantic information, between -3 and 0 bits. This is an internal contradiction if the LLM estimates faithfully implement the conditional probabilities in Eq. (5). The paper describes these as 'wrong estimates' but does not provide an error model or a corrected estimator, and the headline average of about 20 bits/clause is computed over all clauses including these negative values. The authors should either enforce non-negativity with a constrained estimator or provide a quantitative bias analysis showing that the negative values are within the estimation noise and do not affect the reported average.
  2. [Section 2, SI S.2.2-S.2.3] The central identification P(C_i|M_i,c_<i) = P(C_i | prompt containing a ChatGPT rephrasing) is asserted rather than derived or validated. The paper's own Boy Scout example (SI S.2.2) shows that the rephrasing changed the meaning (a swimming race became a running race) and shifted the register from colloquial to formal. Therefore Prompt 2 conditions on a different meaning and on style features, violating the 'single meaning approximation' and the assumption that the rephrasing leaks nothing about phrasing. The consistency checks in SI S.6 demonstrate reproducibility across rephrasings, but reproducibility does not establish correctness; a direct validation of the equality, for instance using human-verified meaning-preserving paraphrases or measuring whether I_W estimates are invariant to style variations, is needed before the reported 20 bits/clause can be accepted.
  3. [Section 3, Fig. 3A] Three clauses that were correctly predicted by human participants have estimated semantic information between 10 and 12 bits. The paper acknowledges that this 'indicates that our method sometimes results in wrong estimates of information,' but offers no explanation or remedy. Because the method is explicitly intended to rank clause predictability (clauses with low I_M should be predictable), these outliers are a direct counterexample for a non-negligible fraction (3/23) of the human-predictable clauses. The authors should report the full distribution of I_M for the 23 human-predictable clauses, including the fraction exceeding a threshold, and discuss whether the method can still be used for ranking despite these failures.
  4. [Section 2 and SI S.5] The headline estimates use the full rephrasing in Prompt 2, which includes rephrased clauses after C_i, so the model's probability is conditioned on information about the future of the narrative, whereas the decomposition in Eq. (5) conditions only on c_<i. The partial-rephrasing test in SI S.5 reports that the wording information differs by -1.1 ± 2.3 bits between the partial and complete rephrasing conditions; this non-zero difference shows a systematic effect of future context, and the sign of the difference appears inconsistent with the stated interpretation that future context further constrains the meaning. The paper neither corrects the headline numbers for this bias nor justifies why the full-rephrasing estimate is preferable for a quantity defined only with respect to preceding context.
minor comments (5)
  1. [Section 2] The paragraph beginning 'In Fig. 1A, we show the cumulative sums of I_i...' appears twice in the main text, once before and once after Eq. (6); the duplication should be removed.
  2. [Figure 1] In Fig. 1B, the inset histogram's axis label is truncated ('Sem. Info. [bits]'); please provide a complete label, and ensure all subplot axes are clearly labeled.
  3. [SI S.2.2] Example 3 (Gloria Stein) is difficult to read because the original and rephrased clauses are not aligned in a table; consider formatting it as a two-column table for clarity.
  4. [SI S.2.1] The token-level information I_x,i is defined but the relationship to the clause-level information I_i is not stated explicitly; please add a sentence clarifying that I_i is the sum of I_x,i over the tokens of the clause.
  5. [References] Reference [18] is to a preprint; if a published version exists, it should be cited instead or in addition.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the semantic-information estimate is an operational definition with an external human benchmark; the only self-citation is incidental and non-load-bearing.

full rationale

The derivation chain is not circular. Total information Ii is computed directly from Llama token log-probabilities over the original narrative; wording information IW,i is computed from a separately generated gpt-4 paraphrase placed in Prompt 2; semantic information is then defined as the difference (Eq. 6). No parameter is fitted to the 20-bits-per-clause result, and the human guessing experiment is an external benchmark that does not depend on the estimator's own parameters. The main weakness, that Prompt 2 may not actually put the LLM in the conditional state P(Ci|Mi,c<i), is an identification or validity problem rather than a circular reduction: the paper's own Boy Scout example shows meaning drift, and the 14 negative IM values in SI S.3 indicate that the estimator sometimes violates the paper's decomposition, but this is evidence of estimator error, not of the output being equivalent to the input by construction. The only self-citation ([18], by co-author Tsodyks) is used for an anecdotal observation that human subjects also misread the Boy Scout 'dash' as running; it does not carry the derivation and is not load-bearing. Thus no substantive circularity is present; score 2 reflects only the minor, non-load-bearing self-citation.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The paper introduces no fitted constants in the classical sense; the only hand-chosen parameter is the rephrasing chunk size. The derivation rests on the single-meaning approximation, on trusting LLM probabilities as language statistics, and on the critical assumption that a paraphrased narrative isolates meaning. No new physical or mathematical entities are postulated; semantic information is an operational definition rather than an invented entity.

free parameters (1)
  • Rephrasing chunk size Lc = 50 clauses, not fitted
    Hand-chosen threshold for splitting long narratives into chunks for rephrasing. The paper shows partial rephrasing gives similar semantic information (SI S.5), so this parameter is not load-bearing.
assumptions (5)
  • domain assumption Single meaning approximation: given context, each clause has exactly one meaning, so P(Mi|Ci,c<i)=1 if Ci means Mi and 0 otherwise.
    Invoked at Eqs. (4) and (5). If clauses are ambiguous in context, the sum over meanings does not collapse and the semantic information estimate is undefined.
  • domain assumption LLM conditional probabilities approximate the true ensemble probabilities P(Ci|c<i) for natural narratives.
    Central to estimating Ii; the paper uses Llama3.1-70B-Instruct as a proxy for human language statistics without quantifying the approximation error.
  • ad hoc to paper The rephrasing in Prompt 2 fixes the meaning of each original clause without leaking extra information, so P(Ci|rephrasal,c<i) = P(Ci|Mi,c<i).
    This is the load-bearing identification of the method (SI S.2.3). If the rephrasing changes meaning or leaks style, IW,i is wrong and IM,i is contaminated.
  • domain assumption The 20 Labov narratives are a representative sample from an ensemble of personal experience narratives.
    Used to compute average slopes and per-position averages in Fig. 3B; the sample is small and may not generalize to other genres.
  • standard math Standard probability decomposition: P(Ci|c<i) = sum over meanings P(Mi|c<i) P(Ci|Mi,c<i).
    A standard application of the law of total probability under the single-meaning approximation; not controversial.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Information Theory of Meaningful Communication." pith.science (2026). https://pith.science/paper/UF63P4OB

@misc{pith2026241112728,
  author       = {Pith},
  title        = {Pith review of: Information Theory of Meaningful Communication},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UF63P4OB}},
  note         = {Machine review of arXiv:2411.12728}
}
read the original abstract

In Shannon's seminal paper, entropy of printed English, treated as a stationary stochastic process, was estimated to be roughly 1 bit per character. However, considered as a means of communication, language differs considerably from its printed form: (i) the units of information are not characters or even words but clauses, i.e. shortest meaningful parts of speech; and (ii) what is transmitted is principally the meaning of what is being said or written, while the precise phrasing that was used to communicate the meaning is typically ignored. In this study, we show that one can leverage recently developed large language models to quantify information communicated in meaningful narratives in terms of bits of meaning per clause.

Figures

Figures reproduced from arXiv: 2411.12728 by the authors.

Figure 1
Figure 1. Semantic information in Labov narratives. (A) Cumulative total and semantic information (Pi j=1 Ij and Pi j=1 IM,j , respectively) up to clause Ci , as a function of clause number, i, for 20 Labov narratives (each narrative is represented by a unique color, with matching colors corresponding to the same narrative). The average slope of these curves is approximately 40 and 20 bits/clause, for total and semantic infor… view at source ↗
Figure 2
Figure 2. Estimating semantic information of a clause in a narrative. We illustrate our method for estimating the semantic information of clause 8 for the "Boy scout" narrative [1]. (A) Total information, I, is estimated by summing the information contents of each token in the clause, conditional on the preceding clauses. (B) Wording information, IW , is estimated by summing the information contents of each token in the claus… view at source ↗
Figure 3
Figure 3. Predictable and surprising clauses in Labov narratives. (A) Semantic information (IM) histogram of predictable clauses, which are clauses whose meaning can be guessed from the preceding part of the narrative with high probability (see text). Predictable clauses have mostly semantic information close to zero. (B) Semantic information as a function of clause number, averaged over the 20 Labov narratives. On average, s… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

107 extracted references · 70 canonical work pages

  1. [1]

    Narrative Analysis: Oral Versions of Personal Experience

    William Labov and Joshua Waletzky. Narrative Analysis: Oral Versions of Personal Experience. In June Helm, editor, Essays on the V erbal and Visual Arts: Proceedings of the 1966 annual spring meeting of the American ethnological society, pages 12–44, 1966

    1966

  2. [2]

    The language of life and death: The transformation of experience in oral narrative

    William Labov. The language of life and death: The transformation of experience in oral narrative . Cambridge University Press, 2013

    2013

  3. [3]

    Claude E. Shannon. Prediction and entropy of printed English. The Bell system technical journal , 30(1):50–64, 1951

    1951

  4. [4]

    Dhillon, Zhaohui Yang, Aylin Yener, Kai Kit Wong, and Chan-Byoung Chae

    Deniz Gündüz, Zhijin Qin, Inaki Estella Aguerri, Harpreet S. Dhillon, Zhaohui Yang, Aylin Yener, Kai Kit Wong, and Chan-Byoung Chae. Beyond transmitting bits: Context, semantics, and task-oriented communications. IEEE Journal on Selected Areas in Communications , 41(1):5–41, 2022

    2022

  5. [5]

    Jacqueline S. Sachs. Recognition memory for syntactic and semantic aspects of connected discourse. Perception & Psychophysics, 2(9):437–442, 1967

    1967

  6. [6]

    Claude E. Shannon. A mathematical theory of communication. The Bell system technical journal , 27(3):379–423, 1948. 5 A PREPRINT - NOVEMBER 20, 2024

    1948

  7. [7]

    A convergent gambling estimate of the entropy of English

    Thomas Cover and Roger King. A convergent gambling estimate of the entropy of English. IEEE Transactions on Information Theory, 24(4):413–421, 1978

    1978

  8. [8]

    Brown, Stephen A

    Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, Jennifer C. Lai, and Robert L. Mercer. An estimate of an upper bound for the entropy of English. Computational Linguistics, 18(1):31–40, 1992

    1992

Show all 107 references
  1. [9]

    Entropy estimation of symbol sequences

    Thomas Schürmann and Peter Grassberger. Entropy estimation of symbol sequences. Chaos: An Interdisciplinary Journal of Nonlinear Science, 6(3):414–427, 1996

    1996

  2. [10]

    Matthew V . Mahoney. Text compression as a test for artificial intelligence. AAAI, 1999

    1999

  3. [11]

    Le, and Ruslan Salakhutdinov

    Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V . Le, and Ruslan Salakhutdinov. Transformer-XL: Attentive language models beyond a fixed-length context. arXiv:1901.02860, 2019

    1901 arXiv

  4. [12]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  5. [13]

    Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, and others (OpenAI)

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, and others (OpenAI). Language models are few-shot learners. arXiv:2005.14165, 2020

    2005 arXiv

  6. [14]

    The Llama 3 Herd of Models

    Meta AI Llama Team. The Llama 3 Herd of Models. arXiv.2407.21783, 2024

    2024 arXiv

  7. [15]

    Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv:2302.09664, 2023

    2023 arXiv

  8. [16]

    A mathematical theory of semantic communication

    Kai Niu and Ping Zhang. A mathematical theory of semantic communication. arXiv:2401.13387, 2024

    2024 arXiv

  9. [17]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. arXiv:2303.08774, 2023

    2023 arXiv

  10. [18]

    Using large language models to study human memory for meaningful narratives

    Antonios Georgiou, Tankut Can, Mikhail Katkov, and Misha Tsodyks. Using large language models to study human memory for meaningful narratives. bioRxiv:2023.11.03.565484, 2023

    2023

  11. [19]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  12. [20]

    Quantifying memorization across neural language models

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. arXiv:2202.07646, 2022

    2022 arXiv

  13. [21]

    Preventing verbatim memorization in language models gives a false sense of privacy

    Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A Choquette-Choo, and Nicholas Carlini. Preventing verbatim memorization in language models gives a false sense of privacy. arXiv:2210.17546, 2022

    arXiv

  14. [22]

    Jang and Thomas Lukasiewicz

    Myeongjun E. Jang and Thomas Lukasiewicz. Consistency analysis of ChatGPT. arXiv:2303.06273, 2023

    2023 arXiv

  15. [23]

    Mistral 7B

    Mistral AI team. Mistral 7B. arXiv:2310.06825, 2023

    2023 arXiv

  16. [24]

    In vain I tried to tell you

    Mistral AI team. Mistral NeMo. mistral.ai/news/mistral-nemo, 2024. 6 A PREPRINT - NOVEMBER 20, 2024 Supplementary Information Contents S.1 Labov narratives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 S.2 Estimation of semantic ...

    2024

  17. [25]

    We moved down to Third and uh Wharton. [ ind.]

  18. [26]

    And then of course they – they – well they stoned the house [ ind.]

  19. [27]

    And I had sense enough to get the kids into the stairway [ ind.] so that they wouldn’t get hurt by flyin’ gla- glass. [dep.]

  20. [28]

    but I remember saying a prayer [ ind.] as I went out the front door [dep.] because I didn’t know [dep.] what was gonna hit me [dep.] when I went out that door. [dep.]

  21. [30]

    Yeah I was in the boy scouts at the time. 1. Indeed, I was a member of the boy scouts during that period

  22. [31]

    We were participating in a 50-yard sprint

    And we was doing the 50-yard dash 2. We were participating in a 50-yard sprint

  23. [32]

    at a designated pier

    but we was at the pier marked off 4. at a designated pier

  24. [33]

    and so we was doing the 50-yard dash. 5. Hence, we were engaged in the 50-yard sprint

  25. [34]

    There was about 8 or 9 of us you know, going down, coming back

  26. [35]

    There were approximately 8 or 9 of us, running forth and back

  27. [36]

    On the third round, I was hit with cramps

    And going down the third time I caught cramps 7. On the third round, I was hit with cramps

  28. [37]

    and I began to shout for assistance

    and I started yelling "Help!" 8. and I began to shout for assistance. 9 A PREPRINT - NOVEMBER 20, 2024

    2024

  29. [38]

    but the fellows didn’t believe me you know. 9. However, my peers didn’t take me seriously

  30. [39]

    They thought I was just trying to catch up because I was going on or slowing down

  31. [40]

    wording information prompt

    And then when he understood what the situation was [ dep.] he turned to my mother [ind.] . . . S.2 Estimation of semantic information The semantic information IM was estimated using the following procedure. We generated a narrative rephrasing using ChatGPT, and then calculated...

    2024

  32. [41]

    They assumed I was merely trying to keep pace because I was either advancing or slowing down

  33. [42]

    So all of them kept going. 11. Consequently, they all continued running

  34. [43]

    They leave me. 12. They abandoned me

  35. [44]

    And so I started going down. 13. As a result, I began to sink

  36. [45]

    Scoutmaster was up there. 14. The scoutmaster was present

  37. [46]

    He was watching me. 15. He was observing me

  38. [47]

    But he didn’t pay me no attention either. 16. But he also disregarded me

  39. [48]

    And for no reason at all there was another guy who had just walked up that minute

  40. [49]

    Unexpectedly, another individual who had just arrived at that moment

  41. [50]

    He spontaneously leaped over

    He just jumped over 18. He spontaneously leaped over

  42. [51]

    They stoned the house

    and grabbed me. 19. and rescued me. Example 3 Gloria Stein: "They stoned the house" Original Rephrased

  43. [52]

    We moved down to Third and uh Wharton. 1. We relocated to the intersection of Third and Whar- ton

  44. [53]

    Our move had been completed,

    We had gotten moved 2. Our move had been completed,

  45. [54]

    but it was like – you remember I told you my father drank

  46. [55]

    but it was during the day - recall I mentioned my father’s drinking habit

  47. [56]

    Despite it being daytime,

    but it was still day 4. Despite it being daytime,

  48. [57]

    and after he had gotten in and put the furniture where it was supposed to go my mother left, to go to market to get some food to bring into the house

  49. [58]

    after my father had settled in and arranged the fur- niture, my mother departed to the market to purchase groceries for our new home

  50. [59]

    And my father went to take the horses – take the team of horses back

  51. [60]

    [After the mother left for the market, the house was stoned

    Meanwhile, my father left to return the horses - the pair that had helped us move. . . . . . . [After the mother left for the market, the house was stoned. Gloria ran to her, and they met a policeman who warned them about the neighborhood. The mother refused to move and decide...

  52. [61]

    She took me along to the police station

  53. [63]

    They even had police wagons drawn by horses in those days

  54. [65]

    What action will you take?

    My mother asked the captain, "What action will you take?"

  55. [66]

    And we went – she took me and we went to the police station

  56. [67]

    Believe it or not in those days

    And they also had police wagons with horses attached to them. Believe it or not in those days

  57. [68]

    What are you going to do about it?

    So my mother said to the captain, "What are you going to do about it?"

  58. [69]

    Well lady there’s nothing we can do about it

    And he says "Well lady there’s nothing we can do about it."

  59. [70]

    Unfortunately, there’s nothing we can do

    He replied, "Unfortunately, there’s nothing we can do."

  60. [71]

    You mean to tell me that I have to live under those circumstances and there’s nothing that you can do about it?

    And she says "You mean to tell me that I have to live under those circumstances and there’s nothing that you can do about it?"

  61. [72]

    Are you implying that I must endure these conditions and you’re incapable of providing any assistance?

    She retorted, "Are you implying that I must endure these conditions and you’re incapable of providing any assistance?"

  62. [73]

    Well uh uh the only thing I can tell you to do is to move

    She says – he says "Well uh uh the only thing I can tell you to do is to move."

  63. [74]

    The only advice I can offer is to relocate

    He suggested, "The only advice I can offer is to relocate." . . . . . . [The mother refused and insisted on a police escort home. They arrived in front of curious neighbors, and the mother declared they were staying.]

  64. [75]

    During our time there, my father fell ill

  65. [76]

    The neighbors turned out to be as kind as anyone could hope for

  66. [77]

    10 A PREPRINT - NOVEMBER 20, 2024

    They reacted to trouble just like anyone else would. 10 A PREPRINT - NOVEMBER 20, 2024

    2024

  67. [81]

    ”’ original clause ”’

  68. [82]

    The notice in the New York Times

    ”’ proposed clause ”’ Do they convey essentially the same meaning (wording/phrasing may differ)? Answer in a step-by-step manner. At the end of your answer provide a True/False decision in the following format: **Same meaning: True/False**. In Prompt 4 we used (zero-shot) chai...

  69. [83]

    Yeah there is an instance

  70. [84]

    Dad was being driven out from town – ah – by his chauffeur

  71. [85]

    This was a good many years ago

  72. [86]

    And he had the New York Times

  73. [87]

    and he read in the New York Times

  74. [88]

    and noticed that – the death of a person whom he knew but he knew was a very close friend of George Jensen

  75. [89]

    George Jensen lived in Chestnut Hill

  76. [90]

    Stop at Mr. Jensen’s house on the way home ’cause I want to commiserate with him

    So Dad said to the chauffeur "Stop at Mr. Jensen’s house on the way home ’cause I want to commiserate with him."

  77. [91]

    The chauffeur nodded and changed the route accordingly

    So they did stop 12 A PREPRINT - NOVEMBER 20, 2024 9’. ChatGPT: "The chauffeur nodded and changed the route accordingly." (semantic information of 5.4 bits)

    2024

  78. [92]

    George, I’m so sorry to hear about the death of —

    and Dad said "George, I’m so sorry to hear about the death of —"

  79. [93]

    I don’t know his name

  80. [94]

    and George Jensen said "I don’t know what you’re talking about

  81. [95]

    If he had died I would have been one of the first people to know."

  82. [96]

    And Dad said "Well it’s in the newspaper,

  83. [97]

    I’ll go out to the car

  84. [98]

    and get the newspaper."

  85. [99]

    and retrieved the newspaper

    and got the newspaper 19’. ChatGPT: "and retrieved the newspaper." (semantic information of 1.8 bits)

  86. [100]

    and brought it back inside

    came back 20’. ChatGPT: "and brought it back inside." (semantic information of 3.4 bits)

  87. [101]

    and he and George went through the newspaper

  88. [102]

    No sign of this death notice

  89. [103]

    And just as they were finished perusing it the telephone rang from somebody in New York telling George Jensen that, guess he’d died

  90. [104]

    But there was nothing in the newspaper

  91. [105]

    Dad brought the newspaper home

  92. [106]

    My sister – guess George was home at the time – and I all went through the newspaper meticulously

  93. [107]

    Still, there was no mention of the death notice

    Couldn’t find anything. 27’. ChatGPT: "Still, there was no mention of the death notice." (semantic information of 0.74 bits) As discussed in the paper, a possible concern is that maybe the LLM had seen the narratives during training and thus is biased to output similar meaning...

    2024

  94. [125]

    So in the meantime – not in the meantime but during during the period of time that we were there, my father became sick

  95. [126]

    And the neighbors were as nice as anybody could ever have been

  96. [127]

    When there was trouble they responded just like anyone else

  97. [128]

    But that was our welcoming to the neighborhood. 128. But that was our introduction to the neighbor- hood

  98. [129]

    It was a terrible experience,

    And it was terrible 129. It was a terrible experience,

  99. [130]

    Boy scout

    and it certainly was frightening for me. 130. and it was certainly scary for me. S.2.3 wording and semantic information We use Prompt 2 to estimate the wording information of the clauses in the narrative, whererephrased is the rephrased narrative and original is the original o...

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.