On the Persistent Effects of Lexicality in Large Language Models

A.B. Siddique; Hammad Rizwan; Hassan Sajjad; Mona T. Diab; Muhammad Umair Haider; Nishant Subramani

REVIEW 1 major objections 1 minor 20 references

On the Persistent Effects of Lexicality in Large Language Models

T0 review · 1 major / 1 minor · reviewed 2026-06-28 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Lexical overlap shapes LLM representations more than semantic content across all layers and model types.

desk verdict Lexical effects on LLM representations hold across depths in the reported tests, but the mid-depth transitional regime claim rests on stress tests whose construction details are missing from the abstract. read the letter →

arxiv 2606.02750 v2 pith:3JYWLKTW submitted 2026-06-01 cs.CL

Hammad Rizwan , Muhammad Umair Haider , Nishant Subramani , Mona T. Diab , A.B. Siddique , Hassan Sajjad This is my paper · ORCID

classification cs.CL

keywords lexicalinfluencesemanticrepresentationslargelanguagemodelsadversarialstresstestsmodeldepthrepresentationalanalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper measures how much hidden states in large language models reflect shared words versus actual meaning. It applies adversarial sentence pairs that keep lexical overlap high while varying semantic content. Lexical signals remain strong from early layers to the final ones in every architecture and training setup tested, including models optimized for semantic similarity. A middle depth range appears where both lexical form and semantic meaning signals weaken at once. This pattern holds implications for any task that relies on the extracted representations.

What carries the argument

Adversarial semantic stress tests that generate sentence pairs with controlled lexical overlap but differing meaning to quantify surface-form versus semantic contributions layer by layer.

What would settle it

A new set of stress tests or model runs showing that semantic similarity scores remain high in the presence of lexical overlap at all depths would contradict the persistence finding.

Watch

Extended reading notes

Core claim

Lexical influence extends across the depth of models, consistently across architectures, training regimes, and objective functions, including the models trained for semantic similarity. Moreover, we observe a mid-depth region in which both lexical and semantic signals degrade simultaneously, indicating a transitional regime where representations are poor for both surface form and meaning.

Load-bearing premise

The stress tests isolate lexical overlap from semantic content without introducing other differences that change how the models process the sentences.

Editorial extensions

If this is right

Lexical effects appear in downstream applications such as summarization and model editing.
The mid-depth transitional regime produces representations that are weak for both surface and meaning tasks.
The pattern appears regardless of whether the model was trained with next-token prediction or semantic similarity objectives.
Architectural differences do not remove the lexical dominance across layers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Applications that pool or compare representations at mid-depths may need extra steps to compensate for the simultaneous drop in both signals.
The same layer-wise measurement could be applied to other sequence models to check whether the transitional regime is general.
If lexical influence is this stable, techniques that aim to remove surface-form bias may need to target every layer rather than only early ones.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The main point to take away is that lexical overlap keeps shaping representations even in deeper layers and even in models trained specifically for semantic similarity. The authors also report a mid-depth zone where both lexical and semantic signals drop together.

What the work does is run the same measurements across several architectures, training regimes, and objectives, then link the patterns to an information-theoretic view. The summarization and model-editing case studies give a concrete sense of downstream impact. That breadth is the useful part.

The soft spot is the adversarial semantic stress tests. The abstract gives no information on how sentence pairs were built, what was matched for length or syntax, or how other surface confounds were ruled out. If those pairs differ systematically on anything besides the intended lexical-semantic contrast, the observed degradation could be an artifact. The stress-test note flags exactly this, and it matters because the persistence and transitional-regime claims depend on the tests being clean.

This paper is aimed at people who extract and use LLM representations for semantic tasks and want to know where surface form still leaks through. A reader already working on embedding diagnostics would find the depth-wise results worth looking at.

It should go to peer review once the full methods and quantitative results are checked; the question is practical and the scope is reasonable, even if the current write-up leaves the key controls underspecified.

Referee Report

1 major / 1 minor

Summary. The manuscript investigates the persistent effects of lexical overlap versus semantic content in representations extracted from LLMs. Using several adversarial semantic stress tests, it claims that lexical influence extends across all model depths consistently across architectures, training regimes, and objective functions (including semantic-similarity models). It identifies a mid-depth transitional regime where both lexical and semantic signals degrade simultaneously, connects the observations to an information-theoretic perspective, and demonstrates downstream effects via case studies on summarization and model editing.

Significance. If the stress tests validly isolate lexical overlap from semantic content without confounds, the results would be significant for the field: they would demonstrate that lexical effects are difficult to eliminate even in models explicitly trained for semantic similarity and would identify a specific transitional depth where representations are weak for both surface and meaning. This has direct implications for layer selection in downstream applications and for the reliability of LLM embeddings in semantic tasks. The information-theoretic framing and case studies on editing/summarization add practical and theoretical value.

major comments (1)

[adversarial semantic stress tests / experimental setup] The section describing the adversarial semantic stress tests (and associated experimental setup): the central claims—that lexical influence persists across depths and regimes and that a distinct mid-depth transitional regime exists—rest on the tests successfully holding lexical overlap constant while varying semantic content (or vice versa) without correlated differences in length, syntactic complexity, negation, or entity salience. The provided abstract contains no description of pair construction, matching criteria, or controls, and any such details in the methods must be evaluated for rigor (e.g., explicit matching statistics or human validation). Without this, observed degradations could reflect construction artifacts rather than a genuine lexical-vs-semantic dissociation in the representations.

minor comments (1)

[Abstract] Abstract: states directional findings but reports no quantitative effect sizes, error bars, or statistical significance, which reduces the ability to gauge the practical magnitude of the lexical effects or the mid-depth degradation.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the detailed and constructive review. We address the single major comment on the adversarial semantic stress tests below, providing clarification on the experimental controls described in the manuscript and committing to expanded documentation in revision.

read point-by-point responses

Referee: [adversarial semantic stress tests / experimental setup] The section describing the adversarial semantic stress tests (and associated experimental setup): the central claims—that lexical influence persists across depths and regimes and that a distinct mid-depth transitional regime exists—rest on the tests successfully holding lexical overlap constant while varying semantic content (or vice versa) without correlated differences in length, syntactic complexity, negation, or entity salience. The provided abstract contains no description of pair construction, matching criteria, or controls, and any such details in the methods must be evaluated for rigor (e.g., explicit matching statistics or human validation). Without this, observed degradations could reflect construction artifacts rather than a genuine lexical-vs-semantic dissociation in the representations.

Authors: We agree the abstract omits these methodological specifics. Section 3 of the manuscript describes the adversarial pair construction: lexical-overlap pairs were generated via minimal-edit templates preserving length (within 3 tokens) and syntactic structure while altering semantics through synonym substitution or entity replacement; semantic-variation pairs used paraphrases with controlled lexical overlap below 20%. We explicitly avoided negations and matched entity salience via frequency and type. To strengthen rigor, we will add (i) quantitative matching statistics (mean length difference, syntactic tree-edit distance, negation count) and (ii) human validation results (inter-annotator agreement on semantic dissimilarity) to the methods section in the revised version. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical measurements with no self-referential derivations or fitted predictions

full rationale

The paper presents an empirical investigation using adversarial semantic stress tests to measure lexical vs. semantic influence in LLM representations across layers. No equations, parameters fitted to subsets then renamed as predictions, or self-citations invoked as uniqueness theorems appear in the provided text. The central claims rest on experimental observations rather than any derivation that reduces to its own inputs by construction. The method description in the abstract does not exhibit self-definition or smuggling of ansatzes. This is a standard empirical study whose validity hinges on experimental controls (addressable via replication or critique of test construction), not on circular logic in any claimed derivation chain.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Empirical study of existing models with no new mathematical axioms, free parameters, or invented entities; relies on standard assumptions about representation extraction and adversarial test validity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the Persistent Effects of Lexicality in Large Language Models." pith.science (2026). https://pith.science/paper/3JYWLKTW

@misc{pith2026260602750,
  author       = {Pith},
  title        = {Pith review of: On the Persistent Effects of Lexicality in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3JYWLKTW}},
  note         = {Machine review of arXiv:2606.02750}
}

read the original abstract

Representations extracted from large language models (LLMs) play an important role in many downstream applications. However, the structure of these representations is often influenced by lexical overlap rather than semantic content. Our understanding of the relationship between this lexical influence and semantic content, and its implications for downstream tasks, remains limited. In this work, we investigate representations to quantify the effect of lexical overlap relative to semantic content. We consider several adversarial semantic stress tests and further connect our findings to the information theory perspective. We find that lexical influence extends across the depth of models, consistently across architectures, training regimes, and objective functions, including the models trained for semantic similarity. Moreover, we observe a mid-depth region in which both lexical and semantic signals degrade simultaneously, indicating a transitional regime where representations are poor for both surface form and meaning. We further demonstrate the effect of lexical influence on downstream uses of LLMs using summarization and model editing as a case study.

Figures

Figures reproduced from arXiv: 2606.02750 by the authors.

**Figure 2.** Layer-wise probe performance for lexical decodability on WikiText across model depths [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** MTEB task performance across the full model depth for all evaluated models. For each [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Results for SCPP average token embeddings [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

**Figure 5.** Figure 5: Results for SCPP last token embeddings 0% 20% 40% 60% 80% 100% Relative layer depth (% of model) 0.55 0.60 0.65 0.70 0.75 0.80 0.85 0.90 Success Rate Counterfact (Average Token Embedding) Inst Rept Query Paper 0% 20% 40% 60% 80% 100% Relative layer depth (% of model) 0…

**Figure 6.** Figure 6: Results of prompt variations on the lexical influence test using the Gemma-12B-IT model [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

**Figure 7.** Figure 7: Results for WSD on Gemma-12B-IT and Gemma-12B-PT model. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]

**Figure 8.** Figure 8: Performance of the models on MTEB tasks. sts (a), pairwise classification (b), and [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 2 canonical work pages

[1]

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

PMLR, 2019. Vedang Lad, Jin Hwa Lee, Wes Gurnee, and Max Tegmark. Remarkable robustness of llms: Stages of inference?Advances in Neural Information Processing Systems, 38:130050–130083, 2026. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequen...

work page Pith review arXiv 2019
[2]

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

doi: 10.18653/V1/2023.EACL-MAIN.148. URL https://doi.org/10.18653/v1/2023. eacl-main.148. Shashi Narayan, Shay B Cohen, and Mirella Lapata. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797–18...

work page Pith review arXiv doi:10.18653/v1/2023.eacl-main.148 2023
[3]

- Do not i n t r o d u c e new t e r m i n o l o g y

Lexical Overlap : - Reuse d i s t i n c t i v e words , phrases , and named en ti tie s from the source . - Do not i n t r o d u c e new t e r m i n o l o g y . - Try to use all words from the source
[4]

- Do NOT p re ser ve relations , causality , chronology , or a r g u m e n t a t i o n

Sem an ti c O r t h o g o n a l i t y : - Do NOT describe , summarize , or restate any part of the source ’s meaning . - Do NOT p re ser ve relations , causality , chronology , or a r g u m e n t a t i o n . - The text must be about s o m e t h i n g else entirely , despite using the same words
[5]

l e x i c a l _ n o n _ s u m m a r y

Length : - A p p r o x i m a t e l y { t a r g e t _ l e n g t h } c h a r a c t e r s . V A L I D A T I O N CHECK ( int er na l ) : If the output allows a reader to ide nt if y the subject or purpose of the source document , it fails . OUTPUT FORMAT : Return a JSON object with exactly one field : {{ " l e x i c a l _ n o n _ s u m m a r y " : " < text > ...

2021
[6]

Ask for the exact same fact as o r i g i n a l _ l o c a l i t y _ p r o m p t : same entity name ( s ) + same a t t r i b u t e / rel at io n
[7]

Do not replace en ti tie s with d e s c r i p t i o n s

Copy entity name ( s ) EXACTLY as in o r i g i n a l _ l o c a l i t y _ p r o m p t ( same s pe ll in g / casing ) . Do not replace en ti tie s with d e s c r i p t i o n s
[8]

capital

Do NOT add / remove q u a l i f i e r s ( time , certainty , of fi cia l / current , " capital " , " city of " , etc .) . No hints , no a s s u m p t i o n s . Cloze f o r m a t t i n g ( each variant ) :

Show all 20 references

[9]

Each prompt must be a PREFIX such that the correct answer should be the next text g e n e r a t e d i m m e d i a t e l y after the final space
[10]

, ? , ! , : , ; , ,)

End with exactly ONE t rai li ng ASCII space and NO tra il in g p u n c t u a t i o n ( no . , ? , ! , : , ; , ,)
[11]

Do NOT end with the entity name ( s )
[12]

Use d e c l a r a t i v e stems only

No i n t e r r o g a t i v e s : do NOT use WH - words ( what / which / where / who / when / how ) . Use d e c l a r a t i v e stems only
[13]

Anti - copy / ma xi mi ze lexical d i f f e r e n c e from e d i t _ p r o m p t ( each variant ) :

Val id it y check : a p p e n d i n g the correct answer i m m e d i a t e l y after the final space must yield a co mp le te g r a m m a t i c a l result without adding any extra words . Anti - copy / ma xi mi ze lexical d i f f e r e n c e from e d i t _ p r o m p t ( each v...
[14]

Hard ban : Do NOT reuse any c o n t i g u o u s 2+ word se qu en ce found in e d i t _ p r o m p t ( e x c e p t i o n : entity tokens )
[15]

Content - word taboo : Avoid reusing ANY non - entity content word from e d i t _ p r o m p t ( verbs / nouns / a d j e c t i v e s / adverbs ) . - Allowed function - word set ( may repeat ) : { the , a , an , of , to , in , on , at , from , for , with , by , is , are , was , ...
[16]

< ENTITY > < connector >

Target : mi ni mi ze overlap of non - entity content words with e d i t _ p r o m p t to 0 wh en ev er p os sib le . M A N D A T O R Y s t r u c t u r a l d i v e r s i t y ( across the 3 va ria nt s ) : Ge ne ra te exactly one from each frame : A ) ENTITY - SUBJECT frame : 22...
[17]

No shared opening 3 tokens across va ri an ts
[18]

Use three d i f f e r e n t c o n n e c t o r tails ( the last 1 to 4 words before the final space must differ across v ari an ts )
[19]

Prefer d i f f e r e n t synonym sets

Across the three outputs , do not reuse the same key content words ( besides the entity and u n a v o i d a b l e a t t r i b u t e noun ) . Prefer d i f f e r e n t synonym sets . Search - and - select r e q u i r e m e n t ( i m p o r t a n t ) :
[20]

o r i g i n a l _ l o c a l i t y _ p r o m p t

For EACH frame ( A / B / C ) , i n t e r n a l l y draft at least 5 c a n d i d a t e rewrites , then choose the best one that : - s a t i s f i e s all constraints , - has the fewest non - entity content words o v e r l a p p i n g with edit_prompt , - and differs most from t...

Pith tools

Reviewed June 28, 2026 · model on record in the stance chip above.

[1] [1]

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

PMLR, 2019. Vedang Lad, Jin Hwa Lee, Wes Gurnee, and Max Tegmark. Remarkable robustness of llms: Stages of inference?Advances in Neural Information Processing Systems, 38:130050–130083, 2026. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequen...

work page Pith review arXiv 2019

[2] [2]

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

doi: 10.18653/V1/2023.EACL-MAIN.148. URL https://doi.org/10.18653/v1/2023. eacl-main.148. Shashi Narayan, Shay B Cohen, and Mirella Lapata. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797–18...

work page Pith review arXiv doi:10.18653/v1/2023.eacl-main.148 2023

[3] [3]

- Do not i n t r o d u c e new t e r m i n o l o g y

Lexical Overlap : - Reuse d i s t i n c t i v e words , phrases , and named en ti tie s from the source . - Do not i n t r o d u c e new t e r m i n o l o g y . - Try to use all words from the source

[4] [4]

- Do NOT p re ser ve relations , causality , chronology , or a r g u m e n t a t i o n

Sem an ti c O r t h o g o n a l i t y : - Do NOT describe , summarize , or restate any part of the source ’s meaning . - Do NOT p re ser ve relations , causality , chronology , or a r g u m e n t a t i o n . - The text must be about s o m e t h i n g else entirely , despite using the same words

[5] [5]

l e x i c a l _ n o n _ s u m m a r y

Length : - A p p r o x i m a t e l y { t a r g e t _ l e n g t h } c h a r a c t e r s . V A L I D A T I O N CHECK ( int er na l ) : If the output allows a reader to ide nt if y the subject or purpose of the source document , it fails . OUTPUT FORMAT : Return a JSON object with exactly one field : {{ " l e x i c a l _ n o n _ s u m m a r y " : " < text > ...

2021

[6] [6]

Ask for the exact same fact as o r i g i n a l _ l o c a l i t y _ p r o m p t : same entity name ( s ) + same a t t r i b u t e / rel at io n

[7] [7]

Do not replace en ti tie s with d e s c r i p t i o n s

Copy entity name ( s ) EXACTLY as in o r i g i n a l _ l o c a l i t y _ p r o m p t ( same s pe ll in g / casing ) . Do not replace en ti tie s with d e s c r i p t i o n s

[8] [8]

capital

Do NOT add / remove q u a l i f i e r s ( time , certainty , of fi cia l / current , " capital " , " city of " , etc .) . No hints , no a s s u m p t i o n s . Cloze f o r m a t t i n g ( each variant ) :