pith. sign in

arxiv: 2408.04596 · v2 · submitted 2024-08-08 · 💻 cs.CL

Code-switching in text and speech challenges information-theoretic speaker design

Debasmita Bhattacharya , Marten van Schijndel This is my paper

Pith reviewed 2026-05-23 21:58 UTC · model grok-4.3

classification 💻 cs.CL
keywords code-switchinglanguage modelingpredictabilitybilingual speechChinese-Englishinformation theoryspeaker design
0
0 comments X

The pith

Code-switching to English occurs even when English is less predictable than equivalent Chinese phrases, rejecting purely speaker-driven accounts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines whether insertional code-switching is driven only by the need to make speech or writing easier for the bilingual speaker. It replicates the finding that switches from Chinese to English tend to occur where Chinese text or speech is hard to predict. However, when the actual English insertions are compared to what a Chinese continuation with the same meaning would have been, the English turns out to be even harder to predict. This pattern appears in both online forum writing and spontaneous speech transcripts, implying that speakers are not switching languages merely to lower their own production cost.

Core claim

Low predictability in the primary language (Chinese) correlates with switches to the secondary language (English), yet the English material actually produced has lower predictability than meaning-equivalent Chinese alternatives; therefore the switches do not reduce production difficulty and the purely speaker-driven account of code-switching is rejected for both written and spoken data.

What carries the argument

Language-model perplexity on primary-language text, used to quantify predictability at potential switch points and to compare the actual secondary-language insertion against a meaning-matched primary-language continuation.

If this is right

  • Code-switching in both writing and speech serves communicative goals beyond reducing speaker effort.
  • Speakers may insert the secondary language to direct listener attention rather than to simplify their own output.
  • The same rejection of a purely speaker-driven account holds for both online forum posts and spontaneous speech transcripts.
  • Information-theoretic models of speaker design must incorporate listener-oriented or social functions to explain observed switching patterns.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar comparisons could be run on other language pairs to test whether the pattern is specific to Chinese-English bilinguals.
  • Listener comprehension studies could check whether the harder-to-predict English insertions actually improve understanding or signal emphasis.
  • The results leave open the possibility that code-switching frequency changes with audience expectations or conversational setting.

Load-bearing premise

Language-model perplexity on the primary language serves as a valid stand-in for the real-time production difficulty a bilingual speaker faces when choosing which language to use next.

What would settle it

A dataset of Chinese-English code-switches in which the English insertions show higher predictability than their meaning-equivalent Chinese alternatives would contradict the central observation.

Figures

Figures reproduced from arXiv: 2408.04596 by Debasmita Bhattacharya, Marten van Schijndel.

Figure 1
Figure 1. Figure 1: Schematic illustrating the shorthand terminology used throughout the remainder [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparing surprisal of CS1 words in code-switched and non-code-switched [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparing CS1 in English and monolingual (ML) English across (a) word length, [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comparing normalized CS1 surprisal in English to Chinese in writing. [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparing CS1 in English and monolingual (ML) English across (a) word length, [PITH_FULL_IMAGE:figures/full_fig_p022_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparing normalized CS1 surprisal in English to Chinese in speech. [PITH_FULL_IMAGE:figures/full_fig_p023_6.png] view at source ↗
read the original abstract

In this work, we use language modeling to investigate the factors that influence insertional code-switching. Code-switching occurs when a speaker alternates between one language variety (the primary language) and another (the secondary language), and is widely observed in multilingual contexts. Recent work has shown that code-switching is often correlated with areas of low predictability in the primary language, but it is unclear whether low primary language predictability only makes the secondary language relatively easier to produce at code-switching points - that is, purely speaker-driven code-switching - or whether code-switching is additionally used by speakers for other purposes, for instance to signal the need for greater attention on the part of listeners. In this paper, we use bilingual Chinese-English online forum posts and transcripts of spontaneous Chinese-English speech to replicate prior findings that low primary language (Chinese) predictability is correlated with insertional switches to the secondary language (English). We then demonstrate that the predictability of the English productions is even lower than that of meaning-equivalent Chinese alternatives, and these are therefore not easier to produce, rejecting the purely speaker-driven theory of code-switching in both writing and speech.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper uses language modeling on Chinese-English bilingual online forum posts and spontaneous speech transcripts to replicate that low primary-language (Chinese) predictability correlates with insertional switches to English. It then claims that the English productions exhibit even lower predictability than meaning-equivalent Chinese alternatives and are therefore not easier to produce, rejecting a purely speaker-driven information-theoretic account of code-switching in both text and speech.

Significance. If the cross-lingual predictability comparison holds after proper calibration, the result would meaningfully challenge speaker-centric accounts by showing that switches do not occur at points of relative production ease. The replication of the primary-language predictability correlation plus the use of meaning-equivalent alternatives across both written and spoken modalities constitute a clear empirical contribution via new corpus measurements.

major comments (2)
  1. [Abstract and Results] Abstract and central results: the directional claim that English productions have lower predictability than meaning-equivalent Chinese alternatives (and are therefore not easier) rests on direct comparison of perplexities from separate monolingual LMs. No bits-per-character normalization, joint multilingual modeling, or other calibration for differing baseline entropies and tokenization is described, so the inequality does not follow from the reported numbers. This comparison is load-bearing for the rejection of the speaker-driven theory.
  2. [Methods] Methods and analysis sections: details on LM training (architecture, data, tokenization), the procedure for constructing and aligning meaning-equivalent Chinese alternatives, and any statistical controls or significance tests on the perplexity differences are not supplied. Without these, the robustness of the key negative result cannot be evaluated.
minor comments (2)
  1. [Abstract] The abstract and introduction could more explicitly flag the cross-lingual comparability issue and how it is addressed (or why separate monolingual perplexities suffice).
  2. Notation for predictability/perplexity should be defined once and used consistently when switching between Chinese and English contexts.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and constructive report. The two major comments identify important gaps in the presentation of our cross-lingual predictability comparison and in the methodological documentation. We address each point below and commit to a major revision that supplies the missing details and strengthens the empirical claim.

read point-by-point responses

  1. Referee: [Abstract and Results] Abstract and central results: the directional claim that English productions have lower predictability than meaning-equivalent Chinese alternatives (and are therefore not easier) rests on direct comparison of perplexities from separate monolingual LMs. No bits-per-character normalization, joint multilingual modeling, or other calibration for differing baseline entropies and tokenization is described, so the inequality does not follow from the reported numbers. This comparison is load-bearing for the rejection of the speaker-driven theory.

    Authors: We agree that the manuscript does not describe any cross-lingual calibration and that this omission weakens the central negative result. In the revision we will (1) report bits-per-character normalized perplexities for both languages, (2) train and evaluate a single multilingual model on the combined data to provide a joint baseline, and (3) include an explicit discussion of why the directional inequality survives these controls. If the calibrated comparison no longer supports the claim, we will qualify or retract the rejection of the purely speaker-driven account. We therefore treat this as a required methodological correction rather than a minor clarification. revision: yes

  2. Referee: [Methods] Methods and analysis sections: details on LM training (architecture, data, tokenization), the procedure for constructing and aligning meaning-equivalent Chinese alternatives, and any statistical controls or significance tests on the perplexity differences are not supplied. Without these, the robustness of the key negative result cannot be evaluated.

    Authors: We acknowledge that the current Methods section is insufficiently detailed. The revised manuscript will add: (a) full LM specifications (architecture, training corpora, tokenizers, and hyper-parameters), (b) the exact procedure used to generate and align meaning-equivalent Chinese alternatives (including any translation or parallel-corpus resources), and (c) statistical tests (paired Wilcoxon signed-rank tests with effect sizes and confidence intervals) on the perplexity differences. These additions will make the negative result reproducible and allow readers to assess its robustness directly. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical perplexity measurements on new corpora are independent of inputs

full rationale

The paper's chain consists of (1) replicating a correlation between low Chinese predictability and English inserts using language-model perplexity on bilingual forum posts and speech transcripts, then (2) comparing perplexity of observed English tokens against meaning-equivalent Chinese alternatives. Both steps are direct computations on held-out data; no parameter is fitted to a subset and then relabeled as a prediction, no equation reduces to its own definition, and no load-bearing premise rests on a self-citation whose content is itself unverified. The central rejection of speaker-driven code-switching therefore follows from external measurements rather than from any self-referential construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim depends on treating LM perplexity as a proxy for speaker production cost and on the validity of meaning-equivalent Chinese alternatives; no free parameters or invented entities are described.

axioms (1)
  • domain assumption Language-model perplexity on the primary language accurately reflects the production difficulty experienced by bilingual speakers.
    Used to interpret low Chinese predictability at switch points and to compare English vs. Chinese alternatives.

pith-pipeline@v0.9.0 · 5725 in / 1105 out tokens · 18334 ms · 2026-05-23T21:58:11.436558+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in ":" * " " * FUNCTION f...

    work page

  2. [2]

    , author Marcotte, K

    author Ansaldo, A.I. , author Marcotte, K. , author Scherer, L. , author Raboyeau, G. , year 2008 . title Language therapy and bilingual aphasia: Clinical implications of psycholinguistic and neuroimaging research . journal Journal of Neurolinguistics volume 21 , pages 539--557 . https://www.sciencedirect.com/science/article/pii/S0911604408000146, :https:...

    work page doi:10.1016/j.jneuroling.2008.02.001 2008

  3. [3]

    , author Navarro-Torres, C.A

    author Beatty-Martínez, A.L. , author Navarro-Torres, C.A. , author Dussias, P.E. , year 2020 . title Codeswitching: A bilingual toolkit for opportunistic speech planning . journal Frontiers in Psychology volume 11 . https://www.frontiersin.org/article/10.3389/fpsyg.2020.01699, :10.3389/fpsyg.2020.01699

    work page doi:10.3389/fpsyg.2020.01699 2020

  4. [4]

    , year 1984

    author Bell, A. , year 1984 . title Language style as audience design . journal Language in Society volume 13 , pages 145--204 . http://www.jstor.org/stable/4167516

    work page arXiv 1984

  5. [5]

    , year 2009

    author Broersma, M. , year 2009 . title Triggered codeswitching between cognate languages . journal Bilingualism: Language and Cognition volume 12 , pages 447--462 . :10.1017/S1366728909990204

    work page doi:10.1017/s1366728909990204 2009

  6. [6]

    , author Fang, L

    author Calvillo, J. , author Fang, L. , author Cole, J. , author Reitter, D. , year 2020 . title Surprisal predicts code-switching in C hinese- E nglish bilingual text , in: booktitle Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , publisher Association for Computational Linguistics , address Online . pp. p...

    work page doi:10.18653/v1/2020.emnlp-main.330 2020

  7. [7]

    , author Graff, D

    author Canavan, A. , author Graff, D. , author Zipperlen, G. , year 1997 . title CALLHOME A merican E nglish S peech

    work page 1997

  8. [8]

    , year 2018

    author Consortium, L.D. , year 2018 . title HUB 5 M andarin T elephone S peech and T ranscripts S econd E dition LDC 2018 S 18

    work page 2018

  9. [9]

    , year 2009

    author Dahl, K.L. , year 2009 . title Audience design and code-switching in Bayside, Texas . Ph.D. thesis. The University of Texas at Austin

    work page 2009

  10. [10]

    , author Devescovi, A

    author D'Amico, S. , author Devescovi, A. , author Bates, E. , year 2001 . title Picture naming and lexical access in italian children and adults . journal Journal of Cognition and Development - J COGN DEV volume 2 . :10.1207/S15327647JCD0201_4

    work page doi:10.1207/s15327647jcd0201_4 2001

  11. [11]

    , year 1978

    author Dornic, S. , year 1978 . title The Bilingual's Performance: Language Dominance, Stress, and Individual Differences . publisher Springer US , address Boston, MA . chapter chapter null . pp. pages 259--271 . https://doi.org/10.1007/978-1-4615-9077-4_23, :10.1007/978-1-4615-9077-4_23

    work page doi:10.1007/978-1-4615-9077-4_23 1978

  12. [12]

    , author Chambers, S.M

    author Forster, K.I. , author Chambers, S.M. , year 1973 . title Lexical access and naming time . journal Journal of Verbal Learning and Verbal Behavior volume 12 , pages 627--635 . https://www.sciencedirect.com/science/article/pii/S0022537173800428, :https://doi.org/10.1016/S0022-5371(73)80042-8

    work page doi:10.1016/s0022-5371(73)80042-8 1973

  13. [13]

    , year 2009

    author Gardner-Chloros, P. , year 2009 . title Code-switching . publisher Cambridge University Press

    work page 2009

  14. [14]

    , year 2019

    author Hassan, A.A. , year 2019 . title English- A rabic C ode- S witching of the A rabic L anguage S peakers in I nstant M essaging: M otivations and S tructure . journal Cairo Studies in English volume 2019 , pages 39--59 . :10.21608/cse.2019.66656

    work page doi:10.21608/cse.2019.66656 2019

  15. [15]

    , author Marinis, T

    author Hofweber, J. , author Marinis, T. , year 2023 . title What sentence repetition tasks can reveal about the processing effort associated with different types of code-switching . journal Languages volume 8 . https://www.mdpi.com/2226-471X/8/1/70, :10.3390/languages8010070

    work page doi:10.3390/languages8010070 2023

  16. [16]

    , author Carlsson, F

    author Isbister, T. , author Carlsson, F. , author Sahlgren, M. , year 2021 . title Should we stop training more monolingual models, and simply use machine translation instead? , in: booktitle Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa) , publisher Link \"o ping University Electronic Press, Sweden , address Reykjavik,...

    work page 2021

  17. [17]

    , year 1982

    author Joshi, A.K. , year 1982 . title Processing of sentences with intra-sentential code-switching , in: booktitle C oling 1982: Proceedings of the N inth I nternational C onference on C omputational L inguistics , p. pages null . https://aclanthology.org/C82-1023

    work page 1982

  18. [18]

    , author Liu, Z

    author Liu, H. , author Liu, Z. , author Yuan, M. , author Chen, T. , year 2023 . title The effect of cognitive load on code-switching . journal International Journal of Bilingualism volume 0 , pages 13670069231170142 . https://doi.org/10.1177/13670069231170142, :10.1177/13670069231170142, http://arxiv.org/abs/https://doi.org/10.1177/13670069231170142 arX...

    work page doi:10.1177/13670069231170142 2023

  19. [19]

    , year 2021

    author Liu, K. , year 2021 . title Language choices as audience design strategies in C hinese multilingual speakers’ W echat posts . journal Global Media and China volume 6 , pages 391--415 . https://doi.org/10.1177/20594364211035201, :10.1177/20594364211035201

    work page doi:10.1177/20594364211035201 2021

  20. [20]

    , author Tan, T.P

    author Lyu, D.C. , author Tan, T.P. , author Chng, E. , author Li, H. , year 2010 . title Mandarin– E nglish code-switching speech corpus in S outh- E ast A sia: SEAME , in: booktitle Language Resources and Evaluation , pp. pages 1986--1989 . :10.1007/s10579-015-9303-x

    work page doi:10.1007/s10579-015-9303-x 2010

  21. [21]

    , year 1994

    author Meisel, J.M. , year 1994 . title Code-switching in young bilingual children: The acquisition of grammatical constraints . journal Studies in Second Language Acquisition volume 16 , pages 413--439 . http://www.jstor.org/stable/44487780

    work page arXiv 1994

  22. [22]

    , year 2022

    author Misra, K. , year 2022 . title minicons: Enabling flexible behavioral and representational analyses of transformer language models . journal arXiv preprint arXiv:2203.13112

    work page arXiv 2022

  23. [23]

    , year 2000

    author Muysken, P. , year 2000 . title Bilingual speech: a typology of code-mixing . publisher Cambridge University Press

    work page 2000

  24. [24]

    , year 1993

    author Myers-Scotton, C. , year 1993 . title Duelling Languages . publisher Oxford: Clarendon Press

    work page 1993

  25. [25]

    , author Levy, R.P

    author Mysl \'i n, M. , author Levy, R.P. , year 2015 . title Codeswitching and predictability of meaning in discourse . journal Language volume 91 , pages 871 -- 905

    work page 2015

  26. [26]

    , year 2005

    author Owens, J. , year 2005 . title Bare forms and lexical insertions in code-switching: A processing-based account . journal Bilingualism: Language and Cognition volume 8 , pages 23--38

    work page 2005

  27. [27]

    , year 1980

    author Poplack, S. , year 1980 . title Sometimes I 'll start a sentence in S panish Y TERMINO EN ESPAÑOL : toward a typology of code-switching . journal Linguistics volume 18 , pages 581--618 . :10.1515/ling.1980.18.7-8.581

    work page doi:10.1515/ling.1980.18.7-8.581 1980

  28. [28]

    , author Wu, J

    author Radford, A. , author Wu, J. , author Child, R. , author Luan, D. , author Amodei, D. , author Sutskever, I. , year 2019 . title Language models are unsupervised multitask learners . journal null

    work page 2019