Beyond Frequency: The Role of Redundancy in Large Language Model Memorization

Chi-ho Lin; Jie Zhang; Lei Li; Qinghua Zhao; Zhongfeng Kang

REVIEW 3 major objections 5 minor 13 references

Beyond Frequency: The Role of Redundancy in Large Language Model Memorization

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Information redundancy, not just token frequency, shapes what large language models memorize: about 79 percent of memorized samples are low-redundancy and roughly twice as fragile to perturbation.

desk verdict A real differential observation about memorized versus non-memorized samples, but the 'beyond frequency' claim collapses because redundancy is unigram entropy and frequency is never jointly controlled. read the letter →

arxiv 2506.12321 v2 pith:ZJU6JFDO submitted 2025-06-14 cs.LG cs.AI

Jie Zhang , Qinghua Zhao , Chi-ho Lin , Zhongfeng Kang , Lei Li This is my paper

classification cs.LGcs.AI

keywords largelanguagemodelsmemorizationinformationredundancyunigramentropyprefixperturbationPythiaprivacyrisk

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that information redundancy—how much of a text can be predicted from context—governs what large language models memorize, beyond the previously studied factors of token frequency and repetition. The authors report that about 79% of memorized samples are low-redundancy, that these low-redundancy samples lose memorization scores at about twice the rate of high-redundancy ones under the same prefix perturbation, and that memorized samples drop by 0.6 while non-memorized samples drop by only 0.01. If true, the finding turns redundancy into a practical lever: curating or perturbing training text by its redundancy could reduce privacy leakage and memorization-driven bias, and it would explain why frequency manipulations move memorized and non-memorized samples so differently.

What carries the argument

The paper's load-bearing tool is an entropy-based redundancy measure, $Re(s) = 1 - \frac{1}{\log_2 |V|} \sum_{v\in V} p(v) \log_2 p(v)$, a unigram predictability score where lower values mean most tokens are unpredictable and hence 'low redundancy.' Around it, the paper builds (i) four formal perturbation operators—insertion, deletion, replacement, and shuffling—and a unified magnitude $\mathcal{P} = \alpha P + (1-\alpha) R$ combining absolute token displacement with relative-order disruption, and (ii) an n-gram overlap memorization score $M_n(O,G)$ that classifies outputs as memorized above threshold $\theta = 0.5$. The redundancy score assigns samples to high/low groups; the perturbation magnitude makes different error types comparable for measuring vulnerability; the memorization score defines the outcome.

What would settle it

A matched-pair or joint-regression study that varies the entropy-based redundancy score while holding average token frequency fixed: if within frequency bins the redundancy measure shows no residual association with memorization status or perturbation-induced score drops, the 'beyond frequency' conclusion fails.

Watch

Extended reading notes

Core claim

The central claim is that models preferentially memorize low-redundancy content, and that this content is disproportionately fragile. Testing the eight Pythia model scales (70M to 12B) on extractable sequences from The Pile, the paper finds that measured by unigram entropy, roughly 79% of memorized samples are low-redundancy across thresholds and scales (with χ² test, p<0.001), and that under identical insertion, deletion, replacement, or shuffling perturbations, low-redundancy samples show a 2-fold higher memorization drop than high-redundancy samples. The paper concludes that information redundancy is a deep-level factor underlying memorization, with conventional surface-level characteristics like frequency acting as symptoms rather than primary drivers, and that models prefer low-redundancy content first, gradually memorizing higher-redundancy content as scale and training progress.

Load-bearing premise

The load-bearing premise is that the unigram-entropy score in Eq. (1) measures information redundancy and is distinct from token frequency and rarity: the paper never tests redundancy against frequency in the same model, so if low redundancy is mainly a proxy for low frequency, the central conclusion does not follow.

Editorial extensions

If this is right

Redundancy can be used as a preprocessing signal during data curation: filtering or augmenting low-redundancy text before training should reduce memorization and the privacy risks that come with it.
Perturbation-vulnerability scoring can be applied to any model family, because it reduces four different error types to one comparable strength measure rather than treating typos, deletions, and shuffles as incommensurable.
Stratifying memorization analyses by memorized versus non-memorized status exposes effects that aggregate correlations hide, so future studies should report subgroup patterns alongside pooled results.
Because the low-redundancy preference appears early in training and weakens only gradually with scale, redundancy-based defenses would need to act on the corpus rather than on model size or training duration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If redundancy is the operative variable, deduplication-based defenses may not address the risk: deduplication removes repeated, often high-redundancy text, whereas the memorized and fragile strings reported here are dense and unique.
A direct test of the 'beyond frequency' framing would re-run the same perturbation experiments on samples matched for token frequency but split into high/low entropy; the paper leaves that comparison unperformed.
The redundancy score is unigram-based, so a bigram or contextual variant could plausibly reassign many samples; reporting how much the 79% figure moves under such a variant would sharpen the claim.
The unified perturbation magnitude contains a free weighting hyperparameter α; the reported drops may shift with α, so a sensitivity sweep would clarify whether the 2-fold ratio is robust.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

There is a real observation hiding in here, but the paper's headline claim, that redundancy explains memorization beyond frequency, is not supported by the experiments. The redundancy measure in Eq. (1) is simply 1 minus normalized unigram entropy, which is a direct function of the token frequency distribution. Low redundancy means high entropy, i.e., diverse or rare tokens. Since the paper never conditions on token frequency or repetition when comparing high- and low-redundancy groups, the 79% low-redundancy dominance and the 2-fold vulnerability are exactly what the existing frequency/rarity literature predicts. The 'beyond frequency' conclusion collapses.

What is actually new and worth taking seriously is the differential finding in Section 5: memorized samples respond far less to frequency and repetition changes than non-memorized samples across Pythia scales. That is a clean, potentially useful observation. The perturbation quantification framework, using positional displacement plus relative order across insertion, deletion, replacement, and shuffle, is a reasonable formalization, and the validation against output uncertainty (R^2 around 0.85) is solid. The paper is also transparent about using a 10k-sample subset and provides a pilot validation.

The soft spots are load-bearing and not minor. First, the construct validity of Eq. (1) as 'redundancy' is never established against frequency; a joint analysis or partial correlation is missing. Second, the abstract's final sentence says 'more redundant content becomes both more memorable and more fragile,' which is the opposite of what the paper's own data show, since the data say low-redundancy content is memorable and fragile. That is a direct internal contradiction. Third, the perturbation weighting alpha and the high/low redundancy split threshold are never reported, so the headline numbers (79%, 2-fold, 0.6 vs 0.01) are not reproducible as written. Fourth, Section 7.3 is a consistency check on the same data, not an independent prediction, even though the paper frames it as testable.

Who is this for? Researchers working on memorization factors might get value from the differential frequency/repetition analysis and from the perturbation method. But the paper would need substantial revision, including joint frequency and redundancy modeling, full reporting of alpha and threshold, fixing the abstract, and reframing Section 7.3, before the central claim is credible.

My recommendation: send it to peer review, but with a clear expectation of heavy revision. A good referee could salvage the differential observation and the perturbation method, and that would be worth having. As written, the central claim does not stand.

Referee Report

3 major / 5 minor

Summary. The paper studies verbatim memorization in Pythia LLMs (70M to 12B) using a dataset of 32-token extractable sequences. It first reports that token frequency and repetition affect memorized and non-memorized samples differently, then proposes a unified perturbation quantification based on positional changes, defines sequence redundancy via normalized unigram entropy (Eq. 1), and claims that roughly 79% of memorized samples are low-redundancy, that low-redundancy samples are about twice as vulnerable to prefix perturbations, and that this demonstrates a role for information redundancy beyond surface-level factors such as frequency. The paper closes with a proposed practical application to data preprocessing for privacy and bias mitigation.

Significance. The empirical setup is attractive: it spans eight Pythia scales and multiple training checkpoints, uses a public extraction test set, and formalizes four perturbation types in a unified framework. If the redundancy account were properly supported, it would add a useful dimension to memorization research and motivate redundancy-guided data curation. The paper also makes a fair methodical point that aggregate frequency/repetition correlations can hide opposite or thresholded behavior within memorized versus non-memorized subsets. However, the central contribution—redundancy as an explanatory factor beyond frequency—is not demonstrated, because the redundancy analysis never jointly controls for frequency, repetition, or perplexity. The abstract's final sentence also states a conclusion opposite to the paper's own reported findings. The manuscript does not mention releasing code or data, which limits reproducibility of the perturbation and redundancy computations.

major comments (3)

[Section 3.2, Eq. (1); Sections 7.1-7.2] The redundancy measure Re(s) in Eq. (1) is a deterministic transform of the empirical unigram distribution of the prefix, so "low redundancy" corresponds to high entropy, i.e., sequences with many distinct or rare tokens. The paper never conditions on token frequency, repetition count, or prefix perplexity when comparing high- and low-redundancy groups, even though those factors are established memorization drivers and are plotted in the paper's own Figure 3. The reported 79% low-redundancy dominance among memorized samples and the 2-fold vulnerability difference may therefore be entirely explained by rarity or repetition, which would collapse the title's "beyond frequency" claim. A concrete joint analysis, such as stratifying by frequency bins or fitting a model with both frequency and redundancy as predictors, is needed to support the central claim.
[Abstract and Section 7.2] The final sentence of the abstract says that "more redundant content becomes both more memorable and more fragile," which is the opposite of the paper's own results. Section 7.2 reports that about 79% of memorized samples are low-redundancy, and Section 7.1 reports that low-redundancy samples are the more vulnerable ones. Section 8 correctly concludes that models preferentially memorize low-redundancy content. This internal contradiction in the abstract's statement of the central conclusion must be fixed; as written, the paper's headline finding is not supported by its reported data.
[Section 7.3] The "testable prediction" in Section 7.3 is derived from the findings of Sections 7.1 and 7.2 and then tested on the same dataset and models that produced those findings. The resulting observation that memorized samples drop by 0.6 while non-memorized samples drop by 0.01 is therefore a consistency check, not an independent confirmation. To be a genuine prediction test, the authors should use held-out data, a different model family, or a pre-registered threshold; otherwise the prediction language overstates the evidential value of the result.

minor comments (5)

[Section 3.1] The unified perturbation magnitude and the absolute position change are both denoted by P, which is confusing; consider renaming one of them (e.g., U for the unified magnitude).
[Section 6] The text states that R2 values "over 85% of uncertainty variance" are explained, but the reported mean is 0.85 and the range is 0.82-0.91; the correct wording would be "on average, about 85%," not "over 85%."
[Section 3.2] The distribution p(v) in Eq. (1) is not explicitly defined; the authors should state that it is the empirical unigram distribution of the prefix and discuss the effect of the short 32-token prefix length on the stability of the entropy estimate.
[Section 7.2] The chi-square test is reported only as p < 0.001; the test statistic, degrees of freedom, and effect size should be given so the reader can assess the magnitude of the 79%/21% imbalance.
[Section 3.1] The phrase "control factor r" is misleading; r is a perturbation intensity ratio (the fraction of tokens modified), not a control variable in the statistical sense, and the terminology should be changed accordingly.

Circularity Check

1 steps flagged · score 6.0 of 10

Section 7.3's 'testable prediction' is a consistency check: the predicted pattern is assembled from the paper's own Sections 7.1 and 7.2 and confirmed on the same test set. The 'beyond frequency' framing is also untested because Eq. (1) is unigram entropy, never jointly controlled against token frequency.

fitted input called prediction [Section 7.3, 'Prediction of Low-Redundancy Memorization'; Figure 7]
"The convergent evidence from vulnerability analysis (Section 7.1) and distribution analysis (Section 7.2) enables us to formulate a testable prediction: if low-redundancy samples exhibit higher vulnerability (Section 7.1) and memorized samples contain 79% low-redundancy content (Section 7.2), then memorized samples as a whole should demonstrate higher perturbation sensitivity compared to non-memorized samples. We tested this prediction by directly comparing memorization drops between memorized and non-memorized groups under identical perturbations."

The 'prediction' is not an independent forecast from a mechanism; it is the logical conjunction of the paper's own in-sample findings. Section 7.2 already establishes that memorized content is 79% low-redundancy, and Section 7.1 already establishes that low-redundancy content has larger memorization drops. Combining these two facts trivially implies that memorized samples, as a group, will show larger drops. Figure 7 therefore confirms the arithmetic consequence of the authors' own partitions rather than testing a new, falsifiable prediction. No held-out data or separate model is used, so the claimed confirmation is forced by construction and does not provide independent support for the redundancy mechanism.

full rationale

There is no load-bearing self-citation chain: the redundancy measure is adopted from Fedorenko et al. (2024), an external source, and the perturbation experiments are self-contained measurements on Pythia models. The main empirical content (differential drops between high- and low-redundancy groups across scales) is real and does not reduce to a fitted parameter. However, the paper overstates its inferential status in Section 7.3: the 'testable prediction' is a restatement of Sections 7.1 and 7.2 tested on the same 10,000-sequence dataset, so its confirmation is a consistency check, not an independent test. Separately, the abstract's closing clause ('more redundant content becomes both more memorable and more fragile') is the opposite of the paper's own stated numbers (79% of memorized samples are low-redundancy; low-redundancy samples are the fragile ones). That is an internal contradiction rather than circularity, but it further undermines the conclusion as written. Finally, the title claim 'Beyond Frequency' is not established: Eq. (1) defines redundancy as one minus normalized unigram entropy, a deterministic function of the token-frequency distribution, and no analysis conditions on token frequency or rarity when comparing high- and low-redundancy groups. This is a confound concern, not a circularity, but it means the central 'beyond frequency' conclusion should be treated as unverified. Because one 'prediction' reduces by construction to prior in-sample results, a score of 6 is appropriate.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on two unevaluated modeling choices: the unigram-entropy definition of redundancy, and the comparability of perturbation types via r*T. The paper introduces no new entities, but it does introduce an unused hyperparameter alpha and an unreported group-split rule.

free parameters (2)

alpha (perturbation weighting)
The unified perturbation magnitude P = alpha*P_bar + (1-alpha)*R requires a weight alpha in [0,1]; the paper never reports the value used. If it was tuned to align perturbation types, it is a fitted free parameter.
high/low redundancy split threshold
Section 7.1 partitions samples into high and low redundancy groups but does not state the cutoff rule. The 79% and 2x numbers depend on this choice.

assumptions (3)

domain assumption Unigram entropy of the prefix as defined in Eq. (1) is a valid measure of information redundancy for memorization.
Section 3.2 adopts Fedorenko et al.'s redundancy measure without validating it against more sophisticated language-model-based predictability. The entire analysis inherits this premise.
domain assumption The memorization threshold theta=0.5, and the threshold sweep, cleanly separate memorized from non-memorized samples.
Section 4 sets theta following Carlini et al.; the paper's stratified analyses rely on this dichotomy.
ad hoc to paper The perturbation strength r*T is comparable across insertion, deletion, shuffle, and replacement without the alpha-weighted unified metric.
Section 6 validates this via uncertainty correlation, but the alpha-weighted metric defined in Section 3.1 is not used in the main memorization experiments, so the validation does not test the actual claimed quantification.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Beyond Frequency: The Role of Redundancy in Large Language Model Memorization." pith.science (2026). https://pith.science/paper/ZJU6JFDO

@misc{pith2026250612321,
  author       = {Pith},
  title        = {Pith review of: Beyond Frequency: The Role of Redundancy in Large Language Model Memorization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZJU6JFDO}},
  note         = {Machine review of arXiv:2506.12321}
}

read the original abstract

Memorization in large language models poses critical risks for privacy and fairness as these systems scale to billions of parameters. While previous studies established correlations between memorization and factors like token frequency and repetition patterns, we revealed distinct response patterns: frequency increases minimally impact memorized samples (e.g. 0.09) while substantially affecting non-memorized samples (e.g., 0.25), with consistency observed across model scales. Through counterfactual analysis by perturbing sample prefixes and quantifying perturbation strength through token positional changes, we demonstrate that redundancy correlates with memorization patterns. Our findings establish that: about 79% of memorized samples are low-redundancy, these low-redundancy samples exhibit 2-fold higher vulnerability than high-redundancy ones, and consequently memorized samples drop by 0.6 under perturbation while non-memorized samples drop by only 0.01, indicating that more redundant content becomes both more memorable and more fragile. These findings suggest potential redundancy-guided approaches for data preprocessing, thereby reducing privacy risks and mitigating bias to ensure fairness in model deployments.

Figures

Figures reproduced from arXiv: 2506.12321 by the authors.

**Figure 1.** The framework of our method. increases from 0 to 8, in Pythia 12B, memorized samples’ scores decrease by 0.09 while non-memorized samples’ scores increase by 0.25. This disparity suggests that surface-level characteristics are symptoms and deep-level characteristics are needed. To explore this disparity, we move beyond surface-level characteristics to investigate information redundancy as a more implicit factor inf… view at source ↗

**Figure 2.** Memorization scaling validation. Pilot study (N=1,000) demonstrates consistent [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Data characteristic effects on memorization scores across Pythia 410M, 6.9B, [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Impacts of perturbation strength on output uncertainty across Pythia models [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]

**Figure 5.** Figure 5: Different impacts of perturbation strength on sequence redundancy across Pythia [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

**Figure 6.** Figure 6: Redundancy ratio at different memorization thresholds ( [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]

**Figure 7.** Figure 7: Differential Vulnerability of Memorized and Non-Memorized Samples to Pertur [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

**Figure 8.** Figure 8: Evolution of Memorization Preferences with Model Scale and Training. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 6 canonical work pages

[1]

Carlini, F

N. Carlini, F. Tram` er, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, ´U. Erlingsson, A. Oprea, C. Raf- fel, Extracting training data from large language models, in: 30th USENIX Security Symposium (USENIX Security 21), USENIX Associa- tion, 2021, pp. 2633–2650. URL: https://www.usenix.org/conference/ usenixsecurity21/p...

work page arXiv 2024
[3]

URL: https://openreview.net/forum?id= MGKDBuyv4p. N. Kandpal, E. Wallace, C. Raffel, Deduplicating training data mitigates privacy risks in language models, in: International Conference on Machine Learning, PMLR, 2022, pp. 10697–10707. A. Kassem, O. Mahmoud, S. Saad, Preserving privacy through dememo- rization: An unlearning technique for mitigating memor...

2022
[5]

arXiv:2412.18497

URL: https://arxiv.org/abs/2412.18497. arXiv:2412.18497. Y. Bai, G. Pei, J. Gu, Y. Yang, X. Ma, Special characters attack: Toward scalable training data extraction from large language models,

arXiv
[6]

arXiv:2405.05990

URL: https://arxiv.org/abs/2405.05990. arXiv:2405.05990. K. Tirumala, A. H. Markosyan, L. Zettlemoyer, A. Aghajanyan, Memoriza- tion without overfitting: analyzing the training dynamics of large language 20 models, in: Proceedings of the 36th International Conference on Neu- ral Information Processing Systems, NIPS ’22, Curran Associates Inc., Red Hook, NY, USA,

arXiv
[9]

Fedorenko, S

E. Fedorenko, S. T. Piantadosi, E. A. Gibson, Language is primarily a tool for communication rather than thought, Nature 630 (2024) 575–586. T.-Y. Chang, J. Thomason, R. Jia, Do localization methods actually local- ize memorized data in LLMs? a tale of two benchmarks, in: K. Duh, H. Gomez, S. Bethard (Eds.), Proceedings of the 2024 Conference of the North...

2024
[10]

Biderman, U

S. Biderman, U. PRASHANTH, L. Sutawika, H. Schoelkopf, Q. Anthony, S. Purohit, E. Raff, Emergent and predictable memorization in large language models, in: A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine (Eds.), Advances in Neural Information Processing Systems, volume 36, Curran Associates, Inc., 2023, pp. 28072–28090. URL: https://procee...

arXiv 2024
[11]

arXiv:2410.23123

URL: https://arxiv.org/abs/2410.23123. arXiv:2410.23123. L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, et al., The pile: An 800gb dataset of di- verse text for language modeling, arXiv preprint arXiv:2101.00027 (2020). URL: https://arxiv.org/abs/2101.00027. S. Biderman, H. Schoelkopf, Q. Anthony, ...

arXiv 2020
[12]

Gurnee, N

W. Gurnee, N. Nanda, M. Pauly, K. Harvey, D. Troitskii, D. Bertsimas, Find- ing neurons in a haystack: Case studies with sparse probing, Transactions on Machine Learning Research (2023). URL: https://openreview.net/ forum?id=JYs1R9IMJr. P. Lesci, C. Meister, T. Hofmann, A. Vlachos, T. Pimentel, Causal esti- mation of memorisation profiles, in: L.-W. Ku, A...

work page 2023

Show all 13 references

[13]

arXiv:2505.22945

URL: https://arxiv.org/abs/2505.22945. arXiv:2505.22945. 24
[2022]

URL: http://dl.acm.org/doi/10.5555/ 3600270.3603043. N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, C. Zhang, Quanti- fying memorization across neural language models, in: The Eleventh In- ternational Conference on Learning Representations,
[2023]

URL: https: //openreview.net/forum?id=TatRHT_1cK. X. Lu, X. Li, Q. Cheng, K. Ding, X. Huang, X. Qiu, Scaling laws for fact memorization of large language models, in: Y. Al-Onaizan, M. Bansal, Y.- N. Chen (Eds.), Findings of the Association for Computational Linguistics: EMNLP ...

2024 doi
[2024]

URL:https: //openreview.net/forum?id=kmn0BhQk7p. J. Aalmoes, V. Duddu, A. Boutet, On the alignment of group fairness with attribute privacy, in: Web Information Systems Engineering – WISE 2024: 25th International Conference, Doha, Qatar, December 2–5, 2024, Proceedings, Part I...

2024 doi
[2025]

arXiv:2505.15722

URL: https:// arxiv.org/abs/2505.15722. arXiv:2505.15722. 18 K. Lee, D. Ippolito, A. Nystrom, C. Zhang, D. Eck, C. Callison-Burch, N. Carlini, Deduplicating training data makes language models better, in: S. Muresan, P. Nakov, A. Villavicencio (Eds.), Proceedings of the 60th A...

2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Carlini, F

N. Carlini, F. Tram` er, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, ´U. Erlingsson, A. Oprea, C. Raf- fel, Extracting training data from large language models, in: 30th USENIX Security Symposium (USENIX Security 21), USENIX Associa- tion, 2021, pp. 2633–2650. URL: https://www.usenix.org/conference/ usenixsecurity21/p...

work page arXiv 2024

[2] [3]

URL: https://openreview.net/forum?id= MGKDBuyv4p. N. Kandpal, E. Wallace, C. Raffel, Deduplicating training data mitigates privacy risks in language models, in: International Conference on Machine Learning, PMLR, 2022, pp. 10697–10707. A. Kassem, O. Mahmoud, S. Saad, Preserving privacy through dememo- rization: An unlearning technique for mitigating memor...

2022

[3] [5]

arXiv:2412.18497

URL: https://arxiv.org/abs/2412.18497. arXiv:2412.18497. Y. Bai, G. Pei, J. Gu, Y. Yang, X. Ma, Special characters attack: Toward scalable training data extraction from large language models,

arXiv

[4] [6]

arXiv:2405.05990

URL: https://arxiv.org/abs/2405.05990. arXiv:2405.05990. K. Tirumala, A. H. Markosyan, L. Zettlemoyer, A. Aghajanyan, Memoriza- tion without overfitting: analyzing the training dynamics of large language 20 models, in: Proceedings of the 36th International Conference on Neu- ral Information Processing Systems, NIPS ’22, Curran Associates Inc., Red Hook, NY, USA,

arXiv

[5] [9]

Fedorenko, S

E. Fedorenko, S. T. Piantadosi, E. A. Gibson, Language is primarily a tool for communication rather than thought, Nature 630 (2024) 575–586. T.-Y. Chang, J. Thomason, R. Jia, Do localization methods actually local- ize memorized data in LLMs? a tale of two benchmarks, in: K. Duh, H. Gomez, S. Bethard (Eds.), Proceedings of the 2024 Conference of the North...

2024

[6] [10]

Biderman, U

S. Biderman, U. PRASHANTH, L. Sutawika, H. Schoelkopf, Q. Anthony, S. Purohit, E. Raff, Emergent and predictable memorization in large language models, in: A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, S. Levine (Eds.), Advances in Neural Information Processing Systems, volume 36, Curran Associates, Inc., 2023, pp. 28072–28090. URL: https://procee...

arXiv 2024

[7] [11]

arXiv:2410.23123

URL: https://arxiv.org/abs/2410.23123. arXiv:2410.23123. L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, et al., The pile: An 800gb dataset of di- verse text for language modeling, arXiv preprint arXiv:2101.00027 (2020). URL: https://arxiv.org/abs/2101.00027. S. Biderman, H. Schoelkopf, Q. Anthony, ...

arXiv 2020

[8] [12]

Gurnee, N

W. Gurnee, N. Nanda, M. Pauly, K. Harvey, D. Troitskii, D. Bertsimas, Find- ing neurons in a haystack: Case studies with sparse probing, Transactions on Machine Learning Research (2023). URL: https://openreview.net/ forum?id=JYs1R9IMJr. P. Lesci, C. Meister, T. Hofmann, A. Vlachos, T. Pimentel, Causal esti- mation of memorisation profiles, in: L.-W. Ku, A...

work page 2023