Pith. sign in

REVIEW 3 major objections 4 minor 41 references

Memorization in Language Models through the Lens of Intrinsic Dimension

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A sequence's intrinsic dimension, a geometric measure of its latent complexity, suppresses how often a language model reproduces it verbatim, especially under sparse exposure.

desk verdict Novel descriptive claim, thin statistics, unvalidated BERT-ID transfer—worth peer review but not ready to cite. read the letter →

arxiv 2506.09591 v1 pith:2XF7V3ZK submitted 2025-06-11 cs.CL

Stefan Arnold This is my paper · ORCID
classification cs.CL
keywords memorizationintrinsicdimensionlanguagemodelsprivacyleakageTwoNNlatentspacegeometryduplicationfrequencyoverparameterization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the intrinsic dimension (ID) of a text sequence—a geometric estimate of how many latent directions its embeddings occupy—systematically modulates the likelihood that a language model memorizes that sequence verbatim. Using 150-token samples from the Pile, IDs estimated from BERT embeddings via the TwoNN method, and memorization measured as greedy continuation matching across GPT-Neo models of roughly 0.1B to 6B parameters, the author reports that high-ID sequences are memorized less than low-ID sequences under sparse exposure. The effect is strongest for larger models and diminishes as duplication frequency rises, with small models showing a reversal in the medium-duplication regime. If correct, this would add a geometric, sequence-level covariate to the known drivers of unintended memorization and could inform privacy-preserving data filtering.

What carries the argument

The carrying object is an estimated per-sequence intrinsic dimension: each 150-token text is treated as a point cloud of token embeddings produced by BERT, and the TwoNN estimator (Facco et al., 2017) infers the local dimension from the ratio of distances to each point's first and second nearest neighbours. This number is compared, in quantile bins, against a discoverable-memorization probe in which each sequence is split into prefix and suffix, the prefix is prompted with greedy decoding, and the sequence counts as memorized if the continuation matches the suffix verbatim. The comparisons run across the GPT-Neo family (about 0.1B, 1.3B, 2.7B, and 6B parameters) on 1,000 Pile sequences stratified by duplication count in three log-scaled regimes, with sequence length fixed at 150 to stabilize the ID estimates.

What would settle it

Compute ID from each GPT-Neo model's own final-layer token embeddings instead of BERT and rerun the quantile-binned memorization analysis; if the inverse relationship between ID and memorization under sparse exposure disappears or flips sign, the central claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that intrinsic dimension acts as a suppressive signal for memorization: compared with low-ID sequences, high-ID sequences are less likely to be reproduced verbatim, particularly in overparameterized models and under sparse exposure. Concretely, in the low-duplication regime (fewer than 10 occurrences), memorization rates decline inversely with intrinsic dimensionality across all four model sizes; in the medium-duplication regime (10 to 100 occurrences) the inverse relationship persists for larger models but reverses for small models, which memorize slightly more of the structurally complex sequences; in the high-duplication regime (100 to 1000 occurrences), memorization saturates and becomes nearly invariant to ID. The paper interprets this as an interaction among scale, exposure, and structural complexity, where capacity to compress complex sequences protects them from memorization until repeated exposure overrides that protection.

Load-bearing premise

The load-bearing premise is that the intrinsic dimension estimated from BERT embeddings of a single 150-token sequence is the same complexity signal the GPT-Neo models 'see' when memorizing it.

Editorial extensions

If this is right

  • Sequences with low estimated intrinsic dimension are the highest verbatim-memorization risk under sparse exposure, so privacy audits could prioritize low-ID unique strings.
  • Scaling model capacity sharpens the suppression: larger models memorize proportionally less of the high-ID content unless that content is duplicated many times.
  • Deduplication is not a complete mitigation because low-ID sequences are memorized even with few occurrences.
  • Duplication frequency and model scale eventually override structural complexity: at 100 to 1000 copies, memorization no longer tracks the intrinsic dimension.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported effect is measured with BERT-based ID estimates; whether the same ordering holds in the memorizing models' own latent spaces is untested. A direct check is recomputing IDs from each GPT-Neo model's hidden states and rerunning the analysis.
  • The small-model reversal hints at a capacity threshold: models that cannot compress complex sequences may memorize them instead, which would predict a rise in per-sequence loss for high-ID samples before their memorization rate rises.
  • If the suppression is causal, perturbing low-ID sequences to raise their effective complexity could serve as a data-level privacy intervention, complementing deduplication.
  • The exposure-dependent interaction parallels established long-tail learning results, suggesting that memorization of rare complex items behaves like the learning of rare classes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper asks whether the intrinsic dimension (ID) of a text sequence, estimated by treating its BERT contextual embeddings as a point cloud and applying the TwoNN estimator, predicts how likely GPT-Neo language models (≈0.1B, 1.3B, 2.7B, 6B parameters) are to reproduce the sequence verbatim under greedy decoding. Using 1,000 sequences of length 150 from the Pile, stratified into three duplication-frequency regimes, the authors quantile-bin ID into 25 intervals and plot mean memorization per bin, disaggregated by model size. They report that in the low-duplication regime memorization declines with ID across all model sizes; in the medium regime the decline persists for large models but reverses for small models; and in the high-duplication regime memorization is nearly ID-invariant. The paper concludes that ID is a suppressive signal for memorization, especially in overparameterized models under sparse exposure, and interprets this through a complexity-compression lens.

Significance. If the central claim is correct, ID would be a newly identified, text-intrinsic covariate of verbatim memorization, relevant for privacy auditing and for understanding the interaction between model scale, data duplication, and latent structure. The study has notable strengths: ID and memorization are measured independently with no parameter fitted to the memorization data, sequence length is fixed, duplication counts are controlled by stratification, and the analysis covers a meaningful range of model scales and exposure regimes. However, the evidence currently rests on descriptive bin plots without uncertainty quantification, and the ID is measured in a representation space (BERT) different from the models whose memorization is scored (GPT-Neo), leaving the central construct-validity claim unestablished.

major comments (3)
  1. [§4, Figure 3] The 25-bin averaging scheme yields, by the paper's own design, roughly 1,000/3 ≈ 333 sequences per duplication regime and hence about 13 sequences per ID bin. Memorization is a binary outcome, so per-bin rates are subject to large sampling variability. No error bars, confidence intervals, or significance tests are reported, yet the central claim that memorization 'declines inversely with intrinsic dimensionality' is a claim about a monotone relationship. A handful of high-ID bins with zero memorization could drive the visual trend. Please report per-bin confidence intervals (e.g., binomial or bootstrap), or fit a logistic regression of memorization on ID with standard errors, and test whether the slope is significantly negative in the low-duplication regime.
  2. [§3, Methodology] The paper estimates ID from BERT contextual embeddings and TwoNN, while memorization is measured on GPT-Neo autoregressive models with a different tokenizer, architecture, and training objective, and greedy continuation depends only on the prefix whereas BERT embeddings encode bidirectional context. The Contribution section describes ID as 'as perceived by language models,' but no evidence is given that BERT's latent geometry ranks these 150-token sequences in the same order as GPT-Neo's. Without validation (for example, estimating ID from GPT-Neo hidden states on a subset and reporting the rank correlation with BERT-based ID, or showing that the reported relationship survives controlling for text domain/boilerplate), the claimed suppressive signal may be an artifact of BERT-ID correlating with an uncontrolled property that also affects memorability. The Limitations section does not mention this cross-model transfer gap.
  3. [§4, Figure 3] The claims of a 'reversal in trend' for small models at medium duplication and of saturation ('almost invariant') at high duplication are based on visual inspection of bin plots. No quantitative trend estimates, slope coefficients, or interaction tests are reported to support statements such as 'this is not the case for small models' and 'memorization is increasingly governed by exposure and scale.' Please report, for each model size and duplication regime, the estimated relationship between log-ID and memorization (with uncertainty), and test the relevant interactions (e.g., model size × ID, duplication regime × ID) rather than relying on the visual pattern in the figure.
minor comments (4)
  1. [§3, Table 1] Table 1 shows only two example sequences; it would be helpful to report the distribution of estimated IDs (range, quantiles) and the rank correlation between ID and duplication count, since these contextualize the stratification.
  2. [Figure 3] The x-axis label in the caption reads 'intrinsic memorization' but should presumably read 'intrinsic dimension'; also, in the provided manuscript the figure content appears garbled with placeholder tokens, so please ensure the final figure renders properly.
  3. [§2.1] There is a typo: 'Noteable definitions' should be 'Notable definitions.'
  4. [References] The reference 'Wang and Komatsuzaki (2021)' is cited for GPT-Neo, but the listed title is 'GPT-J-6B: A 6 billion parameter autoregressive language model'; please cite the correct model card or clarify the relationship between GPT-Neo and GPT-J.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: intrinsic dimension and memorization are measured by independent pipelines, with no fitted parameters or definitional identity linking them.

full rationale

The paper's central claim—that per-sequence intrinsic dimension estimated from BERT embeddings via TwoNN negatively correlates with GPT-Neo verbatim memorization—is not circular. Memorization is operationalized independently following Carlini et al. (2023): prompting with a prefix and checking verbatim continuation under greedy decoding. ID is computed from a separate representation space (BERT contextual embeddings) and a separate estimator (TwoNN), following Tulchinskii et al. (2024). No parameter of the ID estimator is fitted to the memorization data, and memorization is not defined in terms of ID; the two measurements are independent. The reuse of Carlini et al.'s sample and the GPT-Neo family is methodological inheritance, not circular reasoning. The only serious vulnerability is construct validity—whether BERT's latent geometry ranks sequences as GPT-Neo would—but that is an empirical validity concern, not a reduction of the result to its inputs. The paper includes no self-citation chain and no definitional identity; therefore score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim is an empirical correlation, so the ledger records the hand-chosen analysis settings (sequence length, binning, duplication bands, sample size) and the domain assumptions that carry the interpretation: the manifold hypothesis, the transfer of BERT-based ID estimates to GPT-Neo, verbatim greedy decoding as the memorization measure, and exact-duplicate-only exposure control.

free parameters (4)
  • sequence truncation length = 150 tokens
    All sequences are truncated to 150 tokens to stabilize ID estimates; ID values, and therefore the reported relationship, depend on this choice.
  • number of ID quantile bins = 25
    ID is quantile-binned into 25 intervals before averaging memorization; with 1,000 sequences this leaves roughly 13 sequences per bin per condition, so the visual trends depend on this binning.
  • duplication stratification bands = [1,10), [10,100), [100,1000)
    Hand-chosen log-scale boundaries define the low, medium, and high exposure regimes and determine where the inverse ID-memorization trend is claimed to hold.
  • subsample size = 1,000 sequences
    The sample size sets the per-bin counts; no power analysis is given for detecting the claimed monotone trends.
assumptions (4)
  • domain assumption The manifold hypothesis holds for text: token embeddings of a sequence concentrate near a low-dimensional manifold, and its dimension is a meaningful proxy for sample complexity.
    Invoked in Section 1 and Section 3 to justify interpreting ID as the structural complexity of a sequence as perceived by LMs.
  • domain assumption BERT embeddings plus TwoNN yield a valid, transferable ID estimate for single 150-token texts, applicable to the GPT-Neo models under test.
    Section 3 computes ID on BERT embeddings while memorization is measured on GPT-Neo; the transfer is not validated.
  • domain assumption Verbatim match under greedy decoding adequately operationalizes memorization for this comparison.
    Adopted in Section 2.1; the paper's Limitations section concedes greedy decoding is atypical and verbatim matching understates leakage.
  • domain assumption Stratification by exact duplication counts suffices to control exposure.
    Section 3 stratifies on exact duplicates; the Limitations section concedes near-duplicates account for most memorized content, so exposure control is incomplete.
invented entities (1)
  • None
    purpose: The paper introduces no new entities.
    Intrinsic dimension is a pre-existing construct, and the analysis relies only on existing models, datasets, and estimators.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Memorization in Language Models through the Lens of Intrinsic Dimension." pith.science (2026). https://pith.science/paper/2XF7V3ZK

@misc{pith2026250609591,
  author       = {Pith},
  title        = {Pith review of: Memorization in Language Models through the Lens of Intrinsic Dimension},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2XF7V3ZK}},
  note         = {Machine review of arXiv:2506.09591}
}
read the original abstract

Language Models (LMs) are prone to memorizing parts of their data during training and unintentionally emitting them at generation time, raising concerns about privacy leakage and disclosure of intellectual property. While previous research has identified properties such as context length, parameter size, and duplication frequency, as key drivers of unintended memorization, little is known about how the latent structure modulates this rate of memorization. We investigate the role of Intrinsic Dimension (ID), a geometric proxy for the structural complexity of a sequence in latent space, in modulating memorization. Our findings suggest that ID acts as a suppressive signal for memorization: compared to low-ID sequences, high-ID sequences are less likely to be memorized, particularly in overparameterized models and under sparse exposure. These findings highlight the interaction between scale, exposure, and complexity in shaping memorization.

Figures

Figures reproduced from arXiv: 2506.09591 by the authors.

Figure 1
Figure 1. Overview of the post-hoc assessment of mem [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Distribution of memorization rate and intrinsic [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Memorization as a function of intrinsic memorization, binned into equally-sized intervals and disaggregated [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 26 canonical work pages

  1. [1]

    Laurent Amsaleg, Oussama Chelly, Teddy Furon, St \'e phane Girard, Michael E Houle, Ken-ichi Kawarabayashi, and Michael Nett. 2018. Extreme-value-theoretic estimation of local intrinsic dimensionality. Data Mining and Knowledge Discovery, 32(6):1768--1805

    work page 2018

  2. [2]

    Alessio Ansuini, Alessandro Laio, Jakob H Macke, and Davide Zoccolan. 2019. Intrinsic dimension of data representations in deep neural networks. Advances in Neural Information Processing Systems, 32

    work page 2019

  3. [3]

    Devansh Arpit, Stanis aw Jastrz e bski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al. 2017. A closer look at memorization in deep networks. In International conference on machine learning, pages 233--242. PMLR

    work page 2017

  4. [4]

    Tolga Birdal, Aaron Lou, Leonidas J Guibas, and Umut Simsekli. 2021. Intrinsic dimension, persistent homology and generalization in neural networks. Advances in Neural Information Processing Systems, 34:6776--6789

    work page 2021

  5. [5]

    Gavin Brown, Mark Bun, Vitaly Feldman, Adam Smith, and Kunal Talwar. 2021. When is memorization of irrelevant training data necessary for high-accuracy learning? In Proceedings of the 53rd annual ACM SIGACT symposium on theory of computing, pages 123--132

    work page 2021

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  7. [7]

    Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations

    work page 2023

  8. [8]

    Nicholas Carlini, Chang Liu, \'U lfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Security Symposium (USENIX Security 19), pages 267--284

    work page 2019

Show all 41 references
  1. [9]

    Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21),...

    2021

  2. [10]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1--113

    2023

  3. [11]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  4. [12]

    Elena Facco, Maria d’Errico, Alex Rodriguez, and Alessandro Laio. 2017. Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Scientific reports, 7(1):12140

    2017

  5. [13]

    Amir Massoud Farahmand, Csaba Szepesv \'a ri, and Jean-Yves Audibert. 2007. Manifold-adaptive dimension estimation. In Proceedings of the 24th international conference on Machine learning, pages 265--272

    2007

  6. [14]

    Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. 2016. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4):983--1049

    2016

  7. [15]

    Vitaly Feldman. 2020. Does learning require memorization? a short tale about a long tail. In Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing, pages 954--959

    2020

  8. [16]

    Vitaly Feldman and Chiyuan Zhang. 2020. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33:2881--2891

    2020

  9. [17]

    Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. 2020. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027

    2020 arXiv

  10. [18]

    Jamie Hayes, Marika Swanberg, Harsh Chaudhari, Itay Yona, and Ilia Shumailov. 2024. Measuring memorization through probabilistic discoverable extraction. arXiv preprint arXiv:2410.19482

    2024 arXiv

  11. [19]

    Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.148 Are large pre-trained language models leaking your personal information? In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2038--2047, Ab...

    2022 doi

  12. [20]

    Daphne Ippolito, Florian Tramer, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher Choquette Choo, and Nicholas Carlini. 2023. Preventing generation of verbatim memorization in language models gives a false sense of privacy. In Proceedings of the 16th In...

    2023

  13. [21]

    Ian T Jolliffe and IT Jolliffe. 1986. Mathematical and statistical properties of sample principal components. Springer

    1986

  14. [22]

    Nikhil Kandpal, Eric Wallace, and Colin Raffel. 2022. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning, pages 10697--10707. PMLR

    2022

  15. [23]

    Hirokazu Kiyomaru, Issa Sugiura, Daisuke Kawahara, and Sadao Kurohashi. 2024. A comprehensive analysis of memorization in large language models. In Proceedings of the 17th International Natural Language Generation Conference, pages 584--596

    2024

  16. [24]

    Jooyoung Lee, Thai Le, Jinghui Chen, and Dongwon Lee. 2023. Do language models plagiarize? In Proceedings of the ACM Web Conference 2023, pages 3637--3647

    2023

  17. [25]

    Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2022. https://doi.org/10.18653/v1/2022.acl-long.577 Deduplicating training data makes language models better . In Proceedings of the 60th Annual Meeting of t...

    2022 doi

  18. [26]

    Elizaveta Levina and Peter Bickel. 2004. Maximum likelihood estimation of intrinsic dimension. Advances in neural information processing systems, 17

    2004

  19. [27]

    Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tram \`e r, and Katherine Lee. 2023. Scalable extraction of training data from (production) language models. arXiv preprint ar...

    2023 arXiv

  20. [28]

    Milad Nasr, Javier Rando, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Florian Tram \`e r, and Katherine Lee. 2025. Scalable extraction of training data from aligned, production language models. In The Thi...

    2025

  21. [29]

    Phillip Pope, Chen Zhu, Ahmed Abdelkader, Micah Goldblum, and Tom Goldstein. 2021. The intrinsic dimension of images and its impact on learning. 9th International Conference on Learning Representations, ICLR

    2021

  22. [30]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  23. [31]

    Benjamin Schweinhart. 2021. Persistent homology and the upper box dimension. Discrete & Computational Geometry, 65(2):331--364

    2021

  24. [32]

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3--18. IEEE

    2017

  25. [33]

    Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. 2022. Memorization without overfitting: Analyzing the training dynamics of large language models. Advances in Neural Information Processing Systems, 35:38274--38290

    2022

  26. [34]

    Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, Sergey Nikolenko, Evgeny Burnaev, Serguei Barannikov, and Irina Piontkovskaya. 2024. Intrinsic dimension estimation for robust detection of ai-generated texts. Advances in Neural Information Process...

    2024

  27. [35]

    Ben Wang and Aran Komatsuzaki. 2021. Gpt-j-6b: A 6 billion parameter autoregressive language model

    2021

  28. [36]

    Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. 2025. Generalization v.s. memorization: Tracing language models capabilities back to pretraining data. In The Thirteenth International Conference on Learning Repr...

    2025

  29. [37]

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. 2018. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF), pages 268--282. IEEE

    2018

  30. [38]

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2022. Understanding deep learning requires rethinking generalization. In International Conference on Learning Representations

    2022

  31. [39]

    Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tram \`e r, and Nicholas Carlini. 2023. Counterfactual memorization in neural language models. Advances in Neural Information Processing Systems, 36:39321--39362

    2023

  32. [40]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  33. [41]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.