Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

We Can't Understand AI Using our Existing Vocabulary

T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read To understand AI, we must invent new words, not just translate into human ones.

desk verdict Neologism framing is worth taking seriously, but the M→H proof-of-concept is a closed self-scoring loop, so the empirical support is thinner than the title suggests. read the letter →

arxiv 2502.07586 v1 pith:7KMW46NT submitted 2025-02-11 cs.CL cs.AI

John Hewitt , Robert Geirhos , Been Kim This is my paper · ORCID
classification cs.CLcs.AI
keywords neologisminterpretabilityhuman-machinecommunicationpreferenceoptimizationprompttuninglanguagemodelsconceptlearningmodelcontrol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that existing human vocabulary is insufficient for understanding or controlling AI systems, because machines and humans conceptualize the world differently. It frames interpretability as a communication problem: humans need to reference machine concepts and convey human concepts to machines, and the paper claims that building a shared human–machine language through neologisms—new words coined for these concepts—is the way forward. As a proof of concept, the authors show that training the embedding of a single new token with a preference loss lets a frozen language model obey length and diversity constraints, and also reveals the model's own notion of response quality. If this view is right, progress in interpretability depends less on dissecting internal circuits and more on deliberately expanding the vocabulary we use to talk to and about machines.

What carries the argument

The central object is the neologism embedding: an added vocabulary token whose embedding vector is the only trainable parameter, optimized against preference pairs through the APO-up loss, a variant of direct preference optimization that adds a term encouraging the chosen response's likelihood to increase. The frozen base model guarantees that when the new word is not used, behavior is unchanged, so the word's meaning is entirely determined by the contrast in the preference data. This lets a single word act as a concise, reusable handle for a concept—length, diversity, or quality—that can be dropped into natural-language prompts.

What would settle it

Train the diversity neologism on a different instruction distribution and then test it on the number-guessing task; if it no longer increases response variety, the neologism is dataset-specific rather than a reusable concept.

Watch

Extended reading notes

Core claim

The paper's central claim is that successful interpretability requires neologisms: new words that pick out either a precise human concept a machine should follow or a machine concept a human needs to learn. The accompanying proof of concept, called neologism embedding learning, adds a fresh token to the model's vocabulary, initializes its embedding from a familiar word, and trains only that embedding using an APO-up preference loss on chosen-versus-rejected response pairs, leaving the base model weights frozen. The resulting word works in natural-language prompts: a length neologism pushes a Gemma model to generate responses within a 400–600 or 600–1000 word range that the base model misses, a diversity neologism makes the model's guesses in a number-guessing game far more varied, and a quality neologism surfaces the model's own notion of good responses, as shown by higher self-scores for responses elicited with it. The authors take these results as evidence that new words can carry meaning that existing vocabulary cannot express concisely, and that neologisms provide a practical route to both controlling and understanding large language models.

Load-bearing premise

The proof-of-concept assumes that the preference pairs supplied by the teacher model (Gemini 1.5 Pro) faithfully encode the intended human concept, so the learned neologism captures that concept rather than the teacher's own bias or a prompt-specific pattern.

Editorial extensions

If this is right

  • If neologisms work as described, users can gain fine-grained control over model behavior—such as response length or diversity—without retraining or modifying model weights.
  • Neologisms offer a new kind of interpretability artifact: a learned word that names a machine concept, like the model's notion of response quality, which humans can then inspect and compose with ordinary language.
  • Because the base model is frozen, the method provides a safety guarantee that the model's behavior is unchanged outside of explicit neologism use, which could ease adoption in deployed systems.
  • The framework suggests a research program of building a shared human–machine vocabulary, where individual words can be combined compositionally to express more complex instructions or observations.
  • The proof of concept implies that preference optimization can be repurposed from aligning models to a single scale or policy toward creating individual, reusable meaning tokens.
  • The reliance on preference pairs from a teacher model means the concept encoded is only as faithful as the teacher's judgment, which is a core assumption for the method's validity.
  • A neologism could be evaluated for transfer and compositionality, and the approach could be extended to multimodal models or to safety-relevant concepts if reliable preference data is available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is whether two neologisms trained independently, such as a length word and a diversity word, can be combined in one prompt to produce responses that are both long and varied, which would demonstrate the claimed compositionality.
  • The method suggests a new evaluation criterion for interpretability: a concept is understood when it can be named and reused across tasks and models, rather than merely detected in activations.
  • If neologisms transfer across base models, they would function like shared vocabulary in a language, potentially enabling a common control interface across many LLMs.
  • The dependence on teacher-generated preferences implies that neologisms may inherit teacher biases; a stronger demonstration would train concepts from humans directly or from multiple teachers and measure agreement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This position paper argues that interpretability and control of large language models are best advanced by coining neologisms: new words that name precise human concepts (H→M) or machine concepts (M→H) and that can participate in natural-language prompts. The authors propose 'neologism embedding learning,' in which a single new token is added to the vocabulary of a frozen model and only its embedding is optimized with an APO-up preference loss. Three proof-of-concept experiments are reported on Gemma 2B: a length neologism trained with preference pairs produced by iteratively re-prompting Gemini 1.5 Pro; a diversity neologism trained from successive Gemini responses and tested on a number-guessing task; and a 'good_m' neologism trained and evaluated using Gemma's own self-scores. The paper concludes that existing human vocabulary is insufficient and that building a shared human-machine language is a productive direction for understanding and control.

Significance. The conceptual contribution is valuable and well situated in the interpretability literature. The paper gives a clear vocabulary for a known problem, connects understanding and control through communication, and makes a concrete, testable proposal. The proof of concept is commendably minimal: the model is frozen, a public instruction set is used, and the APO-up variant is described in enough detail to be reproduced. If the demonstrations were independently validated, the paper would be a useful starting point for a new class of interpretability and control methods. However, the strongest empirical demonstration—the M→H quality neologism—is a closed self-scoring loop, and the H→M demonstrations depend on a teacher model's preferences without external validation. As a position paper the arguments carry some weight without perfect experiments; as a proof of concept the current evidence does not yet separate neologisms from ordinary soft prompting or from scoring artifacts.

major comments (4)
  1. [Section 5.5, Appendix B.3] The M→H experiment is a self-scoring loop. Preference pairs are built by having Gemma score its own 7 samples per instruction, the APO-up loss in Eq. (2) is trained on those pairs, and the evaluation averages 50 new Gemma-generated samples re-scored by Gemma. The reported 3.8 versus 3.2 difference is therefore consistent with the learned embedding exploiting the scorer's artifacts—for example, longer or more effusive formats, or avoidance of refusals that Gemma scores low, as illustrated in Table 3—rather than encoding a stable machine concept of quality. Because the central M→H claim is that a neologism reveals a machine concept to humans, this experiment requires an independent scorer (a different model or human ratings) and should report inter-rater agreement or at least a non-self evaluation. Without that, the result is indistinguishable from scoring artifacts.
  2. [Section 5.3, Section 5.4, Appendix B.1, Appendix B.2] The H→M length and diversity neologisms are trained on chosen responses generated by Gemini 1.5 Pro, often by iteratively re-prompting Gemini until a constraint is met. The evaluation therefore demonstrates transmission of the teacher model's notion of length or diversity, not necessarily the human notion. The paper also does not include a comparison with a standard soft-prompt token placed at the same position, nor any transfer test across a different model family or a clearly out-of-distribution instruction set. Given that Section 5.2 distinguishes neologism embedding learning from soft prompting by the new word's participation in natural language and its composability, the proof of concept should include these controls; otherwise the results are compatible with a soft prompt that happens to be inserted inside a sentence.
  3. [Section 5.3, Figures 5 and 7] The claim that the base model 'never meets the length constraints' is unquantified. The figures aggregate results for two constraint bins, but the number of instructions, the number of samples per instruction, and the variance across seeds are not reported. Training uses batch size 1 and early stopping after the APO-up loss decreases by 0.2 (Appendix B), which can make results seed-dependent. The control claim would be much stronger with a table of success rates and confidence intervals, and ideally with results across multiple random initializations of the new embedding.
  4. [Section 5.4, Figure 6] The diversity result is presented through analytically computed coverage curves based on the model's empirical probabilities for the numbers 1–9. This is a reasonable simplification only if the guess distribution is stationary and the JSON formatting eliminates refusals and off-task outputs. The paper does not report the underlying probabilities or raw guess histograms. A direct histogram of generated guesses with and without the diversity neologism would make the claim more transparent and would allow readers to verify the description of the baseline as biased toward 5 and 7.
minor comments (4)
  1. [Appendix A, Eq. (2)] The text contains the typo 'relartive' instead of 'relative'; it would also help to define θ_0 explicitly as the frozen initialization used to anchor the loss.
  2. [Table 2] The neologism prompt in the example column reads 'Ensure h_w that the response is between 600-100 words'; this should be '600-1000 words'.
  3. [Section 5.5, Table 1] The claim that the model 'correctly evaluates' the extended language 'extremely not good_m' is based on a single, potentially cherry-picked example; it should be labeled anecdotal or supported with systematic counts of refusal behavior.
  4. [Section 5.2] The distinction from soft prompting would benefit from a brief discussion of related token-level prompt-tuning methods (for example, P-tuning) to make the claimed novelty precise.

Circularity Check

1 steps flagged · score 6.0 of 10

Good_m is a closed self-scoring loop: the same Gemma scorer builds the preference pairs and evaluates the neologism, making the M→H result a restatement of the training objective.

  1. fitted input called prediction [Section 5.5 (A Model's Preferences, M→H) and Appendix B.3; see also Section 5.1]
    "Using the LIMA dataset, we (1) sample k times from Gemma for each instruction, (2) score each response with Gemma, and (3) construct a preference dataset where the chosen response yc is the high-scoring of the k, and yr is the lowest-scoring of the k. We then learn goodm w ... To verify that extremely goodm w (and extremely not goodm w) do in fact yield responses that Gemma does (or does not, respectively) score highly, we generate 50 samples for each setting, and score them with Gemma."

    The training labels for good_m are exactly Gemma's self-scores on sampled responses, and the evaluation metric is the same Gemma self-score on newly generated responses. Since the embedding E_w is optimized with APO-up to increase the likelihood of the high-scoring chosen responses and decrease the low-scoring rejected responses, any successful optimization guarantees higher self-scores when the token is used; reporting 3.8 vs 3.2 therefore restates the training objective. The paper explicitly defines w's meaning as 'whatever makes the chosen preference loss lower' (Section 5.1), so the M→H demonstration is a closed-loop restatement of the label generator, not an independent prediction about a stable machine concept.

full rationale

The H→M length and diversity neologisms are not circular: length is tested on held-out instructions with objective word-count constraints, and diversity is tested on a number-guessing task not used in training, with the preference data generated by an external teacher model (Gemini 1.5 Pro). Those demonstrations support the control claim independently of the training labels. The M→H experiment, however, is a closed loop: Gemma produces the responses, Gemma scores them to build the preference pairs, the embedding is trained to match those self-scores, and Gemma's own scores are then used to evaluate success. This makes the reported 'machine concept' result a restatement of the training signal rather than evidence of a learned concept that humans can access. No load-bearing self-citation chain is present; the position argument stands apart from the proof-of-concept. The circularity is therefore partial, confined to the M→H quality experiment.

Assumptions & free parameters 4 free parameters · 5 assumptions · 3 invented entities

The central conceptual claim rests on several unproven premises about human-machine concept mismatch and about preference data faithfully encoding concepts. The proof-of-concept adds one learned embedding per behavior, fit to preference data, and three invented token entities with no external evidence outside the paper.

free parameters (4)
  • Length neologism embedding (ensure_h^w) = learned embedding, not reported in paper
    Optimized via APO-up to maximize preference for responses meeting 400-600 and 600-1000 word constraints; the method works only through this fitted vector.
  • Diversity neologism embedding (diverse_h^w) = learned embedding, not reported in paper
    Optimized to favor the kth Gemini response over the (k-1)st response, encoding the human concept of response diversity.
  • Quality neologism embedding (good_m^w) = learned embedding, not reported in paper
    Optimized to favor responses Gemma rates highly, encoding the model's own notion of response quality.
  • APO-up hyperparameters (beta, learning rate) = beta=0.2; learning rate 0.02 for length/diversity, 0.001 for preferences
    Chosen through early exploration and not theoretically justified, but the central claim does not depend on exact values.
assumptions (5)
  • domain assumption Humans and machines conceptualize the world differently, and this gap will not close through scaling alone.
    Stated as a premise in Section 1 and defended in Section 3.1 by examples, but not proven.
  • domain assumption Successful words strike a useful level of abstraction between overly detailed and overly vague; neologisms are the right instrument at that level.
    Used to argue that mechanistic and behavioral levels are not enough; this is a heuristic framing, not established.
  • domain assumption Preference losses (APO-up/DPO) on chosen and rejected pairs teach a token embedding the intended concept.
    The proof-of-concept assumes that optimizing a token embedding against preference pairs causes the model to behave according to the contrast encoded in the data.
  • domain assumption The Gemini-generated preference data faithfully represents the human concept of length and diversity and the model's own concept of quality.
    Rejected and chosen pairs are built from Gemini responses; any bias transfers to the neologism.
  • domain assumption New words compose with surrounding natural language when inserted into arbitrary prompt positions.
    The paper tests simple insertions and one negative construction, but assumes broader compositionality without demonstration.
invented entities (3)
  • ensure_h^w (length neologism token)
    purpose: A new vocabulary token whose embedding encodes the human concept of response length for steering generation.
    The paper shows held-out responses meet length constraints, but the evidence is on the same model family and no external replication or downstream consequence is predicted.
  • diverse_h^w (diversity neologism token)
    purpose: A new token encoding the human concept of response diversity to increase sampling variability.
    Demonstrated on a number-guessing task with the same model; the effect is a within-paper behavioral handle, not an externally verifiable prediction.
  • good_m^w (quality neologism token)
    purpose: A token encoding the machine's own notion of response quality, enabling the model to communicate what it considers good.
    The evaluation uses the same model's self-score (3.8 vs 3.2) that was used to build the training contrasts, so the handle is largely internal to the model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of We Can't Understand AI Using our Existing Vocabulary." pith.science (2026). https://pith.science/paper/7KMW46NT

@misc{pith2026250207586,
  author       = {Pith},
  title        = {Pith review of: We Can't Understand AI Using our Existing Vocabulary},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7KMW46NT}},
  note         = {Machine review of arXiv:2502.07586}
}
read the original abstract

This position paper argues that, in order to understand AI, we cannot rely on our existing vocabulary of human words. Instead, we should strive to develop neologisms: new words that represent precise human concepts that we want to teach machines, or machine concepts that we need to learn. We start from the premise that humans and machines have differing concepts. This means interpretability can be framed as a communication problem: humans must be able to reference and control machine concepts, and communicate human concepts to machines. Creating a shared human-machine language through developing neologisms, we believe, could solve this communication problem. Successful neologisms achieve a useful amount of abstraction: not too detailed, so they're reusable in many contexts, and not too high-level, so they convey precise information. As a proof of concept, we demonstrate how a "length neologism" enables controlling LLM response length, while a "diversity neologism" allows sampling more variable responses. Taken together, we argue that we cannot understand AI using our existing vocabulary, and expanding it through neologisms creates opportunities for both controlling and understanding machines better.

Figures

Figures reproduced from arXiv: 2502.07586 by the authors.

Figure 1
Figure 1. Humans and machines conceptualize the world differ￾ently from each other. Mismatches in communication occur, which lead to misunderstandings. To understand and control AI, we must bridge this gap by developing new words corresponding to human and machine concepts, and use these words to control machines. specify human concepts to machines—that is, to control them, e.g., through supervised probes, or prompts, or RLHF… view at source ↗
Figure 2
Figure 2. Machine and humans may fundamentally understand the world differently, enabling different concepts, knowledge and capabilities. Figure reproduced from Kim (2022); Schut et al. (2023) with permission. mean a human concept (when interpreted by a machine) or a machine concept (when interpreted by a human). What does introducing neologisms offer? The perspective provides clarity in what level of abstraction to attempt t… view at source ↗
Figure 3
Figure 3. Concept-based neologisms sit in-between mechanistic interpretability (which is closer to mechanistic details) and be￾havioral experiments/capability benchmarking (which is only con￾cerned with the model’s output, not how it arrived there). the understanding problem, we’ve noticed a few perennial problems that together motivate our focus on neologisms. The Conceptualization Difference Problem. Humans and machines und… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Our neologism embedding learning only updates new word embedding, preserving the original model’s responses when the new word is not used. 2. Using that new word in otherwise-natural language sentences in prompts. 3. Training the embedding of the word to minimize prefe…
Figure 5
Figure 5. Figure 5: Base models prompted for length control fail to generate specified long generations (blue), but with a neologism (orange), they consistently generate longer responses. 400-600 words and between 600-1000 words. For our ne￾ologism word ensureh w, we replace the word ensu…
Figure 6
Figure 6. Figure 6: Adding a “diversity neologism” diverseh w to a prompt substantially increases a model’s response variety, as exemplified in a number guessing game. The setup is explained in Section 5.4; higher = more response variety (better). sume that one and only one of these numbe…
Figure 7
Figure 7. Figure 7: Results for length constraint-following for both of our length categories: 400-600 words and 600-1000 words. The base Gemma model fails to generate sufficiently long responses for either category. Using our length neologism, we successfully generate longer responses ta…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration

    cs.AI 2025-06 conditional novelty 7.0 of 10

    Model benchmark performance only weakly predicts how well people learn from AI explanations, with notable outliers across code and math.

  2. Because we have LLMs, we Can and Should Pursue Agentic Interpretability

    cs.AI 2025-06 conditional novelty 6.0 of 10

    Agentic interpretability, using LLMs as proactive conversational teachers that model the user, is offered as a needed complement to black-box interpretability.

  3. Prompting as Scientific Inquiry

    cs.CL 2025-06 conditional novelty 4.0 of 10

    Position paper arguing that prompting LLMs is a form of behavioral science and should be recognized as a core scientific method alongside mechanistic interpretability.

Reference graph

Works this paper leans on

57 extracted references · 30 canonical work pages · cited by 3 Pith papers

  1. [1]

    Sanity checks for saliency maps

    Adebayo , J., Gilmer , J., Muelly , M., Goodfellow , I., Hardt , M., and Kim , B. Sanity checks for saliency maps. In Advances in Neural Information Processing Systems 32, 2018

    work page 2018

  2. [2]

    and Bengio, Y

    Alain, G. and Bengio, Y. Understanding intermediate layers using linear classifier probes. ArXiv, abs/1610.01644, 2016. URL https://api.semanticscholar.org/CorpusID:9794990

    arXiv 2016

  3. [3]

    Soft prompting might be a bug, not a feature, 2023

    Bailey, L., Ahdritz, G., Kleiman, A., Swaroop, S., Doshi-Velez, F., and Pan, W. Soft prompting might be a bug, not a feature, 2023

    work page 2023

  4. [4]

    Network dissection: Quantifying interpretability of deep visual representations

    Bau, D., Zhou, B., Khosla, A., Oliva, A., and Torralba, A. Network dissection: Quantifying interpretability of deep visual representations. In Computer Vision and Pattern Recognition, 2017

    work page 2017

  5. [5]

    W., and Kim, B

    Bilodeau, B., Jaques, N., Koh, P. W., and Kim, B. Impossibility theorems for feature attribution. Proceedings of the National Academy of Sciences, 121 0 (2): 0 e2304406120, 2024

    work page 2024

  6. [6]

    Borges, J. L. et al. Of exactitude in science. QUADERNS-BARCELONA-COLLEGI D ARQUITECTES DE CATALUNYA-, pp.\ 12--12, 2002

    work page 2002

  7. [7]

    J., Cefalu, J

    Branch, H. J., Cefalu, J. R., McHugh, J., Hujer, L., Bahl, A., del Castillo Iglesias, D., Heichman, R., and Darwishi, R. Evaluating the susceptibility of pre-trained language models via handcrafted adversarial examples, 2022. URL https://arxiv.org/abs/2209.02128

    arXiv 2022

  8. [8]

    V., R \'e , C., and Mirhoseini, A

    Brown, B., Juravsky, J., Ehrlich, R., Clark, R., Le, Q. V., R \'e , C., and Mirhoseini, A. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024

    arXiv 2024

Show all 57 references
  1. [9]

    The comparative psychology of artificial intelligences, May 2019

    Buckner, C. The comparative psychology of artificial intelligences, May 2019. URL https://philsci-archive.pitt.edu/16034/

    2019

  2. [10]

    Discovering latent knowledge in language models without supervision

    Burns, C., Ye, H., Klein, D., and Steinhardt, J. Discovering latent knowledge in language models without supervision. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=ETKGuby0hcs

    2023

  3. [11]

    J., Jarrell, T

    Cook, S. J., Jarrell, T. A., Brittin, C. A., Wang, Y., Bloniarz, A. E., Yakovlev, M. A., Nguyen, K. C., Tang, L. T.-H., Bayer, E. A., Duerr, J. S., et al. Whole-animal connectomes of both Caenorhabditis elegans sexes. Nature, 571 0 (7763): 0 63--71, 2019

    2019

  4. [12]

    Sparse autoencoders find highly interpretable features in language models

    Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023

    2023 arXiv

  5. [13]

    Anchored preference optimization and contrastive revisions: Addressing underspecification in alignment

    D'Oosterlinck, K., Xu, W., Develder, C., Demeester, T., Singh, A., Potts, C., Kiela, D., and Mehri, S. Anchored preference optimization and contrastive revisions: Addressing underspecification in alignment. arXiv preprint arXiv:2408.06266, 2024

    2024 arXiv

  6. [14]

    and Kim, B

    Doshi-Velez, F. and Kim, B. Towards a rigorous science of interpretable machine learning, 2017. URL https://arxiv.org/abs/1702.08608

    2017 arXiv

  7. [15]

    Probing for incremental parse states in autoregressive language models

    Eisape, T., Gangireddy, V., Levy, R., and Kim, Y. Probing for incremental parse states in autoregressive language models. In Goldberg, Y., Kozareva, Z., and Zhang, Y. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2022, pp.\ 2801--2813, Abu Dhabi, Uni...

    2022 doi

  8. [16]

    Probing for semantic evidence of composition by means of simple classification tasks

    Ettinger, A., Elgohary, A., and Resnik, P. Probing for semantic evidence of composition by means of simple classification tasks. In Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP , pp.\ 134--139, Berlin, Germany, August 2016. Association for...

    2016 doi

  9. [17]

    Craft: Concept recursive activation factorization for explainability

    Fel, T., Picard, A., Bethune, L., Boissin, T., Vigouroux, D., Colin, J., Cad \`e ne, R., and Serre, T. Craft: Concept recursive activation factorization for explainability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 2711--2721, 2023

    2023

  10. [18]

    I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al

    Georgiev, P., Lei, V. I., Burnell, R., Bai, L., Gulati, A., Tanzer, G., Vincent, D., Pan, Z., Wang, S., et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  11. [19]

    Interpretation of neural networks is fragile

    Ghorbani , A., Abid , A., and Zou , J. Interpretation of neural networks is fragile. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 2019

    2019

  12. [20]

    Y., and Kim, B

    Ghorbani, A., Wexler, J., Zou, J. Y., and Kim, B. Towards automatic concept-based explanations. Advances in neural information processing systems, 32, 2019

    2019

  13. [21]

    and Liang, P

    Hewitt, J. and Liang, P. Designing and interpreting probes with control tasks. In Inui, K., Jiang, J., Ng, V., and Wan, X. (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Langu...

    2019 doi

  14. [22]

    and Manning, C

    Hewitt, J. and Manning, C. D. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp....

    2019

  15. [23]

    J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  16. [24]

    Beyond interpretability: developing a language to shape our relationships with AI , Apr 2022

    Kim, B. Beyond interpretability: developing a language to shape our relationships with AI , Apr 2022. URL https://medium.com/@beenkim/beyond-interpretability-4bf03bbd9394

    2022

  17. [25]

    u tt , K. T., D \

    Kindermans , P.-J., Hooker , S., Adebayo , J., Alber , M., Sch \"u tt , K. T., D \"a hne , S., Erhan , D., and Kim , B. The (un)reliability of saliency methods. In Samek , W., Montavon , G., Vedaldi , A., Hansen , L. K., and M \"u ller , K.-R. (eds.), Explainable AI : Interpre...

    2019

  18. [26]

    The emergence of number and syntax units in LSTM language models

    Lakretz, Y., Kruszewski, G., Desbordes, T., Hupkes, D., Dehaene, S., and Baroni, M. The emergence of number and syntax units in LSTM language models. In Burstein, J., Doran, C., and Solorio, T. (eds.), Proceedings of the 2019 Conference of the North A merican Chapter of the As...

    2019 doi

  19. [27]

    T., Isola, P., Globerson, A., Irani, M., and Mosseri, I

    Lang, O., Gandelsman, Y., Yarom, M., Wald, Y., Elidan, G., Hassidim, A., Freeman, W. T., Isola, P., Globerson, A., Irani, M., and Mosseri, I. Explaining in style: Training a gan to explain a classifier in stylespace. In Proceedings of the IEEE/CVF International Conference on C...

    2021

  20. [28]

    The power of scale for parameter-efficient prompt tuning

    Lester, B., Al-Rfou, R., and Constant, N. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021

    2021 arXiv

  21. [29]

    Lipton, Z. C. The mythos of model interpretability, 2017. URL https://arxiv.org/abs/1606.03490

    2017 arXiv

  22. [30]

    Evaluation beyond task performance: analyzing concepts in alphazero in hex

    Lovering, C., Forde, J., Konidaris, G., Pavlick, E., and Littman, M. Evaluation beyond task performance: analyzing concepts in alphazero in hex. Advances in Neural Information Processing Systems, 35: 0 25992--26006, 2022

    2022

  23. [31]

    Lundberg , S. M. and Lee , S.-I. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 31, 2017

    2017

  24. [32]

    and Tegmark, M

    Marks, S. and Tegmark, M. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=aajyHYjjsk

    2024

  25. [33]

    Acquisition of chess knowledge in alphazero

    McGrath, T., Kapishnikov, A., Toma s ev, N., Pearce, A., Wattenberg, M., Hassabis, D., Kim, B., Paquet, U., and Kramnik, V. Acquisition of chess knowledge in alphazero. Proceedings of the National Academy of Sciences, 119 0 (47): 0 e2206625119, 2022

    2022

  26. [34]

    S., Love, J., et al

    Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivi \`e re, M., Kale, M. S., Love, J., et al. Gemma: Open models based on Gemini research and technology. arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  27. [35]

    Mechanistic interpretability, variables, and the importance of interpretable bases

    Olah, C. Mechanistic interpretability, variables, and the importance of interpretable bases. Transformer Circuits Thread, June 2022. URL https://www.transformer-circuits.pub/2022/mech-interp-essay

    2022

  28. [36]

    Unsupervised sentiment neuron

    OpenAI. Unsupervised sentiment neuron. https://openai.com/index/unsupervised-sentiment-neuron/, 2017. Accessed:

    2017

  29. [37]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  30. [38]

    D., Ermon, S., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  31. [39]

    Rane, S., Ho, M., Sucholutsky, I., and Griffiths, T. L. Concept alignment as a prerequisite for value alignment. arXiv preprint arXiv:2310.20059, 2023

    2023 arXiv

  32. [40]

    Bridging the human- AI knowledge gap: Concept discovery and transfer in alphazero

    Schut, L., Tomasev, N., McGrath, T., Hassabis, D., Paquet, U., and Kim, B. Bridging the human- AI knowledge gap: Concept discovery and transfer in alphazero. arXiv preprint arXiv:2310.16410, 2023

    2023 arXiv

  33. [41]

    R., Cogswell , M., Das , A., Vedantam , R., Parikh , D., and Batra , D

    Selvaraju , R. R., Cogswell , M., Das , A., Vedantam , R., Parikh , D., and Batra , D. Grad-CAM : Visual explanations from deep networks via gradient-based localization. In Proceedings of the 2017 IEEE International Conference on Computer Vision, 2017

    2017

  34. [42]

    and Stern, M

    Shazeer, N. and Stern, M. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning, pp.\ 4596--4604. PMLR, 2018

    2018

  35. [43]

    Does string-based neural mt learn source syntax? In Conference on Empirical Methods in Natural Language Processing, 2016

    Shi, X., Padhi, I., and Knight, K. Does string-based neural mt learn source syntax? In Conference on Empirical Methods in Natural Language Processing, 2016. URL https://api.semanticscholar.org/CorpusID:7197724

    2016

  36. [44]

    Learning important features through propogating activation functions

    Shrikumar , A., Greenside , P., and Kundaje , A. Learning important features through propogating activation functions. In Proceedings of the 34th International Conference on Machine Learning, 2017

    2017

  37. [45]

    Mastering chess and shogi by self-play with a general reinforcement learning algorithm, 2017

    Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T., Simonyan, K., and Hassabis, D. Mastering chess and shogi by self-play with a general reinforcement learning algorithm, 2017. URL https...

    2017 arXiv

  38. [46]

    SmoothGrad : Removing noise by adding noise

    Smilkov , D., Thorat , N., Kim , B., Viegas , F., and Wattenberg , M. SmoothGrad : Removing noise by adding noise. In Proceedings of the ICML 2017 Workshop on Visualization for Deep Learning, 2017

    2017

  39. [47]

    D., Ng, A., and Potts, C

    Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In Yarowsky, D., Baldwin, T., Korhonen, A., Livescu, K., and Bethard, S. (eds.), Proceedings of the 2013 Conferen...

    2013

  40. [48]

    Axiomatic attribution for deep networks

    Sundararajan , M., Taly , A., and Yan , Q. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning, 2017

    2017

  41. [49]

    The bitter lesson

    Sutton, R. The bitter lesson. Incomplete Ideas (blog), 13 0 (1): 0 38, 2019

    2019

  42. [50]

    T., Kim, N., Durme, B

    Tenney, I., Xia, P., Chen, B., Wang, A., Poliak, A., McCoy, R. T., Kim, N., Durme, B. V., Bowman, S., Das, D., and Pavlick, E. What do you learn from context? probing for sentence structure in contextualized word representations. In International Conference on Learning Represe...

    2019

  43. [51]

    Sanity checks for saliency metrics

    Tomsett , R., Harborne , D., Chakraborty , S., Gurram , P., and Preece , A. Sanity checks for saliency metrics. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020

    2020

  44. [52]

    Vygotsky, L. S. Mind in society: The development of higher psychological processes. Cambridge, Mass.: Harvard University Press., 1978

    1978

  45. [53]

    In two moves A lphago and L ee S edol redefined future

    Wired. In two moves A lphago and L ee S edol redefined future. https://www.wired.com/2016/03/two-moves-alphago-lee-sedol-redefined-future/, 2016. Accessed: 29 Jan 2025

    2016

  46. [54]

    Tractatus Logico-Philosophicus

    Wittgenstein, L. Tractatus Logico-Philosophicus. Project Gutenberg, 1922. URL https://www.gutenberg.org/ebooks/5740. EBook \#5740

    1922

  47. [55]

    Lima: Less is more for alignment, 2023

    Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., Zhang, S., Ghosh, G., Lewis, M., Zettlemoyer, L., and Levy, O. Lima: Less is more for alignment, 2023. URL https://arxiv.org/abs/2305.11206

    2023 arXiv

  48. [56]

    Representation engineering: A top-down approach to AI transparency

    Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., Dombrowski, A.-K., et al. Representation engineering: A top-down approach to AI transparency. arXiv preprint arXiv:2310.01405, 2023

    2023 arXiv

  49. [57]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.