REVIEW 6 cited by
Explainability for Large Language Models: A Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. However, their internal mechanisms are still unclear and this lack of transparency poses unwanted risks for downstream applications. Therefore, understanding and explaining these models is crucial for elucidating their behaviors, limitations, and social impacts. In this paper, we introduce a taxonomy of explainability techniques and provide a structured overview of methods for explaining Transformer-based language models. We categorize techniques based on the training paradigms of LLMs: traditional fine-tuning-based paradigm and prompting-based paradigm. For each paradigm, we summarize the goals and dominant approaches for generating local explanations of individual predictions and global explanations of overall model knowledge. We also discuss metrics for evaluating generated explanations, and discuss how explanations can be leveraged to debug models and improve performance. Lastly, we examine key challenges and emerging opportunities for explanation techniques in the era of LLMs in comparison to conventional machine learning models.
Forward citations
Cited by 6 Pith papers
-
Large Language Models for Interpretable Mental Health Diagnosis
An LLM plus expert review plus a Datalog engine can encode ICD-11 mood disorder criteria into runnable rules that agree with the authors' manual encoding on 30 synthetic patients.
-
The Process of Categorical Clipping at the Core of the Genesis of Concepts in Synthetic Neural Cognition
Words that strongly activate both a lower-layer neuron and its strongly connected upper-layer neuron in GPT-2XL form more semantically similar clusters, which the paper interprets as a clipping process.
-
Putnam's Critical and Explanatory Tendencies Interpreted from a Machine Learning Perspective
The paper argues that Putnam's critical (prediction) and explanatory (explanation) tendencies in science necessarily depend on each other, and interprets this through the input-output structure of machine learning models.
-
Finding Needles in Emb(a)dding Haystacks: Legal Document Retrieval via Bagging and SVR Ensembles
An ensemble of support vector regressors over embedding-space neighbors reaches recall 0.849 on German legal passage retrieval, higher than the reported GerDaLIR baselines.
-
How Do Artificial Intelligences Think? The Three Mathematico-Cognitive Factors of Categorical Segmentation Operated by Synthetic Neurons
The paper names three components of a neuron's aggregation function as cognitive factors and reports near-unity correlations in GPT-2XL, but the effects are largely true by construction.
-
The State of Post-Hoc Local XAI Techniques for Image Processing: Challenges and Motivations
A review of post-hoc local XAI techniques for images, covering motivations, challenges, and suggested future directions.
Discussion (0). Continue with ORCID to comment.