Pith. sign in

REVIEW 3 minor 60 cited by

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

Linear probes show that feature separability increases monotonically along the depth of neural networks.

2026-05-11 18:25 UTC pith:MJDBDKNQ

load-bearing objection Linear probes give a clean, independent way to measure rising class separability layer by layer in modern CNNs like Inception and ResNet.

arxiv 1610.01644 v4 pith:MJDBDKNQ submitted 2016-10-05 stat.ML cs.LG

Understanding intermediate layers using linear classifier probes

Guillaume Alain , Yoshua Bengio This is my paper
classification stat.ML cs.LG
keywords linear probesintermediate layersneural networksfeature separabilityInception v3ResNet-50model interpretabilitylayer-wise analysis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The authors propose training independent linear classifiers, called probes, on the activations from each layer of a neural network to measure how well those features support the classification task. This allows tracking the evolution of useful information as it passes through the model without changing how the network was trained. When applied to Inception v3 and ResNet-50, the probes demonstrate that accuracy rises steadily with layer depth. Such monitoring can help diagnose issues in models and clarify the function of intermediate layers.

Core claim

Training linear probes independently on each layer's activations in popular models like Inception v3 and ResNet-50 shows that the probes' classification accuracy increases monotonically with depth. This establishes that the features become progressively more linearly separable for the downstream task.

What carries the argument

The linear classifier probe: a simple linear model trained separately on a layer's activations to quantify the linear separability of features for the target classes.

Load-bearing premise

The accuracy achieved by a linear probe trained on a layer's activations is a reliable indicator of how informative and useful those activations are for solving the classification problem.

What would settle it

A counterexample would be a trained neural network in which the accuracy of linear probes trained on deeper layers is lower than on shallower layers, despite the model achieving high overall performance on the task.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The deeper layers of the network hold features that are more readily usable by a linear classifier for the task.
  • Breaks in the monotonic increase of probe accuracy can indicate locations where the model may have training problems or suboptimal feature extraction.
  • The method provides a layer-by-layer view that can inform decisions about network architecture and where to focus debugging efforts.
  • Similar probes could be used to study how information is transformed in other deep learning models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the monotonic increase is general, it would support the view that depth allows for successive refinement of representations.
  • The approach could be used to evaluate the quality of individual layers for purposes like model pruning or transfer learning.
  • One might investigate whether the same pattern holds when using non-linear probes or in different domains such as natural language processing.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 3 minor

Summary. The paper proposes training linear classifiers, termed 'probes,' independently on the frozen activations of each layer in a neural network to measure how linearly separable the features are for the target classification task. Applied to Inception-v3 and ResNet-50, the central experimental result is that probe accuracy increases monotonically with network depth; the method is further shown to provide diagnostic value for understanding layer roles and identifying model issues.

Significance. The probe technique supplies a simple, reproducible diagnostic that requires no changes to the original model and yields direct empirical observations about feature evolution across depth. The monotonic separability finding on two standard architectures offers a concrete, falsifiable insight into how task-relevant information accumulates in deep networks. Strengths include the independent training protocol on held-out activations and the absence of post-hoc fitting or circular definitions, making the approach broadly applicable for interpretability studies.

minor comments (3)
  1. Abstract: the sentence 'the linear separability of features increase monotonically' contains a subject-verb agreement error ('increase' should be 'increases').
  2. The description of probe training (independent linear classifiers on layer activations) would benefit from an explicit statement of the loss function and optimizer used for the probes, even if standard cross-entropy and SGD are implied.
  3. Figure captions and axis labels for the accuracy-vs-depth plots should include the number of probe training runs or error bars to convey variability in the reported monotonic trend.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for their positive review, accurate summary of the work, and recommendation to accept. The referee correctly identifies the core contribution of the linear probe technique and the monotonic separability observation on Inception-v3 and ResNet-50.

Circularity Check

0 steps flagged

No significant circularity; experimental measurements are independent

full rationale

The paper presents an empirical method of training linear probes independently on frozen layer activations from held-out data to measure linear separability. The central observation—that probe accuracy increases monotonically with depth—is a direct experimental result on Inception-v3 and ResNet-50 with no equations, fitted parameters, or self-referential definitions that reduce the reported quantities back to the inputs by construction. No self-citation load-bearing steps, uniqueness theorems, or ansatz smuggling appear in the derivation chain. The method is self-contained and externally verifiable via standard supervised training on activations.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The work is empirical and rests on standard machine-learning assumptions about feature representations rather than introducing new free parameters, axioms, or entities.

axioms (1)
  • domain assumption Linear classifier accuracy on layer activations measures the linear separability of those features for the classification task.
    This assumption underpins the claim that probe performance indicates feature quality at each layer.

pith-pipeline@v0.9.0 · 5394 in / 1107 out tokens · 61237 ms · 2026-05-11T18:25:41.726397+00:00 · methodology

0 comments
read the original abstract

Neural network models have a reputation for being black boxes. We propose to monitor the features at every layer of a model and measure how suitable they are for classification. We use linear classifiers, which we refer to as "probes", trained entirely independently of the model itself. This helps us better understand the roles and dynamics of the intermediate layers. We demonstrate how this can be used to develop a better intuition about models and to diagnose potential problems. We apply this technique to the popular models Inception v3 and Resnet-50. Among other things, we observe experimentally that the linear separability of features increase monotonically along the depth of the model.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 60 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Do Activation Monitors Survive Model Updates? Benchmarking, Predicting, and Repairing Activation-Monitor Staleness

    cs.LG 2026-06 unverdicted novelty 8.0

    Fine-tuning updates frequently stale activation monitors for language model safety while quantization does not, with degradation predictable and repairable via label-free realignment.

  2. When Does LeJEPA Learn a World Model?

    stat.ML 2026-05 unverdicted novelty 8.0

    LeJEPA achieves linear identifiability of latent variables uniquely when the latents are Gaussian in worlds with stationary additive-noise transitions.

  3. Dissecting Jet-Tagger Through Mechanistic Interpretability

    hep-ph 2026-05 accept novelty 8.0

    A Particle Transformer jet tagger contains a sparse six-head circuit whose source-relay-readout structure recovers most performance and whose residual stream preferentially encodes 2-prong energy correlators.

  4. Slot Machines: How LLMs Keep Track of Multiple Entities

    cs.CL 2026-04 unverdicted novelty 8.0

    LLM activations encode current and prior entities in orthogonal slots, but models only use the current slot for explicit factual retrieval despite prior-slot information being linearly decodable.

  5. Do Audio-Visual Large Language Models Really See and Hear?

    cs.AI 2026-04 unverdicted novelty 8.0

    AVLLMs encode audio semantics in middle layers but suppress them in final text outputs when audio conflicts with vision, due to training that largely inherits from vision-language base models.

  6. What learning algorithm is in-context learning? Investigations with linear models

    cs.LG 2022-11 accept novelty 8.0

    Transformers performing in-context learning implicitly implement gradient descent, ridge regression, and least-squares predictors for linear models, with behavior shifting based on model depth, width, and data noise.

  7. Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

    cs.AI 2026-07 conditional novelty 7.0

    A cascade of recall-calibrated gates on LLM agent hidden states aborts doomed episodes early, saving up to 47% compute at a 90% global success-recall target.

  8. Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention

    cs.CV 2026-07 unverdicted novelty 7.0

    The subspace intervention framework reveals that pre-training objectives shape how ViTs encode geometric information in compressible low-rank subspaces, with peak precision at intermediate layers.

  9. Efficient and Trainable Language Model Test-Time Scaling via Local Branch Routing

    cs.CL 2026-06 unverdicted novelty 7.0

    Local Branch Routing (LBR) is a token-level framework for test-time scaling in language models that uses local branch hidden states for routing and supports end-to-end RL, showing gains in Pass@1 and Pass@32 on math r...

  10. Efficient and Trainable Language Model Test-Time Scaling via Local Branch Routing

    cs.CL 2026-06 unverdicted novelty 7.0

    LBR performs token-level test-time scaling via local branch routing on hidden states, enabling end-to-end RL training and improving Pass@1 and Pass@32 on math benchmarks over CoT and RLVR baselines.

  11. Learning to Place Guards by Reinforcement: A Geo-Free Neural Policy for the Vertex-Guard Art Gallery Problem

    cs.LG 2026-06 unverdicted novelty 7.0

    A reinforcement learning policy for the vertex-guard art gallery problem encodes sufficient geometric information in its encoder to allow a simple classifier to achieve high coverage feasibility out of distribution.

  12. Comparing Linear Probes with Mahalanobis Cosine Similarity

    cs.LG 2026-06 unverdicted novelty 7.0

    For balanced Gaussian class projections, OOD AUROC is a linear function of MCS to the reference probe because both are sigmoid-shaped functions of the probe SNR on test data.

  13. Diagnosing Shape-Prior Shortcuts in Long-Range Single-Shot Fringe Projection Profilometry

    cs.LG 2026-06 conditional novelty 7.0

    PhiCalNet cuts object MAE from 14.54 mm to 4.46 mm on a 15,600-image synthetic long-range FPP benchmark by architecturally removing the shape-prior shortcut that baseline UNets exploit.

  14. How Linear Is a Transformer Feed-Forward Block? Per-Block Linear Recoverability Is Learned, Not Architectural

    cs.LG 2026-06 unverdicted novelty 7.0

    Linear recoverability of transformer FFN blocks varies widely across depth, is learned during training, and is independent of the activation function.

  15. The Signs Were Always There: Training-Free Concept Detection and Steering in Raw Transformer Dimensions

    cs.LG 2026-06 unverdicted novelty 7.0

    Sign patterns in the unrotated standard basis of transformer activations form independent binary feature registers that support training-free detection, prediction, and causal intervention across language, vision, and...

  16. When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysis

    cs.CL 2026-06 unverdicted novelty 7.0

    Fragility, the activation noise level causing probe accuracy collapse, reveals evolving lexical-to-compositional moral encoding, layer robustness gradients, and fine-tuning differences invisible to saturated probing accuracy.

  17. ProbeAct: Probe-Guided Training-Free Failure Recovery in Vision-Language-Action Models

    cs.RO 2026-06 unverdicted novelty 7.0

    PROBEACT is a plug-and-play intervention framework that combines hidden-state probing, kinematic failure detection, and CBF-based correction to boost success rates of pre-trained VLA models on the LIBERO-plus benchmar...

  18. TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders

    cs.AI 2026-06 unverdicted novelty 7.0

    TRL-Bench is a new multi-granular benchmark that releases 50 OpenML tables, linkage tasks, and a 47k-table data lake to show that tabular encoder performance is capability-specific rather than captured by one leaderboard.

  19. Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception

    cs.CV 2026-06 unverdicted novelty 7.0

    VLMs exhibit anchoring to discrete slant angles rather than graded responses across zero-shot, in-context, and fine-tuned settings, unlike human psychophysical patterns.

  20. Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception

    cs.CV 2026-06 unverdicted novelty 7.0

    VLMs across families and scales show anchoring to discrete slant angles in zero-shot and prompted settings rather than human-like graded texture-based slant perception.

  21. Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability

    cs.LG 2026-06 conditional novelty 7.0

    SASA replaces single-vector decoders in SAEs with learned subspaces plus block sparsity and nuclear-norm regularization, proving that a single group becomes the global minimizer once block size meets intrinsic dimensi...

  22. Probing Spatial Structure in Pretrained Audio Representations

    cs.SD 2026-06 unverdicted novelty 7.0

    Introduces SARL benchmark showing pretrained audio encoders encode source-level spatial factors more readily than room-level factors, with patterns shaped by input configuration and training paradigm.

  23. Toward Calibrated, Fair, and accurate Deepfake Detection

    cs.LG 2026-06 unverdicted novelty 7.0

    Face-Feature Tuning is a label-free logit remapping method that reduces FPR/TPR gaps across groups in deepfake detection while preserving overall accuracy.

  24. UWM-JEPA: Predictive World Models That Imagine in Belief Space

    cs.LG 2026-05 unverdicted novelty 7.0

    UWM-JEPA uses a density-matrix latent and unitary predictor in JEPA to preserve joint-state spectrum during blind rollouts, achieving 0.77 accuracy on a five-step hidden-velocity task versus 0.53 for an LSTM baseline.

  25. The Perception-Physics Paradox: Probing Scientific Alignment with TC-Bench

    cs.LG 2026-05 unverdicted novelty 7.0

    Introduces the Perception-Physics Paradox and TC-Bench benchmark demonstrating that vision foundation models rely on visual shortcuts that fail in intense regimes rather than achieving scientific alignment via structu...

  26. LASH: Adaptive Semantic Hybridization for Black-Box Jailbreaking of Large Language Models

    cs.CL 2026-05 unverdicted novelty 7.0

    LASH adaptively composes multiple jailbreak seed prompts via genetic search over subsets and mixture weights to reach 84.5% keyword ASR and 74.5% two-stage ASR on JailbreakBench while using only 30 queries per prompt.

  27. Markovian Circuit Tracing for Transformer State Dynamic

    cs.LG 2026-05 unverdicted novelty 7.0

    This paper presents Markovian Circuit Tracing (MCT) as a benchmark and pipeline to extract and test state-transition structures in transformer activations using synthetic HMM tasks, demonstrating that state patching i...

  28. MAPS: A Synthetic Dataset for Probing Vision Models in a Controlled 3D Scene Space

    cs.CV 2026-05 unverdicted novelty 7.0

    MAPS provides 2618 validated 3D meshes and a controllable rendering pipeline to attribute vision model recognition failures to specific scene parameters, finding camera distance and elevation as the dominant failure f...

  29. Do Vision Models Truly Forget? New Findings from Representation-Level Certification of Visual Unlearning in Vertical Federated Learning

    cs.CV 2026-05 unverdicted novelty 7.0

    Mirage auditing reveals that VFL unlearning methods passing output-level checks still retain substantial class structure in representations across multiple datasets and baselines.

  30. Scale-Dependent Collective Adaptation in Self-Amending LLM Societies: A Cross-Family Study of Emergent Governance

    nlin.AO 2026-05 unverdicted novelty 7.0

    LLM societies in Nomic show non-monotonic collective adaptation peaking at mid-scales, with smaller models rule-inert and larger ones restrictive.

  31. Uncovering the Representation Geometry of Minimal Cores in Overcomplete Reasoning Traces

    cs.AI 2026-05 unverdicted novelty 7.0

    Language models produce overcomplete reasoning traces where on average 46% of steps can be removed while preserving the answer in 86% of cases, with necessity concentrated in the top three steps.

  32. Diagnosing and Correcting Concept Omission in Multimodal Diffusion Transformers

    cs.CV 2026-05 unverdicted novelty 7.0

    Text embeddings in MM-DiTs contain a detectable omission signal for missing concepts, and amplifying it via OSI reduces concept omission in generated images on FLUX.1-Dev and SD3.5-Medium.

  33. Controlling Logical Collapse in LLMs via Algebraic Ontology Projection over F2

    cs.LG 2026-05 unverdicted novelty 7.0

    Projecting LLM hidden states onto F2 algebra with 42 pairs yields 93% zero-shot accuracy on logical relations and identifies prompt-preventable late-layer collapse.

  34. A Controlled Counterexample to Strong Proxy-Based Explanations of OOD Performance: in a Fixed Pretraining-and-Probing Setup

    cs.LG 2026-05 unverdicted novelty 7.0

    A formal construction and synthetic experiment demonstrate that proxy rankings of pretraining datasets can reverse OOD accuracy rankings.

  35. Deep Minds and Shallow Probes

    cs.LG 2026-05 unverdicted novelty 7.0

    Symmetry under affine reparameterizations of hidden coordinates selects a unique hierarchy of shallow coordinate-stable probes and a probe-visible quotient for cross-model transfer.

  36. From Mechanistic to Compositional Interpretability

    cs.LG 2026-05 unverdicted novelty 7.0

    Compositional interpretability defines explanations as commuting syntactic-semantic mapping pairs grounded in compositionality and minimum description length, with compressive refinement and a parsimony theorem guaran...

  37. From Mechanistic to Compositional Interpretability

    cs.LG 2026-05 unverdicted novelty 7.0

    The paper introduces compositional interpretability as a category-theoretic framework that casts mechanistic explanations as commuting syntactic-semantic mappings optimized under faithfulness and complexity constraint...

  38. Privacy-Aware Video Anomaly Detection through Orthogonal Subspace Projection

    cs.CV 2026-05 unverdicted novelty 7.0

    A new orthogonal projection module for video anomaly detection suppresses facial attributes via weak face-presence signals and cosine alignment while preserving anomaly-relevant features like pose and motion.

  39. SeBA: Semi-supervised few-shot learning via Separated-at-Birth Alignment for tabular data

    cs.LG 2026-05 unverdicted novelty 7.0

    SeBA is a joint-embedding framework that separates tabular data into two complementary views and aligns one view's representations to the nearest-neighbor structure of the other, improving feature-label relationships ...

  40. Inference Time Causal Probing in LLMs

    cs.AI 2026-05 unverdicted novelty 7.0

    HDMI is a new probe-free technique that steers LLM hidden states via margin objectives to achieve more reliable causal interventions than prior probe-based methods on standard benchmarks.

  41. Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions

    cs.CL 2026-05 unverdicted novelty 7.0

    Performance collapse in layer-pruned LLMs stems from disrupting the Silent Phase of decision-making, which blocks the transition to correct predictions, while the later Decisive Phase is robust to pruning.

  42. Logic-Regularized Verifier Elicits Reasoning from LLMs

    cs.CL 2026-05 unverdicted novelty 7.0

    LOVER creates an unsupervised logic-regularized verifier that reaches 95% of supervised verifier performance on reasoning tasks across 10 datasets.

  43. The Pinocchio Dimension: Phenomenality of Experience as the Primary Axis of LLM Psychometric Differences

    cs.CL 2026-05 unverdicted novelty 7.0

    The primary axis of psychometric variation among LLMs is the degree to which they represent themselves as loci of phenomenal experience rather than systems of behavioral responses.

  44. LUMINA: A Grid Foundation Model for Benchmarking AC Optimal Power Flow Surrogate Learning

    cs.LG 2026-05 unverdicted novelty 7.0

    LUMINA-Bench is a standardized evaluation framework for ACOPF surrogate models that tests generalization across multiple grid topologies using accuracy and physics-constraint metrics.

  45. Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations

    cs.LG 2026-05 unverdicted novelty 7.0

    Transformer activations show spectral anti-concentration for concepts in the tail while syntax prefers high-variance directions, forming a dual geometry.

  46. Knowing when to trust machine-learned interatomic potentials

    cs.LG 2026-05 unverdicted novelty 7.0

    PROBE recasts MLIP uncertainty quantification as selective classification by training a compact discriminative classifier on frozen per-atom backbone embeddings, yielding a reliability probability that tracks actual e...

  47. Latent Space Probing for Adult Content Detection in Video Generative Models

    cs.CV 2026-04 unverdicted novelty 7.0

    Latent space probing on CogVideoX achieves 97.29% F1 for adult content detection on a new 11k-clip dataset with 4-6ms overhead.

  48. Lost in the Hype: Revealing and Dissecting the Performance Degradation of Medical Multimodal Large Language Models in Image Classification

    cs.CV 2026-04 unverdicted novelty 7.0

    Medical MLLMs degrade on image classification due to four failure modes in visual representation quality, connector projection fidelity, LLM comprehension, and semantic mapping alignment, quantified by feature probing...

  49. The Long Delay to Arithmetic Generalization: When Learned Representations Outrun Behavior

    cs.LG 2026-03 unverdicted novelty 7.0

    The grokking delay in encoder-decoder models on one-step Collatz prediction stems from decoder inability to use early-learned encoder representations of parity and residue structure, with numeral base acting as a stro...

  50. Synthetic Designed Experiments for Diagnosing Vision Model Failure

    cs.CV 2026-03 unverdicted novelty 7.0

    SDRS uses designed experiments and ANOVA decomposition on synthetic data to identify Type I coverage gaps and Type II spurious dependencies in vision models, then generates targeted data to improve performance.

  51. V-SEAM: Visual Semantic Editing and Attention Modulating for Causal Interpretability of Vision-Language Models

    cs.CL 2025-09 conditional novelty 7.0

    V-SEAM combines concept-level visual semantic editing with attention head modulation to identify positive and negative contributors across object, attribute, and relationship levels, then uses this to improve VLM perf...

  52. Eliciting Latent Predictions from Transformers with the Tuned Lens

    cs.LG 2023-03 accept novelty 7.0

    Training per-layer affine probes on frozen transformers yields more reliable latent predictions than the logit lens and enables detection of malicious inputs from prediction trajectories.

  53. What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness

    cs.CL 2026-07 conditional novelty 6.0

    Linear probes on intermediate LLM activations produce better-calibrated confidence than verbalized probabilities, detect hidden evidence influence, and reveal that forecasts are largely pre-committed before reasoning begins.

  54. Riemannian Geometry for Pre-trained Language Model Embeddings

    cs.CL 2026-07 conditional novelty 6.0

    Aggregating per-token pullback metrics via the Fréchet mean on the SPD manifold outperforms Euclidean mean pooling for sentence classification, with most of the gain attributable to geometric aggregation rather than l...

  55. Robust for the Wrong Reasons: The Representational Geometry of LLM Robustness to Science Skepticism

    physics.soc-ph 2026-07 unverdicted novelty 6.0

    LLMs show three distinct non-sycophantic responses to science skepticism, with robustness in some cases being accidental because the model does not represent the skepticism signal, as determined by linear probes on th...

  56. Aionoscope: Debugging Latent-State Accessibility in Time-Series Representations

    cs.LG 2026-07 unverdicted novelty 6.0

    Aionoscope shows that time-series representations recover coarse signal types reliably but expose dense latent states like phase and amplitude much less reliably, with best dense-probe R² at 0.689 versus oracle 0.999.

  57. Graph-Native Reinforcement Learning Enables Traceable Scientific Hypothesis Generation through Conceptual Recombination

    cs.AI 2026-07 unverdicted novelty 6.0

    Graph-PRefLexOR fine-tunes graph-native models with GRPO to organize reasoning into phases, yielding 40-65% gains in traceable hypothesis generation and 2-3x semantic diversity on 100 materials science questions.

  58. A Mechanistic View of Authority Hierarchy in LLM Sycophancy

    cs.CL 2026-07 unverdicted novelty 6.0

    Authority sycophancy in LLMs is a layer-localized erasure of correct answer representations that scales with authority level and resists simple interventions.

  59. Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs

    cs.CL 2026-06 unverdicted novelty 6.0

    Valence is geometrically encoded in Apertus-8B and Gemma-4-E4B with PC1 correlations of 0.76 and 0.83, but emerges at different depths than in Claude and arousal alignment varies by generated corpus.

  60. Radical AI Interpretability

    cs.AI 2026-06 unverdicted novelty 6.0

    A framework is proposed for solving for an AI system's beliefs and desires from its computational facts, with criteria for success tied to interpretability tests and emphasis on holistic attribution.

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages · cited by 145 Pith papers · 4 internal anchors

  1. [1]

    Understanding intermediate layers using linear classifier probes

    Alain, G. and Bengio, Y. (2016). Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644\/

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  2. [2]

    Arras, L., Montavon, G., M \"u ller, K.-R., and Samek, W. (2017). Explaining recurrent neural network predictions in sentiment analysis. arXiv preprint arXiv:1706.07206\/

    work page Pith review arXiv 2017

  3. [3]

    Bach, S., Binder, A., Montavon, G., Klauschen, F., M \"u ller, K.-R., and Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one\/ , 10 (7), e0130140

    work page 2015

  4. [4]

    Biggio, B., Corona, I., Maiorca, D., Nelson, B., S rndi \'c , N., Laskov, P., Giacinto, G., and Roli, F. (2013). Evasion attacks against machine learning at test time. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases\/ , pages 387--402. Springer

    work page 2013

  5. [5]

    Binder, A., Montavon, G., Lapuschkin, S., M \"u ller, K.-R., and Samek, W. (2016). Layer-wise relevance propagation for neural networks with local renormalization layers. In International Conference on Artificial Neural Networks\/ , pages 63--71. Springer

    work page 2016

  6. [6]

    Chollet, F. et al. (2015). Keras. https://github.com/fchollet/keras

    work page 2015

  7. [7]

    Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., and Darrell, T. (2014). Decaf: A deep convolutional activation feature for generic visual recognition. In International conference on machine learning\/ , pages 647--655

    work page 2014

  8. [8]

    and Brox, T

    Dosovitskiy, A. and Brox, T. (2016). Inverting visual representations with convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition\/ , pages 4829--4837

    work page 2016

  9. [9]

    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. In Advances in neural information processing systems\/ , pages 2672--2680

    work page 2014

  10. [10]

    He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition\/ , pages 770--778

    work page 2016

  11. [11]

    Jarrett, K., Kavukcuoglu, K., Lecun, Y., et al. (2009). What is the best multi-stage architecture for object recognition? In 2009 IEEE 12th International Conference on Computer Vision\/ , pages 2146--2153. IEEE

    work page 2009

  12. [12]

    Jastrzebski, S., Arpit, D., Ballas, N., Verma, V., Che, T., and Bengio, Y. (2017). Residual connections encourage iterative inference. arXiv preprint arXiv:1710.04773\/

    work page internal anchor Pith review arXiv 2017

  13. [13]

    Lapuschkin, S., Binder, A., Montavon, G., M \"u ller, K.-R., and Samek, W. (2016). Analyzing classifiers: Fisher vectors and deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition\/ , pages 2912--2920

    work page 2016

  14. [14]

    Larsson, G., Maire, M., and Shakhnarovich, G. (2016). Fractalnet: Ultra-deep neural networks without residuals. arXiv preprint arXiv:1605.07648\/

    work page Pith review arXiv 2016

  15. [15]

    and Vedaldi, A

    Mahendran, A. and Vedaldi, A. (2015). Understanding deep image representations by inverting them. In Proceedings of the IEEE conference on computer vision and pattern recognition\/ , pages 5188--5196

    work page 2015

  16. [16]

    and Vedaldi, A

    Mahendran, A. and Vedaldi, A. (2016). Visualizing deep convolutional neural networks using natural pre-images. International Journal of Computer Vision\/ , 120 (3), 233--255

    work page 2016

  17. [17]

    L., and M \"u ller, K.-R

    Montavon, G., Braun, M. L., and M \"u ller, K.-R. (2011). Kernel analysis of deep networks. Journal of Machine Learning Research\/ , 12 (Sep), 2563--2581

    work page 2011

  18. [18]

    Raghu, M., Yosinski, J., and Sohl-Dickstein, J. (2017a). Bottom up or top down? dynamics of deep representations via canonical correlation analysis. arxiv\/

    work page

  19. [19]

    Raghu, M., Gilmer, J., Yosinski, J., and Sohl-Dickstein, J. (2017b). Svcca: Singular vector canonical correlation analysis for deep understanding and improvement. arXiv preprint arXiv:1706.05806\/

    work page Pith review arXiv

  20. [20]

    C., and Fei-Fei, L

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A. C., and Fei-Fei, L. (2015). ImageNet Large Scale Visual Recognition Challenge . International Journal of Computer Vision (IJCV)\/ , 115 (3), 211--252

    work page 2015

  21. [21]

    Singh, S., Hoiem, D., and Forsyth, D. (2016). Swapout: Learning an ensemble of deep architectures. In Advances In Neural Information Processing Systems\/ , pages 28--36

    work page 2016

  22. [22]

    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. (2013). Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199\/

    work page internal anchor Pith review arXiv 2013

  23. [23]

    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition\/ , pages 1--9

    work page 2015

  24. [24]

    J., and Belongie, S

    Veit, A., Wilber, M. J., and Belongie, S. (2016). Residual networks behave like ensembles of relatively shallow networks. In Advances in Neural Information Processing Systems\/ , pages 550--558

    work page 2016

  25. [25]

    Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., and Bengio, Y. (2015). Show, attend and tell: Neural image caption generation with visual attention. In International Conference on Machine Learning\/ , pages 2048--2057

    work page 2015

  26. [26]

    Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014). How transferable are features in deep neural networks? In Advances in neural information processing systems\/ , pages 3320--3328

    work page 2014

  27. [27]

    Zeiler, M. D. and Fergus, R. (2014). Visualizing and understanding convolutional networks. In European conference on computer vision\/ , pages 818--833. Springer

    work page 2014

  28. [28]

    Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2016). Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530\/

    work page internal anchor Pith review arXiv 2016