Pith. sign in

REVIEW 3 minor 238 cited by

The information bottleneck method

T0 review · 0 major / 3 minor · reviewed 2026-05-11 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Compressing a signal X through limited codewords can preserve all the information it provides about another signal Y.

desk verdict This is the original paper that introduced the information bottleneck as a variational generalization of rate-distortion theory, and the derivation is clean and direct. read the letter →

arxiv physics/0004057 v1 pith:C5KGNOKK submitted 2000-04-24 physics.data-an cond-mat.dis-nncs.LGnlin.AO

Naftali Tishby (Hebrew University , NEC Research Institute) , Fernando C. Pereira (ATT Shannon Laboratory) , William Bialek (NEC Research Institute) This is my paper · ORCID
classification physics.data-ancond-mat.dis-nncs.LGnlin.AO
keywords informationbottleneckmutualratedistortiontheorydatacompressionrelevantfeatureextractionsignalprocessinglearning
checked against Cost.FunctionalEquationbilinear_family_forceddefect_zero_iff_one
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to formalize the extraction of relevant information from one signal about another as an optimization task. Relevant information means the part of X that helps predict Y, such as speech sounds helping identify spoken words. The authors propose squeezing X into a compressed representation T so that T carries as much information about Y as possible while using as little information from X as possible. This matters because it supplies a concrete mathematical procedure to decide which features of a signal are worth keeping for a given prediction task. The approach treats the tradeoff as a generalization of rate-distortion ideas in which the cost of error arises automatically from the observed relationship between X and Y.

What carries the argument

The bottleneck variable T, the compressed representation of X that is found by optimizing the tradeoff between the information lost in compression and the information retained about Y.

What would settle it

Running the re-estimation procedure on a dataset whose joint distribution p(x,y) is known exactly and finding that the resulting coding rules fail to satisfy the self-consistent equations or achieve the predicted levels of information preservation about Y.

Watch

Extended reading notes

Core claim

We define the relevant information in a signal x as the information it provides about y. We formalize the task of finding a short code for x that preserves the maximum information about y as squeezing that information through a bottleneck formed by a limited set of codewords t. This constrained optimization can be seen as a generalization of rate distortion theory in which the distortion measure emerges from the joint statistics of x and y. The variational principle yields an exact set of self-consistent equations for the coding rules from x to t and from t to y, which can be solved by a convergent re-estimation method that generalizes the Blahut-Arimoto algorithm.

Load-bearing premise

The joint distribution p(x,y) is known or can be estimated reliably from data so that the mutual information quantities can be computed exactly.

Editorial extensions

If this is right

  • The optimal coding rules X to T and T to Y are given by the fixed points of the self-consistent equations.
  • These equations are solved by an iterative re-estimation algorithm that converges to the solution.
  • The effective distortion measure in the equivalent rate-distortion problem is determined directly by the joint statistics p(x,y).
  • The same variational principle supplies a framework for analyzing problems in signal processing and learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • When the joint distribution must be estimated from finite samples, the method may need additional regularization to remain stable.
  • Choosing different target signals Y could turn the same optimization into a tool for supervised or semi-supervised feature extraction.
  • The framework suggests that clustering or dimensionality reduction can be performed by treating class labels or future observations as the Y variable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 3 minor

Summary. The paper defines the relevant information in a signal X about another signal Y as the information preserved through a compressed bottleneck representation T. It formalizes this as a constrained optimization problem maximizing I(T;Y) subject to a bound on I(X;T), shows that this is a generalization of rate-distortion theory in which the distortion measure emerges from the joint p(x,y), derives the exact self-consistent equations for the optimal mappings p(t|x) and p(y|t), and presents a convergent iterative re-estimation algorithm that generalizes the Blahut-Arimoto procedure.

Significance. If the central derivation holds, the work supplies a principled, parameter-light variational framework for relevance-preserving compression with direct applicability to signal processing and learning tasks. Its strengths include the clean derivation of the fixed-point equations from standard mutual-information identities and the Markov chain X–T–Y, the explicit generalization of rate-distortion theory, and the guarantee of monotonic improvement and convergence for finite alphabets.

minor comments (3)
  1. The abstract states that applications 'will be described in detail elsewhere'; a brief forward reference or one-sentence outline of the intended follow-up would improve self-contained readability.
  2. Notation for the bottleneck variable alternates between T and tX in the abstract; consistent use of a single symbol (e.g., T) throughout the manuscript would reduce minor confusion.
  3. The weakest assumption—that p(x,y) is known or reliably estimated—is stated clearly but could be highlighted with a short remark on practical estimation procedures in the main text.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive summary of our manuscript, the recognition of its strengths, and the recommendation to accept. The referee's description accurately captures the central contributions of the work.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; derivation is self-contained from mutual information definitions and variational calculus

full rationale

The paper's central derivation starts from the definitions of mutual information I(X;T) and I(T;Y) under the Markov chain X–T–Y, formulates the bottleneck as a constrained optimization problem, introduces a Lagrange multiplier for the I(X;T) term, and obtains the fixed-point equations via functional derivatives. These steps rely only on standard information-theoretic identities and calculus of variations; no parameters are fitted and then relabeled as predictions, no self-citations carry load-bearing uniqueness claims, and the generalization of rate-distortion theory is presented as an interpretive analogy rather than a renaming that substitutes for derivation. The iterative re-estimation procedure is shown to be a valid alternating optimization that monotonically decreases the functional, but this is a consequence of the variational setup rather than a circular reduction. The joint p(x,y) is an external input, matching the stated weakest assumption.

Assumptions & free parameters 1 free parameters · 2 assumptions · 1 invented entities

The framework rests on standard information-theoretic definitions plus one free trade-off parameter and the introduction of the bottleneck variable T.

free parameters (1)
  • beta
    Lagrange multiplier that trades off compression rate against preserved mutual information with Y; its value is chosen according to the desired operating point.
assumptions (2)
  • standard math Mutual information I(X;Y) = H(X) - H(X|Y) is the measure of relevance.
    Used to quantify both the compression cost and the preserved relevance.
  • domain assumption The mapping from X to T is a stochastic kernel p(t|x) that can be optimized independently of the downstream mapping from T to Y.
    Enables the Markov chain structure X–T–Y required for the bottleneck.
invented entities (1)
  • bottleneck variable T
    purpose: Compressed representation of X that retains maximal information about Y
    New random variable introduced to enforce the rate constraint; no external falsifiable prediction is supplied for T itself.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The information bottleneck method." pith.science (2026). https://pith.science/paper/C5KGNOKK

@misc{pith2026physics0004057,
  author       = {Pith},
  title        = {Pith review of: The information bottleneck method},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C5KGNOKK}},
  note         = {Machine review of arXiv:physics/0004057}
}
abstract

We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$. Examples include the information that face images provide about the names of the people portrayed, or the information that speech sounds provide about the words spoken. Understanding the signal $x$ requires more than just predicting $y$, it also requires specifying which features of $\X$ play a role in the prediction. We formalize this problem as that of finding a short code for $\X$ that preserves the maximum information about $\Y$. That is, we squeeze the information that $\X$ provides about $\Y$ through a `bottleneck' formed by a limited set of codewords $\tX$. This constrained optimization problem can be seen as a generalization of rate distortion theory in which the distortion measure $d(x,\x)$ emerges from the joint statistics of $\X$ and $\Y$. This approach yields an exact set of self consistent equations for the coding rules $X \to \tX$ and $\tX \to \Y$. Solutions to these equations can be found by a convergent re-estimation method that generalizes the Blahut-Arimoto algorithm. Our variational principle provides a surprisingly rich framework for discussing a variety of problems in signal processing and learning, as will be described in detail elsewhere.

Discussion (0). Continue with ORCID to comment.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • IndisputableMonolith.Cost.FunctionalEquation washburn_uniqueness_aczel echoes
    ?
    echoes

    ECHOES: this paper passage has the same mathematical shape or conceptual pattern as the Recognition theorem, but is not a direct formal dependency.

    This constrained optimization problem can be seen as a generalization of rate distortion theory in which the distortion measure d(x, x̃) emerges from the joint statistics of X and Y. This approach yields an exact set of self consistent equations for the coding rules X → X̃ and X̃ → Y.

  • IndisputableMonolith.Foundation.DAlembert.Inevitability bilinear_family_forced echoes
    ?
    echoes

    ECHOES: this paper passage has the same mathematical shape or conceptual pattern as the Recognition theorem, but is not a direct formal dependency.

    Our variational principle provides a surprisingly rich framework for discussing a variety of problems in signal processing and learning

  • IndisputableMonolith.Foundation.LawOfExistence defect_zero_iff_one echoes
    ?
    echoes

    ECHOES: this paper passage has the same mathematical shape or conceptual pattern as the Recognition theorem, but is not a direct formal dependency.

    the information that this signal provides about another signal y∈Y

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Showing 60 of 238 Pith papers that cite this

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 1,869 citations worldwide. See all 238 Pith citations

  1. Textual Belief States for World Models: Identifiable Representation Learning Under Strict Mediation

    cs.LG 2026-06 unverdicted novelty 8.0 of 10

    Introduces textual belief states and factorized GRPO to enforce strict latent state mediation in text-based world models, yielding preserved prediction accuracy with large gains in representation quality and rollout p...

  2. Learning 1-Bit LiDAR-based Localization with Auxiliary Objective

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    BiLoc is the first binary neural network framework for 6-DoF LiDAR pose estimation that uses an auxiliary objective to adaptively regulate information retention and achieve SOTA among BNNs on large outdoor datasets.

  3. S-JEPA : Soft Clustering Anchors for Self-Supervised Speech Representation Learning

    cs.SD 2026-06 unverdicted novelty 7.0 of 10

    S-JEPA uses soft GMM posteriors in a JEPA framework for self-supervised speech learning, achieving lowest WER below 90M parameters without offline re-clustering.

  4. In Defense of Information Leakage in Concept-based Models

    cs.LG 2026-06 conditional novelty 7.0 of 10

    Concept-based models can use controlled 'benign' information leakage to remain accurate and intervenable under real-world concept incompleteness by reframing their training objective.

  5. P$^2$-DPO: Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    P²-DPO generates on-policy preference pairs targeting focus-and-enhance perception and visual robustness, combined with a calibration loss, to reduce hallucinations in LVLMs more effectively than human-feedback baselines.

  6. A Fiber Criterion for Representation Identifiability in Supervised Learning

    cs.LG 2026-05 conditional novelty 7.0 of 10

    A representation property is identifiable from the induced predictor iff it is constant on the fibers of the map from admissible (representation, head) pairs to the composite predictor.

  7. Quantum Subliminal Learning

    quant-ph 2026-05 unverdicted novelty 7.0 of 10

    QNNs retain most hidden-task signals through public-task interfaces while classical networks transmit little, with transmission governed by teacher drift magnitude and the visible fraction of hidden drift in a unified...

  8. MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning

    cs.AI 2026-05 unverdicted novelty 7.0 of 10

    MuCRASP prunes VLMs in a CoT-aware manner, outperforming baselines by preserving reasoning quality at 30-50% compression rates on models like Qwen2.5-VL-7B.

  9. When Interpretability Becomes a Liability: Adversarial Attacks on CBM Concept Layers

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Concept-level adversarial attacks exploit CBM interpretability on the CUB dataset, but SPECTRA raises required perturbation norm from 0.46 to over 4200 while keeping accuracy loss under 2.2%.

  10. AffectVerse: Emotional World Models for Multimodal Affective Computing

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    AffectVerse improves multimodal emotion recognition by at least 2.57% on nine benchmarks through an Emotion World Module that performs short-horizon latent affective prediction via cross-modal temporal imagination and...

  11. Optimising Neural Speech Codecs for 300bps Communication using Reinforcement Learning

    cs.SD 2026-05 unverdicted novelty 7.0 of 10

    ClariCodec applies GRPO reinforcement learning to a 300 bps neural speech codec, using ASR word-error rate as reward to cut LibriSpeech test-clean WER from 4.64% to 3.55%.

  12. Entropy Across the Bridge: Conditional-Marginal Discretization for Flow and Schr\"odinger Samplers

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Derives a conditional-marginal entropy-rate objective for bridge-aware discretization that yields U-shaped schedules and improves low-NFE sample quality on 2D, CIFAR-10, and protein tasks.

  13. Decoupled and Divergence-Conditioned Prompt for Multi-domain Dynamic Graph Foundation Models

    cs.LG 2026-05 conditional novelty 7.0 of 10

    DyGFM introduces decoupled pre-training and divergence-conditioned prompts to create the first multi-domain dynamic graph foundation model that outperforms baselines on node classification and link prediction.

  14. On the Generalization of Knowledge Distillation: An Information-Theoretic View

    cs.IT 2026-05 unverdicted novelty 7.0 of 10

    Knowledge distillation generalization bounds are derived via a new distillation divergence measuring teacher-student kernel difference, with tighter bounds from teacher loss flatness.

  15. Lost and Found in Translation: Variational Diagnostics for Neural Codebook Channels

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Defines the neural codebook channel K_{e→d}(j|i) and proves a Bernoulli-KL bound on encoder-decoder mismatch in VAEs that cannot be recovered from marginal histograms or mutual information.

  16. JEDI: Joint Embedding Diffusion World Model for Online Model-Based Reinforcement Learning

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    JEDI is the first online end-to-end latent diffusion world model that trains latents from denoising loss rather than reconstruction, achieving competitive Atari100k results with 43% less VRAM and over 3x faster sampli...

  17. HEPA: A Self-Supervised Horizon-Conditioned Event Predictive Architecture for Time Series

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    HEPA pretrains via horizon-conditioned JEPA on unlabeled data then fine-tunes only the predictor for event survival CDFs, outperforming PatchTST, iTransformer, MAE and Chronos-2 on at least 10 of 14 benchmarks with fi...

  18. The Wittgensteinian Representation Hypothesis: Is Language the Attractor of Multimodal Convergence?

    cs.AI 2026-05 unverdicted novelty 7.0 of 10

    Language representations serve as the asymptotic attractor for convergence in independently trained multimodal neural networks due to feature density asymmetry.

  19. Neural Information Causality

    quant-ph 2026-05 unverdicted novelty 7.0 of 10

    Neural-IC separates embedding inequalities from capacity bounds in query-separated computations, with one-bit RAC benchmarks and CHSH-layer stability selecting the Tsirelson threshold for quantum enhancements.

  20. Privacy-Aware Video Anomaly Detection through Orthogonal Subspace Projection

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    A new orthogonal projection module for video anomaly detection suppresses facial attributes via weak face-presence signals and cosine alignment while preserving anomaly-relevant features like pose and motion.

  21. Skill-CMIB: Multimodal Agent Skill for Consistent Action via Conditional Multimodal Information Bottleneck

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    CMIB uses a conditional multimodal information bottleneck to create reusable agent skills that separate verbalizable text content from predictive perceptual residuals, improving execution stability.

  22. Factorization Regret mediates compositional generalization in latent space

    cs.LG 2026-03 unverdicted novelty 7.0 of 10

    Factorization Regret — the divergence between joint and naive Bayesian goal beliefs — tracks why reservoir networks plateau at naive accuracy, and a classifier-generator chain learns interactions from sparse reward to...

  23. PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence

    cs.AI 2026-03 conditional novelty 7.0 of 10

    PACED applies student pass-rate weighting w(p)=p(1-p) to distillation, concentrating on the zone of proximal development and delivering up to +8.2 gains on AIME tasks with reduced forgetting.

  24. Semantic Level of Detail for Knowledge Graphs: Discovering Abstraction Boundaries via Spectral Heat Diffusion

    cs.LG 2026-03 unverdicted novelty 7.0 of 10

    SLoD detects emergent scale boundaries in knowledge graphs by applying spectral heat diffusion to Poincare embeddings, recovering planted hierarchies in synthetic data and aligning with taxonomic depths in WordNet wit...

  25. From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents

    cs.CV 2026-03 unverdicted novelty 7.0 of 10

    MM-Mem distills video input through a hierarchical memory of sensory buffer, episodic stream, and symbolic schema, optimized by a semantic information bottleneck and SIB-GRPO, to achieve SOTA on long-horizon video benchmarks.

  26. Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework

    cs.CL 2026-02 unverdicted novelty 7.0 of 10

    Prune-then-Merge combines adaptive pruning of low-signal patches with hierarchical merging to achieve higher compression rates and better performance than prior single-stage methods in visual document retrieval.

  27. Perfect Privacy and Strong Stationary Times for Markovian Sources

    cs.IT 2026-01 unverdicted novelty 7.0 of 10

    For Markov sources, redaction up to strong stationary times achieves perfect privacy with optimal utility using constant average redactions independent of length.

  28. Semantic Identity Compression: Zero-Error Laws, Rate-Distortion, and Neurosymbolic Necessity

    cs.IT 2026-01 accept novelty 7.0 of 10 full

    Collision fiber sizes determine precise zero-error compression bounds and rate-distortion laws for semantic identity, establishing symbolic mechanisms as necessary complements to non-injective neural representations.

  29. Bandwidth-constrained Variational Message Encoding for Cooperative Multi-agent Reinforcement Learning

    cs.LG 2025-12 unverdicted novelty 7.0 of 10

    BVME uses variational Gaussian message encoding with KL regularization to maintain or improve multi-agent coordination performance while using 67-83% fewer message dimensions than naive compression on SMAC and MPE benchmarks.

  30. A Markov Categorical Framework for Language Modeling

    cs.LG 2025-07 unverdicted novelty 7.0 of 10

    A Markov category framework for language models provides an information-theoretic rationale for speculative decoding and shows that a quadratic surrogate to negative log-likelihood induces generalized CCA alignment in...

  31. PASS: Private Attributes Protection with Stochastic Data Substitution

    cs.LG 2025-06 conditional novelty 7.0 of 10

    PASS learns a stochastic substitution mapping that drives private-attribute inference to chance level across images, audio, and sensor data while keeping useful attributes mostly intact.

  32. Beyond Topological Self-Explainable GNNs: A Formal Explainability Perspective

    cs.LG 2025-02 accept novelty 7.0 of 10

    Self-explainable GNNs provably optimize minimal explanations that match prime implicants only for motif-based tasks, and a dual-channel extension recovers better rules.

  33. Nonasymptotic Oblivious Relaying and Variable-Length Noisy Lossy Source Coding

    cs.IT 2025-01 unverdicted novelty 7.0 of 10

    Establishes nonasymptotic achievability for the information bottleneck channel with fixed- and variable-length relaying and introduces a novel variable-length noisy lossy source coding bound.

  34. Structural Entropy Guided Probabilistic Coding

    cs.AI 2024-12 conditional novelty 7.0 of 10

    SEPC uses a maximizing-structural-entropy regularizer for probabilistic embeddings and a soft-label encoding tree for regression, reporting state-of-the-art results on 12 NLU tasks.

  35. Dream to Control: Learning Behaviors by Latent Imagination

    cs.LG 2019-12 accept novelty 7.0 of 10

    Dreamer learns to control from images by imagining and optimizing behaviors in a learned latent world model, outperforming prior methods on 20 visual tasks in data efficiency and final performance.

  36. A neuromorphic vision system for open-world visual intelligence

    eess.IV 2026-07 conditional novelty 6.5 of 10

    A polarization imager plus RRAM array implements a three-stage task-traction front end that reports ~193 μs task latency and large accuracy/latency gains over software vision baselines in eight hard open-world scenes.

  37. $\mathbf{\lambda}$-VAE: Variance Equalization for Posterior Collapse

    cs.LG 2026-07 conditional novelty 6.5 of 10

    Scaling VAE reparameterization noise by a per-dimension exponent while keeping KL on the original variance equalizes latent variances and reduces posterior collapse.

  38. Information Bottleneck under Perfect Privacy

    cs.IT 2026-08 conditional novelty 6.0 of 10

    A perturbed ADMM algorithm with convergence-rate guarantees is developed for finite-alphabet information bottleneck problems that impose exact independence from a sensitive variable.

  39. What should a linear optical frontend compute? Assessing the role of meta-optics, nonlocality, and coherence in hybrid inference systems

    physics.optics 2026-08 conditional novelty 6.0 of 10

    A linear optical frontend improves classification at a small sensor bottleneck through coherent, nonlocal field mixing, producing quadratic features that can outperform a trained linear preprocessor.

  40. Learning Latent Memory States from Longitudinal Athlete Monitoring Data

    stat.CO 2026-08 reject novelty 6.0 of 10

    The paper proposes the Latent Memory Table, a reusable summary of longitudinal histories, but its key empirical validation is undermined by an unfair baseline comparison.

  41. Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environments

    cs.RO 2026-08 conditional novelty 6.0 of 10

    Prior-SG uses an LLM-generated probabilistic prior graph and graph-cut inference to segment robot maps into task-relevant functional regions.

  42. Auditing Data Provenance in LLM Fine-tuning via Intrinsic Distributional Fingerprints

    cs.AI 2026-08 conditional novelty 6.0 of 10

    A black-box audit detects unauthorized fine-tuning by measuring a joint semantic-lexical distributional fingerprint in model outputs, robust to paraphrasing and distillation.

  43. Are Prompt Optimizers Blind? Cross-Modal Visual Feedback for Automatic Prompt Optimization

    cs.AI 2026-07 accept novelty 6.0 of 10

    Giving prompt optimizers failed images only at train time yields reusable text checklists that beat text-only APO by ~2.4 pp mean across 12 VQA datasets and 4 VLMs.

  44. Rethinking Layer-Wise Information Allocation for Vision Foundation Model Adaptation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    PIB regularizes prompt-tuned frozen ViTs with layer-wise compression and sufficiency losses, improving VTAB-1k transfer to 77.33% with about 0.35-0.51% trainable parameters.

  45. C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    C-PTQ weights quantization error by per-channel Fisher information of the task loss, improving low-bit accuracy of multimodal LLMs by small margins over existing channel-wise scaling methods.

  46. QuantiBias: Benchmarking Quantization-Induced Bias in LLMs

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Quantization leaves refusal and multiple-choice bias checks flat while open-ended stereotype endorsement remains high (~24–27% under an independent judge), a gap standard safety evaluations miss.

  47. ChannelGuard: Safe Models Do Not Compose into Safe Multi-Agent Systems

    cs.CR 2026-07 conditional novelty 6.0 of 10

    An undefended multi-agent pipeline's apparent perfect attack-safety is mostly the cloud provider's hidden filter, silently re-sourcing to model alignment on other backends; embedding gates restore deterministic applic...

  48. When Do Multi-Agent Systems Help? An Information Bottleneck Perspective

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Multi-agent LLM systems help exactly when the context removed by compressed relays outweighs the downstream-relevant information those relays discard.

  49. Presentation, Not Mechanism: A Render Confound in Deprecation-Aware Memory Evaluation

    cs.LG 2026-07 accept novelty 6.0 of 10

    A render-matched control explains away almost all (+0.159 of +0.184) of a fine-grained revision-ledger's apparent advantage over a flat baseline, leaving a near-zero mechanism residual and making coarse invalidation t...

  50. DAGR: State-Conditioned Goal Representations via Difference-Aware Goal Cross-Attention

    cs.LG 2026-07 conditional novelty 6.0 of 10

    State-conditioning an offline goal-conditioned agent's goal embedding via a near-identity gated residual improves navigation (Dual 28 to 82% on AntMaze-large), and the gain is carried by the gate, not the difference-a...

  51. Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction

    cs.AI 2026-07 conditional novelty 6.0 of 10

    Strict stage isolation that passes only a compressed symbolic schema and rule between LLM calls improves few-shot inductive reasoning more than self-refinement or explicit verbalization alone.

  52. Interpreting learning dynamics of autoencoders: Transient scaling and emerging concepts of the Ising model

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Unsupervised autoencoders on Ising configurations form magnetization then energy representations in two dynamical regimes, with recursive error flow fields sharing topology across layers.

  53. What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents

    cs.LG 2026-07 conditional novelty 6.0 of 10

    KV-cache eviction, prompt compression, recurrent state bounding, and agent memory consolidation are unified as one rate-distortion problem with a shared lower bound, shared failure mode, and transferable mechanisms.

  54. Steering Optimisation Trajectories in Diffusion Representation Learning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    SteeringDRL identifies two optimization regimes in diffusion autoencoders and uses gated residual U-Nets with a log SNR curriculum to steer training toward disentangled representations, improving performance across mu...

  55. Thermodynamics of Quantum Reservoir Computing

    quant-ph 2026-07 unverdicted novelty 6.0 of 10

    Continuous temporal processing in open quantum reservoirs is shown to obey a generalized Landauer bound, with predictive performance tied to resonant energy-gap matching and to quantum coherence.

  56. ScaleAware-JEPA: Latent Representation for Discovery in Multiscale Physical Fields

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    ScaleAware-JEPA combines Constrained Diffusion Decomposition with a scale-tied JEPA objective to learn label-free latent coordinates that recover coherent morphology in multiscale fields such as MHD turbulence and int...

  57. Explaining Temporal Graph Neural Networks via Feature-induced Information Flow

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Event Relevance, built on modularized Normalized Relevance Measure, attributes ETGNN predictions to events by quantifying the entire event-induced information flow, including messages, and outperforms prior explainers...

  58. Semi-Supervised Vision-Language-Action Model

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    SemiVLA improves VLA adaptation under 10% labeled trajectories via self-distilled pseudo-actions, reaching 89% success on LIBERO with OpenVLA backbone.

  59. Tri-Info: Generalizable, Interpretable Failure Prediction for VLA Models via Information Theory

    cs.RO 2026-06 unverdicted novelty 6.0 of 10

    Tri-Info uses three information theory signals on action diversity, temporal consistency, and state coupling to predict VLA model failures with cross-domain generalization to 83% real-world accuracy.

  60. LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    Latent reasoning through eight bottleneck tokens gives multimodal retrieval performance close to explicit chain-of-thought models at a fraction of the inference cost.

See all 238 Pith citations

Reference graph

Works this paper leans on

6 extracted references · 6 canonical work pages · cited by 238 Pith papers (see all)

  1. [1]

    Extracting relevant informati on,

    W. Bialek and N. Tishby, “Extracting relevant informati on,” in prepara- tion. 15

    work page

  2. [2]

    T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley, New York, 1991)

    work page 1991

  3. [3]

    Information geometry and alternating mini- mization procedures,

    I. Csisz´ ar and G. Tusn´ ady, “Information geometry and alternating mini- mization procedures,” Statistics and Decisions Suppl. 1, 205–237 (1984)

    work page 1984

  4. [4]

    Computation of channel capacity and rate d istortion func- tion,

    R. E. Blahut, “Computation of channel capacity and rate d istortion func- tion,” IEEE Trans. Inform. Theory IT-18, 460–473 (1972)

    work page 1972

  5. [5]

    Agglomerative information bot tleneck,

    N. Slonim and N. Tishby, “Agglomerative information bot tleneck,” To appear in Advances in Neural Information Processing systems (NIPS-1 2) 1999

    work page 1999

  6. [6]

    Distributional clu stering of En- glish words,

    F. C. Pereira, N. Tishby, and L. Lee, “Distributional clu stering of En- glish words,” in 30th Annual Mtg. of the Association for Computational Linguistics, pp. 183–190 (1993). 16

    work page 1993

Pith tools

Reviewed May 11, 2026 · model on record in the stance chip above.