Mistletoe introduces a stealthy attack on speculative decoding that collapses acceleration by reducing average accepted length while preserving output semantics.
hub Mixed citations
Alphaedit: Null-space constrained knowledge editing for language models
Mixed citation behavior. Most common role is background (67%).
25
Pith papers citing it
Background
67% of classified citations
abstract
Large language models (LLMs) often exhibit hallucinations due to incorrect or outdated knowledge. Hence, model editing methods have emerged to enable targeted knowledge updates. To achieve this, a prevailing paradigm is the locating-then-editing approach, which first locates influential parameters and then edits them by introducing a perturbation. While effective, current studies have demonstrated that this perturbation inevitably disrupt the originally preserved knowledge within LLMs, especially in sequential editing scenarios. To address this, we introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters. We theoretically prove that this projection ensures the output of post-edited LLMs remains unchanged when queried about the preserved knowledge, thereby mitigating the issue of disruption. Extensive experiments on various LLMs, including LLaMA3, GPT2-XL, and GPT-J, show that AlphaEdit boosts the performance of most locating-then-editing methods by an average of 36.7% with a single line of additional code for projection solely. Our code is available at: https://github.com/jianghoucheng/AlphaEdit.
hub tools
citation-role summary
background 4
baseline 1
method 1
citation-polarity summary
representative citing papers
Muon outperforms Adam by reducing curvature penalty via lower Normalized Directional Sharpness, as shown via Taylor approximation on LLM training and proven on stylized quadratic problems with heterogeneous curvature.
Introduces Latent Adversarial Robustification and Rank-Constrained Subspace Learning to enable robust generalization in multimodal knowledge editing through adversarial subspace alignment.
GenRecEdit injects cold-start items into generative recommendation models via context-aware token editing and interference-reducing triggers, boosting cold-start accuracy while using only 9.5% of retraining time.
Norm-Anchor Scaling breaks the norm-feedback loop in sequential LLM editing by anchoring value vectors to original norms, improving long-run performance by 72.2% and extending the editing horizon over 4x.
DECODE identifies and separately edits modality-specific neurons in MLLMs to prevent knowledge edits from reverting under unimodal queries.
LOKI enables memory-free lifelong knowledge editing via HSIC-based dynamic layer selection and null-space constrained updates, reporting up to 14% higher average accuracy than prior methods.
AnyEdit++ proposes Bayes-Chunk, an adaptive segmentation method based on Bayesian Surprise, with theoretical claims of structural independence and causal locality, reporting superior results over baselines on math, code, and narrative tasks.
MemGraphRAG uses a memory-based multi-agent system for globally consistent graph construction from fragmented corpora plus a memory-aware hierarchical retriever, claiming better benchmark performance than prior GraphRAG methods at similar cost.
A MEMIT-style knowledge editing framework for MoE LLMs that formulates per-expert updates via tensor structure and applies Woodbury identity for low-rank inversions, achieving up to 6x speedup with comparable editing quality.
EchoDistill applies noisy-to-clean self-distillation with GRPO to boost Audio LLM robustness, reporting 4.18% average GSR gains under strong noise.
A contrastive visual forgetting technique constrained to the null space of retained knowledge enables targeted unlearning of visual concepts in MLLMs while preserving non-target visual and all textual knowledge.
CURaTE performs continual unlearning in LLMs in real time by using sentence embeddings to detect and refuse forget requests without changing model parameters, achieving effective forgetting and perfect knowledge preservation.
Rule knowledge in LLMs is localized by form across layers; a distributed multi-layer editing method improves instance portability by 13.91 and rule understanding by 50.19 percentage points over baselines on multiple models.
FlashEdit delivers real-time localized text-guided image editing under 0.2 seconds via cycle-consistent one-step inversion, background shield, and sparsified spatial cross-attention, achieving over 150x speedup on PIE-Bench.
MemoryAgentBench is a multi-turn benchmark covering four memory competencies, and current memory agents fail at selective forgetting and long-range understanding.
AlphaEdit's null-space editing guarantee holds only within its original models and ~3k-edit regime; it fails on Phi-3/Gemma-2 and degrades after ~5k edits while harming downstream and safety metrics.
Iterative self-improving codebooks enhance safety in autoregressive multimodal models by self-identifying unsafe generations and updating the codebook to eliminate harmful visual token mappings without external feedback.
Introduces 5WBENCH balanced benchmark across 5W categories and MAAT three-phase adapter unlearning method that targets causal Why-type knowledge.
M2A uses null-space model merging to combine mathematical and agentic reasoning in LLMs, raising SWE-Bench Verified performance from 44.0% to 51.2% on Qwen3-8B without retraining.
HoReN is a parameter-preserving editor that wraps an MLP with a Hopfield codebook memory and scales to 50K sequential edits on ZsRE while maintaining performance above 0.93.
LightEdit enables scalable lifelong knowledge editing in LLMs via selective knowledge retrieval and probability suppression during decoding, outperforming prior methods on ZSRE, Counterfact, and RIPE while reducing training costs.
A neuron-targeted safety tuning method for VLLMs reduces attack success rates from ~20-30% to ~4-6% on average across ten languages while using less than 0.03% of parameters.
MemOS introduces a unified memory management framework for LLMs using MemCubes to handle and evolve different memory types for improved controllability and evolvability.
citing papers explorer
-
Mistletoe: Stealthy Acceleration-Collapse Attacks on Speculative Decoding
Mistletoe introduces a stealthy attack on speculative decoding that collapses acceleration by reducing average accepted length while preserving output semantics.
-
Why Muon Outperforms Adam: A Curvature Perspective
Muon outperforms Adam by reducing curvature penalty via lower Normalized Directional Sharpness, as shown via Taylor approximation on LLM training and proven on stylized quadratic problems with heterogeneous curvature.
-
Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment
Introduces Latent Adversarial Robustification and Rank-Constrained Subspace Learning to enable robust generalization in multimodal knowledge editing through adversarial subspace alignment.
-
GenRecEdit: Adapting Model Editing for Generative Recommendation with Cold-Start Items
GenRecEdit injects cold-start items into generative recommendation models via context-aware token editing and interference-reducing triggers, boosting cold-start accuracy while using only 9.5% of retraining time.
-
Norm Anchors Make Model Edits Last
Norm-Anchor Scaling breaks the norm-feedback loop in sequential LLM editing by anchoring value vectors to original norms, improving long-run performance by 72.2% and extending the editing horizon over 4x.
-
Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs
DECODE identifies and separately edits modality-specific neurons in MLLMs to prevent knowledge edits from reverting under unimodal queries.
-
LOKI: Memory-Free Null-Space Constrained Lifelong Knowledge Editing
LOKI enables memory-free lifelong knowledge editing via HSIC-based dynamic layer selection and null-space constrained updates, reporting up to 14% higher average accuracy than prior methods.
-
AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise
AnyEdit++ proposes Bayes-Chunk, an adaptive segmentation method based on Bayesian Surprise, with theoretical claims of structural independence and causal locality, reporting superior results over baselines on math, code, and narrative tasks.
-
MemGraphRAG: Memory-based Multi-Agent System for Graph Retrieval-Augmented Generation
MemGraphRAG uses a memory-based multi-agent system for globally consistent graph construction from fragmented corpora plus a memory-aware hierarchical retriever, claiming better benchmark performance than prior GraphRAG methods at similar cost.
-
Scalable Knowledge Editing for Mixture-of-Experts LLMs via Tensor-Structured Updates
A MEMIT-style knowledge editing framework for MoE LLMs that formulates per-expert updates via tensor structure and applies Woodbury identity for low-rank inversions, achieving up to 6x speedup with comparable editing quality.
-
EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs
EchoDistill applies noisy-to-clean self-distillation with GRPO to boost Audio LLM robustness, reporting 4.18% average GSR gains under strong noise.
-
Null Space Constrained Contrastive Visual Forgetting for MLLM Unlearning
A contrastive visual forgetting technique constrained to the null space of retained knowledge enables targeted unlearning of visual concepts in MLLMs while preserving non-target visual and all textual knowledge.
-
CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge
CURaTE performs continual unlearning in LLMs in real time by using sentence embeddings to detect and refuse forget requests without changing model parameters, achieving effective forgetting and perfect knowledge preservation.
-
Distributed Multi-Layer Editing for Rule-Level Knowledge in Large Language Models
Rule knowledge in LLMs is localized by form across layers; a distributed multi-layer editing method improves instance portability by 13.91 and rule understanding by 50.19 percentage points over baselines on multiple models.
-
FlashEdit: Decoupling Speed, Structure, and Semantics for Precise Image Editing
FlashEdit delivers real-time localized text-guided image editing under 0.2 seconds via cycle-consistent one-step inversion, background shield, and sparsified spatial cross-attention, achieving over 150x speedup on PIE-Bench.
-
Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
MemoryAgentBench is a multi-turn benchmark covering four memory competencies, and current memory agents fail at selective forgetting and long-range understanding.
-
Reproducibility Study of "AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models"
AlphaEdit's null-space editing guarantee holds only within its original models and ~3k-edit regime; it fails on Phi-3/Gemma-2 and degrades after ~5k edits while harming downstream and safety metrics.
-
Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks
Iterative self-improving codebooks enhance safety in autoregressive multimodal models by self-identifying unsafe generations and updating the codebook to eliminate harmful visual token mappings without external feedback.
-
MAAT: Multi-phase Adapter-Aware Targeted Unlearning
Introduces 5WBENCH balanced benchmark across 5W categories and MAAT three-phase adapter unlearning method that targets causal Why-type knowledge.
-
M2A: Synergizing Mathematical and Agentic Reasoning in Large Language Models
M2A uses null-space model merging to combine mathematical and agentic reasoning in LLMs, raising SWE-Bench Verified performance from 44.0% to 51.2% on Qwen3-8B without retraining.
-
HoReN: Normalized Hopfield Retrieval for Large-Scale Sequential Model Editing
HoReN is a parameter-preserving editor that wraps an MLP with a Hopfield codebook memory and scales to 50K sequential edits on ZsRE while maintaining performance above 0.93.
-
Towards Scalable Lifelong Knowledge Editing with Selective Knowledge Suppression
LightEdit enables scalable lifelong knowledge editing in LLMs via selective knowledge retrieval and probability suppression during decoding, outperforming prior methods on ZSRE, Counterfact, and RIPE while reducing training costs.
-
Targeted Interpretable Safety Neuron Enhancement for Multilingual Vision-Language Large Models
A neuron-targeted safety tuning method for VLLMs reduces attack success rates from ~20-30% to ~4-6% on average across ten languages while using less than 0.03% of parameters.
-
MemOS: A Memory OS for AI System
MemOS introduces a unified memory management framework for LLMs using MemCubes to handle and evolve different memory types for improved controllability and evolvability.
-
Low-Rank Adaptation Redux for Large Models
An overview revisits LoRA variants by categorizing advances in architectural design, efficient optimization, and applications while linking them to classical signal processing tools for principled fine-tuning.