Pith. sign in

REVIEW 5 cited by

Understanding Catastrophic Forgetting in Language Models via Implicit Inference

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2309.10105 v2 pith:FDWBSYFX submitted 2023-09-18 cs.CL cs.LG

Suhas Kotha , Jacob Mitchell Springer , Aditi Raghunathan This is my paper
classification cs.CLcs.LG
keywords fine-tuningdistributiontaskscapabilitiescapabilityconjugateinferencelanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We lack a systematic understanding of the effects of fine-tuning (via methods such as instruction-tuning or reinforcement learning from human feedback), particularly on tasks outside the narrow fine-tuning distribution. In a simplified scenario, we demonstrate that improving performance on tasks within the fine-tuning data distribution comes at the expense of capabilities on other tasks. We hypothesize that language models implicitly infer the task of the prompt and that fine-tuning skews this inference towards tasks in the fine-tuning distribution. To test this, we propose Conjugate Prompting, which artificially makes the task look farther from the fine-tuning distribution while requiring the same capability, and we find that this recovers some of the pretraining capabilities in our synthetic setup. Since real-world fine-tuning distributions are predominantly English, we apply conjugate prompting to recover pretrained capabilities in LLMs by simply translating the prompts to different languages. This allows us to recover in-context learning abilities lost via instruction tuning, natural reasoning capability lost during code fine-tuning, and, more concerningly, harmful content generation suppressed by safety fine-tuning in chatbots like ChatGPT.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Helpfulness Hurts: Domain-Dependent Degradation of Mid-Trained Compassion Values Under Post-Training

    cs.CL 2026-04 unverdicted novelty 6.0 of 10

    Helpfulness SFT/GRPO degrades mid-trained animal compassion far more than coding SFT/GRPO on ANIMA, with the compassion effect transferring cross-lingually while general moral-reasoning gains do not.

  2. Transformers Don't In-Context Learn Least Squares Regression

    cs.LG 2025-07 conditional novelty 6.0 of 10

    In-context regression transformers do not approximate OLS: they underperform it even in-distribution, fail on out-of-subspace prompts, and their failures correlate with a low-rank spectral signature in the residual stream.

  3. Scaling Point-in-Time Language Models

    cs.CL 2026-04 conditional novelty 5.5 of 10

    Scaling point-in-time LLMs to 4B parameters and 1T temporally filtered tokens narrows the gap to unrestricted models to about 8–11 average points and yields positive out-of-sample Sharpe ratios from news embeddings.

  4. SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling

    cs.LG 2026-06 conditional novelty 5.0 of 10

    A LoRA update split into several fixed, differently-scaled low-rank experts with orthogonal input directions improves fine-tuning accuracy at the same parameter count.

  5. Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting

    cs.LG 2025-10 conditional novelty 5.0 of 10

    RL post-training forgets less than SFT because it trains on on-policy data; refreshing SFT data each epoch also reduces forgetting.

Pith tools