{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2024:CKQSJSYXKAYGDX6EIU3EIOSV2E","short_pith_number":"pith:CKQSJSYX","schema_version":"1.0","canonical_sha256":"12a124cb17503061dfc44536443a55d12e762122b897a676bc828a2646543932","source":{"kind":"arxiv","id":"2401.06121","version":1},"attestation_state":"computed","paper":{"title":"TOFU: A Task of Fictitious Unlearning for LLMs","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Unlearning methods for large language models fail to make them behave as if specific training data was never seen.","cross_cats":["cs.CL"],"primary_cat":"cs.LG","authors_text":"Avi Schwarzschild, J. Zico Kolter, Pratyush Maini, Zachary C. Lipton, Zhili Feng","submitted_at":"2024-01-11T18:57:12Z","abstract_excerpt":"Large language models trained on massive corpora of data from the web can memorize and reproduce sensitive or private data raising both legal and ethical concerns. Unlearning, or tuning models to forget information present in their training data, provides us with a way to protect private data after training. Although several methods exist for such unlearning, it is unclear to what extent they result in models equivalent to those where the data to be forgotten was never learned in the first place. To address this challenge, we present TOFU, a Task of Fictitious Unlearning, as a benchmark aimed "},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2401.06121","kind":"arxiv","version":1},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.LG","submitted_at":"2024-01-11T18:57:12Z","cross_cats_sorted":["cs.CL"],"title_canon_sha256":"edb1708c592326bd601188ead943b4eab018c8639fed0b61984ea5e117188084","abstract_canon_sha256":"2483e31e651042a2060498bdf47b4980cad60624fef2c3c2d56e03d2cb15ce13"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:48.074970Z","signature_b64":"MKA/crtRZwidg5Vfirl0xQcuyJxNqTw5g2APt8KwSKgxrpaMRsW8da4kWvi75eLt3qQNSqtYzdgCYSLAyGZiCw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"12a124cb17503061dfc44536443a55d12e762122b897a676bc828a2646543932","last_reissued_at":"2026-05-17T23:38:48.074436Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:48.074436Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"TOFU: A Task of Fictitious Unlearning for LLMs","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Unlearning methods for large language models fail to make them behave as if specific training data was never seen.","cross_cats":["cs.CL"],"primary_cat":"cs.LG","authors_text":"Avi Schwarzschild, J. Zico Kolter, Pratyush Maini, Zachary C. Lipton, Zhili Feng","submitted_at":"2024-01-11T18:57:12Z","abstract_excerpt":"Large language models trained on massive corpora of data from the web can memorize and reproduce sensitive or private data raising both legal and ethical concerns. Unlearning, or tuning models to forget information present in their training data, provides us with a way to protect private data after training. Although several methods exist for such unlearning, it is unclear to what extent they result in models equivalent to those where the data to be forgotten was never learned in the first place. To address this challenge, we present TOFU, a Task of Fictitious Unlearning, as a benchmark aimed "},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Importantly, none of the baselines we consider show effective unlearning motivating continued efforts to develop approaches for unlearning that effectively tune models so that they truly behave as if they were never trained on the forget data at all.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The assumption that results on synthetic fictitious author profiles will generalize to the difficulty of unlearning real sensitive information from actual large-scale training corpora.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"TOFU is a new benchmark with synthetic profiles and metrics demonstrating that existing unlearning algorithms for LLMs fail to achieve effective forgetting of targeted information.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Unlearning methods for large language models fail to make them behave as if specific training data was never seen.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"50c9f0eeff59d3ad6db1367425e43caeabfb4a4a28c4e8b1708a0fd3f81d14f5"},"source":{"id":"2401.06121","kind":"arxiv","version":1},"verdict":{"id":"f24f607f-f42e-49bb-919f-ba54b0e6714d","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-16T11:03:31.010641Z","strongest_claim":"Importantly, none of the baselines we consider show effective unlearning motivating continued efforts to develop approaches for unlearning that effectively tune models so that they truly behave as if they were never trained on the forget data at all.","one_line_summary":"TOFU is a new benchmark with synthetic profiles and metrics demonstrating that existing unlearning algorithms for LLMs fail to achieve effective forgetting of targeted information.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The assumption that results on synthetic fictitious author profiles will generalize to the difficulty of unlearning real sensitive information from actual large-scale training corpora.","pith_extraction_headline":"Unlearning methods for large language models fail to make them behave as if specific training data was never seen."},"references":{"count":44,"sample":[{"doi":"","year":2021,"title":"Machine unlearning","work_id":"4d425956-f000-49d7-9d36-97aa726d7a61","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2021,"title":"Extracting training data from large language models","work_id":"ed991696-818d-409a-87e0-a4c7da18320b","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2022,"title":"Membership inference attacks from first principles","work_id":"0ffe7056-ca0e-4fcf-81a6-073d0c49bd83","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"Unlearn what you want to forget: Efficient unlearning for llms, 2023","work_id":"74bd5caf-472e-4e1e-8c53-267458d855fa","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"10.3115/v1/w14-4012","year":2014,"title":"On the properties of neural machine translation: Encoder-decoder approaches","work_id":"62cd99e9-1990-4d03-bfe7-00e9b69af44b","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":44,"snapshot_sha256":"86b2914ab211ef364f30265387a40383fb0c056c921ac98548406c2fe46ed22d","internal_anchors":6},"formal_canon":{"evidence_count":2,"snapshot_sha256":"d0cee415386baf0819dbc07dd821e319a7c65dab5dc2d54a9a14bf217d426316"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2401.06121","created_at":"2026-05-17T23:38:48.074541+00:00"},{"alias_kind":"arxiv_version","alias_value":"2401.06121v1","created_at":"2026-05-17T23:38:48.074541+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2401.06121","created_at":"2026-05-17T23:38:48.074541+00:00"},{"alias_kind":"pith_short_12","alias_value":"CKQSJSYXKAYG","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"CKQSJSYXKAYGDX6E","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"CKQSJSYX","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":34,"internal_anchor_count":34,"sample":[{"citing_arxiv_id":"2605.18879","citing_title":"ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2605.20915","citing_title":"Calibration vs Decision Making: Revisiting the Reliability Paradox in Unlearned Language Models","ref_index":56,"is_internal_anchor":true},{"citing_arxiv_id":"2605.12765","citing_title":"Inference-Time Machine Unlearning via Gated Activation Redirection","ref_index":1,"is_internal_anchor":true},{"citing_arxiv_id":"2605.15687","citing_title":"ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18879","citing_title":"ZeroUnlearn: Few-Shot Knowledge Unlearning in Large Language Models","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2605.17373","citing_title":"FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics","ref_index":32,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18891","citing_title":"Auditing Reasoning-Trace Memorization Claims after Unlearning with Head-Conditioned Canaries","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18253","citing_title":"Machine Unlearning for Masked Diffusion Language Models","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2605.20005","citing_title":"Fine-Tuning Without Forgetting via Loss-Adaptive Learning Rates","ref_index":43,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16746","citing_title":"State Contamination in Memory-Augmented LLM Agents","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2506.14387","citing_title":"SEAT: Sparse Entity-Aware Tuning for Knowledge Adaptation while Preserving Epistemic Abstention","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2506.20941","citing_title":"Revisiting the Past: Data Unlearning with Model State History","ref_index":27,"is_internal_anchor":true},{"citing_arxiv_id":"2509.22483","citing_title":"OFMU: Optimization-Driven Framework for Machine Unlearning","ref_index":15,"is_internal_anchor":true},{"citing_arxiv_id":"2510.00761","citing_title":"Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearning","ref_index":21,"is_internal_anchor":true},{"citing_arxiv_id":"2404.05868","citing_title":"Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning","ref_index":15,"is_internal_anchor":true},{"citing_arxiv_id":"2601.02631","citing_title":"Copyright Laundering Through the AI Ouroboros: Adapting the 'Fruit of the Poisonous Tree' Doctrine to Recursive AI Training","ref_index":1,"is_internal_anchor":true},{"citing_arxiv_id":"2602.23798","citing_title":"MPU: Towards Secure and Privacy-Preserving Knowledge Unlearning for Large Language Models","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2605.14404","citing_title":"Knowledge Beyond Language: Bridging the Gap in Multilingual Machine Unlearning Evaluation","ref_index":12,"is_internal_anchor":true},{"citing_arxiv_id":"2605.14514","citing_title":"Defenses at Odds: Measuring and Explaining Defense Conflicts in Large Language Models","ref_index":31,"is_internal_anchor":true},{"citing_arxiv_id":"2605.12765","citing_title":"Inference-Time Machine Unlearning via Gated Activation Redirection","ref_index":1,"is_internal_anchor":true},{"citing_arxiv_id":"2605.12705","citing_title":"Early Data Exposure Improves Robustness to Subsequent Fine-Tuning","ref_index":10,"is_internal_anchor":true},{"citing_arxiv_id":"2604.03114","citing_title":"Can VLMs Truly Forget? Benchmarking Training-Free Visual Concept Unlearning","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2605.08800","citing_title":"PPU-Bench:Real World Benchmark for Personalized Partial Unlearning in Vision Language Models","ref_index":21,"is_internal_anchor":true},{"citing_arxiv_id":"2605.03547","citing_title":"Erase Persona, Forget Lore: Benchmarking Multimodal Copyright Unlearning in Large Vision Language Models","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2605.05938","citing_title":"ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models","ref_index":30,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/CKQSJSYXKAYGDX6EIU3EIOSV2E","json":"https://pith.science/pith/CKQSJSYXKAYGDX6EIU3EIOSV2E.json","graph_json":"https://pith.science/api/pith-number/CKQSJSYXKAYGDX6EIU3EIOSV2E/graph.json","events_json":"https://pith.science/api/pith-number/CKQSJSYXKAYGDX6EIU3EIOSV2E/events.json","paper":"https://pith.science/paper/CKQSJSYX"},"agent_actions":{"view_html":"https://pith.science/pith/CKQSJSYXKAYGDX6EIU3EIOSV2E","download_json":"https://pith.science/pith/CKQSJSYXKAYGDX6EIU3EIOSV2E.json","view_paper":"https://pith.science/paper/CKQSJSYX","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2401.06121&json=true","fetch_graph":"https://pith.science/api/pith-number/CKQSJSYXKAYGDX6EIU3EIOSV2E/graph.json","fetch_events":"https://pith.science/api/pith-number/CKQSJSYXKAYGDX6EIU3EIOSV2E/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/CKQSJSYXKAYGDX6EIU3EIOSV2E/action/timestamp_anchor","attest_storage":"https://pith.science/pith/CKQSJSYXKAYGDX6EIU3EIOSV2E/action/storage_attestation","attest_author":"https://pith.science/pith/CKQSJSYXKAYGDX6EIU3EIOSV2E/action/author_attestation","sign_citation":"https://pith.science/pith/CKQSJSYXKAYGDX6EIU3EIOSV2E/action/citation_signature","submit_replication":"https://pith.science/pith/CKQSJSYXKAYGDX6EIU3EIOSV2E/action/replication_record"}},"created_at":"2026-05-17T23:38:48.074541+00:00","updated_at":"2026-05-17T23:38:48.074541+00:00"}