{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2026:QOPZW2B2YFYM2D6YOCSNSC5YUO","short_pith_number":"pith:QOPZW2B2","schema_version":"1.0","canonical_sha256":"839f9b683ac170cd0fd870a4d90bb8a3a891d73cca8c323b9994b23576dabaee","source":{"kind":"arxiv","id":"2601.07372","version":1},"attestation_state":"computed","paper":{"title":"Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Engram introduces conditional memory as a new sparsity axis that lets large language models perform direct O(1) knowledge lookups instead of computing retrieval.","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Bingxuan Wang, Damai Dai, Dongyan Zhao, Han Zhang, Huishuai Zhang, Kezhao Huang, Qinyu Chen, Wangding Zeng, Wenfeng Liang, Xin Cheng, Xingkai Yu, Yukun Li, Zhenda Xie, Zhewen Hao","submitted_at":"2026-01-12T09:54:49Z","abstract_excerpt":"While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation. To address this, we introduce conditional memory as a complementary sparsity axis, instantiated via Engram, a module that modernizes classic $N$-gram embedding for O(1) lookup. By formulating the Sparsity Allocation problem, we uncover a U-shaped scaling law that optimizes the trade-off between neural computation (MoE) and static memory (Engram). Guided by this law, we scale Engram to 27B par"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":true},"canonical_record":{"source":{"id":"2601.07372","kind":"arxiv","version":1},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CL","submitted_at":"2026-01-12T09:54:49Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"f356246ba4b44007608f772ef51593fefac15a455acd4eda5c5b20e0d67de9a2","abstract_canon_sha256":"d190569143980c41c63739ea5ab93edafb302492a5bf8dd424afdb714a6040b2"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-17T23:38:49.049202Z","signature_b64":"t4RVKKFyaXX+eMXfD20S3/DBu7wi2LxxZuTVN8tadK5jrYX+wUALzG9oOnqL8YMiRAhHFhPrP1RYVZyiDA5EAA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"839f9b683ac170cd0fd870a4d90bb8a3a891d73cca8c323b9994b23576dabaee","last_reissued_at":"2026-05-17T23:38:49.048549Z","signature_status":"signed_v1","first_computed_at":"2026-05-17T23:38:49.048549Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Engram introduces conditional memory as a new sparsity axis that lets large language models perform direct O(1) knowledge lookups instead of computing retrieval.","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Bingxuan Wang, Damai Dai, Dongyan Zhao, Han Zhang, Huishuai Zhang, Kezhao Huang, Qinyu Chen, Wangding Zeng, Wenfeng Liang, Xin Cheng, Xingkai Yu, Yukun Li, Zhenda Xie, Zhewen Hao","submitted_at":"2026-01-12T09:54:49Z","abstract_excerpt":"While Mixture-of-Experts (MoE) scales capacity via conditional computation, Transformers lack a native primitive for knowledge lookup, forcing them to inefficiently simulate retrieval through computation. To address this, we introduce conditional memory as a complementary sparsity axis, instantiated via Engram, a module that modernizes classic $N$-gram embedding for O(1) lookup. By formulating the Sparsity Allocation problem, we uncover a U-shaped scaling law that optimizes the trade-off between neural computation (MoE) and static memory (Engram). Guided by this law, we scale Engram to 27B par"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Scaling Engram to 27B parameters achieves superior performance over a strictly iso-parameter and iso-FLOPs MoE baseline, with notable gains in reasoning (BBH +5.0, ARC-Challenge +3.7) and long-context retrieval (Multi-Query NIAH: 84.2 to 97.0).","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The U-shaped scaling law for sparsity allocation between MoE computation and Engram memory generalizes beyond the tested model sizes and tasks, and the observed mechanistic benefits (relieving early layers, freeing attention) are causally due to the memory module rather than confounding factors in the experimental setup.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Engram adds conditional memory via scalable lookup to LLMs, outperforming iso-parameter MoE baselines on reasoning and long-context tasks by following a U-shaped scaling law for allocating between computation and memory.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Engram introduces conditional memory as a new sparsity axis that lets large language models perform direct O(1) knowledge lookups instead of computing retrieval.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"a145764756cd9bd88e0cffbe98c0fde785ed2b626bc47c3eb146e930678aa54f"},"source":{"id":"2601.07372","kind":"arxiv","version":1},"verdict":{"id":"769f63b4-a10e-4d8a-bec5-7c7cab3fbd4e","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-16T04:50:55.810210Z","strongest_claim":"Scaling Engram to 27B parameters achieves superior performance over a strictly iso-parameter and iso-FLOPs MoE baseline, with notable gains in reasoning (BBH +5.0, ARC-Challenge +3.7) and long-context retrieval (Multi-Query NIAH: 84.2 to 97.0).","one_line_summary":"Engram adds conditional memory via scalable lookup to LLMs, outperforming iso-parameter MoE baselines on reasoning and long-context tasks by following a U-shaped scaling law for allocating between computation and memory.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The U-shaped scaling law for sparsity allocation between MoE computation and Engram memory generalizes beyond the tested model sizes and tasks, and the observed mechanistic benefits (relieving early layers, freeing attention) are causally due to the memory module rather than confounding factors in the experimental setup.","pith_extraction_headline":"Engram introduces conditional memory as a new sparsity axis that lets large language models perform direct O(1) knowledge lookups instead of computing retrieval."},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"52a4a16841deb854fa05fa6ad541d62379d1bb2674cd11ab0c6baf8ac188b567"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2601.07372","created_at":"2026-05-17T23:38:49.048634+00:00"},{"alias_kind":"arxiv_version","alias_value":"2601.07372v1","created_at":"2026-05-17T23:38:49.048634+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2601.07372","created_at":"2026-05-17T23:38:49.048634+00:00"},{"alias_kind":"pith_short_12","alias_value":"QOPZW2B2YFYM","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"QOPZW2B2YFYM2D6Y","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"QOPZW2B2","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":29,"internal_anchor_count":29,"sample":[{"citing_arxiv_id":"2604.08216","citing_title":"MemCoT: Test-Time Scaling through Memory-Driven Chain-of-Thought","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18848","citing_title":"Exact Linear Attention","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2605.20309","citing_title":"Tiny-Engram: Trigger-Indexed Concept Tables for Generative Vision","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2605.20948","citing_title":"Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2605.20613","citing_title":"HRM-Text: Efficient Pretraining Beyond Scaling","ref_index":60,"is_internal_anchor":true},{"citing_arxiv_id":"2605.21463","citing_title":"Mem-$\\pi$: Adaptive Memory through Learning When and What to Generate","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18848","citing_title":"Exact Linear Attention","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2605.18556","citing_title":"Key-Gram: Extensible World Knowledge for Embodied Manipulation","ref_index":23,"is_internal_anchor":true},{"citing_arxiv_id":"2605.16893","citing_title":"NGM: A Plug-and-Play Training-Free Memory Module for LLMs","ref_index":8,"is_internal_anchor":true},{"citing_arxiv_id":"2603.23516","citing_title":"MSA: Memory Sparse Attention for Efficient End-to-End Memory Model Scaling to 100M Tokens","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2604.27263","citing_title":"Decoupling the Benefits of Subword Tokenization for Language Model Training via Byte-level Simulation","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2603.25414","citing_title":"Decidable By Construction: Design-Time Verification for Trustworthy AI","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2603.28342","citing_title":"Kernel-Smith: A Unified Recipe for Evolutionary Kernel Optimization","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2605.13179","citing_title":"Does Engram Do Memory Retrieval in Autoregressive Image Generation?","ref_index":2,"is_internal_anchor":true},{"citing_arxiv_id":"2605.13370","citing_title":"Phasor Memory Networks: Stable Backpropagation Through Time for Scalable Explicit Memory","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2605.12426","citing_title":"Geometric Factual Recall in Transformers","ref_index":51,"is_internal_anchor":true},{"citing_arxiv_id":"2605.11447","citing_title":"Conditional Memory Enhanced Item Representation for Generative Recommendation","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2604.27263","citing_title":"Decoupling the Benefits of Subword Tokenization for Language Model Training via Byte-level Simulation","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2605.10739","citing_title":"Geospatial-Temporal Sensemaking of Remote Sensing Activity Detections with Multimodal Large Language Model","ref_index":36,"is_internal_anchor":true},{"citing_arxiv_id":"2605.06615","citing_title":"When and Why SignSGD Outperforms SGD: A Theoretical Study Based on $\\ell_1$-norm Lower Bounds","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2605.04400","citing_title":"Contextual Memory-Enhanced Source Coding for Low-SNR Communications","ref_index":19,"is_internal_anchor":true},{"citing_arxiv_id":"2605.00814","citing_title":"Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs","ref_index":14,"is_internal_anchor":true},{"citing_arxiv_id":"2604.08224","citing_title":"Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering","ref_index":22,"is_internal_anchor":true},{"citing_arxiv_id":"2604.08519","citing_title":"Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts","ref_index":17,"is_internal_anchor":true},{"citing_arxiv_id":"2604.08216","citing_title":"MemCoT: Test-Time Scaling through Memory-Driven Chain-of-Thought","ref_index":7,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":2,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/QOPZW2B2YFYM2D6YOCSNSC5YUO","json":"https://pith.science/pith/QOPZW2B2YFYM2D6YOCSNSC5YUO.json","graph_json":"https://pith.science/api/pith-number/QOPZW2B2YFYM2D6YOCSNSC5YUO/graph.json","events_json":"https://pith.science/api/pith-number/QOPZW2B2YFYM2D6YOCSNSC5YUO/events.json","paper":"https://pith.science/paper/QOPZW2B2"},"agent_actions":{"view_html":"https://pith.science/pith/QOPZW2B2YFYM2D6YOCSNSC5YUO","download_json":"https://pith.science/pith/QOPZW2B2YFYM2D6YOCSNSC5YUO.json","view_paper":"https://pith.science/paper/QOPZW2B2","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2601.07372&json=true","fetch_graph":"https://pith.science/api/pith-number/QOPZW2B2YFYM2D6YOCSNSC5YUO/graph.json","fetch_events":"https://pith.science/api/pith-number/QOPZW2B2YFYM2D6YOCSNSC5YUO/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/QOPZW2B2YFYM2D6YOCSNSC5YUO/action/timestamp_anchor","attest_storage":"https://pith.science/pith/QOPZW2B2YFYM2D6YOCSNSC5YUO/action/storage_attestation","attest_author":"https://pith.science/pith/QOPZW2B2YFYM2D6YOCSNSC5YUO/action/author_attestation","sign_citation":"https://pith.science/pith/QOPZW2B2YFYM2D6YOCSNSC5YUO/action/citation_signature","submit_replication":"https://pith.science/pith/QOPZW2B2YFYM2D6YOCSNSC5YUO/action/replication_record"}},"created_at":"2026-05-17T23:38:49.048634+00:00","updated_at":"2026-05-17T23:38:49.048634+00:00"}