{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2026:5GDYLWOL7NGIY3SRLTSKHFM63X","short_pith_number":"pith:5GDYLWOL","schema_version":"1.0","canonical_sha256":"e98785d9cbfb4c8c6e515ce4a3959eddff2175548d26e17b1e05dfdf0e48e916","source":{"kind":"arxiv","id":"2605.12843","version":1},"attestation_state":"computed","paper":{"title":"Bayesian Model Merging","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Bayesian Model Merging fuses task-specific models into one via inner Bayesian regression under anchor priors and outer Bayesian optimization of per-module hyperparameters.","cross_cats":["cs.AI"],"primary_cat":"cs.LG","authors_text":"Kaiyang Li, Qing Su, Shaobo Han, Shihao Ji","submitted_at":"2026-05-13T00:36:47Z","abstract_excerpt":"Model merging aims to combine multiple task-specific expert models into a single model without joint retraining, offering a practical alternative to multi-task learning when data access or computational budget is limited. Existing methods, however, face two key limitations: (1) they overlook the valuable inductive bias of strong anchor models and estimate the merged weights from scratch, and (2) they rely on a shared hyperparameter setting across different modules of the network, lacking a global optimization strategy. This paper introduces Bayesian Model Merging (BMM), a plug-and-play bi-leve"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":true},"canonical_record":{"source":{"id":"2605.12843","kind":"arxiv","version":1},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.LG","submitted_at":"2026-05-13T00:36:47Z","cross_cats_sorted":["cs.AI"],"title_canon_sha256":"7d511b61ff1cd74c70bf56c33d21e3963dd51d07833af05b906bc12ee40a9b8d","abstract_canon_sha256":"c31255ea55fe59d514aec810f8bb685f645b85e58b45ffc7ddd80805d91a6a11"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-05-18T03:09:11.934739Z","signature_b64":"2tKE5AB/YqjJ1TVADBoc4/sYutUCWoeGpfnokQY+czKQi8QZNXkbw8kjUMR1q8iKiwPBUrSkl/SWdjYe5memCA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"e98785d9cbfb4c8c6e515ce4a3959eddff2175548d26e17b1e05dfdf0e48e916","last_reissued_at":"2026-05-18T03:09:11.934066Z","signature_status":"signed_v1","first_computed_at":"2026-05-18T03:09:11.934066Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Bayesian Model Merging","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Bayesian Model Merging fuses task-specific models into one via inner Bayesian regression under anchor priors and outer Bayesian optimization of per-module hyperparameters.","cross_cats":["cs.AI"],"primary_cat":"cs.LG","authors_text":"Kaiyang Li, Qing Su, Shaobo Han, Shihao Ji","submitted_at":"2026-05-13T00:36:47Z","abstract_excerpt":"Model merging aims to combine multiple task-specific expert models into a single model without joint retraining, offering a practical alternative to multi-task learning when data access or computational budget is limited. Existing methods, however, face two key limitations: (1) they overlook the valuable inductive bias of strong anchor models and estimate the merged weights from scratch, and (2) they rely on a shared hyperparameter setting across different modules of the network, lacking a global optimization strategy. This paper introduces Bayesian Model Merging (BMM), a plug-and-play bi-leve"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Across extensive benchmarks, including up to 20-task merging in vision and 5-task merging in language, BMM consistently outperforms all plug-and-play anchor baselines (e.g., TA, WUDI-Merging, and TSV). In particular, on the ViT-L/14 benchmark for 8-task merging, a single merged model reaches 95.1, closely matching the average performance of eight task-specific experts (95.8).","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The claimed alignment between activation statistics and task vectors that enables the data-free Gram-matrix estimation, together with the assumption that the inner-level Bayesian regression under the anchor prior produces a solution that generalizes without hidden post-hoc adjustments.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Bayesian Model Merging introduces a bi-level optimization framework that merges task-specific models via closed-form Bayesian regression with an anchor prior and global hyperparameter search, outperforming baselines and nearly matching expert averages on up to 20-task vision and 5-task language Merg","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Bayesian Model Merging fuses task-specific models into one via inner Bayesian regression under anchor priors and outer Bayesian optimization of per-module hyperparameters.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"ab7c2ad3c1596973df5628ea20ea381dffce5e00899de90a989efa72921045f4"},"source":{"id":"2605.12843","kind":"arxiv","version":1},"verdict":{"id":"755c71a3-bfc8-40b5-9114-45252ba5e03b","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-14T20:38:04.112438Z","strongest_claim":"Across extensive benchmarks, including up to 20-task merging in vision and 5-task merging in language, BMM consistently outperforms all plug-and-play anchor baselines (e.g., TA, WUDI-Merging, and TSV). In particular, on the ViT-L/14 benchmark for 8-task merging, a single merged model reaches 95.1, closely matching the average performance of eight task-specific experts (95.8).","one_line_summary":"Bayesian Model Merging introduces a bi-level optimization framework that merges task-specific models via closed-form Bayesian regression with an anchor prior and global hyperparameter search, outperforming baselines and nearly matching expert averages on up to 20-task vision and 5-task language Merg","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The claimed alignment between activation statistics and task vectors that enables the data-free Gram-matrix estimation, together with the assumption that the inner-level Bayesian regression under the anchor prior produces a solution that generalizes without hidden post-hoc adjustments.","pith_extraction_headline":"Bayesian Model Merging fuses task-specific models into one via inner Bayesian regression under anchor priors and outer Bayesian optimization of per-module hyperparameters."},"references":{"count":66,"sample":[{"doi":"","year":2017,"title":"An Overview of Multi-Task Learning in Deep Neural Networks","work_id":"2c849184-06ce-40df-8143-ec29eb925f39","ref_index":1,"cited_arxiv_id":"1706.05098","is_internal_anchor":true},{"doi":"","year":2022,"title":"Matena and Colin A","work_id":"f85bb7a2-7dff-4a41-aa1a-a2013fc51ef4","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"Editing models with task arithmetic","work_id":"a4f3855c-b66f-4915-8b5b-581e494a8cb5","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"Raffel, and Mohit Bansal","work_id":"ac02f012-a7c9-4546-a950-364c16bc8719","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2026,"title":"The hugging face hub","work_id":"051718a0-462f-4455-962a-aa2930577451","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":66,"snapshot_sha256":"507badcd941774027e9060eec0a645288c2ed7293cb430fdf90147a4c0ac252a","internal_anchors":11},"formal_canon":{"evidence_count":1,"snapshot_sha256":"c3a047cd8172fed4c4a0b52f3b10ba64430131dd805daceefaa009725d2019bf"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2605.12843","created_at":"2026-05-18T03:09:11.934167+00:00"},{"alias_kind":"arxiv_version","alias_value":"2605.12843v1","created_at":"2026-05-18T03:09:11.934167+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2605.12843","created_at":"2026-05-18T03:09:11.934167+00:00"},{"alias_kind":"pith_short_12","alias_value":"5GDYLWOL7NGI","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_16","alias_value":"5GDYLWOL7NGIY3SR","created_at":"2026-05-18T12:33:37.589309+00:00"},{"alias_kind":"pith_short_8","alias_value":"5GDYLWOL","created_at":"2026-05-18T12:33:37.589309+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":0,"internal_anchor_count":0,"sample":[]},"formal_canon":{"evidence_count":1,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/5GDYLWOL7NGIY3SRLTSKHFM63X","json":"https://pith.science/pith/5GDYLWOL7NGIY3SRLTSKHFM63X.json","graph_json":"https://pith.science/api/pith-number/5GDYLWOL7NGIY3SRLTSKHFM63X/graph.json","events_json":"https://pith.science/api/pith-number/5GDYLWOL7NGIY3SRLTSKHFM63X/events.json","paper":"https://pith.science/paper/5GDYLWOL"},"agent_actions":{"view_html":"https://pith.science/pith/5GDYLWOL7NGIY3SRLTSKHFM63X","download_json":"https://pith.science/pith/5GDYLWOL7NGIY3SRLTSKHFM63X.json","view_paper":"https://pith.science/paper/5GDYLWOL","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2605.12843&json=true","fetch_graph":"https://pith.science/api/pith-number/5GDYLWOL7NGIY3SRLTSKHFM63X/graph.json","fetch_events":"https://pith.science/api/pith-number/5GDYLWOL7NGIY3SRLTSKHFM63X/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/5GDYLWOL7NGIY3SRLTSKHFM63X/action/timestamp_anchor","attest_storage":"https://pith.science/pith/5GDYLWOL7NGIY3SRLTSKHFM63X/action/storage_attestation","attest_author":"https://pith.science/pith/5GDYLWOL7NGIY3SRLTSKHFM63X/action/author_attestation","sign_citation":"https://pith.science/pith/5GDYLWOL7NGIY3SRLTSKHFM63X/action/citation_signature","submit_replication":"https://pith.science/pith/5GDYLWOL7NGIY3SRLTSKHFM63X/action/replication_record"}},"created_at":"2026-05-18T03:09:11.934167+00:00","updated_at":"2026-05-18T03:09:11.934167+00:00"}