{"work":{"id":"41b4c732-3a63-4106-9e7c-50d61cfb2126","openalex_id":"https://openalex.org/W4390962868","doi":"10.48550/arxiv.2401.08281","arxiv_id":"2401.08281","raw_key":null,"title":"The Faiss library","authors":null,"authors_text":"Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazar\\'e","year":2024,"venue":"cs.LG","abstract":"Vector databases typically manage large collections of embedding vectors. Currently, AI applications are growing rapidly, and so is the number of embeddings that need to be stored and indexed. The Faiss library is dedicated to vector similarity search, a core functionality of vector databases. Faiss is a toolkit of indexing methods and related primitives used to search, cluster, compress and transform vectors. This paper describes the trade-off space of vector search and the design principles of Faiss in terms of structure, approach to optimization and interfacing. We benchmark key features of the library and discuss a few selected applications to highlight its broad applicability.","external_url":"https://arxiv.org/abs/2401.08281","cited_by_count":65,"metadata_source":"pith","metadata_fetched_at":"2026-08-05T02:28:24.338817+00:00","pith_arxiv_id":"2401.08281","created_at":"2026-05-09T05:50:28.221627+00:00","updated_at":"2026-08-05T02:28:24.338817+00:00","title_quality_ok":false,"display_title":"The Faiss library","render_title":"The Faiss library"},"hub":{"state":{"work_id":"41b4c732-3a63-4106-9e7c-50d61cfb2126","tier":"super_hub","tier_reason":"100+ Pith inbound or 10,000+ external citations","pith_inbound_count":102,"external_cited_by_count":65,"distinct_field_count":15,"first_pith_cited_at":"2024-07-18T06:06:53+00:00","last_pith_cited_at":"2026-07-08T17:59:45+00:00","author_build_status":"needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-08-23T02:29:35.226079+00:00","tier_text":"super_hub"},"tier":"super_hub","role_counts":[{"context_role":"background","n":11},{"context_role":"method","n":10},{"context_role":"baseline","n":1},{"context_role":"other","n":1}],"polarity_counts":[{"context_polarity":"background","n":11},{"context_polarity":"use_method","n":10},{"context_polarity":"baseline","n":1},{"context_polarity":"unclear","n":1}],"runs":{"ask_index":{"job_type":"ask_index","status":"succeeded","result":{"title":"The Faiss library","claims":[{"claim_text":"Vector databases typically manage large collections of embedding vectors. Currently, AI applications are growing rapidly, and so is the number of embeddings that need to be stored and indexed. The Faiss library is dedicated to vector similarity search, a core functionality of vector databases. Faiss is a toolkit of indexing methods and related primitives used to search, cluster, compress and transform vectors. This paper describes the trade-off space of vector search and the design principles of Faiss in terms of structure, approach to optimization and interfacing. We benchmark key features of","claim_type":"abstract","evidence_strength":"source_metadata"},{"claim_text":"We conduct experiments on three datasets: Natural Questions (NQ) [23], HotpotQA [56], and MSMARCO [3]. We chunk each document into smaller text blocks, treating each block as a single record to be embedded and stored in the vector database. For each dataset, we sample 100,000 vectors to populate the database. 6.1.3 Vector Database Settings.We employ FAISS [ 6] as our vector database backend and evaluate our approach using four distinct index types: (1) Flat (exact brute-force search), (2) graph-","claim_type":"method","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"Additional models are considered in the supplementary material (Section A.3). For each image and text sample, we extract the representations from all layers of their respective encoders and follow the experimental protocol from [40]. Details about additional models used in Section 4 are provided in Section E.3.2 in the supplementary material. We use Faiss [17] for nearest neighbor computation at scale. Specifically, we use their exact nearest neighbor implementation with IndexFlatL2 which is equ","claim_type":"method","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"conversational agents with persona-driven user simulations based on large language models: A sales bot case study. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 230-245, Suzhou (China), November 2025. Association for Computational Linguistics. ISBN 979-8-89176-333-3. doi: 10.18653/v1/ 2025.emnlp-industry.16. URL https://aclanthology.org/2025.emnlp-industry.16/. [13] Zhiyuan Hu, Yue Feng, Anh Tuan Luu, Bryan Hooi, and Aldo Lipani. ","claim_type":"other","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"Fabbri, Patrick Ng, Zhiguo Wang, Ramesh Nallapati, and Bing Xiang. 2020. Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) . 4508-4513. [37] Facebook. 2013. RocksDB. https://github.com/facebook/rocksdb [38] Angela Fan and Claire Gardent. 2022. Generating Full Length Wikipedia Bi- ographies: The Impact of Gender Bias on the Retrieval-Based Genera","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"pairs can reach large Hamming distances, overlapping the range of unrelated images and precluding a reliable threshold (see Fig. 4). SSCD near-duplicate detectionTo handle near-duplicates, we rely on Self-Supervised Copy Detection (SSCD) [66]. We compute 512-d SSCD embeddings with the public sscd_disc_mixup model [67] and retrieve the k= 64 nearest neighbors per image using a FAISS index [18]; k= 64 trades off search speed against cluster recall, and is large enough to cover the maximum near-dup","claim_type":"method","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"index node as a (soft or hard) cluster of items, then we can model the item-index mapping as the hidden variables and user and item embedding as observation variables, such that the problem can be approximated as a Gaussian mixture model. Then, the E-step can be assigning each item its closest cluster and can be implemented by the GPU-enabled FAISS [8, 24] parallel cluster- ing library, e.g., K-Means; and the M-step can be training the foundation model according to current index embeddings. This","claim_type":"method","confidence":0.9,"evidence_strength":"citation_context"}],"why_cited":"Pith tracks The Faiss library because it crossed a citation-hub threshold. Current citing contexts most often use it as background evidence (11 contexts).","role_counts":[{"n":11,"context_role":"background"},{"n":10,"context_role":"method"},{"n":1,"context_role":"baseline"},{"n":1,"context_role":"other"}]},"error":null,"updated_at":"2026-07-04T14:36:56.029432+00:00"},"author_expand":{"job_type":"author_expand","status":"succeeded","result":{"authors_linked":[{"id":"96e2176b-e87e-4d35-a6ca-9c873f56bca3","orcid":null,"display_name":"Matthijs Douze"},{"id":"d91193cf-2dc0-43d1-8ad3-af310429798b","orcid":null,"display_name":"Alexandr Guzhva"},{"id":"a0831d7e-3d25-45df-a070-33fa58a339ac","orcid":null,"display_name":"Chengqi Deng"},{"id":"8d4383cd-e16e-4415-ac3a-1fff1481ead4","orcid":null,"display_name":"Jeff Johnson"},{"id":"6cae09ad-6999-46c2-b0f5-3d6119ef8dd0","orcid":null,"display_name":"Gergely Szilvasy"},{"id":"31d00841-52d1-4c8c-b98a-26afa01a6a0f","orcid":null,"display_name":"Pierre-Emmanuel Mazar\\'e"}]},"error":null,"updated_at":"2026-07-04T14:36:56.027029+00:00"},"context_extract":{"job_type":"context_extract","status":"succeeded","result":{"enqueued_papers":25},"error":null,"updated_at":"2026-05-14T17:48:47.702983+00:00"},"graph_features":{"job_type":"graph_features","status":"succeeded","result":{"co_cited":[{"title":"Qwen3 Technical Report","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","shared_citers":8},{"title":"Dense passage retrieval for open-domain question answering","work_id":"083391f8-812d-430f-8d08-89a03031ce6c","shared_citers":5},{"title":"doi: 10.18653/v1/2022.naacl-main.272","work_id":"bfa57bc7-9275-4c55-a608-477c2f68ca23","shared_citers":4},{"title":"doi: 10.18653/v1/ 2024.findings-acl.586","work_id":"8d675bdd-79ca-48d6-9163-fc17ce0e8ece","shared_citers":4},{"title":"OpenAI GPT-5 System Card","work_id":"ca87689a-0d29-4476-b504-b65dbbb08af4","shared_citers":4},{"title":"Qwen3-VL Technical Report","work_id":"1fe243aa-e3c0-4da6-b391-4cbcfc88d5c0","shared_citers":4},{"title":"The Llama 3 Herd of Models","work_id":"1549a635-88af-4ac1-acfe-51ae7bb53345","shared_citers":4},{"title":"BERT : Pre-training of deep bidirectional transformers for language understanding","work_id":"3e3c8ac8-b858-4b22-af32-393d98c883e0","shared_citers":3},{"title":"Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory","work_id":"a5aed26c-a248-48b6-a59e-f7693fcb180a","shared_citers":3},{"title":"Mistral 7B","work_id":"eb5e1305-ad11-4875-ad8d-ad8b8f697599","shared_citers":3},{"title":"MS MARCO: A Human Generated MAchine Reading COmprehension Dataset","work_id":"78d498ce-11db-4f88-8eb0-40e0f86af615","shared_citers":3},{"title":"Passage Re-ranking with BERT","work_id":"562fbfab-d6fe-48e1-a06d-e5d078c70945","shared_citers":3},{"title":"Qwen Technical Report","work_id":"bb1fd52f-6b2f-437c-9516-37bdf6eb9be8","shared_citers":3},{"title":"Retrieval-Augmented Generation for Large Language Models: A Survey","work_id":"b80d2790-6cd9-4c87-b3c4-de404f99a80e","shared_citers":3},{"title":"The probabilistic relevance framework: BM25 and beyond.Foundations and Trends in Information Retrieval, 3(4):333–389","work_id":"3dfaa21d-3751-420b-84f7-aeceda058b63","shared_citers":3},{"title":"Towards General Text Embeddings with Multi-stage Contrastive Learning","work_id":"861a61de-66fe-49d1-b1ab-11f8b082a4cc","shared_citers":3},{"title":"Zep: A Temporal Knowledge Graph Architecture for Agent Memory","work_id":"515c933e-12ae-439d-a7ff-c07fee482dfb","shared_citers":3},{"title":"2021 , url =","work_id":"f6bab950-d25e-4dc8-bd18-7262dc50b62a","shared_citers":2},{"title":"Adam: A Method for Stochastic Optimization","work_id":"1910796d-9b52-4683-bf5c-de9632c1028b","shared_citers":2},{"title":"A-MEM: Agentic Memory for LLM Agents","work_id":"3b98feb2-fdb1-479a-bbe4-2c298a4592e2","shared_citers":2},{"title":"BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models","work_id":"c5f7f027-ac36-4b07-b824-0eca2f310641","shared_citers":2},{"title":"Bernstein, Badrish Chan- dramouli, Richard Wen, and Harsha Vardhan Simhadri","work_id":"5df76726-c438-4944-8487-1967e1ecdd6a","shared_citers":2},{"title":"Datasets: A community library for natural language processing","work_id":"f8e5e705-9ef6-4c07-a011-66da7b3af48f","shared_citers":2},{"title":"Decoupled Weight Decay Regularization","work_id":"07ef7360-d385-4033-83f7-8384a6325204","shared_citers":2}],"time_series":[{"n":41,"year":2026}],"dependency_candidates":[]},"error":null,"updated_at":"2026-05-14T17:49:15.009743+00:00"},"identity_refresh":{"job_type":"identity_refresh","status":"succeeded","result":{"items":[{"title":"Qwen3 Technical Report","outcome":"unchanged","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","resolver":"local_arxiv","confidence":0.98,"old_work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e"}],"counts":{"fixed":0,"merged":0,"unchanged":1,"quarantined":0,"needs_external_resolution":0},"errors":[],"attempted":1},"error":null,"updated_at":"2026-05-14T17:49:18.117102+00:00"},"role_polarity":{"job_type":"role_polarity","status":"succeeded","result":{"title":"The Faiss library","claims":[{"claim_text":"Vector databases typically manage large collections of embedding vectors. Currently, AI applications are growing rapidly, and so is the number of embeddings that need to be stored and indexed. The Faiss library is dedicated to vector similarity search, a core functionality of vector databases. Faiss is a toolkit of indexing methods and related primitives used to search, cluster, compress and transform vectors. This paper describes the trade-off space of vector search and the design principles of Faiss in terms of structure, approach to optimization and interfacing. We benchmark key features of","claim_type":"abstract","evidence_strength":"source_metadata"},{"claim_text":"We conduct experiments on three datasets: Natural Questions (NQ) [23], HotpotQA [56], and MSMARCO [3]. We chunk each document into smaller text blocks, treating each block as a single record to be embedded and stored in the vector database. For each dataset, we sample 100,000 vectors to populate the database. 6.1.3 Vector Database Settings.We employ FAISS [ 6] as our vector database backend and evaluate our approach using four distinct index types: (1) Flat (exact brute-force search), (2) graph-","claim_type":"method","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"Additional models are considered in the supplementary material (Section A.3). For each image and text sample, we extract the representations from all layers of their respective encoders and follow the experimental protocol from [40]. Details about additional models used in Section 4 are provided in Section E.3.2 in the supplementary material. We use Faiss [17] for nearest neighbor computation at scale. Specifically, we use their exact nearest neighbor implementation with IndexFlatL2 which is equ","claim_type":"method","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"conversational agents with persona-driven user simulations based on large language models: A sales bot case study. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 230-245, Suzhou (China), November 2025. Association for Computational Linguistics. ISBN 979-8-89176-333-3. doi: 10.18653/v1/ 2025.emnlp-industry.16. URL https://aclanthology.org/2025.emnlp-industry.16/. [13] Zhiyuan Hu, Yue Feng, Anh Tuan Luu, Bryan Hooi, and Aldo Lipani. ","claim_type":"other","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"Fabbri, Patrick Ng, Zhiguo Wang, Ramesh Nallapati, and Bing Xiang. 2020. Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) . 4508-4513. [37] Facebook. 2013. RocksDB. https://github.com/facebook/rocksdb [38] Angela Fan and Claire Gardent. 2022. Generating Full Length Wikipedia Bi- ographies: The Impact of Gender Bias on the Retrieval-Based Genera","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"pairs can reach large Hamming distances, overlapping the range of unrelated images and precluding a reliable threshold (see Fig. 4). SSCD near-duplicate detectionTo handle near-duplicates, we rely on Self-Supervised Copy Detection (SSCD) [66]. We compute 512-d SSCD embeddings with the public sscd_disc_mixup model [67] and retrieve the k= 64 nearest neighbors per image using a FAISS index [18]; k= 64 trades off search speed against cluster recall, and is large enough to cover the maximum near-dup","claim_type":"method","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"index node as a (soft or hard) cluster of items, then we can model the item-index mapping as the hidden variables and user and item embedding as observation variables, such that the problem can be approximated as a Gaussian mixture model. Then, the E-step can be assigning each item its closest cluster and can be implemented by the GPU-enabled FAISS [8, 24] parallel cluster- ing library, e.g., K-Means; and the M-step can be training the foundation model according to current index embeddings. This","claim_type":"method","confidence":0.9,"evidence_strength":"citation_context"}],"why_cited":"Pith tracks The Faiss library because it crossed a citation-hub threshold. Current citing contexts most often use it as background evidence (11 contexts).","role_counts":[{"n":11,"context_role":"background"},{"n":10,"context_role":"method"},{"n":1,"context_role":"baseline"},{"n":1,"context_role":"other"}]},"error":null,"updated_at":"2026-07-04T14:36:55.317367+00:00"},"summary_claims":{"job_type":"summary_claims","status":"succeeded","result":{"title":"The Faiss library","claims":[{"claim_text":"Vector databases typically manage large collections of embedding vectors. Currently, AI applications are growing rapidly, and so is the number of embeddings that need to be stored and indexed. The Faiss library is dedicated to vector similarity search, a core functionality of vector databases. Faiss is a toolkit of indexing methods and related primitives used to search, cluster, compress and transform vectors. This paper describes the trade-off space of vector search and the design principles of Faiss in terms of structure, approach to optimization and interfacing. We benchmark key features of","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks The Faiss library because it crossed a citation-hub threshold.","role_counts":[]},"error":null,"updated_at":"2026-05-14T17:49:00.353326+00:00"}},"summary":{"title":"The Faiss library","claims":[{"claim_text":"Vector databases typically manage large collections of embedding vectors. Currently, AI applications are growing rapidly, and so is the number of embeddings that need to be stored and indexed. The Faiss library is dedicated to vector similarity search, a core functionality of vector databases. Faiss is a toolkit of indexing methods and related primitives used to search, cluster, compress and transform vectors. This paper describes the trade-off space of vector search and the design principles of Faiss in terms of structure, approach to optimization and interfacing. We benchmark key features of","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks The Faiss library because it crossed a citation-hub threshold.","role_counts":[]},"graph":{"co_cited":[{"title":"Qwen3 Technical Report","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","shared_citers":8},{"title":"Dense passage retrieval for open-domain question answering","work_id":"083391f8-812d-430f-8d08-89a03031ce6c","shared_citers":5},{"title":"doi: 10.18653/v1/2022.naacl-main.272","work_id":"bfa57bc7-9275-4c55-a608-477c2f68ca23","shared_citers":4},{"title":"doi: 10.18653/v1/ 2024.findings-acl.586","work_id":"8d675bdd-79ca-48d6-9163-fc17ce0e8ece","shared_citers":4},{"title":"OpenAI GPT-5 System Card","work_id":"ca87689a-0d29-4476-b504-b65dbbb08af4","shared_citers":4},{"title":"Qwen3-VL Technical Report","work_id":"1fe243aa-e3c0-4da6-b391-4cbcfc88d5c0","shared_citers":4},{"title":"The Llama 3 Herd of Models","work_id":"1549a635-88af-4ac1-acfe-51ae7bb53345","shared_citers":4},{"title":"BERT : Pre-training of deep bidirectional transformers for language understanding","work_id":"3e3c8ac8-b858-4b22-af32-393d98c883e0","shared_citers":3},{"title":"Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory","work_id":"a5aed26c-a248-48b6-a59e-f7693fcb180a","shared_citers":3},{"title":"Mistral 7B","work_id":"eb5e1305-ad11-4875-ad8d-ad8b8f697599","shared_citers":3},{"title":"MS MARCO: A Human Generated MAchine Reading COmprehension Dataset","work_id":"78d498ce-11db-4f88-8eb0-40e0f86af615","shared_citers":3},{"title":"Passage Re-ranking with BERT","work_id":"562fbfab-d6fe-48e1-a06d-e5d078c70945","shared_citers":3},{"title":"Qwen Technical Report","work_id":"bb1fd52f-6b2f-437c-9516-37bdf6eb9be8","shared_citers":3},{"title":"Retrieval-Augmented Generation for Large Language Models: A Survey","work_id":"b80d2790-6cd9-4c87-b3c4-de404f99a80e","shared_citers":3},{"title":"The probabilistic relevance framework: BM25 and beyond.Foundations and Trends in Information Retrieval, 3(4):333–389","work_id":"3dfaa21d-3751-420b-84f7-aeceda058b63","shared_citers":3},{"title":"Towards General Text Embeddings with Multi-stage Contrastive Learning","work_id":"861a61de-66fe-49d1-b1ab-11f8b082a4cc","shared_citers":3},{"title":"Zep: A Temporal Knowledge Graph Architecture for Agent Memory","work_id":"515c933e-12ae-439d-a7ff-c07fee482dfb","shared_citers":3},{"title":"2021 , url =","work_id":"f6bab950-d25e-4dc8-bd18-7262dc50b62a","shared_citers":2},{"title":"Adam: A Method for Stochastic Optimization","work_id":"1910796d-9b52-4683-bf5c-de9632c1028b","shared_citers":2},{"title":"A-MEM: Agentic Memory for LLM Agents","work_id":"3b98feb2-fdb1-479a-bbe4-2c298a4592e2","shared_citers":2},{"title":"BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models","work_id":"c5f7f027-ac36-4b07-b824-0eca2f310641","shared_citers":2},{"title":"Bernstein, Badrish Chan- dramouli, Richard Wen, and Harsha Vardhan Simhadri","work_id":"5df76726-c438-4944-8487-1967e1ecdd6a","shared_citers":2},{"title":"Datasets: A community library for natural language processing","work_id":"f8e5e705-9ef6-4c07-a011-66da7b3af48f","shared_citers":2},{"title":"Decoupled Weight Decay Regularization","work_id":"07ef7360-d385-4033-83f7-8384a6325204","shared_citers":2}],"time_series":[{"n":41,"year":2026}],"dependency_candidates":[]},"authors":[{"id":"d91193cf-2dc0-43d1-8ad3-af310429798b","orcid":null,"display_name":"Alexandr Guzhva","source":"manual","import_confidence":0.72},{"id":"a0831d7e-3d25-45df-a070-33fa58a339ac","orcid":null,"display_name":"Chengqi Deng","source":"manual","import_confidence":0.72},{"id":"6cae09ad-6999-46c2-b0f5-3d6119ef8dd0","orcid":null,"display_name":"Gergely Szilvasy","source":"manual","import_confidence":0.72},{"id":"8d4383cd-e16e-4415-ac3a-1fff1481ead4","orcid":null,"display_name":"Jeff Johnson","source":"manual","import_confidence":0.72},{"id":"96e2176b-e87e-4d35-a6ca-9c873f56bca3","orcid":null,"display_name":"Matthijs Douze","source":"manual","import_confidence":0.72},{"id":"31d00841-52d1-4c8c-b98a-26afa01a6a0f","orcid":null,"display_name":"Pierre-Emmanuel Mazar\\'e","source":"manual","import_confidence":0.72}]}}