IFCMemoryBench gives LLM agents 4,016 prior chat sessions plus live IFC model queries; the best vector-, graph-, or file-based memory system reaches only 32.4% answer accuracy, versus 83.2% when all relevant user messages are shown directly.
BIM Information Extraction Through LLM-based Adaptive Exploration
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
BIM models provide structured representations of building geometry, semantics, and topology, yet extracting specific information from them remains remarkably difficult. Current approaches translate natural language into structured queries by assuming a fixed data organization (static approach), which BIM heterogeneity eventually invalidates. We address this with a new paradigm, adaptive exploration, where an LLM-based agent iteratively executes code to extract information from a BIM model, discovering its structure at runtime instead of assuming it. We evaluate this approach on ifc-bench v2, an open-source BIM question-answering benchmark introduced alongside this work, comprising 1,027 tasks across 37 IFC models from 21 projects. A factorial ablation across two LLM capability levels and four augmentation strategies shows that adaptive exploration significantly outperforms static query generation across all configurations, regardless of the augmentation strategy. These results indicate that BIM heterogeneity is best addressed at the paradigm level, not by further optimizing static approaches.
fields
cs.IR 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
IFCMemoryBench: Evaluating Long-Term Memory of LLM-Based Agents in BIM Information Retrieval
IFCMemoryBench gives LLM agents 4,016 prior chat sessions plus live IFC model queries; the best vector-, graph-, or file-based memory system reaches only 32.4% answer accuracy, versus 83.2% when all relevant user messages are shown directly.