Fine-Tuning Large Language Models and Evaluating Retrieval Methods for Improved Question Answering on Building Codes

Mohammad Aqib; Mohd Hamza; Qipei Mei; Ying Hei Chui

REVIEW 4 major objections 5 minor 1 cited by

Fine-Tuning Large Language Models and Evaluating Retrieval Methods for Improved Question Answering on Building Codes

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Elasticsearch is the most reliable retriever for building-code queries, and LoRA fine-tuning on code-derived data sharply improves LLM answers.

desk verdict Useful NBCC dataset and retriever comparison; the fine-tuning gains may be inflated by same-page train/test leakage that the paper must rule out. read the letter →

arxiv 2505.04666 v1 pith:2BNAWTAE submitted 2025-05-07 cs.CL cs.IRcs.LG

Mohammad Aqib , Mohd Hamza , Qipei Mei , Ying Hei Chui This is my paper · ORCID

classification cs.CLcs.IRcs.LG

keywords retrieval-augmentedgenerationbuildingcodesElasticsearchLoRAfine-tuningquestionansweringBERTScoreNationalCodeofCanadaparameter-efficient

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to settle two practical questions for building a question-answering system over the National Building Code of Canada: which retriever pulls out the right code passages, and whether fine-tuning makes a large language model better at turning those passages into answers. It reports that Elasticsearch, a lexical search engine built on BM25 ranking, beats all dense and sparse competitors on BERT F1 at every top-k setting. It also reports that lightweight LoRA fine-tuning on roughly a thousand code-derived context-question-answer triples improves every model's answer quality, with Llama-3.1-8b more than doubling its BLEU score. If these results hold, a workable recipe is a lexical retriever plus a lightly fine-tuned open model, which matters because building codes are long, technical, and frequently queried by practitioners.

What carries the argument

The load-bearing mechanism is the two-stage RAG pipeline. In the retrieval stage, Elasticsearch's inverted index preprocesses text through tokenization, lowercasing, stemming, and lemmatization, then ranks passages with the BM25 scoring formula, which proved better at capturing the exact technical wording of building-code clauses than dense vector search. In the generation stage, LoRA injects small trainable low-rank matrices into the attention and feed-forward projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj) of a frozen, 4-bit-quantized transformer, so each model adapts to the NBCC while updating less than 1.2% of its parameters. The evaluation is carried out with BERTScore for retrieved passages and with F1, BLEU, ROUGE-1, SMS, METEOR, and BERT scores for generated answers.

What would settle it

Ask building-code experts to audit a random sample of the 445 test triplets against the NBCC text; if a sizable fraction of ground-truth answers are not supported by their paired contexts, the reported rankings lose their basis. A lighter check is to rerun the retriever comparison using human relevance judgments instead of BERTScore and see whether Elasticsearch still leads.

Watch

Extended reading notes

Core claim

The paper's central claim is that for the National Building Code of Canada, retrieval works best when it is lexical rather than semantic: Elasticsearch posts the highest BERT F1 scores at every top-k setting (0.845, 0.849, 0.847, and 0.844 at top-1, top-3, top-5, and top-10), ahead of TF-IDF, BM25, DPR with FAISS, Sentence-BERT with cosine similarity, and DPR with Annoy. It further claims that LoRA fine-tuning on an NBCC-derived dataset improves LLM answer generation across all six models tested, with the largest relative gains in Llama-3.1-8b (F1 from 0.326 to 0.517, BLEU from 0.176 to 0.354) and the highest absolute scores in Mistral-Small-24b-Instruct-2501 (fine-tuned F1 0.688, BLEU 0.549). Taken together, the two findings imply that a RAG system for building codes can be optimized by pairing Elasticsearch with a lightly fine-tuned open LLM.

Load-bearing premise

The evaluation stands or falls on the 1,436 context-question-answer triplets generated by a vision-language model from page images and filtered only for obvious problems; if those triplets contain wrong or misaligned answers, both the retriever ranking and the fine-tuning gains inherit the error.

Editorial extensions

If this is right

A RAG system for the NBCC can combine Elasticsearch with a LoRA-fine-tuned LLM and likely answer code queries more accurately than either component alone.
Retrieving only the top-3 or top-5 passages is enough; the paper's scores show top-10 retrieval adds no contextual benefit and can introduce irrelevant text.
Light fine-tuning, touching less than 1.2% of parameters, can produce large generative gains; Llama-3.1-8b's BLEU more than doubled after 10 epochs on 991 triples.
Models with stronger pretrained baselines benefit less from fine-tuning, as seen in Mistral-Small-24b-Instruct-2501, where gains were smaller but absolute performance was highest.
Dense retrievers appear less suitable than lexical ones for technical regulatory text when relevance depends on exact clause wording, at least on this dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because 1,354 of the 1,436 answers are extractive, much of the measured gain may reflect the model learning to copy relevant spans from context; abstractive reasoning over codes is a harder test the current metrics could miss.
A hybrid retriever that merges Elasticsearch's lexical hits with dense semantic scores is a natural next experiment; the paper does not test this, but its own data suggests the two signal types are complementary.
The same pipeline can likely be transferred to other regulatory corpora, but a test on a different building code or standards document would be needed to confirm that Elasticsearch's edge is not specific to the NBCC's layout and vocabulary.
A human-verified gold set would strengthen confidence: the current ground truth is generated by a vision-language model and manually filtered only for low relevance, not for correctness.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know: this is an applied RAG paper, not a methods paper. The new piece is a 1,436-triplet CQA dataset derived from the National Building Code of Canada, plus benchmark numbers for six retrievers and six LLMs before and after LoRA fine-tuning. That dataset could genuinely save time for people building code-compliance QA systems; the retriever comparison is straightforward and honestly reported.

The retriever story is plausible but weaker than the paper claims. Elasticsearch comes out on top, but the gap over TF-IDF, BM25, and S-BERT is tiny (BERT F1 differences are roughly 0.005–0.010). Without error bars or significance tests, “most robust” is too strong. The dense retrievers (DPR-FAISS, DPR-AI) trail clearly, but that is likely a configuration issue as much as an inherent limitation.

The bigger problem is the fine-tuning half. The dataset was generated by prompting InternVL2.5-8B to produce exactly two CQA triplets per NBCC page image, and 94% of answers are extractive, nearly verbatim spans of their context. The paper reports only a 991/445 train/test count, not how the split was constructed. If triplets were split at the triplet level without grouping by source page, then roughly 69% of test items share a page with a training item, making the reported F1 and BLEU gains (58.59% and 101.14% for Llama-3.1-8b) potentially a memorization artifact rather than domain adaptation. This is directly checkable and must be addressed before the generation claims are taken seriously.

There are smaller soft spots. The dataset is machine-generated with no human verification of correctness beyond manual filtering, so the ground truth itself is untrusted. The retrieval evaluation is also self-referential: questions were generated from the same pages that define relevance, which inflates retrieval scores. No code or data is released, only “on request,” and no end-to-end RAG system is actually built or measured, despite the title and conclusion implying one.

Who is this for? Practitioners in construction AI who want a building-code QA recipe, and researchers who might use the dataset as a benchmark. It deserves a serious referee, but the revision needs to document the split construction, rerun fine-tuning with page-level grouping, add significance testing, and release the data. If the split leakage is confirmed, the generation half collapses; if not, the paper is a modest but useful empirical contribution. I would send it to review, with the split issue as the main requested revision.

Referee Report

4 major / 5 minor

Summary. This paper presents an experimental study of retrieval-augmented generation (RAG) components for question answering over the National Building Code of Canada (NBCC). The authors generate a Context-Question-Answer (CQA) dataset from NBCC page images using InternVL2.5-8B, compare six retrievers (TF-IDF, BM25, Elasticsearch, DPR+FAISS, S-BERT+cosine, DPR+Annoy) using BERTScore precision/recall/F1, and evaluate six LLMs before and after LoRA fine-tuning on a train split of the dataset. They report that Elasticsearch is the most robust retriever and that fine-tuning improves generation, with the largest relative gains for Llama-3.1-8b (F1 +58.59%, BLEU +101.14%).

Significance. If the results are robust, the paper provides a useful domain-specific benchmark and practical guidance for building-code QA: the retriever comparison covers standard baselines, the dataset-construction prompt is specified, and the fine-tuning protocol is described in reasonable detail. The paper is less valuable as a scientific claim about retrieval or fine-tuning because the evaluation is entirely internal to the machine-generated dataset and lacks statistical uncertainty quantification. The main strengths are the breadth of compared methods and the transparency of the dataset-construction prompt; the main weaknesses are the absence of external validation, the lack of error bars, and the unresolved question of train/test leakage in the fine-tuning split.

major comments (4)

[Section 3.2, Figure 7, Table 3] The train/test split for fine-tuning is reported only as counts (991 for training, 445 for testing), but the dataset was created by prompting InternVL2.5-8B to generate exactly two CQA triplets per page image. If the split was performed at the triplet level without grouping by source page, a large fraction of test triplets (roughly 69% under random assignment) would share a source page and nearly all of its context with a training sibling, especially given that 1354 of 1436 answers are extractive (Section 3.3). In that case the reported gains (e.g., Llama-3.1-8b F1 +58.59%, BLEU +101.14%) could largely reflect memorization of near-identical context rather than domain adaptation. The authors must specify whether the split was page-exclusive and, if it was not, rerun the generation evaluation on a page-exclusive split.
[Section 4.1, Table 2] The claim that Elasticsearch is the most robust retriever is based on absolute BERT F1 differences that are very small (e.g., 0.845 vs 0.840 for TF-IDF at top-1, and 0.849 vs 0.847 at top-3). No standard deviations, confidence intervals, or paired significance tests are reported, so these differences could be within noise. The authors should report per-query variability and a paired test (e.g., bootstrap or Wilcoxon) to support the ranking.
[Section 3.2, Section 4.2] The CQA dataset is generated by InternVL2.5-8B and only 'manually filtered,' with no human verification of question naturalness or answer correctness reported in the paper. Because this dataset is the ground truth for both retrieval and generation evaluation, the reported absolute scores measure agreement with machine-generated references rather than actual correctness against the code. The authors should include a human-validated subset (even a few hundred triplets) or an external check against the source code text to confirm the dataset's accuracy and representativeness.
[Section 4.1, Section 3.2] The retrieval evaluation uses the same CQA triplets to define both queries and ground-truth contexts; since the questions were generated from those contexts, the retrieval task is easier than open retrieval and the absolute BERT scores are inflated. This does not by itself invalidate the relative comparison among retrievers, but the paper should explicitly acknowledge this circularity and temper the language that describes Elasticsearch as 'the most robust retriever for the NBCC' without qualification.

minor comments (5)

[Equation (4)] The BM25 formula appears to have a missing closing parenthesis in the denominator; please fix the typesetting so that the formula is unambiguous.
[Section 2.4.4] Sentence Mover's Similarity is described as a similarity metric, but the paper notes that lower scores indicate greater similarity; please clarify that it is a distance-like measure (e.g., 'Sentence Mover's Distance') to avoid confusion.
[Table 3 and Section 4.2] The terminology for 'F1 Score' and 'BERT F1 Score' is used inconsistently; please define both terms clearly and use the same names in the text and table so that the reported improvements are unambiguous.
[Table 2] The TF-IDF row reports '0.84' for top-1 BERT F1 while other entries use three decimals; please standardize the decimal places.
[Figure 8] The sample context contains the phrase 'bylbaws' (likely a typo for 'bylaws') and 'sloping use' (likely 'for use'); please correct these transcription errors.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the retrieval and fine-tuning claims rest on held-out empirical evaluation rather than on inputs defined by the claims.

full rationale

The paper's two central claims—Elasticsearch as the best retriever and LoRA fine-tuning improving LLM answers—are empirical comparisons, not derivations from their inputs. In Section 2.2, each retriever is evaluated by retrieving top-k chunks for questions from the CQA set and comparing the retrieved chunks to the triplet's context with BERTScore; the ground truth is external to the retriever definitions, so no retriever wins by construction (Table 2 reports scores, not identities). In Section 3.2, the dataset is split into 991 training and 445 test CQA pairs; the fine-tuned models are evaluated on held-out test pairs with standard lexical and semantic metrics (Table 3). No model parameter is fitted to the test answers, and no claimed output is defined in terms of the metric used to report it. The paper invokes no uniqueness theorem and no load-bearing self-citation; citations to LoRA, BERTScore, InternVL, and model cards are standard external references. The dataset is self-generated, which raises external-validity and possible page-level leakage concerns, but those are correctness risks, not circular reductions: a properly held-out split still measures generalization, and using the source context as retrieval ground truth is the normal construction of a retrieval benchmark, not an equation that forces the result.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claims rest primarily on the trustworthiness of the machine-generated dataset and the validity of the chosen evaluation metrics. No fitted constants or new theoretical entities are introduced.

assumptions (4)

domain assumption The CQA triplets generated by InternVL2.5-8B from page images are accurate extractions of NBCC content.
Section 3.2 describes using InternVL2.5-8B to generate triplets, with manual filtering of incomplete or low-relevance entries; no human verification of correctness is reported.
domain assumption BERTScore is a valid measure of retrieval relevance for this task.
Section 2.4.6 uses BERTScore to compare retrieved chunks to reference contexts; the paper does not validate BERTScore against human judgments on building code text.
domain assumption The held-out test split is representative of real user queries on building codes.
Section 3.2 states a 991/445 split; questions were generated by an LLM from the same contexts, so the distribution may not match real-world usage patterns.
domain assumption Fine-tuning on the NBCC dataset for 10 epochs with LoRA is a reasonable proxy for optimizing RAG generation, without end-to-end RAG evaluation.
Section 2.3 describes the fine-tuning setup; Section 5 extrapolates to RAG performance without testing a complete retrieval-plus-generation pipeline.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Fine-Tuning Large Language Models and Evaluating Retrieval Methods for Improved Question Answering on Building Codes." pith.science (2026). https://pith.science/paper/2BNAWTAE

@misc{pith2026250504666,
  author       = {Pith},
  title        = {Pith review of: Fine-Tuning Large Language Models and Evaluating Retrieval Methods for Improved Question Answering on Building Codes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2BNAWTAE}},
  note         = {Machine review of arXiv:2505.04666}
}

read the original abstract

Building codes are regulations that establish standards for the design, construction, and safety of buildings to ensure structural integrity, fire protection, and accessibility. They are often extensive, complex, and subject to frequent updates, making manual querying challenging and time-consuming. Key difficulties include navigating large volumes of text, interpreting technical language, and identifying relevant clauses across different sections. A potential solution is to build a Question-Answering (QA) system that answers user queries based on building codes. Among the various methods for building a QA system, Retrieval-Augmented Generation (RAG) stands out in performance. RAG consists of two components: a retriever and a language model. This study focuses on identifying a suitable retriever method for building codes and optimizing the generational capability of the language model using fine-tuning techniques. We conducted a detailed evaluation of various retrieval methods by performing the retrieval on the National Building Code of Canada (NBCC) and explored the impact of domain-specific fine-tuning on several language models using the dataset derived from NBCC. Our analysis included a comparative assessment of different retrievers and the performance of both pre-trained and fine-tuned models to determine the efficacy and domain-specific adaptation of language models using fine-tuning on the NBCC dataset. Experimental results showed that Elasticsearch proved to be the most robust retriever among all. The findings also indicate that fine-tuning language models on an NBCC-specific dataset can enhance their ability to generate contextually relevant responses. When combined with context retrieved by a powerful retriever like Elasticsearch, this improvement in LLM performance can optimize the RAG system, enabling it to better navigate the complexities of the NBCC.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Automating structural reliability analysis with a multi-agent large language model framework
stat.AP 2026-07 conditional novelty 6.0 of 10

A multi-agent LLM framework automates structural reliability analysis from natural-language input to interpreted β and Pf, with a QLoRA-tuned method-planning agent.

Reference graph

Works this paper leans on

64 extracted references · 29 canonical work pages · cited by 1 Pith paper

[1]

AN IMPACT ANALYSIS OF CONSTRUCTION SECTOR ON ECONOMIC GROWTH AND HOUSELOLD INCOME IN SOUTH AFRICA

M. J. M. Mosenogi, “AN IMPACT ANALYSIS OF CONSTRUCTION SECTOR ON ECONOMIC GROWTH AND HOUSELOLD INCOME IN SOUTH AFRICA”

work page
[2]

Construction Sector Contribution to Economic Stability: Malaysian GDP Distribution,

W. S. Alaloul, M. A. Musarat, M. B. A. Rabbani, Q. Iqbal, A. Maqsoom, and W. Farooq, “Construction Sector Contribution to Economic Stability: Malaysian GDP Distribution,” Sustainability, vol. 13, no. 9, p. 5012, Apr. 2021, doi: 10.3390/su13095012

work page doi:10.3390/su13095012 2021
[3]

Loose coupling of GIS and BIM data models for automated compliance checking against zoning codes,

Y . Demir Altıntaş and M. E. Ilal, “Loose coupling of GIS and BIM data models for automated compliance checking against zoning codes,” Autom. Constr., vol. 128, p. 103743, Aug. 2021, doi: 10.1016/j.autcon.2021.103743

work page arXiv 2021
[4]

Invariant Signature, Logic Reasoning, and Semantic Natural Language Processing (NLP)-Based Automated Building Code Compliance Checking (I- SNACC) Framework,

J. Wu, X. Xue, and J. Zhang, “Invariant Signature, Logic Reasoning, and Semantic Natural Language Processing (NLP)-Based Automated Building Code Compliance Checking (I- SNACC) Framework,” J. Inf. Technol. Constr., vol. 28, pp. 1–18, Jan. 2023, doi: 10.36680/j.itcon.2023.001

doi:10.36680/j.itcon.2023.001 2023
[5]

Intelligent Construction Case Study Illustration System Using Natural Language Processing and Image Searching,

J. Zhang and A. Akanmu, “Intelligent Construction Case Study Illustration System Using Natural Language Processing and Image Searching,” 2016

work page 2016
[6]

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects,

M. U. Hadi et al., “Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects,” Nov. 16, 2023. doi: 10.36227/techrxiv.23589741.v4

doi:10.36227/techrxiv.23589741.v4 2023
[7]

Datasets for Large Language Models: A Comprehensive Survey,

Y . Liu, J. Cao, C. Liu, K. Ding, and L. Jin, “Datasets for Large Language Models: A Comprehensive Survey,” Feb. 28, 2024, arXiv: arXiv:2402.18041. doi: 10.48550/arXiv.2402.18041

arXiv doi:10.48550/arxiv.2402.18041 2024
[8]

Potential of Large Language Models in Health Care: Delphi Study,

K. Denecke, R. May, LLMHealthGroup, and O. Rivera Romero, “Potential of Large Language Models in Health Care: Delphi Study,” J. Med. Internet Res., vol. 26, p. e52399, May 2024, doi: 10.2196/52399

doi:10.2196/52399 2024

Show all 64 references

[9]

Fully Autonomous Programming with Large Language Models,

V . Liventsev, A. Grishina, A. Härmä, and L. Moonen, “Fully Autonomous Programming with Large Language Models,” in Proceedings of the Genetic and Evolutionary Computation Conference, Lisbon Portugal: ACM, Jul. 2023, pp. 1146–1155. doi: 10.1145/3583131.3590481

2023
[10]

Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models,

A. Louis, G. Van Dijck, and G. Spanakis, “Interpretable Long-Form Legal Question Answering with Retrieval-Augmented Large Language Models,” Proc. AAAI Conf. Artif. Intell., vol. 38, no. 20, pp. 22266–22275, Mar. 2024, doi: 10.1609/aaai.v38i20.30232

2024 doi
[11]

A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics,

K. He et al., “A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics,” Inf. Fusion, vol. 118, p. 102963, Jun. 2025, doi: 10.1016/j.inffus.2025.102963

2025
[12]

A Survey on Evaluation of Large Language Models,

Y . Chang et al., “A Survey on Evaluation of Large Language Models,” ACM Trans. Intell. Syst. Technol., vol. 15, no. 3, pp. 1–45, Jun. 2024, doi: 10.1145/3641289

2024 doi
[13]

GPT-4 Technical Report,

OpenAI et al., “GPT-4 Technical Report,” Mar. 04, 2024, arXiv: arXiv:2303.08774. doi: 10.48550/arXiv.2303.08774

2024 doi arXiv
[14]

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” May 24, 2019, arXiv: arXiv:1810.04805. doi: 10.48550/arXiv.1810.04805

2019 doi arXiv
[15]

PaLM: Scaling Language Modeling with Pathways

A. Chowdhery et al., “PaLM: Scaling Language Modeling with Pathways”
[16]

LLaMA: Open and Efficient Foundation Language Models,

H. Touvron et al., “LLaMA: Open and Efficient Foundation Language Models,” Feb. 27, 2023, arXiv: arXiv:2302.13971. doi: 10.48550/arXiv.2302.13971

2023 doi arXiv
[17]

A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets,

M. T. R. Laskar, M. S. Bari, M. Rahman, M. A. H. Bhuiyan, S. Joty, and J. X. Huang, “A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets,” Jul. 05, 2023, arXiv: arXiv:2305.18486. doi: 10.48550/arXiv.2305.18486

2023 doi arXiv
[18]

Applications of Large Language Models,

A. Zharovskikh, “Applications of Large Language Models,” InData Labs. Accessed: Feb. 12, 2025. [Online]. Available: https://indatalabs.com/blog/large-language-model-apps

2025
[19]

Using Vision Language Models for Safety Hazard Identification in Construction

M. Adil, G. Lee, V . A. Gonzalez, and Q. Mei, “Using Vision Language Models for Safety Hazard Identification in Construction”
[20]

Intelligent Checking Method for Construction Schemes via Fusion of Knowledge Graph and Large Language Models,

H. Li, R. Yang, S. Xu, Y . Xiao, and H. Zhao, “Intelligent Checking Method for Construction Schemes via Fusion of Knowledge Graph and Large Language Models,” Buildings, vol. 14, no. 8, p. 2502, Aug. 2024, doi: 10.3390/buildings14082502

2024 doi
[21]

Enhancing Visual-Llm Through Prompt Engineering and a Two-Stage Retrieval-Augmented Generation Algorithm for Construction Site Safety Compliance Checking,

X. Guo, P. K.-Y . Wong, J. C. P. Cheng, J. C. F. Chan, P.-H. Leung, and X. Tao, “Enhancing Visual-Llm Through Prompt Engineering and a Two-Stage Retrieval-Augmented Generation Algorithm for Construction Site Safety Compliance Checking,” 2025, SSRN. doi: 10.2139/ssrn.5097308

2025 doi
[22]

ErgoChat – a Visual Query System for the Ergonomic Risk Assessment of Construction Workers

“ErgoChat – a Visual Query System for the Ergonomic Risk Assessment of Construction Workers.”
[23]

Construction contract risk identification based on knowledge-augmented language models,

S. Wong, C. Zheng, X. Su, and Y . Tang, “Construction contract risk identification based on knowledge-augmented language models,” Comput. Ind., vol. 157–158, p. 104082, May 2024, doi: 10.1016/j.compind.2024.104082

2024
[24]

Meet2Mitigate: An LLM- powered framework for real-time issue identification and mitigation from construction meeting discourse,

G. Chen, A. Alsharef, A. Ovid, A. Albert, and E. Jaselskis, “Meet2Mitigate: An LLM- powered framework for real-time issue identification and mitigation from construction meeting discourse,” Adv. Eng. Inform., vol. 64, p. 103068, Mar. 2025, doi: 10.1016/j.aei.2024.103068

2025
[25]

Automated Building Information Modeling Compliance Check through a Large Language Model Combined with Deep Learning and Ontology,

N. Chen, X. Lin, H. Jiang, and Y . An, “Automated Building Information Modeling Compliance Check through a Large Language Model Combined with Deep Learning and Ontology,” Buildings, vol. 14, no. 7, p. 1983, Jul. 2024, doi: 10.3390/buildings14071983

1983 doi
[26]

In Defense of RAG in the Era of Long-Context Language Models,

T. Yu, A. Xu, and R. Akkiraju, “In Defense of RAG in the Era of Long-Context Language Models,” Sep. 03, 2024, arXiv: arXiv:2409.01666. doi: 10.48550/arXiv.2409.01666

2024 doi arXiv
[27]

Empower Large Language Model to Perform Better on Industrial Domain- Specific Question Answering,

F. Yang et al., “Empower Large Language Model to Perform Better on Industrial Domain- Specific Question Answering,” Oct. 16, 2023, arXiv: arXiv:2305.11541. doi: 10.48550/arXiv.2305.11541

2023 doi arXiv
[28]

Hallucination is Inevitable: An Innate Limitation of Large Language Models,

Z. Xu, S. Jain, and M. Kankanhalli, “Hallucination is Inevitable: An Innate Limitation of Large Language Models,” Jan. 22, 2024, arXiv: arXiv:2401.11817. doi: 10.48550/arXiv.2401.11817

2024 doi arXiv
[29]

Retrieval Augmentation Reduces Hallucination in Conversation,

K. Shuster, S. Poff, M. Chen, D. Kiela, and J. Weston, “Retrieval Augmentation Reduces Hallucination in Conversation,” Apr. 15, 2021, arXiv: arXiv:2104.07567. doi: 10.48550/arXiv.2104.07567

2021 doi arXiv
[30]

Merging Generated and Retrieved Knowledge for Open-Domain QA,

Y . Zhang, M. Khalifa, L. Logeswaran, M. Lee, H. Lee, and L. Wang, “Merging Generated and Retrieved Knowledge for Open-Domain QA,” Oct. 22, 2023, arXiv: arXiv:2310.14393. doi: 10.48550/arXiv.2310.14393

2023 doi arXiv
[31]

Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey,

Z. Han, C. Gao, J. Liu, J. Zhang, and S. Q. Zhang, “Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey,” Sep. 16, 2024, arXiv: arXiv:2403.14608. doi: 10.48550/arXiv.2403.14608

2024 doi arXiv
[32]

A fine-tuning enhanced RAG system with quantized influence measure as AI judge,

K. Rangan and Y . Yin, “A fine-tuning enhanced RAG system with quantized influence measure as AI judge,” Sci. Rep., vol. 14, no. 1, p. 27446, Nov. 2024, doi: 10.1038/s41598- 024-79110-x

2024 doi
[33]

Parameter-efficient fine-tuning of large-scale pre-trained language models,

N. Ding et al., “Parameter-efficient fine-tuning of large-scale pre-trained language models,” Nat. Mach. Intell., vol. 5, no. 3, pp. 220–235, Mar. 2023, doi: 10.1038/s42256-023-00626- 4

2023 doi
[34]

Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning,

V . Lialin, V . Deshpande, X. Yao, and A. Rumshisky, “Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning,” Nov. 22, 2024, arXiv: arXiv:2303.15647. doi: 10.48550/arXiv.2303.15647

2024 doi arXiv
[35]

Parameter Efficient Fine-Tuning on Selective Parameters for Transformer-Based Pre-Trained Models,

Q. Li, “Parameter Efficient Fine-Tuning on Selective Parameters for Transformer-Based Pre-Trained Models,” in 2024 IEEE International Conference on Multimedia and Expo (ICME), Niagara Falls, ON, Canada: IEEE, Jul. 2024, pp. 1–6. doi: 10.1109/ICME57554.2024.10688138

2024
[36]

Avidan, G

S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, Eds., Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXIII, vol. 13693. in Lecture Notes in Computer Science, vol. 13693. Cham: Springer Nature S...

2022 doi
[37]

LoRA: Low-Rank Adaptation of Large Language Models,

E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” Oct. 16, 2021, arXiv: arXiv:2106.09685. doi: 10.48550/arXiv.2106.09685

2021 doi arXiv
[38]

AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition

S. Chen et al., “AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition”
[39]

Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment,

L. Xu, H. Xie, S.-Z. J. Qin, X. Tao, and F. L. Wang, “Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment,” Dec. 19, 2023, arXiv: arXiv:2312.12148. doi: 10.48550/arXiv.2312.12148

2023 doi arXiv
[40]

National Building Code of Canada 2020

N. R. C. Canada, “National Building Code of Canada 2020.” Accessed: Mar. 09, 2025. [Online]. Available: https://nrc.canada.ca/en/certifications-evaluations-standards/codes- canada/codes-canada-publications/national-building-code-canada-2020

2020
[41]

Retrieval-Augmented Generation for AI-Generated Content: A Survey,

P. Zhao et al., “Retrieval-Augmented Generation for AI-Generated Content: A Survey,” Jun. 21, 2024, arXiv: arXiv:2402.19473. doi: 10.48550/arXiv.2402.19473

2024 doi arXiv
[42]

The Probabilistic Relevance Framework: BM25 and Beyond,

S. Robertson and H. Zaragoza, “The Probabilistic Relevance Framework: BM25 and Beyond,” Found. Trends® Inf. Retr., vol. 3, no. 4, pp. 333–389, 2009, doi: 10.1561/1500000019

2009 doi
[43]

Document retrieval using term term frequency inverse sentence frequency weighting scheme,

M. T. Mohammed and O. F. Rashid, “Document retrieval using term term frequency inverse sentence frequency weighting scheme,” Indones. J. Electr. Eng. Comput. Sci., vol. 31, no. 3, p. 1478, Sep. 2023, doi: 10.11591/ijeecs.v31.i3.pp1478-1485

2023 doi
[44]

Dense Passage Retrieval for Open-Domain Question Answering,

V . Karpukhin et al., “Dense Passage Retrieval for Open-Domain Question Answering,” Sep. 30, 2020, arXiv: arXiv:2004.04906. doi: 10.48550/arXiv.2004.04906

2020 doi arXiv
[45]

SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval,

Y . Bai et al., “SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval,” Oct. 02, 2020, arXiv: arXiv:2010.00768. doi: 10.48550/arXiv.2010.00768

2020 doi arXiv
[46]

Indonesian- English Textual Similarity Detection Using Universal Sentence Encoder (USE) and Facebook AI Similarity Search (FAISS),

L. D. Krisnawati, A. W. Mahastama, S.-C. Haw, K.-W. Ng, and P. Naveen, “Indonesian- English Textual Similarity Detection Using Universal Sentence Encoder (USE) and Facebook AI Similarity Search (FAISS),” CommIT Commun. Inf. Technol. J., vol. 18, no. 2, pp. 183–195, Sep. 2024, ...

2024 doi
[47]

Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,

N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hon...

2019 doi
[48]

Enhanced support for citations on GitHub,

A. Smith, “Enhanced support for citations on GitHub,” The GitHub Blog. Accessed: Feb. 20, 2025. [Online]. Available: https://github.blog/news-insights/company-news/enhanced- support-citations-github/

2025
[49]

Attention is All you Need

A. Vaswani et al., “Attention is All you Need”
[50]

A Comprehensive Study on Quantization Techniques for Large Language Models,

J. Lang, Z. Guo, and S. Huang, “A Comprehensive Study on Quantization Techniques for Large Language Models,” in 2024 4th International Conference on Artificial Intelligence, Robotics, and Communication (ICAIRC), Xiamen, China: IEEE, Dec. 2024, pp. 224–231. doi: 10.1109/ICAIRC6...

2024
[51]

Meta Llama 2,

“Meta Llama 2,” Meta Llama. Accessed: Mar. 14, 2025. [Online]. Available: https://www.llama.com/llama2/

2025
[52]

Introducing Meta Llama 3: The most capable openly available LLM to date,

“Introducing Meta Llama 3: The most capable openly available LLM to date,” Meta AI. Accessed: Mar. 14, 2025. [Online]. Available: https://ai.meta.com/blog/meta-llama-3/

2025
[53]

Qwen LLMs

“Qwen LLMs.” Accessed: Mar. 14, 2025. [Online]. Available: https://www.alibabacloud.com/help/en/model-studio/developer-reference/what-is-qwen-llm

2025
[54]

mistralai/Mistral-Small-24B-Instruct-2501 · Hugging Face

“mistralai/Mistral-Small-24B-Instruct-2501 · Hugging Face.” Accessed: Mar. 29, 2025. [Online]. Available: https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501

2025
[55]

Introducing Phi-3: Redefining what’s possible with SLMs,

M. Bilenko, “Introducing Phi-3: Redefining what’s possible with SLMs,” Microsoft Azure Blog. Accessed: Mar. 14, 2025. [Online]. Available: https://azure.microsoft.com/en- us/blog/introducing-phi-3-redefining-whats-possible-with-slms/

2025
[56]

Thresholding Classifiers to Maximize F1 Score,

Z. C. Lipton, C. Elkan, and B. Narayanaswamy, “Thresholding Classifiers to Maximize F1 Score,” May 14, 2014, arXiv: arXiv:1402.1892. doi: 10.48550/arXiv.1402.1892

2014 doi arXiv
[57]

BLEU: a method for automatic evaluation of machine translation,

K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: a method for automatic evaluation of machine translation,” in Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL ’02, Philadelphia, Pennsylvania: Association for Computational Linguis...

2001
[58]

ROUGE: A Package for Automatic Evaluation of Summaries

C.-Y . Lin, “ROUGE: A Package for Automatic Evaluation of Summaries”
[59]

Sentence Mover’s Similarity: Automatic Evaluation for Multi-Sentence Texts,

E. Clark, A. Celikyilmaz, and N. A. Smith, “Sentence Mover’s Similarity: Automatic Evaluation for Multi-Sentence Texts,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy: Association for Computational Linguistics, 2019...

2019 doi
[60]

METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments,

S. Banerjee and A. Lavie, “METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments,” Jun. 2005, [Online]. Available: https://aclanthology.org/W05-0909/

2005
[61]

BERTScore: Evaluating Text Generation with BERT,

T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “BERTScore: Evaluating Text Generation with BERT,” Feb. 24, 2020, arXiv: arXiv:1904.09675. doi: 10.48550/arXiv.1904.09675

2020 doi arXiv
[62]

OpenGVLab/InternVL2_5-8B · Hugging Face

“OpenGVLab/InternVL2_5-8B · Hugging Face.” Accessed: Apr. 04, 2025. [Online]. Available: https://huggingface.co/OpenGVLab/InternVL2_5-8B

2025
[63]

mistralai/Mistral-Small-24B-Instruct-2501 · Hugging Face

“mistralai/Mistral-Small-24B-Instruct-2501 · Hugging Face.” Accessed: Apr. 23, 2025. [Online]. Available: https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501

2025
[64]

Introducing Meta Llama 3: The most capable openly available LLM to date,

“Introducing Meta Llama 3: The most capable openly available LLM to date,” Meta AI. Accessed: Apr. 29, 2025. [Online]. Available: https://ai.meta.com/blog/meta-llama-3/

2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

AN IMPACT ANALYSIS OF CONSTRUCTION SECTOR ON ECONOMIC GROWTH AND HOUSELOLD INCOME IN SOUTH AFRICA

M. J. M. Mosenogi, “AN IMPACT ANALYSIS OF CONSTRUCTION SECTOR ON ECONOMIC GROWTH AND HOUSELOLD INCOME IN SOUTH AFRICA”

work page

[2] [2]

Construction Sector Contribution to Economic Stability: Malaysian GDP Distribution,

W. S. Alaloul, M. A. Musarat, M. B. A. Rabbani, Q. Iqbal, A. Maqsoom, and W. Farooq, “Construction Sector Contribution to Economic Stability: Malaysian GDP Distribution,” Sustainability, vol. 13, no. 9, p. 5012, Apr. 2021, doi: 10.3390/su13095012

work page doi:10.3390/su13095012 2021

[3] [3]

Loose coupling of GIS and BIM data models for automated compliance checking against zoning codes,

Y . Demir Altıntaş and M. E. Ilal, “Loose coupling of GIS and BIM data models for automated compliance checking against zoning codes,” Autom. Constr., vol. 128, p. 103743, Aug. 2021, doi: 10.1016/j.autcon.2021.103743

work page arXiv 2021

[4] [4]

Invariant Signature, Logic Reasoning, and Semantic Natural Language Processing (NLP)-Based Automated Building Code Compliance Checking (I- SNACC) Framework,

J. Wu, X. Xue, and J. Zhang, “Invariant Signature, Logic Reasoning, and Semantic Natural Language Processing (NLP)-Based Automated Building Code Compliance Checking (I- SNACC) Framework,” J. Inf. Technol. Constr., vol. 28, pp. 1–18, Jan. 2023, doi: 10.36680/j.itcon.2023.001

doi:10.36680/j.itcon.2023.001 2023

[5] [5]

Intelligent Construction Case Study Illustration System Using Natural Language Processing and Image Searching,

J. Zhang and A. Akanmu, “Intelligent Construction Case Study Illustration System Using Natural Language Processing and Image Searching,” 2016

work page 2016

[6] [6]

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects,

M. U. Hadi et al., “Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects,” Nov. 16, 2023. doi: 10.36227/techrxiv.23589741.v4

doi:10.36227/techrxiv.23589741.v4 2023

[7] [7]

Datasets for Large Language Models: A Comprehensive Survey,

Y . Liu, J. Cao, C. Liu, K. Ding, and L. Jin, “Datasets for Large Language Models: A Comprehensive Survey,” Feb. 28, 2024, arXiv: arXiv:2402.18041. doi: 10.48550/arXiv.2402.18041

arXiv doi:10.48550/arxiv.2402.18041 2024

[8] [8]

Potential of Large Language Models in Health Care: Delphi Study,

K. Denecke, R. May, LLMHealthGroup, and O. Rivera Romero, “Potential of Large Language Models in Health Care: Delphi Study,” J. Med. Internet Res., vol. 26, p. e52399, May 2024, doi: 10.2196/52399

doi:10.2196/52399 2024