Large Language Models in Fire Engineering: An Examination of Technical Questions Against Domain Knowledge

Haley Hostetter; John Gales; M.Z. Naser; Xinyan Huang

Large Language Models in Fire Engineering: An Examination of Technical Questions Against Domain Knowledge

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.04795 v1 pith:UTR4E26Q submitted 2024-03-04 cs.CL cs.AIcs.LG

Haley Hostetter , M.Z. Naser , Xinyan Huang , John Gales This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords fireengineeringchatbotschatgptcommunicationperformancequestionssome

verification ladder T0 review T1 audit T2 compute T3 formal

0 comments

read the original abstract

This communication presents preliminary findings from comparing two recent chatbots, OpenAI's ChatGPT and Google's Bard, in the context of fire engineering by evaluating their responses in handling fire safety related queries. A diverse range of fire engineering questions and scenarios were created and examined, including structural fire design, fire prevention strategies, evacuation, building code compliance, and fire suppression systems (some of which resemble those commonly present in the Fire Protection exam (FPE)). The results reveal some key differences in the performance of the chatbots, with ChatGPT demonstrating a relatively superior performance. Then, this communication highlights the potential for chatbot technology to revolutionize fire engineering practices by providing instant access to critical information while outlining areas for further improvement and research. Evidently, and when it matures, this technology will likely be elemental to our engineers' practice and education.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment
cs.CY 2025-06 reject novelty 6.0 of 10

The authors show that prompting LLMs with earthquake parameters, local building, demographic, and street view data yields Modified Mercalli Intensity estimates that track USGS 'Did You Feel It?' reports for the 2014 N...

Pith tools