Pith. sign in

REVIEW 3 major objections 5 minor 23 references

Building Safer Sites: A Large-Scale Multi-Level Dataset for Construction Safety Research

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces CSDataset, a multi-level OSHA dataset of 150,000+ construction safety records, and uses it to estimate that complaint-driven inspections are associated with a 17.3% reduction in later incidents.

desk verdict The dataset is a real contribution; the 17.3% causal claim is not yet supported and needs either proper diagnostics or a reframe as associational. read the letter →

arxiv 2508.09203 v1 pith:JOTQMFJQ submitted 2025-08-09 cs.LG

Zhenhui Ou , Dawei Li , Zhen Tan , Wenlin Li , Huan Liu , Siyuan Song This is my paper
classification cs.LG
keywords constructionsafetydatasetOSHAmulti-leveldatainjuryseveritypredictionlargelanguagemodelspropensityscorematchinginspection-incidentcausalanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to solve a data bottleneck in construction safety research: existing datasets cover only one level of safety records and are too small for modern ML/LLM methods. It introduces CSDataset, which links over 50,000 OSHA incident records with over 100,000 inspection and violation records from 2013 to 2022, and demonstrates its value through two analyses. First, on injury-severity prediction, LLMs beat traditional ML baselines by using the narrative text. Second, using the linked structure, the authors estimate that complaint-driven inspections are associated with a 17.3% reduction in the likelihood of a subsequent incident, suggesting worker-initiated enforcement has a preventive effect.

What carries the argument

The dataset's multi-level linkage: activity_nr joins incidents to inspections, violator_id joins inspections to violations. This relational structure lets the authors align inspection and incident timing and control for site-level confounders. The causal estimate uses propensity-score matching to compare complaint-driven versus non-complaint inspections on the post-inspection incident outcome.

What would settle it

A credible falsifier: find a setting where inspection assignments are quasi-random (e.g., an OSHA scheduling cutoff or an instrumental variable for complaints) and re-estimate the effect; if the estimate moves materially toward zero, the PSM result is explained by confounding. A sensitivity analysis such as an E-value would also quantify how strong an unmeasured confounder would need to be.

Watch

Extended reading notes

Core claim

The paper's central claim is that a purpose-built multi-level dataset, CSDataset, which links OSHA incident, inspection, and violation records through shared activity numbers, enables both reliable model benchmarking and cross-level causal analysis in construction safety. Using it, the authors report that complaint-driven inspections are associated with a 17.3% decrease in the likelihood of a subsequent incident within 90 days at the same site, estimated via propensity-score matching on employer size, industry, region, prior violations, and project duration.

Load-bearing premise

The 17.3% estimate assumes that after propensity-score matching, complaint-driven and non-complaint inspections are otherwise exchangeable—that no unmeasured factor such as site safety culture or contractor quality drives both worker complaints and later incidents.

Editorial extensions

If this is right

  • CSDataset provides a shared benchmark for injury severity prediction, with GPT-4.1-mini and Qwen2.5-7B outperforming traditional ML models.
  • The dataset enables inspection-to-incident predictive modeling and worker-level risk profiling across three linked levels.
  • The 17.3% association suggests worker-initiated inspections have a preventive effect, supporting enforcement policies.
  • The temporal alignment of inspections and incidents within 90 days allows quasi-experimental studies of regulatory interventions.
  • The dataset's combination of structured and narrative fields supports multi-modal LLM prompting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 17.3% estimate is an association; without sensitivity analysis or balance diagnostics, it may reflect unobserved site safety culture, so a skeptical reader should treat it as a motivating correlation rather than a proven causal effect.
  • The dataset's linkage keys could enable follow-up studies on inspection timing, violation recurrence, and geographic variation in enforcement effectiveness.
  • LLM gains over ML likely stem from narrative text; controlled experiments separating text from tabular features would isolate which modality drives performance.
  • The dataset could be extended with near-miss reports or fatality data to test whether inspection effects persist beyond 90 days.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces CSDataset, a multi-level dataset built from OSHA incident, inspection, and violation records covering 2013–2022, with structured fields and unstructured narratives linked by activity numbers and violator IDs. The authors report a benchmark of injury-severity prediction on this dataset using classical ML models and two LLMs, and a cross-level analysis that claims complaint-driven inspections are associated with a 17.3% reduction in the probability of a subsequent incident within 90 days. The dataset and code are publicly released.

Significance. If the data curation and linkage are sound, CSDataset is a valuable community resource: it is substantially larger than most existing construction-safety datasets, combines multiple administrative levels, and includes free-text narratives that support both classical ML and LLM-based research. The public release and the emphasis on multi-level relational structure are concrete strengths. The predictive benchmark is a useful sanity check, and the inspection-incident analysis points to an important policy question. However, the headline causal claim is currently under-supported, and the evaluation protocol has avoidable leakage risks, so the paper's scientific contribution is not yet fully established.

major comments (3)
  1. [§4.2, Abstract] The 17.3% reduction estimate is the paper's headline quantitative result, but the propensity-score-matching analysis is specified only at the level of 'we adjusted for variables like employer size, industry, region, past violations, and project duration.' No matching algorithm, caliper, ratio, with/without replacement, balance diagnostics (e.g., standardized mean differences), common-support/overlap check, standard errors, confidence intervals, or sensitivity analysis is reported. The threat to identification is concrete: complaint-driven inspections are initiated by worker reports, so complaint status plausibly correlates with latent site safety culture, reporting culture, contractor quality, and management commitment. The comparison group is also under-defined: 'non-complaint inspections' may include accident-triggered inspections, making the 90-day post-inspection incident outcome non
  2. [§4.1, Table 2] The random 7:3 train/test split is problematic because the dataset contains multiple incident records per construction site, and the unit of analysis is the incident, not the site. Random splitting can place incidents from the same site in both training and test sets, leaking site-level information and inflating reported accuracy/F1. I recommend grouping by site (or by activity_nr/project_id) when splitting, or using a temporal split. In addition, Table 2 reports point estimates only, with no variance across multiple runs, no confidence intervals, and no per-class distribution. The text says the split 'ensuring balanced representation across severity classes' but later discusses 'sensitivity to class imbalance, notably affecting rare classes'; these statements are inconsistent unless the classes are balanced only within each split, which should be stated explicitly.
  3. [§3.2, §3.4, Table 1] The dataset size is described inconsistently: the abstract says 'more than 50,000 incident records, 100,000 inspection and associated violation data,' while Table 1 says '150K' and Figure 1 says '150k+ Records.' It is unclear whether the 150K figure is the sum of all records at all levels or the number of incidents alone, and whether every incident is linked to an inspection or violation. Please report exact counts at each level (incidents, inspections, violations), the number of unique sites/activity_nr values, the number of records with successful links, and the period coverage. The linking semantics of activity_nr and violator_id should also be described precisely: for instance, does an incident link to one inspection, or can one inspection have multiple incidents?
minor comments (5)
  1. [Throughout] The dataset name is misspelled as 'CSDatase' in several places (§3.1, §3.3, §3.4, §4).
  2. [§4.2] 'textttcomplaint_type' should be formatted as \texttt{complaint_type} or plain code font; also define this field in §3.
  3. [§3.1] 'cident severity' should be 'incident severity'.
  4. [§3.3] 'weather condtion' should be 'weather condition'.
  5. [References] References [4] and [5] appear to be the same paper (Alkaissy et al., 2023, 'Enhancing construction safety: Machine learning-based classification of injury types'). Please merge or de-duplicate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the dataset is externally sourced, benchmarks use held-out evaluation, and the 17.3% estimate is an observational PSM result, not an input-derived tautology.

full rationale

The paper's derivation chain is self-contained and does not reduce to its own inputs. CSDataset is constructed from publicly available OSHA records, linked through activity_nr and violator_id, so the dataset is not defined in terms of the paper's findings. The injury-severity benchmarking uses a 7:3 train/test split and evaluates models on held-out data; no prediction is fitted then relabeled as a benchmark result. The inspection-incident analysis estimates a 17.3% reduction using propensity score matching, comparing complaint-driven vs. non-complaint inspections; this is a statistical estimate from the data, not a parameter fitted to the outcome and then called a prediction. There are no load-bearing self-citations: reference [14] is an external prior implementation source, and none of the authors' prior work is invoked to justify the central claims. The main scientific weaknesses—potential unconfoundedness violations, lack of balance diagnostics, missing confidence intervals—are threats to causal validity and reproducibility, not circularity. No quoted passage exhibits a step where an equation or result is equivalent to its inputs by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The causal claim depends on the 90-day outcome window and on the assumption that PSM removes all confounding. The benchmark depends on a random split that ignores site-level clustering. Neither is defended with diagnostics.

free parameters (3)
  • Post-inspection outcome window = 90 days
    Defines whether a subsequent incident is attributed to an inspection in Section 4.2; chosen without a stated justification or sensitivity analysis.
  • Train/test split ratio = 7:3
    Used for all benchmark models in Section 4.1; arbitrary and may leak same-site records across folds.
  • PSM covariate set
    The matching model is not specified numerically; covariates listed in Section 4.2 are hand selected and no matching algorithm or caliper is reported.
assumptions (4)
  • domain assumption OSHA records provide an accurate and sufficiently complete account of construction incidents, inspections, and violations
    The dataset is built entirely from OSHA public records (Section 3.2); underreporting or inconsistent enforcement would bias both the dataset and the causal estimate.
  • domain assumption activity_nr and violator_id correctly link incidents to inspections and violations
    Section 3.2 states these identifiers enable multi-level integration; if they are noisy or ambiguous, the cross-level analyses fail.
  • domain assumption Unconfoundedness: complaint status is independent of potential incident outcomes given observed covariates
    Section 4.2's PSM identifies the 17.3% effect only if no unobserved confounders remain; this is unstated and untested.
  • domain assumption Incidents in the benchmark split are independent observations
    Section 4.1 uses a random 7:3 split; incidents from the same site or project are likely correlated, so test accuracy may be overestimated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Building Safer Sites: A Large-Scale Multi-Level Dataset for Construction Safety Research." pith.science (2026). https://pith.science/paper/JOTQMFJQ

@misc{pith2026250809203,
  author       = {Pith},
  title        = {Pith review of: Building Safer Sites: A Large-Scale Multi-Level Dataset for Construction Safety Research},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JOTQMFJQ}},
  note         = {Machine review of arXiv:2508.09203}
}
read the original abstract

Construction safety research is a critical field in civil engineering, aiming to mitigate risks and prevent injuries through the analysis of site conditions and human factors. However, the limited volume and lack of diversity in existing construction safety datasets pose significant challenges to conducting in-depth analyses. To address this research gap, this paper introduces the Construction Safety Dataset (CSDataset), a well-organized comprehensive multi-level dataset that encompasses incidents, inspections, and violations recorded sourced from the Occupational Safety and Health Administration (OSHA). This dataset uniquely integrates structured attributes with unstructured narratives, facilitating a wide range of approaches driven by machine learning and large language models. We also conduct a preliminary approach benchmarking and various cross-level analyses using our dataset, offering insights to inform and enhance future efforts in construction safety. For example, we found that complaint-driven inspections were associated with a 17.3% reduction in the likelihood of subsequent incidents. Our dataset and code are released at https://github.com/zhenhuiou/Construction-Safety-Dataset-CSDataset.

Figures

Figures reproduced from arXiv: 2508.09203 by the authors.

Figure 1
Figure 1. Overview of the motivation, characteristics, collec [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Workflow for Inspection-Incident Causal Analysis [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 20 canonical work pages

  1. [1]

    Hamidreza Abbasianjahromi, Emadaldin Mohammadi Golafshani, and Mehdi Aghakarimi. 2022. A prediction model for safety performance of construction sites using a linear artificial bee colony programming approach. International Journal of Occupational Safety and Ergonomics 28, 2 (2022), 1265–1280

    work page 2022

  2. [2]

    Gonzalez, and Qipei Mei

    Muhammad Adil, Gaang Lee, Vicente A. Gonzalez, and Qipei Mei. 2025. Using Vision Language Models for Safety Hazard Identification in Construction. arXiv preprint arXiv:2504.09083 (2025)

    arXiv 2025

  3. [3]

    Ehsan Ahmadi, Shashank Muley, and Chao Wang. 2025. Automatic construction accident report analysis using large language models (LLMs).Journal of Intelligent Construction 3, 1 (2025), 1–10

    work page 2025

  4. [5]

    Reza Hosseini, Sadegh Khanmohammadi, Yu Bai, and Haibo Feng

    Maryam Alkaissy, Mehrdad Arashpour, Emadaldin Mohammadi Golafshani, M. Reza Hosseini, Sadegh Khanmohammadi, Yu Bai, and Haibo Feng. 2023. En- hancing construction safety: Machine learning-based classification of injury types. Safety Science 162 (2023), 106102

    work page 2023

  5. [6]

    Construction Safety Research Group. 2020. Safety Management Dataset, SMD . Technical Report. Construction Safety Research Group, Unknown. 1–5 pages

    work page 2020

  6. [7]

    Goecks and Nicholas R

    Vinicius G. Goecks and Nicholas R. Waytowich. 2023. Disasterresponsegpt: Large language models for accelerated plan of action development in disaster response scenarios. arXiv preprint arXiv:2306.17271 (2023)

    arXiv 2023

  7. [8]

    Robertson, Angela Garabet, Jin Lee, and Lauren A

    Yueng hsiang Huang, Dov Zohar, Michelle M. Robertson, Angela Garabet, Jin Lee, and Lauren A. Murphy. 2013. Development and validation of safety climate scales for lone workers using truck drivers as exemplar. Transportation Research Part F: Traffic Psychology and Behaviour 17 (2013), 5–19

    work page 2013

  8. [9]

    Hayoung Kim, June-Seong Yi, and YeEun Jang. 2022. Analyzing Patterns of Multi- cause Accidents From KOSHA’s Construction Injury Case Reports Utilizing Text Mining Methodology. Journal of the Architectural Institute of Korea 38, 4 (2022), 237–244

    work page 2022

Show all 23 references
  1. [10]

    Shuang Li, Mengjie You, Dingwei Li, and Jiao Liu. 2022. Identifying coal mine safety production risk factors by employing text mining and Bayesian network techniques. Process Safety and Environmental Protection 162 (2022), 1067–1081

    2022

  2. [11]

    Jiajing Liu, Hanbin Luo, and Henry Liu. 2022. Deep learning-based data analytics for safety in construction. Automation in Construction 140 (2022), 104302

    2022

  3. [12]

    Kai Liu, Yuming Liu, and Yuanyuan Kou. 2024. Study on construction safety management in megaprojects from the perspective of resilient governance.Safety Science 173 (2024), 106442

    2024

  4. [13]

    Fatemeh Mostofi, Vedat Toğan, Yunus Emre Ayözen, and Onur Behzat Tokdemir

  5. [14]

    Clive QX Poh, Chalani Udhyami Ubeynarayana, and Yang Miang Goh. 2018. Safety leading indicators for construction sites: A machine learning approach. Automation in Construction 93 (2018), 375–386

    2018

  6. [15]

    Ahmed Bin Kabir Rabbi and Idris Jeelani. 2024. AI integration in construction safety: Current state, challenges, and future opportunities in text, vision, and audio based applications. Automation in Construction 164 (2024), 105443

    2024

  7. [16]

    Mason Smetana, Lucio Salles de Salles, Igor Sukharev, and Lev Khazanovich. 2024. Highway construction safety analysis using large language models. Applied Sciences 14, 4 (2024), 1352

    2024

  8. [17]

    Kailai Sun, Tianxiang Lan, Yang Miang Goh, and Yueng-Hsiang Huang. 2024. Overcoming Imbalanced Safety Data Using Extended Accident Triangle. arXiv preprint arXiv:2408.07094 (2024)

    2024 arXiv

  9. [18]

    Tixier and Matthew R

    Antoine J-P. Tixier and Matthew R. Hallowell. 2023. Safer together: Machine learning models trained on shared accident datasets predict construction injuries better than company-specific models. arXiv preprint arXiv:2301.03567 (2023)

    2023 arXiv

  10. [19]

    SM Jamil Uddin, Alex Albert, Anto Ovid, and Abdullah Alsharef. 2023. Leveraging ChatGPT to aid construction hazard recognition and support safety education and training. Sustainability 15, 9 (2023), 7121

    2023

  11. [20]

    Department of Labor, Occupational Safety and Health Administration

    U.S. Department of Labor, Occupational Safety and Health Administration. 2022. Construction Industry Statistics . Technical Report. U.S. Department of Labor, Washington, DC, USA. 1–10 pages. doi:10.1002/osha.2022.stats

    2022 doi

  12. [21]

    Department of Labor, Occupational Safety and Health Administration

    U.S. Department of Labor, Occupational Safety and Health Administration. 2022. Severe Injury Reports (SIR) Data Catalog . Technical Report. U.S. Department of Labor, Washington, DC, USA. 1–15 pages. doi:10.1002/osha.2022.sir

    2022 doi

  13. [22]

    Guangbin Wang, Muyang Liu, Dongping Cao, and Dan Tan. 2022. Identify- ing high-frequency–low-severity construction safety risks: An empirical study based on official supervision reports in Shanghai. Engineering, Construction and Architectural Management 29, 2 (2022), 940–960

    2022

  14. [23]

    Na Xu, Ling Ma, Qing Liu, Li Wang, and Yongliang Deng. 2021. An improved text mining approach to extract safety risk factors from construction accident reports. Safety Science 138 (2021), 105216

    2021

  15. [2022]

    Sustainability 14, 23 (2022), 15906

    Construction safety risk model with construction accident network: A graph convolutional network approach. Sustainability 14, 23 (2022), 15906

    2022

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.