pith. sign in

arxiv: 2505.19358 · v2 · submitted 2025-05-25 · 💻 cs.CE

RoofNet: A Global Multimodal Dataset for Roof Material Classification

Noelle Law , Yuki Miura This is my paper

Pith reviewed 2026-05-19 12:50 UTC · model grok-4.3

classification 💻 cs.CE
keywords roof material classificationsatellite imagerymultimodal datasetvision-language modelsnatural hazard modelingglobal exposure datasetsEarth observationbuilding vulnerability
0
0 comments X

The pith

RoofNet supplies over 51,500 satellite images labeled with 14 roof materials from 184 diverse global locations.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper presents RoofNet as a new multimodal dataset that pairs high-resolution satellite imagery with text annotations for classifying roof materials on a global scale. Accurate information on roofing types is currently missing but needed to model how buildings will hold up against earthquakes, floods, wildfires, and hurricanes. The authors built the dataset by selecting images from many different climates and building styles, annotating a portion with experts, and using a vision-language model with special prompts to label the rest while checking results with rules and human review. This resource is meant to make global datasets of building exposure more reliable for disaster planning.

Core claim

The central claim is that RoofNet constitutes the largest and most geographically diverse multimodal dataset for roof material classification, with more than 51,500 samples drawn from 184 sites around the world, each combining Earth Observation imagery and curated annotations for 14 roofing types, created to support vision-language modeling and improve the accuracy of global building exposure data.

What carries the argument

The key mechanism is the process of geographic sampling from climatically and architecturally distinct regions, expert annotation of 6,000 images, application of geographic- and material-aware prompt tuning to a vision-language model, and subsequent rule-based and human-in-the-loop verification to generate labels and additional metadata for the full set of tiles.

If this is right

  • Improved modeling of building vulnerability to natural hazards becomes possible with better roof material data.
  • Vision-language models can be trained more effectively on global roof classification tasks.
  • Global exposure datasets gain higher fidelity through the inclusion of this labeled imagery and metadata.
  • Additional attributes like roof shape, area, solar panel presence, and mixed materials are available for each sample.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Such a dataset could allow researchers to track roof material changes over time by comparing images from different years.
  • Integration into disaster simulation tools might refine estimates of economic losses from future events.
  • Similar approaches could be applied to classify other building components, such as foundations or walls, using satellite data.

Load-bearing premise

The selection of 184 sites from climatically and architecturally distinct regions creates a representative sample of global roof materials, and the combination of expert annotations with rule-based and human verification produces accurate labels free from major bias.

What would settle it

Ground surveys or independent high-resolution inspections in several of the sampled sites that reveal a substantially different distribution of the 14 roof material types than what the dataset reports.

Figures

Figures reproduced from arXiv: 2505.19358 by Noelle Law, Yuki Miura.

Figure 1
Figure 1. Figure 1: Overview of RoofNet and downstream applications. (1) High-resolution EO imagery is annotated with prompts describing roof material and location. Materials are labeled using expert￾validated and VLM-assisted classification (2) A VLM is trained using RoofNet to label a subset of the xBD [9] dataset. (3) RoofNet is hosted online to allow for open-access to advance risk modeling frameworks and other downstream… view at source ↗
Figure 2
Figure 2. Figure 2: RoofNet dataset construction and classification pipeline. Architecturally and geograph￾ically diverse cities are selected to collect geocoded metadata and representative satellite imagery. Roofs are centered using GroundingDINO [26], with a 6k-sample subset manually verified for fine￾tuning RemoteCLIP ViT-L/14 [24]. Remaining samples are classified using the model, followed by rule-based and human-in-the-l… view at source ↗
Figure 3
Figure 3. Figure 3: RoofNet material classes. RoofNet includes 14 distinct roof material classes to support downstream modeling of vulnerability to natural hazards. 3.1 Data Collection and Preprocessing Approximately 200 buildings per city were geocoded and matched with high-resolution satellite imagery. To enhance coverage of underrepresented (“long-tail") material classes (i.e., polycarbonate sheets, wood shingles, and glas… view at source ↗
Figure 4
Figure 4. Figure 4: Geographic and categorical distribution of roof material classes in the RoofNet dataset. The four world maps (left) visualize spatial coverage for each major roof material group (i.e., Manufactured Tiles, Sheet Materials, Synthetic/Flat Roofs, and Traditional/Natural) colored by material class and spanning 184 geographically diverse sites across 112 countries. These maps illustrate RoofNet’s architectural,… view at source ↗
Figure 5
Figure 5. Figure 5: Prompts were selected to maximize distance in material classifications while remaining in the 77 token limit. The image above shows how class names cause confusion and similar confidence levels in the out-of-the-box remoteCLIP, while carefully selecting prompts that use simple, common language with diverse descriptions allows for a greater separation between classes. The EO image uploaded is a Concrete Til… view at source ↗
Figure 6
Figure 6. Figure 6: Impact of Fine-tuning and Class-Rebalanced Training on Roof Material Classification Accuracy. We compare material classification confusion matrices across four models: (left) CLIP ViT￾L/14 [29], (center-left) RemoteCLIP ViT-L/14 [24], (center-right) RemoteCLIP ViT-L/14 fine-tuned on the RoofNet dataset subset, and (right) RemoteCLIP ViT-L/14 fine-tuned using class-rebalanced sampling. While the pretrained … view at source ↗
read the original abstract

Natural disasters are increasing in frequency and severity, causing hundreds of billions of dollars in damage annually and posing growing threats to infrastructure and human livelihoods. Accurate data on roofing materials is critical for modeling building vulnerability to natural hazards such as earthquakes, floods, wildfires, and hurricanes, yet such data remain unavailable. To address this gap, we introduce RoofNet, the largest and most geographically diverse novel multimodal dataset to date, comprising over 51,500 samples from 184 geographically diverse sites pairing high-resolution Earth Observation (EO) imagery with curated text annotations for global roof material classification. RoofNet includes geographically diverse satellite imagery labeled with 14 key roofing types and is designed to enhance the fidelity of global exposure datasets through vision-language modeling (VLM). We sample EO tiles from climatically and architecturally distinct regions to construct a representative dataset. A subset of 6,000 images was annotated in collaboration with domain experts to fine-tune a VLM. We used geographic- and material-aware prompt tuning to enhance class separability. The fine-tuned model was then applied to the remaining EO tiles, with predictions refined through rule-based and human-in-the-loop verification. In addition to material labels, RoofNet provides rich metadata including roof shape, footprint area, solar panel presence, and indicators of mixed roofing materials (e.g., HVAC systems). The dataset used in earlier experiments has been removed due to licensing constraints related to imagery sources. Results based on this dataset should be interpreted with caution. Updated experiments using compliant data are in progress.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces RoofNet as the largest and most geographically diverse multimodal dataset for roof material classification, comprising over 51,500 samples from 184 sites that pair high-resolution Earth Observation imagery with curated text annotations for 14 roofing types. It describes sampling EO tiles from climatically and architecturally distinct regions, expert annotation of a 6,000-image subset, fine-tuning a vision-language model via geographic- and material-aware prompt tuning, and subsequent application with rule-based and human-in-the-loop verification. The work also provides rich metadata including roof shape, footprint area, solar panel presence, and mixed-material indicators. The abstract explicitly states that the dataset used in earlier experiments has been removed due to licensing constraints on imagery sources, with updated experiments using compliant data stated to be in progress.

Significance. If a fully compliant and accessible version of the dataset were released with verified results, RoofNet would offer a substantial contribution to global building exposure datasets for natural hazard modeling. Its scale, geographic diversity across 184 sites, and multimodal design supporting vision-language models could improve fidelity in vulnerability assessments for earthquakes, floods, wildfires, and hurricanes, while the additional metadata would enable more granular analyses.

major comments (2)
  1. [Abstract] Abstract: The explicit statement that 'The dataset used in earlier experiments has been removed due to licensing constraints related to imagery sources' and that 'Results based on this dataset should be interpreted with caution' with 'Updated experiments using compliant data are in progress' directly undermines the central claim of introducing a usable RoofNet dataset. The core contribution is the paired EO imagery and labels, which are currently unavailable, preventing any verification or use of the claimed scale, diversity, or utility.
  2. [Abstract] Abstract: The premise that sampling from climatically and architecturally distinct regions yields a representative global distribution of roof materials, combined with expert annotations and rule-based verification producing accurate labels without significant bias, is load-bearing for the dataset's claimed value but receives only high-level description without quantitative validation or bias assessment.
minor comments (1)
  1. [Abstract] Abstract: The description of the VLM fine-tuning and prompt tuning process would benefit from explicit details on the number of classes, prompt templates, or performance metrics to clarify how class separability was enhanced.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive review and for highlighting important issues regarding dataset availability and validation. We address each major comment below, indicating revisions where appropriate.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The explicit statement that 'The dataset used in earlier experiments has been removed due to licensing constraints related to imagery sources' and that 'Results based on this dataset should be interpreted with caution' with 'Updated experiments using compliant data are in progress' directly undermines the central claim of introducing a usable RoofNet dataset. The core contribution is the paired EO imagery and labels, which are currently unavailable, preventing any verification or use of the claimed scale, diversity, or utility.

    Authors: We agree that the cautionary language in the abstract regarding the removed dataset creates uncertainty about immediate usability and verifiability. This change was necessitated by licensing restrictions discovered after initial experiments. In the revised manuscript we have updated the abstract to state that a fully compliant version of RoofNet, constructed from alternative licensed imagery sources, is now the primary contribution, with the original non-compliant data removed entirely. We have added a dedicated paragraph in the methods section describing the new data sources, acquisition process, and preliminary scale achieved with the compliant imagery. We also include a clear statement that the full dataset and updated results will be released upon completion of verification. This revision directly addresses the concern by shifting emphasis to the compliant resource while acknowledging the transitional status. revision: partial

  2. Referee: [Abstract] Abstract: The premise that sampling from climatically and architecturally distinct regions yields a representative global distribution of roof materials, combined with expert annotations and rule-based verification producing accurate labels without significant bias, is load-bearing for the dataset's claimed value but receives only high-level description without quantitative validation or bias assessment.

    Authors: We accept that the original manuscript relied on high-level descriptions of the sampling strategy and verification pipeline without sufficient quantitative support. To strengthen this foundation we have added a new subsection on dataset representativeness that includes: (1) a breakdown of roof-material class frequencies stratified by the 184 sites and major Köppen climate zones, (2) inter-annotator agreement statistics (Cohen’s kappa) from the 6,000-image expert subset, and (3) a preliminary bias analysis comparing observed material distributions against publicly available national building statistics for a subset of countries. These additions provide measurable evidence for the sampling and annotation approach and will be expanded with the compliant dataset release. revision: yes

Circularity Check

0 steps flagged

No circularity: dataset introduction paper with no derivation or fitted predictions

full rationale

This is a data-collection paper whose central contribution is the creation and description of RoofNet (51,500+ samples, 184 sites, 14 roof classes, VLM fine-tuning plus rule-based verification). The abstract and provided text contain no equations, no parameter fitting presented as out-of-sample prediction, no uniqueness theorems, and no self-citations that bear the load of any claimed result. Sampling from climatically distinct regions and expert annotation are procedural choices, not self-referential definitions or reductions of outputs to inputs. The licensing-removal note affects reproducibility but does not create any circular step in a derivation chain. The work is therefore self-contained as an empirical resource rather than a mathematical or predictive model.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper contributes primarily through data curation and sampling rather than new theoretical constructs or fitted parameters.

axioms (1)
  • domain assumption High-resolution Earth observation imagery combined with expert text annotations can reliably identify roof material types across diverse global regions.
    Invoked in the description of sampling EO tiles and the annotation workflow.

pith-pipeline@v0.9.0 · 5799 in / 1397 out tokens · 49071 ms · 2026-05-19T12:50:46.558696+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages · 2 internal anchors

  1. [1]

    Costello, Cécile L’Hermitte, and Nan Li

    Kenan Liu, Alice Chang-Richards, Seosamh B. Costello, Cécile L’Hermitte, and Nan Li. What factors influence building material supply for post-disaster reconstruction and recovery? A systematic review applying systems thinking.International Journal of Disaster Risk Reduction, 114:104981, November 2024

    work page 2024

  2. [2]

    Hurricane Costs, May 2025

    NOAA Office for Coastal Management. Hurricane Costs, May 2025

    work page 2025

  3. [3]

    Hurricane Ian – Response and Recovery, January 2025

    FEMA. Hurricane Ian – Response and Recovery, January 2025

    work page 2025

  4. [4]

    National Centers for Environmental Information (NCEI). U.S. Billion-dollar Weather and Climate Disasters, 1980 - present (NCEI Accession 0209268). Last Modified: 2025-05-02

    work page 1980

  5. [5]

    Natural Catastrophe and Climate Report: 2024

    Gallagher Re. Natural Catastrophe and Climate Report: 2024. 2024

    work page 2024

  6. [6]

    Patrick Aravena Pelizari, Christian Geiß, Paula Aguirre, Hernán Santa María, Yvonne Merino Peña, and Hannes Taubenböck. Automated building characterization for seismic risk assessment using street-level imagery and deep learning.ISPRS Journal of Photogrammetry and Remote Sensing, 180:370–386, October 2021

    work page 2021

  7. [7]

    Cool Roofs

    Department of Energy. Cool Roofs

    work page

  8. [8]

    Acevedo, Juan C

    Daniela Gonzalez, Diego Rueda-Plata, Ana B. Acevedo, Juan C. Duque, Raúl Ramos-Pollán, Alejandro Betancourt, and Sebastian García. Automatic detection of building typology using deep learning methods on street level images.Building and Environment, 177:106805, June 2020

    work page 2020

  9. [9]

    xBD: A Dataset for Assessing Building Damage from Satellite Imagery, November 2019

    Ritwik Gupta, Richard Hosfelt, Sandra Sajeev, Nirav Patel, Bryce Goodman, Jigar Doshi, Eric Heim, Howie Choset, and Matthew Gaston. xBD: A Dataset for Assessing Building Damage from Satellite Imagery, November 2019. arXiv:1911.09296 [cs]

    work page arXiv 2019

  10. [10]

    Deierlein, Frank McKenna, Adam Zsarnóczay, Tracy Kijewski-Correa, Ahsan Kareem, Wael Elhaddad, Laura Lowes, Matthew J

    Gregory G. Deierlein, Frank McKenna, Adam Zsarnóczay, Tracy Kijewski-Correa, Ahsan Kareem, Wael Elhaddad, Laura Lowes, Matthew J. Schoettler, and Sanjay Govindjee. A Cloud-Enabled Application Framework for Simulating Regional-Scale Impacts of Natural Hazards on the Built Environment.Frontiers in Built Environment, 6, November 2020. Publisher: Frontiers

    work page 2020

  11. [11]

    NHERI-SimCenter/BRAILS: Version 3.1.1, February 2024

    Barbaros Cetiner, Charles Wang, Frank McKenna, SaschaHornauer, Jinyan Zhao, Claudio Perez, and yunhuiguo. NHERI-SimCenter/BRAILS: Version 3.1.1, February 2024

    work page 2024

  12. [12]

    OpenStreetMap

    OpenStreetMap. OpenStreetMap

    work page

  13. [13]

    Residential building type classification from street-view imagery with convolutional neural networks.Signal, Image and Video Processing, 18(2):1949–1958, March 2024

    Ryan Murdoch and Ala’a Al-Habashna. Residential building type classification from street-view imagery with convolutional neural networks.Signal, Image and Video Processing, 18(2):1949–1958, March 2024

    work page 1949

  14. [14]

    Model Fusion for Building Type Classification from Aerial and Street View Images.Remote Sensing, 11(11):1259, January

    Eike Jens Hoffmann, Yuanyuan Wang, Martin Werner, Jian Kang, and Xiao Xiang Zhu. Model Fusion for Building Type Classification from Aerial and Street View Images.Remote Sensing, 11(11):1259, January

    work page

  15. [15]

    Number: 11 Publisher: Multidisciplinary Digital Publishing Institute

    work page

  16. [16]

    Building instance classification using street view images.ISPRS Journal of Photogrammetry and Remote Sensing, 145:44–59, November 2018

    Jian Kang, Marco Körner, Yuanyuan Wang, Hannes Taubenböck, and Xiao Xiang Zhu. Building instance classification using street view images.ISPRS Journal of Photogrammetry and Remote Sensing, 145:44–59, November 2018

    work page 2018

  17. [17]

    Building Information Modeling and Classification by Visual Learning At A City Scale, October 2019

    Qian Yu, Chaofeng Wang, Barbaros Cetiner, Stella Yu, Frank Mckenna, Ertugrul Taciroglu, and Kincho Law. Building Information Modeling and Classification by Visual Learning At A City Scale, October 2019. Publisher: Zenodo. 9

    work page 2019

  18. [18]

    Building usage prediction in complex urban scenes by fusing text and facade features from street view images using deep learning.Building and Environment, 267:112174, January 2025

    Surya Prasath Ramalingam and Vaibhav Kumar. Building usage prediction in complex urban scenes by fusing text and facade features from street view images using deep learning.Building and Environment, 267:112174, January 2025

    work page 2025

  19. [19]

    Zhuo Zheng, Yanfei Zhong, Junjue Wang, Ailong Ma, and Liangpei Zhang. Building damage assessment for rapid disaster response with a deep object-based semantic change detection framework: From natural disasters to man-made disasters.Remote Sensing of Environment, 265:112636, November 2021

    work page 2021

  20. [20]

    Mooney, Daniel A

    Dustin Fry, Stephen J. Mooney, Daniel A. Rodríguez, Waleska T. Caiaffa, and Gina S. Lovasi. Assessing Google Street View Image Availability in Latin American Cities.Journal of Urban Health, 97(4):552–560, August 2020

    work page 2020

  21. [21]

    Timo Lüddecke and Alexander S. Ecker. Image Segmentation Using Text and Image Prompts, March 2022. arXiv:2112.10003 [cs]

    work page arXiv 2022

  22. [22]

    Kruitwagen, K

    L. Kruitwagen, K. T. Story, J. Friedrich, L. Byers, S. Skillman, and C. Hepburn. A global inventory of photovoltaic solar energy generating units.Nature, 598(7882):604–610, October 2021. Publisher: Nature Publishing Group

    work page 2021

  23. [23]

    Thomas Y . Chen. Interpretability in Convolutional Neural Networks for Building Damage Classification in Satellite Imagery, January 2022. arXiv:2201.10523 [cs]

    work page arXiv 2022

  24. [24]

    PreDisM: Pre-Disaster Modelling With CNN Ensembles for At-Risk Communities, December 2021

    Vishal Anand and Yuki Miura. PreDisM: Pre-Disaster Modelling With CNN Ensembles for At-Risk Communities, December 2021. arXiv:2112.13465 [cs]

    work page arXiv 2021

  25. [25]

    RemoteCLIP: A Vision Language Foundation Model for Remote Sensing, April 2024

    Fan Liu, Delong Chen, Zhangqingyun Guan, Xiaocong Zhou, Jiale Zhu, Qiaolin Ye, Liyong Fu, and Jun Zhou. RemoteCLIP: A Vision Language Foundation Model for Remote Sensing, April 2024. arXiv:2306.11029 [cs]

    work page arXiv 2024

  26. [26]

    Overpass API, November 2024

    Roland Olbricht. Overpass API, November 2024

    work page 2024

  27. [27]

    Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, and Lei Zhang. Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection, July 2024. arXiv:2303.05499 [cs]

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  28. [28]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled Weight Decay Regularization, January 2019. arXiv:1711.05101 [cs] version: 3

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  29. [29]

    Contrastive Vision-Language Pre-training with Limited Resources, July 2022

    Quan Cui, Boyan Zhou, Yu Guo, Weidong Yin, Hao Wu, Osamu Yoshie, and Yubo Chen. Contrastive Vision-Language Pre-training with Limited Resources, July 2022. arXiv:2112.09331 [cs]

    work page arXiv 2022

  30. [30]

    2021 International Building Code, October 2020

    International Code Council, Inc. 2021 International Building Code, October 2020

    work page 2021

  31. [31]

    M. F. Huang, Qiang Li, C. M. Chan, W. J. Lou, K. C. S. Kwok, and G. Li. Performance-based design optimization of tall concrete framed structures subject to wind excitations.Journal of Wind Engineering and Industrial Aerodynamics, 139:70–81, April 2015

    work page 2015

  32. [32]

    Innovative Building Materials for Disaster Resilience

    ACMS. Innovative Building Materials for Disaster Resilience

    work page

  33. [33]

    Bracing for impact: how shifting precipitation extremes may influence physical climate risks in an uncertain future.Scientific Reports, 14(1):17398, July

    Saiful Haque Rahat, Shah Saki, Ummul Khaira, Nishan Kumar Biswas, Ishrat Jahan Dollan, Asphota Wasti, Yuki Miura, Md Abul Ehsan Bhuiyan, and Patrick Ray. Bracing for impact: how shifting precipitation extremes may influence physical climate risks in an uncertain future.Scientific Reports, 14(1):17398, July

    work page

  34. [34]

    Publisher: Nature Publishing Group

    work page

  35. [35]

    Why Polycarbonate Is The Right Roofing Material For A Greenhouse, September 2023

    ACME Plastics. Why Polycarbonate Is The Right Roofing Material For A Greenhouse, September 2023

    work page 2023

  36. [36]

    Study on the Cultural Impact of Local Materials in Regional Architectural Design.Commu- nications in Humanities Research, 45:9–14, August 2024

    Lingwen Sun. Study on the Cultural Impact of Local Materials in Regional Architectural Design.Commu- nications in Humanities Research, 45:9–14, August 2024. 10 A Technical Appendices and Supplementary Material Figure 5:Prompts were selected to maximize distance in material classifications while remaining in the 77 token limit. The image above shows how cl...

    work page 2024