pith. sign in

arxiv: 2407.08145 · v3 · submitted 2024-07-11 · 💻 cs.SE

Engineering for Crisis Management: A User-Centred Analysis of Disaster Mobile Applications

Muhamad Syukron , Anuradha Madugalla , Mojtaba Shahin , John Grundy This is my paper

Pith reviewed 2026-05-23 22:47 UTC · model grok-4.3

classification 💻 cs.SE
keywords disaster mobile appsuser reviewscrisis managementdisaster lifecycletopic modellingsentiment analysisusabilityaccessibility
0
0 comments X

The pith

Disaster mobile apps mostly prioritize response functions while providing limited support for preparedness and recovery phases.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The study reviews 70 disaster mobile applications drawn from research literature and major app stores to map their features against the full disaster lifecycle. It finds heavy emphasis on immediate response tools and weak coverage of preparation before events or recovery afterward. Analysis of user reviews through topic modeling and sentiment analysis surfaces repeated complaints about app crashes, confusing interfaces, barriers for people with disabilities, and unclear alerts. The authors then outline practical steps for developers and agencies to address these gaps.

Core claim

Analysis of 70 apps shows prioritization of response-related functionalities with limited support for preparedness and recovery; topic modelling and sentiment analysis of user reviews reveals critical challenges related to technical reliability, usability, accessibility, and information clarity.

What carries the argument

Categorization of apps by disaster focus, geographic scope, popularity, monetization, and features across the disaster lifecycle, followed by extraction, translation, topic modelling, and sentiment analysis of user reviews.

If this is right

  • Developers should adopt lifecycle-oriented design that balances features across preparedness, response, and recovery.
  • Apps need stronger multilingual support and technical robustness to reduce crashes and improve reliability.
  • Integrating ongoing user feedback into development processes can improve accessibility and information clarity.
  • Emergency management agencies can use these insights when commissioning or endorsing apps for public use.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Agencies might consider creating shared open data standards so apps can pull verified hazard information from official sources rather than duplicating efforts.
  • Testing apps with diverse user groups during non-emergency periods could surface accessibility issues earlier than post-disaster reviews allow.
  • Long-term maintenance funding models may be needed because one-time development often leaves apps outdated when hazard data or operating systems change.

Load-bearing premise

The 70 apps identified from literature and app stores form a representative sample of existing disaster apps, and the collected reviews plus topic modelling faithfully reflect genuine user concerns without selection or translation bias.

What would settle it

A broader survey of additional apps or a fresh set of user reviews that shows equal or greater support for preparedness and recovery phases, or identifies different primary complaints, would undermine the reported priorities and challenges.

Figures

Figures reproduced from arXiv: 2407.08145 by Anuradha Madugalla, John Grundy, Mojtaba Shahin, Muhamad Syukron.

Figure 1
Figure 1. Figure 1: Overview of our study approach functionalities of these apps and evaluate their effectiveness for disaster risk reduction [30]. They did not explore the user reviews that are available in these apps for insights into user perception of features and app performance. Studies analyzing app reviews, described in Section 2.2, typically do not cover disaster apps. This is mainly because these studies choose the … view at source ↗
Figure 2
Figure 2. Figure 2: App pricing scheme, number of downloads, and rating Analysis 5- Monetization Strategies: Not all disaster support apps are made freely available by government agen￾cies. To gather data about how app developers may be sup￾ported we examined the homepage of each app. The home page indicated whether the app contained advertising or offers in-app purchases (payments within the app to access certain features or… view at source ↗
Figure 3
Figure 3. Figure 3: Features of Preparation Stage • Features of Response Stage: When a disaster occurs, several features can be utilized. 1. Real Time Alerts: This alert appears as some form of pop-up, informing the user of a disaster happening in the area according to their radius settings or specified area of interest. This feature differs from an early warning alert because notifications can arrive during a disaster, not b… view at source ↗
Figure 4
Figure 4. Figure 4: Features of Response Stage is provided in ‘QuakeFeed Earthquake Tracker’ app (see Figure 4a). 2. Disaster Maps: Such a feature aims to provide some form of geographic-based information to users regarding a disaster occurring at a specific location or coverage area, presented in the form of an interactive map. Such an example can be seen in ‘Hazards Near Me NSW’ app in Figure 4b). 3. Real-Time Tracking: Thi… view at source ↗
Figure 5
Figure 5. Figure 5: Features of Recovery Stage • Features of Recovery Stage: The key features available in the post-disaster stage. 1. Disaster Updates: Information regarding disaster events can be presented in the form of short updates. This usually consist of a list format, such as a list of all the earthquakes in the past month. In such a feature, users can view disaster statistics, such as the magnitude of the disaster, t… view at source ↗
Figure 6
Figure 6. Figure 6: Features of Mitigation Stage • Features of Mitigation Stage: This stage is about taking steps to prevent the impact of future disasters. 1. Help/Shelter Info: In preparation for the next dis￾aster, users can search for the nearest shelter using some apps. This will help them prepare evacuation routes and evacuation modes in preparation for a future disaster. Such an example if found in ‘Emer￾gency: Severe … view at source ↗
Figure 7
Figure 7. Figure 7: Topic Modelling Process Subsequently, the dimensionality of the embedding vector is reduced using UMAP (Uniform Manifold Approximation and Projection) [38]. UMAP was selected for dimension reduction in BERTopic for its superior embedding stability and significantly faster runtime compared to other methods such as T-SNE [39]. Once reduced, the embedding vector is clustered using HDBSCAN (Hierarchical densit… view at source ↗
Figure 8
Figure 8. Figure 8: Sentiment Analysis Process Sentiment Analysis: This RQ examines which topics have the highest percentage of negative reviews. This in￾formation can help developers identify dominant user com￾plaints and prioritize their resolution. Ratings serve as an indicator to assess user satisfaction with the app. However, there is potential for Text-Rating-Inconsistency (TRI) to arise because some users may give rati… view at source ↗
read the original abstract

Disaster mobile apps play an increasingly important role in disseminating hazard information and supporting communities during emergency situations. This study presents a comprehensive analysis of these mobile applications, focusing on their features, user-reported challenges, and opportunities for improvement. We first examined the landscape of disaster mobile apps by analysing 70 apps identified through a combination of methods, including those from the literature, the Google Play Store, and the App Store. The analysis categorised apps based on disaster focus, geographic coverage, popularity, monetisation strategies, and features across the disaster lifecycle. We then extracted, translated and analysed user reviews using topic modelling and sentiment analysis to identify key concerns and recurring issues. The results show that most applications prioritise response-related functionalities, with limited support for preparedness and recovery. User feedback highlights critical challenges related to technical reliability, usability, accessibility, and information clarity. Based on these findings, we propose a set of recommendations for developers and emergency management agencies to improve the reliability, inclusiveness, and overall effectiveness of disaster mobile apps. These include adopting lifecycle-oriented design approaches, strengthening multilingual support, improving technical robustness, and integrating user feedback into development processes. This work contributes to the growing body of research on human-centred disaster risk reduction by providing empirical insights and actionable guidance for the design of more reliable and inclusive disaster communication systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper analyzes 70 disaster mobile apps sourced from literature, Google Play Store, and App Store searches. Apps are categorized by disaster focus, geographic coverage, popularity, monetization, and features mapped to the disaster lifecycle phases. User reviews are extracted, translated, and analyzed via topic modeling and sentiment analysis to surface challenges in technical reliability, usability, accessibility, and information clarity. The central claims are that apps heavily prioritize response-phase functionalities with limited preparedness and recovery support, and that user feedback reveals recurring issues; recommendations for lifecycle-oriented design, multilingual support, and robustness are offered.

Significance. If the sample and review-analysis concerns are resolved, the work supplies empirical evidence on feature distribution and user pain points in disaster apps, supporting human-centered design in crisis management. The mixed-methods combination of feature taxonomy and review mining is a constructive contribution; the lifecycle framing and concrete recommendations for developers and agencies add practical value.

major comments (2)
  1. [Section 3] Section 3 (App Identification and Selection): No search strings, inclusion/exclusion criteria, geographic/language scope, or count of screened/discarded apps are reported for the literature + store searches that yielded the final 70 apps. This directly affects the representativeness claim and the generalizability of the lifecycle-feature counts.
  2. [Section 4.2] Section 4.2 (Review Processing and Topic Modelling): Non-English reviews are translated before topic modelling and sentiment analysis, yet no validation (back-translation, bilingual coder agreement, or error quantification) is described. Systematic distortion of topics or polarity cannot be ruled out and undermines the reported user challenges.
minor comments (2)
  1. [Abstract] Abstract and Section 2: The phrase 'comprehensive analysis' should be qualified given the acknowledged gaps in selection transparency.
  2. [Results] Tables/Figures showing feature distributions: Add explicit sample sizes per category and note any overlaps between the three data sources.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. The two major comments highlight important areas for improving methodological transparency, which we will address in the revision.

read point-by-point responses

  1. Referee: [Section 3] Section 3 (App Identification and Selection): No search strings, inclusion/exclusion criteria, geographic/language scope, or count of screened/discarded apps are reported for the literature + store searches that yielded the final 70 apps. This directly affects the representativeness claim and the generalizability of the lifecycle-feature counts.

    Authors: We agree that the app identification and selection process requires more detailed reporting to support claims of representativeness. In the revised manuscript, Section 3 will be expanded to include the exact search strings used for the literature search and app store queries, the full set of inclusion/exclusion criteria applied, the geographic and language scope considered, and the counts of apps screened and discarded at each stage. These additions will allow readers to better assess the sample and the generalizability of the feature distribution findings. revision: yes

  2. Referee: [Section 4.2] Section 4.2 (Review Processing and Topic Modelling): Non-English reviews are translated before topic modelling and sentiment analysis, yet no validation (back-translation, bilingual coder agreement, or error quantification) is described. Systematic distortion of topics or polarity cannot be ruled out and undermines the reported user challenges.

    Authors: We recognize that explicit validation of the translation step is necessary to strengthen confidence in the topic modeling and sentiment results. The revised Section 4.2 will describe any validation procedures performed (e.g., back-translation of a sample of reviews or inter-coder agreement checks) and, if formal validation was limited, will explicitly discuss this as a methodological limitation along with its potential implications for the identified user challenges. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical aggregation of external app data and reviews

full rationale

The paper performs a landscape analysis of 70 disaster apps sourced from literature, Google Play, and App Store, followed by review extraction, translation, topic modelling, and sentiment analysis. No equations, fitted parameters, derivations, or self-citation chains appear in the provided text. All claims (response-phase prioritization, user challenges in reliability/usability) are direct outputs of the described external-data processing steps rather than reductions to prior author work or definitional inputs. The analysis is self-contained against external benchmarks and does not invoke uniqueness theorems or ansatzes from the authors' prior publications.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is an empirical analysis paper with no mathematical derivations, fitted parameters, background axioms, or postulated entities.

pith-pipeline@v0.9.0 · 5774 in / 1236 out tokens · 28870 ms · 2026-05-23T22:47:21.938392+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 43 canonical work pages · 2 internal anchors

  1. [1]

    W. W. Attribution, Heavy precipitation hitting vulnerable communities in the uae and oman becoming an increasing threat as the climate warms – world weather attribution, https://www.worldweatherattribution.org/heavy- precipitation-hitting-vulnerable-communities-in- the-uae-and-oman-becoming-an-increasing-threat-as- the-climate-warms/, (Accessed on 07/10/2024)

    work page 2024

  2. [2]

    Topics, How many people own smartphones? (2024-2029), https://explodingtopics.com/blog/smartphone-stats, (Ac- cessed on 03/19/2024) (2023)

    E. Topics, How many people own smartphones? (2024-2029), https://explodingtopics.com/blog/smartphone-stats, (Ac- cessed on 03/19/2024) (2023)

    work page 2024

  3. [3]

    M. L. Tan, R. Prasanna, K. Stock, E. E. Doyle, G. Leonard, D. Johnston, Usability factors influencing the continuance intention of disaster apps: A mixed-methods study, International Journal of Disaster Risk Reduction 50 (2020) 101874

    work page 2020

  4. [4]

    M. L. Tan, R. Prasanna, K. Stock, E. E. Doyle, G. Leonard, D. John- ston, Understanding end-users’ perspectives: Towards developing usability guidelines for disaster apps, Progress in Disaster Science 7 (2020) 100118

    work page 2020

  5. [5]

    Dallo, M

    I. Dallo, M. Marti, Why should i use a multi-hazard app? assessing the public’s information needs and app feature preferences in a participatory process, International journal of disaster risk reduction 57 (2021) 102197

    work page 2021

  6. [6]

    Cfa app glitch: New vic emergency app provides incorrect fire ban information - abc news, https://www.abc.net.au/news/2016- 11-22/cfa-app-glitch-vic-emergency-app-provides- incorrect-fire-ban/8042572, (Accessed on 05/13/2024)

    work page arXiv 2016

  7. [7]

    U. F. E. M. Agency, 04 unit 4.doc, https://training. fema.gov/emiweb/downloads/is111_unit%204.pdf, (Accessed on 05/14/2024)

    work page 2024

  8. [8]

    Poser, D

    K. Poser, D. Dransch, V olunteered geographic information for disas- ter management with application to rapid flood damage estimation, Geomatica 64 (1) (2010) 89–98

    work page 2010

  9. [9]

    M. L. Tan, R. Prasanna, K. Stock, E. Hudson-Doyle, G. Leonard, D. Johnston, Mobile applications in crisis informatics literature: A systematic review, International Journal of Disaster Risk Reduction 24 (2017) 297–311. doi:10.1016/j.ijdrr.2017.06.009

    work page doi:10.1016/j.ijdrr.2017.06.009 2017

  10. [10]

    G. D. Haddow, K. S. Haddow, Chapter four - disaster coverage past and present, in: G. D. Haddow, K. S. Haddow (Eds.), Disaster Communications in a Changing Media World (Second Edition), second edition Edition, Butterworth-Heinemann, 2014, pp. 53–70

    work page 2014

  11. [11]

    D ˛ abrowski, E

    J. D ˛ abrowski, E. Letier, A. Perini, A. Susi, Analysing app reviews for software engineering: a systematic literature review, Empirical Software Engineering 27 (2) (2022) 43

    work page 2022

  12. [12]

    Genc-Nayebi, A

    N. Genc-Nayebi, A. Abran, A systematic literature review: Opinion mining studies from mobile app store user reviews, Journal of Systems and Software 125 (2017) 207–219

    work page 2017

  13. [13]

    Johann, C

    T. Johann, C. Stanik, W. Maalej, et al., Safe: A simple approach for feature extraction from app descriptions and app reviews, in: 2017 IEEE 25th international requirements engineering conference (RE), IEEE, 2017, pp. 21–30

    work page 2017

  14. [14]

    A. F. de Araújo, R. M. Marcacini, Re-bert: automatic extraction of software requirements from app reviews using bert language model, in: Proceedings of the 36th annual ACM symposium on applied computing, 2021, pp. 1321–1327

    work page 2021

  15. [15]

    N. Jha, A. Mahmoud, Mining non-functional requirements from app store reviews, Empirical Software Engineering 24 (2019) 3659–3695

    work page 2019

  16. [16]

    M. Lu, P. Liang, Automatic classification of non-functional require- ments from augmented app user reviews, in: Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering, 2017, pp. 344–353

    work page 2017

  17. [17]

    Y . Man, C. Gao, M. R. Lyu, J. Jiang, Experience report: Understand- ing cross-platform app issues from user reviews, in: 2016 IEEE 27th international symposium on software reliability engineering (ISSRE), IEEE, 2016, pp. 138–149

    work page 2016

  18. [18]

    Q. Chen, C. Chen, S. Hassan, Z. Xing, X. Xia, A. E. Hassan, How should i improve the ui of my app? a study of user reviews of popular apps in the google play, ACM Transactions on Software Engineering and Methodology (TOSEM) 30 (3) (2021) 1–38

    work page 2021

  19. [19]

    Maalej, H

    W. Maalej, H. Nabil, Bug report, feature request, or simply praise? on automatically classifying app reviews, in: 2015 IEEE 23rd inter- national requirements engineering conference (RE), IEEE, 2015, pp. 116–125

    work page 2015

  20. [20]

    McIlroy, N

    S. McIlroy, N. Ali, H. Khalid, A. E. Hassan, Analyzing and automat- ically labelling the types of user issues that are raised in mobile app reviews, Empirical Software Engineering 21 (2016) 1067–1106

    work page 2016

  21. [21]

    Martens, W

    D. Martens, W. Maalej, Towards understanding and detecting fake reviews in app stores, Empirical Software Engineering 24 (6) (2019) 3316–3355

    work page 2019

  22. [22]

    E. Noei, F. Zhang, Y . Zou, Too many user-reviews! what should app developers look at first?, IEEE Transactions on Software Engineering 47 (2) (2019) 367–378

    work page 2019

  23. [23]

    Martens, T

    D. Martens, T. Johann, On the emotion of users in app reviews, in: 2017 IEEE/ACM 2nd International workshop on emotion awareness in software engineering (SEmotion), IEEE, 2017, pp. 8–14

    work page 2017

  24. [24]

    W. Luiz, F. Viegas, R. Alencar, F. Mourão, T. Salles, D. Carvalho, M. A. Gonçalves, L. Rocha, A feature-oriented sentiment rating for mobile app reviews, in: Proceedings of the 2018 world wide web conference, 2018, pp. 1909–1918

    work page 2018

  25. [25]

    Shahin, M

    M. Shahin, M. Zahedi, H. Khalajzadeh, A. R. Nasab, A study of gender discussions in mobile apps, in: 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR), IEEE, 2023, pp. 598–610

    work page 2023

  26. [26]

    H. O. Obie, H. Du, K. Madampe, M. Shahin, I. Ilekura, J. Grundy, L. Li, J. Whittle, B. Turhan, H. Khalajzadeh, Automated detection, categorisation and developers’ experience with the violations of honesty in mobile apps, Empirical Software Engineering 28 (6) (2023) 134

    work page 2023

  27. [27]

    R. A. Shams, W. Hussain, G. Oliver, A. Nurwidyantoro, H. Perera, J. Whittle, Society-oriented applications development: Investigating users’ values from bangladeshi agriculture mobile applications, in: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Society, 2020, pp. 53–62

    work page 2020

  28. [28]

    Bowie-DaBreo, C

    D. Bowie-DaBreo, C. Sas, H. Iles-Smith, S. Sünram-Lea, User per- spectives and ethical experiences of apps for depression: a qualitative analysis of user reviews, in: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022, pp. 1–24

    work page 2022

  29. [29]

    P. Nema, P. Anthonysamy, N. Taft, S. T. Peddinti, Analyzing user perspectives on mobile app privacy at scale, in: Proceedings of the 44th International Conference on Software Engineering, 2022, pp. 112–124

    work page 2022

  30. [30]

    Navarro De Corcuera, M

    L. Navarro De Corcuera, M. D. M. Barbero-Barrera, A. Campos Hi- dalgo, J. Recio Martínez, Assessment of the adequacy of mobile applications for disaster reduction 24 (5) 6197–6223. Muhamad et al.: Preprint submitted to Elsevier Page 17 of 19 A Comprehensive Study of Disaster Support Mobile Apps

    work page

  31. [31]

    Chembakottu, H

    B. Chembakottu, H. Li, F. Khomh, A large-scale exploratory study of android sports apps in the google play store, Information and Software Technology 164 (2023) 107321.doi:10.1016/j.infsof. 2023.107321

    work page doi:10.1016/j.infsof 2023

  32. [32]

    Navarro de Corcuera, M

    L. Navarro de Corcuera, M. d. M. Barbero-Barrera, A. Campos Hi- dalgo, J. Recio Martínez, Assessment of the adequacy of mobile applications for disaster reduction, Environment, Development and Sustainability 24 (5) (2021) 6197–6223. doi:10.1007/s10668- 021-01697-2

    work page doi:10.1007/s10668- 2021

  33. [33]

    Lewis, 8 free apps that could save your life in japan, medium (Sep 2018)

    R. Lewis, 8 free apps that could save your life in japan, medium (Sep 2018). URL https://medium.com/@robintlewis/8-free-apps-that- could-save-your-life-in-japan-8a8b6b61b955

    work page 2018

  34. [34]

    of Encyclopaedia Britannica, List of countries (November 11 2019)

    E. of Encyclopaedia Britannica, List of countries (November 11 2019). URL https://www.britannica.com/topic/list-of- countries-1993160

    work page 2019

  35. [35]

    Sherif, Global market share held by mobile operating systems from 2009 to 2021, accessed: March 18, 2024 (2024)

    A. Sherif, Global market share held by mobile operating systems from 2009 to 2021, accessed: March 18, 2024 (2024). URL https://www.statista.com/statistics/272698/ global-market-share-held-by-mobile-operating- systems-since-2009/

    work page 2009

  36. [36]

    BERTopic: Neural topic modeling with a class-based TF-IDF procedure

    M. Grootendorst, Bertopic: Neural topic modeling with a class-based tf-idf procedure (2022). arXiv:2203.05794

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  37. [37]

    M. P. Grootendorst, Parameter tuning - bertopic, GitHub.io (2023). URL https://maartengr.github.io/BERTopic/getting_ started/parameter%20tuning/parametertuning.html#n_ components

    work page 2023

  38. [38]

    UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

    L. McInnes, J. Healy, J. Melville, Umap: Uniform manifold approx- imation and projection for dimension reduction (2020). arXiv: 1802.03426

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  39. [39]

    van der Maaten, G

    L. van der Maaten, G. Hinton, Visualizing data using t-sne, Journal of Machine Learning Research 9 (86) (2008) 2579–2605. URL http://jmlr.org/papers/v9/vandermaaten08a.html

    work page 2008

  40. [40]

    McInnes, J

    L. McInnes, J. Healy, J. Melville, hdbscan: Hierarchical density based clustering, Journal of Open Source Software 2 (11) (2017) 205. doi:10.21105/joss.00205

    work page doi:10.21105/joss.00205 2017

  41. [41]

    B. Fu, J. Lin, L. Li, C. Faloutsos, J. Hong, N. Sadeh, Why people hate your app: making sense of user feedback in a mobile app store, in: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’13, Association for Computing Machinery, New York, NY , USA, 2013, p. 1276–1284. doi:10.1145/2487575.2488202. URL ...

    work page doi:10.1145/2487575.2488202 2013

  42. [42]

    Hutto, E

    C. Hutto, E. Gilbert, Vader: A parsimonious rule-based model for sentiment analysis of social media text, in: Proceedings of the International AAAI Conference on Web and Social Media, V ol. 8, 2014, pp. 216–225. URL https://doi.org/10.1609/icwsm.v8i1.14550

    work page doi:10.1609/icwsm.v8i1.14550 2014

  43. [43]

    J. L. Campbell, C. Quincy, J. Osserman, O. K. Pedersen, Coding in-depth semistructured interviews: Problems of unitization and in- tercoder reliability and agreement, Sociological methods & research 42 (3) (2013) 294–320. Muhamad et al.: Preprint submitted to Elsevier Page 18 of 19 A Comprehensive Study of Disaster Support Mobile Apps Table 9 Region-Speci...

    work page 2013