Pith. sign in

REVIEW 3 major objections 5 minor 27 references

Sentiment Analysis of Typhoon Related Tweets using Standard and Bidirectional Recurrent Neural Networks

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An RNN-based analysis of 35,967 Typhoon Yolanda tweets finds that 51.1% were positive, 29% neutral, and 19.8% negative.

desk verdict Useful new dataset and a plausible qualitative result on Yolanda tweets, but the exact sentiment percentages are not reliable because the classifier was trained on balanced data and applied to the full corpus without calibration. read the letter →

arxiv 1908.01765 v1 pith:KL4OKBL7 submitted 2019-08-03 cs.NE cs.CL

Joseph Marvin Imperial , Jeyrome Orosco , Shiela Mae Mazo , Lany Maceda This is my paper · ORCID
classification cs.NEcs.CL
keywords sentimentanalysisrecurrentneuralnetworksbidirectionalRNNTwitterTyphoonYolandadisasterresponsenaturallanguageprocessingfine-grainedclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper analyzes 39,867 tweets collected before, during, and after Typhoon Yolanda (Haiyan) in the Philippines, aiming to measure public sentiment using standard and bidirectional recurrent neural networks. The authors manually annotated a balanced sample of 3,900 tweets as positive, negative, or neutral, trained RNN classifiers under several hyperparameter settings, and applied the best model to the remaining tweets. They report that 51.1% of the tweets were positive (support, prayers, encouragement), 29% neutral (news, announcements, observations), and 19.8% negative (sadness, despair, anger at officials). The best classifiers reached 81.79% accuracy for three-way classification and 87.69% for binary classification. If these figures hold, they indicate that public sentiment during a major disaster was predominantly supportive rather than panicked.

What carries the argument

The central mechanism is a pair of recurrent-neural-network classifiers, one standard and one bidirectional, trained on Word2Vec vector representations of cleaned tweets. In a standard RNN, recurrent connections keep a hidden state that carries information from earlier words into later predictions; a bidirectional RNN stacks a reversed second layer so the model draws on both past and future context for each word. Training used a balanced gold standard of 3,900 manually labeled tweets (1,300 per class), with dropout, batch size, and backpropagation-through-time (truncated for the standard RNN, full for the bidirectional) as the varied hyperparameters. The trained standard RNN's mapping from tweet text to positive, neutral, or negative, validated at 81.79% test accuracy, is what produces the sentiment distribution reported for the full dataset.

What would settle it

Have independent annotators label a fresh random sample of tweets from the same collection period and compare the positive proportion to 51.1%; a large gap would refute the paper's central finding.

Watch

Extended reading notes

Core claim

The paper reports that public sentiment in Yolanda-related tweets was predominantly positive: when the best standard RNN model was applied to the 35,967 tweets remaining after preprocessing, 18,395 (51.1%) were classified as positive, 10,441 (29%) as neutral, and 7,131 (19.8%) as negative. Positive tweets were those expressing prayers, sympathy, support, and gratitude; neutral tweets were mostly news and relief announcements; negative tweets conveyed sadness, despair, and anger toward officials. The same experiments showed that both RNN variants perform well, with the standard RNN reaching 81.79% three-way accuracy and the bidirectional RNN reaching 87.69% two-way accuracy.

Load-bearing premise

The load-bearing premise is that the 3,900 hand-labeled tweets are a fair and accurate sample, so the classifier's measured accuracy transfers to the 35,967 tweets it labels automatically.

Editorial extensions

If this is right

  • If the 51.1% positive figure is right, disaster-response tools should not assume social-media sentiment in a crisis is dominated by fear; support and encouragement can be the largest class.
  • The 29% neutral share, mostly news and relief announcements, is a separable operational channel, so a three-way sentiment model can help filter actionable announcements from emotional posts.
  • The hyperparameter results give concrete starting points for similar tweet-sentiment tasks: batch size 64 outperformed 128 across both model types, and dropout helped the standard RNN's three-way model.
  • Because false negatives dominated the errors, improving recall for the missed class is the clearest path to better accuracy in follow-up work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not separate the effect of the collection hashtags themselves; #BangonPH and #BangonPilipinas ('rise up, Philippines') are rallying phrases that could inflate the positive share, so a hashtag-neutral collection would test this.
  • Given the false-negative error pattern, the 51.1% positive estimate may understate the true positive share when applied to the full dataset.
  • The same annotation-and-training pipeline could be applied to other Philippine disaster events to see whether a supportive majority is a general pattern of crisis tweeting rather than a Yolanda-specific result.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper describes a sentiment analysis study of Typhoon Yolanda-related tweets in the Philippines. The authors collected 92,040 tweets via hashtag search, preprocessed them down to 39,867 tweets, manually annotated a balanced sample of 3,900 tweets into positive, negative, and neutral classes, and trained standard and bidirectional RNNs for both fine-grained (three-class) and binary classification. The best reported accuracies are 81.79% (fine-grained, standard RNN) and 87.69% (binary, bidirectional RNN). The fine-grained standard RNN is then applied to the remaining 35,967 unlabeled tweets, producing a claimed distribution of 51.1% positive, 29% neutral, and 19.8% negative, which the paper interprets as evidence that Filipinos predominantly expressed support and encouragement during the disaster.

Significance. If the estimated sentiment distribution were valid, the paper would provide a useful descriptive account of public sentiment during a major natural disaster in a low-resource language setting, and the standard-versus-bidirectional RNN comparison would be of modest interest to applied NLP. Strengths of the paper include a clearly specified data collection and preprocessing pipeline, a manually annotated gold standard rather than purely automatic labels, and full reporting of the hyperparameter grid in Tables 1 and 2, which supports replication. However, the central quantitative claim is not yet supported: the distribution is computed from raw predictions of a classifier trained on a balanced sample without calibration or prior correction, and the reported accuracies lack uncertainty quantification. These issues are fixable, but they affect the headline result.

major comments (3)
  1. [Section 4 and Table 1] The central sentiment distribution is computed as raw predicted counts from the fine-grained standard RNN applied to the 35,967 unlabeled tweets, and the classifier was trained on a class-balanced sample of 1,300 tweets per class (§3.4). For any imperfect classifier, the expected predicted class proportion is a convex combination of the true class proportions through the confusion matrix; unless the classifier is calibrated to the true prior, the reported 51.1% / 29% / 19.8% split is a biased estimator of the true sentiment distribution whenever the true prior is not uniform. The paper reports only overall accuracy and F1 (Table 1), not per-class precision/recall or a confusion matrix, so the bias cannot be computed or corrected from the reported results. Please provide the confusion matrix and either apply a prior correction or calibrate the model before aggregating, and report uncertainty intervals for the final percentages.
  2. [Sections 3.4, 3.5, and Tables 1-2] The reported accuracies are point estimates on a single 780-tweet test set, and the best model appears to be selected by comparing these test-set accuracies without a held-out validation set. The claim in Section 4 that batch size 64 produced 'significantly higher' accuracy is not supported by any significance test or confidence interval; with 780 test instances, the difference between 0.8179 and 0.8141 is within sampling error. Please separate model selection (via validation) from final evaluation (via a held-out test set), and report confidence intervals or bootstrap resampling for all accuracy and F1 estimates.
  3. [Section 3.3 (Data Annotation)] The gold standard consists of 3,900 tweets labeled by the authors, but no inter-annotator agreement measure (e.g., Cohen's kappa) is reported, and the annotation procedure is not described in terms of number of annotators per tweet or conflict resolution. If a single annotator labeled each tweet, the reported accuracy partly measures agreement with one person's subjective judgments, which weakens the claim that the 51.1% distribution reflects tweet content rather than annotator idiosyncrasy. Please report annotation reliability and procedure.
minor comments (5)
  1. [Abstract vs. Section 3.1] The abstract states that the collection window ends on January 31, 2014, while Section 3.1 states January 1, 2014; please reconcile the date discrepancy.
  2. [Section 3.4] The paper does not specify how the binary classification dataset was formed from the three annotated classes, e.g., whether neutral tweets were discarded or relabeled.
  3. [Section 3.2.1] The Word2Vec feature description reports example neighboring words but not the embedding dimension, window size, minimum count, or training algorithm, which prevents replication of the feature representation.
  4. [Figure 5] Figure 5 is described as showing the change of sentiment over months, but the figure lacks axis labels and numeric values, so the monthly trend cannot be verified; please add a table or annotated plot.
  5. [References] Several references are incomplete or informal (for example, references [10], [16], and [17] lack complete publication details); please provide full citations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the sentiment distribution is an empirical classifier output, not a quantity defined by the fitted parameters.

full rationale

The paper's central result, the 51.1% positive / 19.8% negative / 29% neutral breakdown (Section 4), is obtained by applying a trained standard RNN to 35,967 previously unlabeled tweets. The model is fitted on 3,900 human-annotated tweets, and the reported distribution is a direct aggregation of predicted labels; no equation in the paper defines the distribution in terms of the training labels or fitted weights, and no parameter is renamed as a prediction. The balanced training set and lack of prior calibration are genuine external-validity/statistical-bias concerns about whether the predicted proportions estimate the true sentiment prior, but they are not circularity: the human annotations are independent ground truth, and the classifier output is not forced by construction to match the reported percentages. The paper contains no self-citations that are load-bearing, no imported uniqueness theorem, and no ansatz smuggled in via citation. The derivation chain is self-contained in the sense that the empirical claim is a measurement, not a tautology.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the quality and representativeness of the manually labeled sample and on the transferability of test accuracy to unlabeled data. No new physical or conceptual entities are introduced.

free parameters (1)
  • Hyperparameter configuration (batch size, learning rate, dropout) = batch_size=64, learning_rate=1.80e-03, dropout=0.5 (for best models)
    Chosen via test-set accuracy comparison without a held-out validation set; the reported sentiment distribution depends on this model selection.
assumptions (3)
  • domain assumption The 3,900 manually annotated tweets are a representative and correctly labeled sample of the full dataset.
    Used as gold standard in Section 3.3; no inter-annotator agreement or sampling protocol described.
  • ad hoc to paper Test accuracy on the balanced 780-tweet test set generalizes to the 35,967 unlabeled tweets.
    The model is applied to all remaining tweets in Section 4, but the unlabeled data may have a different class distribution.
  • domain assumption The three hashtags fully capture Yolanda-related Filipino tweets.
    Data collection in Section 3.1 uses only #YolandaPH, #BangonPH, #BangonPilipinas; other relevant conversations may be missed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sentiment Analysis of Typhoon Related Tweets using Standard and Bidirectional Recurrent Neural Networks." pith.science (2026). https://pith.science/paper/KL4OKBL7

@misc{pith2026190801765,
  author       = {Pith},
  title        = {Pith review of: Sentiment Analysis of Typhoon Related Tweets using Standard and Bidirectional Recurrent Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KL4OKBL7}},
  note         = {Machine review of arXiv:1908.01765}
}
read the original abstract

The Philippines is a common ground to natural calamities like typhoons, floods, volcanic eruptions and earthquakes. With Twitter as one of the most used social media platform in the Philippines, a total of 39,867 preprocessed tweets were obtained given a time frame starting from November 1, 2013 to January 31, 2014. Sentiment analysis determines the underlying emotion given a series of words. The main purpose of this study is to identify the sentiments expressed in the tweets sent by the Filipino people before, during, and after Typhoon Yolanda using two variations of Recurrent Neural Networks; standard and bidirectional. The best generated models after training with various hyperparameters achieved a high accuracy of 81.79% for fine-grained classification using standard RNN and 87.69% for binary classification using bidirectional RNN. Findings revealed that 51.1% of the tweets sent were positive expressing support, love, and words of courage to the victims; 19.8% were negative stating sadness and despair for the loss of lives and hate for corrupt officials; while the other 29% were neutral tweets from local news stations, announcements of relief operations, donation drives, and observations by citizens.

Figures

Figures reproduced from arXiv: 1908.01765 by the authors.

Figure 1
Figure 1. 3.1 Data Collection Typhoon Yolanda related tweets for training and testing were gathered using the keywords ‘#YolandaPH’, ‘#BangonPH’, and ‘#BangonPilipinas’ starting from November 1, 2013 to January 1, 2014. A total of 92,040 tweets were successfully obtained from the given time period. 3.2 Data Preprocessing Dataset that will be used should be cleaned to avoid inclusion of noise and unnecessary information during… view at source ↗
Figure 2
Figure 2. Sample words with their neighboring words in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 26 canonical work pages

  1. [1]

    INTRODUCTION The Philippines is a common birthplace of natural disasters and calamities. Due to its unfortunate geographical location along the western rim of Pacific Ocean, the Philippine Area of Responsibility (PAR) is vulnerable to frequent storm surges and formations of low pressure areas (LPA) . [3] In a single year, around 20 tropical cyclones visit...

    work page 2013

  2. [2]

    [8] Facebook and Twitter have become indispensable tools for organizations, especially during times of disaster or crisis

    RELATED WORKS 2.1 Twitter as Common Grounds for Sentiments Social media platforms like Twitter and Facebook offer s an environment called ‘participatory media’ which allows its users not only to look and take in the media but also to actively participate in the creation and distribution of content. [8] Facebook and Twitter have become indispensable tools ...

    work page

  3. [3]

    METHODOLOGY This study follows the process of sentiment ana lysis shown in Figure 1. 3.1 Data Collection Typhoon Yolanda related tweets for training and testing were gathered using the keywords ‘#YolandaPH’, ‘#BangonPH’, and ‘#BangonPilipinas’ starting from November 1, 2013 to January 1,

    work page 2013

  4. [4]

    DISCUSSION OF FINDINGS Table 1 and 2 below shows the result of extensive configuration and testing of different hyperparameters of standard and bidirectional Recurrent Neural Networks and their yielded accuracy for fine-grained and binary classification. Model Batch Size Learning Rate Drop Out BPTT Type ACC F1 Score Standard RNN 64 1.80E- 03 - tBPTT 0.803...

    work page 2013

  5. [5]

    CONCLUSION AND FUTURE WORK The main purpose of this study is to analyze and determine the sentiments of the tweets sent by the Filipino people before, during, and after Typhoon Yolanda. This process was made possible using the sequential process of standard and bidi rectional Recurrent Neural Networks (RNN) with relatively high accuracy generated given pr...

    work page

  6. [6]

    Nathaniel Oco and his team at National University for providing continuous support and advice in this study

    ACKNOWLEDGMENTS The proponents would like to thank Prof. Nathaniel Oco and his team at National University for providing continuous support and advice in this study

    work page

  7. [7]

    Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts

    Dos Santos, C., & Gatti, M. Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts. Brazilian Research Lab

    work page

  8. [8]

    (2017, July 19)

    A Gentle Introduction to Backpropagation Through Time. (2017, July 19). Retrieved January 12, 2018, from https://machinelearningmastery.com/gentle-introduction- backpropagation-time/

    work page 2017

Show all 27 references
  1. [9]

    (2016, July 08)

    Britz, D. (2016, July 08). Recurrent Neural Networks Tutorial, Part 1 – Introduction to RNNs. Retrieved November 27, 2017, from http://www.wildml.com/2015/09/recurrent -neural- networks-tutorial-part-1-introduction-to-rnns/

    2016

  2. [10]

    (2013, November 11)

    Brown, S. (2013, November 11). The Philippines Is the Most Storm-Exposed Country on Earth. Retrieved November 21, 2017, from http://world.time.com/2013/11/11/the - philippines-is-the-most-storm-exposed-country-on-earth/

    2013

  3. [11]

    Cruz, G. D. IN NUMBERS: Typhoons in the Philippines and the 201 6 polls. Retrieved November 21, 2017, from https://www.rappler.com/move-ph/issues/disasters/126001- typhoons-enter-philippines-fast-facts

    2017

  4. [12]

    PLoS ONE 11(3): e0150190

    David CC, Ong JC, Legara EFT (2016) Tweeting Supertyphoon Haiyan: Evolving Functions of Twitter during and after a Disaster Event. PLoS ONE 11(3): e0150190

    2016

  5. [13]

    Twitter Might Make Tweets Longer. Here’s Why They’re 140 Characters to Begin with

    Dewey, Caitlin. 2015. “Twitter Might Make Tweets Longer. Here’s Why They’re 140 Characters to Begin with.” Washington Post, October 1. https://www.washingtonpost.com/news/the-intersect/ wp/2015/10/01/twitter-might-make-tweets-longer-heres- why-theyre-140-characters-to-beginwit...

    2015

  6. [14]

    (2015, August 05)

    Philippines Typhoon Facts and Figures. (2015, August 05). Retrieved November 21, 2017, from https://www.dec.org.uk/articles/facts-and-figures

    2015

  7. [15]

    Fire, Wind and Water: Social Networks in Natural Disasters

    Freeman, Mark. 2011. “Fire, Wind and Water: Social Networks in Natural Disasters.” Journal of Cases on Information Technology 13 (2): 69–79

    2011

  8. [16]

    Graves, A. (2012). Supervised Sequence Labelling with Recurrent Neural Networks (Ser. 385). Springer-Verlag Berlin Heidelberg. DOI:10.1007/978-3-642-24797-2

    2012 doi

  9. [17]

    Montecillo, P. (2012). Philippines has 9.5M twitter users, ranks 10th. Retrieved from http://technology.inquirer.net/15189/philippines-has9-5m- twitter-users-ranks-10th

    2012

  10. [18]

    Mozetič I, Grčar M, Smailović J (2016) Multilingual Twitter Sentiment Classification: The Role of Human Annotators

    2016

  11. [19]

    Dropout:A simple way to prevent neural networks from overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout:A simple way to prevent neural networks from overfitting. JMLR, 2014

    2014

  12. [20]

    Pang, B., & Lee, L. (2002). Opinion Mining and Sen timent Analysis. Now Publisher Inc

    2002

  13. [22]

    (2013, November 16)

    Quick facts: What you need to know about Super Typhoon Haiyan. (2013, November 16). Retrieved November 21, 2017, from https://reliefweb.int/report/philippines/quick -facts- what-you-need-know-about-super-typhoon-haiyan

    2013

  14. [23]

    PH spends most time online and on social media – report

    R. PH spends most time online and on social media – report. Retrieved November 22, 2017, from https://www.rappler.com/technology/features/159720-ph- spends-most-time-online-and-on-social-media-report

    2017

  15. [24]

    Retrieved November 24, 2017, from https://www.brandwatch.com/blog/understanding-sentiment- analysis/

    Sentiment Analysis: How Does It Work? Why Should We Use It? (2016, August 16). Retrieved November 24, 2017, from https://www.brandwatch.com/blog/understanding-sentiment- analysis/

    2016

  16. [25]

    Retrieved November 21, 2017, from http://www.gmanetwork.com/news/scitech/science/334571/s uper-typhoon-yolanda-is-strongest-storm-ever-to-make- landfall-in-recorded-history/story/

    Super Typhoon Yolanda is strongest storm ever to make landfall in recorded history. Retrieved November 21, 2017, from http://www.gmanetwork.com/news/scitech/science/334571/s uper-typhoon-yolanda-is-strongest-storm-ever-to-make- landfall-in-recorded-history/story/

    2017

  17. [26]

    Retrieved November 21, 2017, from https://www.omnicoreagency.com/twitter-statistics/

    Twitter by the Numbers: Stats, Demographics & Fun Facts. Retrieved November 21, 2017, from https://www.omnicoreagency.com/twitter-statistics/

    2017

  18. [27]

    D., Burnett, E., Lamm, A., & Lucas, J

    Waters, R. D., Burnett, E., Lamm, A., & Lucas, J. (2009). Engaging stakeholders through social networki ng: How nonprofit organizations are using Facebook. Public Relations Review

    2009

  19. [2014]

    3.2 Data Preprocessing Dataset that will be used should be cleaned to avoid inclusion of noise and unnecessary information during training

    A total of 92,040 tweets were successfully obtained from the given time period. 3.2 Data Preprocessing Dataset that will be used should be cleaned to avoid inclusion of noise and unnecessary information during training. Duplicate tweets, usernames, external links, URLs, retwee...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.