RECORD
Social Media Text Processing and Semantic Analysis for Smart Cities
Social and Information Networks · Submitted 11 Sep 2017
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Results are in line with research work in other application areas by showing the robustness of using word embeddings to learn word similarities that bag-of-words is not able to capture.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
With the rise of Social Media, people obtain and share information almost instantly on a 24/7 basis. Many research areas have tried to gain valuable insights from these large volumes of freely available user generated content. With the goal of extracting knowledge from social media streams that might be useful in the context of intelligent transportation systems and smart cities, we designed and developed a framework that provides functionalities for parallel collection of geo-located tweets from multiple pre-defined bounding boxes (cities or regions), including filtering of non-complying tweets, text pre-processing for Portuguese and English language, topic modeling, and transportation-specific text classifiers, as well as, aggregation and data visualization. We performed an exploratory data analysis of geo-located tweets in 5 different cities: Rio de Janeiro, S\~ao Paulo, New York City, London and Melbourne, comprising a total of more than 43 million tweets in a period of 3 months. Furthermore, we performed a large scale topic modelling comparison between Rio de Janeiro and S\~ao Paulo. Interestingly, most of the topics are shared between both cities which despite being in the same country are considered very different regarding population, economy and lifestyle. We take advantage of recent developments in word embeddings and train such representations from the collections of geo-located tweets. We then use a combination of bag-of-embeddings and traditional bag-of-words to train travel-related classifiers in both Portuguese and English to filter travel-related content from non-related. We created specific gold-standard data to perform empirical evaluation of the resulting classifiers. Results are in line with research work in other application areas by showing the robustness of using word embeddings to learn word similarities that bag-of-words is not able to capture.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | With the rise of Social Media, people obtain and share information almost instantly on a 24/7 basis. | background 1.00 | 0.04 | 0.01 | asserted 1.00 |
| s1 | Many research areas have tried to gain valuable insights from these large volumes of freely available user generated content. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s2 | With the goal of extracting knowledge from social media streams that might be useful in the context of intelligent transportation systems and smart cities, we designed and developed a framework that provides functionalities for parallel collection of geo-located tweets from multiple pre-defined bounding boxes (cities or regions), including filtering of non-complying tweets, text pre-processing for Portuguese and English language, topic modeling, and transportation-specific text classifiers, as well as, aggregation and data visualization. | contribution 0.54 | 0.96 | 0.07 | asserted 0.90 |
| s3 | We performed an exploratory data analysis of geo-located tweets in 5 different cities: Rio de Janeiro, S\~ao Paulo, New York City, London and Melbourne, comprising a total of more than 43 million tweets in a period of 3 months. | method 0.77 | 0.98 | 0.03 | asserted 1.00 |
| s4 | Furthermore, we performed a large scale topic modelling comparison between Rio de Janeiro and S\~ao Paulo. | method 0.51 | 0.97 | 0.03 | asserted 0.93 |
| s5 | Interestingly, most of the topics are shared between both cities which despite being in the same country are considered very different regarding population, economy and lifestyle. | result 0.55 | 0.89 | 0.17 | asserted 0.99 |
| s6 | We take advantage of recent developments in word embeddings and train such representations from the collections of geo-located tweets. | method 0.97 | 0.94 | 0.03 | asserted 0.97 |
| s7 | We then use a combination of bag-of-embeddings and traditional bag-of-words to train travel-related classifiers in both Portuguese and English to filter travel-related content from non-related. | method 0.99 | 0.97 | 0.04 | asserted 0.97 |
| s8 | We created specific gold-standard data to perform empirical evaluation of the resulting classifiers. | method 0.93 | 0.97 | 0.03 | asserted 0.98 |
| s9 | Results are in line with research work in other application areas by showing the robustness of using word embeddings to learn word similarities that bag-of-words is not able to capture. | interpretation 0.70 | 0.78 | 0.50 | asserted 0.83 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
With the rise of Social Media, people obtain and share information almost instantly on a 24/7 basis. Many research areas have tried to gain valuable insights from these large volumes of freely available user generated content. With the goal of extracting knowledge from social media streams that might be useful in the context of intelligent transportation systems and smart cities, we designed and developed a framework that provides functionalities for parallel collection of geo-located tweets from multiple pre-defined bounding boxes (cities or regions), including filtering of non-complying tweets, text pre-processing for Portuguese and English language, topic modeling, and transportation-specific text classifiers, as well as, aggregation and data visualization. We performed an exploratory data analysis of geo-located tweets in 5 different cities: Rio de Janeiro, S\~ao Paulo, New York City, London and Melbourne, comprising a total of more than 43 million tweets in a period of 3 months. Furthermore, we performed a large scale topic modelling comparison between Rio de Janeiro and S\~ao Paulo. Interestingly, most of the topics are shared between both cities which despite being in the same country are considered very different regarding population, economy and lifestyle. We take advantage of recent developments in word embeddings and train such representations from the collections of geo-located tweets. We then use a combination of bag-of-embeddings and traditional bag-of-words to train travel-related classifiers in both Portuguese and English to filter travel-related content from non-related. We created specific gold-standard data to perform empirical evaluation of the resulting classifiers. Results are in line with research work in other application areas by showing the robustness of using word embeddings to learn word similarities that bag-of-words is not able to capture.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 4 sentences without a confident label
Cites as background 1 paper
-
SVM, OLS, RF, MLP, NB and DT J48 are some of the supervised classification models used to analyse social media data over fields such as health [ref] and pharmacovigilance, political opinion [ref] , transportation (travel classification [ref] , traffic and incidents detection [ref] ), financial sentiment analysis [ref] and online reputation monitoring [ref] .
FEUP at SemEval-2017 Task 5: Predicting Sentiment Polarity and Intensity with Financial Word Embeddings · Text Classification · 0.85
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.