Pith. sign in

REVIEW 3 major objections 6 minor 32 references

A Large-Scale Empirical Study of Geotagging Behavior on Twitter

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Geotagged tweets are not a random sample of public opinion: language alone predicts geotagging from under 3% (Korean) to over 40% (Indonesian).

desk verdict Large descriptive study with a valuable cautionary message, but the headline homophily claim is confounded by degree and activity and needs reanalysis. read the letter →

arxiv 1908.10948 v1 pith:7N2ECJ3D submitted 2019-08-28 cs.SI cs.HC

Binxuan Huang , Kathleen M. Carley This is my paper · ORCID
classification cs.SIcs.HC
keywords geotaggingTwitteruserbehaviorempiricalstudyhomophilylocationpredictionlanguagedifferencessocialmedia
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tests a quiet assumption behind much social-media research: that tweets carrying a location tag are a fair sample of what people are saying. Using 41 billion tweets from 20 million Twitter users, it shows the assumption fails. Geotagging is sharply uneven across groups—fewer than 3% of Korean-speaking users ever geotag, while more than 40% of Indonesian-speaking users do—and it tracks other disclosure choices: people who write a recognizable home location in their profile are far more likely to share real-time coordinates. The same preference also runs in social ties, since users connected to a geotagging friend are about six times more likely to geotag than users with no such friend. If this is right, opinion estimates and location-prediction systems built only on geotagged users carry a built-in bias toward people who choose to be visible.

What carries the argument

The central object is a user-level geotagging label: after collecting each user's timeline, the authors call a user geotagged if any tweet has a place tag or precise coordinates, and coordinates-tagged if any tweet has precise coordinates. Around that label they build three partitions—major tweet source, major tweet language, and whether a free-text profile location resolves to a place through a public gazetteer—and compare geotagging rates across partitions. The argument is carried by the combination of these partitions with standard statistical comparisons (chi-square tests for rate differences and Kolmogorov-Smirnov tests for the distribution of geotagged friends), plus the ego-alter conditional probabilities that express the homophily effect as a sixfold increase.

What would settle it

Use Twitter's full-archive search or an account's downloadable data to inspect the complete tweet history, including tweets older than the platform's 3,200-tweet limit, for a random sample of accounts the paper classifies as non-geotagged. If a meaningful fraction of those accounts have geotagged earlier, then the ever-geotagged rate and the friend-homophily conditional probabilities are truncated-history artifacts rather than true behavioral patterns.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that geotagging behavior on Twitter is a systematic user-level trait rather than a random property of tweets. Over the collected 41.27 billion tweets, only 2.31% carry a place tag or coordinates, but when aggregated per user, 24.38% of the 19.98 million users have geotagged at least once and 12.93% have shared precise coordinates. The rate varies by platform (iPhone users geotag more than Android users), by country (Indonesia and Turkey show exceptionally high coordinates-tagging), and sharply by language: 2.94% of Korean-speaking users are geotagged versus 42.47% of Indonesian-speaking users. Users who report a recognizable location in their profile are also much more likely to be geotagged (33.03%) than users with an empty profile field (15.91%). Finally, geotagging is homophilous: geotagged users' followees and followers are geotagged at roughly 12 to 15 percentage points higher than the corresponding ties of non-geotagged users, and the conditional probability that a user geotags jumps from about 4% to roughly 26-31% once at least one connected user geotags. The authors take these patterns as evidence against the common assumption that geotagged and non-geotagged users are exchangeable.

Load-bearing premise

The paper assumes that the 20 million users drawn from Twitter's streaming API are a random sample of all Twitter users, and that the most recent 3,200 tweets per user reveal every user's true geotagging behavior; if active users or truncated histories are unrepresentative, every rate and the homophily effect could be off.

Editorial extensions

If this is right

  • Regional opinion studies that use geotagged tweets as a proxy for local sentiment should reweight text by language- and country-specific geotagging likelihood instead of treating all geotags as equal draws.
  • Location-prediction classifiers trained on geotagged users inherit a selection bias, because users who geotag are also disproportionately the users who fill in their profile location, making the training labels non-representative of the non-geotagged users being predicted.
  • Graph-based location inference becomes harder than assumed: if non-geotagged users cluster with non-geotagged users, then friendship ties carry less location signal for exactly the users who most need location inference.
  • Cross-country comparisons of tweet volume on a topic can be misleading, since the same real-world concern can produce far more geotagged discussion in Turkey or Indonesia than in Japan or Korea purely because of geotagging propensity.
  • The strong friend correlation suggests geotagging behavior may spread through social influence, so studies of location-based social behavior should account for network autocorrelation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The person-level correlation between profile disclosure and geotagging points to a stable privacy-sensitivity trait; one could test this by checking whether users who disable geotagging also withhold profile details and engage less with location-based features.
  • The same selection mechanism likely distorts other volunteered geographic information beyond Twitter, such as check-in services or photo geotags, so reweighting by user-level propensity could improve cross-platform location studies.
  • The observed homophily could be confounded by spatial homophily: people who live near each other share language and culture and both geotag; a direct test would compare geotagging similarity between friends who live far apart, where social influence is the only channel.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper reports a large-scale descriptive study of geotagging behavior on Twitter, based on roughly 40 billion tweets from about 20 million users. It makes three main claims: (1) different groups of users have very different geotagging rates, e.g., by language and client platform; (2) users who fill in a recognizable profile location are more likely to geotag their tweets; and (3) there is a strong homophily effect in geotagging preferences, so that geotagged users tend to be connected to geotagged users. The authors argue these findings challenge common assumptions behind opinion inference from geotagged tweets and behind location prediction systems. The paper is organized at three levels: tweet-level analysis, user-level analysis, and graph-level analysis, with the homophily claim supported by degree-based comparisons and conditional probabilities in Section VI.

Significance. If the findings hold, they are practically important: they imply that geotagged tweets are not a representative sample of public opinion and that classifiers trained on geotagged users may not transfer to non-geotagged users. The scale of the dataset is a genuine strength, and the descriptive tables are useful reference material. I also credit the authors for clearly separating tweet-level and user-level definitions of geotagging and for reporting the source-platform and language breakdowns in detail. The study is descriptive rather than model-based, so circularity is not a concern. However, two load-bearing issues prevent me from endorsing the strong conclusions as stated: the sampling and user-geotagging definitions are not as representative as claimed, and the homophily analysis does not control for degree or activity, even though the paper's own Figure 3 shows those variables are strongly associated with geotagging.

major comments (3)
  1. [Section VI, Figure 4, Tables IX-X] The 'strong homophily effect' claim is confounded by degree and activity. Figure 3 shows that the probability of being geotagged rises rapidly with the number of followers/followees before declining, so geotagged and non-geotagged users have systematically different degree distributions. The comparisons in Figure 4 and Tables IX-X do not condition on the ego's degree, tweet volume, or activity level. This matters for two reasons. First, if networks are degree-assortative, a geotagged ego with many alters tends to have alters with many alters, and those alters have a higher baseline geotag rate for reasons unrelated to geotagging-preference homophily. Second, the conditional-probability framing in Tables IX-X compares P(ego geotagged | at least one alter geotagged) with P(ego geotagged | no alter geotagged); the former increases mechanically with the number of alters, so the reported 'at least 6 times' ratios conflate tie count with preference similarity. I recommend re-analyzing the homophily claim with degree-stratified comparisons, controlling for the number of alters, or using a matched/regression design that holds degree and activity fixed.
  2. [Section III.A and Section III.B, Table I] The sampling frame is not a uniform random sample of the Twitter population. The paper states in Section III.A, 'We consider this sampled streaming data representing a random sample of the Twitter population,' but the data actually consist of users who appear in Twitter's public sample stream and whose timelines and following lists are publicly accessible. Users with protected accounts, very low activity, or accounts not captured by the stream are excluded, and the stream itself is not a uniform random sample of users. In addition, the user-level definition of geotagging relies on the most recent 3,200 tweets per user, so heavy users' older geotags are systematically missed. These issues directly affect the headline user-level rates reported in Section III.B (e.g., 24.38% geotagged users) and the language/source breakdowns in Tables VI and VII, because activity level is correlated with tweet volume. Please either provide explicit evidence for the representativeness assumption, restrict the claims to the sampled population, or report robustness checks stratified by tweet volume and account age.
  3. [Section VI, KS tests] The Kolmogorov-Smirnov tests in Section VI are run on millions of observations, so p<0.001 is expected and not informative. The reported KS statistics (0.255 to 0.431) are moderate, but the text interprets them as showing 'significant difference' without discussing effect sizes or practical significance. I recommend reporting standardized mean differences, overlapping coefficients, or other effect-size measures, and acknowledging that with this sample size any small distributional shift will be statistically significant.
minor comments (6)
  1. [Abstract] The phrase 'may affects the generability' should be corrected to 'may affect the generalizability.'
  2. [Table II and Section III.A] The text spells the bot platform as 'twitterbot.net' while Table II lists 'twittbot.net'; please make the names consistent. The footnote URL in Section III.A is also malformed ('en9/docs').
  3. [Figure 1] The figure has dual axes but the axes are not labeled; please label the left and right axes explicitly so the reader can interpret the bar chart and the trend lines.
  4. [Section IV.B, Table V] Table V reports coordinates-tagged percentages among geotagged tweets only, because non-geotagged tweets lack country labels. The text should state this more prominently so readers do not interpret the table as country-level geotagging rates.
  5. [Section VI] The analysis restricts to users with at least five followers/followees, but the threshold is not justified. Please report sensitivity of the homophily results to this threshold.
  6. [Section VII] The concluding sentence says the study is based on '20 million randomly sampled Twitter users,' but given the caveats in Section III.A, 'randomly sampled' is too strong; please qualify the description.

Circularity Check

0 steps flagged · score 0.0 of 10

Descriptive empirical study with direct data summaries; no fitted inputs, equations, or self-citation chains make the claims equivalent to their inputs.

full rationale

This paper is a descriptive empirical study. Its three central findings—geotagging varies by language and platform, users with self-reported profile locations are more likely to geotag, and geotagging preferences are associated between connected users—are all direct summaries of the collected Twitter data. There are no fitted parameters, no model equations, and no derived quantity that is later presented as a prediction. The user-level definition of geotagged (at least one geotagged tweet in the collected timeline) and the graph-level comparisons (percentages and conditional probabilities) are operationalizations of the data, not reductions of a claimed result to its own inputs. The authors' self-citations (e.g., references [10], [15], [18], [19], [22], and [23]) appear only as related work or background for existing geolocation techniques, not as load-bearing evidence for the present findings. The potential confound of degree or activity in the homophily analysis is a correctness and validity concern, not a circularity concern: the observed association between ego and alter geotagging status is an empirical quantity independent of how it is interpreted. No circular step can be quoted and exhibited, so the appropriate finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No fitted parameters are used in this descriptive study, but several data-collection and filtering choices act as assumptions that the reported percentages and homophily effects depend on. The minimum-degree threshold of five is an arbitrary cutoff that affects the graph-level results.

free parameters (1)
  • minimum degree threshold = 5
    Users with fewer than five followers or followees are excluded from the homophily analysis (Section VI). This hand-chosen cutoff affects all friend-percentage distributions and conditional probabilities reported in Tables IX and X.
assumptions (4)
  • domain assumption The Twitter sample streaming API returns a random sample of the Twitter population.
    Stated in Section III.A; all user-level percentages and generalizability claims depend on this.
  • domain assumption The 3,200 most recent tweets per user are sufficient to determine whether a user has ever geotagged.
    Section III.A notes the API cap but still uses 'ever posted a geotagged tweet' as the user-level definition; heavy users with early geotags may be misclassified.
  • domain assumption Twitter's BCP-47 language detection is accurate enough to assign a user's major language.
    Used to compute language-specific geotagging rates in Tables IV and VII; misdetection would change the reported contrasts.
  • domain assumption Users with at least five followers/followees are representative of the full graph for homophily analysis.
    Section VI restricts the study to users with at least five links and does not assess whether this filters out a biased subpopulation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Large-Scale Empirical Study of Geotagging Behavior on Twitter." pith.science (2026). https://pith.science/paper/7N2ECJ3D

@misc{pith2026190810948,
  author       = {Pith},
  title        = {Pith review of: A Large-Scale Empirical Study of Geotagging Behavior on Twitter},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7N2ECJ3D}},
  note         = {Machine review of arXiv:1908.10948}
}
read the original abstract

Geotagging on social media has become an important proxy for understanding people's mobility and social events. Research that uses geotags to infer public opinions relies on several key assumptions about the behavior of geotagged and non-geotagged users. However, these assumptions have not been fully validated. Lack of understanding the geotagging behavior prohibits people further utilizing it. In this paper, we present an empirical study of geotagging behavior on Twitter based on more than 40 billion tweets collected from 20 million users. There are three main findings that may challenge these common assumptions. Firstly, different groups of users have different geotagging preferences. For example, less than 3% of users speaking in Korean are geotagged, while more than 40% of users speaking in Indonesian use geotags. Secondly, users who report their locations in profiles are more likely to use geotags, which may affects the generability of those location prediction systems on non-geotagged users. Thirdly, strong homophily effect exists in users' geotagging behavior, that users tend to connect to friends with similar geotagging preferences.

Figures

Figures reproduced from arXiv: 1908.10948 by the authors.

Figure 1
Figure 1. Left axis is associated with the bar chart of the [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The cumulative distribution for distance between rec [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The percentages of different types of users with various [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 30 canonical work pages

  1. [1]

    Analysis of a location-based social network,

    N. Li and G. Chen, “Analysis of a location-based social network,” in Computational Science and Engineering, 2009. CSE’09. International Conference on, vol. 4. Ieee, 2009, pp. 263–270

    work page 2009

  2. [2]

    An empirical study of geographic user activity patterns in foursquare

    A. Noulas, S. Scellato, C. Mascolo, and M. Pontil, “An empirical study of geographic user activity patterns in foursquare.” ICwSM, vol. 11, no. 70-573, p. 2, 2011

    work page 2011

  3. [3]

    Spatial topic modeling in online social media for location recommendation,

    B. Hu and M. Ester, “Spatial topic modeling in online social media for location recommendation,” in Proceedings of the 7th ACM conference on Recommender systems . ACM, 2013, pp. 25–32

    work page 2013

  4. [4]

    Decision tree analysis of tourists’ preferences regarding tourist attractions using geotag data from social media,

    T. N. Maeda, M. Yoshida, F. Toriumi, and H. Ohashi, “Decision tree analysis of tourists’ preferences regarding tourist attractions using geotag data from social media,” in Proceedings of the Second International Conference on IoT in Urban Space . ACM, 2016, pp. 61–64

    work page 2016

  5. [5]

    Location-based event detection using geotagged semantic graphs,

    Y . Wei and L. Singh, “Location-based event detection using geotagged semantic graphs,” in KDD Workshop Mining and Learning with Graphs, 2017

    work page 2017

  6. [6]

    # earth- quake: Twitter as a distributed sensor system,

    A. Crooks, A. Croitoru, A. Stefanidis, and J. Radzikowski, “# earth- quake: Twitter as a distributed sensor system,” Transactions in GIS , vol. 17, no. 1, pp. 124–147, 2013

    work page 2013

  7. [7]

    Friendship and mobility: user movement in location-based social networks,

    E. Cho, S. A. Myers, and J. Leskovec, “Friendship and mobility: user movement in location-based social networks,” inProceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2011, pp. 1082–1090

    work page 2011

  8. [8]

    Using geolocated twitter data to monitor the prevalence of healthy and unhealthy food references across the us,

    M. J. Widener and W. Li, “Using geolocated twitter data to monitor the prevalence of healthy and unhealthy food references across the us,” Applied Geography, vol. 54, pp. 189–197, 2014

    work page 2014

Show all 32 references
  1. [9]

    Where in the world are you? geolocation and language identification in twitter,

    M. Graham, S. A. Hale, and D. Gaffney, “Where in the world are you? geolocation and language identification in twitter,” The Professional Geographer, vol. 66, no. 4, pp. 568–578, 2014

    2014

  2. [10]

    On predicting geolocation of tweets using convolutional neural networks,

    B. Huang and K. M. Carley, “On predicting geolocation of tweets using convolutional neural networks,” in International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation . Springer, 2017, pp. 281– 291

    2017

  3. [11]

    Birds of a feather: Homophily in social networks,

    M. McPherson, L. Smith-Lovin, and J. M. Cook, “Birds of a feather: Homophily in social networks,” Annual review of sociology , vol. 27, no. 1, pp. 415–444, 2001

    2001

  4. [12]

    Find me if you can: improving geographical prediction with social and spatial proximity,

    L. Backstrom, E. Sun, and C. Marlow, “Find me if you can: improving geographical prediction with social and spatial proximity,” in Proceed- ings of the 19th international conference on World wide web . ACM, 2010, pp. 61–70

    2010

  5. [13]

    Twitter user geolocation us- ing a unified text and network prediction model,

    A. Rahimi, T. Cohn, and T. Baldwin, “Twitter user geolocation us- ing a unified text and network prediction model,” arXiv preprint arXiv:1506.08259, 2015

    2015 arXiv

  6. [14]

    Understanding human mobility from twitter,

    R. Jurdak, K. Zhao, J. Liu, M. AbouJaoude, M. Cameron, and D. Newth, “Understanding human mobility from twitter,” PloS one, vol. 10, no. 7, p. e0131469, 2015

    2015

  7. [15]

    Location order recovery in trails with low temporal resolution,

    B. Huang and K. M. Carley, “Location order recovery in trails with low temporal resolution,” IEEE Transactions on Network Science and Engineering, 2018

    2018

  8. [16]

    Event-based analysis of people’s activities and behavior using flickr and panoramio geotagged photo collections,

    S. Kisilevich, M. Krstajic, D. Keim, N. Andrienko, and G. Andrienko, “Event-based analysis of people’s activities and behavior using flickr and panoramio geotagged photo collections,” in Information visualisation (IV), 2010 14th international conference . IEEE, 2010, pp. 289–296

    2010

  9. [17]

    Geo-located twitter as proxy for global mobility patterns,

    B. Hawelka, I. Sitko, E. Beinat, S. Sobolevsky, P. Kazakopoulos, and C. Ratti, “Geo-located twitter as proxy for global mobility patterns,” Cartography and Geographic Information Science , vol. 41, no. 3, pp. 260–271, 2014

    2014

  10. [18]

    Parameterized convolutional neural networks for aspect level sentiment classification,

    B. Huang and K. Carley, “Parameterized convolutional neural networks for aspect level sentiment classification,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , 2018, pp. 1091–1096

    2018

  11. [19]

    Aspect level sentiment classifi- cation with attention-over-attention neural networks,

    B. Huang, Y . Ou, and K. M. Carley, “Aspect level sentiment classifi- cation with attention-over-attention neural networks,” in International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation . Spr...

    2018

  12. [20]

    Earthquake shakes twitter users: real-time event detection by social sensors,

    T. Sakaki, M. Okazaki, and Y . Matsuo, “Earthquake shakes twitter users: real-time event detection by social sensors,” in Proceedings of the 19th international conference on World wide web. ACM, 2010, pp. 851–860

    2010

  13. [21]

    Geolocation prediction in social me- dia data by finding location indicative words,

    B. Han, P. Cook, and T. Baldwin, “Geolocation prediction in social me- dia data by finding location indicative words,” Proceedings of COLING 2012, pp. 1045–1062, 2012

    2012

  14. [22]

    Rate: Overcoming noise and sparsity of textual features in real-time location estimation,

    Y . Zhang, W. Wei, B. Huang, K. M. Carley, and Y . Zhang, “Rate: Overcoming noise and sparsity of textual features in real-time location estimation,” in Proceedings of the 2017 ACM on Conference on Infor- mation and Knowledge Management . ACM, 2017, pp. 2423–2426

    2017

  15. [23]

    A probabilistic framework for location inference from social media,

    Y . Qian, J. Tang, Z. Yang, B. Huang, W. Wei, and K. M. Carley, “A probabilistic framework for location inference from social media,” arXiv preprint arXiv:1702.07281, 2017

    2017 arXiv

  16. [24]

    What drives you to check in on facebook? motivations, privacy concerns, and mobile phone involvement for location-based information sharing,

    H.-S. Kim, “What drives you to check in on facebook? motivations, privacy concerns, and mobile phone involvement for location-based information sharing,” Computers in Human Behavior , vol. 54, pp. 397– 406, 2016

    2016

  17. [25]

    State of the geotags: Motivations and recent changes

    D. Tasse, Z. Liu, A. Sciuto, and J. I. Hong, “State of the geotags: Motivations and recent changes.” in ICWSM, 2017, pp. 250–259

    2017

  18. [26]

    Capturing, sharing, and using local place information,

    P. J. Ludford, R. Priedhorsky, K. Reily, and L. Terveen, “Capturing, sharing, and using local place information,” in Proceedings of the SIGCHI conference on Human factors in computing systems . ACM, 2007, pp. 1235–1244

    2007

  19. [27]

    Who tweets with their location? understanding the relationship between demographic characteristics and the use of geoservices and geotagging on twitter,

    L. Sloan and J. Morgan, “Who tweets with their location? understanding the relationship between demographic characteristics and the use of geoservices and geotagging on twitter,” PloS one , vol. 10, no. 11, p. e0142209, 2015

    2015

  20. [28]

    Snowball sampling: Problems and tech- niques of chain referral sampling,

    P. Biernacki and D. Waldorf, “Snowball sampling: Problems and tech- niques of chain referral sampling,” Sociological methods & research , vol. 10, no. 2, pp. 141–163, 1981

    1981

  21. [29]

    Knowing the tweeters: Deriving sociologically relevant demographics from twitter,

    L. Sloan, J. Morgan, W. Housley, M. Williams, A. Edwards, P. Burnap, and O. Rana, “Knowing the tweeters: Deriving sociologically relevant demographics from twitter,” Sociological research online, vol. 18, no. 3, pp. 1–11, 2013

    2013

  22. [30]

    Text-based twitter user geolocation prediction,

    B. Han, P. Cook, and T. Baldwin, “Text-based twitter user geolocation prediction,” Journal of Artificial Intelligence Research, vol. 49, pp. 451– 500, 2014

    2014

  23. [31]

    A stacking-based approach to twitter user geolocation prediction,

    ——, “A stacking-based approach to twitter user geolocation prediction,” in Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics: System Demonstrations , 2013, pp. 7–12

    2013

  24. [32]

    Network studies of social influence,

    P. V . Marsden and N. E. Friedkin, “Network studies of social influence,” Sociological Methods & Research , vol. 22, no. 1, pp. 127–151, 1993

    1993

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.