pith. sign in

arxiv: 2509.11443 · v1 · submitted 2025-09-14 · 💻 cs.CL · cs.SI

A Transformer-Based Cross-Platform Analysis of Public Discourse on the 15-Minute City Paradigm

Gaurab Chhetri , Darrell Anderson , Boniphace Kutela , Subasish Das This is my paper

Pith reviewed 2026-05-18 16:11 UTC · model grok-4.3

classification 💻 cs.CL cs.SI
keywords sentiment analysis15-minute citytransformer modelspublic discoursecross-platform analysissocial mediaurban planningcompressed models
0
0 comments X

The pith

Compressed transformer models classify public sentiment on 15-minute cities across Twitter, Reddit and news sources.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets up a pipeline to measure how people discuss the 15-minute city idea by pulling text from Twitter, Reddit, and news outlets. It first uses Llama-3-8B to assign sentiment labels, then trains five smaller transformer models on those labels and tests them with cross-validation. DistilRoBERTa reaches the highest F1 score of 0.8292 while TinyBERT runs fastest and MiniLM stays most consistent when the same model moves between platforms. A reader would care because these urban planning debates happen in real time on many sites, and reliable tools to track sentiment can show where support or resistance is strongest. The work also finds that news texts inflate scores due to uneven class balance, Reddit loses detail in summaries, and Twitter sits in the middle for difficulty.

Core claim

This study presents the first multi-platform sentiment analysis of public opinion on the 15-minute city concept across Twitter, Reddit, and news media. Using compressed transformer models and Llama-3-8B for annotation, the authors classify sentiment across heterogeneous text domains. The pipeline handles long-form and short-form text, supports consistent annotation, and enables reproducible evaluation. Benchmarking five models with stratified 5-fold cross-validation shows DistilRoBERTa with the highest F1 of 0.8292, TinyBERT with best efficiency, and MiniLM with best cross-platform consistency. Results indicate that news data yields inflated performance due to class imbalance, Reddit suffers

What carries the argument

A cross-platform evaluation pipeline that generates ground-truth sentiment labels with Llama-3-8B and then benchmarks compressed transformers (DistilRoBERTa, MiniLM, TinyBERT, DistilBERT, ELECTRA) on mixed-length texts from social media and news.

If this is right

  • News articles produce higher accuracy scores mainly because positive or negative classes are unevenly distributed.
  • Reddit threads lose critical context when shortened, which reduces model reliability on that platform.
  • Twitter posts create a moderate but realistic test case that sits between the other two sources.
  • Compressed models reach performance levels close to larger systems while using far less training time and memory.
  • Urban planning researchers can now choose models based on the trade-off between accuracy, speed, and platform consistency.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same labeling and benchmarking steps could be reused to track public reaction to other local policies such as bike lanes or zoning changes.
  • Planners could run ongoing scans on these platforms to spot rising concerns before they appear in formal comments.
  • Adding non-English posts would test whether the observed platform differences hold across languages and regions.
  • Tiny models that run quickly open the possibility of live dashboards that update sentiment scores daily.

Load-bearing premise

The sentiment labels automatically produced by Llama-3-8B are accurate and consistent enough to serve as reliable ground truth for training the smaller models on all three platforms.

What would settle it

A side-by-side comparison where human annotators re-label a held-out sample of the Twitter, Reddit, and news texts and the agreement rate with Llama-3-8B falls below 70 percent on posts that contain sarcasm or mixed signals.

Figures

Figures reproduced from arXiv: 2509.11443 by Boniphace Kutela, Darrell Anderson, Gaurab Chhetri, Subasish Das.

Figure 1
Figure 1. Figure 1: Study overview: data collection, summarization, LLM [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: ROC curves for all model–dataset combinations. (Each subplot shows the 5-fold averaged performance for one model [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
read the original abstract

This study presents the first multi-platform sentiment analysis of public opinion on the 15-minute city concept across Twitter, Reddit, and news media. Using compressed transformer models and Llama-3-8B for annotation, we classify sentiment across heterogeneous text domains. Our pipeline handles long-form and short-form text, supports consistent annotation, and enables reproducible evaluation. We benchmark five models (DistilRoBERTa, DistilBERT, MiniLM, ELECTRA, TinyBERT) using stratified 5-fold cross-validation, reporting F1-score, AUC, and training time. DistilRoBERTa achieved the highest F1 (0.8292), TinyBERT the best efficiency, and MiniLM the best cross-platform consistency. Results show News data yields inflated performance due to class imbalance, Reddit suffers from summarization loss, and Twitter offers moderate challenge. Compressed models perform competitively, challenging assumptions that larger models are necessary. We identify platform-specific trade-offs and propose directions for scalable, real-world sentiment classification in urban planning discourse.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript claims to present the first multi-platform sentiment analysis of public discourse on the 15-minute city concept using data from Twitter, Reddit, and news media. It employs Llama-3-8B to generate sentiment labels, then benchmarks five compressed transformer models (DistilRoBERTa, DistilBERT, MiniLM, ELECTRA, TinyBERT) via stratified 5-fold cross-validation, reporting metrics such as F1-score (highest 0.8292 for DistilRoBERTa), AUC, and training time. The work highlights platform effects including class imbalance in news data, summarization loss on Reddit, and moderate challenge from Twitter, while arguing that compressed models perform competitively.

Significance. If the central performance claims hold under verified labels, the paper offers useful empirical benchmarks on cross-platform sentiment classification for urban planning topics and demonstrates practical trade-offs in model efficiency and consistency. The emphasis on reproducible evaluation and platform-specific insights could inform scalable discourse analysis in social sciences, though the absence of label validation limits immediate impact.

major comments (2)
  1. [Abstract] Abstract and methods description: The pipeline treats Llama-3-8B outputs as sole ground-truth labels for training and evaluating all five models without any reported human validation, inter-annotator agreement, or accuracy assessment on a held-out subset. Given the heterogeneous domains (noisy short Twitter posts, summarized Reddit threads, longer news articles), unverified label noise or platform bias directly undermines the reported F1 of 0.8292, cross-platform consistency claims for MiniLM, and all downstream comparisons.
  2. [Abstract] Abstract: Insufficient detail is provided on data collection procedures, exact preprocessing steps, and how summarization loss on Reddit was quantified or measured. These omissions make it impossible to evaluate potential confounds or reproduce the stratified 5-fold results that support the platform-effect conclusions.
minor comments (2)
  1. Consider adding an explicit table or figure breaking down per-platform F1/AUC scores for each model to strengthen the cross-platform consistency claim.
  2. Clarify the exact prompting strategy and temperature settings used with Llama-3-8B for annotation to improve reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We have carefully reviewed each major comment and provide point-by-point responses below. Revisions to the manuscript will be made to improve transparency and address concerns about label quality and reproducibility.

read point-by-point responses

  1. Referee: [Abstract] Abstract and methods description: The pipeline treats Llama-3-8B outputs as sole ground-truth labels for training and evaluating all five models without any reported human validation, inter-annotator agreement, or accuracy assessment on a held-out subset. Given the heterogeneous domains (noisy short Twitter posts, summarized Reddit threads, longer news articles), unverified label noise or platform bias directly undermines the reported F1 of 0.8292, cross-platform consistency claims for MiniLM, and all downstream comparisons.

    Authors: We agree that the absence of human validation for the Llama-3-8B labels represents a limitation, particularly across heterogeneous platforms where label noise could influence results. In the revised manuscript, we will add a dedicated subsection on label quality assessment. This will include a human annotation study on a held-out stratified sample of 300 instances (100 per platform), reporting accuracy, inter-annotator agreement via Fleiss' kappa, and platform-specific error analysis. These additions will provide empirical grounding for the reported metrics and allow direct evaluation of potential biases. revision: yes

  2. Referee: [Abstract] Abstract: Insufficient detail is provided on data collection procedures, exact preprocessing steps, and how summarization loss on Reddit was quantified or measured. These omissions make it impossible to evaluate potential confounds or reproduce the stratified 5-fold results that support the platform-effect conclusions.

    Authors: We acknowledge that additional methodological detail is required for full reproducibility and to evaluate potential confounds. The revised manuscript will substantially expand the Methods section with: precise data collection queries, APIs, and temporal scopes for each platform; a complete step-by-step preprocessing pipeline (including cleaning, normalization, and deduplication); and explicit quantification of Reddit summarization loss using metrics such as average token reduction, ROUGE overlap with originals, and sentiment label stability checks. These changes will directly support replication of the stratified 5-fold cross-validation and platform-effect analyses. revision: yes

Circularity Check

0 steps flagged

No circularity in derivation or performance claims

full rationale

The paper generates sentiment labels via Llama-3-8B and reports standard 5-fold cross-validation F1/AUC metrics for five compressed transformers on held-out folds of those labels. These are conventional empirical benchmarks measuring agreement with the external LLM annotations; no equations, fitted parameters, or self-citations reduce the reported scores to a re-expression of the inputs by construction. The pipeline contains no self-definitional steps, ansatzes smuggled via citation, or uniqueness theorems. The derivation chain is self-contained as a comparative ML benchmarking study.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The study rests on standard NLP assumptions about the validity of transformer-based classification and the utility of LLM-generated labels as proxies for human annotation. No new mathematical axioms or physical entities are introduced.

pith-pipeline@v0.9.0 · 5721 in / 1258 out tokens · 55733 ms · 2026-05-18T16:11:02.774452+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Retrieval-Augmented Generation Must Move Beyond Factual Grounding to Represent Diverse Opinions

    cs.AI 2026-04 unverdicted novelty 5.0

    Opinion-aware RAG with LLM opinion extraction and entity-linked graphs improves retrieval diversity by 26-42% over factual baselines on e-commerce forum data.

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages · cited by 1 Pith paper · 2 internal anchors

  1. [1]

    Stanley, J

    J. Stanley, J. Stanley, and S. Davis,Connecting Neighbourhoods: The 20 Minute City, 2015, 28p

    work page 2015

  2. [2]

    Institute for Transportation and Development Policy,Pedestrians First: Tools for a Walkable City, 2018, 75p

    work page 2018

  3. [3]

    Unlocking urban sentiments about 15-min city through hashtags,

    S. Das, J. Tipsword, and B. Kutela, “Unlocking urban sentiments about 15-min city through hashtags,” inInternational Conference on Transportation and Development 2024: Transportation Planning, Operations, and Transit. American Society of Civil Engineers, 2024, pp. 158–169. [Online]. Available: https://doi.org/10.1061/9780784485521.015

    work page doi:10.1061/9780784485521.015 2024

  4. [4]

    The 15-minute city: In- terpreting the model to bring out urban resiliencies,

    L. Abdelfattah, D. Deponte, and G. Fossa, “The 15-minute city: In- terpreting the model to bring out urban resiliencies,” inTransportation Research Procedia, no. 60, 2022, pp. 330–337

    work page 2022

  5. [5]

    PwC,30-Minute Melbourne: The Road (and Rail) to the Future of the World’s Most Liveable City, 2017, 11p

    work page 2017

  6. [6]

    The concept of a walkable city as an alternative form of urban mobility,

    K. Turo ´n, P. Czech, and M. Juzek, “The concept of a walkable city as an alternative form of urban mobility,”Scientific Journal of Silesian University of Technology. Series Transport, vol. 95, pp. 223–230, 2017

    work page 2017

  7. [7]

    Walkable green cities in mediterranean countries: City of alexandria, egypt,

    M. M. Imam, A. F. Bakr, and Z. El-Sayad, “Walkable green cities in mediterranean countries: City of alexandria, egypt,” inWIT Transactions on the Built Environment, no. 186, 2019, pp. 109–119

    work page 2019

  8. [8]

    Designing walkable streets in congested touristic cities: The case of cartagena de indias, colombia,

    H. Gonzalez-Urango, M. Le Pira, G. Inturri, M. Ignaccolo, and M. Garc ´ıa-Mel´on, “Designing walkable streets in congested touristic cities: The case of cartagena de indias, colombia,” inTransportation Research Procedia, no. 45, 2020, pp. 309–316

    work page 2020

  9. [9]

    Strategic design of walkable places in the urban core area of nagoya city,

    A. Ito, K. Nakamura, M. Iryo, and T. Noji, “Strategic design of walkable places in the urban core area of nagoya city,”Journal of the City Planning Institute of Japan, vol. 56, no. 3, pp. 819–826, 2021

    work page 2021

  10. [10]

    Urban accessi- bility in a 15-minute city: A measure in the city of naples, italy,

    F. Gaglione, C. Gargiulo, F. Zucaro, and C. Cottrill, “Urban accessi- bility in a 15-minute city: A measure in the city of naples, italy,” in Transportation Research Procedia, no. 60, 2022, pp. 378–385

    work page 2022

  11. [11]

    Realizing a 15-minute city in metropolitan tokyo,

    H. Shimizu, T. Murooka, and M. Taniguchi, “Realizing a 15-minute city in metropolitan tokyo,”Journal of the City Planning Institute of Japan, vol. 57, no. 3, pp. 592–598, 2022

    work page 2022

  12. [12]

    Is the 15-minute city within reach? evaluating walking and cycling accessibility to grocery stores in vancouver,

    K. Hosford, J. Beairsto, and M. Winters, “Is the 15-minute city within reach? evaluating walking and cycling accessibility to grocery stores in vancouver,”Transportation Research Interdisciplinary Perspectives, vol. 14, p. 100602, 2022

    work page 2022

  13. [13]

    Awareness and knowledge levels of engineering and planning students and practitioners about the 15-minute city concept in a developing country,

    M. Abdullah, N. Ali, M. A. Javid, and Q. Hussain, “Awareness and knowledge levels of engineering and planning students and practitioners about the 15-minute city concept in a developing country,”Journal of Urban Mobility, vol. 2, p. 100037, 2022

    work page 2022

  14. [14]

    The x-minute city: Measuring the 10, 15, 20-minute city and an evaluation of its use for sustainable urban design,

    T. M. Logan, M. H. Hobbs, L. C. Conrow, N. L. Reid, R. A. Young, and M. J. Anderson, “The x-minute city: Measuring the 10, 15, 20-minute city and an evaluation of its use for sustainable urban design,”Cities, vol. 131, p. 103924, 2022

    work page 2022

  15. [15]

    The 15-minute city for all? – measuring individual and temporal variations in walking accessibility,

    E. Willberg, C. Fink, and T. Toivonen, “The 15-minute city for all? – measuring individual and temporal variations in walking accessibility,” Journal of Transport Geography, vol. 106, p. 103521, 2023

    work page 2023

  16. [16]

    Creating livable cities for healthy ageing: Cognitive health in older adults and their 15-minute walkable neighbourhoods,

    S. Zhang, W. Wu, Z. Xiao, S. Wu, Q. Zhao, D. Ding, and L. Wang, “Creating livable cities for healthy ageing: Cognitive health in older adults and their 15-minute walkable neighbourhoods,”Cities, vol. 137, p. 104312, 2023

    work page 2023

  17. [17]

    Extracting patterns from twitter to promote biking,

    S. Das, A. Dutta, G. Medina, L. Minjares-Kyle, and Z. Elgart, “Extracting patterns from twitter to promote biking,”IATSS Research, vol. 43, no. 1, pp. 51–59, 2019. [Online]. Available: https: //doi.org/10.1016/j.iatssr.2018.09.002

    work page doi:10.1016/j.iatssr.2018.09.002 2019

  18. [18]

    Characterizing public emotions and sentiments in covid-19 environment: A case study of india,

    S. Das and A. Dutta, “Characterizing public emotions and sentiments in covid-19 environment: A case study of india,”Journal of Human Behavior in the Social Environment, 2020, published online: 14 July 2020

    work page 2020

  19. [19]

    Uses of social media in public transportation,

    S. Das, T. N. Fahmida, I. N. Sener, and M. Walk, “Uses of social media in public transportation,” Transportation Research Board, National Academies of Sciences, Engineering, and Medicine, Washington, D.C., TCRP Synthesis Report 156, 2021. [Online]. Available: https://nap.nationalacademies.org/catalog/26451

    work page 2021

  20. [20]

    Bruns and J

    A. Bruns and J. Burgess,Twitter and society. Peter Lang, 2013

    work page 2013

  21. [21]

    A guide to 15-minute cities: why are they so controversial?

    University of Birmingham, “A guide to 15-minute cities: why are they so controversial?” 2024. [Online]. Available: https://www.ube.ac. uk/whats-happening/articles/15-minute-city/

    work page 2024

  22. [22]

    Debunking the 15-minute city conspiracy,

    R. Steuteville, “Debunking the 15-minute city conspiracy,”Public Square: A CNU Journal, mar 2023. [Online]. Available: https://www. cnu.org/publicsquare/2023/03/03/debunking-15-minute-city-conspiracy

    work page 2023

  23. [23]

    What are 15 minute cities and why are they controversial?

    Ferret Journalists, “What are 15 minute cities and why are they controversial?”The Ferret, feb 2023. [Online]. Available: https://theferret.scot/15-minute-cities-why-are-they-controversial/

    work page 2023

  24. [24]

    DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

    V . Sanh, L. Debut, J. Chaumond, and T. Wolf, “Distilbert, a dis- tilled version of bert: smaller, faster, cheaper and lighter,”ArXiv, vol. abs/1910.01108, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1910

  25. [25]

    Minilm: Deep self-attention distillation for task-agnostic compression of pre- trained transformers,

    W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, and M. Zhou, “Minilm: Deep self-attention distillation for task-agnostic compression of pre- trained transformers,” 2020

    work page 2020

  26. [26]

    ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

    K. Clark, M.-T. Luong, Q. V . Le, and C. D. Manning, “Electra: Pre- training text encoders as discriminators rather than generators,”arXiv preprint arXiv:2003.10555, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2003

  27. [27]

    Tinybert: Distilling bert for natural language understanding

    X. Jiao, Y . Yin, L. Shang, X. Jiang, X. Chen, L. Li, F. Wang, and Q. Liu, “Tinybert: Distilling bert for natural language understanding,” arXiv preprint arXiv:1909.10351, 2019

    work page arXiv 1909

  28. [28]

    The llama 3 herd of models,

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fanet al., “The llama 3 herd of models,”arXiv e-prints, pp. arXiv–2407, 2024

    work page 2024

  29. [29]

    City transit rider tweets: Understanding sentiments and politeness,

    S. Das and H. A. Zubaidi, “City transit rider tweets: Understanding sentiments and politeness,”Journal of Urban Technology, vol. 30, no. 2, pp. 111–126, 2021, published online April 26, 2021. [Online]. Available: https://doi.org/10.1080/10630732.2021.1903288

    work page doi:10.1080/10630732.2021.1903288 2021

  30. [30]

    User sentiment analysis with louisiana social media data for effective crash countermeasures,

    S. Das and X. Sun, “User sentiment analysis with louisiana social media data for effective crash countermeasures,” Final Report, June 2015, conducted in cooperation with the U.S. Department of Transportation — Federal Highway Administration

    work page 2015

  31. [31]

    Tweets about self-driving cars: Deep sentiment analysis using long short-term memory network (lstm),

    A. Dutta and S. Das, “Tweets about self-driving cars: Deep sentiment analysis using long short-term memory network (lstm),” inInternational Conference on Innovative Computing and Communications, ser. Ad- vances in Intelligent Systems and Computing. Springer, Singapore, 2021, vol. 1165, pp. 515–523

    work page 2021