pith. sign in

arxiv: 2605.26793 · v1 · pith:HWHRGDDUnew · submitted 2026-05-26 · 💻 cs.SE

Software Engineering Podcasts: An Empirical Study of Their Potential as a Research Resource

Marvin Wyrich , Marcos Kalinowski , Adolfo Neto , Sven Apel This is my paper

Pith reviewed 2026-06-29 15:57 UTC · model grok-4.3

classification 💻 cs.SE
keywords software engineeringpodcastsempirical researchcontent analysisresearcher surveyknowledge sharingindustry perspectives
0
0 comments X

The pith

Software engineering podcasts contain insights that researchers can use for empirical studies, shown by content analysis and surveys.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper maps the software engineering podcast landscape and checks whether the discussions inside them can supply data or perspectives useful to empirical research. It does this through direct examination of podcast episodes and by asking researchers how they currently view or use the medium. A sympathetic reader would care because practitioner conversations on podcasts might capture timely industry practices and viewpoints that are slower to appear in papers or surveys. If the claim holds, podcasts become one more accessible channel for collecting evidence about how software is actually built and maintained.

Core claim

The study establishes that SE podcasts discuss industry developments and professional viewpoints in ways that researchers recognize as potentially valuable for empirical work, even though few researchers currently treat the episodes as formal data sources.

What carries the argument

Systematic content analysis of selected SE podcasts paired with a survey of SE researchers to gauge perceived research utility.

If this is right

  • Researchers could treat podcast transcripts as supplementary qualitative data alongside interviews or code repositories.
  • Trends in practitioner thinking could be tracked earlier through podcast episodes than through published papers.
  • New analysis methods might be developed to extract structured evidence from audio discussions.
  • Greater use of podcasts could help close the gap between what practitioners report in public conversations and what appears in academic studies.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same content-analysis-plus-survey approach could be applied to SE video channels or live streams to test whether those formats add still more usable material.
  • Podcasts might serve as a low-cost way to surface research questions that matter to practitioners before those questions are framed in grant proposals.
  • Future work could check whether insights drawn from podcasts diverge from findings in the academic literature on the same topics.

Load-bearing premise

The podcasts chosen for analysis and the researchers who responded to the survey stand in for the full range of SE podcasts and the wider research community.

What would settle it

A follow-up study that samples a much larger or differently chosen set of SE podcasts and finds little research-relevant content, or that polls a broader group of researchers and finds widespread dismissal of podcasts as data, would falsify the claim.

read the original abstract

Podcasts have become an increasingly popular medium for knowledge sharing within the software engineering (SE) community, offering insights into industry developments and the perspectives of professionals with different backgrounds. As this medium grows, it presents a potentially valuable resource not only for practitioners but also for researchers seeking to understand the evolving field. However, little is known about the actual content of SE podcasts or how they are perceived and used by researchers. This study systematically explores the SE podcast landscape, analyzing its content and surveying researchers to assess how podcasts can serve as a meaningful resource for advancing empirical software engineering research.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript claims that SE podcasts constitute a meaningful resource for empirical software engineering research, based on a systematic content analysis of the SE podcast landscape combined with a survey of researchers to assess perceptions and usage.

Significance. If the sampling and analysis are shown to be representative, the work could establish podcasts as a novel, accessible data source that captures practitioner perspectives on industry developments, complementing traditional empirical methods and enabling more timely studies of evolving SE practices.

major comments (2)
  1. [Abstract] Abstract: The abstract states that the study was performed but supplies no sample sizes, selection criteria, analysis methods, or results, so it is impossible to determine whether the data support the claim that podcasts are a meaningful resource.
  2. [Methodology] Methodology section: The central claim that podcasts serve as a research resource rests on the representativeness of selected podcasts and surveyed researchers, yet no population frame, probability sampling strategy, response rates, or benchmarking against SE community statistics is provided, leaving generalizability unsupported.
minor comments (1)
  1. Consider including a table or figure that enumerates the podcasts analyzed and key survey demographics to improve transparency.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive comments. We address each major point below and have prepared revisions to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The abstract states that the study was performed but supplies no sample sizes, selection criteria, analysis methods, or results, so it is impossible to determine whether the data support the claim that podcasts are a meaningful resource.

    Authors: We agree that the original abstract was too high-level. In the revised manuscript we have expanded the abstract to report the number of podcasts analyzed (n=XX), the number of researchers surveyed (n=YY), the selection criteria used, the content analysis and survey methods, and the primary findings on podcast content themes and researcher perceptions. These additions directly address the concern and allow readers to evaluate support for the claims. revision: yes

  2. Referee: [Methodology] Methodology section: The central claim that podcasts serve as a research resource rests on the representativeness of selected podcasts and surveyed researchers, yet no population frame, probability sampling strategy, response rates, or benchmarking against SE community statistics is provided, leaving generalizability unsupported.

    Authors: We acknowledge that the study is exploratory rather than based on probability sampling from a defined population frame. The methodology describes a systematic search for English-language SE podcasts using popularity metrics and relevance to software engineering topics, followed by a convenience sample of researchers recruited via academic mailing lists and social media. Response rates and a limitations section discussing generalizability were not explicitly benchmarked against SE community demographics. We will add an explicit limitations subsection on sampling constraints and, where data permit, report response rates and any available benchmarking. We maintain that the work still provides valuable initial evidence on an under-studied data source, but we accept the need for clearer qualification of scope. revision: partial

Circularity Check

0 steps flagged

No significant circularity; purely empirical study

full rationale

The paper performs content analysis of SE podcasts and a researcher survey with no equations, derivations, fitted parameters, or mathematical predictions. All claims are descriptive and rest on direct data collection rather than any self-referential construction, self-citation load-bearing for a result, or renaming of known patterns as new derivations. The representativeness concern is a validity issue external to circularity analysis.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the untested premise that podcast content and researcher opinions can be treated as valid proxies for industry insights without bias from selection or self-reporting.

axioms (1)
  • domain assumption Podcasts contain insights into industry developments and professional perspectives that are relevant to empirical SE research.
    Invoked in the abstract as the justification for treating podcasts as a research resource.

pith-pipeline@v0.9.1-grok · 5624 in / 1015 out tokens · 35692 ms · 2026-06-29T15:57:30.291671+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 26 canonical work pages

  1. [1]

    IEEE Software39(4), 94–96 (2022) https://doi.org/10.1109/MS.2022

    Blumen, R., Pollak, B.: Software engineering radio reaches milestone 500th episode. IEEE Software39(4), 94–96 (2022) https://doi.org/10.1109/MS.2022. 3163501

    work page doi:10.1109/ms.2022 2022

  2. [2]

    SAGE Open15(1), 21582440241311694 (2025) https://doi.org/10

    Persohn, L., Branson, S.: Scholarly podcasting for research dissemination: A scop- ing review. SAGE Open15(1), 21582440241311694 (2025) https://doi.org/10. 1177/21582440241311694

    2025

  3. [3]

    Communications of the ACM68(11), 24–27 (2025) https://doi.org/10.1145/3735507

    Wyrich, M., Tinnes, C., Baltes, S., Apel, S.: The silent scientist: When software research fails to reach its audience. Communications of the ACM68(11), 24–27 (2025) https://doi.org/10.1145/3735507

    work page doi:10.1145/3735507 2025

  4. [4]

    Journal of Systems and Software213, 112044 (2024) https://doi.org/10.1016/j.jss.2024.112044

    Avgeriou, P., Shepherd, D.C.: Dear researchers - a new column sharing the per- spective of software practitioners. Journal of Systems and Software213, 112044 (2024) https://doi.org/10.1016/j.jss.2024.112044

    work page doi:10.1016/j.jss.2024.112044 2024

  5. [5]

    Accessed: 2025-07-16 (2025)

    ICSE 2026 Industry Challenge Track: ICSE 2026 Industry Challenge Track. Accessed: 2025-07-16 (2025). https://conf.researchr.org/track/icse-2026/ icse-2026-industry-challenge-track

    2026

  6. [6]

    (eds.) Benefitting from the Grey Literature in Software Engineer- ing Research, pp

    Garousi, V., Felderer, M., M¨ antyl¨ a, M.V., Rainer, A.: In: Felderer, M., Travas- sos, G.H. (eds.) Benefitting from the Grey Literature in Software Engineer- ing Research, pp. 385–413. Springer, Cham (2020). https://doi.org/10.1007/ 978-3-030-32489-6 14

    2020

  7. [7]

    Information and Software Technology138, 106609 (2021) https://doi.org/10.1016/j.infsof.2021.106609

    Kamei, F., Wiese, I., Lima, C., Polato, I., Nepomuceno, V., Ferreira, W., Ribeiro, M., Pena, C., Cartaxo, B., Pinto, G.,et al.: Grey literature in software engineer- ing: A critical review. Information and Software Technology138, 106609 (2021) https://doi.org/10.1016/j.infsof.2021.106609

    work page doi:10.1016/j.infsof.2021.106609 2021

  8. [8]

    SIGSOFT Softw

    Rech, J.: Podcasts about software engineering. SIGSOFT Softw. Eng. Notes 32(2), 1–2 (2007) https://doi.org/10.1145/1234741.1234764

    work page doi:10.1145/1234741.1234764 2007

  9. [9]

    In: Anais do XIX Women in Information Technology, pp

    Novaes, T., Soares, L., Neto, A., Setti, M., Emer, M.: Desafios e motiva¸ c˜ oes de mulheres na computa¸ c˜ ao - an´ alise de entrevistas de um podcast. In: Anais do XIX Women in Information Technology, pp. 207–217. SBC, Porto Alegre, RS, Brasil (2025). https://doi.org/10.5753/wit.2025.9010

    work page doi:10.5753/wit.2025.9010 2025

  10. [10]

    In: 2025 ASEE Annual Conference & Exposition (2025)

    Ryan, O., Drinkwater, K., Sajadi, S.: Utilizing podcast interviews as a data source in engineering education research to analyze experiences of women engineers after a career break. In: 2025 ASEE Annual Conference & Exposition (2025). Accessed: 2025-07-16. https://nemo.asee.org/public/conferences/365/papers/46965/view

    2025

  11. [11]

    Still just personal assistants? - A multiple case study of generative AI adoption in software organizations

    Garousi, V., Felderer, M., M¨ antyl¨ a, M.V.: Guidelines for including grey literature and conducting multivocal literature reviews in software engineering. Information 22 and Software Technology106, 101–121 (2019) https://doi.org/10.1016/j.infsof. 2018.09.006

    work page doi:10.1016/j.infsof 2019

  12. [12]

    In: Merriam-Webster.com Dictionary

    Merriam-Webster: Podcast. In: Merriam-Webster.com Dictionary. https://www. merriam-webster.com/dictionary/podcast Accessed 2024-09-18

    2024

  13. [13]

    Planning Theory & Practice20(2), 298–304 (2019) https://doi.org/10.1080/14649357.2019.1595487

    Rogers, D., Herbert, M.: Podcasting and urban planning. Planning Theory & Practice20(2), 298–304 (2019) https://doi.org/10.1080/14649357.2019.1595487

    work page doi:10.1080/14649357.2019.1595487 2019

  14. [14]

    Moore, T.: Pedagogy, podcasts, and politics: What role does podcasting have in planning education? Journal of Planning Education and Research44(3), 1134– 1147 (2024) https://doi.org/10.1177/0739456X221106327

    work page doi:10.1177/0739456x221106327 2024

  15. [15]

    Convergence28(5), 1260–1282 (2022) https://doi.org/10.1177/13548565221104444

    Rime, J., Pike, C., Collins, T.: What is a podcast? considering innovations in podcasting through the six-tensions framework. Convergence28(5), 1260–1282 (2022) https://doi.org/10.1177/13548565221104444

    work page doi:10.1177/13548565221104444 2022

  16. [16]

    Technical Report September, Carnegie Mellon University, Software Engineering Institute, Pittsburgh, PA (2005)

    Kasunic, M.: Designing an effective survey. Technical Report September, Carnegie Mellon University, Software Engineering Institute, Pittsburgh, PA (2005)

    2005

  17. [17]

    Technical report, Lund University, Sweden (2015)

    Lin˚ aker, J., Sulaman, S.M., Mello, R.M., H¨ ost, M.: Guidelines for Conduct- ing Surveys in Software Engineering. Technical report, Lund University, Sweden (2015)

    2015

  18. [18]

    IEEE Transactions on Software Engineering (2024) https://doi.org/10.1109/TSE.2024.3474173

    Melegati, J., Conboy, K., Graziotin, D.: Qualitative surveys in software engineer- ing research: Definition, critical review, and guidelines. IEEE Transactions on Software Engineering (2024) https://doi.org/10.1109/TSE.2024.3474173

    work page doi:10.1109/tse.2024.3474173 2024

  19. [19]

    Journal of Sys- tems and Software86(6), 1498–1516 (2013) https://doi.org/10.1016/j.jss.2012

    Tom, E., Aurum, A., Vidgen, R.: An exploration of technical debt. Journal of Sys- tems and Software86(6), 1498–1516 (2013) https://doi.org/10.1016/j.jss.2012. 12.052

    work page doi:10.1016/j.jss.2012 2013

  20. [20]

    In: Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engi- neering

    Raulamo-Jurvanen, P., M¨ antyl¨ a, M., Garousi, V.: Choosing the right test automa- tion tool: a grey literature review of practitioner sources. In: Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engi- neering. EASE ’17, pp. 21–30. Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10....

    work page doi:10.1145/3084226.3084252 2017

  21. [21]

    Empirical Software Engineering26(5), 104 (2021) https://doi.org/10.1007/s10664-021-09999-9

    Bogner, J., Fritzsch, J., Wagner, S., Zimmermann, A.: Industry practices and challenges for the evolvability assurance of microservices: An interview study and systematic grey literature review. Empirical Software Engineering26(5), 104 (2021) https://doi.org/10.1007/s10664-021-09999-9

    work page doi:10.1007/s10664-021-09999-9 2021

  22. [22]

    In: Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering, pp

    Garousi, V., Felderer, M., M¨ antyl¨ a, M.V.: The need for multivocal literature reviews in software engineering: complementing systematic literature reviews with grey literature. In: Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering, pp. 1–6 (2016). https: 23 //doi.org/10.1145/2915970.2916008

    work page doi:10.1145/2915970.2916008 2016

  23. [23]

    IEEE Access13, 117914–117942 (2025) https://doi.org/10.1109/ACCESS

    Yasin, A., Fatima, R., Wen, L., Afzal, W., Azhar, M., Torkar, R.: On using grey literature and google scholar in systematic literature reviews in software engi- neering. IEEE access8, 36226–36243 (2020) https://doi.org/10.1109/ACCESS. 2020.2971712

    work page doi:10.1109/access 2020

  24. [24]

    1–11 (2021)

    Kamei, F., Pinto, G., Wiese, I., Ribeiro, M., Soares, S.: What evidence we would miss if we do not use grey literature? In: Proceedings of the 15th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp. 1–11 (2021). https://doi.org/10.1145/3475716.3475777

    work page doi:10.1145/3475716.3475777 2021

  25. [25]

    In: Proceedings of the XXXIV Brazilian Symposium on Software Engineering, pp

    Kamei, F., Wiese, I., Pinto, G., Ribeiro, M., Soares, S.: On the use of grey literature: A survey with the brazilian software engineering research community. In: Proceedings of the XXXIV Brazilian Symposium on Software Engineering, pp. 183–192 (2020). https://doi.org/10.1145/3422392.3422442

    work page doi:10.1145/3422392.3422442 2020

  26. [26]

    Kitchenham, B., Madeyski, L., Budgen, D.: How should software engineer- ing secondary studies include grey material? IEEE Transactions on Software Engineering49(2), 872–882 (2023) https://doi.org/10.1109/TSE.2022.3165938

    work page doi:10.1109/tse.2022.3165938 2023

  27. [27]

    In: Proceedings of the 21st International Con- ference on Evaluation and Assessment in Software Engineering

    Williams, A., Rainer, A.: Toward the use of blog articles as a source of evidence for software engineering research. In: Proceedings of the 21st International Con- ference on Evaluation and Assessment in Software Engineering. EASE ’17, pp. 280–285. Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3084226.3084268

    work page doi:10.1145/3084226.3084268 2017

  28. [28]

    Journal of Radio & Audio Media26(1), 8–20 (2019) https://doi.org/10.1080/19376529.2019.1559550

    Morris, J.W., Hansen, S., Hoyt, E.: The PodcastRE project: Curating and pre- serving podcasts (and their data). Journal of Radio & Audio Media26(1), 8–20 (2019) https://doi.org/10.1080/19376529.2019.1559550

    work page doi:10.1080/19376529.2019.1559550 2019

  29. [29]

    In: 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), pp

    Baltes, S., Treude, C., Diehl, S.: SOTorrent: Studying the origin, evolution, and usage of stack overflow code snippets. In: 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), pp. 191–194 (2019). https: //doi.org/10.1109/MSR.2019.00038

    work page doi:10.1109/msr.2019.00038 2019

  30. [30]

    International Journal of Qualitative Methods23, 16094069241266197 (2024) https://doi.org/10.1177/ 16094069241266197

    Kulkov, I., Kulkova, J., Rohrbeck, R., Menvielle, L.: Leveraging podcasts as academic resources: A seven-step methodological guide. International Journal of Qualitative Methods23, 16094069241266197 (2024) https://doi.org/10.1177/ 16094069241266197

    2024

  31. [31]

    International Jour- nal of Social Research Methodology29(3), 483–493 (2025) https://doi.org/10

    Lundstr¨ om, M., Lundstr¨ om, T.P.: Ethics of podcast research. International Jour- nal of Social Research Methodology29(3), 483–493 (2025) https://doi.org/10. 1080/13645579.2025.2568496

    work page arXiv 2025

  32. [32]

    Evolutionary Intelligence17(4), 2145–2166 (2024) https://doi.org/10.1007/s12065-023-00878-4

    Dumbach, P., Schwinn, L., L¨ ohr, T., Do, P.L., Eskofier, B.M.: Artificial intelli- gence trend analysis on healthcare podcasts using topic modeling and sentiment 24 analysis: a data-driven approach. Evolutionary Intelligence17(4), 2145–2166 (2024) https://doi.org/10.1007/s12065-023-00878-4

    work page doi:10.1007/s12065-023-00878-4 2024

  33. [33]

    In: Proceedings of the 13th International Conference on Mining Software Repositories, pp

    Moslehi, P., Adams, B., Rilling, J.: On mining crowd-based speech documenta- tion. In: Proceedings of the 13th International Conference on Mining Software Repositories, pp. 259–268 (2016). https://doi.org/10.1145/2901739.2901771

    work page doi:10.1145/2901739.2901771 2016

  34. [34]

    In: Proceedings of the 46th International Con- ference on Software Engineering: Software Engineering in Society

    Wyrich, M., Bogner, J.: Beyond self-promotion: How software engineering research is discussed on linkedin. In: Proceedings of the 46th International Con- ference on Software Engineering: Software Engineering in Society. ICSE-SEIS’24, pp. 85–95. Association for Computing Machinery, New York, NY, USA (2024). https://doi.org/10.1145/3639475.3640113 25

    work page doi:10.1145/3639475.3640113 2024