Leveraging Machine Learning Techniques to Investigate Media and Information Literacy Competence in Tackling Disinformation
Pith reviewed 2026-05-15 09:00 UTC · model grok-4.3
The pith
Machine learning models predict media and information literacy competencies against disinformation using student survey data.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Complex machine learning models outperform simpler approaches when predicting media and information literacy competencies in the context of disinformation, with academic year and prior training significantly improving prediction accuracy on data from 723 students.
What carries the argument
Classification and regression algorithms trained on validated survey responses to predict MIL competencies and isolate influencing variables such as academic year and prior training.
If this is right
- Targeted educational interventions can be designed using the identified key variables to build disinformation resistance.
- Personalized strategies can be created to improve students' critical navigation of digital environments.
- Training programs for future educators and communicators can be adjusted based on predicted competency levels.
Where Pith is reading between the lines
- The approach could be tested on working professionals or general adult populations to check if the same factors remain predictive.
- Combining the models with ongoing digital behavior data might allow real-time adjustment of literacy support.
- Curriculum planners could use the variable rankings to prioritize certain years or training modules in media education.
Load-bearing premise
The validated survey accurately measures actual media and information literacy competencies related to disinformation and the sample of 723 students generalizes to future educators and communicators.
What would settle it
A follow-up study on a new group of students in which simpler models achieve equal or higher accuracy than complex models would indicate the claimed superiority does not hold.
Figures
read the original abstract
This study develops machine learning models to assess Media and Information Literacy (MIL) skills specifically in the context of disinformation among students, particularly future educators and communicators. While the digital revolution has expanded access to information, it has also amplified the spread of false and misleading content, making MIL essential for fostering critical thinking and responsible media engagement. Despite its relevance, predictive modeling of MIL in relation to disinformation remains underexplored. To address this gap, a quantitative study was conducted with 723 students in education and communication programs using a validated survey. Classification and regression algorithms were applied to predict MIL competencies and identify key influencing factors. Results show that complex models outperform simpler approaches, with variables such as academic year and prior training significantly improving prediction accuracy. These findings can inform the design of targeted educational interventions and personalized strategies to enhance students' ability to critically navigate and respond to disinformation in digital environments.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper develops machine learning models to predict Media and Information Literacy (MIL) competencies in relation to disinformation using survey data from 723 students in education and communication programs. It applies classification and regression algorithms to these data, reports that complex models outperform simpler ones, and identifies academic year and prior training as key variables that improve prediction accuracy. The work aims to inform targeted educational interventions.
Significance. If the survey measures actual competencies with predictive validity and the models are shown to generalize via proper out-of-sample validation, the identification of modifiable factors such as prior training could support the design of more effective MIL curricula for future educators and communicators. The data-driven framing is a modest strength.
major comments (3)
- [Methods] Methods section: The abstract and text refer to a 'validated survey' but supply no information on content validity, reliability statistics (e.g., Cronbach's alpha), pilot testing, or—most critically—any behavioral validation linking self-reports to objective performance on disinformation-detection tasks such as source evaluation or manipulated-image identification. Because all ML targets derive from these labels, the reported model superiority is only as reliable as the label quality.
- [Results] Results section: The claim that 'complex models outperform simpler approaches' is presented without details on train/test splits, cross-validation scheme, performance metrics with uncertainty estimates, or confirmation that accuracy reflects out-of-sample rather than in-sample fit. This information is required to evaluate whether the performance gains are robust.
- [Discussion] Discussion section: The assumption that findings from the 723-student sample generalize to future educators and communicators is not supported by any analysis of sample representativeness or external validity checks.
minor comments (2)
- [Abstract] Abstract: The specific classification and regression algorithms employed are not named; listing them would improve reproducibility.
- [Discussion] The manuscript would benefit from a limitations subsection that explicitly addresses social-desirability bias in self-reported MIL data.
Simulated Author's Rebuttal
We thank the referee for their thorough review and constructive comments. We address each of the major comments below, indicating the revisions we plan to make to the manuscript.
read point-by-point responses
-
Referee: [Methods] Methods section: The abstract and text refer to a 'validated survey' but supply no information on content validity, reliability statistics (e.g., Cronbach's alpha), pilot testing, or—most critically—any behavioral validation linking self-reports to objective performance on disinformation-detection tasks such as source evaluation or manipulated-image identification. Because all ML targets derive from these labels, the reported model superiority is only as reliable as the label quality.
Authors: The survey instrument was adapted from previously validated MIL scales in the literature. However, we agree that the manuscript lacks explicit details on validation procedures. In the revised version, we will expand the Methods section to describe the survey development, including references to source instruments, pilot testing conducted, and reliability statistics such as Cronbach's alpha. We acknowledge that behavioral validation against objective tasks was not performed in this study; we will add this as a limitation in the Discussion and suggest it as an avenue for future research. revision: partial
-
Referee: [Results] Results section: The claim that 'complex models outperform simpler approaches' is presented without details on train/test splits, cross-validation scheme, performance metrics with uncertainty estimates, or confirmation that accuracy reflects out-of-sample rather than in-sample fit. This information is required to evaluate whether the performance gains are robust.
Authors: We appreciate this point and agree that methodological transparency is crucial. The revised manuscript will include a detailed account of the data splitting procedure (e.g., 70/30 train/test split with stratified sampling), the cross-validation approach (e.g., 5-fold CV), and report performance metrics including accuracy, F1-score, RMSE for regression, along with uncertainty estimates such as standard deviations across folds to confirm out-of-sample performance. revision: yes
-
Referee: [Discussion] Discussion section: The assumption that findings from the 723-student sample generalize to future educators and communicators is not supported by any analysis of sample representativeness or external validity checks.
Authors: The sample was drawn specifically from education and communication programs to target future educators and communicators. To address concerns about generalizability, the revised Discussion will include a subsection on sample characteristics, demographic comparisons to national statistics where available, and explicit discussion of limitations regarding external validity. We will also propose directions for multi-institutional studies to enhance generalizability. revision: partial
Circularity Check
No significant circularity: empirical ML fitting on survey data
full rationale
The paper collects survey responses from 723 students using a validated instrument, then applies standard classification and regression algorithms to predict MIL competency scores and identify influential variables such as academic year. No derivation step reduces a claimed prediction to its own inputs by construction, no self-citation chain supports a uniqueness claim, and no ansatz is smuggled in. Model performance comparisons are data-driven and self-contained against the collected dataset; any out-of-sample evaluation would be independent of the fitting process itself. Survey validity and behavioral grounding are separate methodological issues outside the circularity analysis.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption The validated survey accurately measures Media and Information Literacy competencies in the context of disinformation
Lean theorems connected to this paper
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Classification and regression algorithms were applied to predict MIL competencies... Results show that complex models outperform simpler approaches, with variables such as academic year and prior training significantly improving prediction accuracy.
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
validated survey... Cronbach’s alpha of 0.77... exploratory factor analysis
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Marcos-Vílchez, J.M.; Sánchez-Martín, M.; Muñiz-Velázquez, J.A. Effectiveness of training actions aimed at improving critical thinking in the face of disinformation: A systematic review protocol.Thinking Skills and Creativity2024,51. doi:10.1016/j.tsc.2024.101474
-
[2]
Ciudadanía alfabetizada en medios e información: Pensar críticamente, hacer clic sabiamente
UNESCO. Ciudadanía alfabetizada en medios e información: Pensar críticamente, hacer clic sabiamente. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000385119
-
[3]
Mateus De Oro, C.; Jabba, D.; Erazo-Coronado, A.M.; Aguaded, I.; Campis Carrillo, R. Educommunication and ICT: from a corpus to a model of educational intervention for critical attitude.Technology, Pedagogy and Education2024,33, 235–254. doi:10.1080/1475939X.2024.2309950
-
[4]
Media competencies of university professors and students
Romero-Rodríguez, L.M.; Contreras-Pulido, P .; Pérez-Rodríguez, M.A. Media competencies of university professors and students. Comparison of levels in Spain, Portugal, Brazil and Venezuela.Culture and Education2019,31, 326–368. doi:10.1080/11356405.2019.1597564
-
[5]
Alcolea-Díaz, G.; Reig, R.; Mancinas-Chávez, R. UNESCO’s Media and Information Literacy curriculum for teachers from the perspective of Structural Considerations of Information.Comunicar2020,28, 103–114. doi:10.3916/C62-2020-09. I
-
[6]
Beijing Consensus on Artificial Intelligence and Education
UNESCO. Beijing Consensus on Artificial Intelligence and Education. Available online: https://unesdoc.unesco.org/ark: /48223/pf0000368303
-
[7]
OECD.Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots; OECD Digital Education Outlook, 2021
work page 2021
-
[8]
Zhai, X. Practices and Theories: How Can Machine Learning Assist in Innovative Assessment Practices in Science Education. Journal of Science Education and Technology2021,30, 139–149. doi:10.1007/s10956-021-09901-8
-
[9]
Aïmeur, E.; Amri, S.; Brassard, G. Fake news, disinformation and misinformation in social media: a review.Social Network Analysis and Mining2023,13, 39. doi:10.1007/s13278-023-01028-5
-
[10]
Sanaullah, A.R.; Das, A.; Das, A.; Kabir, M.; Shu, K. Applications of machine learning for COVID-19 misinformation: A systematic review.Social Network Analysis and Mining2022,12. doi:10.1007/s13278-022-00921-9
-
[11]
Korkmaz, C.; Correia, A.P . A review of research on machine learning in educational technology.Educational Media International 2019,56, 250–267. doi:10.1080/09523987.2019.1669875
-
[12]
https://doi.org/10.432 4/9781003045366
Fastrez, P .; Landry, N.Media Literacy and Media Education Research Methods: A Handbook; Routledge, 2023. https://doi.org/10.432 4/9781003045366
work page 2023
-
[13]
Knaus, T. Technology criticism and data literacy: The case for an augmented understanding of media literacy.Journal of Media Literacy Education2020,12, 6–16. doi:10.23860/JMLE-2020-12-3-2
-
[14]
Gross, E.C.; Balaban, D.C. The Effectiveness of an Educational Intervention on Countering Disinformation Moderated by Intellectual Humility.Media and Communication2025,13, 1–18. doi:10.17645/mac.9109
-
[15]
Huang, G.; Jia, W.; Yu, W.Media Literacy Interventions Improve Resilience to Misinformation: A Meta-Analytic Investigation of Overall Effect and Moderating Factors; Communication Research, 2024. doi:10.1177/00936502241288103
-
[16]
Cooper, T. Calling out ‘alternative facts’: curriculum to develop students’ capacity to engage critically with contradictory sources. Teaching in Higher Education2019,24, 444–459. doi:10.1080/13562517.2019.1566220Literacy
-
[17]
Media literacy between primary and secondary students in Andalusia (Spain)
Aguaded, I.; Marin-Gutierrez, I.; Diaz-Parejo, E. Media literacy between primary and secondary students in Andalusia (Spain). Ried-Revista Iberoamericana de Educación a Distancia2015,18, 275–298. Information2025,1, 0 19 of 20
-
[18]
Measuring the implementation of media literacy statewide: A validation study
Hobbs, R.; Moen, M.; Tang, R.; Steager, P . Measuring the implementation of media literacy statewide: A validation study. Educational Media International2022,59, 189–208. doi:10.1080/09523987.2022.2136083
-
[19]
Defining media education policies: Building blocks, scope, and characteristics
Landry, N.; Caneva, C. Defining media education policies: Building blocks, scope, and characteristics. InThe Handbook of Media Education Research; Frau-Meigs, D.; Kotilainen, S.; Pathak-Shelat, M.; Hoechsmann, M.; Poyntz, S.R., Eds.; Wiley Online Library, 2020; pp. 289–308
work page 2020
-
[20]
Educommunication.The international encyclopedia of media literacy2019, pp
Aguaded, I.; Delgado-Ponce, A. Educommunication.The international encyclopedia of media literacy2019, pp. 1–6. doi:10.1002/9781118978238.ieml0061
-
[21]
Educación mediática y formación del profesorado
Osuna-Acedo, S.; Frau-Meigs, D.; Marta-Lazo, C. Educación mediática y formación del profesorado. Educomunicación más allá de la alfabetización digital.Revista interuniversitaria de formación del profesorado2018,32, 29–42
-
[22]
Meta-reflexivity for resilience against disinformation.Comunicar2021,29, 107–118
Golob, T.; Makaroviˇ c, M.; Rek, M. Meta-reflexivity for resilience against disinformation.Comunicar2021,29, 107–118. doi:10.3916/C66-2021-09
-
[23]
Herrero-Diz, P .; Conde-Jiménez, J.; Tapia-Frade, A.; Varona-Aramburu, D. The credibility of online news: an evaluation of the information by university students / La credibilidad de las noticias en Internet: una evaluación de la información por estudiantes universitarios.Culture and Education2019,31, 407–435. doi:10.1080/11356405.2019.1601937
-
[24]
Almenar, E.; Aran-Ramspott, S.; Suau, J.; Masip, P . Gender Differences in Tackling Fake News: Different Degrees of Concern, but Same Problems.Media and Communication2021,9, 229–238. https://doi.org/10.17645/mac.v9i1.3523
-
[25]
Mansoori, A.; Tahat, K.; Tahat, D.; Habes, M.; Salloum, S.; Mesbah, H.; Elareshi, M. Gender as a moderating variable in online misinformation acceptance during COVID-19.Heliyon2023,9, e19425. https://doi.org/10.1016/j.heliyon.2023.e19425
-
[26]
Online search is more likely to lead youth to validate true news than to refute false ones
Bouleimen, A.; Luceri, L.; Cardoso, F.; Botturi, L.; Hermida, M.; Addimando, L.; Beretta, C.; Galloni, M.; Giordano, S. Online search is more likely to lead youth to validate true news than to refute false ones. Proceedings of the ICWSM Workshop on Data for the Wellbeing of Most Vulnerable. Available online: https://bit.ly/3PuhgfY
-
[27]
Pennycook, G.; Rand, D.G. Who falls for fake news? The roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking.Journal of personality2020,88, 185–200. doi:10.1111/jopy.12476
-
[28]
Algunos mitos más difundidos sobre las TUC en la educación
Avello-Martínez, R.; Villalba-Condori, K.O.; Arias-Chávez, D. Algunos mitos más difundidos sobre las TUC en la educación. ¿Cómo evitarlos? Mendive2021,19, 1359–1375
-
[29]
Herrero-Curiel, E.; González-Aldea, P . Impacto de las fake news en estudiantes de periodismo y comunicación audiovisual de la Universidad Carlos III de Madrid. Vivat Academia.Revista de Comunicación2022,155, 1–21. doi:10.15178/va.2022.155.e1415
-
[30]
Cherner, T.S.; Curry, K. Preparing Pre-Service Teachers to Teach Media Literacy: A Response to” Fake News".Journal of Media Literacy Education2019,11, 1–31. doi:10.23860/JMLE-2019-11-1-1
-
[31]
Kahne, J.; Bowyer, B. Education for democracy in a partisan age: Confronting the challenges of motivated reasoning and misinformation.American Educational Research Journal2017,54, 3–34. doi:10.3102/000283121667981
-
[32]
Are really digital natives so good? Relationship between digital skills and digital reading
Fajardo, I.; Villalta, E.; Salmerón, L. Are really digital natives so good? Relationship between digital skills and digital reading. Annals of Psychology2015,32, 89–97. doi:10.6018/analesps.32.1.185571
-
[33]
Adjin-Tettey, T.D. Combating fake news, disinformation, and misinformation: Experimental evidence for media literacy education. Cogent Arts & Humanities2022,9, 2037229. doi:10.1080/23311983.2022.2037229
-
[34]
Application and theory gaps during the rise of Artificial Intelligence in Education
Chen, X.; Xie, H.; Zou, D.; Hwang, G.J. Application and theory gaps during the rise of Artificial Intelligence in Education. Computers and Education: Artificial Intelligence2020,1, 100002. https://doi.org/10.1016/j.caeai.2020.100002
-
[35]
Machine learning for the educational sciences.Review of Education2021,9, e3310
Hilbert, S.; Coors, S.; Kraus, E.; Bischl, B.; Lindl, A.; Frei, M.; Wild, J.; Krauss, S.; Goretzko, D.; Stachl, C. Machine learning for the educational sciences.Review of Education2021,9, e3310. https://doi.org/https://doi.org/10.1002/rev3.3310
-
[36]
Ersozlu, Z.; Taheri, S.; Koch, I. A review of machine learning methods used for educational data.Education and Information Technologies2024,29, 22125–22145. doi:10.1007/s10639-024-12704-0
-
[37]
Preparing the next generation of education researchers for big data in higher education
Gibson, D.C.; Ifenthaler, D. Preparing the next generation of education researchers for big data in higher education. InBig data and learning analytics in higher education; Daniel, B.K., Ed.; Springer, 2017; pp. 29–42. doi:10.1007/978-3-319-06520-5_4
-
[38]
Munir, H.; Vogel, B.; Jacobsson, A. Artificial intelligence and machine learning approaches in digital education: A systematic revision.Information2022,13. doi:10.3390/info13040203
-
[39]
Luan, H.; Tsai, C.C. A review of using machine learning approaches for precision education.Educational Technology and Society 2021,24, 250–266
work page 2021
-
[40]
The applications of machine learning in computational thinking assessments: a scoping review
Tan, B.; Jin, H.Y.; Cutumisu, M. The applications of machine learning in computational thinking assessments: a scoping review. Computer Science Education2023,34, 193–221. doi:10.1080/08993408.2023.2245687
-
[41]
Ibarra-Vázquez, G.; Ramírez-Montoya, M.S.; Terashima, H. Gender prediction based on university students’ complex thinking competency: An analysis from machine learning approaches.Education and Information Technologies2024,29, 2721–2739. doi:10.1007/s10639-023-11831-4
-
[42]
The Use of Deep Learning in Open Learning: A Systematic Review (2019 to 2023)
Estrada-Molina, O.; Mena, J.; López-Padrón, A. The Use of Deep Learning in Open Learning: A Systematic Review (2019 to 2023). The International Review of Research in Open and Distributed Learning2024,25, 370–393. doi:10.19173/irrodl.v25i3.7756. Information2025,1, 0 20 of 20
-
[43]
Forero-Corba, W.; Negre Bennasar, F. Diseño y simulación de un modelo de predicción para la evaluación de la competencia digital docente usando técnicas de Machine Learning. Edutec.Revista Electrónica De Tecnología Educativa2024,89, 18–43. doi:10.21556/edutec.2024.89.3201
-
[44]
Bernardo, A.B.I.; Cordel, M.O.I.I.; Ricardo, J.G.E.; Galanza, M.A.M.C.; Almonte-Acosta, S. Global Citizenship Competencies of Filipino Students: Using Machine Learning to Explore the Structure of Cognitive, Affective, and Behavioral Competencies in the 2019 Southeast Asia Primary Learning Metrics.Education Sciences2022,12, 547. doi:10.3390/educsci12080547
-
[45]
Miao, X.; Nadaf, A.; Zhou, Z. Machine learning evidence from PISA 2018 data to integrate global competence intervention in UAE K–12 public schools.International Review of Education2023,69, 675–690. doi:10.1007/s11159-023-10026-w
-
[46]
Predicting Media Literacy Level of Secondary School Students in Fiji
Reddy, P .; Chaudhary, K.; Sharma, B. Predicting Media Literacy Level of Secondary School Students in Fiji. In Proceedings of the 2019 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE). IEEE, December 2019, pp. 1–7. https://doi.org/10.1109/csde48274.2019.9162411
-
[47]
Wusylko, C.; Weisberg, L.; Opoku, R.A.; Abramowitz, B.; Williams, J.; Xing, W.; others.; Vu, M. Using machine learning techniques to investigate learner engagement with TikTok media literacy campaigns.Journal of Research on Technology in Education2024, 56, 72–93. doi:10.1080/15391523.2023.2266518
-
[48]
Mohd Nadzir, M.; Abu Bakar, J. A digital literacy predictive model in the context of distance education.Journal of ICT in Education 2023,10, 118–134. doi:10.37134/jictie.vol10.1.10.2023
-
[49]
OECD.La definición y selección de competencias clave; Resumen ejecutivo. OECD, 2005
work page 2005
-
[50]
Catalina-García, B.; Sousa, J.P .; Silva Sousa, L.C. Consumo de noticias y percepción de fake news entre estudiantes de Comuni- cación de Brasil, España y Portugal.Revista de Comunicación2019,18, 123–135. doi:10.26441/RC18.2-2019-A5
-
[51]
Figueira, J.; Santos, S. Percepción de las noticias falsas en universitarios de Portugal: Análisis de su consumo y actitudes.El Profesional de la Información2019,28, e280315. doi:10.3145/epi.2019.may.15
-
[52]
Marín Suelves, D.; García Tort, E.; Gabarda Méndez, V . La elección de los grados de Maestro/a: análisis de tendencias e incidencia del género y la titulación.Educatio Siglo XXI2021,39, 301–324. doi:10.6018/educatio.408581
-
[53]
Scikit-learn: Machine Learning in Python.Journal of Machine Learning Research2011,12, 2825–2830
Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V .; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P .; Weiss, R.; Dubourg, V .; et al. Scikit-learn: Machine Learning in Python.Journal of Machine Learning Research2011,12, 2825–2830
-
[54]
Shi, Y.; Ke, G.; Soukhavong, D.; Lamb, J.; Meng, Q.; Finley, T.; Wang, T.; Chen, W.; Ma, W.; Ye, Q.; et al.lightgbm: Light Gradient Boosting Machine, 2025. R package version 4.6.0.99. Disclaimer/Publisher’s Note:The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI a...
work page 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.