REVIEW 3 major objections 5 minor 13 references
Impacto de Treinamento em Programa\c{c}\~ao Competitiva no Ensino M\'edio: Resultados e Desafios
T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read Competitive-programming training can teach programming and promote computational thinking in high school.
desk verdict Honest experience report with new 2024 OBI data, but the Section 7 conclusion overreaches: the evidence supports 'suggestive of' rather than 'demonstrates.' read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a volunteer training cycle: outreach to students, synchronous in-person and remote classes built around C++ study tracks on an online learning platform, practice on an online judge system, and intensified simulations before the OBI. The mechanism that carries the claim is the contrast between OBI participation metrics in 2023 and 2024: campus count, per-level enrollment, female participation, and phase-2 qualifiers.
What would settle it
Compare the same participation and qualification metrics in similar schools or campuses that did not receive the training over the same 2023-2024 period; if the increases match untrained schools, the program's causal role is unsupported. Alternatively, administer a validated pre/post computational-thinking test to trained students: if scores do not move while participation rises, the proxy claim that participation reflects skill growth would be contradicted.
Extended reading notes
Core claim
The paper's central claim is that competitive-programming training is a relevant, workable tool for teaching programming and promoting computational thinking in Brazilian middle and high school. Its evidence is early and correlational: participation by campuses grew from four to six, P1 qualifiers for the second phase rose by eight, and female P1 enrollment on one campus grew from three to eight. The authors describe these as preliminary signs that volunteer training with online judges can both widen access to scientific competitions and support the computational-thinking goals of Brazil's national curriculum.
Load-bearing premise
The load-bearing premise is that the observed year-over-year increases in OBI participation and qualifier counts are caused by the training program, rather than by unrelated trends, school-level differences, or changes in who chose to enroll.
Editorial extensions
If this is right
- If the preliminary results are representative, one volunteer training cycle can bring additional campuses into the OBI and increase the number of students reaching the second phase.
- Extending the training to five more cycles through 2026 should produce stronger evidence on qualifiers, medals, and the methodology's generalizability.
- The more than two-fold rise in female P1 participation on the campus with a female instructor suggests that outreach and instruction style may matter for gender inclusion, a direction the project's next cycles can test.
- The authors' decision to keep training students who did not qualify implies that repeated exposure, not just one competition, is part of the proposed pathway to improved performance.
- The lack of average score gains in 2024 is not treated by the authors as a failure; they attribute it to novice students and a strike that reduced attendance, framing participation growth as the current success metric.
Reading between the lines
- Because enrollment was voluntary and no control group was used, the observed year-over-year growth could also reflect preexisting interest or other school-level trends rather than the training itself; a comparison with untrained schools would separate these explanations.
- The jump in female participation on the campus taught by a woman instructor suggests a testable hypothesis: visible female role models in competitive programming may lower the participation barrier, which later cycles could test by varying instructor gender across campuses.
- Participation counts and qualification numbers are the only outcome measures so far; a validated pre/post computational-thinking test would tell whether the training builds skill or mainly attracts and mobilizes students who already have interest.
- If the model proves causal in further cycles, it would offer a low-cost, volunteer-run blueprint for expanding computer science education in school systems with limited computing infrastructure.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports on an ongoing Design-Based Research project to develop a methodology for teaching programming to Brazilian middle and high school students through competitive programming training, centered on the Brazilian Informatics Olympiad (OBI). It describes the training setup (voluntary enrollment, presencial and remote modalities, Neps Academy and Beecrowd tools), presents descriptive statistics comparing 2023 and 2024 OBI participation across IFTM campuses, and concludes that preliminary results demonstrate the relevance of programming training for teaching programming and promoting computational thinking.
Significance. If the causal claim were supported, the paper would provide a useful, low-cost model for increasing OBI participation and potentially fostering computational thinking in Brazilian basic education. The paper's strengths include an honest acknowledgment that the research is preliminary, a clear description of the training methodology that can be replicated by others, and the explicit caveat in Section 5 that a more detailed analysis is still pending. The descriptive data on campus expansion, female participation, and P1 qualifiers are useful starting points. However, the central claim as worded in Section 7 is not supported by the evidence, which is entirely descriptive and lacks a control group or independent skill measure.
major comments (3)
- [Section 5, Figures 7-8] The claim that 'os resultados preliminares demonstram que treinamento em programação é uma ferramenta relevante para o ensino de programação e promoção do pensamento computacional' overreaches. The supporting evidence in Section 5 is year-over-year growth in the number of participating campuses (4 to 6), an increase of 8 P1 qualifiers, and an increase in female P1 participation at one campus (3 to 8). Because enrollment was voluntary (Section 4.2), there is no control group, no pre-test, and no statistical test; these observations are equally compatible with self-selection, national OBI trends, or school-level differences. The authors themselves hedge in Section 5 ('Estes dados podem sugerir um reflexo dos treinamentos oferecidos... no entanto, uma análise mais detalhada ainda será realizada'), but this hedge is dropped in the conclusion. The conclusion should be softened to 'consistent with' or 'suggestive of' and explicitly framed as preliminary pending the planned DBR cycles and formal analysis.
- [Section 5, Figures 7-8] The paper assumes that OBI participation and phase-2 qualification are valid proxies for programming skill and computational thinking, but this assumption is weakened by the paper's own finding that average OBI scores in 2024 were not significantly different from 2023. If learning had occurred, one would expect some improvement in scores, especially among returning or trained students. The increase in qualifiers could reflect larger cohort sizes, cohort composition, familiarity with the exam format, or favorable contest conditions rather than skill gains. The manuscript should either report a direct pre/post measure of skill (e.g., scores on training exercises or a separate test) or restrict the central claim to outcomes of participation and interest, which the data can more directly support.
- [Section 5, Table 2 and Figures 2-6] The quantitative analysis is limited to raw counts and percentages, with no confidence intervals, effect sizes, or significance tests. The observed increases involve small numbers (e.g., female P1 participation rising from 3 to 8), making it difficult to judge whether these changes are meaningful or stable. Furthermore, the paper does not report how many of the students who took the OBI exam were actually participants in the training program, so the link between the training and the observed outcomes is not established. The authors should provide a breakdown of trained versus untrained test-takers and, at minimum, report the uncertainty around the reported counts before drawing any causal or even associational conclusions.
minor comments (5)
- [Section 5, paragraph on female participation] The phrase 'em relação ao ano de 2024' appears to be a typo for 'em relação ao ano de 2023', since the sentence compares 2023 (10 female participants) with 2024 (14 female participants).
- [Section 5, paragraph on campus expansion] The text first says that 'O Campus Uberlândia Centro e Uberaba participaram da prova em 2023' and later says that 'É a primeira participação na OBI na história do Campus Uberlândia.' Clarify whether 'Campus Uberlândia' and 'Campus Uberlândia Centro' are distinct campuses, and if so, which one is which, because the current wording is confusing.
- [Section 5, Figures 7 and 8] The captions for Figures 7 and 8 are identical to those of Figures 5 and 6 ('Participantes por nível - 2023' and '2024'), but the text states that Figures 7 and 8 show average scores. The figure captures should be corrected to reflect that they show performance means, not participant counts.
- [Abstract and Section 7] The abstract and conclusion use the verb 'demonstra', while Section 5 explicitly hedges the interpretation. Align the abstract and conclusion with the more cautious language used in the results section, or provide the additional analysis needed to justify the stronger wording.
- [Section 5, Table 2] The text mentions 'um aumento de 8 alunos' for P1 qualifiers, but Table 2 itself is not described with the baseline and post values. State the absolute numbers (e.g., from N to N+8) in the text so the reader does not have to infer them from the table alone.
Circularity Check
No significant circularity: the empirical claims rest on external OBI data, not on the authors' prior results.
full rationale
The paper contains no derived prediction, no fitted parameter, and no equation whose output is equivalent to its input by construction. The evidence presented consists of externally generated OBI participation counts, qualifier counts, and gender-disaggregated enrollment across IFTM campuses, compared between 2023 and 2024. These data come from an independent competition organized by the SBC, not from the paper's training model or from the authors' earlier publications. The training design is said to be 'baseado no treinamento proposto por [Menezes 2024]', and the choice of Neps Academy cites the authors' own 2021 medalist study, but neither citation defines the outcome measures or the year-over-year comparison. The conclusion in Section 7 ('os resultados preliminares demonstram que treinamento em programação é uma ferramenta relevante') does assert causal relevance, yet the paper itself hedges in Section 5 ('Estes dados podem sugerir um reflexo dos treinamentos oferecidos... no entanto, uma análise mais detalhada ainda será realizada') and explicitly reports that average scores in 2024 were not significantly different from 2023. That is an overstatement of inference strength based on weak observational evidence, not a circular reduction: the observed increases in participation and qualifiers remain independent, externally falsifiable observations that do not reduce to the cited prior work or to any definition in the paper. Self-citations are present but not load-bearing, so no circular step can be exhibited and the appropriate score is 0.
Assumptions & free parameters
assumptions (3)
- domain assumption OBI participation and advancement are valid proxies for programming skill and computational thinking
- domain assumption Volunteer enrollment produces comparable groups across years and does not bias observed changes
- domain assumption The increase in participating campuses is caused by the training and outreach program
Cite this review
Pith. "Pith review of Impacto de Treinamento em Programa\c{c}\~ao Competitiva no Ensino M\'edio: Resultados e Desafios." pith.science (2026). https://pith.science/paper/VXGNNKSN
@misc{pith2026250304732,
author = {Pith},
title = {Pith review of: Impacto de Treinamento em Programa\cc\~ao Competitiva no Ensino M\'edio: Resultados e Desafios},
year = {2026},
howpublished = {\url{https://pith.science/paper/VXGNNKSN}},
note = {Machine review of arXiv:2503.04732}
}
read the original abstract
This article presents an ongoing research aiming to develop an effective methodology for teaching programming, focusing on participation in the Brazilian Informatics Olympiad (OBI), for elementary and high school students. The training conducted with students from the Federal Institute and state schools, demonstrates the importance of programming training programs as a way to promote interest in computing, stimulate the development of computational skills, and increase participation in competitions such as the OBI. The next steps of the research include conducting more training cycles and analyzing the results obtained in the competitions.
Reference graph
Works this paper leans on
-
[1]
Conselho Nacional de Educação (2022). Parecer cne/ceb nº:2/2022. http://portal.mec.gov.br/index.php?option=com\_docman&view=download&alias=235511-pceb002-22&category\_slug=fevereiro-2022-pdf&Itemid=30192 . Acesso em: 16 jun. 2023
work page 2022
-
[2]
de Menezes, G. R., de Souza Pereira, J. H., and Theodoro, L. C. (2021). Análise do perfil dos medalhistas da olimpíada brasileira de informática 2019. Revista de Sistemas e Computação , 11:4--16
work page 2021
-
[3]
Herrington, J., McKenney, S., Reeves, T. C., and Oliver, R. (2007). Design-based research and doctoral students: Guidelines for preparing a dissertation proposal. In Montgomerie, C. and Seale, J., editors, Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2007 , pages 4089--4097. World Conference on Educational M...
work page 2007
-
[4]
Laaksonen, A. (2017). Guide to Competitive Programming , volume 1. Springer International Publishing, 1 edition
work page 2017
-
[5]
An Empirical Evaluation of Competitive Programming AI: A Case Study of AlphaCode
Lertbanjongam, S., Chinthanet, B., Ishio, T., Kula, R. G., Leelaprute, P., Manaskasemsak, B., Rungsawang, A., and Matsumoto, K. (2022). An empirical evaluation of competitive programming ai: A case study of alphacode. arXiv preprint arXiv:2208.08603
work page Pith review arXiv 2022
-
[6]
Martins, J. G. (2002). Aprendizagem baseada em Problemas Aplicada a Ambiente Virtual de Aprendizagem . PhD thesis, Universidade Federal de Santa Catarina
work page 2002
-
[7]
Menezes, G. (2024). Desenvolvimento de um método para potencializar os estudos de programação competitiva no ensino médio e fundamental. Master's thesis, Universidade Federal de Uberlândia
work page 2024
-
[8]
Piekarski, A. E. T., Miazaki, M., Junior, A. L. R., Militão, E. P., and Silva, J. V. P. (2023). Programação competitiva em um projeto de extensão para o ensino técnico em informática. Revista Conexao UEPG
work page 2023
Show all 13 references
-
[9]
Olimpíada brasileira de informática
SBC (2022). Olimpíada brasileira de informática. https://www.sbc.org.br/educacao/312-olimpiada-brasileira-de-informatica. Acesso em: 19 jun. 2024
2022
-
[10]
R., Silva, G., Lima, V., Tavares, W., and Bezerra, C
Sousa, P., Costa, J. R., Silva, G., Lima, V., Tavares, W., and Bezerra, C. (2021). Preparação para olimpíada brasileira de informática nível sênior: Um relato de experiência. In Anais do XXIX Workshop sobre Educação em Computação , pages 101--110, Porto Alegre, RS, Brasil. SBC
2021
-
[11]
Vitorino, M., Silva, H., Sampaio, L., and Gheyi, R. (2018). Perfil dos premiados em olimpíadas de informática e sua influência sobre a educação em computação. Brazilian Symposium on Computers in Education (Simpósio Brasileiro de Informática na Educação - SBIE) , 29(1):228
2018
-
[12]
Wing, J. M. (2006). Computational thinking. Commun. ACM , 49(3):33–35
2006
-
[13]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.