Pith. sign in

REVIEW 4 major objections 7 minor 7 references

Research and Analysis of Employers' Opinion on the Necessary Skills that Students in the Field of Web Programming Should Possess

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper reports a survey of 19 IT employer representatives and 10 graduating students showing that employers rank algorithmic and logical thinking as the top graduate skill, prefer deep programming-language knowledge over framework…

desk verdict A small, honest survey pilot whose conclusions outrun its sample, and whose own Table 4 contradicts Conclusion 6. read the letter →

arxiv 2506.11084 v1 pith:ATVDJDIZ submitted 2025-06-04 cs.SE

Yordan Kalmukov This is my paper
classification cs.SE
keywords webprogrammingemployersurveyalgorithmicthinkinglanguagesversusframeworksAI-assistedcodingstudentskillspracticalexams
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports a survey of 19 employer representatives and 10 graduating students on what web-programming graduates must know. Its central finding is that employers put algorithmic and logical thinking above all else, treat deep knowledge of a programming language as more valuable than framework fluency when teaching hours are limited, and want graduates who can still write code on their own. The paper also finds that 84% of employers consider it unacceptable for a graduate to write code only with AI assistance, although almost half of the surveyed students accept that. A practical exam taken on a computer is the preferred form of assessment, while AI tools are welcomed as part of learning. The stakes are curriculum design: whether universities should teach ready-made frameworks and AI workflows or foundational programming from scratch.

What carries the argument

The survey instrument: a 14-question questionnaire grouped into four topics (algorithmic thinking, built-in functions and APIs versus from-scratch implementation, frameworks versus language fundamentals, and AI use in teaching and exams) administered to 19 employer representatives and 10 graduating students. The paper reports percentage distributions per question and compares employer and student responses; for the exam-hypothetical question it also poses the same scenario to three AI chatbots and reports their recommended grades. The comparison structure carries the argument by showing where employer and student priorities diverge.

What would settle it

A larger survey using a documented random sample of IT employers in the same market would settle the claim: if a majority of that sample did not rank algorithmic and logical thinking as the most important graduate skill, or if fewer than half rejected the idea of AI-only coding, the paper's central conclusions would not survive.

Watch

Extended reading notes

Core claim

The paper's claim is that employers' requirements have shifted but not in the direction of tool fluency. Based on its survey, employers regard algorithmic and logical thinking as the single most important trait distinguishing good from mediocre programmers; most want graduates able to implement basic algorithms even when built-in functions or external APIs exist; and when forced to choose under a fixed teaching-hour budget, both employers and students prefer solid command of the programming language over knowledge of its frameworks. The sharpest result is that 84% of employers find it unacceptable for a graduate to write code with AI but not alone, while 45% of students accept it; and if a student can only produce an AI-generated solution during an exam, employers, students, and the three consulted AI chatbots converge on at most a passing grade of D, because the student does not understand the solution.

Load-bearing premise

The load-bearing premise is that the 19 employer representatives and 10 graduating students surveyed, drawn from unspecified Bulgarian companies by convenience rather than by a documented sampling frame, represent the wider population of IT employers and students well enough for the reported percentages to generalize.

Editorial extensions

If this is right

  • University web-programming courses should protect time for algorithmic and logical thinking and from-scratch implementation, since employers rank these above framework coverage.
  • When contact hours are limited, curricula should prioritize one programming language in depth over any single framework; framework familiarity can follow on the job.
  • Exams in web-programming subjects should be practical and taken on a computer, because both employers and a majority of students favor that format.
  • AI tools should enter the classroom, but assessment must verify unassisted coding, because 84% of employers reject graduates who can code only with AI.
  • Grading policies for AI-assisted exams should cap performance at a passing but low grade, since the student lacks understanding of the solution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the survey sample is representative, hiring screens in the studied market should test unassisted algorithmic problem-solving rather than framework familiarity.
  • The employer preference for language depth over frameworks suggests that framework-specific courses may depreciate quickly, while foundational skills are the durable part of a curriculum.
  • The gap between employers and students on AI-only coding points to a generational disagreement about assessment norms that universities will need to negotiate explicitly.
  • A testable extension would be to repeat the survey with a probability sample and a documented response rate, then check whether the 84% and 89% figures hold outside the initial convenience sample.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. Reporting on a survey conducted among Bulgarian IT employers (19 responses from managers and team leads) and graduating web-programming students (10 responses), the paper tabulates opinions on the importance of algorithmic thinking, implementing basic algorithms versus using built-in functions and APIs, teaching frameworks versus teaching a language in depth, and the use of AI tools in education and examinations. Its main claims are that employers overwhelmingly rank algorithmic and logical thinking as the most important skill (89%), prefer deep knowledge of a programming language over framework proficiency when teaching hours are limited, and overwhelmingly reject hiring graduates who can code only with AI assistance (84% unacceptable). The paper further asks three AI chatbots (ChatGPT, Claude, DeepSeek) to grade an AI-assisted exam scenario and includes their answers in the conclusions. The results are presented as descriptive percentages without statistical uncertainty, and several conclusions generalize the findings to the population of employers.

Significance. The topic is timely and practically relevant: curriculum decisions in web programming directly affect graduate employability, and the paper documents a useful warning signal about employers' concerns regarding AI-assisted coding skills. The paper's strengths are its readable tabulation of raw responses and its explicit reporting of the employer-student disagreement on AI-only coding; for the actual 29 respondents the arithmetic of the tables is mostly consistent. However, the paper has an evidentiary ceiling: a convenience sample of 19 employers with no sampling frame and no uncertainty quantification cannot sustain the unqualified population-level conclusions stated in the introduction and conclusions. If the authors reframe the claims as a descriptive pilot study of these 29 respondents, the findings retain genuine value for follow-up work.

major comments (4)
  1. [§Respondents (p. 260); Conclusions 1, 5; Fig. 3] All population-level claims ('89% of employers state...', '84% of employers find it not acceptable', conclusions 1 and 5) rest on 19 employer representatives recruited from unspecified companies, with no sampling frame, selection criteria, response rate, or comparison with the Bulgarian IT employer population. For n=19, the Clopper-Pearson 95% confidence interval for the 16/19 (84%) Q12 figure spans approximately 60-96%, and for 17/19 (89%) in Q1 approximately 65-99%; the data therefore cannot distinguish a modest majority from near-unanimity. The conclusions should either be explicitly restricted to the 29 respondents or be backed by a sampling methodology and confidence intervals before any generalization to employers as a population is made.
  2. [Table 4 vs. Conclusion 6] Conclusion 6 states that 'according to students, employers and artificial intelligence (ChatGPT, Claude and DeepSeek)... the student should be assessed with a maximum grade of Sufficient, D', but Table 4 reports that 56% of students chose 'Fail, F' for exactly this scenario, with only 44% choosing 'Sufficient, D' or 'Good, C'. The students in the authors' own table are stricter than the employers, not in agreement with a D maximum. The conclusion misreads its own data and must be corrected or the table re-verified.
  3. [§The same question is given to... (pp. 263-264); Conclusion 6] The ChatGPT, Claude, and DeepSeek responses are treated in Conclusion 6 as corroborating survey evidence ('according to students, employers and artificial intelligence'). These outputs were generated from a single prompt authored by the researchers, whose exact wording is not given, so they cannot serve as independent evidence about fair grading; at most they illustrate what current chatbots say when so prompted. The manuscript should label this section as an informal probe of AI opinions, disclose the full prompt, and remove the AI answers from any conclusion framed as an empirical finding.
  4. [p. 262 ('Due to the page limitation...'); Conclusion 3] Conclusion 3 asserts that 'according to 2/3 of employers and half of students, the exam in subjects related to web programming should be practical on a computer', yet the paper explicitly states that the analyses of these questions are omitted, and no table or figure in the text reports these percentages. A conclusion without any presented supporting evidence cannot be checked by the reader; either include a compact table for the exam-related questions (Q7-Q10) or delete conclusion 3.
minor comments (7)
  1. [p. 260, Introduction] The sentence 'To answer the abovementioned important question objectively, I have developed and conducted a survey' would read more smoothly as 'To answer this question, I developed and conducted a survey.'
  2. [Table 3 (p. 262)] The phrase 'how it could make their live as developers easier' contains a spelling error ('live' should be 'lives').
  3. [Table 1 (p. 261)] In Q3, the student column shows 67% and 33% for two responses while the third response ('No, there's no point...') has no student percentage; please clarify whether that option was offered to students or selected by none of them, and apply the same clarification to Q4, where the students have no value for 'It's not that important'.
  4. [Fig. 1 (p. 260)] The text says 'almost half of the students do not think this is the most important thing', but the exact student percentages from Fig. 1 are never given in the text; please state the figures explicitly.
  5. [Fig. 2 (p. 262)] The discussion of Q6 says both groups 'overwhelmingly agree' on the importance of language knowledge, but the actual percentages in Fig. 2 are not stated in the text; please report them for reproducibility.
  6. [References] References [2]-[7] are listed as bare URLs with access dates; please provide the full publication titles and, where available, the publication dates for these online sources.
  7. [p. 260, Respondents] The description of companies as 'large and small local companies' is vague; a short breakdown by company size or sector would help readers assess the sample composition.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a direct survey tabulation with no derivation loop, and the AI-chatbot responses are additional queries rather than self-referential inputs.

full rationale

The paper reports the results of a 14-question survey among 19 employer representatives and 10 students. Every stated conclusion is a direct percentage of the tabulated responses (e.g., Q1: 89% of employers; Q12: 84% of employers; Q13 and Table 4 distributions). There is no mathematical derivation, no fitted parameter that is later renamed as a prediction, and no load-bearing self-citation that would make the conclusions equivalent to their inputs by construction. The AI-chatbot section in the Q13 discussion asks ChatGPT, Claude, and DeepSeek the same exam question and then reports their answers in Conclusion 6; this is an independent query, not a circular use of inputs, although it is methodologically weak as corroborating evidence. Concerns about the convenience sample, the absence of sampling methodology, and wide confidence intervals are validity and generalizability limitations, not circularity. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No numerical fitting occurs, so there are no free parameters or invented entities. The analysis rests on three domain assumptions about the validity of survey self-reports, sample representativeness, and the relevance of chatbot answers.

assumptions (3)
  • domain assumption Respondents' self-reported opinions are an accurate proxy for actual employer hiring preferences and actual student beliefs.
    The survey only records stated opinions; no behavioral data (hiring decisions, test scores) validates this mapping. Used in all result tables and conclusions.
  • domain assumption The convenience sample of 19 employers and 10 students is representative of Bulgarian IT employers and graduating students.
    The Respondents section provides only roles and company types; no sampling frame, randomization, or response rate is reported. The conclusions generalize beyond this sample.
  • ad hoc to paper Answers generated by ChatGPT, Claude, and DeepSeek to the authors' single prompt constitute meaningful, independent evidence about fair grading of AI-assisted exams.
    Introduced in the Q13 discussion and Conclusion item 6; these outputs are produced from the authors' prompt and are not independent empirical observations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Research and Analysis of Employers' Opinion on the Necessary Skills that Students in the Field of Web Programming Should Possess." pith.science (2026). https://pith.science/paper/ATVDJDIZ

@misc{pith2026250611084,
  author       = {Pith},
  title        = {Pith review of: Research and Analysis of Employers' Opinion on the Necessary Skills that Students in the Field of Web Programming Should Possess},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ATVDJDIZ}},
  note         = {Machine review of arXiv:2506.11084}
}
read the original abstract

In the era of artificial intelligence (AI) and chatbots, based on large language models that can generate programming code in any language, write texts and summarize information, it is obvious that the requirements of employers for graduating students have already changed. The modern IT world offers significant automation of programming through software frameworks and a huge set of third-party libraries and application programming interfaces (APIs). All these tools provide most of the necessary functionality out of the box (already implemented), and quite naturally the question arises as to what is more useful for students - to teach how to use these ready-made tools or the basic principles of working and development of web applications from scratch. This paper analyzes the results of a survey conducted among IT employers, aimed to identify what, in their opinion, are the necessary technical skills that graduating students in the field of Web Programming should possess in order to join the company's work as quickly and effectively as possible.

Figures

Figures reproduced from arXiv: 2506.11084 by the authors.

Figure 1
Figure 1. Question 1 and percentage distribution of answers of employers and students [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. What is more important: knowing the programming language in details [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. According to 84% of employers it is not acceptable that students can write code [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

7 extracted references · 7 canonical work pages

  1. [1]

    Real-time statistics on Developers, March 2025, https://mycodelesswebsite.com/developer-statistics/ (Accessed April 2025)

    work page 2025

  2. [2]

    Economic.bg. Какви IT специалисти търсят технологичните компании в България?, 31.03.2025, https://www.economic.bg/bg/a/view/ai-robotika-i- kibersigurnost-kakvi-it-specialisti-tyrsjat-tehnologichnite-kompanii-v-bylgarija (Accessed April 2025)

    work page 2025

  3. [3]

    Top Web Developer Skills You Should Learn in 2025, January 2025, https://www.browserstack.com/guide/top-web-developer-skills (Accessed April 2025)

    BrowserStack. Top Web Developer Skills You Should Learn in 2025, January 2025, https://www.browserstack.com/guide/top-web-developer-skills (Accessed April 2025)

    work page 2025

  4. [4]

    7 IT Skills That Will Boost Your Resume in 2025, December 2024, https://career.softserveinc.com/en-us/stories/top-it-skills-for-resume (Accessed April 2025)

    SoftServe. 7 IT Skills That Will Boost Your Resume in 2025, December 2024, https://career.softserveinc.com/en-us/stories/top-it-skills-for-resume (Accessed April 2025)

    work page 2025

  5. [5]

    Introducing ChatGPT, November 30, 2022, https://openai.com/index/chatgpt/ (Accessed April 2025)

    OpenAI. Introducing ChatGPT, November 30, 2022, https://openai.com/index/chatgpt/ (Accessed April 2025)

    work page 2022

  6. [6]

    Meet Claude – the AI for all of us, https://www.anthropic.com/claude (Accessed April 2025)

    Anthropic PBC. Meet Claude – the AI for all of us, https://www.anthropic.com/claude (Accessed April 2025)

    work page 2025

  7. [7]

    DeepSeek, https://www.deepseek.com/en (Accessed April 2025)

    work page 2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.