{"id":"e4d292e0-dce4-4590-8621-ef12daaf95fc","arxiv_id":"1909.00699","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A field study in one Italian high school suggests that combining a gamified energy challenge with hands-on IoT sensor activities increased student engagement and enabled a measured lighting-energy saving of roughly 21 kWh per day.","lead":"The paper reports two years of classroom experiments in an Italian high school where students used a gamified web app and an IoT sensor kit to learn about energy use. The authors claim the activities raised engagement and produced a measured lighting-energy saving of about 21 kilowatt-hours per day in the school hall.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 21 kWh/day savings estimate depends on an uncalibrated 400 lux threshold and an underspecified before/after power comparison; a calibration check could substantially change the headline figure.","rationale":"The paper is a credible experience report with internally consistent qualitative findings and correct arithmetic. The load-bearing issue is the quantitative energy-savings claim in Section 6, which is used in the abstract as evidence that increased engagement led to actual energy savings. That claim rests on two unverified quantities: the 400 lux threshold, which is admitted to be a rough estimation from poorly oriented sensors, and the before/after lighting power averages, whose definition and measurement context are not given. Neither is fatal to the paper's educational contribution, but both are necessary to support the headline saving figure. A calibration check and inspection of the underlying power time series would settle whether the 21 kWh/day figure is robust. Because the concern is concrete and addressable, the conditional verdict stands.","tokens_in":10703,"tokens_out":4033,"duration_ms":44045,"concrete_test":"Calibrate at least one of the hall luminosity sensors against a handheld lux meter at multiple representative positions and times across a week, including cloudy and sunny days; map the raw 400 threshold to true illuminance and recompute the daily interval during which 150 lux is already available. Then extract the hall power time series from the GAIA IoT platform for the exact baseline and intervention dates, confirm the meter covers only hall lighting, and recompute the 4.9 kW / 1.9 kW averages over matched weekday and hour subsets. If the calibrated interval is only 4 hours instead of 7, the daily saving is 12 kWh, not 21 kWh.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 6 reports that students set a 400 lux threshold as a 'rough estimation' because luminosity sensor readings 'are highly related to their orientation' and are 'not optimal for calculating a luminosity average value.' The paper provides no calibration linking the raw sensor value to the 150 lux recommended illuminance for circulation areas. If the true illuminance corresponding to a raw reading of 400 is, say, only 100 lux in the occupied zone, then the 10:00-17:00 interval is not a period when lights can be off without violating the guideline, and the 3 kW / 21 kWh per day figure is an overestimate. Conversely, if the raw threshold corresponds to much more than 150 lux, the method may leave the hall over-lit on sunny days, which is not the claimed saving. The before/after power comparison is also underspecified: the paper states average values of 4.9 kW and 1.9 kW without giving measurement dates, averaging windows, whether the hall meter is dedicated solely to lighting circuits, or whether the two periods are matched for occupancy and season. The arithmetic 3 kW x 7 h = 21 kWh is correct only if both the interval and the two power averages are valid. Thus the central quantitative claim is a potential saving, not a demonstrated achieved saving with a bounded uncertainty.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports on two years of educational activities in an Italian high school aimed at energy awareness through IoT-based tools and gamification. The authors describe a web-based Challenge with quests and competitions, and a hands-on lab kit (Raspberry Pi, GrovePi sensors, Node-RED with a GaiaNode plugin) integrated into a computer science curriculum. They report high student engagement, positive questionnaire responses, and a teacher-reported improvement in exam performance. Section 6 presents a behavioral intervention in which students, after analyzing luminosity data, set a 400 lux threshold and concluded that hall lights could be turned off between 10:00 and 17:00, reducing measured power consumption from 4.9 kW to 1.9 kW and saving an estimated 21 kWh per day. The paper's central claim is that combining gamification and IoT-based data-driven activities is an effective way to engage students and produce both learning outcomes and energy savings.","tokens_in":11063,"tokens_out":2268,"duration_ms":28208,"significance":"If the claims are accepted, the paper is a useful addition to the small but growing literature on IoT-enabled energy education in secondary schools. It reports on a real, long-running deployment, and it makes practical artifacts available: the Node-RED GaiaNode plugin, the LabKit documentation, and a reproducible template for classroom activities. The qualitative observations about gamification—especially the students' reverse-engineering of scoring and their strong reaction to perceived unfairness—are valuable and transferable lessons for similar interventions. The questionnaire and exam results, though modest, support the feasibility of embedding such activities in existing curricula. However, the quantitative energy-savings claim in Section 6 is not yet supported by the evidence presented, and the causal language in the abstract overstates what a single-school case study can establish.","major_comments":[{"comment":"The 400 lux threshold is the load-bearing element of the savings calculation, yet it is explicitly described as a 'rough estimation' made by students because the luminosity sensors' readings 'are highly related to their orientation' and are 'not optimal for calculating a luminosity average value.' The paper provides no calibration or validation linking the raw sensor reading of 400 to the recommended 150 lux for circulation areas. If the raw threshold corresponds to a much lower or higher illuminance in the occupied zone, the 10:00–17:00 interval and the resulting 3 kW and 21 kWh figures change substantially. The manuscript should either provide a calibration check, report the savings as an illustrative student-driven estimate rather than a demonstrated achieved saving, or state explicitly the uncertainty in the interval selection.","section":"Section 6, Figure 4"},{"comment":"The before/after comparison is underspecified. The paper states average lighting power values of 4.9 kW and 1.9 kW without giving the measurement dates, the length of the averaging windows, whether the monitored circuit is dedicated solely to hall lighting, or whether the two periods are matched for occupancy, season, and weather conditions. The arithmetic 3 kW x 7 hours = 21 kWh is internally consistent, but it is only as meaningful as the two power averages and the seven-hour interval. Please specify the measurement protocol and provide the raw or aggregated data for both periods, or reduce the strength of the quantitative conclusion.","section":"Section 6, power comparison"},{"comment":"The abstract states that increased engagement 'led to both actual energy savings and positive learning outcomes,' but the study design does not support a causal claim. This is a single school with no control group and no counterfactual analysis; the observed drop from 4.9 kW to 1.9 kW could be influenced by changes in occupancy, season, maintenance, or other building operations coincident with the intervention. The paper should be reframed as a case study reporting observed changes and participant-reported outcomes, with the causal attribution explicitly marked as an interpretation rather than a demonstrated effect.","section":"Abstract and Section 7"}],"minor_comments":[{"comment":"There is a repeated word in 'installed installed inside classrooms'; please correct this typo.","section":"Section 3"},{"comment":"The sentence 'The scores obtained by the class were are: 2 excellent, 3 good, 6 satisfactory, 7 sufficient, 4 insufficient' contains a typo ('were are') and should be reworded for clarity.","section":"Section 5.3"},{"comment":"The sentence 'On average the class performed better with respect to previous computer science were better than previous exams' is garbled and should be rewritten, for example to say that the class performed better than in previous computer science exams, according to the teacher.","section":"Section 5.3"},{"comment":"The phrase 'the sensors produce that are highly related to their orientation' appears to be missing a noun (e.g., 'values that are highly related'). Please clarify the sentence.","section":"Section 6"},{"comment":"The GaiaNode plugin and LabKit documentation are said to be available on GitHub, but no URL is provided in the text; adding the repository link would improve reproducibility.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a borderline case: it is a useful experience report, but the quantitative energy-saving claim in Section 6 is not yet reliable and the abstract overstates the causal conclusion. With a calibration discussion, a clearer measurement protocol, and softened causal language, the paper could be acceptable for an applied/education-oriented venue. If the authors cannot supply the missing measurement details, the quantitative claims should be removed or explicitly labeled as illustrative student work rather than validated savings."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid, honest experience report from the GAIA project, and the most valuable part is not the numbers. The genuinely new material is the documented dynamics of the gamified Challenge: students reverse-engineering the scoring, disputing fairness, proposing a fix, and nearly withdrawing over perceived ethical issues. That is a real observation about competition mechanics in schools, and it is reported with enough detail to be useful. The hands-on Node-RED/LabKit curriculum design is also concrete and transferable, with a sensible template and a plausible account of what worked and what did not.\n\nThe quantitative energy-saving claim is where I would pump the brakes. The arithmetic is fine: 4.9 kW minus 1.9 kW is 3 kW, times 7 hours is 21 kWh. But that is a potential saving, not a demonstrated achieved saving. The 400 lux threshold was a rough estimation made by students from sensors whose readings depend on orientation; the paper acknowledges the sensors are 'not optimal for calculating a luminosity average value.' There is no calibration linking the raw threshold to the 150 lux guideline for circulation areas. And the before/after comparison gives no dates, no averaging window, no detail on whether the hall power channel is exclusively lighting, and no control for season or occupancy. So the 21 kWh/day figure is best-treated as an illustrative exercise.\n\nThe causal claim in the abstract—that increased engagement led to actual energy savings and positive learning outcomes—is stronger than the evidence. The learning outcomes are supported by questionnaire answers plus the teacher's subjective assessment, which is fine for an experience report. The energy savings are not causally identified. The paper does not overclaim in the body at the same level; Section 6 carefully says 'could be saved,' and later calls it 'potential energy savings.' So the soft spot is mostly in the abstract and the framing, not in a hidden fatal flaw.\n\nCitation pattern looks honest; the GAIA-related self-citations are to prior work that actually is the basis of the lab kit. No machine-checked proofs or code artifacts are shipped, but that is not what this venue asks for.\n\nWho should read it? People designing school-based IoT sustainability interventions, and researchers studying gamification in classrooms. It is a worthwhile peer-review candidate as an experience report, but I would ask the authors to clearly label the energy figure as an order-of-magnitude estimate and to remove or weaken the causal phrasing in the abstract.","headline":"A readable, honest experience report whose real value is the qualitative gamification dynamics; the 21 kWh/day savings figure is a rough potential estimate, not a demonstrated result.","tokens_in":11484,"tokens_out":2251,"would_cite":false,"duration_ms":24593,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper reports that combining gamified competition with hands-on IoT lab activities kept high school students engaged and produced measurable energy savings, including a 3 kW drop in a school hall's lighting load.","keywords":["Internet of Things","energy awareness","gamification","competition","sustainability","STEM education","Node-RED","behavior change"],"falsifier":"Measure the hall's illuminance at working height with a calibrated meter and record power for two full weeks with the usual lighting schedule; if the 10:00–17:00 interval does not stay above a genuine 150-lux equivalent, or if the daytime power draw is already near 1.9 kW before the intervention, the claimed 21 kWh daily saving is not real.","tokens_in":10492,"feed_emoji":"💡","tokens_out":6497,"duration_ms":48175,"temperature":0.7,"pith_summary":"The paper tries to establish that combining gamified competition with hands-on IoT lab activities can keep high school students engaged enough to produce measurable energy savings and better learning outcomes. It reports on two consecutive school years in one Italian high school: classes used an online quest-based Challenge with leaderboards, and a later computer-science module used Raspberry Pi sensor kits with Node-RED to monitor their own school. The authors' central demonstration is a behavior-change episode in which students set a 400 lux threshold, identified that the school hall's lights were unnecessary between 10 am and 5 pm, and persuaded staff to turn them off, dropping measured lighting power from 4.9 kW to 1.9 kW—about 21 kWh saved per day. A sympathetic reader would take the contribution as a concrete template, not a controlled experiment: the same toolkit, embedded in a normal curriculum, produced both engagement spikes and an observable drop in consumption.","feed_headline":"Gamification and IoT lessons cut hall lighting power by 3 kW","feed_subtitle":"Students set a 400 lux threshold, found needless daytime lights, and acted — 21 kWh saved per day.","key_machinery":"The central objects are two complementary tools. The Challenge is a web application in which students complete Quests grouped into five sustainability areas, earn scores, and compete across classes and schools; its role is to raise engagement through competition and peer evaluation. The LabKit is a low-cost sensor board built around a Raspberry Pi with a GrovePi add-on, light/temperature/humidity/sound sensors, an LCD, and Node-RED, a flow-based programming environment in which students wire together nodes to fetch data from the school's IoT platform and push new sensor readings into it. The combined mechanism is data-driven behavior change: students observe real measurements from their own building, set an explicit threshold (400 lux), spot the wasteful interval, act on the physical environment, and then measure the effect.","core_discovery":"On the paper's own terms, the central discovery is that 'soft' engagement (gamification and inter-school competition) and 'hands-on' data-driven lab work are complementary mechanisms that work in a real classroom: the Challenge produced visibly higher activity during competition windows and even led students to reverse-engineer the scoring system, while the Node-RED lab activity let 22 students analyze their own school's temperature and light data and translate findings into energy-saving actions. The concrete quantitative claim is the hall-lighting intervention: with usual lighting the hall drew about 4.9 kW; after switching off what was unnecessary it drew 1.9 kW, implying about 21 kWh saved over the seven wasteful hours in a day. The paper also reports positive questionnaire responses and exam scores better than previous computer-science exams, which it interprets as signs of increased engagement and positive learning outcomes.","pith_inferences":["If the 400 lux threshold is calibrated against a proper illuminance meter, the 3 kW saving may turn out larger or smaller; a simple repeat measurement at another time of year would sharpen the central number without needing a control group.","The study's lack of a control school means the observed exam improvement and questionnaire positivity could partly reflect novelty or teacher enthusiasm; a useful extension would be to randomize classes within the same school to Challenge-plus-lab versus traditional lessons.","The students' 'reverse engineering' of the scoring system suggests that gamification's main cognitive payoff may be data literacy and systems thinking, not just energy awareness; this spillover is worth measuring directly in future work.","Because the intervention combined gamification and lab work in sequence, the paper cannot separate which ingredient caused the savings; a factorial design with gamification only, lab only, and both would isolate the active mechanism."],"forward_implications":["If the approach works as reported, schools without budgets for actuators or retrofits can still achieve energy savings by coupling low-cost sensing with curriculum-integrated activities.","The same data-threshold-act-measure loop could be transferred to other resource-use problems, such as water, heating, or waste, since the mechanism only requires sensor data, a threshold, and a motivated class.","The gamification findings imply that competition drives engagement quickly, but designers must build safeguards, such as limited registration windows and identity verification, to prevent perceived unfairness from destroying trust.","The 21 kWh-per-day figure, if it generalizes across a school year, would make behavior-only school interventions a meaningful, cheap complement to building retrofits.","Teacher involvement and embedding the activity in an existing subject, here computer science, appear to be prerequisites for reaching students, not optional extras."],"supporting_citations":[{"why":"Supplies the educational lab kit and energy-awareness scenarios that this paper extends into a high-school computer-science curriculum.","marker":"[18]"},{"why":"Shows a prior rapid-prototyping toolkit use with high school students, the methodological precedent for the hands-on activity.","marker":"[13]"},{"why":"Provides the survey approach and secondary-students' stance data used to shape the questionnaire evaluation.","marker":"[14]"},{"why":"Surveys gamified systems for energy and water sustainability, framing the gamification design space.","marker":"[8]"},{"why":"Precedent for data-driven IoT educational scenarios in secondary schools, here applied to water rather than energy.","marker":"[22]"},{"why":"Supplies recommendations for plug-and-play physical computing toolkits that informed the LabKit design.","marker":"[16]"},{"why":"Defines Node-RED, the flow-based programming environment that carries the hands-on activity.","marker":"[3]"}],"fun_headline_variants":["Gamified IoT classes help students cut school hall energy use","Students find needless lights, save 21 kWh daily via IoT lessons","Competition and hands-on IoT labs drive real energy savings in schools","Data-driven lab activity lets teens trim hall lighting waste"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 3 kW and 21 kWh daily savings depend on the students' rough 400 lux threshold correctly marking when hall lighting was genuinely unnecessary, and on the assumption that the drop from 4.9 kW to 1.9 kW came from the educational activities rather than from seasonal light, occupancy, or other building changes.","fun_headline_variants_meta":{"raw":{"variants":["Gamified IoT classes help students cut school hall energy use","Students find needless lights, save 21 kWh daily via IoT lessons","Competition and hands-on IoT labs drive real energy savings in schools","Data-driven lab activity lets teens trim hall lighting waste"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1434,"prompt_tokens":927,"completion_tokens":507,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":435}},"tokens_in":543,"tokens_out":507,"duration_ms":126923,"temperature":1.0,"reasoning_tokens":435,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:38:18.188314+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the hall's illuminance at working height with a calibrated meter and record power for two full weeks with the usual lighting schedule; if the 10:00–17:00 interval does not stay above a genuine 150-lux equivalent, or if the daytime power draw is already near 1.9 kW before the intervention, the claimed 21 kWh daily saving is not real.","supporting_citations":[{"cited_title":"International Journal of Child-Computer Interaction 20, 43 – 53 (2019)","cited_arxiv_id":null,"evidence_quote":"Supplies the educational lab kit and energy-awareness scenarios that this paper extends into a high-school computer-science curriculum."},{"cited_title":"In: European Conference on Ambient Intelligence","cited_arxiv_id":null,"evidence_quote":"Shows a prior rapid-prototyping toolkit use with high school students, the methodological precedent for the hands-on activity."},{"cited_title":"In: Kameas, A., Stathis, K","cited_arxiv_id":null,"evidence_quote":"Provides the survey approach and secondary-students' stance data used to shape the questionnaire evaluation."},{"cited_title":"Games 9(3) (2018)","cited_arxiv_id":null,"evidence_quote":"Surveys gamified systems for energy and water sustainability, framing the gamification design space."},{"cited_title":"In: European Conference on Ambient Intelligence","cited_arxiv_id":null,"evidence_quote":"Precedent for data-driven IoT educational scenarios in secondary schools, here applied to water rather than energy."},{"cited_title":"International Journal of Child-Computer Interaction 17, 72–82 (2018)","cited_arxiv_id":null,"evidence_quote":"Supplies recommendations for plug-and-play physical computing toolkits that informed the LabKit design."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines Node-RED, the flow-based programming environment that carries the hands-on activity."}],"review_version":1}