{"id":"85647747-1a44-4633-94ce-dee93f09d9bb","arxiv_id":"2412.04045","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"AI4EF is a modular, open-source decision-support platform for building energy retrofits and photovoltaic installation assessment, described here with architecture and screenshots but without quantitative validation.","lead":"AI4EF is an open-source web platform that uses machine learning to help building owners and consultants estimate retrofit costs, savings, and emissions impacts. The paper describes the architecture and screenshots, but provides no benchmarks or quantitative validation of the underlying predictions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central reliability claim rests on unquantified 'real-world validation'; no model metrics, dataset, or user evaluation are reported, so the platform's predictive value is not established.","rationale":"The reader's weakest_assumption identifies exactly the load-bearing gap: the paper claims real-world validation and user satisfaction without providing any supporting evidence. My independent reading reaches the same conclusion. The most central claim is not merely that a software platform exists but that its ML models produce reliable retrofit and PV recommendations; no performance metrics, dataset description, or evaluation protocol appear anywhere in the manuscript. The architecture and MLOps pipeline are described in reasonable detail, and the public repository is a concrete artifact, which supports the 'software description' aspect. However, the stated advantage over existing tools depends on predictive quality and real-user uptake, both of which are asserted rather than demonstrated. A concrete reproducibility check on the repository can settle whether the validation claim has substance. Since the reader already issued a conditional verdict, no change in verdict is needed; the condition should explicitly require a public evaluation of model accuracy and a user study.","tokens_in":10034,"tokens_out":3096,"duration_ms":33534,"concrete_test":"Clone the public repository at version 0.0.2 (https://github.com/epu-ntua/enershare-ai4ef), locate the LEIF dataset and the trained artifacts referenced in Table 4 (bestMLPClassifier.ckpt, MLPClassifierscalers.pkl). If those artifacts exist, rerun the Evaluation pipeline on the saved test split and report MAE/RMSE for regression targets and accuracy/precision/confusion matrix for the classifier targets, then compare with a mean/majority-class baseline. If the artifacts or LEIF data are absent, attempt to retrain from the documented data URL and repeat the evaluation; if no data or metrics can be produced, the reliability claim is unverified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution is not just an architecture but a decision-support tool whose ML models 'predict energy consumption, retrofit costs, and environmental impacts' and give tailored retrofit/PV recommendations. For that claim to hold, the deployed models must generalize to real buildings. The manuscript never provides evidence of that. Section 1 bullet 3 and Section 4 assert that AI4EF 'was built on real data based on the needs of LEIF' and 'was evaluated and utilised by real users satisfying their needs,' but no dataset, sample size, feature distributions, train/test split, or accuracy/satisfaction measures are given. Section 3 shows only illustrative screenshots and example inputs (Tables 3 and 4); Table 2 lists inputs and targets but not any quantitative output. The Training Playground can produce metrics, but no metrics from the actual LEIF deployment are reported. Thus the load-bearing premise, model reliability and user satisfaction, is an unsupported assertion. If the real-data/user claim is false or the models have poor out-of-sample accuracy, the stated advantage over OpenStudio, Retrofit Advisor, and Aurora Solar collapses. This is a missing-evidence concern rather than an identified error; it can be settled by examining the released repository and running the training/evaluation pipeline.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"AI4EF is presented as a modular, AI-based decision-support platform for building energy retrofitting and photovoltaic installation assessment. The paper describes the system architecture (Docker microservices, React frontend, Python/MLOps backend), the Training Playground for custom model training, and integration with the Enershare Data Space. The introduction claims that AI4EF is open-source, customizable, validated with real users at the Latvian Environmental Investment Fund (LEIF), and integrated with European energy data sharing. The body includes illustrative UI screenshots and example input tables but no quantitative evaluation of the platform's predictive models or user satisfaction.","tokens_in":10261,"tokens_out":5297,"duration_ms":49563,"significance":"The paper's main contribution is the architectural and software engineering description of AI4EF, which combines building retrofit recommendations, PV assessment, and an MLOps training environment in one platform. If the claimed real-world validation were documented with data and metrics, this would be a useful addition to the building energy efficiency software landscape, especially for integrating European Data Space standards. The code repository and use of standard ML tools are commendable. However, the current version lacks the evidence needed to substantiate the central claims of reliable predictions and user satisfaction, limiting the paper's immediate impact.","major_comments":[{"comment":"The third contribution bullet in Section 1 and the Impact section state that AI4EF 'was built on real data based on the needs of LEIF' and 'was evaluated and utilised by real users satisfying their needs.' However, the manuscript provides no dataset description, no sample size, no evaluation protocol, no prediction accuracy or satisfaction measures, and no user study details. This is a load-bearing claim: without such evidence, the assertion that AI4EF meets real-world demands and offers reliable recommendations is unsupported. Please either add a validation section with quantitative results (e.g., model performance on a held-out LEIF dataset, user surveys, or case studies) or revise the claims to reflect the current evidence level.","section":"Section 1 (third bullet), Section 4"},{"comment":"The illustrative examples in Section 3 show only screenshots and input parameter tables; no quantitative results from the Building Retrofitting or Photovoltaic models are reported. The Training Playground is said to produce metrics and visualizations (Figures 5 and 6), but no metrics from the actual LEIF deployment or any benchmark are given. Without prediction accuracy (e.g., MAE, RMSE, classification accuracy, F1), error bars, or a comparison to existing tools (OpenStudio, Aurora Solar, etc.), the central claim that AI4EF provides reliable 'tailored recommendations' is not established. Please include an evaluation section with concrete numbers and, ideally, ablations or baseline comparisons.","section":"Section 3, Tables 2 and 3, Figures 3, 5, 6"},{"comment":"The first contribution bullet describes 'Open-Source Accessibility' and contrasts AI4EF with proprietary tools, but Table 1 reports the legal license as 'Attribution-Noncommercial 4.0 International' (CC BY-NC). This license restricts commercial use and is not an OSI-approved open-source license, which conflicts with the paper's open-source claim. Please clarify the licensing terms and align the description with the actual license, or reconsider the claim.","section":"Section 1 (first bullet), Table 1"}],"minor_comments":[{"comment":"The heading 'Software functionalities' is immediately followed by '2.3 Interface Features' with no content; please either provide the intended discussion or remove the empty subsection.","section":"Section 2.2"},{"comment":"'HV AC' should be 'HVAC' (the term is used inconsistently in the text).","section":"Nomenclature"},{"comment":"Reference [12] contains a garbled author list; please fix the citation.","section":"References"},{"comment":"Figure numbering appears out of order: Section 3.2 refers to 'Figure 7' before Figure 8, and some screenshots are only in a later appendix; consider renumbering for clarity.","section":"Section 3.2, Figures"},{"comment":"Table 1 lists the code and software licenses as 'Attribution-Noncommercial 4.0 International', but the text uses 'open-source' in the introduction; please use consistent terminology.","section":"Table 1, Section 1"},{"comment":"In Section 3.1, the text says users can leave the energy generation field blank in the PV sub-service, but the example input in Table 3 shows a placeholder '-' for that field; verify consistency.","section":"Section 3.1, Table 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a system description rather than a validation study. The missing evaluation evidence for the real-world validation and model performance claims is the central issue; it is fixable within the scope of the paper. The authors should either add a proper evaluation section or reframe the contributions as architectural design and implementation, avoiding over-claiming demonstrated user satisfaction and predictive accuracy. The code availability is a strength and should be highlighted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a legitimate engineering contribution—a modular, open-source platform (MLP models, Dagster pipelines, Optuna tuning, Keycloak, Enershare data space) with a public repo. The architecture is described clearly and the paper is honest about being a systems description. If you want to know how to assemble these components into a retrofit decision-support tool, this is a useful reference.\n\nWhat it does well: the choice of MLOps practices is sensible, the separation of services (retrofitting, PV, training playground) is clean, and the screenshots and parameter tables give a concrete sense of the user flow. Shipping the code under an open license is real: a reader can go run the pipeline. That is more than many software papers do.\n\nWhere it falls short: the central claim in Section 1—that the tool \"was built on real data based on the needs of LEIF\" and \"was evaluated and utilised by real users satisfying their needs\"—appears without a single number. No dataset description, no sample size, no accuracy metric, no error bar, no user satisfaction score. The Training Playground screenshots show metrics, but none from the actual LEIF deployment. This is a load-bearing evidence gap: the stated advantage over OpenStudio, Retrofit Advisor, and Aurora Solar rests on the platform producing reliable recommendations. The gap is fixable—either by reporting evaluation results from the repository or by rewriting the claim as \"a demonstration platform with illustrative examples.\" As written, the validation claim overreaches.\n\nThe stress-test note about circularity is on target. The authors assert their own tool works, but that is an evidence gap, not a logical error. There are no fitted parameters or predictions that reduce to themselves, so circularity isn't the right frame.\n\nWho is this for: readers in building energy efficiency who want a concrete template for a modular ML decision-support system, and reviewers at a software journal (SoftwareX type). It doesn't break new scientific ground, but it doesn't need to.\n\nRecommendation: send it to peer review, but require the authors to either add a quantitative evaluation section (or at minimum a link to reproducible metrics in the repo) and soften the real-world validation claim to match the evidence. The engineering deserves attention; the marketing doesn't.","headline":"A well-described open-source building retrofit tool whose stated real-world validation is asserted without numbers; the engineering is solid, the evidence is not.","tokens_in":10786,"tokens_out":1845,"would_cite":false,"duration_ms":16831,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AI4EF claims its ML services turn a few building parameters into reliable retrofit and PV recommendations.","keywords":["energy efficiency","machine learning","building retrofitting","photovoltaic installation","decision support","predictive modeling","MLOps","European energy data space"],"falsifier":"Inspect the released version 0.0.2 and rerun the Training Playground pipeline on a held-out set of building records with known post-retrofit outcomes; if predicted energy savings, payback periods, or CO2 reductions are not compared against measured results or a simple baseline and show large errors, the platform's central promise of reliable, user-satisfying recommendations is false.","tokens_in":9867,"feed_emoji":"⚡","tokens_out":7340,"duration_ms":71831,"temperature":0.7,"pith_summary":"AI4EF is a software platform for building energy decision-making. The paper's central claim is that its machine-learning services can turn a few building parameters into tailored recommendations for retrofits and photovoltaic installations, estimating energy savings, costs, payback periods, and CO2 reductions. A training environment lets data scientists replace the default models with custom ones, and integration with a European energy data space provides additional data for training. The authors assert the tool was built on real data and used by actual stakeholders, although the manuscript itself reports no user study or accuracy numbers.","feed_headline":"One platform predicts retrofit savings from minimal data","feed_subtitle":"New tool pairs ML retrofit and solar advice with an environment for retraining models on your own data.","key_machinery":"The load-bearing mechanism is a modular microservices backend with two MLApp sub-services, Building Retrofitting and Photovoltaic Installation, plus a Training Playground orchestrated by a workflow-management tool. The Training Playground runs an Ingestion-Training-Evaluation pipeline: it retrieves and cleans data, trains a multilayer perceptron with a deep-learning training library while a hyperparameter optimizer searches configurations, and evaluates predictions with regression or classification metrics. This pipeline is what gives AI4EF its claimed flexibility: the default models can be retrained on user-provided data, and the versioned model checkpoints are what the two dashboards call to return recommendations. An open-source identity-management layer and a data-space connector support the platform's claimed suitability for secure, multi-organization deployment.","core_discovery":"The paper claims that AI4EF closes a gap left by both open-source and proprietary building-energy tools. Its Building Retrofitting service takes inputs such as building total area, number of floors, current energy class, and energy consumption, and returns recommended construction works, expected primary-energy reduction, and CO2 savings; its Photovoltaic Installation service takes consumption, electricity price, and equipment costs and predicts generation, self-consumption, financial savings, and payback period. Unlike tools that require full building geometry and HVAC details, AI4EF is designed to produce initial insights from minimal data. Its Training Playground wraps data ingestion, hyperparameter optimization, training, and evaluation in an MLOps pipeline so users can retrain the underlying multilayer-perceptron models on their own datasets. The paper further claims the platform was developed using real data based on the needs of a Latvian environmental fund and was evaluated and used by real users satisfying their needs.","pith_inferences":["Editorial extension: if the released models were run against measured post-retrofit consumption data, the platform's minimal-data advantage could be quantified; the paper does not supply that benchmark.","Editorial extension: the data-space connector implies a path toward continuously retrained models fed by live meter streams, turning the Training Playground into a monitoring-and-updating loop rather than a one-off training tool; the paper presents this only implicitly as future work.","Editorial extension: the same input-parameter-to-recommendation pattern could be lifted to other investment decisions such as heat-pump sizing or district-heating connections, since the ML core is not tied to retrofit-specific features; the paper does not claim this."],"forward_implications":["Building owners and managers can obtain retrofit and photovoltaic recommendations from a small set of inputs rather than full building geometry and HVAC specifications.","Data scientists can retrain the underlying models on their own datasets through the Training Playground, making recommendations specific to a building stock or region.","Government representatives and energy consultants can use the platform to support evidence-based policy and advisory decisions on energy-efficiency investments.","Connection to a European energy data space allows the platform to draw on shared, large-scale datasets for training and analysis.","The open-source, modular architecture makes the platform adaptable to different regulatory and organizational contexts."],"supporting_citations":[{"why":"Supplies the released code version 0.0.2 that the open-source accessibility claim points to.","marker":"[1]"},{"why":"Open-source building-energy modeling baseline contrasted with AI4EF for lacking user-defined model training.","marker":"[40]"},{"why":"Proprietary retrofit tool that the paper contrasts with AI4EF's flexible, trainable models.","marker":"[41]"},{"why":"Proprietary photovoltaic design tool that the paper contrasts with AI4EF's minimal-data approach.","marker":"[42]"},{"why":"Latvian environmental fund cited as the source of real data and user needs behind the real-world validation claim.","marker":"[43]"},{"why":"European energy data space project whose connector grounds the platform's data-sharing integration.","marker":"[44]"}],"fun_headline_variants":["AI4EF: retrofit and solar predictions from minimal inputs","Predict building retrofit savings without full geometry","Machine learning guides retrofits from few building metrics","AI platform: ML forecasts for retrofit costs and solar payoff","Minimal-data AI for building energy retrofitting decisions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that AI4EF was actually built on real data and evaluated by real users whose needs it satisfied, because the paper's stated contribution over its competitors rests on that validation; the manuscript offers no user study, dataset, or metrics to back it up.","fun_headline_variants_meta":{"raw":{"variants":["AI4EF: retrofit and solar predictions from minimal inputs","Predict building retrofit savings without full geometry","Machine learning guides retrofits from few building metrics","AI platform: ML forecasts for retrofit costs and solar payoff","Minimal-data AI for building energy retrofitting decisions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000282,"raw_usage":{"total_tokens":1655,"prompt_tokens":922,"completion_tokens":733,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":658}},"tokens_in":538,"tokens_out":733,"duration_ms":8366,"temperature":1.0,"reasoning_tokens":658,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:47:47.873896+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the released version 0.0.2 and rerun the Training Playground pipeline on a held-out set of building records with known post-retrofit outcomes; if predicted energy savings, payback periods, or CO2 reductions are not compared against measured results or a simple baseline and show large errors, the platform's central promise of reliable, user-satisfying recommendations is false.","supporting_citations":[{"cited_title":"epu-ntua/enershare-ai4ef: 0.0.2, November 2024","cited_arxiv_id":null,"evidence_quote":"Supplies the released code version 0.0.2 that the open-source accessibility claim points to."},{"cited_title":"Openstudio, 2023","cited_arxiv_id":null,"evidence_quote":"Open-source building-energy modeling baseline contrasted with AI4EF for lacking user-defined model training."},{"cited_title":"Retrofit advisor, 2023","cited_arxiv_id":null,"evidence_quote":"Proprietary retrofit tool that the paper contrasts with AI4EF's flexible, trainable models."},{"cited_title":"Folsom Labs","cited_arxiv_id":null,"evidence_quote":"Proprietary photovoltaic design tool that the paper contrasts with AI4EF's minimal-data approach."},{"cited_title":"Latvian environmental invest- ment fund (leif), 2024","cited_arxiv_id":null,"evidence_quote":"Latvian environmental fund cited as the source of real data and user needs behind the real-world validation claim."},{"cited_title":"Enershare — the energy data space for europe","cited_arxiv_id":null,"evidence_quote":"European energy data space project whose connector grounds the platform's data-sharing integration."}],"review_version":1}