{"id":"a1a8b6fd-93bc-4edf-a3b8-bf02de9cbd00","arxiv_id":"2504.13624","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"PV-VLM fuses visual, textual, and temporal features via a vision-language model and cross-modal attention to improve intra-hour photovoltaic power forecasts by roughly 5 to 9 percent in RMSE and MAE.","lead":"This paper introduces PV-VLM, a forecasting model that combines sky images, text descriptions, and past power readings to predict solar panel output up to an hour ahead. It reports 5 to 9 percent lower errors than four deep learning baselines on two university datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Random 70/10/20 split on autocorrelated 2-min PV data lets training samples surround each test point; the Table 3 gains may measure leakage, not forecasting skill. A chronological split is needed.","rationale":"I weighed the reader's baseline-fairness concern against the evaluation-design issue. Baseline under-tuning is real and explicitly conceded in Section 3.2, but it is secondary: even with perfectly tuned baselines, a random split on 2-min PV data tests interpolation, not intra-hour forecasting. The transfer study provides some out-of-sample evidence, yet the headline within-dataset percentages are computed under the suspect split, and model selection and ablation also depend on it. I therefore make temporal leakage the primary concern. The paper does not explicitly say the split is random; the proposed test should settle this. If the split is already chronological, this concern evaporates, and the baseline-tuning and missing-error-bar issues still warrant the reader's CONDITIONAL verdict. Hence I recommend no change to the verdict: the paper should be accepted only after a temporal-split evaluation, baseline tuning checks, and release of code and split indices. This is consistent with the reader's overall assessment, though our emphasis differs.","tokens_in":13755,"tokens_out":7358,"duration_ms":74798,"concrete_test":"Re-run Table 3 and Table 4 with a strict chronological split: train on the first 70% of Dataset A's contiguous 2-min record, validate on the next 10%, and test on the last 20%; additionally use a day-disjoint split where complete days are assigned to training, validation, or testing. Report RMSE, MAE, and R² for every method across 3 seeds with standard deviations. If the proposed model's average RMSE/MAE advantage over the best baseline falls below about 1-2% or reverses, the claimed 5%/6% improvement is an artifact of random-split leakage. This test requires only the existing preprocessed data, the split indices, and the training code.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 describes a 70/10/20 allocation without stating that the split is chronological; the default reading in such pipelines is random splitting. Dataset A is resampled 2-minute PV power with strong autocorrelation and cloud-event persistence over 20-60 min forecast horizons. Under a random split, for every test timestamp there are training samples minutes to hours earlier and later from the same weather episode, so the model can interpolate or memorize rather than predict the future. This changes what Table 3 measures. The proposed model is the highest-capacity contender (frozen SigLIP-2 plus GPT-2 backbone, PatchTST-style encoder, cross-modal attention), so it is the most likely to exploit such leakage, and the reported 3.7-5.9% RMSE and 0.7-13.5% MAE advantages over re-implemented baselines could partly reflect this capacity advantage. The same split underlies the ablation study in Table 4, so module attributions are also suspect. A zero-shot transfer to Dataset B is a useful out-of-sample check, but it is small (~3000 samples) and the within-dataset evidence remains the primary support for the claimed 5%/6% improvements. The paper should either confirm that the split is temporal or provide split indices.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces PV-VLM, a multimodal framework for intra-hour photovoltaic power forecasting that fuses time-series patches via a PatchTST-inspired transformer, textual prompts generated from historical statistics and dataset descriptors via an LLM, and sky-image features extracted by a pretrained vision-language model. The proposed architecture is evaluated on a Stanford dataset (Dataset A) and via a cross-dataset transfer study on two PV plants at the University of Wollongong (Dataset B). The authors report average RMSE reductions of approximately 5% on Dataset A and about 7% in the transfer study, with MAE improvements of nearly 6% and 9.5%, respectively, compared with four established baselines. The paper also includes a systematic ablation study examining the contribution of each modality and a visual heatmap illustration of the extracted sky-image features.","tokens_in":14028,"tokens_out":8052,"duration_ms":61670,"significance":"If the reported results are robust, PV-VLM makes a useful contribution to solar forecasting by demonstrating that a frozen vision-language model and an LLM-based prompt encoder can be combined with a time-series transformer for intra-hour predictions, and the cross-continental zero-shot transfer is a practically interesting result. The ablation study is systematic, and the heatmap visualization of SigLIP-2 feature activations is a nice qualitative check. However, the empirical claims are currently weakened by the ambiguous data split, the unequal hyperparameter tuning between the proposed model and baselines, and the absence of any uncertainty quantification or significance tests. These issues must be addressed before the reported relative improvements can be taken at face value.","major_comments":[{"comment":"The paper states that a 70/10/20 allocation was used for training, validation, and testing, but it does not specify whether this split is chronological. Dataset A is a 2-minute resampled PV power time series with strong autocorrelation and cloud-event persistence over 20-60 minute horizons. Under a random split, test timestamps would have training samples from the same weather episodes both before and after them, allowing the model to interpolate or memorize rather than forecast the future. The authors should either confirm that the split is temporal (e.g., train on the first 70% of the time series, validate on the next 10%, and test on the final 20%) or provide the exact split indices so that temporal independence can be verified. If the split is random, the results in Tables 3 and 4, as well as the module attributions from the ablation study, may be substantially inflated.","section":"Section 3.2, data split"},{"comment":"The baselines were re-implemented 'strictly according to the methodological descriptions in their original publications, without additional dataset-specific adaptations or extensive hyperparameter tuning,' while the proposed PV-VLM's hyperparameters (Table 2) were evidently selected for these datasets. This asymmetry makes the comparison unfair: the reported 3.7-5.9% RMSE and 0.7-13.5% MAE improvements over the best baseline per horizon could partly reflect under-tuned baselines. The authors should perform a hyperparameter search for each baseline using the same validation set and computational budget, and report the best configuration for each method, or at least provide a sensitivity analysis showing that the conclusions are robust to baseline hyperparameters.","section":"Section 3.2, baseline tuning"},{"comment":"The paper reports that 'three experiments' were conducted and outcomes averaged, but no standard deviations or significance tests are provided anywhere. Given that some differences are very small (e.g., 0.71% MAE improvement over the best baseline at the 20-minute horizon in Table 3), the reported gains may not be statistically meaningful. The authors should report the mean and standard deviation across runs for each model and horizon, and perform a paired significance test (such as the Diebold-Mariano test or a paired t-test) to establish that the proposed method is significantly better than each baseline.","section":"Section 3.2 and Tables 3-6, statistical significance"},{"comment":"There is an inconsistency in the reported data sizes: the text states that Dataset A comprises approximately 349,000 samples after preprocessing, while Table 1 reports 62,910 samples; similarly, Dataset B is described as around 6,000 samples in the text but 3,000 in Table 1. Please clarify which numbers are correct and describe the preprocessing and quality-filtering steps, because the sample size directly affects the reliability of the comparisons and the transfer study results.","section":"Section 3.1, data description"},{"comment":"In the transfer study, the model is trained on Dataset A and tested on Dataset B without fine-tuning. The Prompt-Aware Module generates prompts from 'historical statistics and dataset descriptors.' If the dataset descriptor for the target dataset B (e.g., camera specifications or location) is included when constructing prompts at test time, the zero-shot evaluation is not purely agnostic to the target distribution. Please clarify whether the prompts are computed solely from the input time-series statistics (which is legitimate) or also from metadata of Dataset B, and if the latter, discuss the implications for the claimed zero-shot transfer.","section":"Section 3.5, transfer study setup"}],"minor_comments":[{"comment":"The equation for bilinear interpolation is garbled in the manuscript (the summation notation and the description of coordinates are corrupted); please rewrite it in standard notation and define all variables.","section":"Eq. (2-1)"},{"comment":"The statement that 'only the 2019 data were retained in Dataset A, matching Dataset B’s temporal coverage' is unclear because Dataset B's time range is not given; please specify the exact temporal windows of both datasets.","section":"Section 3.1"},{"comment":"The caption refers to '< > and < >' which appear to be missing placeholders; please complete the caption.","section":"Fig. 4"},{"comment":"The units RMSE(10^4 kW) and RMSE(10^3 kW) are unusual and make cross-comparison difficult; please report all errors in kW consistently.","section":"Tables 5 and 6"},{"comment":"In Table 4, the variant without the soft prompt gives a lower MAE than the full model at the 20-minute horizon (0.688 vs 0.698) and at the 40-minute horizon (0.845 vs 0.857), yet the text states that the soft prompt 'provides some minor improvements'; please reconcile this observation.","section":"Section 3.4"},{"comment":"Reference [36] is cited for Early Stopping, but that reference is a paper on sky-image-based forecasting, not a standard source for early stopping; please cite an appropriate reference or remove the citation.","section":"Reference [36]"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely topic and proposes a novel architecture. The main technical concerns are the unclear temporal split, the asymmetric hyperparameter tuning, and the lack of statistical significance testing. I would suggest the editor ask the authors to provide the exact split protocol (ideally chronological) and the code for reproducibility. If the split turns out to be random, the core comparison could be invalid; if it is temporal, the paper may be salvageable after the other issues are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's actual contribution is a new fusion architecture: frozen SigLIP-2 for sky images, GPT-2 for textual prompts, and a PatchTST-style temporal encoder, combined with cross-modal attention. That specific combination for intra-hour PV forecasting does not appear in the literature, and the transfer study to a different continent is a good idea. The ablation results are internally consistent, with the temporal module clearly carrying most of the weight, which is plausible.\n\nThe problem is the evaluation. Section 3.2 describes a 70/10/20 split without saying it is chronological. On 2-minute PV data with strong autocorrelation and cloud persistence over 20-60 minute horizons, a random split lets training samples surround each test point. That means the within-dataset results in Table 3, and the ablation in Table 4, may be measuring leakage rather than forecasting skill. The proposed model has the largest capacity, so it would benefit most. The zero-shot transfer to Dataset B is a genuine out-of-sample check and it does show consistent gains, which is the main reason I am not dismissing the paper entirely. But Dataset B is small, and the primary evidence still rests on the within-dataset comparison.\n\nThe baseline comparison is also unfair as stated. The authors re-implemented baselines 'strictly according to the methodological descriptions' without dataset-specific tuning, while the proposed model's hyperparameters were selected for these datasets. That alone could explain a few percentage points of RMSE improvement. On top of that, results are reported as averages of three runs with no standard deviations or significance tests. A 0.7% MAE difference at the 20-minute horizon is well within noise.\n\nThere are smaller issues too. The text says Dataset A has ~349,000 samples but Table 1 lists 62,910 after processing; that needs clarification. No code or data are provided, and the split indices are not given, so the main experimental result is not reproducible as reported.\n\nMy bottom line: the architecture is interesting and the transfer study shows enough promise to merit a proper look, but the core claim of 5-9% error reduction is not established until the authors either confirm a chronological split or provide split indices, tune the baselines fairly, and add error bars. This deserves peer review, but it needs major revision before publication.","headline":"A plausible multimodal PV-forecasting architecture with modest reported gains, but the main evidence is compromised by an apparently random split and under-tuned baselines, so the actual improvement over a fair comparison is unproven.","tokens_in":14554,"tokens_out":1849,"would_cite":false,"duration_ms":18190,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multimodal model that fuses sky images, text prompts, and power history reports about 5–7% lower error in intra-hour solar forecasting.","keywords":["photovoltaic power forecasting","vision-language model","multimodal fusion","sky images","intra-hour forecasting","transfer learning","time series forecasting","large language model"],"falsifier":"Retrain all four baselines with the same hyperparameter search budget, early stopping, learning-rate schedule, and input lengths as PV-VLM on both datasets, and compare mean RMSE and MAE over the same three-run protocol; the claimed superiority fails if the tuned baselines become statistically indistinguishable or better. A sharper test is to replace the sky images with scrambled versions that preserve color statistics, or to remove the Vision-Aware Module while holding everything else fixed—if forecast error does not rise, the visual modality is not carrying the claimed information.","tokens_in":13541,"feed_emoji":"☀️","tokens_out":6094,"duration_ms":54676,"temperature":0.7,"pith_summary":"This paper claims that intra-hour photovoltaic power forecasts can be made more accurate by combining three modalities: historical power readings, descriptive text prompts about the dataset and recent statistics, and sky images processed by a pretrained vision-language model. Existing methods typically process sky images and power series separately or with shallow feature concatenation, so they miss the deep interactions among cloud motion, system state, and domain knowledge. PV-VLM instead aligns all three modalities in a shared embedding space and lets the temporal representation query the vision-language representation through cross-modal attention. If the claim holds, grid operators and energy traders would gain a practical way to reduce 20-to-60-minute forecast errors, and the reported transfer results suggest the features keep working across climates without retraining. The headline numbers are roughly 5% RMSE and 6% MAE improvement on the primary dataset, and about 7% RMSE and 9.5% MAE improvement in a zero-shot transfer study.","feed_headline":"Sky images plus text prompts cut solar forecast error ~5%","feed_subtitle":"A three-way multimodal model beats deep-learning baselines and keeps the edge when moved to a new climate.","key_machinery":"The load-bearing object is PV-VLM's three-branch encoder plus a cross-modal attention fusion. A PatchTST-style transformer divides the PV power history into overlapping patches and encodes both local fluctuations and long-range dependencies; a frozen pretrained vision-language encoder converts each sky image into semantic embeddings that highlight cloud edges, dense formations, and irradiance gradients; a language-model branch encodes a text prompt assembled from dataset descriptions and power statistics, together with a learnable soft prompt. The temporal embedding acts as the query in multi-head cross-modal attention, while the concatenated vision-language embedding acts as key and value, and a residual connection with layer normalization stabilizes the fused representation before a linear output head produces the forecast. This mechanism lets the time series selectively retrieve relevant visual and textual evidence for each forecast step rather than merely concatenating features.","core_discovery":"PV-VLM claims to be the first systematic integration of a frozen pretrained vision-language encoder, a language-model prompt branch, and a PatchTST-style temporal encoder for intra-hour PV forecasting. On a 30-kW rooftop array dataset, it reports average RMSE reductions of about 5% and MAE improvements of nearly 6% over four deep-learning baselines across 20-, 40-, and 60-minute horizons. In a zero-shot transfer study to a second dataset in a different climate, it reports about 7% RMSE and 9.5% MAE reductions without any fine-tuning on the target site. The ablation study attributes the core of the gain to the temporal module, with text prompts and sky-image features adding complementary improvements, and a learnable soft prompt contributing smaller gains.","pith_inferences":["Beyond the paper: the vision module's contribution is not isolated with a scrambled-image or random-image control, so it remains an open question how much of the gain comes from genuine cloud semantics rather than generic image statistics.","Beyond the paper: because the baselines were not hyperparameter-tuned for these datasets while the proposed model was, a fair equal-budget comparison might show a smaller real-world gap than the reported 5–9.5%.","Beyond the paper: the same three-branch architecture could be extended to wind, solar irradiance, or building-load nowcasting, where camera imagery and text metadata can be added as prompts.","Beyond the paper: the prompt branch offers a cheap route to inject exogenous information, such as weather alerts or market signals, into an already-trained model by simply extending the textual prompt."],"forward_implications":["If the reported numbers hold, intra-hour solar forecasting at 20-to-60-minute horizons can be improved by roughly 5% RMSE and 6% MAE over current deep-learning baselines without any change to the physical PV plant.","Because the transfer study reports 7% RMSE and 9.5% MAE gains with no fine-tuning on the target site, the vision-language and prompt features appear to generalize across climates, PV sizes, and camera hardware.","The ablation shows the time-series history is the most load-bearing modality, with text prompts and sky images as complementary enhancements, so a partial deployment is possible when sky cameras are unavailable.","The learnable soft prompt preserves most of the full-model benefit, suggesting that prompt design contributes only a minor share of the overall improvement.","The framework introduces a concrete way to inject domain knowledge and dataset context into a forecasting model without changing its parameters: extend the text prompt with new statistics or descriptions."],"supporting_citations":[{"why":"Supplies the PatchTST-style patched Transformer design used by the temporal module.","marker":"[18]"},{"why":"Provides the pretrained vision-language encoder used to extract sky-image semantic features.","marker":"[21]"},{"why":"Contributes the prompt-as-prefix mechanism for encoding statistical text prompts.","marker":"[23]"},{"why":"Provides the language model used to encode textual prompts and produce the multimodal backbone representation.","marker":"[25]"},{"why":"Supplies the primary sky-image and PV power dataset from a 30-kW rooftop array used for training and evaluation.","marker":"[29]"},{"why":"Supplies the second sky-image and PV power dataset used for the zero-shot transfer study.","marker":"[30]"},{"why":"Serves as a baseline that fuses multi-temporal images and radiation data with convolutional LSTM layers.","marker":"[32]"},{"why":"Serves as a baseline using bi-level spatiotemporal analysis on sky images for PV nowcasting.","marker":"[33]"},{"why":"Serves as a baseline combining convolution and LSTM to maintain spatial integrity and sequential dynamics.","marker":"[34]"},{"why":"Serves as a baseline tailored to short-term solar electricity trend prediction with customized preprocessing and network components.","marker":"[35]"}],"fun_headline_variants":["Sky-aware AI cuts solar forecast error ~5% and holds in new climates","Multimodal model with sky images cuts PV forecast RMSE ~5%, MAE ~6%","PV-VLM fuses sky images and text to cut solar forecast error ~5%","Time, text, and sky: multimodal model trims solar forecast error ~5%","Cloud-savvy multimodal AI improves solar forecasts 5-9% across climates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison treats the re-implemented baselines as fair competition even though their hyperparameters were not tuned for these datasets, while PV-VLM's hyperparameters were selected for these datasets; if the baselines were under-tuned or misconfigured, the reported error reductions would not show a genuine advantage.","fun_headline_variants_meta":{"raw":{"variants":["Sky-aware AI cuts solar forecast error ~5% and holds in new climates","Multimodal model with sky images cuts PV forecast RMSE ~5%, MAE ~6%","PV-VLM fuses sky images and text to cut solar forecast error ~5%","Time, text, and sky: multimodal model trims solar forecast error ~5%","Cloud-savvy multimodal AI improves solar forecasts 5-9% across climates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001563,"raw_usage":{"total_tokens":6255,"prompt_tokens":967,"completion_tokens":5288,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":5177}},"tokens_in":583,"tokens_out":5288,"duration_ms":36545,"temperature":1.0,"reasoning_tokens":5177,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:02:35.165161+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain all four baselines with the same hyperparameter search budget, early stopping, learning-rate schedule, and input lengths as PV-VLM on both datasets, and compare mean RMSE and MAE over the same three-run protocol; the claimed superiority fails if the tuned baselines become statistically indistinguishable or better. A sharper test is to replace the sky images with scrambled versions that preserve color statistics, or to remove the Vision-Aware Module while holding everything else fixed—if forecast error does not rise, the visual modality is not carrying the claimed information.","supporting_citations":[{"cited_title":"Time-LLM: Time Series Forecasting by Reprogramming Large Language Models","cited_arxiv_id":null,"evidence_quote":"Contributes the prompt-as-prefix mechanism for encoding statistical text prompts."},{"cited_title":"Language Models are Unsupervised Multitask Learners","cited_arxiv_id":null,"evidence_quote":"Provides the language model used to encode textual prompts and produce the multimodal backbone representation."},{"cited_title":"SKIPP’D: A SKy Images and Photovoltaic Power Generation Dataset for short -term solar forecasting","cited_arxiv_id":null,"evidence_quote":"Supplies the primary sky-image and PV power dataset from a 30-kW rooftop array used for training and evaluation."},{"cited_title":"Sky Images and PV Power Measurements for Irradiance Forecasting 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the second sky-image and PV power dataset used for the zero-shot transfer study."},{"cited_title":"Short-term solar power forecast with deep learning: Exploring optimal input and output configuration","cited_arxiv_id":null,"evidence_quote":"Serves as a baseline tailored to short-term solar electricity trend prediction with customized preprocessing and network components."}],"review_version":1}