{"id":"88ddd19b-69a2-4b7c-bbd0-4e8388c1e3ec","arxiv_id":"2412.16842","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A low-cost rain-gauge IoT system plus a graph neural network is proposed for heavy rainfall prediction in Bolivia, but the evaluation metrics are internally inconsistent and no baseline is reported.","lead":"A researcher built low-cost rain gauges that send SMS data and a graph neural network that tries to forecast heavy rainfall from 41 Bolivian weather stations. The paper claims useful predictions for data-poor regions, but the reported evaluation contains impossible statistics (correlation over 1) and no baseline, so the claim is not supported.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table I reports Pearson correlation coefficients greater than 1 (r = 5.891, 6.002, 7.204, 8.771), which is mathematically impossible for the coefficient defined in Eq. 4; the effectiveness claim therefore rests on invalid evaluation metrics.","rationale":"The reader's verdict was REJECT, and I agree: the impossible Pearson correlation values in Table I are decisive. The reader's stated weakest_assumption concerned the inverse-distance graph construction, but the rationale correctly identified the r > 1 issue; I treat that issue as the single load-bearing concern because it invalidates the evaluation independent of any external assumptions about rainfall spatial dependence. Even if the graph were perfectly suited to Bolivian orography, an internally impossible metric means the reported experimental support is broken. In good faith, the authors could still salvage the claim by releasing code and predictions and showing corrected metrics within [-1,1]; as written, the paper's own numbers undermine its central conclusion. I therefore recommend no change to the reader's REJECT verdict.","tokens_in":7509,"tokens_out":2704,"duration_ms":25186,"concrete_test":"Recompute Pearson r on the exact test-set predictions and ground-truth values used for Table I, using the standard formula r = sum((p_i - mean_p)(y_i - mean_y)) / sqrt(sum((p_i - mean_p)^2) * sum((y_i - mean_y)^2)). If any value falls outside [-1,1] — or if the model's outputs were standardized/transformed before computing r — the reported r column is not a Pearson correlation and Table I cannot support the effectiveness claim. A minimal unit test with a known perfect/anti-correlated pair would also confirm the implementation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the GNN 'effectively' predicts heavy rainfall, supported by Table I. For that claim to hold, the reported test metrics must be valid. Eq. 4 defines r as a Pearson correlation coefficient, which is bounded in [-1,1] for any two real vectors. All four rows of Table I violate this bound (r = 5.891, 6.002, 7.204, 8.771). This is not an assumption about Bolivian orography or graph construction; it is an internal inconsistency in the evidence itself. The pattern is systematic — every model has an impossible r, and r increases as MSE worsens — which suggests the r column was computed by a different formula or on different quantities than stated (e.g., not on per-station predictions and targets, or after some transformation). Because the central empirical claim is supported only by these metrics, and one of the headline metrics is impossible, the evaluation is not trustworthy as reported. The absence of baselines and ablations further weakens the claim, but the impossible correlation alone invalidates the demonstrated effectiveness.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a low-cost IoT rainfall monitoring system with 3D-printed tipping-bucket rain gauges, GSM/SMS data transmission, and a MEAN-stack web application, together with a graph neural network (GNN) for regional heavy rainfall prediction. The GNN models 41 Bolivian weather stations as nodes with inverse-distance-weighted edges and uses four graph convolution layers followed by a regression head to predict daily rainfall; a prediction above 8 mm is flagged as a heavy rainfall event. The authors report results on a daily precipitation dataset and claim that the proposed method is effective based on MSE, MAE, and Pearson correlation on a held-out test set.","tokens_in":7736,"tokens_out":6107,"duration_ms":53573,"significance":"The hardware contribution is concrete and potentially useful: the paper describes a complete low-cost sensing and communication pipeline for rural environmental monitoring, and the evaluation uses real multi-year data from 41 stations rather than a synthetic benchmark. If the prediction claims were valid, the approach could be attractive for flood early warning in data-sparse regions. However, the central empirical claim is not supported as reported: the headline correlation coefficients are mathematically impossible, the test error exceeds the event threshold, no baselines or ablations are provided, and the data split is not shown to be chronological. These issues place the paper's main contribution, namely the demonstrated effectiveness of GNN-based heavy-rainfall prediction, in question.","major_comments":[{"comment":"The Pearson correlation coefficients reported in Table I (r = 5.891, 6.002, 7.204, 8.771) are mathematically impossible for the coefficient defined in Eq. (4), which is bounded in [-1, 1] for any two real vectors. Since r is one of the three reported performance metrics and the abstract's effectiveness claim rests on these results, the evaluation as reported is invalid; the r column appears to have been computed by a different formula or on different quantities than stated. This is a load-bearing internal inconsistency, not a presentation issue.","section":"Section V, Table I and Eq. (4)"},{"comment":"The best model's MAE is 9.512 mm, which is larger than the 8 mm heavy-rain threshold defined in Section IV-B. With an average absolute error exceeding the event threshold, the proposed flagging rule (predict rainfall above 8 mm) cannot be expected to separate heavy-rain days from other days. The paper reports no event-level metrics such as precision, recall, F1, or a confusion matrix, so the central claim about predicting heavy rainfall events is not demonstrated.","section":"Section V, Table I and Section IV-B"},{"comment":"No baseline or ablation is reported. The manuscript does not compare against persistence, climatology, linear regression, an LSTM, or a non-graph machine learning model, nor does it ablate the graph construction (e.g., inverse-distance weighting, number of neighbors, number of layers). Without such comparisons, statements that the GNN 'outperformed' other models and 'effectively captured spatial dependencies' are unsupported.","section":"Section V"},{"comment":"The train/validation/test split is described only as 70/20/10 percentages. For daily precipitation time series with strong temporal autocorrelation, a random split can leak future information into training and inflate performance. The authors need to specify whether the split is chronological and report evaluation on temporally disjoint periods; as written, the generalization claim is not assured.","section":"Section V, data split"},{"comment":"The model input is underspecified: the node feature vector is described only as 'meteorological data from that station (mainly past rainfall)' with no statement of the number of lag days, the feature dimension, the standardization procedure, or the missing-value interpolation method. This makes the experiment unreproducible and prevents assessment of whether the model is predicting from past rainfall or from contemporaneous information.","section":"Section IV-B and IV-A"}],"minor_comments":[{"comment":"The notation cov(X, y) is dimensionally inconsistent; Pearson correlation should be defined on paired scalar observations, and it would be clearer to state explicitly whether r is pooled across stations and days or computed per station.","section":"Eq. (4)"},{"comment":"There are small typographical issues: 'W AN' should be 'WAN', and 'ReLu' should be 'ReLU'.","section":"Section III-B and IV-B"},{"comment":"Figure 7 is referenced only as 'Best performing GNN model' without a descriptive caption; it is unclear what the plot shows and how it supports the reported results.","section":"Figure 7"},{"comment":"The 8 mm/day heavy-rainfall threshold is introduced without justification or citation; a reference or a brief rationale from Bolivian meteorological practice would help.","section":"Section IV-B"},{"comment":"The dataset duration is inconsistent: the abstract and Section V say 72 months, but the data collection period stated in Section IV-A (1 January 2017 to 30 April 2024) is 88 months.","section":"Section IV-A, abstract, and Section V"},{"comment":"Several references are incomplete, including [1], [2], [9], [10], and [24], which lack full author, title, or access details; the bibliography needs to be brought to journal style.","section":"References"}],"recommendation":"reject","confidential_remarks":"The hardware and deployment story is a useful engineering contribution, but the machine learning evaluation would need to be redone from scratch, including valid metrics, a proper chronological split, baselines, ablations, and event-level scores. The impossible correlation coefficients alone invalidate the reported effectiveness claim, and the MAE exceeding the event threshold further undermines the operational argument. I do not see a path to acceptance without substantial new experiments; if the authors can supply those, a fresh submission would be more appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Edwin Salcedo's paper pairs a low-cost tipping-bucket rain gauge network with a graph convolutional network for daily rainfall prediction across 41 Bolivian stations. The hardware section is the most honest part: the device design is described in detail, costs are stated, and the GSM/SMS data path is sensible for rural areas. That part is credible engineering. The GNN application is also straightforwardly presented, and the paper cites the relevant GNN rainfall literature (refs 20-23).\n\nThe problem is the evaluation table. Table I reports Pearson r values of 5.891, 6.002, 7.204, and 8.771. The coefficient defined in Eq. 4 is bounded between -1 and 1 for any real vectors. These numbers are impossible. The pattern – r getting larger as MSE gets worse – suggests the r column was computed by something other than the stated formula, maybe on different quantities or after some transformation. Either way, the central claim that the method 'effectively' predicts heavy rainfall rests on this table, and the table is internally inconsistent with the paper's own math.\n\nThere are additional problems, but they're secondary. No baseline comparison, no error bars, no guarantee that the 70/20/10 split is chronological (for daily rainfall, random splitting leaks information between training and test days). The model is a static four-layer GCN with no temporal recurrence and no exogenous variables, which is a weak inductive bias for an 8 mm/day event detection problem. And the MAE of 9.5 mm exceeds the 8 mm heavy-rain threshold, so even under the reported metrics, the model would struggle to flag the events it's designed to warn about.\n\nWhat's genuinely useful here is the hardware and the dataset description: a low-cost, solar-powered gauge with SMS backhaul, plus the scraped SENAMHI data from 41 stations over 72 months. That could be a contribution, but it's not evaluated as a system (the experiments use scraped data, not the proposed gauges).\n\nI would not send this to peer review as is. The impossible r values are a fatal, load-bearing flaw that no referee report can repair; the author needs to redo the evaluation. If a corrected version appears with baselines, a chronological split, and honest metrics, it's worth a second look. For now, it's a desk reject.","headline":"The paper's central effectiveness claim collapses on its own Table I: Pearson r values above 1 are impossible, so the evaluation pipeline is broken, despite a plausible low-cost IoT hardware story.","tokens_in":8250,"tokens_out":1785,"would_cite":false,"duration_ms":15659,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","86A10"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that a graph neural network over 41 distance-linked rain-gauge stations can predict daily rainfall and flag heavy events, making low-cost flood warning feasible where radar is scarce.","keywords":["heavy rainfall prediction","graph neural networks","IoT rain gauges","tipping bucket rain gauge","low-cost weather stations","spatial rainfall dependencies","flood early warning","Bolivia"],"falsifier":"Recompute the Pearson correlation between the held-out predictions and observed values: the reported r of 5.891 is mathematically impossible because r must lie within [-1, 1], so a corrected calculation would immediately show whether the headline accuracy claim survives. Separately, train the identical four-layer GNN with the adjacency matrix replaced by the identity matrix (each station isolated) and by an elevation-difference-weighted graph; if the inverse-distance graph does not beat both on days with 8 mm or more, then the spatial-dependence mechanism the paper relies on is not the source of its results.","tokens_in":7286,"feed_emoji":"🌧️","tokens_out":5729,"duration_ms":49700,"temperature":0.7,"pith_summary":"This paper proposes a complete low-cost rainfall monitoring and prediction chain: 3D-printed tipping-bucket rain gauges with GSM/SMS reporting, a central web platform, and a graph neural network (GNN) that treats weather stations as nodes connected by inverse distance. The GNN is trained on 72 months of daily precipitation from 41 Bolivian stations to predict rainfall amounts at each station and flag events of 8 mm/day or more. The authors report that the best four-layer graph convolution model reaches an MSE of 16.344 and an MAE of 9.512 on held-out data, and they argue this makes GNN-based prediction feasible where radar or dense station coverage is unavailable. If reliable, the system would give rural flood-prone regions a low-cost way to issue early warnings and plan water resources.","feed_headline":"Graph network predicts heavy rain from cheap Bolivian gauges","feed_subtitle":"A four-layer GNN on distance-linked stations flags daily rainfall above 8 mm, offering flood alerts where radar is sparse.","key_machinery":"The load-bearing object is the station graph: nodes are weather stations, node features are each station's meteorological history (mainly past rainfall), and edge weights are the inverse geographic distances between stations. On top of this graph the model stacks four graph convolution layers (GCLs) of the form $H^{l+1} = \\sigma(D^{-1/2} A D^{-1/2} H^l W^l)$, followed by a fully connected regression layer, trained with Adam at learning rate 0.01 and early stopping; the 8 mm/day threshold converts the regression output into a heavy-rainfall flag.","core_discovery":"The paper's central claim is that a static graph neural network whose adjacency matrix is the inverse distance between 41 rain-gauge stations can learn enough spatial structure from daily rainfall histories to forecast heavy rainfall regionally. The model applies four graph convolution layers with ReLU activations to the normalized adjacency matrix, then a fully connected regression layer that outputs a rainfall amount per station; any prediction above 8 mm counts as a heavy rainfall event. On a 70/20/10 temporal split of 72 months of daily data from the national hydrology service, the authors report model A as best, with MSE 16.344, MAE 9.512, and a Pearson r of 5.891 on the testing set. In the paper's telling, this demonstrates that low-cost gauge networks plus graph learning can substitute for expensive infrastructure in data-sparse regions.","pith_inferences":["The inverse-distance graph is a strong prior: it assumes geographic closeness controls rainfall dependence. In mountainous Bolivia, elevation and slope are likely at least as influential, so adding altitude as a node feature or edge weight is a natural extension the paper does not test.","The reported Pearson coefficient r = 5.891 lies outside the possible range [-1, 1], which suggests either a misreported formula or a typographical error; correcting this and re-evaluating the model is the first check any adopter should make.","No comparison is made to a station's own historical climatology or to a non-spatial baseline, so the paper's claim that the graph structure itself helps remains unquantified; a persistence baseline would isolate the GNN's contribution.","If heavy-rain events are rare, MSE and MAE can be dominated by many dry days; reporting event-level precision/recall or hits for the 8 mm threshold would make the practical alerting value much clearer."],"forward_implications":["If the reported accuracy holds, 41 low-cost stations connected by GSM could support daily heavy-rainfall alerts across Bolivia without radar.","SMS-based data transfer means the same prediction pipeline can run in other regions where 3G/4G coverage and internet are unavailable.","Because the GNN outputs a rainfall amount per node, the method can flag stations expected to exceed 8 mm/day, giving authorities a concrete trigger for warnings.","The end-device design (tipping bucket plus solar power) keeps per-unit cost around $280–$320, making dense deployment financially conceivable."],"supporting_citations":[{"why":"The national hydrology service's public archive, which supplies the 72-month daily rainfall record across 41 stations that the whole experiment depends on.","marker":"[2]"},{"why":"Provides the tipping-bucket calibration method that the low-cost end devices are built on.","marker":"[8]"},{"why":"Gives the taxonomy of low-cost rain gauge types and their capacity limitations, motivating the tipping-bucket choice.","marker":"[6]"},{"why":"Shows GNNs capturing spatial dependencies from weather station data for regional heatwave prediction, the direct precedent for using GNNs here.","marker":"[20]"},{"why":"Introduces a structured GNN that improves rainfall prediction, supporting the paper's graph-based approach.","marker":"[21]"},{"why":"Demonstrates coupling physical variables through graph structures for precipitation forecasting, another precedent for the GNN methodology.","marker":"[22]"}],"fun_headline_variants":["GNN on 41 cheap gauges predicts Bolivian heavy rain","Distance-linked gauge GNN flags heavy rain over 8mm","Graph neural net + cheap gauges for heavy rain alerts","Low-cost gauge network + graph learning predicts heavy rain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method depends on the idea that rainfall at one station is best explained by rainfall at geographically nearby stations, so a graph whose links are just geographic distances carries the information needed to predict heavy rain; if in Bolivia's mountains rainfall is controlled more by elevation and local terrain than by distance, the graph gives the model the wrong inductive bias and the reported prediction ability would not transfer.","fun_headline_variants_meta":{"raw":{"variants":["GNN on 41 cheap gauges predicts Bolivian heavy rain","Distance-linked gauge GNN flags heavy rain over 8mm","Graph neural net + cheap gauges for heavy rain alerts","Low-cost gauge network + graph learning predicts heavy rain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000596,"raw_usage":{"total_tokens":2767,"prompt_tokens":904,"completion_tokens":1863,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":1794}},"tokens_in":520,"tokens_out":1863,"duration_ms":11659,"temperature":1.0,"reasoning_tokens":1794,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:15:15.672676+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the Pearson correlation between the held-out predictions and observed values: the reported r of 5.891 is mathematically impossible because r must lie within [-1, 1], so a corrected calculation would immediately show whether the headline accuracy claim survives. Separately, train the identical four-layer GNN with the adjacency matrix replaced by the identity matrix (each station isolated) and by an elevation-difference-weighted graph; if the inverse-distance graph does not beat both on days with 8 mm or more, then the spatial-dependence mechanism the paper relies on is not the source of its results.","supporting_citations":[{"cited_title":"Informaci ´on nacional de datos hidrometeorol ´ogicos","cited_arxiv_id":null,"evidence_quote":"The national hydrology service's public archive, which supplies the 72-month daily rainfall record across 41 stations that the whole experiment depends on."},{"cited_title":"Calibrating low-cost rain gauge sensors for their applications in iot infrastructures to densify environ- mental monitoring networks,","cited_arxiv_id":null,"evidence_quote":"Provides the tipping-bucket calibration method that the low-cost end devices are built on."},{"cited_title":"Global precipitation measurement: Methods, datasets and applications,","cited_arxiv_id":null,"evidence_quote":"Gives the taxonomy of low-cost rain gauge types and their capacity limitations, motivating the tipping-bucket choice."},{"cited_title":"Regional heatwave prediction using graph neural network and weather station data,","cited_arxiv_id":null,"evidence_quote":"Shows GNNs capturing spatial dependencies from weather station data for regional heatwave prediction, the direct precedent for using GNNs here."},{"cited_title":"A structured graph neural network for improving the numerical weather prediction of rainfall,","cited_arxiv_id":null,"evidence_quote":"Introduces a structured GNN that improves rainfall prediction, supporting the paper's graph-based approach."},{"cited_title":"Coupling physical factors for precipitation forecast in china with graph neural network,","cited_arxiv_id":null,"evidence_quote":"Demonstrates coupling physical variables through graph structures for precipitation forecasting, another precedent for the GNN methodology."}],"review_version":1}