{"id":"a1c9012e-2343-4971-9255-4e718afed188","arxiv_id":"2411.15764","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A GSP-based denoiser plus a GPT-4o mini prompted with neighbor values reconstructs missing graph signals under Gaussian noise, outperforming graph baselines in most tested settings.","lead":"This paper combines graph signal processing with large language models to fill in missing sensor measurements in traffic and weather networks. The GPT-4o mini version of the pipeline beats several graph-based baselines on noisy benchmarks, though the evaluation lacks key controls.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No control for the GSP handler alone: the experiments compare LLM-OSR to full baselines but never show what the learned graph filter already reconstructs, so the LLM's contribution to the headline accuracy is untested.","rationale":"The reader's weakest assumption focuses on recursive feedback of the LLM's own previous estimates. That is a real stability concern, but the GSP-only control is more fundamental: it tests whether the component being highlighted, GPT-4o mini, is actually doing the work. The GSP handler alone is a nontrivial reconstructor because the graph convolution assigns a value to every node, including the missing ones. Without comparing against that output, the reported gains over GLMS/GNLMS/RGDAN could be due entirely to the learned graph filter and graph smoothness, with the LLM acting as a near-identity wrapper around the filtered values. The reader's rationale does mention the absent GSP-only and LLM-only ablations, so there is partial overlap, but the reader does not elevate this to the weakest assumption. If the proposed ablation shows the GSP-only baseline matches LLM-OSR-4, the paper's central claim about the LLM would need to be substantially reframed; if the LLM clearly improves on the GSP-only output, the claim is strengthened. Either way, the conditionality of the verdict is appropriate, so I do not recommend changing the reader's conditional assessment.","tokens_in":17601,"tokens_out":8246,"duration_ms":78006,"concrete_test":"Add an ablation to the Seattle experiment at noise variance 1.0: for each missing node, set the reconstruction equal to the GSP-filtered value at that node, i.e., x_hat_i[t] = o_tilde_i[t], and otherwise keep the experiment identical; report RMSE/MAE on the same test split. Also run a second ablation with x_hat_i[t] = average of the GSP-filtered observed neighbor values plus a fraction of the previous estimate, mirroring the LLM's prompt content. If either baseline is within 10% of LLM-OSR-4's reported values in Table I (RMSE 4.05, MAE 2.88), the LLM is not established as the source of accuracy and the central claim should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that GPT-4o mini inside LLM-OSR is accurate and robust. The evidence compares LLM-OSR-4 with full baselines (GLMS, GNLMS, GNS, GCN, GVARMA, GGARCH, RGDAN), but no experiment isolates the GSP handler. The GSP handler (Algorithm 2, Eq. 4) is a learned graph filter: applied to o[t], the graph convolution U diag(h(Λ)) U^T o[t] produces a value at every node, including the 30% missing nodes. Those filtered missing-node values are already a reconstruction. The LLM prompt (Eq. 11) is built from the same filtered observed-neighbor values plus the previous estimate, so the LLM could simply be re-emitting the graph filter's smooth interpolation. If a parameter-free baseline using the same inputs, or simply reading the GSP-filtered value at the missing node, matches LLM-OSR-4's RMSE/MAE, then the reported accuracy is attributable to GSP smoothness rather than to the LLM's reasoning. This is a missing control, not an internal inconsistency, but it is load-bearing because the paper's stated contribution is the combination.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LLM-OSR, an online framework that combines a learned graph-spectral filter (GSP-based spatial-temporal signal handler) with a pretrained LLM predictor to reconstruct missing graph signal values from noisy, partially observed time-varying signals. The GSP filter is trained by minimizing the MAE between the filtered training signals and ground truth; the LLM (GPT-3.5-turbo or GPT-4o mini) is prompted with the filtered observed neighbor values and the previous node estimate to output each missing node value. Experiments on the Seattle Loop traffic dataset and NOAA wind-speed/temperature datasets under additive Gaussian noise report RMSE/MAE for LLM-OSR variants and seven baselines (GLMS, GNLMS, GNS, GCN, GVARMA, GGARCH, RGDAN). The results show that LLM-OSR-4 achieves the best or second-best accuracy in most settings, and the authors acknowledge that its performance degrades relative to RGDAN at higher noise levels.","tokens_in":79,"tokens_out":4658,"duration_ms":106159,"significance":"If the claimed gain is real, the paper demonstrates a novel and practical integration of a pretrained LLM with graph signal processing for online reconstruction tasks, a direction that is currently of wide interest. The manuscript is also transparent about limitations (numerical instability of LLM outputs, one-node-per-prompt cost, recursive memory issues, and scalability), which is a strength. However, the evidence is not yet sufficient to support the central claim: no experiment isolates the GSP handler's contribution, the recursive feedback loop is not stress-tested, and the comparison with deep-learning baselines rests on very short training sequences. The scientific value therefore depends on completing the missing control experiments; the concept is interesting but the current validation is incomplete. The paper does not ship code or reproducibility details, which further limits verification.","major_comments":[{"comment":"The experiments compare the full LLM-OSR pipeline against baselines but never report the reconstruction obtained by the GSP-based spatial-temporal signal handler alone. Applying the learned filter from Eq. (4), U diag(h(Λ)) U^T o[t], yields a value at every node, including the missing nodes, so its missing-node output is already a reconstruction. Without this control, it is unclear how much of LLM-OSR-4's accuracy comes from the LLM predictor and how much from the GSP filter's smooth interpolation. This is a load-bearing omission for the central claim that GPT-4o mini is an accurate and robust predictor.","section":"Section IV, Tables I-VI"},{"comment":"The predictor feeds the previous estimate x̂[t-1] as temporal context, creating a recursive loop that can accumulate errors over long horizons (up to 1440 test time steps for traffic). The paper provides no experiment replacing predicted past values with ground-truth past values, nor any analysis of error growth over time. Without such a test, the stability of the online scheme is unverified, and a single bad prediction could contaminate all subsequent reconstructions.","section":"Algorithm 1 line 7 and Algorithm 3 line 5"},{"comment":"There is a train/test mismatch in how the GSP filter is used. In Algorithm 2, the filter is learned by applying the graph convolution to full training signals x[t] and minimizing MAE against xg[t]. In the test phase, however, the filter is applied to observations o[t] = M(xg[t] + ϵ[t]) with zero-masked missing nodes (Eq. (5)). The filter is never trained or validated under the masked observation model, so its behavior on the actual deployment input is unknown; this weakens the premise that the GSP handler 'enhances' the signals that the LLM receives.","section":"Algorithm 2 and Section IV-A"},{"comment":"The method is developed for undirected, unweighted graphs with adjacency entries in {0,1} (Eq. (1)), but the weather experiments use a k-nearest-neighbor graph with Gaussian-kernel edge weights. The graph Laplacian in Eq. (2) and the GFT in Eq. (3) are not defined for weighted adjacency matrices, and no normalized Laplacian or alternative treatment is specified. This makes the spectral filter implementation for the weather datasets (Tables III–VI) under-specified and potentially inconsistent with the theory.","section":"Section II-A and Section IV-A1"},{"comment":"The offline deep baselines (GCN, GVARMA, GGARCH, RGDAN) are trained on very short sequences: 24 time steps for weather and 576 for traffic, with no architecture or hyperparameter details reported. For a fair comparison, the training configuration, validation procedure, and early stopping criteria should be stated; otherwise the reported advantage over these baselines may reflect insufficient training rather than genuine superiority of the proposed method.","section":"Section IV-A2 and IV-A1"}],"minor_comments":[{"comment":"The notation \"PF f=1 h(λ)f\" is malformed; it should be written as a sum over filter components (e.g., ∑_{f=1}^F h(λ_f), with a clear definition of h(λ)). This makes the graph convolution formula hard to parse.","section":"Equation (4)"},{"comment":"The symbol x̂ is used for the processed signal in the text before Eq. (8) but x̃ is used in the equation; please use one consistent notation for the filtered signal and define it at first use.","section":"Equation (8) and surrounding text"},{"comment":"The text says \"During the training phase of LLM-SRO\" but the algorithm is called LLM-OSR; this typo should be corrected.","section":"Section III-B"},{"comment":"The sentence \"The limitations and some potential future extensions of the proposed LLM-OSR are discussed in Section V provides an in-depth discussion...\" is grammatically broken; please rephrase to a single coherent statement.","section":"Section I"},{"comment":"The entries \"19.22 ± e+00\" are missing the numerical value before the exponent; they should read e.g., \"19.22 ± 1.0e+00\" with the correct standard deviation.","section":"Table II, GCN row"},{"comment":"The abstract claims robustness under Gaussian noise, but Section IV-C notes that LLM-OSR-4 degrades more quickly than RGDAN as noise variance increases. The claim of robustness should be qualified to the low-to-moderate noise regime in the abstract and conclusion.","section":"Section IV-B and IV-C"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable candidate after substantive revision, but the missing GSP-only ablation is a serious gap that prevents the current version from supporting its central claim. I would also ask the authors to address the recursive-feedback stability and the weighted-graph consistency before considering acceptance. The honest limitations section is a positive sign, and the topic is timely."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nQuick take: this paper reports good numbers but never shows what the GSP filter alone already does, so the LLM's contribution is unproven. The stress-test note is on target. I read the method carefully: the GSP handler applies a learned graph filter to the zero-masked observation o[t], producing a value at every node, including the 30% missing ones. Those filtered values are reconstructions in their own right, and the LLM prompt is built from the same filtered neighbor values plus the previous estimate. A control that simply reads off the GSP-filtered value at the missing node, or a parameter-free smoother using the same inputs, is missing. Without that, the headline accuracy could come from GSP smoothness rather than LLM reasoning.\n\nWhat's genuinely new: using a pretrained LLM prompted with neighbor and past values as the online predictor after GSP denoising is, to my knowledge, a fresh combination. The writing is clear, the algorithms are explicit, and the limitation section is honest—they admit high-noise degradation, numerical instability, and scalability problems. That gives me some confidence the authors are not hiding the warts.\n\nThe soft spots, in order of weight. First, the missing GSP-only control is load-bearing. Second, the baselines are trained on very short windows (24 time steps for weather) while the LLM brings pretrained world knowledge, so the comparison is asymmetric. Third, the recursive use of the model's own previous estimate (x-hat[t-1]) is never stress-tested by replacing it with ground truth; a bad early prediction could poison the whole horizon. Fourth, no code or data is released. Minor point: the 'reverse embedding' contribution is really just prompt construction—fine, but it's oversold as a technique.\n\nThe abstract's 'accurate and robust' is also overstated: the weather results show LLM-OSR-4's performance degrades more than RGDAN as noise increases. The paper's own discussion concedes this.\n\nNet: the central idea is plausible and the combination is worth publishing if the evidence is strengthened. As it stands, the evaluation does not pin down where the gain comes from. I'd send it to review with a clear request for the GSP-only ablation and fairer baselines, but I wouldn't reject it outright.\n\nFor you: if you work on GSP+LLM or on evaluation methodology for LLM predictors, worth a skim; otherwise you can skip. I'd bring it to a reading group that debates 'what is a control'—it is a good case study.\n\nMy recommendation: engage with it, but insist on the ablation before believing the central claim.","headline":"Promising GSP+LLM combination for online graph signal reconstruction, but a load-bearing missing control: the GSP filter alone already outputs values for missing nodes, so the LLM's contribution is not actually isolated.","tokens_in":18383,"tokens_out":2787,"would_cite":false,"duration_ms":25011,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a pretrained GPT-4-o mini LLM, given graph-denoised neighbor values and its own previous estimate in a text prompt, reconstructs missing spatial-temporal signals more accurately than graph filters, graph time-series…","keywords":["Large Language Models","Graph Signal Processing","online reconstruction","spatial-temporal signals","Gaussian noise","zero-shot prediction","graph signal denoising","traffic forecasting"],"falsifier":"Run the same LLM-OSR-4 pipeline twice on the traffic data: once feeding the LLM's previous estimate $\\hat{x}[t-1]$ in each prompt and once feeding the true previous value $x_g[t-1]$; a large drop in RMSE in the second run would show that recursive error accumulation, not robust spatial-temporal understanding, drives the reported performance. A complementary control shuffles the neighbor values in the prompts, because if reconstruction error does not rise when neighbor context is destroyed, the LLM is not using spatial information at all.","tokens_in":17403,"feed_emoji":"📈","tokens_out":9018,"duration_ms":69673,"temperature":0.7,"pith_summary":"This paper tries to establish that an off-the-shelf large language model, GPT-4-o mini, can act as an accurate and robust online predictor for missing values in noisy, time-varying graph signals, provided the signals are first cleaned by a learned graph filter. The proposed LLM-OSR pipeline denoises the partial observation with a graph convolution trained on the graph Laplacian, then rewrites each missing node's task as a short English prompt containing the node's previous estimated value and its observed neighbors' current values. On traffic and weather datasets corrupted by Gaussian noise, the GPT-4-o mini variant reports lower RMSE and MAE than adaptive graph filters (GLMS, GNLMS, GNS), graph time-series models (GVARMA, GGARCH), and GNN baselines (GCN, RGDAN), while the GPT-3.5-turbo variant performs much worse. The paper reads this as evidence that modern LLMs can treat graph-encoded numeric data as language and compete with dedicated graph predictors with no fine-tuning.","feed_headline":"Graph denoising plus GPT-4o mini reconstructs missing signals best","feed_subtitle":"GSP learns a denoiser; a zero-shot LLM predicts missing nodes and beats graph baselines on traffic and weather.","key_machinery":"The central object is the LLM-OSR pipeline: an undirected graph $\\mathcal{G}$ with Laplacian $L = D - A$; a learned graph filter $h(\\lambda)$ applied through the graph convolution $\\tilde{x} = U \\mathrm{diag}(h(\\lambda)) U^{\\top} x$ to denoise the noisy, partially observed signal $o[t] = M(x_g[t] + \\epsilon[t])$; and a reverse-embedding step that turns each missing node's prediction into the text prompt $T(v_i) = \\mathrm{agg}(\\{(\\hat{x}_i[t-1], \\tilde{x}_j[t]) \\mid j \\in (A v_i \\cup O)\\})$. The filter parameters are trained by gradient descent on the MAE loss, and the LLM is used zero-shot with a system role that demands one numeric output per node and a user role that supplies the numeric context. The load-bearing mechanism is the LLM performing self-aggregation over neighbor values plus the previous estimate, converting graph signal prediction into a language-completion task.","core_discovery":"On the paper's own terms, the central discovery is that combining a GSP denoiser with a zero-shot LLM predictor gives the most accurate one-step reconstruction of missing graph signals among the tested methods under Gaussian noise. The GSP handler learns filter parameters by gradient descent on the graph Laplacian to minimize MAE against clean training signals, and the LLM then outputs the missing node values from prompts of the form: time index, entity index, previous estimated value, and observed neighbor values. In the reported numbers, LLM-OSR-4 reaches RMSE 4.05 and 4.69 on the Seattle traffic dataset for noise variances 1.0 and 1.5, ahead of RGDAN's 5.32 and 6.61, and leads on wind and temperature RMSE at low noise while degrading faster than RGDAN as noise variance rises. The authors attribute the success to the LLM using the spatial smoothness of graph signals through natural-language descriptions rather than through explicit topological reasoning.","pith_inferences":["A testable implication the paper leaves implicit: if the LLM truly uses spatial information, shuffling or replacing the neighbor values in the prompts should increase error; if error is unchanged, the gains may come mostly from temporal persistence plus denoising.","Because the temporal context is the LLM's own previous estimate, errors can compound over long horizons; an ablation that feeds the true past value would separate one-step skill from long-horizon stability.","The prompt design could be extended to few-shot learning by inserting numeric examples, which the paper identifies as a likely path to better accuracy; this extension would test whether in-context examples beat the current zero-shot format.","For impulsive or heavy-tailed noise, the Gaussian observation model and the MAE-trained graph filter would need to be reworked, and the paper suggests $\\alpha$-stable noise as the natural next case."],"forward_implications":["If the reported margins hold, an off-the-shelf LLM plus a learned graph filter can outperform dedicated graph filters, GNNs, and graph time-series models for one-step missing-data reconstruction.","The method requires no LLM fine-tuning, so it can be deployed with API access alone on any graph where node neighborhoods and a denoising filter are available.","The large gap between GPT-4-o mini and GPT-3.5-turbo inside the same pipeline indicates that LLM capability, not just the GSP handler, drives the result.","Performance degrades as Gaussian noise variance grows, so the practical niche is moderate noise; the paper attributes the degradation to residual noise left by the GSP filter.","The retry-and-error-check mechanism for invalid LLM outputs is a practical requirement for online deployment."],"supporting_citations":[{"why":"Supplies the Seattle Loop Detector traffic dataset and the highway graph topology used for the main experiments.","marker":"[39]"},{"why":"Supplies the NOAA wind speed and temperature station data used for the weather experiments.","marker":"[40]"},{"why":"Defines the GLMS adaptive filter that serves as a baseline for online graph signal estimation under Gaussian noise.","marker":"[42]"},{"why":"Defines the GNLMS baseline and the spectral-normalization idea, and its kNN graph construction is used for the weather datasets.","marker":"[43]"},{"why":"Defines the GNS adaptive filter baseline for online graph signal estimation under impulsive noise.","marker":"[44]"},{"why":"Defines the GCN baseline, a graph neural network the LLM-OSR is compared against.","marker":"[22]"},{"why":"Defines the GVARMA graph time-series baseline.","marker":"[45]"},{"why":"Defines the GGARCH graph time-series baseline.","marker":"[46]"},{"why":"Defines the RGDAN attention-based GNN baseline that is the strongest non-LLM competitor in the reported tables.","marker":"[17]"},{"why":"Describes the GPT-4 model family from which the GPT-4-o mini predictor is drawn.","marker":"[32]"}],"fun_headline_variants":["LLM-OSR: GSP denoiser + GPT-4o mini beats graph baselines on traffic and weather","Zero-shot LLM pairs with GSP denoiser to top noise-robust reconstruction","GSP-LLM framework reconstructs missing signals better than graph baselines","GPT-4o mini + GSP lead in traffic and weather missing-data reconstruction","Graph-plus-language model recovers lost nodes with top accuracy under noise"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the system can keep feeding the LLM its own earlier guesses as temporal context without the errors accumulating and spoiling later predictions; the paper does not test this by swapping in ground-truth past values.","fun_headline_variants_meta":{"raw":{"variants":["LLM-OSR: GSP denoiser + GPT-4o mini beats graph baselines on traffic and weather","Zero-shot LLM pairs with GSP denoiser to top noise-robust reconstruction","GSP-LLM framework reconstructs missing signals better than graph baselines","GPT-4o mini + GSP lead in traffic and weather missing-data reconstruction","Graph-plus-language model recovers lost nodes with top accuracy under noise"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000488,"raw_usage":{"total_tokens":2368,"prompt_tokens":875,"completion_tokens":1493,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":1380}},"tokens_in":491,"tokens_out":1493,"duration_ms":11118,"temperature":1.0,"reasoning_tokens":1380,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:56:24.128532+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same LLM-OSR-4 pipeline twice on the traffic data: once feeding the LLM's previous estimate $\\hat{x}[t-1]$ in each prompt and once feeding the true previous value $x_g[t-1]$; a large drop in RMSE in the second run would show that recursive error accumulation, not robust spatial-temporal understanding, drives the reported performance. A complementary control shuffles the neighbor values in the prompts, because if reconstruction error does not rise when neighbor context is destroyed, the LLM is not using spatial information at all.","supporting_citations":[{"cited_title":"Seattle loop detector data,","cited_arxiv_id":null,"evidence_quote":"Supplies the Seattle Loop Detector traffic dataset and the highway graph topology used for the main experiments."},{"cited_title":"National oceanic and atmospheric administration (noaa) weather data,","cited_arxiv_id":null,"evidence_quote":"Supplies the NOAA wind speed and temperature station data used for the weather experiments."},{"cited_title":"Adaptive least mean squares estimation of graph signals,","cited_arxiv_id":null,"evidence_quote":"Defines the GLMS adaptive filter that serves as a baseline for online graph signal estimation under Gaussian noise."},{"cited_title":"Normalized lms algorithm and data-selective strategies for adaptive graph signal estimation,","cited_arxiv_id":null,"evidence_quote":"Defines the GNLMS baseline and the spectral-normalization idea, and its kNN graph construction is used for the weather datasets."},{"cited_title":"Adaptive message passing sign algorithm,","cited_arxiv_id":null,"evidence_quote":"Defines the GNS adaptive filter baseline for online graph signal estimation under impulsive noise."},{"cited_title":"Forecasting Time Series With V ARMA Recursions on Graphs,","cited_arxiv_id":null,"evidence_quote":"Defines the GVARMA graph time-series baseline."},{"cited_title":"Multivariate time series forecasting with GARCH models on graphs,","cited_arxiv_id":null,"evidence_quote":"Defines the GGARCH graph time-series baseline."},{"cited_title":"RGDAN: A random graph diffusion attention network for traffic prediction,","cited_arxiv_id":null,"evidence_quote":"Defines the RGDAN attention-based GNN baseline that is the strongest non-LLM competitor in the reported tables."}],"review_version":1}