{"id":"5738d3b8-c4df-49cd-b1ab-dd3f0319c98d","arxiv_id":"2605.28153","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"ObsCast produces skillful short-term high-resolution weather analyses and forecasts over the contiguous US and Europe using only observational data, outperforming operational NWP without relying on NWP-derived data for training or inference.","lead":"ObsCast is a machine learning system that generates weather analyses and short-term forecasts using only observational data, without any numerical weather prediction inputs or reanalyses. A smart generalist might read it to understand a potentially simpler route to accurate regional forecasts built directly from local measurements.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Data provenance may embed NWP-derived information despite the independence claim","rationale":"The reader's weakest assumption correctly flags the core difficulty of learning dynamics from observations alone. The data-provenance issue is a more specific, prior failure mode that would invalidate the independence premise before the learning question even arises. Because the full text was not supplied for direct inspection of the data pipeline, the reader's UNVERDICTED stance remains appropriate; the proposed audit is the minimal check that would resolve the concern.","tokens_in":1698,"tokens_out":323,"duration_ms":17485,"concrete_test":"In the data section, list every input variable and its exact source/version; for each, confirm whether its production pipeline (as documented by the provider) uses NWP retrievals or assimilation. If any does, retrain ObsCast on the corresponding raw unprocessed measurements and recompute the 0-18 h skill scores for near-surface variables; a drop >10 % relative to the reported numbers would falsify the independence claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that both training and inference use exclusively raw observational inputs with no NWP-derived data at any stage. Many standard high-resolution observational products (satellite radiances, radar composites, surface station QC) are generated via retrieval algorithms or assimilation steps that incorporate NWP backgrounds or physical constraints. If ObsCast's input pipelines rely on any such pre-processed fields, the 'without using any NWP-derived data' assertion fails even if the model itself never sees reanalysis grids.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.3","summary":"The manuscript introduces ObsCast, a regional machine-learning system that produces both analyses and short-term forecasts at high resolution. It claims to operate entirely from raw observational inputs with no NWP-derived data used in training or inference, while achieving state-of-the-art performance over the contiguous United States and Europe: outperforming operational NWP for near-surface variables through 18 h and delivering skillful precipitation forecasts.","tokens_in":1775,"tokens_out":387,"duration_ms":22455,"significance":"If the independence claim and quantitative performance results are substantiated with full methodological detail, the work would offer a materially simpler pathway for building regional forecasting systems that do not require access to or inheritance from NWP reanalyses. This could be especially relevant for domains lacking mature NWP infrastructure.","major_comments":[{"comment":"Abstract: the central performance claims (state-of-the-art skill, outperformance through 18 h, skillful precipitation) are stated without any accompanying metrics, datasets, validation periods, or error bars, rendering the empirical assertions unevaluable from the supplied text.","section":"Abstract"},{"comment":"Data and Methods sections: the load-bearing assertion that 'no NWP-derived data' enters training or inference requires an exhaustive accounting of every input field and its provenance. Standard high-resolution observational products (satellite radiances, radar composites, surface QC) frequently embed NWP backgrounds or physical constraints via retrieval or assimilation steps; without explicit verification that ObsCast inputs avoid all such steps, the independence claim cannot be assessed.","section":"Data and Methods"}],"minor_comments":[{"comment":"The abstract and title would benefit from a concise operational definition of 'NWP-derived data' to bound the independence claim.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their constructive feedback and recommendation for major revision. We address each major comment below and will revise the manuscript accordingly to improve clarity and substantiation of our claims.","responses":[{"response":"We agree that the abstract would benefit from greater specificity. In the revised manuscript we will incorporate key quantitative metrics (e.g., RMSE and anomaly correlation for 2 m temperature and 10 m wind), the exact validation periods and domains (e.g., 2022–2023 over CONUS and Europe), and reference to error bars or significance testing so that the performance claims can be directly evaluated from the abstract.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the central performance claims (state-of-the-art skill, outperformance through 18 h, skillful precipitation) are stated without any accompanying metrics, datasets, validation periods, or error bars, rendering the empirical assertions unevaluable from the supplied text."},{"response":"We acknowledge that an exhaustive provenance table is necessary to fully substantiate the independence claim. The revised Data and Methods section will include a comprehensive table enumerating every input field, its exact observational source, and explicit confirmation that no NWP background, retrieval, or assimilation step is involved (e.g., direct ASOS surface observations, raw radar reflectivity, and unprocessed satellite brightness temperatures). This will allow readers to verify that no NWP-derived information enters training or inference.","revision_made":"yes","referee_comment":"[Data and Methods] Data and Methods sections: the load-bearing assertion that 'no NWP-derived data' enters training or inference requires an exhaustive accounting of every input field and its provenance. Standard high-resolution observational products (satellite radiances, radar composites, surface QC) frequently embed NWP backgrounds or physical constraints via retrieval or assimilation steps; without explicit verification that ObsCast inputs avoid all such steps, the independence claim cannot be assessed."}],"tokens_in":1312,"tokens_out":421,"duration_ms":23176,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that ObsCast is built to generate both analysis and short-term high-resolution regional forecasts using only observational inputs for training and inference, with no NWP reanalysis involved at any stage. It reports outperforming operational NWP on near-surface variables out to 18 hours and producing skillful precipitation over the contiguous US and Europe.\n\nWhat is new is the complete break from reanalysis supervision. Most existing ML weather models still train against ERA5-style products and inherit their biases and limits. This one tries to operate directly on observations, which would matter for areas where suitable reanalysis is unavailable or too costly to maintain.\n\nThe paper does a reasonable job naming the practical constraint that current end-to-end ML systems still depend on NWP-derived data. Framing the result as a simpler path for local services is a clear motivation.\n\nThe soft spots are large and start with the abstract itself. There are no quantitative scores, no error bars, no model architecture, no training procedure, and no description of the exact observation sources or preprocessing. Without those, the state-of-the-art claim is just an assertion. The stress-test point on data provenance is worth taking seriously: many satellite, radar, and surface products already embed NWP backgrounds through retrieval or quality control steps. The paper must show that its inputs avoid this contamination or the independence claim does not hold.\n\nThe central assumption that ML can learn the necessary dynamics from observations alone, without physical constraints or reanalysis guidance, is plausible but untested in the provided text.\n\nThis is for people working on regional ML forecasting or on reducing reliance on traditional NWP pipelines. A reader interested in observation-only methods would get value if the full results are solid. It deserves a serious referee because the claim is specific and addresses a real limitation, even though heavy revision is likely needed once the methods and data checks are examined.","headline":"ObsCast claims to forecast from raw observations alone without any NWP data, but the abstract gives no methods, datasets, or numbers, so the performance claims cannot be checked.","tokens_in":2317,"tokens_out":462,"would_cite":false,"duration_ms":27355,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A machine learning system produces skillful high-resolution regional weather forecasts and analyses using only observational data, without any numerical weather prediction models or reanalyses.","keywords":["weather forecasting","machine learning","observational data","regional modeling","high-resolution forecasts","numerical weather prediction","analysis and prediction"],"falsifier":"Running ObsCast on a new geographic region with no training observations and comparing its 18-hour forecasts against independent high-resolution measurements would show whether skill falls below that of operational NWP.","tokens_in":2596,"feed_emoji":"🌦️","tokens_out":579,"duration_ms":16164,"temperature":0.7,"pith_summary":"The paper introduces ObsCast as a regional system that generates both initial conditions and short-term predictions directly from observations. It trains and runs without any NWP-derived data in either stage yet matches or exceeds operational NWP skill for near-surface variables through 18 hours over the contiguous United States and Europe, including for precipitation. This independence removes the need to inherit biases or resolution limits from existing reanalysis products. The approach therefore offers a direct path to build and update regional forecasting services from local observations alone.","feed_headline":"ML system forecasts weather from observations alone without NWP","feed_subtitle":"ObsCast matches operational skill for near-surface variables through 18 hours over the US and Europe using only local data.","key_machinery":"ObsCast, the regional machine learning system that performs both analysis and forecasting directly from observations.","core_discovery":"ObsCast is an end-to-end machine learning model that learns to produce both weather analyses and forecasts solely from observational datasets and achieves state-of-the-art performance in short-term high-resolution regional modeling without using any NWP-generated reanalyses for supervision.","pith_inferences":["The same observation-only training approach could be tested on global scales if dense enough observational coverage becomes available.","Performance on rare extreme events would indicate whether the model has captured dynamics that physical constraints normally enforce.","Combining ObsCast outputs with sparse physical constraints might improve longer-range skill while retaining the independence benefit."],"forward_implications":["Forecasts are generated faster than traditional NWP while maintaining higher skill for near-surface variables.","Precipitation forecasts remain skillful without inheriting resolution limits from reanalysis data.","Regional services can be built and refined directly from local observations without developing full NWP pipelines.","The system adapts more easily to locations where suitable reanalysis products are unavailable or expensive."],"fun_headline_variants":["ML weather forecasts from observations only without NWP","End-to-end model for weather analysis and prediction from data","High-resolution regional forecasting independent of NWP","ObsCast achieves skillful forecasts using observational data alone","No reanalyses: ML system learns weather directly from observations"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Machine learning models can learn the complex atmospheric dynamics sufficiently well from observational data alone to produce accurate forecasts without guidance from physical models or reanalysis products.","fun_headline_variants_meta":{"raw":{"variants":["ML weather forecasts from observations only without NWP","End-to-end model for weather analysis and prediction from data","High-resolution regional forecasting independent of NWP","ObsCast achieves skillful forecasts using observational data alone","No reanalyses: ML system learns weather directly from observations"]},"model":"grok-4.3","cost_usd":0.005478,"raw_usage":{"total_tokens":2523,"prompt_tokens":610,"num_sources_used":0,"completion_tokens":72,"cost_in_usd_ticks":54778000,"prompt_tokens_details":{"text_tokens":610,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1841,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":610,"tokens_out":72,"duration_ms":16880,"temperature":1.0,"reasoning_tokens":1841,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T09:37:04.808712+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Running ObsCast on a new geographic region with no training observations and comparing its 18-hour forecasts against independent high-resolution measurements would show whether skill falls below that of operational NWP.","supporting_citations":[],"review_version":1}