{"id":"06c611a9-b4f9-4f81-9b0f-de531303ddd6","arxiv_id":"2412.18097","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"LangYa is a single AI model that forecasts global ocean temperature, salinity, and currents for 1 to 7 days at 1/12° resolution, with reported RMSE improvements over numerical systems and the XiHe AI model.","lead":"LangYa is a large AI model trained on 27 years of ocean reanalysis data that forecasts ocean temperature, salinity, and currents for 1 to 7 days ahead at 1/12° resolution. It combines an LLM-based time embedding, an atmosphere sampling strategy, a self-attention module, and a thermocline-focused loss to beat numerical models and the XiHe AI model in forecast accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline accuracy numbers are computed against the same GLORYS12 reanalysis used for training; the claimed advantage over numerical models may reflect reanalysis bias rather than true skill.","rationale":"The reader's weakest assumption correctly identifies the evaluation reference as the most fragile part of the paper. I agree that the GLORYS12-vs-GLORYS12 evaluation is the single most load-bearing concern: the central claim is about forecast skill relative to numerical models and XiHe, but the comparison is not made against a common, independent ground truth. The paper itself acknowledges the limitation in the Discussion, and the lack of error bars leaves the reported differences without statistical support. The architecture and training details are described with enough specificity that the work is plausible and potentially reproducible once code is released, and the authors are transparent about data sources. However, the headline numbers cannot be accepted as established until a common-reference evaluation is performed. This does not move the verdict because the reader already imposed a CONDITIONAL verdict contingent on independent validation; my analysis reinforces that condition rather than changing it. I did not find an internal inconsistency that would justify rejection, and the self-reported limitations are consistent with my reading. The most valuable next step is therefore the concrete common-reference test described above, which would settle whether the claimed advantage is genuine forecast skill or an artifact of evaluating on the training reanalysis.","tokens_in":14468,"tokens_out":2620,"duration_ms":26277,"concrete_test":"Run a single common-reference evaluation for the 2020–2021 test period: compute LangYa forecasts from GLORYS12/ERA5 initial conditions, then verify temperature and salinity profiles against Argo float profiles and currents against moored/satellite observations (e.g., IV-TT Class 4) at the same collocated points. Also compute the same RMSE metrics for PSY4, BLK, GIOPS, FOAM, and XiHe against GLORYS12 (not only against observations) to quantify how much of LangYa's apparent advantage is an artifact of the training/test reference. Report bootstrap confidence intervals for the mean RMSE differences. If LangYa's advantage over numerical models disappears or falls below significance under this common reference, the headline claim requires revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that LangYa outperforms numerical models and XiHe rests on RMSE comparisons in Figs. 2 and 5, but the ground truth for LangYa is GLORYS12 reanalysis (Methods, 'Model Training and Metrics'), the same product used for training (1993–2019) and the test period (2020–2021). The numerical models (PSY4, BLK, GIOPS, FOAM) and XiHe are instead compared partly via the IV-TT Class 4 observational framework, so the two evaluations do not share a common reference. Because GLORYS12 is itself a model-based reanalysis that assimilates observations, a trained model can score well against it by reproducing reanalysis artifacts (e.g., smooth fields, assimilation increments) rather than physical ocean state. The Discussion even concedes that LangYa is 'trained and tested on reanalysis data' and that cross-data-source performance may degrade. In addition, no confidence intervals or significance tests are reported for the RMSE differences, and the thermocline '50% improvement' is tied to a loss term (Eq. 8) that explicitly weights thermocline RMSE, so the improvement is partly by construction and its physical relevance depends on the reanalysis thermocline being accurate. These issues do not disprove the architecture's value, but they make the headline accuracy comparison currently unverified against an independent reference.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript presents LangYa, a data-driven global ocean forecasting system trained on GLORYS12 and ERA5 reanalyses (1993-2019) and evaluated on 2020-2021. The model forecasts temperature, salinity, and zonal/meridional currents on 32 depth levels at 1/12 degree resolution for lead times of 1-7 days with a single set of weights. The four claimed contributions are an LLM-based time embedding that encodes initial date and lead time, an asynchronous cross-iterative random sampling strategy that ingests 10-day atmospheric histories with learned scalar weights, a cosine-attention ocean self-attention module, and an adaptive thermocline loss that weights temperature error by the vertical temperature gradient. The authors report lower RMSE than XiHe, XiHe-AR, PSY4, BLK, GIOPS, and FOAM, and a roughly 50% improvement in thermocline forecast accuracy.","tokens_in":14860,"tokens_out":5283,"duration_ms":50684,"significance":"If the performance claims were verified against an independent observational reference, LangYa would be a significant contribution to AI-based ocean forecasting: a single model spanning multiple lead times at eddy-resolving resolution, with inference of about one second and a thermocline-focused loss, is exactly the kind of system the community needs. The paper also makes useful design choices that are clearly described, uses publicly available data, compares against several operational systems, and adds MAE and PSNR metrics to RMSE. The strengths are, however, currently offset by the evaluation design: the headline metrics appear to use the same reanalysis product for training and verification, the comparison to numerical models is not on a common reference, and the thermocline result is tied to the loss function in a way that makes the improvement partly by construction. These issues are fixable with additional experiments and reporting, but they are central to the paper's claims.","major_comments":[{"comment":"The headline comparisons in Figs. 2 and 5 are not defined against a common reference. Equation (10) computes RMSE against X_t^*, and the training/evaluation description indicates that X_t^* is GLORYS12 reanalysis for LangYa, while the numerical models and XiHe are evaluated through the IV-TT Class 4 observational framework. A lower RMSE against reanalysis does not establish lower error against observations, especially because GLORYS12 is itself a model-based product and the Discussion concedes that LangYa is 'trained and tested on reanalysis data.' Please report, for every system and every lead time, RMSE/MAE computed on the same grid and on the same observational points, and provide confidence intervals or significance tests.","section":"Methods, Model Training and Metrics; Experimental Setting and Main Results"},{"comment":"The claim of 'approximately a 50% improvement in thermocline forecast accuracy' is not quantified in the main text and is partly by construction. Equation (8) adds a loss term proportional to the vertical temperature gradient of the ground truth, and Eq. (9) trains with that term; therefore a lower thermocline RMSE relative to a model trained without the term is expected. To support the physical claim, evaluate thermocline error against independent Argo profiles, report the exact percentage and how it is computed from Fig. 5, and show that the improvement persists when the thermocline region is defined independently of the loss weighting.","section":"Adaptive Thermocline Loss Function, Eqs. (8)-(9)"},{"comment":"The asynchronous cross-iterative random sampling strategy is a load-bearing component, but the paper does not validate that it captures atmosphere-ocean coupling rather than acting as stochastic augmentation. The learned scalar weights H(A_k) and the random sampling over a 10-day window need diagnostics: report the learned weights as a function of lag, compare with a fixed-window or no-atmosphere baseline, and check physical consistency (for example, wind-forced response in ocean currents). Without such evidence, the claim that the module 'effectively simulates air-sea interactions' is unsupported.","section":"Atmospheric Forcing Ocean Module, Eq. (6)"},{"comment":"The paper states that LangYa was evaluated 'with both reanalysis and observational datasets,' but no figure or table reports the observational evaluation, and the IV-TT Class 4 framework is not connected to any LangYa curve. Please add a dedicated observational verification section with masks, sample sizes, and a clear statement of which systems are compared at observation points and which are compared on the reanalysis grid. This is necessary to support the abstract's claim of superiority over numerical and AI-based systems.","section":"Experimental Setting and Main Results; Comparative Methods"}],"minor_comments":[{"comment":"The MSE formula has malformed exponent notation and the MAE expression appears to be missing the outer average over grid points; please fix the typography and confirm the metric definitions.","section":"Methods, Model Training and Metrics, Eq. (10)"},{"comment":"The label 'T Yp e K ey M odules' has unintended spacing and should likely read 'Type Key Modules'.","section":"Fig. 1"},{"comment":"The symbol H is used both for the whole air-sea interaction module in Eq. (1) and for the learned scalar weight in Eq. (6); please use distinct notation to avoid ambiguity.","section":"Eqs. (1) and (6)"},{"comment":"The attribution of FOAM to the Institute of Atmospheric Physics, CAS, conflicts with standard references, which identify FOAM as the UK Met Office's Forecast Ocean Assimilation Model; please verify and cite the correct source.","section":"Comparative Methods"},{"comment":"The code availability section promises future release of trained models and inference code; for a paper making strong performance claims, please provide the evaluation scripts and trained model at submission, or state explicitly that the results are not yet reproducible.","section":"Code Availability"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for an AI-for-Earth-systems venue, but the main risk is overclaiming from reanalysis-only verification. I would encourage the editor to require the common-reference evaluation against observations before acceptance, and to ask the authors to separate the thermocline claim from the construction of the loss function."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: LangYa is a genuinely engineered system, and the paper is honest about its limits, but the headline accuracy claims are not yet supported. The RMSE numbers for LangYa are computed against GLORYS12 reanalysis—the same product used for training—while the numerical models and XiHe are evaluated partly through the IV-TT Class 4 observational framework. That mismatch means the 'lower than PSY4/BLK/GIOPS/FOAM' claim is not a like-for-like comparison, and the magnitude of the claimed advantage is unverified.\n\nWhat's new and good: the LLM-based time embedding (Llama3.1-8B to encode the year, plus discrete month/day/lead embeddings) is a sensible way to get one model to handle 1-7 day leads without autoregressive chaining. The asynchronous cross-iterative random sampling for atmospheric forcing is a plausible mechanism, and the ocean self-attention with cosine attention plus post-norm is a reasonable stability fix. Training a single model at 1/12° for seven leads over 27 years, with a 1-second inference, is a real engineering accomplishment. The paper also deserves credit for stating in the Discussion that it is trained and tested on reanalysis and that cross-data-source performance may degrade.\n\nThe soft spots are real but not disqualifying. The thermocline '50% improvement' is partly built into the loss function (Eq. 8 explicitly weights thermocline regions); reporting an improvement there is to some extent circular, though it does show the loss works as intended. More importantly, no error bars or significance tests are reported, so we cannot tell if the differences from XiHe are meaningful or noise. The asynchronous sampling strategy's learned scalar weights are also not physically validated, which is a lesser concern. The code and models are promised but not yet released, so independent reproduction is currently impossible. These are fixable in revision: add an independent observational comparison for LangYa, report confidence intervals, and actually release the code.\n\nWho this is for: anyone building AI ocean or weather forecasting systems will want to read it, and the methodology discussion (evaluation of learned models against the training reanalysis) is a useful cautionary tale. It deserves a serious referee—conditional acceptance with major revisions is the right call.","headline":"A well-engineered AI ocean forecasting system with honest limitations, but the headline accuracy claims are not yet verified because LangYa's RMSE is computed against the same GLORYS12 reanalysis used for training.","tokens_in":15273,"tokens_out":2288,"would_cite":false,"duration_ms":22619,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LangYa, a single AI model, claims 1–7 day global ocean forecasts at 1/12° resolution with about a 50% thermocline accuracy gain over existing numerical and AI systems.","keywords":["ocean forecasting","large AI models","cross-spatiotemporal forecasting","air-sea coupling","thermocline","LLM time embedding","GLORYS12","ocean self-attention"],"falsifier":"Score LangYa's 2020–2021 forecasts against independent Argo float profiles, not GLORYS12 reanalysis, and check whether the reported ~50% thermocline RMSE improvement over XiHe and the numerical models survives; if it does not, the central accuracy claim is falsified.","tokens_in":14253,"feed_emoji":"🌊","tokens_out":11027,"duration_ms":98943,"temperature":0.7,"pith_summary":"LangYa is an attempt to show that one neural network, rather than a stack of single-day or single-variable models, can forecast the global ocean's temperature, salinity, and currents for lead times of 1 to 7 days at 1/12° daily resolution. The paper argues that explicit time information supplied by an LLM-based embedding, a stochastic sampling scheme for atmosphere-ocean coupling, cosine self-attention, and an adaptive thermocline loss together make this possible. A sympathetic reader would care because the claimed result—roughly 50% better thermocline accuracy and lower RMSE than the leading numerical and AI systems, at about 10,000 times lower inference cost—would make AI ocean forecasting a practical alternative to operational numerical models.","feed_headline":"AI ocean model: 7-day forecasts, 50% better thermocline skill","feed_subtitle":"One neural net maps temperature, salinity, and currents across the global ocean 10,000x faster than numerical systems.","key_machinery":"Four mechanisms carry the argument. The LLM-based time embedding encodes the initial date and lead time as $E(t,\\tau) = E_Y(t_Y) \\oplus E_M(t_M) \\oplus E_D(t_D) + E_\\tau(\\tau)$, with the year coefficients drawn from an instruction-tuned LLM, so a single model can forecast any lead time from 1 to 7 days. The asynchronous cross-iterative random sampling strategy writes the air-sea coupling as $H(X_t, A_{(t-10):t}) \\triangleq X_t \\oplus SS_t \\oplus \\sum_{k=t-10}^{t} H(A_k) * A_k$, where each $H(A_k)$ is a learned scalar weight for a past day's atmospheric field; this is meant to capture the stochastic, lagged influence of the atmosphere on the ocean. The ocean self-attention module replaces dot-product attention with cosine attention plus post-normalization in Swin Transformer blocks to stabilize training. The adaptive thermocline loss multiplies the temperature error by a normalized vertical-gradient term $\\mathrm{Norm}(\\partial [X_T]^*_t/\\partial z) + 1$, forcing the network to spend capacity where the thermocline gradient is sharp.","core_discovery":"The central claim is that LangYa, a single trained model, produces deterministic global 1/12° daily forecasts of ocean temperature, salinity, and zonal and meridional currents for 1–7 day lead times in one inference pass, avoiding both the cumulative drift of autoregressive forecasting and the cost of training a separate model for each lead time. The paper further claims that its four modules each earn their place: the LLM-based time embedding enables cross-spatiotemporal forecasts; the asynchronous cross-iterative random sampling strategy simulates lagged atmospheric forcing; the ocean self-attention module stabilizes training; and the adaptive thermocline loss yields about a 50% improvement in thermocline forecast accuracy. On the 2020–2021 test period, LangYa is reported to achieve lower RMSE than XiHe, XiHe-AR, PSY4, BLK, GIOPS, and FOAM for temperature and currents, with salinity comparable to the best of them, and to run 10,000 times faster than operational numerical models.","pith_inferences":["Editorial inference: If time embedding is as general as claimed, the same single-model design could extend to lead times beyond 7 days or to other ocean variables such as sea ice and biogeochemical tracers without adding sub-models.","Editorial inference: The random-sampling coupling treats air-sea interaction as a learned stochastic perturbation; a natural test the paper does not report is whether the spread of LangYa ensemble members across sampling draws tracks the actual forecast error, which would show the coupling is capturing real uncertainty rather than noise.","Editorial inference: Because the headline verification uses the same reanalysis family as the training data, the paper's own stated limitation about cross-data-source adaptability is the key risk; evaluating LangYa initialized from an operational data assimilation system would be a more demanding test than the GLORYS12-based comparison."],"forward_implications":["One consequence is that a single LangYa model can replace the usual suite of one-model-per-lead-time or autoregressive chains for 1–7 day global ocean forecasts.","A second is that the adaptive thermocline loss, if it carries over to independent conditions, gives AI forecasts their clearest advantage in the 50–300 m layer where vertical temperature gradients govern mixing and heat exchange.","A third is that one-second inference on a single GPU makes global daily ocean-state forecasting cheap enough for real-time operational use and for large ensemble runs.","A fourth is that treating forecast lead time as an explicit conditioning variable, rather than as a separate training target, is what allows one model to exhibit cross-spatiotemporal skill.","Finally, day-7 forecasts are made in a single pass, so cumulative error from chaining daily forecasts is removed by construction."],"supporting_citations":[{"why":"Defines the GLORYS12 1/12-degree, 50-level reanalysis that supplies every training and test ocean field, so the forecast skill claims inherit its accuracy and biases.","marker":"[11]"},{"why":"XiHe is the released AI baseline; LangYa's central comparison claim is lower RMSE than XiHe and XiHe-AR for temperature, salinity, and currents.","marker":"[27]"},{"why":"Provides the ERA5 atmospheric and surface fields that the asynchronous cross-iterative random sampling strategy uses to force the ocean forecasts.","marker":"[34]"},{"why":"Supplies the observational Argo and satellite-altimeter data used in the verification against the numerical models.","marker":"[36]"},{"why":"The Swin Transformer windowed self-attention blocks that form LangYa's deep feature extraction backbone.","marker":"[38]"},{"why":"The shifted-window variant used to introduce cross-window connections in LangYa's ST blocks.","marker":"[39]"},{"why":"The denoising-diffusion time-embedding idea that inspired LangYa's explicit encoding of initial time and forecast lead time.","marker":"[40]"}],"fun_headline_variants":["LangYa: One AI model, 1-7 day ocean forecasts, 10,000x faster","AI ocean forecaster: single model, 7-day lead, better thermocline","LangYa: single model forecasts ocean 1-7 days, 10,000x faster","One AI model: 1-7 day ocean forecasts, 50% better thermocline"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that GLORYS12 reanalysis, the same model-based product used for training, is an adequate stand-in for the true ocean when measuring forecast error, so reanalysis-aligned biases could inflate the reported skill.","fun_headline_variants_meta":{"raw":{"variants":["LangYa: One AI model, 1-7 day ocean forecasts, 10,000x faster","AI ocean forecaster: single model, 7-day lead, better thermocline","LangYa: single model forecasts ocean 1-7 days, 10,000x faster","One AI model: 1-7 day ocean forecasts, 50% better thermocline"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000839,"raw_usage":{"total_tokens":3695,"prompt_tokens":1023,"completion_tokens":2672,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":2573}},"tokens_in":639,"tokens_out":2672,"duration_ms":18449,"temperature":1.0,"reasoning_tokens":2573,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:00:27.227457+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Score LangYa's 2020–2021 forecasts against independent Argo float profiles, not GLORYS12 reanalysis, and check whether the reported ~50% thermocline RMSE improvement over XiHe and the numerical models survives; if it does not, the central accuracy claim is falsified.","supporting_citations":[{"cited_title":"Frontiers in Earth Science 9, 698876 (2021) 11","cited_arxiv_id":null,"evidence_quote":"Defines the GLORYS12 1/12-degree, 50-level reanalysis that supplies every training and test ocean field, so the forecast skill claims inherit its accuracy and biases."},{"cited_title":"Quarterly Journal of the Royal Meteorological Society 147(741), 4186–4227 (2021)","cited_arxiv_id":null,"evidence_quote":"Provides the ERA5 atmospheric and surface fields that the asynchronous cross-iterative random sampling strategy uses to force the ocean forecasts."},{"cited_title":"Journal of Operational Oceanography 8(sup1), 98–111 (2015)","cited_arxiv_id":null,"evidence_quote":"Supplies the observational Argo and satellite-altimeter data used in the verification against the numerical models."},{"cited_title":"Advances in neural information processing systems 33, 6840–6851 (2020) Methods Dataset The OSVs data used in LangYa are derived from the GLORYS12 reanalysis data","cited_arxiv_id":null,"evidence_quote":"The denoising-diffusion time-embedding idea that inspired LangYa's explicit encoding of initial time and forecast lead time."}],"review_version":1}