{"id":"65ced66d-a842-49b1-891e-b11f5601bf3b","arxiv_id":"2608.04792","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A benchmark of geospatial foundation models for biomass regression shows that pre-computed embedding products, especially AlphaEarth Foundations, outperform both frozen weight-distributed GFMs and a fully supervised state-of-the-art model.","lead":"This paper benchmarks satellite-image AI models for estimating above-ground biomass, and finds that ready-made embedding products beat both frozen foundation models and a fully supervised baseline. The practical consequence is that carbon monitoring from space may need geospatial AI models delivered as analysis-ready features rather than as model weights users must run themselves.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim may hinge on AEF's GEDI-aligned pre-training: AEF reconstructs GEDI L2A height metrics, the parent signal of the GEDI L4A labels; the decisive check is a TESSERA full-regime control.","rationale":"The paper is a thorough, honest benchmark that clearly separates two delivery modes for GFMs, runs controlled spatial and temporal generalization experiments, and validates against independent AGBref data. The reader's weakest_assumption identifies the AEF circularity concern, and we agree this is the single most load-bearing issue. The frozen-encoder input-support confound is real but the authors transparently attribute the gap to it, consistent with their delivery-mode framing, so it does not threaten the central claim. The AEF circularity is more dangerous because the abstract's central quantitative win is AEF-specific, and AEF is also the backbone for the generalization and AGBref claims. The paper partially acknowledges this in Section 6.2, but its decomposition argument rests on the Lite-regime TESSERA comparison. A full-regime TESSERA run directly tests whether non-GEDI-aligned pre-computed embeddings also beat the supervised baseline; if it passes, the central claim holds with only minor caveats, and if it fails, the strongest claims collapse to a narrow statement about AEF specifically. The reader's CONDITIONAL verdict is therefore appropriate, and the proposed TESSERA full run makes the condition concrete and executable.","tokens_in":28397,"tokens_out":10203,"duration_ms":108848,"concrete_test":"Run fcn_film on TESSERA embeddings in the exact Full regime used for AEF in Table 1: train on the full AGBD training set and evaluate on the full AGBD test set. If TESSERA's full-data RMSE is at or below the AGBD-features baseline of 53.73 Mg/ha, the embedding-product advantage survives without GEDI-target alignment. If TESSERA's full-data RMSE exceeds roughly 55 Mg/ha, the AEF-specific gains are largely attributable to reconstructing GEDI L2A and ALOS PALSAR-2 in pre-training, and the headline claim must be qualified accordingly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest quantitative claims (abstract; Table 1) are AEF-specific: an MLP on AEF embeddings achieves 52.22 Mg/ha versus 53.73 for fcn_film on AGBD features, and fcn_film on AEF+ is best overall at 50.79 Mg/ha. AEF's pre-training objective (Table A.6, Section 6.2) reconstructs GEDI L2A relative-height metrics and ALOS PALSAR-2 L-band as dense per-pixel targets. GEDI L4A biomass, the regression label in AGBD (Section 2.1), is derived directly from GEDI L2A waveform metrics (Duncanson et al., 2022). Thus AEF's embedding space is explicitly optimized to encode the structural signal that generates the labels. Excluding AEF training locations from test sets prevents spatial leakage but not representational alignment: the difficult extraction of canopy structure from optical/SAR imagery has been pre-solved in the embedding. The observed margins are small: 50.92 vs 53.73 Mg/ha, roughly 5%. The paper's own Lite-regime control (AEF 53.70 vs TESSERA 56.43, Section 6.2) suggests a 2.7 Mg/ha advantage from target alignment; applying a similar gap at full scale would put a non-GEDI-aligned product at roughly parity with the SOTA baseline. TESSERA, which does not use GEDI targets, was evaluated only in the Lite regime (Table C.8: TESSERA fcn_film 56.43 vs AGBD fcn_film 59.02), not the full regime. Without a full-data TESSERA result, the claim that pre-computed embedding products are highly effective cannot be separated from AEF's GEDI-aligned pre-training.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper benchmarks geospatial foundation models (GFMs) for global above-ground biomass (AGB) regression on the AGBD dataset. It distinguishes weight-distributed GFMs, evaluated as frozen encoders within PANGAEA (11 models), from pre-computed embedding products (AEF and TESSERA), and compares both against a fully supervised fcn_film baseline. The central claims are: (i) frozen GFMs substantially underperform the supervised baseline; (ii) pre-computed embeddings, especially AEF, are highly effective, with an MLP on AEF embeddings beating the supervised baseline and fcn_film on AEF+ achieving the best overall RMSE; (iii) AEF embeddings generalize better across space and time and reach near-parity with the ESA CCI biomass product on the independent AGBref reference data. The paper concludes that the delivery mode of GFM features, not just model quality, determines their utility for quantitative regression.","tokens_in":28780,"tokens_out":7896,"duration_ms":88252,"significance":"If fully substantiated, the paper would make a useful contribution: it provides a broad, global-scale comparison that most existing GFM benchmarks avoid for continuous regression, ships public code and data, reports three-seed means with small standard deviations for the main AEF and baseline comparisons, excludes AEF pre-training locations from evaluation, and includes an independent validation against AGBref. However, the central delivery-mode claim is weakened by a known confound: AEF's pre-training explicitly reconstructs GEDI L2A structural metrics, the signal family from which the GEDI L4A regression labels are derived. The paper is candid about this alignment in Section 6.2, but the abstract and conclusion nevertheless assert the stronger claim that pre-computed embedding products as a class are highly effective. The contribution is therefore best viewed as a careful demonstration that AEF embeddings are informative for GEDI-derived AGB prediction, with the broader conclusion about delivery format still requiring a non-GEDI-aligned embedding control at full scale.","major_comments":[{"comment":"The central attribution of AEF's success to delivery mode is not yet supported. AEF's pre-training reconstruction targets include GEDI L2A relative-height metrics and ALOS PALSAR-2 L-band (Table A.6), while the AGBD regression labels are GEDI L4A biomass, a product derived from the same GEDI L2A waveform family (Section 2.1). Excluding AEF training locations from the test sets prevents spatial leakage, but the representation has still been optimized to encode exactly the structural signal that generates the labels. The only non-GEDI-aligned embedding product, TESSERA, was evaluated only in the Lite regime; there AEF's advantage over TESSERA with the fcn_film head is 2.7 Mg/ha (53.70 vs. 56.43, Table C.8) and with the MLP head 5.9 Mg/ha (53.91 vs. 59.79). The paper's statement that 'the bulk of the gap is attributable to the factors AEF and TESSERA share' therefore extrapolates from a Lite-only control. A full-regime TESSERA fcn_film/MLP result, or an equivalent non-GEDI-aligned embedding product, is needed before the headline 'pre-computed embedding products prove highly effective' can be attributed to delivery format rather than to AEF's task-aligned pre-training.","section":"Section 6.2, Table C.8, Appendix A.6"},{"comment":"The frozen-encoder GFM results are reported as single runs without standard deviations, while all other full-regime results are three-seed means. The full-regime comparison that supports the 'substantially underperform' claim is close: SSL4EO-MoCo (60.56) is only 2.0 Mg/ha worse than fcn_film on Sentinel-2 only (58.57), and in the Lite regime the ordering is reversed (64.34 vs. 66.51, Table C.8). With a single run, the full-regime reversal could be within run-to-run variability. Please provide multiple seeds for at least the best-performing GFMs, such as SSL4EO-MoCo and Prithvi-2, or otherwise quantify the uncertainty of the frozen-encoder results.","section":"Section 5.1, Tables 1 and 2"},{"comment":"The headline that weight-distributed GFMs 'substantially underperform' conflates representation quality with input support. Because none of the benchmarked GFMs accepts L-band SAR or the AGBD ancillary features, the frozen-encoder evaluation is an endpoint comparison under restricted inputs. The paper acknowledges this in Section 6.1, but the abstract and conclusion state the result without the caveat. In the matched-input Lite setting, the best GFM actually outperforms the supervised Sentinel-2-only baseline (64.34 vs. 66.51, Table C.8), and the full-regime reversal is based on single runs. I recommend stating in the abstract and conclusion that the underperformance holds in the realistic input-restricted deployment setting, and that representation quality alone is not the tested quantity.","section":"Sections 3.1 and 6.2, Abstract"}],"minor_comments":[{"comment":"AGBD Lite is described as containing about 600,000 samples and as a subsample of about 5% of the approximately 16 million patches; 5% of 16 million is 800,000, so the numbers should be reconciled, for example by clarifying whether the 5% is applied to GEDI footprints before patch construction.","section":"Section 2.1"},{"comment":"The statement that TESSERA training locations are not publicly available and may overlap with AGBD test locations should be revisited in the discussion; at minimum, report whether the temporal overlap (same-year Sentinel-1/2 imagery) could introduce coincident data in the embedding generation.","section":"Section 3.2"},{"comment":"The caption says the fcn_film baseline rows are shaded, but the shading is not visible in the typeset version; please ensure the final PDF displays the intended shading.","section":"Table 2"},{"comment":"The caption contains a typo ('Here„'); it should be 'Here,'.","section":"Figure 7 caption"},{"comment":"Please clarify whether the all-years temporal baseline is trained on the full joint 2019+2020 training set or on the size-matched subsample; the sentence 'for those experiments, we subsample the 2020 training set to match the 2019 training set size' is ambiguous.","section":"Section 4.2"},{"comment":"The 1500 km buffer radius for the AGBref comparison is selected by visual inspection; this is a reasonable screening device, but the sensitivity of the near-parity result to the buffer radius should be reported or at least acknowledged in the main text.","section":"Section 5.5 and Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The paper is admirably candid in Section 6.2 and in the Limitations section about AEF's GEDI-aligned pre-training, which is to its credit. The problem is that the abstract and conclusion assert the stronger delivery-mode claim without the full-regime non-GEDI-aligned control needed to support it. I would ask for either a full-regime TESSERA result or a carefully reframed central claim that limits the finding to AEF. No concerns about novelty disclosure or citation practice."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, this paper's main claim is genuinely new: pre-computed embedding products, specifically AEF, beat a fully supervised SOTA model on global AGB regression, while frozen weight-distributed GFMs underperform. The delivery-mode distinction is a real contribution, not a repackaging. The benchmark is thorough: 11 GFMs, two embedding products, linear/MLP/fcn heads, geographic and temporal splits, and an AGBref comparison. Code and data are public, main comparisons have three seeds, and the writing is clear and honest about limitations.\n\nThe soft spot is the one the stress-test note names. AEF's pre-training reconstruction targets include GEDI L2A relative-height metrics (Appendix A.6), and the regression labels are GEDI L4A biomass, derived from the same waveform family. The paper acknowledges this in Section 6.2 and calls it the likely explanation for AEF's edge over TESSERA. That is honest, but it means the abstract's general claim about pre-computed embedding products is only directly demonstrated for a product whose pre-training was aligned with the label-generating sensor. The missing experiment is a full-regime TESSERA evaluation; TESSERA does not use GEDI targets. In the Lite regime, TESSERA beats AGBD features but trails AEF by ~2.7 Mg/ha. That small margin suggests the circularity burden could shrink the AEF-specific advantage substantially at full scale.\n\nOther concerns are real but milder. The frozen-encoder GFMs only see Sentinel-2 bands while AEF sees multi-modal time-series; the paper flags this but cannot fully control it. The AGBref comparison uses a 1500 km buffer chosen by visual inspection and excludes Japan post hoc, but the full-all-set results are also reported, so it is transparent.\n\nBottom line: this is a solid, useful benchmark that deserves peer review. It would be stronger with a full-regime TESSERA run and a cleaner separation of pre-training alignment from the embedding-product advantage. I'd engage with it seriously and ask for that experiment, not reject it.","headline":"A genuinely new delivery-mode result for GFM benchmarking, but the AEF-specific advantage is entangled with GEDI-aligned pre-training; the missing TESSERA full-regime control is the decisive experiment.","tokens_in":29348,"tokens_out":3244,"would_cite":true,"duration_ms":32774,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that for global above-ground biomass regression, geospatial foundation models outperform a fully supervised state-of-the-art baseline only when delivered as pre-computed embedding products, not as frozen encoders; a…","keywords":["above-ground biomass","geospatial foundation models","embedding products","frozen encoders","remote sensing regression","satellite imagery","global biomass mapping","generalization"],"falsifier":"Train the same linear-probe, MLP, and fcn_film heads on embeddings from a model with comparable multi-modal, multi-temporal pre-training but no GEDI-family or vegetation-structure reconstruction targets, and compare against the supervised baseline on the same AGBD test split. If the margin shrinks to near zero, the representation-quality reading fails; if the margin persists, the paper's delivery-mode conclusion survives. A complementary check is to fine-tune the best weight-distributed GFM with a dense per-pixel reconstruction objective over the same modalities and see whether it reaches AEF-level accuracy.","tokens_in":28184,"feed_emoji":"🌲","tokens_out":10980,"duration_ms":113177,"temperature":0.7,"pith_summary":"The paper tries to establish that the delivery format of a geospatial foundation model—weights the user runs, or embeddings the user consumes—decides whether the model helps at all for a hard quantitative regression task: mapping above-ground biomass from satellite data. It benchmarks eleven weight-distributed models as frozen encoders and two pre-computed embedding products (AlphaEarth Foundations and TESSERA) against a fully supervised convolutional baseline on the global AGBD dataset. The result is one-sided: relative to the baseline trained on the full raw feature stack, every frozen encoder underperforms, while an MLP trained on AEF embeddings reaches 52.22 Mg/ha RMSE versus 53.73 Mg/ha for the supervised baseline, and the same supervised network on AEF embeddings plus a few raw covariates reaches 50.79 Mg/ha, the best result. The authors attribute the gap to the input richness and multi-temporal context distilled into the embedding products, with AEF's reconstruction targets—including GEDI height metrics and L-band radar—a secondary refinement. If the claim holds, foundation-model builders should treat analysis-ready embedding layers as a first-class product, because a user who must run a frozen encoder is currently worse off than one who uses no foundation model at all.","feed_headline":"Pre-computed embeddings beat frozen AI models for biomass maps","feed_subtitle":"A simple MLP on AEF embeddings outdoes the supervised state of the art and generalizes across space and time.","key_machinery":"The load-bearing object is the pre-computed embedding layer: a per-pixel vector field (64 dimensions for AEF, 128 for TESSERA) distilled from a full year of multi-modal satellite observations, delivered as data rather than as model weights. The benchmark machinery is a frozen-encoder protocol in which eleven weight-distributed GFMs are paired with a standard dense-prediction decoder (UPerNet) and trained only on that head, while the embedding products are paired with a linear probe, an MLP, and the fully convolutional fcn_film network (a feature-wise linear-modulation architecture used both as supervised baseline and as high-capacity head). The mechanism proposed for the results is that pre-computation carries temporal depth into a single-date benchmark for free, and that AEF's dense per-pixel autoencoding objective—reconstructing targets that include GEDI relative-height metrics and L-band radar—preserves continuous structural gradients that patch-level masked-image-modeling or contrastive objectives do not. AGBD Lite, a distribution-matched 5% subset of the 16-million-sample dataset, is the screening device that makes benchmarking tractable while preserving model rankings.","core_discovery":"On the paper's own terms, the central discovery is a sharp dichotomy in how geospatial foundation models behave on a continuous regression task. Run as frozen encoders, none of the eleven weight-distributed GFMs matched the supervised baseline: the best, SSL4EO-MoCo, reaches 60.56 Mg/ha RMSE on the full test set versus 53.73 Mg/ha for the fully convolutional fcn_film baseline trained on AGBD features, and this gap is mainly blamed on restricted input support—most models accept only a subset of Sentinel-2 bands and cannot ingest the radar and ancillary variables in AGBD. Consumed as pre-computed embeddings, the picture inverts: a single-hidden-layer MLP on the 64-dimensional AEF embeddings reaches 52.22 Mg/ha, outperforming the supervised baseline, and fcn_film on AEF embeddings augmented with land cover, topography, and geographic coordinates reaches 50.79 Mg/ha, the best overall result, with lower RMSE in every region under within-region, cross-region, and general protocols and with much smaller degradation when a one-year gap separates training and evaluation data. The paper further argues that the bulk of this advantage comes from rich multi-modal, multi-temporal input context and analysis-ready delivery shared by AEF and TESSERA, while AEF's edge over TESSERA (2.7 Mg/ha with matched heads) is best explained by its dense per-pixel reconstruction targets, which include GEDI LiDAR height metrics and L-band radar. A side result is that a model trained on AEF embeddings over only eleven regions achieves near-parity with the globally calibrated ESA CCI biomass product when both are scored against independent AGBref plots.","pith_inferences":["Editorial extension: if the delivery-mode explanation is right, benchmarks that evaluate GFMs only as frozen encoders will systematically undervalue the models for regression; future evaluation suites should include an embedding-product track, and model releases should ship both weights and embeddings.","Editorial extension: because AEF's reconstruction targets include GEDI height metrics and the benchmark labels are GEDI L4A biomass, one testable prediction follows—a multi-modal embedding product pre-trained with no GEDI-family or vegetation-structure targets would show a smaller margin over the supervised baseline; the current design cannot fully separate alignment from representation quality.","Editorial extension: the South Asia zero-shot failure suggests an operational checklist: before deploying an embedding-based model in a new region, compare the distribution of the target region in embedding space to the training coverage, and keep raw features when the target appears out-of-distribution.","Editorial extension: the paper's own temporal test only spans 2019–2020; a natural next experiment is to test embeddings on a year after a disturbance event such as fire, harvest, or storm, to see whether the multi-temporal context helps or hurts when land cover has changed."],"forward_implications":["A 10 m biomass map can now be produced by a lightweight MLP over AEF embeddings at a fraction of the compute of running a frozen encoder, and it will beat a supervised model trained on the full raw feature stack.","Weight-distributed GFMs need to become input-flexible—accepting L-band SAR, ancillary covariates, variable patch sizes, and documented preprocessing—before they can serve quantitative regression tasks.","Embedding-based models degrade far less when training and evaluation years differ, so they can be updated near-real-time without waiting for new labels.","Zero-shot geographic transfer works for embedding-based models in most carbon-dense regions (17–25% RMSE reduction in Africa and South America), but fails in ecologically distinct regions such as South Asia unless raw features are retained alongside the embeddings.","A model trained on a handful of regions with AEF embeddings matches a globally calibrated operational product on independent reference data, suggesting that representation quality can substitute for geographic coverage in training data."],"supporting_citations":[{"why":"Provides the AGBD dataset of about 16 million labelled patches and the fcn_film supervised baseline used throughout the study.","marker":"Sialelli et al. 2025"},{"why":"Defines AlphaEarth Foundations and its pre-computed embeddings; its pre-training targets include GEDI height metrics and L-band radar, which the paper identifies as the likely source of AEF's edge over TESSERA.","marker":"Brown et al. 2025"},{"why":"Supplies the TESSERA pre-computed embeddings, the second embedding product benchmarked against the supervised baseline.","marker":"Feng et al. 2026"},{"why":"Supplies the benchmarking framework and frozen-encoder evaluation protocol with a standard dense-prediction decoder used for the eleven weight-distributed GFMs.","marker":"Marsocci et al. 2024"},{"why":"Supplies the GEDI L4A footprint biomass labels that are the regression target of AGBD.","marker":"Dubayah et al. 2022b"},{"why":"Provides the ESA CCI biomass product used for the operational-product comparison.","marker":"Santoro and Cartus 2025"},{"why":"Provides the AGBref reference dataset on which the independent validation against AGBD and ESA CCI is performed.","marker":"Araza 2025"},{"why":"Documents how GEDI L4A biomass estimates are calibrated and where their biases and uncertainties lie, grounding the paper's caveat about label-dependent evaluation.","marker":"Duncanson et al. 2022"}],"fun_headline_variants":["Frozen AI encoders fail, embedding products win for biomass","For biomass, pre-computed embeddings outdo frozen models","Biomass mapping: pre-computed embeddings trump frozen AI","Geospatial AI: use embeddings, not frozen models, for biomass","Pre-computed embeddings beat frozen models on biomass task"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results assume that AEF's advantage is a fair measure of representation quality rather than an artifact of its pre-training having already been pointed at the same kind of signal being predicted: its training targets include GEDI height measurements, and the labels are GEDI biomass estimates, so the representation has effectively practiced encoding the very structure it is then asked to predict.","fun_headline_variants_meta":{"raw":{"variants":["Frozen AI encoders fail, embedding products win for biomass","For biomass, pre-computed embeddings outdo frozen models","Biomass mapping: pre-computed embeddings trump frozen AI","Geospatial AI: use embeddings, not frozen models, for biomass","Pre-computed embeddings beat frozen models on biomass task"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000531,"raw_usage":{"total_tokens":2687,"prompt_tokens":1204,"completion_tokens":1483,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":820,"completion_tokens_details":{"reasoning_tokens":1409}},"tokens_in":820,"tokens_out":1483,"duration_ms":11261,"temperature":1.0,"reasoning_tokens":1409,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:27:51.349737+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same linear-probe, MLP, and fcn_film heads on embeddings from a model with comparable multi-modal, multi-temporal pre-training but no GEDI-family or vegetation-structure reconstruction targets, and compare against the supervised baseline on the same AGBD test split. If the margin shrinks to near zero, the representation-quality reading fails; if the margin persists, the paper's delivery-mode conclusion survives. A complementary check is to fine-tune the best weight-distributed GFM with a dense per-pixel reconstruction objective over the same modalities and see whether it reaches AEF-level accuracy.","supporting_citations":[{"cited_title":"2025 , copyright =","cited_arxiv_id":null,"evidence_quote":"Provides the AGBref reference dataset on which the independent validation against AGBD and ESA CCI is performed."},{"cited_title":"and Armston, John and Dubayah, Ralph and Minor, David M","cited_arxiv_id":null,"evidence_quote":"Documents how GEDI L4A biomass estimates are calibrated and where their biases and uncertainties lie, grounding the paper's caveat about label-dependent evaluation."}],"review_version":1}