{"id":"87e37047-60db-40e8-94b4-a048d8f5ffb8","arxiv_id":"2504.19742","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper introduces EcoWikiRS, a dataset of aerial images paired with GBIF species observations and Wikipedia habitat text, and WINCEL, a weighted InfoNCE loss that improves zero-shot EUNIS ecosystem classification.","lead":"Researchers built a dataset that links Swiss aerial photos with citizen-science species observations and Wikipedia habitat descriptions, and a new loss function (WINCEL) that makes vision-language models focus on the most relevant text. Fine-tuning with WINCEL improved zero-shot ecosystem type classification over standard contrastive learning on three of four models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"WINCEL's sentence-selection mechanism is not validated: the loss computes weights with the same model it trains, and the paper provides only qualitative evidence that selected sentences are visually relevant. A quantitative relevance check is needed.","rationale":"The reader's weakest assumption is that each image has at least one relevant sentence; my concern sharpens this into a specific, testable claim about WINCEL's weighting mechanism. The target-leakage issue between GBIF observations and EUNIS labels is plausible but would inflate both InfoNCE and WINCEL similarly, so it does not explain the method's relative advantage. The distinguishing contribution of the paper is WINCEL, and its behavior is the load-bearing point: the paper's own evidence for relevance selection is qualitative, and the loss's self-referential weighting could produce gains without genuine ecological grounding. A quantitative relevance evaluation would settle whether the proposed mechanism actually works as described. Since the reader already issued a conditional verdict centered on supervision quality and evaluation rigor, my concern does not move the verdict; it reinforces the need for the conditional acceptance criteria. The empirical comparisons in Table 1 are internally consistent, and the ablations in Table 3 are informative, so no rejection is warranted. The proposed annotator study is feasible with a modest sample and would directly test the mechanism's core assumption.","tokens_in":18391,"tokens_out":4821,"duration_ms":55018,"concrete_test":"Randomly sample 300 images from the EcoWikiRS test split, stratified by EUNIS class. For each image, extract its K=15 habitat sentences. Have three annotators (or one expert ecologist) label each sentence as visually relevant to the image or not, using only the image and the sentence. Compute: (i) the fraction of images with at least one relevant sentence; (ii) the precision@1 of WINCEL's highest-weight sentence versus a random sentence and versus the pretrained model's highest-similarity sentence; (iii) an oracle upper bound trained only on sentences labeled relevant. If the fraction in (i) is low (e.g., below 50%) or WINCEL's precision@1 is not significantly above random, the core mechanism is unsupported and the improvement should be attributed to a different effect.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4 defines the weighting α_{n,k} = σ(V_n·T_{n,k}/τ) and the weighted text representation G_n = Σ_k α_{n,k}·T_{n,k} (Eqs. 2–4), then optimizes WINCEL (Eq. 5) to pull V_n toward G_n. Because α is computed from the very model being trained, this is a soft self-training loop: any pretrained-encoder bias in cross-modal similarity is reinforced, and there is no ground-truth relevance signal. The paper states the assumption that at least one of K sentences per image is visually pertinent (Section 4), but never measures how often it holds. The only evidence that WINCEL selects relevant sentences is qualitative—Figures 4 and 9 show a handful of examples where fine-tuned scores rank ecological sentences above generic ones; no precision/recall of the selected sentences is reported. If WINCEL's gain over InfoNCE stems from a soft-averaging or regularization effect rather than from identifying visually grounded habitat text, then the central claim of integrating ecological knowledge is not established, and the mechanism may fail on datasets where no relevant sentence exists for many images.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces EcoWikiRS, a dataset pairing 91,801 high-resolution aerial images (100 m × 100 m, 50 cm resolution, Switzerland) with geolocated GBIF species observations and filtered Wikipedia habitat sentences describing the observed species. It proposes WINCEL, a weighted variant of InfoNCE, in which each image is contrasted against a convex combination of the K associated sentence embeddings, with weights computed from the model's own image–sentence similarities. The authors fine-tune four pretrained vision–language models (CLIP, GeoRSCLIP, RemoteCLIP, SkyCLIP) on this dataset and evaluate zero-shot EUNIS ecosystem type classification. They report that WINCEL outperforms InfoNCE fine-tuning on three of four backbones (CLIP 30.9% vs 25.3% OA; SkyCLIP 30.1% vs 27.1%; GeoRSCLIP 29.5% vs 27.6%), while RemoteCLIP degrades (20.9% vs 22.2%). Ablations compare text source types, fine-tuning strategies, and alternative noise-robust losses (bootstrap, sampling, substring augmentation). Qualitative visualizations show improved cross-modal similarity maps over Switzerland and example sentence-selection behavior.","tokens_in":18587,"tokens_out":4456,"duration_ms":45978,"significance":"If the central claims hold, the paper offers a scalable weak-supervision source for remote sensing vision–language models in ecology, moving beyond land-cover labels toward habitat and ecosystem semantics. The dataset and code are released, which is a concrete community resource. The evaluation on EUNIS, a standardized European habitat taxonomy, is a sensible benchmark, and the comparison across four backbones with reported means and standard deviations strengthens the empirical case. However, two load-bearing aspects need closer scrutiny: (i) WINCEL's sentence-weighting mechanism is a soft self-training loop that lacks external validation, and (ii) the EUNIS downstream task is constructed from plant species communities, the same ecological signal that supplies the training text, so the reported gains may reflect recovery of species-associated text patterns rather than a general 'ecological understanding' of imagery. The paper's contribution is valuable, but the mechanism claim and the generality of the ecological representation need additional evidence before the results can be fully accepted.","major_comments":[{"comment":"The WINCEL weighting α_{n,k} = σ(V_n·T_{n,k}/τ) is computed from the model's own image–text similarity and then used to construct the target G_n that the same model is trained to align with. This is a soft self-training loop: the model's prior biases in cross-modal similarity are reinforced, and there is no external relevance signal. The paper's claim that WINCEL 'learn[s] to recognize text passages that are relevant to the images' (Figure 1 caption) is therefore not established by the current experiments, which provide only qualitative examples (Figures 4 and 9). Please add a quantitative validation of the selection mechanism, for example: (a) measure precision/recall or rank correlation of the learned weights against human annotations of sentence relevance on a held-out subset; (b) compare WINCEL against an 'oracle-weighted' variant that uses the true EUNIS class to select the most relevant sentence; or (c) show that WINCEL's improvement over InfoNCE persists when the model is initialized with a different pretrained encoder whose similarity rankings are decorrelated. Without such evidence, the gain could be attributed to soft averaging or regularization rather than to identifying visually grounded habitat text.","section":"Section 3.2 and Section 6.1"},{"comment":"The EUNIS ground-truth map is constructed from plant species communities (Section 3.2), and the training signal in EcoWikiRS is also derived from species observations (GBIF) and Wikipedia habitat descriptions of those species. The zero-shot EUNIS task thus measures the model's ability to map aerial images to species-derived ecological descriptions—essentially the same semantic axis as the training text. This does not invalidate the positive results, but it weakens the conclusion that the model has learned a general 'ecologically meaningful' representation. To support the generalization claim, please add at least one downstream evaluation whose labels are not directly generated from species occurrences, such as independent land-cover mapping (e.g., from authoritative land-cover products), or prediction of climate/soil attributes from held-out stations. Alternatively, explicitly reframe the contribution as recovering species-habitat associations from imagery, and temper the ecological-understanding language accordingly.","section":"Section 4, hypothesis statement"},{"comment":"The method's key assumption—'at least one sentence per article is likely to describe the content of the image'—is load-bearing but never measured. If for many images the sentence set J_n contains no visually pertinent sentence (e.g., because the observed species is a generalist, the species is not visible from above, or the image depicts a habitat that does not match the article text), WINCEL's softmax weighting has no correct sentence to select, and the learned alignment cannot recover the ecological signal. The paper should quantify how often a relevant sentence exists, for example by checking whether the top-weighted sentence under a trained WINCEL model matches the EUNIS habitat description of the image on a validation subset, or by sampling a small set of images and having annotators judge whether at least one of the K sentences is visually relevant. This would also help interpret the RemoteCLIP result, where WINCEL underperforms InfoNCE, perhaps because the pretrained encoder's similarity rankings are too poor to provide useful weights.","section":"Section 4"}],"minor_comments":[{"comment":"There is a typo: 'such as such as “sandy”, “urban”, “acidic”' should be 'such as “sandy”, “urban”, “acidic”'.","section":"Section 6.3"},{"comment":"Section 3.1 describes swissIMAGE as 'openly available', while Section 6.4 states that 'the lack of freely available high-resolution aerial imagery hinders the direct generalization'. Please reconcile these statements; if the imagery is openly available but not freely redistributable, say so explicitly.","section":"Section 3.1 vs Section 6.4"},{"comment":"In Eq. (4), the summation index is written as i in the first factor (σ(V_n·T_{n,k}/τ)) and k in the second factor (T_{n,k}); this should be a single consistent index, presumably k.","section":"Equation (4)"},{"comment":"The supplementary material alternates between the names 'EcoWikiRS' and 'WikiRS' (e.g., Table 5, Table 7). Please standardize the dataset name throughout.","section":"Supplementary, Tables 5 and 7"},{"comment":"The temperature hyperparameters are set to 0.07 for InfoNCE and 0.15 for WINCEL after separate grid searches. Since temperature directly controls the softmax sharpness in both the contrastive loss and the weighting, it would be informative to report a sensitivity analysis over τ for both methods, or at least to state the ranges searched, to ensure the comparison is not driven by this hyperparameter.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the empirical results are interesting, but the WINCEL mechanism needs quantitative validation and the EUNIS evaluation has a potential circularity with the training signal. These are fixable within the manuscript's scope. I would support acceptance after a revision that adds the proposed analyses. The release of code and dataset is a definite strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: this paper is worth a serious look if you work on remote-sensing vision-language models or ecological mapping. The EcoWikiRS dataset is the main deliverable: 91k very high-resolution aerial images over Switzerland, each linked via GBIF to Wikipedia habitat sentences of co-located species. That is a genuinely useful resource, and the construction is careful: spatial block split, GBIF uncertainty filtering, and a sensible habitat-sentence extraction step. The WINCEL loss is a simple weighted InfoNCE that averages sentence embeddings with weights from current image-text similarity. The empirical claim holds up: fine-tuning on EcoWikiRS improves zero-shot EUNIS classification over pretrained baselines on all four backbones, and WINCEL beats InfoNCE on three of them, with non-overlapping standard deviations. That is real evidence, especially since they compare against bootstrapping baselines and report standard deviations over five runs.\n\nThe soft spots, in proportion. The stress-test concern is valid but not fatal. The weighting is computed with the same model being trained, so it is a soft self-training loop. The paper never measures how often the assumption \"at least one Wikipedia sentence per image is visually pertinent\" actually holds; the only support is qualitative (Figures 4, 9). So the mechanistic story that WINCEL learns to pick ecologically relevant sentences is under-supported. The gain could come partly from soft averaging or regularization. That does not kill the central claim, because the evaluation is external, but a quantitative relevance check is the clear next step. The EUNIS evaluation may be optimistically biased: the EUNIS map is built from plant species communities, and the training text comes from exactly those species' Wikipedia descriptions, so vocabulary overlap between training sentences and class prompts is high. The spatial block split prevents location leakage but not semantic label leakage. An analysis of how directly the habitat sentences match EUNIS class definitions would clarify this; it is a genuine confound, but not a dealbreaker, because the model still has to visually identify the habitat from the image. Finally, absolute performance is modest (macro F1 around 20% on 25 classes), which the paper honestly acknowledges; the supervised upper bound is 39.8%, so there is headroom.\n\nThe paper is honest about limitations and gives enough detail to reproduce the pipeline. It is a solid engineering-and-data contribution for a niche subfield, and it deserves a serious referee. I would accept it for review and push for the relevance analysis and the leakage discussion.","headline":"Solid dataset-plus-loss paper for RS ecology: the empirical gain is real but modest, and the mechanistic story is weaker than the headline.","tokens_in":19160,"tokens_out":3042,"would_cite":true,"duration_ms":29321,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that fine-tuning remote sensing vision-language models on aerial images paired with co-located species observations and Wikipedia habitat text, using a weighted contrastive loss, improves zero-shot ecosystem…","keywords":["remote sensing","vision-language models","weak supervision","contrastive learning","species observations","Wikipedia habitat text","EUNIS ecosystem classification","zero-shot classification"],"falsifier":"Take a held-out set of images and inspect the single sentence WINCEL gives the highest weight: if, for most images, that sentence's habitat keywords do not match the ground-truth EUNIS ecosystem type while overall accuracy stays high, the gain is coming from the text set rather than from genuine image-text alignment. A sharper version is to train on images whose paired sentences are swapped across unrelated habitats; if zero-shot accuracy is unchanged, the pairing mechanism is not doing the work.","tokens_in":18164,"feed_emoji":"🛰️","tokens_out":6228,"duration_ms":59670,"temperature":0.7,"pith_summary":"The paper aims to show that the ecological character of a landscape can be read from aerial imagery without hand-labeled ecological maps, by using species observations as the link: where a species was recorded, the image of that 100-meter patch is paired with sentences from the species' Wikipedia habitat description. Because most of those sentences describe the species generally rather than the particular patch, the supervision is weak and noisy; the paper's proposed WINCEL loss is a weighted InfoNCE that lets the model itself concentrate on the few relevant sentences. On zero-shot classification of 25 EUNIS ecosystem types in Switzerland, this raises CLIP's overall accuracy from 14.7% to 30.9% and SkyCLIP's from 19.2% to 30.1%, exceeding standard InfoNCE fine-tuning. A sympathetic reader would care because, if the claim holds, text written by naturalists becomes a scalable source of ecological supervision for overhead imagery.","feed_headline":"Weighted loss lifts zero-shot ecosystem mapping from 14.7% to 30.9%","feed_subtitle":"Species observations and Wikipedia habitat text teach remote-sensing models to recognize ecosystems from aerial images.","key_machinery":"The central object is WINCEL, a weighted version of the InfoNCE contrastive loss. For an image $I_n$, the visual encoder produces embedding $V_n$ and the text encoder produces $K$ sentence embeddings $T_{n,k}$; each sentence receives weight $\\alpha_{n,k}=\\sigma(V_n\\cdot T_{n,k}/\\tau)$, the softmax of image-sentence similarity over the $K$ sentences, and the text representation is $G_n=\\sum_k \\alpha_{n,k}T_{n,k}$. The loss is then the InfoNCE loss between $V_n$ and $G_n$. This works as a form of text-space interpolation, analogous to mixup, in which the model's own relevance estimates decide how much each sentence contributes, thereby down-weighting false positives and, by pulling text closer to images, reducing false negatives in the batch.","core_discovery":"The paper's central claim is that a pretrained vision-language model, fine-tuned on the EcoWikiRS dataset with WINCEL, learns an embedding in which aerial images are aligned with ecologically pertinent text rather than with the full noisy set of Wikipedia sentences attached to a location. The discovery is that a softmax-weighted mixture of the K sentence embeddings, with weights set by the model's current image-text similarity, can extract the ecological signal from weak supervision: the paper starts from the hypothesis that at least one sentence per species article describes the image content, and WINCEL learns to select it during training. The reported evidence is the zero-shot EUNIS ecosystem classification performance, where fine-tuning with WINCEL improves all pretrained backbones tested and beats InfoNCE fine-tuning on three of four backbones.","pith_inferences":["If the alignment transfers beyond Switzerland, the same recipe of co-located species observations plus Wikipedia habitat text could produce continent-scale ecological maps from freely available coarser satellite imagery; the paper itself flags coarse-resolution free imagery as a scaling bottleneck, so this is an extension rather than a result.","A per-species reliability prior could strengthen the weighting: species with broad habitat ranges, the generalists, are exactly where the at-least-one-relevant-sentence assumption is weakest, and the paper's blackbird examples show the model's margin there is thin.","Because WINCEL's weights depend on the model's current similarity estimates, an iterative or self-distilled version might push accuracy further by re-weighting with an improved encoder; the paper evaluates a single pass."],"forward_implications":["Zero-shot ecosystem mapping becomes feasible from high-resolution aerial images alone, since after fine-tuning the model can be prompted with EUNIS habitat names without any supervised training on ecosystem labels.","General-purpose CLIP benefits most from this fine-tuning, indicating that ecological semantics can be added to a model that was never pretrained on remote sensing imagery.","The type of input text matters: sentences extracted from Wikipedia habitat sections outperform keyword-filtered, random, and species-name texts, so the dataset construction protocol is part of the method's effectiveness.","WINCEL also changes what the model judges relevant: fine-tuning shifts top-ranked sentences toward ecologically specific descriptions such as soils, urban noise, and alpine zones, and gives negative scores to irrelevant passages, which supports text-based interpretation and retrieval from aerial images."],"supporting_citations":[{"why":"Supplies the InfoNCE contrastive objective that WINCEL weights.","marker":"[37]"},{"why":"Provides the CLIP pretrained backbone and the standard contrastive image-text alignment setup that the paper fine-tunes.","marker":"[39]"},{"why":"Motivates weighting samples by estimated noise level in contrastive learning, the starting point for WINCEL's relevance weighting.","marker":"[34]"},{"why":"Contributes the idea of learning from sentences relevant to visual content rather than full texts, which underlies the habitat-sentence selection.","marker":"[38]"},{"why":"Closest prior work aligning satellite images with species descriptions; EcoWikiRS extends it to very high resolution imagery and filtered ecological sentences.","marker":"[10]"},{"why":"Provides the SkyCLIP backbone and its pretraining dataset, used both as a baseline and as the model for the main ablations.","marker":"[52]"},{"why":"Source of the geolocated species observations that define which Wikipedia sentences are paired with which aerial image.","marker":"[43]"},{"why":"Defines the EUNIS habitat classification used as the zero-shot downstream evaluation task.","marker":"[9]"},{"why":"Provides the Ecosystem Type Map v3.1 layer from which the EUNIS labels and image tiling grid are taken.","marker":"[54]"}],"fun_headline_variants":["Weighted loss doubles zero-shot ecosystem mapping accuracy","EcoWikiRS: Weak supervision from species and Wikipedia maps ecosystems better","WINCEL loss selects habitat text, lifting zero-shot ecosystem classification","From 14.7% to 30.9%: Weighted InfoNCE improves ecosystem zero-shot mapping"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that for each aerial image, at least one of the Wikipedia sentences attached through co-located species observations actually describes what is visible in that specific 100 by 100 meter image; if most sentences are visually irrelevant, WINCEL's weighting has no correct sentence to select and the learned alignment cannot recover the ecological signal.","fun_headline_variants_meta":{"raw":{"variants":["Weighted loss doubles zero-shot ecosystem mapping accuracy","EcoWikiRS: Weak supervision from species and Wikipedia maps ecosystems better","WINCEL loss selects habitat text, lifting zero-shot ecosystem classification","From 14.7% to 30.9%: Weighted InfoNCE improves ecosystem zero-shot mapping"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00075,"raw_usage":{"total_tokens":3328,"prompt_tokens":924,"completion_tokens":2404,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":2322}},"tokens_in":540,"tokens_out":2404,"duration_ms":18935,"temperature":1.0,"reasoning_tokens":2322,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:44:20.437697+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of images and inspect the single sentence WINCEL gives the highest weight: if, for most images, that sentence's habitat keywords do not match the ground-truth EUNIS ecosystem type while overall accuracy stays high, the gain is coming from the text set rather than from genuine image-text alignment. A sharper version is to train on images whose paired sentences are swapped across unrelated habitats; if zero-shot accuracy is unchanged, the pairing mechanism is not doing the work.","supporting_citations":[{"cited_title":"Eunis habitat classification: Expert system, character- istic species combinations and distribution maps of european habitats","cited_arxiv_id":null,"evidence_quote":"Defines the EUNIS habitat classification used as the zero-shot downstream evaluation task."},{"cited_title":"Learning transferable visual models from natural language supervi- sion","cited_arxiv_id":null,"evidence_quote":"Provides the CLIP pretrained backbone and the standard contrastive image-text alignment setup that the paper fine-tunes."},{"cited_title":"Ro- bust audio-visual instance discrimination","cited_arxiv_id":null,"evidence_quote":"Motivates weighting samples by estimated noise level in contrastive learning, the starting point for WINCEL's relevance weighting."},{"cited_title":"ZEST: Zero-shot Learning from Text Descrip- tions using Textual Similarity and Visual Summarization","cited_arxiv_id":null,"evidence_quote":"Contributes the idea of learning from sentences relevant to visual content rather than full texts, which underlies the habitat-sentence selection."},{"cited_title":"SkyScript: A Large and Seman- tically Diverse Vision-Language Dataset for Remote Sens- ing","cited_arxiv_id":null,"evidence_quote":"Provides the SkyCLIP backbone and its pretraining dataset, used both as a baseline and as the model for the main ablations."},{"cited_title":"The gbif integrated publishing toolkit: facilitating the efficient publishing of biodiversity data on the internet","cited_arxiv_id":null,"evidence_quote":"Source of the geolocated species observations that define which Wikipedia sentences are paired with which aerial image."},{"cited_title":"Ecosystem type map v3","cited_arxiv_id":null,"evidence_quote":"Provides the Ecosystem Type Map v3.1 layer from which the EUNIS labels and image tiling grid are taken."}],"review_version":1}