{"id":"3a2394ed-015b-48ed-b582-0fef6c4062fe","arxiv_id":"2504.12753","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"DepthForge fuses frozen DINOv2 or EVA02 visual features with frozen Depth Anything V2 depth features via depth-aware learnable tokens and a refinement decoder to improve domain-generalized semantic segmentation.","lead":"DepthForge adds depth estimates from a frozen AI model to visual features from another frozen AI model to help self-driving segmentation systems handle unseen night and snow scenes. The method reports state-of-the-art results on standard benchmarks, though the gains are modest and partly tuned on the test domains.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Core depth-quality premise is unverified on target domains, and Table 4's +DA+AO vs DepthForge gap suggests an undisclosed component, weakening the geometric-consistency attribution.","rationale":"The reader's weakest assumption is that Depth Anything V2 produces accurate depth on the unseen target domains, and I agree this is a load-bearing unverified premise. However, I find an even more concrete internal inconsistency in the ablation evidence: Table 4's '+DA+AO' row should, by the paper's own definition of DepthForge (DA+AO+DRD), be identical to the final method, yet there is a 3.2 mIoU gap on Snow. This suggests an undocumented component or target-set hyperparameter selection, which would undermine the paper's component attribution and potentially the reported extreme-condition gains. The proposed test directly checks the released code and the sensitivity to λ, settling whether the unexplained gap is real and whether the gains are due to geometric consistency or to target-aware tuning. The reader's verdict of CONDITIONAL remains appropriate: the paper's central idea is plausible, but the current evidence is not fully transparent. I therefore keep the verdict unchanged while strengthening the specific grounds for conditionality.","tokens_in":15028,"tokens_out":7958,"duration_ms":81201,"concrete_test":"Use the released code to reproduce Table 4's '+DA+AO' and 'DepthForge' configurations and diff the exact code/configs. If no diff exists, the discrepancy is a reporting error; if a diff exists, identify the missing component. Additionally, re-run the full method with λ (Eq. 5) selected on a source-only validation split (e.g., a held-out portion of Cityscapes) rather than on ACDC, and confirm the snow/night improvements (75.4/60.4 on ACDC) persist. This distinguishes a genuine geometric-consistency effect from target-domain selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DepthForge's gains come from geometric consistency requires that Depth Anything V2's depth features are informative on the unseen target domains, especially night and snow. The paper never measures depth quality on these domains (Sections 4, 5.3), so this premise is unverified. More concretely, the ablation table is internally inconsistent: Table 4 shows the row '+DA+AO' (which should be the full model since DepthForge is defined as DA+AO+DRD in Section 4) achieving 72.2 mIoU on Snow and 60.4 on Night, while the final 'DepthForge' row achieves 75.4 and 60.4. The Snow gap of 3.2 mIoU between a supposedly complete configuration and the final model is unexplained and is larger than any single component's contribution. This indicates either an untracked component (e.g., the scale factor λ in Eq. 5 or a fusion ε in Eq. 3) or selection of hyperparameters on the target test set. Without resolving this, the attribution of the extreme-condition gains to geometric consistency is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DepthForge, a fine-tuning framework for domain-generalized semantic segmentation (DGSS) that combines features from a frozen visual VFM (DINOv2 or EVA02) with features from a frozen depth VFM (Depth Anything V2). The method introduces depth-aware learnable tokens, an attention-optimization module, and a multi-scale depth refinement decoder, and it is evaluated under three DGSS protocols over Cityscapes, ACDC, BDD100k, Mapillary, and GTA5. The experiments report consistent mIoU gains over REIN and FADA across all settings, with the largest gains in extreme conditions, e.g., 75.4 vs. 70.6 on Snow and 60.4 vs. 55.9 on Night under Cityscapes-to-ACDC (Table 1). The abstract and introduction additionally claim an approximate +4% mIoU improvement in the GTA5-to-Cityscapes+BDD100k+Mapillary setting and roughly +5% in extreme scenes.","tokens_in":15315,"tokens_out":15906,"duration_ms":140462,"significance":"If the reported gains hold up, DepthForge is a useful contribution: it achieves consistent improvements over strong VFM-based DGSS baselines across three protocols and two visual backbones at constant parameter cost (2.99M trainable parameters, equal to REIN and well below FADA's 11.65M, Tables 6-7), and the comparison set is thorough, including ResNet, Transformer, and VFM baselines as well as PEFT methods such as LoRA, VPT, and AdaptFormer (Table 7). The direction of using frozen depth-VFM features as a geometric prior for DGSS is timely and the code is promised. The main caveats are attribution and reproducibility: the headline +4% is not supported by Table 3, the ablation record is internally inconsistent (Tables 4 and 5 report different numbers for the same configuration), and a key fusion parameter in Eq. (5) is never specified. These issues are fixable within the scope of the manuscript but need to be resolved before the geometric-consistency mechanism can be accepted as the cause of the gains.","major_comments":[{"comment":"The abstract and the introduction claim approximately +4% mIoU improvement in the GTA5-to-Cityscapes+BDD100k+Mapillary setting, but Table 3 shows average gains of 2.73 points over REIN (2.64, 2.42, and 3.12 per domain) and under 1.1 points over FADA; no single domain reaches +4%. The accompanying '+5% in extreme scenes' claim refers to the Cityscapes-to-ACDC setting in Table 1 (Snow +4.8, Night +4.5 over REIN), not to the GTA5 setting, so the introduction conflates two evaluation protocols, and it does not state which baseline the percentages are measured against. These claims should be revised to match the tables.","section":"Abstract, Section 1, Table 3"},{"comment":"The ablation table does not state whether its rows are cumulative additions to REIN or independent configurations. Under the natural cumulative reading, the final increment (+DA+AO to DepthForge) contributes 3.2 points on Snow and 3.0 points on Night, which are the two headline extreme-condition gains, whereas the depth-awareness module (+DA+AO over REIN) contributes only 1.5-1.7 points on those conditions. This is at odds with Section 5.4, which attributes 'an approximate 2% improvement' primarily to the depth-aware tokens and never discusses the decoder's dominant contribution to the extreme-condition gains. Additionally, the same configuration is reported with different numbers in Tables 4 and 5: '+concat f_d^i' is 70.8/56.1/79.4/72.8 in Table 4 but appears as 69.8/55.0/78.3/71.2 under 'Concatenation' in Table 5. The row semantics should be stated explicitly, the two tables reconciled, and the decomposition of the extreme-condition gains discussed.","section":"Table 4, Section 5.4"},{"comment":"The depth-attention weight lambda in Eq. (5) and the gating threshold for 'discarding high-weight features' in Section 4.2 are never assigned values or selection procedures in Section 5.2. Table 5 shows that removing the scale factor changes the average gain from 3.5 to 2.8 points, so this parameter is material to the result. Without reporting its value and the threshold, the central fusion mechanism is not reproducible, and one cannot rule out that these constants were selected on the target test domains. Please specify both, and state whether any hyperparameter was chosen using validation performance on the unseen target sets.","section":"Eq. (5), Section 4.2, Table 5, Section 5.2"},{"comment":"The paper's mechanism claim is that Depth Anything V2 supplies geometric consistency because 'the underlying geometry remains stable' across domains while visual cues fail, but no evidence is provided that the depth features are actually accurate or informative on the unseen target domains, particularly night and snow, where monocular depth estimates are known to degrade. The construction also assumes that the layer indices of the frozen visual and depth encoders align semantically, which is asserted without justification. Please add direct evidence, such as depth maps on target-domain samples, a quantitative depth-quality measure, or an ablation that perturbs or corrupts the depth features to verify sensitivity. Without this, the geometric-consistency attribution rests on an unverified premise.","section":"Sections 4 and 5.3"}],"minor_comments":[{"comment":"The notation in Eq. (4) is garbled: 'Hθ⟨(FΘv,Θd,θD(xi)),yi' should be a standard loss over the segmentation head, and the parameters θ_h and Hθ are used inconsistently.","section":"Eq. (4)"},{"comment":"The column header 'Rain Night Snow Rog' is scrambled and mistyped; the numbers are in Snow/Night/Fog/Rain order, and 'Rog' should be 'Fog'. The row labels (No Depth vs. REIN, Depth Token vs. the rows of Table 4) should also be harmonized with Table 4.","section":"Table 5"},{"comment":"The text reports GTA gains of 3.12% and 1.13% over REIN and FADA under the Cityscapes-to-BDD+Map+GTA setting, but Table 2 implies gains of 4.83 and 3.46 points (67.24 vs. 62.41 and 63.78); the printed numbers appear to be copied from Table 3 and should be corrected.","section":"Section 5.3, Table 2"},{"comment":"No variance or multiple-seed information is reported, and several gains over the second-best method are at or below one point (e.g., 0.07 on Mapillary in Table 2 and 1.0 on Fog in Table 1), so the claim of significant improvement needs error bars or repeated runs to be meaningful.","section":"Tables 1-3"},{"comment":"The Config.1/Config.2/Config.3 definitions in the Figure 2 caption should be aligned with the Config1/Config2/Config3 labels in Table 5 to avoid confusion about which configuration is which.","section":"Figure 2, Table 5"},{"comment":"There are several typos and inconsistencies: 'datsets' in the Abstract, 'optimaiton' in Section 4, duplicated references [39] and [46]/[47], inconsistent GitHub URLs between the abstract (anonymouse-xzrptkvyqc) and the body (SY-Ch), and 'qualitatively evaluating' in Section 4.1 should be 'quantitatively evaluating'.","section":"Throughout"},{"comment":"The claim of being 'the first depth-aware framework specifically designed for DGSS' is too strong given that Section 2.2 cites existing RGB-D domain-generalization works [19, 27, 38, 55]; the novelty claim should be scoped relative to that literature.","section":"Section 1, Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The main barrier to acceptance is the internal inconsistency between Tables 4 and 5, the ambiguity of the ablation row semantics, and the undisclosed value of the depth-attention weight lambda. If the authors clarify the row semantics, reconcile the conflicting numbers, report the hyperparameters, and correct the overclaimed +4% figure, the paper is likely acceptable. I would also ask the editor to have the 'first depth-aware DGSS framework' claim checked against the cited RGB-D DG literature during revision. The consistent gains across protocols and backbones, together with the parameter-efficient design, are genuine strengths."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a credible, consistently positive empirical result: frozen DINOv2/EVA02 plus frozen Depth Anything V2, with depth-aware learnable tokens and a refinement decoder, beats REIN and FADA on all three standard DGSS benchmarks, usually by 1–3 mIoU, at the same parameter budget as REIN. Second, the paper overstates its own numbers and makes a \"first\" claim that its own references contradict.\n\nWhat is new: the specific fusion machinery in Eqs. (3) and (5) is not identical to prior RGB-D domain generalization work, and the ablation table shows each added component buys something. The parameter efficiency is a nice point: same 2.99M trainable parameters as REIN with a consistent gain. That is a useful contribution to the DGSS subfield.\n\nSoft spots, in order of importance. The abstract's \"approximately +4%\" improvement is not in the tables; averaging the gains over the three settings gives roughly 2.7–3.5 points. The \"first depth-aware framework specifically designed for DGSS\" claim is contradicted by their own citations [19,27,38,55] to RGB-D domain generalization methods. The paper never measures whether Depth Anything V2's depth on night and snow actually is reliable; the geometric-consistency story rests on that assumption. And several hyperparameters—lambda in Eq. (5), the gating threshold in Section 4.2, the scale factor in Table 5—are not given enough detail and appear to be selected on the target test domains, a common DG pitfall.\n\nOne thing the stress-test note flags does not hold up: the \"+DA+AO\" row in Table 4 is not the full model; it is the configuration without the Depth Refinement Decoder. The gap to \"DepthForge\" is the DRD contribution, which the table already documents. So there is no hidden component there.\n\nBottom line: worth engaging with. The recipe is simple, reproducible-sounding, and gets steady gains across benchmarks. It deserves a serious referee, but the authors need to correct the +4% claim, soften the novelty claim, and ideally report depth feature quality on the target domains. I would bring it to a reading group and would cite it if I were working in this area.","headline":"Useful, incremental DGSS paper combining frozen visual and depth VFMs with learnable tokens; the +4% headline claim is inflated and depth quality on target domains is unverified, but the core recipe holds up.","tokens_in":15807,"tokens_out":3518,"would_cite":true,"duration_ms":32362,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DepthForge claims that fusing frozen depth features into frozen visual foundation models, through depth-aware learnable tokens, yields domain-generalized semantic segmentation that is stronger in normal scenes and dramatically better…","keywords":["domain generalized semantic segmentation","vision foundation models","depth-aware learnable tokens","geometric consistency","multi-modal fusion","adverse conditions","parameter-efficient fine-tuning","DINOv2"],"falsifier":"Compare DepthForge's mIoU on the Cityscapes to ACDC night and snow splits when the frozen depth features are replaced at test time by corrupted versions, such as Gaussian noise or a constant plane. If the segmentation barely degrades, the claimed geometric-consistency mechanism is not what produces the gain; if it degrades sharply, the gain is tied to depth quality as argued. A complementary check is to compute Depth Anything V2's depth error on ACDC night and snow images and see whether image-level gains correlate with depth accuracy.","tokens_in":14863,"feed_emoji":"🌨️","tokens_out":9145,"duration_ms":82074,"temperature":0.7,"pith_summary":"DepthForge sets out to prove that geometry, not pixels, is the ingredient that lets a segmentation model survive a domain shift. The paper argues that RGB cues are the first thing to fail in night, snow, fog, or rain, while relative depth stays roughly stable, so it builds a fine-tuning strategy that injects depth cues from a frozen Depth Anything V2 model into the frozen features of DINOv2 or EVA02 at every layer. The injection happens through depth-aware learnable tokens and an attention-bias term that adds a depth-based spatial relationship to the visual attention map. On the standard DGSS benchmarks, the method reports consistent gains over prior VFM-based methods, with the largest jumps under extreme conditions: 75.4 vs. 70.6 mIoU on snow and 60.4 vs. 55.9 at night in the Cityscapes-to-ACDC setting. If the claim holds, the practical payoff is that reliable segmentation under adverse weather can be obtained by reusing frozen foundation models plus a small set of tunable tokens, rather than retraining on target-domain data.","feed_headline":"Depth cues boost night and snow segmentation by ~5 points","feed_subtitle":"Frozen monocular depth stabilizes visual attention when RGB cues fail, lifting mIoU on the hardest unseen domains.","key_machinery":"The load-bearing mechanism is the depth-aware learnable token $T_i \\in \\mathbb{R}^{m\\times c}$ and the attention rule $A_i = \\mathrm{Softmax}(f^v_i (T^v_i)^\\top / \\sqrt{c}) + \\lambda \\mathrm{Softmax}(f^d_i (T^d_i)^\\top / \\sqrt{c})$, which adds a depth-derived spatial bias to the visual attention map so that pixels are tied to real-world locations even when RGB features are ambiguous. The layer update $f_{i+1} = W^v_i f_i + \\varepsilon(\\epsilon_v(T_i(W^v_i f^v_i)) + \\epsilon_d(T_i(W^d_i f^d_i)))$ lets each modality contribute independently through the same token set, preventing one cue from dominating. Around these two identities, DepthForge wraps an Attention Optimization module that rescales and gates the attention output, and a Depth Refinement Decoder that fuses multi-layer enhanced features through MLPs, concatenation, a convolution, and Transformer layers. The depth-aware token is what carries the paper's claim: it is the place where visual and depth cues are decoupled and recombined, and where the geometric-consistency bias enters the frozen VFM at every layer.","core_discovery":"The central claim is that a frozen depth vision foundation model can be used as a stable geometric prior for domain-generalized semantic segmentation, and that the correct way to combine it with a frozen visual VFM is per-layer, through learnable tokens that carry both visual and depth awareness. DepthForge updates the layer feature not by adding depth features directly, but by generating an enhanced feature from visual and depth tokens via Eq. (3), and it forges geometric consistency by adding a depth-attention bias to the visual attention map in Eq. (5). The paper shows that this combination outperforms simpler fusion strategies (concatenation, depth-prompts, direct token injection) and that the gains concentrate precisely where visual cues are weakest: snow, night, fog, and rain. The authors argue the resulting visual-spatial attention is steadier across domains, and they support that with t-SNE and affinity-map visualizations showing more separated class clusters and cleaner attention maps.","pith_inferences":["Beyond the paper: because the design leans on relative depth rather than absolute distance, any monocular depth model with reliable ordinal relationships (other depth VFMs, or even stereo or disparity networks) should be able to substitute for Depth Anything V2, which would make the recipe model-agnostic.","Beyond the paper: the same depth-aware token fusion could be transferred to other dense prediction tasks that suffer under domain shift, such as monocular depth estimation itself or panoptic and instance segmentation, since the mechanism only modifies how frozen features are combined, not the task head.","Beyond the paper: a natural stress test is to degrade or zero out the depth branch for a subset of target images; if the paper's story is right, the model's attention maps should become visibly less stable exactly on those images, linking the qualitative steadier-attention claim to a quantitative input.","Beyond the paper: because the authors never measure target-domain depth quality, the most useful follow-up is a calibration study that gates the depth bias $\\lambda$ when the depth model is uncertain; if the gate recovers performance when depth is poor, the approach becomes more robust than the fixed-bias version."],"forward_implications":["On Cityscapes to ACDC, DepthForge reports the best results among VFM-based DGSS methods on all four adverse conditions, with the largest margins on snow and night (+4.8 and +4.5 mIoU over REIN).","Across GTA5 to Cityscapes+BDD+Mapillary and Cityscapes to BDD+Mapillary+GTA5, DepthForge improves mIoU over REIN by roughly 2.4 to 3.1 points and over FADA by about 0.1 to 1.1 points while using the same or fewer trainable parameters.","The component ablations attribute the gain to the full DepthForge configuration (depth-aware tokens plus attention optimization plus refinement decoder); simply concatenating depth features or adding depth prompts gives little or no improvement.","The gains hold when the visual backbone is changed to EVA02, CLIP, or SAM, and relative-depth cues from Depth Anything V2 beat absolute-depth prompts from Prompt Depth Anything.","The per-layer token design means the same frozen VFMs can be reused for different segmentation tasks with only the small tunable token set and the head retrained."],"supporting_citations":[{"why":"Supplies the frozen depth model that provides the stable geometric prior at every layer.","marker":"[49, 50]"},{"why":"The frozen visual backbone whose layer features are enhanced by depth-aware tokens.","marker":"[30]"},{"why":"Alternative frozen visual backbone (EVA02) used to show the method is backbone-agnostic.","marker":"[15]"},{"why":"The token-based parameter-efficient fine-tuning mechanism this paper extends, and the main VFM baseline it must beat.","marker":"[44]"},{"why":"A recent VFM-based DGSS baseline whose results DepthForge compares against across all benchmarks.","marker":"[2]"},{"why":"The ACDC benchmark's night, snow, fog, and rain splits that define the extreme-condition claim.","marker":"[40]"},{"why":"The Cityscapes source domain used for the extreme-condition training setting.","marker":"[12]"}],"fun_headline_variants":["Depth priors forge steadier segmentation in snow and night","Frozen depth model steadies visual attention for domain shift","Per-layer depth tokens stabilize segmentation across domains","Geometry beats color: depth cues lift segmentation in fog"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's edge depends on the depth model producing trustworthy relative depth on the unseen target domains: if Depth Anything V2's estimates are noisy or wrong under night, fog, and snow, the spatial-consistency bias becomes noise and the advantage should shrink or vanish. The paper never measures depth quality on those target domains, so this premise is asserted rather than verified.","fun_headline_variants_meta":{"raw":{"variants":["Depth priors forge steadier segmentation in snow and night","Frozen depth model steadies visual attention for domain shift","Per-layer depth tokens stabilize segmentation across domains","Geometry beats color: depth cues lift segmentation in fog"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000272,"raw_usage":{"total_tokens":1659,"prompt_tokens":1002,"completion_tokens":657,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":594}},"tokens_in":618,"tokens_out":657,"duration_ms":7348,"temperature":1.0,"reasoning_tokens":594,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:22:55.934217+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare DepthForge's mIoU on the Cityscapes to ACDC night and snow splits when the frozen depth features are replaced at test time by corrupted versions, such as Gaussian noise or a constant plane. If the segmentation barely degrades, the claimed geometric-consistency mechanism is not what produces the gain; if it degrades sharply, the gain is tied to depth quality as argued. A complementary check is to compute Depth Anything V2's depth error on ACDC night and snow images and see whether image-level gains correlate with depth accuracy.","supporting_citations":[{"cited_title":"Stronger fewer & superior: Harnessing vision foundation models for domain generalized semantic segmentation","cited_arxiv_id":null,"evidence_quote":"The token-based parameter-efficient fine-tuning mechanism this paper extends, and the main VFM baseline it must beat."},{"cited_title":"Learning frequency- adapted vision foundation model for domain generalized se- mantic segmentation","cited_arxiv_id":null,"evidence_quote":"A recent VFM-based DGSS baseline whose results DepthForge compares against across all benchmarks."},{"cited_title":"Acdc: The adverse conditions dataset with correspondences for se- mantic driving scene understanding","cited_arxiv_id":null,"evidence_quote":"The ACDC benchmark's night, snow, fog, and rain splits that define the extreme-condition claim."}],"review_version":1}