{"id":"91a0fd2f-2b0f-4527-8cb6-352f3e3c194b","arxiv_id":"2501.12824","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Training a frozen DINOv2 backbone with a shared decoder on auxiliary multi-label dense classification improves monocular depth accuracy by about 11 percent on in-domain benchmarks.","lead":"Monocular depth estimation gets more accurate when the model is trained jointly on depth labels and simpler auxiliary tasks such as scene classification. The method lifts depth accuracy by about 11 percent on average and keeps quality while using 80 to 99 percent less depth-labeled data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains confound auxiliary-task signal with a doubled optimization budget; no same-update-count baseline is reported.","rationale":"The reader's conditional verdict is appropriate, but the weakest assumption they identified (best-task-per-dataset selection in Table 3) is not the most load-bearing risk. A fixed MLDC task still improves all six in-domain datasets, so the average gain is not merely an upper envelope over task choices. The more decisive gap is the missing control for optimization budget: the proposed method performs two gradient updates per global step, while the baseline performs one. Because the paper's contribution is specifically that auxiliary tasks, not extra training or data, drive the improvement, the absence of a same-update-count MDE-only baseline leaves an alternative explanation open. The paper does include useful controls (LR scaling in Sec. 5.1, task comparisons in Table 3, single-source ablation in Appendix C.2), but none of these equates the number of gradient updates or data batches seen by the baseline and the proposed method. A single control experiment with a 2x-step baseline would settle this directly. I do not recommend changing the reader's conditional verdict; the concern should be added as a condition for acceptance, together with the code-release condition the reader already identified.","tokens_in":16204,"tokens_out":7953,"duration_ms":95640,"concrete_test":"Train the DINOv2-only baseline on NYUv2 for 76,800 optimizer updates (2x the 38,400 steps used in Sec. 4 and Appendix A), using the same cosine schedule, batch size, seeds, and evaluation protocol, and compare AbsRel to the proposed method's Table 3 MLDC result of 696 +/- 3 (x10^-4). Run a second control with two MDE mini-batches per step instead of one MDE plus one auxiliary mini-batch, to isolate the effect of duplicated updates from the effect of auxiliary data. If the doubled baseline reaches roughly 696e-4, the reported gains and the 80-99% data-efficiency claim cannot be attributed to auxiliary task information; if it stays near the baseline's 809 +/- 10e-4, this concern is resolved and the auxiliary interpretation is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the ~11% average AbsRel gain comes from auxiliary-task information. But as defined in Sec. 3, Eqs. (4)-(5), one 'global' training step of the proposed method consists of two consecutive optimizer steps: an MDE update with learning rate alpha*eta, then an auxiliary update with (1-alpha)*eta. With alpha=0.9 and Sec. A's '38400 steps', the proposed model performs 38400 MDE updates plus 38400 auxiliary updates, i.e., roughly twice as many total gradient updates as the DINOv2 baseline, which is trained for 38400 single-task steps. The paper states that the total learning rate per global step is comparable, but that does not control the number of updates, the number of data batches seen, or the effect of a second optimizer with its own AdamW state and weight decay. The ablations in Sec. 5.1 vary the learning-rate scale and beta, and the task-type comparisons in Table 3 vary the auxiliary loss, but there is no control where the baseline is trained for twice as many MDE-only updates. Without that control, the improvement could be partly or wholly due to additional optimization, extra data batches, or doubled optimizer state, rather than to the auxiliary task itself. This is load-bearing because the paper's novelty and its statement that gains are 'not achieved by merely adding data' depend on isolating the auxiliary signal. The reader's concern about per-dataset best-task selection is real but secondary: even a fixed MLDC classification task improves all six in-domain datasets in Table 3, so the central claim does not collapse solely because of task selection.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an alternating training scheme for monocular depth estimation (MDE) in which a frozen DINOv2 ViT-G backbone with a shared DPT decoder is trained on MDE batches and on batches from auxiliary vision tasks (semantic segmentation, multi-label dense classification (MLDC), or image reconstruction), with the MDE update weighted by a task-focusing parameter α and the auxiliary update by 1−α. The method is evaluated on six in-domain MDE datasets using MIX6 (a union of six auxiliary datasets) and compared against the DINOv2 zero-shot baseline. The authors report an average ~11% AbsRel improvement when selecting the best auxiliary task per dataset, show that the same scheme preserves or improves quality when the MDE training set is reduced by 80–99%, and provide an additional validation with a Depth Anything backbone. They also claim that the gains come from the auxiliary task rather than from simply adding data, based on comparisons across tasks on the same auxiliary data.","tokens_in":16492,"tokens_out":8262,"duration_ms":81923,"significance":"If the central effect is real, this is a useful practical result: it shows that a frozen foundation model's decoder can be improved by auxiliary dense classification datasets without fine-tuning the backbone, and it identifies MLDC as an often-effective auxiliary task. The empirical effort is substantial: six in-domain MDE datasets, three auxiliary tasks, four independent runs per condition with standard errors, and ablations over the task-focusing parameter, learning-rate scaling, and backbone choice. Code release is promised upon acceptance. However, the missing control for the doubled optimization budget and the post-hoc per-dataset task selection mean that the magnitude and attribution of the headline gain are not yet established. These issues are fixable with additional experiments and a re-analysis of the reported averages.","major_comments":[{"comment":"The proposed global step consists of two consecutive optimizer updates: one MDE update at learning rate αη and one auxiliary update at (1−α)η, whereas the DINOv2 baseline is trained for 38400 single-update steps. Thus the proposed method performs 38400 MDE updates plus 38400 auxiliary updates on the shared decoder, i.e., twice as many total gradient updates, twice as many data batches, and a second optimizer/AdamW state. The ablations in Section 5.1 (Figure 5) vary the learning-rate scaling γ and the auxiliary scaling β, but they never include a baseline trained for the same number of total updates using only MDE data (e.g., 76800 MDE-only steps, or two MDE updates per global step). Without this control, the observed ~11% gain cannot be attributed solely to the auxiliary task; it may be partly a consequence of extra optimization steps or data presentations. This is load-bearing for the paper's central claim that improvements are not achieved by merely adding data and that the gains come from the selected auxiliary task.","section":"Section 3, Eqs. (4)-(5); Appendix A"},{"comment":"The reported ~11% average gain is computed by selecting, for each MDE dataset, the best-performing auxiliary task among classification, segmentation, and reconstruction. The paper provides no rule for choosing the auxiliary task for a new dataset, and some fixed choices are much worse than the best (e.g., reconstruction on NYUv2 is worse than baseline). The 'consistent improvement' claim therefore applies to a post-hoc per-dataset oracle, not to a fixed procedure. I recommend reporting averages for each fixed auxiliary task; for instance, using MLDC fixed across all six in-domain datasets yields roughly a 9.5% average gain, which is still positive but lower than the headline 11%. The 11% figure should be described as an upper envelope over task choices, and any claim of consistency should be tied to a predetermined task-selection rule.","section":"Table 3; Section 4.1"},{"comment":"The data-efficiency claim inherits the same optimization-budget confound. When comparing proposed models trained on 1–20% of the MDE data against the full-data DINOv2 baseline, the proposed models receive auxiliary updates in addition to their MDE updates. To support the '80–99% less data' statement, the comparison should include a control that uses the same total number of gradient updates on MDE data only (for example, a baseline trained for twice as many MDE-only steps, or a reduced-data baseline with the same total update budget). Without this control, part of the apparent data-efficiency gain could again be an artifact of the doubled update count rather than of the auxiliary task information.","section":"Section 4.2; Figure 4"}],"minor_comments":[{"comment":"The gradient notation ∇_{θ_{t-1}, φ_{t-1}} suggests an update to both θ and φ, but the equation only updates θ. Please write the θ-component explicitly (e.g., ∇_θ L_D) or clarify that the φ-component is handled by the separate task-head learning rate.","section":"Section 3, Eqs. (4)-(5)"},{"comment":"The phrase 'without decreasing the performance' is ambiguous: it should explicitly state that the reference is the full-data DINOv2 baseline, not the reduced-data baseline.","section":"Section 4.2"},{"comment":"The figure caption does not state that error bars represent standard errors; please add this for consistency with the rest of the paper.","section":"Figure 4"},{"comment":"The pseudo-labeling procedure for Taskonomy, NYUv2, Matterport3D, and DIODE is described only by reference to PolyMax; please specify the model used to generate the pseudo labels and any preprocessing, since the quality of these labels directly affects the single-source ablation.","section":"Appendix C.2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid experimental backbone and the auxiliary-task idea is plausible, but the missing same-update-count baseline is a serious gap for the paper's central attribution claim. I would encourage the editor to require this control experiment in revision, along with fixed-task averages for the headline gain. The per-dataset best-task selection is a secondary but important reporting issue. With these additions, the paper could be a useful contribution to the MDE literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the quick read on arXiv:2501.12824. The paper proposes alternating MDE and auxiliary-task training on a frozen DINOv2 ViT-G with a shared DPT decoder, and reports ~11% average AbsRel improvement on six in-domain datasets plus large data-efficiency gains. The genuinely new piece is Multi-Label Dense Classification (MLDC): converting segmentation labels to per-image multi-label vectors and using BCE. The comparison across tasks – MLDC, segmentation, reconstruction – on six datasets with four random seeds and standard errors is careful and useful. I believe the finding that MLDC consistently beats dense segmentation is real and worth knowing.\n\nThe soft spot is the one the stress-test flags. Each 'global step' of the proposed method is two optimizer steps: one depth update at alpha*LR, then one auxiliary update at (1-alpha)*LR. The baseline gets 38,400 MDE-only updates; the proposed method gets 38,400 MDE updates plus 38,400 auxiliary updates. The paper argues the total learning rate per global step is comparable, but that doesn't control for the number of updates, the amount of data seen, or the extra AdamW state. No baseline is trained with twice as many MDE-only updates, or with meaningless auxiliary labels. Without that control, the claim that gains come from auxiliary-task information rather than from additional optimization is not established.\n\nThat said, the paper is not refuted. The reconstruction task performs no better than baseline on some datasets, which would be strange if extra optimization alone were the driver. But the confounding is real and load-bearing because the paper's own conclusion asserts improvements are 'due to the selected auxiliary task, not only due to adding more data.' A referee should demand the missing control.\n\nA secondary issue: the headline 11% is the best task per dataset; the paper doesn't give a rule for choosing the task. The reader's note is right that MLDC alone improves all six in-domain datasets, so this is not fatal, but the claim as written oversells a procedure that has not been fully specified.\n\nThe abstract says code is available, but the reproducibility statement says it will be published upon acceptance. That needs reconciling. Otherwise, the experimental hygiene is good: multiple runs, standard errors, ablations, and a validation with Depth Anything.\n\nWho this is for: anyone working on data-efficient monocular depth or multi-task auxiliary learning. It deserves a serious referee; acceptance should be conditional on the same-update-count control and a clear statement on task selection.","headline":"Solid empirical study of auxiliary-task training for monocular depth, but the headline gain is undercut by a missing control: the proposed method takes twice as many optimizer steps as the baseline.","tokens_in":17041,"tokens_out":4663,"would_cite":false,"duration_ms":46767,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A shared decoder trained alternately on depth and auxiliary vision tasks improves monocular depth estimation by about 11% over the DINOv2 baseline, and allows reducing depth training data by 80-99%.","keywords":["monocular depth estimation","multi-task learning","auxiliary tasks","multi-label dense classification","DINOv2","DPT decoder","data efficiency","semantic segmentation"],"falsifier":"Run the exact same training scheme with MLDC fixed as the auxiliary task on all six in-domain datasets, choosing nothing per dataset, and compare average AbsRel to the DINOv2 baseline; if the average gain is near zero or negative, the headline gain depends on post-hoc task selection rather than on the training scheme.","tokens_in":16032,"feed_emoji":"📏","tokens_out":6320,"duration_ms":56168,"temperature":0.7,"pith_summary":"The paper tries to establish that monocular depth estimation—predicting depth from a single image—can be improved cheaply by training a shared decoder on auxiliary vision data from other tasks, without fine-tuning the large frozen backbone. It claims that alternating depth steps with segmentation, multi-label dense classification, or reconstruction steps, weighted toward depth, improves absolute relative error by about 11% on average across six in-domain datasets over a DINOv2 baseline. It further claims that the same scheme lets one cut MDE training data by 80-99% without losing quality. The paper also argues that the improvement comes from the task itself, not merely from adding data, since using the same dataset with different auxiliary tasks gives different results.","feed_headline":"Auxiliary vision tasks cut depth-error by ~11 percent.","feed_subtitle":"Frozen DINOv2 plus shared decoder beats the baseline and needs 80-99% less depth data.","key_machinery":"The load-bearing mechanism is an alternating gradient-step schedule with a task-focusing parameter α. A frozen DINOv2 ViT-G extracts features; a single shared DPT decoder plus two small task-specific heads produces depth and auxiliary outputs. Each global update first takes an MDE step scaled by α, then an auxiliary step scaled by 1−α, so the decoder is reshaped by both signals while depth dominates. MLDC is defined by spatially averaging the segmentation output to a multi-label vector and training with binary cross-entropy, which isolates class-presence information from precise positioning.","core_discovery":"The central discovery is that a frozen vision foundation model can be made a better monocular depth estimator by training a single shared DPT decoder on depth and auxiliary tasks alternately, with separate task heads and a task-focusing parameter α = 0.9 that weights depth updates more heavily. On six in-domain depth datasets, the best auxiliary task per dataset reduces absolute relative error by an average of about 11% relative to the DINOv2 baseline; with the MIX6 collection of six segmentation datasets, multi-label dense classification (MLDC)—averaging the segmentation map over space and classifying which classes are present—usually gives the largest gain. The same scheme also improves the stronger Depth Anything backbone by about 2.5% on average. The paper further reports that the method preserves or improves quality while using 80-99% less MDE training data on the tested datasets.","pith_inferences":["A practical follow-up would test whether a fixed rule such as 'always use MLDC' preserves most of the gain; the paper's ~11% is computed with the best task chosen per dataset, so a fixed procedure may give less.","Because image reconstruction sometimes wins on outdoor scenes, the most useful auxiliary signal may depend on scene type; dynamic task weighting could outperform any single task.","If class-presence information is what helps, then cheap image-level classifiers trained on large classification corpora may substitute for dense segmentation labels, potentially extending the data-efficiency result.","The same shared-decoder, frozen-backbone recipe may transfer to other dense prediction targets, such as surface normals or optical flow, whenever a related auxiliary dataset is available."],"forward_implications":["If the central claim holds, MDE datasets can be cut by 80-99% with no quality loss, directly lowering the cost of collecting dense depth labels.","Existing semantic segmentation datasets become reusable as depth-training resources, especially when converted to MLDC.","Auxiliary task choice matters more than raw data volume, so future systems should treat task selection as a first-class design decision.","The gain is not tied to one backbone: the same alternating scheme improves Depth Anything as well as DINOv2 ViT-G.","Out-of-domain auxiliary data can hurt, as shown by KITTI, so auxiliary data should be matched to the target scene domain."],"supporting_citations":[{"why":"supplies the frozen DINOv2 ViT-G backbone and the baseline training procedure against which gains are measured","marker":"[46]"},{"why":"supplies the shared DPT decoder architecture that the alternating MDE-auxiliary training reshapes","marker":"[51]"},{"why":"provides the Depth Anything baseline and backbone used to validate that the scheme transfers to a stronger MDE model","marker":"[69]"},{"why":"is one of the six segmentation datasets composing MIX6, the main auxiliary data source","marker":"[73]"},{"why":"is the largest auxiliary segmentation dataset in MIX6","marker":"[7]"},{"why":"serves both as an in-domain MDE test set and as one of the MIX6 auxiliary segmentation datasets","marker":"[58]"}],"fun_headline_variants":["Auxiliary tasks improve depth estimation by 11%","Depth accuracy up 11% with auxiliary task training","Multi-task training boosts depth estimation and cuts data 80%","Auxiliary tasks slash depth error and data needs","Shared decoder plus auxiliary tasks lifts depth accuracy 11%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported ~11% average gain belongs to the method itself, although the table computes it by selecting, after the fact, whichever auxiliary task worked best for each dataset; for a new dataset there is no rule for making that choice.","fun_headline_variants_meta":{"raw":{"variants":["Auxiliary tasks improve depth estimation by 11%","Depth accuracy up 11% with auxiliary task training","Multi-task training boosts depth estimation and cuts data 80%","Auxiliary tasks slash depth error and data needs","Shared decoder plus auxiliary tasks lifts depth accuracy 11%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000304,"raw_usage":{"total_tokens":1735,"prompt_tokens":923,"completion_tokens":812,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":732}},"tokens_in":539,"tokens_out":812,"duration_ms":7907,"temperature":1.0,"reasoning_tokens":732,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:44:43.258556+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact same training scheme with MLDC fixed as the auxiliary task on all six in-domain datasets, choosing nothing per dataset, and compare average AbsRel to the DINOv2 baseline; if the average gain is near zero or negative, the headline gain depends on post-hoc task selection rather than on the training scheme.","supporting_citations":[{"cited_title":"Vi- sion transformers for dense prediction","cited_arxiv_id":null,"evidence_quote":"supplies the shared DPT decoder architecture that the alternating MDE-auxiliary training reshapes"},{"cited_title":"Scene parsing through ade20k dataset","cited_arxiv_id":null,"evidence_quote":"is one of the six segmentation datasets composing MIX6, the main auxiliary data source"},{"cited_title":"Coco- stuff: Thing and stuff classes in context","cited_arxiv_id":null,"evidence_quote":"is the largest auxiliary segmentation dataset in MIX6"}],"review_version":1}