{"id":"ea31fa49-273d-4feb-87e8-177836ab5d7e","arxiv_id":"2411.16131","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A co-learning matrix over imitation-learning branches, plus classification-style steering losses, raises reach-destination success in unseen CARLA towns by about 62% over the CIL baseline.","lead":"This paper trains self-driving steering models in the CARLA simulator and connects the specialist branches of conditional imitation learning with a learned co-learning matrix, reporting a 62% success-rate gain in unseen towns. The work matters because it targets the known generalization weakness of end-to-end driving models without adding LIDAR, maps, or modular perception.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 62% generalization gain rests on a single unseeded run of a narrowed benchmark (single-turn routes, steering filtered to |s|<=0.8); extending evaluation to multi-turn or sharp-curve routes is the decisive test.","rationale":"I read the paper in good faith: the proposed co-learning architecture is coherent, the tables are internally consistent, and the reported 62% relative improvement is arithmetically correct for the selected cells. The central claim, however, is about generalization to unseen environments, and the evaluation protocol is the weakest link. Section 4 explicitly narrows the CoRL benchmark to single-turn routes and discards training samples with |steering| > 0.8, so the measured success rates characterize a restricted distribution of driving situations. The abstract and conclusion generalize this to unseen environments without evidence that the advantage persists on multi-turn routes or sharp curves. The absence of reported seeds and route definitions means the headline number cannot be independently reproduced, and the apparent selection of W and R on the test benchmark adds further uncertainty. These are external-validity and reproducibility concerns rather than internal inconsistencies, and they do not disprove the claim. They do, however, justify keeping the conditional verdict: acceptance should require the extended evaluation and seed reporting described in the concrete test.","tokens_in":9185,"tokens_out":8132,"duration_ms":84247,"concrete_test":"Release the exact route definitions and rerun the published CIC and CIL models on (a) the original CoRL benchmark's navigation task with multiple consecutive turns and (b) a new suite of single-turn routes containing at least one curve requiring |steering| > 0.8, with at least 5 seeds per condition and a pre-registered comparison. If the GTU model's success-rate advantage over CIL on these tasks is not statistically significant (e.g., a 95% CI for the difference excluding zero), the 62% claim should be restricted to the single-turn, moderate-curvature benchmark.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline number is 62%: (75.00 - 46.25)/46.25 for GTU co-learning versus CIL regression in new-town/new-weather conditions (Tables 1 and 4). The load-bearing problem is that the benchmark producing this number is deliberately narrowed in Section 4: only the single-turn task of the CoRL benchmark is kept, and training samples with steering magnitude above 0.8 are discarded. The 38+40 start/destination pairs are all single-turn routes, so the vehicle never faces a change of navigational command en route or a sharp curve. The claim of improved success rate in unseen environments is therefore demonstrated only for moderate single turns in two towns and two weathers, not for the general unseen-environment claim. Additionally, no seeds are reported, so 75.00% versus 46.25% (60/80 versus 37/80) is a single draw; without route definitions or code the result cannot be reproduced. Hyperparameter values W in Tables 2-3 and the R matrix in the first co-learning variant appear to be selected on the test benchmark, further inflating the reported variants. These issues do not make the arithmetic wrong, but they make the central generalization claim fragile.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes conditional imitation co-learning (CIC), an extension of the conditional imitation learning (CIL) architecture of Codevilla et al. (2018) for end-to-end steering. CIC lets the specialist branches (left, right, straight, follow-lane) share output-level information through a co-learning matrix, with coefficients generated either from a manually fixed binary relationship matrix R multiplied by learned coefficients or from gated tanh units (GTUs). The paper also recasts steering regression as classification, adding a hybrid cross-entropy/MSE loss (Eq. 3) and a co-existence-probability loss based on a Gaussian spatial relationship matrix (Eq. 4). Experiments on a modified CoRL benchmark in CARLA report that the GTU-based co-learning model achieves 75.00% reach-destination success in new-town/new-weather conditions versus 46.25% for the CIL regression baseline, a 62% relative improvement, while the hybrid-loss and co-existence-loss variants show smaller gains. The paper concludes that CIC improves generalization robustness in unseen environments.","tokens_in":9513,"tokens_out":3425,"duration_ms":33144,"significance":"If the reported gain is robust, the paper makes a useful empirical contribution by showing that a simple output-level mixing mechanism between CIL specialist branches can yield substantial generalization improvements over the standard architecture, and by providing ablations of classification-based steering losses with baselines including mixture of experts, sluice networks, and soft parameter sharing. The idea of learning branch relationships through GTUs is plausible and the comparison against multiple MTL baselines is informative. However, the central quantitative claim currently rests on a single unseeded run of a deliberately narrowed benchmark, and the hyperparameter selection procedure is not described as respecting a clean train/validation/test split. These issues make the headline 62% claim fragile as it stands.","major_comments":[{"comment":"The headline 62% improvement is computed as (75.00 - 46.25) / 46.25 from single point estimates, with no standard deviations, confidence intervals, or multiple runs/seeds reported. CARLA evaluation includes stochasticity in weather, traffic, and autopilot behavior; a single draw (60/80 versus 37/80 successful routes) is not sufficient evidence for the claimed generalization advantage. Please provide results over multiple seeds and report statistics (mean and variance or a significance test) for all key comparisons.","section":"Section 4, Tables 1 and 4"},{"comment":"The evaluation is restricted to single-turn routes only, and training samples with steering magnitude above 0.8 are discarded. The claim in the abstract and conclusion that the model 'improve[s] autonomous driving success rate in unseen environment' is therefore demonstrated only for moderate single turns in two towns and two new weathers. The decisive test is evaluation on multi-turn routes and sharp-curve scenarios from the full CoRL benchmark; without this, the generalization claim is overstated. Please either extend the evaluation or explicitly scope the claim to the tested distribution.","section":"Section 4, benchmark design"},{"comment":"Hyperparameter W for the hybrid loss and the co-existence loss appears to be selected by testing a set of values and reporting the best-performing rows on the same benchmark (W=10 for the hybrid loss, W=0.6 for the co-existence loss), and the binary relationship matrix R in the first co-learning variant is described as manually fine-tuned. This procedure can inflate the reported performance of the proposed variants. Please describe the hyperparameter selection protocol, and ideally use a separate validation set or cross-validation to choose W and R before evaluating on the held-out test routes.","section":"Section 3.3, Section 3.4, Tables 2 and 3"}],"minor_comments":[{"comment":"The abstract states '62% on average', but the tables show this exact relative improvement only in the new-town/new-weather cell; the averaging basis over conditions should be clarified.","section":"Abstract"},{"comment":"The symbol y is used for both the continuous steering value and the one-hot vector of the discretized steering; please use distinct notation (e.g., y_true and y_onehot) to avoid ambiguity.","section":"Equation (3)"},{"comment":"The row labeled 'stitch network' appears to refer to cross-stitch networks (Misra et al., 2016); please correct the terminology and ensure the reference is cited consistently.","section":"Table 4"},{"comment":"The paper does not state whether the route definitions, trained models, or code will be made available; providing these would significantly improve reproducibility of the reported success rates.","section":"Section 4"},{"comment":"The choice of Gaussian distribution with sigma^2 = 1 for the co-existence matrix is asserted without sensitivity analysis; please justify this choice or show that performance is stable over a range of sigma values.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a revised conference-style contribution with a plausible idea and a potentially interesting empirical finding, but the experimental evidence is currently too thin for a journal-level claim of a 62% generalization improvement. The major revisions I request (multiple seeds, full benchmark evaluation, and a clean hyperparameter selection protocol) are substantial but feasible within the scope of the paper. I would not recommend rejection, as the central mechanism is clearly described and the baseline comparisons are informative. The authors should also clarify the novelty of GTU-based co-learning relative to soft parameter sharing and cross-stitch networks; the current discussion positions CIC as output-oriented, but the empirical comparison alone may not fully establish this distinction."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nQuick read: this is a workshop-grade empirical paper that proposes a genuinely new architectural variant—conditional imitation co-learning (CIC)—where the specialist branch outputs are mixed via a gated tanh co-learning matrix. That is new relative to the cited cross-stitch/sluice/MoE baselines, and the comparisons against those baselines are a real strength. The paper is also transparent about its setup, including that it keeps only the single-turn task from the CoRL benchmark and discards steering samples with magnitude above 0.8. That transparency is good, but it makes the abstract's '62% improvement in unseen environment' narrower than it sounds: 75% vs 46.25% success on new-town/new-weather over 80 routes per condition, from a single unseeded run.\n\nThe arithmetic is consistent, and the internal numbers add up. But the load-bearing generalization claim is fragile for three reasons. First, no seeds or confidence intervals are reported, so the 28.75 point gap could easily be within run-to-run variance. Second, the loss weight W and the binary relationship matrix R appear to be chosen after seeing test performance on the same benchmark used for the headline number. Third, the route set is single-turn only and the training data is filtered to moderate steering, so 'unseen environment' covers gentle turns in two towns and two weathers, not multi-turn intersections or sharp curves. The authors acknowledge the filtering, but they don't discuss how it limits the claim.\n\nThat said, I don't think this is a bad paper. The CIC idea is worth taking seriously, the comparison set is appropriate, and the writing is clear. What it needs is more evidence: multiple seeds, a held-out hyperparameter selection procedure, route definitions or released code, and ideally the full CoRL benchmark rather than the single-turn subset.\n\nWho should read it: people working on end-to-end imitation driving or output-space parameter sharing for multi-task networks. It's a reasonable workshop/conference submission, not a major journal result.\n\nMy recommendation: send it to peer review. The architecture is novel and the evaluation, while under-powered, is not misleading about what was measured. A competent reviewer can push for the missing evidence. I would not cite the 62% number in my own work, but I'd cite the CIC approach if it survives a re-run.","headline":"A novel co-learning architecture that plausibly improves CIL generalization in simulation, but the headline 62% rests on a single unseeded run of a deliberately narrowed benchmark.","tokens_in":10022,"tokens_out":1744,"would_cite":false,"duration_ms":16461,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Letting specialist branches share features lifts unseen-town driving success by 62%.","keywords":["Autonomous Driving","End-to-end","Conditional Imitation Learning","Co-learning Matrix","Co-existence Probability Matrix","Steering Model","CARLA","Generalization"],"falsifier":"Take the same GTU co-learning model and the unmodified CIL baseline, and evaluate both on the full route set without dropping multi-turn routes or samples with steering magnitude above 0.8. If the gap in reach-destination success over unseen towns and weathers shrinks to a small margin, the 62% claim is specific to the filtered single-turn, moderate-steering task.","tokens_in":8963,"feed_emoji":"🚗","tokens_out":8301,"duration_ms":66796,"temperature":0.7,"pith_summary":"This paper tries to improve how end-to-end driving models generalize to unseen towns and weather conditions. It argues that the specialist branches of conditional imitation learning should not be treated as independent: each branch's predictions for one navigational command can benefit from features extracted by branches trained on other commands. The proposed conditional imitation co-learning (CIC) model uses a co-learning matrix, generated by gated tanh units, to mix the branches' output features, and the paper reports that this raises reach-destination success in fully unseen town-plus-weather conditions to 75% from 46.25% for the unmodified model. The paper also tests two ways of casting steering regression as classification, finding that a hybrid cross-entropy plus mean-squared-error loss helps while plain classification does not.","feed_headline":"Co-learning branches lift unseen-town driving success 62 percent","feed_subtitle":"GTU co-learning between CIL specialist branches reaches 75% success in new towns vs 46% baseline.","key_machinery":"The load-bearing mechanism is the co-learning matrix $C_t$ that linearly mixes the four specialist branches' feature vectors, $\\hat{A}_t$, before the action is read out: $A_t = C_t \\hat{A}_t$. The diagonal entries of $C_t$ are fixed at 1 and the off-diagonal coefficients are produced by gated hyperbolic tangent units (GTUs), so the network can dynamically decide, per driving scenario, how much each branch's features should influence every other branch's prediction. The paper also introduces a classification-regression hybrid loss (categorical cross-entropy plus mean-squared error on the expected steering) and a co-existence probability matrix loss that nudges output scores toward a Gaussian spatial distribution over steering classes, but it is the GTU-driven co-learning matrix that carries the headline generalization gain.","core_discovery":"On the paper's own terms, the central discovery is that the conditional imitation learning (CIL) assumption that each navigational-command branch should act independently is the bottleneck to generalization. By computing an $N \\times N$ co-learning matrix at each time step and multiplying it with the vector of branch-specific output features, the proposed conditional imitation co-learning (CIC) model lets a branch trained mainly on right turns, for example, draw on features learned from left-turn and straight-driving data. In the benchmark, the GTU-generated co-learning matrix reaches a 75.00% reach-destination success rate in an unseen town with unseen weather, versus 46.25% for the baseline CIL regression model, the 62% improvement cited in the abstract. The paper further reports that classifying steering into nine discrete classes with a hybrid cross-entropy plus mean-squared-error loss improves unseen-environment success by 21% over the regression baseline, while plain classification alone does not beat regression.","pith_inferences":["Because the benchmark drops multi-turn routes and samples with steering magnitude above 0.8, the reported 62% gain is established only for moderate single-turn maneuvers; extending the evaluation to the full route distribution would test whether the co-learning benefit persists.","The co-learning matrix idea is not specific to steering: any multi-branch policy or task module that shares an output layer could use the same GTU-generated mixing coefficients, so the mechanism could transfer to other imitation-learning or multi-task settings.","The spatial-class loss via co-existence probability with a Gaussian kernel is one choice; comparing it to learned kernels or to the sine-wave encoding on the same benchmark would separate the benefit of the spatial prior from the kernel's shape.","If the GTU coefficients are interpreted per scenario, they may be inspected to see which branch-to-branch transfers matter most, potentially guiding data collection toward under-connected navigation commands."],"forward_implications":["A network trained in one town and two weather conditions reaches 75% success in a new town under new weather conditions, versus 46.25% for the unmodified conditional imitation learning model.","The co-learning mechanism is output-oriented: it mixes the final branch features rather than hidden-layer parameters, so it can be added atop an existing CIL-style architecture without changing the feature extractor.","Posing steering regression as classification with a hybrid cross-entropy plus mean-squared-error loss improves unseen-environment success by 21% over the regression baseline, though classification alone does not.","Existing multi-task sharing methods (mixture of experts, soft parameter sharing, sluice networks) improve over CIL in new towns but none of them reaches the GTU co-learning model's success rates."],"supporting_citations":[{"why":"Supplies the CIL branched architecture and the three-camera, noise-injected data collection procedure that the paper modifies.","marker":"(Codevilla et al., 2018)"},{"why":"Provides the CARLA simulator and the CoRL benchmark whose single-turn variant is used for evaluation.","marker":"(Dosovitskiy et al., 2017)"},{"why":"Introduces the gated tanh units used to generate the co-learning matrix coefficients.","marker":"(Dauphin et al., 2017)"},{"why":"Establishes the regression-as-classification approach that the paper adapts to steering.","marker":"(Rothe et al., 2015)"},{"why":"Applies regression-as-classification to steering with sine-wave spatial encoding, the comparison baseline for the co-existence matrix.","marker":"(Eraqi et al., 2017)"},{"why":"Introduces the co-existence probability matrix loss that the paper uses to enforce spatial structure among steering classes.","marker":"(Bengio et al., 2013)"},{"why":"Provides the hybrid classification-regression loss formulation that the paper combines into the CCE plus MSE objective.","marker":"(Kourbane and Genc, 2021)"},{"why":"Supplies sluice networks, one of the multi-task sharing baselines CIC is compared against.","marker":"(Ruder et al., 2019)"},{"why":"Supplies mixture of experts, another multi-task baseline in the comparison table.","marker":"(Jacobs et al., 1991)"}],"fun_headline_variants":["Co-learning matrix lifts driving success 62% in unseen towns","GTU co-learning boosts autonomous driving 62% in new environments","75% unseen-town success via conditional imitation co-learning","Co-learning branches improve CIL generalization by 62%","End-to-end steering co-learning: 62% gain in unfamiliar locales"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 62% generalization gain is measured on a benchmark that keeps only single-turn routes and drops steering samples with magnitude above 0.8, so the claim depends on those filters matching what 'unseen environment' means in practice.","fun_headline_variants_meta":{"raw":{"variants":["Co-learning matrix lifts driving success 62% in unseen towns","GTU co-learning boosts autonomous driving 62% in new environments","75% unseen-town success via conditional imitation co-learning","Co-learning branches improve CIL generalization by 62%","End-to-end steering co-learning: 62% gain in unfamiliar locales"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000167,"raw_usage":{"total_tokens":1266,"prompt_tokens":963,"completion_tokens":303,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":217}},"tokens_in":579,"tokens_out":303,"duration_ms":3676,"temperature":1.0,"reasoning_tokens":217,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:31:18.704440+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same GTU co-learning model and the unmodified CIL baseline, and evaluate both on the full route set without dropping multi-turn routes or samples with steering magnitude above 0.8. If the gap in reach-destination success over unseen towns and weathers shrinks to a small margin, the 62% claim is specific to the filtered single-turn, moderate-steering task.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the CIL branched architecture and the three-camera, noise-injected data collection procedure that the paper modifies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CARLA simulator and the CoRL benchmark whose single-turn variant is used for evaluation."},{"cited_title":"N., Fan, A., Auli, M., and Grangier, D","cited_arxiv_id":null,"evidence_quote":"Introduces the gated tanh units used to generate the co-learning matrix coefficients."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the regression-as-classification approach that the paper adapts to steering."},{"cited_title":"End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies","cited_arxiv_id":"1710.03804","evidence_quote":"Applies regression-as-classification to steering with sine-wave spatial encoding, the comparison baseline for the co-existence matrix."},{"cited_title":"Using Web Co-occurrence Statistics for Improving Image Categorization","cited_arxiv_id":"1312.5697","evidence_quote":"Introduces the co-existence probability matrix loss that the paper uses to enforce spatial structure among steering classes."},{"cited_title":"A hybrid classification-regression approach for 3D hand pose estimation using graph convolutional networks","cited_arxiv_id":"2105.10902","evidence_quote":"Provides the hybrid classification-regression loss formulation that the paper combines into the CCE plus MSE objective."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies sluice networks, one of the multi-task sharing baselines CIC is compared against."}],"review_version":1}