{"id":"e1bf6670-a045-422e-b477-46172b0a1ee9","arxiv_id":"2603.00919","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"DriveCode's continuous number projector and regression number head reduce control-signal errors in LLM autonomous driving compared with text-token and xVal baselines on the tested datasets.","lead":"DriveCode replaces discrete number tokens in LLM-based driving with continuous number embeddings, letting the model output speeds, angles, and waypoints as regressed values. If the reported gains hold up under stronger evaluation, numerical encoding could become a standard component for precise control in autonomous-driving LLMs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DriveCode's central claim rests on an unvalidated number-extraction/classification pipeline; the paper's own Limitations section concedes the dependency without quantifying its reliability.","rationale":"The reader's weakest assumption correctly identifies the number-extraction pipeline as the most fragile link in the chain. The central claim of superior performance depends on the model being trained with correctly aligned numerical values for the number projector and head. The paper itself acknowledges this limitation but provides no quantitative assessment of the pipeline's accuracy or robustness. This is a concrete, testable concern that should be resolved before the method can be fully trusted. However, the concern does not invalidate the architectural idea—it calls for validation and possibly a more robust preprocessing implementation. The reader's conditional verdict is therefore appropriate; no change is needed.","tokens_in":11260,"tokens_out":13894,"duration_ms":142279,"concrete_test":"Run the preprocessing pipeline on a random sample of 100 dialogues per dataset (DriveGPT4, DriveGPT4-V2, OmniDrive). For each dialogue, manually list the physically meaningful numbers (speed, angle, waypoints, time, etc.) and compare against the extracted list and the <number token> count. Verify that the order of the extracted list matches placeholder order and that no descriptive numbers (e.g., '8-frame', '5 vehicles') are converted. Compute the mismatch rate. Then retrain DriveCode with a more conservative pipeline that converts numbers only in clearly numeric contexts (e.g., after 'Speed:' or 'Turning angle:') and compare the resulting metrics to the reported ones. If the gains over xVal/text disappear, the original results are artifacts of extraction; if they persist, the concern is minor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The preprocessing described in Section III-B replaces selected numbers with <number token> and builds an ordered list aligned with placeholder positions. However, the selection of 'meaningful' numbers is a heuristic (e.g., excluding 'number of camera views') that is not specified precisely, and the regex patterns are not given. If a physically meaningful number is missed or a descriptive number is incorrectly converted, the model either loses the continuous encoding for that quantity or receives a spurious one. Because the number list drives both the input projector and the regression supervision (Eqs. 3, 6-8), a single misclassification can inject wrong targets into the number head for that sample. The paper provides no error-rate analysis, manual verification, or code to reproduce the pipeline. The Limitations section (IV-G) explicitly flags 'mismatches, missing numbers, or inconsistent formats' but gives no evidence that such cases are rare. Without this evidence, the reported improvements over xVal and text baselines could be partly an artifact of the preprocessing rather than the architectural idea. This is load-bearing because the method's core claim is the superiority of continuous numerical encoding; if the encoding pipeline is unreliable, the method is not generally applicable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"DriveCode proposes a continuous numerical encoding for LLM-based autonomous driving. In the preprocessing stage, numerical quantities in prompts and targets are replaced by a special <number_token> placeholder, with the actual values stored in an ordered list. A learned number projector maps each value into the LLM embedding space, and a number head regresses numerical outputs from hidden states, so that numbers are processed as continuous quantities rather than discrete text tokens. The model is evaluated on three datasets (OmniDrive, DriveGPT4, DriveGPT4-V2) against text-token and xVal baselines, reporting lower RMSE and point/speed errors in most settings, with ablations separating input- and output-side numeric conditioning. The central claim is that this dedicated numeric modality improves trajectory prediction and control-signal generation.","tokens_in":11558,"tokens_out":6859,"duration_ms":62954,"significance":"The idea of treating numbers as a dedicated modality in driving LLMs is timely and practically relevant for control-signal regression, where small absolute errors matter. The architecture is clearly specified, and the ablation design (Text / Variant / DriveCode) is a sensible way to isolate the contributions of numeric conditioning on the input and regression supervision on the output. The paper also provides an efficiency analysis and explicitly acknowledges limitations. If the empirical claims held up with adequate statistical support and a validated preprocessing pipeline, the method would be a credible component for real-time LLM-based driving systems. However, the current evidence is insufficient: results are single-run point estimates without confidence intervals, the preprocessing heuristic is unquantified, and the training loss directly matches the evaluation metric, which partially explains the reported improvements.","major_comments":[{"comment":"The preprocessing pipeline is load-bearing but unvalidated. The regex-based selection of 'meaningful' numbers is not specified (no patterns or decision rule for excluding terms like 'number of camera views'), and no precision/recall statistics are reported. Since Eqs. (3)-(8) use the ordered number list for both input embeddings and regression targets, a mis-extraction or order misalignment injects corrupted supervision. The Limitations (IV-G) concede this dependency but do not quantify its effect. Please report extraction accuracy on a random sample, provide the exact patterns/rules, and analyze failure cases, especially on natural-language DriveGPT4 versus templated DriveGPT4-V2.","section":"III-B, IV-G"},{"comment":"All results are single-run point estimates without error bars, confidence intervals, or seed variability. The claimed improvements are often small (e.g., speed RMSE 1.08 vs 1.13; DriveGPT4-V2 point error 0.01137 vs 0.01166; latency 3.1798 vs 3.1920 s), and Table I shows DriveCode losing to DriveGPT4 at the tight threshold A0.1 for both speed (27.50 vs 30.09) and angle (57.18 vs 59.23). Without paired significance tests over multiple seeds, the headline 'superior performance' is not supported.","section":""},{"comment":"The numerical loss L_num (Eqs. 7-8) is, up to sign, the same as the reported metrics (L1 error, L2 error, heading error). Thus DriveCode directly optimizes the evaluation metric, whereas text-token baselines are trained only with cross-entropy (Eq. 5). Part of the observed gain is therefore a consequence of the supervision signal, not of the continuous encoding per se. Please add a baseline that applies the same regression head (or an auxiliary L1/L2 loss) to hidden states derived from text numbers, and/or discuss this confound explicitly.","section":"III-E, IV-C"},{"comment":"Table V shows that 'Variant' achieves a lower theta error (0.07078 degrees) than DriveCode (0.07377 degrees) on DriveGPT4-V2, while DriveCode wins on point and speed errors. The text says DriveCode 'yields the best point and speed accuracy' but omits the theta result. The abstract and conclusion claim 'superior performance in trajectory prediction and control signal generation'; this is not consistent with Table V. The claims should be limited to the metrics on which the method actually wins, or the discrepancy explained (e.g., variance).","section":"IV.E, Table V"}],"minor_comments":[{"comment":"Figure 4 appears to show 'number: [...]' appended to the answer; please clarify whether this list is part of the input/target text or only an illustration of the stored aligned values. If it is part of the target, the LM could copy numbers from the list, circumventing the number head.","section":"Fig. 4"},{"comment":"Specify whether training uses teacher forcing with ground-truth numbers for the next-step embedding or the predicted number from the number head; the current text describes inference only.","section":"III-D"},{"comment":"Define the relation between the normalized L2 norm in Eq. (15) and the RMSE/MAE used in the tables; the notation is ambiguous.","section":"Eq. (15)"},{"comment":"No normalization details for input numbers; the Limitations mention sensitivity to scale/outliers, but the training procedure does not state how x_k is normalized (e.g., min-max or z-score).","section":"III-D"},{"comment":"Minor typos and formatting issues: 'LLaV A' should be 'LLaVA', 'V arious' etc. Please proofread.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is attractive, and the experimental framework is reasonable. However, the empirical support is currently too thin: no error bars, an unvalidated preprocessing pipeline that the authors themselves flag, and a training loss that directly matches the evaluation metrics. The abstract overstates the results given Table V and the A0.1 numbers in Table I. I would be willing to review a revision that addresses these points, especially the preprocessing validation and statistical rigor."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this paper is a direct extension of xVal, but it adds two things xVal doesn't have — an input number projector that maps scalar numbers into the LLM embedding space, and a parallel output number head that regresses numerical tokens instead of decoding digits. That combination, applied to autonomous driving, is a genuine architectural contribution. The comparisons against text and xVal baselines, plus the Variant ablation that isolates input vs. output numeric conditioning, are the right way to test the idea.\n\nThe paper is mostly sound. The architecture is coherent, the training objective that combines LM loss with a scalar or trajectory regression loss is plausible, and the efficiency argument is correct in principle: one regression step replaces several autoregressive tokens. The OmniDrive trajectory result (3.08 to 2.83 L2) is the most convincing single number. The limitations section is honest, which is more than many papers offer. The authors clearly know the weak spots.\n\nThe soft spots are real. There are no error bars, confidence intervals, or multiple seeds anywhere. Several gains are tiny — Table II shows differences of 0.0003 and 0.0003 in theta and point error on DriveGPT4-V2, which could easily be noise. The per-metric results are mixed: DriveGPT4 beats DriveCode at the tightest A0.1 threshold on both speed and angle, and the Variant (output-only numeric regression) actually beats DriveCode on theta in Table V. The abstract's \"superior performance\" overstates what the tables show; the honest summary is \"better on most, not on all, and sometimes by a hair.\"\n\nThe stress-test worry about the regex-based number extraction pipeline is legitimate but, I think, not load-bearing. Yes, the pipeline is under-specified — no regex patterns, no error rate, no manual verification — and the paper's own limitations section admits \"mismatches, missing numbers, or inconsistent formats\" are possible. That is a genuine reproducibility problem and a robustness concern. But the architecture's value does not collapse if extraction is perfect; the ablations comparing text vs. numeric input and output are independent of the extraction pipeline's accuracy, and those ablations show consistent, if small, improvements. I would not call the pipeline the \"central claim.\" The central claim is that continuous encoding helps, and that claim is supported, weakly and needing more seeds, but supported.\n\nWho this is for: someone working on numeric encodings for LLMs, or on LLM-based driving control, will want to read this. It is not a breakthrough paper, but it is a useful, concrete step. The lack of code is the biggest practical obstacle to citing it seriously.\n\nRecommendation: send it to peer review. It deserves a real referee, but the reviewers should demand error bars, multiple seeds, a full specification (or release) of the preprocessing pipeline, and a toned-down abstract. As is, I would not cite it in the next year without code.","headline":"DriveCode is a sensible xVal-style extension for LLM driving that delivers small, mixed gains; the architecture is coherent and the ablations are well designed, but the results need statistical grounding and the preprocessing pipeline needs to be specified before the claims can be fully trusted.","tokens_in":866,"tokens_out":1115,"would_cite":false,"duration_ms":35124,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DriveCode claims that representing numbers as continuous embeddings rather than text tokens makes LLM-based autonomous driving predict speed, steering, and trajectories more precisely.","keywords":["DriveCode","numerical encoding","continuous number embeddings","LLM autonomous driving","trajectory prediction","control signal generation","number projector","number head"],"falsifier":"Run the same training recipe on a dialogue where numbers are written in varied formats (e.g., '1.5m', 'one point five', '≈2') and compare alignment errors; or construct a dataset where the regex misses a number and show the model silently ignores it. A direct test: take a trained DriveCode model and feed a prompt whose placeholder order has been shuffled relative to the number list—if performance collapses, the method depends on exact alignment rather than on continuous number semantics.","tokens_in":11198,"feed_emoji":"🚗","tokens_out":3441,"duration_ms":33657,"temperature":0.7,"pith_summary":"DriveCode argues that the reason LLM-based self-driving models make imprecise control predictions is that numbers are tokenized as text, which destroys their magnitude and forces digit-by-digit generation. The paper proposes a number projector that maps each numerical value into the model's embedding space, and a number head that regresses output numbers in a single step. On three autonomous-driving datasets, this continuous representation reduces trajectory, speed, and steering-angle errors compared with text-token and scaled-embedding baselines. If correct, it gives a way to make LLM-generated driving commands numerically reliable without changing the language model itself.","feed_headline":"Numbers as embeddings beat text tokens for driving LLMs","feed_subtitle":"Continuous number projector and regression head cut speed, steering, and trajectory errors on three driving benchmarks.","key_machinery":"The load-bearing mechanism is the paired number projector and number head. The projector maps a scalar x to a high-dimensional embedding through a two-layer MLP with GELU, inserted where a <number> placeholder sits in the token sequence; the number head regresses a scalar from the hidden state one position before each placeholder. Together they let the LLM attend over numeric values alongside image and text features, and generate numbers continuously rather than token by token. The authors also use an ordered number list extracted from each dialogue to keep every placeholder aligned with its real value.","core_discovery":"The central claim is that treating numbers as a dedicated continuous modality—rather than as discrete vocabulary tokens—improves both the perception and the generation of physical quantities in LLM-based driving. Input numbers are extracted from prompts, replaced by placeholders, and injected as learned embeddings via a two-layer projector; output numbers are predicted by regressing from the hidden state immediately before each placeholder, so the model emits a value in one decoding step instead of one token per digit. The paper reports that this scheme outperforms text-token and scaled-embedding baselines on speed, steering angle, point, and trajectory metrics across three driving datasets,","pith_inferences":["A likely consequence beyond driving: any LLM task that needs exact scalar outputs—robotics control, sensor calibration, financial quantities—could use the same projector/head pattern.","The reliance on regex extraction implies the method's benefit shrinks on free-form text where numbers aren't in the expected format; a learned number span detector would be a natural extension.","The paper's speed advantage is modest; if bigger models with more tokens per number shrink the gap, the main value may be precision rather than latency.","Testable extension: use the number projector in a math word-problem setting with unseen number scales to see if continuous encoding improves generalization beyond the driving domain."],"forward_implications":["If numbers are processed continuously, LLM driving models can produce speed, waypoint, and steering commands with lower mean error than text-based number generation, on the three tested datasets.","Because a number is produced in a single decoding step, inference latency drops slightly, which matters for real-time control.","The method is architecture-agnostic: it only replaces numeric tokens, so it can be grafted onto existing vision-language driving models without retraining the vision or language backbone from scratch.","The output-side regression head alone (without the input projector) already reduces numeric error, suggesting that direct numeric supervision is the main driver; the input side adds further gains."],"fun_headline_variants":["DriveCode: Numbers as continuous embeddings for driving LLMs","Continuous number embeddings beat text tokens in driving LLMs","DriveCode: Dedicated number embeddings improve LLM driving control","DriveCode: Continuous number embeddings for one-step driving control"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"DriveCode assumes that a simple regex can find every meaningful number in every prompt and that the extracted list stays in exactly the same order as the placeholders in the token sequence; if either fails, the number supervision is corrupted.","fun_headline_variants_meta":{"raw":{"variants":["DriveCode: Numbers as continuous embeddings for driving LLMs","Continuous number embeddings beat text tokens in driving LLMs","DriveCode: Dedicated number embeddings improve LLM driving control","DriveCode: Continuous number embeddings for one-step driving control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001042,"raw_usage":{"total_tokens":4178,"prompt_tokens":662,"completion_tokens":3516,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":406,"completion_tokens_details":{"reasoning_tokens":3449}},"tokens_in":406,"tokens_out":3516,"duration_ms":25131,"temperature":1.0,"reasoning_tokens":3449,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T05:54:10.438944+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same training recipe on a dialogue where numbers are written in varied formats (e.g., '1.5m', 'one point five', '≈2') and compare alignment errors; or construct a dataset where the regex misses a number and show the model silently ignores it. A direct test: take a trained DriveCode model and feed a prompt whose placeholder order has been shuffled relative to the number list—if performance collapses, the method depends on exact alignment rather than on continuous number semantics.","supporting_citations":[],"review_version":1}