{"id":"88de47f1-9d15-42a8-a4ae-a822c4d779b8","arxiv_id":"2505.10083","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"ChronoSteer steers a frozen time series foundation model with one of nine LLM-generated revision instructions, trained on synthetic instruction-series pairs, and beats unimodal and LLM baselines on a fresh multimodal benchmark.","lead":"ChronoSteer adds a small text-reading branch to a frozen time series forecasting model, so a language model's weather or holiday notes can adjust the forecast. The paper reports accuracy gains on fresh energy, traffic, and air quality benchmarks using training data that never includes real text-series pairs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fine-tuning uses real future series as supervised targets, so the 'trained exclusively on synthetic data' claim is internally contradicted and a substantial part of the reported gain is not attributable to synthetic data alone.","rationale":"I agree with the reader's bottom line (CONDITIONAL) and with several of its supporting observations, especially that code/data are not yet released and that the headline percentages are not straightforwardly recoverable from Tables 1-3. However, I identify a more load-bearing concern than the nine-anchor limitation: the paper explicitly trains on real future series during fine-tuning, while the abstract and contributions repeatedly claim exclusive use of synthetic data. This is not a scope limitation like the codebook restriction; it is an internal inconsistency in the central claim. The reader did flag this point in passing, but their stated weakest assumption is the sufficiency of nine anchors. Both issues are real: Appendix D candidly acknowledges that the nine-anchor codebook restricts generalization, and the anchor-matching mechanism in Eq. 4 quantizes every unseen instruction to one of nine fixed templates. Yet because the collected benchmark contexts (weather, weekend effects, fog) plausibly map onto those nine templates, the anchor concern is harder to settle without new benchmark domains. In contrast, the synthetic-data contradiction is already evidenced within the paper: Section 3.5.1 uses real y as targets, and Table 3's rm Finetune shows that this real-label stage contributes 8.3-16.6% accuracy. A controlled ablation replacing real y with synthetic targets would decisively quantify the dependence. Credit is due where warranted: the Function baseline comparison and the rm Contrastive ablation provide real evidence that learned instruction-seeking adds value beyond hand-designed transformations, and the fresh January-2025 benchmark is a meaningful step against leakage. Those strengths do not repair the stated data-exclusivity claim, so the verdict should remain conditional: the central mechanism is plausible, but the paper must be revised to state precisely which training signals are synthetic and which are real, and the headline numbers must be recomputed under that corrected description.","tokens_in":28318,"tokens_out":9315,"duration_ms":100018,"concrete_test":"Re-run the fine-tuning stage on ChronoSteer-100K-FT with targets replaced by the synthetic function outputs used in pre-training, so no real future series y enters any gradient computation, then evaluate on MTSFBench-300. If the resulting MSE is substantially worse than the full model and close to the rm Finetune row of Table 3, the reported 25.7% gain depends on real future labels and the 'exclusively synthetic data' wording must be corrected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.5.1 states that 'we incorporate the real future series y as a supervised target during fine-tuning' and builds ChronoSteer-100K-FT from 100K triplets (x, ĉ, y). This directly contradicts the abstract's repeated claim that ChronoSteer is 'trained exclusively on synthetic data.' The contradiction is load-bearing because Table 3's rm Finetune ablation shows that removing this real-label fine-tuning degrades accuracy by 10.1% (Energy), 8.3% (Traffic), and 16.6% (Weather). Thus a substantial part of the headline 25.7% improvement over the unimodal backbone comes from exposure to real future observations, not from the synthetic two-stage procedure alone. The narrower contribution, avoiding real cross-modal event-text pairs, may survive, but the stronger data-efficiency and 'exclusively synthetic' claims are not supported as written, and the 25.7%/22.5% headline percentages should be re-reported with this dependence stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"ChronoSteer proposes a decoupled multimodal forecasting framework: an LLM converts textual events into one of nine anchor revision instructions, and a lightweight alignment module steers the output of a frozen time series foundation model (Chronos-Bolt-base) conditioned on that instruction. To avoid collecting real event-series pairs, the authors design a two-stage training procedure: first, synthetic pretraining on triplets (x, c, \\tilde{y}) where \\tilde{y} is produced by applying hand-designed transformation functions to the unimodal forecast; second, pseudo-label-guided fine-tuning on triplets (x, \\hat{c}, y) where \\hat{c} is selected among the nine anchors by closeness of the model's candidate predictions to the real future series y. The paper also introduces MTSFBench-300, a January-2025 benchmark in Energy, Traffic, and Weather. The headline claim is that ChronoSteer, trained exclusively on synthetic data, improves prediction accuracy by 25.7% over the unimodal backbone and by 22.5% over the previous state-of-the-art multimodal method.","tokens_in":28584,"tokens_out":6293,"duration_ms":55704,"significance":"If the claims held as stated, the contribution would be practically valuable: it would show that a small trained adapter can steer a frozen TSFM through textual revision instructions, avoiding the need for real cross-modal event-text pairs, and it would provide a benchmark designed to mitigate LLM data leakage. The design is internally coherent, and Table 2's Function baseline is a good control demonstrating that learned steering outperforms directly applying the transformation functions. The robustness tables (Table 5) and the backbone/embedding/LLM ablations in Appendix B are useful. However, the significance is materially reduced by two load-bearing problems: the 'exclusively synthetic data' claim is contradicted by the fine-tuning stage, which uses real future series as supervised targets, and the headline 25.7%/22.5% numbers cannot be reproduced from the reported tables. The core idea is defensible, but the paper as written overstates what is demonstrated.","major_comments":[{"comment":"The repeated claim that ChronoSteer is 'trained exclusively on synthetic data' is contradicted by the method itself. Section 3.5.1 states that 'we incorporate the real future series y as a supervised target during fine-tuning' and constructs ChronoSteer-100K-FT from 100K triplets (x, c\\u0302, y) with real y. The fine-tuning stage is not synthetic. This is load-bearing because Table 3's 'rm Finetune' ablation shows that removing this real-label fine-tuning degrades accuracy by 10.1% (Energy), 8.3% (Traffic), and 16.6% (Weather). A substantial part of the reported 25.7% improvement over the unimodal backbone therefore comes from exposure to real future observations, not from the synthetic two-stage procedure. The abstract, Section 4.2, and Section 5 should be rephrased to state that pretraining is synthetic while fine-tuning uses real future series, and the relative contributions should be re-reported accordingly.","section":"Abstract and Section 3.5.1"},{"comment":"The headline percentages (25.7% improvement over the unimodal backbone and 22.5% gain over DeepSeek-R1) cannot be recovered from the reported tables. Using the MSE values in Table 2, the per-domain improvements over the unimodal backbone are approximately 24.7% (Energy), 23.2% (Traffic), and 34.3% (Weather); averaging across rows gives 28.0%, while averaging across domains gives 27.4%. None of these equals 25.7%. For the comparison against DeepSeek-R1 in Table 1, a per-row average of relative MSE improvements is 26.2%, not 22.5%. The paper does not state how the aggregate percentages are computed (e.g., macro-average over domains, micro-average over rows, or a different metric such as MAE). Since these numbers appear in the abstract and conclusion, the authors should specify the exact aggregation formula and provide numbers consistent with the tables, or report per-domain results only.","section":"Section 4.2 and Tables 1/2"},{"comment":"The framework's coverage of textual guidance is restricted to nine hand-crafted anchor revision instructions. Section 3.4.1 derives these from 1K open-text instructions by expert distillation, and Appendix D explicitly acknowledges that 'its generalization is still restricted.' This is a central assumption rather than a peripheral detail: the anchor matching mechanism of Eq. (4) maps any LLM-generated instruction to the nearest anchor, so instructions outside the codebook are silently replaced. The paper provides no analysis of how often the LLM's instructions on MTSFBench-300 fall far from the nine anchors, no ablation varying the codebook size, and no error analysis of the cosine-similarity retrieval. Given the load-bearing role of the nine anchors for the claimed domain generality, the authors should either quantify coverage on their own benchmark or explicitly scope the empirical claims to instructions expressible by the nine anchors.","section":"Section 3.4.1 and Appendix D"},{"comment":"The pseudo-label generation strategy uses the real future series y to select the instruction whose candidate prediction is closest to y. This is a reasonable training heuristic, but it means the instruction labels are derived from the prediction target. The paper should disclose this clearly in the abstract and method overview; the current phrasing 'trained exclusively on synthetic data' obscures it. Moreover, since the selection is among only nine candidates produced by the pretrained model, the fine-tuning signal can only reinforce what those nine anchors can express, compounding the coverage concern raised above. This does not invalidate the method, but the data-efficiency claim should be restated as 'no real cross-modal event-text pairs are required' rather than 'trained exclusively on synthetic data.'","section":"Section 3.5.1 and Eq. (9)"}],"minor_comments":[{"comment":"There is a typo in the model name 'V oyage-3-lite'; it should be 'Voyage-3-lite'.","section":"Appendix B.4.2"},{"comment":"The text 'LLaV A' should be 'LLaVA' for consistency with the reference [22].","section":"Section 4.4"},{"comment":"The table header uses 'Hist Pred' without a space and does not list the prediction lengths for the Energy dataset (which are 48 in the text but the column says 48 for all rows); please clarify whether 'Pred' values are prediction lengths or something else.","section":"Table 4"},{"comment":"The description of MTSFBench-300 says the benchmark 'incorporates data updated to January 2025,' but the test period is January 2025; please clarify whether the data includes only January 2025 or also earlier months and how the train/validation/test splits are defined for the benchmark.","section":"Section 4.1.1"},{"comment":"Table 5 reports standard deviations only for ChronoSteer, not for the baselines. Since the headline comparisons are against DeepSeek-R1 and other baselines, reporting baseline variance or confidence intervals would strengthen the claim that the observed improvements exceed run-to-run variability.","section":"Appendix B.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the two-stage design is largely coherent, but the 'exclusively synthetic data' claim is factually contradicted by the fine-tuning stage and appears in the abstract, contributions, and conclusion. The headline percentages also need a precise derivation. These are fixable in revision, so I am not recommending rejection. Please ask the authors to re-run their aggregation, clarify the role of real future series in fine-tuning, and either quantify or soften the nine-anchor coverage claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — read ChronoSteer. The core idea is genuinely new and worth your time: an LLM compresses free-text events into a small set of revision instructions, and a lightweight adapter steers a frozen TSFM with those instructions. The Function baseline in Table 2 is the right control, and it shows learned steering beats directly applying the hand-designed transformations. The two-stage training (synthetic pretraining, then pseudo-label fine-tuning) is sensible, the fresh benchmark addresses real leakage concerns, and the ablations are careful. Credit also for the honest limitations section: it admits the nine-anchor restriction.\n\nThe soft spot is load-bearing. The abstract, intro, and conclusion all say 'trained exclusively on synthetic data.' Section 3.5.1 says the opposite: fine-tuning uses real future series y as supervised targets, and Eq. 9 picks pseudo-labels by closeness to y. The rm Finetune ablation shows removing that stage costs 10.1% (Energy), 8.3% (Traffic), and 16.6% (Weather). So a big part of the headline 25.7% gain comes from real future observations, not the synthetic procedure. That doesn't kill the method — the actual contribution is avoiding real cross-modal event-text pairs — but the central data-efficiency claim is overstated as written.\n\nAlso two smaller issues. First, the headline 25.7% and 22.5% are not exactly recoverable from Table 1; simple averaging across the twelve settings gives ~28%, so either a different aggregation or a rounding mismatch needs stating. Second, no code or data is released despite promises, and baseline variance isn't reported. The nine anchors are a genuinely narrow bridge, as Appendix D concedes.\n\nOn balance: the central argument holds up well enough to deserve serious referee time. The fix is re-framing the 'exclusively synthetic' claim, reporting exact aggregation, and releasing artifacts.","headline":"A genuinely new steering idea with a solid control, but the 'exclusively synthetic' claim is contradicted by real-label fine-tuning and the headline numbers don't exactly reproduce — still worth refereeing.","tokens_in":29085,"tokens_out":3476,"would_cite":false,"duration_ms":29299,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ChronoSteer claims that a frozen time-series foundation model can be steered by natural-language revision instructions through a small trained adapter, and that training this adapter solely on synthetic instruction-series pairs beats both…","keywords":["multimodal time series forecasting","time series foundation model","large language model","revision instructions","synthetic training data","pseudo-label fine-tuning","zero-shot forecasting","information leakage"],"falsifier":"Find a set of events whose effect on the series is not one of the nine transformations—for example, a one-day strike that shifts the morning rush-hour peak two hours later while leaving its size unchanged—and ask the LLM to rewrite them as revision instructions. If the full ChronoSteer pipeline cannot reduce MSE below the unimodal forecast on a held-out batch of such events, the nine-anchor assumption is falsified.","tokens_in":28126,"feed_emoji":"📈","tokens_out":10613,"duration_ms":98441,"temperature":0.7,"pith_summary":"ChronoSteer attacks the shortage of paired event-text and time-series data by splitting multimodal forecasting into two roles: a large language model rewrites messy textual context into a short, explicit \"revision instruction\" (for example, \"lower the peaks\"), and a small adapter on top of a frozen time-series foundation model applies that instruction to the model's forecast. The paper claims that training this adapter exclusively on synthetic pairs—generated by applying nine hand-coded revision functions to unimodal forecasts—transfers to real-world zero-shot forecasting. In experiments across energy, traffic, and weather benchmarks, the full system beats its unimodal backbone by 25.7% in MSE and the previous best multimodal method by 22.5%, at nearly the same inference cost as the backbone. The same paper contributes MTSFBench-300, a leakage-conscious benchmark with data updated to January 2025, and a pseudo-label fine-tuning stage that lets real future series correct imprecise instructions.","feed_headline":"Time-series model cuts error 25.7% with text instructions","feed_subtitle":"A small adapter lets a frozen time-series model follow weather and calendar text, trained only on synthetic pairs.","key_machinery":"The load-bearing mechanism is the anchor codebook plus its paired transformation functions: nine instructions—\"keep unchanged, increase/reduce trend, expand/compress amplitude, elevate/lower peaks, raise/deepen troughs\"—each with a hand-designed analytic transformation of the unimodal forecast. At inference, the LLM's revision instruction is embedded, matched by cosine similarity to the nearest anchor, and converted by the aligned MLP into one token placed at the beginning of the time-series token sequence, exploiting the tendency of Transformers to attend strongly to early tokens. During pre-training, these nine transformations create dense supervision (900K synthetic triplets from a public time-series archive), and a contrastive loss forces outputs tied to different anchors apart. During fine-tuning, optimal matching against real future series replaces function-transformed targets with real targets, letting the model exceed the ceiling of the hand-coded functions.","core_discovery":"On the paper's own terms, the central discovery is that cross-modal alignment for forecasting does not require the model to understand open-ended text; it can be reduced to choosing one of nine canonical \"revision instructions\" that reshape a unimodal forecast. ChronoSteer maps a text embedding to the nearest anchor in a pre-constructed codebook of nine instruction embeddings, projects it through a two-layer MLP into a single token, and prepends that token to the frozen TSFM's patch embeddings. Training proceeds without any real text-series pairs: pre-training uses function-transformed synthetic futures with a contrastive loss that separates different anchors, and fine-tuning uses pseudo-labels chosen as the instruction whose candidate forecast is closest to the real future series. After this synthetic-only training, the model steered by an LLM-generated instruction reduces prediction MSE by 25.7% relative to its unimodal backbone and by 22.5% relative to the previous state-of-the-art multimodal method.","pith_inferences":["The same architecture could support a larger or hierarchical codebook: the nine anchors are a training-data choice, not a structural limit, so richer instruction dictionaries are a natural next step that the paper does not explore.","A clean test would replace nearest-neighbour anchor retrieval with a soft mixture of anchors; if that helps on out-of-distribution events, the hard nine-way discretisation is the bottleneck rather than the frozen TSFM.","The pseudo-label fine-tuning stage is effectively using real but unlabelled series to correct instruction errors, which suggests a semi-supervised route to scaling cross-modal forecasting without any paired annotations."],"forward_implications":["Plugging a better reasoning LLM into the instruction-generation step should improve forecasts directly, because the trained ChronoSteer itself never sees raw text at inference.","Any domain with useful textual context can adopt the same recipe—frozen TSFM, text embedder, anchor codebook, two-stage synthetic training—without collecting paired event-series data.","The new benchmark offers a leakage-conscious way to measure multimodal forecasting on data issued after LLM knowledge cutoffs.","Ablation results show that each ingredient matters: removing context, contrastive loss, fine-tuning, or the nonlinear alignment module degrades accuracy, with fine-tuning removal costing 8–17% depending on domain."],"supporting_citations":[{"why":"Supplies the frozen TSFM backbone and the unimodal baseline used for the headline 25.7% improvement claim.","marker":"[1]"},{"why":"Provides the public time-series archive from which synthetic instruction-series triplets are sliced for pre-training.","marker":"[12]"},{"why":"Serves as the previous state-of-the-art multimodal baseline against which the reported 22.5% gain is measured.","marker":"[34]"},{"why":"Is the reasoning LLM that converts textual events into revision instructions in the deployed system.","marker":"[38]"},{"why":"Provides the text embedding model used before anchor matching and the MLP alignment module.","marker":"[5]"},{"why":"Is the prior multimodal TSFM baseline and supplies the clustering and evaluation paradigm adopted for benchmark construction.","marker":"[39]"},{"why":"Supplies the contrastive loss used in pre-training to separate different anchor instructions.","marker":"[32]"},{"why":"Justifies prepending the instruction token at the front of the time-series token sequence.","marker":"[19]"}],"fun_headline_variants":["Synthetic-only training lets LLM steer time-series forecasts, 25.7% better","Adapter + 9 text anchors: frozen time-series model follows LLM, 25.7% error cut","LLM tells time-series model what to fix: 9 instruction types, 25.7% better","Synthetic pairs only: ChronoSteer steers frozen TSFM with LLM, 25.7% gain","No real text pairs: codebook of 9 instructions lets LLM steer TSFM, 25.7%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every consequential textual event can be compressed into one of nine hand-designed revision instructions, each implemented as a fixed mathematical reshaping of the unimodal forecast; if a real event needs a different kind of adjustment, the bridge from language to forecast stops working.","fun_headline_variants_meta":{"raw":{"variants":["Synthetic-only training lets LLM steer time-series forecasts, 25.7% better","Adapter + 9 text anchors: frozen time-series model follows LLM, 25.7% error cut","LLM tells time-series model what to fix: 9 instruction types, 25.7% better","Synthetic pairs only: ChronoSteer steers frozen TSFM with LLM, 25.7% gain","No real text pairs: codebook of 9 instructions lets LLM steer TSFM, 25.7%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001174,"raw_usage":{"total_tokens":4871,"prompt_tokens":981,"completion_tokens":3890,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":597,"completion_tokens_details":{"reasoning_tokens":3754}},"tokens_in":597,"tokens_out":3890,"duration_ms":25173,"temperature":1.0,"reasoning_tokens":3754,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:16:49.709830+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find a set of events whose effect on the series is not one of the nine transformations—for example, a one-day strike that shifts the morning rush-hour peak two hours later while leaving its size unchanged—and ask the LLM to rewrite them as revision instructions. If the full ChronoSteer pipeline cannot reduce MSE below the unimodal forecast on a held-out batch of such events, the nine-anchor assumption is falsified.","supporting_citations":[{"cited_title":"Webb, Rob J","cited_arxiv_id":null,"evidence_quote":"Provides the public time-series archive from which synthetic instruction-series triplets are sliced for pre-training."},{"cited_title":"Qwq-32b: Embracing the power of reinforcement learning, 2025","cited_arxiv_id":null,"evidence_quote":"Is the reasoning LLM that converts textual events into revision instructions in the deployed system."},{"cited_title":"Chattime: A unified multimodal time series foundation model bridging numerical and textual data","cited_arxiv_id":null,"evidence_quote":"Is the prior multimodal TSFM baseline and supplies the clustering and evaluation paradigm adopted for benchmark construction."},{"cited_title":"Learning transferable visual models from natural language supervision","cited_arxiv_id":null,"evidence_quote":"Supplies the contrastive loss used in pre-training to separate different anchor instructions."}],"review_version":1}