{"id":"5053ab9c-1c0d-45e4-b13f-e7f220ecc897","arxiv_id":"2608.05238","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CGTime trains a time-series-language model on deterministically computed statistics, using them as both supervision and reward, and reports strong performance on its own multivariate benchmark compared with larger general-purpose models.","lead":"This paper presents CGTime, a 4-billion-parameter model that describes time series by first computing statistical facts with deterministic code and then letting a language model put those facts into words. The method is designed to avoid the trap where AI labels data using the very perception skills it is supposed to learn, and the authors report large gains over bigger models on their own multivariate understanding benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark validity is the load-bearing weakness: all evaluations are aligned with training/reward (TSQA is also in the training mix), so the multivariate advantage may reflect in-distribution optimization, not general understanding.","rationale":"The reader's weakest assumption identifies the same structural issue I find most load-bearing: the internal benchmark is not independent of the training signal. I agree with that assessment and add two sharpening observations. First, the supposed external benchmark TSQA is included in training (Joint-SFT/Joint-GRPO), so it does not provide an out-of-distribution test; the cleaned evaluation split is only disjoint by time-series signature, not by task type or question format. Second, the family analysis in Section L shows the advantage is concentrated in the Joint multivariate family, which is precisely the family the reward was designed to optimize; on Direct univariate readout the model is dramatically worse than GPT-5.4-nano, indicating that the method instills specific statistical estimation skills rather than a general ability to read multivariate series. I also flag a concrete internal contradiction: the paper excludes causal links but includes Granger causality p-values in the metric library (Table 10), which undercuts the 'deterministic, no model-based inference' framing. These concerns do not invalidate the engineering contribution: the ablation replacing computed statistics with GPT-perceived supervision (Table 20) is a strong internal control, and the release plans are detailed. But the central comparative claim remains conditional on a truly external benchmark with unseen statistics and scoring. Therefore I keep the reader's CONDITIONAL verdict.","tokens_in":51579,"tokens_out":7728,"duration_ms":86244,"concrete_test":"Build a transfer evaluation from the same open-source multivariate series (or new sources) using 30-50 statistical features not in the 169-property inventory, computed with an independent implementation (e.g., tsfresh) and scored with a tolerance never used as a reward. Run the released CGTime checkpoint and the same ten baselines on this set. If the multivariate advantage over GPT-5.4-nano (Table 2) does not replicate, the central claim is an artifact of training-evaluation alignment; if it persists, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline advantage is established only on benchmarks whose construction is aligned with the training objective. Metric-QA and Captioning use the same 169-metric library and the same Gaussian kernel for ground truth, training reward, and evaluation (Sections 2.6, G.1, H.1); the sole external benchmark, TSQA, is itself part of the training mixture in Joint-SFT and Joint-GRPO (Sections 2.6, D.1). No evaluation is performed on a benchmark whose question set, metric definitions, and scoring function were unseen during training. The multivariate advantage in Table 2 may therefore reflect in-distribution optimization over the exact properties and tolerances the model was rewarded to emit, rather than general multivariate understanding. The statistic-family analysis (Tables 17-19) is consistent with this: CGTime's gain is concentrated in the Joint relational/system family (PCA, synchrony, lead-lag) emphasized in training, while it scores far below GPT-5.4-nano on Direct level/scale readout (0.137 vs 0.590). A separate internal inconsistency weakens the design claim: the paper states it excludes causal links (Section 1) yet the metric inventory (Table 10) includes granger_xy_pvalue and granger_yx_pvalue. A true transfer test on unseen statistics is required before the central claim can be accepted beyond the authors' benchmark.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CGTime, a 4B-parameter time-series–language model trained by decoupling perception from description: deterministic code computes 169 statistical properties from real multivariate series, an LLM verbalizes those precomputed facts, and the same computed facts serve as supervised-training targets, RL rewards, and evaluation references. The authors claim that CGTime outperforms much larger general-purpose and specialist models on multivariate time-series understanding, with the headline result being a Metric-QA mean fact score of 0.283 on the multivariate held-out set versus 0.203 for GPT-5.4-nano and 0.173 for GPT-4o-mini, with Holm-corrected paired significance. They also report higher caption precision/recall and competitive TSQA accuracy. The paper includes detailed appendices on data construction, prompts, scoring, significance testing, ablations, and reproducibility artifacts.","tokens_in":51919,"tokens_out":3725,"duration_ms":39901,"significance":"If the empirical claim held on a genuinely unseen benchmark, the decoupling principle would be a valuable contribution to time-series–language alignment: replacing LLM-perceived labels with deterministic, verifiable statistics is a sensible way to escape the self-supervision trap, and the paper's open pipeline, reproducible scoring, and ablation comparing computed versus GPT-perceived supervision (Appendix M) are concrete strengths. However, the central evidence is currently confined to benchmarks constructed from the same 169-metric pipeline and scored with the same Gaussian kernel used as the training reward, and the only external benchmark (TSQA) is itself included in training. The claimed general multivariate understanding therefore is not yet established beyond in-distribution optimization. The paper is transparent about this anchoring (Appendices G and H explicitly state that M_i grounds training and evaluation), which is commendable, but transparency does not remove the need for a valid transfer test.","major_comments":[{"comment":"The evaluation is circular with respect to the training objective. Metric-QA and Captioning are generated from the same 169-metric pipeline that produced training labels and RL rewards, and the evaluation scorer is the same Gaussian kernel used as the training reward (Eq. (2) and Eq. (4), with evaluation fixing m=0.5 while training uses split-specific multipliers). The paper's own protocol states that 'Metric-QA and Captioning use the computed statistics in M_i for supervision, RL rewards, and evaluation' (App. G.1). The only external benchmark, TSQA, is explicitly included in the training mixture during Joint-SFT and Joint-GRPO (App. D.1), and its evaluation split is disjoint from training only at the series-signature level, not at the level of question construction or metric definitions. Consequently, the headline multivariate advantage (Table 2) may reflect optimization toward the exact metric inventory, tolerances, and scoring function, rather than general multivariate time-series understanding. A true transfer test is required: for example, evaluating on a benchmark whose question set, metric definitions, and scoring function were entirely unseen during training, or training without TSQA and then evaluating on TSQA. Until such a test is provided, the central claim should be read as in-distribution benchmark performance, not general capability.","section":"§3.2, App. L, Tables 17–18"},{"comment":"The statistic-family analysis weakens the interpretation of the multivariate advantage as evidence of general understanding. Table 17 shows that CGTime scores 0.137 on Direct level/scale readout questions, far below GPT-5.4-nano's 0.590, while its strong performance is concentrated in the Joint relational/system family (0.283), which coincides with the PCA, synchrony, and lead-lag statistics that are emphasized in training and rewarded during RL. Because Direct and Marginal questions appear only in the univariate split and Joint questions only in the multivariate split (Table 16), the family comparison is nested with variable count, and the Joint-family advantage is not separable from the multivariate training emphasis. This pattern is exactly what would be expected if the model had learned to reproduce the specific computed statistics it was rewarded to emit, rather than acquiring a general multivariate understanding.","section":"§1 vs Table 10"},{"comment":"There is an internal inconsistency in the exclusion of causal links. Section 1 states: 'We exclude causal links, which require assumptions beyond observational data.' Yet the metric inventory in Table 10 includes granger_xy_pvalue and granger_yx_pvalue, which are Granger-causality test p-values. If these metrics are computed, used in training, or evaluated, the stated exclusion is violated; if they are not used, the discrepancy should be clarified in the inventory and in the main text. This matters because the 'no causal assumptions' claim is part of the paper's methodological framing.","section":"§2.3, §3.1, App. E/F"}],"minor_comments":[{"comment":"The term 'held-out benchmark' is used for Metric-QA and Captioning, but held-out refers only to time-series signatures not seen in training; the question templates, metric definitions, extraction rules, and scoring function are the same as those used to generate training labels and RL rewards. Consider renaming this to 'series-disjoint internal benchmark' and explicitly stating the remaining alignment with the training pipeline in the main text.","section":"Table 3"},{"comment":"The TSQA comparison with PATRA is footnoted as not being head-to-head because PATRA uses a different split and evaluation protocol, but the main text says CGTime is 'numerically close to PATRA’s strongest reported results.' This phrasing invites a direct comparison that the protocol does not support; please add an explicit caveat in the main text or move the PATRA comparison entirely to the appendix.","section":"App. I.4, App. H.1"},{"comment":"The captioning extractor is the same rule-based extractor used to compute the RL reward, and the caption score uses the same Gaussian kernel. This means caption scores partially reflect how well a model's phrasing matches the extractor's patterns, not only the factual content. A short discussion of this extraction-conditioned nature in the main text would improve interpretability.","section":"§2.6, App. G.1"},{"comment":"The paper states in Appendix G.1 that 'Metric-QA and Captioning use the computed statistics in M_i for supervision, RL rewards, and evaluation.' This is transparent, but in the main text the same fact is only implicit. A prominent limitation paragraph in the main text would help readers avoid overinterpreting the internal benchmark results.","section":"§2.6, App. G.1"}],"recommendation":"major_revision","confidential_remarks":"The core issue is benchmark validity. The paper's strongest contribution—the decoupling principle and the reproducible pipeline—is undermined by the fact that the headline empirical claim is demonstrated only on in-distribution benchmarks. I would not reject the paper outright because the design idea is sound and the authors have already included useful ablations (e.g., computed vs. GPT-perceived supervision). However, major revision is necessary: a transfer experiment on a genuinely unseen benchmark (or a training-free TSQA evaluation) is needed to establish general multivariate understanding. If such an experiment is impossible within the current scope, the authors should substantially soften the general claims and reposition the paper as a demonstration of in-distribution statistical fact retrieval."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things. The core idea is real: computing a fixed library of 169 deterministic statistics from real multivariate series, verbalizing them with an LLM, and reusing those same statistics as SFT targets and GRPO rewards is a clean answer to the self-supervision trap. The paper executes it carefully, with a sensible PCA-based O(rK) description scheme, strict-vs-permissive parser asymmetry that favors the baselines, a full metric inventory with per-metric tolerances, and a reproducibility plan that includes the checkpoint and evaluation scripts. That is more than most papers in this area ship.\n\nThe ablation comparing computed statistics against GPT-perceived pseudo-labels is the strongest evidence in the paper: replacing computed targets with GPT-5-nano generations drops multivariate Metric-QA from .282 to .157 and caption precision from .383 to .121, which supports the decoupling thesis independent of the benchmark debate.\n\nNow the soft spot, and it is load-bearing. Metric-QA and Captioning use the same metric definitions, the same Gaussian kernel, and largely the same pool of series for training, reward, and evaluation. TSQA, the only external benchmark, is itself part of the Joint-SFT and Joint-GRPO training mixture. No evaluation is done on a question set whose statistics and scoring were unseen during training. The statistic-family analysis makes the circularity visible: CGTime's advantage is concentrated in the Joint relational/system family (PCA, synchrony, lead-lag) that training emphasizes, while on Direct level/scale readout it scores .137 versus GPT-5.4-nano's .590. So the paper's own numbers suggest the model is optimized for the inventory it was trained on, not for general multivariate understanding. The authors acknowledge the direct-readout bottleneck in Appendix L and the Limitations, which is honest, but they do not provide a true transfer test.\n\nOne smaller internal inconsistency: Section 1 says causal links are excluded, yet the metric listing in Table 10 includes granger_xy_pvalue and granger_yx_pvalue. Granger causality is not structural causation, but it is squarely a causal-link statistic, so the wording needs fixing.\n\nWho gets value from this: anyone working on time-series–language alignment, RLVR, or benchmark design. The paper deserves a serious referee, but the referee should ask for an independent evaluation—ideally on statistics not present in the 169-metric family, or at minimum a held-out benchmark whose construction is fully independent of the training pipeline. I would not cite the headline comparative numbers without that; I would cite the decoupling idea and the supervision ablation.","headline":"Genuinely new supervision idea, thoroughly executed, but the headline comparison is measured on benchmarks built from the same 169-metric pipeline that provides training and reward, so the multivariate advantage is not yet established.","tokens_in":52405,"tokens_out":1818,"would_cite":true,"duration_ms":21500,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 4B-parameter model called CGTime, trained to verbalize deterministically computed statistics rather than LLM-written descriptions, outperforms far larger general-purpose models at retrieving multivariate time-series facts.","keywords":["computation-grounded supervision","multivariate time series","time-series language models","verifiable reward","representation alignment","PCA summarization","statistical fact retrieval","metric QA"],"falsifier":"Run CGTime on an external benchmark whose questions target statistical properties outside the 169-metric inventory, or whose ground truth comes from human expert annotation on unseen domains; if its score falls to parity with GPT-4o-mini or GPT-5.4-nano, the claimed advantage is benchmark familiarity rather than decoupled perception.","tokens_in":51427,"feed_emoji":"📈","tokens_out":5069,"duration_ms":45850,"temperature":0.7,"pith_summary":"Time-series–language models are usually trained on descriptions written by LLMs that read the series, so the labels cannot be more perceptually skilled than the model being trained. This paper argues that supervision should decouple perception from description: deterministic code computes a fixed library of 169 statistics from real multivariate series, and an LLM only verbalizes those precomputed facts. The result is CGTime, a 4B-parameter model whose supervised fine-tuning and reinforcement-learning reward are both anchored to the same computed facts. On a held-out multivariate Metric-QA set, CGTime scores 0.283 versus 0.203 for GPT-5.4-nano and 0.173 for GPT-4o-mini, with the advantage surviving Holm-corrected paired significance tests. If this holds, it means multivariate statistical fact retrieval can be instilled cheaply by computation-grounded supervision rather than by scaling model size.","feed_headline":"4B model outreads GPT-4o-mini on multivariate series facts","feed_subtitle":"Computation-grounded training lifts its multivariate fact score to 0.283 versus 0.173 for GPT-4o-mini.","key_machinery":"The central mechanism is the metric-function library $\\mathcal{F}=\\{f_a\\}$ with $A=169$ deterministic statistic functions, each computing one marginal, cross-channel, or system-level property from a masked multivariate series. To avoid the $O(K^2)$ blow-up of describing every channel pair, PCA is applied to extract $r\\ll K$ principal components and the description covers each channel's correlation with retained components plus selected lead-lag relations, reducing complexity to $O(rK)$. The same computed facts are dual-use: they supervise SFT targets across a four-level curriculum and define a Gaussian-kernel verifiable reward in GRPO, whose factual component is computed by a rule-based extractor matching generated claims against $M_i$. The architecture is a frozen MOMENT encoder, a trainable channel–patch alignment module that compresses into temporal and channel-summary tokens, a scale encoder adding per-channel mean and standard-deviation tokens, and an unfrozen Qwen3-4B language model.","core_discovery":"The paper's central claim is that the reliability–realism–scalability trilemma of time-series–language supervision can be resolved by splitting the two roles that LLM-written labels conflate. Perception is the part LLMs do badly, so CGTime assigns it to deterministic computation: a library of 169 metric functions computes verifiable statistics from each real series, including PCA-based summaries of cross-channel structure, lead-lag relations, and risk indicators. Description is what the language model actually learns: it is trained to verbalize those precomputed facts, with the same statistics serving as ground truth for supervised fine-tuning and as a verifiable reward during GRPO. The author's evidence is that the 4B model outperforms far larger general-purpose and specialist baselines on the multivariate metric-QA benchmark and writes captions with more accurate and more numerous numerical claims. On the paper's own terms, this establishes that computation-grounded supervision instills multivariate statistical fact retrieval into a small model better than zero-shot prompting of large LLMs.","pith_inferences":["If the benchmark anchors evaluation to the same facts used in training, the headline advantage may partly reflect task familiarity; a human-annotated, out-of-distribution statistic benchmark would be needed to confirm general understanding.","The decoupling principle transfers to other modalities where perception is hard but verification is cheap, such as audio, video, or sensor streams, provided a deterministic fact library exists.","The Direct-readout bottleneck suggests a testable extension: adding a raw-value readout path or explicit value tokens could close the gap on exact level and scale statistics without discarding the compact representation.","The 0.186 caption recall (versus 0.018 to 0.141 for external models) suggests that the density of verifiable claims, not just their correctness, is a trainable property that could be pushed further through reward shaping on coverage."],"forward_implications":["Supervision for multivariate time-series language models can be generated automatically at scale, because deterministic code replaces human or LLM labeling of every channel pair.","The PCA reduction from $O(K^2)$ to $O(rK)$ description terms makes cross-channel structure describable without enumerating all pairs, supporting models on high-dimensional series.","Using the same computed facts for SFT targets and RL rewards gives a consistent, verifiable optimization signal, so numerical claims in generated text can be audited against the source series.","The model's multivariate score holds at 0.283 versus an overall 0.288, while the strongest baselines drop from their overall scores, indicating the gain concentrates where cross-channel structure matters.","Some statistical families remain weak: direct level and scale readout scores 0.137 on forced-answer Metric-QA, so exact raw-value recovery is the identified bottleneck."],"supporting_citations":[{"why":"Time-MQA supplies the LLM-labeled QA data and one of the main time-series specialist baselines, illustrating the supervision bottleneck CGTime is designed to avoid.","marker":"[Kong et al. 2025]"},{"why":"Documents that LLMs struggle to zero-shot reason about time series, motivating the decoupling of perception from description.","marker":"[Merrill et al. 2024]"},{"why":"MOMENT provides the frozen pretrained time-series encoder that produces the patch embeddings CGTime aligns with text.","marker":"[Goswami et al. 2024]"},{"why":"DeepSeekMath introduces the GRPO objective used to optimize CGTime with group-relative advantages and no value model.","marker":"[Shao et al. 2024]"},{"why":"DeepSeek-R1 demonstrates large-scale reinforcement learning from verifiable rewards, the paradigm underlying the paper's dual-use computed-fact reward.","marker":"[DeepSeek-AI 2025]"},{"why":"The Monash Forecasting Archive is one of the two main sources of the real, open-source multivariate series used to construct statistics-grounded supervision.","marker":"[Godahewa et al. 2021]"},{"why":"ChatTS represents the synthetic-series route to controllable labels and serves as a baseline, showing the realism gap CGTime targets.","marker":"[Xie et al. 2025]"}],"fun_headline_variants":["4B model beats GPT-4o-mini on multivariate series facts","Computation-grounded training outdoes larger LLMs on series facts","Decoupling perception from description lifts small model's multivariate skills","Smaller model, bigger multivariate fact accuracy via computed grounding","4B model beats 5.4B-nano and 4o-mini on multivariate facts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the internal Metric-QA and Captioning benchmarks, whose questions and scoring are anchored to the same computed facts used for training and reward, are a valid and unbiased measure of multivariate time-series understanding; if they over-reward task familiarity, the headline advantage over zero-shot baselines does not establish general understanding.","fun_headline_variants_meta":{"raw":{"variants":["4B model beats GPT-4o-mini on multivariate series facts","Computation-grounded training outdoes larger LLMs on series facts","Decoupling perception from description lifts small model's multivariate skills","Smaller model, bigger multivariate fact accuracy via computed grounding","4B model beats 5.4B-nano and 4o-mini on multivariate facts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000431,"raw_usage":{"total_tokens":2250,"prompt_tokens":1045,"completion_tokens":1205,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":661,"completion_tokens_details":{"reasoning_tokens":1110}},"tokens_in":661,"tokens_out":1205,"duration_ms":11583,"temperature":1.0,"reasoning_tokens":1110,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T17:26:59.828851+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CGTime on an external benchmark whose questions target statistical properties outside the 169-metric inventory, or whose ground truth comes from human expert annotation on unseen domains; if its score falls to parity with GPT-4o-mini or GPT-5.4-nano, the claimed advantage is benchmark familiarity rather than decoupled perception.","supporting_citations":[{"cited_title":"2024 , note =","cited_arxiv_id":null,"evidence_quote":"MOMENT provides the frozen pretrained time-series encoder that produces the patch embeddings CGTime aligns with text."}],"review_version":1}