TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning

Dehai Min; Fangxu Yu; Ge Liu; Lu Cheng; Tao Feng; Tianyi Zhou

A heterogeneous graph that scores every modality–model pair for each time-series query raises accuracy 16–46% over fixed and model-only routers.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 05:37 UTC pith:FXKNWDE3

load-bearing objection Solid systems paper: joint modality–model routing via a four-type heterogeneous graph, with large consistent gains and real ablations; softest part is text-profile node features, but they already stress-tested it. the 3 major comments →

arxiv 2607.08940 v2 pith:FXKNWDE3 submitted 2026-07-09 cs.LG

TSRouter: Dynamic Modality-Model Selection for Time Series Reasoning

Fangxu Yu , Tao Feng , Dehai Min , Lu Cheng , Ge Liu , Tianyi Zhou This is my paper

classification cs.LG

keywords time series reasoningmodality routingmodel routingheterogeneous graphvision-language modelslarge language modelscost-aware selectionzero-shot generalization

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Time series can be shown to foundation models as raw numbers, as charts, or as both, and different language and vision models succeed on different queries and tasks. The paper argues that modality fitness and model expertise interact with task type and query traits in ways that fixed rules and model-only routers miss. TSRouter builds a small heterogeneous graph of task, query, modality, and model nodes, then scores each valid modality–model pair under a user-chosen mix of accuracy and cost. On four reasoning tasks it reaches 51.3% overall accuracy at competitive API cost, and it absorbs unseen models and held-out tasks without retraining. A sympathetic reader cares because the same text-versus-vision trade-off appears wherever numerical precision and global shape both matter, and because the graph turns joint routing into a lightweight plug-and-play decision rather than another heavy model.

Core claim

Jointly selecting the input modality and the foundation model for each time-series query, by learning scores on a heterogeneous graph of tasks, queries, modalities, and models, substantially outperforms always using the largest model or existing model-only routers, delivering 16–46% relative gains across four time-series reasoning tasks while supporting cost-aware control and zero-shot insertion of new models and tasks.

What carries the argument

TSRouter: a heterogeneous graph whose nodes are tasks, queries, modalities, and models, with edges for task–query membership, query–modality and query–model links, modality–model compatibility, and query–query nearest neighbors; node features are frozen embeddings of short natural-language profiles; a Heterogeneous Graph Transformer plus MLP scorer ranks candidates under an effectiveness score that mixes correctness and normalized cost, trained with KL soft targets.

Load-bearing premise

That short natural-language profiles of tasks, models, and modalities plus a few series statistics, without the raw series or model internals, give the graph enough signal to predict which candidate will succeed on a new query.

What would settle it

On a held-out time-series query set, if a strong non-graph router or a single best fixed modality–model pair matches or exceeds TSRouter accuracy at equal cost after the same training budget, the claim that the heterogeneous interaction graph is essential fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Solid systems paper: joint modality–model routing via a four-type heterogeneous graph, with large consistent gains and real ablations; softest part is text-profile node features, but they already stress-tested it.

read the letter

Punchline: this is a clean empirical systems paper that actually does joint modality and model selection for time-series reasoning, not just another model-only router. The four-type heterogeneous graph (task / query / modality / model) plus soft ranking on an accuracy–cost effectiveness score is the real novelty relative to GraphRouter, Hybrid-LLM, RouterDC, etc., which all fix the modality.

What works: the motivation is concrete (Figure 1 shows modality and model complementarity on the same queries). Main table is strong—51.33% overall, 16–46% relative over ten adapted baselines, competitive cost. They ablate the heterogeneous structure, query–query edges, MLP head, and modality–model edges (Table 5); show data efficiency (10% data already beats full-data Hybrid-LLM); Pareto curves under α; and zero-shot insertion of two new models and two held-out tasks. Code is public. Math is ordinary GNN + KL on soft targets; no circularity. Citations look fair.

Soft spots, in proportion: the node features are LLM-written profiles plus a few scalar series stats, never the raw series or model internals. That is the weakest design choice. But they already measured it—dropping the descriptions only costs ~0.7 points and still beats the best baseline (Table 7), and the held-out model/task results still move in the right direction. So it is a real limitation, not a load-bearing hole under the main claim. Free parameters (α, k, layers, dim) are standard and swept. No missing baseline of consequence that I can see.

Who it is for: people building multi-LLM/VLM time-series pipelines who care about cost–accuracy routing. Not a theory paper. I would bring it to reading group if the group does routers or time-series foundation models. It deserves a serious referee; I would accept for peer review and expect mostly experimental polishing, not a rewrite.

Referee Report

3 major / 6 minor

Summary. The paper proposes TSRouter, a heterogeneous-graph router that jointly selects an input modality (text, vision, or both) and a foundation model (LLM or VLM) for each time-series reasoning query. Nodes for tasks, queries, modalities, and models are initialized from frozen text embeddings of short natural-language profiles (plus scalar series statistics for queries); five edge types encode task–query, query–modality, query–model, modality–model, and k-NN query–query relations. A Heterogeneous Graph Transformer produces node embeddings, an MLP scores each valid modality–model candidate under a user-chosen performance–cost trade-off α, and training matches soft rankings via KL divergence. On TSRBench (four task families, 15 subtasks) the method reports 51.33% overall accuracy—16–46% relative gains over ten adapted baselines—at competitive API cost, with zero-shot insertion of two unseen models and two held-out tasks, cost–accuracy Pareto curves, and component ablations.

Significance. Joint modality–model routing for time series is a practically important and under-addressed problem: LLMs and VLMs have complementary numerical vs. global-pattern strengths, and models differ in per-query expertise and cost. The work goes beyond fixed-modality LLM routers by making both dimensions first-class and by modeling multi-type interactions in a single graph. Strengths that should be credited include: a clean problem formulation (Eqs. 1–2), thorough empirical stress-testing (main table, held-out models/tasks, α-sweeps, data-efficiency, GNN/embedding ablations, description-removal study), public code, and an efficiency comparison showing low routing latency and memory. If the gains hold under broader model pools and deployment settings, the framework is a useful systems contribution for cost-aware multi-modal time-series reasoning.

major comments (3)

§2.3 and §3.1 (training-data construction): Building the soft targets requires evaluating every valid modality–model candidate on every training query to obtain I(q,c) and cost(q,c). This labeling cost is load-bearing for practical adoption and is only partially mitigated by the 10% data-efficiency curve in Figure 6. The manuscript should quantify total labeling budget (API spend / number of model calls) relative to baselines and discuss whether active or sparse labeling of candidates can preserve the reported gains.
Table 3 and §3.2: Main accuracy numbers are reported as single point estimates without seed-level variance or confidence intervals, whereas the description ablation (Table 7) does report means±std over three runs. For a journal claim of 16–46% relative improvement, multi-seed means and intervals (or bootstrap CIs) on the overall and per-task metrics are needed to establish that the gap over Hybrid LLM / GraphRouter is stable.
§3.1 baseline adaptation: All baselines are adapted by treating each (modality, model) pair as an independent candidate. For graph methods originally designed for query–model edges (e.g., GraphRouter), this may not be the strongest possible extension (e.g., retaining modality as a typed node). A short note or one stronger re-implementation would make the 51.33% vs. 42.13% comparison more conclusive rather than an artifact of a flat candidate set.

minor comments (6)

Notation alternates among TSROUTER, TSRouter, and TSROUTERconstructs in the abstract and §1–2; standardize the product name and spacing.
Figure 1(b) caption and legend are dense; a short sentence in the main text defining the three disjoint correctness groups would help readers who only skim the figure.
Eq. (4) uses element-wise multiplication after additive composition of modality and model embeddings; a one-line justification (vs. concatenation or bilinear) would improve clarity.
Table 1 lists GraphRouter as capturing query–model interaction but not query–query; the main text should briefly state how TSRouter’s E_QQ edges differ from GraphRouter’s design so the table is self-contained.
Appendix case study (Figure 9 / Table 10) is useful; a pointer from §2.1 would better connect the qualitative motivation to the quantitative routing results.
Cost tables use provider pricing that will drift; note the snapshot date of the price table (Table 9) so future readers can reinterpret absolute USD figures.

Circularity Check

0 steps flagged

No circularity: empirical routing trained on observed correctness/cost and evaluated on held-out queries; no claim reduces to its inputs by construction.

full rationale

TSRouter is a standard supervised routing system. Training labels are measured per-query correctness I(q,c) and API cost (Eq. 1–2); the heterogeneous GNN is trained by KL to the soft ranking of those labels (Eq. 5–7) and is scored on held-out test queries, new models, and new tasks. Nothing in the derivation equates a claimed accuracy gain to a fitted constant or to a self-defined quantity. Self-citations (TSRBench as the evaluation suite; GraphRouter as a baseline) are ordinary related-work/benchmark usage and are not load-bearing uniqueness or ansatz imports. Component ablations and zero-shot insertion of unseen models/tasks further show the reported gains are not forced by construction. Score 0 is the correct honest finding.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 2 invented entities

The central empirical claim rests on standard ML assumptions plus a handful of design choices that are free parameters or domain premises. No new physical entities are postulated; the only invented construct is the four-type routing graph itself.

free parameters (4)

α (performance–cost trade-off weight)
User-chosen scalar in the effectiveness score (Eq. 1); different values produce the Pareto curves in Figure 5. Not fitted to maximize a single reported number, but still a free design knob that affects which candidate is optimal.
k (nearest-neighbor query edges)
Set to 60 after validation search; controls density of E_QQ edges and therefore message-passing neighborhood size.
GNN embedding dimension and layer count
Chosen as 64 and 2 after the sensitivity plots in Figure 7; both are free architectural hyperparameters that affect capacity and oversmoothing.
learning rate and optimizer settings
Adam 1e-3; standard but still free choices that influence final router quality.

axioms (4)

domain assumption LLM and VLM capabilities on time series are complementary and query-dependent (text preserves numbers, vision captures global shape).
Stated as the motivating observation in Section 2.1 and Figure 1; the entire routing problem is predicated on this premise.
ad hoc to paper A frozen text embedding of a short natural-language profile plus a few scalar series statistics is an adequate initial feature for every node type.
Section 2.3 node-set construction; no theoretical guarantee that these features suffice for effectiveness prediction.
domain assumption Soft KL targets derived from observed effectiveness scores provide richer supervision than hard argmax labels.
Training objective (Eqs. 5–7); common ranking practice but still an unproved modeling choice for this setting.
standard math Heterogeneous Graph Transformer layers correctly aggregate the five edge types into useful node representations.
Uses the published HGT architecture (Hu et al., 2020) without modification of its mathematical definition.

invented entities (2)

Four-type heterogeneous routing graph (task / query / modality / model nodes with five relation types) no independent evidence
purpose: To contextualize interactions so that a GNN can score modality–model candidates for each query.
The graph schema is defined by the authors for this problem; it is not a pre-existing object in the literature.
Effectiveness score e(q,c) = α·I − (1−α)·normalized cost no independent evidence
purpose: Scalar objective that turns multi-objective routing into a single ranking problem.
Defined in Eq. 1; the particular linear combination is a design choice of the paper.

pith-pipeline@v1.1.0-grok45 · 24959 in / 2844 out tokens · 34633 ms · 2026-07-13T05:37:11.385559+00:00 · methodology

0 comments

read the original abstract

Time series reasoning is essential for real-world problem-solving. While both Large Language Models (LLMs) and Vision-Language Models (VLMs) can reason about time-series data, their capabilities are complementary: LLMs process time series as text sequences and thus preserve exact numerical understanding, but struggle with global patterns, whereas VLMs efficiently capture these patterns by visualizing time series but may lose fine-grained details. Moreover, models vary significantly in task-specific expertise and inference costs. Dynamically selecting the most suitable modality and model for each query is therefore crucial, yet challenging because it requires modeling the complex interactions among tasks, queries, modalities, and models, which carry rich contextual signals. To this end, we introduce TSRouter, a graph-based dynamic routing framework. TSRouter constructs a heterogeneous graph of task, query, modality, and model nodes to contextualize the interactions among query characteristics, modality attributes, and model capabilities. TSRouter formulates routing as a candidate scoring problem, where each modality-model pair is evaluated based on user-defined performance-cost preferences to select the optimal candidate. Comprehensive evaluations on 4 distinct time series reasoning tasks reveal that TSRouter substantially outperforms diverse baselines with 16\% to 46\% relative improvements. Furthermore, TSRouter demonstrates robust zero-shot plug-and-play generalization to unseen models and novel tasks and preserves high performance while reducing computational overhead through cost-aware optimization. Our code is available at https://github.com/tianyi-lab/TSRouter.