{"id":"92e5baee-e4f0-42aa-9556-ca89af4f4adb","arxiv_id":"2412.03427","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A simulation-based evaluation pipeline shows that Moirai embeddings distort physiological signal structure, with feature entanglement, lost temporal dynamics, and reduced scenario discrimination.","lead":"This paper tests how well a time series foundation model represents simulated patient monitoring signals. The Moirai model distorts the signals, so the authors argue that models need fine-tuning before medical use.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Independent per-feature projection may make the reported distortions artifacts of input formatting rather than intrinsic model limitations.","rationale":"The paper is an honest, early-stage evaluation, and its negative result for Moirai could be true. But the argument's load-bearing inference is from 'this specific embedding protocol distorts signals' to 'foundation models need architectural modification for clinical use.' That inference fails if the protocol is unrepresentative. The reader identified the same weakest assumption, and I agree. The concern is not an ad hominem or a disagreement with a consensus; it is a correctness risk about whether the measured effect is caused by the model or by the input format. The proposed test is decisive because it isolates the independent-projection choice while holding the model and metrics fixed. Given the paper's own framing as work-in-progress and the missing code/data, a conditional verdict remains appropriate; this concern strengthens the condition rather than overturning the result.","tokens_in":7657,"tokens_out":6609,"duration_ms":74021,"concrete_test":"Run the full Section 2.3 evaluation twice on the same seven BioGears scenarios: (1) the paper's Section 2.2 per-feature univariate projection, and (2) a joint multivariate projection in which all seven normalized channels are passed to Moirai simultaneously as one multivariate time series, keeping interpolation, normalization, PCA thresholds, and linear probes identical. If the four headline metrics—cross-feature correlation, reconstruction R-squared, smoothness, and scenario dimensionality—change by less than the within-scenario variability, the reported distortions are robust; if they improve substantially (e.g., reconstruction R-squared above 0.9 or smoothness matching raw signals), the central negative result is an artifact of independent per-feature projection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Moirai's final-layer embeddings systematically distort physiological signals. The evidence rests on Section 2.2's projection step: each of the seven BioGears features is passed through the transformer independently, after linear interpolation to 1000 timesteps. Under this protocol, the model never sees the multivariate physiological state; it sees seven unrelated univariate series. Consequently, the headline results—spurious cross-feature correlations (Section 3.2), poor reconstruction (Section 3.3), loss of temporal smoothness (Section 3.4), and reduced scenario discrimination (Section 3.5)—could reflect how a univariate series is formatted and patched, rather than how the foundation model would represent multivariate physiological signals in a clinical deployment. The paper even computes 'feature independence' using correlations across separately embedded channels; any shared encoder weight can induce correlations between independently processed inputs. The conclusion that 'current foundation models may require substantial architectural modifications' (Section 4.2) therefore overgeneralizes from one model, one projection scheme, and one simulation-only dataset. A control condition is missing: feeding the same signals jointly through the model's native multivariate interface would determine whether the distortions are intrinsic or artifacts of the evaluation pipeline.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a work-in-progress pipeline for assessing how well time-series foundation models transfer to physiological signals in precision medicine. The pipeline generates synthetic physiological data with the BioGears simulator across several clinical scenarios (e.g., hemorrhage, sepsis, multi-organ failure), linearly interpolates each of seven physiological features to 1000 timesteps, independently projects each feature through the Moirai time-series foundation model to obtain final-layer embeddings, and then evaluates those embeddings with linear probes: Pearson correlations among features, linear reconstruction R², PCA-based dimensionality and trajectory smoothness, cosine similarity between scenarios, and logistic-regression decoding of feature identity. The authors report that Moirai's embeddings introduce spurious cross-feature correlations, poorly reconstruct the raw signals, destroy temporal smoothness and dimensionality, reduce scenario discrimination, and lower feature-decoding AUC from 0.96 to 0.78. From these results they conclude that current foundation models may require substantial architectural modification or targeted fine-tuning before clinical deployment.","tokens_in":7886,"tokens_out":3238,"duration_ms":30956,"significance":"If the reported effects were shown to be intrinsic to the model class, the paper would provide a useful, low-cost benchmarking methodology for an important application area. The pipeline is clearly described and the use of linear probes is appropriate for isolating representation quality from classifier power. The authors also deserve credit for explicitly framing the work as a work-in-progress and for naming ongoing extensions. However, the central claim as stated is not yet supported: the evaluation protocol projects each feature independently, so the reported distortions may be artifacts of input formatting rather than intrinsic limitations of the foundation model. The paper also provides no code, no data, no real-physiological-signal validation, and reports almost no error bars or experimental scale, which limits the reproducibility and robustness of the quantitative findings. The contribution is therefore a promising evaluation framework with initial, but not yet conclusive, results.","major_comments":[{"comment":"The evaluation rests on a per-feature projection: each of the seven features is linearly interpolated to 1000 timesteps and passed independently through Moirai's transformer module, so the model never sees the multivariate physiological state. Under this protocol, the reported spurious cross-feature correlations (§3.2), reconstruction loss (§3.3), and temporal dynamics distortions (§3.4) could be artifacts of formatting and patch construction for univariate series rather than intrinsic limitations of the model's representation of physiological signals. A mandatory control is to feed the same signals jointly through Moirai's native multivariate interface, or at least to vary patch size and interpolation length and to compare against a baseline encoder; without such a control, the central conclusion in §4.2 is not supported.","section":"§2.2"},{"comment":"The feature-decoding experiment is interpreted with an arbitrary threshold: AUC > 0.9 is called 'effective disentanglement,' and the drop from raw AUC 0.96 to embedded AUC 0.78 is presented as compromised clinical utility. But pairwise linear separability of feature identity is a measure of feature distinctiveness, not of disentanglement, and a 0.78 AUC is not by itself evidence of clinical non-usability. The paper does not tie decoding performance to any downstream clinical task, so the claim that 'reduced capacity for feature decoding impairs clinical utility' is not established.","section":"§3.6 and §2.3"},{"comment":"The quantitative results are reported without the experimental scale needed to assess them: the number of BioGears virtual patients, scenarios, sessions, and simulation runs is never stated, and only the decoding AUC has error bars. The smoothness metric depends on 1000 random permutations, but no sampling distribution or sensitivity analysis is given; the PCA variance threshold (90%) and the decoding threshold (0.9) are fixed without justification. These omissions make it impossible to know whether the reported distortions are robust or within noise.","section":"§3"},{"comment":"The scope of the empirical evidence is one model (Moirai), one simulation package's default scenarios, and no real physiological recordings. The abstract and §4.2 nevertheless conclude that 'current foundation models may require substantial architectural modifications or targeted fine-tuning before deployment in clinical settings.' This overgeneralizes from a single univariate projection protocol; at minimum, the claims should be restricted to Moirai's zero-shot univariate embeddings and to the BioGears synthetic distribution.","section":"§4.1 and §4.2"}],"minor_comments":[{"comment":"The text repeatedly refers to 'Fig. 2.3' where the only pipeline figure is labeled Fig. 1; all cross-references should be corrected.","section":"§3.2–§3.5"},{"comment":"The model name is spelled inconsistently as both 'Moirai' and 'Moiria' (for example in the abstract, §4.1, §4.2, and the Fig. 1 caption); the latter is a typo.","section":"Throughout"},{"comment":"The linear interpolation to 1000 timesteps is described but its effect is never analyzed; a sensitivity analysis over interpolation lengths would strengthen the claim that the results are not formatting artifacts.","section":"§2.2"},{"comment":"The logistic-regression decoding setup is underspecified: which feature pairs, how many samples, and which regularization were used are not reported, and the pairwise AUC values for particular features mentioned in §3.6 are not shown.","section":"§2.3 and §3.6"},{"comment":"The figure contains many panels with dense small text; the caption does not identify the simulated scenario shown in panels D(left) and E(top-left), and the color and type legends are hard to distinguish in print.","section":"Fig. 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is honest about being work in progress, but the lack of code, data, and real-signal validation, combined with the per-feature projection protocol, makes the headline claim considerably stronger than the evidence. I would advise the editor that the paper is likely to be suitable only after the authors add the multivariate-input control and substantially constrain the generalization claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper proposes a simulation-to-linear-probe pipeline for checking whether a time series foundation model preserves physiological signal structure, and demonstrates it on Moirai. The pipeline idea is genuinely useful: BioGears gives you controllable, clinically themed scenarios, and the linear probing is cheap. If you're a group deciding whether to fine-tune a foundation model for monitoring data, this kind of capability audit is worth having. The paper's negative result is also not trivial: Moirai's embeddings do show feature entanglement, poor reconstruction, distorted temporal dynamics, and worse scenario discrimination in their protocol. That's a plausible red flag.\n\nThe soft spots are real, though. The biggest one is the projection step in Section 2.2: each physiological feature is interpolated and passed through Moirai independently. So the model never sees a multivariate physiological state; it sees seven unrelated univariate series. Under that protocol, the spurious cross-feature correlations could be an artifact of how the model handles input formatting and patching, not an intrinsic property of the representations. The authors never show a control where the features are fed jointly through the model's native multivariate interface, so the central conclusion — that 'current foundation models may require substantial architectural modifications' — overreaches. Also missing: code/data, simulation counts, error bars on most metrics, any real-data validation. The AUC threshold of 0.9 for 'effective disentanglement' is presented without justification. And there's a concrete inconsistency: the text lists acute hemorrhage, sepsis, and multi-organ failure, but Figure 1 shows a burn scenario and a radiation scenario. That needs fixing.\n\nI'm not saying the result is fake; the degradation is consistently reported and the paper is honest about being work-in-progress. But the load-bearing claim about what Moirai does to physiological signals is confounded by the evaluation protocol. The good news is that the confound is addressable with a small number of additional experiments. I'd send this to a serious referee — the pipeline is useful enough and the negative result is relevant to the time series foundation model community — but I'd flag the missing multivariate control as a major revision. If the authors add that control, tighten the conclusions, and release the code, the paper becomes a solid contribution.","headline":"Useful evaluation pipeline, but the per-feature projection confounds the negative result; worth a serious round of revisions.","tokens_in":8366,"tokens_out":2637,"would_cite":false,"duration_ms":25120,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A general-purpose time series foundation model, Moirai, produces embeddings that systematically distort physiological signals, adding spurious cross-feature correlations, destroying temporal smoothness, and reducing the ability to decode…","keywords":["foundation models","time series","physiological signals","precision medicine","transfer learning","embeddings","synthetic physiology simulation","feature entanglement"],"falsifier":"An experiment that feeds the same physiological scenarios as joint multivariate tensors with native patch sizes and then measures the same metrics would settle the question: if the distortions vanish, the paper's conclusion fails; if they persist, the conclusion is robust.","tokens_in":7475,"feed_emoji":"📉","tokens_out":3957,"duration_ms":35453,"temperature":0.7,"pith_summary":"This paper builds a three-stage pipeline to test whether general-purpose time series foundation models can be used for precision medicine: generate clinically relevant synthetic physiological signals, push them through the model's transformer to get embeddings, and probe the embeddings with linear methods. Applying the pipeline to the Moirai time series foundation model, the paper finds that the embeddings are not faithful representations of physiological data. Features that are nearly independent in the raw signals become correlated in embedding space, raw signals cannot be reconstructed from embeddings, temporal smoothness and dimensionality drop, and medical scenarios become harder to tell apart. A logistic-regression probe decodes feature identity from raw signals with mean AUC 0.96 but only 0.78 from embeddings. The paper concludes that general-purpose time series foundation models will need architectural changes or targeted fine-tuning before clinical deployment.","feed_headline":"Time-series foundation model distorts vital-sign data","feed_subtitle":"Embeddings add false feature correlations, erase temporal structure, and cut feature decoding from 0.96 to 0.78 AUC.","key_machinery":"The machinery is a three-stage evaluation pipeline. Stage one uses whole-body physiology simulations to generate diverse, clinically relevant scenarios such as hemorrhage, sepsis, multi-organ failure, burn, and radiation exposure, with signals normalized to zero mean and unit variance. Stage two linearly interpolates each feature to 1000 timesteps and passes each feature independently through the foundation model's transformer module, taking the final-layer embedding. Stage three applies linear probes: Pearson correlations between feature pairs, linear regression with 5-fold cross-validation for reconstruction R2, PCA for dimensionality and a smoothness metric based on velocity ratios relative to randomly permuted trajectories, cosine similarity between scenario embeddings, and logistic regression AUC for pairwise feature identity decoding. These probes are designed to quantify feature entanglement, temporal dynamics preservation, and medical scenario differentiation.","core_discovery":"On the paper's own terms, the central discovery is that feeding each physiological feature independently through Moirai's transformer module produces final-layer embeddings that systematically corrupt the signal structure: spurious cross-feature correlations appear, linear reconstruction of the raw time series from embeddings is poor, the low-dimensional PCA trajectories become erratic and lower-dimensional, inter-scenario correlation rises, and pairwise feature decoding drops from near-perfect on raw signals to 0.78 AUC on embeddings. These are measured on synthetic physiological signals from BioGears covering hemorrhage, sepsis, multi-organ failure, and related scenarios. The paper positions this as evidence that zero-shot transfer of such foundation models to precision medicine is not ready, and that the evaluation pipeline itself is the contribution that makes such assessments fast and inexpensive.","pith_inferences":["A testable extension the paper does not run: feed the same physiological scenarios as joint multivariate channels or vary patch size and interpolation length to see how much of the distortion is due to the independent-feature projection rather than the model itself.","If feature entanglement in embedding space is as strong as reported, a simple linear whitening or decorrelation post-processing step might recover some feature independence, but the reconstruction and temporal-smoothness losses imply information is genuinely discarded, so post-hoc correction alone is unlikely to suffice.","The same evaluation could be applied to wearable photoplethysmography or ECG streams, where temporal smoothness matters for beat-by-beat analysis, to see whether the distortion pattern persists across physiological signal types."],"forward_implications":["If Moirai-level distortions generalize to other time series foundation models, zero-shot use of such embeddings in clinical pipelines will misrepresent physiological state and can mislead downstream models.","The pipeline's simulation-to-linear-probe design makes it possible to benchmark many foundation models quickly and cheaply before any clinical data collection.","The identified failure modes each suggest a fine-tuning objective: controlled-correlation scenarios to combat feature entanglement, scenarios with clinically relevant timescales to restore temporal dynamics, and edge-case scenario design to improve discrimination.","Feature decoding AUC dropping from 0.96 to 0.78 implies that subtle but clinically important features such as central venous pressure and renal blood flow will be the first to be confounded.","Since no fine-tuning was performed, the results bound zero-shot performance, not the ceiling after adaptation."],"supporting_citations":[{"why":"Supplies the Moirai time series foundation model whose embeddings are under evaluation.","marker":"[14]"},{"why":"Supplies the BioGears physiological simulation software used to generate the synthetic clinical scenarios.","marker":"[2]"},{"why":"Frames the data-scarcity problem that motivates transferring foundation models to precision medicine.","marker":"[1]"},{"why":"Provides the broader survey of time series foundation models that the paper draws on for future multi-model evaluation.","marker":"[10]"}],"fun_headline_variants":["Time-series foundation model corrupts vital-sign features","Moirai embeddings distort temporal dynamics and feature independence","Physiological signal transfer: foundation model fails zero-shot","Precision medicine readiness: foundation model distorts vital data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline's verdict depends on the assumption that the way signals were fed to the model—one feature at a time, stretched to a fixed length of 1000 time steps—is a fair test rather than an awkward input format the model was not built for.","fun_headline_variants_meta":{"raw":{"variants":["Time-series foundation model corrupts vital-sign features","Moirai embeddings distort temporal dynamics and feature independence","Physiological signal transfer: foundation model fails zero-shot","Precision medicine readiness: foundation model distorts vital data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1339,"prompt_tokens":910,"completion_tokens":429,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":366}},"tokens_in":526,"tokens_out":429,"duration_ms":4339,"temperature":1.0,"reasoning_tokens":366,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:24:38.936000+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An experiment that feeds the same physiological scenarios as joint multivariate tensors with native patch sizes and then measures the same metrics would settle the question: if the distortions vanish, the paper's conclusion fails; if they persist, the conclusion is robust.","supporting_citations":[{"cited_title":"BioGears: A C++ library for whole body physiology simulations.Journal of Open Source Software, 5(56):2645, December 2020","cited_arxiv_id":null,"evidence_quote":"Supplies the BioGears physiological simulation software used to generate the synthetic clinical scenarios."},{"cited_title":"A survey on data-efficient algorithms in big data era.Journal of Big Data, 8(1):24, January 2021","cited_arxiv_id":null,"evidence_quote":"Frames the data-scarcity problem that motivates transferring foundation models to precision medicine."}],"review_version":1}