{"id":"553408c6-e04c-4d77-aade-8870598c02b0","arxiv_id":"2501.16368","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Current foundation models fall short on CPS-IoT needs in resource efficiency, spatial generalization, long-term context, and knowledge integration; the paper proposes desiderata and a community roadmap.","lead":"This paper argues that generic foundation models and large language models are not yet viable for cyber-physical systems and IoT, and it maps what domain-specific models would need to satisfy. It supports the argument with four small experiments on mobile forecasting, multi-sensor tracking, complex event detection, and building knowledge-graph queries.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evidence for domain-specific FM necessity is confounded: Section 3 contrasts small under-adapted general models with bespoke designs, never controlling for scale, tokenization, or sensor-data pretraining.","rationale":"The reader's weakest_assumption correctly identifies the load-bearing premise: the paper assumes observed FM failures stem from missing domain-specific design rather than from insufficient scale, data, or adaptation of general FMs. My reading of Section 3 confirms this confound in each of the three main experiments. The Llama-3.2-1B result in §3.1 is particularly weak evidence for the architectural thesis because raw digitized physiological signals are far outside the model's pretraining distribution, and no same-scale sensor-pretrained generalist is compared. The CED experiment in §3.3 isolates a reasoning task but does not vary context length, memory architecture, or model scale in a controlled way; the poor conditional F1 could reflect evaluation format or prompt sensitivity more than a fundamental limitation. The KGQA experiment in §3.4, while useful for ecosystem design, demonstrates a tooling bottleneck rather than a model-capability ceiling. None of this refutes the paper's agenda, and domain-specific inductive biases may well be necessary for resource-constrained CPS-IoT deployment; the paper itself frames the work as preliminary. The conditional verdict is therefore appropriate: the direction is compelling, but the empirical support is not yet decisive. The proposed test—a scale- and data-matched general FM baseline—would directly adjudicate the central claim and should be part of the released artifact package. Since the reader's verdict already conditions acceptance on stronger evidence and reproducibility, my concern does not shift the verdict; it sharpens the specific condition that must be met.","tokens_in":27011,"tokens_out":2614,"duration_ms":31238,"concrete_test":"Run the §3.1-3.3 benchmarks under a matched-control design: take a general transformer/LLM backbone of comparable parameter count to the proposed CPS-IoT-specific models, continue pretraining it on a large multi-modal sensor corpus (IMU, audio, ECG, PPG) with patch-based tokenization and long-context memory, and evaluate extrapolation/imputation, spatial tracking, and complex-event detection with identical metrics. If this generic-but-scaled model matches or beats the domain-specific systems (vantage-position embeddings, AE+FSM), the 'must account' claim loses force; if it does not, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3's experiments are offered as evidence that CPS-IoT FMs must embody domain-specific design, but each comparison conflates architecture with scale and adaptation. In §3.1.1, the only general FM tested is Llama-3.2-1B prompted on raw digitized 50 Hz ECG/PPG; it is neither sensor-pretrained nor given a modality-appropriate tokenizer. MOMENT-L, the better time-series FM, is itself a general time-series model, not a demonstration that spatial embeddings or neurosymbolic structure are required, and no frontier-scale general model is included. §3.3.2 gives LLMs perfect atomic labels but tests only prompt-based complex-event detection; the low conditional F1 may reflect output-format brittleness or missing temporal memory mechanisms rather than an architectural ceiling. §3.4.2's KGQA failures are explicitly attributed to subgraph retrieval (AutoKGQA/FAISS), not to the LLM backbone, so they support better KG tooling more than domain-specific FM architecture. Without a scale/tokenization-matched comparison, the central claim that CPS-IoT FMs 'must account for the unique characteristics' (Section 5) remains a plausible agenda rather than an established finding.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper argues that foundation models (FMs) and large language models (LLMs), despite their success in language and vision, are not yet viable for CPS-IoT systems because the domain has four distinguishing characteristics: tight resource/quality trade-offs, spatial embodiment, historical context, and structural constraints. After a survey of perception-focused CPS-IoT FMs organized by sensor modality (unimodal, multimodal, flexible) and task flexibility (fixed, configurable, selectable, run-time specifiable), the authors present four preliminary experiments probing each characteristic: an edge-resource comparison of MOMENT and Llama-3.2-1B against ARIMA on ECG/PPG forecasting and imputation, a multi-vantage masked autoencoder for acoustic/seismic vehicle tracking, LLM and neurosymbolic complex-event detection on a simulated smart-health dataset, and a KGQA study on a Brick-schema building knowledge graph. On this basis, Section 4 lists seven desiderata for CPS-IoT FMs (e.g., sensor-configuration generalizability, sensor tokenization, long-stream models, neurosymbolic structure, language channel, system-service abstractions) and calls for community datasets, benchmarks, and application-specific 'micro foundation models.' The central thesis is that domain-specific FM designs, not direct adaptation of general FMs, are needed to close the gap.","tokens_in":27302,"tokens_out":9825,"duration_ms":98838,"significance":"If the central thesis is correct, the paper provides a useful roadmap: the survey taxonomy is informative, the four characteristics are a plausible organizing framework, and the desiderata in Section 4 are concrete enough to guide research. The call for shared raw-sensor datasets and benchmarks is timely, and the proposal of application-specific micro foundation models is a reasonable middle path between fully general and task-specific models. The authors are candid in labeling their experiments as preliminary and in acknowledging that the spatial dataset is self-collected, which helps calibrate the strength of the claims. However, the empirical support for the strong 'must account for unique characteristics' conclusion is currently confounded: scale, tokenization, and sensor-data pretraining are not controlled, and the released evidence is narrow (one physiology-signal resource study, one six-node tracking study without error bars, one simulated CED study, and one building KGQA study). If the paper is reframed as a research agenda with illustrative case studies, its significance is solid; as an empirical demonstration of domain-specific FM necessity, it is not yet established.","major_comments":[{"comment":"The resource-tradeoff experiment conflates architecture with scale and adaptation. Llama-3.2-1B is a 1B general-purpose LLM given raw 50 Hz digitized waveforms with no sensor-specific tokenizer or sensor pretraining, while MOMENT-L is a large time-series FM; neither comparison controls for parameter count, context length, or training data. The observation that Llama performs worse than ARIMA and MOMENT is consistent with the paper's resource-feasibility argument, but it does not show that a general FM with sufficient scale and sensor-aware tokenization would fail. The conclusion in §3.1.2 that this 'highlights the limitations of today's general LLMs in processing low-level sensory data directly' should be softened to a statement about the specific models tested, or supplemented with a matched comparison.","section":"§3.1.1–3.1.2, Table 1"},{"comment":"The CED experiment's conclusion is stronger than the evidence. LLMs are given perfect atomic labels and are evaluated on a metric (conditional F1) that may be dominated by output-format brittleness; the low scores do not isolate a failure of temporal reasoning from a failure to produce the exact output format. The comparison to Mamba and AE+FSM also conflates model class with hand-coded rules, since AE+FSM receives the true complex-event rules and the other models must learn them. The manuscript should either add controls (e.g., constrained decoding, structured output parsing, or a learned model with similar state-compression capacity) or explicitly limit the claim to 'the specific prompting protocol we used.'","section":"§3.3.2, Table 2 and Figure 5"},{"comment":"The spatial-embedding result, as reported, does not support the strong claim. The evaluation uses a self-collected six-node dataset, reports no error bars or significance tests, and the data and code are not released. The statement that the method 'consistently achieves the highest accuracy' cannot be verified from the figure. For a paper whose central claim is that spatial embeddings are a required domain-specific component, this single proof-of-concept should be presented as an illustrative case study, with the corresponding claims in the text adjusted.","section":"§3.2.3, Figure 3"},{"comment":"The KGQA study's own analysis undermines the architectural conclusion. The failures in Table 3 are attributed in the text to subgraph extraction (FAISS relevance), entity/relation linking, and one-to-many mappings, not to the LLM backbone; indeed, §3.4.3 states that the 'main challenge lies in supplying LLMs with the appropriate relevant subgraph.' As written, the study supports the need for better CPS-IoT KG retrieval and schema design, not the paper's conclusion that CPS-IoT FMs must incorporate structural constraints through neurosymbolic architecture. The paper should separate these two claims and not use this experiment as direct evidence for the latter.","section":"§3.4.2–3.4.3, Table 3"},{"comment":"The conclusion that CPS-IoT FMs 'must account for the unique characteristics' is presented as an established finding, while the evidence in Section 3 is explicitly preliminary and, as noted above, does not rule out the alternative that a sufficiently scaled general FM with sensor-aware tokenization and longer context would close the same gap. The manuscript should either provide such a control or clearly label the domain-specific requirements as hypotheses to be tested. This distinction is load-bearing because the paper's proposed research program (domain-specific architectures, muFMs, and new benchmarks) depends on it.","section":"Abstract and §5"}],"minor_comments":[{"comment":"The figure caption contains the typo 'for ecasting' and the prompt text in the figure contains 'ret ur n' and 'for ecast'; these should be corrected.","section":"Figure 1 caption and §3.1.1"},{"comment":"There is a duplicated 'the' in 'determines if the the complex event labels' and 'few-short' should be 'few-shot' in the discussion.","section":"§3.3.2–3.3.3"},{"comment":"'embeddedings' should be 'embeddings'.","section":"§4.1.2"},{"comment":"The manuscript contains placeholder metadata ('Make sure to enter the correct conference title', 'XXXXXXX', 'Woodstock, NY', fake DOI) and reference [100] includes '<today>'; these must be completed before submission.","section":"Header and references"},{"comment":"The header of Table 1 is garbled (e.g., '𝑬 𝑪 𝑮𝑰 𝒎𝒑 𝑬 𝑪 𝑮𝑬𝒙𝒕 𝑷 𝑷 𝑮𝑰 𝒎𝒑 𝑷 𝑷 𝑮𝑬𝒙𝒕') and Table 3's ✓/× symbols are not defined in the caption; both tables need cleanup.","section":"Table 1 and Table 3"},{"comment":"The metrics 'Length Acc.', 'Coarse F1', and 'Conditional F1' are defined only in the text; the table caption should include brief definitions.","section":"Table 2"},{"comment":"Reference [81] lists 'Ozan Baris Mulayim' while the paper's first author is 'Ozan Baris'; please check the name consistency.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"To the editor: this is a position/agenda paper. I am recommending major revision because the empirical sections are used to support a strong architectural conclusion despite the confounds described above. The authors' heavy reliance on their own prior models (FOCAL, FreqMAE, LLMSense, IoT-LM) as state-of-the-art examples is not a correctness problem, but the external evidence base would be stronger if at least one comparison used an independently implemented baseline or an independent dataset. The manuscript also appears to be a draft (placeholder metadata), which should be checked by the production team."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful roadmap. The section organizing CPS-IoT FMs along modality type (unimodal/multimodal/flexible) and task type (fixed/configurable/selectable/runtime-specifiable) is a clean synthesis I'll reuse. The PCCA framing is a nice corrective to perception-centric surveys.\n\nThe four preliminary experiments add measurements that aren't in the literature. The phone-based comparison of ARIMA vs MOMENT vs Llama-3.2 on ECG/PPG gives concrete numbers on the resource-quality frontier. The multi-vantage MAE with relative geolocation embeddings is a reasonable proof of concept for spatial abstraction. The call for sensor tokenization and long-context/state-space models is well grounded. The KGQA failure taxonomy—one-to-many mapping, subgraph size, relation linking, property extraction—is worth quoting.\n\nThe soft spot is the gap between the data and the 'must' in the conclusion. The stress-test note is right: §3's experiments confound architecture with scale and adaptation. Llama-3.2-1B prompted on raw digitized physiological signals is not a fair representative of 'general FMs'; it has no sensor pretraining and a tokenizer designed for text. MOMENT-L is itself a general time-series FM, so it doesn't establish that spatial embeddings or neurosymbolic structure are required. The KGQA failures come from subgraph retrieval, not the LLM backbone. So the evidence is directionally consistent but not decisive. The paper is honest about calling these 'preliminary,' but the conclusion's 'must account for' overshoots what the experiments can support.\n\nTwo minor things. The spatial experiment uses a self-collected dataset with no release and no error bars; I can't verify the claimed margins. The CED experiment gives LLMs perfect atomic labels and still sees low conditional F1, which might be format brittleness. That's a useful negative result, but again it's not an architectural ceiling.\n\nWho is this for: anyone writing about sensor-based FMs or planning CPS-IoT ML research. I'd read it for the survey and the agenda, and I'd cite the taxonomy. I'd treat the empirical claims as hypotheses to test with matched baselines and released artifacts.\n\nRecommendation: send it to peer review. A serious referee should ask for artifact release and scale/tokenization-matched comparisons, but the paper deserves the attention.","headline":"A useful roadmap with four suggestive but confounded experiments; the agenda is plausible, the 'must' outruns the evidence.","tokens_in":27822,"tokens_out":3335,"would_cite":true,"duration_ms":34451,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Current foundation models are not yet viable for CPS-IoT applications, and the gap will close only through domain-specific architectures and community-built sensor data resources.","keywords":["Cyber-Physical Systems","Internet of Things","foundation models","large language models","sensor data","complex event detection","neurosymbolic learning","time series analysis"],"falsifier":"Scale a generic multimodal foundation model on a large corpus of raw sensor streams and test it on the paper's own benchmarks (ECG/PPG extrapolation-imputation, multi-vantage vehicle tracking, and 5-minute complex event detection); if it matches or beats the domain-specific models at deployable resource cost, the structural-gap claim is falsified.","tokens_in":26826,"feed_emoji":"📡","tokens_out":9242,"duration_ms":86248,"temperature":0.7,"pith_summary":"The paper argues that the current wave of foundation models and large language models, however successful in language and vision, is not yet viable for cyber-physical systems and the Internet of Things (CPS-IoT). Through a survey of the state of the art and four preliminary experiments, it identifies a persistent gap between what general models can do and what CPS-IoT applications require, and it traces that gap to four characteristics of the domain: tight resource and quality trade-offs, spatial embodiment, historical context, and structural constraints. The paper's constructive claim is that bridging the gap demands domain-specific CPS-IoT foundation models designed around these characteristics, plus a community-wide ecosystem of shared sensor datasets, models, and benchmarks. A sympathetic reader would take this as a research agenda: invest in sensor-aware architectures, context compression, and neurosymbolic structure rather than in straightforwardly scaling general-purpose models.","feed_headline":"Generic AI models fail sensor tasks; CPS-IoT needs its own designs","feed_subtitle":"Four experiments show why: latency, memory, and accuracy all point to sensor-specific architectures and shared datasets.","key_machinery":"The argument is carried by the Perception-Cognition-Communication-Action (PCCA) loop as the organizing frame, together with four distinguishing characteristics of CPS-IoT systems that the authors treat as foundational: tight resource/quality trade-offs, spatial embodiment, historical context, and structural constraints. These four characteristics do the analytical work: each is paired with a preliminary experiment that exposes a failure mode of current FMs, and each translates directly into a desideratum (resource-feasible inference, world-state representations, long-context state compression, and neurosymbolic knowledge injection). The inverse problem — recovering the underlying physical state from distributed sensor projections — and the notion of complex events as temporally extended patterns are the concrete mechanisms the paper proposes for turning sensor data into FM training objectives.","core_discovery":"On the paper's own terms, the central discovery is that the gap between current foundation models (FMs) and large language models (LLMs) and the requirements of CPS-IoT applications is real, structural, and measurable. In head-to-head tests on mobile electrocardiogram and photoplethysmogram signals, the largest time-series foundation model tested (MOMENT-L) beat classical ARIMA baselines only at the price of roughly 1.2 gigabytes of memory and a 2.3-second cold-start latency, while a 1-billion-parameter LLM failed to beat ARIMA at all. A multi-vantage masked autoencoder with relative geolocation embeddings recovered physical trajectories better than contrastive baselines, but only because it was designed to learn world state rather than sensor projections. LLMs scored poorly on complex event detection even when given perfect atomic-activity labels, while state-based and neurosymbolic models held up better; and an LLM-based knowledge graph question-answering pipeline struggled to map vague CPS concepts like 'energy' onto building schema. From these results the authors conclude that CPS-IoT FMs must generalize across sensor configurations and tasks, tokenize continuous sensor streams without information loss, handle long and continuous streams, incorporate human knowledge through neurosymbolic layers, expose a rich language channel, and become composable system services rather than standalone per-application models.","pith_inferences":["The paper's framing implies that next-sample prediction benchmarks systematically flatter general time-series models, because they reward matching sensor projections rather than recovering world state; a benchmark that scores tracking or inverse-problem accuracy would likely widen the measured gap.","If historical context is the bottleneck, a fixed-size state compressor (e.g., a small state-space encoder) could be tested as a drop-in front end to any transformer FM, converting unbounded sensor history into a bounded state vector before attention; such an experiment would isolate the context-length effect.","The resource-efficiency results suggest a concrete engineering target: a CPS-IoT FM that beats ARIMA on accuracy and matches its memory and latency would be the practical threshold for mobile health deployment, and today no published model meets both.","The KGQA failure pattern hints that ontology standardization (e.g., consistent naming of 'energy' concepts across buildings) is as important as model capability; a testable extension is measuring query success rate against schema variants, which would tell the community where to invest."],"forward_implications":["General LLMs will not serve as drop-in processors of raw sensor time series; their reliable role is limited to metadata, high-level reasoning, and language-based analytics over sensor data.","CPS-IoT FMs should be pretrained to reconstruct masked signals across vantage points and time, with relative geolocation embeddings, so that their internal representations track physical world state rather than sensor-specific projections.","Long-horizon tasks such as complex event detection require state-compressing architectures (state-space models, neurosymbolic finite state machines) rather than attention over unbounded context windows.","For edge devices, foundation models must become runtime composable services shared across applications, and the community should pursue application-specific micro foundation models rather than a single universal CPS-IoT FM.","A shared ecosystem of raw sensor datasets, benchmarks, and ontologies is a prerequisite for progress; without it, the desiderata cannot be validated or iterated upon."],"supporting_citations":[{"why":"Defines foundation models as large-scale self-supervised, task-agnostic models, the object the paper evaluates and extends.","marker":"[12]"},{"why":"MOMENT time-series FM is the model tested on mobile ECG/PPG extrapolation-imputation and compared against ARIMA baselines in the resource-efficiency experiment.","marker":"[39]"},{"why":"Provides the state-space architecture that generalized best on long-horizon complex event detection, supporting the historical-context desideratum.","marker":"[41]"},{"why":"Contrastive multimodal sensing FM used as a baseline in the multi-vantage tracking comparison, representing current perception-focused CPS-IoT FMs.","marker":"[68]"},{"why":"Provides the neurosymbolic programming formalism used for rule-based complex event detection, supporting the structural-constraints desideratum.","marker":"[74]"},{"why":"Adapter-based approach that maps sensor signals into an LLM's semantic space, representing the beyond-perception trend the paper surveys.","marker":"[79]"},{"why":"LLM-based question-answering pipeline tested on building energy queries in the structural-constraints experiment.","marker":"[6]"},{"why":"Defines the building metadata schema used to construct the knowledge graph, grounding the one-to-many mapping failures observed.","marker":"[7]"},{"why":"Supplies the real-world building knowledge graph on which the KGQA queries were run.","marker":"[31]"}],"fun_headline_variants":["Sensor data defeats generic LLMs; specialized FMs essential","MOMENT-L beats ARIMA on ECG but at 1.2GB memory cost","Generic AI models fail on CPS-IoT; domain-specific designs win","Task-agnostic FMs? Not for sensors—latency and memory bite"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole agenda rests on the premise that the observed failures come from missing CPS-IoT-specific design rather than from current models simply being too small or not pretrained on enough sensor data.","fun_headline_variants_meta":{"raw":{"variants":["Sensor data defeats generic LLMs; specialized FMs essential","MOMENT-L beats ARIMA on ECG but at 1.2GB memory cost","Generic AI models fail on CPS-IoT; domain-specific designs win","Task-agnostic FMs? Not for sensors—latency and memory bite"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000287,"raw_usage":{"total_tokens":1764,"prompt_tokens":1102,"completion_tokens":662,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":718,"completion_tokens_details":{"reasoning_tokens":581}},"tokens_in":718,"tokens_out":662,"duration_ms":7388,"temperature":1.0,"reasoning_tokens":581,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:25:38.997018+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Scale a generic multimodal foundation model on a large corpus of raw sensor streams and test it on the paper's own benchmarks (ECG/PPG extrapolation-imputation, multi-vantage vehicle tracking, and 5-minute complex event detection); if it matches or beats the domain-specific models at deployable resource cost, the structural-gap claim is falsified.","supporting_citations":[{"cited_title":"FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space","cited_arxiv_id":"2310.20071","evidence_quote":"Contrastive multimodal sensing FM used as a baseline in the multi-vantage tracking comparison, representing current perception-focused CPS-IoT FMs."}],"review_version":1}