{"id":"24e83ed8-2c0a-4cd1-8fe2-8b74ba66ed63","arxiv_id":"2411.13441","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A seven-function Listener/Connector API implemented in Lingua Franca with a network driver supports TCP, MQTT, and SST-based encryption, but the MQTT mode has high latency under centralized coordination.","lead":"The paper presents a simple seven-function API that lets Internet of Things devices communicate either directly or through a broker, with optional encryption. A working implementation on top of Lingua Franca and the Secure Swarm Toolkit shows direct connections add only small delay, but the broker mode adds about 188 milliseconds per message.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'reasonably small overhead' claim is contradicted by the paper's own MQTT measurement: 188 ms lag versus 4.4 ms baseline, caused by synchronous QoS-2 writes doubled by RTI relay; decentralized MQTT is explicitly future work.","rationale":"The reader and I converge on the MQTT overhead as the weakest point. The paper's own numbers in §5.2 directly contradict the abstract's 'reasonably small overhead': 188 ms versus 4.4 ms baseline is not small by any ordinary reading, and the authors' future-work note that decentralized MQTT coordination is unsupported confirms the current implementation cannot fix this in the tested configuration. I considered two other concerns: the code is not actually released (only promised), so the experiments are not independently reproducible, and no heterogeneous-protocol or heterogeneous-device interoperation is demonstrated (both federates are Raspberry Pi 4s, and each run uses one comm-type). These are real weaknesses, but they are under-support rather than direct contradiction of the central claim; the MQTT latency is a direct contradiction. The paper does have merits: a concrete API design, a working runtime built on Lingua Franca and SST, a clean binary-size analysis, and an honest admission of the MQTT inefficiency. If the authors were to reword the overhead claim by mode and provide the implementation, the underlying idea could still be a useful design study. As submitted, however, the primary quantitative evidence for the headline claim fails in one of the two explicitly supported communication modes, so I keep the reader's REJECT verdict rather than moving to CONDITIONAL.","tokens_in":14576,"tokens_out":7655,"duration_ms":82036,"concrete_test":"Re-run the §5.2 benchmark while instrumenting the MQTT path to separate (a) time spent in MQTTClient_waitForCompletion() and MQTTClient_receive() from (b) actual broker round-trips, then compare lag with a callback/asynchronous netdrv read/write path and with a decentralized (no-RTI-relay) MQTT federation. If lag drops to near the TCP/SST values, the 188 ms overhead is an implementation artifact of the synchronous QoS-2 API layering and can be removed by re-architecting the driver, preserving a qualified small-overhead claim. If lag stays at tens of milliseconds, the overhead is inherent to the synchronous pub-sub mapping and the current claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the approach yields 'reasonably small overhead' is contradicted by the paper's own published data. In §5.2, the MQTT mode has an average lag of 188 ms, while the TCP and SST modes are 4.4–5.1 ms in the same setup; that is roughly 37–43x larger. The cause is stated by the authors: netdrv write() calls MQTTClient_waitForCompletion() synchronously, and centralized LF coordination relays each message through the RTI, so write() is executed twice per logical message and each call costs about 90 ms. The paper then concedes that 'centralized coordination becomes inefficient' for MQTT and that decentralized MQTT coordination is future work, with §5.4 stating decentralized coordination currently supports only TCP. Since publish-subscribe is one of the two communication models the API explicitly claims to support (§3.2), the abstract's unqualified overhead claim is false for a supported mode. No error bars are reported for the 188 ms figure, but the gap is so large that error bars would not change the qualitative conclusion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a case study of a seven-function API (create_listener, create_connector, wait_for_connection, connect, read, write, close) implemented on top of the Lingua Franca runtime. The API is realized through a network driver (netdriver) that abstracts TCP, MQTT, and the Secure Swarm Toolkit (SST), with optional KDC-based authentication and encryption. The authors evaluate communication lag, message size, binary size, and software-design modularity in a setup with two Raspberry Pi 4 federates and an edge workstation. The paper claims that the approach provides IoT/CPS interoperability and security with reasonably small overhead and better-managed software.","tokens_in":14736,"tokens_out":6153,"duration_ms":66440,"significance":"The main value of the paper is a working, open-source implementation that exposes a uniform API over multiple communication stacks and integrates an existing security toolkit into an established coordination runtime. The netdriver state machine for TCP message-boundary handling and the concrete integration of SST with LF are useful engineering contributions. The evaluation is direct, and the latency results for TCP and SST modes are clean and small. However, the significance is limited by claim overreach: the MQTT measurement contradicts the 'reasonably small overhead' claim, and the interoperability claim is not tested in a mixed-mode federation. If these claims are scoped down and the MQTT path is either improved or clearly presented as a limitation, the case study could be a useful practical report for the community.","major_comments":[{"comment":"The abstract's claim of 'reasonably small overhead' is contradicted by the paper's own MQTT measurement. In the same setup, the average lag for MQTT is 188 ms versus about 4.4–5.1 ms for the baseline, TCP, and SST modes, a factor of roughly 37–43. The text explains that the synchronous MQTTClient_waitForCompletion() call and centralized RTI relaying cause netdrv_write() to run twice per logical message, at about 90 ms per call. Section 5.2 concedes that centralized coordination becomes inefficient for MQTT, and §5.4 states that decentralized coordination currently supports only TCP. Since publish-subscribe is one of the two communication models the API explicitly claims to support in §3.2, the unqualified overhead claim must be restricted to TCP/SST or the MQTT path must be redesigned and re-measured.","section":"§5.2, Fig. 8 and Abstract"},{"comment":"The 'interoperability' claim is not directly demonstrated by the evaluation. In the experiments, a federation is configured with a single comm-type, as in Fig. 5b line 3, so the results show that the same API can be mapped onto TCP, MQTT, or SST individually, not that nodes using different communication models can interact within one federation. The conclusion and future-work section explicitly lists 'allow distributed nodes with different communication modes to join a single federation' as future work. To support the abstract's interoperability claim, the paper should either present a mixed-mode federation experiment or reword the claim as portability across communication stacks.","section":"§3.2 and §6"},{"comment":"The software-design analysis claims that the design 'does not pass any data between modules' and therefore avoids all six types of coupling. This is contradicted by the API itself: read() and write() transfer message payloads, and the netdriver in §4.3 and §4.5 passes data pointers and protocol-specific structures such as socket_priv_t, sst_priv_t, and MQTT_priv_t between the abstraction and implementation layers. The coupling analysis should be revised to acknowledge data coupling through parameters; as written, the qualitative support for the 'better-managed software' claim is not accurate.","section":"§5.5"}],"minor_comments":[{"comment":"The lag results are reported only as averages, with no error bars, confidence intervals, or number of repetitions; please add this information so the reader can assess variability, especially for the 188 ms MQTT figure.","section":"§5.2, Fig. 8"},{"comment":"The experimental setup gives workstation and Raspberry Pi hardware but does not specify the OS, library versions (e.g., Eclipse Paho, Mosquitto, LF, SST), or Wi-Fi conditions; including these would improve reproducibility.","section":"§5.1"},{"comment":"There are minor typos: 'API deisgn' in the §4.4 heading and 'Subscirbeto' in Fig. 3; these should be corrected.","section":"§4.4 and Fig. 3"},{"comment":"The topic name reported for the MQTT message size, 'MQTTTest fed0 to RTI', does not match the naming scheme 'federationID_listenerID' described in §3.2; please clarify the actual topic names used in the measurement.","section":"§5.3"},{"comment":"The binary-size differences for MQTT versus TCP/SST are attributed to speculation about compiler optimizations; since binary-size overhead is one of the stated evaluation metrics, a short verification (e.g., checking linked libraries) would strengthen the discussion.","section":"§5.4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a case study: the authors layer a seven-function socket-like API into Lingua Franca with a network-driver abstraction and add a comm-type property so a program can switch between TCP, MQTT, and SST with a configuration flag. The integration is the genuine contribution. The authors honestly note that Listener and Connector are standard terms; what is new is the working netdriver layer and the conditional compilation that keeps the runtime decoupled from any one protocol. This is real engineering value, and the TCP and SST measurements are clean: sub-millisecond overhead over the LF baseline. The message-boundary state machine for TCP is a thoughtful solution to a practical problem.\n\nThe soft spot is serious. The MQTT path shows an average lag of 188 ms versus 4.4 ms for the baseline — roughly 40 times larger. The authors explain why: synchronous QoS-2 writes plus the RTI relaying every message, so write() is called twice per logical message. They then state that centralized coordination is inefficient for MQTT and that decentralized coordination for MQTT is future work, with Section 5.4 confirming that the decentralized mode currently only supports TCP. So a mode that the API explicitly promises (publish-subscribe) does not come close to 'reasonably small overhead', and the abstract makes that unqualified claim. If the paper had said 'small overhead for point-to-point and secure modes; high latency for MQTT due to current architecture', it would be honest rather than misleading.\n\nOther shortcomings are more minor: a single simple message flow, no error bars, no released code despite saying it \"will be available\", and a qualitative software-design analysis that is reasonable but light. None of these sink the underlying idea, but together they mean the current version overstates what is demonstrated.\n\nI would send this to a serious referee rather than desk-reject. The integration is a concrete artifact, the flaw is clearly diagnosable, and a revision that qualifies the overhead claim by mode and releases the code would make it a solid case study. As written, though, I would not cite the performance numbers or rely on the abstract.","headline":"A real API/runtime integration for Lingua Franca whose headline 'reasonably small overhead' is contradicted by its own MQTT measurement; fixable in revision, but misleading as published.","tokens_in":15304,"tokens_out":2752,"would_cite":false,"duration_ms":30372,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a seven-function API and runtime can give heterogeneous IoT devices both interoperability across communication models and optional security, with small overhead.","keywords":["Internet of Things","Interoperability","Security","API Design","Publish-Subscribe","Point-to-Point Communication","Edge Computing","Overhead Evaluation"],"falsifier":"Re-run the paper's HelloDistributed federation with MQTT in centralized coordination and a 50 ms message period; if the average lag stays near the reported 188 ms rather than approaching the 4.4 ms TCP baseline, the publish-subscribe mode does not achieve the reasonably-small-overhead claim.","tokens_in":14334,"feed_emoji":"🌐","tokens_out":11426,"duration_ms":100749,"temperature":0.7,"pith_summary":"The paper argues that a single small API, with seven functions for connection setup, data transfer, and teardown, can hide the differences between point-to-point and publish-subscribe communication and can add optional authentication and encryption. The aim is to let heterogeneous IoT and cyber-physical systems interoperate without each application being rewritten for a particular protocol or security level. The authors implement the API on top of open-source coordination and security components and measure communication latency, message size, and binary size. Their reported overhead is small for the TCP and encrypted paths, while the publish-subscribe path currently carries a much larger latency, a limitation the paper acknowledges.","feed_headline":"Seven-call API aims to unify IoT communication models and security","feed_subtitle":"The same read/write interface runs over TCP, MQTT, or encrypted sessions, so one app could span heterogeneous devices.","key_machinery":"The carrying mechanism is the network driver (netdriver), a small abstraction layer that gives the seven API functions a uniform interface while hiding protocol details behind a void pointer cast to a protocol-specific struct for TCP, MQTT, or the secure session mode. It also contains the bookkeeping for reading complete application-level messages, using a state machine that distinguishes fixed-length, structured variable-length, and variable-length messages, which is necessary because TCP is a byte stream and the other protocols carry their own lengths. This layer is what lets the same federated program be compiled once with a communication-type choice and run over different stacks without changes to application code.","core_discovery":"The central claim is that the proposed API and runtime can deliver interoperability and security for IoT and distributed cyber-physical systems with reasonably small overhead and better-managed software. The API defines a Listener and a Connector node and exposes seven functions -- create_listener, create_connector, wait_for_connection, connect, read, write, and close -- that map onto a client-server model such as TCP and, through a three-way handshake over named topics, onto a publish-subscribe model such as MQTT. When security is enabled, the runtime obtains a session key from a key distribution center, uses message authentication codes for integrity and block-cipher encryption for confidentiality, and destroys the key on close. The implementation wraps these protocols behind a polymorphic network driver and conditionally compiles only the needed libraries. The measured TCP lag is close to the baseline, and the secure session mode adds only a small increase; the MQTT mode, however, shows an average lag around 188 milliseconds, which the paper attributes to synchronous delivery guarantees and centralized coordination.","pith_inferences":["If the MQTT latency gap is closed, protocol selection could become a deployment-time property, letting constrained devices use pub-sub while control loops use TCP without forking the application code.","The three-way handshake over named topics could be ported to brokerless pub-sub systems, where discovery would replace the centralized coordinator and the 188 ms bottleneck might disappear.","The state-machine approach to message boundaries is a reusable pattern for retrofitting message framing onto any stream transport, not just TCP.","A natural next test is a single federation whose nodes deliberately use different communication modes; the paper states this is future work, so until demonstrated, the interoperability claim is per-protocol rather than cross-protocol within one running system."],"forward_implications":["Application code written against the seven functions does not need to change when the underlying protocol is switched from TCP to MQTT or to the secure mode; the switch happens at compile time through the communication-type option.","Enabling authentication and encryption adds only a small latency cost in the TCP case, so security can be turned on for sessions that need it without redesigning the application.","The message-boundary state machine lets a byte-stream protocol like TCP carry messages that other protocols already delimit, which is a prerequisite for mixing protocols behind one API.","In the current implementation, publish-subscribe over MQTT has substantially higher latency than TCP, so before the reasonably-small-overhead claim extends to pub-sub, the runtime must avoid the double-broker bottleneck or use decentralized coordination.","The decoupled network-driver design is claimed to improve modularity and information hiding; the paper's qualitative analysis finds all six standard coupling types avoided."],"supporting_citations":[{"why":"Defines the MQTT publish-subscribe protocol that the API's pub-sub mode is implemented over.","marker":"[1]"},{"why":"Supplies the MQTT client library used to build the pub-sub runtime.","marker":"[10]"},{"why":"Provides the security toolkit that supplies authentication and encryption for the secure session mode.","marker":"[21]"},{"why":"Defines the local authorization entity that acts as the key distribution center in the secure mode.","marker":"[22]"},{"why":"Provides the MQTT broker used in the evaluation setup.","marker":"[27]"},{"why":"Supplies the coordination language and runtime with federated execution that the API is layered on.","marker":"[30]"},{"why":"Defines the logical-time lag metric used to measure communication overhead.","marker":"[31]"},{"why":"Supplies the design considerations (abstraction, modularity, information hiding) used in the qualitative software-design analysis.","marker":"[50]"}],"fun_headline_variants":["Seven-call API bridges TCP and MQTT with built-in security","Unified IoT API: seven calls for TCP, MQTT, and security","Seven functions unify IoT protocols and add secure sessions","One API, seven calls: secure TCP and MQTT for IoT","Seven-call API merges IoT comms and security"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reasonably-small-overhead claim rests on the assumption that the API's synchronous read() and write() can be layered on an MQTT broker without losing the low latency of the TCP path; the paper's own measurement puts MQTT average lag at about 188 ms versus 4.4 ms for the baseline, so that assumption is not currently met for publish-subscribe.","fun_headline_variants_meta":{"raw":{"variants":["Seven-call API bridges TCP and MQTT with built-in security","Unified IoT API: seven calls for TCP, MQTT, and security","Seven functions unify IoT protocols and add secure sessions","One API, seven calls: secure TCP and MQTT for IoT","Seven-call API merges IoT comms and security"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001094,"raw_usage":{"total_tokens":4560,"prompt_tokens":928,"completion_tokens":3632,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":3541}},"tokens_in":544,"tokens_out":3632,"duration_ms":27565,"temperature":1.0,"reasoning_tokens":3541,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:23:53.605963+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the paper's HelloDistributed federation with MQTT in centralized coordination and a 50 ms message period; if the average lag stays near the reported 188 ms rather than approaching the 4.4 ms TCP baseline, the publish-subscribe mode does not achieve the reasonably-small-overhead claim.","supporting_citations":[{"cited_title":"OASIS Standard (2014), https://docs.oasis- open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html","cited_arxiv_id":null,"evidence_quote":"Defines the MQTT publish-subscribe protocol that the API's pub-sub mode is implemented over."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MQTT client library used to build the pub-sub runtime."},{"cited_title":"In: The 2nd ACM/IEEE International Conference on Internet-of-Things Design and Implementation","cited_arxiv_id":null,"evidence_quote":"Provides the security toolkit that supplies authentication and encryption for the secure session mode."},{"cited_title":"In: The 4th IEEE International Conference on Future Internet of Things and Cloud (FiCloud)","cited_arxiv_id":null,"evidence_quote":"Defines the local authorization entity that acts as the key distribution center in the secure mode."},{"cited_title":"Journal of Open Source Software 2(13), 265 (2017)","cited_arxiv_id":null,"evidence_quote":"Provides the MQTT broker used in the evaluation setup."},{"cited_title":"ACM Transactions on Embedded Computing Systems (TECS) 20(4), 1–27 (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the coordination language and runtime with federated execution that the API is layered on."},{"cited_title":"In: 2020 Forum for Specification and Design Languages (FDL)","cited_arxiv_id":null,"evidence_quote":"Defines the logical-time lag metric used to measure communication overhead."},{"cited_title":"Wiley (2008) 24 D","cited_arxiv_id":null,"evidence_quote":"Supplies the design considerations (abstraction, modularity, information hiding) used in the qualitative software-design analysis."}],"review_version":1}