{"id":"a89d5123-55bf-47e1-b2bd-b8a26ead1d16","arxiv_id":"2509.11239","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"An MLP-enhanced Spray and Wait router improves simulated DTN delivery by 7-8%, but the evaluation uses non-causal features and a self-referential label.","lead":"An MLP-based relay selection router for Delay Tolerant Networks is trained on simulation features and tested in ONE simulator scenarios. The paper reports 7-8% higher delivery probability and lower latency than baseline Spray and Wait, but the real-time feature construction is not causally valid.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Temporal leakage in the feature set breaks the real-time relay-selection claim and invalidates the reported delivery gains as stated.","rationale":"The reader's weakest_assumption identifies exactly the most load-bearing flaw: the real-time routing claim assumes access to features that are only known after simulation completion. I agree with that assessment. The paper's methodology sections describe feature extraction from DeliveredMessagesReport (Section 3.2.1) and then state that MLPBasedSprayRouter computes features in real time during live simulation (Sections 3.1.2 and 3.4). Those statements are mutually inconsistent unless the router uses only a subset of the training features, in which case the trained model is being applied out-of-distribution, or it uses future-derived values, which is leakage. Either way, the headline empirical result in Tables 4.4/4.5 is not established. The concern is not merely a matter of disagreement with an external consensus; it is an internal correctness risk in the paper's own pipeline. Independent support is absent: there is no released code, no formal verification, and no error bars. A fix is conceivable—redefine features from contact history only, retrain, and rerun the simulations—but as written, the central claim cannot be accepted. The reader's REJECT verdict is appropriate, and no change is needed to that verdict.","tokens_in":10667,"tokens_out":2489,"duration_ms":36380,"concrete_test":"Instrument the ONE simulator to log the exact feature vector sent to the Flask /predict endpoint at every contact, together with the current simulation time and the creation/delivery time of the message being routed. Then verify that no feature value (e.g., avg delivery time, relay count, destination count) is derived from any delivery event occurring after the decision time. Re-run the Weekday and Holiday scenarios (Tables 4.4/4.5) with the router restricted to strictly past-encounter features; if the delivery-probability gain over Spray and Wait disappears or shrinks materially, the original gain was caused by temporal leakage.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim—that MLPBasedSprayRouter improves delivery probability by 7-8% and reduces latency by 200-300s (Section 4.4, Tables 4.4-4.5)—depends on the router making online, causally valid forwarding decisions. But the feature set defined in Section 3.2.1 includes Average Delivery Time, Average Hop Count, Relay Count, and Destination Count, all derived from the completed DeliveredMessagesReport. At the moment a forwarding decision is made, these values are not yet known: the message has not been delivered, and relay/destination tallies only exist after the simulation run completes. Thus the router as described either sends a different feature vector at inference time than the one used in training (train/serve skew), or it queries the MLP using post-hoc outcome information, which is lookahead leakage. The labeling strategy compounds the problem: labels are a median split of an aggregate score built from the same delivery-outcome features, so the classifier is trained to predict a post-hoc quality label from features that are themselves only available after the fact. The +7-8% delivery improvement could therefore be an artifact of using future information during forwarding, not evidence that the learned policy is genuinely adaptive. The paper provides no code release and no ablation showing that the deployed router uses only causally available features, so the reported gains are unsupported as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an ML-enhanced variant of Spray and Wait routing for delay-tolerant networks (DTNs). The authors extract node-level features from ONE simulator logs (contact frequency, degree, average contact duration, average hop count, average delivery time, relay count, destination count), label nodes via a median split of an aggregate score, and train MLP, SVM, and Random Forest classifiers. The best model, MLP, is deployed via a Flask REST API and integrated into a custom MLPBasedSprayRouter, which forwards messages only to nodes predicted as high-quality relays. Experiments on a simulated emergency scenario with weekday and holiday mobility report that the MLP-based router improves delivery probability by about 7-8% and reduces average latency by 200-300 seconds compared to baseline Spray and Wait, with an unspecified RandomRouter baseline also shown. The paper claims the approach is practical and real-time, and proposes future online learning and a modular testbed.","tokens_in":11043,"tokens_out":4012,"duration_ms":50186,"significance":"If the reported gains were causally valid, the paper would demonstrate a modest but useful improvement to a well-known DTN routing protocol through a modular ML integration. The comparison of three classifiers and the inclusion of simulated weekday/holiday scenarios are also potentially informative. However, the core evaluation is undermined by a severe temporal-leakage problem: the feature set includes delivery outcomes that are not available at routing time, and the labels are derived from the same features. The reported model accuracy and end-to-end delivery improvements may therefore reflect post-hoc information rather than a genuinely adaptive learned policy. Other issues, including an undefined RandomRouter baseline and the absence of statistical variability measures, further prevent the results from being interpreted. The paper currently does not support its central claim.","major_comments":[{"comment":"The feature set includes Average Delivery Time, Average Hop Count, Relay Count, and Destination Count, all stated to be derived from the DeliveredMessagesReport after simulation completion. At the moment a forwarding decision is made, these outcome statistics are not known for the message being routed; they become available only after delivery. The paper claims 'real-time feature extraction' in Section 3.1.2, but it never specifies how a node could obtain these values for a candidate peer from causally available information. This is a temporal-leakage problem: the classifier is trained on features that encode the future, and the routing decisions in Tables 4.4-4.5 may be exploiting lookahead information. The authors need to either redefine the features in terms of purely historical, locally observable statistics or provide an explicit mechanism explaining how average delivery time and de","section":"Section 3.2.1"},{"comment":"The binary label is assigned by a median split of an aggregate score built from the same normalized features that the classifier consumes. Consequently, the reported classification accuracy (e.g., F1=0.733 for MLP in Table 4.3) measures how well the model reproduces the authors' own scoring rule, not whether it identifies causally good relays. In particular, because the score includes delivery outcomes, a node that has historically delivered messages quickly will be labeled good, and the same historical delivery time is used as a feature. The circularity must be broken by deriving labels from independent ground truth, such as whether forwarding to a node ultimately improved delivery in a held-out causal evaluation, or by using only features available at decision time.","section":"Section 3.2.1 / 3.2.4"},{"comment":"The RandomRouter baseline is not defined anywhere in the paper: no algorithm description, no reference, no explanation of how it differs from MLPBasedRouter, and no statement of its role. Without this information, the comparison is uninterpretable. Additionally, the protocols were run for five random seeds (Section 4.2), but the tables report only point estimates without standard deviations, confidence intervals, or per-seed results. The claimed +7-8% delivery improvement and 200-300s latency reduction could fall within seed-to-seed variability; the paper must report error bars or a statistical test before these improvements can be accepted.","section":"Section 4.4, Tables 4.4 and 4.5"},{"comment":"The manuscript does not state whether the mobility configurations used in the router evaluation (weekday and holiday scenarios) are disjoint from the data used to train the classifiers. The dataset is described as being aggregated from 'nine mobility configurations' (Section 3.2.3), and the same types of scenarios appear in the evaluation. If the weekday/holiday runs overlap with the training configurations, the reported delivery gains may reflect memorization of specific scenario traces rather than generalization. The authors should specify the exact training/test split at the scenario level and, ideally, evaluate on unseen configurations.","section":"Section 4.3"}],"minor_comments":[{"comment":"Typo: 'pedistrains' should be 'pedestrians'.","section":"Section 4.1"},{"comment":"Table 4.4 and Table 4.5 are referred to as 'Tables 4.4 and 4.5' in the text, but the in-text references use the format 'Table 4.4-4.5' and the table captions do not follow a consistent numbering scheme. Please standardize.","section":"Section 4, table numbering"},{"comment":"Reference [34] is listed as 'F. Author and S. Author, \"A modular dtn testbed for opportunistic networks: Moditones\"'—this is a placeholder and must be replaced with the actual citation. Reference [4] also has a typo in the author name ('Vu San').","section":"References"},{"comment":"The definitions of Average Hop Count and Average Delivery Time are ambiguous: do they refer only to messages that have been fully delivered, or also to messages currently in the buffer? This matters for the causal-feature discussion and should be clarified.","section":"Section 3.2.1"},{"comment":"Caption is incomplete: 'Distribution of features across' does not specify across what (presumably high- vs. low-quality nodes) or which scenario/configuration. Please complete the caption.","section":"Figure 3.5"}],"recommendation":"reject","confidential_remarks":"The temporal-leakage issue in Section 3.2.1 is not a cosmetic flaw; it invalidates the central empirical claim. Even a major revision would require a complete redefinition of the feature set, a causal labeling procedure, and a re-run of the simulations with explicit train/test separation. Given that the reported delivery gains are presented as the main contribution, and that these gains are unsupported as stated, I cannot see how a revision within the scope of this manuscript would preserve the current conclusions. I would be open to a future submission that uses only causally available features and reports proper statistical summary."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short take: the MLP-based relay selection is a reasonable extension of existing ML-for-DTN work, but the paper as written doesn't support the real-time routing claim because the features include delivery statistics that are only known after the simulation ends. The reported +7-8% delivery gains are therefore not credible unless the authors clarify how they compute those features online.\n\nWhat's new: they integrate an MLP (and SVM/RF) into Spray and Wait via a Flask API, which is a clean modular design. The comparison across three classifiers and two mobility scenarios is a useful data point. The paper does its homework on prior work (MLProph, CARL-DTN, etc.) and is well structured.\n\nWhere the soft spots are: the biggest is the feature set in Section 3.2.1. Average Delivery Time, Relay Count, and Destination Count are defined from the completed DeliveredMessagesReport. At the time of a forwarding decision, those numbers don't exist for the current message, and the paper doesn't specify a temporal aggregation (e.g., sliding-window averages over past deliveries). So the router either uses different features at inference than at training, or it cheats with lookahead. The label is also a median split of a score built from the same features, which makes the classifier essentially reproduce the scoring rule. That is not necessarily fatal, but it means the classification accuracy doesn't tell us anything about the routing policy.\n\nMinor issues: RandomRouter is never defined, and there are no error bars despite five seeds, so the 7-8% improvement could be within noise. The overhead increase is reported but not discussed much.\n\nThe paper honestly lists some limitations (offline training, API scalability) but does not mention the temporal problem. That is the kind of oversight a thoughtful referee could catch.\n\nWho this is for: people working on applying supervised learning to DTN routing might get value from the architecture and the classifier comparison, but they should treat the reported gains with caution until the feature definitions are fixed.\n\nRecommendation: it deserves a serious referee because the core idea is viable and the flaw is fixable, but I would ask for a major revision that defines causally valid features and re-runs the experiments. If the authors can show the gains hold with only past information, the paper could be worth publishing. As it stands, I would not cite it.","headline":"The real-time relay-selection claim is undercut by temporal leakage in the feature definitions; the reported routing gains are unsupported as stated, but the architecture is clean and the issue is fixable.","tokens_in":11408,"tokens_out":4231,"would_cite":false,"duration_ms":48503,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that replacing Spray and Wait's blind copying with MLP-based relay evaluation improves delivery probability by 7-8 percentage points and cuts average latency by 200-300 seconds in simulated urban emergency scenarios.","keywords":["Delay-Tolerant Networks","relay selection","Spray and Wait","multi-layer perceptron","machine learning routing","emergency communication","opportunistic networks","real-time classification"],"falsifier":"Run the MLPBasedSprayRouter in the same simulator while logging exactly which features are available in each node's local state at each encounter. If the 'real-time' feature set includes values from delivered-message report entries for messages not yet delivered, the claim of real-time prediction is falsified; a concrete test is to remove those three outcome features, retrain on local-only features, and check whether the delivery probability gain over Spray and Wait persists.","tokens_in":10596,"feed_emoji":"🤖","tokens_out":3462,"duration_ms":38237,"temperature":0.7,"pith_summary":"The paper tries to establish that a delay-tolerant network can route messages more reliably by learning, from past contact and delivery logs, which nodes make good relays, rather than spraying copies blindly. It integrates a feedforward neural-network classifier into Spray and Wait, using seven per-node features—contact frequency, degree, average contact duration, average hop count, average delivery time, relay count, and destination count—to label nodes high- or low-quality and forward only to high-quality nodes. In simulated weekday and holiday emergency scenarios with pedestrians and vehicles, the resulting MLPBasedSprayRouter improves delivery probability by about 7-8 percentage points and reduces average latency by 200-300 seconds compared with baseline Spray and Wait. If correct, this would show that lightweight online ML inference can be bolted onto a standard DTN protocol to make it adaptive.","feed_headline":"Neural relay picker beats Spray and Wait by 7-8 percent","feed_subtitle":"A learned classifier decides which nodes forward messages, cutting average delay by 200-300 seconds in simulated emergencies.","key_machinery":"The load-bearing mechanism is a median-split labeling scheme plus a two-hidden-layer multi-layer perceptron classifier served through a lightweight HTTP prediction service and cached inside the router. Seven node-level statistics are computed from contact and delivery logs; each node's features are normalized and scored, nodes above the median are labeled high-quality, and the model learns the decision boundary. At runtime the router queries the service at each encounter and forwards only to predicted high-quality peers. The caching mechanism is what makes per-encounter inference cheap enough to sit inside a forwarding loop.","core_discovery":"The central claim is that relay selection in delay-tolerant networks can be treated as a binary classification problem and solved with a shallow neural network trained offline and queried at encounter time. The authors propose MLPBasedSprayRouter, a modified Spray and Wait router that extracts seven features from simulator logs, sends them over HTTP to a cached prediction service, and forwards a message copy only when the peer is classified as a high-quality relay. Their experiments across nine pedestrian/vehicle configurations and two temporal patterns show consistent delivery gains and latency reductions over the static baseline, with the MLP classifier outperforming the SVM and random for","pith_inferences":["The paper's 'real-time' framing likely overstates what is available at decision time: features such as average delivery time, relay count, and destination count come from a completed-delivery report, so a node in the field would not know them yet; the reported gains may depend on access to future statistics.","A fair field test would restrict features to locally observable encounter history—contact frequency, degree, average duration, and hop counts of messages seen so far—and re-measure delivery; if the gains vanish, the classifier was exploiting offline labels rather than predictive context.","The median-split labeling ties 'high-quality' to an end-of-run summary rather than to a per-forwarding decision, so the architecture is better described as offline feature engineering plus online scoring than as online learning.","The modular HTTP service design suggests a path to online retraining: as delivery reports accumulate, the model could be refreshed without recompiling the router, but that would require solving the feature-availability problem first."],"forward_implications":["If the central claim holds, standard DTN protocols can be upgraded by an external prediction service without rewriting the routing core.","The 7-8 percentage point delivery gain and 200-300 second latency reduction suggest that selective forwarding beats blind replication even when the number of message copies is limited.","MLP beating SVM and random forest at low inference cost hints that shallow neural networks offer a good accuracy-versus-speed trade-off for encounter-time decisions.","Stability across weekday and holiday mobility patterns suggests the model generalizes across temporal traffic conditions, not just one scenario.","Caching repeated feature patterns implies per-contact overhead can be bounded, making ML-based decisions viable in resource-constrained nodes."],"fun_headline_variants":["MLP picks relay nodes, cuts delivery delays in DTNs","Neural relay selection lifts delivery, slashes latency","Train a neural net to pick relays for faster DTN routing","AI-driven relay selection improves emergency messaging"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The strongest claim assumes that the seven features—including average delivery time, relay count, and destination count—can be computed in real time at the moment of a forwarding decision, but those values are outcome statistics that only exist after a delivery report is complete.","fun_headline_variants_meta":{"raw":{"variants":["MLP picks relay nodes, cuts delivery delays in DTNs","Neural relay selection lifts delivery, slashes latency","Train a neural net to pick relays for faster DTN routing","AI-driven relay selection improves emergency messaging"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000663,"raw_usage":{"total_tokens":2893,"prompt_tokens":798,"completion_tokens":2095,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":2040}},"tokens_in":542,"tokens_out":2095,"duration_ms":16602,"temperature":1.0,"reasoning_tokens":2040,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T16:50:05.383346+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the MLPBasedSprayRouter in the same simulator while logging exactly which features are available in each node's local state at each encounter. If the 'real-time' feature set includes values from delivered-message report entries for messages not yet delivered, the claim of real-time prediction is falsified; a concrete test is to remove those three outcome features, retrain on local-only features, and check whether the delivery probability gain over Spray and Wait persists.","supporting_citations":[],"review_version":1}