{"id":"ff423c61-a3e0-470b-8686-4e2d64f203ae","arxiv_id":"2501.15493","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"RLER-TTE adds a reinforcement-learning decision maker that decides when to recompute en-route travel time estimates, reducing model calls while improving accuracy on three taxi datasets.","lead":"This paper proposes a travel-time prediction system that decides, in real time, whether to recompute a route's remaining travel time or reuse the last prediction. By routing only a fraction of requests through the expensive model, it aims to cut computation while preserving or improving accuracy.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (1) rewards choosing the smaller of two predictor outputs, not the one closer to the true remaining time; without ground truth in the reward, the Decision Maker's accuracy gains are ungrounded.","rationale":"The reader's weakest_assumption is exactly the one I find most load-bearing. The central claim has two parts: accuracy and efficiency. Efficiency is directly measured by MUR and latency, and the Decision Maker's low complexity in Sec. 4.3 supports it. Accuracy, however, depends on the learned policy making better invocation decisions than always re-predicting. Eq. (1) rewards whichever action yields the smaller numeric estimate: for a_i = 0, r_p = yhat_rp - yhat_dl, so reward is high when yhat_dl < yhat_rp; for a_i = 1, reward is high when yhat_rp < yhat_dl. No ground-truth term appears in Eqs. (1)-(4), and Algorithm 1 computes r_i only from a_i and Yhat. Under MAE/MAPE, the correct action is to pick the estimate closest to Y_remain; choosing the minimum is not equivalent unless a strong, unstated assumption holds, such as systematic overestimation with monotone errors. The paper even claims the reward 'directly reflects accuracy,' which is contradicted by the equation. A Transformer-based decision maker (ND) performs worse, and removing curriculum or contrastive components (NU/NA) hurts, but none of these ablations establishes that the Double DQN policy is actually optimizing accuracy; a random or always-lookup policy at similar MUR is missing. The proposed concrete test checks whether the signed prediction difference predicts which estimate is closer to ground truth; if not, the reward is ungrounded, and the accuracy gain cannot be attributed to the RL mechanism as designed. I do not see an independent reason to reject the whole framework: the efficiency contribution and the predictor-side training ideas are plausible, and several ablations support component importance. A conditional verdict requesting the reward check, an appropriate policy ablation, and code release is therefore appropriate.","tokens_in":24985,"tokens_out":6269,"duration_ms":58325,"concrete_test":"On the held-out test set, recompute the performance reward with ground truth: for each request, compare the absolute errors of yhat_dl and yhat_rp against Y_remain, and record the action chosen by the trained Decision Maker. If the chosen action is not significantly more often the lower-error action than a random policy would be, or if the sign of (yhat_dl - yhat_rp) does not predict which estimate is closer to Y_remain, then Eq. (1) is not an accuracy signal. As a second arm, retrain the Decision Maker with r_p = -|Y_remain - yhat_action|, keeping all other settings, and compare the resulting MAE and MUR with Table 3; a material change would confirm that the published reward was not optimizing the reported objective.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Sec. 4.1, the performance reward is defined in Eq. (1) as r_p = -(yhat_dl - yhat_rp) for a_i = 0 and r_p = -(yhat_rp - yhat_dl) for a_i = 1. Expanding: for a_i = 0, the reward is yhat_rp - yhat_dl; for a_i = 1, it is yhat_dl - yhat_rp. In both cases, the reward is maximized when the selected action's output is the smaller of the two estimates. There is no term involving the true remaining travel time Y_remain anywhere in Eqs. (1)-(4), and Algorithm 1 line 6 computes r_i from a_i and Yhat only. The paper's assertion that 'the performance reward directly reflects the accuracy' is therefore false as written: MAE/MAPE are minimized by choosing the estimate closest to Y_remain, not by choosing the minimum estimate. If the Predictor is roughly unbiased, the smaller of two noisy estimates is not systematically the more accurate one; if it overestimates, the reward may accidentally correlate with accuracy, but that is an empirical accident, not the stated mechanism. The headline accuracy gain in Table 3 is attributed to the Decision Maker's selective invocation, so this ungrounded reward is load-bearing: it is the only training signal for the agent. The ND/NA/NU ablations show that the framework's components matter, but they do not test whether the learned policy outperforms a random or always-lookup policy, nor whether Eq. (1) aligns with ground-truth error.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RLER-TTE, a framework for en-route travel time estimation that separates online inference into a lightweight Decision Maker and a Predictor. The Decision Maker, trained with Double DQN, chooses at each request whether to invoke the expensive prediction model or reuse the last stored result; the Predictor is an existing ER-TTE model (SSML or MetaER-TTE). The state representation combines offline route features with online driving-behavior features using self-attention, contrastive learning, and cross-attention fusion. Training includes experience replay, an end-to-end multi-request evaluation protocol, and a curriculum-learning scheduler for the Predictor. Experiments on Chengdu, Xi'an, and Porto report lower MAE, RMSE, and MAPE than several baselines, with a model utilization rate around 23-26% and reduced estimation latency.","tokens_in":25371,"tokens_out":6477,"duration_ms":58481,"significance":"If the accuracy and efficiency claims hold, the paper addresses a practically important problem: large-scale TTE services receive many en-route requests, and a learned selective-invocation policy could reduce expensive model calls by roughly a factor of four. The paper has clear strengths: the end-to-end evaluation with multiple requests per trajectory is more realistic than the fixed-proportion protocol used by prior ER-TTE work; the ablations (ND, NU, NA) and the PPO/A3C variants test several design choices; and the use of two different Predictors supports the claim that the framework is predictor-agnostic. The efficiency numbers in Table 4 are directly measured and plausible. However, the accuracy claims are seriously weakened by the reward misspecification in Eq. (1), which trains the Decision Maker to prefer the smaller of two predictor outputs rather than the estimate closer to the ground-truth remaining time. Because that reward is the only accuracy signal for the agent, the paper's central accuracy claim is not currently supported.","major_comments":[{"comment":"The performance reward is r_p = -(y_hat_dl - y_hat_rp) for direct lookup and r_p = -(y_hat_rp - y_hat_dl) for re-prediction. Expanding, the reward is y_hat_rp - y_hat_dl when a_i=0 and y_hat_dl - y_hat_rp when a_i=1, so in both cases the reward is maximized by taking the action whose output estimate is the smaller of the two numbers. No ground-truth remaining time Y_remain appears anywhere in Eqs. (1)-(4) or in the reward computation in Algorithm 1. The statement in Sec. 4.1 that the performance reward 'directly reflects the accuracy' is therefore false as written: minimizing MAE/MAPE requires comparing against Y_remain, not between the two candidate estimates. Because the Decision Maker is the only component that changes invocation decisions, this misspecification is load-bearing for the accuracy improvements claimed in Table 3 and Sec. 6.4. The reward should be redefined using each action's error against the ground truth (the labels Y are already inputs to Algorithm 1), or the authors must demonstrate empirically that choosing the smaller of the two estimates is a valid accuracy proxy for their predictors; in either case the experiments in Table 3 must be rerun.","section":"Sec. 4.1, Eq. (1), and Algorithm 1, line 6"},{"comment":"The experimental comparison lacks policy-level control baselines. The paper compares RLER-TTE against full re-prediction baselines (MUR = 100%) and against architecture ablations (ND, NU, NA), but not against random decision-making, an always-lookup policy, an always-re-predict policy, or the trivial rule that always selects the smaller of the two estimates. Given the reward issue in Eq. (1), the reported accuracy gains could in principle be produced by a simple minimum-selection rule rather than by the learned state-dependent Double DQN policy. Adding these controls, or reporting per-action error conditioned on the learned policy, is necessary to support the attribution in Sec. 6.4, observation (6), and to make the MUR results interpretable.","section":"Sec. 6.4, Table 3"},{"comment":"All accuracy and efficiency numbers are reported from a single run, with no standard deviations or significance tests. The improvements over the strongest baseline are modest in several metrics (e.g., Chengdu MAPE 30.83% vs. MetaER-TTE's 33.87%; MAE 84.76s vs. 90.74s), so seed variability could change the conclusions. Please report means and standard deviations over multiple seeds (at least 3-5) and, where appropriate, paired significance tests for MAE, RMSE, MAPE, and MUR on all three datasets.","section":"Sec. 6.4, Table 3; Sec. 6.6, Table 4"}],"minor_comments":[{"comment":"The difficulty score mu_i = MAE + MAPE sums quantities with different units and scales (seconds and percentage), which makes the score sensitive to the arbitrary scaling of the two terms; please normalize the components or justify the additive combination.","section":"Sec. 5.1, Eq. (8)"},{"comment":"The sentence 'Avg performs worse compared to deep learning methods, because the former can approximately fit any function' is garbled: it should say that deep learning methods can approximate complex functions, while the historical-average method cannot.","section":"Sec. 6.4, observation (2)"},{"comment":"The paper claims in Sec. 6.4, observation (7), a roughly fourfold reduction in computational load based on MUR, but the measured end-to-end latency reductions in Table 4 are closer to 2.5-3x (e.g., Chengdu 0.29s vs. 0.78s; Porto 0.27s vs. 0.86s). Please reconcile the MUR-based claim with the measured latency by reporting the Decision Maker's per-request overhead and end-to-end throughput.","section":"Sec. 6.6, Table 4"},{"comment":"The statement that the Decision Maker's complexity is 'an order of magnitude lower' than the baselines is not supported by the table: the Decision Maker's O(b n^2 d + b n d^2) is asymptotically comparable to MetaER-TTE's O(b n w d^2 + b n^2 d + b d^2 + b K d) when w and K are constants. Please qualify the complexity claim.","section":"Sec. 4.3, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The reward misspecification in Eq. (1) is central but appears fixable within the paper's scope, since the training labels are already available and the experimental framework can be rerun. I recommend major revision rather than rejection. If a rerun shows that the learned policy is essentially equivalent to a minimum-selection rule, the paper's contribution would be reduced to an efficiency mechanism, and the accuracy claims would need substantial revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nWhat you should know: the efficiency story is real, and the Decision Maker/Predictor split is a genuinely new framing for en-route travel time estimation. Prior work (SSML, MetaER-TTE) re-predicts at every request; this paper gates the expensive model behind a lightweight RL agent and measures a fourfold reduction in invocations (MUR ~25%) with latency dropping from ~1s/K to ~0.3s/K. That part is solid.\n\nThe accuracy claim, however, is not supported as written. The performance reward in Eq. (1) is r_p = -(yhat_dl - yhat_rp) for lookup and r_p = -(yhat_rp - yhat_dl) for re-prediction. In both cases the agent is rewarded for picking the action whose output is smaller. There is no ground-truth remaining time in any part of the reward. The text says this reward 'directly reflects the accuracy,' but it doesn't; it reflects which number is smaller. Worse, yhat_dl is a stale prediction from a longer remaining route, so comparing it with yhat_rp (a newer, shorter-route estimate) is apples-to-oranges. If the Predictor systematically overestimates, picking the smaller output can accidentally reduce error, but that is not the stated learning signal. The ablations (ND/NU/NA) show the components matter, but they never compare against random or always-lookup policies, so we can't attribute the accuracy gains to the RL decisions.\n\nThe efficiency findings are independent of this reward bug and are worth taking seriously. The complexity analysis is also credible: the decision gate is an order of magnitude cheaper than the predictor.\n\nMinor: no standard deviations or repeated runs are reported, so the modest MAE/MAPE differences in Table 3 could be noise.\n\nWho should read this: anyone working on selective inference, adaptive gating, or online spatio-temporal prediction. The reward misspecification is localized and fixable—add Y_remain to the reward or use a surrogate that correlates with true error, and run an ablation against random gating. As it stands, the paper overclaims accuracy but delivers a real efficiency contribution.\n\nI'd send this to peer review (major revision expected), and I'd cite it for the gating idea and the end-to-end evaluation protocol, not for the accuracy numbers.\n\nBest","headline":"The efficiency gating is a genuine contribution, but the reward in Eq. (1) does not measure accuracy, so the headline accuracy gains are not attributable to the RL agent.","tokens_in":25894,"tokens_out":4692,"would_cite":true,"duration_ms":39756,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement-learning gate decides when to skip recomputing an en-route travel-time estimate, and the paper reports this cuts expensive model invocations by roughly fourfold while lowering prediction error on three real-world…","keywords":["travel time estimation","en route travel time estimation","reinforcement learning","Double DQN","curriculum learning","spatio-temporal data mining","trajectory"],"falsifier":"Take the trained Decision Maker to a held-out set of en-route requests, and at each state compare its chosen action with the action that actually yields lower absolute error against the true remaining travel time; if the agent's choices track the sign of $\\hat{y}_{dl} - \\hat{y}_{rp}$ rather than which estimate is closer to ground truth, the reward is not measuring accuracy.","tokens_in":24780,"feed_emoji":"🚕","tokens_out":7325,"duration_ms":68272,"temperature":0.7,"pith_summary":"The paper is trying to establish that en-route travel time estimation does not need to recompute a deep prediction at every request. It proposes a two-component pipeline—a lightweight reinforcement-learning Decision Maker that decides, at each request, whether to re-run the heavy Predictor or return the latest cached estimate—and claims this gate preserves or improves accuracy while cutting model invocations to roughly a quarter. The evidence is an end-to-end evaluation on Chengdu, Xi'an, and Porto trajectories, where RLER-TTE reports lower MAE, RMSE, and MAPE than the MetaER-TTE baseline with model utilization rates of 22.8–25.6%, a fourfold efficiency gain. The paper also argues that previous ER-TTE training and evaluation, which uses one fixed traveled-route proportion per trajectory, unfairly understates what an online model can do, and proposes multi-request curriculum training instead.","feed_headline":"RL gate cuts travel-time recomputation ~4x, lifts accuracy","feed_subtitle":"En-route travel-time service skips full re-prediction unless a trained agent deems new driving data worth it.","key_machinery":"The central object is the Decision Maker/Predictor pipeline. The Decision Maker is a Double-DQN network that maps a fused state—offline embeddings of road-network structure, time slots, and historical traffic, plus online embeddings of driving behavior and past decisions—through separate multi-head self-attention encoders, an online-offline attention fusion, and an InfoNCE contrastive-alignment loss, to choose one of two actions: re-predict or look up the cached estimate. The reward (Eqs. 1–4) combines a prediction-difference term $r_p$, an efficiency penalty $r_e$ for using the Predictor, and a linear frequency reward $r_f$ that encourages re-prediction after a long gap. The Predictor is trained end-to-end with a curriculum that partitions trajectories by length and traveled proportion, ranks samples within each metaset by an expert model's MAE plus MAPE, and gradually introduces harder data. The complexity analysis shows the Decision Maker is an order of magnitude cheaper than the Predictor, which is what makes the gate viable.","core_discovery":"The central claim is that en-route travel time estimation should be recast as a learned cache-versus-recompute decision rather than an unconditional re-prediction at every request. The RLER-TTE framework places a lightweight Double-DQN agent, the Decision Maker, in front of a heavy estimation model, the Predictor, instantiated with either MetaER-TTE or SSML. At each request the agent receives a state built from attention-encoded offline route features and online driving-behavior and decision-history features, then chooses between re-prediction and direct lookup from an Inference Memory that stores the most recent prediction. The reward is the signed difference between the two estimates, plus an efficiency penalty and a recency term. On the paper's end-to-end evaluation over all requests along each trajectory, RLER-TTE reports lower MAE, RMSE, and MAPE than MetaER-TTE on all three datasets while using the expensive Predictor for only about a quarter of requests, which the paper attributes to the Decision Maker selectively feeding useful real-time information to the Predictor.","pith_inferences":["The reward in Eq. (1) uses only the signed difference between the two predictor outputs; an obvious unstated test is whether a reward based on true remaining-time error changes the learned policy and the reported accuracy/efficiency trade-off.","The same learned gate could be dropped into other online prediction services where cached outputs can be reused—ETA widgets, route re-optimization, or real-time arrival estimates—since it needs only two outputs, a diff-like reward, and a recency term.","The paper's interval experiments imply system operators could tune the request interval as a compute-quality knob; the authors do not provide a cost model that translates MUR and MAPE into server cost, but the data make such a model straightforward to build.","A useful extension would compare the trained agent against simple threshold heuristics, such as skipping re-prediction when average speed barely changed, to see how much of the gain comes from learning rather than from the gate structure itself."],"forward_implications":["At the claimed utilization rates (22.8–25.6%), an ER-TTE service can serve roughly four times as many real-time requests with the same predictor compute, because only about a quarter of requests trigger re-prediction.","Under the paper's end-to-end protocol, each trajectory contributes many training requests with different traveled proportions, so a model trained this way is evaluated on the full range of online requests rather than one fixed proportion.","The framework is portable across predictors: swapping MetaER-TTE for SSML still improves over the SSML baseline, so the gate does not depend on one estimation model.","Within this two-action discrete setting, the paper's Double-DQN agent beats PPO and A3C variants, which suggests value-based RL is the better fit for cache-versus-recompute decisions.","Longer request intervals raise MAPE for both the proposed method and MetaER-TTE, and raise the model utilization rate, so update frequency is a real operating parameter for ER-TTE systems."],"supporting_citations":[{"why":"Supplies SSML, one of the two ER-TTE predictors the framework is instantiated with and a baseline that defines the ER-TTE few-shot setting.","marker":"[7]"},{"why":"Supplies MetaER-TTE, the primary state-of-the-art baseline and the Predictor used in most experiments; its accuracy and efficiency define the bar the paper claims to beat.","marker":"[6]"},{"why":"Provides the ConST/ConSTGAT backbone whose embedding and graph-attention structure underlie both meta-learning baselines and the complexity comparison.","marker":"[8]"},{"why":"Supplies the Double DQN algorithm that the Decision Maker is built on, including the overestimation-reduction training objective.","marker":"[27]"},{"why":"Provides MAML, the meta-learning method that MetaER-TTE adapts, anchoring the strongest baseline the paper must outperform.","marker":"[9]"},{"why":"Provides the Transformer architecture used in the ND ablation, supporting the claim that the value-based RL decision network, not just any sequence model, drives the efficiency gain.","marker":"[29]"}],"fun_headline_variants":["RL agent decides when to recompute travel-time, boosting speed and accuracy","Travel-time estimator uses RL to skip needless re-predictions, improving accuracy","RL gate reduces travel-time recomputations ~4x, lifts accuracy","En-route travel-time: RL gate triggers re-prediction only when needed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reward the Decision Maker learns from compares the two predictor outputs to each other, never to the true remaining travel time, so the entire policy assumes that choosing the numerically smaller estimate is the same as choosing the more accurate one.","fun_headline_variants_meta":{"raw":{"variants":["RL agent decides when to recompute travel-time, boosting speed and accuracy","Travel-time estimator uses RL to skip needless re-predictions, improving accuracy","RL gate reduces travel-time recomputations ~4x, lifts accuracy","En-route travel-time: RL gate triggers re-prediction only when needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001047,"raw_usage":{"total_tokens":4447,"prompt_tokens":1041,"completion_tokens":3406,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":3323}},"tokens_in":657,"tokens_out":3406,"duration_ms":24319,"temperature":1.0,"reasoning_tokens":3323,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:13:32.925038+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained Decision Maker to a held-out set of en-route requests, and at each state compare its chosen action with the action that actually yields lower absolute error against the true remaining travel time; if the agent's choices track the sign of $\\hat{y}_{dl} - \\hat{y}_{rp}$ rather than which estimate is closer to ground truth, the reward is not measuring accuracy.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies SSML, one of the two ER-TTE predictors the framework is instantiated with and a baseline that defines the ER-TTE few-shot setting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies MetaER-TTE, the primary state-of-the-art baseline and the Predictor used in most experiments; its accuracy and efficiency define the bar the paper claims to beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ConST/ConSTGAT backbone whose embedding and graph-attention structure underlie both meta-learning baselines and the complexity comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides MAML, the meta-learning method that MetaER-TTE adapts, anchoring the strongest baseline the paper must outperform."}],"review_version":1}