Pith. sign in

REVIEW 5 major objections 5 minor 16 references

Robust Taxi Fare Prediction Under Noisy Conditions: A Comparative Study of GAT, TimesNet, and XGBoost

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read XGBoost, not GAT or TimesNet, is the model that keeps taxi fare predictions accurate when the input data are corrupted by noise.

desk verdict XGBoost vs. GAT vs. TimesNet benchmark with code and a public dataset, but impossible R2 values and table-vs-conclusion contradictions make the headline claim unsupported as written. read the letter →

arxiv 2507.20008 v1 pith:QF6A6STG submitted 2025-07-26 cs.LG cs.AI

Padmavathi Moorthy (SUNY Buffalo) This is my paper · ORCID
classification cs.LGcs.AI
keywords taxifarepredictionXGBoostGraphAttentionNetworkTimesNetGaussiannoiseinjectionuncertaintyestimationout-of-distributionrobustnessdatadenoising
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper compares three model families for predicting NYC taxi fares from over 55 million trip records: XGBoost, a graph attention network (GAT), and a temporal model (TimesNet). It tests them on both clean data and data corrupted by injected Gaussian noise, imputation, and autoencoder denoising, measuring accuracy, calibration, uncertainty, and out-of-distribution robustness. The central claim is that XGBoost remains the most accurate and best-calibrated model under both clean and noisy conditions, while GAT and TimesNet degrade substantially, with GAT's R2 turning strongly negative under noise. If true, the practical takeaway is that for tabular fare prediction, a classical gradient-boosted tree baseline should be preferred over these deep architectures, especially when data quality is uncertain.

What carries the argument

The comparison is carried by three artifacts: XGBoost, a gradient-boosted decision tree regressor with L2 regularization and early stopping; a multi-layer graph attention network whose graph nodes are trips connected by fare similarity and temporal adjacency; and TimesNet, a temporal model using 2-D temporal-variation blocks over sliding windows of length 10. Performance is measured along regression metrics (MAE, MSE, R2), calibration curves, ensemble-based uncertainty estimates, and OOD tests on noise-perturbed samples. Gaussian noise injection at a fixed level, IQR outlier removal, KNN imputation, and autoencoder denoising form the data pipeline that defines the noisy and denoised regimes.

What would settle it

Feed the models data with structured corruption instead of Gaussian noise, such as pickup coordinates shifted along a city grid or fares missing in a non-random pattern, and check whether XGBoost still keeps a positive R2 and stable calibration while GAT and TimesNet collapse.

Watch

Extended reading notes

Core claim

The paper reports that XGBoost achieves MAE 0.1040 and R2 0.9910 on clean data and MAE 0.8203 with R2 0.7326 after Gaussian noise injection, whereas GAT drops from R2 0.2153 to -1.1237 and TimesNet remains negative (R2 about -0.09) in both settings. The authors interpret this as evidence that XGBoost 'emerges as a reliable baseline for both structured tabular learning and robustness under data perturbation' and 'demonstrates superior performance under both clean and noisy conditions.' They attribute the difference to XGBoost's tree ensemble structure and built-in regularization, in contrast to GAT's sensitivity to graph topology disruption and TimesNet's sensitivity to periodic patterns and memory demands.

Load-bearing premise

The entire robustness ranking rests on the assumption that adding fixed-level Gaussian noise and removing IQR outliers faithfully mimics the kinds of errors that occur in real taxi fare data, such as GPS drift or transmission failures.

Editorial extensions

If this is right

  • Practitioners building taxi or ride-hailing fare predictors should use gradient-boosted trees as the default baseline, reserving deep graph or sequence models for cases where spatial or temporal structure is explicitly needed.
  • Noise-aware pre-processing such as autoencoder denoising helps deep models, especially TimesNet, but does not close the gap with XGBoost on the metrics studied.
  • GAT's negative R2 under noise implies that end-to-end graph models on trip-level data are brittle unless combined with noise-aware regularization or denoising.
  • The evaluation recipe of calibration, uncertainty, bin-wise MAE, and OOD tests can be reused as a template for robustness benchmarking in other tabular regression tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own numbers suggest XGBoost is the safe choice for tabular fare data, but the gap may narrow if the deep models were given extensive hyperparameter tuning or pretraining, which the paper does not report.
  • A testable extension is to inject realistic error types, such as rounded GPS coordinates or dropped signals, and compare whether the robustness ordering persists.
  • If the fixed-level Gaussian noise model is unrealistic, the ranking could reverse for real-world corruptions, so readers should treat the 'XGBoost superior under noise' claim as specific to Gaussian perturbation.
  • TimesNet's single window size of 10 may understate its temporal modeling capability, so its poor performance should be interpreted with that limitation in mind.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper reports an empirical comparison of GAT, XGBoost, and TimesNet for taxi fare prediction on a 55-million-row NYC taxi dataset. The authors construct clean and noisy variants by injecting fixed-level Gaussian noise, applying KNN imputation, IQR-based outlier removal, and autoencoder denoising, then evaluate the three models with MAE, MSE, R², calibration, uncertainty, and OOD-style robustness tests. The central claim is that XGBoost is the most accurate and robust model, while GAT degrades sharply under noise and TimesNet shows partial resilience. The manuscript includes code availability and an end-to-end pipeline.

Significance. If the reported comparisons were reliable, the study would be a useful practical benchmark for ride-hailing fare prediction under data corruption, and the public code release is a strength. However, the central ranking is not supported by the evidence as written. There are internal inconsistencies in the reported metrics, including an impossible R² value, a sign flip between tables for the same quantity, and a conclusion about TimesNet that directly contradicts the table it summarizes. Because these issues affect the core comparative claim, the current manuscript cannot be used as a basis for the stated practical guidelines.

major comments (5)
  1. [VI-B, Tables I and II] The GAT noisy R² is reported as 1.1237 in Table I but as -1.1237 in Table II for what appears to be the same evaluation; Table VII also uses -1.1237. An R² value above 1 is impossible, and the sign flip cannot both be correct. This is not a cosmetic issue: it undermines confidence in the accuracy of every metric table in the paper and leaves the central comparison unsupported.
  2. [VII vs. Tables III and V] The conclusion states that TimesNet 'demonstrates the highest tolerance to noise (MAE 1.35)', but Table V reports TimesNet noisy MAE as 1.7303 and Table III reports XGBoost noisy MAE as 0.8203. Moreover, TimesNet's R² is negative in both clean (-0.0779) and noisy (-0.0919) conditions. The stated conclusion directly contradicts the reported numbers and must be reconciled before any ranking is claimed.
  3. [VI-B.1] The text claims GAT shows 'near-perfect calibration' on clean data where Table I reports R² = 0.2153. While calibration and R² are conceptually distinct, the paper provides no quantitative calibration error metric, only plots, so the 'near-perfect calibration' claim is not substantiated. A numerical calibration measure (e.g., expected calibration error or reliability diagram statistics) is needed for this claim to be meaningful.
  4. [Tables I–VI and IV-C] All performance tables report single-run metrics with no standard deviations, confidence intervals, or repeated-seed evaluations. Since the paper's conclusions rest on differences between models, and some differences are modest, the absence of any variability estimate means the reported ranking cannot be distinguished from noise. This is especially important for the XGBoost vs. GAT comparison, where clean MAE values differ by an order of magnitude but the noisy R² sign inconsistency already raises doubt.
  5. [IV-A.1 and IV-A.3] The robustness ranking depends entirely on the assumption that fixed-level Gaussian noise injection and IQR-based outlier removal faithfully emulate real-world taxi fare corruption. The paper provides no external validation that this noise model matches actual sensor, GPS, or transmission errors, nor any analysis of structured corruption such as missingness correlated with location or time. As a result, the practical guideline that XGBoost is the safest choice under 'realistic conditions' is not established.
minor comments (5)
  1. [II-B] TimesNet is described as an 'RNN-based multivariate time series forecasting model', but the cited TimesNet architecture uses 2D temporal-variation blocks and attention, not recurrent connections; this description should be corrected.
  2. [IV-D.3] The sentence 'Early stopping: patience=7 on validation MA' appears to be truncated; it should read 'validation MAE' or similar.
  3. [References [1] and [9]] Reference [1] gives a placeholder arXiv identifier (1511.00000) and cites Geras and Sutton's autoencoder paper, which does not appear to be the source of the noise-annealing technique described in Section III; Reference [9] also uses a placeholder identifier (2401.12345). These must be replaced with accurate citations.
  4. [Figure 2 caption] The caption states that the IQR outlier removal plot shows 'the efficacy of denoising', but the figure depicts outlier filtering rather than autoencoder-based denoising; the wording should be aligned with the actual content.
  5. [VI-A.2] The text says 'Figure 2 shows the skewness before and after perturbation' but Figure 2 is split into 'Skewness' and 'IQR Outlier Removal'; the narrative should clarify which panel corresponds to which claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper is an empirical benchmark whose conclusions rest on measured metric tables, not on fitted parameters renamed as predictions or on a self-citation chain.

full rationale

The paper's central claims are empirical comparisons among GAT, XGBoost, and TimesNet on clean and noise-injected taxi fare data. Each model is trained, tuned, and then evaluated on held-out test data, with performance summarized in Tables I-VII. The conclusion that XGBoost is the most reliable baseline is derived directly from these reported metrics (e.g., Table III and Table VII), not from any equation that defines the outcome in terms of the input. Hyperparameter tuning and early stopping are standard model-selection procedures and do not constitute fitting a parameter to the test set, so the 'fitted input called prediction' pattern does not apply. The noise injection and denoising pipeline define an operational notion of robustness, but that is a methodological choice, not a circular reduction: the models' rankings are not true by construction. There are no load-bearing self-citations or imported uniqueness theorems; the related-work references are contextual. The reviewer's noted internal inconsistencies, such as GAT noisy R-squared appearing as +1.1237 in Table I and -1.1237 in Table II, and the conclusion's statement that TimesNet has the 'highest tolerance to noise (MAE 1.35)' despite XGBoost's lower noisy MAE of 0.8203, are serious correctness and reproducibility concerns, but they are not circularity. A wrong or inconsistent measurement can undermine a conclusion without making the argument circular. Therefore, the circularity score is 0.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The benchmark depends on several preprocessing and modeling choices that are stated but not justified or ablated. The XGBoost result is probably robust to many of these choices, but the poor performance of GAT and TimesNet cannot be separated from misconfiguration without ablations and external validation of the noise model.

free parameters (7)
  • XGBoost hyperparameters (tree depth, L2 lambda, learning rate) = grid searched; final values not reported
    Section IV-C.3 states grid search over depth and lambda with early stopping, but the chosen values are absent, so the XGBoost result cannot be recreated or audited.
  • TimesNet window size = 10
    Section IV-D.1 fixes sliding windows of size 10 over normalized features; this choice controls how much temporal context the model sees and is not ablated.
  • TimesNet training hyperparameters = lr=5e-4, weight_decay=1e-5, dropout=0.3, gradient clip=1.0
    Section IV-D.3 fixes these values with no sensitivity analysis; performance may depend on them.
  • GAT architecture (layers, attention heads, hidden sizes) = not specified
    Section IV-B.2 describes a multi-layer GAT with dropout and NeighborLoader but gives no layer count, head count, or hidden dimension, so the GAT comparison is not reproducible.
  • KNN imputation parameters = not reported
    Section IV-A uses KNN for missing coordinates but does not state the number of neighbors, distance metric, or weighting scheme.
  • IQR outlier thresholds and NYC bounds = not reported
    Section IV-A.3 removes outliers with IQR and expected NYC bounds, but the exact thresholds are not given; this filtering changes the target distribution and the noise evaluation.
  • Autoencoder architecture and training = not reported
    Section IV-A.4 trains a denoising autoencoder on noisy columns, but architecture, loss, and training schedule are unspecified; denoised outputs feed all models.
assumptions (5)
  • domain assumption The NYC Yellow Taxi Trip Records are a representative real-world benchmark for taxi fare prediction.
    Section V introduces the TLC dataset; all conclusions are limited to this city and this fare structure.
  • ad hoc to paper Fixed-level Gaussian noise injection emulates realistic sensor, system, and transmission noise in fare data.
    Section IV-A.1 injects Gaussian noise; no external measurement validates that this noise model matches real GPS drift or system errors.
  • ad hoc to paper Representing each trip as a graph node with edges from fare similarity and temporal adjacency is a valid relational encoding for fare prediction.
    Section IV-B.1 constructs the graph this way with no ablation against other graph definitions; GAT's poor performance may stem from this choice.
  • domain assumption IQR-based outlier removal does not introduce selection bias that changes the ranking of models.
    Section IV-A.3 removes spatial and fare anomalies before training and evaluation; without reported thresholds, the effect on extreme fares is unknown.
  • standard math The KS test with p<0.05 is an appropriate check for distribution shift after noise injection.
    Section IV-A.2 uses the KS test to confirm deviations; this is a standard statistical tool.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Robust Taxi Fare Prediction Under Noisy Conditions: A Comparative Study of GAT, TimesNet, and XGBoost." pith.science (2026). https://pith.science/paper/QF6A6STG

@misc{pith2026250720008,
  author       = {Pith},
  title        = {Pith review of: Robust Taxi Fare Prediction Under Noisy Conditions: A Comparative Study of GAT, TimesNet, and XGBoost},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QF6A6STG}},
  note         = {Machine review of arXiv:2507.20008}
}
read the original abstract

Precise fare prediction is crucial in ride-hailing platforms and urban mobility systems. This study examines three machine learning models-Graph Attention Networks (GAT), XGBoost, and TimesNet to evaluate their predictive capabilities for taxi fares using a real-world dataset comprising over 55 million records. Both raw (noisy) and denoised versions of the dataset are analyzed to assess the impact of data quality on model performance. The study evaluated the models along multiple axes, including predictive accuracy, calibration, uncertainty estimation, out-of-distribution (OOD) robustness, and feature sensitivity. We also explore pre-processing strategies, including KNN imputation, Gaussian noise injection, and autoencoder-based denoising. The study reveals critical differences between classical and deep learning models under realistic conditions, offering practical guidelines for building robust and scalable models in urban fare prediction systems.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 13 canonical work pages

  1. [1]

    Autoencoders,

    K. J. Geras and C. Sutton, “Autoencoders,” arXiv preprint arXiv:1511.00000, 2015

    work page arXiv 2015

  2. [2]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907 , 2017

    arXiv 2017

  3. [3]

    Graph attention networks,

    P. Veli ˇckovi´c, G. Cucurull et al. , “Graph attention networks,” in Proc. ICLR, 2018

    work page 2018

  4. [4]

    Failing loudly: An empirical study of methods for detecting dataset shift,

    S. Rabanser et al. , “Failing loudly: An empirical study of methods for detecting dataset shift,” arXiv preprint arXiv:1810.11953 , 2019

    arXiv 2019

  5. [5]

    AmbieGen: A Search-based Framework for Autonomous Systems Testing

    Q. Wu et al. , “Energy-based out-of-distribution detection for graph neural networks,” arXiv preprint arXiv:2301.01234 , 2023

    work page Pith review arXiv 2023

  6. [6]

    Rethinking the Role of Token Retrieval in Multi-Vector Retrieval

    P. Pernot, “Can bin-wise scaling improve consistency and adaptivity of prediction uncertainty for machine learning regression?” arXiv preprint arXiv:2304.01982, 2023

    work page Pith review arXiv 2023

  7. [7]

    On analysis of gan-based image-to-image translation with gaussian noise injection,

    C. Shi et al., “On analysis of gan-based image-to-image translation with gaussian noise injection,” in CVPR Workshops, 2024

    work page 2024

  8. [8]

    Current-driven dynamics of antiferromagnetic skyrmions: from skyrmion Hall effects to hybrid inter-skyrmion scattering

    C. Liu et al. , “Spatial-temporal large language model for traffic fore- casting,” arXiv preprint arXiv:2403.01987 , 2024

    work page Pith review arXiv 2024

Show all 16 references
  1. [9]

    Navig: Natural language-guided analysis with vision for image geo-localization,

    Z. Zhang et al. , “Navig: Natural language-guided analysis with vision for image geo-localization,” arXiv preprint arXiv:2401.12345 , 2025

    2025 arXiv

  2. [10]

    Xgboost: A scalable tree boosting system,

    T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proc. 22nd ACM SIGKDD Conf. , 2016

    2016

  3. [11]

    Catboost: Unbiased boosting with categorical features,

    L. Prokhorenkova, G. Gusev et al. , “Catboost: Unbiased boosting with categorical features,” in Advances in Neural Information Processing Systems (NeurIPS) , vol. 31, 2018

    2018

  4. [12]

    Lightgbm: A highly efficient gradient boosting decision tree,

    G. Ke, Q. Meng, T. Finley et al., “Lightgbm: A highly efficient gradient boosting decision tree,” in Advances in Neural Information Processing Systems (NeurIPS) , vol. 30, 2017

    2017

  5. [13]

    Timesnet: Temporal and inter-sample correlation exploration for multivariate time series forecasting,

    Z. Wu, Q. Fan, C. Ding et al. , “Timesnet: Temporal and inter-sample correlation exploration for multivariate time series forecasting,” in Proc. ICLR, Kigali, 2023

    2023

  6. [14]

    Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting,

    H. Zhou, S. Zhang, J. Peng et al. , “Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting,” in Ad- vances in Neural Information Processing Systems (NeurIPS) , vol. 35, 2022

    2022

  7. [15]

    Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting,

    H. Zeng, M. Ding, X. Yang et al. , “Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting,” in Proc. ICLR, 2023

    2023

  8. [16]

    Informer: Beyond efficient trans- former for long sequence time-series forecasting,

    H. Zhou, S. Zhang, M. Zhu et al. , “Informer: Beyond efficient trans- former for long sequence time-series forecasting,” in Advances in Neural Information Processing Systems (NeurIPS) , vol. 34, 2021

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.