{"id":"a4be4482-b7f1-47aa-907a-48783ceb372e","arxiv_id":"2502.03395","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Gradient-boosted ML models achieved the best accuracy in a 4-restaurant hourly sales forecasting benchmark, with zero-shot Chronos-Bolt foundation models competitive in 3 of 4 cases.","lead":"This paper compares statistical, machine learning, deep learning, and foundation models for forecasting hourly restaurant sales 14 days ahead using data from four German restaurants. It reports that gradient-boosting ML models perform best, that zero-shot Chronos-Bolt foundation models are close behind, and that a hybrid Spark-Pandas approach enables horizontal scalability.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed consistent superiority of gradient boosting over Chronos-Bolt is not statistically supported: with one 14-day test window per restaurant, no reported per-metric scores, and no confidence intervals, the ranking cannot be distinguished from noise.","rationale":"The reader's condition is appropriate: the paper needs more evidence before the central ranking claim can be accepted. My concern is not exactly the representativeness of the four restaurants, though that is related; it is that even for the restaurants studied, the paper provides no numeric results, no error bars, and no repeated-evaluation protocol, so the claimed consistency of the ranking is not verifiable. A conditional verdict is therefore right, but the condition should explicitly include releasing per-metric tables and confidence intervals, not merely adding textual detail. I do not see an internal inconsistency that would justify rejection; the empirical setup is plausible, and the authors may have the missing numbers. The concrete test would settle whether the gradient-boosting advantage survives basic uncertainty quantification.","tokens_in":4134,"tokens_out":3274,"duration_ms":34083,"concrete_test":"Request the authors' per-model, per-restaurant evaluation tables for all six metrics and the forecasting scripts. Then, using the same data pipeline, re-run the comparison with a rolling-window protocol (e.g., 12 non-overlapping 14-day test horizons per restaurant, or alternatively at least 10 additional restaurants) and compute paired differences with 95% bootstrap confidence intervals for MAE, RMSE, and MAPE. If the intervals for the ML-vs-Chronos-Bolt difference include zero in most restaurants, or if per-metric rankings conflict, the 'consistently highest' claim is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section III-B is that gradient-boosting ML models 'consistently achieved the highest performance across all restaurants,' with Chronos-Bolt competitive in 3 of 4 cases. Every load-bearing element of that comparison is missing from the text. First, the evaluation appears to use a single 14-day test horizon per restaurant (Section III-A); no rolling windows, repeated forecasts, or cross-validation are described, so the reported ranking is one draw from a noisy distribution. Second, Section II-C defines six metrics, but the paper never reports numeric scores per model, restaurant, and metric, nor does it state which metric or aggregation is used to declare a 'highest performance' winner. Third, no standard errors, confidence intervals, or pairwise significance tests are given; with n=4 restaurants, a single atypical location could flip the aggregate ranking. These omissions are not cosmetic: the headline superiority of gradient boosting and the '3 out of 4' competitiveness of Chronos-Bolt are exactly the quantities that require exact values and uncertainty quantification. The hybrid Spark-Pandas scalability statement in Section III-C is also unsupported by measured scaling experiments, but the ranking claim is the more central and load-bearing result.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates 12 forecasting models for hourly restaurant sales over a 14-day horizon using real-world data from four German restaurants, one per chain and geographically distributed. Model families include statistical (Prophet), machine learning (linear regression, XGBoost, LightGBM), deep learning (N-Beats), and foundation models (TimesFM, four sizes of Chronos-Bolt). The authors claim that gradient-boosting ML models consistently achieved the highest performance across all restaurants, that Chronos-Bolt matched ML accuracy in 3 out of 4 cases with zero-shot inference, and that a hybrid Spark-Pandas approach provides horizontal scalability. The paper concludes with recommendations favoring ML meta-models for accurate large-scale deployment and foundation models for simplicity when GPUs are available.","tokens_in":4483,"tokens_out":3575,"duration_ms":33521,"significance":"The paper addresses a practically relevant question with real-world data and compares a broad range of model families, including several zero-shot foundation models. If the claims were fully supported, the finding that gradient-boosting ML models remain competitive with or superior to foundation models in this hospitality setting would be a useful data point for practitioners, as would the proposed Spark-Pandas hybrid for large-scale retraining. However, the current evidence is insufficient to establish the headline ranking: no numeric metric tables are provided, there is no uncertainty quantification, and the scalability claim is unsupported by any measurements. The paper is potentially useful but requires substantial additional analysis.","major_comments":[{"comment":"The central claim that gradient-boosting ML models 'consistently achieved the highest performance across all restaurants' is not supported by numeric results in the text: Section II-C defines six metrics, but Figures 2-5 are the only reported outcomes and no table gives per-model, per-restaurant metric values, nor does the text state which metric or aggregation was used to declare a winner. I request tables with all six metrics for every model and restaurant, together with a clear statement of the ranking criterion.","section":"III-B"},{"comment":"The evaluation uses a single 14-day test horizon per restaurant, with no rolling windows, repeated forecasts, or cross-validation described. With n=4 restaurants, the observed ranking and the '3 out of 4 cases' competitiveness claim are one draw from a noisy distribution; without standard errors, confidence intervals, or pairwise comparisons, the ranking cannot be distinguished from chance.","section":"III-A"},{"comment":"The scalability claim for the hybrid Spark-Pandas approach is stated without any measured runtime, throughput, or scaling data; the text reports that the method 'demonstrated excellent stability, horizontal scalability, ease of implementation, and robust performance' but provides no experiment with varying cluster sizes, data volumes, or wall-clock times. Quantitative scaling experiments are needed to support this claim.","section":"III-C"},{"comment":"The paper does not explain how future weather and calendar features are obtained for the 14-day forecast horizon; if actual weather observations from the test period were used, the ML models that consume these features are evaluated under information not available at prediction time, which biases the comparison against zero-shot foundation models that use only the time series. The authors should state whether test-period covariates were known or forecasted, and how.","section":"II-A/III-A"},{"comment":"The claim that the four restaurants constitute 'a Representative Diverse Dataset' is asserted without support: no description of the random selection procedure, no screening for typical sales patterns, and no analysis of how these four locations represent the population of thousands of restaurants is given. This is the basis for the generalization claim, so the selection procedure and representativeness evidence should be reported.","section":"III-A"}],"minor_comments":[{"comment":"The feature table lists 'is_ascension_day' twice; one occurrence should be removed.","section":"II-A"},{"comment":"The heading 'Performance Analyisis' contains a typo and should read 'Performance Analysis'.","section":"III-B"},{"comment":"The metric table uses LaTeX-style notation such as '𝑹𝟐' without defining it as the coefficient of determination in equation form; consider adding the explicit formula for each metric.","section":"II-C"},{"comment":"The figures are referenced collectively but not discussed individually in the text; adding a short discussion of each restaurant's pattern would improve readability.","section":"Figures 2-5"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable fit for an applied machine learning benchmarking venue, but the central claims are currently not verifiable from the text because the quantitative results and scaling measurements are missing. I believe the deficiencies are fixable within the manuscript's scope by adding metric tables, uncertainty-aware comparisons, and scaling experiments, so major_revision is appropriate rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me save you the trouble of reading the whole thing. The new bit is narrow: they benchmarked Chronos-Bolt (four sizes) and TimesFM zero-shot against XGBoost, LightGBM, Prophet, N-Beats, and linear regression on hourly sales from four German restaurants, with a 14-day horizon. As far as I can tell, that specific comparison on that data hasn't been published before. The finding that zero-shot foundation models sit close to tuned gradient boosting in 3 of 4 restaurants is genuinely useful for practitioners in hospitality who are deciding whether to invest in feature engineering.\n\nThe paper does some things well. The experimental setup is clearly described: features grouped into weather, time-of-day, day-of-week, holidays, and event indicators. They use six metrics. The prose is direct and doesn't oversell the foundation models; the recommendation to use ML for accuracy and foundation models for simplicity is measured.\n\nThe soft spots, however, are the load-bearing ones. The central claim — that gradient boosting 'consistently achieved the highest performance across all restaurants' — appears only as figures. No numeric table reports MSE, MAE, RMSE, MAPE, R2, or correlation for each model and restaurant. There is no error bar, no confidence interval, no significance test, and no description of repeated forecasts or rolling windows. The evaluation appears to be a single 14-day test window per restaurant. With n=4, the difference between gradient boosting and Chronos-Bolt could easily be noise, and one atypical restaurant could flip the aggregate ranking. The '3 out of 4' competitiveness claim needs the same quantitative support. Also, the hybrid Spark-Pandas scalability claim in Section III-C is delivered as an assertion, with no runtime measurements, cluster specs, or scaling plots.\n\nOther smaller issues: hyperparameters are undisclosed, so the ML results aren't reproducible; there's no code or data release; the dataset is described as 'representative diverse' with only four restaurants, which is an overstatement; and the reference list is padded with several of the first author's prior proteomics bioinformatics papers that have nothing to do with time series.\n\nBottom line: the paper is a decent industry case study, not a scientific demonstration of model superiority. If the authors add the missing numbers, confidence intervals, and scalability measurements, it would be a solid practitioner-oriented benchmark. As it stands, it is conditionally acceptable at best. I'd send it to a serious referee, because the application domain is underserved and the specific comparison is new, but I'd tell the referee to insist on the quantitative tables and a much more cautious interpretation of the ranking.\n\nFor peer review: yes, it deserves a referee, with heavy revision expected.","headline":"A useful practical benchmark for restaurant sales forecasting, but the headline ranking is asserted rather than demonstrated because the paper withholds the actual numbers.","tokens_in":4844,"tokens_out":2663,"would_cite":false,"duration_ms":24014,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that gradient-boosting machine-learning models, rather than statistical or deep-learning baselines, give the most accurate 14-day hourly sales forecasts for restaurants, while Chronos-Bolt foundation models are…","keywords":["time series forecasting","zero-shot inference","foundation models","gradient boosting","Chronos-Bolt","Spark","restaurant sales","scalability"],"falsifier":"Rerun the same 14-day forecast benchmark on a random sample of fifty or more restaurants from the same network, ideally in a different season, and check whether gradient boosting still beats Chronos-Bolt in most locations; if the ranking reverses on a substantial share of restaurants, the paper's main conclusion would not generalize.","tokens_in":3944,"feed_emoji":"📈","tokens_out":6606,"duration_ms":58159,"temperature":0.7,"pith_summary":"This paper tests whether modern forecasting approaches hold up on a real operational problem: predicting hourly sales for each of thousands of restaurants in Germany, 14 days ahead. Using two years of historical data plus weather, calendar, and time-of-day features from four geographically distributed restaurants, it benchmarks statistical, machine-learning, deep-learning, and pretrained foundation models. The central result is that gradient-boosting machine-learning models (XGBoost, LightGBM) achieve the highest accuracy on all four restaurants, while Chronos-Bolt foundation models come close in three of four cases using only the raw time series, with no feature engineering. Prophet and N-Beats trail the leaders, and a hybrid Spark-Pandas pipeline is presented as the practical way to retrain such models periodically across the full restaurant network. The paper's point is that accuracy and scalability can be had together, with foundation models emerging as a low-effort alternative when GPUs are available.","feed_headline":"Gradient boosting tops foundation models in restaurant sales forecast","feed_subtitle":"Zero-shot Chronos-Bolt matches ML accuracy in three of four cases; Spark-Pandas keeps retraining scalable.","key_machinery":"The carrying mechanism is the controlled benchmark: each model receives the same training input of two years of hourly sales plus weather flags (rain, snow, clear, humidity, temperature), cyclic encodings of hour, day of week and week of year, event flags for lunch, dinner, and peak hours, and German holiday flags, then produces a 14-day hourly forecast scored by MSE, MAE, RMSE, MAPE, R2, and correlation. Against this common ground, the two approaches that matter are gradient boosting meta-models and zero-shot Chronos-Bolt transformers; the hybrid Spark-Pandas setup supplies the distributed retraining loop that makes the ML route practical at network scale.","core_discovery":"The paper's central claim is that gradient-boosting ML models, implemented with libraries such as XGBoost and LightGBM, deliver the highest forecasting accuracy on all four restaurants for the 14-day horizon, beating statistical models (Prophet), deep learning (N-Beats), and foundation models. Chronos-Bolt pretrained transformers match that accuracy in three of the four restaurants while seeing only the raw time series, and TimesFM also runs in zero-shot mode. Prophet underperforms, and N-Beats requires heavy feature engineering and computation for less payoff. The paper further claims that a hybrid Spark-Pandas pipeline, which uses Spark for distributed data handling and Pandas inside user-defined functions for training, gives stable horizontal scalability for periodic retraining across a network of thousands of restaurants without GPUs.","pith_inferences":["A direct extension suggested by the results is to feed the same weather, calendar, and time-of-day regressors to all Chronos-Bolt sizes, not only the Base variant, and see whether zero-shot accuracy differences disappear.","Because the benchmark covers four restaurants, the paper's ranking is best read as a demonstration that the methods work on diverse real patterns, not as a guarantee for every store; replicating on dozens of locations would settle how universal the ranking is.","The Spark-Pandas retraining pattern is not restaurant-specific; it should transfer to other multi-site forecasting problems, such as retail chains, logistics depots, or distributed energy demand, where per-location models must be refreshed on a schedule."],"forward_implications":["CPU-only production systems can get accurate 14-day hourly forecasts from gradient boosting without GPU dependence by using the Spark-Pandas retraining pipeline.","GPU-equipped teams can deploy Chronos-Bolt with almost no feature engineering and expect accuracy close to a tuned ML model in most restaurants.","Foundation models capture seasonality and holiday patterns from the series alone, so the heavy feature-building step can be skipped when they are used.","Prophet and N-Beats are the weaker options here: Prophet lags on accuracy and N-Beats needs heavy feature engineering and compute.","Per-restaurant hourly forecasts support staffing and procurement decisions, which is the operational reason the benchmark matters."],"supporting_citations":[{"why":"Supplies TimesFM, the multivariate transformer foundation model benchmarked against the ML models.","marker":"[14]"},{"why":"Supplies the Chronos and Chronos-Bolt pretrained foundation models, the zero-shot models that match ML accuracy in three of four cases.","marker":"[15]"},{"why":"Supplies N-Beats, the deep-learning baseline whose computational cost and feature needs the paper compares against the winners.","marker":"[16]"}],"fun_headline_variants":["Gradient boosting wins restaurant sales forecast over foundation models","Zero-shot foundation models rival ML in restaurant sales forecast","Gradient boosting tops foundation models in real-world sales forecast","ML beats deep learning and foundation models in sales forecasting","Zero-shot Chronos-Bolt ties top ML model in 3 of 4 sales sets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the four sampled restaurants, one per chain and spread across Germany, represent the sales behavior of the thousands of restaurants in the network; if those four locations are atypical, the performance ranking found here may not hold elsewhere.","fun_headline_variants_meta":{"raw":{"variants":["Gradient boosting wins restaurant sales forecast over foundation models","Zero-shot foundation models rival ML in restaurant sales forecast","Gradient boosting tops foundation models in real-world sales forecast","ML beats deep learning and foundation models in sales forecasting","Zero-shot Chronos-Bolt ties top ML model in 3 of 4 sales sets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000633,"raw_usage":{"total_tokens":2864,"prompt_tokens":832,"completion_tokens":2032,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":448,"completion_tokens_details":{"reasoning_tokens":1947}},"tokens_in":448,"tokens_out":2032,"duration_ms":14334,"temperature":1.0,"reasoning_tokens":1947,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T04:51:49.182504+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the same 14-day forecast benchmark on a random sample of fifty or more restaurants from the same network, ideally in a different season, and check whether gradient boosting still beats Chronos-Bolt in most locations; if the ranking reverses on a substantial share of restaurants, the paper's main conclusion would not generalize.","supporting_citations":[],"review_version":1}