{"id":"67e16c9a-faa2-4393-9a59-8ba462a08b8b","arxiv_id":"2502.07213","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The authors use GANs trained on real data to synthesize concept drifts, including the under-studied incremental type, in regression data streams, and evaluate four regression and two prediction-interval methods on the resulting streams.","lead":"This paper proposes a method to generate synthetic data streams for testing regression models that must adapt to changing data over time, and it recommends a standard set of metrics for judging them. It aims to fill a gap: gradual, incremental changes in streaming data are hard to simulate and under-studied in regression.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Simulated 'concept drift' is never verified: incremental streams discard the drift feature, leaving no evidence that P(Y|X) actually changes.","rationale":"The reader's weakest assumption correctly identifies that the paper does not verify genuine concept drift in the generated streams. I partially agree: the specific worry about 'only covariate shift on an unobserved driver' is less likely because shifting an unobserved feature that is correlated with the target will generally change P(Y|X) as well as P(X). The more precise and more load-bearing issue is the complete absence of any statistical validation that the intended drift type is present in the generated data. The prequential plots and the 'synthetic is harder' comparison in Tables 3 and 4 are insufficient evidence. This concern is addressable: a simple drift-detection and retraining comparison on the released code would settle it. Since the reader already assigned CONDITIONAL and this stress-test does not move that verdict, UNCHANGED is appropriate. If the proposed test fails (no drift detected, or only covariate shift), the central contribution would be substantially undercut and the verdict should move toward REJECT.","tokens_in":17160,"tokens_out":7825,"duration_ms":73824,"concrete_test":"Use the released code to generate BIK2i. Train a regression model (e.g., Linear Regression or the paper's SOKNL) on the first 20k instances (stable concept). Compute prequential residuals on the next 80k. Run ADWIN on the residual sequence and record detection points: if detections align with the two 20k drift windows (instances 20k-40k and 60k-80k), drift exists. To distinguish concept drift from covariate shift, compare, on the final stable segment (80k-100k), the R^2 of the model trained on segment 1 vs a model retrained on that same segment: if retraining improves R^2 by more than 0.05, P(Y|X) changed. Also fit a linear model of target on the drifting feature before discarding it and verify the slope is significantly nonzero in generated data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the GAN-based strategy synthesizes genuine concept drift, especially incremental drift. The load-bearing assumption is that splitting real data by a target-correlated feature and training a separate CTGAN per chunk yields streams whose joint feature-target distribution changes over time in the intended way. The paper never tests this. Section 5.2.3 explicitly discards the drifting feature from incremental streams to avoid leakage; consequently the only thing guaranteed to change is the marginal distribution of an unobserved variable. The visible 'turbulence' in Figure 4c is not evidence of concept drift: it could arise from independent GANs producing incompatible noise, from a shift in the target mean, or from covariate shift. Tables 3 and 4 show synthetic streams are harder than the original data, but 'harder' does not establish the presence or type of drift. No baseline comparison to Hyperplane/RBF or any drift detector is reported. If CTGAN training on small chunks fails to propagate the feature-target correlation into generated samples, the simulated streams may contain no concept drift at all, invalidating the main contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses stream regression evaluation, proposing a standardized protocol (cumulative and prequential metrics, including RMSE, adjusted R2, coverage, and NMPIW) and a drift simulation strategy based on CTGAN. Real datasets are sorted by a target-correlated feature, split into chunks, and a separate CTGAN is trained on each chunk; concepts are then concatenated (abrupt), interleaved (gradual), or sorted across a transition (incremental) to produce 18 synthetic streams. Four streaming regressors and two prediction-interval methods are evaluated on these streams. The central claim is that the simulator produces genuine concept drifts, especially incremental drift, and that the proposed evaluation process is a useful standard for the community.","tokens_in":17381,"tokens_out":3839,"duration_ms":35855,"significance":"If the drift simulation is valid, the paper offers a practical and much-needed resource: it augments real regression datasets to stream lengths, generates three drift types including incremental drift, and provides open-source code, which promotes reproducibility in a field that lacks dedicated regression stream benchmarks. The evaluation protocol itself is not conceptually new (it combines existing streaming metrics), but the drift simulator is the main contribution. However, the load-bearing claim that the simulated streams contain concept drift is not substantiated; the paper does not verify that P(Y|X) changes across the synthesized concepts, and it does not compare against existing simulators. If the required verification were added, the paper could serve as a useful benchmarking contribution.","major_comments":[{"comment":"For incremental drift, the drifting feature is discarded from the generated data to avoid information leakage, so the visible stream has no dimension along which the transition is ordered. The statement in §5.2.3, \"This ensures that the simulated concept drift indeed exists,\" is therefore unsupported: the paper provides no measurement of a change in P(Y|X) across the transition; it verifies only a change in the marginal distribution of a feature that is then removed. Without direct evidence (e.g., comparing conditional distributions on held-out instances from adjacent concepts, or applying a drift detector to model residuals), the incremental streams may constitute covariate shift on an unobserved driver rather than concept drift, which is exactly the central claim of the paper.","section":"5.2.3"},{"comment":"The comparison between the original datasets (Table 4) and the synthesized datasets (Table 3) shows that all algorithms perform worse on the latter, but this is not evidence that the difference is caused by concept drift. The comparison confounds drift simulation with CTGAN artifacts and with the removal of the drifting feature. The paper does not include a control stream (e.g., a random-order concatenation of chunks from different concepts without a transition, or a stream where the concept order is shuffled) that would isolate the effect of the drift itself. Additionally, no comparison against existing incremental drift simulators (e.g., Hyperplane or RBF) is reported, so the novelty and effectiveness of the GAN-based approach relative to the state of the art are not demonstrated.","section":"5.2, Tables 3 and 4"},{"comment":"The prequential RMSE plot for the incremental Abalone dataset shows more \"turbulence\" than the abrupt and gradual cases, and the text interprets this as affirming the effectiveness of the incremental simulation. This inference is not valid: turbulence can result from the independent CTGANs generating incompatible random noise, from a shift in the target marginal distribution, or from covariate shift. The paper does not check whether the performance drops coincide with the known drift boundaries (around 20k and 80k instances), nor whether the change corresponds to a change in the conditional distribution of the target given the observed features. Without such analysis, the visual evidence does not establish that the intended drift type was generated.","section":"6.2, Figure 4c"}],"minor_comments":[{"comment":"The title \"Evaluation for Regressive Analyses on Evolving Data Streams\" appears to contain a grammatical error; \"Regressive Analyses\" should likely be \"Regression Analyses\".","section":"Title"},{"comment":"The typeset of Equation (3) is broken: the square root symbol is rendered as \"vt\" in the provided text, obscuring the RMSE formula.","section":"3.2.1, Equation (3)"},{"comment":"The CTGAN hyperparameters (epochs, batch size, learning rate) are stated, but no sensitivity analysis is provided; since CTGAN is the backbone of the simulator, the robustness of the drift simulation to these choices is unknown.","section":"5.1"},{"comment":"The \"Overflowed\" entries for FIMT-DD on several original datasets prevent a complete comparison between the original and synthetic results for those datasets; the paper should either report alternative robust metrics (e.g., median absolute error) or explicitly state the limitation.","section":"Table 4"},{"comment":"The caption \"Simple Showcase of Feature and Target Values with Simulated Incremental Drifts\" is vague; it does not state which feature is plotted, how the target is displayed, or how the figure relates to the simulation procedure described in §5.2.3.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The main technical weakness is the lack of verification that the synthesized streams exhibit concept drift rather than covariate shift or GAN artifacts. This is fixable within the paper's scope: the authors could add experiments with drift detectors on the generated streams, or compare conditional distributions across concept segments, or include control streams with no drift. If such evidence is added in revision, the paper would be a valuable contribution to the regression stream evaluation literature. The paper also contains several self-citations to prior work by the same group; while this is not problematic per se, the novelty of the drift simulation strategy relative to that work should be clarified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper ships something genuinely useful—a GAN-based simulator that creates regression streams with abrupt, gradual, and incremental drift, plus a bundled evaluation protocol and public code. The empirical comparison across 4 algorithms and 18 datasets is real work. What it doesn't do is verify that its simulated 'concept drift' is actually concept drift, especially for the incremental case.\n\nThe core idea: split a real dataset by sorting on the most target-correlated feature, train a separate CTGAN on each chunk, and concatenate/transition between chunks. For abrupt and gradual drift this is reasonable—each chunk is a different slice of the data, so the joint distribution changes at the seams. For incremental drift, though, Section 5.2.3 discards the drifting feature to avoid leakage. That means the learner never sees the feature that defines the concepts. The smooth change the plots show is then a shift in the marginal distribution of an unobserved variable. That may be legitimate drift, but the paper doesn't test whether P(Y|X) actually changes. The turbulence in Figure 4c could come from independent GANs producing incompatible noise, or from a shift in the target mean. 'Harder than the original data' (Tables 3 vs 4) doesn't establish drift type.\n\nThere's also a confound: the drifting feature is removed only for incremental streams, so drift type and feature availability are tangled. And there's no baseline comparison to Hyperplane/RBF simulators, even though those are the cited alternatives. A drift detector or a direct test of the conditional distribution would have settled the question.\n\nWhat the paper does well: it addresses a real gap (regression streams, not just classification), releases code and data, and runs careful prequential evaluations with multiple algorithms and PI methods. The standardized protocol part is a selection of existing metrics—not new, but clearly stated and easy to reuse. The GAN-per-chunk approach has genuine novelty, even if the paper's own figures show the target trend is faint.\n\nBottom line: this deserves a serious referee. It's plausible, not broken, and the missing validation is fixable. A revision with drift detection or a formal check of P(Y|X) change, plus a comparison to existing simulators, would turn a conditional accept into a solid one. If I were the editor, I'd send it out.","headline":"Novel GAN-based drift simulator for streaming regression, useful but under-validated: the incremental drift claims rest on an assumption that is never tested.","tokens_in":17878,"tokens_out":2774,"would_cite":false,"duration_ms":25943,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces a CTGAN-based drift simulator that converts real regression datasets into streams with abrupt, gradual, or incremental concept drift, and a standardized evaluation protocol for them.","keywords":["data streams","streaming regression","concept drift","incremental drift","drift simulation","prediction intervals","synthetic data","evaluation metrics"],"falsifier":"On a generated incremental stream, estimate the conditional distribution of the target given the remaining features on the first stable segment and on the second stable segment; if the two conditional distributions are statistically indistinguishable, the simulated 'concept drift' is only covariate shift.","tokens_in":16958,"feed_emoji":"📊","tokens_out":5916,"duration_ms":49348,"temperature":0.7,"pith_summary":"Streaming regression has lagged behind streaming classification, in part because there are few dedicated benchmarks and no established way to simulate incremental drift. This paper proposes a standardized evaluation protocol for streaming regression and prediction intervals, together with a drift-simulation strategy that augments real datasets using a conditional tabular generative adversarial network to synthesize abrupt, gradual, and incremental concept drift. If the strategy works as claimed, researchers can generate reproducible regression streams with controlled drift, evaluate them with consistent metrics, and compare algorithms and prediction-interval methods on equal footing.","feed_headline":"A recipe to synthesize concept drift in regression streams","feed_subtitle":"Real-data chunks become abrupt, gradual, or incremental drift, evaluated by one shared protocol.","key_machinery":"The central object is the chunk-wise CTGAN: the real dataset is sorted by the most target-correlated numeric feature, split into chunks, and each chunk trains its own conditional tabular generative adversarial network, so each chunk defines a concept. The load-bearing step for incremental drift is sorting the transition segment by the drifting feature, then deleting that feature from the stream; this creates a smooth, hard-to-detect transition without letting the model see the feature that orders it.","core_discovery":"The paper claims that concept drift in regression streams can be synthesized from real data by selecting the numeric feature most correlated with the target, sorting the data by that feature, splitting it into chunks, and training a separate generative model on each chunk so the chunks act as distinct concepts. Abrupt drift is produced by concatenating concept chunks in random order; gradual drift by interleaving the tail of one concept with the head of the next; incremental drift by sorting the transition segment by the drifting feature and then discarding that feature to avoid leakage. The paper further claims that the resulting incremental streams are genuinely harder, since state-of-the-art regressors show lower adjusted R-squared and more turbulent prequential error than on the original data, and that the proposed evaluation protocol reveals how adaptive prediction intervals recover coverage after drift.","pith_inferences":["Editorial extension: because the drifting feature is discarded, the incremental streams may be better described as covariate shift on a hidden driver; the procedure's claim to produce concept drift could be tested by checking whether the conditional target distribution changes across segments.","Editorial extension: the same chunk-and-CTGAN recipe could be reused to simulate recurrent or mixed drift by cycling or randomly permuting concept order, which would make the benchmark more general.","Editorial extension: the per-chunk generative modeling step means the fidelity of the simulated drift depends on how well the GAN captures joint feature dependencies; a failure there would weaken the realism of the drift regardless of the sorting procedure."],"forward_implications":["Researchers can generate regression streams with abrupt, gradual, or incremental drift of specified length from public real datasets, with all code and scripts released.","The proposed evaluation stack—cumulative and prequential versions of RMSE, adjusted R-squared, coverage, and NMPIW—gives a common yardstick for comparing streaming regressors and prediction-interval methods.","On the synthesized streams, all tested algorithms perform worse than on the original data, and incremental-drift streams produce the most unstable prequential errors, indicating the simulator creates genuinely harder problems.","Adaptive prediction intervals (AdaPI) recover target coverage faster after drift than static mean-and-variance intervals, at the cost of wider intervals.","SOKNL and ARF-Reg outperform FIMT-DD and KNN on these streams, suggesting ensemble methods are better suited to drifting regression data."],"supporting_citations":[{"why":"Supplies CTGAN, the generative model used to create each concept's synthetic data.","marker":"[42]"},{"why":"Introduces GANs, the generative framework that CTGAN extends.","marker":"[14]"},{"why":"Provides the synthetic data generation library used to train CTGAN on each chunk.","marker":"[30]"},{"why":"Motivates selecting a real feature as the drifting feature and documents challenges of real-world stream benchmarks.","marker":"[34]"},{"why":"Provides AdaPI, the adaptive prediction-interval method compared in the experiments.","marker":"[38]"},{"why":"Provides SOKNL, one of the regression baselines.","marker":"[37]"},{"why":"Provides FIMT-DD, the model-tree regression baseline.","marker":"[23]"},{"why":"Provides ARF-Reg, the random-forest regression baseline.","marker":"[9]"}],"fun_headline_variants":["Synthesize abrupt, gradual, and incremental drift in regression","Standard protocol for regression drift evaluation and synthesis","Turn real data into three drift types for regression","One protocol to create and test regression drift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The simulation assumes that sorting real data by the most target-correlated feature and training a separate generative model on each chunk produces genuinely different feature–target relationships, but since the drifting feature is later discarded, the observed change may only be a shift in the distribution of an unobserved input.","fun_headline_variants_meta":{"raw":{"variants":["Synthesize abrupt, gradual, and incremental drift in regression","Standard protocol for regression drift evaluation and synthesis","Turn real data into three drift types for regression","One protocol to create and test regression drift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000881,"raw_usage":{"total_tokens":3711,"prompt_tokens":755,"completion_tokens":2956,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":371,"completion_tokens_details":{"reasoning_tokens":2896}},"tokens_in":371,"tokens_out":2956,"duration_ms":16976,"temperature":1.0,"reasoning_tokens":2896,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T13:26:44.113069+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a generated incremental stream, estimate the conditional distribution of the target given the remaining features on the first stable segment and on the second stable segment; if the two conditional distributions are statistically indistinguishable, the simulated 'concept drift' is only covariate shift.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates selecting a real feature as the drifting feature and documents challenges of real-world stream benchmarks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides AdaPI, the adaptive prediction-interval method compared in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides SOKNL, one of the regression baselines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides FIMT-DD, the model-tree regression baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides ARF-Reg, the random-forest regression baseline."}],"review_version":1}