Everything You Need to Know About CS Education: Open Results from a Survey of More Than 18,000 Participants

Aleksandra Makeeva; Aleksei Rostovskii; Anastasiia Birillo; Anna Potriasaeva; Ilya Vlasov; Katsiaryna Dzialets; Yaroslav Golubev

REVIEW 4 major objections 4 minor 5 references

Everything You Need to Know About CS Education: Open Results from a Survey of More Than 18,000 Participants

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that time-series forecasting can be made sampling-rate equivariant by pairing a state-space encoder with a functional basis decoder, so one small model forecasts at any temporal resolution without retraining.

desk verdict The real paper is a strong TSFM contribution (FlowState) that deserves a serious referee, but the metadata abstract is for a different CS-ed survey and the multi-layer equivariance proof doesn't close. read the letter →

arxiv 2508.05286 v1 pith:EBGU7OAQ submitted 2025-08-07 cs.CY cs.HCcs.SE

Katsiaryna Dzialets , Aleksandra Makeeva , Ilya Vlasov , Anna Potriasaeva , Aleksei Rostovskii , Yaroslav Golubev , Anastasiia Birillo This is my paper

classification cs.CYcs.HCcs.SE

keywords timeseriesforecastingfoundationmodelsstatespacesampling-rateequivariancefunctionalbasisdecoderzero-shotgeneralizationcontinuous-timeGIFT-Eval

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FlowState is a time-series foundation model built so that changing the input sampling rate changes only the output grid, not the underlying continuous forecast. The paper's central claim is that this sampling-rate equivariance can be made an architectural property: an SSM encoder produces nearly identical hidden states for the same signal sampled at different rates, and a novel functional basis decoder reinterprets those states as coefficients of continuous basis functions, yielding a forecast that can be sampled anywhere. The paper proves a first-order bound on the equivariance error, reports best results on a 97-task benchmark with 10.6M-18.6M parameter models, and shows the model adapts to sampling rates it never saw in pretraining. If correct, this means time-series foundation models do not need to see every temporal resolution during training to forecast at any resolution.

What carries the argument

Functional basis decoder (FBD), paired with the SSM encoder's scale-adjusted discretization. The FBD interprets the encoder's final output as coefficients of a continuous basis (Legendre polynomials by default), builds a continuous forecast function, and samples it at spacing $\Delta_F$. Scale factors $s_{\Delta_E}$ and $s_{\Delta_F}$ multiply the learned $\Delta$ parameters so the same model can re-discretize the encoder and re-sample the decoder for a new temporal resolution; the ZOH mismatch bound (Proposition B.3) is the proof engine showing the representation stays approximately invariant across sampling rates.

What would settle it

Subsample the same physical signal at many rates, inject a brief high-frequency spike into the slower samples, and measure the forecast difference on a fixed output grid. The ZOH argument predicts this difference should grow at most linearly in $\max\{\Delta,\Delta'\}$; a plateau, jump, or superlinear growth would falsify the equivariance bound. Rerunning the paper's ETT1 experiment with such added high-frequency content would stress the Lipschitz assumption directly.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that sampling-rate equivariance is a designable property, not an emergent accident. FlowState couples an S5-style state-space encoder with a functional basis decoder (FBD). The encoder's continuous-time recurrence is discretized by zero-order hold, so two discretizations of the same signal at step sizes $\Delta$ and $\Delta'$ differ in final hidden state by at most $\frac{\|B\|M}{\alpha}(1-e^{-\alpha T})\max\{\Delta,\Delta'\}$ under Lipschitz input and exponential stability assumptions. The decoder treats the encoder output as coefficients of Legendre polynomials and produces a continuous forecast, so sampling at any $\Delta_F$ merely evaluates the

Load-bearing premise

The whole design rests on real time series being close to smoothly varying continuous signals sampled at known, regular intervals, and on the seasonality-based scale factor chosen for each dataset being close to correct; if either fails, sampling-rate equivariance and the reported accuracy degrade.

Editorial extensions

If this is right

One pretrained checkpoint can forecast datasets sampled at intervals absent from pretraining, as the ETT1 resampling experiment shows; other TSFMs need exposure to each frequency.
Forecasting horizon becomes an inference-time choice: the FBD samples the same continuous function on any grid, so target length can be changed without retraining.
Parameter efficiency follows: the 18.6M-parameter model beats a 200M-parameter baseline on the benchmark, suggesting equivariance substitutes for model scale.
The strictly causal architecture plus causal RevIN makes the parallel-prediction pretraining scheme leakage-free, producing roughly 4,000 forecasts per context-target pair and speeding training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the equivariance story is right, the FBD should transfer to other causal encoders as a drop-in decoder; the paper notes it is a separate component, so a direct test is to attach it to a non-SSM encoder and measure cross-rate generalization.
The load-bearing practical requirement is knowing the seasonality in advance: the paper's own automatic scale selection on GIFT-Eval degrades MASE from 0.725 to 0.746 (Appendix D.2.2). A natural extension is learning the scale factor end-to-end from the context instead of setting it by dataset heuristics.
The formal guarantee is proven for a single SSM layer; the multi-layer recursion (Appendix B.1) contains unverified constants. I would expect the first-order dependence on $\max\{\Delta,\Delta'\}$ to survive, but with a prefactor that grows with depth, which future work could verify layer by layer.
The Fourier-basis variant underperforms Legendre by a small margin on the benchmark average, yet Fourier is a better match for strictly periodic signals; a testable prediction is that Fourier will win on datasets with strong isolated periodic components.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague — first, the submission metadata is for a CS-education survey of 18,000 people, but the actual text is the FlowState time-series forecasting paper (arXiv 2508.05287). That mismatch needs fixing, but reviewing the substance, the FlowState paper is the real deal.

What is new: the functional basis decoder (FBD) that reads the final SSM hidden state as coefficients of continuous basis functions and produces a forecast that can be sampled at any rate is genuinely new, and coupling it with scale-factor multiplication of Delta in both encoder and decoder to get sampling-rate-equivariant forecasting is a clean idea. The parallel-prediction training scheme (up to 4076 forecasts per sample) is also a nice contribution for any causal architecture. The paper ships real ablations, including three-seed variance on the baseline, and the Appendix B bound is a legitimate ZOH mismatch derivation with stated Lipschitz and stability assumptions. The empirical robustness to unseen sampling rates (Figure 4) is convincing.

Soft spots, in order. First, the theory doesn't cover the evaluated architecture. Proposition B.3 and Eq. (46) are proven for a single SSM layer; the multi-layer recursion (Eq. 53) relies on per-layer Lipschitz constants L_l that are never bounded. If any L_l > 1, the mismatch can grow exponentially in depth and the 'first-order in max{Δ,Δ'}' claim becomes vacuous for the actual 6-layer encoder. Causal RevIN, with its running statistics, is Lipschitz only if you assume non-vanishing variance; that's not analyzed. This is the stress-test and it lands. Second, the headline SOTA margins are tiny — MASE 0.701 vs 0.705, CRPS 0.487 vs 0.490 — and the main table has no error bars or significance tests. With a difference that small, I'd want to know seed variance on the main runs. Third, the scale factor sDelta comes from a domain heuristic (Algorithm 1); the automatic version degrades from 0.725 to 0.746. So the 'unseen sampling rate' story is partly dependent on knowing the seasonality up front.

None of this is fatal. The architecture, ablations, and empirical invariance are strong enough that the central contribution — a small, sampling-rate-flexible TSFM with a continuous decoder — holds up. The theory just needs either a closed multi-layer proof or a more modest claim.

Verdict: send to peer review. The theoretical gap is addressable in revision, and the FBD idea is worth the field's attention. If I were the editor, I'd ask for error bars on Table 1 and a corrected or softened multi-layer statement.

Referee Report

4 major / 4 minor

Summary. The manuscript as submitted is not internally coherent: the title and abstract describe a large-scale computer science education survey (18,032 learners, 173 countries) and promise an open dataset, while the full text is a machine-learning paper titled 'FlowState: Sampling-Rate-Equivariant Time-Series Forecasting.' The body proposes FlowState, an SSM encoder paired with a functional basis decoder, claims approximate sampling-rate equivariance (Section 4, Appendix B), reports state-of-the-art results on the GIFT-Eval benchmark (Table 1), and presents ablation studies (Table 2). Because the claimed survey content is entirely absent, I could not evaluate the stated CS-education contribution; the remainder of this report assesses the actual body on its own merits.

Significance. Considered on its own, the FlowState body contains several genuinely useful ingredients. The functional basis decoder, the causal RevIN construction, and the parallel-prediction training scheme are clearly described and are supported by targeted ablations (Table 2). The single-layer zero-order-hold mismatch bound in Proposition B.3 is a real derivation with explicit assumptions, and the model weights are released. However, the headline SOTA margins on GIFT-Eval are very small and are reported without uncertainty estimates; the formal equivariance proof is incomplete for the multi-layer encoder actually evaluated; and the method depends on externally supplied seasonality information through the scale factor s_Delta. More importantly, the submitted artifact does not contain the CS-education survey promised by the title and abstract. That mismatch is a blocking issue that is independent of the technical merits of the body.

major comments (4)

[Title, Abstract, and Full Text] The submission's title and abstract claim a CS-education survey of 18,032 participants from 173 countries and promise an open dataset, but the full text is a time-series forecasting paper. No survey instrument, methodology, dataset link, or survey results appear anywhere in the body. This is not a presentation issue; the claimed contribution cannot be assessed, and the abstract's central claims are unsupported by the manuscript. The manuscript must be corrected to match its actual content, or resubmitted as the appropriate paper.
[Appendix B.1, Eq. (53)] The formal support for the headline equivariance property is proved only for a single SSM layer. The multi-layer extension is a recursion E_l <= L_l E_{l-1} + C_l max{Delta, Delta'}, E_0 = 0, where L_l and C_l are 'layer-dependent constants independent of Delta and Delta'. No bounds are given on L_l; if any L_l > 1, the conclusion E_K = O(max{Delta, Delta'}) is vacuous because the implied constant absorbs an exponential-in-depth factor. The six-layer encoder includes MLPs, output gates, and causal RevIN, whose Lipschitz regularity with respect to previous-layer outputs is not analyzed. Thus Proposition B.3 does not establish the equivariance property for the architecture actually evaluated.
[Section 6.2, Table 1 / Appendix D.1] The SOTA claim is based on differences of 0.701 vs 0.704 vs 0.705 MASE and 0.487 vs 0.490 CRPS, with no error bars or significance tests on the main variants. Appendix D.1 states that experiments were executed once ('we did not observe a major performance variation'). The reported seed variation for FlowState-3M(2k), +-0.0007 MASE, is much smaller than the headline gaps, but it does not cover the main models. At a minimum, the authors should report multiple seeds for the main variants or soften the SOTA claim.
[Section 6.1.2 / D.2.2] The method's claimed ability to generalize across sampling rates relies on choosing s_Delta from external seasonality knowledge (Algorithm 1). The automatic-selection experiment degrades performance from MASE 0.725 to 0.746 and CRPS 0.502 to 0.521, and Figure 9 shows that FlowState is sensitive to misaligned scale factors on ETTm1. This is acknowledged in the Limitations section, but it is a real condition on the central claim of 'inherently' generalizing across temporal resolutions. The paper should present this dependence as a stated condition rather than as an incidental implementation detail.

minor comments (4)

[Section 3.1, Eq. (3)] The layer indexing is inconsistent: x^{l-1}_t is used for the SSM input, while h^l_t is the SSM output; the notation should be unified (e.g., x^l_t for layer inputs) to avoid confusion.
[Algorithm 1 / Table 7] The 'yearly' case in Algorithm 1 returns 4, but this is not connected to the base seasonality B=24 used in Section 6.1.2 and Table 7. Please clarify the relationship.
[Appendix D.2.2] The automatic seasonality selection is not fully reproducible: the candidate range [s_min, s_max], the prominence threshold, and the peak-finder settings are not specified. Please add these parameters.
[Appendix B.2, Figure 8] The caption says Panel b is 'flipped along the time axis'; this is clearer as 'temporally reversed' or 'time-reversed' so that the misalignment is unambiguous.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the sampling-rate equivariance claim is a mathematical consequence of ZOH discretization under explicit Lipschitz/stability assumptions, not a fitted quantity or self-citation.

full rationale

FlowState's central claim is that the SSM encoder yields approximately sampling-rate-invariant latent states and that the functional basis decoder maps those latents to a rate-agnostic continuous forecast. The formal derivation in Appendix B (Assumptions B.1/B.2, Proposition B.3, Eq. (46)) is a genuine theorem: the two-rate ZOH mismatch is bounded by the input Lipschitz constant, the exponential stability constant, and max{Δ, Δ'}; no term in that bound is fitted to forecast targets. The decoder is rate-agnostic by construction, with the output grid spacing chosen only at evaluation time. The per-dataset scale factors sΔE=sΔF are set from seasonality/domain metadata via Algorithm 1, or estimated from the context in the automatic experiment, and the paper honestly reports that automatic selection degrades MASE from 0.725 to 0.746. This is not a hidden target fit. The GIFT-Eval results are zero-shot comparisons against external baselines and are supported by component ablations. The multi-layer extension (Eq. (53)) relies on layer-dependent Lipschitz constants that are not explicitly bounded, and causal RevIN is not separately analyzed; however, this is an incomplete formal argument and a correctness risk, not a circular reduction. The paper does not assume its target result to prove it, and no load-bearing self-citation chain was found.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The core derivation of sampling-rate equivariance rests on standard ZOH discretization theory plus smoothness and stability assumptions. The only deliberately inserted free parameter that affects the benchmark results is the per-dataset scale factor sDelta, which is a domain-knowledge heuristic rather than a fitted target. No new physical or theoretical entities are introduced; the FBD is a new neural component with an implementation and ablations.

free parameters (3)

sDelta (per-dataset scale factor) = e.g., 0.25 for ETTm1; set by B / seasonality via Algorithm 1
The headline GIFT-Eval results and the equivariance demonstrations rely on an external prior over each dataset's seasonality and domain. The ablation 'w/o equivariance' (MASE 0.799 vs 0.725) and the automatic-selection degradation (0.746 vs 0.725) show that the central results depend on this choice.
Base seasonality B = 24
A hand-set hyperparameter used to define sDelta = B / seasonality for all experiments (Section 6.1.2 and Table 7).
Time noise tau = 0.1
A hand-set hyperparameter that perturbs the decoder sampling locations during training; the ablation without time noise degrades MASE from 0.725 to 0.740.

assumptions (6)

standard math Zero-order-hold discretization of the continuous SSM is exact for piecewise-constant inputs
Invoked in Section 3.1 and Appendix B (Equation 19 and surrounding text) to define the discrete recurrence and to bound the equivariance error.
domain assumption The underlying input signal is Lipschitz continuous (Assumption B.1)
Used in Appendix B to bound the mismatch between two ZOH reconstructions by M * max{Delta, Delta'}. Real discrete time series may not inherit this property from any continuous process.
domain assumption The SSM state matrix is exponentially stable via the negative-real-part parameterization (Assumption B.2)
Assumed in Appendix B so that the state-transition integral is bounded by e^{-alpha t}; the paper parameterizes real parts as -exp(theta_i) to enforce it, which is standard but still an assumption about the learned dynamics.
domain assumption Pointwise encoder components (output gate, MLP, residual, causal RevIN) are Lipschitz on the bounded set of activations
Assumed implicitly in Equation (33) of Appendix B to propagate the SSM mismatch through the rest of the layer.
standard math The HiPPO-initialized hidden state of an SSM provides coefficients of a polynomial basis suitable for continuous reconstruction (Gu et al., 2020)
Borrowed in Section 3.2 to justify the FBD design; the decoder then reinterprets these coefficients for forecasting rather than reconstruction.
ad hoc to paper Seasonality and domain determine the appropriate scale factor via Algorithm 1
Introduced in Section 6.1.2 and Appendix D.2.1. This heuristic maps sampling interval and domain to a seasonality and hence to sDelta, and is the main externally provided input to the evaluation procedure.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Everything You Need to Know About CS Education: Open Results from a Survey of More Than 18,000 Participants." pith.science (2026). https://pith.science/paper/EBGU7OAQ

@misc{pith2026250805286,
  author       = {Pith},
  title        = {Pith review of: Everything You Need to Know About CS Education: Open Results from a Survey of More Than 18,000 Participants},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EBGU7OAQ}},
  note         = {Machine review of arXiv:2508.05286}
}

read the original abstract

Computer science education is a dynamic field with many aspects that influence the learner's path. While these aspects are usually studied in depth separately, it is also important to carry out broader large-scale studies that touch on many topics, because they allow us to put different results into each other's perspective. Past large-scale surveys have provided valuable insights, however, the emergence of new trends (e.g., AI), new learning formats (e.g., in-IDE learning), and the increasing learner diversity highlight the need for an updated comprehensive study. To address this, we conducted a survey with 18,032 learners from 173 countries, ensuring diverse representation and exploring a wide range of topics - formal education, learning formats, AI usage, challenges, motivation, and more. This paper introduces the results of this survey as an open dataset, describes our methodology and the survey questions, and highlights, as a motivating example, three possible research directions within this data: challenges in learning, emerging formats, and insights into the in-IDE format. The dataset aims to support further research and foster advancements in computer education.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

5 extracted references · 1 canonical work pages

[4]

Wang, Y ., Qiu, Y ., Chen, P., Shu, Y ., Rao, Z., Pan, L., Yang, B., and Guo, C

URL https://openreview.net/forum? id=Ai8Hw3AXqks. Wang, Y ., Qiu, Y ., Chen, P., Shu, Y ., Rao, Z., Pan, L., Yang, B., and Guo, C. LightGTS: A lightweight general time series forecasting model. In Singh, A., Fazel, M., Hsu, D., Lacoste-Julien, S., Berkenkamp, 11 FlowState: Sampling-Rate-Equivariant Time-Series Forecasting F., Maharaj, T., Wagstaff, K., an...

arXiv 2024
[5]

12 FlowState: Sampling-Rate-Equivariant Time-Series Forecasting A

doi: 10.48550/arXiv.2205.13504. 12 FlowState: Sampling-Rate-Equivariant Time-Series Forecasting A. Additional Results A.1. Performance on Standard Time Series Benchmarks We additionally evaluate FlowState on standard long-term forecasting benchmarks, including ETT (ETT1m, ETT2m, ETT1h, ETT2h), Traffic, Weather, Exchange, and Electricity. We compare agains...

arXiv doi:10.48550/arxiv.2205.13504 2024
[2022]

Liang, Y ., Wen, H., Nie, Y ., Jiang, Y ., Jin, M., Song, D., Pan, S., and Wen, Q

URL https://openreview.net/forum? id=cGDAkQo1C0p. Liang, Y ., Wen, H., Nie, Y ., Jiang, Y ., Jin, M., Song, D., Pan, S., and Wen, Q. Foundation models for time se- ries analysis: A tutorial and survey. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’24, pp. 6555–6565, New York, NY , USA, 2024. Association for C...

arXiv 2024
[2023]

URL https://openreview.net/forum? id=klK17OQ3KB. Hadi, M. U., Qureshi, R., Shah, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., Mirjalili, S., et al. A survey on large language models: Applications, challenges, limitations, and practical usage.Authorea Preprints, 2023. Kim, T., Kim, J., Tae, Y ., Park, C., Choi, J.-H., and Choo, J. Reversibl...

work page 2023
[2025]

Nie, Y ., Nguyen, N

URL https://proceedings.mlr.press/ v267/liu25be.html. Nie, Y ., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers. InThe Eleventh International Conference on Learning Representations, 2023. URL https:// openreview.net/forum?id=Jbdc0vTOcol. Rahman, M. M., Tutul, A. A., Nath, A., Laishra...

arXiv doi:10.48550/arxiv.2410.03105 2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [4]

Wang, Y ., Qiu, Y ., Chen, P., Shu, Y ., Rao, Z., Pan, L., Yang, B., and Guo, C

URL https://openreview.net/forum? id=Ai8Hw3AXqks. Wang, Y ., Qiu, Y ., Chen, P., Shu, Y ., Rao, Z., Pan, L., Yang, B., and Guo, C. LightGTS: A lightweight general time series forecasting model. In Singh, A., Fazel, M., Hsu, D., Lacoste-Julien, S., Berkenkamp, 11 FlowState: Sampling-Rate-Equivariant Time-Series Forecasting F., Maharaj, T., Wagstaff, K., an...

arXiv 2024

[2] [5]

12 FlowState: Sampling-Rate-Equivariant Time-Series Forecasting A

doi: 10.48550/arXiv.2205.13504. 12 FlowState: Sampling-Rate-Equivariant Time-Series Forecasting A. Additional Results A.1. Performance on Standard Time Series Benchmarks We additionally evaluate FlowState on standard long-term forecasting benchmarks, including ETT (ETT1m, ETT2m, ETT1h, ETT2h), Traffic, Weather, Exchange, and Electricity. We compare agains...

arXiv doi:10.48550/arxiv.2205.13504 2024

[3] [2022]

Liang, Y ., Wen, H., Nie, Y ., Jiang, Y ., Jin, M., Song, D., Pan, S., and Wen, Q

URL https://openreview.net/forum? id=cGDAkQo1C0p. Liang, Y ., Wen, H., Nie, Y ., Jiang, Y ., Jin, M., Song, D., Pan, S., and Wen, Q. Foundation models for time se- ries analysis: A tutorial and survey. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’24, pp. 6555–6565, New York, NY , USA, 2024. Association for C...

arXiv 2024

[4] [2023]

URL https://openreview.net/forum? id=klK17OQ3KB. Hadi, M. U., Qureshi, R., Shah, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., Mirjalili, S., et al. A survey on large language models: Applications, challenges, limitations, and practical usage.Authorea Preprints, 2023. Kim, T., Kim, J., Tae, Y ., Park, C., Choi, J.-H., and Choo, J. Reversibl...

work page 2023

[5] [2025]

Nie, Y ., Nguyen, N

URL https://proceedings.mlr.press/ v267/liu25be.html. Nie, Y ., Nguyen, N. H., Sinthong, P., and Kalagnanam, J. A time series is worth 64 words: Long-term forecasting with transformers. InThe Eleventh International Conference on Learning Representations, 2023. URL https:// openreview.net/forum?id=Jbdc0vTOcol. Rahman, M. M., Tutul, A. A., Nath, A., Laishra...

arXiv doi:10.48550/arxiv.2410.03105 2023