REVIEW 5 major objections 5 minor 8 references
Triple Attention Transformer Architecture for Time-Dependent Concrete Creep Prediction
T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that a triple-attention transformer, which treats concrete creep as an autoregressive sequence like a language model, predicts creep strain with a mean absolute percentage error of 1.63% and an R² of 0.999, outperforming…
desk verdict The headline performance is an artifact of training and testing on synthetic daily values generated from per-specimen curve fits, not raw measurements. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the Triple Attention Transformer Architecture (TATA). It embeds creep and log-transformed time, adds sinusoidal positional encodings, applies a masked multi-head self-attention encoder across the full 160-step sequence, and pools the encoded context with a hybrid of mean, attention, and last-token strategies. In parallel, material features go through three paths: a direct feed-forward transform, feature-wise attention over individually projected properties, and batch attention over other specimens' feature vectors, whose outputs are concatenated and fused with the sequence context before a two-layer predictor emits the next creep value. The autoregressive loop, where each forecast is fed back as the newest token, is what makes partial histories enough to predict the full 160-day curve.
What would settle it
Re-run the evaluation on the original irregularly sampled creep readings, splitting by entire specimens, training on raw values without Eq. (28) interpolation, and measuring against the actual measured strains. If the MAPE rises substantially above 1.63% or regression against raw readings no longer reaches $R^2 \approx 0.999$, the claimed near-perfect predictive accuracy is an artifact of curve-generated targets.
Extended reading notes
Core claim
The paper's central discovery is that representing each daily creep measurement as a token, and letting self-attention condition on the whole history rather than a truncated window, lets a transformer reproduce concrete creep curves almost exactly. The architecture runs three attention streams in parallel: temporal attention over the creep sequence, feature attention over the three material properties (compressive strength, elastic modulus, density), and batch attention over other specimens in the mini-batch, then combines a hybrid of mean, attention-weighted, and last-token pooling before the prediction head. On the dataset of 65 specimens standardized to 160 daily points, the model reports a MAPE of 1.63% and an R² of 0.999 across all partitions, and the ablation study attributes the largest error increase (to 3.58% MAPE) to removal of attention pooling. The paper takes this as evidence that the autoregressive, full-history paradigm is the decisive improvement over previous approaches that used time as an input or limited context to five steps.
Load-bearing premise
The evaluation targets are not raw measurements but daily values produced by fitting each specimen's data to the curve $Creep = a(1 - e^{-b t^c})$ (Eq. 28), so everything rests on that fitted curve representing the true experimental behavior; the 90/5/5 split further means the same specimens appear in training and test sets.
Editorial extensions
If this is right
- Full-history attention removes the fixed 5-step window limitation, so longer creep histories can be exploited directly rather than truncated.
- Only three routinely measured lab parameters—compressive strength, elastic modulus, and density—are needed to achieve the reported accuracy on the studied data range.
- Attention pooling is the component whose removal hurts most, with MAPE rising from 1.63% to 3.58%, so future designs should keep or strengthen it.
- The SHAP ranking of Young's modulus above density and compressive strength offers a pragmatic ordering for feature collection and mix-design screening.
- The autoregressive transformer recipe transfers to other time-dependent material behaviors, which the paper states as the next step.
Reading between the lines
- Because the targets are generated by a smooth three-parameter fitted curve, the near-perfect scores could partly reflect how well the model learns that particular functional family; an independent check would be training on raw, non-uniform measurements to see whether the accuracy survives.
- The batch-attention mechanism suggests that cross-specimen context carries predictive signal beyond within-specimen history, so pooling larger multi-source databases could amplify the reported gains.
- The model is demonstrated only up to 160 days, so extrapolating to multi-year creep remains untested; applying the same architecture to long-term databases would establish whether the full-history advantage persists.
- The token-sequence framing could be applied to shrinkage, thermal deformation, or relaxation, since the paper demonstrates creep but claims the framework is general.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes the Triple Attention Transformer Architecture (TATA), which treats concrete creep prediction as an autoregressive next-token task over 160 daily steps. The architecture combines temporal, feature, and batch attention streams and reports R² = 0.999 with MAPE = 1.63% on a database of 65 creep specimens. The manuscript also includes an ablation study, SHAP interpretability analysis, Bayesian hyperparameter optimization, and a deployed web application. The central problem is that the daily ground-truth values used for both training and evaluation are generated by fitting each specimen to the smooth three-parameter curve in Eq. (28), so the reported accuracy reflects the model's ability to reproduce a fitted curve family rather than to predict experimental creep measurements.
Significance. If the central claim were supported, the paper would be a meaningful contribution: it would be one of the first demonstrations of a transformer-based autoregressive approach to concrete creep, with a clearly documented architecture, a quantitative ablation study, and a practical deployment interface. However, the paper's load-bearing evidence is invalid because the targets are synthetic per-specimen curve fits, not raw measurements, and because no baseline models are compared on the same evaluation protocol. The contribution is therefore architectural rather than empirically established; the claimed superiority over traditional and machine-learning creep models is not demonstrated by the experiments reported here.
major comments (5)
- [Section 3, Eq. (28)] The daily ground-truth values used for both the training loss in Eq. (33) and the reported evaluation metrics are generated by fitting each specimen to the smooth three-parameter curve Creep = a·(1 − exp(−b·t^c)). The reported R² = 0.999 and MAPE = 1.63% therefore measure how accurately the model reproduces this fitted curve family, not how accurately it predicts experimental creep measurements. The paper never reports the model's error against the original irregularly sampled measurements, nor does it report the fit residuals of Eq. (28) itself, so the central predictive claim is unsubstantiated.
- [Section 4.1] The 90/5/5 split is performed at the time-step level: 9504 training, 528 validation, and 528 test samples generated from 160 daily points per specimen mean that the same specimen's generated daily points appear in all three splits. Because all daily points for a specimen lie on the same fitted curve from Eq. (28), the test samples are not independent of the training samples; the model has effectively seen the same per-specimen curve during training. A per-specimen split, such as leave-specimen-out, is required to support any claim of generalization to unseen specimens.
- [Sections 4 and 5] No baseline comparisons are reported. The abstract and conclusion claim substantial outperformance over traditional empirical models and existing machine learning approaches, but the experiments in Sections 4 and 5 contain only the proposed model and its ablations in Table 3. Without comparisons on the same data split to, for example, the Eq. (28) curve fit itself, ANN/RF/XGBoost, or GRU/LSTM architectures, the claimed superiority is unsupported.
- [Fig. 9 and Section 4.2] The caption of Fig. 9 states that predictions commence from zero initial strain and that each subsequent prediction is incorporated as input for forecasting the next increment, which describes an autoregressive rollout rather than a next-step prediction conditioned on observed history. The manuscript does not specify whether the reported test metrics are computed under this rollout protocol or under teacher forcing; the distinction matters because error accumulation is the relevant real-world task and should be reported explicitly.
- [Sections 3 and 4.1] The dataset size is internally inconsistent: the text states 65 creep specimens, but the database is described as 42 plus 25 specimens, i.e., 67; and 65 specimens times 160 daily points equals 10,400 samples, whereas Section 4.1 reports 9504 + 528 + 528 = 10,560 samples. This inconsistency needs clarification because the sample count is the basis of the data-splitting description and of the claimed data leakage.
minor comments (5)
- [Section 4.1, Eqs. (31)-(32)] Equation (32) is used twice: once for logarithmic time normalization and once for z-score feature normalization; the second equation should be renumbered.
- [Section 4.2 and Fig. 7] The text says the optimization used a maximum of 40 epochs per trial with a median of about 22 epochs, but the caption of Fig. 7 refers to a 200-epoch training period; these statements should be reconciled.
- [Section 2 and Section 3] The section numbering is off: the literature review ends with a subsection titled 'Research Gaps' inside Section 2, and the model architecture section is also labeled Section 2; the sections should be renumbered.
- [Section 4.2] The phrase 'transformer-based concrete strength prediction model' appears in the hyperparameter discussion, though the model predicts creep, not strength; this appears to be a copy-paste error and should be corrected.
- [Section 6] The SHAP analysis text contains literal '[reference]' placeholders and a repeated paragraph describing density and compressive strength; the citations should be completed and the duplication removed.
Circularity Check
Near-perfect creep prediction (MAPE 1.63%, R²=0.999) is evaluated against synthetic daily values generated by per-specimen fits of Eq. (28), so the model is scored on reproducing smoothed curves that were themselves fitted to the measurements, not on predicting independent experimental observations.
-
fitted input called prediction
[Section 3, Eq. (28); Section 4, Eq. (33); Figs. 8-9]
"The standardization procedure employed a modified logarithmic model: Creep = a ⋅ (1 − exp(−b ⋅ t^c)) (28)... These calibrated parameters were subsequently utilized to generate uniformly spaced daily creep predictions extending to 160 days. The curve fitting procedure yielded an R² value of 0.9950. The interpolation was configured to produce time steps of 1 day for each sample."
The paper fits Eq. (28) to each specimen's irregular experimental measurements, then uses the fitted curve to generate daily values that become both the training targets in Eq. (33) and the 'actual' values in the reported evaluation (Figs. 8-9). The headline MAPE of 1.63% and R² of 0.999 therefore measure agreement with the authors' own smoothed parametric curves, not with independent experimental creep observations.
-
other
[Section 4.1, data split description]
"For rigorous evaluation, we partition the data into training, validation, and test sets with proportions set at 90%, 5%, and 5%, respectively, resulting in 9504 training samples, 528 validation samples, and 528 test samples."
Because each specimen is represented as a 160-day token sequence and the reported sample counts far exceed the roughly 65 specimens, the partition is per time-step rather than per whole specimen curve. This places points from the same generated daily curve into training, validation, and test simultaneously, so the model has already seen other points of the same fitted curve during training. The test evaluation therefore does not measure generalization to unseen specimens; it measures interpolation within curves whose other points were part of the training set, compounding the synthetic-target problem and forcing the reported near-perfect test statistics.
full rationale
The central claim of the paper is the reported near-perfect accuracy of the Triple Attention Transformer for concrete creep prediction. That claim is not supported by independent experimental data. Section 3 transforms each specimen's sparse, irregular creep measurements into a daily series by fitting Eq. (28), Creep = a*(1-exp(-b*t^c)), and sampling the fitted curve at one-day intervals out to 160 days. Section 4 then uses these generated values as the targets in the training loss (Eq. 33) and as the 'actual' values in the evaluation figures (Figs. 8-9). Consequently, the reported MAPE of 1.63% and R² of 0.999 measure agreement with the authors' own fitted curves, not with raw creep measurements; the curve fit's own R² of 0.9950 caps the information available from the data. The per-time-step split further compounds this by spreading the same specimen's generated points across train, validation, and test, so the test set is not independent. The self-citation to Kongkitkul et al. (2025) is used only as architectural inspiration for batch attention and is not load-bearing for the performance claim. No external benchmark against raw measurements or independent experimental tests is provided. The headline performance is therefore largely forced by the data-generation procedure: fitted inputs are renamed as 'actual' predictions, and the model's near-perfect scores are an artifact of training and evaluating on synthetic smooth curves, not a demonstration of predictive capability for real concrete creep.
Assumptions & free parameters
free parameters (3)
- a (per-specimen curve amplitude) =
not reported
- b (per-specimen rate parameter) =
not reported
- c (per-specimen shape parameter) =
not reported
assumptions (4)
- domain assumption Concrete creep strain follows the functional form Creep = a*(1-exp(-b*t^c)) (Eq. 28)
- domain assumption Daily interpolated/extrapolated values from the fitted curve are equivalent to experimental measurements for training and evaluation
- ad hoc to paper The 90/5/5 split at the time-step level does not leak information between train and test
- domain assumption The three material features (elastic modulus, density, compressive strength) are sufficient predictors of creep
Cite this review
Pith. "Pith review of Triple Attention Transformer Architecture for Time-Dependent Concrete Creep Prediction." pith.science (2026). https://pith.science/paper/OX6NIZLY
@misc{pith2026250604243,
author = {Pith},
title = {Pith review of: Triple Attention Transformer Architecture for Time-Dependent Concrete Creep Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/OX6NIZLY}},
note = {Machine review of arXiv:2506.04243}
}
read the original abstract
This paper presents a novel Triple Attention Transformer Architecture for predicting time-dependent concrete creep, addressing fundamental limitations in current approaches that treat time as merely an input parameter rather than modeling the sequential nature of deformation development. By transforming concrete creep prediction into an autoregressive sequence modeling task similar to language processing, our architecture leverages the transformer's self-attention mechanisms to capture long-range dependencies in historical creep patterns. The model implements a triple-stream attention framework incorporating temporal attention for sequential progression, feature attention for material property interactions, and batch attention for inter-sample relationships. Evaluated on experimental datasets with standardized daily measurements spanning 160 days, the architecture achieves exceptional performance with mean absolute percentage error of 1.63% and R2 values of 0.999 across all datasets, substantially outperforming traditional empirical models and existing machine learning approaches. Ablation studies confirm the critical role of attention mechanisms, with attention pooling contributing most significantly to model performance. SHAP analysis reveals Young's modulus as the primary predictive feature, followed by density and compressive strength, providing interpretability essential for engineering applications. A deployed web-based interface facilitates practical implementation, enabling real-time predictions using standard laboratory parameters. This work establishes the viability of applying transformer architectures to materials science problems, demonstrating the potential for data-driven approaches to revolutionize structural behavior prediction and engineering design practices.
Reference graph
Works this paper leans on
-
[2]
Model Architecture The proposed concrete creep prediction framework employs a multi -pathway transformer architecture that processes temporal measurement sequences and specimen -specific features through parallel computational streams, as illustrated in Fig. 1. The temporal processing pathway transforms historical creep measurements and corresponding time...
work page 2025
-
[3]
Data Characteristics The concrete compressive creep database was obtained from the Concrete and Materials Testing Laboratory at King Mongkut's University of Technology Thonburi, Thailand. This database comprises 42 creep test curves from tests conducted in accordance with ASTM C512/C512M (ASTM International, 2015), along with an additional 25 data sourced...
work page 2015
-
[4]
Experiments 4.1 Data Preparation The model employs an LLM-style framework to process concrete creep measurements with a fixed structure of 160 data points, representing 160 days of observations for each specimen. Our methodology organizes each concrete specimen's data as a structured collection of historical creep measurements, corresponding time points, ...
-
[5]
Ablation Study The ablation study presented in Table 3 systematically evaluates the contribution of individual architectural components within the proposed Triple Attention model for concrete creep prediction. The experimental protocol involved sequential removal of specific model elements to quantify their impact on predictive performance as measured by ...
-
[6]
Model Explainable Lundberg and Lee (Lundberg and Lee, 2017) introduced SHAP (SHapley Additive exPlanation) values as a theoretically sound methodology for interpreting outputs from machine learning models. Based on principles from cooperative game theory, these values allocate the prediction f(x) across the input features through the following additive re...
work page 2017
-
[8]
Discussion The proposed Triple Attention Architecture demonstrates significant improvements in predicting concrete creep behavior compared to traditional approaches, achieving a Mean Absolute Percentage Error (MAPE) of 1.2% for each loading step, which substantially outperforms standard empirical models and conventional machine learning methods. This adva...
work page 2024
-
[9]
Conclusion This research successfully demonstrates the application of a Triple Attention Transformer Architecture for concrete creep prediction, achieving unprecedented accuracy with a Mean Absolute Percentage Error (MAPE) of 1. 63% across loading steps. The proposed architecture fundamentally advances the field by treating concrete creep prediction as an...
-
[10]
Swelling, shrinkage and creep: a mechanical approach to cement hydration
References Acker, P., 2004. Swelling, shrinkage and creep: a mechanical approach to cement hydration. Mat. Struct. 37, 237–243. https://doi.org/10.1007/BF02480632 ASTM International, 2015. Standard Test Method for Creep of Concrete in Compression (No. C512/C512M). ASTM International, West Conshohocken, PA. Ba Ragaa, A., Al -Neshawy, F., Noureldin, M., 202...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.