Pith. sign in

REVIEW 4 major objections 4 minor 47 references

ReFace claims that splitting a facial video into four spatial quadrants before tokenization produces 56.00% test accuracy on AI4Pain — the highest video-only figure among the methods compared — against a 49.67% full-face baseline at the sam

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 11:52 UTC pith:6TPOHWRW

load-bearing objection The quadrant-reorganization trick is clean and honestly reported, but the 56% headline is a single-run number on 12 subjects and the apparent gain over prior work is within sampling noise. the 4 major comments →

arxiv 2607.19722 v2 pith:6TPOHWRW submitted 2026-07-22 cs.CV

ReFace: Reorganizing Facial Spatiotemporal Representations for Improved Pain Assessment

Stefanos Gkikas , Yu Fang , Christian Arzate Cruz , Muhammad Umar Khan , Raul Fernandez Rojas This is my paper
classification cs.CV
keywords automatic pain assessmentfacial video analysisspatial reorganizationquadrant partitioningvision transformertoken mixingAI4Pain benchmarkvideo-only classification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that how facial pixels are arranged before a transformer sees them matters as much as how many there are. Its proposed pipeline, ReFace, cuts a 224×224 face into four 112×112 quadrants, folds time into channels, and concatenates the quadrants into one tensor that is tokenized and processed by a segment-latent attention model. On the AI4Pain benchmark, the four-quadrant configuration reaches 56.00% test accuracy using video only, which the paper reports as the highest among compared methods under the fixed protocol, and it beats a full-face baseline at the same stride by more than six points. A single quadrant, at one-quarter of the pixels, stays competitive, which matters for deployment where compute is scarce. A sympathetic reader would care because the result suggests an inexpensive preprocessing change — not more data, not extra sensors — can improve automatic pain assessment from facial video.

Core claim

On the paper's own terms, the central discovery is that spatial reorganization of facial input before tokenization improves video-only pain classification. The four-quadrant fusion configuration — top-left, top-right, bottom-left, bottom-right concatenated along channels at a temporal stride of 10 — achieves 56.00% accuracy on the AI4Pain test set, the highest figure among methods compared under the fixed benchmark protocol, using no physiological sensors. In the paper's ablation, this exceeds the full-face baseline at the same stride (49.67%) and the best full-face validation configuration when moved to test (54.00%). The configuration consumes the same total pixel budget as the full face,

What carries the argument

ReFace is a spatial reorganization pipeline: the face is divided into four non-overlapping 112×112 quadrants; frames are temporally subsampled at a stride s; each region's temporal axis is folded into channels; the K region tensors are concatenated along the channel dimension (C = 3TK); and the spatial grid is flattened into tokens with Fourier coordinate encodings. This single unified tensor is processed by a segment-latent transformer in which each of 64 latent states cross-attends to one segment of tokens, followed by global self-attention and mean pooling. The central idea is to embed spatial-region structure in the input before tokenization, rather than letting the model discover facial

Load-bearing premise

The load-bearing premise is that one test-set evaluation of a configuration selected on validation — 12 test subjects, no repeated runs, no significance testing — is a reliable estimate of true performance; if that number is sampling noise, the 'highest reported' and 'improvement' claims lose support.

What would settle it

Run the same four-quadrant configuration (TL|TR|BL|BR, stride 10) with five random seeds on the AI4Pain test set. If the seed-averaged accuracy is not reliably above the full-face baseline at the same stride (49.67%) and does not at least match the prior video-only result (55.00%), the claimed improvement would be refuted. A second check: randomize the order of the four quadrant channel groups; if accuracy stays at roughly 56%, the benefit is not coming from the anatomical arrangement the paper proposes.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the central claim holds, a video-only pain classifier can reach or exceed the test accuracy of several multimodal systems on the same protocol, which simplifies deployment where physiological sensors are impractical.
  • The same-pixel-budget result implies that spatial arrangement of input tokens is a lever for accuracy independent of resolution, so the procedure may transfer to other transformer-based video recognition tasks.
  • A single quadrant at one-quarter of the computational cost stays close to the full four-quadrant accuracy, suggesting a low-cost operating point for continuous monitoring on constrained hardware.
  • The strong dependence on temporal stride means the benefit is not automatic: practitioners would need to re-tune stride when changing the spatial partition.
  • Because the headline number is one run on 12 test subjects, the practical gain over the prior video-only result should be re-estimated with repeated runs before relying on it.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural interpretation the paper does not defend: the fixed anatomical ordering of quadrants acts as a strong prior, forcing each latent segment to attend only to one facial region. This could be tested by permuting the four channel groups; if accuracy survives permutation, the benefit comes from finer tokenization, not from spatial layout.
  • The bottom-left quadrant's 55.67% hints that lower-face motion (mouth and jaw) carries most of the discriminative signal in this dataset; the paper does not quantify region salience, so this remains an editorial reading.
  • The reorganization is task-agnostic, so the same pipeline could be tried for other facial-expression recognition problems where cues are spatially localized; this is an extrapolation, not a paper claim.
  • A larger held-out set or repeated subject splits would be needed to see whether the one-point lead over the previous video-only method is systematic; the paper's design cannot resolve this.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes ReFace, a spatial reorganization pipeline for video-based pain assessment. A face crop is partitioned into regions (full, halves, single quadrants, or four quadrants); each region is temporally subsampled at a stride s, axis-folded into channels, concatenated into one tensor, flattened to tokens, and then processed by a segment-latent transformer with Fourier positional encoding. On AI4Pain (three classes; 41 train, 12 validation, 12 test subjects), the four-quadrant configuration at stride 10 attains 56.00% test accuracy, which the authors claim is the highest reported under the fixed AI4Pain benchmark protocol; it exceeds the full-face same-stride test accuracy (49.67%) and the best full-face validation configuration evaluated on test (54.00%). A single bottom-left quadrant reaches 55.67%. The paper includes a validation sweep over 36 configurations with parameters, GFLOPs, and latencies, and explicitly states that no repeated-run statistical significance analysis was performed.

Significance. If robust, the contribution is a simple and compute-conscious design finding: reordering the face into quadrant channel groups before tokenization can outperform whole-face tokenization under the same base architecture and total pixel budget, and a single quadrant remains competitive at much lower cost. The paper's controlled ablations and transparent compute reporting are strengths, as is the explicit limitation statement. However, the empirical case for the headline claims rests on a single test-set evaluation of one configuration selected from 36 validation candidates, on a 12-subject test set, with margins over prior work on the order of one to two video samples and no confidence intervals or significance tests. The significance is therefore promising but not yet established.

major comments (4)
  1. [Section V / Table VI] The central claim that 56.00% is the highest reported test accuracy is not statistically supported. The test set has only 12 subjects; if the test subjects have 12 samples each, a 1-percentage-point difference is about 1.4 samples. The reported margins over [21] (55.69%) and [20] (55.00%) are at or below that scale. Table VI also shows the single bottom-left quadrant at 55.67%, only 0.33 percentage points below the four-quadrant fusion, indicating that the fusion gain is within configuration noise. The paper's own Section VI concedes that 'repeated-run statistical significance analysis was not performed.' Please provide repeated runs across seeds, confidence intervals (ideally subject-level bootstrap), and paired comparisons against full-face and prior configurations, or substantially weaken the 'highest reported' and 'improvement' wording.
  2. [Section IV] The final configuration was selected by validation Average from Tables III–V (4 full-face + 8 half-face + 16 single-quadrant + 8 multi-region = 36 configurations) and then evaluated once on the test set. The reported 56.00% is thus the test outcome for the single best-validation configuration, not an estimate of a pre-specified method's performance. Validation selection among many correlated configurations inflates the chance of an extreme test value. The paper should report the test performance of all configurations, or at least a dispersion measure around the selected configuration, and should discuss the selection effect when making a single-number leaderboard claim.
  3. [Section V] The phrase 'fixed AI4Pain benchmark protocol' is used to justify cross-method comparison, but the protocol is not defined. Prior rows in Table VI may differ in preprocessing, training splits, input modalities, and evaluation details. Please state explicitly what constitutes the fixed protocol (same subject partitions, same labels, same metric) and verify that all cited numbers were obtained under it. If they are merely reported numbers from different papers, the comparison is a convenience leaderboard rather than a controlled benchmark, and the claim should be phrased accordingly.
  4. [Section V] The secondary claim that 'the bottom-left quadrant also remained competitive, achieving 55.67% accuracy' is reported without specifying the stride or quadrant configuration, although Table IV shows strong stride dependence on validation. Please provide the exact configuration, and ideally the test results for all four single quadrants, so that the 'single quadrant remains competitive' claim can be reproduced and assessed.
minor comments (4)
  1. [Abstract] The text contains a rendering error: 'ReF ace' should be 'ReFace'. Please fix in the final version.
  2. [Table VI] The checkmark symbols for modality use and the three rows for [21] are not keyed clearly; readers cannot immediately tell which modality row corresponds to which configuration. Please clarify the table formatting and legend.
  3. [Section IV] The statement 'Each subject has 12 samples in the training set' should clarify whether the 12 validation and 12 test subjects also have 12 samples each; this is important for interpreting how many test videos a 1% accuracy difference corresponds to.
  4. [Section III-B-2-e] The symbol s is used both for temporal stride and for the frequency scales in Eq. (5). Rename the frequency scales (e.g., f_k) to avoid confusion.

Circularity Check

0 steps flagged

No circularity: all headline claims are empirical benchmark comparisons; the shared backbone is fixed across conditions and no fitted parameter is renamed as a prediction.

full rationale

ReFace contains no derivation chain that reduces to its inputs. The central claims—56.00% test accuracy, highest reported under the AI4Pain protocol, and quadrant fusion outperforming full-face—are empirical comparisons against a held-out test set. The final configuration was selected on validation and evaluated once on the test set (Section IV: 'The final configuration was selected according to the validation Average and then evaluated once on the 12 held-out test subjects using the released test labels'), which is standard model selection, not fitting a parameter to the target quantity. The architecture is drawn from the authors' own prior tokenization/segment-latent work, but it is applied identically to the full-face, half-face, single-quadrant, and four-quadrant conditions, so it does not by construction force the quadrant result. No uniqueness theorem, imported ansatz, or self-citation is used to justify the empirical outcome. The paper itself notes the statistical limitation: 'the study is limited to a single dataset with small subject-level validation and test splits, and repeated-run statistical significance analysis was not performed' (Section VI). That is a robustness/correctness concern about sampling noise and selection bias, not a circularity concern. The comparison with prior AI4Pain results is made against external methods on a shared benchmark, and the claimed advantage (e.g., 56.00% vs. 55.69% in Table VI) is an empirical observation, not a consequence of definition or construction. Accordingly, no circular step can be quoted or exhibited, and the appropriate finding is no significant circularity.

Axiom & Free-Parameter Ledger

9 free parameters · 4 axioms · 0 invented entities

No new physical entities are postulated; the only 'invented' structure is the data-reorganization scheme, which is a method, not an entity. Free parameters are mostly architecture/selection choices; they are disclosed in the tables but selected on validation without significance testing.

free parameters (9)
  • Temporal stride s = 10 (final 4Q config); swept over {5,10,20,30}
    Chosen by validation Average; directly controls number of frames and channel dimension C=3TK, altering parameter count and GFLOPs.
  • Region configuration K = TL|TR|BL|BR (K=4)
    Selected from 8 configurations (full, top, bottom, 4 quadrants, Top|Bottom, 4Q) on validation; this is the central design choice.
  • Latent dimension d = 128
    Hand-set architecture hyperparameter (Table I) used across all experiments.
  • Number of latents M = 64
    Hand-set; each of 64 segments gets one latent state (Table I).
  • Segments S = 64
    Hand-set; token sequence partition into contiguous blocks (Table I).
  • Fourier bands Kf and max frequency fmax = Kf=6, fmax=10
    Hand-set; define the positional encoding dimension 2(2Kf+1)=26.
  • Depth L, self-attn blocks R, attention heads = L=1, R=1, cross-attn heads=1, latent self-attn heads=4
    Hand-set architecture choices (Table I) that affect capacity.
  • Training schedule and regularization = LR=2e-5, cosine, weight decay 0.1, epochs 100, warmup 15, batch 32, dropouts 0.10, label smoothing 0.10
    Hand-set training configuration (Table II); not swept in this paper.
  • Augmentation probabilities = Ranges in Table II (e.g., TrivialAugment p in [0.20,0.50])
    Hand-set augmentation policy applied consistently across all configurations.
axioms (4)
  • domain assumption AI4Pain labels (Low/No/High Pain) are reliable ground truth.
    Labels derive from TENS stimulation at self-reported thresholds; used as ground truth for all methods in Table VI.
  • domain assumption MTCNN face detection yields stable, comparable crops so quadrant positions correspond to the same anatomical regions across frames.
    Section III-A; if bounding-box jitter shifts anatomy between quadrants, the reorganization is not spatially consistent.
  • domain assumption A single validation evaluation on 12 subjects is an adequate model-selection signal.
    Section IV; the central claim depends on selecting the best configuration from 32 validation candidates.
  • domain assumption The segment-latent cross-attention architecture from prior work transfers without modification.
    Section III-C; reused from the authors' earlier tokenization framework; no formal guarantees are provided.

pith-pipeline@v1.3.0-alltime-deepseek · 13041 in / 12035 out tokens · 119779 ms · 2026-08-01T11:52:28.983121+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of ReFace: Reorganizing Facial Spatiotemporal Representations for Improved Pain Assessment." pith.science (2026). https://pith.science/paper/6TPOHWRW

@misc{pith2026260719722,
  author       = {Pith},
  title        = {Pith review of: ReFace: Reorganizing Facial Spatiotemporal Representations for Improved Pain Assessment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6TPOHWRW}},
  note         = {Machine review of arXiv:2607.19722}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Automatic pain assessment from facial video remains challenging due to the spatial heterogeneity of pain-related facial cues. This study proposes ReFace, a spatial reorganization pipeline that divides facial input into four spatial quadrants before tokenization, rather than processing the entire face as a single region. Evaluated on the AI4Pain dataset, the proposed approach achieves $56.00\%$ accuracy on the test set using video only, achieving the highest reported accuracy under the fixed AI4Pain benchmark protocol among the compared methods. Notably, the four-quadrant configuration processes the same total pixel budget as the full-face input, yet achieves higher accuracy, suggesting that spatial reorganization can improve performance under the proposed tokenization design. A single quadrant region, processing just one quarter of those pixels, remains competitive at a fraction of the computational cost.

Figures

Figures reproduced from arXiv: 2607.19722 by Christian Arzate Cruz, Muhammad Umar Khan, Raul Fernandez Rojas, Stefanos Gkikas, Yu Fang.

Figure 1
Figure 1. Figure 1: Overview of the proposed reorganization pipeline. The input facial video is spatially partitioned into four quadrant [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Average score (%) and GFLOPs across all face region [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 1 canonical work pages

  1. [1]

    Automatic assessment of pain based on deep learning methods: A systematic review,

    S. Gkikas and M. Tsiknakis, “Automatic assessment of pain based on deep learning methods: A systematic review,”Computer Methods and Programs in Biomedicine, vol. 231, p. 107365, 2023

    2023

  2. [2]

    A pain assessment framework based on multimodal data and deep machine learning methods,

    S. Gkikas, “A pain assessment framework based on multimodal data and deep machine learning methods,” 2025, arXiv preprint arXiv:2505.05396. [Online]. Available: https://arxiv.org/abs/2505.05396

    Pith/arXiv arXiv 2025

  3. [3]

    G. Collaboratorset al., “Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: A systematic analysis for the global burden of disease study 2017,”The Lancet, vol. 392, no. 10159, pp. 1789–1858, nov 2018. TABLE III: Performance and computational cost...

    1990

  4. [4]

    The State of US Health, 1990-2010: Burden of Diseases, Injuries, and Risk Factors,

    U. B. of Disease Collaborators, “The State of US Health, 1990-2010: Burden of Diseases, Injuries, and Risk Factors,”JAMA, vol. 310, no. 6, pp. 591–606, 08 2013

    1990

  5. [5]

    The economic costs of pain in the united states,

    D. J. Gaskin and P. Richard, “The economic costs of pain in the united states,”The Journal of Pain, vol. 13, no. 8, pp. 715–724, 2012

    2012

  6. [6]

    The individual and societal burden of chronic pain in europe: the case for strategic prioritisation and action to improve knowledge and availability of appropriate care,

    H. Breivik, E. Eisenberg, and T. O’Brien, “The individual and societal burden of chronic pain in europe: the case for strategic prioritisation and action to improve knowledge and availability of appropriate care,”BMC public health, vol. 13, pp. 1–14, 2013

    2013

  7. [7]

    Cognition and pain: A review,

    T. Khera and V . Rangasamy, “Cognition and pain: A review,”Frontiers in Psychology, vol. V olume 12 - 2021, 2021

    2021

  8. [8]

    Pathogenesis of pain,

    P. Dinakar and A. M. Stillman, “Pathogenesis of pain,”Seminars in Pediatric Neurology, vol. 23, no. 3, pp. 201–208, aug 2016

    2016

  9. [9]

    Prescription opioid abuse in chronic pain: an updated review of opioid abuse predictors and strategies to curb opioid abuse: part 1,

    A. D. Kaye, M. R. Jones, A. M. Kaye, J. G. Ripoll, V . Galan, B. D. Beakley, F. Calixto, J. L. Bolden, R. D. Urman, and L. Manchikanti, “Prescription opioid abuse in chronic pain: an updated review of opioid abuse predictors and strategies to curb opioid abuse: part 1,”Pain physician, vol. 20, no. 2, p. S93, 2017

    2017

  10. [10]

    The first 24 h: opioid administration in people with spinal cord injury and neurologic recovery,

    A. Stampas, C. Pedroza, J. N. Bush, A. R. Ferguson, J. L. K. Kramer, and M. Hook, “The first 24 h: opioid administration in people with spinal cord injury and neurologic recovery,”Spinal Cord, vol. 58, no. 10, pp. 1080–1089, 2020

    2020

  11. [11]

    Opioid complications and side effects,

    R. Benyamin, A. M. Trescot, S. Datta, R. M. Buenaventura, R. Adlaka, N. Sehgal, S. E. Glaser, and R. Vallejo, “Opioid complications and side effects,”Pain physician, vol. 11, no. 2S, p. S105, 2008

    2008

  12. [12]

    Use of a pain assessment and intervention notation (p.a.i.n.) tool in critical care nursing practice: Nurses’ evaluations,

    K. A. Puntillo, D. Stannard, C. Miaskowski, K. Kehrle, and S. Gleeson, “Use of a pain assessment and intervention notation (p.a.i.n.) tool in critical care nursing practice: Nurses’ evaluations,”Heart & Lung, vol. 31, no. 4, pp. 303–314, 2002

    2002

  13. [13]

    Guidance on the management of pain in older people,

    A. Abdulla, N. Adams, M. Bone, A. M. Elliott, J. Gaffin, D. Jones, R. Knaggs, D. Martin, L. Sampson, P. Schofield, and British Geriatric Society, “Guidance on the management of pain in older people,”Age and ageing, vol. 42 Suppl 1, p. i1—57, March 2013

    2013

  14. [14]

    Analgesic administration, pain intensity, and patient satisfaction in cardiac surgical patients,

    D. Meehan, M. McRae, D. Rourke, C. Eisenring, and F. Imperial, “Analgesic administration, pain intensity, and patient satisfaction in cardiac surgical patients,”American Journal of Critical Care, vol. 4, no. 6, pp. 435–442, 1995

    1995

  15. [15]

    A systematic review of neurophysiological sensing for the assessment of acute pain,

    R. Fernandez Rojas, N. Brown, G. Waddington, and R. Goecke, “A systematic review of neurophysiological sensing for the assessment of acute pain,”NPJ Digital Medicine, vol. 6, no. 1, p. 76, 2023

    2023

  16. [16]

    Automatic pain recognition from video and biomedical signals

    P. Werner, A. Al-Hamadi, R. Niese, S. Walter, S. Gruss, and H. C. Traue, “Automatic pain recognition from video and biomedical signals.” Institute of Electrical and Electronics Engineers Inc., 2014, pp. 4582–4587

    2014

  17. [17]

    Analysis of facial expressiveness during experimentally induced heat pain,

    P. Werner, A. Al-Hamadi, and S. Walter, “Analysis of facial expressiveness during experimentally induced heat pain,” in2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2017, pp. 176–180

    2017

  18. [18]

    Spatio-temporal pain estimation network with measuring pseudo heart rate gain,

    D. Huang, X. Feng, H. Zhang, Z. Yu, J. Peng, G. Zhao, and Z. Xia, “Spatio-temporal pain estimation network with measuring pseudo heart rate gain,”IEEE Transactions on Multimedia, vol. 24, pp. 3300–3313, 2022

    2022

  19. [19]

    A full transformer-based framework for automatic pain estimation using videos,

    S. Gkikas and M. Tsiknakis, “A full transformer-based framework for automatic pain estimation using videos,” in2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2023, pp. 1–6

    2023

  20. [20]

    Transformer with leveraged masked autoencoder for video-based pain assessment,

    M.-D. Nguyen, H.-J. Yang, S.-H. Kim, J.-E. Shin, and S.-W. Kim, “Transformer with leveraged masked autoencoder for video-based pain assessment,” 2024

    2024

  21. [21]

    Painformer: A vision foundation model for automatic pain assessment,

    S. Gkikas, R. F. Rojas, and M. Tsiknakis, “Painformer: A vision foundation model for automatic pain assessment,”IEEE Transactions on Affective Computing, vol. 16, no. 4, pp. 3369–3386, 2025

    2025

  22. [22]

    Mul- timodal automatic assessment of acute pain through facial videos and heart rate signals utilizing transformer-based architectures,

    S. Gkikas, N. S. Tachos, S. Andreadis, V . C. Pezoulas, D. Zaridis, G. Gkois, A. Matonaki, T. G. Stavropoulos, and D. I. Fotiadis, “Mul- timodal automatic assessment of acute pain through facial videos and heart rate signals utilizing transformer-based architectures,”Frontiers in Pain Research, vol. 5, 2024

    2024

  23. [23]

    Twins-painvit: Towards a modality-agnostic vision transformer framework for multimodal automatic pain assessment using facial videos and fnirs,

    S. Gkikas and M. Tsiknakis, “Twins-painvit: Towards a modality-agnostic vision transformer framework for multimodal automatic pain assessment using facial videos and fnirs,” in2024 12th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2024, pp. 13–21

    2024

  24. [24]

    A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalities,

    S. Gkikas, C. A. Cruz, V . Becchetti, M. U. Khan, A. Giuseppi, and R. F. Rojas, “A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalities,” inProceedings of the 28th ACM International Conference on Multimodal Interaction. Association for Computing Machinery, 2026

    2026

  25. [25]

    Giaformer: A gradient-infused attention and transformer for pain assessment with eda-fnirs fusion,

    M. U. Khan, G. Chetty, S. Gkikas, M. Tsiknakis, R. Goecke, and R. Fernandez-Rojas, “Giaformer: A gradient-infused attention and transformer for pain assessment with eda-fnirs fusion,”Information Fusion, vol. 131, p. 104173, 2026

    2026

  26. [26]

    A crossmod-transformer deep learning framework for multi-modal pain detection through eda and ecg fusion,

    J. Farmani, G. Bargshady, S. Gkikas, M. Tsiknakis, and R. Fernandez Rojas, “A crossmod-transformer deep learning framework for multi-modal pain detection through eda and ecg fusion,”Scientific Reports, vol. 15, no. 1, p. 29467, 2025. [Online]. Available: https://doi.org/10.1038/s41598-025-14238-y

    work page doi:10.1038/s41598-025-14238-y 2025

  27. [27]

    A systematic review of multimodal signal fusion for acute pain assessment systems,

    M. U. Khan, G. Chetty, R. Goecke, and R. Fernandez-Rojas, “A systematic review of multimodal signal fusion for acute pain assessment systems,”ACM Comput. Surv., 2025

    2025

  28. [28]

    A spatiotemporal convolutional neural network for automatic pain intensity estimation from facial dynamics,

    M. Tavakolian and A. Hadid, “A spatiotemporal convolutional neural network for automatic pain intensity estimation from facial dynamics,” International Journal of Computer Vision, vol. 127, no. 10, pp. 1413– 1425, oct 2019

    2019

  29. [29]

    Automatic pain intensity estimation based on electrocardiogram and demographic factors

    S. Gkikas, C. Chatzaki, E. Pavlidou, F. Verigou, K. Kalkanis, and M. Tsik- nakis, “Automatic pain intensity estimation based on electrocardiogram and demographic factors.” SciTePress, 2022, pp. 155–162

    2022

  30. [30]

    Multi-task neural networks for pain intensity estimation using electrocardiogram and demographic factors,

    S. Gkikas, C. Chatzaki, and M. Tsiknakis, “Multi-task neural networks for pain intensity estimation using electrocardiogram and demographic factors,” inInformation and Communication Technologies for Ageing Well and e-Health. Springer Nature Switzerland, 2023, pp. 324–337

    2023

  31. [31]

    Synthetic thermal and rgb videos for automatic pain assessment utilizing a vision-mlp architecture,

    S. Gkikas and M. Tsiknakis, “Synthetic thermal and rgb videos for automatic pain assessment utilizing a vision-mlp architecture,” in2024 12th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2024, pp. 4–12

    2024

  32. [32]

    Tiny-biomoe: a lightweight embedding model for biosignal analysis,

    S. Gkikas, I. Kyprakis, and M. Tsiknakis, “Tiny-biomoe: a lightweight embedding model for biosignal analysis,” inCompanion Proceedings of the 27th International Conference on Multimodal Interaction, ser. ICMI Companion ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 117–126

    2025

  33. [33]

    Pain-awareness multistream convolutional neural network for pain estimation,

    D. Huang, Z. Xia, L. Li, K. Wang, and X. Feng, “Pain-awareness multistream convolutional neural network for pain estimation,”Journal of Electronic Imaging, vol. 28, no. 04, p. 1, 2019

    2019

  34. [34]

    The ai4pain grand challenge 2024: Advancing pain assessment with multimodal fnirs and facial video analysis,

    R. Fernandez–Rojas, C. Joseph, N. Hirachan, B. Seymour, and R. Goecke, “The ai4pain grand challenge 2024: Advancing pain assessment with multimodal fnirs and facial video analysis,” in2024 12th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2024, pp. 55–60

    2024

  35. [35]

    Multimodal physiological sensing for the assessment of acute pain,

    R. Fernandez Rojas, N. Hirachan, N. Brown, G. Waddington, L. Murtagh, B. Seymour, and R. Goecke, “Multimodal physiological sensing for the assessment of acute pain,”Frontiers in Pain Research, vol. 4, 2023

    2023

  36. [36]

    Multi-representation diagrams for pain recognition: Integrating various electrodermal activity signals into a single image,

    S. Gkikas, I. Kyprakis, and M. Tsiknakis, “Multi-representation diagrams for pain recognition: Integrating various electrodermal activity signals into a single image,” inCompanion Proceedings of the 27th International Conference on Multimodal Interaction, ser. ICMI Companion ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 162–171

    2025

  37. [37]

    Efficient pain recognition via respiration signals: A single cross- attention transformer multi-window fusion pipeline,

    ——, “Efficient pain recognition via respiration signals: A single cross- attention transformer multi-window fusion pipeline,” inCompanion Pro- ceedings of the 27th International Conference on Multimodal Interaction, ser. ICMI Companion ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 70–79

    2025

  38. [38]

    Pain assessment using multi-kernel-fcn-lstm and haemoglobin difference in fnirs,

    G. Bargshady, S. Aziz, S. Gkikas, M. Tsiknakis, R. Goecke, and R. Fernandez Rojas, “Pain assessment using multi-kernel-fcn-lstm and haemoglobin difference in fnirs,”ACM Trans. Comput. Healthcare, 2025

    2025

  39. [39]

    A lightweight transformer for pain recognition from brain activity,

    S. Gkikas, C. A. Cruz, Y . Fang, L. Cao, M. U. Khan, T. Kassiotis, G. Giannakakis, R. F. Rojas, and R. Gomez, “A lightweight transformer for pain recognition from brain activity,” 2026

    2026

  40. [40]

    Towards a Unified Modality-Agnostic Multimodal Framework for Cognitive Workload Assessment,

    S. Gkikas, C. A. Cruz, C. Joseph, G. Giannakakis, and R. F. Rojas, “Towards a Unified Modality-Agnostic Multimodal Framework for Cognitive Workload Assessment,” 2026

    2026

  41. [41]

    Joint face detection and alignment using multitask cascaded convolutional networks,

    K. Zhang, Z. Zhang, Z. Li, and Y . Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,”IEEE signal processing letters, vol. 23, no. 10, pp. 1499–1503, 2016

    2016

  42. [42]

    Trivialaugment: Tuning-free yet state-of- the-art data augmentation,

    S. G. M ¨uller and F. Hutter, “Trivialaugment: Tuning-free yet state-of- the-art data augmentation,” in2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 754–762

    2021

  43. [43]

    Augmix: A simple data processing method to improve robustness and uncertainty,

    D. Hendrycks, N. Mu, E. D. Cubuk, B. Zoph, J. Gilmer, and B. Laksh- minarayanan, “Augmix: A simple data processing method to improve robustness and uncertainty,”arXiv preprint arXiv:1912.02781, 2019

    Pith/arXiv arXiv 1912

  44. [44]

    Faces of experimental pain: Transferability of deep-learned heat pain features to electrical pain*,

    P. Prajod, D. Schiller, D. W. Don, and E. Andr ´e, “Faces of experimental pain: Transferability of deep-learned heat pain features to electrical pain*,” in2024 12th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), 2024, pp. 31–38

    2024

  45. [45]

    Multimodal model for automated pain assessment: Leveraging video and fnirs,

    J. Vianto, A. Divakaran, H. Yang, S. Yeom, S. Kim, S. Kim, and J. Shin, “Multimodal model for automated pain assessment: Leveraging video and fnirs,”Applied Sciences, vol. 15, no. 9, 2025

    2025

  46. [46]

    ✗ ✓ Handcrafted ENS 53.66 [20]✓ ✗ Deep Transformer 55.00 ✗ ✓52.60 ✓ ✗ 53.67 [21] ✓ ✓ Deep Transformer 55.69 Our ✓ ✗ Deep Transformer 56.00† ✓modality is used ✗ modality is not used ENS: Ensemble Classifier †: Four-quadrant fusion (TL|TR|BL|BR), stride10

  47. [47]

    Empirically transformed energy patterns: A novel approach for capturing fnirs signal dynamics in pain assessment,

    M. U. Khan, S. Aziz, L. Murtagh, G. Chetty, R. Goecke, and R. Fernandez Rojas, “Empirically transformed energy patterns: A novel approach for capturing fnirs signal dynamics in pain assessment,”Computers in Biology and Medicine, vol. 192, p. 110300, 2025

    2025