Pith. sign in

REVIEW 3 major objections 30 references

A multiscale convolutional tokenizer makes transformers work for X-ray fluorescence spectra of pigments under scarce data.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 00:32 UTC pith:UGPZBSC5

load-bearing objection Solid modality-aware tokenizer for 1-D XRF that beats standard spectral transformers on identification and is more parameter-efficient on unmixing, but every number sits on synthetic linear mixtures of single-reference spectra. the 3 major comments →

arxiv 2607.06424 v1 pith:UGPZBSC5 submitted 2026-07-07 cs.CV

XRFormer: Multiscale Tokenization for XRF Representation Learning

Sofiane Daimellah , Sylvie Le H\'egarat-Mascle , Clotilde Boust This is my paper
classification cs.CV
keywords XRF spectroscopytransformerstokenizationself-supervised learningpigment identificationspectral unmixingcultural heritage
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

X-ray fluorescence spectra of pigments are one-dimensional signals with sharp elemental peaks, broader structures, and background that ordinary vision transformers and convolutional nets do not model well. This paper introduces XRFormer, which first runs the spectrum through a multiscale convolutional tokenizer that shrinks spectral resolution while growing embedding dimension, then feeds the resulting compact tokens into a standard transformer encoder. On the Pigments Checker STANDARD v.5 set the model beats ViT, SpectralFormer, and a 1D-CNN baseline at pigment identification and delivers competitive abundance estimates for unmixing while using only 128 tokens and 1.5 M parameters instead of SpectralFormer's 512 tokens and 3.37 M. Self-supervised pretraining with masked spectral reconstruction and a peak-presence prediction task adds further gains. The result is a parameter-efficient, modality-aware foundation for automated pigment analysis when labeled cultural-heritage data are limited.

Core claim

Multiscale, modality-aware tokenization is an effective and parameter-efficient foundation for transformer-based modeling of XRF spectra: XRFormer consistently outperforms ViT, SpectralFormer (with and without CAF), and a 1D-CNN baseline on pigment identification, and achieves robust abundance estimation on unmixing at half the parameters and one-quarter the tokens of SpectralFormer, with further gains from MSM+PPP pretraining.

What carries the argument

The multiscale convolutional tokenizer: three successive 1D convolutional blocks that enlarge effective receptive field, double channel capacity, and downsample spectral resolution by two, followed by adaptive pooling to a fixed 128-token sequence that a standard transformer encoder then processes globally.

Load-bearing premise

Performance rankings obtained on synthetic mixtures of single-reference spectra plus simple intensity and Poisson noise will transfer to real multi-layer or weathered paint samples.

What would settle it

Train and test the same models on a held-out set of genuine multi-acquisition XRF spectra from layered or aged paint samples; if XRFormer's identification accuracy or unmixing A-RMSE advantage disappears relative to SpectralFormer or the 1D-CNN, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper proposes XRFormer, a transformer for 1D XRF spectra whose main novelty is a multiscale convolutional tokenizer that progressively downsamples the spectrum while increasing channel capacity, then feeds a short token sequence (N=128) to a standard transformer encoder. Two self-supervised pretraining objectives are studied: Masked Spectral Modeling (MSM) and a physics-informed Peak Presence Prediction (PPP) task based on prominence-constrained local maxima. On synthetic linear mixtures derived from 22 Pigments Checker STANDARD v.5 reference spectra (plus Infraart pretraining), XRFormer outperforms ViT, SpectralFormer (with/without CAF), and a 1D-CNN on multilabel pigment identification; for unmixing it is competitive with SpectralFormer while using fewer tokens and roughly half the parameters (1.5M vs 3.37M). MSM and MSM+PPP give further gains. Code is released.

Significance. XRF pigment analysis in cultural heritage is genuinely data-limited and underexplored for modern sequence models; a modality-aware tokenizer that targets sharp peaks plus broader background is a sensible and transferable idea. The paper is transparent about parameter/token budgets, reports means±std over five runs, compares against relevant spectral transformers, and ships a public repository—strengths that support reproducibility. If the synthetic rankings transfer, the work would offer a practical, parameter-efficient baseline for XRF representation learning. The significance is currently bounded by the exclusive use of linear-mixture surrogates from single-reference spectra, which is the main caveat on how far the empirical claims can be taken.

major comments (3)
  1. §4.1 and Tables 2–4: Every reported metric is obtained from synthetic linear mixtures of the same 22 single-reference spectra (Dirichlet weights, intensity factor in [0.5,2], Poisson λ=x·10^4). The authors note that elemental contributions combine only “approximately” linearly under typical settings, yet real layered/weathered paint exhibits matrix absorption, secondary fluorescence, and geometry-dependent non-linear peak ratios. Because train/val/test splits are all drawn from this generative process, the AA gains (e.g., 71–77% vs ViT/SpectralFormer) and the efficiency argument (128 tokens / 1.5M vs 512 / 3.37M) may partly reflect match to the synthetic assumptions rather than robustness on physical spectra. This is load-bearing for the central empirical claim. Please either (i) add at least one real multi-acquisition or layered-paint evaluation (even small), or (ii) substantially refra
  2. §3.3 and §4.5 (PPP): The paper states that PPP “further enhances performance … when tuned with an appropriate peak prominence,” and that higher vs lower prominence trades off identification vs unmixing. No prominence values, selection procedure, or sensitivity curve are reported for the MSM+PPP rows in Tables 2–3. Without this, the PPP contribution is not reproducible and the claim that PPP is a useful physics-informed pretext remains under-supported. Please report the prominence(s) used, how they were chosen (validation grid?), and a brief ablation over a small prominence range for both tasks.
  3. §3.1 / §4.2 (tokenizer contribution): The multiscale convolutional tokenizer is the primary architectural claim, yet there is no ablation of its design choices (number of stages, downsampling factors, kernel/stack depth that set the effective receptive field, adaptive pooling to N=128, or a single-scale CNN tokenizer control). Tables 2–4 compare full XRFormer to ViT/GSE-style SpectralFormer, so gains could come from any convolutional front-end rather than multiscale structure specifically. A minimal ablation (single-scale vs multiscale; N∈{64,128,256}) would make the central “multiscale tokenization” claim load-bearing rather than suggestive.

Circularity Check

0 steps flagged

No circularity: empirical architecture comparison on synthetic mixtures with external baselines; no derivation reduces to fitted inputs or self-citation by construction.

full rationale

XRFormer is an empirical ML paper proposing a multiscale convolutional tokenizer + standard transformer encoder, plus optional MSM/PPP pretraining, evaluated via supervised fine-tuning on pigment identification (AA/HA/F1) and unmixing (A-RMSE/R-RMSE/SAM). All claims are comparative performance numbers (Tables 2–4) against independently re-implemented external baselines (ViT, SpectralFormer w/wo CAF, 1D-CNN) under identical capacity settings where possible. Tokenization strategies, loss functions, and the PPP peak-signature construction (local-maxima + prominence on the input spectrum itself) are standard self-supervised or architectural choices; none define a quantity in terms of the reported metric or rename a fit as a prediction. Citations (MViT, SpectralFormer, BERT-style masking, etc.) are to prior external work and are not load-bearing uniqueness theorems. The synthetic linear-mixture protocol (Dirichlet weights, intensity scaling, Poisson noise) is a shared experimental assumption that may limit external validity, but it does not create a circular reduction of any claimed result to its own inputs. The paper is therefore self-contained against its stated benchmarks with zero circular steps.

Axiom & Free-Parameter Ledger

5 free parameters · 3 axioms · 2 invented entities

The paper is an empirical ML architecture paper; its claims rest on standard transformer math, the physical linearity of XRF under typical conditions, and a set of modeling choices for synthetic data generation and peak detection. No new physical entities are postulated.

free parameters (5)
  • peak prominence threshold for PPP ground-truth
    Tuned separately for identification vs unmixing; higher values favor dominant peaks, lower values favor subtle features (Section 3.3).
  • token sequence length N=128
    Fixed design choice that determines both capacity and comparison fairness with SpectralFormer’s 512 tokens.
  • masking ratio 15 % and 80/10/10 replacement schedule for MSM
    Standard BERT-style schedule adopted without ablation on XRF.
  • Dirichlet concentration, intensity range [0.5,2], Poisson rate 10^4 for data augmentation
    Hand-chosen parameters that define the entire synthetic train/test distribution.
  • learning rates 1e-4 (pretrain) / 5e-5 (finetune), batch size 128, early-stopping patience
    Optimizer hyper-parameters that affect final reported numbers.
axioms (3)
  • domain assumption Elemental contributions in XRF combine approximately linearly under typical acquisition settings
    Invoked in Section 4.1 to justify synthetic linear mixtures as training data; cited to Bezur et al. and Moreau et al.
  • standard math Standard pre-norm transformer blocks with MHSA + FFN are sufficient once multiscale tokens are supplied
    Equations (1)–(2) and architecture description in Section 3.2.
  • ad hoc to paper Local-maxima detection with a prominence threshold yields a useful binary peak signature for self-supervision
    Defines the PPP target in Section 3.3; no external validation that the resulting binary vectors are optimal.
invented entities (2)
  • XRFormer multiscale convolutional tokenizer no independent evidence
    purpose: Inject locality and multi-resolution inductive bias into 1-D XRF token sequences before global attention
    Core architectural novelty; three stacked 1-D conv blocks with progressive downsampling and channel growth.
  • Peak Presence Prediction (PPP) pretext no independent evidence
    purpose: Physics-informed self-supervised objective that forces the [CLS] token to encode emission-peak locations
    Defined in Section 3.3 using prominence-based pseudo-labels; no external benchmark of the pretext itself.

pith-pipeline@v1.1.0-grok45 · 15301 in / 2748 out tokens · 34125 ms · 2026-07-11T00:32:55.709014+00:00 · methodology

0 comments
read the original abstract

X-ray fluorescence (XRF) spectroscopy is a key modality for material analysis in cultural heritage. However, automated learning from XRF spectra remains challenging: XRF spectra are complex one-dimensional signals composed of sharp elemental peaks, broader structures, and background variations that are not taken into account by existing learning-based models. This paper introduces XRFormer, a transformer architecture tailored to XRF spectra through a multiscale convolutional tokenizer that injects locality and multi-resolution inductive biases before global self-attention. The tokenizer progressively reduces spectral resolution while increasing embedding dimensionality, and the resulting token sequence is processed by a standard transformer encoder. We further investigate self-supervised pretraining for XRF representation learning using Masked Spectral modeling (MSM) and a physics-informed Peak Presence Prediction (PPP) objective. Experiments on the Pigments Checker STANDARD v.5 dataset for pigment identification and unmixing show that XRFormer consistently outperforms ViT, SpectralFormer (with and without CAF), and a 1D-CNN baseline for pigment identification. For pigment unmixing, XRFormer achieves robust abundance estimation while maintaining significantly higher parameter efficiency than SpectralFormer, operating at a lower token resolution (128 vs. 512 tokens) and with less than half the number of parameters (1.5M vs. 3.37M). MSM yields consistent gains across both tasks, while PPP further enhances performance for both identification and unmixing when tuned with an appropriate peak prominence. These results highlight multiscale, modality-aware tokenization as an effective and parameter efficient foundation for transformer-based XRF modeling under data-limited conditions. A GitHub repository is provided at https://github.com/sofiane1010/XRFormer.

Figures

Figures reproduced from arXiv: 2607.06424 by Clotilde Boust, Sofiane Daimellah, Sylvie Le H\'egarat-Mascle.

Figure 1
Figure 1. Figure 1: Typical XRF spectrum showing the energies of fluorescent emission lines (peaks) corresponding to different elements. Pigment identification and unmixing are common CH applications of XRF, yet automated analysis remains limited. Most existing studies focus on pig￾ment identification and often rely on expert-driven peak interpretation. Recent learning-based approaches operate directly on raw spectra using on… view at source ↗
Figure 2
Figure 2. Figure 2: Three spectral tokenization strategies: (a) linear patch-wise embedding (ViT), (b) group-wise spectral embedding (SpectralFormer), and (c) multiscale convolutional embedding. All tokenizers map a raw XRF spectrum to a token sequence processed by the same transformer encoder. Linear (patch-wise) Embedding Following ViT [8], x is split into N non￾overlapping patches of length P, forming xP ∈ R N×P ( [PITH_F… view at source ↗
Figure 3
Figure 3. Figure 3: XRFormer overview: the multiscale convolutional tokenizer maps the input XRF spectrum to a sequence of token embeddings, which are processed by a standard transformer encoder. The final [CLS] token serves as a global representation for down￾stream tasks. 3.3 Self-Supervised Pretraining Objectives While XRFormer can be trained end-to-end in a supervised manner, we fur￾ther investigate whether self-supervise… view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 30 canonical work pages · 3 internal anchors

  1. [1]

    Getty Conservation Institute (2020)

    Bezur, A.o.: Handheld XRF in cultural heritage: a practical workbook for conser- vators. Getty Conservation Institute (2020)

    work page 2020

  2. [2]

    Machine Learning: Science and Technology6(1), 015058 (2025)

    Bombini, A., et al.: Towards virtual painting recolouring using vision transformer on x-ray fluorescence datacubes. Machine Learning: Science and Technology6(1), 015058 (2025)

    work page 2025

  3. [3]

    Powder Diffraction22(2), 130–137 (2007)

    Börjesson, J., Mattsson, S.: Medical applications of x-ray fluorescence for trace element research. Powder Diffraction22(2), 130–137 (2007). https://doi.org/10. 1154/1.2737468

    work page 2007

  4. [4]

    In: Euro- pean conference on computer vision

    Chen, Y.C., et al.: Uniter: Universal image-text representation learning. In: Euro- pean conference on computer vision. pp. 104–120. Springer (2020)

    work page 2020

  5. [5]

    ACM Journal on Computing and Cultural Heritage16(2), 1–11 (2023)

    Cortea,I.M.,etal.:Infra-art:anopenaccessspectrallibraryofart-relatedmaterials as a digital support tool for cultural heritage science. ACM Journal on Computing and Cultural Heritage16(2), 1–11 (2023)

    work page 2023

  6. [6]

    https://chsopensource

    Cultural Heritage Science Open Source: Pigments Checker. https://chsopensource. org/products/pigments-checker/

    work page

  7. [7]

    In: Proc

    Devlin, J., et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proc. of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). pp. 4171–4186 (2019)

    work page 2019

  8. [8]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

    Dosovitskiy, A., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  9. [9]

    In: Proc

    Fan, H., et al.: Multiscale vision transformers. In: Proc. of the IEEE/CVF inter- national conference on computer vision. pp. 6824–6835 (2021)

    work page 2021

  10. [10]

    GraphCodeBERT: Pre-training Code Representations with Data Flow

    Guo, D., et al.: Graphcodebert: Pre-training code representations with data flow. arXiv preprint arXiv:2009.08366 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2009

  11. [11]

    IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing15, 9266–9277 (2022)

    He, W., et al.: Csit: A multiscale vision transformer for hyperspectral image clas- sification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing15, 9266–9277 (2022)

    work page 2022

  12. [12]

    IEEE Transactions on Geoscience and Remote Sensing60, 1–15 (2021)

    Hong, D., et al.: Spectralformer: Rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing60, 1–15 (2021)

    work page 2021

  13. [13]

    IEEE Transactions on Geoscience and Remote Sens- ing60, 1–14 (2022)

    Ibanez, D., et al.: Masked auto-encoding spectral–spatial transformer for hyper- spectral image classification. IEEE Transactions on Geoscience and Remote Sens- ing60, 1–14 (2022)

    work page 2022

  14. [14]

    Heritage Science10(1), 1–14 (2022)

    Jones, C., et al.: Neural network-based classification of x-ray fluorescence spectra of artists’ pigments: an approach leveraging a synthetic dataset created using the fundamental parameters method. Heritage Science10(1), 1–14 (2022)

    work page 2022

  15. [15]

    Journal of hazardous materials83(1-2), 93–122 (2001)

    Kalnicky, D.J., et al.: Field portable xrf analysis of environmental samples. Journal of hazardous materials83(1-2), 93–122 (2001)

    work page 2001

  16. [16]

    International Jour- nal of Applied Earth Observation and Geoinformation118, 103286 (2023) XRFormer: Multiscale Tokenization for XRF Representation Learning 15

    Khotimah, W.N., et al.: Mce-st: Classifying crop stress using hyperspectral data with a multiscale conformer encoder and spectral-based tokens. International Jour- nal of Applied Earth Observation and Geoinformation118, 103286 (2023) XRFormer: Multiscale Tokenization for XRF Representation Learning 15

    work page 2023

  17. [17]

    X-Ray Spectrometry50(4), 310–319 (2021)

    Kogou, S., et al.: A new approach to the interpretation of xrf spectral imaging data using neural networks. X-Ray Spectrometry50(4), 310–319 (2021)

    work page 2021

  18. [18]

    IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing15, 3891–3903 (2022)

    Liu, K., et al.: Mapping coastal wetlands using transformer in transformer deep network on china zy1-02d hyperspectral satellite images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing15, 3891–3903 (2022)

    work page 2022

  19. [19]

    In: Proc

    Liu, Z., et al.: Swin transformer: Hierarchical vision transformer using shifted win- dows. In: Proc. of the IEEE/CVF international conference on computer vision. pp. 10012–10022 (2021)

    work page 2021

  20. [20]

    In: 2023 23rd International Conference on Transparent Optical Networks (ICTON)

    Moreau, R., et al.: Multi-sensor scanners and machine-learning data processing: a novel instrumentation and data analysis method in heritage science. In: 2023 23rd International Conference on Transparent Optical Networks (ICTON). pp. 1–

    work page 2023

  21. [21]

    Geology, Ecology, and Landscapes 2(2), 148–154 (2018)

    Oyedotun, T.D.T.: X-ray fluorescence (xrf) in the investigation of the composition of earth materials: a review and an overview. Geology, Ecology, and Landscapes 2(2), 148–154 (2018)

    work page 2018

  22. [22]

    In: International conference on machine learning

    Radford, A., et al.: Learning transferable visual models from natural language su- pervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

    work page 2021

  23. [23]

    NPJ digital medicine 4(1), 86 (2021)

    Rasmy, L., et al.: Med-bert: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction. NPJ digital medicine 4(1), 86 (2021)

    work page 2021

  24. [24]

    Applied Physics A122(1), 17 (2016)

    Ravaud, E., et al.: Development of a versatile xrf scanner for the elemental imaging of paintworks. Applied Physics A122(1), 17 (2016)

    work page 2016

  25. [25]

    IEEE Transactions on Geoscience and Remote Sensing60, 1–14 (2022)

    Sun, L., et al.: Spectral–spatial feature tokenization transformer for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing60, 1–14 (2022)

    work page 2022

  26. [26]

    LXMERT: Learning Cross-Modality Encoder Representations from Transformers

    Tan, H., et al.: Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490 (2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1908

  27. [27]

    Advances in neural information processing systems30(2017)

    Vaswani, A., et al.: Attention is all you need. Advances in neural information processing systems30(2017)

    work page 2017

  28. [28]

    Xu, B.J., et al.: Can deep learning assist automatic identification of layered pig- ments from xrf data? Journal of Analytical Atomic Spectrometry37(12), 2672– 2682 (2022)

    work page 2022

  29. [29]

    IEEE Transactions on Pattern Analysis and Machine Intelligence45(10), 12113–12132 (2023)

    Xu, P., et al.: Multimodal learning with transformers: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence45(10), 12113–12132 (2023)

    work page 2023

  30. [30]

    IEEE Geoscience and Remote Sensing Letters19, 1–5 (2022)

    Zhang, J., et al.: Convolution transformer mixer for hyperspectral image classifi- cation. IEEE Geoscience and Remote Sensing Letters19, 1–5 (2022)

    work page 2022