Pith. sign in

REVIEW 3 major objections 6 minor 3 references

Credit Card Fraud Detection Using RoFormer Model With Relative Distance Rotating Encoding

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that replacing absolute integer positions in RoFormer's rotary encoding with relative transaction time distances (ReDRE) improves credit-card fraud detection, lifting AUC-ROC from 0.7288 to 0.740 on the IEEE-CIS dataset.

desk verdict ReDRE is RoPE with timestamps, but the paper never defines which side of the attention dot product gets rotated, so the reported gain is uninterpretable. read the letter →

arxiv 2507.09385 v1 pith:QUES5OG2 submitted 2025-07-12 cs.NE cs.LG

Kevin Reyes , Vasco Cortez This is my paper · ORCID
classification cs.NEcs.LG
keywords creditcardfrauddetectionRoFormerrotarypositionembeddingrelativedistanceencodingIEEE-CISdatasetTransformerAUC-ROCtemporaldependencies
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a Transformer for credit-card fraud detection can be improved by encoding the time gap between transactions directly into the rotary position mechanism, rather than treating that gap as an ordinary input feature. The proposed variant, ReDRE, rotates query and key vectors by an angle proportional to the relative distance between events, replacing the fixed integer positions used by the standard RoFormer. On the IEEE-CIS fraud dataset the variant reaches an AUC-ROC of 0.740, compared with 0.7288 for RoFormer and 0.7286 for a classic Transformer. If the result holds, it suggests that explicitly representing irregular event distances inside the attention geometry is a practical lever for fraud detection.

What carries the argument

The load-bearing mechanism is the rotary matrix $R(p)$, a block-diagonal rotation applied independently to pairs of coordinates of the query and key vectors. RoFormer sets the rotation angle from an absolute position index; ReDRE replaces that index with the relative distance between two transactions, giving $\theta_i = d \cdot \omega_i$. Because attention scores are computed through the dot product of rotated queries and keys, the relative-distance angle is meant to move temporal information into the comparison itself, rather than leaving time gaps as extra columns of the feature vector. The paper presents the general 2x2 rotation $\begin{pmatrix}\cos\theta_i & -\sin\theta_i \\ \sin\theta_i & \cos\theta_i\end{pmatrix}$ applied to each coordinate pair, and this rotation is the only structural difference between the proposed model and the RoFormer baseline.

What would settle it

A controlled reproduction with identical hyperparameters and a fully specified forward pass would settle the claim: if $\theta_i = d \cdot \omega_i$ is applied to both the query and the key, the rotations cancel in the dot product, and the AUC should collapse to the no-relative-encoding baseline; if the gain persists only under an asymmetric application, the paper's stated mechanism is not what is actually being tested. Reporting the exact code path for the rotation is enough to check this.

Watch

Extended reading notes

Core claim

The central claim is that relative distance rotating encoding improves fraud detection because it makes the attention mechanism sensitive to actual time separations between transactions instead of assuming evenly spaced positions. In the paper's setup, a transaction is a token, the sequence is the cardholder's transaction history, and the angle of each rotation pair is set to $\theta_i = d \cdot \omega_i$, where $d$ is the temporal distance between the event of interest and the current event. This replaces the equispaced positional indexing of RoFormer. The reported outcome is that RoFormer with ReDRE attains the highest AUC-ROC (0.740) among the three models tested, with standard RoFormer at 0.7288 and the plain Transformer at 0.7286. The authors conclude that incorporating angle rotation this way better captures temporal dependencies in transactional data and that the approach generalizes to other definitions of event distance.

Load-bearing premise

The paper assumes, without stating it, that the relative-distance rotation is applied in a way that changes how two transactions are compared, and not symmetrically to both sides, since a symmetric rotation would cancel out and leave no relative information.

Editorial extensions

If this is right

  • If the result is right, encoding transaction time gaps directly into the rotation gives a better fraud-detection model than treating those gaps as plain input features.
  • The reported ordering is ReDRE (0.740), RoFormer (0.7288), Transformer (0.7286) under the paper's experimental setup.
  • Because the angle rule is defined through a general distance $d$, the method is claimed to extend to any meaningful event distance, not only time.
  • A better historical detection model is a step toward real-time fraud prevention: once validated on historical data, the encoder can be used to decline transactions before settlement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the exact rotation convention is underspecified; until the authors state whether $d$ is applied to one vector or both, the 0.011 AUC gap cannot be confidently attributed to relative-distance encoding.
  • Editorial inference: because each model's architecture-specific hyperparameters were tuned separately, part of the gap could be tuning rather than the encoding; an ablation with fixed hyperparameters would isolate the contribution.
  • Editorial inference: the same idea could be tried on other irregular event streams, such as medical claim sequences, IoT sensor logs, or general financial timestamped records, where absolute position is a poor proxy for real time.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a variant of the RoFormer model for credit card fraud detection, called Relative Distance Rotating Encoding (ReDRE). Instead of using integer token positions in the rotary position embedding, ReDRE parameterizes the rotation angle by the temporal distance d between transactions, via theta_i = d * omega_i. The authors compare three models—a traditional Transformer, a standard RoFormer, and the proposed RoFormer with ReDRE—on the IEEE-CIS fraud detection dataset, reporting AUC-ROC values of 0.7286, 0.7288, and 0.740, respectively. The central claim is that incorporating true relative transaction-time distances into the rotary encoding improves fraud detection by about 1.1 points of AUC.

Significance. The underlying idea—using irregular real timestamps rather than equispaced integer positions inside rotary position embeddings—is a reasonable direction for transaction data, and the paper has the virtue of evaluating on a public benchmark with a held-out AUC. However, the central mechanism is not precisely defined, and the empirical evidence is not statistically grounded. As written, the manuscript does not establish that ReDRE is responsible for the reported improvement. The paper also provides no code, no confidence intervals, and no repeated-run statistics, so the claimed gain could be noise or an artifact of the unspecified implementation. The direction may be worth pursuing, but this manuscript would need a precise formulation of ReDRE and substantially stronger experimental evidence before its central claim can be accepted.

major comments (3)
  1. [II-B and II-A] The definition of ReDRE is critically underspecified. In Section II-A the position-encoded queries and keys are written as q'_p = R(p) q and k'_p = R(p) k, and in Section II-B the ReDRE angle is theta_i = d * omega_i. The paper never states whether the same relative distance d is applied to both the query and the key. If it is, then in the attention dot product the rotations cancel: (R(d) q)^T (R(d) k) = q^T R(d)^T R(d) k = q^T k, because R(d) is orthogonal. In that case no relative positional information survives and the reported 0.740 AUC cannot be attributed to ReDRE. If, instead, different rotation parameters are intended for the query and the key—as in standard RoPE, where the query uses one position and the key uses another—the assignment of d to each side is not given. This is a load-bearing gap: without a precise specification, the proposed mechanism is undefined and the experiment is not reproducible.
  2. [III-C] The headline result rests on a single AUC difference of 0.011 (0.740 for RoFormer ReDRE versus 0.7288 for RoFormer), reported without variance, confidence intervals, number of seeds, or repeated runs. On an imbalanced dataset with roughly 3.5% positive labels, such a small difference is well within the range of random seed variability for deep Transformer models. The claim that ReDRE 'improves' fraud detection therefore needs mean-and-standard-deviation reporting across multiple runs and, ideally, a significance test. As it stands, the empirical superiority claim is not supported.
  3. [III-B] The comparison is not as controlled as stated. The paper says hyperparameters such as learning rate, batch size, and number of epochs were kept consistent across models, but then says architecture-specific hyperparameters were optimized independently for each model using Optuna. If this tuning was performed on the same validation split used to report the AUC in Section III-C, then the reported scores are selection-biased and the cross-model comparison is not fair. The Optuna search spaces, number of trials, and the final chosen hyperparameters are not reported, so the reader cannot assess whether the 0.011 AUC difference is due to ReDRE or to per-model tuning. This directly affects the central comparison.
minor comments (6)
  1. [II-B and II-C] The symbol d is used both for the relative distance between transactions (Section II-B) and for the dimensionality of the embedding vector (Section II-C). This notation conflict should be resolved, for example by using delta for the temporal distance.
  2. [II-D] The equations in Section II-D contain garbled subscripts and missing formatting (for example, the indices q_{2i} and q_{2i+1} are not rendered clearly), making the general application of the rotary transformation hard to follow. Please rewrite these equations with clear notation.
  3. [III-C] Figure 1 is referenced but the corresponding curves are not visible in the manuscript text provided for review; if the figure exists, it must be included with a legend so that the claimed 'slightly superior performance' can be inspected.
  4. [III-C] The results table is presented as an unformatted line of text; it should be a proper table with model names and AUC-ROC values in separate columns or rows.
  5. [I] There are several grammatical and typographical errors, including the sentence 'It can be represented the queries and keys vectors with the rotary position as follows' and a duplicated sentence about fraud detection being an important challenge. A thorough language edit is needed.
  6. [References] The reference to RoFormer is described as 'developed in 2023,' but the cited Neurocomputing article is dated 2024; please verify the correct publication year and, if relevant, cite the earlier arXiv version.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the reported AUC is an external empirical measurement on IEEE-CIS, not a quantity reconstructed from the method's own inputs.

full rationale

The paper's central claim is an empirical one: RoFormer with ReDRE achieves AUC-ROC 0.740 on the IEEE-CIS Fraud Detection dataset, compared with 0.7288 for RoFormer and 0.7286 for the baseline Transformer. This result is produced by training and evaluating on a public benchmark split, not by a derivation that re-inserts the method's own assumptions as if they were conclusions. The method section defines a rotary encoding with theta_i = d * omega_i, but no fitted parameter from the validation labels is later renamed as a prediction, and no self-citation or imported uniqueness theorem carries the argument. The closest concerns are not circular: (1) the ReDRE specification is ambiguous, because if the same rotation R(d) is applied to both query and key, the orthogonal matrices cancel in the attention dot product and no relative-distance signal survives; that is an underspecification/correctness risk that would undermine the mechanism, but it is not an equation-level circular step. (2) Architecture-specific hyperparameters were optimized with Optuna on the same validation split whose AUC is reported, which is a validation-leakage/overfitting concern for the comparison, not a fitted-input-called-prediction structure in the derivation sense. Since no load-bearing claim is equivalent by construction to its own input, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim depends on a new encoding scheme whose exact operation is not fully specified, on an undisclosed frequency schedule, and on per-model hyperparameter tuning. No code or statistical error estimates are supplied, so the ledger of unstated inputs is larger than the reported result.

free parameters (3)
  • Architecture-specific hyperparameters = not reported
    Each model used Optuna to optimize hyperparameters independently (Section III-B), but the selected values and search spaces are not disclosed; the reported AUCs may result from tuning rather than from ReDRE.
  • Rotary frequency scaling omega_i = not reported
    ReDRE defines theta_i = d * omega_i (Section II-B) but never defines omega_i. Unless it is the standard RoPE schedule, it is an unstated input that the result depends on.
  • Temporal distance scaling or clipping = not reported
    TransactionDT deltas are in seconds over six months; using raw seconds in the rotary angle can produce large angle spans, and no scaling or clipping of d is described (Section II-B).
assumptions (4)
  • domain assumption The FDB/IEEE-CIS dataset labels are correct and TransactionDT values are reliable relative times for all transactions.
    The entire evaluation rests on this public dataset and its fraud labels, as described in Section III-A.
  • ad hoc to paper Rotating query and key vectors by R(d) with d equal to the time gap produces a valid relative position encoding that does not reduce to absolute position information.
    Section II-B defines theta_i = d * omega_i but never proves or specifies how d enters both q and k; if both use the same d the rotation cancels in the dot product, and if only one uses it the encoding is not symmetric.
  • standard math Standard RoPE mathematical identities hold for non-integer, irregular positions.
    The paper applies the rotary matrix machinery of RoFormer to real-valued time deltas without adapting the orthogonality or relative-difference arguments (Sections II-C and II-D).
  • domain assumption Models are trained on chronologically ordered transaction sequences.
    Relative temporal distance d is meaningful only if the sequence order matches transaction time; the paper does not describe sorting or batching by TransactionDT (Section III).
invented entities (1)
  • Relative Distance Rotating Encoding (ReDRE)
    purpose: A positional encoding that rotates query and key embeddings by a temporal distance d instead of integer position.
    ReDRE is the paper's central contribution, but it provides no external falsifiable handle; the only evidence is the in-paper AUC comparison, and the mechanism itself is underspecified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Credit Card Fraud Detection Using RoFormer Model With Relative Distance Rotating Encoding." pith.science (2026). https://pith.science/paper/QUES5OG2

@misc{pith2026250709385,
  author       = {Pith},
  title        = {Pith review of: Credit Card Fraud Detection Using RoFormer Model With Relative Distance Rotating Encoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QUES5OG2}},
  note         = {Machine review of arXiv:2507.09385}
}
read the original abstract

Fraud detection is one of the most important challenges that financial systems must address. Detecting fraudulent transactions is critical for payment gateway companies like Flow Payment, which process millions of transactions monthly and require robust security measures to mitigate financial risks. Increasing transaction authorization rates while reducing fraud is essential for providing a good user experience and building a sustainable business. For this reason, discovering novel and improved methods to detect fraud requires continuous research and investment for any company that wants to succeed in this industry. In this work, we introduced a novel method for detecting transactional fraud by incorporating the Relative Distance Rotating Encoding (ReDRE) in the RoFormer model. The incorporation of angle rotation using ReDRE enhances the characterization of time series data within a Transformer, leading to improved fraud detection by better capturing temporal dependencies and event relationships.

Figures

Figures reproduced from arXiv: 2507.09385 by the authors.

Figure 1
Figure 1. Curve AUC-ROC for the Transformer, RoFormer and RoFormer ReDRE models. IV. CONCLUSIONS In this work, we evaluated the performance of three Transformer-based architectures for fraud detection: a [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

3 extracted references · 1 canonical work pages

  1. [1]

    XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE Credit Card Fraud Detection Using RoFormer Model With Relative Distance Rotating Encoding Kevin Reyes Flow Payment Gateway Santiago, Chile. kreyes@flow.cl Vasco Cortez Flow Payment Gateway Santiago, Chile vcortez@flow.cl Abstract— Fraud detection is one of the most important challenges that financial systems must add...

    work page 2023

  2. [3]

    Credit card fraud detection using machine learning: A survey

    Y. Lucas and J. Jurgovsky, “Credit card fraud detection using machine learning: A survey”, Arxiv Preprint Arxiv:2010.06479, October

    arXiv 2010

  3. [6]

    Fraud Dataset Benchmark and Applications,

    P. Grover, J. Xu, J. Tittelfitz, A. Cheng, Z. Li, J. Zablocki, J. Liu and H. Zhou, “Fraud Dataset Benchmark and Applications,” arXiv:2208.14417, September

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.