Detecting Dark Patterns in User Interfaces Using Logistic Regression and Bag-of-Words Representation

Abdullahi Abdulkadir; Adamu Lawan; Aliyu Umar; Maaruf Lawan; Mukhtar Dahiru

REVIEW 4 major objections 5 minor 19 references

Detecting Dark Patterns in User Interfaces Using Logistic Regression and Bag-of-Words Representation

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A bag-of-words and logistic-regression pipeline detects dark-pattern UI text at 92 percent accuracy and 97 percent AUC on a public dataset.

desk verdict A thin baseline on dark-pattern detection whose reported 92% accuracy is plausible but under-documented; the paper needs a rigorous split protocol, code, and a sensitivity analysis before the numbers can be taken seriously. read the letter →

arxiv 2412.14187 v1 pith:7NYARBDF submitted 2024-12-09 cs.HC cs.LG

Aliyu Umar , Maaruf Lawan , Adamu Lawan , Abdullahi Abdulkadir , Mukhtar Dahiru This is my paper · ORCID

classification cs.HCcs.LG

keywords darkpatternsdeceptivedesignlogisticregressionbag-of-wordstextclassificationuserinterfacesethicalmachinelearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a deliberately simple text-classification pipeline—bag-of-words features fed into logistic regression—can detect deceptive dark-pattern UI copy with 92% accuracy, 93% precision, 94% recall, 93% F1-score, and 97% AUC on a 3,636-instance public dataset. The dataset is balanced, with 1,818 dark-pattern and 1,818 non-dark-pattern UI texts. If the claim holds, any team building an ethical-design checker could start with a cheap, transparent model instead of a large deep network. The paper's broader aim is to make automated dark-pattern detection practical and interpretable enough to embed in UX workflows and governance.

What carries the argument

The engine is the bag-of-words representation: each UI text becomes a vector of word counts or term frequencies, so a linear classifier only sees which words occur. Logistic regression then models the probability of a dark pattern as $h_\theta(x)=\frac{1}{1+e^{-\theta^\top x}}$, and the learned weight vector doubles as a feature-importance ranking. Cross-validation and grid search tune regularization strength and the solver, while accuracy, precision, recall, F1-score, and AUC measure the result.

What would settle it

An independent replication using a documented stratified 80/20 hold-out of the full 3,636-instance dataset—or a fresh annotation of a few hundred live e-commerce UI texts at natural prevalence—would settle it: if the pipeline's accuracy, precision, recall, and F1 fall materially below 92%, 93%, 94%, and 93%, or if two independent labelers cannot reproduce the dataset's labels, the central claim is not portable.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that a logistic-regression classifier trained on bag-of-words feature vectors separates dark-pattern UI text from ordinary UI text with accuracy 92%, precision 93%, recall 94%, F1-score 93%, and AUC 97% on the public dataset behind reference [1]. The paper presents this as evidence that a transparent linear model can do detection work that is usually assigned to deep neural networks, and that the learned coefficients expose which words drive the decision. That is the discovery: deceptive UI copy carries a word-frequency signal strong enough for a simple, explainable classifier to exploit.

Load-bearing premise

The 92 percent figure assumes the public dataset's labels are correct and that a balanced half-dark, half-normal sample reflects real user-interface text; neither assumption is documented in the paper.

Editorial extensions

If this is right

A UI auditor could run this pipeline on app and website copy and flag texts whose logistic score crosses a chosen threshold.
The coefficient list gives designers a concrete inventory of wording most associated with dark patterns, usable as a style checklist.
A lightweight detector can be embedded in browser extensions or design tools without GPU or large-language-model infrastructure.
The same bag-of-words plus logistic-regression setup provides a strong, reproducible baseline that future dark-pattern detectors must beat on this dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: because the dataset is artificially balanced at 50/50, real web prevalence is far lower, so raw 92% accuracy would not transfer to deployment without calibration or threshold adjustment.
Beyond the paper: bag-of-words discards word order, so dark patterns expressed through sentence structure may slip through; the paper mentions trying n-gram ranges, and a systematic test of bigrams and trigrams is a natural next step.
Beyond the paper: the paper states that inter-annotator agreement was ensured but reports no statistic, so a fresh annotation of a random sample of the same texts would test whether the labels themselves are stable.
Beyond the paper: a direct same-split comparison against the cited deep-learning models, such as RoBERTa, would show whether the linear model's reported edge survives identical preprocessing and evaluation conditions.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Hi,

You asked about arXiv:2412.14187. Short version: it's a vanilla logistic regression + bag-of-words pipeline run on Mathur's public dark-pattern dataset. The headline numbers (92% acc., 93% F1) are believable for a balanced text classification task, but the paper gives you almost nothing to verify them with, and the leakage concern from the stress-test is a real one, not a nitpick.

What's genuine: the dataset is public, the method is appropriate as a baseline, and the authors openly cite prior ML baselines and stronger models. The basic approach—preprocess text, BoW, logistic regression, evaluate—is executed without any obviously broken step. If I needed a simple baseline number on this dataset, this is the kind of result I'd expect.

The soft spots are the problem. Section 3.3 says the vectorizer parameters were 'adjusted based on experimentation' but never says on what data, and Sections 3.1/3.3 don't state the split ratio, the random seed, or whether the vectorizer was fit only on the training fold. The Mathur data is full of repeated UI strings; a random instance split can put near-duplicates in both train and test, so the model could be memorizing templates rather than detecting deceptive language. The paper doesn't rule that out, and no code is provided to check. That's load-bearing, because the entire contribution is the performance claim.

There are also presentation problems: Table 3 labels the row 'RL' instead of 'LR', Figures 1 and 2 are referenced but not actually included in the text, and Section 3.4 claims sensitivity analysis was conducted without showing a single result. The abstract calls the approach 'novel,' but it's a standard pipeline that the authors themselves cite earlier work doing. That overclaim should go.

Net: this is a workshop-level baseline report. The dataset and the idea of a logistic regression baseline are useful, but as written the paper doesn't meet the bar for a serious archival venue. I would desk reject it, or at most send back for major revision with a strict requirement to release code, document the split, and address template leakage. I wouldn't cite it in the next year unless it gets substantially revised.

My reading-group interest is low, and I wouldn't send it to a serious referee in this form.

Referee Report

4 major / 5 minor

Summary. The paper proposes a logistic regression classifier with bag-of-words text features to detect dark patterns in user interface text, using the public Mathur dark-patterns dataset (3,636 instances, balanced between dark and non-dark). The authors report an accuracy of 92%, precision of 93%, recall of 94%, F1 of 93%, and AUC of 97% (Table 3), and claim robustness based on a sensitivity analysis. The paper also includes discussion of feature importance, qualitative analysis, and practical implications for ethical UI design.

Significance. If the reported performance is trustworthy, the paper provides a useful, interpretable baseline for automated dark-pattern detection: a simple linear model with bag-of-words features can separate deceptive UI text from benign text on a widely used public dataset. The use of an external dataset and clearly defined metrics is a strength. However, the manuscript omits crucial experimental details—exact data split, vectorizer configuration, hyperparameters, and any actual sensitivity or qualitative results—so the central quantitative claim (Table 3) cannot currently be reproduced or fully assessed. The methodology itself is standard and not novel, but establishing a transparent baseline on this dataset is a legitimate contribution if the evaluation is properly documented.

major comments (4)

[§3.3 and §4.1] The paper never states how the data were split into training and test sets, nor whether the vectorizer (including max_features and n-gram range) and logistic regression hyperparameters were fit only on the training portion. Because the Mathur dataset consists of real UI strings that are highly templated, a random instance-level split can place exact or near-duplicate texts in both training and test sets, so the reported 92% accuracy, 93% precision, 94% recall, 93% F1, and 97% AUC may partly reflect template memorization rather than generalization to unseen deceptive text. The authors should describe the split protocol precisely and, ideally, report results under a source-grouped or deduplicated split.
[§3.3 and §4] Section 3.3 claims that "sensitivity analysis was conducted" and that "qualitative analysis of misclassified instances... was performed," but Section 4 contains no results from either analysis: no variation of max_features, n-gram range, regularization strength, or preprocessing choices, no error bars or confidence intervals, and no example false positives or false negatives. The abstract's assertion of "robustness to variations in dataset composition and model parameters" is therefore unsupported by the presented evidence.
[§3.3] The vectorizer is described as "CountVectorizer or TfidfVectorizer" with parameters "adjusted based on experimentation," but the actual choice, the final max_features, the n-gram range, the regularization strength C, the solver, and the train/test split ratio are not reported. Without these details, the results in Table 3 cannot be reproduced, and the reader cannot rule out that the reported performance was obtained by tuning on the test set.
[§4.1 and Table 3] Table 3 is labeled "Confusion matrix achieved by Logistic Regression" but contains only aggregate metrics; the actual confusion matrix with TP, FP, TN, and FN counts is never reported. The confusion matrix figure (Figure 1) appears to be a generic schematic rather than the experimental result, and the ROC curve (Figure 2) and feature-importance bar chart (Figure 3) are not shown in the manuscript text. Please provide the raw counts and the actual figures.

minor comments (5)

[§4.1, Table 3] The algorithm label "RL" appears to be a typo for "LR" (logistic regression).
[§3.2, Eq. (4)] In the recall equation, "Negetives" should be "Negatives."
[§3.1] The claim that "inter-annotator agreement through rigorous guidelines and regular quality checks were ensured" is vague; please report the inter-annotator agreement measure or cite the original dataset paper where it is documented.
[§3.1 and References] The GitHub repository should be cited with the repository name, version, and access date, and reference [15] (Loshchilov and Hutter) appears unrelated to the surrounding sentence about RoBERTa.
[§5] The Discussion states that the model shows "comparable or superior accuracy" to existing studies, but no quantitative comparison to [11] or other baselines is provided; please add a direct comparison on the same dataset.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: reported results are empirical measurements on an external public dataset, not derivations from the method's own inputs.

full rationale

The paper's central claim is a set of classifier metrics (Table 3: accuracy 92%, precision 93%, recall 94%, F1-score 93%, AUC 97%) obtained by training logistic regression on bag-of-words features extracted from the Mathur et al. dark-pattern text dataset and evaluating on a held-out test set. Nothing in the derivation chain defines the target labels in terms of the model's predictions, and no reported quantity is a renamed fitted parameter: the metrics are empirical outcomes computed from the external labels using the standard equations in Section 3.3. The dataset, labels, vectorizer, and model are external or standard tools rather than constructs defined by this paper's results. The paper cites prior work, including Mathur et al. and Yada et al., but no load-bearing premise is justified only by a self-citation; the authors do not cite themselves. Possible weaknesses such as undocumented train/test splitting, potential template leakage, or unreported sensitivity-analysis results are correctness and reporting risks, not circularity, because the paper does not exhibit any equation or construction that makes a prediction equal to its input. Under the rule that honest non-findings are expected when warranted, no circular step is identified.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the quality and representativeness of the public dataset, the sufficiency of bag-of-words features, and a single unpaired test evaluation. The model's fitted coefficients and unreported hyperparameters are data-fitted quantities the paper does not disclose, making exact reproduction impossible.

free parameters (6)

Logistic regression weight vector theta = not reported
Learned from training data; the reported 92% accuracy depends on these fitted coefficients (Eq. 1).
Vectorizer max_features = not reported
CountVectorizer or TfidfVectorizer parameter adjusted by experimentation, value omitted (Section 3.3).
n-gram range = not reported
Adjusted to optimize feature representation; value omitted (Section 3.3).
Regularization strength C = not reported
Optimized via cross-validation and grid search; value omitted (Section 3.4).
Solver algorithm = not reported
Choice affects numerical convergence; value omitted (Section 3.4).
Train/test split ratio and random seed = not reported
Split construction not described, so test-set metrics cannot be exactly reproduced (Section 3.4).

assumptions (4)

domain assumption Text instances from the Mathur et al. GitHub dataset are correctly labeled as dark or non-dark patterns.
The model is trained and evaluated against these labels; no inter-annotator agreement statistic is reported (Section 3.1).
domain assumption Bag-of-words features capture enough linguistic signal for dark-pattern detection.
The method discards word order and semantics; the paper asserts sufficiency without ablations or feature studies (Section 3.3).
domain assumption The balanced 1,818/1,818 split approximates real-world deployment conditions.
Real UI interfaces contain far fewer dark patterns; balanced evaluation overstates accuracy in the wild (Table 1).
domain assumption Evaluation metrics from a single held-out test set are stable and representative.
No repeated runs, confidence intervals, or cross-fold variance are reported (Section 4.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Detecting Dark Patterns in User Interfaces Using Logistic Regression and Bag-of-Words Representation." pith.science (2026). https://pith.science/paper/7NYARBDF

@misc{pith2026241214187,
  author       = {Pith},
  title        = {Pith review of: Detecting Dark Patterns in User Interfaces Using Logistic Regression and Bag-of-Words Representation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7NYARBDF}},
  note         = {Machine review of arXiv:2412.14187}
}

read the original abstract

Dark patterns in user interfaces represent deceptive design practices intended to manipulate users' behavior, often leading to unintended consequences such as coerced purchases, involuntary data disclosures, or user frustration. Detecting and mitigating these dark patterns is crucial for promoting transparency, trust, and ethical design practices in digital environments. This paper proposes a novel approach for detecting dark patterns in user interfaces using logistic regression and bag-of-words representation. Our methodology involves collecting a diverse dataset of user interface text samples, preprocessing the data, extracting text features using the bag-of-words representation, training a logistic regression model, and evaluating its performance using various metrics such as accuracy, precision, recall, F1-score, and the area under the ROC curve (AUC). Experimental results demonstrate the effectiveness of the proposed approach in accurately identifying instances of dark patterns, with high predictive performance and robustness to variations in dataset composition and model parameters. The insights gained from this study contribute to the growing body of knowledge on dark patterns detection and classification, offering practical implications for designers, developers, and policymakers in promoting ethical design practices and protecting user rights in digital environments.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 7 canonical work pages

[1]

Dark Patterns at Scale,

A. Mathur et al., “Dark Patterns at Scale,” Proc ACM Hum Comput Interact, vol. 3, no. CSCW, pp. 1–32, Nov. 2019, doi: 10.1145/3359183

doi:10.1145/3359183 2019
[2]

‘Nothing comes before profit’: Asshole design in the wild,

S. S. Chivukula , C. Watkins, L. McKay, and C. M. Gray, “‘Nothing comes before profit’: Asshole design in the wild,” Conference on Human Factors in Computing Systems - Proceedings, May 2019, doi: 10.1145/3290607.3312863

arXiv 2019
[3]

Ethical mediat ion in UX practice,

C. M. Gray and S. S. Chivukula, “Ethical mediat ion in UX practice,” Conference on Human Factors in Computing Systems - Proceedings, May 2019, doi: 10.1145/3290605.3300408

arXiv 2019
[4]

Wayback Machine

“Wayback Machine.” Accessed: Nov. 29, 2024. [Online]. Available: https://web.archive.org/web/*/https://www.darkpatterns.org/*

work page 2024
[5]

The dark (patterns) side of UX design,

C. M. Gray, Y. Kou, B. Battles, J. Hoggatt, and A. L. Toombs, “The dark (patterns) side of UX design,” Conference on Human Factors in Computing Systems - Proceedings, vol. 2018-April, Apr. 2018, doi: 10.1145/3173574.3174108

arXiv 2018
[6]

#Darkpatterns: UX Practitioner Conversations About Ethical Design,

M. Fansher, S. S. Chivuk ula, and C. M. Gray, “#Darkpatterns: UX Practitioner Conversations About Ethical Design,” Conference on Human Factors in Computing Systems - Proceedings, vol. 2018-April, Apr. 2018, doi: 10.1145/3170427.3188553. 11

arXiv 2018
[7]

Unmasking Dark Patterns: A Machine Learning Approach to Detecting Deceptive Design in E -commerce Websites,

S. T. G. S. & Prof. R. N. B. Arya Ramtek e, “Unmasking Dark Patterns: A Machine Learning Approach to Detecting Deceptive Design in E -commerce Websites,” ArXiv, May 2024

work page 2024
[8]

A Domain -Independent Holistic Approach to Deception Detection,

S. Shahriar, A. Mukherjee, and O. Gnawali, “A Domain -Independent Holistic Approach to Deception Detection,” International C onference Recent Advances in Natural Language Processing, RANLP, pp. 1308–1317, 2021, doi: 10.26615/978-954- 452-072-4_147

work page doi:10.26615/978-954- 2021

Show all 19 references

[9]

Deceptive Patterns (aka Dark Patterns) - spreading awareness since 2010

“Deceptive Patterns (aka Dark Patterns) - spreading awareness since 2010.” Accessed: Nov. 29, 2024. [Online]. Available: https://www.deceptive.design/

2010
[10]

Nudges for Privacy and Security,

A. Acquisti et al. , “Nudges for Privacy and Security,” ACM Computing Surveys (CSUR), vol. 50, no. 3, Aug. 2017, doi: 10.1145/3054926

2017 doi
[11]

Dark patterns in e -commerce: a dataset and its baseline evaluations,

Y. Yada, J. Feng, T. Matsumoto, N. Fukushima, F. Kido, and H. Yamana, “Dark patterns in e -commerce: a dataset and its baseline evaluations,” Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022, pp. 3015–3022, 2022, doi: 10.1109/BIGDATA55660.2022.10020800

2022
[12]

Detecting Deceptive Dark Patterns in E -commerce Platforms,

A. Ramteke, S. Tembhurne, G. Sonawane, and R. N. Bhimanpallewar, “Detecting Deceptive Dark Patterns in E -commerce Platforms,” May 2024, Accessed: Nov. 29,

2024
[13]

AidUI: Toward Automated Recognition of Dark Patterns in User Interfaces,

S. M. Hasan Mansur, S. Salma, D. Awofisayo, and K. Moran, “AidUI: Toward Automated Recognition of Dark Patterns in User Interfaces,” Proceedings - International Conference on Software Engineering , pp. 1958 –1970, 2023, doi: 10.1109/ICSE48619.2023.00166

1958
[14]

Deception detection using machine learning (ML) and deep learning (DL) techniques: A systematic review,

S. A. Prome, N. A. Ragavan, M. R. Islam, D. Asirvatham, and A. J. Jegathesan, “Deception detection using machine learning (ML) and deep learning (DL) techniques: A systematic review,” Natural Language Processing Journal , vol. 6, p. 100057, Mar. 2024, doi: 10.1016/J.NLP.2024.100057

2024
[15]

Decoupled weight de cay regularization,

I. Loshchilov and F. Hutter, “Decoupled weight de cay regularization,” 7th International Conference on Learning Representations, ICLR 2019 , 2019, Accessed: Nov. 29, 2024. [Online]. Available: https://ar5iv.labs.arxiv.org/html/2401.04119

2019 arXiv
[16]

Text Classification by Augmenting Bag of Words (BOW) Representation with Co -occurrence Feature,

S. George K and S. Joseph, “Text Classification by Augmenting Bag of Words (BOW) Representation with Co -occurrence Feature,” IOSR J Comput Eng , vol. 16, no. 1, pp. 34–38, 2014, doi: 10.9790/0661-16153438

2014 doi
[17]

Classification of EEG signals using neural network and logistic regression,

A. Subasi and E. Erçelebi, “Classification of EEG signals using neural network and logistic regression,” Comput Methods Programs Biomed, vol. 78, no. 2, pp. 87–99, May 2005, doi: 10.1016/J.CMPB.2004.10.009

2005 doi
[18]

Gerber, A

N. Gerber, A. Stöver, and K. Marky, Eds., Human Factors in Privacy Research. Cham: Springer International Publishing, 2023. doi: 10.1007/978-3-031-28643-8. d

2023 doi
[2024]

Available: https://arxiv.org/abs/2406.01608v1

[Online]. Available: https://arxiv.org/abs/2406.01608v1

arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Dark Patterns at Scale,

A. Mathur et al., “Dark Patterns at Scale,” Proc ACM Hum Comput Interact, vol. 3, no. CSCW, pp. 1–32, Nov. 2019, doi: 10.1145/3359183

doi:10.1145/3359183 2019

[2] [2]

‘Nothing comes before profit’: Asshole design in the wild,

S. S. Chivukula , C. Watkins, L. McKay, and C. M. Gray, “‘Nothing comes before profit’: Asshole design in the wild,” Conference on Human Factors in Computing Systems - Proceedings, May 2019, doi: 10.1145/3290607.3312863

arXiv 2019

[3] [3]

Ethical mediat ion in UX practice,

C. M. Gray and S. S. Chivukula, “Ethical mediat ion in UX practice,” Conference on Human Factors in Computing Systems - Proceedings, May 2019, doi: 10.1145/3290605.3300408

arXiv 2019

[4] [4]

Wayback Machine

“Wayback Machine.” Accessed: Nov. 29, 2024. [Online]. Available: https://web.archive.org/web/*/https://www.darkpatterns.org/*

work page 2024

[5] [5]

The dark (patterns) side of UX design,

C. M. Gray, Y. Kou, B. Battles, J. Hoggatt, and A. L. Toombs, “The dark (patterns) side of UX design,” Conference on Human Factors in Computing Systems - Proceedings, vol. 2018-April, Apr. 2018, doi: 10.1145/3173574.3174108

arXiv 2018

[6] [6]

#Darkpatterns: UX Practitioner Conversations About Ethical Design,

M. Fansher, S. S. Chivuk ula, and C. M. Gray, “#Darkpatterns: UX Practitioner Conversations About Ethical Design,” Conference on Human Factors in Computing Systems - Proceedings, vol. 2018-April, Apr. 2018, doi: 10.1145/3170427.3188553. 11

arXiv 2018

[7] [7]

Unmasking Dark Patterns: A Machine Learning Approach to Detecting Deceptive Design in E -commerce Websites,

S. T. G. S. & Prof. R. N. B. Arya Ramtek e, “Unmasking Dark Patterns: A Machine Learning Approach to Detecting Deceptive Design in E -commerce Websites,” ArXiv, May 2024

work page 2024

[8] [8]

A Domain -Independent Holistic Approach to Deception Detection,

S. Shahriar, A. Mukherjee, and O. Gnawali, “A Domain -Independent Holistic Approach to Deception Detection,” International C onference Recent Advances in Natural Language Processing, RANLP, pp. 1308–1317, 2021, doi: 10.26615/978-954- 452-072-4_147

work page doi:10.26615/978-954- 2021