REVIEW 3 major objections 6 minor 26 references
Bridging the Gap in Phishing Detection: A Comprehensive Phishing Dataset Collector
T0 review · 3 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read This paper introduces a webpage-resource collector that pairs PhishTank-sourced URLs with CSS, JavaScript, favicon, image, and screenshot downloads, and releases a dataset of 4,056 legitimate and 5,666 phishing pages.
desk verdict The tool is a genuinely useful scraper, but the paper's dataset is undermined by a load-bearing provenance error: the 4,056 'legitimate' labels come from PhishTank, which is not a legitimate-URL source. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The tool's pipeline: Selenium-driven scraping of PhishTank page listings to extract URLs; requests to fetch landing-page HTML; BeautifulSoup parsing of <script>, <style>, <link>, and <img> tags to discover and download JavaScript, CSS, favicons, and images; and webbrowser-plus-PIL to capture a visible-portion screenshot. Each resource is saved into per-PHISHID directories (HTML, Javascript, CSS, Favicon, Images, Screenshots), which is the design that distinguishes it from PyWebCopy's unstructured output.
What would settle it
Take 100 random URLs from the released dataset, check each URL's live status against an independent benign/malicious label source and manual inspection, and re-crawl each page with the tool to compare downloaded CSS, JavaScript, favicon, and screenshot files against the live page's actual linked resources; a material mismatch in either check would refute the contribution.
Extended reading notes
Core claim
The central claim is that a single tool can assemble, for any URL, the full set of webpage resources that phishing detectors consume — URL, source code, CSS, JavaScript, favicon, images, and screenshot — in a structured layout, and that a dataset built this way is more useful than the URL-only or content-only repositories common today. The paper grounds this in a collection run over PhishTank-sourced URLs, yielding 4,056 legitimate and 5,666 phishing webpages with resources, and in a correlation analysis showing that classic features (anchor URLs, age of domain, URL length, IP address) have shifted weight between the 2015 UCI benchmark and their 2023 collection.
Load-bearing premise
The dataset's class labels hinge on the claim that PhishTank supplies legitimate as well as phishing URLs, but the paper does not describe any separate procedure for collecting or verifying those 4,056 legitimate pages, and PhishTank is primarily a phishing-reporting service.
Editorial extensions
If this is right
- Researchers can use the tool to generate fresh multi-resource datasets on demand instead of relying on static URL-only lists.
- Datasets with CSS, JavaScript, favicon, image, and screenshot resources allow content-based and visual-based detectors to use signals that attackers routinely modify, such as look-alike favicons or CSS-based screenshot replacement.
- The correlation comparison indicates that feature importance changes over time, so detectors trained on older datasets may degrade unless feature selection is revisited.
- The tool's structured directory makes the landing-page source easy to locate, reducing preprocessing overhead, and it can fetch pages where an existing library returns Forbidden, Not Found, or encoded content.
Reading between the lines
- A careful user should independently verify a sample of the 'legitimate' labels, because the paper does not document a separate legitimate-URL workflow and relies on PhishTank for both classes.
- The paper itself notes that some phishing pages are blocked as Dangerous by the browser, so screenshots are missing for those pages, and that only the visible portion of the page is captured; detectors trained on this dataset will need to handle missing visual modalities.
- The temporal drift in feature correlations implies that crawl timestamps and per-page resource-completeness metadata would be valuable additions for future users controlling for collection date.
- The favicon gap between phishing and legitimate pages suggests a cheap, testable signal: a classifier could use favicon presence or favicon-versus-brand match as a weak phishing indicator, though the paper only reports the correlation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a Python-based tool that, given a URL, downloads a set of associated webpage resources: HTML source, CSS, JavaScript, favicon, images, and a screenshot. It uses PhishTank as the source of URLs and contributes a dataset of 4,056 legitimate and 5,666 phishing webpages with their associated resources. The paper also reports a feature-correlation analysis comparing the collected dataset with the UCI phishing dataset and claims that the tool fetches more resources than PyWebCopy with a more organized directory structure. The central claims are that the tool is a comprehensive resource collector and that the shared dataset can support phishing-detection research.
Significance. If the dataset labels are valid and the tool reliably captures the claimed resources, this artifact would be useful to the phishing-detection community: it is one of the few collections that includes multiple resource types per URL, and the authors have made both the tool and a sample dataset public. The tool is straightforward and reproducible, and the descriptive feature-correlation analysis is a useful sanity check for dataset drift. However, the paper currently lacks a credible description of how legitimate URLs were obtained, and it does not validate that the downloaded resources are complete or that the downloaded content matches the intended URL. These gaps directly affect the dataset's ground-truth labels and the 'comprehensive resource set' claim, so the contribution cannot be fully assessed until these issues are resolved.
major comments (3)
- [Sections III, IV-A, V] The provenance of the 4,056 'legitimate' URLs is not credibly established. Section IV-A states that PhishTank was used 'as the primary source for collecting legitimate and live phishing URLs,' and Section III asserts that 'the PhishTank repository comprises phishing and legitimate URLs.' This is factually inaccurate: PhishTank is a phishing-submission repository, not a curated source of legitimate URLs. The paper does not describe any separate legitimate-URL collection mechanism, any filtering of PhishTank 'invalid' submissions, or any external validation of the legitimate labels. The conclusion (Section V) even concedes that PhishTank was used only 'as the primary source for collecting active phishing URLs.' The class labels of the entire dataset depend on this point, so the dataset's central contribution is currently unsupported. The authors should specify exactly how the legitimate UR
- [Section IV-B, Figure 5] The paper does not verify that the downloaded resources are complete or correct. For example, Figure 5 shows that screenshots and favicons are missing for many phishing URLs, and the text attributes some missing screenshots to browser warnings, but there is no systematic report of HTTP status codes, response sizes, content-type checks, or parsing failures. Without per-resource success rates and error analysis, the claim that the tool provides a 'comprehensive resource set' is not substantiated. I recommend reporting, for both classes, how many URLs yielded each resource type, how many failed for each reason category, and whether the downloaded HTML actually corresponds to the requested URL rather than an interstitial warning page.
- [Section II, Table II] The comparison with PyWebCopy is not convincing as presented. Table II uses symbols without a legend, so it is unclear what the check marks in the 'File Not Found' and 'Content Forbidden' columns mean; the 'Favicon Download' and 'Screenshot Download' columns appear empty for both tools, which is confusing given that the tool does download these resources. The timing comparison is also one single run ('PyWebCopy takes 1537.756 seconds whereas our tool takes 5054.288 seconds for 100 URLs'), with no environment details, variance, or explanation that the lower absolute time is expected because the tool collects additional resources. This does not support the statement that the tool 'outperforms' PyWebCopy. Please provide a clear comparison protocol, a legend for Table II, and per-resource download counts for both tools on the same URL set.
minor comments (6)
- [Section V] Typo: 'Our too outperforms PyWebCopy' should be 'Our tool outperforms PyWebCopy'.
- [Section II] The phrase 'A few distinguishing features distinguishing our tool' is redundant; rephrase.
- [Section V] Reference [18] in the conclusion likely should be [24] (the GitHub repository for the dataset); as written, the dataset link points to the PyWebCopy issue tracker.
- [Section IV-A] The resource type is inconsistently named: the text says 'Image directory' but Figure 3 and Section III use 'Images.' Please standardize.
- [Table I] The check-mark table is unclear: it is not obvious which columns correspond to which repositories, and the 'Sample Class' rows are difficult to parse. A clearer layout with labeled repositories and explicit Yes/No cells would help.
- [Section IV-C] The feature-correlation analysis says the same 30 features as UCI were extracted, but no extraction details are given (e.g., who computed the features, which definitions were used, how missing values were handled). This is analysis is presented as a remark rather than a substantive contribution, so please either add detail or clearly label it as illustrative.
Circularity Check
No circular derivation: the paper is a descriptive data-collection tool report; the only self-reference is a data-availability citation that is not load-bearing.
full rationale
The paper contains no predictive or derivational chain that reduces to its own inputs. Its pipeline takes PHISHID links from PhishTank, fetches webpage resources, organizes them into folders, and reports download statistics and feature correlations. The feature-correlation analysis in Section IV-C is computed directly from the collected dataset and is descriptive; it is not used to fit a parameter, to generate a prediction, or to justify the tool's design. The comparison to PyWebCopy is a qualitative feature contrast, not a fitted result. The only self-reference is citation [24], which points to the authors' own GitHub repository hosting the tool and dataset; it is a data-availability pointer, not an argument that derives the paper's claims. The manuscript does identify a limitation in Section V, acknowledging that PhishTank is used only as the primary source for active phishing URLs and that broader sources are left to future work; the provenance of the 4,056 'legitimate' URLs is indeed undocumented (Section IV-A implies PhishTank itself provides legitimate URLs, which is questionable). That is a data-validity and reproducibility concern, not a circularity. No equation is defined in terms of a target, no fitted input is renamed a prediction, and no uniqueness or ansatz is imported from self-citation. Accordingly, the circularity score is 0.
Assumptions & free parameters
assumptions (3)
- domain assumption PhishTank provides accurate labels for phishing and legitimate URLs.
- domain assumption The UCI 30-feature set is applicable to the collected dataset.
- domain assumption The download process captures all relevant resources without significant loss.
Cite this review
Pith. "Pith review of Bridging the Gap in Phishing Detection: A Comprehensive Phishing Dataset Collector." pith.science (2026). https://pith.science/paper/AMAAWROT
@misc{pith2026250909592,
author = {Pith},
title = {Pith review of: Bridging the Gap in Phishing Detection: A Comprehensive Phishing Dataset Collector},
year = {2026},
howpublished = {\url{https://pith.science/paper/AMAAWROT}},
note = {Machine review of arXiv:2509.09592}
}
read the original abstract
To combat phishing attacks -- aimed at luring web users to divulge their sensitive information -- various phishing detection approaches have been proposed. As attackers focus on devising new tactics to bypass existing detection solutions, researchers have adapted by integrating machine learning and deep learning into phishing detection. Phishing dataset collection is vital to developing effective phishing detection approaches, which highly depend on the diversity of the gathered datasets. The lack of diversity in the dataset results in a biased model. Since phishing websites are often short-lived, collecting them is also a challenge. Consequently, very few phishing webpage dataset repositories exist to date. No single repository comprehensively consolidates all phishing elements corresponding to a phishing webpage, namely, URL, webpage source code, screenshot, and related webpage resources. This paper introduces a resource collection tool designed to gather various resources associated with a URL, such as CSS, Javascript, favicons, webpage images, and screenshots. Our tool leverages PhishTank as the primary source for obtaining active phishing URLs. Our tool fetches several additional webpage resources compared to PyWebCopy Python library, which provides webpage content for a given URL. Additionally, we share a sample dataset generated using our tool comprising 4,056 legitimate and 5,666 phishing URLs along with their associated resources. We also remark on the top correlated phishing features with their associated class label found in our dataset. Our tool offers a comprehensive resource set that can aid researchers in developing effective phishing detection approaches.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Digital 2023 July Global Statshot Re- port,
Meltwater, “Digital 2023 July Global Statshot Re- port,” 2023, https://www.meltwater.com/en/resources/ digital-2023-july-global-statshot-report [Accessed: September 23rd, 2023]
2023
-
[2]
What is phishing?
I. Newsletter, “What is phishing?” 2023, https://www.ibm.com/topics/ phishing [Accessed: September 23rd, 2023]
2023
-
[3]
The 5 Biggest Phishing Scams of All Time - IT Governance Blog En,
L. Irwin, “The 5 Biggest Phishing Scams of All Time - IT Governance Blog En,” 2023, https://www.itgovernance.eu/blog/en/ the-5-biggest-phishing-scams-of-all-time [Accessed: September 23rd, 2023]
2023
-
[4]
Phishing Activity Trends Report,
APWG, “Phishing Activity Trends Report,” 2022, https://docs.apwg.org/ reports/apwg trends report q4 2022.pdf [Accessed: September 23rd, 2023]
2022
-
[5]
Anti-phishing based on automated indi- vidual white-list,
Y . Cao, W. Han, and Y . Le, “Anti-phishing based on automated indi- vidual white-list,” inProceedings of the 4th ACM workshop on Digital identity management, 2008
2008
-
[6]
Beyond blacklists: learning to detect malicious web sites from suspicious URLs,
J. Ma, L. K. Saul, S. Savage, and G. M. V oelker, “Beyond blacklists: learning to detect malicious web sites from suspicious URLs,” in Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, 2009
2009
-
[7]
Cantina: a content-based approach to detecting phishing web sites,
Y . Zhang, J. I. Hong, and L. F. Cranor, “Cantina: a content-based approach to detecting phishing web sites,” inProceedings of the 16th international conference on World Wide Web, 2007
2007
-
[8]
Phishzoo: Detecting phishing websites by looking at them,
S. Afroz and R. Greenstadt, “Phishzoo: Detecting phishing websites by looking at them,” inIEEE fifth international conference on semantic computing, IEEE, 2011, pp. 368–375
2011
Show all 26 references
-
[9]
PhishAlert: An Efficient Phishing URL Detection via Hybrid Methodology,
B. Sharma and P. Singh, “PhishAlert: An Efficient Phishing URL Detection via Hybrid Methodology,”International Journal of Innovative Technology and Exploring Engineering (IJITEE), vol. 8, 2019
2019
-
[10]
A machine learning based approach for phishing detection using hyperlinks information,
A. K. Jain and B. B. Gupta, “A machine learning based approach for phishing detection using hyperlinks information,”Journal of Ambient Intelligence and Humanized Computing, Springer, vol. 10, no. 5, pp. 2015–2028, 2019
2015
-
[11]
URLNet: Learning a URL representation with deep learning for malicious URL detection,
H. Le, Q. Pham, D. Sahoo, and S. C. Hoi, “URLNet: Learning a URL representation with deep learning for malicious URL detection,”arXiv preprint arXiv:1802.03162, 2018
2018 arXiv
-
[12]
Visualphishnet: Zero- day phishing website detection by visual similarity,
S. Abdelnabi, K. Krombholz, and M. Fritz, “Visualphishnet: Zero- day phishing website detection by visual similarity,” inProceedings of the 2020 ACM SIGSAC conference on computer and communications security, 2020
2020
-
[13]
Phishpedia: A hybrid deep learning based approach to visually identify phishing webpages
Y . Lin, R. Liu, D. M. Divakaran, J. Y . Ng, Q. Z. Chan, Y . Lu, Y . Si, F. Zhang, and J. S. Dong, “Phishpedia: A hybrid deep learning based approach to visually identify phishing webpages.” inUSENIX Security Symposium, 2021
2021
-
[14]
Comprehensible classification models: a position paper,
A. A. Freitas, “Comprehensible classification models: a position paper,” ACM SIGKDD explorations newsletter, vol. 15, no. 1, pp. 1–10, 2014
2014
-
[15]
Top Sites Ranking,
S. T. M. K. Victor Le Pochat, Tom Van Goethem and W. Joosen, “Top Sites Ranking,” https://tranco-list.eu/ [Accessed: June 22nd, 2023]
2023
-
[16]
OpenPhish,
“OpenPhish,” 2022, https://openphish.com/ [Accessed: June 22nd, 2023]
2022
-
[17]
MillerSmiles,
M. Bright, “MillerSmiles,” 2003, http://www.millersmiles.co.uk/ [Ac- cessed: June 22nd, 2023]
2003
-
[18]
PyWebCopy, GitHub,
R. Tomar, “PyWebCopy, GitHub,” 2023, https://github.com/ rajatomar788/pywebcopy/issues [Accessed: September 23rd, 2023]
2023
-
[19]
Alexa Top Sites,
Amazon, “Alexa Top Sites,” http://www.alexa.com/topsites
-
[20]
Common Crawl,
E. Sayegh, “Common Crawl,” http://index.commoncrawl.org/ [Accessed: June 22nd, 2023]
2023
-
[21]
PhishTank,
C. T. I. Group, “PhishTank,” 2006, http://www.phishtank.com/developer [Accessed: June 22nd, 2023]
2006
-
[22]
UCI1 Benchmark Dataset,
L. M. Rami Mohammad, “UCI1 Benchmark Dataset,” 2015, https:// archive.ics.uci.edu/ml/datasets/phishing+websites [Accessed: June 22nd, 2023]
2015
-
[23]
Mendeley,
S. F. S. Ariyadasa, Subhash; Fernando, “Mendeley,” 2021, https://data. mendeley.com/datasets/n96ncsr5g4 [Accessed: June 22nd, 2023]
2021
-
[24]
Bridging the Gap in Phishing Detection: A Comprehensive Phishing Dataset Collector,
A. K. V . B. Patel Shahil Manishbhai, Shivam Pradip Tirmare and T. Das, “Bridging the Gap in Phishing Detection: A Comprehensive Phishing Dataset Collector,” 2023, [Available at:] https://github.com/ ShahilPatel-IITDh/scrapio
2023
-
[25]
BeautifulSoup,
L. Richardson, “BeautifulSoup,” 2004, https://pypi.org/project/ beautifulsoup4/ [Accessed: September 23rd, 2023]
2004
-
[26]
Modeling hybrid feature-based phishing websites detection using machine learning techniques,
S. Das Guptta, K. T. Shahriar, H. Alqahtani, D. Alsalman, and I. H. Sarker, “Modeling hybrid feature-based phishing websites detection using machine learning techniques,”Annals of Data Science, Springer, pp. 1–26, 2022
2022
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.