{"id":"84d65de1-1204-4262-b081-39055bd5045c","arxiv_id":"2509.09592","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The authors present a phishing webpage resource collection tool and a dataset of 4,056 legitimate and 5,666 phishing URLs with associated resources, plus a feature correlation analysis.","lead":"This paper introduces a tool that downloads multiple types of resources (HTML, CSS, JavaScript, favicons, images, screenshots) for phishing and legitimate URLs, and shares a new dataset of about 10,000 such pages. It also compares feature correlations between its new dataset and the older UCI phishing dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 4,056 'legitimate' labels have no described provenance: PhishTank is a phishing feed, and the paper never states how legitimate URLs were obtained. If these labels are unreliable, the central dataset claim collapses.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the provenance of the 'legitimate' URLs from PhishTank. I agree that this is the critical point. The paper's central contribution is a labeled dataset; if the legitimate labels cannot be verified, the dataset's utility for training or evaluating phishing detectors is undermined. The reader's conditional verdict is appropriate. I did not find another concern that is more load-bearing: the PyWebCopy comparison and correlation analysis are weaker contributions, but the dataset label quality is what the entire resource rests on. The concrete test I propose directly targets this by inspecting the code and externally validating a sample of the 'legitimate' URLs. If the test passes, the central claim is substantially supported; if it fails, the dataset should be treated as unverified rather than as a reliable benchmark.","tokens_in":7834,"tokens_out":2832,"duration_ms":33628,"concrete_test":"Download the linked GitHub repository, inspect the scraper code, and locate the exact code path that produced the 4,056 'legitimate' URLs. If the only source is PhishTank (e.g., 'invalid' submissions or any PHISHID-based scraping), then sample 200 of the dataset's legitimate URLs and check them against an independent legitimate-URL source such as the Tranco top-1M list, plus a phishing/malware checker (Google Safe Browsing or PhishTank lookup). If a nontrivial fraction (>5%) are unreachable, flagged as malicious, or overlapping with known phishing domains, the 'legitimate' class labels are unreliable and the dataset cannot support the central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the tool provides a comprehensive, trustworthy dataset of 4,056 legitimate and 5,666 phishing webpages. The load-bearing assumption is that the 'legitimate' URLs are genuinely legitimate. Section IV-A says: 'we leverage PhishTank as it is an open-source repository widely adopted in phishing webpage detection research. Additionally, the PhishTank repository comprises phishing and legitimate URLs.' This is factually problematic: PhishTank is a phishing-submission repository; it tracks 'valid' and 'invalid' phish submissions, but it is not a curated source of legitimate URLs. No separate legitimate-URL collection mechanism is described anywhere. Section III's pipeline starts from PHISHID links only, and Section V concedes that PhishTank was used 'as the primary source for collecting active phishing URLs,' with broader sources left to future work. If the 4,056 legitimate URLs were derived from PhishTank's 'invalid' submissions or from some undocumented filtering, the class labels are not independently verified. A model trained on this dataset could be silently poisoned by mislabeled phishing or non-representative URLs. This is not an internal inconsistency in the tool's mechanics; it is a missing justification for the dataset's ground truth. The paper's own text highlights the gap: the legitimate collection pipeline is absent, and no validation against an external legitimate-URL source is reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a Python-based tool that, given a URL, downloads a set of associated webpage resources: HTML source, CSS, JavaScript, favicon, images, and a screenshot. It uses PhishTank as the source of URLs and contributes a dataset of 4,056 legitimate and 5,666 phishing webpages with their associated resources. The paper also reports a feature-correlation analysis comparing the collected dataset with the UCI phishing dataset and claims that the tool fetches more resources than PyWebCopy with a more organized directory structure. The central claims are that the tool is a comprehensive resource collector and that the shared dataset can support phishing-detection research.","tokens_in":8158,"tokens_out":2937,"duration_ms":35524,"significance":"If the dataset labels are valid and the tool reliably captures the claimed resources, this artifact would be useful to the phishing-detection community: it is one of the few collections that includes multiple resource types per URL, and the authors have made both the tool and a sample dataset public. The tool is straightforward and reproducible, and the descriptive feature-correlation analysis is a useful sanity check for dataset drift. However, the paper currently lacks a credible description of how legitimate URLs were obtained, and it does not validate that the downloaded resources are complete or that the downloaded content matches the intended URL. These gaps directly affect the dataset's ground-truth labels and the 'comprehensive resource set' claim, so the contribution cannot be fully assessed until these issues are resolved.","major_comments":[{"comment":"The provenance of the 4,056 'legitimate' URLs is not credibly established. Section IV-A states that PhishTank was used 'as the primary source for collecting legitimate and live phishing URLs,' and Section III asserts that 'the PhishTank repository comprises phishing and legitimate URLs.' This is factually inaccurate: PhishTank is a phishing-submission repository, not a curated source of legitimate URLs. The paper does not describe any separate legitimate-URL collection mechanism, any filtering of PhishTank 'invalid' submissions, or any external validation of the legitimate labels. The conclusion (Section V) even concedes that PhishTank was used only 'as the primary source for collecting active phishing URLs.' The class labels of the entire dataset depend on this point, so the dataset's central contribution is currently unsupported. The authors should specify exactly how the legitimate UR","section":"Sections III, IV-A, V"},{"comment":"The paper does not verify that the downloaded resources are complete or correct. For example, Figure 5 shows that screenshots and favicons are missing for many phishing URLs, and the text attributes some missing screenshots to browser warnings, but there is no systematic report of HTTP status codes, response sizes, content-type checks, or parsing failures. Without per-resource success rates and error analysis, the claim that the tool provides a 'comprehensive resource set' is not substantiated. I recommend reporting, for both classes, how many URLs yielded each resource type, how many failed for each reason category, and whether the downloaded HTML actually corresponds to the requested URL rather than an interstitial warning page.","section":"Section IV-B, Figure 5"},{"comment":"The comparison with PyWebCopy is not convincing as presented. Table II uses symbols without a legend, so it is unclear what the check marks in the 'File Not Found' and 'Content Forbidden' columns mean; the 'Favicon Download' and 'Screenshot Download' columns appear empty for both tools, which is confusing given that the tool does download these resources. The timing comparison is also one single run ('PyWebCopy takes 1537.756 seconds whereas our tool takes 5054.288 seconds for 100 URLs'), with no environment details, variance, or explanation that the lower absolute time is expected because the tool collects additional resources. This does not support the statement that the tool 'outperforms' PyWebCopy. Please provide a clear comparison protocol, a legend for Table II, and per-resource download counts for both tools on the same URL set.","section":"Section II, Table II"}],"minor_comments":[{"comment":"Typo: 'Our too outperforms PyWebCopy' should be 'Our tool outperforms PyWebCopy'.","section":"Section V"},{"comment":"The phrase 'A few distinguishing features distinguishing our tool' is redundant; rephrase.","section":"Section II"},{"comment":"Reference [18] in the conclusion likely should be [24] (the GitHub repository for the dataset); as written, the dataset link points to the PyWebCopy issue tracker.","section":"Section V"},{"comment":"The resource type is inconsistently named: the text says 'Image directory' but Figure 3 and Section III use 'Images.' Please standardize.","section":"Section IV-A"},{"comment":"The check-mark table is unclear: it is not obvious which columns correspond to which repositories, and the 'Sample Class' rows are difficult to parse. A clearer layout with labeled repositories and explicit Yes/No cells would help.","section":"Table I"},{"comment":"The feature-correlation analysis says the same 30 features as UCI were extracted, but no extraction details are given (e.g., who computed the features, which definitions were used, how missing values were handled). This is analysis is presented as a remark rather than a substantive contribution, so please either add detail or clearly label it as illustrative.","section":"Section IV-C"}],"recommendation":"major_revision","confidential_remarks":"The dataset/tool is potentially useful, but the legitimate-URL provenance is a load-bearing flaw that must be fixed before the paper can be accepted. The authors should be asked to supply the actual legitimate-URL collection procedure and independent label validation. The PyWebCopy comparison should also be made quantitative and interpretable. If the authors cannot provide a legitimate-URL source, the paper may need to be reframed as a phishing-URL-only collection tool."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the tool itself is a reasonable engineering artifact: it scrapes a URL and pulls down HTML, CSS, JS, favicon, images, and a screenshot into a clean directory structure. That is genuinely convenient and not something you get out of PyWebCopy in one piece. Second, the dataset's ground truth is shaky in a way that matters. The paper says PhishTank 'comprises phishing and legitimate URLs' and that they used it as the primary source for both classes. PhishTank is a phishing-submission feed; it is not a curated source of legitimate URLs. There is no description anywhere of how the 4,056 legitimate URLs were obtained or vetted. If those labels are wrong, every downstream claim about the dataset collapses, and a model trained on it could be silently poisoned.\n\nWhat the paper does well: it identifies a real gap—most existing repositories give you URLs or content, but not a multi-resource set per URL—and it builds a straightforward solution. The directory structure is sensible, the parsing logic is clearly explained, and the authors share both code and data. The observation that feature correlations shift over time (e.g., URL-of-anchor becoming less predictive, domain-age more predictive) is a nice illustrative point, even if it is descriptive only.\n\nThe soft spots, in order of seriousness:\n1. Legitimate URL provenance. This is the big one. Sections III and IV-A both explicitly say PhishTank is the source for both classes, and that is factually wrong. The stress-test gets this exactly right. No external legitimate source is described, and none of the Table I alternatives (Tranco, Common Crawl) are used.\n2. No validation of downloaded resources. The paper never checks whether a downloaded page actually corresponds to the intended URL, whether resources were complete, or whether the screenshot matched the live page. Some screenshots are missing because the browser flagged pages as dangerous—fine—but there is no systematic audit.\n3. The PyWebCopy comparison is under-specified. The table is ambiguous (checkmarks for 'File Not Found' and 'Content Forbidden' appear for both tools), and the timing comparison is a single run. The authors also claim to 'outperform' PyWebCopy while reporting they are three times slower, which is confusing without a clear definition of 'outperform.'\n4. The correlation analysis has no statistical grounding. It is a figure and a few interpretations, not a test.\n5. The dataset link has no commit hash or versioning, making reproducibility weaker.\n\nWho is this for? Phishing researchers who need a dataset with multiple resource types and are willing to re-verify labels. The tool itself is worth having. The dataset, as labeled, is not something I would train on until the legitimate-URL provenance is fixed.\n\nRecommendation: This deserves a serious referee, but only because the artifact is reproducible and the core idea is sound. A referee would need to force a revision where the legitimate URLs come from a credible source (Tranco or Common Crawl) and are validated. If the paper is accepted as-is, that would be a mistake. I would not cite the dataset in its current form, but I would use the tool.","headline":"The tool is a genuinely useful scraper, but the paper's dataset is undermined by a load-bearing provenance error: the 4,056 'legitimate' labels come from PhishTank, which is not a legitimate-URL source.","tokens_in":8588,"tokens_out":1963,"would_cite":false,"duration_ms":22480,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces a webpage-resource collector that pairs PhishTank-sourced URLs with CSS, JavaScript, favicon, image, and screenshot downloads, and releases a dataset of 4,056 legitimate and 5,666 phishing pages.","keywords":["phishing detection","dataset collection","webpage resources","PhishTank","web scraping","feature correlation","screenshots","CSS/JS resources"],"falsifier":"Take 100 random URLs from the released dataset, check each URL's live status against an independent benign/malicious label source and manual inspection, and re-crawl each page with the tool to compare downloaded CSS, JavaScript, favicon, and screenshot files against the live page's actual linked resources; a material mismatch in either check would refute the contribution.","tokens_in":7782,"feed_emoji":"🎣","tokens_out":7804,"duration_ms":69215,"temperature":0.7,"pith_summary":"The paper aims to close a gap in phishing research: most public datasets supply only URLs or page text, while attackers increasingly hide signals in styling, scripts, favicons, and screenshots. It presents a tool that, given a URL, downloads the landing page's HTML, CSS, JavaScript, favicon, images, and a visible screenshot into a clean folder structure. Using that tool, it compiles a sample dataset of 4,056 legitimate and 5,666 phishing URLs with their associated resources, and compares feature correlations against an older benchmark to show that phishing signals drift over time. The paper argues that this gives researchers both a reproducible collector and a multi-resource dataset for training richer detectors.","feed_headline":"Collector pairs 5,666 phishing URLs with full page resources","feed_subtitle":"New tool and dataset give detectors CSS, JS, favicons, images, and screenshots, not just URLs.","key_machinery":"The tool's pipeline: Selenium-driven scraping of PhishTank page listings to extract URLs; requests to fetch landing-page HTML; BeautifulSoup parsing of