PhishBD_2026

Published: 8 July 2026| Version 1 | DOI: 10.17632/8d6zsfwc7z.1
Contributors:
,
,

Description

PhishBD_2026 is a phishing URL detection dataset for Bangladeshi web infrastructure. It contains 131,167 URLs (91,817 phishing, 39,350 legitimate), each represented by 73 engineered features, the raw URL string, and a binary label. Phishing samples were drawn from threat intelligence feeds and incident records provided by a Bangladeshi financial institution. Legitimate samples combine standard sources (Tranco, ISCX, Majestic Million) with manually verified Bangladeshi websites spanning government, banking, mobile financial services (MFS), telecom, education, media, e-commerce, and healthcare. These regional samples were added because global URL datasets underrepresent Bangladeshi domain and brand patterns, which limits the transferability of models trained only on standard sources. The 73 features are organized into six categories: lexical (length, character composition), structural (subdomain and path depth), encoding-related, statistical (entropy, token distributions), security indicators (HTTPS usage, IP-literal hosts, punycode), and features specific to the Bangladeshi context (local brand detection, typosquatting edit distance, BD-TLD identification, government keyword matching). Some features are correlated by construction (for example, slash_count and path_depth, or token_count and special_char_count). These were retained rather than removed, since redundant representations can be useful for some models and are easily filtered out during feature selection for others. The phishing-to-legitimate ratio is approximately 70:30, reflecting the composition of the source feeds and institutional records rather than an artificially balanced sample. Users training classifiers on this dataset should select evaluation metrics accordingly (e.g., precision, recall, or PR-AUC rather than accuracy alone).

Files

Steps to reproduce

Environment: Python 3.10, run in Google Colab without GPU. Main libraries: pandas 2.x, tldextract 3.x, NumPy 1.x, scikit-learn 1.x, tqdm 4.x, openpyxl 3.x. Phishing URL collection: Pulled from four threat intelligence feeds (CSV format, filtered to URL-type indicators only) plus an incident spreadsheet from a partner financial institution. Feed providers are not named, per a confidentiality agreement. URLs from the feeds arrive defanged, so hxxp was converted back to http, bracketed dots and colons were restored, and a scheme was added to any URL missing one. Each URL was parsed with urllib.parse and tldextract to extract domain, subdomain, path, query, and TLD. Legitimate baseline: Started with the top 10,000 Tranco domains, prefixed with http://, and run through the same parsing and feature pipeline as the phishing URLs. Feature engineering: Built 66 features across six passes: lexical and structural (34), entropy and randomness (6), token-level (5), Bangladesh-specific indicators (7), typosquatting and homograph detection (5), and obfuscation/redirection signals (9). Class balancing: The raw collection skewed heavily phishing. Legitimate URLs were added to reach a 70:30 ratio, using n_inject = floor(n_phish × 30/70) − n_legit,existing. Additions came from the full Tranco list, ISCX, Majestic Million filtered to .bd domains, and a hand-compiled list of 260 Bangladeshi sites across government, banking, MFS, telecom, education, media, e-commerce, healthcare, and city corporations, deduplicated against the existing set before merging. Redundancy check: Three features were dropped after auditing the original 66: one constant column, one exact duplicate, one at r ≈ 0.98 with an existing feature, leaving 63. Ten more were added afterward, bringing the total to 73, and the full set was re-audited at a 0.95 correlation threshold, which caught eight correlated pairs. One additional pair correlates at r ≈ 0.93, just under that threshold, and both features were kept since they aren't identical. Final checks: Deduplicated on the URL column, confirmed no missing values, checked outliers past 100x the 99th percentile (none found), dropped two internal-only tracking columns. Final file: 131,167 rows, 75 columns, no missing values, no duplicate URLs.

Institutions

Categories

Cybersecurity, Cyber Attack

Licence