This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2024.Doi Number
Evaluating the Impact of Feature Engineering in
Phishing URL Detection: A Comparative Study
of URL, HTML, and Derived Features
Yanche Ari Kustiawan and Khairil Imran Ghauth
Faculty of Computing and Informatics, Multimedia University, Cyberjaya 63000, Malaysia
Corresponding author: Khairil Imran Ghauth (e-mail: khairil-imran@mmu.edu.my).
ABSTRACT Phishing attacks have evolved into sophisticated threats, making effective cybersecurity
detection strategies essential. While many studies focus on either URL or HTML features, limited work has
explored the comparative impact of engineered feature sets across different machine learning models. This
study aims to bridge that empirical gap by evaluating the effectiveness of URL-based, HTML-based, and
derived features, individually and in combination, on phishing URL detection. The proposed approach utilizes
the PhishOFE dataset of 101,063 phishing and legitimate URLs. Features are organized into four sets: (1)
URL only, (2) HTML only, (3) URL + HTML, and (4) URL + HTML + derived features. Ten machine
learning models are employed, including Random Forest, k-Nearest Neighbors, Logistic Regression, Support
Vector Machine, Naive Bayes, and advanced ensemble methods such as LightGBM, XGBoost, and CatBoost.
Performance is assessed using accuracy, precision, recall, and F1-score, while permutation importance is used
to evaluate feature significance. Experimental results demonstrate that ensemble models outperform
traditional classifiers, with CatBoost achieving the highest accuracy of 99.45% using the complete feature
set. Moreover, URL features like URLLength and NoOfSubDomain consistently rank high in importance,
while derived features such as SuspiciousCharRatio and URLComplexityScore notably enhance detection
performance in specific models.
INDEX TERMS Phishing URL detection, machine learning, feature engineering, URL features, HTML
features, derived features, ensemble learning.
I. INTRODUCTION
Phishing attacks continue to represent the most pervasive
and effective cyber threats confronting individuals and
organizations. This attack primarily aims to deceive users
into divulging sensitive information such as credentials,
financial data, and personal details by impersonating
legitimate websites [1]. The attacker sends a fake message
by propagating over e-mail, short message service (SMS),
social media platforms, VoIP, and mobile messaging apps
[2] containing deceptive links disguised as legitimate
messages from retailers, banks, delivery services,
government agencies, or known brands. Moreover, online
banking and fintech made financial institutions a main
target, and attackers developed a website that looked
similar to an actual bank website [3] [4] but was designed
to steal users' login credentials and obtain access to the
user's financial accounts.
The Anti-Phishing Working Group (APWG) released a
report highlighting phishing activity during the first half of
2024. In Q1 2024, there were 963,994 phishing incidents
recorded [5], predominantly targeting social media
platforms, representing 37.4% of all attacks, followed by
software-as-a-service (SaaS) and webmail providers at
21%. Interestingly, attacks on the banking sector declined
to 9.8%. In Q2 2024 [6], phishing incidents slightly
decreased to 877,536. During this quarter, SaaS and
webmail companies became primary targets, accounting for
25.6%, while attacks on financial institutions slightly
increased to 10%.
Additionally, the growing adoption of smartphones has
led to a rise in mobile phishing (vishing and smishing), with
attackers frequently distributing malicious URLs through
SMS and mobile messaging apps to circumvent traditional
email security measures. In Q3 of 2024 [7], APWG
observed a renewed increase to 932,923 phishing attacks,
up from 877,536 in Q2. Social media platforms remained
the most frequently targeted sector, accounting for 30.5%
VOLUME XX, 2017
1
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
of all phishing attacks. Notably, there was a significant rise
in smishing, with phishing via SMS and text messages
increasing by over 22%.
Furthermore, attackers have started using more
sophisticated tactics, such as sending personalized emails
featuring Google Street View images of victims' homes.
Moreover, Gmail accounts became the primary tool for
perpetrators of Business Email Compromise (BEC) scams,
utilized in 83.1% of such incidents. In the Phishing threat
trends report [8], Egress indicates that in 2024, phishing
attacks have significantly evolved due to the increasing
commoditization of AI and deepfake technologies, with
74.8% of phishing toolkits examined referencing AI and
82.0% mentioning deepfakes. Commodity phishing attacks
saw a remarkable increase, with organizations experiencing
a 2,700% rise in phishing emails during campaigns
compared to normal baseline levels. For instance, an
organization with 2,000 employees received approximately
7,382 phishing emails in 31 days. Advanced Persistent
Threats (APTs) also remain significant, with 52.5%
identified as zero-day attacks.
learning models have demonstrated strong capabilities in
distinguishing phishing URLs from legitimate ones,
showcasing their potential as practical solutions to mitigate
phishing threats [12] [13].
Furthermore, machine learning often relies on feature
engineering to help models learn the patterns and
relationships more effectively, thus resulting in better
accuracy and generalization. Feature engineering is the
process of selecting, transforming, or creating features
from raw data to improve the performance of machine
learning algorithms. This means accurate feature extraction
and selection, which are fundamental to effectively
combating phishing attacks. This study conducts a
comparative analysis of URL-based, HTML-based, and
derived features due to each category's distinct strengths
and detection capabilities. Relying solely on URL features
may miss sophisticated phishing attempts that appear
legitimate on the surface, and HTML-based features will
delve deeper into a webpage's structural and content aspects.
However, the attackers constantly adapt, employing
sophisticated tactics that may evade traditional detection
techniques. Hence, derived features become increasingly
crucial as well, thus enabling security practitioners to adopt
a layered and adaptive approach to securing digital
interactions against phishing attacks.
The rest of this paper is organized as follows: Section 2
provides a comprehensive literature review of related work.
Section 3 outlines the proposed method for comparing the
mentioned features of engineering using machine learning.
Section 4 describes the experimental design and setup.
Section 5 reports the results and discusses the findings from
the experimental evaluation. Finally, Section 6 provides the
conclusion and suggests possible directions for future
research.
II. LITERATURE REVIEW
FIGURE 1. Diagram of how phishing attack works.
Techniques such as typosquatting and combosquatting
are frequently utilized by attackers to manipulate URLs.
Typosquatting involves intentional spelling errors or
inserting unnecessary punctuation marks, such as
hyphens—for instance, using "purcase" instead of
"purchase" [9]. Combosquatting combines a trusted brand
with additional misleading keywords, creating deceptive
URLs
like
"applepay-members[.]com"
or
"facebookfriends[.]com" [10]. Additionally, attackers
occasionally substitute English characters with visually
similar characters from different alphabets to further
deceive users.
Research on phishing website detection has significantly
progressed over recent years, attracting considerable
attention from the academic community. Machine learning
methods, in particular, have been widely adopted to
enhance phishing website detection [11]. Multiple machine
This section summarizes the key findings and features
engineering applied to each machine learning-based method
for detecting the phishing URL and gaining valuable insights
that reveal gaps and opportunities that drive the research
forward.
Gupta et al. [14] proposed a lexical-based machine learning
approach for real-time phishing URL detection that minimizes
computational resources by utilizing only nine lexical features.
These features include the number of tokens in the domain,
number of top-level domains, URL length, number of dots in
the URL, number of delimiters in the domain and path, length
of the longest token in the path, number of digits in a query,
and domain length. They employed several machine learning
classifiers, including Random Forest, k-Nearest Neighbors
(kNN), Logistic Regression, and Support Vector Machine
(SVM). The Random Forest model achieved the highest
accuracy of 99.57% on the ISCXURL-2016 dataset. The study
has limitations such as dependency only on lexical features,
potentially overlooking sophisticated phishing techniques that
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
mimic legitimate URLs closely, and not considering dynamic
webpage features.
Soma Charan et al. [15] proposed a URL-based approach
for phishing website detection utilizing machine learning
techniques. Their methodology extracted ten distinct URL
features from a balanced dataset of 6000 URLs (split evenly
between phishing URLs from PhishTank and legitimate URLs
from Alexa). They applied eight machine learning models,
including Random Forest, Logistic Regression, SVM, and
Multi-layer Perceptron (MLP). Among these, the MLP model
achieved the highest accuracy of 85.41% and an F1 score of
85.17%, outperforming the other algorithms. However, the
moderate accuracy reflects limitations in their approach,
particularly regarding the relatively small dataset and limited
number of URL-based features. Their model did not
incorporate dynamic web pages or HTML content analysis.
Jagdale and Chavan [16] proposed a hybrid ensemble
machine learning approach for detecting phishing URLs by
analyzing URL-based features. Their approach involved a
comparative analysis of standard machine learning algorithms,
including Decision Tree, Naive Bayes, Logistic Regression,
and a Hybrid Ensemble Stacking algorithm, using a dataset
composed of phishing URLs from PhishTank and legitimate
URLs collected from everyday browsing, totaling 3000 URLs.
They extracted nine distinct URL features, like URL length,
presence of HTTPS, suspicious symbols, and phishing-related
terms. The Hybrid Ensemble Stacking algorithm yielded the
highest accuracy at 89.25%, outperforming Decision Tree
(87.04%), Logistic Regression (86.18%), and Naive Bayes
(85.59%). Despite achieving promising results, the approach
had limitations, including the relatively small dataset and the
reliance exclusively on URL-based features, potentially
overlooking dynamic website behaviors and HTML-based
indicators that may provide deeper insights into detecting
sophisticated phishing attacks.
Dantwala et al. [17] proposed a novel approach for phishing
URL detection by evaluating the impact of feature counts on
classification accuracy. They experimented with different
feature subsets, explicitly selecting the top 5, 10, and 15
features from an original set of 88 features, including URLbased, content-based, and external features. The machine
learning models applied in their study were Logistic
Regression, SVM, Decision Tree, Random Forest, and Neural
Networks. Among these, the neural network model trained
with all 88 features achieved the highest performance,
reaching an accuracy, precision, recall, and F1-score of 98.5%,
100%, 100%, and 100%, respectively. Their study suggests
that the feature importance technique did not fully succeed in
identifying a significantly smaller set of features capable of
matching the accuracy obtained with the complete feature set.
Jain et al. [18] proposed an efficient machine learningbased approach named APuML (Anti Phishing using
Machine Learning), specifically aimed at detecting phishing
web pages on mobile devices. They extracted 34 features,
categorized into mobile-specific, JavaScript, HTML, URL,
website-specific, fake form, and Naive Bayes-based
probability features. The study compared various classifiers,
including Logistic Regression, kNN, SVM, Decision Tree,
Neural Network, and Random Forest. The Random Forest
classifier emerged as the best performer, achieving a high
detection accuracy of 93.85%. Despite the promising results,
the proposed approach had limitations, primarily its
dependency on third-party services like Alexa for website
ranking information, potentially affecting its real-time
applicability and accuracy for newly created legitimate
websites.
Rashid et al. [19] proposed a phishing detection approach
using machine learning, emphasizing effective feature
selection to enhance detection performance. They utilized a
combination of internal and external features derived from the
URL and HTML source code, extracting and then reducing
these features using Principal Component Analysis (PCA).
Initially starting with 33 features, PCA reduced the feature set
to about 22.5% of the original features, which equates to
around seven features for their final classification model.
Various machine learning classifiers, including Random
Forest and SVM, were employed, with the SVM classifier
showing the highest accuracy of 95.66%. While their
approach demonstrated significant accuracy improvements, it
is limited by its reliance on principal component analysis,
which may omit specific features that are important for
distinguishing sophisticated phishing websites.
The fusion-based model introduced by Lakshmanarao et al.
[20]. They started with a dataset consisting of 30 features
from the UCI repository that contained 11,055 samples, with
7,738 samples used for training and 3,317 samples used for
testing, covering categories like URL structure, webpage
content, and external attributes such as web traffic and
domain registration length. Feature selection was performed
using ANOVA F-value and Mutual Information methods,
resulting in the removal of nine less relevant features. After
this feature reduction, they trained machine learning models
using the remaining 21 features. Various classification
algorithms were tested, including Logistic Regression,
Decision Tree, Random Forest, AdaBoost, Gradient Boosting,
and SVM. The best individual model was Random Forest,
with a 97% True Positive Rate and 95.9% True Negative Rate.
Their fusion model, combining Random Forest, Decision
Tree, and Gradient Boosting classifiers using priority-based
stacking techniques, achieved an overall accuracy of 97%.
Sinthiya et al. [21] proposed a phishing website detection
approach using machine learning, analyzing a dataset
obtained from Kaggle containing 95,910 samples with 11
features. The features include Domain, Ranking, isIP,
Validity, Active Duration, URL Length, Presence of '@'
Symbol, Redirects, Dashes in Domain Name, Domain Length,
and Number of Subdomains. They applied six machine
learning classifiers, such as Random Forest, Gradient
Boosting, kNN, Decision Tree, SVM, and Logistic
Regression. Among these, Random Forest achieved the
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
highest accuracy of 96.52%, outperforming the other models.
However, the study had limitations, including using only
URL-based features without considering HTML or contentbased features, which may limit its robustness against
sophisticated phishing techniques.
Hu et al. [22] proposed a multi-feature stacking approach
for phishing website detection, incorporating URL and
HTML-based features. Their dataset consisted of 50,000 and
100,000 samples, where legitimate websites were sourced
from Alexa, and phishing websites were obtained from
PhishTank. They extracted 9 URL-based features, including
IP presence, URL length, auspicious symbols, redirections,
subdomain count, and active time. Additionally, they
introduced a Transformer-based HTML string embedding
feature using BERT, which converted webpage HTML
content into a multi-dimensional vector representation. The
study applied a stacking ensemble model consisting of Deep
Forest, XGBoost, and LightGBM as base learners, with
Gradient Boosting Decision Tree (GBDT) as the second-layer
learner. Their BERT-Stacking model achieved an accuracy of
98.52% and an F1-score of 98.81%.
Murshid et al. [23] proposed StackedPhish, a stacked
ensemble framework for phishing website detection, focusing
on lexical-based features. They utilized the "Malicious Phish"
dataset from Kaggle, which contains 300,000 unique
instances with two initial features: URL and type (phishing or
legitimate). The study extracted 40 URL-based features and
applied three feature selection techniques (Chi-Square Test,
Correlation of Features, and Variance Threshold), resulting in
a final selection of 31 features categorized into URL
obfuscation features, third-party-based features, and
hyperlink-based features. The study employed multiple
machine learning models for classification, including
Random Forest, XGBoost, Decision Tree, kNN, Gradient
Boosting, Bagging, Voting, and a Stacking Classifier. The
Stacking Classifier achieved the highest accuracy of 99.88%
for binary classification and 99.39% for multi-class
classification, outperforming individual models. However,
despite its high accuracy, the study had limitations,
particularly in real-time detection feasibility, as it heavily
relied on lexical features and lacked HTML or behavioral
analysis.
Karim et al. [24] introduced a hybrid machine learningbased phishing detection system utilizing URL features. The
study used a dataset containing 11,054 samples and 33
features obtained from Kaggle, with features categorized into
URL structure, domain properties, and external attributes.
The authors applied multiple machine learning models,
including Decision Tree, Linear Regression, Naive Bayes,
Random Forest, Gradient Boosting Machine, Support Vector
Classifier (SVC), and kNN. They introduced a hybrid
ensemble model (LSD), combining Logistic Regression,
SVM, and Decision Tree using soft and hard voting. The
proposed LSD model outperformed other classifiers,
achieved 98.12% accuracy, 97.31% precision, 96.33% recall,
96.55% specificity, and 95.89% F1-score. The canopy feature
selection method was employed to select relevant features,
along with cross-validation and hyperparameter tuning using
Grid Search. However, the study's limitation lies in its
dependence on URL-based features, neglecting HTML and
webpage behavior-based indicators, which might limit its
ability to detect advanced phishing techniques.
Zhu et al. [25] proposed OFS-NN, an effective phishing
website detection model based on optimal feature selection
and neural networks. Their dataset consisted of two sources:
a UCI phishing dataset (11,055 samples) and a custom dataset
compiled from PhishTank and Alexa (14,582 samples). The
study extracted 30 features, categorized into Address Bar
Features, Abnormal Features, HTML and JavaScript Features,
and Domain Features. A novel Feature Validity Value (FVV)
index was introduced to select the most influential features,
reducing overfitting and improving model efficiency. The
optimal feature selection algorithm was applied to prune
insignificant features, leading to a final selection of 28
features. A 7-layer neural network was used for classification,
achieving 98.49% accuracy on the optimized feature set.
Despite its strong performance, the study's limitations include
computational overhead from training deep networks and a
lack of real-time adaptability for newly emerging phishing
techniques.
Pandey and Mishra [26] introduced Phish-Sight, a novel
phishing detection approach that leverages dominant colors
on web pages and optical character recognition (OCR) to
identify phishing websites. The dataset consisted of 3,500
phishing URLs from OpenPhish and 2,700 legitimate URLs
from Alexa, totaling 6,200 samples. Phish-Sight extracted 26
features, including eight dominant RGB colors from the
webpage and 18 brand name features using OCR. Five
machine learning models were evaluated, including Random
Forest, Decision Tree, Logistic Regression, Naive Bayes, and
SVM. Random Forest achieved the highest accuracy of
99.13% and a true positive rate of 98.43%. Despite its high
accuracy, the main limitations of Phish-Sight include high
computational costs due to image processing and OCR
techniques, which may not be ideal for real-time phishing
detection.
Ahammad et al. [27] introduced a phishing URL detection
approach using machine learning methods, focusing on URLbased and domain-based features. The dataset consisted of
3,000 URLs, including 1,500 malicious URLs from
PhishTank and 1,500 legitimate URLs from the University of
New Brunswick dataset. A total of 15 features were extracted
and categorized into Address Bar Features (e.g., presence of
IP address, '@' symbol, URL length, prefix/suffix in the
domain, and HTTPS usage) and Domain-based Features (e.g.,
web traffic, domain age, number of subdomains, and DNS
records). The study applied Random Forest, Decision Tree,
LightGBM, Logistic Regression, and SVM for classification.
Among these, LightGBM achieved the highest accuracy of
86.0%, followed by Random Forest (85.3%) and Decision
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
Tree (85.0%). The main limitation of this study was its small
dataset size, which may impact generalizability, and the lack
of HTML-based and behavioral features, potentially making
the approach less effective against advanced phishing attacks.
Nagunwa et al. [28] proposed a machine learning approach
for detecting fast-flux phishing hostnames, focusing on
phishing websites hosted in Fast Flux Service Networks
(FFSNs). Their methodology introduced a novel set of 56
features derived from DNS, host, and network characteristics,
with 41 newly proposed features. The features were
categorized into temporal, spatial, DNS-based, networkrelated, host-related, and reputation-based features. The study
formulated the problem as binary and multi-class
classification tasks and applied flat and hierarchical
classification techniques. The models tested included
traditional machine learning algorithms (Random Forest,
Gradient Boosting, Decision Tree, and SVM) and deep
learning models (Fully Connected Deep Neural Networks,
LSTM, and 1D CNN). Among these, Random Forest
achieved the highest accuracy of 98.42% for binary
classification and 97.81% for multi-class classification. Their
results demonstrated that DNS-based and temporal features
were the strongest predictors, whereas network and hostrelated features were weaker. However, the study had some
limitations, particularly in real-time applicability, as the
proposed model had a higher-than-desirable prediction time.
PhishNot, introduced by Alani and Tawfik [29], is a cloudbased machine learning approach for phishing URL detection.
Their dataset comprised 88,646 URLs sourced from
PhishTank (phishing URLs) and Alexa (legitimate URLs),
with 111 extracted features. Feature selection was performed
using Recursive Feature Elimination (RFE), reducing the
feature count to 14 features, including URL-based, domainbased, and network-based attributes. The study experimented
with Random Forest, Logistic Regression, Decision Tree,
Gaussian Naive Bayes, and MLP classifiers. The Random
Forest classifier achieved the highest accuracy of 97.5%,
outperforming the other models. The approach was deployed
as a cloud-based API, enabling real-time phishing detection
with an average processing time of 11.5 microseconds per
URL. However, PhishNot has limitations, particularly its
reliance on external features such as domain age and network
response time, which may not always be available, affecting
real-time performance. Additionally, while the study
demonstrated high accuracy, it did not incorporate HTMLbased features, potentially making it less effective against
sophisticated phishing attacks that mimic legitimate web
pages.
Ding et al. [30] introduced a method called Search &
Heuristic Rule & Logistic Regression (SHLR) that achieved
an accuracy of 98.9% with a logistic regression classifier.
Their method consists of three phases: the first phase is search
engine-based filtering, where a webpage is considered
legitimate if its domain appears in the top 10 search results
when querying its title tag on the Baidu search engine. The
second phase is heuristic rule-based detection, which applies
seven heuristic rules to identify obfuscation techniques used
in phishing URLs, and lastly, logistic regression-based
classification, which uses machine learning to analyze
unidentified webpages after the first two steps. The study
extracted 37 lexical features, 7 DNS-based features, 17
HTML-based features, 10 WHOIS-based features, and
phishing vocabulary similarity features, totaling 72 features.
However, the SHLR approach has limitations, including its
dependency on search engine accessibility and potential false
negatives if legitimate websites are not indexed correctly.
The PhishOFE introduced by Kustiawan and Ghauth,
presented a machine learning-based phishing URL detection
framework that optimizes feature engineering to enhance
detection accuracy while minimizing reliance on third-party
data. The dataset comprises 101,063 phishing and legitimate
URLs sourced from PhishTank, OpenPhish, and Open
PageRank. A total of three categories of features were
extracted: URL-based features, HTML-based features, and
derived features. Key derived features include Suspicious
Character Ratio, URL Complexity Score, HTML Content
Density, and Interactive Element Density. The study
evaluated ten machine learning models, including Random
Forest, kNN, Logistic Regression, SVM, Naive Bayes,
Gradient Boosting Machine (GBM), LightGBM, CatBoost,
and XGBoost. CatBoost achieved the highest accuracy of
99.46%, outperforming other models in precision, recall, and
F1-score. The framework was optimized using Boruta feature
selection and GridSearchCV for hyperparameter tuning.
Despite extensive research efforts, there remains an
empirical gap in systematically evaluating the role of feature
engineering, particularly derived features, in improving
phishing website detection accuracy across different machine
learning models. Many existing studies focus solely on URLbased or HTML-based features, often neglecting to
empirically validate how engineered features contribute to
model robustness detection capability. This study aims to
address this gap by conducting a comprehensive feature
importance analysis across various feature categories and
evaluating their impact on detection performance using stateof-the-art machine learning models.
III. PROPOSED METHOD
In this section, we present the workflow of the proposed
research, as illustrated in Figure 2. This study is a continuity
from our previous work, where the PhishOFE dataset will be
used in this study. The next step is identifying features, where
different sets of features are assessed through a series of
controlled experiments. The feature evaluation phase consists
of four experiments: (1) URL-based features, (2) HTMLbased features, (3) a combination of URL and HTML features,
and (4) URL, HTML, and derived features. This structured
approach allows for a comparative analysis of how different
feature sets impact phishing detection performance.
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
Multiple classification models are trained and evaluated,
including Random Forest (RF), Decision Tree (DT), kNN,
GBM, Logistic Regression (LR), LightGBM, SVM,
CatBoost, Naive Bayes (NB), and XGBoost. Each model is
tested using different feature sets to determine the most
compelling feature combinations and algorithms for phishing
detection.
Once classification models are trained, the model
evaluation phase uses standard performance metrics such as
accuracy, precision, recall, and F1-score. Finally, a feature
importance analysis is performed to identify the most
influential features contributing to phishing detection
accuracy. This step helps refine the model by focusing on the
most discriminative attributes, improving efficiency.
TABLE 1. Data dictionary of PhishOFE.
Feature Name
URL
IsHTTPS
IsDomainIP
TLD
URLLength
NoOfSubDomain
NoOfDots
NoOfObfuscatedChar
NoOfEqual
NoOfQmark
NoOfAmp
NoOfDigits
LineLength
HasTitle
HasMeta
HasFavicon
HasExternalFormSubmit
HasCopyright
HasSocialNetworking
HasPasswordField
HasSubmitButton
HasKeywordBank
HasKeywordPay
HasKeywordCrypto
NoOfPopup
NoOfiFrame
NoOfImage
NoOfJS
NoOfCSS
NoOfURLRedirect
NoOfHyperlink
Label
Description
URL address
1 if the URL uses HTTPS, 0 otherwise.
1 if the URL uses an IP address instead of a
domain name.
The top-level domain of the URL.
The length of the full URL.
Number of subdomains in the URL.
Number of dots in the URL, indicating
subdomains or obfuscation.
Count of special characters (%,@,$,!,+,-,~)
used for obfuscation.
Count of = in the URL.
Count of ? in the URL.
Count of & in the URL.
Count of numerical digits in the URL.
Length of the longest line in the webpage
source code.
1 if a <title> tag is present, 0 otherwise.
1 if meta tags are present, 0 otherwise.
1 if a favicon is detected, 0 otherwise.
1 if forms submit data to external domains, 0
otherwise.
1 if a copyright statement is present, 0
otherwise.
1 if social media links are detected, 0 otherwise.
1 if a password input field is detected, 0
otherwise.
1 if a form contains a submit button, 0
otherwise.
1 if the HTML contains the phishing-related
keyword bank, 0 otherwise.
1 if the HTML contains the phishing-related
keyword pay, 0 otherwise.
1 if the HTML contains the phishing-related
keyword crypto, 0 otherwise.
Number of popups detected in the page.
Number of <iframe> elements, which can
indicate embedded malicious content.
Number of <img> elements on the page.
Number of JavaScript <script> elements on the
page.
Number of CSS <link
rel="stylesheet"> elements.
Number of URL redirections detected.
Number of <a> hyperlinks on the webpage.
1 if phishing, 0 otherwise.
B. FEATURES ENGINEERING
FIGURE 2. The workflow of the proposed research.
Through this workflow, the study systematically examined
the impact of feature engineering on phishing URL detection,
offering insights into optimal feature selection strategies and
machine learning model performance.
A. DATASET DESCRIPTION
To evaluate the impact of feature engineering in phishing
URL detection, experiments were conducted using the
PhishOFE dataset [31] sourced from IEEE Dataport. This
dataset comprises the latest collection of URL and HTML
features, containing 101,063 records of legitimate and
phishing websites, providing a comprehensive basis for
analysis. The data structure for an instance, including its
extracted features, is presented in Table 1.
In this section, we focus on three distinct feature categories:
URL features, HTML features, and derived features. We
analyze these features through four different experiments,
each designed to evaluate their individual and combined
impact on phishing URL detection.
1) URL FEATURES
URL features play an essential role in phishing URL
detection [32] by offering key insights into a web page's
legitimacy. The main focus URL features to be used in this
study are IsHTTPS, IsDomainIP, TLD, URLLength,
NoOfSubDomain,
NoOfDots,
NoOfObfuscatedChar,
NoOfEqual, NoOfQmark, NoOfAmp, and NoOfDigits.
2) HTML FEATURES
HTML is used to create web pages, and examining the HTML
code can help detect phishing websites based on their visual
similarity [33] by identifying the elements and code the
attacker uses to replicate legitimate websites. Some of the key
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
HTML features extracted to be used in this study are as
follows: LineLength. HasTitle, HasMeta, HasFavicon,
HasExternalFormSubmit,
HasCopyright,
HasSocialNetworking, HasPasswordField, HasSubmitButton,
HasKeywordBank, HasKeywordPay, HasKeywordCrypto,
NoOfPopup, NoOfiFrame, NoOfImage, NoOfJ, NoOfCSS,
NoOfURLRedirect, and NoOfHyperlink.
3) DERIVED FEATURES
The derived features suggested in our previous work combine
multiple elements based on the URL, and HTML features are
discussed below.
• SuspiciousCharRatio: This feature measures the ratio of
suspicious characters (like %, @, =, etc.) relative to the
URL. It is calculated as the total number of obfuscated
characters, equal signs, question marks, and ampersands
divided by the total length of the URL. This feature is
designed to capture the presence of characters often used
in phishing URLs to manipulate the URL structure, and
below
is
the
equation
(1)
to
calculate
SuspiciousCharRatio.
𝑆!"#$ =
%!"#$ &%%& &%&'()* &%('+
(,)-
(1)
where,
• 𝑆!"#$ represents the Suspicious Character Ratio,
• 𝑁)*+, is the number of obfuscated characters,
• 𝑁-. is the number of equal signs (‘=’),
• 𝑁./#$0 is the number of question marks (‘?’),
• 𝑁#/1 is the number of ampersands (‘&’),
• 𝐿2$3 is the length of the URL.
(,)- &%#," &%!"#$
(,)-
+
%%& &%('+
%&'()* &4
(
(2)
where,
• 𝐶2$3 represents the URL Complexity Score,
• 𝐿2$3 is the URL length,
• 𝑁+2* is the number of subdomains,
• 𝑁)*+, is the number of obfuscated characters,
• 𝑁-. is the number of equal signs (‘=’),
• 𝑁#/1 is the number of ampersands (‘&’),
• 𝑁./#$0 is the number of question marks (‘?’).
• HTMLContentDensity: This feature evaluates an HTML
structure’s ratio of visible elements (text and images) to
&%
𝐷"5/3 = % &%-./%&% .'0
1#
2##
(3)
.$)('% &4
where,
• 𝐷"5/3 represents the HTML Content Density,
• 𝐿367- is the HTML line length,
• 𝑁6/8 is the number of images,
• 𝑁9+ is the number of JavaScript files,
• 𝑁!++ is the number of CSS files,
• 𝑁6,$#/- is the number of iframes.
• InteractiveElementDensity: This feature calculates
interactive element density by dividing the total number
of features "HasSubmitButton", "HasPasswordField", and
"NoOfPopup" by the total number of line lengths and
images. It reflects how concentrated these engagement
features are within the webpage content. Phishing sites
typically employ more interactive components to deceive
users into providing sensitive credentials.
𝐷675 =
• URLComplexityScore: This feature quantifies a URL's
complexity by combining its total length, subdomain
count, and obfuscated characters (like %, @, etc.), along
with the number of equal signs (=) and ampersands (&)
divided by the question mark (?) count. The metric is
designed to identify structural intricacies that may signal
phishing attempts without requiring third-party
verification.
The
calculation
formula
for
URLComplexityScore is presented in equation (2) below.
𝐶2$3 =
non-visible structural components (such as scripts,
iframes, and styles). The calculation method for
HTMLContentDensity is provided in the formula below.
:#,"'.3 &;+(## &%+!+,+
(4)
(-./% &%.'0
where,
• 𝐷675 represents the Interactive Element Density,
• 𝐵+2*/65 is a binary indicator for the presence of
a submit button,
• 𝐹1#++ is a binary indicator for the presence of
a password field,
• 𝑁1)121 is the number of pop-ups detected,
• 𝐿367- is the HTML line length,
• 𝑁6/8 is the number of images.
C. CLASSIFICATION MODEL & EVALUATION
The features undergo training and testing using ten machine
learning algorithms, including RF, kNN, LR, SVM, NB,
GBM, LightGBM, CatBoost, and XGBoost. These models
were chosen to represent a balanced mix of traditional
classifiers, ensemble-based learners, and gradient boosting
techniques, ensuring a comprehensive performance
evaluation across varying algorithmic paradigms.
Each model will be assessed based on four key evaluation
metrics, including accuracy, precision, recall, and F1 score.
These metrics provide a well-rounded understanding of the
classifier’s effectiveness, particularly in handling the class
imbalance commonly found in phishing detection tasks.
Accuracy reflects the overall correctness of predictions, while
precision and recall offer more profound insight into the
model's ability to correctly identify phishing URLs without
misclassifying legitimate ones. The F1 score, as the harmonic
mean of precision and recall, serves as a balanced measure to
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
evaluate model robustness in the presence of both false
positives and false negatives.
All models are implemented using their initial or default
parameter settings as provided by the respective machine
learning libraries. The chosen parameter configurations are
summarized in Table 2, ensuring consistency and
reproducibility throughout the experimental evaluation
process.
TABLE 2. The initial parameter for each model
Model
Random Forest
Naive Bayes
k-Nearest Neighbors
Logistic Regression
Decision Tree
Support Vector Machine
Gradient Boosting
LightGBM
XGBclassifier
CatBoost
Parameter
n_estimators=10
default
n_neighbors=1
default
max_depth=30
kernel='linear', C=1.0, random_state=42
max_depth=4,learning_rate=0.7
random_state=42
max_depth=4,learning_rate=0.7
learning_rate = 0.1
Employed permutation feature importance to assess the
significance of each feature in phishing URL detection. This
method evaluates the contribution of individual features by
randomly shuffling their values and measuring the resulting
change in model performance, highlighting redundant or less
relevant features that may be removed to optimize model
performance.
IV. EXPERIMENTAL DESIGN AND SETUP
In the following section, we outline the tools and system setup
used for the experiments, followed by a discussion of the
performance metrics used to evaluate the model.
A. SYSTEM CONFIGURATION AND TOOLS
All experiments were performed on a 64-bit Windows 11
operating system running on AMD Ryzen 5 7520U @ 2.8
GHz (64-bit) with 16 GB RAM and a 2 GB graphics card.
The implementation was carried out using Python 2.7.16 in a
Jupyter Notebook environment.
B. PERFORMANCE METRICS
Performance metrics are essential for measuring the accuracy
of a machine learning model’s prediction. These metrics offer
a quantitative evaluation of model performance, enabling
comparisons across different models. This study used these
metrics to see how different features affect various machine
learning models. The metrics used in this study include:
𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 =
<+ & </
<+ & </ &;+ &;/
<+
<+ & ;+
+
/
=$-!6+6)7 ∗ ?-!#33
𝐹1𝑆𝑐𝑜𝑟𝑒 = 2 ∗ =$-!6+6)7& ?-!#33
(7)
(8)
where,
• 𝑇1 is legitimate URL correctly classified as legitimate,
• 𝑇7 is phishing URL correctly classified as phishing,
• 𝐹1 is legitimate URL incorrectly classified as phishing,
• 𝐹7 is phishing URL incorrectly classified as legitimate.
C. DATASET VALIDATION
D. FEATURE IMPORTANCE
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 =
<
𝑅𝑒𝑐𝑎𝑙𝑙 = < &+;
(5)
(6)
Data validation enhances the reliability and efficiency of the
developed models. Our research thoroughly examined and
validated a dataset containing 101,063 records to ensure its
quality for training and evaluation purposes. The validation
involved multiple critical checks:
• Null value: There are no null values.
• Missing value: No missing values were found.
• Duplicate records: Each record is unique.
• Class imbalance: The dataset consists of 51.03%
legitimate URLs and 48.97% phishing URLs, reflecting a
balanced class distribution with no significant imbalance.
D. FEATURE IMPORTANCE SCORE
We used the permutation importance method from
sklearn.inspection to measure feature importance by
randomly shuffling feature values and evaluating the
resulting changes in model performance. This method
employed parameters n_repeats=10 and random_state=42 to
ensure consistent results.
V. RESULT & DISCUSSION
After conducting a series of experiments assessing various
models based on different feature sets, including URL
features, HTML features, a combination of URL and HTML
features, and complete feature sets that combine URL,
HTML, and derived features, the results are presented as the
below:
A. SOLELY URL FEATURES
Table 3 shows the performance of various machine learning
models using URL-based features for phishing detection.
Among all tested models, LightGBM achieved the highest
accuracy of 96.74% and an F1 Score of 96.89%, indicating a
superior balance between precision at 97.26% and recall at
96.50%. XGBoost also demonstrated strong performance with
an accuracy of 96.71% and an F1 Score of 96.68%, closely
followed by CatBoost, which had an accuracy of 96.73%,
precision of 97.13%, and an F1 Score of 96.65%.
Random Forest and Decision Tree models showed strong
recall values at 95.95% and 95.91%, respectively, suggesting
effectiveness in identifying positive cases. However, their
overall accuracy and precision were slightly lower compared
to gradient boosting models.
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
TABLE 3. ML model performance on URL-Based Feature
Model
Random Forest
Naive Bayes
k-Nearest Neighbors
Logistic Regression
Decision Tree
Support Vector Machine
Gradient Boosting
LightGBM
XGBoost
CatBoost
Accuracy
Precision
Recall
F1
Score
0.9658
0.7274
0.9459
0.9190
0.9659
0.9219
0.9675
0.9674
0.9667
0.9673
0.9704
0.9942
0.9418
0.9441
0.9711
0.9583
0.9726
0.9726
0.9728
0.9713
0.9595
0.4471
0.9483
0.8874
0.9591
0.8790
0.9609
0.9606
0.9589
0.9618
0.9649
0.6168
0.9450
0.9149
0.9650
0.9170
0.9667
0.9666
0.9658
0.9665
In this first experiment result, Gradient Boosting,
LightGBM, XGBoost, and CatBoost outperformed other
models, such as Logistic Regression, k-nearest Neighbors, and
Support Vector Machines, which highlights the advantage of
ensemble methods for phishing URL detection.
registered TLDs. Features such as NoOfDigits and NoOfDots
exhibit moderate importance across various models,
highlighting their relevance but having a lower comparative
influence. Conversely, features like IsHTTPS have limited
importance, confirming that malicious sites increasingly adopt
secure protocols to appear legitimate.
Moreover, IsDomainIP shows minimal importance,
possibly because this obvious phishing indicator has become
less common. Obfuscation techniques (NoOfObfuscatedChar,
NoOfQmark, NoOfAmp) show low significance but are still
utilized by some models.
B. SOLELY HTML FEATURES
Table 4 presents the performance of different machine
learning models using HTML-based features for detecting
phishing websites. CatBoost achieved the highest accuracy at
95.19% and the highest F1 Score of 95.10%, showcasing
excellent precision (95.04%) and recall (95.17%). LightGBM
closely followed, recording an accuracy of 94.68%, precision
of 92.99%, recall of 94.22%, and an F1 Score of 93.60%.
Random Forest also performed well, particularly regarding
the recall, with a notable value of 94.82%, a substantial
accuracy of 96.00%, and an F1 Score of 95.88%. XGBoost
showed slightly lower performance than CatBoost and
LightGBM, with an accuracy of 94.19%, precision of 93.91%,
recall of 94.27%, and F1 Score of 94.09%.
TABLE 4. ML model performance on HTML-Based Features.
FIGURE 3. Feature importance heatmap of URL-based features across
machine learning models.
Figure 3 illustrates the influence of various URL features
across different machine learning models for phishing
detection. URLLength appears as the most significant feature
influencing model, such as Logistic Regression, Random
Forest, Gradient Boosting, LightGBM, XGBoost, and
CatBoost, with importance values ranging from 0.195 to 0.255.
The NoOfSubDomain also holds substantial importance,
especially in Logistic Regression, Gradient Boosting, and
XGBoost. URL Length and NoOfSubDomain are the
strongest predictors of classification across models,
suggesting malicious URLs often have distinctive structural
patterns as attackers attempt to obscure their real domain
names and may use fake subdomains to appear legitimate.
Features like TLD have moderate importance but are less
impactful compared to length, indicating certain top-level
domains may be more associated with malicious activities;
phishing domains frequently use uncommon or newly
Model
Accuracy
Precision
Recall
F1
Score
Random Forest
Naive Bayes
k-Nearest Neighbors
Logistic Regression
Decision Tree
Support Vector Machine
Gradient Boosting
LightGBM
XGBoost
CatBoost
0.9600
0.7927
0.9329
0.8336
0.9422
0.8557
0.9412
0.9368
0.9419
0.9519
0.9697
0.7229
0.9213
0.7794
0.9344
0.7942
0.9372
0.9299
0.9391
0.9504
0.9482
0.9366
0.9438
0.9217
0.9488
0.9528
0.9434
0.9422
0.9427
0.9517
0.9588
0.8160
0.9324
0.8446
0.9415
0.8663
0.9403
0.9360
0.9409
0.9510
Naive Bayes recorded the lowest accuracy of 79.27% and
the lowest F1 Score of 81.60%, indicating limited
effectiveness with HTML-based features. Similarly, Decision
Tree performance was significantly lower, with an accuracy of
only 42.22% and an F1 Score of 64.15%, suggesting it is not
suitable for HTML-based phishing detection tasks. In this
result, CatBoost and LightGBM demonstrated the most robust
performance, proving highly effective in identifying phishing
websites based on HTML features.
Figure 4 shows the importance of HTML-based features
across machine learning models. The heatmap shows that
NoOfHyperlink is the most crucial feature across all models
(except SVM), with values ranging from 0.148 to 0.340.
LineLength, NoOfJS, and NoOfImage form a second tier of
essential features, particularly in tree-based models (RF, DT)
and kNN. NoOfCSS shows moderate importance in the
Decision Tree model (0.125) but less influence in other
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
models. The number of hyperlinks is consistently the most
potent predictor across models as attackers embed multiple
links to redirect users, followed by JavaScript usage, line
length, and image count. This suggests malicious websites
have distinctive structural patterns compared to legitimate
ones.
Other features like HasTitle, HasMeta, HasFavicon,
HasExternalFormSubmit,
HasCopyright,
HasSocialNetworking, HasPasswordField, HasSubmitButton,
HasKeywordBank, HasKeywordPay, HasKeywordCrypto,
NoOfPopUp, NoOfiFrame and NoOfURLRedirect have
minimal impact across all models.
CatBoost, XGBoost, and LightGBM demonstrated superior
effectiveness in phishing detection tasks.
TABLE 5. ML model performance on URL + HTML-Based Features
Model
Random Forest
Naive Bayes
k-Nearest Neighbors
Logistic Regression
Decision Tree
Support Vector Machine
Gradient Boosting
LightGBM
XGBoost
CatBoost
Accuracy
Precision
Recall
F1
Score
0.9913
0.8970
0.9590
0.9276
0.9857
0.9490
0.9882
0.9932
0.9937
0.9942
0.9955
0.9441
0.9606
0.8999
0.9842
0.9656
0.9853
0.9954
0.9952
0.9960
0.9868
0.8398
0.9557
0.9591
0.9867
0.9292
0.9908
0.9907
0.9919
0.9921
0.9911
0.8889
0.9582
0.9285
0.9854
0.9471
0.9880
0.9931
0.9936
0.9940
Figure 5 presents a combined set of URL and HTML
features across various machine learning models, providing a
more holistic view of feature importance for classification. A
mix of both URL-based features (URLLength, TLD,
NoOfSubDomain)
and
HTML-based
features
(NoOfHyperlink, LineLength, NoOfImage) features show
high importance. Furthermore, features like URLLength,
NoOfHyperlink, and NoOfImage are consistently among the
top features across models. Random Forest, Decision Tree,
and LightGBM highlight these features more prominently.
FIGURE 4. Feature importance heatmap of URL-based features across
machine learning models.
C. URL AND HTML FEATURES
Table 5 outlines the performance of various machine learning
models using combined URL and HTML-based features for
phishing detection. Random Forest achieved the highest
accuracy at 99.13% and the top F1 Score of 99.11%, with
outstanding precision (99.55%) and recall (98.68%),
demonstrating excellent overall performance. CatBoost and
XGBoost showed strong performances. CatBoost recorded an
accuracy of 99.42%, precision of 99.60%, recall of 99.21%,
and an F1 Score of 99.40%, while XGBoost had similar
impressive metrics with an accuracy of 99.37%, precision of
99.52%, recall of 99.17%, and an F1 Score of 99.34%.
LightGBM exhibited slightly lower accuracy at 99.32%,
precision at 99.53%, recall at 99.07%, and an F1 Score of
99.31%. Despite this minor difference, LightGBM performed
exceptionally well compared to traditional models.
Naive Bayes again presented the lowest performance
among the tested models, with an accuracy of 89.70% and an
F1 Score of 88.89%, suggesting it struggles with effectively
leveraging combined URL and HTML features. Overall, when
combining URL and HTML-based features, Random Forest,
FIGURE 5. Feature importance heatmap of URL and HTML features
across machine learning models.
When considering content and URL features together, URL
characteristics, particularly length and subdomain count,
emerge as the most reliable predictors across models. This
suggests that malicious URLs have more consistent patterns
than malicious content. Many HTML features that showed
moderate importance in the previous analysis now appear less
significant when considered alongside URL features. This
indicates that URL features may provide stronger signals for
classification, and Machine learning models that rely on both
URL and HTML features tend to achieve higher accuracy.
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
D. URL, HTML, AND DERIVED FEATURES
Table 6 shows the performance of various machine learning
models using combined URL, HTML, and derived features for
phishing detection. CatBoost achieved the highest accuracy of
99.45% and the best F1 Score of 99.44%, demonstrating
excellent precision at 99.69% and recall at 99.19%. XGBoost
closely followed CatBoost, achieving an accuracy of 99.41%,
precision of 99.61%, recall of 99.18%, and an F1 Score of
99.39%. LightGBM also performed strongly with an accuracy
of 99.33%, precision of 99.51%, recall of 99.12%, and an F1
Score of 99.31%.
Random Forest also showed robust performance, achieving
accuracy at 99.13%, precision of 99.57%, recall of 98.65%,
and an F1 Score of 99.11%. Naive Bayes exhibited the lowest
overall performance with an accuracy of 90.00% and an F1
Score of 89.08%, suggesting limitations in effectively using
complex combined features.
CatBoost) consistently emphasize URL structural features
(URLLength, NoOfSubDomain). Naive Bayes shows a more
distributed importance pattern with URLLength, NoOfDigits,
and URLComplexityScore as primary features.
The high importance of HTMLContentDensity and
InteractiveElementDensity in kNN suggests that the
relationship between content elements provides valuable
signals. This aligns with sophisticated phishing techniques
that balance content density to appear legitimate.
TABLE 6. ML Model performance on URL + HTML + Derived Features
Model
Random Forest
Naive Bayes
k-Nearest Neighbors
Logistic Regression
Decision Tree
Support Vector Machine
Gradient Boosting
LightGBM
XGBoost
CatBoost
Accuracy
Precision
Recall
F1
Score
0.9913
0.9000
0.9536
0.8749
0.9833
0.9548
0.9926
0.9933
0.9941
0.9945
0.9957
0.9589
0.9511
0.8788
0.9816
0.9632
0.9930
0.9951
0.9961
0.9969
0.9865
0.8318
0.9545
0.8642
0.9844
0.9440
0.9919
0.9912
0.9918
0.9919
0.9911
0.8908
0.9528
0.8714
0.9830
0.9535
0.9925
0.9932
0.9939
0.9944
Overall, ensemble methods such as CatBoost, XGBoost,
LightGBM, and Random Forest continue to demonstrate
superior effectiveness, emphasizing their capability to detect
phishing websites accurately through comprehensive feature
sets.
Figure 6 introduces additional derived features and presents
a more comprehensive view of feature importance across
various machine learning models. URLLength remains an
important feature across most models, like Decision Tree,
GBM, LightGBM, XGBoost, and CatBoost. Furthermore,
LineLength and HTMLContentDensity are extremely
important for kNN, both 0.326, suggesting this model relies
heavily on content density metrics. NoOfHyperlink shows
high importance in LR but moderate importance in other
models, and New derived features show significant
importance for particular models; SuspiciousCharRatio is
important for LR, URLComplexityScore is important for kNN
and LR, and InteractiveElementDensity is important for kNN.
Lastly, NoOfSubDomain remains important, particularly
Decision Tree and LightGBM.
kNN shows a distinct pattern focusing on content density
metrics
(LineLength,
HTMLContentDensity),
InteractiveElementDensity, and URLComplexityScore.
Logistic
Regression
prioritizes
NoOfHyperlink,
SuspiciousCharRatio, and URLComplexityScore. Then, treebased models (DT, RF, GBM, LightGBM, XGBoost,
FIGURE 6. Feature importance heatmap of URL, HTML, and Derived
features across machine learning models.
URLComplexityScore is essential across multiple models,
suggesting that composite measures capturing multiple URL
characteristics outperform individual metrics. This indicates
phishing URLs exhibit complexity patterns that are more
detectable through holistic analysis.
SuspiciousCharRatio's importance in LR highlights that
unusual characters in URLs are a meaningful signal,
potentially capturing obfuscation techniques used in phishing.
The strong performance of derived features validates the
approach of creating composite metrics that capture more
sophisticated patterns rather than relying solely on URL or
HTML features.
VI. CONCLUSION
This study on phishing URL detection reveals several valuable
insights. It shows that the choice of features significantly
influences the performance of phishing URL detection and
validates the effectiveness of combining URL, HTML, and
derived features for phishing detection.
By systematically evaluating URL features, HTML features,
combined features, and derived features, we can conclude that
integration of URL, HTML, and derived features produced the
most robust detection, with CatBoost achieving the highest
performance (99.45% accuracy, 99.44% F1 score),
demonstrated that a multi-layered approach to feature
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
engineering significantly enhanced detection capabilities
beyond what any single feature category can achieve.
Ensemble methods consistently outperform traditional
classifiers across all feature configurations. CatBoost,
XGBoost, LightGBM, and Random Forest demonstrated
superior effectiveness, highlighting the advantage of ensemble
learning for this security application. These models effectively
leverage the complementary signals from different feature
types to improve accuracy.
URL features provide the strongest individual signals for
phishing detection. URLLength and NoOfSubDomain
emerged as particularly powerful predictors across most
models, suggesting that malicious URLs exhibit distinctive
structural patterns that can be reliably detected. When URL
features are combined with HTML features, the URL
characteristics remain dominant in importance rankings.
Content density and relationship metrics contribute
significant value when adequately leveraged. The kNN
showed firm reliance on HTMLContentDensity, LineLength,
and InteractiveElementDensity, demonstrating that the
relationships between content elements, rather than just their
presence, provide valuable classification signals.
Derived features that capture complex patterns show
considerable
importance
in
specific
models.
URLComplexityScore and SuspiciousCharRatio add
substantial value by aggregating multiple characteristics into
more holistic metrics, enhancing the detection of sophisticated
phishing attempts that might evade simpler feature sets.
Different models exhibit specialized sensitivity to particular
feature types. kNN excels with content density metrics, and
Logistic Regression leverages character composition and
hyperlink features. At the same time, tree-based models
primarily utilize URL structural features, meaning ensemble
approaches combining multiple model types could further
enhance detection robustness.
The research confirms that effective phishing detection
requires a tiered approach to feature engineering,
incorporating URL, HTML, and Derived features. This multifaceted strategy provides resilience against evolving phishing
techniques while maintaining high accuracy.
These findings suggest deploying a detection system that
utilizes ensemble methods with comprehensive feature sets for
future research. A cascading architecture that applies
computationally efficient URL analysis first, followed by
more intensive content and relationship analysis for borderline
cases, could be implemented. In addition, researchers could
explore and craft different derived features to enhance the
accuracy of phishing URL detection models.
REFERENCES
[1] J. Hong, “The state of phishing attacks,” Commun. ACM, vol. 55, no. 1,
pp. 74–81, Jan. 2012, doi: 10.1145/2063176.2063197.
[2] R. Ahmad, S. Terzis, and K. Renaud, “Getting users to click: a content
analysis of phishers’ tactics and techniques in mobile instant messaging
phishing,” ICS, vol. 32, no. 4, pp. 420–435, Sep. 2024, doi:
10.1108/ICS-11-2023-0206.
[3] W. Li, S. Manickam, S. U. A. Laghari, and Y.-W. Chong, “Uncovering
the Cloak: A Systematic Review of Techniques Used to Conceal
Phishing Websites,” IEEE Access, vol. 11, pp. 71925–71939, 2023, doi:
10.1109/ACCESS.2023.3293063.
[4] H. Cui, Y. Zhou, C. Wang, X. Wang, Y. Du, and Q. Wang, “PPSB: An
Open and Flexible Platform for Privacy-Preserving Safe Browsing,”
IEEE Transactions on Dependable and Secure Computing, vol. 18, no.
4, pp. 1762–1778, Jul. 2021, doi: 10.1109/TDSC.2019.2937783.
[5] “APWG | APWG Q1 Report: Phone-Based Phishing Grows
Explosively, Shifting the Cybercrime Threatscape.” Accessed: Sep. 01,
2024. [Online]. Available: https://apwg.org/apwg-q1-report-phonebased-phishing-grows-explosively-shifting-the-cybercrimethreatscape/
[6] “APWG | Phishing Activity Trends Reports.” Accessed: Sep. 01, 2024.
[Online]. Available: https://apwg.org/trendsreports/
[7] “egress-phishing-threat-trends-report-april-2024.pdf.” Accessed: Sep.
01,
2024.
[Online].
Available:
https://www.egress.com/media/jhtb1m2r/egress-phishing-threattrends-report-april-2024.pdf
[8] “egress_phishing_threat_trends_report_oct_2024.pdf.” Accessed: Mar.
24,
2025.
[Online].
Available:
https://www.egress.com/media/kuvpjdjl/egress_phishing_threat_trend
s_report_oct_2024.pdf
[9] J. Spaulding, S. Upadhyaya, and A. Mohaisen, “You’ve Been Tricked!
A User Study of the Effectiveness of Typosquatting Techniques,” in
2017 IEEE 37th International Conference on Distributed Computing
Systems
(ICDCS),
Jun.
2017,
pp.
2593–2596.
doi:
10.1109/ICDCS.2017.221.
[10] P. Kintis et al., “Hiding in Plain Sight: A Longitudinal Study of
Combosquatting Abuse,” in Proceedings of the 2017 ACM SIGSAC
Conference on Computer and Communications Security, Dallas Texas
USA: ACM, Oct. 2017, pp. 569–586. doi: 10.1145/3133956.3134002.
[11] A. Safi and S. Singh, “A systematic literature review on phishing
website detection techniques,” Journal of King Saud University Computer and Information Sciences, vol. 35, no. 2, pp. 590–611, Feb.
2023, doi: 10.1016/j.jksuci.2023.01.004.
[12] N. A. B. M. Zin, M. F. Ab Razak, A. Firdaus, F. Ernawan, and N. S. A.
Zulkifli, “Machine Learning Technique for Phishing Website
Detection,” in 2023 IEEE 8th International Conference On Software
Engineering and Computer Systems (ICSECS), Aug. 2023, pp. 235–239.
doi: 10.1109/ICSECS58457.2023.10256280.
[13] O. K. Sahingoz, E. Buber, O. Demir, and B. Diri, “Machine learning
based phishing detection from URLs,” Expert Systems with
Applications, vol. 117, pp. 345–357, Mar. 2019, doi:
10.1016/j.eswa.2018.09.029.
[14] B. B. Gupta, K. Yadav, I. Razzak, K. Psannis, A. Castiglione, and X.
Chang, “A novel approach for phishing URLs detection using lexical
based machine learning in a real-time environment,” Computer
Communications, vol. 175, pp. 47–57, Jul. 2021, doi:
10.1016/j.comcom.2021.04.023.
[15] A. N. S. Charan, Y.-H. Chen, and J.-L. Chen, “Phishing Websites
Detection using Machine Learning with URL Analysis,” in 2022 IEEE
World Conference on Applied Intelligence and Computing (AIC), Jun.
2022, pp. 808–812. doi: 10.1109/AIC55036.2022.9848895.
[16] N. Jagdale and P. Chavan, “Hybrid Ensemble Machine Learning
Approach for URL Phishing Detection,” in 2022 2nd Asian Conference
on Innovation in Technology (ASIANCON), Aug. 2022, pp. 1–8. doi:
10.1109/ASIANCON55314.2022.9908667.
[17] V. Dantwala, R. Lakhani, and N. Shekokar, “A Novel Technique to
Detect URL Phishing based on Feature Count,” in 2023 3rd
International Conference on Intelligent Communication and
Computational Techniques (ICCT), Jaipur, India: IEEE, Jan. 2023, pp.
1–5. doi: 10.1109/ICCT56969.2023.10075943.
[18] A. K. Jain, N. Debnath, and A. K. Jain, “APuML: An Efficient
Approach to Detect Mobile Phishing Webpages using Machine
Learning,” Wireless Pers Commun, vol. 125, no. 4, pp. 3227–3248, Aug.
2022, doi: 10.1007/s11277-022-09707-w.
[19] J. Rashid, T. Mahmood, M. W. Nisar, and T. Nazir, “Phishing Detection
Using Machine Learning Technique,” in 2020 First International
Conference of Smart Systems and Emerging Technologies
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2025.3579223
(SMARTTECH), Riyadh, Saudi Arabia: IEEE, Nov. 2020, pp. 43–46.
doi: 10.1109/SMART-TECH49988.2020.00026.
[20] A. Lakshmanarao, P. S. P. Rao, and M. M. B. Krishna, “Phishing
website detection using novel machine learning fusion approach,” in
2021 International Conference on Artificial Intelligence and Smart
Systems (ICAIS), Coimbatore, India: IEEE, Mar. 2021, pp. 1164–1169.
doi: 10.1109/ICAIS50930.2021.9395810.
[21] N. J. Sinthiya, T. A. Chowdhury, and A. B. Haque, “Incorporating
Machine Learning Algorithms to Detect Phishing Websites,” in 2022
International Conference on ICT for Smart Society (ICISS), Bandung,
Indonesia:
IEEE,
Aug.
2022,
pp.
1–5.
doi:
10.1109/ICISS55894.2022.9915211.
[22] Q. Hu, H. Zhou, and Q. Liu, “Phishing Website Detection Based on
Multi-Feature Stacking,” in 2021 2nd International Conference on
Artificial Intelligence and Computer Engineering (ICAICE), Hangzhou,
China:
IEEE,
Nov.
2021,
pp.
716–720.
doi:
10.1109/ICAICE54393.2021.00140.
[23] M. Murshid, M. N. Uddin, and F. Hossain, “StackedPhish: A Stacked
Ensemble Framework for Identification of Phishing Website,” in 2023
IEEE 9th International Women in Engineering (WIE) Conference on
Electrical
and
Computer
Engineering
(WIECON-ECE),
Thiruvananthapuram, India: IEEE, Nov. 2023, pp. 76–81. doi:
10.1109/WIECON-ECE60392.2023.10456518.
[24] A. Karim, M. Shahroz, K. Mustofa, S. B. Belhaouari, and S. R. K. Joga,
“Phishing Detection System Through Hybrid Machine Learning Based
on URL,” IEEE Access, vol. 11, pp. 36805–36822, 2023, doi:
10.1109/ACCESS.2023.3252366.
[25] E. Zhu, Y. Chen, C. Ye, X. Li, and F. Liu, “OFS-NN: An Effective
Phishing Websites Detection Model Based on Optimal Feature
Selection and Neural Network,” IEEE Access, vol. 7, pp. 73271–73284,
2019, doi: 10.1109/ACCESS.2019.2920655.
[26] P. Pandey and N. Mishra, “Phish-Sight: a new approach for phishing
detection using dominant colors on web pages and machine learning,”
Int. J. Inf. Secur., vol. 22, no. 4, pp. 881–891, Aug. 2023, doi:
10.1007/s10207-023-00672-4.
[27] S. H. Ahammad et al., “Phishing URL detection using machine learning
methods,” Advances in Engineering Software, vol. 173, p. 103288, Nov.
2022, doi: 10.1016/j.advengsoft.2022.103288.
[28] T. Nagunwa, P. Kearney, and S. Fouad, “A machine learning approach
for detecting fast flux phishing hostnames,” Journal of Information
Security and Applications, vol. 65, p. 103125, Mar. 2022, doi:
10.1016/j.jisa.2022.103125.
[29] M. M. Alani and H. Tawfik, “PhishNot: A Cloud-Based MachineLearning Approach to Phishing URL Detection,” Computer Networks,
vol. 218, p. 109407, Dec. 2022, doi: 10.1016/j.comnet.2022.109407.
[30] Y. Ding, N. Luktarhan, K. Li, and W. Slamu, “A keyword-based
combination approach for detecting phishing webpages,” Computers &
Security,
vol.
84,
pp.
256–275,
Jul.
2019,
doi:
10.1016/j.cose.2019.03.018.
[31] Y. A. Kustiawan, “PhishFOE Dataset.” IEEE, Mar. 08, 2025. Accessed:
Mar.
24,
2025.
[Online].
Available:
https://ieeedataport.org/documents/phishfoe-dataset
[32] R. S. Rani, N. Silpa, G. N. Satish, N. Amrutha, and G. N. Reddy,
“Optimizing Phishing Detection: Leveraging URL Features with
Machine Learning,” in 2024 10th International Conference on
Advanced Computing and Communication Systems (ICACCS), Mar.
2024, pp. 2094–2099. doi: 10.1109/ICACCS60874.2024.10717158.
[33] G. Brezeanu, A. Archip, and C.-G. Artene, “Phish Fighter: Self
Updating Machine Learning Shield Against Phishing Kits Based on
HTML Code Analysis,” IEEE Access, vol. 13, pp. 4460–4486, 2025,
doi: 10.1109/ACCESS.2025.3525998.
YANCHE ARI KUSTIAWAN received a
bachelor’s degree in multimedia, majoring in
software
development,
from
Swinburne
University in Kuching, Sarawak, Malaysia. He is
currently pursuing a master’s degree with the
Faculty of Computing and Informatics at
Multimedia University. His research interests
include XML query optimization, data
management, requirement engineering and
software quality, and machine learning.
KHAIRIL IMRAN GHAUTH is an Associate
Professor at the Faculty of Computing and
Informatics, Multimedia University. He graduated
his Master of IT from the University of Melbourne
and his PhD from the University of Malaya. His
research interests are in the area of Recommender
Systems, Information Filtering, Data Analytics,
and Service-Oriented Architecture.
8
VOLUME XX, 2017
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/