Available online at website: https://jurnal.
id/index.
php/RESTI JURNAL RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
1347 - 1357
e-ISSN: 2580-0760 Optimizing a Hybrid Deep Learning Model for DDoS Detection Using DBSCAN and PSO Indrastanti Ratna Widiasari1.
Rissal Efendi2 1,2Informatics Engineering Department.
Information Technology Faculty.
Satya Wacana Christian University.
Salatiga.
Indonesia 1indrastanti@uksw.
edu, 2rissal.
efendi@uksw.
Abstract This study proposes a hybrid deep learning that combines Gated Recurrent Units (GRU.
and Convolutional Neural Networks (CNN.
for Distributed Denial of Service (DDoS) cyberattack detection.
The model, called DBSCANAeGRUAeCNN, uses densitybased clustering (DBSCAN) to select relevant features and reduce execution time.
The dataset for this research was collected from penetration testing, in which several simulated attack scenarios were executed on a monitored network.
To evaluate the performance of the proposed model, several comparison models were used, including DBSCANAeGRUAeCNN (Single Hidden Laye.
DBSCANAeGRUAeCNN (Double Hidden Layer.
DBSCANAeGRUAeCNN (With Regularizatio.
DBSCANAeGRUAeCNNAe PSO.
GRUAeCNN.
GRUAeCNN (With Hyperparameter Tunin.
, and Random Forest (Tuned Mode.
Variations of the model tested were made by adding hidden layers, regularization, optimization with Particle Swarm Optimization (PSO), and hyperparameter tuning.
The results of the experiments reveal that the DBSCANAeGRUAeCNNAePSO model provided optimal performance with a 99.
3% accuracy, a 99% precision, a 98.
9% recall, and a 99% F1-score, while the model with hyperparameter tuning achieved a 99% accuracy.
By adding PSO, the model achieved optimized weights, better generalization, and excellent accuracy in DDoS detection.
Keywords: attack detection.
DBSCAN.
DdoS.
hybrid deep learning.
PSO.
How to Cite: I.
Widiasari, and R.
Efendi.
AuOptimizing a Hybrid Deep Learning Model for DDoS Detection Using DBSCAN and PSOAy.
RESTI (Rekayasa Sist.
Teknol.
Inf.
), vol.
9, no.
6, pp.
1347 - 1357.
Dec.
Permalink/DOI: https://doi.
org/10.
29207/resti.
Received: February 8, 2025 Accepted: August 15, 2025 Available Online: December 7, 2025 This is an open-access article under the CC BY 4.
0 License Published by Ikatan Ahli Informatika Indonesia Introduction In recent years.
Artificial Intelligence (AI), an umbrella term encompassing several Machine Learning (ML) and Deep Learning (DL) models, has acquired increased attention in the field of malware detection.
ML algorithms are developed using formatted and categorized data, enabling them to identify forms and classify recent information.
These algorithms have shown significant potential in detecting malicious activities by studying large amounts of cybersecurity DDoS is a type of cyberattacks where attackers try to incapacitate systems and servers, rendering them inaccessible to users .
It causes severe damage to a system, impacting it to a severe extent .
All industries, irrespective of size, continuously face a serious risk to their network security due to the growing frequency, complexity, and volume of these attacks.
As networks continue to move toward a softwarecentric model, they face a greater risk from malicious cyber activities such as DDoS attacks, which exploit performance bottlenecks, and poor handling of exception .
Typically, these attacks overwhelm the targetAia particular network or serverAiwith numerous packets from a vast number of hosts as it becomes compromised by malicious software .
A DDoS attack primarily results in a massive influx of packets directed to the target, depleting all available resources, and eventually causing malfunction of every programmable switch.
Consequently, a DDoS attacker can render a network inoperable.
For that reason, detection of suitable responses to cyberattacks is more vital in programmable networks than in conventional ones .
AI models are key in solving these challenges, with deep learning models, recognized for their ability to detect complex patterns from large datasets, being frequently applied to DDoS attack detection .
, .
Widiasari et al Jurnal RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
DL, which is built on Artificial Neural Networks (ANN.
, is generally more intricate than ML, which includes various algorithms such as K-Nearest Neighbor (KNN).
Random Forest (RF).
Support Vector Machine (SVM).
Logistic Regression (LR), and Nayve Bayes (NB) algorithms.
The artificial neuron serves as the basic building block of an ANN.
When one neuron is combined with others, the network can perform complex classification and forecasting functions as a result of their concurrent structure .
An ANN fundamentally consists of one input layer, one or more hidden layers, and one output layer .
As DL employs multiple hidden layers, it generally needs higher data volumes and computational strength compared to ML.
In supervised DL, data labelling, as in distinguishing between malicious and benign data packets, is Conversely, unsupervised DL operates without data labelling.
DL architectures come in several types, such as Deep Neural Networks (DNN.
Recurrent Neural Networks (RNN.
Deep Belief Networks (DBN.
and Convolutional Neural Networks (CNN.
, .
Data Generation AI, a fast-growing domain, is applied in Large Language Models (LLM.
, which are a variety of DNNs, with several layers that are essential for extracting deep abstractions and dealing with complex challenges, including natural language understanding and generation .
CNNs, renowned for their prowess in feature extraction from complex data structures, offer a promising avenue for enhancing the accuracy and adaptability of DDoS detection mechanisms.
This paper builds upon a rich body of literature addressing DDoS attacks, cloud security, and machine learning applications in Prior research has extensively explored the vulnerabilities and countermeasures associated with DDoS attacks in various contexts.
Additionally, the effectiveness of machine learning, especially CNNs, in handling cybersecurity challenges has been welldocumented .
- .
DL models, which are growing in prevalence, are applied to detect attacks and anomalies in computer Among these.
GRUs prioritize newer information by enabling the model to swiftly customize the most up-to-date data and improving its capability to identify abnormal traffic .
The results obtained from the application of the GRU model across various meteorological forecasting tasks show advantages in precision, predictive ability, and efficiency, making it a favorable choice for this study.
A GRU has a gating mechanism that performs selective updates to its hidden states according to input data.
It operates with an update gate by merging the roles of the forget gate and the input gate, thereby refining the design and decreasing parameter complexity.
As a result, it leads to more efficient computation and reduced model development.
The GRU model incorporates hidden cells and states, which improves the smoothness of information flow, hence its ability to maintain crucial details and discard unrelated ones, making it very suitable for sequence data analysis .
- .
It has been extensively implemented in a wide range of sectors, such as forecasting the movement of landslides, estimating traffic flow .
, predicting electrical load, estimating solar radiation .
, conducting advanced agricultural practices .
, assessing wind speed and temperature conditions .
, predicting carbon dioxide levels .
, and estimating solar energy levels .
This paper proposes two integrated models of DNNs for accurate DDoS attack monitoring and analysis.
The main contribution of this study is an integrated DL approach called AuGRUAeCNNAy is designed to identify DDoS packets and categorize them into various types of DDoS attacks promptly and accurately.
On the other hand.
Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is applied to determine the most relevant attribute from an extensive DDoS attack By detecting the density-based clusters and outliers associated with malicious payload, this method boosts detection accuracy while reducing computation Therefore, potential DDoS attacks can be detected and effectively identified by clustering them based on Previous studies have explored the vulnerabilities of networks targeted by DDoS attacks, as well as the potential of ML and DL in addressing these challenges.
However, the primary contribution in this research lies in the integration of DBSCAN for feature selection and PSO for optimization in a hybrid GRUAeCNN Methods This section details the research flow for the optimization and comparison of hybrid GRUAeCNN models in detecting DDoS attacks using the DBSCAN Through this study, we propose an approach to optimizing detection accuracy by utilizing a combination of deep learning models and optimization This process also integrates clustering methods for efficient data segmentation and anomaly Figure 1 illustrates the steps involved in this process.
The process started from network traffic data that went through a preprocessing stage for cleaning, feature selection, and data balancing.
DBSCAN was used for data segmentation and anomaly detection.
DBSCAN
was chosen because it is superior to K-Means in DDoS It does not assume a spherical or uniform cluster shape, making it capable of handling more complex and unstructured data patterns, such as DDoS In addition, it can automatically detect outliers and noise, which is important for separating scattered attacks from normal traffic data, while K-Means is sensitive to outliers.
It also avoids the need for prior determination of the number of clusters, making it more flexible in handling imbalanced datasets with an unknown number of clusters.
Widiasari et al Jurnal RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
2 Data Preprocessing In this phase, the raw data obtained was processed and organized to prepare it for further analysis.
Data preprocessing in this study included feature normalization using MinAeMax Scaling to ensure that all attributes had a uniform value range between 0 and 1 to reduce bias due to scale differences.
This process included several important tasks, including addressing missing data, excluding irrelevant information, changing data formats as needed, and eliminating unnecessary or redundant data.
This stage aimed to ensure that the data was consistent, accurate, and reliable, so that it could produce valid analysis results in the next stage.
Figure 1.
Flowchart of the Research We then built a GRU model as the basis for temporal The results of the GRU were then input to build a CNN model that was optimized using the PSO algorithm to determine the best parameters, which offered an efficient way to find the optimal parameter space without the need for a grid search which could be more time-consuming and result in more dependency on the choice of pre-defined grid parameters.
PSO is more flexible and effective in dealing with complex optimization problems.
The final step involved evaluating the model based on accuracy, precision, recall, and F1-score metrics before the model was practically implemented.
1 Data Collection Data collection included attributes such as time, sources, destinations, protocols, durations, clusters, and anomalies in a managed network infrastructure to obtain reliability and a realistic modeling of real-world attack behavior.
The data included data on both benign and anomalous network traffic.
The data size obtained was 1,031,892 records.
From the analysis of the class distribution in the dataset, the normal class amounted to about 93%, while the DDoS class did about 7%.
This shows that the dataset was not completely balanced, with a tendency for normal data to dominate.
Data collection is important because it supplies the essential data needed for model training and evaluation.
In this study, the dataset was generated through controlled penetration testing in a lab environment.
simulate realistic DDoS attack scenarios, two tools were employed, namely.
LOIC (Low Orbit Ion Canno.
and Hping3.
LOIC was used to generate HTTP and UDP flood attacks, while Hping3 was configured to simulate SYN flood and other packet-based attacks.
During the attack simulations.
Wireshark was used to capture and analyze the resulting network traffic.
The recorded packet data was later preprocessed and utilized to extract relevant features for training and evaluating the proposed hybrid deep learning model.
In addition to basic data preprocessing, to increase data diversity and improve model performance, a data augmentation technique was performed.
The technique used was SMOTE (Synthetic Minority Over-sampling Techniqu.
, which generated synthetic data to create variations of DDoS attacks that might not have been represented in the original dataset.
This augmentation step aimed to balance the dataset, enrich the training data, and help the model recognize more complex and varied attack patterns.
3 Dataset Split The data used in this study was split into three main The division ratio used was 80% for training data, 10% for validation data, and 10% for test data.
Training data was employed to train the model, validation data was used to select optimal hyperparameters and prevent overfitting, and test data was used to test the validity of the performance that had been trained on data that had not been encountered earlier by the model.
This division aimed to ensure that the model could generalize well and to avoid bias in the data used during training.
The data split with an 80:10:10 ratio was consistently applied throughout the experiments to maintain a balanced evaluation process.
This approach ensured that the training phase contained sufficient data to capture underlying patterns, while the validation phase provided an independent reference for fine-tuning the Meanwhile, the test phase offered an unbiased benchmark for assessing the final performance.
Such a division strategy was widely recognized in machine learning research as it reduced the risk of data leakage and enhanced the reliability of the experimental results.
4 Segmentation and Anomaly Detection Using DBSCAN DBSCAN was used for data segmentation and anomaly detection in network data.
Figure 2 represents data segmentation, which was performed to divide the data based on density patterns, allowing the identification of homogeneous clusters and striking anomalies.
The process began by selecting an unvisited data point, followed by calculating the number of points within an epsilon radius (A).
If the number of points met a Widiasari et al Jurnal RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
minimum threshold .
inPt.
, the data point was marked as a core point, and the cluster was expanded by adding neighboring points.
Otherwise, the data point was marked as noise or anomaly.
This process continued until all data points were visited, resulting in clearly segmented clusters and anomalies.
This approach is effective for understanding complex patterns in high-dimensional network data.
reset gate computes the gate value based on the current input and previous hidden state.
ycyc = E.
cOyc .
ycuyc ycOyc .
EaycOe1 ycayc ) .
The update gate regulated the amount of old hidden state information that should have been retained with new information .
rom ongoing network traffi.
This allowed the model to remember relevant patterns and ignore useless information, which is important in detecting DDoS attacks.
It balanced past and current information to optimize the GRU's prediction This process follows Equation 2.
This process follows Equation 2.
ycsyc = E.
cOyc .
ycuyc ycOyc .
EaycOe1 ycayc ) .
Candidate hidden states combined current input and reset hidden state information, calculated using Equation 3.
This resulted in a new hidden state that took into account the current network traffic conditions, which is important for detecting attack patterns or anomalies in traffic data.
This process allowed the GRU to generate more accurate representations by integrating relevant past and current information.
EaUyc = ycycaycuEa .
cOEa .
ycuyc ycOEa .
cyc o EaycOe1 ) ycaEa Figure 2.
DBSCAN Clustering Process with Anomaly Detection Utilizing Network Traffic Variables 5 Building the GRU Model In this study, the GRU model was developed with hyperparameter adjustments to ensure that the model would generate the best performance in capturing temporal dependencies in network traffic data.
For the learning rate, the value used was 0.
001, which is a value commonly used in optimization with Particle Swarm Optimization (PSO), because it provides stable convergence without causing too large parameter updates .
Meanwhile, the number of hidden units on each GRU layer was set at 128 units, which allowed the model to capture complex patterns in sequential data.
The batch size used in model training was 64, which provided a good balance between memory efficiency and model accuracy.
The model was trained for 50 epochs, with monitoring of convergence and avoidance of overfitting through validation techniques, using a dropout rate of 0.
2 to reduce the risk of overfitting.
The selection of these hyperparameters was carried out based on experiments with the model, as well as adjustments during training, to find the best configuration that could improve accuracy in detecting DDoS attacks.
In the GRU architecture, the reset gate, update gate, candidate hidden state, and final hidden The reset gate regulated the hidden state information which must be forgotten so that the GRU only retained information that was relevant to detecting DDoS attack patterns based on current network traffic conditions as shown in Equation 1.
This mechanism ensured that irrelevant past information did not affect the model's current prediction.
The mathematical formulation of the .
The final hidden state combined the old hidden state EaUyc and the new candidate hidden state.
The GRU model could decide whether to retain the old hidden state or combine it with new information based on the current network traffic input.
This mechanism enabled the model to adaptively update its memory, improving its ability to detect DDoS attack patterns.
Final hidden gate is represented in Equation 4.
Eayc = .
Oe ycyc )o EaycOe1 ycyc o EaUyc To explain how the GRU model was used in DDoS detection, the process began with input in the form of network traffic data .
uch as the number of packets, connection duration, and traffic level.
and output in the form of attack detection .
ttack or normal traffi.
The input data was converted into a time sequence, where the GRU processed the information by retaining relevant data from the previous sequence using the reset gate and update gate.
The GRU then integrated the new input and the previous hidden state to produce the final hidden state that would be processed by the dense layer to predict the DDoS detection state.
Thus, the GRU could capture temporal patterns in traffic data and enhance the effectiveness of DDoS attack detection.
6 Building the CNN Model using GRU Outputs After the results from the GRU were processed, the CNN model was used for DDoS attack classification.
For the CNN model, several hyperparameters were also set to get the best performance in detecting attack On the convolutional layer, the numbers of filters used were 32, 64, and 128 on the first, second, and third convolutional layers, respectively.
The filter .
size used on each Conv1D layer was 3 .
quivalent to 3 x 1 in 1D convolutio.
, as it is adept at Widiasari et al Jurnal RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
identifying temporal or local patterns in sequential data.
To avoid overfitting, the dropout rate on the fully connected layer was set to 0.
In addition, the batch size for the CNN model was set to 32 to speed up the training process, while the activation function used on the convolutional layer was ReLU, which helped improve non-linearity in the model.
On the output layer, the The model utilized a sigmoid activation function for binary classification between DDoS or non-DDoS.
Table 1 shows the CNN architecture for DDoS The model consisted of three convolutional layers (Conv1D) with increasing numbers of filters, followed by a fully connected layer.
The output layer used a Sigmoid activation function since this was a binary classification problem (DDoS or non-DDoS).
Table 2 shows regularization techniques in the CNN architecture for DDoS detection.
MaxPooling1D was applied after each convolutional layer to reduce the output dimensionality and help the model to enhance generalization capability.
A dropout rate of 0.
5 was applied to the dense layers to help minimize overfitting.
Batch normalization was applied after each convolutional layer to speed up the training and optimize the modelAos performance.
Table 1.
CNN Architecture for DDoS Detection Layer Input Convolutional Layer 1 Convolutional Layer 2 Convolutional Layer 3 Fully Connected Output Type Conv1D Conv1D Conv1D Dense Dense Output Shape (None, timesteps, .
(None, timesteps-2, .
(None, timesteps-4, .
(None, timesteps-6, .
(None, unit.
(None, .
Table 2.
CNN Architecture for DDoS Detection Layer Pooling Dropout Input Convolutional Layer 1 Convolutional Layer 2 Convolutional Layer 3 Fully Connected Batch Normalization MaxPooling1D Yes MaxPooling1D Yes MaxPooling1D Yes Output Mathematically, the convolution process can be described by the Equation 5:
yayc = OcyaOe1 ycn=0 ycUyc 1 .
yayc is the the result of a convolution operation that shows the effectiveness of the kernel or filter K, which fits a particular portion of the input X at time .
r positio.
ycUyc 1 is the input .
utput of the GRU) at position yc 1, which contains network packet traffic flow in DDoS detection, or time values in time-series data that has been analyzed by the GRU.
OcyaOe1 ycn=0 ycUyc 1 is the sum of all the products of the input values and the kernel values for each position in the filter.
and K: is a CNN filter or kernel, which is used to detect patterns or anomalies in network traffic.
This convolution operation was used to produce features highlighting patterns that could help identify whether the network traffic was part of a DDoS attack or normal traffic.
7 PSO Optimization In this phase.
PSO was used to optimize the model built using GRUAeCNN.
Each particle in the PSO model represented a potential solution consisting of GRUAe CNN parameters, such as weights and biases.
This process began by initializing the particles randomly in Filters/Units Kernel Size Activation ReLU ReLU ReLU ReLU Sigmoid the specified search space.
This random initialization allowed PSO to explore various possible GRUAeCNN parameter configurations in an attempt to find optimal convergence to the expected solution.
There were some steps to perform optimization using PSO.
First, the velocity of each particle is updated by considering its previous velocity, its best-known position, and the global best position found by the swarm as represented in Equation 6.
Then, the position of each particle was updated according to the new velocity, enabling the swarm to gradually move toward the optimal solution in the search space.
= yc.
cyycn,ycayceycyc Oe ycuycn .
) yca2 .
ciycayceycyc Oe ycuycn .
) .
is the velocity of particle i at time t.
yc is the inertia yca1 and yca2 are the cognitive and social coefficients, respectively.
ycycn and yc2 are the random numbers between 0 and 1, respectively.
ycyycn,ycayceycyc is the best location/position identified by particle i.
yciycayceycyc is the best position found by the entire swarm.
and ycuycn .
is the current position of particle i.
For example, suppose a particle's velocity at time t for parameter TCP packets is 0.
5, yc = 0.
7, yca1 = 1.
5, yca2 = 5, ycycn and yc2 are random values .
, 0.
4 and 0.
ycyycn,ycayceycyc = 0.
8, and yciycayceycyc = 1.
Then, the updated velocity might be:
= 0.
7 UI 0.
5 UI 0.
4 UI .
8 Oe .
5 UI .
0 Oe .
OO 0.
35 (Oe2.
(Oe4.
Oe 6.
Position ycuycn .
is updated using new velocity ycuycn .
= ycuycn .
For example, if the current position of the particle for the TCP packets is 6, then:
= 6 (Oe6.
= Oe0.
Each new position of the particle was evaluated using an objective Widiasari et al Jurnal RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
, malware detection accurac.
The best position .
was updated accordingly.
Then, the steps were repeated for a set number of iterations.
These features would form part of the input vector ycuyc for GRUAeCNN.
PSO optimized GRUAeCNN weights and biases to increase accuracy in detecting types .
enign or maliciou.
8 Model Evaluation In this stage, we applied a confusion matrix to determine the necessary requirement for the model.
Some of the used features were false positives (FP), false negatives (FN), true positives (TP), and true negatives (TN).
The confusion matrix was provided to focus on the correctness of the predictions.
We also evaluated our proposed model with metrics commonly applied in DDoS.
Accuracy describes the precision of prediction of the It was used to calculate abnormal or malicious packets generated by the CNNAeGRU model.
Equation 7 represents the accuracy percentage of any DDoS yaycaycaycycycaycayc = ycNycE ycNycA ycNycE ycNycA yaycE yaycA 1 Results During model development, various configuration variations, such as varying numbers of hidden layers and neurons per layer and dropout rates, were tested to find the best combination that could produce optimal This process aimed to explore the potential of each technique used, as well as to understand how each element of the model could result in improved accuracy and efficiency for DDoS The results of the model development are expected to provide further insights into the strengths and weaknesses of each method in the context of network attack detection.
Figure 3 shows performance comparison metrics of the all models used in the study.
The metrics compared included accuracy, precision, recall, and F1-score, expressed as percentages.
The models tested included methods such as LSTM.
DBSCAN.
SMOTE.
PSO, and Random Forest, with various configurations and The findings indicate that the DBSCANAe LSTM model with PSO showed the highest performance compared to other models, especially in terms of accuracy and F1-score.
Precision is the percentage of misidentified intrusion to the total incidents of attacks that occurred.
It is based on Equation 8, which shows the number of correctly predicted positive DDoS detections:
ycEycyceycaycnycycnycuycu .
cE) = ycNycE yaycE ycNycE Recall is the outcome ratio of detecting adverse incidence compared to the overall number of adverse Equation 9 demonstrates the number of true positives which are precisely identified:
ycIyceycaycaycoyco .
= ycNycE yaycA ycNycE F-score or F1-score provides the following information about network functionality which reflects on both false positives and false negatives.
It is useful especially in situations where class labels are imbalanced.
Equation 10 demonstrates the consistency of recall and precision:
yaycycaycuycyce = 2 y ycEyycI ycE ycI Results and Discussions In this research, we aimed to optimize the effectiveness of a hybrid deep learning model that combines DBSCAN.
GRU, and CNN techniques in detecting DDoS attacks.
This approach was chosen to address the increasingly complex challenges of attack detection by utilizing DBSCAN's ability to handle imbalanced data and GRUsAo ability to recognize temporal patterns in large and dynamic data.
This hybrid model was then optimized with various techniques, including regulation and hyperparameter settings, to improve detection Figure 3.
Performance Metrics of Various Models Table 3 presents the outcomes of the performance evaluation of various models in terms of accuracy, precision, recall, and F1-score.
From this table, it can be seen that the DBSCANAeGRUAeCNNAePSO model produced the best performance compared to all other models, with an accuracy of 99.
3%, a precision of 99%, a recall of 98.
9%, and an F1-score of 99%.
This shows a significant performance improvement compared to baseline models, such as SVM and k-NNs, which only achieved accuracies of 95.
8% and 95.
5% and F1-scores 1% and 94.
7%, respectively.
In addition, popular pre-trained deep learning models such as MobileNet and ResNet also showed good performance but were still below the proposed model.
MobileNet produced an accuracy of 97.
9% and an F1-score of 97.
4%, while ResNet achieved an accuracy of 98.
2% and an F1-score Compared to the baseline CNN model (GRUAeCNN), the use of techniques such as optimization using PSO was shown to consistently improve performance.
This strengthens the argument Widiasari et al Jurnal RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
that the CNN model proposed in this research is not only effective, but also outperforms simpler methods and other deep learning alternatives.
Table 3.
Comparison of metrics Model DBSCANAe GRUAeCNN (Single Hidden Laye.
DBSCANAe GRUAeCNN (Double Hidden Layer.
DBSCANAe GRUAeCNN (With Regularizatio.
DBSCANAe GRUAeCNNAe PSO GRUAeCNN GRUAeCNN (With Hyperparamete r Tunin.
Random Forest (Tuned Mode.
Support Vector Machine (SVM) k-Nearest Neighbors .
NN.
Pre-trained MobileNet (Fine-tune.
Pre-trained ResNet (Finetune.
Accuracy (%) Precision (%) Recall (%) F1Score
(%)
The table provides a deeper overview of the stability and generalization ability of each model tested.
The DBSCANAeGRUAeCNNAePSO model again showed the best performance, with the lowest training loss .
and a very small validation loss .
addition, this model recorded a training accuracy of 3% and a validation accuracy of 98.
9%, indicating that this model is not only very accurate on the training and validation data.
The small difference between training and validation accuracies indicates a very good generalization ability and minimal overfitting.
The GRUAeCNN model and the GRUAeCNN model with performance, with validation accuracies of 97.
8% and 3% and relatively low validation losses .
14 and .
, respectively.
Meanwhile, baseline models, such as SVM and k-NNs, had much higher validation losses .
32 and 0.
and lower validation accuracies .
2%).
This indicates that these traditional models are less able to handle data complexity optimally compared to the CNN-based deep learning architecture used in this study.
Popular pre-trained deep learning models, such as MobileNet and ResNet, showed competitive results with validation accuracies of 97.
4% and 97.
7% and validation losses of 0.
17 and 0.
15, respectively.
Although their performance was quite good, these results were still below the proposed models, especially DBSCANAeGRUAeCNNAePSO, both in terms of accuracy and training efficiency.
Table 4 and Figure 4 show the training and validation evaluation results of various models based on training loss, validation loss, training accuracy, and validation Overall.
Table 4 supports the findings from Table 3 that the proposed CNN model does not only excel in accuracy, but is also stable, efficient, and has an excellent generalization ability compared to other benchmark models.
Table 4.
Training and Validation Outcomes Model Training Loss Validation Loss DBSCANAeGRUAeCNN (Single Hidden Laye.
DBSCANAeGRUAeCNN (Double Hidden Layer.
DBSCANAeGRUAeCNN (With Regularizatio.
DBSCANAeGRUAeCNNAePSO GRUAeCNN GRUAeCNN (With Hyperparameter Tunin.
Random Forest (Tuned Mode.
Support Vector Machine (SVM) k-Nearest Neighbors .
-NN.
Pre-trained MobileNet (Fine-tune.
Pre-trained ResNet (Fine-tune.
The results in Table 3 and Table 4 as a whole show that the developed CNN-based approach not only provides accurate prediction results, but also stability during the training and validation process.
The combination of evaluation metrics .
ccuracy, precision, recall, and F1scor.
with training metrics .
oss and validation accurac.
allowed for a more comprehensive analysis of model performance.
In particular.
CNN-based models combined with optimization and regularization Training Accuracy (%) Validation Accuracy (%) methods showed high performance consistency when tested on both training and validation data.
This performance shows that the model is not only superior in terms of evaluation numbers, but also robust in dealing with data variations, making it worthy of consideration as a more effective approach compared to traditional methods or other common deep learning Widiasari et al Jurnal RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
remain a solid choice in many classification tasks.
However, compared to the hybrid model based on DBSCANAeGRUAeCNN.
Random Forest is less effective in handling DDoS attacks with more dynamic and varied patterns.
This is in line with the study conducted by .
, which showed that although Random Forest can provide good results in some cases, deep-learningbased models tend to be superior in tasks involving more complex and large data.
Figure 4.
Model Performance Metrics 2 Discussion The results of this experiment show that DBSCANAe GRUAeCNN significantly improves accuracy in detecting DDoS attacks compared to other simpler The DBSCANAeGRUAeCNNAePSO model, which utilizes Particle Swarm Optimization (PSO) for hyperparameter adjustment, successfully achieved the highest accuracy of 99.
3%, with a precision of 99% and a recall of 98.
These results indicate that hyperparameter optimization through methods such as PSO greatly contributes to the achievement of the best performance, where PSO is able to optimize parameter values configuration to improve model performance.
comparison to other models.
DBSCANAeGRUAeCNN (With Regularizatio.
showed very good performance, with an accuracy of 98.
The addition of regularization to the model helped the model address overfitting, which is a common difficulty in optimizing models, especially on large and imbalanced datasets, such as those often encountered in DDoS detection.
The use of DBSCAN to handle imbalanced data proved to be very effective, which is consistent with findings from previous studies that showed that DBSCAN can help in identifying hidden attack patterns in irregular data .
, .
DBSCAN also helps in reducing the impact of noise that is often present in DDoS datasets, increasing the modelAos effectiveness to generalize and detect unusual attacks.
However, although the simpler GRUAeCNN model without the DBSCAN component achieved an accuracy 8%, the result was slightly lower compared to that of the hybrid DBSCANAeGRUAeCNN model.
This suggests that GRUs are indeed effective in handling sequence-based or temporal data, such as that often encountered in DDoS attacks, where attack patterns can evolve over time.
However, without the help of techniques such as DBSCAN to handle the problem of data imbalance.
GRUAeCNN's ability to identify more complex or less frequent attacks is somewhat limited.
The use of Random Forest in this study also showed quite competitive results, with an accuracy of 98.
indicating that ensemble models such as Random Forest Overall, the results of this study show that the combination of DBSCAN and GRUs in an CNN architecture is very effective for DDoS attack detection, especially when the model is optimized with techniques such as PSO.
The main advantage of this hybrid approach is its ability to handle imbalanced data and identify temporal patterns that are sometimes missed by other models.
Although models such as GRUAeCNN and Random Forest also perform well, the combination of techniques such as DBSCAN provides advantages in detecting more complex and rare attacks.
Therefore, this study contributes to the development of a more robust and accurate DDoS detection model, which can be implemented in network security systems to identify attacks more effectively.
In this discussion, we will compare the results of several recent studies with a focus on intrusion detection using various methods .
- .
Table 5 reveals the various detection techniques applied in these studies, ranging from hybrid methods and signatures, to machine learning and deep learning techniques.
Each study showed varying results in terms of accuracy, precision, recall, and the advantages and limitations of each The table presents a comparison of various intrusion detection systems (IDS) and their effectiveness in detecting different types of attacks, employing different A clear trend toward hybrid approaches emerged, with models like those proposed by Raza et .
Midi et al.
, and Xu et al.
demonstrating exceptional results for detecting DoS and routing attacks.
Hybrid models appear to be effective in balancing the strengths of different techniques, achieving high accuracy and low false alarm rates.
Moreover, deep-learning-based methods, such as those used by Vanitha et al.
, show a growing interest in leveraging advanced machine learning techniques like BiLSTM.
ELM.
GRU, and transfer learning to improve IDS performance across diverse datasets.
These approaches contribute to more robust systems capable of adapting to a wide range of attack scenarios, highlighting the growing role of deep learning and ensemble learning in cybersecurity.
The results presented in the table highlight significant advancements in IDS performance, with models such as that proposed by Xu et al.
achieving perfect scores in accuracy, precision, recall, and F1-score.
Widiasari et al Jurnal RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
Table 5.
Performance Comparison of IDS Methods Raza et al.
Detection Method Hybrid Kasinathan et al.
Cervantes et al.
Signature DoS Hybrid Sinkhole Midi et al.
Hybrid DoS and routing attacks Sharma et al.
Zero-day attacks Xu et al.
Specificationbased Hybrid Vanitha et al.
BiLSTM.
ELM.
GRU Improved ant colony optimization machine-learning-based ensemble Intrusion Detection Rehman et al.
Signaturebased Anomaly detection for enhanced 92% DR, 8% FNR .
70% DR,
28% FNR .
91% DR, 100% accuracy, 0.
CPU usage FNR = 0.
FPR = 0.
TPR =
Achieved perfect scores: accuracy, precision, recall, and F1 = 1.
The new MLEID classifierAos overall findings are 98.
34%, with precision rates for classifiers like DT.
SVM, and Ensemble at 77.
67%, 89.
34%, respectively.
37% accuracy, 91.
4% precision, 2% recall, and 90.
1% F1-score.
Hussain et al.
Rule-based Ahuja et al.
37% accuracy, 98.
5% precision, 3% recall, and 98.
4% F1-score.
Accuracy: 98.
8%, and a very low
false alarm rate
Our Model
SVC,
Random Forest DBSCANAe GRUAeCNNAe PSO DDoS attack in Cyber-Physical Production Systems (CPPS) Hybrid model combining SVC and Random Forest An intrusion detection system for hybrid DoS attacks Accuracy: 99.
Precision: 99%.
Recall: 98.
F1-Score: 99% Reference Publish Year Encountered Attacks Experiment-al Results Sinkhole, spoofing, altered info 100% TPR .
, 90% TPR .
Reduced FPR DoS The focus on reducing false positive rates (FPR), as seen in the work of Kasinathan et al.
, further emphasizes the importance of minimizing unnecessary alarms in practical deployment.
Furthermore, the latest models, like the DBSCANAeGRUAeCNNAePSO model proposed in 2025, achieved impressive results .
accuracy, 99% precision, and 98.
9% recal.
, showcasing the power of combining clustering algorithms with neural networks and optimization These findings demonstrate that ongoing innovation in hybrid models and machine learning techniques is crucial for addressing increasingly sophisticated and dynamic cybersecurity threats, providing more efficient and adaptive solutions for intrusion detection.
The proposed DBSCANAeGRUAe CNNAePSO model showed better performance in DDoS attack detection, with a 99.
3% accuracy, a 99% precision, and a 98.
9% recall.
Compared with other state-of-the-art models, such as the hybrid models of Xu et al.
and Vanitha et al.
DBSCANAeGRUAeCNNAePSO showed superiority in terms of precision and recall, and was more effective in handling imbalanced data problems using DBSCAN.
Although Xu et al.
achieved a perfect score, the DBSCANAeGRUAeCNNAePSO model outperformed the others in detecting more complex and rare attack patterns due to the combination of clustering and optimization techniques.
This makes the model more adaptable to be applied in real-world scenarios, where attack patterns may continue to evolve over time.
Although the DBSCANAeGRUAeCNNAePSO hybrid model is effective in detecting DDoS attacks, there are some limitations related to the dataset used.
The dataset, although large, is limited to a specific type of attacks and may not cover the latest variations or attack techniques found in the real world.
This may affect the model's ability to generalize to more complex and dynamic attack scenarios.
For real-world deployment, several practical considerations must be addressed, such as inference time, computational cost, and resource usage in live network environments.
Inference time directly impacts the systemAos responsiveness, while efficient resource utilization is crucial to ensuring that the approach can scale and operate effectively under real-time This becomes particularly critical in edge computing environments, where available memory and processing power are limited.
Therefore, deployment strategies should include lightweight model optimization techniques such as pruning, quantization, and distributed inference to support scalability and responsiveness in operational settings such as IoT networks or ISP-level monitoring systems.
In addition, the historical data used may not reflect the rapid evolution of DDoS attacks, so the model needs to be tested with more recent and diverse datasets.
Although DBSCAN is effective in dealing with imbalanced data, it may face challenges in highly dynamic and noisy environments, potentially decreasing detection accuracy.
In the future, it is important to use more diverse realtime datasets and develop hybrid approaches that incorporate anomaly detection, real-time learning, and Widiasari et al Jurnal RESTI (Rekayasa Sistem dan Teknologi Informas.
Vol.
9 No.
deployment-aware performance in evolving, high-volume environments.
Conclusions .
The DBSCANAeGRUAeCNNAePSO model proposed in this study showed high efficiency in DDoS attack detection, achieving a 99.
3% accuracy, a 99% precision, a 98.
9% recall, and a 99% F1-score.
The hybrid approach combining DBSCAN for clustering.
GRUs and CNNs for feature extraction, and PSO for optimization sas shown efficacy in improving the performance of DDoS attack detection, even with an imbalanced dataset .
% normal class, 7% DDoS The data used in this research was collected through real-time penetration testing, in which simulated attacks were carried out on a managed This data provided a realistic and valid picture of real-world attacks, with attributes such as time, sources, destinations, protocols, durations, clusters, and anomalies used for model training and evaluation, providing a strong basis for intrusion detection models.
Although the model showed excellent results based on the data collected through live penetration testing, greater challenges arise when the model is implemented in a real operational network.
One challenge is The modelAos adaptability to the highly dynamic and diverse attacks that often occur in real-world environments, which may not always be fully represented in simulated Furthermore, although the model showed high performance in terms of accuracy and detection, implementation in larger and more complex systems requires considerations regarding scalability and efficiency of computing resource usage, especially in real-time detection.
Future research should focus on testing the model in more dynamic and complex operational environments and consider further optimizations so that the model can be effectively integrated into the network infrastructure of enterprises or Internet service providers (ISP.
to deal with the ever-evolving DDoS threats.
Acknowledgements This research received funding from the Vice Rector for Research.
Innovation, and Entrepreneurship (Contract No.
102/SPK-PF/RIK/07/2.
and supported by the Infrastructure and Digitalization Directorate (DID) of Satya Wacana Christian University.
References