Indonesian Journal on Geoscience Vol.
12 No.
1 April 2025: 43-53
INDONESIAN JOURNAL ON GEOSCIENCE
Geological Agency Ministry of Energy and Mineral Resources Journal homepage: h p://ijog.
ISSN 2355-9314, e-ISSN 2355-9306 Hyperparameter Tuning on Machine Learning-Based Landslide Susceptibility Mapping (Case study: Palu City and Its Surrounding area.
Sukristiyanti Sukristiyanti1.
Pamela Pamela2.
Moch Hilmi Zaenal Putra1.
Yukni Arifianti1.
Andri Fachrur Rozie3.
Hilda Lestiana4.
Tri Muji Susantoro5.
Sumaryono2.
Yohandi Kristiawan2, and Iqbal Eras Putra2,6 Research Center for Geological Disaster.
National Research and Innovation Agency (BRIN).
Indonesia Center of Volcanology and Geological Disaster Mitigation.
Geological Agency.
Indonesia Research Center for Data and Information Sciences.
BRIN.
Indonesia Research Center for Geological Resources.
BRIN.
Indonesia Research Center for Geoinformatics.
BRIN.
Cibinong.
Indonesia Magister Program in Regional and City Planning.
School of Architecture.
Planning and Policy Development.
ITB.
Indonesia Corresponding author: sukristiyanti @brin.
Manuscript received: December, 01, 2023.
revised: January, 19, 2024.
approved: February, 10, 2025.
available online: February, 27, 2025 Abstract - Landslide susceptibility mapping (LSM) produces a zonation map of landslide susceptibility levels, representing the future probability of landslides.
It is necessary to give a guideline regarding spatial planning.
A machine learning method was used, namely a random forest (RF) algorithm to map landslide susceptibility in Python.
The case study is Palu City and its surrounding areas, which were attacked by a big earthquake on September 28th, 2018.
Some earlier LSM studies did not discuss hyperparameter tuning, and several others did not mention the training accuracy.
Therefore, this study is to find out whether the fast model without hyperparameter tuning and frequently overfitting, can well produce landslide susceptibility maps.
The research questions were addressed by comparing two landslide susceptibility maps built with and without hyperparameter tuning using receiver operating characteristics (ROC) and landslide density (LD) analyses.
This study shows that the area under the curve (AUC) of the landslide susceptibility map from the fast RF model without hyperparameter tuning is as high as the AUC from the tuned model map.
It also happened in both landslide density (LD) maps, and there is no anomaly in the fast model map.
Nevertheless, there are strange appearances in the fast model map.
Therefore, hyperparameter tuning to obtain the optimal model with no overfitting is mandatory to predict landslide susceptibility spatially.
Keywords: landslide susceptibility mapping (LSM), random forest, machine learning, hyperparameter tuning.
Python
A IJOG - 2025
How to cite this article:
Sukristiyanti.
Pamela.
Putra.
Arifianti.
Rozie.
Lestiana.
Susantoro.
Sumaryono.
Kristiawan.
, and Putra.
, 2025.
Hyperparameter Tuning on Machine Learning-Based Landslide Susceptibility Mapping (Case study: Palu City and Its Surrounding area.
IndoAnesian Journal on GeoAscience, 12 .
, p.
DOI: 10.
17014/ijog.
Introduction Landslide as a natural disaster needs a lot of attention since its effect can destroy infrastructure, kill people, and cause many more losses that come with it (Harsa et al.
, 2.
A landslide susceptibility map indicates the landslide-prone areas, which can be used by planners or policy-makers, scientists, and the community for the landslide-disaster risk reduction effort.
Machine learning-based landslide Indexed by: SCOPUS Indonesian Journal on Geoscience.
Vol.
12 No.
1 April 2025: 43-53 The causative factors consist of twelve layers, i.
, aspect, elevation, slope gradient, lithology, peak ground acceleration (PGA), land use, distance to roads, distance to rivers, distance to lineaments, road density, river density, and lineament density.
This study utilized the causative factors as independent types with the successfully inventoried landslides to do the LSM.
The output of LSM is a zonation map of landslide susceptibility levels, which depicts the probability of landslides in the future (Tyagi et al.
, 2.
The machine learning method is a novel method to conduct LSM (Reichenbach et al.
belonging to data-driven models, which tend to be able to avoid subjectivity compared to other previous methods.
Machine learning methods perform more see-through calculations and improve accuracy (Chen et al.
, 2.
Bivariate statistical-based LSM is a data-driven model (Nohani et al.
, 2019.
Arifianti et al.
, 2.
However, it needs categorical type data for each landslide causative factor, and there is lack of standard to classify each parameter instead of using heuristic classification (Chen et al.
, 2.
In the machine learning, various data types can be run together, even continuous and nominal-type data (Cao et , 2.
, and then it can avoid subjectivity.
Some continuous data represent geo-environmental values close to actual values, like earth surface altitude, slope gradient, distance to rivers, etc.
susceptibility mapping (LSM) methods have recently become popular (Kavzoglu et al.
, 2.
, since information technology is growing fast.
It makes machine learning modeling can be run lightly and easily nowadays.
The abundance of geospatial data such as satellite images, remotely sensed data, and historical landslide data in the last decade supports This mapping method is suitable on a medium scale (Westen et al.
, 2.
for quite large areas, like in this studied area (Barredo et al.
, 2.
The location of the study is Palu City and its surrounding areas.
The lithology composition of this area mainly consists of Holocene alluvial deposits in the centre, characterized by a plain This area is encompassed by two hills on the right and left, composed of Pleistocene Pakuli Formation, a Miocene granite intrusive rock exposed in the eastern part of the studied area.
The Upper Miocene Lariang Formation, underlain by metamorphic rocks of the Latimojong Formation, dominates the western part (Sukamto, 1.
The studied area is located along the Palu Koro Fault Valley, flanked by two hills on the right and left sides.
Palu City faced a big 7.
5 Richter Scale earthquake on September 28th, 2018, which caused 4,340 fatalities (Arifianti et al.
, 2.
The shakes triggered the landslides in the hills arround the city (Figure .
The characteristics of Palu acted as landslide causative factors were compiled into a geoda- Figure 1.
Landslide distribution map in the studied area.
Hyperparameter Tuning on Machine Learning-Based Landslide Susceptibility Mapping (Case study: Palu City and Its Surrounding area.
(Sukristiyanti et al.
Bagging produces many new training datasets.
Each new training dataset takes a sample from the original dataset with a replacement .
ootstrap sampl.
, i.
, it takes a sample, returns it to the original dataset, and takes another sample at random.
By sampling with replacement, the same sample can be repeated in each new training dataset, and this causes the new dataset to be different for each tree (Taalab et al.
As a supervised learning method.
RF needs a response variable or desired output (Muller and Guido, 2016.
Islam et al.
, 2.
The twelve landslide causative factors were prepared with a cell size of 10 m.
TerraSAR X satellite data with a cell size of 7.
5 m was used to get all the elements that make up the digital elevation model (DEM), such as aspect, elevation, and slope gradient (Arifianti et al.
, 2.
Because LSM is a binary classification, it requires two classes for the response variable: landslide location and non-landslide location (He et al.
, 2.
Landslide locations were obtained from landslide inventory, while nonlandslide locations were constructed from sampling on the two lowest classes of bivariate statistical landslide susceptibility map produced by a previous study (Arifianti et al.
, 2.
The number of landslides is 591 points, and the same number of nonlandslides were generated (Zhang et al.
, 2.
All the thematic layers and the response variables were prepared in the geographic information system (GIS) environment.
Three causative factors have nominal type data, , lithology, aspect, and land use, while the rest have continuous type data.
The slope gradient is one of the mandatory parameters for landslide susceptibility study .
Nellek, 2.
It influences slope stability, where a more significant slope gradient makes the slope likely to collapse if the shear stress exceeds its shear strength.
The elevation factor was chosen in this modeling since it may relate to rainfall and vegetation types, affecting landslides.
Especially for earthquake-induced landslides, the ground tremor will be higher if the elevation increases.
The slope aspect refers to sun exposure, which affects weathering levels and is closely connected to landslide probability.
Many prior LSM studies used machine learning models.
Some researchers used one (Yao et , 2008.
Taalab et al.
, 2.
or more machine learning algorithms (Kalantar et al.
, 2018.
ElMagd et al.
, 2021.
Shahzad et al.
, 2.
Those algorithms are decision trees, random forests (Taalab et al.
, 2018.
El-Magd et al.
, 2021.
Shahzad et al.
, 2.
, support vector machines (Shahzad et al.
, 2.
, nayve Bayes (El-Magd et al.
, 2.
k-nearest neighbour (El-Magd et al.
, 2.
, and many more.
This LSM study applied random forest (RF), considering previous researchers proved this method superior (Chen et al.
, 2.
It is a nonlinear algorithm suitable for overcoming classification problems in complex phenomena like landslide occurrence (Cao et al.
, 2.
The RF model requires hyperparameter tuning to optimize it (Sun et al.
, 2.
An optimum model has test accuracy lower than training accuracy, and the training accuracy is not equal 0 or overfitting (Muller and Guido, 2.
Overfitting occurs when a model is too complex and too fit to the particularities of the training data (Muller and Guido, 2.
Test accuracy compares the prediction of test data with the actual test data, while training accuracy compares the prediction of the training data with the actual training data.
The test data is not involved in the modeling, whereas the training data is used to build a model.
Some previous LSM studies did not mention training accuracy or discuss hyperparameter tuning.
Therefore, this paper aims to study whether the fast model without hyperparameter tuning, which frequently overfits, can produce good landslide susceptibility maps.
This study answered the research question by looking at two landslide susceptibility maps that were made with and without hyperparameter tuning.
The maps were made using receiver operating characteristics (ROC) and landslide density (LD) analysis (Chen et al.
, 2.
Methods Random forest (RF) is an ensemble method that combines many decision trees.
The training dataset was built using the bagging .
ootstrap Indonesian Journal on Geoscience.
Vol.
12 No.
1 April 2025: 43-53 Table 1.
Default Random Forest Parameters Parameter n_estimators max_depth min_samples_split min_samples_leaf min_weight_fraction_leaf max_features max_leaf_nodes min_impurity_decrease oob_score class_weight max_samples Value None None True False None None Meanwhile, lithology influences soil permeability and strength associated with slope gradients.
The lithology map is scaled to 1:50,000 obtained from the Centre for Geological Survey of Indonesia.
Peak ground acceleration (PGA) is assumed to be an earthquake-leading indicator and is essential as a driving factor for earthquakeinduced landslides.
Besides, the lineament element was used as an independent factor.
Faults and fractures were added in the lineament element, which showed the weak zone regarding lithological characteristics.
Lineaments also control the existence of rivers structurally (Ngo et al.
, so river element was used to contribute to this modeling.
Two factors based on rivers used here were the distance to rivers and river density.
Land use may control slope stability.
is enhanced by vegetation regarding mechanical Land use is one of the anthropogenic factors where land-use changes affect the slope stability.
Regarding anthropogenic factors, this study considered roads as an essential element, so distance to roads and road density layers were utilized in this modeling.
The street development alters the slope stability of natural slopes since there is a cut-and-fill activity (Awawdeh et al.
, 2.
A total of 1,182 samples were used as input in Python.
70 % was used as training data, and the rest for test data (Zhang et al.
, 2.
Each data point has a label, whether landslide or nonlandslide, bringing twelve geo-environment values.
Besides, each sample has a geographical location: longitude and latitude.
The coordinates for all data points are essential to trace which data points are landslides for training and test data, etc.
Geographical locations are needed for landslide susceptibility map validation using receiver operating characteristics (ROC) (Pourghasemi et al.
, 2.
Scikit-learn was applied, a Python package widely used for data science, to implement a random forest (RF) algorithm for landslide susceptibility mapping (LSM).
The default values of the model parameters were used to build a fast RF model, as shown in Table 1.
Aside from that, hyperparameter tuning was done to create the tuned The most common method, grid search, was used for this hyperparameter tuning.
Grid search tries all possible combinations of desired model parameters (Muller and Guido, 2.
In this study, five model parameters were tuned as follows: the number of decision trees .
_estimator.
, maximum depth of all trees .
ax_dept.
, minimum number of samples to split .
in_samples_spli.
, minimum number of samples in the leaves .
in_samples_lea.
, and maximum number of independent variables in all trees .
ax_feature.
Model parameters define the random forest and decision tree structure (Figure .
, where numerous decision trees compound a random forest.
Every decision tree in the random forest (RF) is different because of the nature of randomness in it, like the samples, the number of independent variables, and which variables are involved in each decision tree, and so on.
Fast and tuned models were evaluated using the accuracy score in the Python environment.
Two kinds of accuracy scores were performed, , training and test accuracy scores.
The test accuracy is the model accuracy which compares Root node Intemal node Depth = 1 Leaf node Leaf node Leafnode Leaf node Figure 2.
Decision tree diagram.
Depth = 2 Hyperparameter Tuning on Machine Learning-Based Landslide Susceptibility Mapping (Case study: Palu City and Its Surrounding area.
(Sukristiyanti et al.
Data Bivariate statistical landslide susceptibility map Landslide inventory Geodatabase consisting of 12 range of AUC is between 0.
0 to 1.
0 with details as follows: 0.
5 - 0.
, 0.
6 - 0.
, 7 - 0.
, 0.
8 - 0.
ery goo.
, and more 9 is excellent (Pourghasemi et al.
, 2.
Besides, landslide density (LD) analysis was used to compare two landslide susceptibility The previous study used LD analysis to compare the landslide susceptibility maps from three different models (Chen et al.
, 2.
The LD is well-defined as the percentage of landslides in each class (PL) by the percentage of area in each class (PA).
PA is the area of each class divided by the total area of the researched area, and PL is the count of landslides divided by the total number of Accurate landslide susceptibility maps should display a higher LD for classes of higher susceptibility (Chen et al.
, 2.
The flow chart of this method can be seen in Figure 3.
the prediction of test data with the actual test Meanwhile, the training accuracy compares the prediction of the training data with the actual training data.
Whether the model is overfitting can be determined by calculating training and test accuracy.
These two models were used to generate spatial prediction results in the form of spatial landslide susceptibility indices, and to classify each index map into a landslide susceptibility map.
The index ranges from 0.
0 to 1.
0, and is categorized into five susceptibility classes: very low, low, moderate, high, and very high.
To compare the two resulting maps from the fast model and the tuned ones, they were evaluated by calculating the area under the curve (AUC) of the receiver operating characteristic (ROC) (Pourghasemi et , 2.
beyond the Python environment.
The Two lowest classes 591 landslide points 591 non-landslide Random forest (RF) algorithm No hyperparameter tuning With hyperparameter tuning The fast RF model The tuned RF model Evaluation of the fast model Evaluation of the turned model Landslide susceptibility map from the fast RF model Landslide susceptibility map from the tuned RF model Evaluation of the landslide susceptibility Figure 3.
Flow chart of this study.
Indonesian Journal on Geoscience.
Vol.
12 No.
1 April 2025: 43-53 Result In this RF modeling, 1,182 data points were half are landslides, and the other half are Some 827 samples .
%) were used to train the model, and 355 were used to test the model.
The twelve processed causative factors, which are superimposed with landslide and nonlandslide points, can be seen in Figure 4.
99 E 119.
99 E 119.
Distance to lineaments .
Lineament density .
/sq .
Elevation .
Lineament Slope( ) Lineament Distance to rivers .
PGA (Ga.
River Road density .
/sq .
Distance to roads .
River Road Road River density .
/sq .
Lithology Aspect Alluvial Pakuli Formation Lariang Formation Intrusive Rocks Landuse Latimojong Formation Flat River 25 km Grassland RiceAeld Farmland Shrubs Sand Landslide and Non-landslide Landslide Non-landslide Figure 4.
Twelve landslide causative maps were superimposed with landslide and nonlandslide points.
Forest Waste Residential Hyperparameter Tuning on Machine Learning-Based Landslide Susceptibility Mapping (Case study: Palu City and Its Surrounding area.
(Sukristiyanti et al.
for each class using GIS tools.
The values were plotted in a curve in which the x-axis represents the cumulative percentage of the area, and the y-axis represents the cumulative percentage of The results showed that the AUC of landslide susceptibility maps from the fast RF model is 0.
897, and from the tuned model is 893 (Figure .
AUC of Landslide Susceptibility Maps cumulative percentage of landslide Two landslide susceptibility models were conducted, one with hyperparameter tuning and not on another one.
In the fast model, hyperparameter tuning was not employed.
the model is simplistic as it solely utilizes the default values, hence conserving time.
Hyperparameter tuning was done using the grid search method involving five model parameters.
The tuned model is the best constructed by the following parameter values: max_depth was 7, max_features was 3, min_samples_leaf was 2, min_samples_split was 5, and n_estimators was 150.
The tuned model consists of 150 decision trees with a default value of 100.
All decision trees have seven depths maximum, showing that they are not too long to reach the leaves.
The causative factors used in each decision tree maximum is 3 .
ax_features= .
, not much involving most of the variables.
The decision trees in the random forest will differ (Muller and Guido, 2.
, because the randomness in picking the variables is high, considering that the total causative factors involved are 12.
Both RF, the fast RF and the tuned model, have the same model accuracy of 0.
963, but the training accuracy of the fast RF model is 1.
which shows there is overfitting.
Conversely, hyperparameter tuning can avoid overfitting (Table Both models were used to generate landslide susceptibility index maps.
from the fast RF model = 0.
from the tuned RF model = 0.
cumulative percentage of area Table 2.
Performances of the Random Forest Models Training Test accuracy Fast RF model Tuned RF model ROC analysis was performed to obtain the map accuracy identified by their AUCs, which compared the landslides in test data with the landslide susceptibility map.
In this analysis, each landslide susceptibility index map was categorized into thirty-two classes using the geometrical interval classification method.
The number of landslides and the area were calculated Figure 5.
The area under the curve (AUC) represents the quality of landslide susceptibility maps.
Besides, landslide density (LD) was calculated.
The previous study used the formula to compare landslide susceptibility maps, generalized from three different machine learning models (Chen et al.
, 2.
In this study, the LD analysis observed the accuracy of every landslide susceptibility map.
The landslide susceptibility maps (Figure .
were classified into five classes using the geometrical interval classification In this LD analysis, landslides are the total landslides used for the training and test PL means the percentage of landslides in every class.
PA means the percentage of the area of each class, and LD is the landslide density of each class (Table .
Accurate landslide susceptibility maps should display a higher LD for classes of higher susceptibility (Chen et al.
, 2.
, which agrees with LD in the two maps.
Figure 6 presents the maps, zooming in on specific sections.
Indonesian Journal on Geoscience.
Vol.
12 No.
1 April 2025: 43-53 Palu Bay Palu Bay Figure 6.
Landslide susceptibility maps using the RF algorithm, from: .
Fast model, .
Tuned model, and .
the Zoom-in of the anomaly appearances.
Table 3.
Landslide Density (LD) of Two Landslide Susceptibility Maps The map from the fast RF model The map from the tuned RF model Moderate High Very high Landslide susceptibility class Very low Low Discussion The accuracy of the model or the map is relatively high, > 0.
8, while the fast RF model 0, showing overfitting.
It happens when a model is too complex and too fit to the particularities of the training data (Muller and Guido.
The overfitting in the fast RF model can show an anomaly in the resulting map (Figure Conversely, hyperparameter tuning can avoid overfitting, and the resulting map looks better.
The very high class on the susceptibility maps has the highest LD (Table .
Both maps have similar LD, which means that there is no anomaly in the LD of the fast model map.
The fast and tuned model maps are shown in Figure 6a and Figure 6b, respectively.
The zoomed-in map presented in Figure 6a exhibits unusual features, as illustrated in Figure 6c.
Figure 6c displays two crescent shapes, whereas Figure 6d, which provides a zoomed-in view of the tuned model map (Figure 6.
, does not exhibit any such anomalies.
Hyperparameter Tuning on Machine Learning-Based Landslide Susceptibility Mapping (Case study: Palu City and Its Surrounding area.
(Sukristiyanti et al.
Training Sweet spot Accuracy Conclusions This study is to find out whether the fast model without hyperparameter tuning, frequently overfitting, can well produce landslide susceptibility maps.
The result shows that the area under the curve (AUC) of the landslide susceptibility map from the fast RF model without hyperparameter tuning is as high as the AUC from the tuned model map.
It also happened in both map landslide density (LD), and there is no anomaly in the fast model map.
Nevertheless, there are strange appearances in the fast model map.
can be concluded that hyperparameter tuning to obtain the optimal model with no overfitting is mandatory to predict the landslide susceptibility The model accuracy has to be calculated toward training and test data to observe whether there is overfitting.
If overfitting happens, it is encouraged to do more hyperparameter tuning until the model has no overfitting.
Generalization UnderAtting training accuracy of 1.
Therefore, a significant effort in hyperparameter tuning is required to build an optimal random forest model and prevent The calculation of AUC (Figure .
and landslide density (LD) (Table .
for the no-hyperparameter model map did not reveal any anomalies.
However, it is evident that hyperparameter tuning is essential until overfitting is eliminated in the model.
A model is the fast model or the tuned model, which was built based on the training data, pairs of input, and target.
In this study, the input, or predictor variable, is the landslide causative factor, and the target is landslides and non-landslides.
A supervised algorithm, like the random forest (RF), learns the relationship between the input and target to produce a model to predict the new The optimal model lies in the optimal range of model complexity, avoiding both overfitting and underfitting (Figure .
OverAtting Model complexity Figure 7.
Model complexity trade-off versus training and test accuracy (Muller and Guido, 2.
The underfitting model shows the model is too simple, and there is a lot of generalization in creating the model.
The underfitting model has low accuracy both in training and test accuracy.
Conversely, the overfitting model achieves the highest training accuracy, yet it struggles to make accurate predictions.
Although it is already a principle in machine learning modeling, several studies have given little consideration.
Overfitting generally occurs when a model is created without parameter tuning or when tuning is performed but not to the best of its ability.
For example, this research used many model parameters, those are five parameters.
Each parameter utilized multiple input values during the grid search process to ensure that the resulting model did not have a Acknowledgements This research was funded through the Programme House of Research Organization for Electronic And Informatics (OREI) of the National Research and Innovation Agency (BRIN).
The authors would like to thank the Head of Research Centre for Geological Disaster (BRIN) who has encouraged the authors to carry out this research.
References