Inform : Jurnal Ilmiah Bidang Teknologi Informasi dan Komunikasi Vol.
8 No.
2 July 2023.
P-ISSN : 2502-3470.
E-ISSN : 2581-0367 Design and Implementation of Artificial Intelligence-based Real-time Pothole Detection for IoT Smart Infrastructure Rahardhita Widyatra Sudibyo1.
Akhmad Alimudin2 Department of Electronic Engineering.
Politeknik Elektronika Negeri Surabaya.
Indonesia Department of Multimedia Creative Technology.
Politeknik Elektronika Negeri Surabaya.
Indonesia widi@pens.
id (*) 2alioke@pens.
Received: 2023-04-27.
Accepted: 2023-05-30.
Published: 2023-06-05 AbstractAi The Internet of Things (IoT) has been extensively deployed for Smart Cities due to its ability to process many different and heterogeneous end systems.
The IoT innovation encourages artificial intelligence applications to process data.
In Smart City infrastructure, the road is a critical component of transportation infrastructure that supports the economic, social, and cultural things of community life and various aspects of community life.
Road conditions affect a variety of community activities.
Good roads enhance comfort and support local However, many roads remain in bad condition, such as potholes.
Various methods have been attempted to identify potholes, especially the two-dimensional imaging method.
This paper proposes the real-time Artificial Intelligence detection of potholes using the Convolutional Neural Network (CNN), which leverages the Edge Tensor Processing Unit (TPU) with the MobileNet SSD v2.
The system was set up on a Jetson Nano with a few extras, including a camera and GPS, to support the IoT infrastructure.
Evaluation for the model consists of device implementation, model evaluation.
GPS position deviation, and on-road implementation.
The effectiveness is confirmed through experiments using a system test-bed that generates ideal mAP off 0.
22 and recall values.
KeywordsAi IoT.
AI.
Pothole Detection.
CNN.
MobileNet SSDV2.
improves its accuracy by 97.
08 percent .
When analyzing images with the CNN .
deep neural networ.
, its computational power is limited .
Since real-time surveying is portable, having a portable device with adequate processing power is important.
Because of this, the Jetson Nano .
and AI Accelerator .
are used in Systems-on-a-module (SoM.
such as Jetson Nano are popular computing modules for robotics and artificial intelligence use.
In embedded systems.
NVIDIA products, such as Jetson Nano, are frequently used for machine learning because of its small size and low power consumption .
Specifically.
Google's own custom ASIC, the Edge TPU coprocessor .
, the AI accelerator speeds up the infringing process on Machine Learning models .
Google's AI accelerator developers have found success with and recommend using MobileNet SSD V2, combining MobileNet V2 and SSD (Single Shot Detecto.
to form a CNN model .
This paper proposes a systemic design and an implementation for detecting potholes with the Mobilenet SSD V2 model using CNN on Jetson Nano.
The images of the potholes and their GPS coordinates will be sent to the server for analysis and display on the website.
The system was installed on a test bed to perform performance evaluations using the Jetson Nano.
It then conducted experiments to validate the proposal.
The rest of this article is structured as follows: Section II shows the study's related works and discusses the system's Section i evaluates the proposal through test-bed Finally.
Section IV concludes this paper with future works.
INTRODUCTION
The Internet of Things (IoT) has been widely adopted worldwide because it can transparently and seamlessly integrate various disparate end systems .
With the innovation of IoT technology and the increased use of mobile devices, a brand-new computation paradigm, edge computing, is growing rapidly.
Meanwhile, artificial intelligence (AI) applications are thriving due to deep learning breakthroughs and various hardware architecture improvements .
In addition to transportation, roads play a significant role in the community's market, societies, and cultures.
The condition of the roads impacts several activities in the community.
Road conditions will improve the quality of life in a given area and facilitate economic activity.
Despite the efforts to improve, there are still areas of poor road quality, such as potholes.
reported by Indonesia's Ministry of Public Works and Housing, on the north coast road of Java Island, there are 3,338 potholes.
Roads too damaged for repairs must be marked with a traffic sign or signs, according to Indonesian Law Number 22/2009 No.
24 paragraph .
This necessitates a damage survey and a public information system to display those results.
Then, with the use of AI in cameras, it can help locate potholes.
Hence, road damage detection has been explored extensively .
- .
AI is one of the most widely used methods for detecting objects in cameras today .
A Deep Learning algorithm.
Convolution Neural Network (CNN), is widely used in Machine Learning.
In light of .
, using CNN in the introduction yields good results.
When the CNN base and ResNet model are used, the pothole detection system DOI: http://dx.
org/10.
25139/inform.
Inform : Jurnal Ilmiah Bidang Teknologi Informasi dan Komunikasi Vol.
8 No.
2 July 2023.
P-ISSN : 2502-3470.
E-ISSN : 2581-0367 Our proposal is built on the TensorFlow Object Detection API .
In order to train networks on MobileNet SSD V2, this API was used.
II.
RESEARCH METHODOLOGY
Related Works Here, the authors briefly touch on some other works relevant to this paper's topic.
The research .
presents multiple network models for the classification of road damage.
They ascertained which algorithm works better for detecting and classifying road damage.
The harm is categorized into potholes, cracks, and damage that exposes the road.
This research employs R-CNN and faster R-CNN for road damage object detection and uses SVM for classification, and it achieves better results than previous research.
Authors present .
develops a method for retrieving images of asphalt road holes based on their degree of damage by extracting features from the GLCM.
This research aims to develop a retrieval system capable of detecting road damage based on its severity.
Extraction of texture features from the GLCM.
Specifically, 52 features were taken from 13 features at 0 degrees, 45 degrees, 90 degrees, and 135 degrees.
The research .
discusses evaluating CNN has used several models with front-facing smartphone cameras to determine whether or not a road region contains potholes.
They used a DCNN model.
ResNet v2 152.
ResNet v2, and MobileNet v1 are utilized to detect a pothole on the road The study's results indicate that the best-performing models can accurately detect potholes in the road surface 5 percent of the time.
Furthermore, they show that converting input frames to grayscale effectively boosts performance when detecting road potholes.
The research .
proposes a CNN-based approach for classifying images as pothole/non-pothole that achieves better results than common SVM-based algorithms.
A total of 13,244 images were used in the training process with a wide range of lighting, subject matter, and frame dimensions.
The model performed exceptionally well when tested on new data, achieving an accuracy of 99.
80%, a precision of 100%, a recall of 99.
60%, and an F1-Score of 99.
However, results suffered greatly from varying illumination levels and significant gaps in the available data.
emphasizes the importance of transfer learning.
The CNN technique was demonstrated to identify and locate potholes in images.
The proposed model is a spinoff of the YOLO model that reduces the neural network's computational costs and model size.
From 48 million, the YOLO model's parameters are cut down to 18 million across 27 layers.
Compared to the original YOLO architecture, the new modified design is smaller and runs faster while achieving higher average precision and recall scores.
In this review, the CNN model static picture was used by the majority of papers.
However, no paper applies the model to real-world scenarios involving actively moving vehicles.
a result of the survey, we developed a system for detecting potholes by CNN on Jetson Nano to the MobileNet SSD V2 model, which should be able to locate and list road defects Figure 1.
MobileNet SSD V2 Structure The pre-trained SSD MobileNet V2 FPNLite 320x320' model is used.
The data sets were formed from a combination of images taken from the internet, data test/training ratio of 70%:30%.
Each TFRecord in the output directory was generated from the TFRecords in the input directory.
The annotation information for each TFRecord file was included.
Figure 1 displays the framework of a MobileNet SSD V2 .
In order to extract the feature from the input picture data, the network is split into two parts: combined efforts of a MobileNet v2 CNN model and a full-convolutional network using SSD for ROI proposal .
Machine Learning Model Training Addition of an AI Accelerator on the Jetson Nano Figure 2.
System Architecture DOI: http://dx.
org/10.
25139/inform.
Inform : Jurnal Ilmiah Bidang Teknologi Informasi dan Komunikasi Vol.
8 No.
2 July 2023.
P-ISSN : 2502-3470.
E-ISSN : 2581-0367 Pothole detection is achieved in the post-processing step using the quantization technique, which employs a low-power, low-cost device for instantaneous monitoring.
An integer approximation of floating-point computations used in neural networks for quantization during training, model accuracy can be restored to very close to the original by simulating the impact of quantization.
Additionally, there is a size reduction of 4 times and an inference efficiency boost of 15% .
In order to use the AI accelerator, after the quantizing model was complete, further processing was necessary.
The compilation is necessary to pass model-specific operations to the USB accelerator.
the central processing unit handles everything else.
In this study, the TensorFlow Object Detection API was used for inference.
The File Uploader and CNN inference process both listen to the temporary folder and upload the picture to the server over the internet.
As illustrated in Figure 2, the system uses this architecture process.
Figure 4.
Electrical Block Diagram Figure 4 illustrates the electrical block diagram used in this implementation.
Device Implementation Adding an AI accelerator to the Jetson Nano allows the construction of a system with a compact design and ease of processing while maintaining the required processing power.
Figure 5.
Geographic Information System
TABLE I
DEVICE SPECIFICATIONS
Figure 3.
System Integration Architecture Figure 3 illustrates the integration architecture for each Input, processing, connectivity, and output are all system components.
Due to the quality of the source materialAia rear-mounted car cameraAian AV video capture device is required to transform the raw footage into a format that the Jetson Nano can read.
The device employs a small, modified USB connector to reduce the overall size.
The Jetson Nano is integrated into the AI accelerator for its processing.
mini-display unit, an audio-visual capture card, a wireless LAN dongle, and some other peripherals were all wired straight into the controller USB port.
The 3D-printed case provides structural integrity and makes the implemented device more user-friendly.
A battery, controller, and other required parts will round out the finished The necessary on/off button, switches controls, charging, and RCA ports are all set up.
Also included is a mini digital voltmeter for checking battery life.
The photos of the potholes are shown on a dynamic The geographic information system appearance plots the marker concerning the actual pothole location.
In this paper, an interactive map was developed using the free and open-source Leaflet JavaScript library.
Images are uploaded to DOI: http://dx.
org/10.
25139/inform.
Inform : Jurnal Ilmiah Bidang Teknologi Informasi dan Komunikasi Vol.
8 No.
2 July 2023.
P-ISSN : 2502-3470.
E-ISSN : 2581-0367 national road between Surabaya City Ae Banyuwangi City Ae Jember City.
a server using the PHP POST method, with coordinates incorporated into the filename.
Figure 5 displays a website that was set up.
Table I reveals the device and software specifications used in the experiments.
One Jetson Nano is employed as the main controller.
TABLE II
HYPER PARAMETER TO TRAIN CNN
Total Images 1700 Pothole images Data Test i.
RESULT AND DISCUSSION
Data Training Device Implementation This pre-trained model was used to train the Predictor, and the optimal combination of hyper-parameters was obtained by experimenting with different learning rates.
different learning rate was used to train the model, and only a thousand iterations of transfer learning were used to calculate the corresponding loss value.
The training parameter is shown in Table II, where the pothole image is collected on the Batch Size As illustrated in Figure 6, both 0.
01 and 0.
10 make for excellent learning rates.
Seventy-five thousand iterations of Single-Predictor transfer learning at a learning rate of 0.
64-batch maximum was used in the gradient descent update, one of the procedures.
The total loss of a CNN model generated using the COCO Evaluation Metric method is 0.
and the mAP is 0.
Figure 6.
Per-step Lost with Various Learning Rate Values pothole detection result based on the tested data.
The figure also shows how road roughness and engine vibration cause the tested image to blur at high speeds.
Likewise, the testing process demonstrates that the faster the vehicle detects, the fewer correct results it generates.
Evaluation of Model Training The inference procedure was executed on each device following the acquisition of videos of a car driving at various Table i summarizes the results of these experiments.
From this data, the total of FP is clearly relatively high, and night-time result performs extremely poorly, but the detection process itself is quite fast.
Figure 7 demonstrates the TABLE i DETECTION IN DIFFERENT SPEED DOI: http://dx.
org/10.
25139/inform.
Inform : Jurnal Ilmiah Bidang Teknologi Informasi dan Komunikasi Vol.
8 No.
2 July 2023.
P-ISSN : 2502-3470.
E-ISSN : 2581-0367 Speed (Km/.
Total .
Night .
Speed 30-40 km/h Figure 7.
Conditions on A Different Speed GPS Position Deviation To calculate the GPS error, compare the observed and actual locations.
Vincenty's formulas .
determine the distance between two latitudes and longitudes.
We gathered the data from various pothole spots and sent to the server.
The result is shown in Table IV.
a standard deviation of 1.
5 meters exists between the actual location of the pothole and the location due to the evaluation process.
Speed 0-10 km/h TABLE IV
GPS DEVIATION
Speed 10-20 km/h Deployment in the Field Device configuration in the situation represented.
See the evaluations in Figures 8 and 9 for how the device is actually The camera is mounted in the vehicle's front end and controlled from within.
Surabaya.
Indonesia, was chosen as the location for the survey.
Speed 20-30 km/h Figure 8.
Device Implementation on Vehicle DOI: http://dx.
org/10.
25139/inform.
Inform : Jurnal Ilmiah Bidang Teknologi Informasi dan Komunikasi Vol.
8 No.
2 July 2023.
P-ISSN : 2502-3470.
E-ISSN : 2581-0367 forth a fresh approach to pothole warnings and adaptive suspension for future cars.
This tool and a lidar array can determine how rough a road surface is.
A more cohesive outcome calls for both improved datasets and updates to the CNN model are being made.
ACKNOWLEDGMENT
The authors would like to thank Politeknik Elektronika Negeri Surabaya.
PT.
Telekomunikasi Indonesia and the Ministry of Education.
Culture.
Research, and Technology for the support in the realization of the EMSiT (EEPIS Mobile Smart City Based on IoT).
The authors would also like to express their appreciation to the engineers who helped with the system's deployment and supplied the experimental data and technical documentation for the EMSiT project.
Figure 9.
Camera Implementation on Vehicle.
REFERENCES