International Journal of Electrical and Computer Engineering (IJECE) Vol.
No.
June 2025, pp.
ISSN: 2088-8708.
DOI: 10.
11591/ijece.
Investigation of the satellite internet of things and reinforcement learning via complex software defined network modeling Arun Kumar1.
Sumit Chakravarty2.
Aziz Nanthaamornphong3 Department of Electronics and Communication Engineering.
New Horizon College of Engineering.
Bengaluru.
India Department of Electrical Engineering and Computer Engineering.
Kennesaw State University.
Kennesaw.
United States College of Computing.
Prince of Songkla University.
Phuket.
Thailand
Article Info
ABSTRACT
Article history:
The satellite internet of things (SIoT) has emerged as a transformative technology, enabling global connectivity and extending IoT infrastructure to remote and underserved regions.
This paper explores the integration of SIoT with advanced reinforcement learning (RL) techniques through sophisticated software-defined networking (SDN) modeling.
The study emphasizes SDNAos capability to offer flexible, dynamic, and efficient management of satellitebased IoT networks, addressing unique challenges such as high latency, limited bandwidth, and frequent mobility.
To address these challenges, we propose an RL based approach for optimizing network resource allocation, routing, and communication strategies.
The RL algorithm enables autonomous adaptation to real-time network conditions, tackling critical concerns such as spectrum management, energy efficiency, and load balancing, ensuring reliable connectivity while minimizing congestion and power consumption.
Furthermore.
SDN facilitates network programmability, enabling centralized control and streamlined management of SIoT systems.
The proposed RL-driven SDN model is validated through simulation experiments, demonstrating significant improvements in throughput, network efficiency, and quality of service (QoS) metrics compared to traditional network models.
This work advances the development of satellite IoT networks by providing a robust, scalable framework that integrates RL and SDN technologies, offering intelligent and efficient connectivity solutions to meet the growing demands of next-generation SIoT systems.
Received Aug 10, 2024 Revised Jan 17, 2025 Accepted Mar 3, 2025 Keywords:
Complex network modeling Complex software defined Fractal analysis satellite Network virtualization Satellite internet of things This is an open access article under the CC BY-SA license.
Corresponding Author:
Aziz Nanthaamornphong College of Computing.
Prince of Songkla University Kathu 83120.
Thailand Email: aziz.
n@phuket.
INTRODUCTION
The rapid evolution of communication technologies has ushered in a new era of interconnectivity, with the internet of things (IoT) playing a fundamental role in transforming industries worldwide.
The IoT envisions a global ecosystem of interconnected devices capable of seamless data exchange and intelligent decision-making.
As a result, industries across various sectors, such as healthcare, manufacturing, and transportation, are experiencing unprecedented advancements in efficiency, automation, and data-driven insights.
In parallel, satellite communication systems have emerged as essential enablers, extending IoT infrastructure to remote and underserved sections where outmoded global grids are impractical or cautiously unviable .
Despite its potential, managing satellite IoT networks boons momentous tasks due to high latency, inadequate bandwidth, and dynamic channel conditions.
In this context, software-defined networking (SDN) Journal homepage: http://ijece.
Int J Elec & Comp Eng
ISSN: 2088-8708
has appeared as a transformative pattern that introduces programmability, flexibility, and centralized control into network management .
SDN allows dynamic configuration and optimization of satellite IoT networks, accommodating the heterogeneous demands of IoT devices.
The dynamic and distributed nature of satellite IoT introduces additional complexities, such as resource allocation and real-time decision-making under unpredictable conditions.
These challenges arise from the need to manage a large number of devices in highly variable environments, making efficient resource management To address these issues, this study explores the integration of reinforcement learning (RL) within the SDN framework, enabling intelligent agents to adaptively optimize network resources.
This includes optimizing critical parameters such as bandwidth, energy consumption, and computational capacity .
The proposed research investigates how the incorporation of RL with SDN can significantly enhance the gain, reliability, and scalability of satellite IoT networks.
This integration allows for more efficient resource management and optimization in dynamic network environments.
By leveraging advanced RL techniques, such as deep reinforcement learning and multi-agent systems, the study enables real-time optimization and autonomous decision-making, ultimately improving network performance and adaptability in complex scenarios.
Furthermore, the study highlights the role of satellite IoT in critical applications, such as disaster management, environmental monitoring, and precision agriculture, where reliable and robust communication systems are paramount .
These applications demand innovative solutions to ensure seamless connectivity under extreme conditions.
The integration of RL within SDN architectures not only enhances operational efficiency but also provides a pathway for developing scalable and sustainable models capable of accommodating exponential IoT growth .
Despite the opportunities, the study acknowledges key challenges, including the computational complexity of RL algorithms, the need for robust security mechanisms, and the integration of legacy systems with modern SDN architectures.
These challenges arise due to the high processing demands of RL models, the vulnerability of IoT networks to cyber threats, and the difficulty of ensuring seamless interoperability between traditional and software-defined infrastructures.
Addressing these challenges requires a multidisciplinary approach, combining expertise in satellite communication, machine learning, and network engineering to develop efficient, secure, and scalable solutions.
This research investigates the intersection of satellite IoT.
RL, and SDN, proposing innovative solutions to address the challenges of modern connectivity.
By harnessing the strengths of these technologies, this study paves the way for next-generation satellite IoT networks that are intelligent, adaptive, and resilient.
The findings hold the potential to drive advancements across domains, including smart cities, autonomous systems, global healthcare, and environmental sustainability, contributing to the vision of a truly connected world.
The organization of this work is as follows.
The introduction is provided in section 1.
Section 2 will analyze the system architecture of our satellite framework.
Section 3 is a review of prior research.
The simulation developed in MATLAB will be emphasized in section 4.
Section 5 elucidates the results, whereas section 6 presents the conclusion.
The proposed work makes a significant contribution to integrating emerging technologies within the S-IoT domain.
This study introduces an innovative approach to augment the efficiency, scalability, and adaptability of S-IoT systems by leveraging the synergy between SSDN and RL.
The proposed SDN-RL framework models the dynamic complexities of S-IoT environments, addressing critical challenges such as resource allocation, latency optimization, and seamless device interoperability .
The framework enables real-time decision-making and centralized management of satellite networks, offering greater flexibility and efficiency in managing complex communication systems.
RL further enhances adaptive optimization by training intelligent agents to respond dynamically to the active nature of S-IoT traffic and environmental constraints.
This capability allows the system to autonomously adjust network parameters, ensuring optimal performance even under fluctuating conditions and resource limitations.
A core highlight of this work is the introduction of the low latency and joint-jamming mitigation medium access control (LJJ-MAC) protocol, a novel medium access control (MAC) protocol designed for energy-efficient, low-latency communication in S-IoT systems .
Unlike conventional MAC protocols.
LJJMAC incorporates SDN and RL principles to dynamically allocate spectral resources, reduce collision rates, and adaptively prioritize IoT traffic.
The protocolAos design ensures high reliability and efficient bandwidth utilization, which is crucial for dense and heterogeneous device ecosystems typical of S-IoT networks.
Additionally.
LJJ-MAC supports multi-tier hierarchical architectures, enabling seamless communication between satellite relays, terrestrial gateways, and IoT endpoints .
The contributions of this study are validated through comprehensive simulations, demonstrating the superiority of the proposed SDN-RL framework and the LJJ-MAC protocol over traditional solutions.
These simulations highlight significant improvements in key performance metrics such as throughput, latency, and energy efficiency, showcasing the effectiveness of the proposed approach.
By optimizing network performance across multiple dimensions, the study underscores the potential of this framework to transform the S-IoT landscape, making it more efficient and adaptable to future demands.
Investigation of the satellite internet of things and reinforcement learning A (Arun Kuma.
A ISSN: 2088-8708 By addressing critical challenges and offering a scalable, intelligent framework, this work lays the foundation for future research in S-IoT systems.
This framework enables more efficient resource management and dynamic decision-making, paving the way for improved network performance.
Applications of this research include smart agriculture, environmental monitoring, and disaster management, all leveraging satellite-enabled IoT connectivity to provide reliable, real-time data in remote and underserved areas.
A satellite beam antenna focuses electromagnetic signals into narrow beams for efficient communication as shown in Figure 1.
These beams enable precise coverage of specific geographic areas, optimizing signal strength and minimizing interferenceAifeatures essential for applications like S-IoT and broadband services, as shown in Figure 1.
The switch process in satellites involves routing signals between beams or channels, ensuring seamless connectivity and efficient resource utilization within dynamic communication networks, as illustrated in Figure 1.
Figure 1.
Architecture of satellite communication .
satellite beam antenna and .
switch process
SYSTEM ARCHITECTURE
The system architecture integrates satellite communication.
IoT devices.
RL, and SDN for efficient and intelligent data transmission and resource management.
The architecture consists of several key components: IoT devices, satellites, ground stations.
SDN controllers, and RL agents .
IoT devices, distributed over vast geographic areas, generate data that must be transmitted reliably and efficiently.
These devices communicate with satellites in low earth orbit (LEO) to relay data to ground stations.
The SDN framework manages the satellite and terrestrial network infrastructure.
The SDN controller centralizes network management and dynamically configures the data flow paths between IoT devices and ground stations.
achieves optimal performance by leveraging RL-based decision-making algorithms.
The RL agent, integrated into the SDN architecture, learns and adapts to dynamic network conditions, such as changing satellite positions, weather conditions, and traffic loads.
This adaptive mechanism ensures reduced latency, enhanced throughput, and efficient spectrum utilization .
A crucial element in the architecture is the LJJ-MAC protocol, a custom MAC protocol tailored for satellite IoT.
LJJ-MAC addresses the unique challenges of satellite networks, including high propagation delays, intermittent connectivity, and limited bandwidth.
optimizes channel access by prioritizing data packets based on their urgency and the networkAos real-time state.
The protocol employs adaptive time-division multiplexing and contention-based mechanisms to balance fairness and efficiency.
Additionally, it integrates with the SDN controller to dynamically adjust parameters based on RL feedback, enabling seamless interaction between satellite and terrestrial components.
The synergy between RL and SDN fosters proactive and intelligent decision-making, such as optimizing satellite handovers.
Int J Elec & Comp Eng.
Vol.
No.
June 2025: 3506-3518
Int J Elec & Comp Eng
ISSN: 2088-8708
scheduling data transmission, and balancing traffic loads across multiple paths.
This architecture is particularly suited for scenarios like disaster recovery, remote area monitoring, and large-scale IoT deployments, where dynamic adaptability and high reliability are critical.
By combining LJJ-MAC with SDN and RL, the system ensures robust, scalable, and energy-efficient operations in a complex satellite-IoT environment .
RELATED WORKS
The efficient utilization of satellite resources is also demonstrated in the succeeding tests illustrated in Figure 2.
The integration of S-IoT with advanced network technologies has gained significant attention due to its potential in enabling widespread, reliable connectivity for remote and underserved regions.
Existing research highlights the use of SDN to enhance the adaptability, scalability, and efficiency of satellite IoT SDN decouples control and data planes, enabling dynamic reconfiguration of satellite networks to address varying IoT traffic demands.
Studies such as .
have demonstrated how SDN controllers optimize bandwidth allocation and minimize latency in satellite-enabled IoT environments.
RL has emerged as a critical component in enhancing the intelligence of SDN-based satellite IoT systems.
RL techniques enable automated executive procedures in resource apportionment, routing, and traffic management, allowing systems to familiarize to altering network circumstances.
For instance.
Liu et al.
utilized deep RL models to optimize handover decisions between satellite nodes, reducing service interruptions and improving QoS.
Furthermore, hybrid approaches combining RL with other machine learning paradigms, as explored by Tirmizi et al.
enhance network reliability and throughput, especially in dynamic satellite IoT scenarios.
The LJJ-MAC protocol has been specifically proposed as a MAC solution for satellite IoT networks, addressing challenges such as collision minimization and energy efficiency.
It integrates advanced scheduling algorithms with reinforcement learning to prioritize critical IoT data packets, thereby improving overall network performance.
Figure 2.
Simulation parameters Liu et al.
showed that LJJ-MAC significantly reduces packet loss and transmission delays, making it suitable for latency-sensitive applications in satellite IoT.
Additionally, researchers have explored the interplay of SDN.
RL, and MAC protocols to achieve holistic improvements in satellite IoT systems.
In .
implemented an RL-driven SDN controller with adaptive MAC layer functionalities to dynamically allocate satellite resources, catering to varying IoT traffic patterns while maintaining energy efficiency.
In conclusion, the convergence of SDN.
RL, and specialized protocols like LJJ-MAC offers a promising pathway for developing robust satellite IoT networks.
Future work should focus on integrating these technologies with emerging advancements such as 6G communication and edge computing, further enhancing their capability to meet the demands of next-generation IoT applications.
We then used .
to represents the front-end timing, or the time associated with the start or leading part of an event or signal.
ycNfront = ycu ycu2 yc, ycNback = 2ycuycAyc ycuOeycu2 yc.
The equation .
represents the total time required for a set of operations, transmissions, or events to occur without interruption.
From the theoretical results.
C means the RTT of LoRa, the latency of our method is given in .
Investigation of the satellite internet of things and reinforcement learning A (Arun Kuma.
A ISSN: 2088-8708
ycNContiguous = OcAyc=1 ycNLST-MAC = .
a yayc.
ycu yayc ya.
cOe1 yc.
PROBLEM FORMULATION
The LJJ-MAC procedure and the arrangement architecture illustrated in Figure 3.
The integration of satellite internet of things (SIoT) and RL in SSDNs introduces complex challenges that necessitate precise problem formulation.
SIoT, as a vital enabler of global connectivity for IoT devices, demands robust, scalable, and efficient management of satellite resources to support heterogeneous applications.
RL, with its adaptive decision-making capabilities, offers a promising avenue for optimizing network configurations, resource allocation, and QoS under dynamic conditions.
However, the non-deterministic nature of satellite communication environments, characterized by latency, bandwidth constraints, and mobility, complicates this SDNs add another layer of complexity by separating control and data planes, requiring intelligent coordination to ensure network responsiveness and reliability .
A significant aspect of the problem is designing protocols like the LJJ-MAC to address medium access issues in SIoT environments.
LJJ-MAC aims to reduce latency and mitigate interference through dynamic channel allocation and anti-jamming mechanisms, leveraging RL for real-time decision-making.
The protocol integrates seamlessly with SDN architectures to adaptively configure satellite links based on network traffic patterns and interference levels .
This hybrid approach enables the network to maintain efficient communication even under hostile conditions.
The problem formulation centers on balancing the trade-offs between throughput, latency, energy efficiency, and scalability while ensuring security against adversarial attacks.
Modeling the SIoT-RL-SDN framework involves defining state spaces, action sets, and reward functions that reflect the multi-dimensional objectives of the system.
Incorporating LJJ-MAC enhances this model by offering a tailored mechanism for collision and jamming mitigation, thereby optimizing resource utilization and QoS in SIoT systems.
This multi-faceted problem highlights the need for innovative solutions that address the intricacies of integrating SIoT.
RL, and SDN technologies within a cohesive framework .
The steps involved in implantation of LJJ-MAC Joined Beam method in indicated in Algorithm 1.
The satellite gateway, along with its link budget, is illustrated in Figure 4.
To enhance our approach, we incorporated time awareness by training each set with time-constrained purposes over a finite time horizon.
The results demonstrate that, even within a limited time prospect, our method surpasses topographical neighborhood-based and arbitrary division structures, achieving lower system average waiting times and average service wait times .
Ae.
Figure 3.
Satellite diagram The satellite gateway, along with its link budget, is illustrated in Figure 4.
To enhance our approach, we incorporated time awareness by training each set with time-constrained objectives over a finite time horizon.
The results demonstrate that, even within a limited time horizon, our method surpasses geographical neighborhood-based and random allocation schemes, achieving lower system average waiting times and average service wait times .
Int J Elec & Comp Eng.
Vol.
No.
June 2025: 3506-3518
Int J Elec & Comp Eng
ISSN: 2088-8708
Figure 4.
Satellite gateway specs Algorithm 1.
LJJ-MAC joined beam algorithm System initialization:
Oe Initialize all network parameters, including the number of users, base stations, and the wireless Oe Define the beamforming matrices for each base station and user equipment (UE) in the system.
Channel estimation:
Oe Perform a channel estimation phase where base stations and UEs exchange pilot signals.
Oe Estimate the channel state information (CSI) at both the transmitter and receiver ends for each communication link in the network.
Beamforming vector calculation:
Oe Based on the estimated CSI, calculate the optimal beamforming vectors at the base stations to focus the transmission power towards the UEs.
Oe Apply techniques such as singular value decomposition (SVD) or other optimization algorithms to determine the most effective beamforming vectors.
User scheduling:
Oe Use a medium access control (MAC) protocol to schedule users for data transmission based on the current network conditions, including the quality of the channel, network congestion, and fairness considerations.
Oe Prioritize users that are in good channel conditions and those requiring higher data rates.
Transmission time slot allocation:
Oe Allocate time slots for the transmission of data between the base stations and users.
Ensure efficient use of available bandwidth while minimizing interference.
Oe Apply time division multiple access (TDMA) or other scheduling schemes to allocate transmission RL DETAILS The investigation into the SIoT and RL through complex SDN modeling emphasizes efficient resource allocation, scalability, and adaptive decision-making in space-ground communications.
RL algorithms, integrated within SDN frameworks, optimize SIoT performance by enabling autonomous learning of traffic patterns, resource contention resolution, and dynamic reconfiguration in varying network conditions.
This approach enhances throughput, minimizes latency, and supports the massive connectivity demands of IoT devices linked via satellites.
The LJJ-MAC protocol, pivotal in this context, addresses medium access control by leveraging RL for collision avoidance and channel scheduling.
By learning the traffic characteristics and network dynamics.
LJJ-MAC ensures efficient spectrum utilization and energy-aware communication for IoT Together, these innovations showcase how RL and SDN synergize to tackle SIoT challenges, delivering robust, adaptable, and high-performance networking solutions tailored to the intricate demands of satellitebased IoT ecosystems .
Investigation of the satellite internet of things and reinforcement learning A (Arun Kuma.
A ISSN: 2088-8708 Iterate this process, adjusting beamforming angles, transmission power, and scheduling decisions to ensure optimal throughput, fairness, and overall network performance.
Deep reinforcement learning (DRL) optimizes satellite IoT systems by enabling adaptive resource allocation in complex SDNs.
The LJJ-MAC protocol integrates DRL to enhance medium access efficiency, reducing latency and collisions.
By modeling dynamic network states.
DRL provides robust decision-making for scalable and energy-efficient IoT applications in satellite communication environments.
The state equation is given by .
yc = yco1 , yco2 , ya1 , ya2 The inequality =0 < {L1,p,L2,p,m1,p,m2,p,.
suggests that a parameter equals zero and is smaller than a set of variables, including L1,p,L2,p .
ikely related to lengths or level.
, m1,p,m2,p .
ossibly moments or measure.
, and This condition could define boundaries or constraints in a mathematical or physical model.
The action equation is represented as .
= 0 < .
a1,ycy , ya2,ycy , yco1,ycy , yco2,ycy , yc.
The agent determines the course of action by selecting two ground stations .
Ae.
based on the satelliteAos inclination and the mounting angle of its beams.
In this context.
I represent the inclination.
L denotes the ground stations, p signifies the selection process, and M indicates the mounting angle.
The associated reward is defined in .
yc = 1 yca .
a1 ya2 ) = Ocycuycn=1 ya1 , ya2 The measured in bits per second .
, it determines the quality and efficiency of the transmission, impacting audio or video clarity.
Higher bit rates provide better quality but require more bandwidth, while lower bitrates save bandwidth at the cost of reduced quality or resolution .
SDN plays a crucial role in satellite communication by enabling flexible, centralized control over the networkAos infrastructure.
It allows dynamic management of resources, such as bandwidth and connectivity, optimizing satellite links in real-time.
With SDN, satellite networks can efficiently handle traffic, prioritize critical data, and adapt to varying network This enhances scalability, reduces operational costs, and supports advanced applications like IoT and global connectivity .
RESULTS
Our study explores the integration of advanced technologies like SDN and RL in the management and optimization of satellite communication networks, specifically targeting IoT applications in remote areas.
These satellite-based networks, such as LEO satellite constellations, offer an efficient solution for connecting IoT gadgets in environments where terrestrial substructure is unfeasible, such as woods, oceans, and remote A significant challenge in such systems is the effective distribution of resources .
uch as communication channels and powe.
while ensuring that diverse QoS requirements are met.
Different IoT applications have varying needs in terms of spectrum, delay, and consistency, which complicates the resource management task.
The paper uses RL to autonomously determine the optimal allocation of resources in realtime by continuously learning from the systemAos performance.
This enables a more adaptive and scalable solution for IoT deployments where traditional methods fall short.
The paper further discusses the energy efficiency of satellite IoT systems.
Since many IoT devices are battery-powered and operate in remote areas, managing the trade-off between power consumption and transmission efficiency is critical.
The proposed RLbased methods aim to optimize energy use while meeting communication requirements, offering significant improvements over traditional approaches.
For instance, simulations in the paper show that the RL-based approach can increase power efficiency by up to 60.
91% compared to other conventional algorithms.
addition, the paper incorporates SDN to provide a flexible, centralized control mechanism, which is crucial for managing the complexities of satellite IoT systems.
SDN allows for dynamic reconfiguration of network resources, enhancing the networkAos adaptability and performance in various scenarios.
Furthermore, the authors suggest a transfer learning method for efficient deployment in space environments, ensuring that the model can quickly adapt and perform with minimal computational resources.
This study contributes significantly to optimizing resource allocation in S-IoT, offering a more robust, scalable, and energy-efficient solution for satellite communication in remote areas.
In RL, training 2,000 times refers to the agent iteratively interacting with the environment, adjusting its strategies based on rewards received after each action.
Over 2,000 training cycles, the RL model refines its decision-making to maximize its objective, enhancing performance by learning optimal policies.
This repeated exposure improves the agentAos adaptability and efficiency in complex tasks given in Figure 5.
Int J Elec & Comp Eng.
Vol.
No.
June 2025: 3506-3518
Int J Elec & Comp Eng
ISSN: 2088-8708
Figure 5.
RL trained 2000 times Figure 6 illustrates the results of training a RL model 500 times using different formulas.
Each formula likely represents a distinct approach to optimizing the modelAos learning process, focusing on resource allocation in satellite IoT networks.
By training the RL model multiple times, the figure demonstrates how different configurations of the algorithm affect its performance, efficiency, or accuracy.
This setup helps compare the effectiveness of each approach, emphasizing the importance of adapting the learning process to varying network conditions or QoS requirements.
The significance of training a RL model 500 times using a different formula lies in its ability to enhance the robustness and generalization of the model.
Repeated training with varied formulations allows the RL agent to explore different strategies, adapting to diverse conditions and uncertainties in the environment.
This process improves its decision-making abilities, ensuring it can handle a wide range of situations effectively, especially in complex systems like satellite IoT, where resource allocation and network dynamics are highly variable.
Figure 6.
RL trained 500 times using a different formula Investigation of the satellite internet of things and reinforcement learning A (Arun Kuma.
A ISSN: 2088-8708 Figure 7 in the context of RL represents the cumulative reward achieved after testing the RL model 500 times.
The cumulative reward serves as a measure of the modelAos overall performance and learning effectiveness across these trials.
By observing the reward accumulation over time, this figure helps in understanding how well the RL agent is learning the task.
An increasing trend in cumulative reward indicates successful learning, while plateaus or declines might suggest issues such as insufficient exploration, suboptimal policy, or model inefficiency.
This type of analysis is essential for fine-tuning and optimizing RL models.
The cumulative reward reflects the RL modelAos learning progress and efficiency in optimizing resource allocation in the SIoT system.
The results suggest that with increased training episodes, the RL model converges to an optimal policy, achieving higher cumulative rewards.
This demonstrates the modelAos ability to adapt to varying network conditions and meet diverse QoS requirements efficiently.
Figure 7.
RL tested 500 times using Different formula Figure 8 in the paper demonstrates the performance of a RL model that was trained 500 times using the normal formula but with an increased learning rate of 1.
This increased learning rate means that the model makes larger updates to its parameters after each training iteration, potentially speeding up the learning process.
However, while a higher learning rate can accelerate convergence, it may also lead to overshooting the optimal solution or causing instability in the training process.
In the context of this experiment, increasing the learning rate to 1 could result in faster learning, but it also introduces the risk of the model not settling on the optimal policy if the updates are too large.
The figure likely shows how the modelAos performance evolves over the training iterations, comparing it to a model with a standard learning rate.
In practical terms, this adjustment can be beneficial for certain types of problems where quick convergence is more important than gradual fine-tuning.
However, careful tuning is required to prevent the model from missing optimal solutions or encountering Overall, this experiment likely provides insights into the trade-off between learning rate and stability in RL applications, especially in complex domains like satellite IoT networks.
Figure 9 in the study discusses the results of RL with an increased learning rate formula tested under various conditions.
The increased learning rate approach aims to optimize the learning process by adjusting how quickly the model updates its parameters during training.
The results show that a higher learning rate can initially accelerate convergence, enabling the model to reach a solution faster than with standard learning rates.
However, the increased rate also introduces some instability, with the model experiencing sharper fluctuations in performance early in the training process.
By adjusting the learning rate dynamically, the RL model can better respond to the complexities of the environment it is trained in.
The results suggest that the optimal Int J Elec & Comp Eng.
Vol.
No.
June 2025: 3506-3518
Int J Elec & Comp Eng
ISSN: 2088-8708
learning rate depends on the specific task and the trade-off between exploration and exploitation in the RL If set too high, the model may experience overshooting, where it moves past the optimal point too quickly, leading to poor performance.
On the other hand, if the learning rate is too low, the model may take longer to converge, limiting efficiency.
In the context of RL, adaptive learning rates can help achieve faster convergence, but careful tuning is necessary to avoid instability.
The figure likely demonstrates this trade-off through varying performance metrics over the course of training with an increased learning rate.
Thus, further investigation is required to fine-tune the learning rate for optimal results in RL settings.
Figure 8.
RL trained 500 times using normal formula however an increased learning rate of 1 Figure 9.
RL tested results using increased learning rate formula Investigation of the satellite internet of things and reinforcement learning A (Arun Kuma.
A ISSN: 2088-8708
CONCLUSION
In conclusion, this study delves into the integration of SIoT with RL through the framework of complex SDNs.
The investigation highlights the promising capabilities of SIoT in enhancing connectivity for a range of remote and underserved regions, combined with the adaptability and optimization potential offered by RL algorithms.
By leveraging SDN architecture, the research underscores the importance of centralized control, scalability, and flexibility in managing network resources, particularly in the challenging and dynamic environments encountered in satellite-based communication systems.
The application of RL algorithms within this context provides a robust mechanism for automating network management and improving key metrics such as throughput, latency, and energy efficiency.
The study demonstrates how RL can facilitate real-time decisionmaking for dynamic resource allocation and fault tolerance, contributing to the efficient functioning of satellitebased IoT systems.
Future work should focus on addressing these limitations by optimizing RL algorithms for low latency, high-efficiency performance in satellite IoT networks.
More specifically, research could explore hybrid approaches that combine RL with other machine learning techniques or heuristic methods to reduce computational complexity and improve real-time decision-making capabilities.
Another important avenue for future research is the development of more robust models that can accurately predict and mitigate the effects of satellite channel impairments.
ACKNOWLEDGMENTS
We would like to thanks to College of Computing at Prince of Songkla University.
Phuket for their valuable assistance.
FUNDING INFORMATION
Authors state no funding involved.
AUTHOR CONTRIBUTIONS STATEMENT
This journal uses the Contributor Roles Taxonomy (CRediT) to recognize individual author contributions, reduce authorship disputes, and facilitate collaboration.
Name of Author Arun Kumar Sumit Chakravarty Aziz Nanthaamornphong C : Conceptualization M : Methodology So : Software Va : Validation Fo : Formal analysis ue ue ue ue ue ue ue ue I : Investigation R : Resources D : Data Curation O : Writing - Original Draft E : Writing - Review & Editing ue ue ue ue ue ue ue ue ue ue ue ue Vi : Visualization Su : Supervision P : Project administration Fu : Funding acquisition CONFLICT OF INTEREST STATEMENT Authors state no conflict of interest.
DATA AVAILABILITY
Data availability is not applicable to this paper as no new data were created or analyzed in this study.
REFERENCES