– In Chapter 13, “Multi-Layer UAV Ad-Hoc Network Architecture, Protocol and Simulation,” Kamlesh Lakhwani, Tejpreet Singh Orchu and Aruna discuss the use of flying ad hoc networks (FANETs) to provide communication among the UAVs. In FANET, the selection of a suitable mobility model is a critical task for researchers.
– In Chapter 14, “Artificial Intelligence in Logistics and Supply Chain,” Jeyaraju Jayaprakash explains the source of activities in any white product manufacturing and service industry. This logistics and supply chain network will become more complex over the next few decades as a result of pandemic situations, natural disasters, increasing population, and other side entities developing smart strategies.
– In Chapter 15, “Hereditary Factor-Based Multi-Feature Algorithm for Early Diabetes Detection Using Machine Learning,” S. Deepajothi, R. Juliana, Aruna S K and R. Thiagarajan indicate that the predominance of diabetes mellitus among the global population ultimately leads to blindness and death in some cases. The proposed model attempts to design and deliver an intelligent solution for predicting diabetes in the early stages and address the problem of late detection and diagnosis.
– In Chapter 16, “Adaptive and Intelligent Opportunistic Routing Using Enhanced Feedback Mechanism,” V. Sharmila, Mandal K, Shankar Shalani and P. Ezhumalai discuss opportunistic routing of an intercepted packet to provide an effective wireless mesh network. Traditional opportunistic routing algorithms are being used to provide high-speed use batching of packets, which is a complex task. Therefore, an enhanced opportunistic feedback-based algorithm is proposed in this chapter in which individual packet forwarding uses a new route calculation in the proposed work that takes into consideration the cost of transmitting feedback and the capacity of the nodes to choose appropriate rates for monitoring operating conditions.
– In Chapter 17, “Enabling Artificial Intelligence and Cyber Security in Smart Manufacturing,” R. Satheesh Kumar, G. Keerthana, L. Murali, S. Chidambaranathan, C.D. Premkumar and R. Mahaveerakannan propose an efficient green manufacturing approach in SM systems with the aid of AI and cyber security frameworks. The proposed work employs a dual-stage artificial neural network (ANN) to find the design configuration of SM systems in industries. Then, for maintaining data confidentiality while communicating, the data are encrypted using the 3DES approach.
– In Chapter 18, “Deep Learning in 5G Networks,” G. Kavitha, P. Rupa Ezhil Arasi and G. Kalaimani discuss a 3D-CNN model combined with RNN model for analyzing and classifying the network traffic into three various classes, such as maximum, average and minimum traffic, which proves that the combined 3D-CNN and RNN model provides better classification of network traffic.
– In Chapter 19, “EIDR Umpiring Security Models for Wireless Sensor Networks,” A. Kathirvel, S. Navaneethan and M. Subramaniam provide an overview of WSNs with their classification, as well as comparisons between different routing algorithms and the proposed EIDR (enhanced intrusion detection and response). Soundness of proposed EIDR is tested using QualNet 5.0.
– In Chapter 20, “Artificial Intelligence in Wireless Communication,” Prashant Hemrajani, Vijaypal Singh Dhaka, Manoj Kumar Bohra and Amisha Kirti Gupta describe the applications of AI techniques in wireless communication technologies and networking, which can bring these changes through new research. Also, AI/ML techniques can improve the current state of network management, operations and automation. They can further support software-defined networking (SDN) and network function virtualization (NFV), which are considered important wireless communication technology components for the deployment of 5G and higher generation communication systems.
All of the contributors to this book deserve our heartfelt gratitude. With the fervent cooperation of the editorial director and production editor at Scrivener Publishing, we aspire to cross many more milestones to glory in the future academic year.
Prof. Dr. R. Kanthavel
Department of Computer Engineering, King Khalid UniversityAbha, Saudi Arabia
Dr. K. Ananthajothi
Department of Computer Science and Engineering Misrimal Navajee Munoth Jain Engineering CollegeChennai, India
Dr. S. Balamurugan
Founder and Chairman, Albert Einstein Engineering and Research Labs (AEER Labs)Vice-Chairman, Renewable Energy Society of India, India
Dr. R. Karthik Ganesh
Department of Computer Science and EngineeringSCAD College of Engineering and Technology Cheranmahadevi, India January 2022
1
Comprehensive and Self-Contained Introduction to Deep Reinforcement Learning
P. Anbalagan*, S. Saravanan and R. Saminathan
Department of Computer Science and Engineering, Annamalai University, Annamalai Nagar, India
Abstract
Deep reinforcement learning is a type of machine learning and artificial intelligence in which smart robots, similar to the way people make good decisions, can learn from their actions. Implicit in this form of machine learning is that, depending on their behavior, an agent is rewarded or punished. Including unsupervised machine learning and supervised learning, reinforcement learning is yet another common type of artificial intelligence development. Deep reinforcement learning can lead to incredibly impressive results beyond normal reinforcement learning, due to the fact that it incorporates the core qualities of both deep learning and reinforcement learning. Since this is becoming a very broad and rapidly growing field, the entire application landscape will not be explored, but mainly based on comprehensive and self contained introduction to deep reinforcement learning. The goal of this chapter is twofold: (i) to provide a brief guide to the deep reinforcement learning process; (ii) to present detailed applications and research directions.
Keywords: Artificial intelligence, deep learning, machine learning, reinforcement learning
1.1 Introduction
Due to its effectiveness in solving complex sequential decision-making issues, Reinforcement Learning (RL) has become increasingly common over the past few years. Many of these accomplishments are due to the integration of deep learning techniques with RL. But, thanks to its ability to learn various levels of abstractions from data, deep RL has been effective in complex tasks with lower prior knowledge. For example, from visual perceptual inputs made up of thousands of pixels, a deep RL agent can successfully learn [14]. Deep RL also has potential for real-world areas such as medical, self-driving cars, finance and smart grids, to name a few. Nonetheless in implementing deep RL algorithms, many problems arise. The area of machine learning that deals with sequential decision-making