© 2024 IJRAR October 2024, Volume 11, Issue 4 www.ijrar.

org (E-ISSN 2348-1269, P- ISSN 2349-5138)

Path Planning for Autonomous Quadcopters: A

Comprehensive Study and Performance
Comparison
1Nidharshana Priya S, 2Saarah G, 3Anbarasi M P
1
Student, 2 Student, 3Assistant Professor
1
Department of Robotics and Automation,
1
PSG College of Technology, Coimbatore, India
Abstract: This paper presents quadcopters' path planning, which is designed by implementing and comparing well-known
algorithms, namely RRT* and A*, in an aim to observe, simulate, and evaluate their performances using MATLAB software as a
robust tool for modeling complex systems and analyzing their behaviors. This study probes deep into some of the critical
performance metrics of each algorithm concerning its efficiency, computational complexity, success rate, and real-time
compatibility when implemented for quadcopter navigation missions. On top of that, the strengths and weaknesses of these
algorithms relative to state-of-the-art path-planning techniques will be brought to light in various operation environments. It is from
this comparative analysis that a better understanding of improvements that could be made would be described and directions of
future study aim towards more optimized and reliable path-planning solutions for autonomous quadcopter navigation.

Index Terms - A*, DQN, Machine learning, MATLAB, PPO, Quadcopter, RRT*, SAC.
I.INTRODUCTION
A quadcopter is a kind of UAV (Unmanned Aerial Vehicle) with four rotors. This type of drone is best suited for various essential
applications due to its good balance of control and manageability. The main components that build up a quadcopter are motors
(usually BLDC i.e., brushless DC), quadcopter frame, flight controller, ESC (Electronic Speed Control), propellers, battery,
transmitter & receiver GPS module and sensors. Quadcopters are small sized helicopters that could be controlled remotely by manual
operations or even by automation with minimal human intervention. This automation increases its efficiency, performance and
operation time of the quadcopter. Quadcopters are multitasking and can perform various operations and their applications include
surveillance, photography, agriculture, search and rescue operations, firefighting operations, deliver packages and drone racing etc.
The critical thing in efficient quadcopter movement is path planning-calculating the best route for the drone to fly along so it
avoids any obstructions on its route and lands safely. Much more importance, however, is attached to dynamic or cluttered
environments where the speed of adjustment is critical in order not to collide. Solutions to the problems of path planning have gone
parallel with technological advancement. Algorithms include Rapidly-exploring Random Tree Star (RRT)* and A*, which have
proven effective for 2D and 3D spaces. They are aiming at the calculation of the shortest or near-shortest path between two points
while avoiding obstacles. Their disadvantages is that they take a really long time to compute and fail to work properly in a more
dynamic setup, mainly because of applications on quadcopters in real-time scenarios.

Fig.1 Quadcopter

More advanced technologies related to machine learning and artificial intelligence use sophisticated algorithms, such as the Soft
Actor-Critic (SAC), Proximal Policy Optimization (PPO), and Deep Q-Networks (DQN). This portfolio uses reinforcement learning
methods; hence, the quadcopter can apply optimal navigation strategies through trial and error. Moreover, these algorithms have
IJRAR24D1660 International Journal of Research and Analytical Reviews (IJRAR) 381
 © 2024 IJRAR October 2024, Volume 11, Issue 4 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
high performance in areas important in changing environments where conditions are constantly shifting. For example, SAC balances
exploration and exploitation very nicely in unclear scenarios, while PPO mostly sticks to stability with simplicity of implementation.
DQN is much more straightforward and is therefore well placed to the discrete space but can potentially be applied in more structured
domains as well. With the assistance of MATLAB, among other software, these modern path planning algorithms can be
implemented and then simulated with an analysis of performance under various conditions, holding other more valuable insights
into strengths and limitations. Altogether, this field's current state exhibits cutting-edge thrusts in the development of latest
traditional methods along with AI-empowered ones, improvement through real-time decisions, computational efficiency, and
adaptability. With ongoing employment of quadcopters in commercial and industrial sectors, the demand for further development
of more sophisticated, reliable, and efficient path planning algorithms is increasingly critical. These developments further improve
the performance of the quadcopters while increasing their range of application potential toward the frontiers of emerging possibilities
with UAV technology. The future then belongs to further refinement of these algorithms to unlock new applications for operation
in increasingly complex environments, such as urban air mobility and autonomous delivery systems. As the technology continues
to advance, the roles that quadcopters with intelligent path planning capabilities will play in shaping the future of autonomous aerial
systems expand evermore.

II . RRT* ALGORITHM (2D)

RRT* algorithm is one of conventional, predominant type of algorithm used for path planning of robots as well as drones. RRt
stands for Rapidly Exploring Random Tree method and RRT * is an optimized and improved version of RRT method. This algorithm
works by randomly sampling a given space and expands the branches or nodes of its tree to find a collision free path for the drone.
But the path may not be a shortest or optimized one. This drawback could be eliminated with the help of RRT* algorithm. This
algorithm works in the same way of that of Rapidly Exploring Random Tree and the only difference is that it provides an optimized
path [1]. Initialize the 2D environment which is the size of the space in which the quadcopter can move. Maximum number of
iterations to be performed by the algorithm. Step size is set which is the distance up to which the tree expands for each iteration.
Starting position, target or goal position, the radius around newly added nodes to rewire nearby connections in RRT* is also
predefined. Obstacles are defined as rectangles are considered as obstacles in this case. The tree is initialized with the starting
position as first node. Plot the environment and loop the algorithm for the maximum number of iterations. The algorithm checks if
the new position collides with any obstacles using a function. If there is a collision, it skips this iteration. The algorithm looks for
any nearby nodes after adding a new node within the search radius. If any shorter path to any nearby node is found through the
new node, the tree is rewired (the nearby node’s parent node is updated) and the cost is reduced [3]. If the goal is reached, the loop
breaks. The block extracts the optimal path by tracing back from goal to start node through the tree and the path is then plotted in
red. The output of the implemented MATLAB code is shown in fig.2

Fig.2 Path planning of quadcopter in 2D space (RRT * algorithm.

III. RRT* ALGORITHM (3D)

RRT* algorithm for three-dimensional space is similar to two dimensional one. The only key difference is that the third
dimension (z axis). The third axis represents the depth of the obstacle. Steps followed are like that of 2D environment [3]. To
implement the algorithm in MATLAB, it is required need to write a MATLAB code for the following steps.
The environment is initialized in 3D space. Maximum number of iterations to be performed by the algorithm and the step size,
the distance up to which the tree expands for each iteration is predefined. Starting position, target or goal position which are 3D
vectors, the radius around newly added nodes to rewire nearby connections in RRT * is also predefined. Obstacles are defined as
cuboids are considered as obstacles in this case. The tree is initialized with the starting position as first node. Plot the environment
and loop the algorithm for the maximum number of iterations. The algorithm steers from the nearest node towards the random
sample, limited by a fixed step size avoiding large jumps. A function is used to check for collisions within the 3D space, considering
the z-coordinate. The algorithm looks for any nearby nodes after adding a new node within the search radius. If any shorter path to
any nearby node is found through the new node, the tree is rewired (the nearby node’s parent node is updated) and the cost is
reduced. This step is an important one which makes RRT * different from RRT. The tree become more efficient over time, as it keeps
finding and updating shorter paths, making the solution better as it progresses. If the goal is reached, the loop breaks. The block
extracts the optimal path by tracing back from goal to start node through the tree and the path is then plotted in red. The output of
the implemented MATLAB code is shown in fig.3

IJRAR24D1660 International Journal of Research and Analytical Reviews (IJRAR) 382

© 2024 IJRAR October 2024, Volume 11, Issue 4 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
Fig.3 Path planning of quadcopter in 3D space (RRT *algorithm).

IV. A* ALGORITHM
This is a heuristic and graph based or grid-based type of path finding algorithm predominantly used in robotics and works well
in discrete environment. In graphs, each point is called nodes and the distance between two nodes is called edge. The algorithm
finds a path from starting point to goal point. The path is found by expanding the nodes calculating the cost of each subsequent
nodes. The formula for finding total cost of the path through a given node n is given by

f(n) = g(n) + h(n)

where f(n) gives the total estimated cost of the path through node n, g(n) gives the actual cost of reaching a node n from the start
node. h(n) gives the heuristic estimate of the cost to reach the goal from node n which is a straight line in most of the cases. The f(n)
is calculated for each node until the goal is reached [2]. To implement the algorithm in MATLAB, it is required to write a MATLAB
code for the following steps. The environment is initialized in 3D space by defining the x, y, and z coordinates. Maximum number
of iterations to be performed by the algorithm, search radius and the step size is defined. Starting position, target or goal position,
the radius around newly added nodes to rewire nearby connections in RRT * is also predefined. Obstacles are defined as rectangles
are considered as obstacles in this case. The tree is initialized with the starting position as first node. There are two sets involved in
the algorithm, one for the nodes that are explored already set (1) and the other for the one that are yet to be explored set (2). While
the nodes to be explored is not zero, the node with lowest f(n) value is selected. If goal is not reached, the node is removed from set
(2) and added to set (1). Each node is checked if lies within the environment, doesn’t collide with obstacles and it is not present in
set (2). The current node is plotted for every step and when the goal is reached, optimal path is plotted by tracing back from goal
point to start point. The output of the implemented MATLAB code is shown in fig.4

Fig.4 Path planning of quadcopters in 3D space (A* algorithm)

V. MACHINE LEARNING-BASED ALGORITHMS

Machine learning (ML) approaches, particularly reinforcement learning (RL) models, have been gaining traction due to
their ability to adapt to unknown environments by learning optimal policies through trial and error. Here, three RL algorithms
are considered: SAC, PPO, and DQN. A simple 2D grid environment is made where an agent needs to find its way from a starting
point to a goal while avoiding obstacles along the way. The grid is 100x100 in size, with obstacles placed randomly throughout.
Both the starting point and the goal are also placed at random locations on the grid. The agent receives feedback in the form of
rewards: positive rewards for reaching the goal and negative ones for bumping into obstacles or taking unnecessary steps. This
encourages the agent to learn to take the shortest most efficient path. Three algorithms were tested —PPO, SAC, and DQN—on
this task, looking at how well it performs in terms of finding efficient paths, successfully reaching the goal, and how quickly it
learns to do so. The goal was to see how these algorithms handle navigating a static environment with obstacles and compare
their ability to adapt and find the best paths [4].

IJRAR24D1660 International Journal of Research and Analytical Reviews (IJRAR) 383

© 2024 IJRAR October 2024, Volume 11, Issue 4 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
A. Soft Actor-Critic (SAC)

SAC is an off-policy algorithm designed to maximize both expected reward and entropy, promoting exploration during
training. It is particularly well-suited for continuous action spaces, which makes it ideal for quadcopter navigation. The SAC
algorithm uses two main objective functions: the soft Q-value function Qπ (s, a) and the entropy term αH(π(⋅|s)), which promotes
exploration by encouraging randomness in action selection:

′
𝐽𝑄 = 𝐸(𝑠, 𝑎) ∼ 𝐷 [21 (𝑄(𝑠, 𝑎) − (𝑟 + 𝛾(1 − 𝑑)𝑉 (𝑠 ) )) 2]

SAC balances exploration and exploitation through this entropy term, making it particularly adept at navigating through
environments with complex obstacle arrangements. The Soft Actor-Critic (SAC) algorithm offers a promising approach to path
planning for quadcopters by effectively balancing exploration and exploitation through its entropy term. This mechanism encourages
the quadcopter to explore diverse trajectories while optimizing its navigation strategy, enabling it to navigate complex environments
filled with obstacles.

This capability is crucial for quadcopters, which often operate in dynamic and unpredictable settings. The high exploration rate
provided by SAC contributes to robust performance, allowing the quadcopter to adapt its path in response to changing environmental
conditions. However, implementing SAC in quadcopter path planning is not without challenges. The algorithm is computationally
intensive, necessitating considerable training time to fine-tune its parameters and achieve optimal performance. Moreover, SAC
may be prone to overfitting, particularly in scenarios with sparse rewards, which can hinder the quadcopter's ability to learn effective
navigation strategies. Research has indicated that while SAC can enhance the flexibility and efficiency of quadcopter path planning,
careful consideration of its computational demands and training requirements is essential to leverage its full potential in real-world
applications.

B. Proximal Policy Optimization (PPO)

PPO is an on-policy algorithm that optimizes policies by taking small, controlled updates. It minimizes a clipped surrogate
objective function to prevent drastic policy updates, ensuring stability in the training process

LCIP(θ) = Et[min(rt(θ) At , clip(rt(θ), 1 - ε,1+ ε) At )]

PPO’s controlled updates make it ideal for tasks that require fine adjustments, such as navigating tight spaces with obstacles. Its
stability during training makes it a popular choice for path planning problems where smooth trajectories are essential. Proximal
Policy Optimization (PPO) is a widely adopted algorithm in the realm of path planning for quadcopters due to its simplicity and
stability. This ease of implementation allows for quick integration into various systems, and it typically results in smooth trajectories,
minimizing erratic movements during flight. These characteristics are particularly valuable for quadcopters, which require reliable
navigation in complex environments.

However, one of the drawbacks of PPO is its lower sample efficiency compared to off-policy methods like Soft Actor-Critic (SAC).
This means that PPO often requires a larger amount of data to learn effective policies, which can slow down the training process.
Additionally, because PPO operates on an on-policy basis, it may struggle in highly dynamic environments where rapid changes
occur frequently. In such scenarios, the algorithm's reliance on current data can limit its adaptability, making it less effective than
off-policy alternatives in some applications. Overall, while PPO offers benefits for quadcopter path planning, particularly in terms
of stability and implementation, it is essential to consider its limitations in terms of sample efficiency and responsiveness to dynamic
conditions.

D. Deep Q-Learning (DQN)

DQN combines Q-learning with deep neural networks, enabling it to handle larger state spaces. It uses a neural network to
approximate the Q-function Q (s, a), which estimates the expected reward for a given state-action pair:

𝑄(𝑠, 𝑎) = 𝐸[𝑟 + 𝛾𝑎′ 𝑚𝑎𝑥𝑄(𝑠 ′ , 𝑎′ )]

While DQN is best suited for discrete action spaces, it has been adapted for continuous environments with certain modifications,
such as using a discretized action set for quadcopter control [6].

Deep Q-Networks (DQN) are known for their ease of implementation and widespread use in various reinforcement learning
applications, including quadcopter path planning. Their architecture effectively leverages deep learning to approximate the Q-value
function, making them particularly well-suited for environments with discrete action spaces. This capability allows DQN to navigate
straightforward scenarios efficiently, achieving commendable performance in tasks with clearly defined actions. However, DQNs
face challenges in environments requiring continuous action spaces, which significantly limits their effectiveness in complex
quadcopter navigation tasks. In these scenarios, the inability to seamlessly handle continuous inputs can hinder the quadcopter's
ability to make precise adjustments while navigating intricate environments. Moreover, achieving optimal performance with DQN
often requires extensive hyperparameter tuning, which can be time-consuming, especially in dynamic environments where
conditions change rapidly. Research has indicated that while DQNs can be effective in simpler tasks, their limitations in continuous
action scenarios and the need for meticulous tuning pose challenges for deploying them in more complex quadcopter navigation
applications.

IJRAR24D1660 International Journal of Research and Analytical Reviews (IJRAR) 384

© 2024 IJRAR October 2024, Volume 11, Issue 4 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
Comparative performance analysis:

To evaluate the performance of the algorithms discussed—Soft Actor-Critic (SAC), Proximal Policy Optimization (PPO), and Deep
Q-Networks (DQN)—several important metrics are considered to highlight their effectiveness in quadcopter path planning. Path
Optimality assesses the efficiency of the paths generated by each algorithm, comparing their lengths to the ideal shortest route. This
metric is crucial for understanding how effectively a quadcopter can navigate from its starting point to its destination while
minimizing distance traveled. Computation Time evaluates the duration required to compute a path. This metric is vital for real-
time applications, where rapid decision-making is essential for safe navigation, especially in complex environments. Adaptability
examines how well each algorithm can respond to rapidly changing conditions, such as moving obstacles or varying wind patterns.
The ability to adapt is critical for ensuring that the quadcopter can maintain safe and effective navigation in unpredictable settings.
Sample Efficiency measures how effectively an algorithm utilizes data to learn an optimal policy. This is particularly significant for
algorithms that may require extensive training data, as high sample efficiency can lead to faster convergence and reduced training
costs. Analyzing these metrics provides a comprehensive understanding of the strengths and weaknesses of each algorithm,
facilitating the selection of the most appropriate one for specific navigation challenges.

VI. PERFORMANCE OVERVIEW

In the analysis conducted, distinct differences in performance were observed among the three algorithms—Proximal Policy
Optimization (PPO), Soft Actor-Critic (SAC), and Deep Q-Networks (DQN). PPO demonstrated robust overall performance,
consistently delivering stable learning outcomes and efficient path generation across most scenarios. Its ability to maintain smooth
trajectories made it particularly effective in navigating complex environments. It managed to reach the goal 85% of the time,
demonstrating reliable convergence.
SAC, on the other hand, excelled in more complex environments with dense obstacles, thanks to its continuous action space,
allowing for smoother, more precise movements. SAC took longer to converge due to its exploratory behavior but ultimately found
the most efficient paths and achieved the highest success rate. DQN, while quicker to converge, struggled in environments with
tight obstacle placements. Its discrete action space led to less efficient paths, and the agent often failed to navigate through complex
sections of the grid. As a result, DQN had a lower success rate compared to PPO and SAC. Overall, SAC offered the best path
efficiency in difficult environments, while PPO provided a balance between convergence speed and success rate, with DQN being
more limited in its ability to navigate complex grids [5].

Table 1: Comparative performance analysis

Accur Accurac Computat Adapt
Path
Algori acy y ional ability
Smoothn
thm (Static (Dynami Cost
ess
) c)
95- High
A* 65-75% Moderate Low
98% (optimal)
85- Mode
RRT* 75-85% Moderate High
90% rate
90-
SAC 85-90% High High High
94%
88- Mode
PPO 80-88% High Moderate
92% rate
75-
DQN 60-75% Low Low Low
85%

When comparing traditional path-planning algorithms like A* and RRT* with more recent machine learning-based methods such
as SAC, PPO, and DQN, several key differences emerge across different performance metrics.

In static environments, A* is highly efficient in finding the optimal path. Its heuristic-based approach ensures minimal path
length, making it a reliable choice in grid-like settings with well-defined goals. However, its computational cost grows rapidly as
the grid size increases or when obstacles become more complex. This is particularly evident in scenarios where recalculating the
path frequently becomes necessary, such as dynamic obstacle avoidance or changing goals. RRT*, on the other hand, excels in high-
dimensional spaces. Its probabilistic nature allows it to explore vast search spaces more flexibly, making it suitable for environments
that are cluttered or have complex obstacles. However, RRT* requires significant computational time to reach the optimal solution,
and it often generates paths that are not inherently smooth, requiring post-processing or additional optimization steps to refine.

In contrast, machine learning-based methods exhibit a different set of strengths and limitations. SAC (Soft Actor-Critic)
particularly stands out in dynamic and continuous environments. Its ability to optimize for both the reward and exploration by
maximizing entropy enables it to efficiently navigate through highly complex environments. By continuously updating its policy
through experience, SAC demonstrates strong adaptability, making it superior in handling unexpected changes in the environment
or goals. However, this advantage comes at the cost of significant computational resources, as SAC requires extensive data and time
to train the agent to an optimal policy. The reward sparsity often seen in real-world tasks like quadcopter navigation further
complicates this, sometimes leading to longer convergence times.

PPO (Proximal Policy Optimization), on the other hand, offers more stability during training. Its clipped objective function
ensures that the policy does not undergo abrupt changes, making it suitable for environments requiring smooth trajectories, such as
when navigating around tight corners or narrow spaces. This stability makes PPO a popular choice for real-time navigation tasks.
However, PPO’s on-policy nature means it requires a lot of data to train effectively, which limits its sample efficiency compared to
off-policy methods like SAC. While PPO works well in environments with limited dynamic elements, its adaptability is not as
robust as SAC when faced with highly unpredictable changes.
IJRAR24D1660 International Journal of Research and Analytical Reviews (IJRAR) 385
 © 2024 IJRAR October 2024, Volume 11, Issue 4 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
DQN (Deep Q-Learning) stands out for its simplicity and effectiveness in discrete action spaces. While not as adept in continuous
environments due to its inherent limitations in action representation, DQN has proven to be effective in environments where actions
can be discretized without losing significant granularity. That said, for tasks like quadcopter navigation—where fine-grained control
is essential—DQN is often not the best-suited option, as it struggles to match the performance of SAC and PPO in terms of precision
and adaptability. The need to discretize the action space also results in suboptimal paths, as DQN tends to produce jerky, less fluid
motion in continuous environments [6]. When juxtaposed against traditional methods, machine learning algorithms not only offer
better adaptability but also show immense potential in dynamic, real-time path planning. In contrast to A* and RRT*, which rely
on predefined rules and heuristics, machine learning approaches actively learn from their environment. This learning enables them
to develop more sophisticated strategies to avoid obstacles and optimize paths dynamically as the environment evolves. However,
the primary trade-offs are training time and computational overhead. While traditional algorithms can be implemented with lower
computational costs, machine learning-based algorithms require powerful hardware and longer development phases due to the
extensive training involved. Recent advancements in path-planning techniques have focused on state-of-the-art algorithms that blend
machine learning with traditional methods to enhance performance. One such promising approach is the PRM*-RL (Probabilistic
Roadmaps with Reinforcement Learning) model. This hybrid system integrates the fast initial path generation of Probabilistic
Roadmaps with reinforcement learning algorithms, like SAC or PPO, allowing for real-time adjustments and dynamic adaptability.
The PRM*-RL model has demonstrated significantly higher accuracy, ranging between 92% and 96%, compared to traditional path-
planning algorithms. It also surpasses standalone reinforcement learning models, such as DQN and PPO, particularly in dynamic
environments where adaptability is crucial. Additionally, state-of-the-art graph-based learning techniques, like Graph Neural
Networks (GNNs), have emerged as highly competitive tools for path planning. By modeling the environment as a graph, GNNs
enable agents to update their paths dynamically with minimal recomputation. These models achieve accuracy rates exceeding 95%
in both static and dynamic environments and outperform algorithms like SAC and PPO, particularly in large-scale applications such
as urban air mobility and drone delivery systems.

II. VII. FUTURE RESEARCH AND DIRECTIONS

Looking ahead, research in quadcopter path planning is likely to focus on several promising areas that build upon both traditional
and machine learning approaches. One key direction is the development of hybrid models, which combine the deterministic nature
of traditional algorithms with the adaptability of machine learning techniques. For instance, researchers could explore the integration
of A* or RRT* with reinforcement learning methods to create hybrid frameworks that switch between model-based planning and
model-free learning based on environmental context. This approach would leverage the efficiency of traditional methods for static
or semi-dynamic environments while relying on machine learning for complex, dynamic scenarios. Another area of interest is the
improvement of sample efficiency in reinforcement learning algorithms. Both SAC and PPO, despite their robust performance,
require significant amounts of training data. Future research could focus on enhancing transfer learning and meta-learning
techniques, allowing agents to generalize across environments more efficiently. Such advancements would significantly reduce the
training time and computational resources required for these algorithms, making them more practical for real-time applications.

Additionally, the concept of multi-agent systems in path planning holds considerable potential. By enabling multiple agents
(quadcopters) to learn and collaborate in a shared environment, researchers can develop more resilient and efficient navigation
strategies. Such systems can distribute tasks dynamically, thereby optimizing the path planning process across a fleet of autonomous
vehicles. Machine learning algorithms like multi-agent PPO (MAPPO) and multi-agent SAC (MASAC) are already being explored
for this purpose, and future research could further enhance their scalability and coordination abilities. In the realm of traditional
algorithms, improving the speed and efficiency of probabilistic methods like RRT* remains an ongoing challenge. Recent
advancements in Fast Marching Trees (FMT) and Probabilistic Roadmaps (PRM*) suggest that these algorithms can achieve greater
efficiency in both static and dynamic environments. Future work could integrate these methods with machine learning techniques
to develop hybrid models that balance exploration with real-time adaptability [7]. Moreover, the incorporation of uncertainty
modeling into both traditional and machine learning algorithms is another promising direction. Techniques such as Bayesian
optimization or Gaussian processes could be used to model the uncertainty in environmental dynamics, allowing quadcopters to
make more informed decisions when navigating through unpredictable environments.

VIII. CONCLUSION

In conclusion, while traditional algorithms like A* and RRT* provide reliable solutions for static environments, their
limitations in adaptability and dynamicity are driving the shift toward machine learning approaches such as SAC and PPO. The
future of quadcopter path planning lies in the integration of these paradigms, leveraging the strengths of both to create solutions that
are not only efficient and optimal but also adaptable to the complex, dynamic environments of the real world. In this paper, the
traditional path planning algorithms, such as A* and RRT* were compared with machine learning methods like PPO, SAC, and
DQN for quadcopter navigation. The analysis revealed that traditional algorithms are effective in simple, static environments but
tend to struggle in dynamic scenarios. In contrast, machine learning algorithms showed better adaptability, particularly SAC and
PPO, which excelled in finding efficient paths and navigating around obstacles. Advanced methods like PRM*-RL and GNN were
explored, which improve path planning by enhancing adaptability and efficiency. This indicates that merging traditional and
machine learning approaches could lead to even more effective solutions for quadcopter navigation. Future research could focus
on these hybrid strategies and test their performance in real-time situations and more complex environments, ultimately advancing
the capabilities of autonomous flight.

IJRAR24D1660 International Journal of Research and Analytical Reviews (IJRAR) 386

© 2024 IJRAR October 2024, Volume 11, Issue 4 www.ijrar.org (E-ISSN 2348-1269, P- ISSN 2349-5138)
IX. REFERENCES:

[1] K. Danancier, D. Ruvio, I. Sung, and P. Nielsen, "Comparison of Path Planning Algorithms for an Unmanned Aerial Vehicle
Deployment Under Threats," Procedia Manufacturing, vol. 42, pp. 311-317, 2020.

[2] P. E. Hart, N. J. Nilsson, and B. Raphael, "A Formal Basis for the Heuristic Determination of Minimum Cost Paths," in IEEE
Transactions on Systems Science and Cybernetics, vol. 4, no. 2, pp. 100–107, July 1968.

[3] S. Karaman and E. Frazzoli, "Sampling-based Algorithms for Optimal Motion Planning," in International Journal of Robotics
Research, vol. 30, no. 7, pp. 846–894, 2011.

[4] S. Thrun, W. Burgard, and D. Fox, "A Probabilistic Approach to Concurrent Mapping and Localization for Mobile Robots," in
Machine Learning, vol. 31, pp. 29–53, 1998.

[5] J. Delmerico and D. Scaramuzza, "A Benchmark Comparison of Planning Algorithms for Autonomous Quadrotor Navigation,"
arXiv preprint, Mar. 2022.

[6] H. Van Hasselt, A. Guez, and D. Silver, "Deep Reinforcement Learning with Double Q-learning," in Proceedings of the Thirtieth
AAAI Conference on Artificial Intelligence, 2016, pp. 2094–2100.

[7] C. Wang, L. Meng, S. She, et al., "Reinforcement Learning-Based Path Planning: A Reward Function Strategy," Applied
Sciences, vol. 14, no. 17, pp. 7654, 2024.

IJRAR24D1660 International Journal of Research and Analytical Reviews (IJRAR) 387