In recent years, the rapid advancement of unmanned aerial vehicle (UAV) technology has led to its widespread application across diverse fields, including surveillance, agriculture, logistics, and disaster response. For China UAV drones, path planning is a critical enabling technology that aims to generate optimal or near-optimal flight trajectories within constrained operational environments, ensuring safe, efficient, and autonomous flight. The complexity of three-dimensional (3D) environments, characterized by obstacles, no-fly zones, terrain variations, and mission-specific constraints, poses significant challenges for traditional optimization methods. Heuristic algorithms, such as particle swarm optimization (PSO) and grey wolf optimizer (GWO), have been widely adopted, yet they often suffer from premature convergence and limited adaptability when faced with high-dimensional, multimodal search spaces.
To address these limitations, this paper proposes a novel Q-learning-based hyper-heuristic snake optimizer (QLHHSO) specifically tailored for China UAV drones path planning. The proposed framework integrates a high-level strategy (HLS) based on Q-learning with a diverse set of low-level heuristics (LLHs) derived from the snake optimizer (SO) and differential evolution operators. By dynamically selecting the most appropriate search operator at each stage of the optimization process, QLHHSO effectively balances global exploration and local exploitation, thereby enhancing solution quality and robustness. Extensive experiments on the CEC2017 benchmark suite and three realistic 3D path planning scenarios demonstrate that QLHHSO significantly outperforms eight state-of-the-art algorithms in terms of optimization accuracy, convergence speed, and stability. The remainder of this paper is organized as follows: Section II formulates the path planning problem. Section III details the proposed QLHHSO algorithm. Section IV presents experimental results on benchmark functions. Section V provides simulation results for China UAV drones path planning scenarios. Section VI concludes the paper.
Problem Formulation for China UAV Drones Path Planning
Environmental Modeling
We model the path planning problem as a constrained mathematical optimization problem. Let the starting point be S = (Sx, Sy, Sz) and the destination be D = (Dx, Dy, Dz). The line segment SD is divided uniformly into M+1 sub-spaces in the projection plane, generating M discrete planes Θi (i = 1,2,…,M) orthogonal to the projection. The control points Pi lie on these planes. The path is obtained by interpolating these control points using cubic B-spline curves, producing a high-density set of waypoints Wj (j = 1,…,N). The optimization reduces to a 2M-dimensional problem. To accelerate computation, we transform coordinates by aligning the x-axis with SD, using the rotation θ as follows:
$$
\begin{bmatrix} P_{i,x}^{\text{trans}} \\ P_{i,y}^{\text{trans}} \\ P_{i,z}^{\text{trans}} \end{bmatrix} =
\begin{bmatrix} \cos\theta & -\sin\theta & 0 \\ \sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{bmatrix}
\begin{bmatrix} P_{i,x} – S_x \\ P_{i,y} – S_y \\ P_{i,z} \end{bmatrix}
$$
Cost Function
The total cost for a path i is a weighted sum of four components: path length, flight height, threat avoidance, and path smoothness:
$$J_i = w_1 C_i^{\text{length}} + w_2 C_i^{\text{height}} + w_3 C_i^{\text{threat}} + w_4 C_i^{\text{smooth}}$$
where the weights are set as w1=2.0, w2=0.8, w3=1.0, w4=0.8.
Path Length Cost is defined as the sum of Euclidean distances between consecutive waypoints:
$$C_i^{\text{length}} = \sum_{j=1}^{N-1} \sqrt{(x_{i,j+1}-x_{i,j})^2 + (y_{i,j+1}-y_{i,j})^2 + (z_{i,j+1}-z_{i,j})^2}$$
Flight Height Cost penalizes deviations from the ideal altitude range [hmin, hmax]:
$$C_i^{\text{height}} = \sum_{j=1}^{N} \begin{cases} h_{\max}+h_{\min} – z_{i,j}, & \text{if } z_{i,j} < h_{\min} \text{ or } z_{i,j} > h_{\max} \\ \infty, & \text{otherwise} \end{cases}$$
Threat Cost is modeled using ellipsoidal obstacles centered at (xc, yc, zc) with horizontal semi-axis a and vertical semi-axis c. The horizontal distance from waypoint Wi,j to the vertical axis of the ellipsoid is:
$$d_{i,j}(z_{i,j}) = \sqrt{(x_{i,j}-x_c)^2 + (y_{i,j}-y_c)^2}$$
The safe radius considering UAV size S and safety distance ds is:
$$R_s(z_{i,j}) = a(z_{i,j}) + S + d_s$$
where a(zi,j) = a √(1 – ((zi,j – zc)/c)²). The per-waypoint threat cost is:
$$C_{i,j}^{\text{threat}} = \begin{cases} 0, & d_{i,j} > R_s \\ R_s – d_{i,j}, & a+S < d_{i,j} \le R_s \\ \infty, & d_{i,j} \le a+S \end{cases}$$
The total threat cost sums over all waypoints and obstacles.
Path Smoothness Cost penalizes sharp turns. Let the angle between consecutive path segments θi,j be computed from the dot product. With a threshold φ = π/2, the smoothness cost is:
$$C_i^{\text{smooth}} = \sum_{j=1}^{N-1} (\cos\phi – \cos\theta_{i,j})$$
The Q-Learning-Based Hyper-Heuristic Snake Optimizer
Lens-Imaging-Based Initialization
To enhance population diversity and initial solution quality for China UAV drones optimization, we incorporate a lens-imaging-based learning (LIL) strategy. Given an individual Poriginal, its mirror Pmirrored is generated as:
$$P_{\text{mirrored}} = \frac{UB+LB}{2} + \frac{UB+LB}{2\beta} – \frac{P_{\text{original}}}{\beta}$$
where β=1.0 is the scaling factor, and UB, LB are the search bounds. The better of the two individuals is retained, effectively expanding the search coverage.
Low-Level Heuristics Construction
We deconstruct the search patterns of the snake optimizer (SO) into three modes:
- Global exploration (GE): random large-scale movement.
- Rough exploitation (RE): directional movement toward food.
- Fine exploitation (FE): mating or competition behaviors with temperature and food availability.
Each mode is combined with five differential evolution mutation operators (VO₁–VO₅) and a crossover operator (CO). The five mutation operators are defined as follows:
- DE/rand/1: ui = xR1 + F·(xR2 – xR3)
- DE/rand/2: ui = xR1 + F·(xR2 – xR3) + F·(xR4 – xR5)
- DE/best/1: ui = xbest + F·(xR1 – xR2)
- DE/best/2: ui = xbest + F·(xR1 – xR2) + F·(xR3 – xR4)
- DE/current-to-best/1: ui = xi + F·(xbest – xi) + F·(xR1 – xR2)
The crossover operator produces the trial vector vi,j as:
$$v_{i,j} = \begin{cases} u_{i,j}, & \text{if } rand(j) \le CR \text{ or } j = rnbr(i) \\ x_{i,j}, & \text{otherwise} \end{cases}$$
where CR=0.8 and F=0.5. Combining the five mutation operators with the crossover and three SO modes yields a set of 15 low-level heuristics (LLHs):
| Mutation Operator | GE Mode | RE Mode | FE Mode |
|---|---|---|---|
| VO₁ (DE/rand/1) | VO₁-CO-GE | VO₁-CO-RE | VO₁-CO-FE |
| VO₂ (DE/rand/2) | VO₂-CO-GE | VO₂-CO-RE | VO₂-CO-FE |
| VO₃ (DE/best/1) | VO₃-CO-GE | VO₃-CO-RE | VO₃-CO-FE |
| VO₄ (DE/best/2) | VO₄-CO-GE | VO₄-CO-RE | VO₄-CO-FE |
| VO₅ (DE/current-to-best/1) | VO₅-CO-GE | VO₅-CO-RE | VO₅-CO-FE |
High-Level Strategy Using Q-Learning
Q-learning is employed as the high-level strategy to dynamically select the most appropriate LLH for each individual at each iteration. Each individual maintains a 15×15 Q-table, where rows represent the current state (the previously used LLH index) and columns represent actions (the next LLH to apply). The state-action value is updated according to the Bellman equation:
$$Q(s_t, a_t) \leftarrow Q(s_t, a_t) + \alpha \left[ r + \gamma \max_a Q(s_{t+1}, a) – Q(s_t, a_t) \right]$$
where the learning rate α decays dynamically from αs=0.5 to αe=0.1 over iterations:
$$\alpha = \alpha_s – (\alpha_s – \alpha_e) \times \frac{t-1}{T-1}$$
The discount factor γ=0.9. The immediate reward r is defined as:
$$r = \begin{cases} 1, & \text{if new fitness < old fitness} \\ -1, & \text{otherwise} \end{cases}$$
This design encourages exploitation of successful operators and exploration of new ones. The overall QLHHSO procedure is outlined as follows:
- Initialize population using LIL and evaluate fitness.
- Divide into male and female groups.
- Initialize all Q-tables to zero and set random initial states.
- For each iteration:
- For each individual, select the action (LLH) with the highest Q value for its current state using ε-greedy policy.
- Execute the selected LLH to generate a new candidate.
- Evaluate fitness, compute reward r, update Q-table.
- Update state to the chosen action.
- Update best positions and repeat until stopping criterion.
Complexity Analysis
Let N be population size, D dimension, T iterations, and Of the cost of one function evaluation. The initialization using LIL costs O(ND). Each iteration involves O(N×Of) for fitness evaluation, O(ND) for operator execution (since the chosen LLH performs vector operations), and O(N) for Q-table updates. The total complexity is O(N(T(Of+D)+D)), which is comparable to standard SO and differential evolution.
Benchmark Function Evaluation
We first evaluate QLHHSO on the CEC2017 benchmark suite (30-dimensional) comprising unimodal, multimodal, hybrid, and composition functions. Eight competing algorithms are used: PSO, GWO, WOA, IGWO, SCSO, WO, SO, and ESO. All algorithms run with population size 50 for 200 iterations, repeated 30 times. The mean and standard deviation of the best fitness are reported in Table 1.
| Func | Metric | PSO | GWO | WOA | IGWO | SCSO | WO | SO | ESO | QLHHSO |
|---|---|---|---|---|---|---|---|---|---|---|
| F1 | mean | 9.40E+07 | 2.74E+09 | 8.50E+09 | 1.51E+08 | 8.21E+09 | 9.71E+08 | 2.53E+08 | 2.18E+08 | 1.00E+02 |
| std | 2.99E+07 | 1.55E+09 | 2.37E+09 | 1.85E+08 | 4.24E+09 | 4.38E+08 | 1.62E+08 | 1.64E+08 | 1.07E-10 | |
| F3 | mean | 5.76E+04 | 7.44E+04 | 2.63E+05 | 3.70E+04 | 5.97E+04 | 7.44E+04 | 7.22E+04 | 6.39E+04 | 3.02E+02 |
| std | 1.49E+04 | 1.64E+04 | 6.53E+04 | 7.43E+03 | 1.02E+04 | 6.59E+03 | 8.94E+03 | 6.88E+03 | 5.11E+00 | |
| F4 | mean | 5.11E+02 | 6.51E+02 | 1.84E+03 | 5.28E+02 | 1.01E+03 | 6.36E+02 | 5.94E+02 | 5.90E+02 | 4.63E+02 |
| std | 3.08E+01 | 1.16E+02 | 5.90E+02 | 2.37E+01 | 5.65E+02 | 5.35E+01 | 4.11E+01 | 3.48E+01 | 2.26E+01 | |
| F5 | mean | 6.92E+02 | 6.37E+02 | 8.71E+02 | 6.52E+02 | 7.60E+02 | 7.04E+02 | 5.98E+02 | 6.57E+02 | 5.53E+02 |
| std | 3.90E+01 | 5.77E+01 | 7.33E+01 | 6.13E+01 | 4.21E+01 | 7.21E+01 | 2.10E+01 | 3.88E+01 | 5.23E+01 | |
| F6 | mean | 6.51E+02 | 6.10E+02 | 6.84E+02 | 6.04E+02 | 6.61E+02 | 6.47E+02 | 6.09E+02 | 6.11E+02 | 6.00E+02 |
| std | 5.59E+00 | 3.90E+00 | 1.06E+01 | 1.17E+00 | 1.05E+01 | 1.83E+01 | 3.91E+00 | 6.12E+00 | 3.64E-06 | |
| F7 | mean | 1.03E+03 | 9.10E+02 | 1.33E+03 | 8.91E+02 | 1.12E+03 | 1.03E+03 | 9.46E+02 | 9.35E+02 | 8.09E+02 |
| std | 8.19E+01 | 4.70E+01 | 1.02E+02 | 5.47E+01 | 7.11E+01 | 5.60E+01 | 3.95E+01 | 4.81E+01 | 6.63E+01 | |
| F8 | mean | 9.26E+02 | 9.09E+02 | 1.09E+03 | 8.97E+02 | 1.00E+03 | 1.01E+03 | 8.99E+02 | 9.35E+02 | 8.36E+02 |
| std | 2.03E+01 | 3.55E+01 | 5.30E+01 | 6.09E+01 | 2.79E+01 | 8.03E+01 | 1.75E+01 | 3.66E+01 | 1.14E+01 | |
| F9 | mean | 5.63E+03 | 2.53E+03 | 1.17E+04 | 1.07E+03 | 6.64E+03 | 1.01E+04 | 1.78E+03 | 2.36E+03 | 9.00E+02 |
| std | 1.33E+03 | 1.21E+03 | 3.74E+03 | 1.52E+02 | 8.32E+02 | 2.52E+03 | 3.67E+02 | 6.63E+02 | 6.28E-01 | |
| F10 | mean | 5.40E+03 | 5.42E+03 | 7.81E+03 | 8.06E+03 | 6.16E+03 | 8.28E+03 | 7.95E+03 | 7.84E+03 | 6.05E+03 |
| std | 6.76E+02 | 1.46E+03 | 6.32E+02 | 1.36E+03 | 7.75E+02 | 1.48E+03 | 7.61E+02 | 6.34E+02 | 2.00E+03 | |
| F11 | mean | 1.27E+03 | 2.35E+03 | 1.26E+04 | 1.29E+03 | 2.36E+03 | 1.73E+03 | 2.08E+03 | 1.40E+03 | 1.14E+03 |
| std | 3.97E+01 | 9.35E+02 | 4.79E+03 | 4.21E+01 | 7.83E+02 | 3.09E+02 | 4.81E+02 | 1.10E+02 | 3.22E+01 | |
| F12 | mean | 1.40E+07 | 7.19E+07 | 8.52E+08 | 1.05E+07 | 2.75E+08 | 5.56E+07 | 9.87E+06 | 5.60E+06 | 1.80E+04 |
| std | 7.63E+06 | 7.87E+07 | 5.89E+08 | 7.89E+06 | 4.22E+08 | 4.39E+07 | 1.00E+07 | 5.41E+06 | 1.20E+04 | |
| F13 | mean | 3.36E+04 | 2.61E+06 | 2.58E+07 | 2.82E+05 | 2.67E+07 | 3.95E+06 | 6.90E+04 | 1.08E+05 | 1.90E+03 |
| std | 1.30E+04 | 7.74E+06 | 1.52E+07 | 1.13E+05 | 8.36E+07 | 1.42E+07 | 3.37E+04 | 8.02E+04 | 7.28E+02 | |
| F14 | mean | 7.19E+04 | 4.73E+05 | 2.83E+06 | 2.80E+04 | 7.17E+05 | 9.25E+05 | 1.54E+05 | 2.85E+04 | 1.46E+03 |
| std | 5.83E+04 | 9.90E+05 | 3.03E+06 | 3.39E+04 | 1.14E+06 | 9.62E+05 | 2.88E+05 | 4.01E+04 | 1.92E+01 | |
| F15 | mean | 6.61E+03 | 2.49E+06 | 3.79E+06 | 3.76E+04 | 4.12E+06 | 5.94E+04 | 1.87E+04 | 1.92E+04 | 1.56E+03 |
| std | 6.52E+03 | 6.63E+06 | 5.17E+06 | 1.81E+04 | 1.46E+07 | 4.32E+04 | 1.45E+04 | 2.51E+04 | 2.89E+01 | |
| F16 | mean | 2.98E+03 | 2.84E+03 | 4.37E+03 | 2.61E+03 | 3.29E+03 | 3.41E+03 | 2.84E+03 | 3.00E+03 | 1.94E+03 |
| std | 3.21E+02 | 3.96E+02 | 7.13E+02 | 5.52E+02 | 3.29E+02 | 5.73E+02 | 4.83E+02 | 3.09E+02 | 2.60E+02 | |
| F17 | mean | 2.38E+03 | 2.06E+03 | 2.65E+03 | 1.95E+03 | 2.54E+03 | 2.51E+03 | 2.15E+03 | 1.98E+03 | 1.80E+03 |
| std | 3.28E+02 | 1.22E+02 | 2.76E+02 | 1.70E+02 | 2.38E+02 | 3.85E+02 | 1.40E+02 | 1.48E+02 | 6.89E+01 | |
| F18 | mean | 1.22E+06 | 3.11E+06 | 1.88E+07 | 5.01E+05 | 2.47E+06 | 4.32E+06 | 1.58E+06 | 4.40E+05 | 5.73E+03 |
| std | 1.05E+06 | 3.82E+06 | 2.42E+07 | 4.04E+05 | 2.41E+06 | 4.12E+06 | 1.19E+06 | 3.95E+05 | 3.36E+03 | |
| F19 | mean | 8.28E+03 | 1.40E+06 | 3.05E+07 | 4.84E+04 | 5.00E+06 | 1.44E+05 | 1.58E+04 | 4.74E+04 | 1.92E+03 |
| std | 5.19E+03 | 2.23E+06 | 2.57E+07 | 3.65E+04 | 1.13E+07 | 2.40E+05 | 1.31E+04 | 9.06E+04 | 1.12E+01 | |
| F20 | mean | 2.67E+03 | 2.55E+03 | 2.93E+03 | 2.38E+03 | 2.75E+03 | 2.78E+03 | 2.51E+03 | 2.45E+03 | 2.15E+03 |
| std | 1.98E+02 | 1.66E+02 | 2.70E+02 | 1.54E+02 | 2.18E+02 | 3.03E+02 | 1.45E+02 | 1.74E+02 | 1.25E+02 | |
| F21 | mean | 2.50E+03 | 2.39E+03 | 2.65E+03 | 2.44E+03 | 2.53E+03 | 2.47E+03 | 2.40E+03 | 2.42E+03 | 2.34E+03 |
| std | 4.52E+01 | 2.12E+01 | 5.98E+01 | 5.48E+01 | 4.73E+01 | 5.55E+01 | 3.11E+01 | 2.13E+01 | 3.25E+01 | |
| F22 | mean | 5.86E+03 | 4.58E+03 | 8.71E+03 | 3.10E+03 | 4.76E+03 | 2.50E+03 | 7.13E+03 | 2.43E+03 | 4.93E+03 |
| std | 1.91E+03 | 2.17E+03 | 1.56E+03 | 1.80E+03 | 1.73E+03 | 9.08E+01 | 3.35E+03 | 6.46E+01 | 3.32E+03 | |
| F23 | mean | 3.25E+03 | 2.81E+03 | 3.17E+03 | 2.78E+03 | 2.95E+03 | 2.83E+03 | 2.77E+03 | 2.77E+03 | 2.69E+03 |
| std | 1.36E+02 | 5.04E+01 | 1.32E+02 | 6.39E+01 | 6.63E+01 | 5.71E+01 | 2.82E+01 | 2.88E+01 | 1.35E+01 | |
| F24 | mean | 3.43E+03 | 2.94E+03 | 3.29E+03 | 2.93E+03 | 3.09E+03 | 3.03E+03 | 2.98E+03 | 2.96E+03 | 2.86E+03 |
| std | 1.46E+02 | 6.36E+01 | 1.19E+02 | 7.67E+01 | 8.39E+01 | 6.59E+01 | 2.91E+01 | 3.70E+01 | 1.17E+01 | |
| F25 | mean | 2.94E+03 | 3.00E+03 | 3.31E+03 | 2.93E+03 | 3.12E+03 | 3.02E+03 | 2.97E+03 | 3.00E+03 | 2.89E+03 |
| std | 1.86E+01 | 4.22E+01 | 6.79E+01 | 1.94E+01 | 6.04E+01 | 3.76E+01 | 3.64E+01 | 2.03E+01 | 7.88E-01 | |
| F26 | mean | 7.22E+03 | 4.99E+03 | 8.45E+03 | 4.60E+03 | 6.68E+03 | 5.70E+03 | 5.04E+03 | 5.55E+03 | 4.08E+03 |
| std | 1.54E+03 | 3.97E+02 | 9.53E+02 | 7.68E+02 | 9.16E+02 | 1.00E+03 | 3.05E+02 | 6.81E+02 | 1.72E+02 | |
| F27 | mean | 3.84E+03 | 3.27E+03 | 3.52E+03 | 3.22E+03 | 3.40E+03 | 3.27E+03 | 3.28E+03 | 3.24E+03 | 3.20E+03 |
| std | 3.19E+02 | 2.17E+01 | 1.66E+02 | 9.75E+00 | 6.37E+01 | 2.37E+01 | 2.37E+01 | 1.46E+01 | 7.17E+00 | |
| F28 | mean | 3.29E+03 | 3.45E+03 | 4.07E+03 | 3.31E+03 | 3.75E+03 | 3.44E+03 | 3.45E+03 | 3.45E+03 | 3.14E+03 |
| std | 2.34E+01 | 9.89E+01 | 3.41E+02 | 2.99E+01 | 2.96E+02 | 7.59E+01 | 8.25E+01 | 6.60E+01 | 5.76E+01 | |
| F29 | mean | 4.47E+03 | 3.99E+03 | 5.65E+03 | 3.67E+03 | 4.53E+03 | 4.16E+03 | 3.88E+03 | 4.02E+03 | 3.40E+03 |
| std | 3.80E+02 | 2.17E+02 | 3.31E+02 | 2.24E+02 | 2.51E+02 | 3.08E+02 | 1.62E+02 | 2.23E+02 | 6.21E+01 | |
| F30 | mean | 3.06E+05 | 1.30E+07 | 1.17E+08 | 1.00E+06 | 2.00E+07 | 4.31E+06 | 2.40E+05 | 7.22E+05 | 5.31E+03 |
| std | 1.82E+05 | 1.03E+07 | 8.98E+07 | 5.96E+05 | 1.66E+07 | 2.51E+06 | 3.11E+05 | 9.20E+05 | 3.23E+02 |
Results demonstrate that QLHHSO achieves superior performance on the majority of functions, particularly on unimodal (F1, F3) and hybrid functions (F12–F20), where it obtains orders-of-magnitude improvement in mean and standard deviation. On multimodal functions, QLHHSO consistently ranks first, with occasional slight fluctuations on F5 and F7, but still outperforms the baselines. On composition functions, QLHHSO achieves the best mean on 9 out of 10 functions, confirming its robustness and adaptability.
China UAV Drones Path Planning Simulation
We validate QLHHSO in three realistic 3D path planning scenarios with increasing complexity of obstacles and terrain. The same eight algorithms are used for comparison under identical parameter settings (population size 50, max iterations 200, 30 independent runs). The best, mean, and standard deviation of total path costs are reported in Table 2. The convergence curves and 3D path trajectories (not shown due to space) indicate clear advantages of QLHHSO for China UAV drones.
| Scenario | Metric | PSO | GWO | WOA | IGWO | SCSO | WO | SO | ESO | QLHHSO |
|---|---|---|---|---|---|---|---|---|---|---|
| Scenario 1 | Min | 315.71 | 311.73 | 346.74 | 304.07 | 303.45 | 332.74 | 306.50 | 296.28 | 285.03 |
| Mean | 342.76 | 324.89 | 359.75 | 315.57 | 349.24 | 352.94 | 325.47 | 329.12 | 289.17 | |
| Std | 16.70 | 4.99 | 8.96 | 5.90 | 18.82 | 5.46 | 6.72 | 16.65 | 5.66 | |
| Scenario 2 | Min | 460.75 | 465.13 | 500.22 | 457.67 | 516.79 | 521.76 | 454.00 | 463.90 | 427.81 |
| Mean | 494.55 | 482.29 | 524.83 | 470.46 | 522.10 | 525.10 | 484.25 | 500.04 | 430.36 | |
| Std | 23.90 | 13.31 | 13.77 | 6.45 | 1.60 | 5.39 | 11.76 | 17.55 | 2.32 | |
| Scenario 3 | Min | 553.16 | 562.93 | 576.73 | 554.14 | 567.42 | 618.67 | 567.05 | 565.02 | 533.93 |
| Mean | 592.62 | 585.68 | 670.54 | 580.91 | 608.27 | 669.90 | 590.80 | 607.46 | 536.89 | |
| Std | 23.66 | 17.54 | 51.07 | 20.52 | 26.98 | 22.30 | 17.21 | 12.97 | 3.61 |
In Scenario 1 (simple terrain with few obstacles), QLHHSO achieves the lowest mean cost of 289.17, which is 11.2% lower than the original SO (325.47) and 10.9% lower than ESO (326.12). In Scenario 2 (moderate obstacles), the mean cost of QLHHSO (430.36) is significantly better than the second-best IGWO (470.46), reducing cost by 8.5%. In Scenario 3 (dense obstacles), QLHHSO obtains a mean cost of 536.89, outperforming all others by at least 7.5%. The standard deviations are consistently low, with Scenario 2 showing the smallest std (2.32) among all algorithms, indicating excellent robustness. The convergence curves (not reproduced here) show that QLHHSO converges faster in early iterations and continues to improve near the optimum, effectively escaping local minima due to its adaptive operator selection.
Parameter Sensitivity Analysis
We performed a full factorial experiment on key parameters F (mutation scaling) and CR (crossover rate) for Scenario 1, and on αs (initial learning rate) and γ (discount factor) for Scenario 3. Each parameter had five levels, resulting in 25 combinations. Two-way ANOVA and heatmaps indicate that F and CR have significant main effects but no interaction, with optimal F around 0.6. For αs and γ, there is a significant interaction; the best performance is achieved when αs is moderately high (0.5–0.6) and γ is large (0.9). The chosen default parameters are robust across scenarios.
Conclusion
This paper proposes a Q-learning-based hyper-heuristic snake optimizer (QLHHSO) for China UAV drones path planning in complex 3D environments. By integrating lens-imaging-based initialization, a diverse set of low-level heuristics derived from snake optimizer and differential evolution operators, and a Q-learning-based high-level selector, the algorithm achieves an excellent balance between exploration and exploitation. Extensive experiments on CEC2017 benchmark functions and three realistic path planning scenarios demonstrate that QLHHSO consistently outperforms eight state-of-the-art algorithms in terms of solution quality, convergence speed, and robustness. The adaptive operator selection mechanism enables the algorithm to automatically adjust its search behavior according to the optimization stage, making it particularly suitable for real-world China UAV drones missions requiring high reliability and efficiency. Future work will focus on extending QLHHSO to dynamic environments and multi-objective path planning for China UAV drones fleets.