The rapid evolution of the Unmanned Aerial Vehicle (UAV) industry towards informatization and large-scale applications has significantly expanded its footprint in the civil domain. Traditional segregated airspace operations can no longer accommodate the burgeoning demand for UAV utilization. Consequently, regulatory frameworks worldwide are evolving to permit the integrated or mixed operation of manned aircraft and drones within shared airspace. This integration, while promising enhanced operational flexibility and economic benefits, inevitably introduces new layers of complexity to the airspace environment and resource management, posing substantial safety challenges. Therefore, developing effective and feasible safety assessment methodologies for Mixed Operation of Manned Aircraft and UAVs (MOMAU) is paramount. The core of such methodologies lies in the systematic identification and analysis of factors influencing safety, enabling proactive risk mitigation and disturbance management. This article conducts an analysis and research on MOMAU safety assessment from the following key aspects.
The foundational step in safety assessment is a thorough risk factor identification. Guided by the classic “Four-Element” model (Man, Machine, Environment, Management), potential risk factors in MOMAU are systematically analyzed.
| Element | Category | Identified Risk Factors (Examples) |
|---|---|---|
| Man | Air Traffic Controller (ATC) | Lack of MOMAU procedural experience/skills; High workload; “Error, Omission, Forgetfulness” regarding flight dynamics; Unstandardized phraseology; Inadequate coordination. |
| Manned Aircraft Crew | Misinterpreting or violating ATC instructions; Insufficient emergency response capability; Inadequate pre-flight preparation. | |
| UAV Operator | Deficient situational awareness; Inadequate control skills; Poor crew coordination; Improper control handover procedures. | |
| Machine | Equipment & Systems | Manned aircraft avionics failure; UAV airborne system/communication link failure; ATC system/radar failure; UAV Ground Control Station (GCS) failure; Lack of timely maintenance. |
| Environment | Internal & External | Adverse weather; Low accuracy of weather forecasts; Radio frequency interference; Complex airspace structure; Poor ATC/GCS workplace environment. |
| Management | Organizational & Regulatory | Insufficient technical and safety drone training; Inadequate MOMAU emergency plans and drills; Incomplete regulatory framework for mixed operations; Unreasonable staff scheduling. |
Not all identified factors carry equal risk significance. To prioritize risk management resources, the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method is employed to screen and rank these factors based on three metrics: probability of occurrence, severity of consequence, and controllability. Experts rate each factor on these metrics (e.g., using scales of 1-5). The TOPSIS process involves constructing a decision matrix, normalizing it, applying weights to the metrics (e.g., via AHP: $\omega = (0.3196, 0.5584, 0.1220)$ for probability, severity, controllability respectively), and calculating the relative closeness ($C_i$) of each factor to the ideal solution. Factors with higher $C_i$ are more critical. The key calculation steps are shown below.
Normalization (for benefit-type metric like severity, higher is worse):
$$ h_{ij} = \frac{x_{ij} – \min(x_j)}{\max(x_j) – \min(x_j)} $$
For cost-type metric like controllability (higher controllability is better):
$$ h_{ij} = \frac{\max(x_j) – x_{ij}}{\max(x_j) – \min(x_j)} $$
Then, the normalized matrix is weighted: $V = [w_j \cdot r_{ij}]$, where $r_{ij}=h_{ij} / \sum_{i} h_{ij}$. The positive ideal solution ($Y^+$) and negative ideal solution ($Y^-$) are determined:
$$ Y^+ = (\max(f_{1}), \max(f_{2}), …, \max(f_{n})), \quad Y^- = (\min(f_{1}), \min(f_{2}), …, \min(f_{n})) $$
The separation measures are calculated using the Euclidean distance:
$$ d_i^+ = \sqrt{\sum_{j=1}^{n}(f_{ij} – Y_j^+)^2}, \quad d_i^- = \sqrt{\sum_{j=1}^{n}(f_{ij} – Y_j^-)^2} $$
The relative closeness to the ideal solution is:
$$ C_i = \frac{d_i^-}{d_i^+ + d_i^-} $$
Applying this method, critical risk factors such as UAV operator situational awareness deficiency, ATC lack of MOMAU experience, and inadequate mixed operation regulations are typically ranked highest. Based on these critical factors and principles of systematicity, scientific rigor, and practicality, a multi-level safety assessment indicator system is constructed, as summarized below.
| Primary Indicator (U) | Secondary Indicator | Tertiary Indicator (Examples) |
|---|---|---|
| MOMAU Safety Assessment | Personnel (A) | A1: ATC MOMAU Competency; A2: ATC “Error, Omission, Forgetfulness”; A3: ATC Regulation Compliance; A4: ATC Workload; A5: ATC Phraseology Standardization; A6: ATC Attention Distribution; A7: Manned-UAV Separation Appropriateness; A8: Pilot Instruction Compliance; A9: Pilot Emergency Response; A10: UAV Operator Situational Awareness; A11: UAV Control Handover; A12: UAV Operator Control Skill; A13: UAV Crew Coordination. |
| Equipment (B) | B1: Manned Aircraft Airborne System Integrity; B2: UAV Airborne System Integrity; B3: ATC System Integrity; B4: UAV GCS Integrity. | |
| Environment (C) | C1: Weather Mutation; C2: Radio Interference; C3: Airspace Complexity. | |
| Management (D) | D1: Technical & Safety Drone Training Status; D2: MOMAU Emergency Plan & Drill Development; D3: Completeness of Mixed Operation Rules. |
A robust safety assessment requires an appropriate model to handle uncertainty and integrate multi-source information. The first model proposed combines the CRiteria Importance Through Intercriteria Correlation (CRITIC) method for objective weighting with the Cloud Model for comprehensive evaluation.
1. Weight Determination using CRITIC Method: The CRITIC method determines weights based on the contrast intensity and conflicting character of evaluation criteria. For ‘m’ alternatives and ‘n’ criteria, the weight $W_j$ for criterion j is calculated as follows, where $\sigma_j$ is the standard deviation and $r_{jt}$ is the correlation coefficient between criteria j and t.
Calculate the amount of information $C_j$:
$$ C_j = \sigma_j \sum_{t=1}^{n} (1 – |r_{jt}|) $$
Then, the objective weight is:
$$ W_j = \frac{C_j}{\sum_{t=1}^{n} C_t} $$
2. Safety Evaluation using Cloud Model: The Cloud Model, characterized by three digital features – Expectation ($Ex$), Entropy ($En$), and Hyper-Entropy ($He$) – effectively handles the randomness and fuzziness in safety evaluations. A set of standard clouds representing safety grades (e.g., Poor, Fair, Medium, Good, Excellent) is first established over the universe [0,10] using the Golden Section method. The digital features for the “Medium” level are typically set as $(Ex=5, En=0.39, He=0.05)$.
For a given indicator, expert ratings (e.g., on a 0-10 scale) are collected. The comprehensive cloud digital features ($Ex_i, En_i, He_i$) for a tertiary indicator are computed by aggregating expert input clouds. The cloud for a higher-level indicator (e.g., Personnel factor ‘A’) is synthesized from its child indicators’ clouds and their CRITIC-derived weights $\omega_i$:
$$ Ex_A = \frac{\sum_{i=1}^{k} \omega_i \cdot Ex_i \cdot En_i}{\sum_{i=1}^{k} \omega_i \cdot En_i}, \quad En_A = \sum_{i=1}^{k} \omega_i \cdot En_i, \quad He_A = \frac{\sum_{i=1}^{k} \omega_i \cdot He_i \cdot En_i}{\sum_{i=1}^{k} \omega_i \cdot En_i} $$
The final comprehensive safety cloud for the entire MOMAU system ($U$) is calculated similarly using the secondary indicators’ clouds and their weights. By comparing the generated cloud diagram of the comprehensive assessment with the standard safety grade clouds visually (e.g., using MATLAB), the overall safety level is determined. An assessment often reveals that indicators like ATC workload (A4), UAV operator situational awareness (A10), and completeness of mixed operation rules (D3) tend to have lower $Ex$ values, indicating higher risk. Effective countermeasures include enhancing specialized drone training for ATCs and UAV operators, implementing advanced decision support tools to reduce workload, and expediting the development of robust mixed operation regulations and contingency protocols.
The second model employs a combined weighting approach integrated with Fuzzy Comprehensive Evaluation (FCE) to enhance the scientific rigor of weight assignment and manage assessment ambiguity.
1. Combined Weighting based on Game Theory: This approach seeks Nash equilibrium between subjective weights (from Improved Analytic Hierarchy Process – IAHP) and objective weights (from Entropy Weight Method – EWM).
- Improved AHP: Instead of the standard 1-9 scale, an improved scale synthesizing multiple scaling methods (1-9, exponential scales $e^{0/4}$~$e^{8/4}$, $e^{0/5}$~$e^{8/5}$) is used to construct judgment matrices that better reflect expert expectations. The weights are calculated using the arithmetic mean, geometric mean, and eigenvector methods, and then averaged to improve robustness. Consistency is verified ($CR < 0.1$).
- Entropy Weight Method: For ‘m’ evaluation objects and ‘n’ indicators, the entropy $e_j$ and weight $\omega_j^{ent}$ for indicator j are calculated as:
$$ p_{ij} = \frac{d_{ij}}{\sum_{i=1}^{m} d_{ij}}, \quad e_j = -\frac{1}{\ln(m)} \sum_{i=1}^{m} p_{ij} \ln(p_{ij}), \quad \omega_j^{ent} = \frac{1 – e_j}{\sum_{j=1}^{n} (1 – e_j)} $$
where $d_{ij}$ is the standardized value. - Game Theory Combination: The optimal combined weight vector $\omega^*$ is a linear combination of the weight vectors from IAHP ($\omega^{ahp}$) and EWM ($\omega^{ent}$): $\omega^* = \alpha_1^* \cdot \omega^{ahp} + \alpha_2^* \cdot \omega^{ent}$. The coefficients $\alpha_1^*, \alpha_2^*$ are determined by solving the following minimization problem to achieve equilibrium:
$$ \min \left\| \alpha_1 \cdot (\omega^{ahp})^T \omega^{ahp} + \alpha_2 \cdot (\omega^{ahp})^T \omega^{ent} – (\omega^{ahp})^T \omega^{ahp} \right\| $$
$$ \min \left\| \alpha_1 \cdot (\omega^{ent})^T \omega^{ahp} + \alpha_2 \cdot (\omega^{ent})^T \omega^{ent} – (\omega^{ent})^T \omega^{ent} \right\| $$
After obtaining $\alpha_1, \alpha_2$, they are normalized to get $\alpha_1^*, \alpha_2^*$.
2. Fuzzy Comprehensive Evaluation: The evaluation space is defined by a factor set (the established indicator system) and a comment set $V = \{$Poor, Fair, Medium, Good, Excellent$\}$, corresponding to score ranges [0,2), [2,4), [4,6), [6,8), [8,10]. Based on expert ratings, a fuzzy membership matrix $R$ is constructed for each indicator subset, where each element $r_{ij}$ denotes the degree to which the indicator belongs to comment level $j$.
The fuzzy evaluation for a secondary factor (e.g., Personnel ‘A’) is obtained by synthesizing its combined weight vector $\omega_A^*$ with its fuzzy matrix $R_A$:
$$ P_A = \omega_A^* \circ R_A = (\omega_{A1}^*, \omega_{A2}^*, …, \omega_{Ak}^*) \circ \begin{bmatrix} r_{11} & … & r_{1m} \\ \vdots & \ddots & \vdots \\ r_{k1} & … & r_{km} \end{bmatrix} = (p_{A1}, p_{A2}, …, p_{Am}) $$
where ‘$\circ$’ denotes a fuzzy composition operator, typically the weighted average model. The evaluations for all secondary factors form the fuzzy matrix $R_U$ for the primary goal. The final comprehensive evaluation vector $P_U$ is calculated using the combined weights of the secondary factors $\omega_U^*$: $P_U = \omega_U^* \circ R_U$. The comprehensive score $G$ is derived by multiplying $P_U$ with the transpose of the score vector $H$ (e.g., $H^T = [1, 3, 5, 7, 9]$): $G = P_U \cdot H^T$. This score maps to a safety grade. Application of this model to a case study typically yields a result like “Good” (score around 6.0-6.5), aligning with the findings from the CRITIC-Cloud model and validating the reliability of both approaches. It consistently highlights that management factors, particularly the state of drone training and regulatory frameworks (D1, D2, D3), often receive lower scores, underscoring their criticality.
In conclusion, this research systematically addresses the safety assessment challenge in MOMAU. It establishes a comprehensive risk factor framework and a multi-level indicator system by integrating the “Four-Element” model with TOPSIS screening. Two distinct yet complementary assessment methodologies are developed and validated. The CRITIC-Cloud Model excels in handling fuzziness and providing intuitive visualization of safety states. The Game Theory-based Combined Weighting with Fuzzy Comprehensive Evaluation enhances weight objectivity and effectively processes qualitative judgments. Both models converge on similar findings, identifying personnel competency (especially in MOMAU procedures), UAV operator proficiency, equipment reliability for drones, and the maturity of management systems—including comprehensive drone training programs and operational regulations—as pivotal areas requiring focused risk mitigation. Future work should involve dynamic updates to the indicator system with operational data, exploration of real-time assessment techniques, and deeper integration of human factor analysis, particularly concerning the interaction between ATCs, pilots, and remote UAV operators in the shared airspace loop.