The evolution of aerial warfare is inexorably moving towards intelligent, collaborative systems where manned aircraft and unmanned aerial vehicles (UAV drones) operate as a cohesive force. This paradigm of Manned-Unmanned Teaming (MUM-T) promises to amplify combat effectiveness, enhance operational flexibility, and reduce risk to human pilots. At the heart of realizing this vision lies Deep Reinforcement Learning (DRL), a powerful technique enabling UAV drones to learn complex, dynamic combat strategies through autonomous interaction with their environment. DRL algorithms can master intricate air combat maneuvers, cooperative tactics, and adaptive decision-making far beyond the scope of pre-programmed rules. However, a significant barrier impedes the seamless integration of these intelligent agents into critical military operations: their inherent “black-box” nature. The deep neural networks that underpin DRL policies are notoriously opaque. While a UAV drone may execute a brilliant tactical maneuver, the human pilot or commander in the loop often cannot comprehend *why* that particular decision was made. This lack of transparency erodes trust, hampers effective human-machine collaboration, and poses serious safety and accountability risks in high-stakes environments. Therefore, developing eXplainable Artificial Intelligence (XAI) methods for DRL is not merely an academic pursuit but a critical operational necessity for the future of intelligent air combat.
This article addresses this pivotal challenge by proposing a novel, two-tiered interpretability framework designed specifically for the context of manned-UAV cooperative air combat. Our approach synergistically combines the causal reasoning power of Dynamic Bayesian Networks (DBNs) with the equitable contribution analysis of Shapley values. The framework first answers the “what” by identifying the high-level tactical intent of the UAV drones (e.g., flanking, climbing, evasion) from its low-level control sequence. It then answers the “why” by quantifying and ranking the contribution of each piece of situational data (e.g., enemy position, friendly aircraft status) to every critical decision, providing a human-understandable rationale. Furthermore, we validate the fidelity of our explanations and demonstrate their practical utility by using the interpretability results to guide and improve the DRL training process itself.
The Manned-UAV Cooperative Air Combat Problem
We consider a representative beyond-visual-range air combat scenario involving a heterogeneous red team and a blue adversary. The red team comprises one manned fighter aircraft (the leader) and two autonomous UAV drones (the wingmen). The blue team consists of a single, potentially superior, adversary aircraft. The core objective for the red team is to cooperatively engage and neutralize the blue threat while ensuring the survival of the high-value manned asset. The UAV drones act as force multipliers—they can scout, distract, absorb enemy fire, and execute coordinated attacks, all orchestrated to create a tactical advantage for the manned fighter.
The mathematical foundation models the kinematics and relative combat geometry. Each aircraft’s motion in 3D space is governed by simplified point-mass kinetics. The state of an aircraft \(i\) at time \(t\) is defined by its position \((x_i^t, y_i^t, h_i^t)\), velocity \(v_i^t\), flight path angle \(\theta_i^t\), and heading angle \(\psi_i^t\). The control inputs are the normalized load factors in the longitudinal, lateral, and vertical axes: \(a_i^t = [n_x^t, n_y^t, n_z^t]\). The dynamics are given by:
$$
\begin{aligned}
\dot{v}_i &= g (n_x – \sin\theta_i) \\
\dot{\theta}_i &= \frac{g}{v_i} (n_y \cos\phi_i – \cos\theta_i) \\
\dot{\psi}_i &= -\frac{g}{v_i} \frac{n_y \sin\phi_i}{\cos\theta_i}
\end{aligned}
$$
where \(g\) is gravity and \(\phi_i\) is the bank angle. The combat geometry between a red aircraft \(r\) and the blue aircraft \(b\) is characterized by the relative range \(d_{rb}\), line-of-sight (LOS) angles, the aspect angle \(\varphi_{rb}\) (the angle between the red aircraft’s velocity vector and the LOS to blue), and the antenna train angle \(q_{rb}\) (the angle between the blue aircraft’s velocity vector and the LOS to red). These are critical for assessing who has the tactical advantage for a missile launch. A launch opportunity is deemed valid when the target is within both sensor range and missile kinematic range, formalized as a set of conditions on \(d_{rb}\), \(\varphi_{rb}\), and \(q_{rb}\). The success condition for the red team is to maintain this launch condition on the blue aircraft for a sustained period while protecting the manned fighter from an analogous threat.
A Deep Reinforcement Learning Framework for Cooperative Tactics
To train the red team’s cooperative policy, we employ the Multi-Agent Twin Delayed Deep Deterministic Policy Gradient (MATD3) algorithm. MATD3 is well-suited for this continuous control problem in a multi-agent setting, as it mitigates overestimation bias and improves learning stability. Each agent (manned aircraft, UAV drone 1, UAV drone 2) has its own actor-network (policy) and critic-network (value function), which are trained concurrently using a centralized-training-with-decentralized-execution paradigm.
The joint state observation \(S_t\) available to all red agents via data link is comprehensive, encompassing:
$$ S_t = \{ \mathbf{P}_r, \mathbf{P}_b, \mathbf{D}_{rel}, \mathbf{V}, \mathbf{A}_{red}, \mathbf{O}_{uav} \} $$
where \(\mathbf{P}_r, \mathbf{P}_b\) are red and blue team positions; \(\mathbf{D}_{rel}\) includes all inter-agent ranges and LOS angles; \(\mathbf{V}\) contains velocities and attitudes; \(\mathbf{A}_{red}\) is the history of red team’s own control actions; and \(\mathbf{O}_{uav}\) is the survival status of the UAV drones.
The action space for each agent is its three-axis load factor command: \(a_i^t = [n_x, n_y, n_z]\).
The reward function \(R_t\) is meticulously designed to shape the desired cooperative behavior. It is a weighted sum of terminal and step-wise rewards:
$$ R_t = R_{terminal} + R_{step} $$
\(R_{terminal}\) provides a large positive reward for a successful blue kill, a large negative reward for the loss of the manned aircraft, and a smaller negative reward for the loss of a UAV drone. \(R_{step}\) is a composite guiding signal:
$$ R_{step} = w_a R_{angle} + w_d R_{distance} + w_e R_{energy} + w_c R_{coop} + R_{penalty} $$
- \(R_{angle}\) encourages owning the positional advantage (minimizing own aspect angle while maximizing the enemy’s).
- \(R_{distance}\) encourages closing within effective weapon range.
- \(R_{energy}\) encourages maintaining a higher total energy state (altitude + kinetic) relative to the enemy.
- \(R_{coop}\) is crucial for UAV drone teamwork. It encourages the two UAV drones to approach the target from different bearing angles (flanking) and to time their attacks coherently.
- \(R_{penalty}\) discourages mid-air collisions between friendly aircraft.
Through curriculum learning and self-play, the MATD3 algorithm learns a policy where the red team exhibits sophisticated emergent behavior. The manned fighter often acts as a commander and final arbitrator, while the UAV drones execute high-risk maneuvers, draw fire, and set up kill opportunities. However, this policy remains an inscrutable neural network.
The Bayesian-Shapley Explainability Framework
Our framework decomposes the explanation task into two logically sequential questions addressed by separate modules: the Intent Explainer and the Decision Explainer.
1. Intent Explainer: What is the UAV Drone Doing? (Dynamic Bayesian Network)
The continuous stream of low-level load factor commands is not directly interpretable. The Intent Explainer’s role is to segment the UAV drone’s trajectory into semantically meaningful chunks labeled with recognized tactical maneuvers. We formulate this as an online inference problem using a Dynamic Bayesian Network (DBN).
The DBN models the temporal evolution of a hidden variable—the maneuver type \(M_t\)—based on observable flight state features \(E_t\). The feature set \(E_t\) is derived from the UAV’s state and includes: velocity trend \((\Delta v)\), heading rate \((\Delta \psi)\), altitude rate \((\Delta h)\), and their higher-level integrals. The library of maneuvers \(M_t\) includes: Level Flight, Climb, Dive, Left/Right Turn, Immelmann, Split-S, High-G Turn, etc.
The DBN’s structure links time slices, allowing it to reason about the most likely sequence of maneuvers given the observed evidence. The conditional probability tables linking features to maneuvers are informed by expert knowledge and historical data. At each time step, the DBN performs inference to compute the posterior distribution over maneuver classes:
$$ P(M_t | E_{1:t}) \propto P(E_t | M_t) \sum_{M_{t-1}} P(M_t | M_{t-1}) P(M_{t-1} | E_{1:t-1}) $$
The maneuver with the highest posterior probability is assigned as the intent label for that time segment. The output is a time-stamped log: “UAV-1 executed a Climb (t=0-2s), followed by a High-G Right Turn (t=2-5s)…”. This provides the first layer of explanation, translating actuator commands into pilot-understandable tactical actions.
2. Decision Explainer: Why Did the UAV Drone Change its Intent? (Shapley Value Analysis)
Knowing the “what” is incomplete without the “why.” Why did the UAV drone initiate a turn at that precise moment? The Decision Explainer focuses on the critical transition points between intents identified by the DBN. At these junctures, we employ Shapley values from cooperative game theory to attribute the change in the policy’s output (the action \(a_t\)) to changes in the input state features.
Shapley values provide a theoretically sound method for fair attribution. In our context, the “players” are the individual components of the state vector \(S_t\) (e.g., blue aircraft’s x-coordinate, friendly UAV’s altitude, relative bearing). The “game” is the computation of the action by the DRL policy network \(f(\cdot)\). The “payout” is the specific numerical value of one of the output load factors \((n_x, n_y, n_z)\). The Shapley value \(\phi_j(v)\) for feature \(j\) measures its average marginal contribution to the output, considering all possible subsets of other features:
$$ \phi_j(v) = \sum_{Q \subseteq S \setminus \{j\}} \frac{|Q|! (|S|-|Q|-1)!}{|S|!} [v(Q \cup \{j\}) – v(Q)] $$
where \(v(Q)\) is the value of the policy output when only the features in subset \(Q\) are used (others are set to a baseline value).
Computing exact Shapley values is intractable for our high-dimensional state. We therefore use a Monte Carlo sampling approximation:
$$ \hat{\phi}_j \approx \frac{1}{m} \sum_{k=1}^{m} \left[ f(\mathbf{x}_{+j}^k) – f(\mathbf{x}_{-j}^k) \right] $$
where \(m\) is the number of samples, \(\mathbf{x}_{+j}^k\) is a sample where feature \(j\) takes its current value, and \(\mathbf{x}_{-j}^k\) is a sample where it takes a baseline value (we use the state from the previous timestep, providing a local, causal contrast).
The result is a ranked list for each control axis at each intent-switch moment. For example, the explanation might be: “UAV-1’s decision to start a right turn (increase \(n_y\)) was 65% influenced by the blue aircraft moving left (change in \(x_b\)), 20% by UAV-2 closing in from the right, and 15% by the manned aircraft dropping in energy.” This state-level attribution directly answers the pilot’s question of “why now?”
Simulation, Validation, and Model Improvement
We trained the MATD3 policy in a simulated environment. The red team’s assets had inferior performance characteristics compared to the blue adversary, necessitating intelligent cooperation. After training, the red team achieved a high success rate, with UAV drones effectively flanking and distracting the blue aircraft to set up shots for the manned fighter.
Applying our Bayesian-Shapley framework to these engagements yielded clear, post-hoc explanations. The DBN successfully segmented complex trajectories into sequences of basic fighter maneuvers. The Shapley analysis at transition points consistently highlighted key situational triggers. For instance, a UAV drone’s turn initiation was most frequently attributed to changes in the enemy’s lateral position or the status of its teammate, validating that the learned policy was indeed responsive to cooperative cues and enemy motion.
To quantitatively validate the correctness of our Shapley-based attributions, we conducted a state ablation study. We calculated the average absolute Shapley value for every state feature across many episodes, creating a global ranking of feature importance.
| Rank | High-Importance Features (Most Influential) | Low-Importance Features (Least Influential) |
|---|---|---|
| 1 | UAV-2’s Vertical Load (\(n_{z}^{U2}\)) | LOS Bearing: Manned ↔ UAV-1 |
| 2 | UAV-2’s Longitudinal Load (\(n_{x}^{U2}\)) | LOS Elevation: UAV-1 ↔ UAV-2 |
| 3 | Manned Aircraft’s Vertical Load (\(n_{z}^{M}\)) | Range: Manned ↔ UAV-1 |
| 4 | UAV-1’s Vertical Load (\(n_{z}^{U1}\)) | LOS Elevation: Manned ↔ UAV-1 |
| 5 | UAV-2’s Heading (\(psi_{U2}\)) | LOS Bearing: Manned ↔ UAV-2 |
We then retrained three separate DRL models from scratch: 1) a baseline with the full state, 2) a model with the top-5 important features masked, and 3) a model with the bottom-5 important features masked. The results were compelling:
- Masking High-Importance Features: Training convergence was severely hampered. The model struggled to learn an effective policy, confirming that the features Shapley identified as critical were indeed essential for decision-making.
- Masking Low-Importance Features: Training proceeded almost identically to the baseline, often converging slightly faster. This proves that the low-Shapley features were largely redundant, and their removal streamlined the learning process without harming performance.
This ablation study serves a dual purpose: it validates the accuracy of our Shapley value explanations, and it provides a practical methodology for state space optimization. By pruning low-contribution features from the observation vector, we can design more efficient and faster-training DRL models for UAV drones, a direct practical benefit derived from explainability analysis.
Conclusion and Future Outlook
The integration of intelligent, autonomous UAV drones into manned combat teams is inevitable. For this integration to be safe, effective, and trusted, the decision-making processes of these UAV drones must be transparent. This work presents a robust, two-level explainability framework that makes significant strides toward this goal. By using Dynamic Bayesian Networks to infer tactical intent and Shapley values to attribute decisions to precise situational factors, we can provide human operators with clear answers to “what is the UAV drone doing?” and “why is it doing that?”.
Furthermore, we demonstrated that the outputs of this explanatory framework are not just informative narratives but are empirically verifiable and actionable. The feature importance rankings can be used to refine the DRL system itself, leading to more efficient and potentially more robust policies. Future work will focus on scaling this approach to larger teams of UAV drones, extending the intent recognition to higher-level tactical goals (e.g., “Suppress,” “Cover,” “Engage”), and developing real-time explanation interfaces that can seamlessly integrate into a pilot’s cockpit display. The path to true cognitive teaming between humans and intelligent UAV drones is built on the foundation of mutual understanding, and explainable AI is the cornerstone of that foundation.