In recent years, drone technology has rapidly evolved from a niche hobbyist tool into a critical component of both civilian and military operations. The integration of deep reinforcement learning (DRL) has enabled autonomous decision-making in complex environments, ranging from surveillance and reconnaissance to collaborative combat and low-altitude logistics. However, the inherent vulnerabilities of DRL models pose significant security risks, particularly through backdoor attacks and adversarial policy attacks. As drone technology becomes increasingly autonomous, ensuring the robustness and security of DRL algorithms is paramount. This survey, written from the perspective of our research group, systematically reviews the current state of attack and defense mechanisms in DRL-based drone technology. We focus on the unique challenges posed by drone platforms, such as limited computational resources, dynamic operational environments, and the need for real-time responses. Through extensive analysis of training-phase backdoor attacks and testing-phase policy attacks, we highlight the evolution of adversarial techniques and the corresponding defense strategies. We also introduce a federated collaborative attack-defense architecture tailored for drone swarms. Finally, we outline future research directions, emphasizing lightweight algorithms, multi-agent coordination, and the integration of emerging technologies like quantum communication and digital twins. Our goal is to provide a comprehensive reference for securing drone technology against DRL-based threats.
Introduction
The advent of deep reinforcement learning has significantly enhanced the autonomous capabilities of drone technology. By interacting with the environment and optimizing policies through trial-and-error, DRL enables drones to perform complex tasks such as obstacle avoidance, target tracking, and cooperative formation flight. However, the reliance on neural network models introduces a new attack surface. Malicious actors can exploit vulnerabilities in the training data, model parameters, or input sensors to hijack drone behavior, leading to mission failure or even physical damage. Recent conflicts, such as the war in Ukraine, have demonstrated the real-world impact of such attacks: GPS spoofing and adversarial inputs have been used to deceive drone navigation systems. As drone technology continues to expand into critical infrastructure, public safety, and defense, securing DRL algorithms becomes an urgent priority.
In this survey, we address the security challenges of DRL in drone technology from both offensive and defensive perspectives. We categorize attacks into two main types: backdoor attacks during training and policy attacks during testing. Corresponding defenses are reviewed, including data sanitization, adversarial training, and federated robust aggregation. Additionally, we propose a hierarchical federated collaborative architecture that leverages the distributed nature of drone swarms to enhance resilience. We also discuss evaluation methodologies and open challenges. The remainder of this paper is organized as follows: Section 2 reviews attack techniques; Section 3 presents defense mechanisms; Section 4 describes the federated collaborative framework; Section 5 outlines future research directions; Section 6 concludes the paper.
Attack Techniques in DRL for Drone Technology
DRL attacks directed at drone technology can be broadly classified by the phase in which they occur: training-phase backdoor attacks and testing-phase policy attacks. Each type exploits different weaknesses within the DRL pipeline. Below we elaborate on their evolution, implementation methods, and limitations.
Training-Phase Backdoor Attacks
Backdoor attacks contaminate the training dataset with poisoned samples containing a specific trigger pattern. During inference, when the trigger is present, the model produces adversary-desired outputs while behaving normally otherwise. Early works such as TrojDRL demonstrated that a 0.025% poisoning rate could mislead drone navigation by injecting pixel-pattern triggers into images. However, such out-of-distribution triggers are easily detected by simple feature-based filters. To evade detection, researchers proposed temporal-pattern triggers embedded in flight attitude sequences, and later, in-distribution triggers that leverage natural environmental features like wind speed or electromagnetic signatures. These advanced triggers blend seamlessly into the drone’s operational context, making detection significantly harder.
More recently, sparse targeted backdoor attacks (e.g., BadRL) inject poisoned signals into only 0.003% of training steps, leaving minimal footprint. Another line of work focuses on supply-chain-level attacks: TrojanentRL implants backdoor modules directly into the DRL network architecture during development, while InfectroRL remotely manipulates model parameters via communication channel vulnerabilities. Table I summarizes key backdoor attack methods, their trigger types, and operational constraints for drone technology.
| Attack Name | Trigger Type | Poisoning Rate | Key Feature | Limitations for Drones |
|---|---|---|---|---|
| TrojDRL | Pixel pattern (out-of-distribution) | 0.025% | Minimal data modification | Easily detected; not robust to dynamic drone environments |
| Temporal-pattern attack | Time-sequence anomaly | ~0.1% | Embedded in flight dynamics | Fails under strong turbulence or interference |
| In-distribution trigger | Natural feature (e.g., wind speed) | ~0.05% | Blends with environment | High modeling cost; unstable across varied terrains |
| BadRL (sparse targeted) | Unseen state signature | 0.003% of steps | Extremely low footprint | Requires precise control of injection timing; not tested on swarms |
| TrojanentRL | Network component | N/A (model-level) | No dataset modification needed | Requires physical access to development pipeline |
| InfectroRL | Remote parameter tampering | N/A (parameter-level) | Remote exploitation of communication | Depends on insecure wireless links; high risk of detection |
The core mathematical formulation of a backdoor attack in Q-learning can be expressed as a modification to the Bellman update:
$$ Q(s, a) \leftarrow Q(s, a) + \alpha \left[ r + \gamma \max_{a’} Q(s’, a’) – Q(s, a) \right] + \beta \cdot \mathbb{1}_{\{\text{trigger present}\}} \cdot \delta(s,a) $$
where $\beta$ controls the strength of the backdoor perturbation $\delta(s,a)$ that shifts the Q-value toward an adversary-chosen target. For drone technology, $\delta(s,a)$ may encode a malicious direction or altitude command. Current backdoor attacks face significant challenges when scaled to drone swarms: parameter sharing dilutes the poison signal, and the dynamic flight environment reduces trigger reliability. Moreover, the computational overhead of advanced attack algorithms often exceeds the capabilities of micro-drones, limiting their practical deployment.
Testing-Phase Policy Attacks
Policy attacks (also known as adversarial attacks) inject imperceptible perturbations into the input of the DRL agent during inference, causing the model to output incorrect actions. Unlike backdoor attacks, these do not require corrupting the training process and can be performed in real-time. In drone technology, such attacks can target sensor data (e.g., camera images, LIDAR, GPS) or communication signals. For example, GPS spoofing has been widely used in the Russia-Ukraine conflict to gradually deviate drones from their intended paths. Observation-space attacks generate adversarial examples that increase obstacle collision rates by over 30% in 3D navigation tasks.
More advanced attacks have evolved into multi-modal coordinated strategies: Projected Gradient Descent (PGD) attacks can simultaneously corrupt visual and radar inputs, exploiting the fusion of heterogeneous sensors. Communication-channel attacks add adversarial noise to command links, causing the drone to lose connection with the ground station and fall back to a compromised local policy. The attack effect can be quantified by the degradation of the expected cumulative reward:
$$ J(\pi) = \mathbb{E}_{\tau \sim p(\tau|\pi)} \left[ \sum_{t=0}^{T} \gamma^t r_t \right] \quad \text{vs.} \quad J_{\text{adv}}(\pi) = \mathbb{E}_{\tau \sim p(\tau|\pi)} \left[ \sum_{t=0}^{T} \gamma^t \left( r_t + \eta_t \right) \right] $$
where $\eta_t$ is the adversary-induced reward shift. Table II compares representative policy attack methods and their practical constraints on drone technology.
| Attack Method | Target Modality | Success Rate (Simulation) | Real-World Feasibility | Main Limitation |
|---|---|---|---|---|
| GPS spoofing | Positioning | ~80% | High (used in conflict) | Mitigated by multi-sensor fusion |
| Observation-space attack (PGD) | Camera/LIDAR | ~70% | Medium | Requires accurate environment model |
| Communication-channel attack | Control link | ~60% | Medium | Easily detected by spectrum monitoring |
| Multi-modal coordinated attack | Vision + GPS | ~85% | Low | High computational cost; resource-intensive for small drones |
A major limitation of current policy attacks is their use of fixed perturbation steps, which fail to adapt to the dynamic state of a flying drone. Additionally, many attacks target a single sensor, making them vulnerable to sensor redundancy checks. Finally, most evaluations are conducted in ideal simulation environments, leaving significant uncertainty about their effectiveness in real-world combat scenarios with electromagnetic interference and terrain occlusion. Addressing these gaps requires the development of adaptive, resource-efficient attack strategies that can match the computational budget of drone platforms.
Defense Mechanisms for DRL in Drone Technology
Defenses against DRL attacks on drone technology aim to detect, purify, or robustly resist adversarial manipulations. Given the resource constraints of drone platforms, defense algorithms must be lightweight and real-time. We discuss two major categories: backdoor defenses and policy attack defenses.
Backdoor Defenses
Backdoor defenses operate throughout the DRL lifecycle: from data collection and training to deployment. Early methods like Senti-Net extract statistical features from training data to flag outliers, but they fail against in-distribution triggers. Meta-neural analysis frameworks learn the distribution of benign data to detect anomalies, yet they require large amounts of clean data for meta-training. Neural Cleanse analyzes neuron activation patterns to identify backdoor signatures, though its computational cost is prohibitive for onboard detection on small drones.
Recent advances have moved toward full-chain defenses. Neural attention distillation (NAD) removes backdoor features by transferring attention maps from a teacher model to a student model. This technique has been successfully applied to civilian drone maintenance tasks, achieving a balance between purification and task performance. The FMADRL model from Beihang combines federated learning with an attention mechanism to detect backdoors in distributed training with 97.3% accuracy. Component-level security verification prevents backdoor module implantation by checking the integrity of each network layer. Additionally, data encryption (both symmetric and asymmetric) secures training data transmission, preventing tampering during offloading. Table III summarizes the pros and cons of these defenses in the context of drone technology.
| Defense Method | Detection/Purification | Computational Cost | Effectiveness Against In-Distribution Triggers | Suitability for Micro Drones |
|---|---|---|---|---|
| Senti-Net | Detection | Low | Poor | High |
| Meta-neural analysis | Detection | Medium | Fair | Medium |
| Neural Cleanse | Detection | High | Good | Low |
| Attention distillation | Purification | Medium | Good | Medium |
| FMADRL (federated) | Detection | Low (distributed) | Excellent | High (swarm) |
| Component verification | Prevention | Low | N/A (prevents injection) | High |
Despite these advances, several gaps persist: detection of in-distribution triggers remains unreliable; purification algorithms often degrade task performance; and distributed defense mechanisms are vulnerable to malicious nodes that falsify their local updates. Future work needs to develop anomaly detection that is both sensitive to natural triggers and robust to adversarial manipulation.
Policy Attack Defenses
Defenses against adversarial policy attacks primarily focus on enhancing model robustness through adversarial training, feature smoothing, and sensor fusion. PGD-based adversarial training has been successfully applied to military drone obstacle avoidance, increasing resilience to common perturbations. However, this approach incurs a high training overhead and may reduce nominal performance. Hybrid deep ensemble architectures combine multiple independent DRL models to vote on actions, providing defense against communication-channel attacks, but the ensemble size increases computational load.
More recently, antifragile reinforcement learning frameworks have emerged. By incorporating controllable adversarial examples during training, the drone learns to become stronger under attack. For example, the curriculum-guided antifragile RL proposed by Panda et al. reduced collision conflicts by over 30% under observation-space attacks. Another promising direction is multi-modal fusion defense: using LSTM-based anomaly detectors on sensor data can identify adversarial perturbations with 91% accuracy. The fusion of heterogeneous sensors (e.g., visual + radar + inertial) naturally mitigates attacks that target a single modality. The general defense framework can be formalized as solving a robust optimization problem:
$$ \max_{\pi} \min_{\delta \in \Delta} \mathbb{E}_{s \sim \mathcal{D}} \left[ Q^{\pi}(s, \pi(s + \delta)) \right] $$
where $\Delta$ is the perturbation set and $\mathcal{D}$ is the state distribution. Table IV compares the key attributes of these defenses.
| Defense Method | Robustness Gain | Training Overhead | Inference Speed | Sensor Dependency |
|---|---|---|---|---|
| PGD adversarial training | +25% success rate | High (3× training time) | Fast | Single modality |
| Hybrid ensemble | +20% success rate | Very high (multiple agents) | Slow | All modalities |
| Antifragile RL | +30% collision reduction | Medium | Fast | Observation-space |
| Multi-modal fusion + LSTM | +35% detection rate | Low (on-device LSTM) | Moderate | Multiple sensors |
A critical open challenge is balancing defense effectiveness with task performance. Many defenses that achieve strong robustness also degrade the drone’s ability to complete its primary mission (e.g., path smoothness, energy efficiency). As highlighted by recent studies, the dilemma “defense works but mission fails” must be resolved through joint optimization of security and task objectives.
Attack-Defense Evaluation Methodologies
Reliable validation of attack and defense techniques is essential for their practical deployment in drone technology. Current evaluation pipelines typically include simulation testing, hardware-in-the-loop (HIL) experiments, and operational wargaming. AirSim-based platforms offer high-fidelity physics simulation and customizable sensor models, enabling reproducible comparisons. However, simulation scenarios often lack the unpredictability of real-world electromagnetic and atmospheric conditions. HIL setups, while more realistic, are expensive and time-consuming. Standardized metrics such as expected mission completion rate, attack success probability, and false positive rate have been proposed, but they do not fully capture mission-specific requirements (e.g., stealth, latency). For drone technology, evaluation must also consider energy consumption and real-time constraints. A comprehensive evaluation framework should include adaptive scenario generation, adversarial robustness curves, and resource utilization profiling.
Federated Collaborative Attack-Defense Architecture for Drone Swarms
Traditional centralized defense architectures suffer from single points of failure, high communication overhead, and data privacy issues. For large-scale drone swarms, federated learning offers a natural solution by keeping data local and aggregating only model parameters. We propose a “three-level, four-layer” federated collaborative attack-defense architecture that enables distributed resilience. The three levels (node-level, cross-node-level, and global-level) provide progressive defense depth, while the four layers (sensing, execution, aggregation, and decision) form a closed-loop security pipeline.
Three-Level Collaborative Logic
At the node level, each drone independently runs a local DRL model with built-in anomaly detectors. This allows the drone to continue functioning even if the central controller is compromised. Cross-node collaboration involves encrypted parameter sharing among nearby drones to form a local consensus model, which improves detection of distributed stealth attacks. The global level aggregates all local models into a system-wide robust policy that can coordinate swarm behavior and allocate defensive resources. The hierarchy ensures that decisions are made with appropriate granularity and latency. For example, if a drone detects a local perturbation, it can immediately take evasive action (node-level), while simultaneously sending a compressed alert to neighboring units (cross-node) and to the ground station (global).
Four-Layer Architecture
The sensing layer comprises all onboard sensors (camera, radar, IMU, GPS). Raw data is preprocessed (denoised, normalized) and encrypted before being passed to the execution layer. The execution layer hosts the local DRL policy and defense modules (e.g., an LSTM-based attack detector). It performs local training, inference, and anomaly evaluation. The aggregation layer resides on edge nodes (e.g., high-altitude relay drones or ground vehicles) and performs secure aggregation of model updates using techniques like secure multi-party computation or differential privacy. This layer generates a global collaborative model and publishes a threat summary. Finally, the decision layer (typically a command center or lead drone) integrates global threat reports with mission objectives to issue coordinated attack-defence strategies, such as redistributing tasks among the swarm or activating countermeasures. The entire architecture is summarized in Figure 2 (conceptual) and the key functional requirements are listed in Table V.
| Layer | Function | Security Mechanism | Communication Demand | Example Implementation |
|---|---|---|---|---|
| Sensing | Data acquisition & preprocessing | Encryption, anomaly filtering | Low (local) | AES-256 encryption on sensor streams |
| Execution | Local DRL + defense agents | Model distillation, local detection | Medium (parameter upload) | Onboard Jetson Nano with lightweight LSTM |
| Aggregation | Secure federated averaging | Differential privacy, Byzantine-robust aggregation | High (model updates) | Federated averaging with trimmed mean |
| Decision | Global strategy optimization | Reinforcement learning on meta-level | Low (strategy broadcast) | Centralized Q-table for task reassignment |
We have mathematically formulated the collaborative defense as a constrained optimization:
$$ \min_{\theta_1,\dots,\theta_N} \sum_{i=1}^N w_i \mathcal{L}\left( \theta_i; \mathcal{D}_i^{\text{clean}} \right) + \lambda \sum_{i=1}^N \sum_{j\in \mathcal{N}(i)} \|\theta_i – \theta_j\|^2 $$
where $\theta_i$ is the local policy parameter, $w_i$ is a trust weight, $\mathcal{D}_i^{\text{clean}}$ is clean local data, and the second term enforces consistency among neighboring nodes to mitigate Byzantine attacks. The architecture faces three core challenges: cross-layer latency (which may exceed the reaction time required for fast-moving drones), fault tolerance (a single compromised layer can cascade failures), and dynamic environment adaptation (the architecture must reconfigure on-the-fly as drones join or leave the swarm). Addressing these requires advances in 6G ultra-reliable low-latency communication, self-healing network protocols, and online meta-learning.
Future Research Directions
As drone technology continues to evolve toward swarm intelligence, micro-size platforms, and wide-area operations, the following five research directions are critical for securing DRL-based autonomy.
Multi-Agent Collaborative Attack and Defense. Current attack and defense techniques are primarily designed for single-agent scenarios. In a swarm, adversaries can launch coordinated attacks that exploit the interdependence of multiple drones. Future work should develop game-theoretic frameworks where each drone’s policy accounts for the actions of others. The challenge is to design scalable algorithms that can handle hundreds or thousands of agents while maintaining real-time performance.
Lightweight Algorithm and Hardware Co-Design. Many advanced defenses are too computationally heavy for micro-drones. Model compression (pruning, quantization), knowledge distillation, and edge-cloud collaboration are promising pathways. For example, a DDPG-based path planner augmented with an artificial potential field (as demonstrated in prior work) can be further optimized for low-precision arithmetic. The goal is to fit a robust DRL agent within a few megabytes of memory and a few watts of power.
Cross-Scenario Adaptive Defense. Drone missions vary widely (surveillance, strike, logistics, search-and-rescue). A defense that works well in one scenario may fail in another due to different sensor modalities, environmental conditions, or threat models. Transfer learning and meta-learning can enable a single defense framework to adapt rapidly across scenarios. 6G connectivity will allow model updates to be pushed to drones in real-time as mission context changes.
Joint Optimization of Security, Mission, and Platform. The trade-off between defense and mission performance must be formalized. We propose a multi-objective optimization that simultaneously maximizes mission effectiveness, minimizes vulnerability, and respects platform constraints (energy, compute, weight). For instance, path planning under attack may sacrifice optimality for safety; the optimal trade-off can be learned via constrained MDPs.
Integration with Emerging Technologies. Quantum communication can provide theoretically unbreakable encryption for drone-to-ground links. Generative adversarial networks (GANs) can create realistic adversarial examples for robust training. Digital twins of drone swarms can simulate attack-defense scenarios at scale without risking physical hardware. These technologies promise to radically improve the security posture of DRL-based drone technology.
Conclusion
In this survey, we have systematically examined the security challenges of deep reinforcement learning algorithms in drone technology. From training-phase backdoor attacks to testing-phase policy attacks, we documented the evolution of adversarial techniques and the corresponding defense mechanisms. We highlighted that while significant progress has been made, practical deployment is hindered by limitations in scenario fidelity, computational efficiency, and the balance between security and mission effectiveness. The proposed federated collaborative architecture offers a systematic approach to enhance resilience in drone swarms, but it requires further research in low-latency communication and fault tolerance. Future work should focus on multi-agent coordination, lightweight algorithm design, adaptive defenses, and the integration of quantum and digital twin technologies. By addressing these challenges, we can ensure that drone technology remains robust and secure in the face of evolving adversarial threats.