In recent years, the rapid development of UAV drone technology has enabled widespread applications in civilian, industrial, surveying, and many other fields, providing new pathways for improving the efficiency and quality of production activities. However, the proliferation of UAV drones has also been accompanied by lagging regulatory system construction. Some unauthorized UAV drone flights not only easily cause personal privacy leakage but may also intrude into no-fly zones, triggering public safety hazards and posing a direct threat to social governance and public security. Therefore, quickly and accurately identifying low-altitude illegal UAV drones and promptly investigating safety risks has become an important research topic in the field of low-altitude security. Especially in low-altitude control scenarios of small and medium-sized cities, constrained by hardware deployment costs and computing resources, there is an urgent need for a UAV drone detection scheme that balances detection accuracy, inference efficiency, and deployment economy. With continuous breakthroughs in deep learning technology in the computer vision field, visual-based object detection methods have become the mainstream technical route for UAV drone detection due to their low cost, fast detection speed, and strong adaptability.
Our work addresses these challenges by proposing AU-YOLO, a lightweight multi-scale edge information fusion algorithm specifically designed for low-altitude UAV drone detection. The algorithm is built upon YOLOv11 and introduces two novel modules: the Multi-Scale Edge Selection (MSES) module for enhancing edge feature extraction, and the Dynamic Multi-Temporal Self-Attention (DMTSA) module for refining high-level features with temporal awareness. Extensive experiments on the Anti-UAV dataset demonstrate that AU-YOLO achieves significant improvements in Precision (P), Recall (R), and mAP50 while maintaining extremely low computational overhead.
1. Related Work
In the current field of visual UAV drone detection, the combination of deep learning and computer vision technology has become the mainstream solution. The challenge of detecting UAV drones with large size variations has been addressed by various approaches. Zeng et al. proposed a UAV drone detection network based on RetinaNet, using Res2Net as the backbone to extract UAV drone features through multiple receptive fields, and designed an innovative hybrid feature pyramid structure convolutional neural network. Magouliantis et al. followed this direction by employing super-resolution technology to double the image size before inputting it into the detection system, making the UAV drone appear clearer in the image, and combining it with the Faster R-CNN object detection algorithm to effectively improve the recall rate during detection. Lv et al. proposed a detection scheme combining background subtraction technology with an enhanced YOLOv5s model. Although this method can effectively eliminate background interference and improve detection efficiency, it is highly dependent on static scenes and is still insufficient in complex environments. Despite these breakthroughs, UAV drone detection still faces dual challenges: small UAV drone targets are difficult to accurately identify, and they are easily disturbed by complex backgrounds.
Our proposed AU-YOLO algorithm addresses these challenges by introducing edge information enhancement and dynamic temporal attention, achieving superior performance in real-world scenarios and effectively detecting small UAV drones and even swarms.
2. Proposed Method
We propose AU-YOLO, a lightweight algorithm based on YOLOv11, specifically designed for low-altitude UAV drone detection. The overall architecture consists of a backbone enhanced by the MSES module and a neck/head enhanced by the DMTSA module. The key innovations are described below.
2.1 MSES Module
UAV drone targets in images typically exhibit low pixel count and weak edge features. The original C3k2 block in YOLOv11 uses a fixed combination of 1×1 and 3×3 convolutions in its Bottleneck units, without any specialized design for the edge structure of UAV drones. This leads to a single dimension of feature extraction, causing severe loss of edge details and consequently affecting detection accuracy. To address this, we propose the MSES module, which combines a Dual Domain Selection Mechanism (DSM) with an Edge Feature Extraction (EFE) sub-module to enhance edge feature extraction capability and solve the problem of small target missed detection.
The DSM module, originally designed for image restoration, is adapted here to locate spatial regions of the UAV drone, suppress background noise, enhance high-frequency features of the UAV drone, and improve detail capture of distant UAV drones. DSM consists of two parts: the Spatial Selection Module (SSM) and the Frequency Selection Module (FSM). SSM performs average pooling and convolution along the channel dimension on the input feature \( F_0 \in \mathbb{R}^{H \times W \times C} \) to obtain a universal degradation location feature \( F_1 \in \mathbb{R}^{H \times W \times 1} \). Then \( F_1 \) is tiled to \( F_{\text{Tile}} \in \mathbb{R}^{H \times W \times C} \) to align dimensions, and depth-wise convolution is applied to \( F_0 \) to generate channel-level degradation features \( F_{\text{channel}} \). The final spatial selection feature \( F_g \in \mathbb{R}^{H \times W \times C} \) is obtained by element-wise multiplication:
$$
F_g = F_{\text{channel}} \odot F_{\text{Tile}}
$$
FSM enhances high-frequency critical information by removing low-frequency components and amplifying high-frequency ones. FSM generates low-frequency features \( F_{\text{low}} \in \mathbb{R}^{1 \times 1 \times C} \) via global average pooling, broadcasts them to \( \mathbb{R}^{H \times W \times C} \) to obtain \( F_{\text{Broad}} \), and then performs high-frequency separation and fusion to obtain the enhanced feature \( F_{\text{sh}} \).
The MSES module is not a direct replacement of the Bottleneck in C3k2, but rather a deep adaptation to the lightweight design of C3k2. Depth-wise convolutions in MSES are consistent with the grouped convolution mechanism in C3k2, both reducing redundant inter-channel computations to lower GFLOPS. MSES preserves the residual design of the original Bottleneck, ensuring continuity of feature transmission. By using only simple convolution and pooling operations, MSES maintains the original lightweight positioning.
2.2 DMTSA Module
The original C2PSA module often encounters two types of problems in UAV drone detection tasks: first, C2PSA typically uses static activation functions, but UAV drones sometimes perform flight tasks under backlight or cloudy conditions, where static activation has poor adaptability to complex scenes, leading to the algorithm’s inability to adapt to the “weak feature, strong background noise” feature distribution of UAV drones; second, the original PSA is a single-frame spatial attention mechanism, and UAV drones have fast motion characteristics. If the motion trend of the UAV drone cannot be captured, attention failure may occur. To address these problems, we propose the DMTSA dynamic temporal feature refinement module, which integrates the Dynamic Tanh activation function, a joint temporal-sparse self-attention (TSSA + InfLLM-V2), and a lightweight Mona adapter, adapting to dynamic scenarios such as complex weather and UAV drone swarms to refine features and improve detection robustness.
For the issue of poor environmental adaptability due to static activation, we introduce Dynamic Tanh (DyT), an improved dynamic activation function derived from Tanh. The formula is:
$$
\text{DyT}(x) = \tanh\left(\alpha \cdot \frac{x – \mu}{\sigma} + \beta\right)
$$
where \( x \) is the input feature tensor; \( \alpha \) and \( \beta \) are learnable parameters controlling scaling and shifting; \( \mu \) and \( \sigma \) are the mean and standard deviation of the input features; \( \tanh(\cdot) \) is the nonlinear activation function. For weak response features under cloudy or backlight conditions, DyT can dynamically amplify them. We introduce this function into the main path of each residual structure to achieve dynamic adaptation of activation characteristics, partially solving the poor adaptability of static activation in complex scenes.
To address the second problem, we introduce a joint module called STSSA, which combines Mona (Multi-cognitive Visual Adapter), TSSA (Token Statistics Self-Attention), and InfLLM-V2 (zero-parameter sparse attention). TSSA completely abandons the traditional Query-Key-Value calculation of self-attention. Instead, for each attention head, TSSA computes the second-order moment statistics of tokens within each group, using the statistics to represent the feature distribution strength of the group, rather than the similarity between tokens. This avoids the complexity of \( \mathcal{O}(n^2) \). After that, subspace projection and diagonal weighting are used to achieve precise screening of features within the group. However, this alone still does not fully exploit the local relevance of temporal features, and the statistical calculation within groups still relies on a global static mode. To address this, we incorporate the “sparse screening” concept from InfLLM-V2 to optimize the statistical calculation logic, further enhancing the targeting and efficiency advantages of temporal features. During the second-order moment statistics calculation phase of TSSA grouping, we introduce a parameter-free adaptive threshold based on feature similarity. Each token within a group is sparsely screened, and only tokens with similarity to the UAV drone target features above the threshold participate in the second-order moment statistics. The remaining redundant tokens undergo simple mean pooling without participating in complex statistical calculations. This reduces the original group statistics complexity from \( \mathcal{O}(n) \) to \( \mathcal{O}(k) \), where \( k \) is the number of effective feature tokens, and enables subsequent subspace projection and diagonal weighting to focus more precisely on the motion trend features of the UAV drone, avoiding the problem of target features being obscured by background due to global static statistics.
Mona is a lightweight adapter for visual tasks that dynamically calibrates input features through scaling layer normalization to compensate for the lack of local differentiated details that TSSA, focusing on global common features, may miss. The calibration formula is:
$$
x_{\text{norm}} = s_1 \cdot \| x_0 \|_{\text{LN}} + s_2 \cdot x_0
$$
where \( x_0 \) is the original feature, \( \| \cdot \|_{\text{LN}} \) denotes layer normalization, and \( s_1, s_2 \) are learnable weights. After calibration, Mona uses multi-scale depthwise separable convolutions for detail extraction, followed by feature aggregation and upscaling.
The DMTSA module seamlessly integrates with the backbone network. Its channel dimensions remain consistent with those of C2PSA, requiring no additional adjustments. Due to the linear complexity of TSSA, the computational overhead of Dynamic Tanh and Mona is effectively offset, ensuring that overall GFLOPS remains low. The output features of DMTSA are fully compatible with subsequent network feature dimensions, guaranteeing smooth feature fusion.
3. Experiments and Analysis
3.1 Experimental Setup
All experiments were conducted on a single NVIDIA RTX 4060 GPU with Windows 11, Python 3.10, and CUDA 11.8. Table 1 summarizes the key training hyperparameters.
| Parameter | Value |
|---|---|
| Image size | 640 |
| Epochs | 400 |
| Optimizer | SGD |
| Momentum | 0.937 |
3.2 Dataset
We used the Anti-UAV-Detection dataset, collected by the laboratory of Dalian University of Technology. The dataset contains 5200 training images, 2600 validation images, and 2200 test images. These images include various challenging scenarios such as small UAV drone targets, complex backgrounds, and different lighting conditions.
3.3 Evaluation Metrics
We employed Precision (P), Recall (R), mean Average Precision at IoU 0.5 (mAP50), number of parameters, and computational cost in GFLOPS. The formulas are:
$$
P = \frac{TP}{TP + FP}, \quad R = \frac{TP}{TP + FN}
$$
$$
\text{mAP50} = \frac{1}{C} \sum_{k=1}^{C} \text{AP}_{0.5}^{(k)}
$$
3.4 Ablation Studies
To evaluate the contribution of each component, we conducted ablation experiments on the Anti-UAV dataset. Table 2 shows the ablation study for the DMTSA module.
| Group | DyT | Mona | STSSA | P (%) | R (%) | mAP50 (%) | GFLOPS | Params (M) |
|---|---|---|---|---|---|---|---|---|
| 1 | 92.36 | 75.74 | 84.75 | 6.3 | 2.582 | |||
| 2 | ★ | 92.19 | 75.96 | 84.91 | 6.4 | 2.590 | ||
| 3 | ★ | ★ | 92.44 | 76.33 | 85.04 | 6.5 | 2.643 | |
| 4 | ★ | ★ | ★ | 92.75 | 76.62 | 85.19 | 6.6 | 2.652 |
From Table 2, the baseline YOLOv11n achieves 84.75% mAP50. Adding Dynamic Tanh (DyT) increases mAP50 to 84.91% with a slight computation increase. Incorporating Mona further improves mAP50 to 85.04%. The full DMTSA (with STSSA) reaches 85.19% mAP50, demonstrating that each component contributes to better UAV drone detection.
Table 3 presents the ablation for both MSES and DMTSA modules.
| Group | MSES | DMTSA | P (%) | R (%) | mAP50 (%) | GFLOPS | Params (M) |
|---|---|---|---|---|---|---|---|
| 1 | 92.36 | 75.74 | 84.75 | 6.3 | 2.582 | ||
| 2 | ★ | 92.93 | 76.99 | 84.69 | 6.5 | 2.573 | |
| 3 | ★ | 92.75 | 76.62 | 85.19 | 6.6 | 2.652 | |
| 4 | ★ | ★ | 93.26 | 76.84 | 85.83 | 6.8 | 2.661 |
The combined AU-YOLO (with both MSES and DMTSA) achieves the best mAP50 of 85.83%, with only 0.5 GFLOPS and 0.079M parameter increase over the baseline. This validates that the two modules are complementary and effectively enhance UAV drone detection performance.
3.5 Complex Scenario Testing
To verify performance under realistic challenging conditions, we selected two subsets from the test set: 300 images of UAV drone swarms (multiple UAV drones close together) and 200 images of complex weather (snow, low illumination). Results are shown in Table 4.
| Model | Scenario | P (%) | R (%) | mAP50 (%) |
|---|---|---|---|---|
| YOLOv11n | UAV swarm | 92.16 | 75.48 | 84.56 |
| YOLOv11n | Complex weather | 89.23 | 70.49 | 76.65 |
| AU-YOLO | UAV swarm | 92.92 | 76.32 | 85.68 |
| AU-YOLO | Complex weather | 92.57 | 69.47 | 77.47 |
AU-YOLO significantly outperforms YOLOv11n in UAV drone swarm scenarios due to the MSES module’s improved edge discrimination, and also shows better robustness in complex weather owing to the dynamic activation of DMTSA.
3.6 Comparison with State-of-the-Art
We compared AU-YOLO with several lightweight object detectors: Rtmdet-tiny, SSD-Mobilenetv2, Mamba-YOLO, Hyper-YOLO, and YOLO-world. All models were trained and tested under the same settings. Results are in Table 5.
| Model | P (%) | R (%) | mAP50 (%) | GFLOPS | Params (M) |
|---|---|---|---|---|---|
| Rtmdet-tiny | 92.80 | 76.68 | 85.03 | 8.0 | 4.876 |
| SSD-Mobilenetv2 | 90.12 | 71.13 | 81.81 | 6.0 | 3.542 |
| Mamba-YOLO | 93.10 | 77.22 | 85.53 | 12.3 | 5.662 |
| Hyper-YOLO | 93.28 | 78.32 | 85.63 | 9.5 | 3.620 |
| YOLO-world | 91.54 | 76.38 | 83.97 | 12.8 | 4.047 |
| AU-YOLO | 93.26 | 76.84 | 85.83 | 6.8 | 2.661 |
AU-YOLO achieves the highest mAP50 (85.83%) while maintaining the lowest parameter count (2.661M) and competitive GFLOPS (6.8). This demonstrates its excellent balance between accuracy and efficiency for UAV drone detection.
3.7 Deployment on Embedded Platform
To validate real-world deployability, we applied LAMP pruning and CWD knowledge distillation to lightweight the model. The results are shown in Table 6.
| Model | P (%) | R (%) | mAP50 (%) | GFLOPS | Params (M) |
|---|---|---|---|---|---|
| Base (AU-YOLO) | 92.36 | 75.74 | 84.75 | 6.3 | 2.582 |
| After Prune | 91.58 | 74.59 | 83.58 | 2.9 | 1.244 |
| After Distill | 92.06 | 75.15 | 84.53 | 2.9 | 1.244 |
After pruning and distillation, the model size reduces by 51% (from 2.582M to 1.244M) and GFLOPS drop from 6.3 to 2.9, while mAP50 only decreases by 0.22% (from 84.75% to 84.53%). We then converted the distilled model to ONNX and tested it on a Jetson Orin NX embedded platform. The results are in Table 7.
| Platform | P (%) | R (%) | mAP50 (%) | GFLOPS | Params (M) | FPS |
|---|---|---|---|---|---|---|
| RTX 4060 | 92.06 | 75.15 | 84.53 | 2.9 | 1.244 | 76 |
| Jetson Orin NX | 91.12 | 74.44 | 83.68 | 2.9 | 1.244 | 41 |
On the Jetson Orin NX, AU-YOLO achieves 41 FPS, fully satisfying real-time detection requirements, with mAP50 of 83.68%. This demonstrates its practical value for low-altitude no-fly zone surveillance applications where cost-effective embedded deployment is crucial.
4. Conclusion
We proposed AU-YOLO, an improved model based on YOLOv11 specifically optimized for low-altitude UAV drone detection. By introducing the MSES edge feature enhancement module and the DMTSA dynamic temporal feature refinement module, we significantly improved detection accuracy and robustness in complex scenarios while maintaining model lightweightness. Ablation studies confirmed that each component contributes positively and that the modules work well together. Comparative experiments showed that AU-YOLO outperforms other lightweight detectors. Embedded deployment tests further validated its practical engineering value, achieving 41 FPS on Jetson Orin NX with minimal accuracy loss.
In summary, our proposed algorithm excels in low-altitude UAV drone detection tasks, effectively improving detection precision while maintaining lightweight characteristics, making it suitable for resource-constrained deployment and applications. Future work will focus on further optimizing the model for dynamic video sequences, reducing computational consumption, and expanding its application potential in more real-world security scenarios.