In the rapidly evolving field of drone technology, the accurate and efficient detection of unmanned aerial vehicles (UAVs) is of paramount importance for airspace security, surveillance, and autonomous navigation. Drones operating at medium to long distances often appear as small, weak targets in visual imagery, characterized by minimal pixel area, lack of distinct texture, and susceptibility to motion blur, varying illumination, and complex backgrounds. Traditional object detection methods struggle to reliably identify such small targets due to insufficient feature representation and background interference. To address these challenges in drone technology, I propose an improved detection algorithm based on the RT-DETR framework, which integrates three novel modules: a Cross-Scale Context Enhancement (CSCE) module, an Adaptive Gated Feature Fusion (AGFF) module, and an IoU-Adaptive weighted loss function. The proposed method aims to enhance multi-scale feature interaction, suppress background noise, and optimize bounding box regression specifically for small drone targets. Extensive experiments on the Drone Detection V2 and VisDrone2019 datasets demonstrate significant improvements in detection accuracy and robustness while maintaining real-time inference speed.
The core architecture of the proposed model is illustrated by the following structure, which replaces the original static feature fusion with adaptive and context-aware mechanisms. The modifications are applied to the encoder part of RT-DETR, where multi-scale features from the backbone are processed through the CSCE and AGFF modules before being fed into the transformer decoder. The IoU-Adaptive loss is employed during training to dynamically balance the contributions of classification and regression branches.
1. Proposed Method
1.1 Cross-Scale Context Enhancement (CSCE) Module
In RT-DETR, the original CCFF (CNN-based Cross-scale Feature Fusion) module uses a cascaded fusion strategy where low-level features are first fused with medium-level features and then indirectly propagated to high levels. This indirect interaction leads to the attenuation of fine-grained details from shallow layers, which are critical for small drone detection. To overcome this limitation, I design the CSCE module that establishes direct semantic interaction between low-level and high-level features via cross-scale self-attention.
Given the multi-scale features \( F_3, F_4, F_5 \) from the backbone (with decreasing spatial resolutions), I first upsample \( F_4 \) and \( F_5 \) to the same spatial size as \( F_3 \), creating aligned features \( P_3, P_4, P_5 \). The low-level feature \( F_3 \) is projected as the query \( Q \), while all aligned features serve as keys \( K \) and values \( V \). The cross-scale attention is computed as:
$$
\text{Attention}(Q, K, V) = \text{softmax}\left( \frac{Q K^T}{\sqrt{d_k}} \right) V
$$
where \( d_k \) is the dimension of the key vector. By attending to all scales, the low-level query can gather contextual information from both medium and high levels, capturing object semantics that help distinguish small drones from background clutter. The resulting context is then fused with the original low-level feature via a 1×1 convolution:
$$
F_{\text{enhanced}} = \text{Conv}_{1\times1}(\text{Concat}(F_3, \text{Context}))
$$
The CSCE module significantly improves the model’s ability to represent small drone features by directly linking fine spatial details with rich semantic cues.
1.2 Adaptive Gated Feature Fusion (AGFF) Module
In the standard feature pyramid, deep features from high-level layers contain strong semantic information but also include substantial background noise due to their large receptive fields. Directly adding these features to shallow layers can dilute the target signal. To address this, I propose the AGFF module that leverages a lightweight gating mechanism to dynamically filter the deep features before fusion.
Given the enhanced low-level feature \( F_{\text{enhanced}} \) from the CSCE module and the original high-level feature \( F_5 \), the gate is computed as:
$$
\text{gate} = \sigma(\text{MLP}(\text{GAP}(\text{Concat}(F_{\text{enhanced}}, F_5))))
$$
where GAP denotes global average pooling, MLP is a two-layer perceptron with a hidden dimension of 128, and \( \sigma \) is the sigmoid function. The gate value ranges from 0 to 1, indicating the proportion of target-relevant information in the deep feature. The final fused feature is:
$$
F_{\text{out}} = F_{\text{enhanced}} + \text{gate} \cdot F_5
$$
When the deep feature contains strong target semantics, the gate is high, allowing effective injection of semantic context; when dominated by background noise, the gate suppresses the noisy component, preserving the integrity of low-level details. This adaptive mechanism is particularly beneficial for drone technology, where small targets are often embedded in cluttered environments.
1.3 IoU-Adaptive Weighted Loss Function
The original RT-DETR uses Varifocal Loss for classification and GIoU Loss for regression, treating all matched positive samples equally. However, in drone detection, the Hungarian matching often produces positive samples with varying IoU values. Low IoU samples introduce noisy gradients that dilute the optimization signals from high-quality matches. To mitigate this, I introduce an IoU-Adaptive weighting strategy that assigns sample-specific weights to classification and regression losses.
For the classification branch, the weight for the \( i \)-th positive sample is defined as:
$$
w_{\text{cls}, i} = \max(\text{IoU}_i – \tau_{\text{cls}}, 0)
$$
where \( \tau_{\text{cls}} \) is a threshold (set to 0.5) to retain only samples with sufficient overlap. The weighted classification loss becomes:
$$
L_{\text{cls}} = \frac{1}{N_{\text{pos}}} \sum_{i=1}^{N_{\text{pos}}} w_{\text{cls}, i} \cdot L_{\text{vf}, i}
$$
For the regression branch, the weight is defined oppositely to focus on poorly localized samples:
$$
w_{\text{reg}, i} = \max(\tau_{\text{reg}} – \text{IoU}_i, 0)
$$
with \( \tau_{\text{reg}} = 0.7 \), encouraging the model to improve low-IoU predictions. The weighted regression loss is:
$$
L_{\text{reg}} = \frac{1}{N_{\text{pos}}} \sum_{i=1}^{N_{\text{pos}}} w_{\text{reg}, i} \cdot L_{\text{giou}, i}
$$
The total loss is then:
$$
L_{\text{total}} = L_{\text{cls}} + \lambda L_{\text{reg}}
$$
where \( \lambda \) is a balancing coefficient (set to 2 in experiments). This adaptive weighting effectively mitigates gradient dilution, allowing the model to focus on high-quality matches for classification and on corrections for regression, leading to more robust learning in drone technology.
2. Experiments and Results
2.1 Dataset and Settings
I evaluate the proposed method on the Drone Detection V2 dataset, which contains 5534 annotated images of drones in diverse backgrounds including urban, rural, sky, and sea scenes. The dataset covers a wide range of scales, with a majority of small objects. Additionally, I use the VisDrone2019 dataset to test cross-dataset generalization. All models are trained from scratch without pretrained weights. The training configuration is summarized in Table 1.
Table 1: Training hyperparameters.
| Parameter | Value |
|---|---|
| Epochs | 300 |
| Batch size | 16 |
| Image size | 640 |
| Workers | 4 |
| Optimizer | AdamW |
| Learning rate | 0.001 |
| Momentum | 0.9 |
| Weight decay | 0.0001 |
2.2 Comparison with State-of-the-Art Methods
I compare the proposed model with several mainstream detectors, including two-stage (Faster R-CNN), single-stage (SSD, YOLOv7, YOLOv8, YOLOv11), and transformer-based methods (Deformable-DETR, DINO, RT-DETR). All models are trained and tested under the same conditions on Drone Detection V2. Results are shown in Table 2.
Table 2: Comparison of different detection methods on Drone Detection V2.
| Model | Model Size (MB) | GFLOPs | mAP0.5 (%) | mAP0.5:0.95 (%) | FPS |
|---|---|---|---|---|---|
| SSD | 90.07 | 136.59 | 93.9 | 58.0 | 141.38 |
| Faster R-CNN | 107.90 | 470.47 | 87.8 | 44.2 | 34.50 |
| YOLOv7 | 141.89 | 52.56 | 94.6 | 55.8 | 152.00 |
| YOLOv8s | 42.60 | 28.4 | 93.9 | 59.7 | 271.89 |
| YOLOv8m | 98.81 | 78.7 | 94.8 | 60.1 | 203.23 |
| YOLOv8l | 166.67 | 82.97 | 94.7 | 60.5 | 164.26 |
| YOLOv11 | 10.01 | 3.31 | 94.3 | 59.1 | 198.75 |
| Deformable-DETR | 157.03 | 152.2 | 90.3 | 58.7 | 64.71 |
| DINO | 163.43 | 165.2 | 92.5 | 59.2 | 75.44 |
| RT-DETR-R50 | 155.96 | 67.90 | 92.9 | 60.1 | 73.40 |
| Ours | 238.83 | 122.12 | 95.7 | 61.8 | 75.75 |
The proposed method achieves the best mAP0.5 and mAP0.5:0.95 among all compared methods, outperforming the baseline RT-DETR by 2.8% and 1.7% respectively. Notably, although the model size and GFLOPs increase, the inference speed (FPS) improves from 73.40 to 75.75 due to the end-to-end architecture without NMS. This demonstrates the effectiveness of the proposed modules in balancing accuracy and efficiency within drone technology.
2.3 Ablation Study
To validate the contribution of each module, I perform ablation experiments on Drone Detection V2. The baseline is RT-DETR-R50. Modules are added incrementally, and results for mAP, AP at different scales (APs, APm, APl), parameters, and FPS are reported in Table 3.
Table 3: Ablation study results. Tick marks indicate the module is included.
| Exp | CSCE | AGFF | IoU-Adaptive | mAP0.5 (%) | mAP0.5:0.95 (%) | APs (%) | APm (%) | APl (%) | Param (MB) | FPS |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 92.9 | 60.1 | 40.8 | 60.0 | 78.5 | 42.9 | 73.6 | |||
| 2 | √ | 93.8 | 60.9 | 42.1 | 65.2 | 80.1 | 51.1 | 74.3 | ||
| 3 | √ | 93.9 | 59.8 | 41.2 | 62.1 | 78.9 | 46.9 | 74.6 | ||
| 4 | √ | 92.6 | 60.5 | 42.3 | 64.5 | 79.2 | 44.1 | 78.2 | ||
| 5 | √ | √ | 93.7 | 61.6 | 43.3 | 66.0 | 80.2 | 65.3 | 81.0 | |
| 6 | √ | √ | 93.5 | 61.3 | 43.6 | 65.7 | 78.5 | 63.6 | 73.8 | |
| 7 | √ | √ | 92.8 | 61.1 | 42.5 | 64.6 | 79.8 | 58.1 | 77.8 | |
| 8 | √ | √ | √ | 95.7 | 61.8 | 45.1 | 65.8 | 80.7 | 62.6 | 75.8 |
The ablation results confirm that each module contributes positively. The CSCE module improves APs from 40.8% to 42.1%, indicating enhanced small-target feature representation. AGFF alone increases mAP0.5 by 1.0% but slightly reduces mAP0.5:0.95, suggesting that noise suppression aids recall but not precision without the loss weighting. The IoU-Adaptive loss alone boosts APs by 1.5% and improves mAP0.5:0.95 to 60.5%, demonstrating its effect on boundary regression. The combination of all three modules yields the best overall performance: mAP0.5 = 95.7%, mAP0.5:0.95 = 61.8%, and APs = 45.1% (a 4.3% improvement over baseline). Furthermore, the inference speed increases, showing that the modules not only improve accuracy but also enable more efficient computation through better feature propagation.
2.4 Generalization Analysis
To verify the cross-dataset generalization of the proposed method, I evaluate the baseline RT-DETR and the improved model on the VisDrone2019 dataset without fine-tuning. The results are shown in Table 4.
Table 4: Generalization performance on Drone Detection V2 and VisDrone2019 datasets.
| Dataset | Metric | RT-DETR (%) | Ours (%) | Improvement (%) |
|---|---|---|---|---|
| Drone Detection V2 | mAP0.5 | 92.9 | 95.7 | +2.8 |
| mAP0.5:0.95 | 60.1 | 61.8 | +1.7 | |
| VisDrone2019 | mAP0.5 | 47.9 | 49.5 | +1.6 |
| mAP0.5:0.95 | 29.3 | 30.5 | +1.2 |
The proposed model achieves consistent improvement on both datasets, confirming its generalization capability across different drone detection scenarios. This robustness is crucial for practical deployment of drone technology in diverse environments.
2.5 Training Convergence and Visualization
I further analyze the training dynamics. The improved model converges faster than the baseline, achieving stable performance by around 200 epochs, while the baseline requires about 300 epochs. The final mAP0.5:0.95 curves show a higher plateau for the proposed method, indicating more effective learning. Qualitative results on test images demonstrate that the model accurately localizes drones even under challenging conditions such as complex backgrounds, low contrast, and small scales. The detected bounding boxes tightly fit the targets with minimal false positives. The improvements in recall and precision are visually evident.
3. Conclusion
In this work, I addressed the critical challenges in drone technology for small target detection by enhancing the RT-DETR framework with three novel components. The Cross-Scale Context Enhancement module directly connects low-level details with high-level semantics, improving feature representation for small drones. The Adaptive Gated Feature Fusion module dynamically suppresses background noise, enabling cleaner feature integration. The IoU-Adaptive weighted loss focuses training on high-quality matches and poorly localized samples, alleviating gradient dilution. Extensive experiments on the Drone Detection V2 and VisDrone2019 datasets demonstrate that the proposed method significantly improves detection accuracy, especially for small targets (APs improvement of 4.3%), while maintaining real-time inference speed. These contributions advance the state of the art in drone technology, offering a robust solution for real-world UAV detection applications. Future work will explore further lightweighting and adaptation to diverse aerial platforms.