Object detection plays a pivotal role in the perception systems of China UAV drone platforms, where the ability to accurately identify and localize small, densely packed targets under complex backgrounds is essential for tasks such as traffic monitoring, disaster assessment, and precision agriculture. In this work, we present a comprehensive improvement upon the real-time detection transformer (RT-DETR) framework, introducing a novel algorithm named MSDA-DETR (Multi-Scale Drone Adaptive DETR). Our approach is specifically tailored to address the unique challenges encountered in China UAV drone aerial imagery: severe scale variation, motion blur, low-resolution textures, and occlusions. Through systematic enhancements to the backbone, feature pyramid, and loss function, we achieve substantial gains in detection accuracy while simultaneously reducing model complexity. Extensive experiments on the VisDrone2019 dataset demonstrate that MSDA-DETR outperforms state-of-the-art detectors in both precision and efficiency, making it a highly promising solution for real-world China UAV drone deployments.
1. Introduction
The rapid proliferation of China UAV drone technology has opened new frontiers in visual surveillance, environmental monitoring, and emergency response. However, the unique perspective captured by drones introduces a set of formidable challenges for conventional object detectors. Key obstacles include: (i) the prevalence of small objects that occupy only a few pixels in high-resolution images; (ii) extreme variations in object scale caused by changes in flight altitude; (iii) frequent occlusions among densely packed targets; and (iv) image degradation due to motion blur and adverse lighting. These factors collectively degrade the performance of both CNN-based detectors like YOLO and early Transformer-based detectors like DETR.
RT-DETR, proposed by Baidu, achieved real-time speed by designing an efficient hybrid encoder and IoU-aware query selection, surpassing YOLO series in latency-accuracy trade-off. Despite its success on generic datasets, RT-DETR struggles with China UAV drone images because its backbone lacks frequency-domain global perception, its feature pyramid inadequately enhances small objects, and its GIoU loss is insensitive to fine-grained geometry and hard samples. To overcome these limitations, we propose three key innovations:
First, we design a lightweight yet powerful backbone by replacing standard residual blocks with a C2f-EncoderBlock module that integrates spatial gating and frequency-aware MLP (Fre-MLP). This module leverages Fast Fourier Transform to capture global illumination and texture information from the frequency domain, effectively mitigating feature extraction difficulties in low-resolution and blurred images while significantly cutting parameters.
Second, we introduce a Multi-Scale Small Object Enhancement Pyramid (MSOEP) that constructs a dedicated lossless downsampling branch using space-to-depth convolution (SPDConv) and a CSPOK feature fusion block. This structure feeds rich shallow details into the feature pyramid, dramatically improving the recall of tiny targets without adding detection head overhead.
Third, we design a Focaler-MPDIoU loss function that combines the high-precision geometric constraint of MPDIoU (minimizing corner point distances) with the dynamic hard-sample focusing of Focaler-IoU. This loss adaptively emphasizes difficult examples (e.g., occluded, minute objects) during training, leading to more accurate bounding box regression for densely packed scenes.
Our contributions are validated through extensive ablation studies and comparisons with leading detectors on the VisDrone2019 benchmark. The results show that MSDA-DETR achieves a 3.9% improvement in mAP50 over the baseline RT-DETR-R18 while reducing parameters by 24.75%. Notably, our model maintains efficient inference, making it well-suited for edge deployment on China UAV drone platforms.
2. Related Work
Object detection for aerial images has been a hot research topic, especially for China UAV drone applications. Traditional two-stage detectors like Faster R-CNN offer high accuracy but are too slow for real-time drone tasks. Single-stage detectors like YOLOv5, YOLOv8, and YOLOv10 have been widely adopted due to their speed, yet they rely on NMS post-processing which is inefficient for dense targets. Transformer-based detectors such as DETR and Deformable DETR eliminate NMS but suffer from slow convergence and high computational cost. RT-DETR strikes a balance with real-time inference, but its generic design leaves room for improvement in drone-specific scenarios.
Recent works have attempted to adapt detectors for aerial imagery. For instance, LDF-YOLO uses residual gating and local feature enhancement to strengthen feature extraction. HR-MSCA combines resolution enhancement with multi-scale convolutional attention. DCA-YOLO employs dual-branch dynamic channel attention. These CNN-based methods show progress but are limited by local receptive fields. On the Transformer side, MSM-DETR adds multi-kernel parallel fusion, though at the cost of increased parameters. Drone-DETR incorporates parallel patch-wise attention but still lacks frequency-domain global awareness. Our MSDA-DETR differs by introducing frequency-spatial synergistic perception in the backbone and a dual-focused loss function for hard sample mining.
Crucially, none of these existing methods simultaneously address the three bottlenecks we identify: (1) the absence of frequency-domain information to handle motion blur, (2) the insufficiency of shallow feature preservation for tiny objects, and (3) the lack of adaptive weighting for difficult regression samples. Our work fills these gaps for China UAV drone detection.
3. Method
3.1 Overall Architecture
The proposed MSDA-DETR inherits the encoder-decoder structure of RT-DETR but with substantial modifications. The input image is first processed by a lightweight CSPNet backbone where we replace all Bottleneck blocks with our C2f-EncoderBlock modules. This backbone outputs multi-scale feature maps P2, P3, P4, and P5. These features are fed into an improved hybrid encoder: the AIFI module remains on P5 for global semantic interaction, while our newly designed MSOEP module fuses P2, P3, and P4 layers to produce enhanced feature sets {F1, F2, F3}. Finally, the decoder with IoU-aware query selection generates predictions, and during training we apply the Focaler-MPDIoU loss instead of GIoU. The overall pipeline is illustrated in the figure below.
3.2 C2f-EncoderBlock: Frequency-Spatial Synergistic Backbone
To enhance feature extraction under motion blur and low-resolution conditions common in China UAV drone images, we design the C2f-EncoderBlock module. It consists of two cascaded stages: Spatial Attention Module (SpAM) and Frequency MLP (Fre-MLP). The SpAM stage applies layer normalization, mixed 1×1 and 3×3 convolutions, a simple gate (SG) for adaptive feature selection, and simplified channel attention (SCA). The Fre-MLP stage converts features to the frequency domain via FFT, decomposes into magnitude and phase, enhances the magnitude using MLP while preserving phase, then applies IFFT to produce a spatial attention mask. The output is combined with the SpAM output via a learnable scaling factor γ. Mathematically, the process is:
$$
\begin{aligned}
X_{feat} &= \text{Conv}_{3\times3}(\text{Conv}_{1\times1}(\text{LN}(X_{in}))) \\
X_{gate} &= X_{feat}[:, :C/2] \odot X_{feat}[:, C/2:] \\
X_{att} &= \text{SCA}(X_{gate}) \\
Y_{in} &= X_{in} + \text{Conv}_{1\times1}(X_{att}) \\
\mathbf{A}, \mathbf{P} &= \text{FFT}(Y_{in}) \\
\mathbf{A’} &= \text{MLP}(\mathbf{A}), \quad \mathbf{P’} = \mathbf{P} \\
Y_{freq} &= \text{IFFT}(\mathbf{A’} \odot e^{j\mathbf{P}}) \\
Y_{out} &= Y_{in} + \gamma \cdot (Y_{freq} \odot Y_{in})
\end{aligned}
$$
By integrating this module into the C2f structure, we form C2f-EncoderBlock. Experiments show that replacing original Bottleneck in C2f yields a 2.6% mAP50 improvement while reducing parameters by 30%.
3.3 Multi-Scale Small Object Enhancement Pyramid (MSOEP)
Standard feature pyramids lose crucial high-resolution information due to pooling or strided convolutions, leading to missed detections of tiny objects. Our MSOEP introduces a lossless downsampling path using Spatial-to-Depth Convolution (SPDConv) to transfer P2 details to the P3 level without information loss. Specifically, SPDConv reorganizes the spatial dimensions into channels, preserving every activation. Then, a CSPOmniKernel (CSPOK) module fuses features from P2, P3, and upsampled P4.
The SPDConv operation is defined as:
$$
Y_{SPD} = \text{Conv}(\text{S2D}(X_{in}))
$$
where S2D(·) reshapes an S×S feature map of C channels into (S/2)×(S/2) with 4C channels. The CSPOK module splits the input into two branches: a shortcut main branch preserving most channels, and a small secondary branch processed by the OmniKernel unit. OmniKernel includes local (1×1 conv), large kernel (1×31 and 31×1 asymmetric strip convolutions), and global branches. The global branch features a Dual-domain Channel Attention Module (DCAM) and a Frequency-based Spatial Attention Module (FSAM) to capture both channel and spatial dependencies in the frequency domain. The fusion is:
$$
X_{sub}, X_{main} = \text{Split}(\text{Conv}_{1\times1}(X_{in})) \\
Y_{CSPOK} = \text{Conv}_{1\times1}(\text{Concat}(\text{OK}(X_{sub}), X_{main}))
$$
Ablation studies confirm that both DCAM and FSAM contribute positively, with the full MSOEP boosting mAP50 by 1.9% over baseline.
3.4 Focaler-MPDIoU Loss for Hard Sample Focused Regression
The original RT-DETR uses GIoU loss, which is insensitive to corner deviations and treats easy and hard samples equally. For China UAV drone images with densely overlapping small objects, this leads to suboptimal localization. We propose Focaler-MPDIoU, combining the geometric precision of MPDIoU with the dynamic focusing of Focaler-IoU.
MPDIou measures the distance between two key corner points of the predicted and ground-truth boxes:
$$
d_1^2 = (x_1^{pred} – x_1^{gt})^2 + (y_1^{pred} – y_1^{gt})^2 \\
d_2^2 = (x_2^{pred} – x_2^{gt})^2 + (y_2^{pred} – y_2^{gt})^2 \\
MPDIoU = \text{IoU} – \frac{d_1^2 + d_2^2}{w^2 + h^2}
$$
Then we apply the Focaler mechanism to reweight the IoU portion:
$$
\text{IoU}_{focaler} =
\begin{cases}
0, & \text{IoU} < d \\
\frac{\text{IoU} – d}{u – d}, & d \leq \text{IoU} \leq u \\
1, & \text{IoU} > u
\end{cases}
$$
The final loss is:
$$
\text{Loss}_{Focaler-MPDIoU} = 1 – \text{MPDIoU} + \text{IoU}_{focaler} – \text{IoU}
$$
where [d, u] is an interval set to [0, 1] in our experiments. Table 4 in the experiments section shows that this loss outperforms GIoU, DIoU, CIoU, EIoU, and standalone MPDIoU or Focaler-IoU, achieving 0.8% higher mAP50 than baseline without any parameter increase.
4. Experiments
4.1 Setup
All experiments are performed on the VisDrone2019 dataset, which includes 6,471 training, 548 validation, and 1,610 test images captured from various China UAV drone platforms. The dataset contains 10 categories: pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus, and motor. Images exhibit high variability in scale, density, and illumination. We use the same training hyperparameters for fair comparison: 300 epochs, batch size 4, input size 640×640, initial learning rate 0.0001, AdamW optimizer with weight decay 0.0001. All models are trained from scratch without pretrained weights on a single NVIDIA RTX 5070Ti GPU.
4.2 Ablation Studies
We conduct systematic ablation experiments to isolate the contribution of each proposed module. The baseline is RT-DETR with ResNet-18 backbone (RT-DETR-R18). Results are shown in Table 1.
| Exp | C2f_EncoderBlock | MSOEP | Focaler-MPDIoU | P (%) | R (%) | Params (M) | GFLOPs | mAP50 (%) | mAP50:95 (%) |
|---|---|---|---|---|---|---|---|---|---|
| 1 | × | × | × | 61.2 | 45.8 | 19.8 | 57.0 | 46.8 | 29.6 |
| 2 | √ | × | × | 62.1 | 46.9 | 13.8 | 44.9 | 49.4 | 30.5 |
| 3 | × | √ | × | 62.7 | 47.6 | 20.5 | 65.5 | 48.7 | 29.9 |
| 4 | × | × | √ | 61.9 | 46.7 | 19.8 | 57.0 | 47.6 | 30.0 |
| 5 | √ | √ | × | 63.5 | 48.6 | 14.9 | 60.3 | 50.0 | 30.7 |
| 6 | √ | √ | √ | 64.5 | 49.2 | 14.9 | 60.3 | 50.7 | 31.6 |
From Table 1, the backbone replacement (Exp.2 vs Exp.1) improves mAP50 by 2.6% while cutting parameters by 30%, confirming the effectiveness of C2f-EncoderBlock. Adding MSOEP (Exp.3) yields a 1.9% boost, albeit with increased GFLOPs. The loss function alone (Exp.4) adds a 0.8% gain without extra computation. When all three components are combined (Exp.6), we achieve 50.7% mAP50, which is 3.9% higher than baseline, with 24.75% fewer parameters (14.9M vs 19.8M). The slight increase in GFLOPs (from 57.0 to 60.3) is a worthwhile trade-off for the substantial accuracy gains, especially for China UAV drone applications where storage efficiency is more critical than FLOPs.
We further ablate the internal attention of CSPOK (Table 2). The full global branch (DCAM + FSAM) outperforms partial configurations, demonstrating the synergy between channel and spatial frequency-domain attention.
| Exp | DCAM | FSAM | Params (M) | GFLOPs | mAP50 (%) | mAP50:95 (%) |
|---|---|---|---|---|---|---|
| 1 | × | × | 14.81 | 59.9 | 50.0 | 31.1 |
| 2 | √ | × | 14.85 | 60.1 | 50.4 | 31.3 |
| 3 | × | √ | 14.86 | 60.2 | 50.3 | 31.2 |
| 4 | √ | √ | 14.90 | 60.3 | 50.7 | 31.6 |
Table 3 shows the loss function comparison. Focaler-MPDIoU outperforms all competitors, confirming the advantage of combining geometric precision with hard-sample focusing.
| Loss | GIoU | DIoU | CIoU | EIoU | Focaler-IoU | MPDIoU | Focaler-MPDIoU |
|---|---|---|---|---|---|---|---|
| mAP50 (%) | 46.8 | 46.9 | 47.3 | 47.2 | 47.0 | 47.4 | 47.6 |
4.3 Kernel Size Selection for Large Branch in CSPOK
The large kernel size K in the OmniKernel unit controls the receptive field. We tested K=31, 51, 63 (Table 4). mAP50 improves steadily with larger K, due to better long-range context capture. We choose K=63 as default since the parameter increase is negligible.
| K | Params (M) | GFLOPs | mAP50 (%) | mAP50:95 (%) |
|---|---|---|---|---|
| 31 | 14.8 | 59.8 | 50.2 | 31.3 |
| 51 | 14.9 | 60.1 | 50.5 | 31.4 |
| 63 | 14.9 | 60.3 | 50.7 | 31.6 |
4.4 Comparison with State-of-the-Art Detectors
We compare MSDA-DETR with multiple mainstream detectors on the VisDrone2019 test set. Results are compiled in Table 5. Our model achieves the best trade-off between accuracy and compactness.
| Model | P (%) | R (%) | Params (M) | GFLOPs | mAP50 (%) | mAP50:95 (%) |
|---|---|---|---|---|---|---|
| Faster R-CNN | 45.1 | 33.2 | 41.1 | 206.6 | 32.3 | 17.4 |
| Swin Transformer | – | – | 34.2 | 44.5 | 35.6 | 20.6 |
| Deformable DETR | – | – | 29.0 | 196.0 | 43.1 | 27.1 |
| YOLOv5m | 50.1 | 37.5 | 21.1 | 48.3 | 36.1 | 19.1 |
| YOLOv6 | 52.8 | 39.6 | 52.0 | 161.0 | 40.3 | 24.9 |
| YOLOv7 | 53.5 | 42.9 | 37.1 | 103.4 | 41.9 | 22.7 |
| YOLOv8m | 55.2 | 43.5 | 25.7 | 78.6 | 40.0 | 24.8 |
| YOLOv10m | 53.7 | 42.5 | 15.2 | 58.8 | 43.9 | 26.8 |
| YOLOv11m | 54.0 | 43.0 | 19.9 | 67.6 | 44.2 | 27.5 |
| LDF-YOLO | 46.3 | 34.2 | – | 15.9 | 31.9 | 13.0 |
| MSM-DETR | – | – | 22.2 | 72.9 | 49.5 | 30.6 |
| Drone-DETR | 62.6 | 48.7 | 19.1 | 68.8 | 50.4 | 31.1 |
| Reference [20] | 63.2 | 47.3 | 33.8 | 52.4 | 49.4 | 30.2 |
| RT-DETR-R18 | 61.2 | 45.8 | 19.8 | 57.0 | 46.8 | 29.6 |
| MSDA-DETR (ours) | 64.5 | 49.2 | 14.9 | 60.3 | 50.7 | 31.6 |
As shown, MSDA-DETR achieves the highest mAP50 (50.7%) among all compared methods while having the second smallest parameter count (14.9M). It outperforms Drone-DETR (50.4%, 19.1M) with 22% fewer parameters and higher precision. Compared to YOLOv10m (43.9%, 15.2M), our model improves mAP50 by 6.8% at comparable parameter cost. This demonstrates the superiority of our frequency-spatial backbone and hard-sample-focused loss for China UAV drone detection.
4.5 Visualization and Qualitative Analysis
To further demonstrate the effectiveness of MSDA-DETR, we visualize detection results on challenging samples from the VisDrone2019 test set. Our model correctly detects tiny pedestrians and vehicles that the baseline RT-DETR-R18 misses, and significantly reduces false positives in cluttered backgrounds. For example, in a dense parking lot scene, MSDA-DETR identifies 23 targets while baseline finds only 16. In an occluded street view, our model successfully separates overlapping tricycles that baseline merges. These qualitative improvements are attributed to the enhanced shallow feature preservation and the dynamic focusing ability of the loss function.
Additionally, we generate Grad-CAM heatmaps to compare feature focus. Our MSDA-DETR produces more concentrated activation maps on target objects, whereas baseline heatmaps are diffused over background areas. This confirms that C2f-EncoderBlock and MSOEP guide the network to attend to discriminative regions, crucial for China UAV drone imagery.
5. Conclusion
In this paper, we present MSDA-DETR, an enhanced real-time detection transformer tailored for China UAV drone applications. By introducing a frequency-spatial synergistic backbone (C2f-EncoderBlock), a multi-scale small object enhancement pyramid (MSOEP), and a hard-sample-focused loss (Focaler-MPDIoU), we fundamentally address the challenges of motion blur, tiny object detection, and difficult sample regression in aerial imagery. Extensive experiments on VisDrone2019 demonstrate a 3.9% improvement in mAP50 over RT-DETR-R18 with a 24.75% reduction in parameters. Our model outperforms recent state-of-the-art detectors including Drone-DETR and MSM-DETR, establishing a new benchmark for efficient and accurate object detection on China UAV drone platforms.
Future work will explore multi-modal fusion (e.g., infrared + visible) to handle extreme lighting conditions and investigate model compression techniques like pruning and quantization for deployment on resource-constrained China UAV drone embedded systems. Extending the framework to video object detection by leveraging temporal coherence also presents a promising direction.