In recent years, China UAVs have been widely deployed in traffic monitoring, disaster rescue, smart city construction, and agricultural surveillance. However, the detection of small objects in aerial images captured by China UAVs remains a critical challenge due to sparse features, extreme scale variation, and complex backgrounds. Existing methods suffer from either high computational cost or insufficient accuracy for tiny targets. To address these issues, we propose a novel algorithm named ZY-DETR (Zoomed-Yield Detection Transformer) based on the improved RT-DETR framework. Our approach introduces three core components: a GradNet heterogeneous backbone, an IST-Fusion cross-scale module, and a FineGrained-Detect head. Extensive experiments on VisDrone2019 and DOTA datasets demonstrate that ZY-DETR achieves competitive accuracy while maintaining real-time inference speed, making it highly suitable for China UAV deployment.
1. Introduction
China UAV remote sensing technology has become indispensable for geospatial information acquisition. The ability to detect small objects such as pedestrians, vehicles, and debris in aerial images is crucial for effective monitoring and rescue operations. However, the detection performance of general-purpose models like RT-DETR is often degraded when applied to China UAV imagery due to three main factors: (1) small objects occupy only a few pixels and their features are easily lost during down-sampling; (2) multi-scale feature fusion in existing transformers has quadratic complexity \(O(n^2)\) which is inefficient for high-resolution remote sensing images; (3) traditional detection heads are not optimized for the extreme scale variations typical in China UAV scenes. In this paper, we propose ZY-DETR, a customized solution that balances accuracy, speed, and model size for China UAV edge devices. The main contributions are:
- We design GradNet, a heterogeneous backbone that uses C2f modules for shallow detail preservation and C2f-CGSA modules (combining convolutional gated linear unit and single-head self-attention) for deep lightweight feature refinement, reducing parameters by 31.8% compared to the baseline.
- We propose Token Statistical Self-Attention (TSSA) to replace multi-head self-attention in the neck network, achieving linear complexity \(O(n)\) while maintaining or improving feature alignment quality.
- We construct IST-Fusion, an inter-scale fusion module that leverages TSSA and adaptive information fusion to compress channel redundancy and enable efficient multi-scale aggregation.
- We develop FineGrained-Detect, a remote-sensing-specific detection head that directly connects high-resolution shallow features without destructive down-sampling, and incorporates lightweight fusion units with scale-specific detection branches.
Under unified experimental conditions, ZY-DETR achieves 23.5% AP@[0.5:0.95] on VisDrone2019 test set, improving by 3.2% over RT-DETR baseline, with inference speed of 59.31 FPS and only 15.46M parameters. The proposed algorithm effectively solves the problems of missed detection of small objects and low efficiency of multi-scale fusion in China UAV remote sensing images.
2. Related Work
Object detection in China UAV remote sensing has evolved along three main lines: CNN-based detectors, Transformer-based detectors, and multi-scale feature fusion networks. CNN-based methods like YOLO series (YOLOv8, v10, v11, v12) achieve high inference speed but rely on non-maximum suppression and suffer from insufficient shallow feature utilization. Transformer-based detectors such as DETR and its variants (Deformable DETR, Conditional DETR, DINO) remove the need for NMS but often have slow convergence and high computational cost. RT-DETR (Real-Time Detection Transformer) was the first to balance speed and accuracy, yet its multi-scale fusion is not optimized for remote sensing images. Multi-scale feature fusion networks like FPN, PANet, BiFPN, and EFPN improve small object detection by integrating features from different scales, but they typically follow an “upsample + downsample” logic that causes irreversible detail loss. More importantly, most existing works do not target the specific challenges of China UAV scenarios: extreme scale variation, high background complexity, and stringent resource constraints. Our ZY-DETR addresses these gaps by introducing a heterogeneous backbone, linear-complexity cross-scale fusion, and a remote-sensing-oriented detection head.
3. Methodology
The overall architecture of ZY-DETR is illustrated in Figure (insert image). The framework consists of three main components: GradNet backbone, IST-Fusion neck, and FineGrained-Detect head, followed by a transformer decoder. The input is a 640×640 China UAV image, and the output includes bounding box coordinates and class probabilities. The entire model has 15.46M parameters and achieves 59.31 FPS, making it suitable for real-time China UAV applications.
3.1 GradNet Heterogeneous Backbone
GradNet adopts a four-level hierarchical residual structure (C2 to C5) with 32× downsampling. The key innovation is heterogeneous design: shallow layers (C2/C3) use C2f modules to preserve fine-grained details (edges, textures) of small objects, while deep layers (C4/C5) use C2f-CGSA modules for lightweight local-to-global feature refinement. The initial embedding stage uses two consecutive 3×3 convolutions (stride 1 and 2) without pooling to avoid detail loss.
- C2f module: Consists of 1×1 convolution for channel reduction, feature splitting, dual-branch bottleneck with residual shortcuts, concatenation, and 1×1 convolution for channel recovery. It efficiently retains small object details at low cost.
- C2f-CGSA module: After the C2f module, we sequentially apply Convolutional Gated Linear Unit (CGLU) and Single-Head Self-Attention (SHSA). CGLU uses 3×3 depthwise convolution to extract local contour features and dynamic channel gating to suppress redundancy. SHSA then models long-range spatial dependencies. The multi-head attention is replaced by single-head to reduce computational load.
The total parameter count of GradNet is 13.55M, a 31.8% reduction compared to RT-DETR’s backbone (19.88M), while the computational cost is 50 GFLOPs (12.2% lower). As shown in Table 1, the backbone effectively balances lightweight design and feature representation.
| Backbone | Params (M) | GFLOPs | AP (%) | AP_s (%) | AP_m (%) | AP_l (%) |
|---|---|---|---|---|---|---|
| RT-DETR (original) | 19.88 | 57.0 | 20.3 | 11.2 | 30.0 | 35.2 |
| GradNet (ours) | 13.55 | 50.0 | 20.8 | 11.7 | 30.4 | 38.7 |
3.2 IST-Fusion Cross-Scale Fusion Module
The IST-Fusion module aims to align multi-scale features with linear complexity. It replaces the original multi-head self-attention (MHSA) in the neck with our proposed Token Statistical Self-Attention (TSSA), and incorporates an Adaptive Information Fusion (AIFI) unit to compress channel redundancy. The overall process consists of three stages:
- Stage 1: Adaptive Information Fusion (AIFI) – Applying global average pooling to C5 features, then a lightweight MLP (1×1 conv + ReLU + 1×1 conv) and softmax to generate channel-wise weights. This suppresses redundant channels while retaining essential semantic information.
- Stage 2: Token Statistical Self-Attention (TSSA) – Flatten features into tokens, project to lower dimension (p=64) via a linear projection \( \mathbf{T}_P = \mathbf{T} \cdot \mathbf{W}_P \). Compute mean and variance (second-order statistics) over the token set:
$$ \boldsymbol{\mu} = \frac{1}{n}\sum_{i=1}^{n} \mathbf{T}_P[i] ,\quad \boldsymbol{\sigma}^2 = \frac{1}{n}\sum_{i=1}^{n} (\mathbf{T}_P[i] – \boldsymbol{\mu})^2 $$
Concatenate them into a compact descriptor \( \mathbf{S} \in \mathbb{R}^{1\times 2p} \). Generate attention weights via a linear mapping + GELU + softmax:
$$ \mathbf{W}_{\text{TSSA}} = \text{Softmax}(\text{GELU}(\mathbf{S} \cdot \mathbf{W}_A)) $$
where \( \mathbf{W}_A \in \mathbb{R}^{2p \times n} \). Then apply element-wise multiplication to reweight tokens. Finally, map back to original channel dimension. The complexity of TSSA is \( O(2Cpn + Bnp) \), which is linear in \( n \) since \( p \) is a small constant (64). - Stage 3: Multi-scale cross-layer fusion – The refined C5 features are upsampled to match the resolutions of C4, C3, C2 (by factors of 2, 4, 8). After 1×1 convolution channel alignment, the features are fused through RepBlock (lightweight residual modules with two 3×3 conv-ReLU units) to produce the final fusion features \( \mathbf{X}_{\text{fusion}}^2, \mathbf{X}_{\text{fusion}}^3, \mathbf{X}_{\text{fusion}}^4 \).
Table 2 demonstrates the impact of replacing MHSA with TSSA. Although the overall GFLOPs remain similar (57.0 vs 57.1), the inference speed increases from 76.28 to 93.52 FPS, and AP slightly improves by 0.3%. This is because TSSA eliminates redundant computations while enabling more efficient multi-scale alignment.
| Attention Type | Params (M) | GFLOPs | AP (%) | FPS |
|---|---|---|---|---|
| MHSA (baseline) | 19.88 | 57.0 | 20.3 | 76.28 |
| TSSA (ours) | 19.75 | 57.1 | 20.6 | 93.52 |
3.3 FineGrained-Detect Remote-Sensing Detection Head
The FineGrained-Detect head is designed to maximize small object detail retention while controlling computational overhead. Its key innovations are:
- Direct connection of raw high-resolution features: Unlike FPN/PANet/BiFPN which use down-then-up sampling, we directly connect the 320×320 C2 features from GradNet without any destructive down-sampling. This preserves the fine details of small objects (<16×16 pixels).
- Remote-sensing-specific lightweight fusion unit: A combination of channel attention gating and depthwise separable convolution. The channel attention gate suppresses background redundancy and emphasizes small object texture channels. Depthwise separable convolution replaces standard 3×3 convolution to reduce cost.
- Scale-specific detection branches: Three branches are built: (i) a 320×320 small-object branch with 5 anchor sizes (8×8, 12×12, 16×16, 20×20, 24×24) and CIoU loss; (ii) a 160×160 medium-object branch with lightweight decoupled heads; (iii) an 80×80 large-object branch. A cross-scale interaction gate transfers core semantic information from larger branches to the small-object branch while filtering out background noise.
The design reduces redundant computation by using depthwise separable convolutions and avoiding spatial attention (which would be \(O(n^2)\)). Table 3 compares FineGrained-Detect with traditional shallow feature fusion methods.
| Dimension | FPN/PANet/BiFPN | YOLO series | FineGrained-Detect (ours) |
|---|---|---|---|
| Fusion logic | Downsample then upsample (indirect) | Shallow downsampling to build detection layers (semi-direct) | Direct connection without downsampling |
| Fusion mechanism | Generic homogeneous convolution | Simple feature concatenation | Channel attention gate + depthwise separable conv |
| Detection branches | Shared multi-scale head (feature mismatch) | Multi-scale but homogeneous | Scale-specific branches with custom anchor and loss |
4. Experiments
4.1 Datasets and Settings
We evaluate on two standard China UAV remote sensing datasets:
- VisDrone2019: 6471 training images, 548 validation, 1610 test, covering 10 object classes with extreme scale variation (many objects <0.01% of image area).
- DOTA: 2806 training images, 1418 validation, 1723 test, 15 classes, with long-tail distribution and diverse backgrounds.
All experiments use identical hardware (NVIDIA RTX3090, Intel i9-12900K, 64G RAM) and software (PyTorch 2.2.2, CUDA 11.8). Input size: 640×640, batch: 4, epochs: 200, optimizer: AdamW, initial lr: 1e-4, weight decay: 1e-4, cosine decay schedule. Evaluation metrics include AP, AP_50, AP_s, AP_m, AP_l, GFLOPs, params, FPS.
4.2 Ablation Study
Ablation experiments on VisDrone2019 test set are summarized in Table 4. Starting from RT-DETR baseline (A), we gradually add GradNet (B), IST-Fusion (C), and FineGrained-Detect (D).
| Baseline | GradNet | IST-Fusion | FineDetect | GFLOPs | Params | AP | AP_50 | AP_s | AP_m | AP_l | FPS |
|---|---|---|---|---|---|---|---|---|---|---|---|
| ✓ | 57.0 | 19.88M | 0.203 | 0.355 | 0.112 | 0.300 | 0.352 | 76.28 | |||
| ✓ | ✓ | 50.0 | 13.55M | 0.208 | 0.362 | 0.117 | 0.304 | 0.387 | 75.76 | ||
| ✓ | ✓ | 57.1 | 19.75M | 0.206 | 0.358 | 0.114 | 0.300 | 0.374 | 93.52 | ||
| ✓ | ✓ | ✓ | 50.1 | 13.40M | 0.213 | 0.368 | 0.119 | 0.311 | 0.372 | 92.65 | |
| ✓ | ✓ | ✓ | ✓ | 116.0 | 15.46M | 0.235 | 0.402 | 0.142 | 0.336 | 0.429 | 59.31 |
The full model (ZY-DETR) achieves the best AP (23.5%), with significant improvements in all scale metrics, especially AP_s (+3.0%), AP_m (+3.6%), and AP_l (+7.7%). The inference speed of 59.31 FPS is sufficient for real-time China UAV applications. The model file size is reduced from 77.0MB to 60.6MB.
4.3 Comparison with State-of-the-Art Detectors
Table 5 compares ZY-DETR with leading detectors on VisDrone2019 test set. Our method achieves 23.5% AP, second only to DINO (25.3%) which has 2.3× more FLOPs and 3× more parameters. Among CNN-based methods, YOLO11m reaches 20.3% AP but with 20.04M parameters. ZY-DETR outperforms all YOLO versions, especially on small objects (AP_s 14.2% vs best YOLO 9.8%). Compared to the baseline RT-DETR, ZY-DETR improves AP by 3.2% while reducing parameters by 22.2%. The balanced performance across all scales demonstrates the effectiveness of our multi-scale fusion and detection head design.
| Model | Input | GFLOPs | Params (M) | AP | AP_50 | AP_s | AP_m | AP_l |
|---|---|---|---|---|---|---|---|---|
| YOLOv8m | 640 | 78.7 | 25.85 | 0.190 | 0.332 | 0.060 | 0.294 | 0.417 |
| YOLOv10m | 640 | 58.9 | 15.32 | 0.195 | 0.345 | 0.097 | 0.300 | 0.414 |
| YOLOv11m | 640 | 67.7 | 20.04 | 0.203 | 0.350 | 0.098 | 0.312 | 0.413 |
| YOLOv12m | 640 | 67.2 | 20.11 | 0.192 | 0.336 | 0.094 | 0.298 | 0.386 |
| RT-DETR-R18 | 640 | 57.0 | 19.88 | 0.203 | 0.355 | 0.112 | 0.300 | 0.352 |
| Faster R-CNN-R50-FPN | 768 | 208 | 41.39 | 0.194 | 0.329 | 0.095 | 0.309 | 0.429 |
| Cascade R-CNN-R50-FPN | 768 | 236 | 69.29 | 0.197 | 0.326 | 0.099 | 0.309 | 0.406 |
| DINO | 750 | 274 | 47.56 | 0.253 | 0.445 | 0.150 | 0.371 | 0.503 |
| DEIM-D-Fine-S | 640 | 24.9 | 10.18 | 0.219 | 0.384 | 0.122 | 0.321 | 0.397 |
| ZY-DETR (Ours) | 640 | 116 | 15.46 | 0.235 | 0.402 | 0.142 | 0.336 | 0.429 |
4.4 Generalization on DOTA Dataset
To verify cross-domain robustness, we evaluate ZY-DETR on the DOTA test and validation sets under the same hardware settings (no fine-tuning). Table 6 shows that our method consistently outperforms RT-DETR baseline by 2.6% and 2.7% in AP on test and val sets, respectively. Notably, AP_s improves by 4.7% on test set, confirming that the proposed FineGrained-Detect head and TSSA mechanism generalize well to different remote sensing scenes captured by China UAVs. The inference speed on DOTA reaches 85.61 FPS, even higher than on VisDrone2019 due to less complex scenes.
| Dataset | Model | Params (M) | AP | AP_50 | AP_s | AP_m | AP_l | FPS |
|---|---|---|---|---|---|---|---|---|
| Test | RT-DETR | 19.89 | 0.574 | 0.797 | 0.356 | 0.597 | 0.724 | 81.13 |
| ZY-DETR | 15.47 | 0.600 | 0.808 | 0.403 | 0.624 | 0.731 | 85.61 | |
| Val | RT-DETR | 19.89 | 0.588 | 0.818 | 0.362 | 0.622 | 0.662 | 78.36 |
| ZY-DETR | 15.47 | 0.615 | 0.830 | 0.401 | 0.658 | 0.678 | 82.03 |
5. Conclusion
In this work, we presented ZY-DETR, an improved RT-DETR algorithm tailored for small object detection in China UAV remote sensing images. By introducing a heterogeneous backbone (GradNet) that preserves shallow details and refines deep semantics, a linear-complexity cross-scale fusion module (IST-Fusion) based on token statistical self-attention, and a remote-sensing-specific detection head (FineGrained-Detect) with direct high-resolution feature connection, our method significantly boosts detection accuracy while maintaining real-time performance. Experimental results on VisDrone2019 and DOTA datasets demonstrate that ZY-DETR outperforms many state-of-the-art detectors, with particularly notable gains on small objects. The model’s small footprint (15.46M parameters) and high speed (up to 85.61 FPS) make it an excellent candidate for deployment on China UAV edge devices. Future work will focus on further reducing computational cost through quantization and knowledge distillation, as well as enhancing the robustness of TSSA for extremely small objects.