In recent years, the proliferation of small unmanned aerial vehicles (UAVs), or drones, has revolutionized various industries, but it has also introduced significant threats to low-altitude security. To mitigate risks such as unauthorized surveillance, smuggling, or potential attacks, effective anti-drone systems have become critical in both civilian and military applications. Among these systems, video-based anti-drone detection and tracking offer advantages like moderate range, low cost, and adaptability to diverse environments. However, tracking drones in video sequences presents unique challenges: drones often appear as small objects with low pixel occupancy, fly at low altitudes amidst complex backgrounds like buildings and trees, and exhibit rapid motion. These factors complicate traditional object tracking methods, leading to issues like drift or loss of target. In this work, I address these challenges by proposing an enhanced Siamese neural network algorithm, termed SiamAU, which integrates an improved backbone network and a feature rearrangement mechanism for robust anti-drone tracking.
Object tracking is a core component of video-based anti-drone tasks, enabling continuous localization of drones in real-time. Early tracking approaches relied on generative models or correlation filters, but deep learning-based methods, particularly Siamese networks, have shown superior performance. For instance, SiamFC introduced fully convolutional Siamese networks for efficient tracking, while SiamRPN incorporated region proposal networks to regress target boundaries. Subsequent advancements like SiamRPN++ leveraged deep backbones like ResNet-50 to enhance feature representation. However, in anti-drone scenarios, these methods still struggle due to the small size of drones and cluttered backgrounds. Drones often occupy few pixels in frames, making deep features less discriminative, while shallow features, rich in spatial details, are underutilized. Additionally, complex low-altitude environments introduce distractions like similar objects or occlusions. To overcome these limitations, I design SiamAU by augmenting the backbone with attention mechanisms and optimizing feature fusion for small targets. This approach aims to improve tracking accuracy and robustness in anti-drone applications, where precise drone localization is paramount for threat neutralization.
The proposed SiamAU algorithm builds upon SiamRPN++ as a baseline. I modify the ResNet-50 backbone by integrating the ECA-Net attention module and replacing the activation function with HardSwish, resulting in an enhanced HE-ResNet-50 network. This improves feature representation in complex anti-drone scenes by focusing on relevant channels and enhancing gradient flow. Furthermore, I introduce a feature rearrangement mechanism that reduces the dimensionality of shallow features and fuses them with deeper layers, leveraging spatial details crucial for small drone tracking. The overall architecture consists of twin HE-ResNet-50 branches for template and search frames, followed by multi-level feature fusion and region proposal networks for bounding box regression. Experiments on UAV123 and DUT Anti-UAV datasets demonstrate that SiamAU outperforms state-of-the-art trackers in anti-drone tasks, achieving higher success and precision rates. This work contributes to the field of anti-drone technology by providing a tailored solution for drone tracking, which can be integrated into broader surveillance systems to enhance low-altitude security.
Related Work in Anti-Drone Tracking
Anti-drone tracking has evolved alongside general object tracking techniques. Initial methods used audio, radar, or RF signals, but video-based approaches offer cost-effectiveness and versatility. In computer vision, trackers can be categorized into generative, discriminative, and deep learning-based models. Generative models like MOSSE learn target appearances, while discriminative methods like KCF employ correlation filters for efficiency. However, these struggle with drones due to scale variations and background clutter. Deep Siamese networks, such as SiamFC and SiamRPN, use similarity learning but lack deep feature exploitation. SiamRPN++ addressed this with ResNet-50, yet its deep features may ignore small drones. Recent attention-based trackers like SiamAttn or TransT improve accuracy but increase computation. For anti-drone scenarios, algorithms must balance speed and precision, as drones are fast-moving and small. Datasets like UAV123 provide aerial views, while DUT Anti-UAV focuses on ground-based anti-drone tracking with complex backgrounds. My work bridges this gap by enhancing feature utilization specifically for drones, ensuring robustness in real-world anti-drone operations.
Methodology: SiamAU for Anti-Drone Tracking
The SiamAU algorithm is designed to tackle anti-drone tracking challenges through two key innovations: an enhanced backbone network and a feature rearrangement scheme. I first describe the baseline SiamRPN++ framework, then detail my modifications.
Baseline: SiamRPN++ Architecture
SiamRPN++ employs a Siamese network with a ResNet-50 backbone for feature extraction. Given a template frame $z$ and a search frame $x$, it computes features via shared convolutional layers, then uses region proposal networks (RPN) for classification and regression. The features from the last three residual blocks (conv3, conv4, conv5) are adjusted to 256 channels and fused for multi-level representation. However, in anti-drone scenarios, shallow features from earlier layers (e.g., conv2) contain fine-grained details beneficial for small drones but are discarded in SiamRPN++. This limitation motivates my enhancements.
Enhanced Backbone Network: HE-ResNet-50
To improve feature discrimination in complex anti-drone backgrounds, I augment ResNet-50 with ECA-Net attention and HardSwish activation. The ECA-Net module performs local cross-channel interaction without dimensionality reduction, enhancing focus on informative features. For an input feature map $F \in \mathbb{R}^{C \times H \times W}$, global average pooling yields a channel descriptor $g \in \mathbb{R}^C$, followed by a 1D convolution with adaptive kernel size $k$ to capture channel dependencies. The attention weights $\alpha$ are computed as:
$$ g_c = \frac{1}{H \times W} \sum_{i=1}^H \sum_{j=1}^W F_c(i,j), $$
$$ \alpha = \sigma(\text{Conv1D}(g, k)), $$
where $\sigma$ is the sigmoid function. The output feature map is scaled by $\alpha$. I insert ECA-Net after conv3, conv4, and conv5 layers, producing enhanced features $E\_conv3$, $E\_conv4$, and $E\_conv5$.
Additionally, I replace the ReLU activation with HardSwish to facilitate better gradient propagation in deep networks. HardSwish is defined as:
$$ \text{HardSwish}(x) = x \cdot \frac{\text{ReLU6}(x + 3)}{6}, $$
where $\text{ReLU6}(x) = \min(\max(0, x), 6)$. This activation reduces computational cost compared to Swish while maintaining performance. The modified residual blocks in HE-ResNet-50 thus combine attention and improved non-linearity, boosting anti-drone feature extraction. Table 1 summarizes the backbone outputs.
| Layer | Output Size | Channels | Enhancement |
|---|---|---|---|
| conv1 | 61×61 | 64 | None |
| conv2 | 31×31 | 256 | HardSwish |
| conv3 | 15×15 | 512 | ECA-Net + HardSwish |
| conv4 | 15×15 | 1024 | ECA-Net + HardSwish |
| conv5 | 15×15 | 2048 | ECA-Net + HardSwish |
Feature Rearrangement Mechanism
For anti-drone tracking, shallow features are crucial due to drones’ small size. I propose a feature rearrangement (FR) module to leverage conv2 features, which have high spatial resolution (31×31) but are omitted in baseline fusion. First, I apply a dimensionality reduction to conv2 using a stride-2 convolution, reducing its spatial size to 15×15 and increasing channels to 1024. This is followed by a 1×1 convolution to obtain $Ep\_conv2$ with 512 channels. The process can be expressed as:
$$ Ep\_conv2 = \text{Conv}_{1\times1}(\text{Conv}_{\text{stride=2}}(E\_conv2)), $$
where $E\_conv2$ is the conv2 output with HardSwish. Next, I rearrange the channels of all enhanced features to balance contributions. Specifically, $Ep\_conv2$, $E\_conv3$, $E\_conv4$, and $E\_conv5$ are adjusted to 256 channels each using 1×1 convolutions, yielding $F\_conv2$, $F\_conv3$, $F\_conv4$, and $F\_conv5$. These are then concatenated along the channel dimension to form a fused feature map $F_{\text{fused}} \in \mathbb{R}^{15 \times 15 \times 1024}$. The fusion enhances small drone representation by combining spatial details from shallow layers with semantic information from deep layers. Mathematically, for feature maps $F_i$ with $i \in \{2,3,4,5\}$, the fusion is:
$$ F_{\text{fused}} = \text{Concat}(\text{Conv}_{1\times1}(F_i); \text{channels}=256). $$
This contrasts with SiamRPN++, which only uses conv3-conv5. The rearrangement ensures that anti-drone tracking benefits from multi-scale features, improving localization accuracy.
Overall Pipeline
The SiamAU pipeline processes template and search frames through HE-ResNet-50 branches. Features from conv2-conv5 are enhanced and rearranged as described, then fed into RPN heads for classification and bounding box regression. The loss function combines cross-entropy for classification and smooth L1 for regression, similar to SiamRPN++. During inference, the template is initialized from the first frame, and the search region is cropped from subsequent frames to predict drone positions. This design optimizes anti-drone tracking by addressing both background complexity and small target size.
Experimental Evaluation for Anti-Drone Tracking
I evaluate SiamAU on two public datasets to assess its anti-drone performance. All experiments are conducted on a system with Intel i7-11700 CPU, NVIDIA GTX3060 GPU, and PyTorch 1.8.0.
Datasets and Metrics
Two datasets are used: UAV123 and DUT Anti-UAV. UAV123 contains 123 sequences from drone perspectives, simulating aerial tracking with challenges like occlusion and scale change. DUT Anti-UAV is specifically designed for anti-drone tracking, with 20 sequences of drones flying in low-altitude environments like urban or wooded areas; it includes 24,804 frames with small targets and complex backgrounds. Both datasets are essential for benchmarking anti-drone algorithms.
I employ standard metrics: precision plot and success plot. Precision measures the percentage of frames where the center location error (CLE) is below a threshold (e.g., 20 pixels). CLE is computed as:
$$ \text{CLE} = \sqrt{(x_t – x_0)^2 + (y_t – y_0)^2}, $$
where $(x_t, y_t)$ is the predicted center and $(x_0, y_0)$ is the ground truth. Success measures the overlap ratio between predicted and ground truth bounding boxes, defined as:
$$ \text{IoU} = \frac{A_t \cap A_{gt}}{A_t \cup A_{gt}}, $$
where $A_t$ and $A_{gt}$ are areas. The success rate is the percentage of frames with IoU > 0.5. These metrics comprehensively evaluate anti-drone tracking accuracy.
Implementation Details
SiamAU is trained on a combination of LaSOT, DET, and YouTube-BB datasets, following SiamRPN++ protocol. The backbone is initialized from ImageNet pre-trained weights, and the network is trained for 20 epochs with stochastic gradient descent. For anti-drone specialization, I fine-tune on drone-related data if available. During testing, input sizes are 127×127 for template and 255×255 for search regions. I compare against seven state-of-the-art trackers: KCF, SiamFC, SiamRPN, SiamFC++, SiamAPN++, SiamRPN++, and ATOM. These represent diverse approaches, from correlation filters to deep Siamese networks.
Quantitative Results
Table 2 presents the success and precision scores on DUT Anti-UAV, the core anti-drone dataset. SiamAU achieves the best performance, with 60.5% success and 88.1% precision, outperforming others by significant margins. This demonstrates its effectiveness in anti-drone scenarios. On UAV123, SiamAU also performs well, though ATOM leads due to its generalization to larger objects. However, in anti-drone contexts where small targets dominate, SiamAU’s advantages are clear.
| Tracker | Success | Precision | Notes |
|---|---|---|---|
| KCF | 40.2 | 65.3 | Correlation filter-based |
| SiamFC | 48.7 | 70.1 | Basic Siamese network |
| SiamRPN | 52.3 | 75.6 | With region proposals |
| SiamFC++ | 55.8 | 79.4 | Anchor-free design |
| SiamAPN++ | 57.1 | 81.2 | Optimized for UAVs |
| SiamRPN++ | 54.9 | 80.0 | Baseline with ResNet-50 |
| ATOM | 57.7 | 83.0 | Discriminative model |
| SiamAU (Ours) | 60.5 | 88.1 | With HE-ResNet-50 and FR |
To analyze contributions, I conduct ablation studies on DUT Anti-UAV, as shown in Table 3. The baseline SiamRPN++ scores 54.9% success and 80.0% precision. Adding HardSwish activation alone improves precision by 0.4%, while adding ECA-Net attention boosts success by 0.8% and precision by 4.1%. The combined HE-ResNet-50 (with both enhancements) yields 56.3% success and 84.3% precision. Feature rearrangement alone increases success by 1.8% over baseline. When integrating both HE-ResNet-50 and feature rearrangement (full SiamAU), performance peaks at 60.5% success and 88.1% precision, confirming the synergy of these components for anti-drone tracking.
| Variant | Success | Precision | Description |
|---|---|---|---|
| SiamRPN++ | 54.9 | 80.0 | Baseline |
| + HardSwish | 55.0 | 80.4 | Improved activation |
| + ECA-Net | 55.7 | 84.1 | Attention added |
| HE-ResNet-50 | 56.3 | 84.3 | Combined backbone |
| + Feature Rearrangement | 56.7 | 81.0 | FR alone on baseline |
| SiamAU (Full) | 60.5 | 88.1 | All enhancements |
Qualitative Analysis
I visualize tracking results on four DUT Anti-UAV sequences: v05, v09, v12, and v17. These represent typical anti-drone scenarios with trees, buildings, and small drones. SiamAU maintains accurate tracks even when drones move across complex backgrounds, while baseline methods often drift or lose targets. For example, in v05, as a drone transitions from building to tree background, SiamAU successfully follows it, whereas SiamRPN++ fails. In v09, with temporary drone disappearance, SiamAU re-acquires the target quickly upon reappearance. These observations highlight SiamAU’s robustness in real-world anti-drone operations.
The feature heatmaps from HE-ResNet-50 show higher activation on drone regions compared to vanilla ResNet-50, thanks to ECA-Net’s focus. This enhances discrimination against distractions, a key requirement in anti-drone tracking. Additionally, the feature rearrangement ensures that shallow details are preserved, aiding small drone localization.
Computational Efficiency
While SiamAU introduces additional modules, its speed remains practical for real-time anti-drone systems. On a GTX3060, it processes at ~40 FPS, comparable to SiamRPN++ (~45 FPS). The ECA-Net adds minimal overhead due to its lightweight design, and HardSwish reduces activation latency. Thus, SiamAU balances accuracy and speed for deployment in anti-drone applications.
Discussion and Future Work
The proposed SiamAU algorithm demonstrates significant improvements in anti-drone tracking by addressing small target size and background complexity. The enhanced backbone with ECA-Net and HardSwish effectively captures discriminative features, while feature rearrangement leverages multi-scale information. However, challenges remain for long-term anti-drone tracking, such as handling full occlusions or extreme scale changes. Future work could integrate temporal modeling or adaptive template updating to further boost performance. Moreover, expanding to multi-drone scenarios or combining with other sensors (e.g., radar) could enhance anti-drone systems. The methodology presented here provides a foundation for developing advanced anti-drone technologies to safeguard low-altitude airspace.
Conclusion
In this work, I presented SiamAU, a Siamese network-based tracker tailored for anti-drone object tracking. By enhancing the backbone with ECA-Net attention and HardSwish activation, and introducing a feature rearrangement mechanism, SiamAU improves feature representation for small drones in complex environments. Extensive experiments on UAV123 and DUT Anti-UAV datasets show that SiamAU outperforms state-of-the-art trackers in anti-drone tasks, achieving success and precision rates of 60.5% and 88.1% on DUT Anti-UAV. This represents a step forward in video-based anti-drone systems, offering a robust solution for real-time drone tracking. As drone usage grows, such algorithms will be crucial for security and surveillance, contributing to safer skies in the era of ubiquitous aerial devices.