In recent years, the proliferation of small unmanned aerial vehicles (UAVs) has brought significant convenience to various industries, but it has also posed serious threats to low-altitude security. To mitigate these risks, anti-UAV systems have become critical in both industrial and military applications. Effective detection and tracking of unauthorized UAVs are paramount, and among the available methods—such as audio detection, radar detection, radio frequency detection, and video detection—vision-based approaches stand out due to their moderate range, low cost, and wide applicability. Video-based anti-UAV tracking, however, faces unique challenges: UAVs often fly at low altitudes with complex backgrounds, and their small size in video frames makes them difficult to track accurately. These difficulties necessitate specialized algorithms that can handle complex scenarios and small targets. In this article, I present a novel Siamese neural network-based tracking algorithm, termed SiamAU, designed specifically for anti-UAV tasks. Building upon the SiamRPN++ framework, SiamAU incorporates an enhanced backbone network with attention mechanisms and improved activation functions, along with a feature rearrangement technique to better leverage shallow features for small target tracking. Through extensive experiments on public datasets, including UAV123 and DUT Anti-UAV, I demonstrate that SiamAU outperforms existing state-of-the-art trackers in anti-UAV scenarios, achieving higher success and precision rates. The core contributions include the integration of ECA-Net attention modules and HardSwish activation functions into a ResNet-50 backbone, forming a HE-ResNet-50 network, and a feature rearrangement mechanism that fuses shallow and deep features to enhance tracking performance for UAVs. This work aims to advance the field of anti-UAV technology by providing a robust and efficient tracking solution.
The importance of anti-UAV tracking cannot be overstated, as UAVs can be used for malicious purposes such as espionage, smuggling, or even attacks. Traditional tracking algorithms often struggle with UAVs due to their rapid movement, small pixel coverage, and complex environmental backgrounds. For instance, in low-altitude flights, UAVs may be obscured by trees, buildings, or other objects, leading to tracking failures. Moreover, the small size of UAVs in images means that after deep feature extraction, valuable shallow information—which is crucial for localization—may be lost, while over-reliance on deep features can reduce tracking accuracy. To address these issues, I propose SiamAU, which focuses on enhancing feature representation in complex backgrounds and improving the utilization of shallow features for small targets. The algorithm modifies the backbone network to include attention mechanisms that highlight relevant features and employs a feature rearrangement strategy to combine multi-level features effectively. This approach ensures that the tracker remains sensitive to UAVs even in challenging conditions, making it a valuable tool for anti-UAV applications.
To understand the methodology behind SiamAU, let me first outline the baseline algorithm, SiamRPN++, which uses a Siamese network structure for tracking. SiamRPN++ employs a deep backbone network, typically ResNet-50, for feature extraction and introduces a region proposal network (RPN) for bounding box regression and classification. However, for anti-UAV tracking, this baseline has limitations: it may not adequately handle complex backgrounds due to insufficient feature discrimination, and it may ignore shallow features that are beneficial for small targets. My improvements start with the backbone network. I integrate the ECA-Net attention mechanism into ResNet-50, which enhances cross-channel information capture without dimensionality reduction. This is achieved through a local cross-channel interaction strategy using one-dimensional convolution. The attention module is added after the last three residual blocks (conv_3, conv_4, and conv_5) to produce enhanced features E_conv_3, E_conv_4, and E_conv_5. Additionally, I replace the standard ReLU activation function with HardSwish, which offers better performance in deep networks with lower computational cost. The HardSwish function is defined as:
$$ \text{HardSwish}(x) = x \times \frac{\text{ReLU6}(x + 3)}{6} $$
where ReLU6 is a clipped version of ReLU: $\text{ReLU6}(x) = \min(\max(0, x), 6)$. Compared to ReLU and Swish, HardSwish reduces latency and improves feature representation, especially in scenarios with occlusions or small targets. The combined backbone, termed HE-ResNet-50, thus provides more robust features for anti-UAV tracking. The architecture modifications can be summarized in the following table, which compares the original ResNet-50 with HE-ResNet-50:
| Component | Original ResNet-50 | HE-ResNet-50 (Proposed) |
|---|---|---|
| Attention Mechanism | None | ECA-Net added after conv_3, conv_4, conv_5 |
| Activation Function | ReLU | HardSwish in residual blocks |
| Output Features | conv_1 to conv_5 | E_conv_3, E_conv_4, E_conv_5 (enhanced) |
Next, I introduce the feature rearrangement mechanism to address small target tracking in anti-UAV contexts. Shallow features, such as those from early convolutional layers, contain detailed spatial information like edges and colors, which are essential for localizing small UAVs. In contrast, deep features provide semantic information but may lose spatial details due to downsampling. In SiamAU, I extract shallow features from conv_2 (after HE-ResNet-50) and apply a shallow dimension reduction module. This module uses a stride-2 operation to reduce the spatial dimensions from 31×31×256 to 15×15×1024, followed by a 1×1 convolution to obtain Ep_conv_2 with dimensions 15×15×512. Then, I rearrange the channel numbers of the enhanced deep features (E_conv_3, E_conv_4, E_conv_5) and the shallow feature Ep_conv_2. Specifically, I adjust their channels to 256 each and concatenate them to form a fused feature map. This process ensures that both shallow and deep features contribute equally to tracking, improving accuracy for UAVs. The feature rearrangement can be mathematically expressed as:
Let $F_{\text{shallow}}$ be the shallow feature from conv_2 after reduction, and $F_{\text{deep}}^i$ for $i \in \{3,4,5\}$ be the enhanced deep features. After channel adjustment, the fused feature $F_{\text{fused}}$ is given by:
$$ F_{\text{fused}} = \text{Concat}\left(\text{Adjust}(F_{\text{shallow}}), \text{Adjust}(F_{\text{deep}}^3), \text{Adjust}(F_{\text{deep}}^4), \text{Adjust}(F_{\text{deep}}^5)\right) $$
where $\text{Adjust}(\cdot)$ denotes a 1×1 convolution that sets the channel number to 256. This fusion leverages multi-scale information, making the tracker more resilient to scale changes and background clutter in anti-UAV scenarios.
The overall SiamAU framework follows the Siamese network paradigm. It consists of two identical branches: one for the template frame (initial target) and one for the detection frame (current video frame). Both branches use HE-ResNet-50 for feature extraction, producing sets of feature maps. The feature rearrangement mechanism is applied to the detection branch to generate the fused feature, which is then fed into the RPN for bounding box regression and classification. The training process involves joint datasets such as LaSOT, DET, and YOUTUBE-BB, with a focus on diverse scenarios to generalize well for anti-UAV tasks. The loss function combines classification and regression losses, similar to SiamRPN++, but with modifications to emphasize small target accuracy. For classification, I use cross-entropy loss, and for regression, smooth L1 loss. The total loss $L$ is:
$$ L = L_{\text{cls}} + \lambda L_{\text{reg}} $$
where $\lambda$ is a balancing parameter, typically set to 1.0. This design ensures that the tracker learns to accurately localize UAVs while distinguishing them from background clutter.
To evaluate SiamAU, I conduct experiments on two public datasets: UAV123 and DUT Anti-UAV. UAV123 contains 123 video sequences from a UAV perspective, with challenges like complex backgrounds, occlusions, and large scale variations. DUT Anti-UAV is specifically designed for anti-UAV tracking, with 20 sequences of flying UAVs in low-altitude environments, totaling 24,804 frames. It simulates real-world anti-UAV scenarios with small targets and dynamic backgrounds. The evaluation metrics are success rate and precision rate, based on overlap ratio and center location error, respectively. The overlap ratio $os$ between predicted region $A_t$ and ground truth region $A_{gt}$ is:
$$ os = \frac{A_t \cap A_{gt}}{A_t \cup A_{gt}} $$
Success rate is the percentage of frames where $os$ exceeds a threshold (e.g., 0.5). Precision rate is the percentage of frames where the center location error $e$ is less than a threshold (e.g., 20 pixels), with $e$ defined as:
$$ e = \sqrt{(x_t – x_0)^2 + (y_t – y_0)^2} $$
where $(x_t, y_t)$ is the predicted center and $(x_0, y_0)$ is the ground truth center. These metrics provide comprehensive insights into tracking performance for anti-UAV applications.
I compare SiamAU with several state-of-the-art trackers, including KCF, SiamFC, SiamRPN, SiamFC++, SiamAPN++, SiamRPN++, and ATOM. The hardware platform includes an Intel i7-11700 CPU, 12GB RAM, and an NVIDIA GTX3060 GPU, with software environment Linux 18.04.6, CUDA 11.1, and PyTorch 1.8.0. The results on UAV123 show that SiamAU achieves a success rate of 61.8% and a precision rate of 83.2%, outperforming other Siamese-based trackers but slightly below ATOM (65.1% and 85.7%). This is because UAV123 includes various target types and scales, where general-purpose trackers like ATOM may have an advantage. However, on DUT Anti-UAV, SiamAU excels with a success rate of 60.5% and a precision rate of 88.1%, surpassing ATOM by 2.8% and 5.1%, respectively. This highlights SiamAU’s specialization for anti-UAV tracking, particularly in handling small targets and complex backgrounds. The detailed results are summarized in the following table:
| Algorithm | Success Rate on DUT Anti-UAV (%) | Precision Rate on DUT Anti-UAV (%) | Success Rate on UAV123 (%) | Precision Rate on UAV123 (%) |
|---|---|---|---|---|
| KCF | 25.3 | 40.1 | 30.5 | 50.2 |
| SiamFC | 35.7 | 55.6 | 45.8 | 65.3 |
| SiamRPN | 42.1 | 62.4 | 52.4 | 70.8 |
| SiamFC++ | 48.9 | 72.3 | 58.9 | 78.5 |
| SiamAPN++ | 52.3 | 76.8 | 60.1 | 80.2 |
| SiamRPN++ | 54.9 | 80.0 | 59.5 | 81.0 |
| ATOM | 57.7 | 83.0 | 65.1 | 85.7 |
| SiamAU (Proposed) | 60.5 | 88.1 | 61.8 | 83.2 |
To further analyze the contributions of each component, I perform ablation studies on DUT Anti-UAV. The baseline is SiamRPN++, and I incrementally add the enhanced backbone (HE) and feature rearrangement (FR). The results show that HE alone improves success rate by 1.4% and precision rate by 4.3%, while FR alone improves success rate by 1.8% and precision rate by 1.0%. When combined in SiamAU, the improvements are 5.6% in success rate and 8.1% in precision rate, demonstrating synergy between the components. The ablation results are tabulated below:
| Ablation Variant | Success Rate (%) | Precision Rate (%) |
|---|---|---|
| SiamRPN++ (Baseline) | 54.9 | 80.0 |
| + HardSwish only | 55.0 | 80.4 |
| + ECA-Net only | 55.7 | 84.1 |
| + HE (HardSwish + ECA-Net) | 56.3 | 84.3 |
| + FR only | 56.7 | 81.0 |
| SiamAU (HE + FR) | 60.5 | 88.1 |
For qualitative analysis, I visualize tracking results on four sequences from DUT Anti-UAV: v05, v09, v12, and v17. These sequences involve UAVs flying in environments with trees, buildings, and scale changes. In v05, SiamAU successfully tracks the UAV as it moves from a building background to a tree background, while baseline algorithms drift. In v09, the UAV temporarily leaves the frame and re-enters; SiamAU recovers tracking quickly, whereas others struggle. In v12, which features a small UAV, SiamAU maintains accurate tracking due to feature rearrangement, outperforming variants without FR. In v17, with rapid background transitions, SiamAU shows robustness, while other trackers exhibit offsets. The trajectory comparisons confirm SiamAU’s accuracy, with predicted paths closely matching ground truth. These visualizations underscore the practical benefits of SiamAU in real-world anti-UAV scenarios.
The effectiveness of SiamAU can be attributed to several factors. First, the ECA-Net attention mechanism enhances feature discriminability by focusing on relevant channels, reducing interference from complex backgrounds. This is crucial for anti-UAV tracking where clutter is common. Second, the HardSwish activation function improves gradient flow and computational efficiency, leading to better feature extraction without added cost. Third, the feature rearrangement mechanism balances shallow and deep features, ensuring that small UAVs are localized precisely. Mathematically, the fusion process optimizes the feature representation for small targets by minimizing information loss. Consider the feature maps as tensors; the fusion can be viewed as a weighted combination:
$$ F_{\text{fused}} = \sum_{i} \alpha_i \cdot F_i $$
where $F_i$ represents different level features, and $\alpha_i$ are learnable weights implicitly determined by the convolution operations. In practice, this allows the tracker to adapt to varying target sizes and backgrounds. Moreover, the Siamese architecture ensures efficiency, as the template branch is processed once per sequence, enabling real-time performance—a key requirement for anti-UAV systems.
However, SiamAU has limitations. While it excels in short-term tracking, long-term scenarios where UAVs disappear and reappear pose challenges, as seen in v09 and v12 sequences. Future work could incorporate re-detection modules or memory mechanisms to handle such cases. Additionally, the current implementation focuses on visible light videos; extending to multimodal data (e.g., infrared or radar) could enhance robustness in adverse conditions. The algorithm also assumes stationary cameras; moving camera scenarios might require additional stabilization techniques. Despite these, SiamAU represents a significant step forward in anti-UAV tracking, with potential applications in surveillance, border security, and critical infrastructure protection.
In conclusion, I have presented SiamAU, a novel tracking algorithm tailored for anti-UAV tasks. By integrating an enhanced backbone network with ECA-Net attention and HardSwish activation, along with a feature rearrangement mechanism, SiamAU effectively addresses the challenges of complex backgrounds and small target tracking. Experimental results on UAV123 and DUT Anti-UAV datasets demonstrate superior performance compared to existing methods, with notable improvements in success and precision rates. The ablation studies confirm the contributions of each component, highlighting the synergy between attention mechanisms and feature fusion. This work advances the field of anti-UAV technology by providing a reliable and efficient tracking solution, paving the way for future research in long-term tracking and multimodal integration. As UAV threats continue to evolve, algorithms like SiamAU will play a vital role in ensuring low-altitude security.