I have focused my research on the design and implementation of a deep learning based target tracking system for UAV drone platforms. The rapid development of microelectronics and embedded systems has enabled UAV drone technology to advance significantly. Because of their small size, flexible maneuverability, and hovering capability, UAV drones are now widely used in aerial photography, security surveillance, path planning, and many other fields. Target tracking is one of the core technologies for intelligent UAV drone operation, and it plays a critical role in autonomous navigation and target following tasks. However, visual target tracking in real UAV drone scenarios remains highly challenging. The tracked targets are diverse and uncertain, so the tracking algorithm must generalize well without prior knowledge of target appearance. In addition, UAV drone tracking scenes are complex and dynamic, with frequent camera motion, viewpoint changes, motion blur, occlusion, and scale variation. These factors greatly increase tracking difficulty. Therefore, a UAV drone tracking algorithm must achieve both strong anti-interference ability and real-time performance.
Traditional tracking algorithms based on correlation filters have achieved good real-time performance, but they often fail in complex scenes with deformation, occlusion, and background clutter. Deep learning based trackers have become the mainstream because convolutional neural networks can extract deep features that are more robust to appearance changes and environmental interference. Representative methods include fully convolutional Siamese networks, Siamese region proposal networks, and Transformer based trackers. Although these methods achieve high accuracy, most of them require powerful GPUs and large memory, which makes them difficult to deploy on UAV drone platforms. A UAV drone usually carries embedded computing devices with limited power, weight, and computation. Therefore, I designed a lightweight deep learning tracker named SiamLT that combines Siamese networks and Transformers. I also studied model quantization for edge deployment and built a quadrotor UAV drone platform for onboard tracking experiments.
In this article, I present the complete research work in a structured manner. I first review the background and related theory. Then I describe the proposed SiamLT algorithm in detail, including the lightweight CNN backbone, the LMFF-Trans feature fusion network, the DW_LMFF cross-correlation layer, and the uncertainty-based loss function. After that, I report extensive experiments on UAV123, LaSOT, and GOT-10k datasets. I further discuss edge deployment on CPU and NPU devices, including quantization strategies and hardware acceleration. Finally, I describe the quadrotor UAV drone platform design and onboard tracking experiments. Throughout the article, I use tables and mathematical formulas to summarize key concepts and results.
Background and Motivation
Visual target tracking aims to continuously locate a target in a video sequence given its initial position. Unlike object detection, tracking exploits temporal correlation between consecutive frames. For UAV drone tracking, the main challenges include small target size, low resolution, frequent scale changes, camera motion, viewpoint changes, occlusion, and background clutter. The following table summarizes twelve common challenge attributes in UAV drone tracking benchmarks.
| Number | Challenge | Description |
|---|---|---|
| 1 | SV | Scale variation |
| 2 | ARC | Aspect ratio change |
| 3 | LR | Low resolution |
| 4 | FM | Fast motion |
| 5 | FOC | Full occlusion |
| 6 | POC | Partial occlusion |
| 7 | OV | Out-of-view |
| 8 | BC | Background clutter |
| 9 | IV | Illumination variation |
| 10 | VC | Viewpoint change |
| 11 | CM | Camera motion |
| 12 | SOB | Similar object |
For UAV drone platforms, the onboard processor has limited computing resources and cannot support high-complexity deep models. High-altitude flight also reduces target resolution, making feature extraction difficult. Wind and other external factors cause camera shake, viewpoint change, and motion blur, leading to tracking drift or loss. Therefore, I aimed to design a tracker that can run in real time on edge devices while maintaining high accuracy.
Related Work
Target tracking algorithms can be broadly divided into correlation filter methods and deep learning methods. Correlation filter trackers such as KCF and MOSSE use frequency-domain correlation to estimate target position. They are efficient but struggle with deformation, occlusion, fast motion, and complex background. Deep learning trackers use CNNs to model target appearance. SiamFC formulated tracking as similarity learning and introduced a fully convolutional Siamese network. SiamRPN added a region proposal network to improve scale and aspect ratio adaptation. Later methods such as SiamCAR, SiamBAN, SiamGAT, and SiamAttn further improved performance. However, CNN based trackers have limited global modeling ability and long-range dependency capture.
Vision Transformer introduced self-attention to computer vision and enabled global feature interaction. SwinTrack, SparseTT, ARTrack, MixFormer, HIPTrack, and DiffusionTrack are representative Transformer based trackers. They achieve high accuracy but often have large parameter counts and computation costs. For UAV drone tracking, lightweight methods such as SiamAPN++, TCTrack, AutoTrack, and SiamTPN have been proposed. SiamAPN++ achieved real-time tracking on an embedded GPU. TCTrack introduced temporal context and achieved 27 FPS on an embedded platform. AutoTrack achieved 60 FPS on CPU using spatio-temporal regularization. SiamTPN used ShuffleNet and a feature pyramid for CPU tracking at 30 FPS. Despite these advances, a lightweight and accurate tracker for UAV drone edge platforms remains an open problem.
Proposed SiamLT Algorithm
I proposed SiamLT, a CNN-Transformer hybrid tracker for UAV drone target tracking. The overall architecture contains three parts: a lightweight CNN backbone, a multi-scale feature fusion network, and a prediction head. The backbone extracts multi-scale features from the template and search regions. The fusion network integrates shallow and deep features. The prediction head performs classification and regression. I designed the network to balance accuracy and efficiency. The following sections describe each component.
Lightweight CNN Backbone
For UAV drone deployment, the backbone must be lightweight. I adopted GhostNet as the backbone because it generates redundant feature maps with cheap linear operations instead of standard convolutions. A Ghost module first applies a small convolution to produce part of the output, then applies linear transformations to generate more feature maps. These parts are concatenated to approximate the output of a standard convolution with lower cost. I designed the backbone to output three feature levels, denoted as \(F_1\), \(F_2\), and \(F_3\). The detailed structure is summarized in the following table.
| Input | Operation | Stride | Channels | Label |
|---|---|---|---|---|
| 320×320×3 | Conv | 2 | 16 | – |
| 160×160×16 | Ghost | 1 | 16 | – |
| 160×160×16 | Ghost | 2 | 24 | – |
| 80×80×24 | Ghost | 1 | 24 | – |
| 80×80×24 | Ghost | 2 | 40 | – |
| 40×40×40 | Ghost | 1 | 40 | – |
| 40×40×40 | Ghost | 2 | 80 | 1 |
| 20×20×80 | Ghost | 1 | 80 | – |
| 20×20×80 | Ghost | 1 | 80 | – |
| 20×20×80 | Ghost | 1 | 80 | – |
| 20×20×80 | Ghost | 1 | 112 | – |
| 20×20×112 | Ghost | 1 | 112 | – |
| 20×20×112 | Ghost | 2 | 160 | 2 |
| 10×10×160 | Ghost | 1 | 160 | – |
| 10×10×160 | Ghost | 1 | 160 | – |
| 10×10×160 | Ghost | 1 | 160 | – |
| 10×10×160 | Ghost | 1 | 160 | 3 |
The backbone provides multi-scale features that contain both low-level details and high-level semantics. This is important for UAV drone tracking because small targets and low-resolution frames require fine-grained information.
LMFF-Trans Design
Traditional feature pyramid networks directly upsample and add features from different levels. This can destroy semantic information and is not effective when scale differences are large. Vision Transformer can model global dependencies, but its attention computation is heavy. I therefore designed a lightweight multi-scale feature fusion Transformer called LMFF-Trans. The core idea is to keep the query token count unchanged while reducing the key and value token counts through feature compression. This preserves the output scale and reduces computation.
Given an input feature \(X \in \mathbb{R}^{H \times W \times C}\), I split it into \(X_Q\) and \(X_{KV}\). \(X_Q\) generates the query, and \(X_{KV}\) generates the key and value. I compress \(X_{KV}\) to a smaller spatial size:
$$ X_Q = X, \quad X_{KV} = p(X, r) $$
where \(p\) is a compression operation and \(r\) is the compression factor. The compressed feature has size \(h \times w \times C\), with \(h \le H\) and \(w \le W\). The query, key, and value are computed as:
$$ Q = X_Q W_Q, \quad K = X_{KV} W_K, \quad V = X_{KV} W_V $$
The standard multi-head attention computation is:
$$ \text{Attention}(Q,K,V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V $$
The computational cost of standard multi-head attention is:
$$ \Omega(\text{MHA}) = 2HWC^2 + 4(HW)^2 C $$
After my compression, the cost becomes:
$$ \Omega(\text{Ours-MHA}) = 2HWC^2 + 2hwC^2 + 2HWhwC $$
Because \(h \le H\) and \(w \le W\), the cost is significantly reduced. The compression also balances the contribution of features from different scales during concatenation. I visualized the attention maps and found that LMFF-Trans focuses more on the target region than a standard Transformer encoder. The following table compares the two designs.
| Method | Query Source | Key/Value Source | Output Scale | Relative Cost |
|---|---|---|---|---|
| Standard Transformer | All tokens | All tokens | \(HW\) | High |
| LMFF-Trans | \(X_Q\) | Compressed \(X_{KV}\) | \(HW\) | Low |
Fusion Block
I built a Fusion Block based on LMFF-Trans. Each Fusion Block contains one cross-attention LMFF-Trans block and three self-attention LMFF-Trans blocks. The first block fuses features from the backbone, and the remaining blocks enhance the fused representation. The computation can be written as:
$$ \text{output}_1 = LT_1((F_1,F_2,F_3), (4,2,1), F_3) $$
$$ \text{output}_n = LT_n(\text{output}_{n-1}, 2, \text{output}_{n-1}), \quad n = 2,3,4 $$
where \(LT_n\) denotes the \(n\)-th LMFF-Trans block, and the numbers in parentheses are compression factors. The output of one Fusion Block is fed into the next Fusion Block as part of the input. By adjusting the number of Fusion Blocks, I can trade off speed and accuracy. This is useful for different UAV drone edge devices.
DW_LMFF Cross-Correlation
Siamese trackers usually use cross-correlation to match template and search features. Cross-correlation is efficient and captures local information, but it can break semantic information and fall into local optima. Transformer based matching captures global information but has weak local inductive bias. I combined both by designing DW_LMFF. It concatenates the output of depthwise cross-correlation and LMFF-Trans matching:
$$ \text{DW\_LMFF}(X_t, X_s) = \text{Concat}(D(X_t, X_s), L(X_t, X_s)) $$
where \(D\) is depthwise cross-correlation and \(L\) is LMFF-Trans matching. This design integrates local and global information. The prediction head has two branches: a classification branch and a regression branch. The classification branch distinguishes foreground from background, and the regression branch predicts the bounding box.
Loss Function
The total loss consists of three subtasks: cross-entropy loss \(L_{cls}\), L1 loss \(L_{reg}\), and GIoU loss \(L_{giou}\). The cross-entropy loss is:
$$ L_{cls} = -\sum_i y_i \log(\hat{y}_i) $$
The L1 loss is:
$$ L_{reg} = \sum_{j \in \{x,y,w,h\}} |y_j – \hat{y}_j| $$
The GIoU loss is:
$$ GIoU = IoU – \frac{|C \setminus (B_{pred} \cup B_{gt})|}{|C|} $$
$$ L_{giou} = 1 – GIoU $$
Instead of manually tuning the weights, I used uncertainty to weigh the losses automatically. The total loss is:
$$ L_{sum} = \sum_{i=1}^n \left( \frac{1}{2\sigma_i^2} L_i + \ln(1+\sigma_i^2) \right) $$
where \(\sigma_i\) is a learnable parameter. This allows the model to adaptively balance classification and regression during training.
Experiments and Results
I trained SiamLT on a combination of LaSOT, TrackingNet, COCO, and GOT-10k. The template size was \(80 \times 80\), and the search size was \(320 \times 320\). I used AdamW with a learning rate of \(10^{-4}\) for most modules and \(10^{-5}\) for the backbone. The initial value of \(\sigma_i\) was 0.25. I trained for 500 epochs with batch size 32. I evaluated the tracker on UAV123, LaSOT, and GOT-10k. These datasets cover diverse UAV drone scenarios and long-term tracking challenges.
Ablation Study
I conducted ablation studies on UAV123 to evaluate the contribution of each component. The following table summarizes the results. In the table, Backbone is the feature extractor, Fusion is the fusion method, \(F_i\) are the selected feature levels, \(N\) is the number of Fusion Blocks, and Corr. is the correlation method.
| Backbone | Fusion | Features | N | Corr. | Params (M) | FLOPs (G) | AUC (%) | Precision (%) | FPS |
|---|---|---|---|---|---|---|---|---|---|
| AlexNet | None | – | – | DW | 3.9 | 6.73 | 50.9 | 72.2 | 149 |
| GhostNet | None | – | – | DW | 0.8 | 0.28 | 52.9 | 75.1 | 156 |
| GhostNet | None | – | – | DW_LMFF | 1.5 | 0.56 | 57.7 | 78.6 | 131 |
| GhostNet | FPN | 1~3 | 2 | DW_LMFF | 7.3 | 2.93 | 61.2 | 81.0 | 79 |
| GhostNet | Transformer | 1~3 | 2 | DW_LMFF | 8.7 | 3.16 | 61.8 | 80.7 | 74 |
| GhostNet | LMFF-Trans | 1~3 | 2 | DW_LMFF | 9.1 | 2.76 | 64.6 | 84.5 | 87 |
| GhostNet | LMFF-Trans | 1~3 | 1 | DW_LMFF | 7.2 | 2.06 | 63.1 | 83.5 | 96 |
| GhostNet | LMFF-Trans | 2 | 2 | DW_LMFF | 9.1 | 2.71 | 62.7 | 82.6 | 89 |
| GhostNet | LMFF-Trans | 1~3 | 2 | DW | 7.0 | 1.94 | 61.6 | 81.4 | 98 |
| GhostNet | LMFF-Trans | 1~3 | 2 | LMFF | 7.5 | 2.12 | 62.3 | 82.4 | 96 |
The results show that GhostNet reduces parameters and FLOPs compared with AlexNet while improving AUC and precision. LMFF-Trans achieves the best fusion performance. Using only a single Fusion Block reduces AUC by 1.5%. Using only \(F_2\) reduces AUC by 1.9%. DW_LMFF improves AUC by 3.0% over DW correlation and by 2.3% over LMFF-only matching.
Comparison with State-of-the-Art
I compared SiamLT with Transformer based trackers, Siamese trackers, and UAV drone specific trackers. The following table lists parameters and FLOPs. SiamLT has 9.1M parameters and 2.8G FLOPs, which is much lower than Transformer based methods.
| Category | Tracker | Params (M) | FLOPs (G) |
|---|---|---|---|
| Transformer | SparseTT | 46.3 | 9.2 |
| Transformer | ARTrack | 173.1 | 37.2 |
| Transformer | MixFormer | 189.9 | 113.0 |
| Siamese | SiamFC | 2.3 | 3.2 |
| Siamese | HiFT | 11.1 | 6.5 |
| Siamese | SiamAPN++ | 15.4 | 7.6 |
| Siamese | TCTrack | 10.4 | 4.5 |
| Siamese | SiamCAR | 51.4 | 59.3 |
| Siamese | SiamBAN | 53.9 | 59.6 |
| Siamese | SiamLT | 9.1 | 2.8 |
On UAV123, SiamLT achieves 64.6% AUC and 84.5% precision. It outperforms traditional Siamese trackers and recent UAV drone trackers such as TCTrack, while using far fewer FLOPs. The following table shows detailed results on twelve challenge attributes.
| Tracker | SV AUC | SV Precision | ARC AUC | ARC Precision | LR AUC | LR Precision | FM AUC | FM Precision | FOC AUC | FOC Precision | POC AUC | POC Precision |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SparseTT | 68.2 | 87.2 | 67.2 | 86.7 | 54.1 | 77.0 | 64.9 | 83.7 | 50.0 | 74.2 | 62.5 | 83.6 |
| ARTrack | 67.1 | 87.7 | 66.6 | 87.4 | 53.4 | 78.7 | 66.1 | 86.9 | 50.2 | 75.5 | 63.2 | 85.6 |
| MixFormer | 68.4 | 89.8 | 69.3 | 91.2 | 54.2 | 80.3 | 67.1 | 88.7 | 52.6 | 78.8 | 64.7 | 87.8 |
| SiamFC | 47.6 | 69.3 | 43.2 | 65.8 | 35.4 | 60.8 | 41.5 | 63.6 | 29.7 | 55.4 | 41.7 | 63.1 |
| HiFT | 57.1 | 76.8 | 53.7 | 73.3 | 42.8 | 65.5 | 55.4 | 77.8 | 35.8 | 58.6 | 48.8 | 68.4 |
| SiamAPN++ | 55.4 | 73.6 | 52.9 | 71.2 | 41.5 | 63.3 | 55.8 | 75.2 | 33.4 | 56.1 | 47.6 | 65.8 |
| TCTrack | 58.6 | 77.8 | 56.6 | 75.3 | 43.8 | 67.9 | 55.3 | 75.7 | 39.1 | 62.3 | 51.8 | 71.6 |
| SiamCAR | 60.5 | 79.1 | 58.0 | 75.9 | 46.5 | 69.3 | 54.9 | 74.2 | 41.9 | 66.0 | 53.3 | 72.4 |
| SiamBAN | 61.3 | 81.3 | 59.1 | 79.6 | 47.2 | 71.9 | 59.3 | 80.5 | 41.8 | 67.1 | 55.0 | 76.5 |
| SiamLT | 62.7 | 82.6 | 62.8 | 83.5 | 50.9 | 75.3 | 57.0 | 76.8 | 49.0 | 75.5 | 58.2 | 79.2 |
| Tracker | OV AUC | OV Precision | BC AUC | BC Precision | IV AUC | IV Precision | VC AUC | VC Precision | CM AUC | CM Precision | SOB AUC | SOB Precision |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SparseTT | 67.8 | 87.3 | 46.0 | 66.0 | 62.6 | 82.5 | 71.8 | 89.7 | 70.4 | 90.2 | 66.0 | 87.0 |
| ARTrack | 68.0 | 87.5 | 48.9 | 71.4 | 64.2 | 86.2 | 71.2 | 90.9 | 70.7 | 92.6 | 64.8 | 88.5 |
| MixFormer | 67.3 | 87.7 | 53.6 | 77.8 | 65.4 | 88.4 | 72.6 | 92.8 | 72.4 | 95.0 | 67.8 | 92.1 |
| SiamFC | 45.7 | 67.9 | 30.2 | 50.3 | 39.0 | 58.8 | 46.5 | 66.4 | 50.6 | 72.8 | 47.3 | 71.2 |
| HiFT | 52.2 | 70.0 | 39.2 | 59.4 | 50.2 | 70.0 | 58.8 | 76.4 | 60.0 | 79.9 | 51.4 | 71.3 |
| SiamAPN++ | 53.9 | 71.3 | 38.7 | 58.4 | 51.6 | 70.8 | 59.2 | 76.1 | 58.9 | 77.3 | 48.3 | 66.4 |
| TCTrack | 57.5 | 75.2 | 39.7 | 59.1 | 51.8 | 71.0 | 61.6 | 77.1 | 60.5 | 79.7 | 53.1 | 74.1 |
| SiamCAR | 56.4 | 73.5 | 45.8 | 65.9 | 56.6 | 74.8 | 64.6 | 80.7 | 61.2 | 79.7 | 56.3 | 75.4 |
| SiamBAN | 58.9 | 78.9 | 43.1 | 64.5 | 56.5 | 76.6 | 63.9 | 82.4 | 63.9 | 84.8 | 56.6 | 77.7 |
| SiamLT | 62.3 | 82.2 | 51.8 | 75.2 | 56.2 | 76.0 | 65.9 | 84.3 | 65.9 | 86.0 | 60.3 | 81.3 |
SiamLT achieves strong performance in ten of twelve challenge categories. It is particularly effective for low resolution, scale variation, and occlusion. On LaSOT, SiamLT achieves 56.8% AUC and 56.7% precision, outperforming SiamGAT and SiamRN. It also improves over TCTrack by 11.2% AUC. On GOT-10k, SiamLT achieves 59.9% AO, 69.8% SR0.5, and 50.0% SR0.75, while running at 36.3 FPS on CPU. The following table summarizes GOT-10k results.
| Tracker | AO (%) | SR0.5 (%) | SR0.75 (%) | FPS | FLOPs (G) |
|---|---|---|---|---|---|
| SiamFC | 34.8 | 35.3 | 9.8 | 9.8 | 9.2 |
| SiamBAN | 54.7 | 64.9 | 40.6 | 1.0 | 59.6 |
| HiFT | 49.6 | 58.8 | 27.5 | 5.7 | 6.5 |
| SiamAPN++ | 46.7 | 53.5 | 26.4 | 9.3 | 7.6 |
| TCTrack | 48.2 | 55.6 | 31.0 | 11.5 | 4.5 |
| MixFormer | 75.6 | 85.7 | 72.8 | 0.3 | 113.0 |
| SiamLT | 59.9 | 69.8 | 50.0 | 36.3 | 2.8 |
Qualitative Analysis
I analyzed five representative sequences: bird1, group2, wakeboard6, boat8, and bike2. In bird1, the target is a flying bird with low resolution and fast motion. Only SiamLT re-locates the target after it leaves and re-enters the view. In group2, the target is fully occluded by a building. SiamLT maintains a larger search region and successfully recovers the target. In wakeboard6, the camera viewpoint changes sharply. Most trackers jitter, but SiamLT remains stable. In boat8, the target scale and aspect ratio change rapidly. SiamLT predicts the scale accurately because of multi-scale fusion. In bike2, a similar object interferes with the target. SiamLT avoids confusion due to its shallow feature fusion. These results confirm that SiamLT is effective for UAV drone tracking.
Edge Deployment and Quantization
UAV drone platforms usually cannot carry high-power GPUs. Therefore, I studied deployment on CPUs and NPUs. For CPUs, I converted the model to ONNX and tested on several Intel processors. The following table lists the CPU devices.
| Model | Base Frequency | Cores | Threads |
|---|---|---|---|
| Intel i9-13900 | 3.00 GHz | 24 | 32 |
| Intel i7-12700 | 2.40 GHz | 14 | 20 |
| Intel i5-9300 | 2.40 GHz | 4 | 8 |
SiamLT achieves 52 FPS on i9, 36 FPS on i7, and 23 FPS on i5. In contrast, MixFormerV2 and SiamBAN run at only 7 FPS and 5 FPS on i9. This shows that SiamLT is suitable for CPU-only UAV drone platforms.
For NPU deployment, I used two edge chips: Rockchip RK3588 and Huawei Ascend 310. RK3588 has 6 TOPS, and Ascend 310 has 8 TOPS. Both support INT8 quantization. Quantization reduces model size and computation. The quantization formula is:
$$ Q = \text{round}\left(\frac{X}{S}\right) + Z $$
$$ Q = \text{clip}\left(\text{round}\left(\frac{X}{S}\right)+Z, -2^{b-1}, 2^{b-1}-1\right) $$
$$ \text{clip}(x, min, max) = \begin{cases} min, & x < min \\ x, & min \le x \le max \\ max, & x > max \end{cases} $$
$$ X’ = S(Q – Z) = S\left( \text{clip}\left(\text{round}\left(\frac{X}{S}\right)+Z, -2^{b-1}, 2^{b-1}-1\right) – Z \right) $$
I used asymmetric quantization because weights are not always symmetrically distributed. I also used per-channel quantization for better accuracy. I compared three quantization algorithms: Normal, KL, and MMSE. The following table summarizes their characteristics.
| Property | Normal | KL | MMSE |
|---|---|---|---|
| Principle | Max/min | KL divergence | Minimum MSE |
| Outlier sensitivity | High | Low | Low |
| Complexity | Low | Medium | High |
| Quantization error | Large | Medium | Small |
| Use case | Uniform distribution | Non-uniform | High accuracy |
I used a calibration dataset from OTB with 100 image samples. The samples were cropped to \(320 \times 320\) and \(80 \times 80\). I separated the template and search branches during quantization to avoid redundant computation. The following table compares quantization algorithms on template and search branches.
| Algorithm | Template COS | Template EUC | Search COS | Search EUC |
|---|---|---|---|---|
| Normal | 0.9857 | 125.31 | 0.9953 | 0.6462 |
| KL | 0.9865 | 121.94 | 0.9952 | 0.7108 |
| MMSE | 0.9929 | 88.80 | 0.9968 | 0.5518 |
MMSE achieves the best similarity. I therefore used MMSE for final deployment. The following tables show operator statistics for the template and search branches on RK3588 NPU.
| Operator | Calls | CPU (us) | NPU (us) | Total (us) | Ratio (%) |
|---|---|---|---|---|---|
| Conv | 79 | 0 | 2298 | 2298 | 21.67 |
| exNorm | 38 | 0 | 1908 | 1908 | 17.99 |
| ConvRelu | 40 | 0 | 1209 | 1209 | 11.40 |
| Concat | 36 | 0 | 1105 | 1105 | 10.42 |
| exSDPAttention | 8 | 0 | 916 | 916 | 8.64 |
| ConvAdd | 21 | 0 | 760 | 760 | 7.17 |
| ConvExGelu | 8 | 0 | 390 | 390 | 3.68 |
| Reshape | 38 | 0 | 344 | 344 | 3.24 |
| Mul | 14 | 0 | 304 | 304 | 2.87 |
| AveragePool | 10 | 0 | 293 | 293 | 2.76 |
| Split | 8 | 0 | 281 | 281 | 2.65 |
| ConvClip | 7 | 0 | 271 | 271 | 2.56 |
| exGelu | 8 | 0 | 264 | 264 | 2.49 |
| Add | 11 | 0 | 250 | 250 | 2.36 |
| OutputOperator | 1 | 9 | 0 | 9 | 0.08 |
| InputOperator | 1 | 4 | 0 | 4 | 0.04 |
| Operator | Calls | CPU (us) | NPU (us) | Total (us) | Ratio (%) |
|---|---|---|---|---|---|
| exSDPAttention | 9 | 0 | 13875 | 13875 | 28.17 |
| exNorm | 43 | 0 | 11276 | 11276 | 22.89 |
| Conv | 91 | 0 | 5975 | 5975 | 12.13 |
| ConvRelu | 45 | 0 | 4639 | 4639 | 9.42 |
| Concat | 38 | 0 | 4022 | 4022 | 8.17 |
| ConvAdd | 23 | 0 | 2374 | 2374 | 4.82 |
| ConvExGelu | 9 | 0 | 2234 | 2234 | 4.54 |
| Reshape | 43 | 92 | 1086 | 1178 | 2.39 |
| AveragePool | 11 | 0 | 867 | 867 | 1.76 |
| Mul | 14 | 0 | 592 | 592 | 1.20 |
| Transpose | 3 | 0 | 527 | 527 | 1.07 |
| Add | 11 | 0 | 518 | 518 | 1.05 |
| Split | 9 | 0 | 467 | 467 | 0.95 |
| exGelu | 9 | 0 | 425 | 425 | 0.86 |
| ConvClip | 7 | 0 | 270 | 270 | 0.55 |
| OutputOperator | 2 | 13 | 0 | 13 | 0.03 |
| InputOperator | 2 | 5 | 0 | 5 | 0.01 |
The NPU handles nearly all computation, with CPU time close to zero. The search branch takes longer because of its larger input size and the cross-correlation and prediction head. The following table compares model format, size, accuracy, and latency.
| Format | Device | Weight Size (M) | AO (%) | SR0.5 (%) | SR0.75 (%) | Latency (ms) |
|---|---|---|---|---|---|---|
| PyTorch | RTX3090 GPU | 104.9 | 59.9 | 69.8 | 50.0 | 11.49 |
| ONNX | RK3588 CPU | 59.8 | 59.7 | 70.0 | 50.2 | 159.49 |
| RKNN | RK3588 NPU | 23.9 | 58.5 | 68.8 | 48.1 | 54.70 |
The quantized model runs at 20 FPS on RK3588 and 28 FPS on Ascend 310. With one Fusion Block, the frame rate increases to 27 FPS and 33 FPS. If I replace LMFF-Trans with a standard Transformer, the frame rate drops to 18 FPS and 8 FPS. This confirms the efficiency of LMFF-Trans for UAV drone edge deployment.
Quadrotor UAV Drone Platform
I designed and built a quadrotor UAV drone platform to validate the tracking system. The quadrotor uses four rotors in an X configuration. By adjusting rotor speeds, it can perform vertical take-off and landing, pitch, roll, and yaw. The flight principles are summarized in the following table.
| Motion | Rotor Adjustment | Result |
|---|---|---|
| Vertical | All rotors increase/decrease together | Ascend/descend/hover |
| Pitch | Increase front rotors, decrease rear rotors | Forward/backward |
| Roll | Increase right rotors, decrease left rotors | Left/right |
| Yaw | Increase diagonal rotors, decrease other diagonal | Rotate |
I used a cascade PID controller for stable flight. The outer loop generates desired angular velocity, and the inner loop controls motor speed. The PID output is:
$$ u(t) = K_p e(t) + K_i \int e(t) dt + K_d \frac{de(t)}{dt} $$
The cascade PID outer loop is:
$$ w(t) = K_{p,outer} e_\theta(t) + K_{i,outer} \int e_\theta(t) dt + K_{d,outer} \frac{de_\theta(t)}{dt} $$
The inner loop is:
$$ u(t) = K_{p,inner} e_w(t) + K_{i,inner} \int e_w(t) dt + K_{d,inner} \frac{de_w(t)}{dt} $$
The hardware includes an STM32F407 MCU, an RK3588 edge computing device, an ICM20602 six-axis sensor, an SPL06 barometer, and a W25Q32 flash memory. The MCU collects sensor data, performs attitude estimation, and generates PWM signals. The RK3588 processes camera images and runs SiamLT. The following table summarizes the platform components.
| Component | Model | Function |
|---|---|---|
| MCU | STM32F407 | Sensor fusion, attitude control, PWM generation |
| Edge device | RK3588 | Image acquisition and target tracking |
| IMU | ICM20602 | Angular velocity and acceleration |
| Barometer | SPL06 | Altitude estimation |
| Flash | W25Q32 | Parameter storage |
I conducted onboard tracking experiments in both normal and complex scenes. In the normal scene, the target was a pedestrian. The target was briefly occluded by trees, but SiamLT maintained accurate scale estimation and tracking at about 20 FPS. In the complex scene, the target was a vehicle with cluttered background and rapid aspect ratio changes. SiamLT still tracked the target accurately. These experiments confirm that the UAV drone system works reliably in real-world conditions.
Conclusion and Future Work
I designed a deep learning based target tracking system for UAV drone platforms. The proposed SiamLT tracker combines a lightweight GhostNet backbone, the LMFF-Trans multi-scale feature fusion network, and the DW_LMFF cross-correlation layer. I also used uncertainty-based loss weighting to balance classification and regression. Experiments on UAV123, LaSOT, and GOT-10k show that SiamLT achieves high accuracy with low computation. I further studied edge deployment and quantization. The quantized model runs in real time on CPU, RK3588 NPU, and Ascend 310 NPU. Finally, I built a quadrotor UAV drone platform and verified the system in onboard tracking experiments. The results show that the system is robust in both normal and complex scenes.
For future work, I plan to optimize NPU scheduling and computation graph partitioning to further improve inference speed. I also intend to integrate target position information with flight control to achieve autonomous target following and obstacle avoidance. Multimodal perception could further enhance the autonomy of UAV drone systems. I believe that lightweight deep learning trackers will play an increasingly important role in UAV drone applications.