Research on UAV Small Object Detection

Unmanned aerial vehicles, commonly referred to as UAVs, have become an indispensable tool for intelligent transportation, urban surveillance, and emergency response. The ability to extract accurate traffic information from aerial imagery is essential, and the detection of small objects from an unmanned aerial vehicle platform is one of the most challenging computer vision tasks. In this thesis, I present a systematic investigation into deep-learning-based small object detection methods tailored for unmanned aerial vehicle imagery. I propose a high-accuracy detection model, a task-oriented lightweight strategy, and an application system for traffic congestion evaluation. The entire workflow covers algorithm design, model compression, and real-world scenario verification.

1. Background and Problem Statement

With the rapid development of smart cities, intelligent transportation monitoring has become a critical component for mitigating urban traffic congestion and improving mobility. Traditional fixed cameras have limited coverage and are expensive to install and maintain. In contrast, an unmanned aerial vehicle offers a broad field of view, high maneuverability, and flexible deployment. These advantages make UAV-based traffic perception a promising supplement to conventional roadside sensing infrastructure.

However, aerial images captured by an unmanned aerial vehicle introduce severe challenges for object detection. Because the platform operates at high altitude, pedestrians and vehicles often occupy only a few pixels in the image. Formally, small objects are defined in the MS COCO dataset as objects with a pixel area smaller than \(32 \times 32\) pixels. In many UAV scenes, the actual target size is even smaller. In addition, targets are densely distributed, heavily occluded, and frequently confused with complex background structures such as trees, shadows, and road markings. Therefore, generic object detectors often suffer from missed detections and false alarms, making them unsuitable for accurate traffic flow statistics.

Another key constraint is the limited computational resources on board an unmanned aerial vehicle. Battery endurance, thermal dissipation, and payload weight impose strict limits on the embedded computing platform. High-accuracy deep learning models usually require substantial memory and computation, making real-time inference difficult on edge devices. Moreover, transmitting high-resolution video to ground stations introduces severe latency and privacy risks. Consequently, there is an urgent need for detection algorithms that are both accurate for small objects and efficient enough for deployment on UAV platforms.

2. Overview of Existing Work

Object detection has evolved from traditional hand-crafted feature methods to modern deep convolutional neural network paradigms. Early approaches used SIFT or HOG descriptors combined with support vector machines, which lacked robustness in complex aerial scenes. Later, two-stage detectors such as Faster R-CNN introduced region proposal networks and anchor boxes, achieving high localization accuracy but at the cost of speed. One-stage detectors, especially the YOLO family, directly regress bounding boxes and class probabilities, offering a much better trade-off between speed and accuracy for real-time applications.

In the context of small object detection from unmanned aerial vehicle imagery, feature pyramid networks, path aggregation networks, and attention mechanisms have been widely explored. Feature pyramids fuse high-resolution shallow features with semantically strong deep features, alleviating the loss of small-object information during downsampling. Attention modules like SE and CBAM re-calibrate channel and spatial features, enabling the network to focus on relevant regions. In recent years, Transformer-based detectors have also shown promising results by capturing global context, yet their computational cost remains prohibitive for embedded UAV platforms.

On the model-compression side, lightweight network architectures such as MobileNet, ShuffleNet, and GhostNet have been proposed to reduce parameters and multiply-accumulate operations. Additionally, model pruning, quantization, and knowledge distillation are widely adopted to compress existing models. However, most of these methods are designed for generic detection tasks and do not adequately account for the small-object characteristics in UAV imagery. Uniform pruning rates may accidentally remove important channels that encode fine-grained details in high-resolution feature maps, causing a severe performance drop. Therefore, I argue that lightweight strategies must be task-specific and resolution-aware when applied to unmanned aerial vehicle detection.

3. Proposed High-Accuracy Detection Network

In this section, I present a high-accuracy detection model named TSD-YOLO, which is built upon the state-of-the-art YOLOv11s baseline. The model consists of three improved components: a context-aware multi-scale backbone called CMSNet, an enhanced neck named ENeck, and an improved detection head called IHead. These components are designed together to address the challenge of detecting small, dense, occluded objects in unmanned aerial vehicle imagery.

3.1 Baseline: YOLOv11s

YOLOv11 is the latest iteration of the YOLO series. Its architecture follows the classic three-part structure: a backbone network for feature extraction, a neck network for multi-scale feature fusion, and a head network for final classification and regression. YOLOv11 introduces the C3k2 module, which replaces the earlier C2f module, and adds a C2PSA module at the end of the backbone to improve feature representation. The model uses an anchor-free detection head and a decoupled design for classification and regression. I selected YOLOv11s as the baseline because it offers a favorable balance between accuracy and efficiency, making it a reasonable starting point for UAV edge deployment.

Despite its strong performance on generic datasets, YOLOv11s struggles with small objects in UAV scenarios. The repeated downsampling operations reduce the spatial resolution of feature maps, which may erase the weak signals from tiny objects. In addition, the original neck fuses features at three scales — \(P_3\), \(P_4\), and \(P_5\), corresponding to 8×, 16×, and 32× downsampling—but shallow high-resolution features are often not sufficiently exploited. The detection head also lacks the ability to emphasize subtle target responses under cluttered backgrounds. These observations motivate my modifications.

3.2 CMSNet: Context-Aware Multi-Scale Backbone

To improve the joint modeling of local details and global context, I propose the Contextual Multi-Scale Convolution module, abbreviated as CMSConv. Based on this module, I construct a new backbone network called CMSNet. The core idea is to replace most of the standard convolution units in the original YOLOv11 backbone with CMSConv, while retaining the overall hierarchical structure. In the shallow layers, CMSConv focuses on preserving high-resolution local details; in the deeper layers, it aggregates multi-scale contextual information to enhance semantic representations.

The structure of CMSConv is composed of three parts: a base convolution unit, a multi-scale contextual feature extraction branch, and a global attention module. The input feature is first processed by a \(1 \times 1\) convolution to compress the channel dimension. Then, three parallel dilated convolutions with dilation rates \(d=1,2,3\) capture context at different receptive fields. The outputs are concatenated along the channel dimension, and a \(1 \times 1\) convolution is applied to fuse and compress the concatenated feature.

Let \(x\) denote the input feature map. The multi-scale contextual feature can be written as:

\[
F_{msc}(x) = \mathrm{Concat}\big(F_{d=1}(x), F_{d=2}(x), F_{d=3}(x)\big),
\]

where \(F_{d=k}(x)\) represents the output of a dilated convolution with dilation rate \(k\). The subsequent projection operation is defined as:

\[
F_{proj}(x) = W_{proj} \cdot F_{msc}(x),
\]

where \(W_{proj}\) denotes the weights of the \(1 \times 1\) projection convolution. After that, the module applies global average pooling to obtain the global feature descriptor:

\[
F_{gap}(x) = \frac{1}{H \cdot W} \sum_{i=1}^{H} \sum_{j=1}^{W} F_{proj}(x)_{i,j}.
\]

The global descriptor is then passed through two fully connected layers and a sigmoid activation to produce channel-wise attention weights \(A(x)\). The final output of the CMSConv is computed as:

\[
F_{out}(x) = A(x) \cdot F_{proj}(x).
\]

This design allows the backbone to simultaneously preserve fine-grained texture information and model long-range contextual dependencies. As a result, CMSNet produces more discriminative intermediate representations, which benefit small object detection in complex UAV images.

3.3 ENeck: Enhanced Cross-Scale Feature Fusion

The neck network is critical for multi-scale feature fusion. In the original YOLOv11, the neck adopts a combination of FPN and PAN structures. FPN conveys top-down semantic information to high-resolution layers, while PAN propagates localization cues bottom-up. Although this design is effective, the lack of a dedicated shallow high-resolution branch limits the detection of extremely small objects. Therefore, I introduce an enhanced neck, called ENeck, which adds a \(P_2\) branch into the feature pyramid. The \(P_2\) feature map has a downsampling factor of 4, preserving richer spatial details about edges, textures, and positions of small targets.

Another important improvement is the replacement of traditional bilinear interpolation upsampling with the dynamic upsampling module DySample. Unlike fixed interpolation, DySample generates content-adaptive sampling offsets through a lightweight linear layer. Let \(x\) be the input feature map. The offsets are computed as:

\[
I = \mathrm{Linear}(x).
\]

The final sampling set is formed by adding the offsets to the original sampling grid \(G\):

\[
S = G + I.
\]

Then the upsampled feature \(x’\) is obtained via grid sampling:

\[
x’ = \mathrm{grid\_sample}(x, S).
\]

To constrain the offset range and improve the initial sampling distribution, DySample uses bilinear interpolation to initialize the sampling points and multiplies the offsets by a constant scaling factor \(\alpha\). The dynamic range modulation is further expressed as:

\[
I = \sigma\big(\mathrm{Linear}_1(x)\big) \cdot \mathrm{Linear}_2(x),
\]

where \(\sigma\) denotes the sigmoid function. This mechanism enables the sampling process to adaptively adjust its receptive scope, leading to better upsampling quality for multi-scale feature maps.

The ENeck fuses features from \(P_2\), \(P_3\), \(P_4\), and the original deeper levels. In my final high-accuracy model, I keep all four scales to maximize detection performance. The enhanced fusion path strengthens the interaction between high-resolution details and deep semantic features, significantly reducing missed detections for small objects.

3.4 IHead: Improved Detection Head

The detection head governs the final classification and regression results. For small objects, a standard convolution head may not emphasize the weak responses sufficiently. I design IHead based on multi-scale attention mechanisms. The structure first generates query, key, and value features from the input feature map \(X’\):

\[
Q = W_q X’, \quad K = W_k X’, \quad V = W_v X’.
\]

I then apply dilated attention with multiple dilation rates to model spatial dependencies at different scales. The outputs from each dilation branch are concatenated along the channel dimension:

\[
F_{cat} = \mathrm{Concat}\big(F_1, F_2, F_3, F_4\big).
\]

After that, a projection module composed of a \(1 \times 1\) convolution, ReLU activation, and normalization compresses the fused feature. A squeeze-and-excitation module is then used to re-calibrate the channel responses, allowing the head to focus on target-relevant channels while suppressing background noise. Finally, dropout is applied for regularization, and the enhanced features are fed into two separate branches for bounding box regression and classification.

IHead improves the discriminative power of the detection head, especially in densely populated UAV scenes. It complements CMSNet and ENeck by enhancing the final prediction stage, forming a coherent architecture for small object detection.

4. Task-Oriented Lightweight Strategy for UAV Deployment

Although TSD-YOLO achieves high accuracy, its increased parameter count and computational cost hinder direct deployment on resource-constrained UAV platforms. To meet the requirements of edge devices such as Jetson Nano, Jetson Xavier NX, and Jetson Orin Nano, I propose a three-stage lightweight pipeline: scale-adaptive structure reconstruction, resolution-aware structured pruning, and channel-wise knowledge distillation recovery. This pipeline is designed with the small-object detection task in mind, rather than using generic compression techniques.

4.1 Scale-Adaptive Structure Reconstruction

In UAV aerial scenes, the target scale distribution is dominated by small and medium objects. The deep high-level feature map \(P_5\), which corresponds to 32× downsampling, mainly encodes large-object semantics. Owing to the low resolution, small-object information is nearly absent in \(P_5\). Therefore, I decide to remove the \(P_5\) detection branch from the original TSD-YOLO structure, focusing the computational budget on \(P_2\), \(P_3\), and \(P_4\). This process is not a random layer pruning but a task-oriented structure reallocation.

After removing \(P_5\), the neck starts the top-down fusion from \(P_4\), and the detection head only outputs predictions for three scales. This structural reconstruction reduces both parameters and FLOPs while preserving the capability to detect small objects because \(P_2\) is retained. The resulting model is referred to as TSD-YOLO(-P5). I use this model as the starting point for the subsequent pruning and distillation steps.

4.2 Resolution-Aware Structured Pruning

Even after removing \(P_5\), the remaining network still contains redundant channels. A straightforward approach would apply a uniform pruning ratio to all layers. However, such an approach may overly compress high-resolution feature layers that are crucial for small-object detection. I therefore introduce a resolution-aware pruning strategy that differentiates the pruning strength according to the spatial resolution of each feature layer.

First, I define a channel importance score. For a convolution layer followed by batch normalization, the scaling factor \(\gamma\) of the BN layer is used. For other layers, the magnitude of the convolution weights is used. Specifically, the importance score of the \(i\)-th channel in layer \(l\) is:

\[
S_l^{(i)} = \begin{cases}
|\gamma_l^{(i)}|, & \text{if BN exists}\\[4pt]
\sum_{r=1}^{C_{in}} \sum_{k=1}^{K} |W_l^{(i,r,k)}|, & \text{otherwise}
\end{cases}
\]

where \(C_{in}\) is the number of input channels and \(K\) is the kernel size. Within each module, I sort the channel importance scores and keep only the top-ranked channels according to the pruning ratio.

Second, I assign adaptive pruning ratios based on the feature-map resolution \(h \times w\). The pruning ratio function \(\rho(h)\) is defined in a piecewise manner:

\[
\rho(h) = \begin{cases}
0.15 \sim 0.35, & h \ge 160 \\
0.35 \sim 0.40, & 80 \le h < 160 \\
0.30 \sim 0.55, & 40 \le h < 80 \\
0.30 \sim 0.65, & h < 40
\end{cases}
\]

Here, the highest-resolution feature maps receive the most conservative pruning, while lower-resolution layers are compressed more aggressively. This prevents the loss of spatial details that matter most for small object localization.

Third, I implement module-level pruning to maintain structural consistency. In the C3k2/C2f modules, I prune only the hidden channels of the internal bottleneck without changing the final output width. In the detection head, I prune the intermediate hidden channels while keeping the final prediction dimension intact. For the CMSConv module, a joint pruning scheme is applied so that all coupled branches follow the same channel indices. All pruned channel counts are aligned to multiples of eight to ensure hardware-friendly inference.

4.3 Channel-Wise Knowledge Distillation Recovery

Direct pruning leads to a significant accuracy drop because the network weights are not adapted to the reduced channel capacity. To recover the lost feature representations, I adopt a channel-wise feature distillation approach. The teacher network is the unpruned TSD-YOLO(-P5), and the student network is the pruned lightweight model. Both networks share the same structural hierarchy, which makes intermediate feature alignment meaningful.

For a given channel \(c\) in the \(k\)-th layer, let \(T_c \in \mathbb{R}^{H \times W}\) be the teacher feature map and \(S_c\) be the student feature map. The spatial distribution is normalized using a softmax function:

\[
\phi(T_c) = \mathrm{softmax}\left(\frac{T_c}{\tau}\right) = \frac{\exp(T_{c,i}/\tau)}{\sum_{j=1}^{H \times W} \exp(T_{c,j}/\tau)},
\]

where \(\tau\) is a temperature parameter. The channel-wise distillation loss is then defined as the KL divergence between the teacher and student distributions averaged over all channels:

\[
L_{feat} = \frac{\tau^2}{C} \sum_{c=1}^{C} \sum_{i=1}^{H \times W} \phi(T_{c,i}) \log \frac{\phi(T_{c,i})}{\phi(S_{c,i})}.
\]

This loss encourages the student to mimic the spatial attention patterns of the teacher, rather than merely fitting the absolute values. Because small objects rely on fine spatial response peaks, this distribution alignment helps the student recover the key discriminative cues lost during pruning.

The total training objective combines the original detection loss \(L_{det}\) and the distillation loss over a set of selected layers \(\mathcal{L}\):

\[
L_{total} = L_{det} + \lambda \sum_{l \in \mathcal{L}} L_{feat}^{(l)},
\]

where \(\lambda\) is a balancing weight. During distillation training, the teacher parameters are frozen, and a \(1 \times 1\) convolution is inserted after the student feature maps to align the channel dimensions when necessary.

5. Experiments and Analysis

5.1 Datasets and Evaluation Metrics

I evaluate the proposed methods on the VisDrone2019 dataset, a widely used benchmark for UAV object detection. The dataset contains 8,599 images with over 540,000 annotated bounding boxes across 10 categories: pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus, and motor. The images are captured at various locations, altitudes, and lighting conditions, making the dataset highly representative of real unmanned aerial vehicle scenes. I also use the TinyPerson dataset as an additional benchmark to verify generalization towards extremely small pedestrian detection. In TinyPerson, the target sizes are much smaller, and the scenes are more challenging.

I use the following metrics: precision (\(P\)), recall (\(R\)), mean average precision at IoU threshold 0.5 (\(mAP@0.5\)), mean average precision averaged over IoU thresholds from 0.5 to 0.95 (\(mAP@0.5:0.95\)), parameter count, and GFLOPs. Precision and recall are defined as:

\[
P = \frac{TP}{TP + FP}, \qquad R = \frac{TP}{TP + FN}.
\]

The average precision for a single class is the area under the precision-recall curve:

\[
AP = \int_0^1 P(R) \, dR.
\]

The mean average precision is the average of AP over all classes.

5.2 High-Accuracy Detection Results

I compare TSD-YOLO with several state-of-the-art detection models on the VisDrone2019 validation set. All models are trained under identical settings: SGD optimizer, initial learning rate 0.01, cosine annealing schedule, and 300 epochs. The input size is consistent across experiments. Table 1 presents the comparison.

Model Precision (%) Recall (%) mAP@0.5 (%) mAP@0.5:0.95 (%)
YOLOv5s 51.3 38.5 39.6 23.7
YOLOv8s 49.9 38.5 39.2 23.6
YOLOv10s 51.6 38.3 39.8 23.9
YOLOv11s 51.0 39.0 40.0 24.0
TSD-YOLO 54.1 43.3 45.4 27.7

Table 1 shows that TSD-YOLO outperforms the baseline YOLOv11s by 3.1 percentage points in precision, 4.3 percentage points in recall, 5.4 percentage points in mAP@0.5, and 3.7 percentage points in mAP@0.5:0.95. The recall improvement is particularly important because it indicates fewer missed detections in densely populated UAV scenes. The improvement in mAP@0.5:0.95 also shows that my model produces more accurate localizations, which is essential for traffic applications.

5.3 Ablation Study

To verify the contribution of each proposed module, I conduct ablation experiments by gradually adding CMSNet, IHead, and ENeck to the baseline. The results are shown in Table 2.

YOLOv11s CMSNet IHead ENeck Precision (%) Recall (%) mAP@0.5 (%) mAP@0.5:0.95 (%)
√ 51.0 39.0 40.0 24.0
√ √ 50.7 40.0 40.9 24.5
√ √ √ 52.1 39.9 41.1 24.7
√ √ √ √ 54.1 43.3 45.4 27.7

As shown in Table 2, each component contributes positively to the overall performance. CMSNet improves recall and mAP, IHead improves precision and stability, and ENeck provides the largest gains, especially in recall. The combination of all three modules achieves the best results, demonstrating that the improvements are complementary.

I further compare CMSConv with other feature enhancement modules, including DWRSeg, ScConv, RFAConv, CAA, SPDConv, and ODConv. Table 3 reports the results.

Method Precision (%) Recall (%) mAP@0.5 (%) mAP@0.5:0.95 (%)
YOLOv11s 51.0 39.0 40.0 24.0
+DWRSeg 51.7 39.1 40.2 24.3
+ScConv 51.2 38.4 39.5 23.6
+RFAConv 51.3 39.4 40.0 23.9
+CAA 51.0 38.3 39.3 23.5
+SPDConv 51.0 37.8 39.1 23.5
+ODConv 50.0 38.8 39.6 23.6
+CMSConv (CMSNet) 50.7 40.0 40.9 24.5

Table 3 demonstrates that several existing modules do not generalize well to small-object detection on unmanned aerial vehicle imagery. Some even degrade the baseline performance. CMSConv, on the other hand, improves recall and mAP, confirming its suitability for the target task.

5.4 Per-Class Performance

Table 4 details the improvement for each category on the VisDrone2019 dataset. I compare YOLOv11s and TSD-YOLO in terms of precision, recall, and mAP@0.5:0.95.

Category YOLOv11s TSD-YOLO
Precision Recall AP50-95 Precision Recall AP50-95
pedestrian 53.5 39.3 19.8 58.2 47.8 25.4
people 52.8 29.4 12.6 55.3 39.5 17.5
bicycle 25.6 16.5 5.7 36.7 18.6 8.6
car 72.5 77.7 57.3 74.2 82.5 61.2
van 54.4 44.3 32.7 55.9 47.6 35.8
truck 54.2 37.5 24.5 55.5 36.3 26.1
tricycle 41.0 30.5 16.1 45.8 34.0 18.5
awning-tricycle 33.9 18.8 10.4 30.4 19.0 11.7
bus 71.8 49.7 40.8 73.3 55.0 46.5
motor 50.0 45.9 20.0 56.4 52.8 25.6

Most categories benefit from the improved architecture. In particular, small and densely distributed categories such as pedestrian, bicycle, and motor show substantial gains. The truck category has a slight recall decrease, but the overall trend strongly supports the effectiveness of the proposed model.

5.5 Cross-Dataset Generalization on TinyPerson

To test generalization beyond VisDrone2019, I evaluate both the baseline and TSD-YOLO on the TinyPerson dataset. All labels are treated as a single person class. Table 5 reports the results.

Model Precision (%) Recall (%) mAP@0.5 (%) mAP@0.5:0.95 (%)
YOLOv11s 40.0 21.0 19.7 6.9
TSD-YOLO 41.8 26.3 23.8 8.0

Although the absolute metrics are lower due to the extreme difficulty of TinyPerson, the proposed model still outperforms the baseline by a clear margin. This confirms that the improvements are not overfitted to a single dataset and can generalize to other small-object detection tasks.

6. Lightweight Model Results

In this section, I evaluate the proposed lightweight strategy step by step. First, I compare different structure reconstruction schemes. Table 6 shows the effect of removing different pyramid levels.

Model mAP@0.5 (%) mAP@0.5:0.95 (%) Parameters GFLOPs
TSD-YOLO 45.4 27.7 13.3M 38.1
TSD-YOLO(-P4,-P5) 44.9 27.4 8.3M 30.7
TSD-YOLO(-P5) 45.5 27.6 9.4M 34.4

Removing both \(P_4\) and \(P_5\) reduces complexity but also harms accuracy. Removing only \(P_5\) keeps the accuracy essentially unchanged while reducing parameters from 13.3M to 9.4M and GFLOPs from 38.1 to 34.4. Thus, TSD-YOLO(-P5) becomes the foundation for further compression.

Next, I apply resolution-aware pruning under three different intensities: light, medium, and strong. Table 7 compares the resulting models without fine-tuning.

Model mAP@0.5 (%) mAP@0.5:0.95 (%) Parameters GFLOPs
TSD-YOLO(-P5) 45.5 27.6 9.4M 34.4
Light pruning 9.1 4.2 7.6M 28.4
Medium pruning 2.4 0.8 7.0M 25.4
Strong pruning 0.0 0.0 6.6M 24.3

Clearly, pruning without retraining severely degrades the model. Strong pruning destroys the feature distribution completely. The medium-pruned model offers a reasonable trade-off between compression and remaining accuracy potential, so I choose it as the student model for distillation recovery.

Finally, I perform channel-wise knowledge distillation using the unpruned TSD-YOLO(-P5) as the teacher. Table 8 shows the restoration result.

Model mAP@0.5 (%) mAP@0.5:0.95 (%) Parameters GFLOPs
TSD-YOLO(-P5) 45.5 27.6 9.4M 34.4
Pruned (before fine-tuning) 2.4 0.8 7.0M 25.4
Pruned + distillation 43.6 26.1 7.0M 25.4

After distillation recovery, the lightweight model achieves 43.6% mAP@0.5 and 26.1% mAP@0.5:0.95. Compared with the original TSD-YOLO, the parameter count is reduced from 13.3M to 7.0M (a 47.4% reduction), and GFLOPs are reduced from 38.1 to 25.4 (a 33.3% reduction). The accuracy drop is only 1.8 percentage points in mAP@0.5 and 1.6 percentage points in mAP@0.5:0.95. This demonstrates that the proposed lightweight strategy successfully balances detection performance and computational cost for resource-constrained unmanned aerial vehicle platforms.

7. Application System for Traffic Perception

To validate the engineering usability of the improved algorithm, I design and implement a multi-object traffic perception and congestion evaluation system based on PyQt6. The system supports image and video inputs, online inference, interactive ROI selection, multi-object classification and counting, congestion-index computation, mixed-traffic risk warning, and result storage/export. It provides a complete closed loop from high-accuracy detection and lightweight deployment to traffic-scenario application.

The system architecture comprises three layers: the interface interaction layer, the business processing layer, and the data management layer. The interface allows users to load a model, open an image or video, adjust detection thresholds, select a ROI, and observe real-time metrics. The processing layer performs object detection, grouping, and congestion assessment. The data management layer uses SQLite to store running configurations, per-frame statistics, and historical results.

7.1 Detection and ROI Filtering

Given a frame, the detector outputs a set of detections \(D = (b_i, c_i, s_i)\), where \(b_i\) is the bounding box, \(c_i\) is the category, and \(s_i\) is the confidence score. The system filters detections whose confidence is below a threshold. To focus on a specific area, the user can draw a ROI rectangle with normalized coordinates.

For each target, its center point is computed as:

\[
p_i = \left( \frac{x_{1i} + x_{2i}}{2}, \frac{y_{1i} + y_{2i}}{2} \right).
\]

The target is considered inside the ROI if \(p_i\) lies within the ROI rectangle. This operation is efficient and robust to scale changes.

7.2 Multi-Object Grouping and Congestion Index

The system maps the 10 detection categories into three groups: pedestrian group, non-motorized vehicle group, and motorized vehicle group. The pedestrian group includes pedestrian and people; the non-motorized group includes bicycle, tricycle, and awning-tricycle; the motorized group includes car, van, truck, bus, and motor. After ROI filtering, the system counts the number of objects in each group.

To account for varying road-space usage, I define an equivalent vehicle number \(W\):

\[
W = \sum_{i=1}^{M} \mathbb{1}(p_i \in ROI) \cdot \mathbb{1}(c_i \in C_{motor}) \cdot w(c_i),
\]

where \(w(c_i)\) is a weight assigned to the vehicle category. The normalized equivalent vehicle number is:

\[
W_{norm} = \min\left(1,\frac{W}{W_{max}(ROI)}\right),
\]

where \(W_{max}(ROI)\) is the capacity parameter scaled by the ROI area:

\[
W_{max}(ROI) = W_{max}^{frame} \cdot \frac{A_{ROI}}{A_{frame}}.
\]

The spatial occupancy density \(D\) is defined as the ratio of the sum of intersection areas between target bounding boxes and the ROI to the ROI area:

\[
D = \frac{A_{occ}}{A_{ROI}}.
\]

Finally, the congestion index \(CI\) is computed as a weighted fusion:

\[
CI = \alpha W_{norm} + (1-\alpha)D,
\]

where \(\alpha \in [0,1]\) is a user-adjustable parameter. Based on two thresholds \(T_1\) and \(T_2\), the system outputs a traffic state: smooth, slow, or congested.

\[
State = \begin{cases}
\text{smooth}, & CI < T_1, \\
\text{slow}, & T_1 \le CI < T_2, \\
\text{congested}, & CI \ge T_2.
\end{cases}
\]

7.3 Mixed-Traffic Risk Warning

To capture potential conflicts between motor vehicles and vulnerable road users, I introduce a mixed-traffic risk indicator \(Risk\). Let \(P_{norm}\) and \(N_{norm}\) denote the normalized pedestrian and non-motorized vehicle counts. Then:

\[
Risk = \beta \cdot P_{norm} + (1-\beta) \cdot N_{norm},
\]

where \(\beta \in [0,1]\) balances the two factors. The risk level is classified as low, medium, or high using two thresholds.

7.4 System Interface and Data Management

The system interface is divided into three columns. The left column contains model loading, input selection, ROI controls, and parameter settings. The middle column displays the video canvas with overlaid detection results and ROI. The right column shows real-time evaluation outputs, multi-object statistics, detection details, and historical records. The system supports pause/resume, screenshot, and clear ROI operations.

All running results are stored in a SQLite database. Each run is assigned a unique run ID. The system stores the model path, input source, parameters, ROI configuration, per-frame counts, congestion index, state, and inference latency. Historical records can be queried and exported as CSV files. This functionality enables reproducible experiments and comparative analysis across different model versions and configurations.

The implemented system demonstrates that the proposed high-accuracy model and the lightweight model can be successfully integrated into an application prototype. Through the use of interactive ROI selection and congestion indicator calculation, the system transforms raw detection results into meaningful traffic information. This provides evidence that the proposed methods are not only theoretically sound but also practically valuable for intelligent transportation with unmanned aerial vehicles.

8. Conclusion and Future Work

In this thesis, I have presented a comprehensive investigation into deep-learning-based small object detection methods for unmanned aerial vehicle imagery. I first proposed a high-accuracy detection model, TSD-YOLO, which integrates a context-aware multi-scale backbone, an enhanced feature pyramid neck, and a multi-scale attention detection head. Extensive experiments on the VisDrone2019 dataset demonstrate that TSD-YOLO significantly outperforms the baseline YOLOv11s and other state-of-the-art models in detecting small, dense, and occluded objects.

Second, I developed a task-oriented lightweight pipeline to deploy the improved model on resource-constrained devices. By removing the redundant deep pyramid branch, applying resolution-aware structured pruning, and recovering feature representations through channel-wise knowledge distillation, I obtained a lightweight model with only 7.0M parameters and 25.4 GFLOPs, while maintaining 43.6% mAP@0.5 and 26.1% mAP@0.5:0.95. This demonstrates a favorable trade-off between detection accuracy and computational cost for unmanned aerial vehicle platforms.

Third, I implemented a PyQt6-based traffic perception system that uses the trained models to detect vehicles, pedestrians, and other road users from UAV images. The system supports ROI selection, congestion index computation, mixed-traffic risk assessment, and historical data management. The successful deployment of the system verifies the engineering applicability of the proposed methods.

There are still several limitations. The experiments are mainly conducted on two public datasets, and real-world UAV on-board deployment has not been fully evaluated. In the future, I plan to test the models on actual embedded platforms such as Jetson devices, measure inference latency and power consumption, and apply TensorRT acceleration. I also intend to expand the application system to support real-time online analysis under realistic flight conditions. The ultimate goal is to establish a complete and reliable unmanned aerial vehicle-based intelligent transportation solution for smart cities.

In summary, the methods and systems presented in this thesis provide a solid foundation for the practical use of small object detection in unmanned aerial vehicle imagery, with promising directions for future research and development.

Scroll to Top