In recent years, the rapid advancement of unmanned aerial vehicles (UAVs) has revolutionized numerous civilian and military applications, including public security, disaster response, urban surveillance, and precision agriculture. UAVs equipped with high-resolution cameras can capture extensive aerial imagery from dynamic perspectives, enabling wide-area situational awareness. However, the unique characteristics of UAV imagery — such as small object sizes, dense target distributions, complex backgrounds, and significant scale variations — pose severe challenges to conventional object detection algorithms. Moreover, UAV platforms are typically resource-constrained edge devices with limited computational power and memory, making it difficult to directly deploy heavy deep learning models. Therefore, the research on achieving both high detection accuracy and real-time inference on UAVs has become a critical and challenging topic.
In this thesis, I focus on addressing these challenges from two complementary angles: (1) designing a high-performance teacher network that is capable of extracting rich multi-scale features, and (2) devising a tailored knowledge distillation framework that transfers the learned knowledge to a lightweight student network without sacrificing inference speed. The proposed methods are evaluated on the VisDrone2019 dataset and the UAVDT dataset, demonstrating significant improvements in detection accuracy while maintaining real-time performance.
Introduction and Motivations
The rapid deployment of unmanned aerial vehicles in real-world missions demands robust and efficient visual perception systems. Object detection from a UAV perspective differs fundamentally from ground-level detection because of several reasons. First, objects in aerial images often occupy only a few pixels; for example, a pedestrian might be represented by a mere 10×20 patch in a 1920×1080 image. Second, the background is highly cluttered with buildings, trees, vehicles, and shadows, which can easily confuse feature extraction. Third, the viewing angle changes abruptly as the UAV maneuvers, leading to large variations in object appearance and orientation. Finally, the strict latency and power constraints of embedded platforms require models that run at high frame rates while consuming minimal energy.
Traditional machine learning approaches relied on hand-crafted features such as HOG, SIFT, and LBP combined with classifiers like SVM or AdaBoost. These methods suffer from limited representational capacity and poor generalization to complex aerial scenes. The emergence of deep convolutional neural networks (CNNs) has significantly advanced the field. Single-stage detectors such as YOLO series, SSD, and two-stage detectors such as Faster R-CNN and Cascade R-CNN have been widely explored. However, most of them are designed for general object detection and may not be optimal for UAV imagery. In particular, the repeated strided convolutions and pooling operations in standard backbones tend to discard fine-grained spatial information that is critical for small object detection. Furthermore, the large model size and high computational cost of accurate detectors hinder their deployment on UAV onboard computers.
To tackle these issues, I present a comprehensive solution consisting of two main components:
- Multi-Scale Feature Enhancement (MSFE): A novel YOLOv10-based detector, named YOLOv10s-WRS, that integrates wavelet convolution, receptive-field and attention modules, and a small-object-oriented feature pyramid.
- Selective Knowledge Distillation: A distillation framework that transfers the knowledge from YOLOv10s-WRS to a lightweight YOLOv10n-WRS student via cross-dimensional feature reconstruction, dynamic mask focusing, and adaptive decoupled loss.
The central theme of this thesis is to demonstrate that a carefully designed feature enhancement scheme for the teacher network, combined with a selective and robust distillation mechanism, can produce a highly accurate yet lightweight detector suitable for unmanned aerial vehicles.
Related Work
Object Detection in UAV Imagery
Research on UAV object detection has followed two major paradigms. The first paradigm consists of two-stage detectors like Faster R-CNN and Cascade R-CNN, which first generate region proposals and then refine their classifications and locations. These methods often achieve higher accuracy but at the cost of slower inference. The second paradigm employs one-stage detectors, such as SSD and the YOLO family, which directly predict bounding boxes and class probabilities in a single pass. Their efficiency makes them attractive for real-time applications. However, their accuracy on small objects remains a challenge due to the lack of high-resolution features in the deep layers.
Many recent studies have proposed modifications specifically for UAV imagery. For instance, some have introduced feature pyramid networks with extra P2 levels to preserve spatial details, while others have incorporated attention mechanisms to suppress background clutter. Dynamic multi-scale fusion and adaptive weighted feature fusion have also been explored. Nevertheless, most of these improvements increase the model complexity, making deployment harder. This raises the need for knowledge distillation methods that can compress a sophisticated model into a lightweight one without losing its representational power.
Knowledge Distillation
Knowledge distillation (KD) is a model compression technique that transfers knowledge from a large teacher model to a small student model. The standard KD framework aligns the softened output probabilities of the teacher and student, enabling the student to mimic the teacher’s decision boundaries. However, for object detection, the output includes both category probabilities and bounding box coordinates, and the intermediate features contain valuable spatial information. Thus, feature-based and relation-based distillation methods have been developed. Feature-based methods align the feature maps of the teacher and student at selected layers, while relation-based methods distill structural relationships between samples or channels. In the context of UAV detection, what matters most is to effectively transfer the teacher’s ability to localize tiny objects and to distinguish them from complex backgrounds. Traditional feature alignment approaches often suffer from channel mismatch and background noise dominance. To overcome those pitfalls, I propose a selective knowledge distillation framework that incorporates both spatial masks and task-decoupled losses.
Proposed Teacher Model: YOLOv10s-WRS
In order to achieve a strong teacher model, I improve the baseline YOLOv10 architecture in three key aspects. The overall structure of YOLOv10s-WRS is shown below. The model consists of an enhanced backbone, a reconstructed neck, and the original decoupled head. The improvements are designed to enhance the representation of high-frequency details, adaptively focus on informative regions, and facilitate multi-scale feature interaction without introducing excessive computational overhead.
Wavelet Convolution in C2f Blocks
Standard convolution operations in CNNs naturally emphasize low-frequency components while often suppressing high-frequency information. In aerial images, high-frequency edges and textures are extremely important for recognizing small objects. To preserve these details, I introduce wavelet convolution (WTConv) into the C2f modules of the backbone. The improved module is denoted as C2f_WTConv. Given an input feature tensor \( \mathbf{X} \in \mathbb{R}^{C \times H \times W} \), I first apply a two-dimensional discrete wavelet transform (DWT) to decompose it into four subbands:
$$
\mathbf{X}_{LL}, \mathbf{X}_{LH}, \mathbf{X}_{HL}, \mathbf{X}_{HH} = \text{DWT}(\mathbf{X}),
$$
where \( \mathbf{X}_{LL} \) represents the low-frequency approximation capturing the overall structure, while \( \mathbf{X}_{LH} \), \( \mathbf{X}_{HL} \), and \( \mathbf{X}_{HH} \) represent detail coefficients in horizontal, vertical, and diagonal directions. Each subband has half the spatial resolution but four times the channel capacity after concatenation. In the wavelet-enhanced convolution, I apply depthwise convolution to the concatenated subband features to model frequency-specific information, followed by learnable channel-wise scaling that adaptively weights the contribution of each frequency band:
$$
\hat{\mathbf{X}} = \text{DWConv}([\mathbf{X}_{LL}; \mathbf{X}_{LH}; \mathbf{X}_{HL}; \mathbf{X}_{HH}]) \cdot \mathbf{s},
$$
where \( \mathbf{s} \) is a learnable scaling vector and \( [\cdot] \) denotes channel concatenation. After processing the frequency-domain components, I apply the inverse wavelet transform (IWT) to reconstruct a feature map with the original resolution:
$$
\mathbf{X}_{\text{wavelet}} = \text{IWT}(\hat{\mathbf{X}}).
$$
Finally, the output of the wavelet branch is combined with a residual connection from the original spatial branch:
$$
\mathbf{X}_{\text{out}} = \mathbf{X} + \mathcal{F}_{\text{spatial}}(\mathbf{X}) + \mathcal{F}_{\text{wavelet}}(\mathbf{X}),
$$
where \( \mathcal{F}_{\text{spatial}} \) is a standard convolution stream and \( \mathcal{F}_{\text{wavelet}} \) is the wavelet-enhanced stream. The C2f_WTConv module thus expands the effective receptive field while explicitly modeling high-frequency textual information, which is crucial for detecting small objects embedded in cluttered backgrounds.
RFCBAMConv: Receptive Field and Channel–Spatial Attention Convolution
To further improve the discriminative ability of the backbone, I design a novel convolutional module called RFCBAMConv. This module introduces variable receptive field modeling and multi-dimensional attention mechanisms into a single convolutional block. The structure is depicted in the figure above (middle part). Given an input \( \mathbf{X} \in \mathbb{R}^{C \times H \times W} \), I first generate multiple receptive-field features using grouped depthwise convolutions with different dilation rates. Let \( K(\cdot) \) denote a depthwise convolution with kernel size \( k \) and expansion factor \( k^2 \) in the channel dimension. The expanded feature is then rearranged to explicitly separate spatial and receptive-field dimensions:
$$
\mathbf{F} = \text{rearrange}(\text{DWConv}(\mathbf{X}), C \times k^2 \times H \times W \rightarrow C \times k^2 \times H \times W).
$$
After the expansion, a Squeeze-and-Excitation (SE) block calculates channel attention weights \( \mathbf{w}_c \in \mathbb{R}^{C} \) to emphasize important channels:
$$
\mathbf{w}_c = \sigma(\mathbf{W}_2 \cdot \delta(\mathbf{W}_1 \cdot \text{GAP}(\mathbf{F}))),
$$
where GAP is global average pooling, \( \delta \) is ReLU, and \( \sigma \) is sigmoid. The weighted feature is then passed through a spatial attention branch that applies max pooling and average pooling along the channel dimension and concatenates them. A \( 3 \times 3 \) convolution generates a spatial attention map \( \mathbf{A}_s \). The entire module can be summarized as:
$$
\mathbf{Y} = \text{Conv}_{1 \times 1} (\mathbf{A}_s \otimes (\mathbf{w}_c \otimes \mathbf{F})).
$$
RFCBAMConv enables the network to dynamically select the most relevant receptive field for each spatial location, thereby reducing background interference and enhancing small object features. It serves as an effective drop-in replacement for certain convolutional layers in the backbone, improving both precision and robustness.
SOEP: Small Object Enhanced Feature Pyramid
The standard feature pyramid network (FPN) in YOLOv10 uses features from P3, P4, and P5 detection layers. While P3 has relatively high resolution, the detection of extremely small objects benefits from even higher spatial details. Directly adding a P2 layer increases the computational cost and slows down inference. Instead, I propose a Small Object Enhanced Pyramid (SOEP) that reuses the high-resolution P2 features and enhances them via a Space-to-Depth Convolution (SPDConv) and a CSP-OmniKernel fusion module.
First, the P2 feature \( \mathbf{F}_{P2} \in \mathbb{R}^{H \times W \times C} \) is processed by an SPD (space-to-depth) transformation. The SPD operation rearranges spatial blocks into the channel dimension:
$$
\mathbf{F}_{spd} = \text{SPD}(\mathbf{F}_{P2}, r),
$$
where \( r \) is the scale factor. Then a pointwise convolution is applied to compress the channels and retain high-resolution information:
$$
\mathbf{F}_{enh} = \phi(\mathbf{W}_{spd} \star \mathbf{F}_{spd}),
$$
where \( \phi \) is the activation function and \( \star \) denotes the convolution operation.
Second, to merge multi-scale contextual information, I introduce a CSP-OmniKernel module. The input feature \( \mathbf{F}_{enh} \) is split into two branches along the channel dimension:
$$
\mathbf{F}_1, \mathbf{F}_2 = \text{Split}(\mathbf{F}_{enh}),
$$
where \( \mathbf{F}_1 \) is passed through a direct shortcut to preserve original information, and \( \mathbf{F}_2 \) is processed by an OmniKernel that contains a global branch, a large-kernel branch, and a local branch:
$$
\mathbf{F}_2′ = \mathcal{K}_g(\mathbf{F}_2) + \mathcal{K}_l(\mathbf{F}_2) + \mathcal{K}_s(\mathbf{F}_2).
$$
Finally, the outputs are concatenated and fused to generate the enhanced feature:
$$
\mathbf{F}_{out} = \text{Conv}(\mathbf{F}_1 \oplus \mathbf{F}_2′).
$$
The SOEP structure improves the preservation of small target information without adding an extra detection head, thereby keeping the computational cost low while substantially boosting detection performance on small objects.
Proposed Knowledge Distillation Framework
After obtaining a strong teacher model YOLOv10s-WRS, I aim to transfer its knowledge to a lightweight student model YOLOv10n-WRS. The student is built by applying the same modifications to YOLOv10n, but since YOLOv10n has fewer channels and layers, its representation capacity is lower. Directly mimicking the teacher’s features is ineffective due to channel dimension mismatch and the overwhelming presence of background information in aerial images. Therefore, I propose a selective knowledge distillation framework that consists of three components: cross-dimensional multi-scale semantic feature reconstruction, dynamic mask-focused distillation, and adaptive decoupled distillation loss. The framework is illustrated in the diagram.
Cross-Dimensional Multi-Scale Semantic Feature Reconstruction
The teacher and student networks share the same YOLOv10 architecture family, which means their feature maps at corresponding layers have the same spatial resolution, but the channel numbers differ. To align the channel dimensions, I insert a 1×1 convolutional projection layer after the student’s selected feature maps. This projection does not alter spatial resolution, ensuring that fine-grained spatial information is strictly preserved. Mathematically, let the teacher’s feature at scale l be \( \mathbf{F}_t^{(l)} \in \mathbb{R}^{C_t \times H \times W} \) and the student’s feature be \( \mathbf{F}_s^{(l)} \in \mathbb{R}^{C_s \times H \times W} \). The projected student feature is computed as:
$$
\tilde{\mathbf{F}}_s^{(l)}(i,j) = \mathbf{W}^{(l)} \mathbf{F}_s^{(l)}(i,j) + \mathbf{b}^{(l)},
$$
where \( \mathbf{W}^{(l)} \in \mathbb{R}^{C_t \times C_s} \) is the learnable transformation matrix. Since the 1×1 convolution operates on each spatial position independently, the spatial alignment remains exact:
$$
\text{res}(\tilde{\mathbf{F}}_s^{(l)}) = \text{res}(\mathbf{F}_t^{(l)}).
$$
This projection is performed on three scales (P3, P4, P5), enabling the student to learn high-level semantic knowledge from the teacher at multiple resolutions.
Dynamic Mask-Focused Distillation
Aerial images contain vast amounts of background regions such as roads, trees, and buildings. When the student mimics the teacher on all pixels, the training objective is dominated by background loss, which reduces the model’s focus on small foreground objects. To address this, I design a dynamic mask that is generated from the teacher’s response map. Specifically, the teacher’s feature map at a certain layer is averaged across channels after taking the absolute value:
$$
\mathbf{M}(h,w) = \frac{1}{C_t} \sum_{c=1}^{C_t} \left| \mathbf{F}_t^{(l)}(h,w,c) \right|.
$$
This response map indicates where the teacher concentrates its attention. I normalize the mask to [0,1]:
$$
\tilde{\mathbf{M}}(h,w) = \frac{\mathbf{M}(h,w) – \min \mathbf{M}}{\max \mathbf{M} – \min \mathbf{M} + \epsilon}.
$$
Then, the feature distillation loss is computed by weighting the per-pixel mean squared error with the mask:
$$
\mathcal{L}_{\text{feat}} = \frac{1}{H W} \sum_{h,w} \tilde{\mathbf{M}}(h,w) \cdot \left\| \mathbf{F}_t^{(l)}(h,w) – \tilde{\mathbf{F}}_s^{(l)}(h,w) \right\|_2^2.
$$
By applying this spatial weighting, the student model is forced to focus on regions with high teacher response, which usually correspond to foreground objects. Background gradient contributions are suppressed, improving the efficiency of knowledge transfer.
Adaptive Decoupled Distillation Loss
The output of the detector consists of a classification branch and a regression branch. For the classification branch, teacher logits contain rich inter-class relationships. I use a temperature-scaled softmax to produce soft labels:
$$
p_t^{(c)} = \frac{\exp(z_t^{(c)} / T)}{\sum_{j=1}^{C} \exp(z_t^{(j)} / T)}, \quad p_s^{(c)} = \frac{\exp(z_s^{(c)} / T)}{\sum_{j=1}^{C} \exp(z_s^{(j)} / T)},
$$
where \( T \) is the temperature, set to 3 in my experiments. The classification distillation loss is defined as the Kullback-Leibler divergence between the two distributions:
$$
\mathcal{L}_{\text{cls}} = T^2 \cdot \frac{1}{N} \sum_{i=1}^{N} \sum_{c=1}^{C} p_t^{(c)}(i) \log \frac{p_t^{(c)}(i)}{p_s^{(c)}(i)}.
$$
For the regression branch, YOLOv10 uses feature maps to predict the probability distribution over discrete offsets for each bounding box. To transfer the localization knowledge, I minimize the KL divergence between the teacher’s and projected student’s box distributions:
$$
\mathcal{L}_{\text{reg}} = \frac{1}{N} \sum_{i=1}^{N} \sum_{k \in \{x,y,w,h\}} \mathcal{D}_{KL}(\beta_t^k(i) \| \beta_s^k(i)),
$$
where \( \beta_t^k \) and \( \beta_s^k \) are the discrete probability distributions for the \( k \)-th coordinate. This distribution-based regression distillation provides smoother gradients than direct coordinate regression, especially for overlapping small objects.
The total distillation loss is a weighted combination of the feature loss and the decoupled head loss:
$$
\mathcal{L}_{\text{KD}} = \lambda_{\text{feat}} \mathcal{L}_{\text{feat}} + \lambda_{\text{cls}} \mathcal{L}_{\text{cls}} + \lambda_{\text{reg}} \mathcal{L}_{\text{reg}}.
$$
In my implementation, I set \( \lambda_{\text{feat}} = 1.0 \), \( \lambda_{\text{cls}} = 0.5 \), and \( \lambda_{\text{reg}} = 0.5 \). The overall training objective for the student model is the sum of the original detection loss and the distillation loss.
Experiments and Results
Dataset and Evaluation Metrics
I evaluate the proposed methods on the VisDrone2019 dataset, which contains 8,629 UAV images spanning 14 cities with diverse scenes, including urban, suburban, and rural areas. The images are captured under varying conditions of illumination, weather, and altitude. There are ten object categories: pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus, and motor. The dataset is split into a training set of 6,471 images, a validation set of 548 images, and a test set of 1,610 images. To assess generalization, I also conduct experiments on the UAVDT dataset.
The evaluation metrics include precision (P), recall (R), mAP@0.5, and mAP@0.5:0.95. Precision and recall are defined as:
$$
\text{Precision} = \frac{TP}{TP + FP}, \quad \text{Recall} = \frac{TP}{TP + FN}.
$$
mAP@0.5 is the mean average precision computed with an IoU threshold of 0.5, and mAP@0.5:0.95 is the mean of AP values over IoU thresholds from 0.5 to 0.95 with a step size of 0.05. I also report the number of parameters, GFLOPs, and FPS (frames per second) to assess model efficiency.
Implementation Details
All experiments are conducted on an NVIDIA RTX 4090 GPU with 24GB memory. The deep learning framework is PyTorch 2.0.0 with CUDA 11.8. Input images are resized to 640×640 pixels. The batch size is set to 8, and the model is trained for 200 epochs. The initial learning rate is 0.01 with a momentum of 0.937 and weight decay of 0.0005 using SGD optimizer. For distillation, the teacher model YOLOv10s-WRS is trained first with the same settings, and then its weights are frozen during student training. The temperature for soft label distillation is set to 3. The experimental environment is summarized in the following table.
| Category | Item | Value |
|---|---|---|
| Hardware | GPU | NVIDIA RTX 4090 (24 GB) |
| Software | OS | Ubuntu 20.04 |
| Software | Python / PyTorch | Python 3.9 / PyTorch 2.0.0 |
| Software | CUDA | 11.8 |
| Training | Input resolution | 640×640 |
| Training | Batch size | 8 |
| Training | Epochs | 200 |
| Training | Optimizer | SGD (momentum=0.937) |
| Training | Initial LR | 0.01 |
| Training | Weight decay | 0.0005 |
| Distillation | Teacher state | Frozen |
| Distillation | Temperature | 3 |
| Distillation | Distilled layers | Feature maps + logits |
Performance of the Teacher Model YOLOv10s-WRS
I first compare the proposed teacher model with the baseline YOLOv10s on the VisDrone2019 validation set. The results are reported in the following table.
| Method | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) |
|---|---|---|---|---|
| YOLOv10s | 48.6 | 38.6 | 39.3 | 23.3 |
| YOLOv10s-WRS | 54.6 | 41.6 | 43.6 | 26.0 |
The improved teacher model achieves a substantial gain of 4.3 percentage points in mAP@0.5 and 2.7 points in mAP@0.5:0.95 compared to the baseline. Precision improves by 6.0 points and recall by 3.0 points. These improvements demonstrate the effectiveness of the multi-scale feature enhancement techniques for UAV imagery.
To further analyze the improvements by category, I list the per-class results of the baseline and the proposed teacher model in the following two tables.
| Class | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) |
|---|---|---|---|---|
| pedestrian | 53.1 | 39.5 | 43.0 | 19.6 |
| people | 52.8 | 31.5 | 34.7 | 14.0 |
| bicycle | 24.9 | 16.9 | 13.2 | 5.4 |
| car | 72.3 | 77.3 | 80.3 | 57.2 |
| van | 51.8 | 45.0 | 45.7 | 31.8 |
| truck | 46.3 | 32.1 | 34.2 | 21.8 |
| tricycle | 38.9 | 29.2 | 27.1 | 14.5 |
| awning-tricycle | 32.8 | 19.0 | 16.6 | 10.1 |
| bus | 62.6 | 48.7 | 53.1 | 38.6 |
| motor | 50.0 | 46.9 | 45.6 | 20.0 |
| Class | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) |
|---|---|---|---|---|
| pedestrian | 60.6 | 41.9 | 47.7 | 22.6 |
| people | 58.5 | 32.5 | 37.9 | 15.6 |
| bicycle | 31.8 | 19.5 | 17.3 | 7.61 |
| car | 78.8 | 77.4 | 82.6 | 59.7 |
| van | 56.6 | 47.7 | 50.1 | 34.8 |
| truck | 54.2 | 38.5 | 41.2 | 25.9 |
| tricycle | 45.5 | 33.6 | 33.2 | 18.2 |
| awning-tricycle | 35.1 | 22.0 | 18.2 | 11.3 |
| bus | 69.9 | 54.0 | 58.8 | 41.2 |
| motor | 55.4 | 48.6 | 49.6 | 23.2 |
It is apparent that all classes benefit from the proposed modifications. For example, the mAP@0.5 of the bicycle class improves from 13.2% to 17.3%, and that of the truck class improves from 34.2% to 41.2%. These gains confirm that the wavelet convolution and attention modules enhance high-frequency detail extraction, while the SOEP pyramid preserves small object information.
Comparison with Other Detectors
I compare my teacher model against several state-of-the-art detectors under the same training settings. The comparison is summarized in the table below.
| Model | mAP@0.5 (%) | mAP@0.5:0.95 (%) | Params (M) | GFLOPs (G) |
|---|---|---|---|---|
| Faster R-CNN | 29.5 | 15.2 | 41.5 | 206 |
| Cascade R-CNN | 31.8 | 16.5 | 69.1 | 235 |
| YOLOv5s | 36.2 | 19.2 | 7.2 | 16.5 |
| YOLOv7-tiny | 35.8 | 18.9 | 6.2 | 13.8 |
| YOLOv8s | 40.5 | 24.1 | 11.2 | 28.6 |
| YOLOv10s | 39.3 | 23.3 | 7.2 | 21.6 |
| YOLOv10s-WRS (Ours) | 43.6 | 26.0 | 9.5 | 35.8 |
The two-stage detectors achieve much lower accuracy while having a significantly larger number of parameters. Lightweight models like YOLOv5s and YOLOv7-tiny have fewer parameters but their accuracy is limited. My YOLOv10s-WRS achieves the highest mAP among all listed methods, with a moderate increase in computational cost compared to YOLOv10s. This teacher model serves as a strong knowledge source for distillation.
Ablation Study on Feature Enhancements
To verify the contribution of each proposed module, I conduct step-by-step ablation experiments on the teacher model. The results are shown in the following table.
| Baseline | C2f_WTConv | RFCBAMConv | SOEP | P (%) | R (%) | mAP@0.5 (%) |
|---|---|---|---|---|---|---|
| √ | 48.6 | 38.6 | 39.3 | |||
| √ | √ | 51.4 | 40.2 | 41.8 | ||
| √ | √ | √ | 53.2 | 41.1 | 42.9 | |
| √ | √ | √ | √ | 54.6 | 41.6 | 43.6 |
Each module contributes positively. Adding C2f_WTConv yields a 2.5 point improvement in mAP@0.5, confirming the value of high-frequency information. RFCBAMConv further boosts mAP@0.5 by 1.1 points, demonstrating its ability to suppress background interference. Finally, SOEP improves mAP@0.5 by another 0.7 points, showing that enhanced high-resolution feature fusion helps small object detection.
Generalization Experiment on UAVDT Dataset
To investigate whether the proposed improvements generalize to other UAV datasets, I evaluate both YOLOv10s and YOLOv10s-WRS on the UAVDT dataset. The results are reported below.
| Method | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) |
|---|---|---|---|---|
| YOLOv10s | 49.4 | 37.7 | 38.6 | 23.2 |
| YOLOv10s-WRS | 51.4 | 41.3 | 42.6 | 25.5 |
The consistent improvement on UAVDT validates that the proposed multi-scale feature enhancement is not dataset-specific and can adapt to diverse aerial scenarios, which is essential for practical unmanned aerial vehicles deployments.
Knowledge Distillation Results
Now I evaluate the proposed knowledge distillation method. The teacher is the pretrained YOLOv10s-WRS, and the student is YOLOv10n-WRS initialized from scratch. The results are compared against the baseline student without distillation and the teacher model.
| Experiment | Core strategy | mAP@0.5 (%) | Params (M) | GFLOPs (G) | FPS |
|---|---|---|---|---|---|
| Student (YOLOv10n-WRS) | Baseline | 38.8 | 2.8 | 11.7 | 126 |
| Teacher (YOLOv10s-WRS) | High capacity | 43.6 | 9.5 | 35.8 | 91 |
| Distilled student (Ours) | Selective KD | 41.5 | 2.8 | 11.7 | 126 |
After applying my distillation framework, the student model improves from 38.8% to 41.5% in mAP@0.5, an absolute gain of 2.7 percentage points. The parameter count and computational cost remain unchanged at 2.8M parameters and 11.7 GFLOPs, and the inference speed stays at 126 FPS, which is much faster than the teacher’s 91 FPS. This demonstrates that the proposed KD method successfully transfers critical knowledge while preserving real-time efficiency.
The convergence curves of mAP@0.5 during training are shown in the figure. The teacher model reaches the highest accuracy, while the student without distillation lags behind. Our distilled student approaches the teacher’s performance closely.
Comparison with Other Distillation Methods
To prove the superiority of my KD approach, I compare it with several mainstream knowledge distillation methods. All methods use the same teacher and student networks. The results are as follows.
| Teacher | Student | Distillation method | mAP@0.5 (%) |
|---|---|---|---|
| YOLOv10s-WRS | YOLOv10n-WRS | KD | 39.8 |
| FitNet | 40.1 | ||
| FGD | 41.0 | ||
| CWD | 40.8 | ||
| MGD | 41.2 | ||
| AT | 40.6 | ||
| Ours | 41.5 |
My method achieves the highest mAP among all distillation approaches, outperforming the recent strong methods MGD and FGD. This improvement is attributed to the dynamic mask that suppresses background gradients and the decoupled loss that handles classification and localization separately, both of which are specially designed for UAV-specific challenges.
Ablation Study on Knowledge Distillation
I perform an ablation study to evaluate the contribution of each KD component. The results are listed in the table below.
| Experiment | Feature Reconstruction | Mask Focusing | Decoupled Loss | mAP@0.5 (%) |
|---|---|---|---|---|
| 1 | 38.8 | |||
| 2 | √ | 40.1 | ||
| 3 | √ | √ | 40.5 | |
| 4 | √ | √ | √ | 41.5 |
Each component contributes positively. Feature reconstruction alone improves the student from 38.8% to 40.1%, confirming that aligning semantic features is beneficial. Adding the dynamic mask yields a further 0.4 point improvement, showing that suppressing background information enhances distillation efficiency. The adaptive decoupled loss brings the largest gain of 1.0 point, emphasizing the importance of task-specific supervision at the output head.
Qualitative Analysis
To visualize the effect of distillation, I illustrate detection results on representative UAV images. The baseline student YOLOv10n-WRS detects most objects but often misses small vehicles in dense scenes and produces low-confidence boxes. The teacher YOLOv10s-WRS detects more small objects with higher localization quality. After distillation, the student is able to recover a substantial portion of the missed detections and the bounding boxes align more closely with the true targets. This qualitative comparison confirms that my selective distillation framework effectively transfers the teacher’s foreground-aware feature representation to the lightweight student model.
Discussion and Future Work
In this thesis, I have presented a complete pipeline for improving UAV object detection on resource-constrained platforms. The proposed YOLOv10s-WRS teacher model demonstrates the power of multi-scale feature enhancement through wavelet convolution, variable receptive field attention, and a small-object-oriented pyramid. However, there are still limitations. First, my method relies on fully supervised training, which requires large amounts of annotated aerial data. In real-world scenarios, annotating images is expensive and time-consuming. Future work should explore semi-supervised or unsupervised domain adaptation techniques to reduce the annotation cost. Second, while the student model runs at 126 FPS on a powerful desktop GPU, the actual deployment on embedded platforms such as NVIDIA Jetson may further reduce inference speed. Combining my KD approach with model pruning and quantization could achieve even lower latency and memory usage. Finally, the current framework only uses RGB images. Multi-modal sensors, such as thermal infrared cameras, can provide complementary information for detecting small objects in low-light or adverse weather conditions. Integrating multi-modal knowledge distillation is a promising direction for future research.
Conclusion
This thesis addresses the twin challenges of small object detection and model compression in unmanned aerial vehicles. I first proposed YOLOv10s-WRS, a high-performance detection network that employs wavelet convolution to capture high-frequency details, a receptive-field and attention module to adaptively focus on informative regions, and a small-object-enhanced feature pyramid to preserve multi-scale information. This teacher model achieves 43.6% mAP@0.5 on VisDrone2019, significantly outperforming the baseline YOLOv10s. Subsequently, I developed a selective knowledge distillation framework consisting of cross-dimensional multi-scale semantic feature reconstruction, dynamic mask-focused distillation, and adaptive decoupled distillation loss. The resulting lightweight student YOLOv10n-WRS achieves 41.5% mAP@0.5 while maintaining only 2.8M parameters and 126 FPS inference speed, demonstrating an excellent trade-off between accuracy and efficiency. The proposed methods provide a practical and effective solution for real-time object detection on unmanned aerial vehicles, and I believe they can be extended to broader edge-computing applications in the future.