With the rapid advancement of unmanned aerial vehicle technology, aerial imaging has become an indispensable tool in diverse applications such as public safety, disaster monitoring, urban governance, and traffic management. However, unmanned aerial vehicle imagery exhibits distinctive challenges compared to ground-level vision, including extremely small object scales, dense target distributions, complex backgrounds, and varying illumination conditions. Furthermore, the limited computational resources and power constraints of embedded unmanned aerial vehicle platforms impose strict requirements on model efficiency. This thesis focuses on improving the detection accuracy of small objects in unmanned aerial vehicle scenes while maintaining real-time inference capability. I propose a series of network modifications based on the YOLOv10 baseline, and then develop a knowledge distillation framework that transfers rich semantic information from a high-capacity teacher model to a lightweight student network. The experimental results demonstrate that the proposed methods achieve a promising trade-off between accuracy and speed.
1. Introduction
The last decade has witnessed an explosive growth in the deployment of unmanned aerial vehicles. Equipped with high-resolution cameras, these platforms can rapidly collect large-scale visual data over vast areas. In emergency response, unmanned aerial vehicles provide situational awareness after earthquakes, floods, and wildfires; in urban management, they enable efficient traffic monitoring, crowd analysis, and infrastructure inspection. A core component of these intelligent systems is object detection, which aims to locate and classify objects of interest in every video frame. However, achieving robust detection from a high-altitude perspective is not trivial. Objects often appear as small as a few pixels, and the background contains abundant texture that can act as noise.
Generic detection models trained on natural images, such as those based on standard convolutional neural networks, frequently suffer from severe performance degradation when applied to unmanned aerial vehicle imagery. The repeated stride operations in the backbone discard fine-grained spatial details that are crucial for recognizing tiny objects. Moreover, the limited on-board compute resources of unmanned aerial vehicle platforms make it impossible to deploy heavy models with billions of floating-point operations. Therefore, it is necessary to design a detection algorithm that simultaneously enhances small-object feature representation, suppresses background interference, and reduces model complexity for edge deployment.
In this thesis, I investigate two complementary directions. First, I modify the YOLOv10 architecture by incorporating wavelet convolution, a receptive-field adaptive convolution module with channel-spatial attention, and a small-object-oriented feature pyramid. This enhanced model, named YOLOv10s-WRS, serves as a powerful teacher network. Second, I design a selective knowledge distillation framework that transfers multi-scale semantic knowledge from YOLOv10s-WRS to a lightweight YOLOv10n-WRS student model. The proposed distillation method comprises a cross-dimensional feature reconstruction module, a dynamic mask focus mechanism, and an adaptive decoupled distillation loss. Extensive experiments on the VisDrone and UAVDT datasets verify the effectiveness of the proposed approaches.
2. Related Work
2.1 Object Detection for Unmanned Aerial Vehicles
Early approaches for unmanned aerial vehicle object detection relied on manually designed features such as HOG, SIFT, and LBP, combined with classifiers like SVM and Adaboost. These methods achieved moderate success in simple scenes but lacked robustness under occlusion, scale variation, and complex backgrounds. With the advent of deep convolutional neural networks, detection frameworks shifted toward two-stage and one-stage paradigms. Two-stage detectors, such as Faster R-CNN and Cascade R-CNN, first generate candidate regions and then classify them. These models typically achieve high accuracy but at the cost of heavy computation, rendering them unsuitable for real-time unmanned aerial vehicle systems. In contrast, one-stage detectors such as the YOLO family and SSD predict bounding boxes and class probabilities directly from the feature map in a single pass. They offer excellent speed, which is critical for unmanned aerial vehicle applications. However, vanilla YOLO models still struggle with small objects and complex background suppression when applied to aerial images.
2.2 Knowledge Distillation
Knowledge distillation is an effective model compression technique that transfers knowledge from a large teacher network to a compact student network. The most common form of knowledge distillation matches the softened output distributions via the Kullback-Leibler divergence. In object detection, distillation strategies can be applied at different levels: logit level, feature level, and relation level. Feature-level distillation aligns intermediate feature maps and forces the student to mimic the teacher’s representations. Attention-based distillation transfers spatial attention maps, and relational distillation explores sample and channel relationships. Despite their success in generic detection tasks, applying knowledge distillation to unmanned aerial vehicle object detection remains challenging because of the extreme class imbalance, the small target scale, and the need to preserve precise localization information. In this thesis, I propose a specialized distillation framework that selectively emphasizes high-value foreground regions and decouples the classification and regression constraints.
3. Multi-Scale Feature Enhancement for Unmanned Aerial Vehicle Detection
The baseline of this thesis is YOLOv10, a state-of-the-art end-to-end detector that eliminates the need for non-maximum suppression through a consistent dual-assignment strategy. YOLOv10 demonstrates a favorable balance between speed and accuracy. However, when applied to unmanned aerial vehicle scenes, its feature extraction backbone still loses critical high-frequency details, and its neck network lacks sufficient multi-scale fusion granularity. To address these issues, I propose the YOLOv10s-WRS model, which integrates three novel components: wavelet-enhanced C2f (C2f_WTConv), a receptive field and channel-spatial attention convolution module (RFCBAMConv), and a small-object feature pyramid (SOEP).
3.1 Wavelet-Enhanced C2f Module
Standard convolutions operate in the spatial domain and tend to encode low-frequency information such as overall shape and semantic content. High-frequency information, which is important for small object edges and textures, is often neglected. In unmanned aerial vehicle imagery, tiny targets rely heavily on high-frequency cues. Therefore, I enhance the C2f module by embedding wavelet convolution (WTConv) into its bottleneck layers. The proposed C2f_WTConv module performs a two-dimensional discrete wavelet transform (DWT) on the input feature map $X_{l-1} \in \mathbb{R}^{H \times W \times C}$ to decompose it into four sub-bands:
$$
\begin{aligned}
X_{ll} &= A X_{l-1} A^T, \\
X_{lh} &= V X_{l-1} A^T, \\
X_{hl} &= A X_{l-1} V^T, \\
X_{hh} &= V X_{l-1} V^T,
\end{aligned}
$$
where $A$ and $V$ denote the low-pass and high-pass filter matrices derived from the wavelet basis, respectively. The sub-bands $X_{ll}$, $X_{lh}$, $X_{hl}$, and $X_{hh}$ respectively represent the low-frequency approximation, horizontal high-frequency, vertical high-frequency, and diagonal high-frequency components. After the decomposition, these sub-bands are concatenated along the channel dimension:
$$
X_{freq} = \text{Concat}\left( X_{ll}, X_{lh}, X_{hl}, X_{hh} \right).
$$
The concatenated feature is then processed by a depthwise convolution and a learnable channel scaling operation to adaptively balance the contributions of different frequency components:
$$
\tilde{X}_{freq} = S^{(l)} \left( \text{DWConv}\left( X_{freq} \right) \right),
$$
where $S^{(l)}$ is a learnable channel-wise scaling vector. Finally, the enhanced frequency-domain representation is converted back to the spatial domain via the inverse discrete wavelet transform (IWT):
$$
X_{wavelet} = \text{IWT}\left( \tilde{X}_{freq} \right).
$$
The wavelet-enhanced module is combined with the original spatial convolution path through a residual connection:
$$
X_{out} = F_{spatial}(X_{l-1}) + F_{wavelet}(X_{l-1}),
$$
where $F_{spatial}(\cdot)$ denotes a standard convolution-based spatial feature extraction branch and $F_{wavelet}(\cdot)$ denotes the wavelet-based frequency feature extraction branch. This design enables the model to preserve fine-grained details that are essential for detecting small objects in unmanned aerial vehicle imagery without significantly increasing the parameter count.
3.2 RFCBAMConv: Receptive Field and Channel-Spatial Attention Convolution
Unmanned aerial vehicle images contain complex backgrounds with abundant edges, vegetation, buildings, and shadows that can interfere with object detection. A conventional convolution kernel applies a fixed receptive field to all spatial locations and all channels equally, which limits its ability to adapt to varying object scales and background clutter. To tackle this issue, I propose a convolution module that combines variable receptive field modeling with channel-spatial attention, called RFCBAMConv. Its structure is illustrated conceptually below.
Given an input feature $X \in \mathbb{R}^{H \times W \times C}$, the module first applies a depthwise convolution with a $k \times k$ kernel and expands the output channels to $C \times k^2$, enabling each channel to capture multiple receptive field responses:
$$
X_{rf} = \phi \left( \text{DWConv}(X) \right),
$$
where $\phi$ denotes the batch normalization followed by a ReLU activation. The tensor $X_{rf}$ is then rearranged so that the receptive field information is explicitly unfolded along the spatial dimension. This transformation produces feature maps that encode the local patch information for every position.
After the receptive field expansion, the module applies a Squeeze-and-Excitation (SE) channel attention mechanism to highlight semantically important channels. The channel weights are computed by
$$
s = \sigma \left( W_2 \, \delta \left( W_1 \, \text{GAP}(X_{rf}) \right) \right),
$$
where $\text{GAP}(\cdot)$ is global average pooling, $W_1$ and $W_2$ are weights of the two fully connected layers, $\delta$ is ReLU, and $\sigma$ is the sigmoid function. The channel-weighted feature is then obtained by
$$
\tilde{X}_{rf} = X_{rf} \otimes s,
$$
where $\otimes$ denotes channel-wise multiplication. Next, to adaptively focus on informative spatial positions, the module applies max and average pooling along the channel axis, concatenates the two resulting descriptors, and passes them through a convolution layer followed by sigmoid to generate a spatial attention map:
$$
A_{rf} = \sigma \left( \text{Conv}_{3\times3}\left( \left[ \text{MaxPool}(\tilde{X}_{rf}), \, \text{AvgPool}(\tilde{X}_{rf}) \right] \right) \right).
$$
The spatial attention map is then applied to the feature:
$$
\tilde{X}_{sp} = \tilde{X}_{rf} \otimes A_{rf}.
$$
Finally, the module includes a standard convolution with stride to project the feature to the desired output dimension and perform spatial downsampling:
$$
Y = \text{Conv}_{1\times1}(\tilde{X}_{sp}).
$$
The RFCBAMConv module enables the network to select the appropriate receptive field for each spatial location and to emphasize important channels and regions. In the context of unmanned aerial vehicle detection, this mechanism effectively suppresses background distraction and improves the discriminability of small targets.
3.3 SOEP: Small Object Feature Pyramid
In a standard FPN, the high-level semantic features are produced at low spatial resolutions, which causes small objects to lose their already limited pixel information. A direct solution is to add a P2 detection layer with higher resolution, but this significantly increases computation and memory. To avoid that overhead, I propose a small-object-oriented feature pyramid structure named SOEP, which leverages the rich spatial details of the P2 feature while not adding an extra detection head.
Let $F_{P2} \in \mathbb{R}^{H \times W \times C}$ denote the high-resolution feature output from the backbone. The SOEP structure first applies a space-to-depth operation to rearrange the spatial information into the channel dimension:
$$
F_{SPD} = \text{SpaceToDepth}(F_{P2}, r),
$$
where $r$ is the rearrangement factor. After this operation, a convolution layer followed by a non-linear activation compresses and reorganizes the channel-wise information:
$$
F_{P2}^{‘} = \varphi \left( \text{Conv}(F_{SPD}) \right),
$$
where $\varphi$ denotes the activation function.
To further enhance multi-scale feature fusion, SOEP introduces a CSP-OmniKernel module. The input feature is first split into two branches along the channel dimension:
$$
F_1, F_2 = \text{Split}(F_{P2}^{‘}, \text{dim}=1).
$$
$F_1$ is passed through a cross-stage connection to preserve original information, while $F_2$ is sent to the OmniKernel block. The OmniKernel block processes the feature using multiple kernels with different receptive field sizes, including global, large-scale, and small-scale branches. Its output is denoted as
$$
F_{Omni} = \sum_{K \in \{K_g, K_l, K_s\}} K * F_2,
$$
where $K_g$, $K_l$, and $K_s$ represent the global, large, and small kernel operations, respectively. The final output of the CSP-OmniKernel module is obtained by concatenating the two branches:
$$
F_{out} = \text{Concat}\left( F_1, F_{Omni} \right).
$$
The SOEP structure strengthens the high-resolution feature representation and enables effective multi-scale interaction without introducing an extra detection layer. This is especially beneficial for tiny objects in unmanned aerial vehicle scenes, as it preserves the spatial cues that are often lost in deeper layers.
3.4 Experimental Setup for the Teacher Model
All experiments are conducted on a workstation with an NVIDIA RTX 4090 GPU (24 GB), Ubuntu 20.04, PyTorch 2.0.0, and CUDA 11.8. The input image size is 640×640, the batch size is set to 8, and the models are trained for 200 epochs with an initial learning rate of 0.01 and a momentum of 0.937. The optimizer is stochastic gradient descent (SGD). The detailed environment is listed in the following table.
| Category | Item | Value |
|---|---|---|
| Hardware/Software | GPU | NVIDIA RTX 4090 (24 GB) |
| OS | Ubuntu 20.04 | |
| Framework | PyTorch 2.0.0, CUDA 11.8 | |
| Training | Input size | 640×640 |
| Batch size | 8 | |
| Epochs | 200 | |
| Initial learning rate | 0.01 | |
| Optimizer | SGD (momentum=0.937) |
I evaluate the models on two public unmanned aerial vehicle datasets. The VisDrone2019 dataset contains 8,629 images with 10 object categories, including pedestrian, people, bicycle, car, van, truck, tricycle, awning-tricycle, bus, and motor. The dataset is split into 6,471 training images, 548 validation images, and 1,610 test images. The UAVDT dataset is used for cross-scene generalization experiments. It contains aerial images of traffic scenarios captured by a drone.
| Dataset | Train | Validation | Test |
|---|---|---|---|
| VisDrone2019 | 6471 | 548 | 1610 |
| UAVDT | 6000 | 1000 | 1000 |
3.5 Evaluation Metrics
I adopt the following metrics to evaluate the detection performance:
- Precision $P$: the ratio of correctly detected objects over all detected objects.
- Recall $R$: the ratio of correctly detected objects over all ground-truth objects.
- mean Average Precision (mAP) at IoU threshold 0.5 and 0.5:0.95.
- Model parameters, GFLOPs, and FPS for efficiency.
The precision is defined as
$$
P = \frac{TP}{TP + FP},
$$
and recall as
$$
R = \frac{TP}{TP + FN}.
$$
AP for a single class is the area under the precision-recall curve:
$$
AP = \int_0^1 p(r) \, dr.
$$
The mean average precision is then
$$
mAP = \frac{1}{N}\sum_{i=1}^{N} AP_i,
$$
where $N$ is the category count. The detection speed is defined as
$$
FPS = \frac{1}{t_{avg}},
$$
where $t_{avg}$ is the average inference time per image.
3.6 Results and Analysis
The overall detection results comparing YOLOv10s and the improved YOLOv10s-WRS are shown in the following table. The improved model achieves significantly better precision, recall, and mAP across the board.
| Model | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) |
|---|---|---|---|---|
| YOLOv10s | 48.6 | 38.6 | 39.3 | 23.3 |
| YOLOv10s-WRS | 54.6 | 41.6 | 43.6 | 26.0 |
The improved model improves mAP@0.5 by 4.3 percentage points and mAP@0.5:0.95 by 2.7 percentage points. The precision increases by 6.0 percentage points, which indicates that the enhanced feature extraction can suppress false positives caused by complex backgrounds.
I further report the per-class results for the baseline and the improved model. The CF_WTConv module is particularly beneficial for categories whose targets are small and textured, such as truck, bus, pedestrian, and bicycle. For example, the truck category achieves an mAP@0.5 improvement from 34.2% to 41.2%, and the bus category improves from 53.1% to 58.8%. This confirms that wavelet convolution helps to preserve edge and texture information.
| Class | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) |
|---|---|---|---|---|
| pedestrian | 53.1 | 39.5 | 43.0 | 19.6 |
| people | 52.8 | 31.5 | 34.7 | 14.0 |
| bicycle | 24.9 | 16.9 | 13.2 | 5.4 |
| car | 72.3 | 77.3 | 80.3 | 57.2 |
| van | 51.8 | 45.0 | 45.7 | 31.8 |
| truck | 46.3 | 32.1 | 34.2 | 21.8 |
| tricycle | 38.9 | 29.2 | 27.1 | 14.5 |
| awning-tricycle | 32.8 | 19.0 | 16.6 | 10.1 |
| bus | 62.6 | 48.7 | 53.1 | 38.6 |
| motor | 50.0 | 46.9 | 45.6 | 20.0 |
| Class | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) |
|---|---|---|---|---|
| pedestrian | 60.6 | 41.9 | 47.7 | 22.6 |
| people | 58.5 | 32.5 | 37.9 | 15.6 |
| bicycle | 31.8 | 19.5 | 17.3 | 7.61 |
| car | 78.8 | 77.4 | 82.6 | 59.7 |
| van | 56.6 | 47.7 | 50.1 | 34.8 |
| truck | 54.2 | 38.5 | 41.2 | 25.9 |
| tricycle | 45.5 | 33.6 | 33.2 | 18.2 |
| awning-tricycle | 35.1 | 22.0 | 18.2 | 11.3 |
| bus | 69.9 | 54.0 | 58.8 | 41.2 |
| motor | 55.4 | 48.6 | 49.6 | 23.2 |
3.7 Comparison with State-of-the-Art Detectors
I compare YOLOv10s-WRS with several mainstream detectors under the same experimental conditions. The results are presented in the table below. The two-stage detectors such as Faster R-CNN and Cascade R-CNN are accurate but have large parameter counts and high GFLOPs that are unsuitable for unmanned aerial vehicle deployment. The lightweight one-stage detectors YOLOv5s and YOLOv7-tiny suffer from lower detection accuracy. YOLOv8s and YOLOv10s have moderate accuracy but are still inferior to the proposed YOLOv10s-WRS, which achieves the highest mAP while maintaining acceptable complexity.
| Model | mAP@0.5 (%) | mAP@0.5:0.95 (%) | Params (M) | GFLOPs |
|---|---|---|---|---|
| Faster RCNN | 29.5 | 15.2 | 41.5 | 206 |
| Cascade RCNN | 31.8 | 16.5 | 69.1 | 235 |
| YOLOv5s | 36.2 | 19.2 | 7.2 | 16.5 |
| YOLOv7-tiny | 35.8 | 18.9 | 6.2 | 13.8 |
| YOLOv8s | 40.5 | 24.1 | 11.2 | 28.6 |
| YOLOv10s | 39.3 | 23.3 | 7.2 | 21.6 |
| YOLOv10s-WRS | 43.6 | 26.0 | 9.5 | 35.8 |
The comparison demonstrates that the proposed YOLOv10s-WRS achieves the best balance between accuracy and computational cost. Although its parameter count and GFLOPs are slightly higher than those of the slim YOLOv5s and YOLOv7-tiny, the substantial accuracy improvement makes this trade-off favorable for high-precision unmanned aerial vehicle applications.
3.8 Ablation Study for the Teacher Model
I conduct a set of ablation experiments to evaluate the individual contributions of the proposed components. Starting from the baseline YOLOv10s, I incrementally add C2f_WTConv, RFCBAMConv, and SOEP. The results are reported in the following table.
| Baseline | C2f_WTConv | RFCBAMConv | SOEP | P (%) | R (%) | mAP@0.5 (%) |
|---|---|---|---|---|---|---|
| √ | 48.6 | 38.6 | 39.3 | |||
| √ | √ | 51.4 | 40.2 | 41.8 | ||
| √ | √ | √ | 53.2 | 41.1 | 42.9 | |
| √ | √ | √ | √ | 54.6 | 41.6 | 43.6 |
Each module contributes positively to the final detection performance. The C2f_WTConv module brings the largest gain in mAP, confirming the importance of frequency-domain information for small objects. The RFCBAMConv module further improves precision by suppressing background noise. The SOEP structure adds another 0.7% mAP improvement by enhancing high-resolution feature fusion.
3.9 Generalization Experiment on UAVDT
To verify the generalization capability of YOLOv10s-WRS across different unmanned aerial vehicle imagery, I evaluate it on the UAVDT dataset. The results in the following table show that the proposed model consistently outperforms the baseline YOLOv10s on this dataset as well, proving that the architectural improvements are not dataset-specific.
| Algorithm | P (%) | R (%) | mAP@0.5 (%) | mAP@0.5:0.95 (%) |
|---|---|---|---|---|
| YOLOv10s | 49.4 | 37.7 | 38.6 | 23.2 |
| YOLOv10s-WRS | 51.4 | 41.3 | 42.6 | 25.5 |
4. Knowledge Distillation for Lightweight Unmanned Aerial Vehicle Detection
Although YOLOv10s-WRS achieves high detection accuracy, its parameter count (9.5M) and GFLOPs (35.8G) are still relatively high for edge deployment on unmanned aerial vehicle platforms. To create a lightweight model that can run in real time with minimal resource consumption, I design a knowledge distillation framework that transfers the knowledge of the powerful teacher model to a compact student model. The student model is YOLOv10n-WRS, which shares the same architectural improvements as YOLOv10s-WRS but uses a narrower channel width and fewer layers. The student model has only 2.8M parameters and 11.7 GFLOPs, yet it can achieve a promising detection accuracy after distillation.
The proposed distillation framework consists of three key strategies:
- Cross-dimensional multi-scale semantic feature reconstruction.
- Dynamic mask-focused distillation for suppressing background interference.
- Adaptive decoupled distillation loss for classification and localization.
4.1 Cross-Dimensional Multi-Scale Semantic Feature Reconstruction
Teacher and student networks differ in channel dimensions, which makes it difficult to directly align their feature maps. A naive alignment using upsampling or convolution may lose spatial resolution or introduce unwanted distortions. Since the teacher and student share the same YOLOv10 architecture family, their feature maps at each level have identical spatial resolutions but different channel numbers. Let the teacher feature at scale $l$ be
$$
F_t^{(l)} \in \mathbb{R}^{H_l \times W_l \times C_t^{(l)}},
$$
and the student feature be
$$
F_s^{(l)} \in \mathbb{R}^{H_l \times W_l \times C_s^{(l)}}.
$$
To align the channels, I insert a $1\times1$ convolution layer after the student feature. The weights of the projection layer are $W^{(l)} \in \mathbb{R}^{C_t^{(l)} \times C_s^{(l)}}$ and bias $b^{(l)}$. The projected student feature is
$$
\tilde{F}_s^{(l)}(i,j) = W^{(l)} F_s^{(l)}(i,j) + b^{(l)},
$$
for every spatial position $(i,j)$. Because the convolutional kernel size is $1\times1$, the spatial resolution remains unchanged. Therefore, the teacher feature and the projected student feature can be directly compared using a pixel-wise loss. This reconstruction operation allows the lightweight student to learn the teacher’s high-order semantic representation without sacrificing spatial detail, which is critical for small objects in unmanned aerial vehicle imagery.
4.2 Dynamic Mask-Focused Distillation
In aerial images, the background often occupies a large portion of the feature map. If the distillation loss treats all spatial locations equally, the student will be forced to fit irrelevant background responses, which wastes its limited capacity. I therefore propose a dynamic mask mechanism that uses the teacher’s feature response to generate a spatial soft mask. The teacher’s activation map is computed by averaging the absolute values along the channel axis:
$$
M_t(h,w) = \frac{1}{C} \sum_{c=1}^{C} \left| F_t^{(l)}(h,w,c) \right|.
$$
Then, Min-Max normalization is applied to obtain the spatial mask:
$$
M_{spatial}(h,w) = \frac{M_t(h,w) – \min(M_t)}{\max(M_t) – \min(M_t) + \epsilon}.
$$
The original mean squared error between the teacher and student features is
$$
L_{MSE} = \frac{1}{HW}\sum_{h,w} \sum_{c=1}^{C} \left( \tilde{F}_s^{(l)}(h,w,c) – F_t^{(l)}(h,w,c) \right)^2.
$$
The dynamic mask loss is then defined as
$$
L_{mask} = \frac{1}{HW}\sum_{h,w} M_{spatial}(h,w) \cdot \sum_{c=1}^{C} \left( \tilde{F}_s^{(l)}(h,w,c) – F_t^{(l)}(h,w,c) \right)^2.
$$
This loss assigns higher weights to pixels where the teacher has strong responses (typically corresponding to objects) and lower weights to background areas. In this way, the distillation process focuses on valuable foreground information and suppresses the disturbing gradient from background clutter.
4.3 Adaptive Decoupled Distillation Loss
The output layer of the detector contains two branches: classification and regression. These two tasks have different optimization properties and therefore should be constrained separately. I design an adaptive decoupled distillation loss that combines a classification soft-label distillation term and a boundary box distribution distillation term.
For classification, the teacher and student logits are denoted by $Z_t$ and $Z_s$. The softened probability distributions are obtained by applying the Softmax function with temperature $T$:
$$
p_t^{(c)} = \frac{\exp(Z_t^{(c)}/T)}{\sum_{j=1}^{C}\exp(Z_t^{(j)}/T)}, \quad
p_s^{(c)} = \frac{\exp(Z_s^{(c)}/T)}{\sum_{j=1}^{C}\exp(Z_s^{(j)}/T)}.
$$
The classification distillation loss is the KL divergence between the two distributions:
$$
L_{cls} = T^2 \cdot \frac{1}{N}\sum_{i=1}^{N} \sum_{c=1}^{C} p_t^{(c)} \log \frac{p_t^{(c)}}{p_s^{(c)}},
$$
where $N$ is the number of samples and $T^2$ is used to keep the gradient scale consistent.
For regression, YOLOv10 models the bounding box offset as a probability distribution via the distribution focal loss. Let the boundary box distributions of the teacher and the student be $\beta_t$ and $\beta_s$. The regression distillation loss is defined as
$$
L_{reg} = D_{KL}(\beta_t \parallel \beta_s) = \sum_{i} \beta_t(i) \log \frac{\beta_t(i)}{\beta_s(i)}.
$$
This distribution-level alignment provides smoother gradient signals and helps the student learn more precise localization, especially for overlapping tiny objects.
The total detection-head distillation loss is a weighted combination of the two terms:
$$
L_{head} = \lambda_{cls} L_{cls} + \lambda_{reg} L_{reg},
$$
where $\lambda_{cls}$ and $\lambda_{reg}$ balance the importance of classification and regression, respectively.
4.4 Experimental Results on Distillation
The distillation experiment environment is the same as the teacher training environment, except that the teacher model is frozen during distillation. The temperature $T$ is set to 3, and the distillation loss weight is set to 0.5. The complete experimental settings are summarized in the following table.
| Category | Parameter | Value |
|---|---|---|
| Environment | GPU | NVIDIA RTX 4090 (24 GB) |
| OS / Framework | Ubuntu 20.04 / PyTorch 2.0.0 | |
| Training | Input size / Batch size | 640×640 / 8 |
| Optimizer | SGD, momentum=0.937, lr=0.01 | |
| Distillation | Teacher frozen | Yes |
| Temperature | 3 | |
| Loss weight | 0.5 |
The overall detection results for the student model before and after distillation are presented in the following table. The original lightweight student YOLOv10n-WRS achieves a mAP@0.5 of 38.8% with 2.8M parameters and 126 FPS. After applying the proposed distillation method, the mAP@0.5 increases to 41.5% while maintaining the same parameter count and inference speed. The teacher model YOLOv10s-WRS achieves 43.6%, but with a considerably larger computational footprint.
| Experiment | mAP@0.5 (%) | Params (M) | GFLOPs | FPS |
|---|---|---|---|---|
| Student baseline (YOLOv10n-WRS) | 38.8 | 2.8 | 11.7 | 126 |
| Teacher (YOLOv10s-WRS) | 43.6 | 9.5 | 35.8 | 91 |
| Student + proposed distillation | 41.5 | 2.8 | 11.7 | 126 |
The distillation process improves the student model’s mAP@0.5 by 2.7 percentage points, significantly narrowing the gap between the teacher and student models (from 4.8% to 2.1%). Moreover, the distilled student runs at 126 FPS, which is suitable for real-time unmanned aerial vehicle applications.
4.5 Comparison with Existing Distillation Methods
To further verify the superiority of the proposed distillation strategy, I compare it with several classical distillation algorithms: KD, FitNet, FGD, CWD, MGD, and AT. All methods use the same teacher and student models. The results are listed in the following table. The proposed method achieves the highest mAP@0.5 among all compared approaches.
| Method | mAP@0.5 (%) |
|---|---|
| KD | 39.8 |
| FitNet | 40.1 |
| FGD | 41.0 |
| CWD | 40.8 |
| MGD | 41.2 |
| AT | 40.6 |
| Proposed method | 41.5 |
The proposed method outperforms the strongest baseline (MGD) by 0.3 percentage points. This improvement is attributed to the selective attention mechanism that emphasizes foreground regions and the decoupled output supervision that treats classification and regression separately.
4.6 Ablation Study on Distillation Modules
I perform another ablation experiment to validate the contribution of each distillation module. The results are shown in the following table. The baseline student model without any distillation achieves 38.8%. Adding the feature reconstruction module improves the mAP to 40.1%. The dynamic mask module further increases the mAP to 40.5%. Finally, integrating the adaptive decoupled distillation loss yields the best result of 41.5%. Each module contributes positively and the combination produces the largest improvement.
| Feature Reconstruction | Mask Focus | Decoupled Loss | mAP@0.5 (%) |
|---|---|---|---|
| 38.8 | |||
| √ | 40.1 | ||
| √ | √ | 40.5 | |
| √ | √ | √ | 41.5 |
4.7 Visual Analysis of Distillation
By visualizing the detection results on representative unmanned aerial vehicle frames, I observe that the original student model often misses small objects in crowded scenes. The teacher model detects these objects more reliably. After distillation, the student model’s detections become more complete, with more small objects identified and more accurate bounding boxes. The visual comparison confirms that the knowledge distillation framework successfully transfers the teacher’s capability to perceive fine details and suppress background errors.
5. Conclusion and Future Work
In this thesis, I studied the problem of unmanned aerial vehicle object detection from the perspectives of network design and model compression. I proposed the YOLOv10s-WRS teacher model, which improves small-object detection through wavelet-enhanced feature extraction, receptive-field and channel-spatial attention, and a small-object feature pyramid. Experiments on VisDrone2019 and UAVDT demonstrate that this model achieves superior accuracy compared with many mainstream detectors. I then proposed a knowledge distillation framework that enables a lightweight student model to achieve a detection accuracy close to that of the teacher, while retaining a high inference speed of 126 FPS and a parameter count of only 2.8M. The distillation framework includes cross-dimensional feature reconstruction, dynamic mask-focused loss, and decoupled output supervision. Each component contributes to the overall performance, and the combination yields the best result.
Several directions remain open for future work. First, current detection models require large amounts of labeled data. In real-world unmanned aerial vehicle flights, the environment changes drastically under different weather and lighting conditions. Unsupervised and semi-supervised learning methods could reduce the need for manual annotation and improve the model’s generalization ability. Second, deploying the proposed model on actual edge devices could be further optimized via pruning, quantization, and other model compression techniques. In addition, integrating multi-modal data, such as visible and thermal infrared images, could help detect targets under occlusions and low-light conditions. Finally, the interaction between feature enhancement and distillation is a fertile area for future exploration, especially for extremely dense small objects in high-altitude unmanned aerial vehicle scenes. I intend to pursue these directions in future research.