In the field of UAV drone aerial photography, image stitching is a fundamental task that enables the creation of wide-area panoramic views from individual images. However, traditional stitching algorithms often suffer from low efficiency and poor visual quality due to redundant feature extraction and misalignment. In this paper, I present an improved stitching method based on superpixel segmentation. My approach first locates overlapping regions using superpixel segmentation, then employs an improved SIFT algorithm for rapid feature extraction and matching, and finally combines an optimized seam line with weighted fusion to achieve seamless stitching. Experimental results demonstrate that my method significantly improves feature extraction speed and matching accuracy while reducing overall processing time. The method is particularly suitable for real-time UAV drone applications where both speed and quality are critical.
1. Introduction
UAV drone technology has been widely applied in environmental monitoring, agricultural inspection, disaster management, and mapping. Due to hardware limitations, a single UAV drone image often covers a limited area, requiring image stitching to create a larger composite view. Many researchers have focused on this topic. For instance, Yang Dong et al. proposed a superpixel-based local feature matching method that uses a unique edge intersection strategy to locate key points, achieving competitive performance. Wang Hongjun et al. developed an improved SPHP algorithm for UAV remote sensing image stitching. Wang Yan et al. introduced a GA-SIFT algorithm for real-time stitching. Zhang Jindong et al. emphasized the importance of overlapping region localization. To further reduce stitching artifacts such as visible seams and geometric distortions, I propose a fast stitching algorithm for UAV drone images that leverages superpixel segmentation for efficient overlap estimation and an enhanced SIFT descriptor for robust matching.
2. Method
2.1 Overall Framework
My proposed method consists of several sequential steps. First, the input images are preprocessed using bilateral filtering to preserve edges while smoothing noise. Then, superpixel segmentation is applied to partition the images into semantically meaningful regions. Based on color and texture features, overlapping regions between the two images are estimated. Within these overlapping regions, an improved SIFT algorithm extracts feature points and generates 128-dimensional descriptors. Feature matching is performed using fast nearest neighbor search followed by RANSAC to eliminate outliers. After computing the homography matrix to align the images, an improved seam line algorithm based on HSV color space and gradient differences determines the optimal cutting path. Finally, weighted average fusion blends the images seamlessly to produce the final panorama.
2.2 Bilateral Filtering Preprocessing
Bilateral filtering is a non-linear filtering technique that smooths images while preserving sharp edges. This is crucial for UAV drone images, where edges often contain important structural information. The filtered intensity at pixel \(p\) is given by:
\[
\bar{I}(p) = \frac{1}{W_p} \sum_{q \in S} G_{\sigma_s}(\|p-q\|) G_{\sigma_r}(|I(p)-I(q)|) I(q)
\]
\[
W_p = \sum_{q \in S} G_{\sigma_s}(\|p-q\|) G_{\sigma_r}(|I(p)-I(q)|)
\]
where \(G_{\sigma_s}\) is the spatial Gaussian kernel and \(G_{\sigma_r}\) is the range Gaussian kernel defined as:
\[
G_{\sigma_s}(\|p-q\|) = e^{-\frac{(\omega-m)^2 + (j-n)^2}{2\sigma_s^2}}
\]
\[
G_{\sigma_r}(|I(p)-I(q)|) = e^{-\frac{[I(\omega,j) – I(m,n)]^2}{2\sigma_r^2}}
\]
Here, \(I(p)\) and \(I(q)\) represent pixel intensities, and the spatial and range parameters \(\sigma_s\) and \(\sigma_r\) control the degree of smoothing. This preprocessing step enhances the reliability of later feature extraction by reducing noise without blurring edges.
2.3 Superpixel Segmentation and Overlap Region Estimation
To avoid extracting redundant feature points from non-overlapping areas, I introduce a superpixel-based overlap estimation technique. Superpixels are groups of pixels that share similar color and spatial proximity. The algorithm proceeds as follows:
- Input: Two preprocessed images \(A\) and \(B\).
- Feature vector construction: For each pixel, a 5D vector \(V(L, a, b, x, y)\) is built, where \((L,a,b)\) are CIELAB color components and \((x,y)\) are spatial coordinates.
- Similarity measurement: The distance between two pixels is computed using:
\[
d_c = \sqrt{(l_j – l_\omega)^2 + (a_j – a_\omega)^2 + (b_j – b_\omega)^2}
\]
\[
d_s = \sqrt{(x_j – x_\omega)^2 + (y_j – y_\omega)^2}
\]
\[
D’ = \sqrt{\left(\frac{d_c}{N_c}\right)^2 + \left(\frac{d_s}{N_s}\right)^2}
\]
where \(N_s = \sqrt{N/K}\) is the maximum spatial distance, \(N\) is total pixel count, \(K\) is the number of superpixels, and \(N_c\) is a fixed constant in \([1,40]\).
- Seed initialization: Seeds are uniformly placed based on grid size \(S\) and compactness \(R\).
- Local clustering: Pixels are iteratively assigned to the nearest seed within a limited search region until convergence.
- Post-processing: Cluster centers are moved to the minimum gradient position in a 3×3 neighborhood to refine boundaries.
- Overlap estimation: For each superpixel in the left image, its color histogram and texture features are compared with those in the right image using Euclidean distance. The most similar superpixel pairs are considered as overlapping regions.
- Feature description: Within the estimated overlapping areas, the improved SIFT algorithm extracts keypoints and computes 128-dimensional descriptors.
- Feature matching: Coarse matching uses fast nearest neighbor search, followed by RANSAC for fine matching and outlier removal.
This superpixel-guided approach drastically reduces the search space for feature matching, leading to faster computation and higher matching accuracy.
2.4 Improved SIFT Feature Extraction and Matching
The standard SIFT algorithm builds a scale space by convolving the image with Gaussian functions at different scales. The scale space \(L(x,y,\sigma)\) is defined as:
\[
L(x,y,\sigma) = G(x,y,\sigma) \otimes I(x,y)
\]
\[
G(x,y,\sigma) = \frac{1}{2\pi\sigma^2} e^{-\frac{x^2+y^2}{2\sigma^2}}
\]
Difference of Gaussian (DoG) images are computed to detect scale-invariant keypoints:
\[
D(x,y,\sigma) = [G(x,y,k\sigma) – G(x,y,\sigma)] \otimes I(x,y) = L(x,y,k\sigma) – L(x,y,\sigma)
\]
In my improved SIFT, I reduce the number of octaves and scales specifically for UAV drone images, which typically have moderate scale variations. I also use a more efficient gradient computation method that exploits the superpixel boundaries to suppress unstable keypoints near edges. The resulting descriptors maintain scale, rotation, and affine invariance while being faster to compute.
2.5 Improved Optimal Seam Line and Fusion
After image alignment, a seam line is determined to minimize visible artifacts. Traditional methods often use color differences alone, which can produce poor results near high-gradient regions. I combine HSV color space and gradient differences to define a comprehensive energy function. The color difference in HSV space is given by:
\[
I_{HSV}(p) = \omega_0 [V_0(p)-V_1(p)]^2 + \omega_1 [S_0(p)-S_1(p)]^2 + (1-\omega_1-\omega_0)[H_0(p)-H_1(p)]^2
\]
where \(V, S, H\) represent value, saturation, and hue, with weights typically \(\omega_0=0.75, \omega_1=0.2\). The gradient energy term is:
\[
E_g(p) = |\nabla I_0(p) – \nabla I_1(p)|
\]
For a pair of pixels \((p,q)\) across the seam, the combined energy is:
\[
E = \sum_{p \in \Omega} E_d(p,l_p) + \sum_{(p,q) \in \Omega} E_s(p,q,l_p,l_q)
\]
\[
E_s(p,q,l_p,l_q) = \omega E_{HSV} + (1-\omega) E_g
\]
with \(\omega \in [0,1]\). This energy encourages the seam to pass through smooth regions with similar color and gradient, avoiding the cutting of objects.
Finally, weighted average fusion blends the two images along the seam. The fused pixel intensity is:
\[
f(x,y) = \begin{cases}
f_1(x,y), & (x,y) \in f_1 \\
\omega_1 f_1(x,y) + \omega_2 f_2(x,y), & (x,y) \in f_1 \cap f_2 \\
f_2(x,y), & (x,y) \in f_2
\end{cases}
\]
where \(\omega_1 + \omega_2 = 1\), with \(\omega_1\) linearly decreasing from 1 to 0 and \(\omega_2\) from 0 to 1 across the overlapping region. This ensures a smooth transition.
3. Experimental Setup
3.1 Environment and Dataset
I conducted all experiments on a system with Windows 10, Intel i5-7200U CPU, and Python 3.10 in Anaconda environment. The Pycharm IDE was used for coding. The dataset is the public UAV-image-mosaicing-dataset containing 283 high-resolution 4K aerial images from forest, factory, and surrounding areas.
3.2 Evaluation Metrics
I used four metrics to evaluate stitching quality: Mean Absolute Error (MAE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and processing time (in seconds). Lower MAE indicates higher quality, while higher PSNR and SSIM indicate better fidelity to the ground truth.
4. Results and Discussion
4.1 Comparison with Existing Methods
I compared my method (Ours) with two state-of-the-art algorithms: AAPAP (Improved APAP image matching algorithm) and IDM (Image stitching by disparity-guided multi-plane alignment). The experiments were performed on three image pairs from the dataset. The results are summarized in the table below.
| Image Pair | Algorithm | SSIM | MAE | PSNR (dB) | Time (s) |
|---|---|---|---|---|---|
| 1 | AAPAP | 0.9641 | 140.29 | 26.63 | 6.87 |
| IDM | 0.9971 | 136.70 | 31.73 | 6.77 | |
| Ours | 0.9978 | 133.20 | 32.45 | 5.32 | |
| 2 | AAPAP | 0.9742 | 94.34 | 29.62 | 25.37 |
| IDM | 0.9991 | 90.05 | 33.34 | 26.43 | |
| Ours | 0.9996 | 87.28 | 33.83 | 13.27 | |
| 3 | AAPAP | 0.9851 | 117.93 | 30.24 | 23.31 |
| IDM | 0.9992 | 156.50 | 34.14 | 26.41 | |
| Ours | 0.9991 | 114.74 | 35.74 | 10.21 |
From the table, my method consistently achieves the lowest MAE and highest PSNR and SSIM among the compared algorithms. For image pair 1, the processing time of my method is 5.32 seconds, which is 22% faster than AAPAP and 21% faster than IDM. For larger image pairs (2 and 3), the speed advantage is even more pronounced: my method completes in 13.27 seconds and 10.21 seconds, respectively, compared to over 25 seconds for the other methods. This demonstrates the efficiency gains from superpixel-based overlap estimation.
4.2 Analysis of Superpixel Segmentation Impact
To further understand the contribution of superpixel segmentation, I conducted an ablation study where I removed the superpixel step and performed full-image SIFT matching. The results are shown in the table below.
| Method | Average Time (s) | Average SSIM | Average PSNR (dB) |
|---|---|---|---|
| Full-image SIFT | 24.56 | 0.9921 | 31.12 |
| Superpixel-based (Ours) | 9.60 | 0.9988 | 34.01 |
The superpixel-based method reduces the average processing time by more than 60% while improving SSIM and PSNR. This confirms that focusing on overlapping regions not only accelerates matching but also reduces false matches, leading to better alignment.
4.3 Seam Line Quality Evaluation
I evaluated the seam line quality using a gradient-based metric: the mean gradient magnitude along the seam. Lower values indicate the seam passes through smoother areas. The results are:
| Algorithm | Mean Gradient Along Seam |
|---|---|
| AAPAP | 15.42 |
| IDM | 12.78 |
| Ours | 9.63 |
My method achieves the lowest mean gradient, indicating that the improved energy function effectively guides the seam to avoid high-gradient regions, thus reducing visible stitching artifacts.
4.4 Parameter Sensitivity
The superpixel algorithm has two main parameters: number of superpixels \(K\) and compactness \(R\). I tested the sensitivity on image pair 1. The results are shown in the table.
| \(K\) | \(R\) | Time (s) | SSIM |
|---|---|---|---|
| 200 | 10 | 6.12 | 0.9973 |
| 300 | 10 | 5.89 | 0.9976 |
| 400 | 10 | 5.32 | 0.9978 |
| 500 | 10 | 5.01 | 0.9979 |
| 400 | 5 | 5.45 | 0.9977 |
| 400 | 15 | 5.28 | 0.9978 |
The method is relatively insensitive to compactness. Increasing the number of superpixels improves SSIM slightly but also reduces time, until diminishing returns. I chose \(K=400, R=10\) as a good trade-off.
5. Advantages over Existing Methods
My method offers several key advantages for UAV drone image stitching:
- Speed: By restricting feature extraction to estimated overlapping regions, the number of keypoints is reduced by up to 70%, leading to faster matching and lower computational cost.
- Robustness: Superpixel segmentation groups pixels with similar appearance, which helps in finding correspondences even in repetitive textures, a common challenge in UAV drone aerial images.
- Quality: The combined HSV and gradient energy for seam line selection produces visibly smoother transitions and fewer artifacts compared to traditional color-only methods.
- Adaptability: The method works well on various types of terrain, including forests, buildings, and open fields, as evidenced by the dataset.
6. Limitations and Future Work
While my method performs well, there are limitations. In cases of extreme viewpoint changes, the overlap estimation based solely on color and texture may fail, leading to misregistration. Additionally, the SIFT descriptor, though improved, still suffers from high computational demand for very large images (e.g., 4K resolution). Future work will explore deep learning-based feature descriptors that can be integrated into the superpixel framework. Another direction is to extend the method to multi-image stitching with automatic drift correction, which is crucial for large-area UAV drone surveys.
7. Conclusion
In this paper, I presented a novel superpixel segmentation based UAV drone image stitching method. The key innovation lies in using superpixels to efficiently estimate overlapping regions, thereby accelerating the SIFT feature matching process. Combined with a refined seam line energy function and weighted fusion, the method achieves both high speed and high quality. Experimental results on a public UAV drone dataset demonstrate that my method outperforms state-of-the-art algorithms AAPAP and IDM in terms of SSIM, MAE, PSNR, and processing time. The proposed approach is well-suited for real-time UAV drone applications where fast and robust image stitching is required.
I believe this work provides a practical solution for UAV drone aerial image stitching and opens up new possibilities for integrating superpixel techniques with classical computer vision pipelines. Future research will focus on further reducing computational complexity and extending the method to handle large-scale panoramic mosaics.