The rapid advancement of drone technology has significantly expanded the application scope of unmanned aerial vehicles (UAVs) in both military and civilian domains, including intelligence, surveillance, and reconnaissance (ISR). For such autonomous operations, the ability to accurately and robustly estimate the ego-motion of a drone is a fundamental prerequisite. While traditional Global Navigation Satellite Systems (GNSS) provide a straightforward solution, they are inherently vulnerable to signal occlusion and interference in challenging environments like canyons, forests, and urban canyons. Vision-based pose estimation, which leverages environmental features, offers a compelling alternative for GNSS-denied scenarios. However, existing visual methods, particularly those relying on point features like ORB, face severe performance degradation in scenes with low texture, such as water surfaces, deserts, or uniform building facades. To address this critical limitation in drone technology, we propose a novel method named Multi-Feature Fusion and Memory Correction (MF2MC). Our approach enhances the robustness and accuracy of pose estimation by synergistically integrating complementary visual features and employing a temporal correction mechanism.
The core contribution of this work, which advances the state-of-the-art in drone technology, is a three-pronged framework. First, we develop a multi-feature cooperative modeling strategy that unifies geometric point features, geometric line features, and photometric features to provide a richer and more reliable set of constraints for pose estimation, especially in texture-poor environments. Second, we propose a unified representation and type-guided matching strategy for these diverse features, ensuring efficient and robust cross-frame correspondences. Finally, we introduce a novel short-term memory correction mechanism that detects and corrects local estimation failures by leveraging temporal continuity, thereby suppressing drift and improving the long-term stability of the estimated trajectory.
1. Methodology of the MF2MC Framework
Our proposed MF2MC method is structured into three main modules: multi-feature extraction and description, multi-feature fusion and matching, and pose estimation with memory correction. This architecture is designed to maximize information utilization in challenging visual conditions, a key requirement for reliable drone technology.
1.1 Multi-Feature Extraction and Description
To overcome the limitations of single-feature methods, we extract three distinct types of features from each image frame.
Point Feature Extraction: We utilize XFeat, a lightweight convolutional network, for initial point feature extraction. XFeat is chosen for its excellent balance between performance and computational efficiency, making it suitable for resource-constrained drone platforms. It outputs a set of keypoints, descriptors, and a confidence map. To avoid the common problem of keypoints clustering in high-response areas, we enhance the selection process with a multi-dimensional scoring mechanism. The combined score, $S_{combined}^i$, for each keypoint $p_i$ is defined as:
$$S_{combined}^i = \omega_1 s_i + \omega_2 \|d_i\| + \omega_3 H(d_i) + \omega_4 f_i$$
where $s_i$ is the confidence score, $\|d_i\|$ is the descriptor norm, $H(d_i)$ is the descriptor entropy, and $f_i$ is a spatial distribution factor. Weights $\omega_1$ to $\omega_4$ balance these contributions. An adaptive threshold $T$ is then applied to dynamically select the most informative keypoints, where $T = \mu_{S} + \alpha \delta_{S}$, with $\mu_{S}$ and $\delta_{S}$ being the mean and standard deviation of the combined scores.
Line Feature Extraction: In weakly textured scenes, continuous structural information is often more reliable than isolated points. We therefore incorporate line segments extracted using the Line Segment Detector (LSD). To make these line features matchable, we propose a structured representation. The descriptor for a line segment $l_i$ is obtained by uniformly sampling ten points along it, extracting their feature vectors from the XFeat network’s dense feature map via bilinear interpolation, and then aggregating them through mean pooling and L2 normalization, resulting in a descriptor $d_i$. The saliency score $s_i$ for the line is its length, calculated as:
$$s_i = \sqrt{(x_2 – x_1)^2 + (y_2 – y_1)^2}$$
Redundant line segments are filtered out using a cosine similarity threshold of 0.95 to ensure a clean set of structural features.
Photometric Feature Extraction: To further enrich the feature set, we include photometric features that are sensitive to local intensity variations. Candidate points are first identified using the Harris corner detector. The response is refined using a Sobel gradient magnitude and Non-Maximum Suppression (NMS) to select high-response regions. For each candidate point $(x_i, y_i)$, a local $8 \times 8$ image patch $P_{x,y}$ is extracted. The saliency score $s_i$ is defined as the variance of pixel intensities within that patch:
$$s_i = \delta^2(P_{x,y}) = \frac{1}{M^2} \sum \sum (I(u,v) – \mu_i)^2$$
A lightweight convolutional network encodes each image patch into a robust, fixed-length descriptor $d_i \in \mathbb{R}^{64}$, providing strong discriminative power even in low-texture areas.
1.2 Multi-Feature Fusion and Matching
Having extracted point, line, and photometric features, the next challenge is to fuse and match them effectively. Directly matching all features as a single set can lead to cross-type interference. Our strategy addresses this by first unifying all features into a common point-like representation and then applying a type-guided matching process. The key features and their unified representation are summarized in the table below.
| Feature Type | Source | Unified Elements | Description |
|---|---|---|---|
| Point | XFeat | $(p_i, d_i, s_i, t_i=0)$ | Keypoint coordinates, descriptor, score, type label |
| Line | LSD | $(p_i, d_i, s_i, t_i=1)$ | Midpoint coordinates of line segment, descriptor, length score, type label |
| Photometric | Harris + CNN | $(p_i, d_i, s_i, t_i=2)$ | Point coordinates, encoded descriptor, variance score, type label |
Intra and Inter-Class Double Filtering: Before matching, we perform a two-stage filtering process. In the intra-class stage, for photometric features, we retain only points whose distance to the nearest XFeat point is greater than a threshold $\tau_d$ and whose saliency score exceeds a mean-based threshold $\bar{s}$. In the inter-class stage, spatial conflicts between the surviving point features and line features are resolved. If a point feature is very close to a line segment (distance $d(p_i, l_j) < \tau_l$), the point is considered redundant and is removed. However, to preserve information, the descriptor of the removed point is used to enhance the line’s descriptor via a convex combination: $d_l’ = \alpha d_l + (1-\alpha) \mu_{d_i}$, where $\mu_{d_i}$ is the mean descriptor of the removed points and $\alpha \in [0,1]$ is a fusion weight.
Type-Guided Matching: The final feature set $\mathcal{F} = \bigcup_i \mathcal{F}_{t_i}$ is partitioned by feature type $t_i \in \{0,1,2\}$. For each type, a Mutual Nearest Neighbor (MNN) matching strategy is employed. A match between a feature $p_i$ in image $I$ and $p_j$ in image $J$ is established if and only if $p_j = \text{NN}(p_i)$ and $p_i = \text{NN}(p_j)$. This initial match set is then filtered by requiring a high cosine similarity between their descriptors, i.e., $\cos(d_i, d_j) > \tau$. This process yields a robust set of matches for each feature type, which are then combined to form the final matching pool. For line features, the matched midpoints are mapped back to their corresponding line segment matches.
1.3 Pose Estimation and Memory Correction
The robust set of 2D-2D correspondences obtained from the fusion and matching stage is used to estimate the relative pose between consecutive frames. This is a crucial step in drone technology for reconstructing the flight trajectory.
Relative Pose Estimation: Given a set of matched points, the fundamental matrix $\mathbf{F}$ is estimated using the epipolar constraint $p_j^T \mathbf{F} p_i = 0$. With known camera intrinsics $\mathbf{K}$, the essential matrix $\mathbf{E}$ is computed as $\mathbf{E} = \mathbf{K}^T \mathbf{F} \mathbf{K}$. The relative rotation matrix $\mathbf{R}$ and translation vector $\mathbf{t}$ are then recovered by decomposing $\mathbf{E}$, i.e., $\mathbf{E} = \mathbf{t}^* \mathbf{R}$, where $\mathbf{t}^*$ is the skew-symmetric matrix of $\mathbf{t}$. This pose is then transformed or integrated with prior information to obtain the 6-DoF global pose $\mathbf{P}_k$ in the ENU coordinate frame, consisting of latitude, longitude, altitude, and orientation angles.
Memory Correction Mechanism: To mitigate local estimation failures and drift, especially in challenging low-texture scenarios, we integrate a Short-Term Memory Correction (MC) module. This mechanism operates as a post-processing step on the pose estimates, analyzing temporal continuity. The translational error $e_t$ and rotational error $e_R$ between consecutive poses are monitored against empirically defined thresholds $\tau_t$ and $\tau_R$:
$$e_t = \| \mathbf{t}_k – \mathbf{t}_{k-1} \|, \quad e_R = \arccos\left(\frac{\text{trace}(\mathbf{R}_{k-1}^T \mathbf{R}_k) – 1}{2}\right)$$
If $e_t > \tau_t$ or $e_R > \tau_R$, the current pose estimate $\mathbf{P}_k$ is deemed unreliable. The system then performs a rollback and a smoothed correction:
$$\mathbf{P}_k \leftarrow \alpha \mathbf{P}_{k-1} + (1-\alpha) \mathbf{P}_k$$
Here, $\alpha \in [0,1]$ is a smoothing factor. This simple but effective mechanism uses the assumption of motion smoothness over short time intervals to suppress abrupt jitters and large errors, significantly enhancing the reliability of the pose estimation. The effectiveness of this module is a key contribution to the robustness of our drone technology.
| Scene | Method | Keypoint Count | Grid Coverage (%) | Match Count | Inlier Ratio (%) |
|---|---|---|---|---|---|
| Water Surface | ORB | 4885 | 19.84 | 1022 | 86.61 |
| Water Surface | XFeat | 2317 | 25.71 | 1409 | 52.09 |
| Water Surface | MF2MC (ours) | 4498 | 30.16 | 3088 | 78.02 |
| Sandy Terrain | ORB | 4930 | 53.17 | 599 | 80.63 |
| Sandy Terrain | XFeat | 4096 | 81.27 | 1402 | 34.88 |
| Sandy Terrain | MF2MC (ours) | 7261 | 93.81 | 2528 | 63.29 |
| Building Surface | ORB | 2447 | 51.50 | 613 | 73.08 |
| Building Surface | XFeat | 2524 | 72.56 | 1376 | 61.34 |
| Building Surface | MF2MC (ours) | 2540 | 80.83 | 1564 | 69.81 |
2. Experimental Evaluation and Results
We extensively evaluated our MF2MC method using the challenging UAV_VisLoc dataset. This dataset, which features low-frame-rate imagery from real drone flights over water, sand, and buildings, is ideal for testing the limits of drone technology in GNSS-denied environments.
The experiments assessed three key capabilities: feature extraction, feature matching, and overall pose estimation. The results, as shown in Table 2, demonstrate the superior performance of our method. In feature extraction, MF2MC consistently achieved higher grid coverage compared to ORB and XFeat, indicating a more uniform and comprehensive spatial distribution of features, which is critical for reliable pose estimation. For example, on sandy terrain, the coverage improved from 81.27% (XFeat) to 93.81%. In feature matching, MF2MC produced the highest number of matches while maintaining a high inlier ratio. This balance between quantity and quality is a direct result of our multi-feature fusion and type-guided matching strategy. On the water surface scene, we obtained 3088 matches with a 78.02% inlier ratio, far exceeding the performance of both ORB and XFeat.
| Scene | Method | Translation Error (km) | Rotation Error (°) | General ACC (%) | ||||
|---|---|---|---|---|---|---|---|---|
| RMSE ↓ | STD ↓ | AUC/ACC@20 ↑ | RMSE ↓ | STD ↓ | AUC/ACC@10 ↑ | |||
| drone_01 | ORB | 0.028 | 11.460 | (3.633, 18.63) | 24.920 | 22.870 | (32.67, 68.63) | 82.10 |
| drone_01 | XFeat | 0.032 | 16.330 | (5.756, 20.96) | 14.070 | 11.050 | (34.90, 72.79) | 97.30 |
| drone_01 | MF2MC (ours) | 0.022 | 5.041 | (5.737, 32.72) | 7.345 | 4.237 | (42.28, 91.18) | 98.77 |
| drone_06 | ORB | 0.019 | 7.932 | (19.040, 71.43) | 22.740 | 20.560 | (33.70, 77.26) | 96.79 |
| drone_06 | XFeat | 0.018 | 6.796 | (19.040, 70.55) | 16.060 | 13.630 | (33.87, 76.38) | 97.68 |
| drone_06 | MF2MC (ours) | 0.013 | 2.910 | (34.850, 96.79) | 10.460 | 7.390 | (34.59, 84.26) | 97.68 |
Table 3 shows the quantitative results for pose estimation. MF2MC achieved the lowest RMSE for both translation and rotation errors across all tested sequences. On the ‘drone_01’ sequence, the translation error RMSE was 0.022 km and the rotation error RMSE was 7.345°, representing a significant improvement over ORB (0.028 km and 24.92°) and XFeat (0.032 km and 14.07°). The general accuracy (ACC@0.4) also reached 98.77%, the highest among all methods compared.
| Method | Translation Error (km) | Rotation Error (°) | General ACC (%) | ||||
|---|---|---|---|---|---|---|---|
| RMSE ↓ | STD ↓ | AUC/ACC@20 ↑ | RMSE ↓ | STD ↓ | AUC/ACC@10 ↑ | ||
| drone_01 (-MC) | 0.035 | 18.890 | (4.370, 20.10) | 12.380 | 9.500 | (36.43, 76.35) | 98.28 |
| drone_01 (+MC) | 0.022 | 5.041 | (5.737, 32.72) | 7.345 | 4.237 | (42.28, 91.18) | 98.77 |
| drone_06 (-MC) | 0.018 | 6.651 | (19.000, 70.85) | 11.500 | 8.540 | (33.86, 76.68) | 97.67 |
| drone_06 (+MC) | 0.013 | 2.910 | (34.850, 96.79) | 10.460 | 7.390 | (34.59, 84.26) | 97.68 |
The ablation study in Table 4 confirms the crucial role of the Memory Correction (MC) mechanism. By comparing the performance with and without MC, we observed a substantial reduction in both error and its standard deviation. For instance, in the ‘drone_01’ scene, the introduction of MC reduced the translation error RMSE from 0.035 km to 0.022 km and the rotation error from 12.38° to 7.345°. The standard deviation also dropped dramatically, indicating increased stability and reduced jitter. This validates that the temporal smoothing effect of MC effectively counters the local failures and drift that plague single-frame methods.
| Configuration | Coverage ↑ | ATE (km) ↓ | Rotation Error (°) ↓ | ACC@0.4 ↑ |
|---|---|---|---|---|
| Full MF2MC | 0.570 | 0.022 | 7.345 | 98.77 |
| w/o Lines | 0.521 | 0.023 | 8.985 | 98.77 |
| w/o Photometric | 0.567 | 0.022 | 9.911 | 98.52 |
| w/o MC | 0.570 | 0.025 | 8.573 | 98.28 |
| w/o Multi-Score | 0.551 | 0.022 | 10.850 | 98.52 |
Table 5 presents a more detailed system-level ablation study. Removing each component led to performance degradation in specific metrics. Removing line features (‘w/o Lines’) decreased coverage and increased the rotation error. Removing photometric features (‘w/o Photometric’) had a notable negative impact on the rotation error, highlighting its role in stabilizing orientation estimation. Disabling the multi-score selection (‘w/o Multi-Score’) caused a significant increase in rotation error, confirming its importance in selecting robust and stable point features. The most critical component for reducing absolute trajectory error (ATE) was the Memory Correction module (‘w/o MC’), whose removal increased the ATE from 0.022 to 0.025 km. This comprehensive analysis confirms that each module in our MF2MC framework contributes synergistically to the overall robustness and accuracy of the system, effectively advancing the capabilities of drone technology for autonomous navigation in challenging environments.
3. Conclusion
We have presented MF2MC, a novel visual pose estimation method specifically designed to enhance the robustness and accuracy of drone technology in visually challenging, low-texture environments. By moving beyond single-feature approaches, our framework synergistically fuses geometric points, lines, and photometric features to create a rich and redundant set of constraints for pose estimation. The introduction of a short-term memory correction mechanism provides a crucial temporal safeguard against local estimation failures and drift. Our comprehensive experiments on the UAV_VisLoc dataset demonstrate that MF2MC significantly outperforms state-of-the-art methods like ORB and XFeat, achieving lower translation and rotation errors and a higher trajectory estimation success rate. This work provides a more reliable and robust solution for autonomous drone navigation in GNSS-denied or feature-poor scenarios, representing a significant step forward in the practical application of vision-based drone technology.
