We present a comprehensive study on the design and validation of an integrated cooperative system for multi-task payloads carried by medium-to-large unmanned aerial vehicles (UAV drones). The system addresses critical challenges in emergency surveying, including low multi-payload coordination efficiency and insufficient environmental adaptability under complex meteorological conditions. Our approach introduces a dynamic multi-source heterogeneous payload coordination architecture combined with a dual-mode route optimization algorithm. The entire hardware–control–data collaborative system was built upon a UAV drone platform (CH-4), integrating a photogrammetric camera, a lightweight synthetic aperture radar (SAR), and an electro-optical (EO) pod. Through a time-window-constrained payload triggering mechanism and a feature-point-density-adaptive rapid mosaicking model, flight experiments were conducted at a test site in Southwest China. Results demonstrate: (1) a 42 % improvement in per-mission multi-payload cooperative efficiency, reducing mission time to 0.64 h; (2) data completeness of 98.7 % under adverse weather, with visible-light and SAR rapid mosaicking accuracy (RMSE) of 6.464 m and 2.871 m, respectively, representing a 51 % enhancement over traditional single-payload sequential operations; (3) real-time data processing efficiency of 5 min/km², meeting the timeliness requirements of emergency surveying. This work provides an engineering-ready technical solution for multi-task cooperative operations using UAV drones.
1. Introduction
Unmanned aerial vehicles (UAV drones) have become indispensable tools for rapid acquisition of situational information during emergencies such as natural disasters, infrastructure failures, and public security incidents. Aerial photogrammetry using UAV drones enables high-resolution mapping and real-time monitoring. However, existing UAV drone emergency surveying technologies still face two major bottlenecks. Lightweight single-payload UAV drones suffer from low data acquisition efficiency—multiple sorties are required to cover the same area. Medium-to-large UAV drones capable of carrying multiple payloads lack dynamic optimization methods for multi-sensor coordination, especially under rapidly changing environmental conditions (e.g., clouds, rain, wind). To overcome these limitations, we propose a systemic solution: (1) a dynamic multi-source heterogeneous payload coordination architecture with high-precision spatiotemporal synchronization and adaptive communication protocols, enabling millisecond-level collaborative control among the photogrammetric camera, SAR, and EO pod; (2) a dual-mode route planning algorithm that dynamically switches between optical and radar acquisition modes based on mission priority and sensor characteristics, achieving full-element data collection (optical, SAR, infrared) in a single flight, improving operational efficiency by 40 %; (3) an edge-based real-time rapid mosaicking engine that leverages lightweight neural networks and parallel computing, compressing data processing latency from hours to minutes. Flight tests in a representative Southwestern test site verified system performance under 10 m/s wind and moderate rain conditions, with data integrity exceeding 93 % and fast-mosaicking accuracy reaching sub-meter levels for SAR. This paper details the system design, algorithm formulation, experimental validation, and comparative analysis, offering a new paradigm for multi-task payload cooperative operations using UAV drones.
2. System Architecture
We designed a modular hardware–control–data collaborative framework centered on a medium-to-large UAV drone platform. The core payloads include a high-resolution photogrammetric camera (IXU 1000-RS90), a Ku-band lightweight SAR, and an EO pod with both visible and infrared channels. Key hardware specifications are summarized in Table 1.
| Payload | Parameter | Value |
|---|---|---|
| Photogrammetric camera | Focal length | 89.5072 mm |
| Pixel size | 4.6 µm | |
| Image resolution | 16 470 × 11 540 px | |
| Lightweight SAR | Band | Ku |
| Look angle | 10° – 30° (right side) | |
| Resolution | 0.15 – 0.3 m | |
| EO pod (visible) | Focal length (optical) | 6.7 – 129 mm |
| Pixel pitch | 2.59 µm | |
| Frame size | 1920 × 1080 | |
| EO pod (IR) | Focal length | 50 mm |
| Pixel pitch | 17 µm | |
| Frame size | 640 × 512 |
The payloads are rigidly mounted on a three‑axis stabilized platform via a mechanical slip ring, achieving microsecond‑level time synchronization between the IMU and the sensors. Dual redundant power supply ensures uninterrupted operation even if the main power source fails, maximizing UAV drone flight safety.
2.1 Cooperative Control Logic
We developed a time‑window‑constrained payload triggering mechanism. The total mission time T is divided into N consecutive time windows of equal length ΔT = T/N. For each window j (j = 1, 2, …, N), the system decides whether to trigger each of the M payloads (here M = 3: camera, SAR, EO pod). The decision variable is defined as
$$
x_{ij} = \begin{cases} 1 & \text{if payload } i \text{ is triggered in window } j, \\ 0 & \text{otherwise.} \end{cases}
$$
The constraints are:
$$
\sum_{i=1}^{M} x_{ij} \leq K, \quad \forall j,
$$
where K represents the maximum number of payloads that can be triggered simultaneously (determined by the hardware parallel capacity; in our system K = 2). Furthermore, a conflict matrix C is introduced: Cik = 1 if payloads i and k conflict (e.g., simultaneous operation exceeding power or bandwidth limits), otherwise 0. The triggering rule must satisfy xij + xkj ≤ 1 for all conflicting pairs. To maximize mission utility, each payload is assigned a priority weight wi. For emergency mapping, the photogrammetric camera receives the highest weight (0.6) due to its ultra‑high resolution (0.1 m) needed for post‑disaster damage assessment. SAR receives weight 0.3 because of its all‑weather capability (crucial under clouds/rain). The EO pod, mainly providing real‑time video streams (0.68 m ground resolution), gets weight 0.1. The objective function maximizes total weighted triggering
$$
\max \sum_{j=1}^{N} \sum_{i=1}^{M} w_i x_{ij}
$$
subject to the time‑window capacity and conflict constraints. Solving this per window using a greedy heuristic achieves near‑optimal coordination with complexity O(N × M).
3. Dual‑Mode Route Optimization Algorithm
To exploit the distinct characteristics of optical and SAR sensors, we designed a two‑mode route generation method that adapts to terrain variations and sensor geometry.
3.1 Topography‑Adaptive Acquisition
For the photogrammetric camera and EO pod, the required ground sample distance (GSD) and overlap are maintained by adjusting flight height H and line spacing L based on real‑time digital elevation data. The relationship is given by
$$
GSD = \frac{f}{F \cdot H} \quad \Rightarrow \quad H = \frac{f}{F \cdot GSD},
$$
where f is the focal length and F is the pixel pitch. For each sample point p (typically 5000 points per km²), we compute the required H and side‑overlap Os using
$$
O_s = 1 – \frac{L}{W},
$$
with W being the swath width. A greedy selection of P = 10 candidate lines maximizes coverage while minimizing cross‑track variation. The computational complexity O(P log P + P) satisfies real‑time planning requirements (planning time < 30 s).
3.2 Cooperative Route Generation
Considering the right‑looking geometry of SAR (always imaging to the right of the UAV drone’s flight path), we developed a route‑reuse strategy. The infrared video acquisition lines are embedded into the redundant segments of the SAR flight pattern, reducing non‑productive flight time by 30 %. Table 2 summarizes the route designs for the two modes.
| Mode | Parameter | Camera/EO mode | SAR/IR mode |
|---|---|---|---|
| Photogrammetric camera | GSD | 0.1 m | – |
| Forward overlap | 70 % | – | |
| Side overlap | 35 % | – | |
| Number of lines | 5 | – | |
| Exposure points | 97 | – | |
| Line spacing | 1220 m | – | |
| Mission time | 0.26 h | – | |
| Visible EO camera | GSD | 0.68 m | – |
| Focal length | 6.7 mm | – | |
| Side overlap | 35 % | – | |
| Line spacing | 1220 m | – | |
| Area covered | 28 km² | – | |
| Mission time | 0.26 h | – | |
| Lightweight SAR | Resolution | – | 0.15 m |
| Number of lines | – | 6 | |
| Side overlap | – | 50 % | |
| Forward overlap | – | 70 % | |
| Flight altitude | – | 2000 m | |
| Line spacing | – | 1220 m | |
| Mission time | – | 0.64 h | |
| Infrared video camera | GSD | – | 0.68 m (IR) |
| Focal length | – | 50 mm | |
| Side overlap | – | 35 % | |
| Line spacing | – | 1220 m | |
| Mission time | – | 0.64 h (simultaneous with SAR) |
The cooperative route effectively increases area coverage by 28 % compared to separate flights, with total mission duration for the combined optical‑SAR‑IR acquisition reaching only 0.64 h.
4. Real‑Time Data Processing and Mosaicking
We implemented an edge‑based rapid mosaicking pipeline that runs onboard the UAV drone or a ground station in real time. The processing flow consists of keyframe extraction from video streams, feature‑based image registration, and seamless blending.
4.1 Video Stream Processing
For visible and infrared videos, frames are automatically extracted at intervals that guarantee the desired overlap (typically 70 % forward). Keyframes are then aligned using an adaptive density of feature points (SIFT or lightweight neural descriptors). The mosaicking model minimizes the reprojection error E:
$$
E = \sum_{k=1}^{K} \sum_{m=1}^{M_k} \| p_{km} – \hat{p}_{km} \|^2,
$$
where pkm and ŝkm are the observed and predicted feature point coordinates, K is the number of overlapping frames, and Mk is the number of matches per frame pair. The optimization is performed using a Levenberg‑Marquardt solver accelerated on an embedded GPU. The resulting mosaic accuracy for visible video reached an RMSE of 6.464 m against ground control points; for infrared video the RMSE was 33.609 m (primarily due to the lower resolution and narrower field of view). These values meet the operational standards for rapid emergency mapping.
4.2 Photogrammetric Camera Image Mosaicking
The camera images (16 470 × 11 540 pixels, JPEG format) are imported into a dedicated software module that converts POS data from WGS84 to the CGCS2000 planar coordinate system and performs fast stitching using a bundle‑adjustment‑free approach. The RMSE of the camera mosaic was 2.413 m, well within the required tolerance for 1:10 000 scale mapping.
4.3 SAR Image Processing
SAR raw data are processed onboard using motion compensation and polarimetric calibration algorithms. The resulting SAR images maintain a geometric resolution of 0.15 m. The mosaicking of multiple SAR strips employs a phase‑based correlation method to achieve sub‑pixel alignment. The final mosaic accuracy was measured as an RMSE of 2.871 m (planimetric). Table 3 summarizes the accuracy and processing efficiency achieved by the system.
| Metric | Traditional single‑payload sequential mode | Proposed cooperative mode | Improvement |
|---|---|---|---|
| Per‑mission operation time (h) | 1.10 | 0.64 | 42 % reduction |
| Data completeness ( %) | 89.2 | 98.7 | +9.5 % |
| Hardware conflict events | 3 | 0 | Eliminated |
| Visible mosaic RMSE (m) | 13.2 | 6.464 | 51 % improvement |
| SAR mosaic RMSE (m) | 5.86 | 2.871 | 51 % improvement |
| Real‑time processing rate (min/km²) | 8.6 | 5.0 | 42 % faster |
5. Experimental Validation
We conducted flight experiments over a 28 km² rectangular area in a test region characterized by moderate terrain undulation and variable weather. The UAV drone autonomous flight altitude was set to 2000 m above mean sea level. To verify the robustness of the proposed system, three typical conditions were encountered during the trials: moderate rain (intensity 2 mm/h), surface wind gusting to 10 m/s, and partial cloud cover below the flight altitude. The flight mission was fully pre‑programmed and executed without manual intervention.
During the flights, the time‑window‑constrained payload triggering mechanism operated without any hardware conflicts. The camera acquired 97 high‑resolution images; the SAR generated six strips; and the visible and infrared EO cameras recorded continuous video streams from which 500 keyframes each were extracted and mosaicked. Post‑flight inspection confirmed data completeness of 98.7 % (four images were slightly cloud‑obscured, but the SAR data compensated fully). The real‑time processing pipeline produced mosaics within 5 minutes per square kilometer, satisfying the 30‑minute turnaround requirement for emergency response.
Accuracy assessment was performed using 30 independent ground control points measured with RTK‑GPS. The RMSE values for each payload type are reported in Table 3. In every case, the accuracy surpasses what could be obtained by flying the same payloads separately (the traditional baseline was approximated by processing data collected in previous single‑payload sorties over the same area). The dual‑mode route optimization contributed to the accuracy improvement by ensuring more homogeneous coverage and better overlap consistency.
6. Discussion
The results confirm that our integrated cooperative system significantly enhances the operational efficiency and data quality of UAV drone‑based emergency mapping. The 42 % reduction in mission time is directly attributable to the concurrent acquisition of optical and radar data in a single sortie. The time‑window‑constrained triggering logic prevents resource contention while maximizing the utility of each payload. The dual‑mode route algorithm is particularly effective: by embedding the infrared video lines into the SAR flight pattern, we eliminated 30 % of non‑productive flight segments without sacrificing data coverage.
Comparing with traditional approaches, the accuracy gain of 51 % for both visible and SAR mosaics stems from two factors: (1) the increased overlap achieved through topography‑adaptive route design, and (2) the higher‑quality features extracted from the uninterrupted, dense keyframe sequences. The real‑time processing rate of 5 min/km² is enabled by the lightweight neural network‑based feature matcher and the parallel onboard GPU, a substantial improvement over the 8.6 min/km² baseline achieved by offline desktop processing. This speed is critical for disaster response where situational awareness must be updated rapidly.
Despite the promising results, some limitations remain. The system currently assumes a fixed priority weighting scheme; future work could incorporate adaptive weighting based on real‑time weather and mission phase. The SAR mosaic accuracy, while meeting operational needs, is still limited by uncompensated motion errors at high wind speeds; advanced autofocus algorithms may further improve it. Also, the integration of AI‑driven autonomous task allocation could allow the UAV drone to dynamically reassign payloads (e.g., switch from camera to SAR when clouds block the view) without ground operator intervention.
7. Conclusion
We have designed, implemented, and validated an integrated cooperative system for multi‑task payloads on medium‑to‑large UAV drones dedicated to emergency surveying. The dynamic coordination architecture, dual‑mode route optimization, and edge‑based rapid mosaicking engine collectively enable single‑mission acquisition of high‑resolution optical, SAR, and infrared data with 42 % efficiency improvement and 51 % accuracy enhancement over traditional sequential operation. Flight experiments under adverse weather demonstrated 98.7 % data completeness and real‑time processing speeds suitable for time‑critical applications. This research provides a ready‑to‑deploy technical framework for UAV drone‑based emergency mapping, with clear potential for extension to forest fire monitoring, flood assessment, and other disaster response scenarios. Future work will focus on AI‑driven autonomous mission planning and multi‑sensor data fusion to further elevate the intelligence level of UAV drone cooperative operations.