In modern firefighting operations, the integration of intelligent technologies is crucial for enhancing rescue efficiency and safety. Fire UAVs (unmanned aerial vehicles) equipped with visual sensors have emerged as powerful tools for real-time monitoring of fire scenes, providing comprehensive environmental data that ground-based equipment often lacks. This paper presents a fire UAV-based pre-aiming system for fire monitors, which leverages visual information from UAVs to enable automated aiming adjustments. The system involves a three-stage control process: fire scene perception, pose calculation, and angle adjustment. By addressing challenges such as small target sizes and smoke interference in UAV imagery, we develop a lightweight perception model that combines haze removal and object detection. This approach not only improves image quality under smoky conditions but also ensures real-time performance suitable for emergency response. The proposed system demonstrates practical applicability in industrial fire scenarios, with experimental validation showing high accuracy in target detection and pre-aiming angle estimation. Below, we detail the methodologies, experimental setups, and results, emphasizing the role of fire UAVs in advancing intelligent firefighting solutions.
The fire UAV-based pre-aiming system comprises a UAV with a gimbal camera and communication modules, along with a firefighting robot equipped with a fire monitor and industrial computer. The workflow begins with the fire UAV capturing live video of the fire scene, which is transmitted via a 2.4 GHz wireless link to the firefighting robot. The robot processes the video frames sequentially, applying haze removal algorithms to enhance image clarity, followed by object and keypoint detection to identify fire sources and the robot’s own position. Key frames are selected from a short-term sliding window to mitigate transient干扰. Based on the perception results, the camera imaging model is used to compute the three-dimensional position of fire sources in the world coordinate system. The relative pose between the fire monitor and fire source is then derived, allowing for the calculation of ideal horizontal and pitch angles for pre-aiming. The fire monitor adjusts accordingly, directing water jets to cover the fire source, thus completing a “detection-processing-strike”灭火流程. This integrated system highlights the synergy between fire UAVs and ground robots, enabling rapid and accurate fire suppression.
In fire scenarios, smoke and uneven lighting from flames often degrade image quality, posing challenges for visual perception. Traditional haze removal methods, such as those based on the atmospheric scattering model (ASM), assume uniform haze distribution and fixed haze grayscale values, which are inadequate for fire environments where smoke exhibits non-uniform patterns and diverse grayscale levels. To address this, we propose an improved ASM that incorporates a learnable haze grayscale parameter. The model is expressed as:
$$ I(x) = J(x) t(x) + c(x)(1 – t(x)) $$
Here, \( I(x) \) is the input hazy image, \( J(x) \) is the ideal haze-free image, \( t(x) \) is the transmission map representing the portion of light reaching the camera, and \( c(x) \) denotes the haze grayscale value, which varies spatially. Unlike standard ASM with a constant atmospheric light, this formulation allows for adaptive haze characterization in fire UAV imagery. To solve for \( t(x) \) and \( c(x) \), we design a lightweight encoder-decoder neural network. The encoder uses ResNet-inspired shallow layers and FasterNet-Blocks in deeper layers for efficient feature extraction, while the decoder consists of two branches outputting \( t(x) \) and \( c(x) \), incorporating spatial attention and efficient channel attention mechanisms to enhance feature representation. The FasterNet-Block employs partial convolution (PConv), which reduces computational costs by convolving only a subset of input channels, as shown in the FLOPs calculation:
$$ \text{FLOPs}_{\text{FNB}} = h \times w \times (k^2 + 2) \times c_p^2 $$
where \( h \) and \( w \) are feature map dimensions, \( k \) is the kernel size, and \( c_p \) is the number of channels involved in convolution. The network is trained using a combined loss function:
$$ \mathcal{L} = \gamma_1 \mathcal{L}_1 + \gamma_2 \mathcal{L}_p + \gamma_3 \mathcal{L}_{\text{MS-SSIM}} $$
with \( \gamma_1 = 1 \), \( \gamma_2 = 0.01 \), and \( \gamma_3 = 0.4 \). The \( \mathcal{L}_1 \) loss ensures pixel-wise accuracy, perceptual loss \( \mathcal{L}_p \) leverages VGG-16 features for semantic similarity, and MS-SSIM loss \( \mathcal{L}_{\text{MS-SSIM}} \) preserves structural integrity. This haze removal approach effectively enhances fire UAV images, facilitating subsequent detection tasks.
For target detection in fire UAV imagery, we optimize the YOLOv8s baseline to handle small targets like fire sources and firefighting robots. The improved network includes modifications to the backbone and neck architectures. In the backbone, shallow convolutional operations are replaced with PSConv (Pinwheel-Shaped Convolution) modules, which offer concentrated receptive fields ideal for small target feature retention. Deep convolutional modules are substituted with GhostConv, reducing redundancy and computational load. Additionally, the deepest C2f structure is replaced with a SimA-Former module, which uses L1 normalization instead of softmax in attention mechanisms, lowering complexity while maintaining performance. The SimA-Former attention is computed as:
$$ O = \hat{Q}\hat{K}^T V, \quad \hat{Q}_i = \frac{Q_i}{\|Q_i\|_1}, \quad \hat{K}_i = \frac{K_i}{\|K_i\|_1} $$
where \( Q \), \( K \), and \( V \) are query, key, and value matrices, and \( \|\cdot\|_1 \) denotes the L1 norm. In the neck network, a high-resolution feature fusion module is added, integrating a 160×160 small target detection head (P2 layer) and multi-scale fusion paths inspired by BiFPN. A coordinate attention (CA) mechanism enhances feature aggregation by capturing spatial dependencies along horizontal and vertical directions. The fusion module combines backbone features \( X_B \) (enhanced via CA) with neck features \( X_1 \) and \( X_2 \) using learnable weights \( w \). This design improves detection accuracy for small targets in fire UAV footage, which is critical for reliable fire scene perception.
The pre-aiming process relies on pose calculation derived from detection results. Using the camera imaging model, the 3D position of fire sources is computed. Let the fire monitor’s position be the origin \( O_w(0,0,0) \) in the world coordinate system, and a fire source point be \( X_w(x_w, y_w, 0) \). The mapping to pixel coordinates \( X_p(u, v) \) involves multiple transformations. First, world coordinates are converted to camera coordinates \( X_c(x_c, y_c, H) \) via rigid transformation:
$$ X_c = R X_w + t, \quad R = \begin{bmatrix} \cos \alpha & -\sin \alpha & 0 \\ \sin \alpha & \cos \alpha & 0 \\ 0 & 0 & 1 \end{bmatrix} $$
where \( R \) is the rotation matrix dependent on UAV gimbal yaw angle \( \alpha \), \( t = [x_u, y_u, H]^T \) is the translation vector, and \( H \) is UAV altitude. Projection onto the image plane with focal length \( f \) yields image coordinates \( X_i(x_i, y_i) \):
$$ x_i = f \cdot \frac{x_c}{H}, \quad y_i = f \cdot \frac{y_c}{H} $$
Conversion to pixel coordinates involves pixel dimensions \( dx \), \( dy \) and offset \( (u_0, v_0) \):
$$ u = \frac{x_i}{dx} + u_0, \quad v = \frac{y_i}{dy} + v_0 $$
Combining these steps, the imaging equation is:
$$ H \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = K \cdot [R \ t] \cdot \begin{bmatrix} X_w \\ 1 \end{bmatrix} $$
where \( K \) is the intrinsic matrix. Distortion correction is applied using coefficients from camera calibration. For fire monitor orientation, keypoints on the firefighting robot are detected: a rear point \( P_b \) and four corner points \( P_{lu}, P_{ru}, P_{ld}, P_{rd} \) forming a quadrilateral. Vectors \( \vec{r}_1, \vec{r}_2, \vec{r}_3, \vec{r}_4 \) are derived perpendicular to the quadrilateral sides, and the robot’s orientation \( \vec{r} \) is computed as their sum. The angle \( \theta \) between \( \vec{r} \) and the vector to the fire source determines the horizontal adjustment, while the distance \( O_wX_w \) guides pitch angle adjustment based on water jet range. This method enables precise pre-aiming, with the fire UAV providing essential visual data for real-time calculations.
Experiments were conducted in a custom-built fire scenario outside an industrial facility, involving gasoline-ignited fire sources and a firefighting robot with a water jet rate of 30 L/s and pressure of 0.11 MPa. The fire UAV was equipped with a gimbal camera of 24 mm focal length and 1280×720 resolution. A dataset of 960 images was collected, with 780 for training and 180 for testing, augmented via rotations, scaling, and splicing. The haze removal network was trained on synthetic hazy images generated using Perlin noise with random grayscale values (120–255) and transmission rates (0.1–0.8). Evaluation metrics for haze removal included Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM), defined as:
$$ \text{PSNR} = 10 \times \log_{10}\left(\frac{\text{MAX}_I^2}{\text{MSE}}\right) $$
and SSIM as in Equation (6) earlier. For object detection, precision (P), recall (R), average precision (AP), and mean average precision (mAP) were used, with mAP50 and mAP50:95 reported. Computational metrics included parameters, GFLOPs, and inference time. The fire UAV’s role was central in data acquisition, underscoring its value in dynamic fire environments.
| Method | PSNR | SSIM | Params (M) | GFLOPs |
|---|---|---|---|---|
| Original Image | 13.15 | 0.66 | – | – |
| DCP | 15.69 | 0.72 | – | – |
| AODNet | 18.78 | 0.74 | 1.76K | 1.68 |
| GridDehazeNet | 19.43 | 0.77 | 702.47K | 271.95 |
| FFA-Net | 24.32 | 0.81 | 4.46M | 287.50 |
| TNN | 29.76 | 0.85 | 50.35M | 68.36 |
| DeHamer | 26.24 | 0.88 | 29.44M | 48.93 |
| Our Method | 27.91 | 0.86 | 8.20M | 24.81 |
The haze removal results show that our method balances performance and efficiency, with a PSNR of 27.91 dB and SSIM of 0.86, outperforming traditional methods while maintaining lower computational costs. This is vital for fire UAV applications where real-time processing is needed. For object detection, the improved YOLOv8s model achieved a mAP50 of 92.3% when combined with haze removal, compared to 86.1% for baseline YOLOv8s without haze removal, representing a 6.2% improvement. Detection results for various targets are summarized below:
| Method | Fire | Firefighting Robot | Person | Water Jet | mAP50 (%) |
|---|---|---|---|---|---|
| YOLOv8s | 74.2 | 98.7 | 82.7 | 88.6 | 86.1 |
| Improved YOLOv8s | 79.5 | 98.8 | 86.6 | 90.2 | 88.8 |
| Dehaze + YOLOv8s | 80.4 | 98.8 | 83.0 | 92.5 | 88.7 |
| Dehaze + Improved YOLOv8s | 85.1 | 99.3 | 88.8 | 95.9 | 92.3 |
Ablation studies on object detection components are presented, highlighting the contributions of each modification:
| Modules | mAP50 (%) | mAP50:95 (%) | Params (M) | GFLOPs | Inference Time (ms) |
|---|---|---|---|---|---|
| Baseline | 88.7 | 54.8 | 11.1 | 28.6 | 7.9 |
| + High-res Fusion | 91.5 | 61.0 | 13.4 | 41.3 | 11.2 |
| + Backbone Mods | 90.1 | 55.4 | 8.5 | 22.4 | 6.7 |
| + SimA-Former | 88.9 | 55.0 | 9.6 | 24.7 | 7.9 |
| + Fusion & Backbone | 92.2 | 62.5 | 10.8 | 35.1 | 8.6 |
| All Improvements | 92.3 | 62.8 | 9.3 | 31.2 | 8.6 |
Comparative experiments with other detection models further validate our approach:
| Method | mAP50 (%) | mAP50:95 (%) | Params (M) | GFLOPs | Inference Time (ms) |
|---|---|---|---|---|---|
| YOLOv5m | 86.2 | 57.0 | 21.2 | 49.0 | 15.3 |
| YOLOv8m | 89.8 | 58.6 | 25.9 | 78.9 | 14.1 |
| YOLOv9m | 90.9 | 59.4 | 20.1 | 76.8 | 13.8 |
| YOLOv10m | 90.8 | 60.1 | 15.4 | 59.1 | 12.4 |
| Our Method | 92.3 | 62.8 | 17.5 | 56.0 | 13.7 (total) |
For pre-aiming evaluation, horizontal angle error and fire source distance error were measured. Results show that horizontal angle error remains within ±4°, and distance error is below 6%, meeting the requirements for initial aiming adjustments. The fire UAV’s visual data proved instrumental in achieving these accuracies, enabling the fire monitor to effectively cover fire sources with water jets. In tests, the system successfully directed水流 to extinguish multiple fire sources, demonstrating robustness in real-world conditions. The integration of fire UAVs not only enhances perception but also facilitates autonomous decision-making, reducing response times in emergency scenarios.
In conclusion, we have developed a fire UAV-based pre-aiming system that combines advanced haze removal and object detection for intelligent firefighting. The system addresses key challenges in fire scene perception, such as smoke interference and small target detection, through lightweight neural network designs. Experimental results confirm the effectiveness of our methods, with significant improvements in detection accuracy and pre-aiming precision. The fire UAV serves as a critical component, providing high-quality visual information that drives automated adjustments in fire monitors. Future work will focus on enhancing system reliability under extreme conditions, such as heavy smoke or high temperatures, by incorporating reinforcement learning for UAV path planning and end-to-end training for joint haze removal and detection. Additionally, instance segmentation could be explored for better handling of irregular fire sources. This research underscores the potential of fire UAVs in transforming firefighting operations, paving the way for more autonomous and efficient救援 solutions. The continuous evolution of fire UAV technology will likely lead to even more integrated systems, where multiple UAVs collaborate with ground units for comprehensive fire management.
The fire UAV-based approach not only improves operational efficiency but also enhances safety by minimizing human exposure to hazardous environments. As demonstrated, the pre-aiming system reduces manual intervention, allowing firefighters to focus on strategic决策. The use of fire UAVs for real-time monitoring and data transmission is a testament to the growing role of aerial platforms in emergency response. With further advancements in AI and sensor technology, fire UAVs could be equipped with multimodal sensors, such as thermal cameras, to detect hidden fire sources or assess structural integrity. Moreover, swarm coordination among fire UAVs could enable large-scale scene coverage and redundant data collection, ensuring robustness against communication failures. These prospects highlight the importance of continued research in this domain, driven by the need for smarter and safer firefighting tools. Ultimately, the synergy between fire UAVs and intelligent algorithms will redefine how we combat fires, making communities more resilient to such disasters.