The increasing frequency of fire accidents highlights the critical need for intelligent rescue technologies. In firefighting operations, ground-based equipment often suffers from limited field of view, making it difficult to obtain comprehensive situational awareness. Unmanned Aerial Vehicles (UAVs), or fire drones, equipped with gimbal cameras and communication modules, can provide a superior aerial perspective for real-time monitoring. They capture rich, panoramic information about the fire scene and transmit it to ground units. By processing this data, ground-based firefighting apparatus, such as monitors, can adjust their aiming behavior. The aiming process typically involves an initial pre-aiming adjustment to direct the water stream, followed by dynamic fine-angle control based on extinguishing effectiveness. This work focuses on fire scene perception from a UAV perspective and explores a pre-aiming methodology for fire monitors based on the visual information acquired by the fire drone.
In fire scenarios, non-uniform smoke significantly degrades image quality. Image dehazing methods can be broadly categorized into prior-based and learning-based approaches. The Atmospheric Scattering Model (ASM) is a classical prior model, described by:
$$I(x) = J(x) t(x) + A (1 – t(x))$$
where $I(x)$ is the observed hazy image, $J(x)$ is the latent clear image, $t(x)$ is the transmission map, and $A$ is the global atmospheric light. Traditional methods like the Dark Channel Prior (DCP) estimate these parameters using statistical rules but often cause color distortion. With advancements in deep learning, convolutional neural networks (CNNs) have been employed for direct parameter estimation or end-to-end haze removal. However, in fire scenarios, smoke exhibits non-uniform distribution and diverse grayscale values, unlike the typically high-grayscale atmospheric haze. Furthermore, the presence of flickering flames adds to the complexity. Therefore, conventional dehazing algorithms often underperform in firefighting contexts.
Target detection is another crucial technology for UAV-based fire scene perception. Deep learning-based object detectors are mainly two-stage (e.g., R-CNN, Faster R-CNN) or single-stage (e.g., SSD, YOLO series). Single-stage methods offer a better balance between speed and accuracy, making them suitable for real-time applications. Keypoint detection, essential for estimating object orientation, can be achieved via heatmap-based or regression-based methods. While significant progress has been made in general vision tasks, UAV-captured images present unique challenges such as complex backgrounds and a high proportion of small objects, leading to missed or false detections by generic algorithms. Research has addressed this through specialized datasets (e.g., VisDrone), attention mechanisms (e.g., CBAM, BiFormer), multi-modal fusion, model pruning, and dedicated small-object detection heads.
Despite these advances, two key issues remain for visual perception in firefighting scenarios: 1) The diverse and often lower grayscale values of smoke from combustion reduce the effectiveness of traditional dehazing algorithms. 2) The real-time requirements of firefighting preclude the use of overly complex neural network architectures that increase computational latency. There is a pressing need to develop a lightweight yet effective real-time perception model tailored to the complexities of fire rescue operations.
To address these challenges, this work proposes a fire cannon pre-aiming system based on visual information from a fire drone. The system comprises a UAV and a firefighting robot. The workflow involves a reconnaissance phase where the UAV streams video to the robot. The robot processes the frames through a perception model, selects key frames, and during the decision-execution phase, calculates the required pan and tilt angles for the fire monitor to direct the water stream onto the fire source, completing the “detect-process-strike” workflow.
Joint Perception Model: Dehazing and Detection
The core of the system is a cascaded perception model that first applies image dehazing, followed by object and keypoint detection.
1. Image Dehazing for Fire Scenes
Conventional ASM assumes uniformly distributed haze with a fixed, high grayscale value (often white), which is not suitable for fire scenes with non-uniform smoke and varied light. We propose an improved ASM:
$$I(x) = J(x) t(x) + c(x)(1 – t(x))$$
Here, $c(x)$, representing the spatially varying haze color/intensity, replaces the global atmospheric light $A$. Solving for $t(x)$ and $c(x)$ allows recovery of $J(x)$.
To estimate $t(x)$ and $c(x)$, we design a lightweight encoder-decoder neural network. The encoder uses a ResNet-inspired shallow structure and deep layers with FasterNet-Blocks (based on Partial Convolution, PConv) for efficient feature extraction. The decoder has two branches, each with a U-Net-like structure comprising transposed convolutions, skip connections, and residual blocks. The shallow residual blocks incorporate Spatial Attention (SA) and Efficient Channel Attention (ECA) mechanisms to enhance feature representation. The network is trained using a combined loss function:
$$\mathcal{L} = \gamma_1 \mathcal{L}_{1} + \gamma_2 \mathcal{L}_{p} + \gamma_3 \mathcal{L}_{MS-SSIM}$$
where $\mathcal{L}_{1}$ is the L1 loss, $\mathcal{L}_{p}$ is the perceptual loss using VGG-16 features, and $\mathcal{L}_{MS-SSIM}$ is the multi-scale structural similarity loss. Training data is synthesized by adding Perlin noise-based haze layers with random grayscale values (120-255) and transmission rates (0.1-0.8) to clear fire scene images.
2. Enhanced Object and Keypoint Detection
For the detection task, we build upon the YOLOv8s architecture, modifying it to better handle small targets like fire sources and firefighting robots from a fire drone view, while maintaining real-time performance.
- Backbone Network Modifications: In the shallow layers (B1-B3), standard convolutions are replaced with PSConv (Pinwheel-Shaped Convolution). PSConv offers a concentrated, Gaussian-like receptive field, making it more adept at capturing features of small objects. In the deeper layers (B4-B5), convolutions are replaced with GhostConv, which generates redundant “ghost” features via cheap linear operations, significantly reducing computational load. The deepest C2f module is replaced with a SimA-Former block. SimA-Former uses L1 normalization on the Query and Key matrices instead of the computationally expensive softmax in standard Transformers, maintaining global context modeling capabilities with lower cost.
- Neck Network Enhancement: We augment the FPN-PAN structure with a high-resolution feature fusion mechanism. An additional detection head (P2) at 160×160 resolution is added for small objects. Inspired by BiFPN, we introduce extra cross-scale connections from backbone features B3 and B4 to the neck. A dedicated Fusion module combines these features using learnable weights and an enhanced Coordinate Attention (CA) mechanism. The enhanced CA replaces average pooling with combined average and max pooling for richer feature aggregation.
The complete improved YOLOv8 architecture integrates these components to form a lightweight yet powerful detector.
Fire Cannon Pre-Aiming Calculation
Pre-aiming requires adjusting the monitor’s pan (horizontal) and tilt (pitch) angles based on the relative pose between the fire monitor and the fire source, derived from the fire drone‘s perception.
1. Fire Source Localization
We assume the UAV gimbal is adjusted so its imaging plane is parallel to the ground. The world coordinate system $O_w$ has its origin at the fire monitor’s position. A fire source at $X_w = (x_w, y_w, 0)$ is projected onto the pixel coordinate $X_p = (u, v)$ through the camera model:
$$ \lambda \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = \mathbf{K} [\mathbf{R} | \mathbf{t}] \begin{bmatrix} X_w \\ 1 \end{bmatrix} $$
where $\lambda$ is a scale factor, $\mathbf{K}$ is the camera intrinsic matrix, $\mathbf{R}$ is the rotation matrix (dependent on UAV yaw when the imaging plane is parallel to ground), and $\mathbf{t} = [x_u, y_u, H]^T$ is the translation vector with $H$ being the UAV altitude. Lens distortion is corrected using coefficients $[k_1, k_2, p_1, p_2, k_3]$ obtained via calibration. Given the detected pixel coordinates $X_p$ (after distortion correction), the 3D world coordinates $X_w$ of the fire source can be solved using the above model. The distance $d = ||O_w X_w||$ is then calculated for pitch angle adjustment.
2. Monitor Orientation and Pan Angle Calculation
The firefighting robot’s orientation is determined via keypoint detection. Five keypoints are defined: one at the rear ($P_b$) and four at the corners ($P_{lu}, P_{ru}, P_{ld}, P_{rd}$) forming a quadrilateral $Q$. The intersection of its diagonals ($O$) is the monitor’s pivot point. Vectors $\vec{r_1} \perp \overline{P_{lu}P_{ru}}$, $\vec{r_2} \perp \overline{P_{ld}P_{rd}}$, $\vec{r_3} \perp \overline{P_{lu}P_{ru}}$ (from $P_b$), and $\vec{r_4} \perp \overline{P_{ld}P_{rd}}$ (from $P_b$) are computed. The robot’s forward orientation $\vec{r}$ is estimated as the sum of these four vectors to average out small detection errors:
$$\vec{r} = \vec{r_1} + \vec{r_2} + \vec{r_3} + \vec{r_4}$$
The required pan angle $\theta$ is the angle between $\vec{r}$ and the vector $\overrightarrow{O X_w}$ from the monitor to the fire source.
Experimental Results and Analysis
Experiments were conducted in a custom-built outdoor fire scenario. A fire drone monitored the scene, transmitting video to a firefighting robot which processed the data to control the water monitor.
1. Image Dehazing Performance
The proposed dehazing network was trained on synthesized hazy images and compared against several state-of-the-art methods on a fire scene test set. Performance was evaluated using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), number of parameters, and GFLOPs.
| Method | PSNR | SSIM | Params (M) | GFLOPs |
|---|---|---|---|---|
| Hazy Input | 13.15 | 0.66 | – | – |
| DCP | 15.69 | 0.72 | – | – |
| AOD-Net | 18.78 | 0.74 | 0.0018 | 1.68 |
| GridDehazeNet | 19.43 | 0.77 | 0.70 | 271.95 |
| FFA-Net | 24.32 | 0.81 | 4.46 | 287.50 |
| TNN | 29.76 | 0.85 | 50.35 | 68.36 |
| DeHamer | 26.24 | 0.88 | 29.44 | 48.93 |
| Ours | 27.91 | 0.86 | 8.20 | 24.81 |
Our method achieves a favorable balance between restoration quality (high PSNR/SSIM) and computational efficiency (moderate Params and GFLOPs), making it suitable for the real-time pipeline of the fire drone system.
2. Object Detection Performance
The improved YOLOv8 model was trained on a custom fire scene dataset (960 images) to detect fire sources, the firefighting robot, people, and water stream impact points. Testing was performed on images with synthetic smoke interference, with and without the dehazing pre-processing step. Mean Average Precision at IoU=0.5 (mAP@50) is the primary metric.
| Detection Method | mAP@50 (%) |
|---|---|
| YOLOv8s (Baseline) | 86.1 |
| Improved YOLOv8s | 88.8 |
| Dehaze + YOLOv8s | 88.7 |
| Dehaze + Improved YOLOv8s (Ours) | 92.3 |
The cascaded model (our full perception system) achieves the highest mAP@50 of 92.3%, a 6.2% improvement over the baseline without dehazing. This demonstrates the synergistic effect of the dedicated dehazing module and the enhanced detector.
An ablation study on the detection model improvements confirms the contribution of each component:
| High-Res Fusion (A) | Backbone Conv (B) | SimA-Former (C) | mAP@50 (%) | Params (M) | GFLOPs |
|---|---|---|---|---|---|
| 88.7 | 11.1 | 28.6 | |||
| ✓ | 91.5 | 13.4 | 41.3 | ||
| ✓ | 90.1 | 8.5 | 22.4 | ||
| ✓ | 88.9 | 9.6 | 24.7 | ||
| ✓ | ✓ | 92.2 | 10.8 | 35.1 | |
| ✓ | ✓ | ✓ | 92.3 | 9.3 | 31.2 |
The final model achieves superior accuracy (92.3% mAP@50) with reduced parameters (9.3M) and manageable GFLOPs compared to the baseline.
Comparison with other recent detectors on our fire scene dataset further validates the effectiveness of our cascaded approach:
| Method | mAP@50 (%) | Params (M) | GFLOPs |
|---|---|---|---|
| YOLOv5m | 86.2 | 21.2 | 49.0 |
| YOLOv8m | 89.8 | 25.9 | 78.9 |
| YOLOv9m | 90.9 | 20.1 | 76.8 |
| YOLOv10m | 90.8 | 15.4 | 59.1 |
| Ours (Cascaded) | 92.3 | 17.5 | 56.0 |
3. Pre-Aiming Accuracy
The overall system was tested in real-world fire suppression experiments. After pre-aiming, the water stream successfully covered the target fire source. Quantitative evaluation measured the pan angle error and the fire source distance estimation error across multiple trials.
- Pan Angle Error: Remained within ±4° across the tested range (-70° to 70°).
- Distance Estimation Error: Remained below 6% for distances between 15m and 60m.
These errors are within an acceptable range for the initial pre-aiming adjustment, providing a solid foundation for subsequent fine control.
Conclusion
This study presents a comprehensive fire cannon pre-aiming system based on visual information from a fire drone. The core innovation is a joint perception model that cascades a dedicated image dehazing algorithm with an enhanced object detector. The dehazing algorithm introduces an improved atmospheric scattering model with a spatially varying haze term, estimated via a lightweight encoder-decoder network, effectively handling non-uniform smoke typical of fire scenes. The detector, built upon YOLOv8s, incorporates PSConv and GhostConv for efficient small-object feature extraction, a high-resolution multi-scale fusion neck with coordinate attention, and a SimA-Former block for lightweight global context modeling. The perception output is fed into a geometric model to calculate the relative pose between the fire monitor and the fire source, enabling the computation of necessary pan and tilt angles for pre-aiming.
Experimental results demonstrate the effectiveness of each component. The dehazing algorithm provides quality enhancement suitable for real-time use. The improved detector achieves a mAP@50 of 92.3% on fire scene data, significantly outperforming the baseline. The integrated system achieves a pan angle error within ±4° and a distance error under 6%, successfully directing the water stream to cover the fire source for initial attack. This work provides a viable “perception-decision” framework for intelligent firefighting, enhancing situational awareness and response efficiency through the use of a fire drone.
Limitations include potential communication degradation under intense dynamic interference (thick smoke, heat) and challenges with highly irregular fire shapes. Future work will focus on enhancing system robustness, potentially using reinforcement learning for UAV path planning to optimal viewpoints, exploring end-to-end joint training of the dehazing and detection networks, and employing instance segmentation for more precise localization of irregular fire sources.