In the context of emergency disaster response, particularly flooding, the ability to rapidly assess building inundation status from aerial imagery is critical. Traditional approaches rely heavily on offboard processing, requiring high-resolution image transmission and complex spatial overlay analysis, which introduces significant latency and bandwidth constraints. To address these challenges, we propose FloodSAM-Duo (Dual-scale Prompt-guided SAM for Flooded Building Extraction), a lightweight framework designed for real-time, object-level detection of flooded buildings directly onboard a UAV drone. Our method integrates dual-scale weak semantic reasoning with prompt-guided instance segmentation, enabling the UAV drone to generate structured disaster information in milliseconds without continuous image downlink. In this paper, we detail the methodology, present extensive experimental results on real flood datasets, and demonstrate superior performance in accuracy, inference speed, and robustness compared to existing lightweight models.
The core innovation of FloodSAM-Duo lies in its ability to transform a single UAV drone image into actionable building-level flood information through a coarse-to-fine collaborative inference mechanism. First, a lightweight Pyramid Scene Parsing Network (PSPNet) extracts weak semantic responses for buildings and water at two complementary scales, capturing both global spatial relationships and local boundary details. These responses are used to infer candidate flooded building regions without requiring pixel-level ground truth or pre‑disaster reference imagery. Second, within these candidate regions, semantically stable prompt points are automatically generated to guide a lightweight instance segmentation model, FastSAM, to produce structurally complete and boundary‑consistent building masks. Finally, the instance masks are converted into a structured JSON-based information set containing the spatial positions and counts of flooded buildings, which can be transmitted and aggregated without high-resolution image streaming.
We evaluate our method on UAV drone imagery acquired during recent flood events in Suizhou and Xinxiang, China, using a Jetson Xavier NX onboard computing platform. The results demonstrate that FloodSAM-Duo achieves a single-frame inference latency of only 15–20 ms, outperforming representative lightweight models such as ENet (18–25 ms), YOLOv8n-seg (25–40 ms), and RT-DETR Tiny (40–60 ms). In terms of detection accuracy, FloodSAM-Duo improves F1-score by approximately 18% and mean Intersection over Union (mIoU) by about 17% over these baselines, while also achieving higher boundary consistency as measured by BF1. Moreover, the method exhibits strong robustness in negative‑sample scenarios (e.g., non‑flooded urban areas or water‑only scenes), with a false‑positive rate as low as 0.25 per image. These findings confirm that FloodSAM-Duo provides a practical, light‑weight, and efficient solution for real‑time building damage assessment from UAV drones during flood emergencies.
1. Introduction
Unmanned Aerial Vehicles (UAV drones) have become indispensable tools for emergency response due to their high mobility, rapid deployment, and ability to capture very high‑resolution imagery at low altitudes. In flood disaster scenarios, the UAV drone can quickly overfly affected areas and provide near‑real‑time visual data that is crucial for assessing the extent of inundation and prioritizing rescue efforts. Among the various types of damage, the status of buildings—whether they are partially or fully flooded—directly indicates potential human casualties, the need for evacuation, and the allocation of relief resources. Therefore, detecting flooded buildings from UAV drone imagery is a key task in disaster informatics.
Conventional approaches to flooded building detection often rely on offline, ground‑based processing. They require the UAV drone to continuously transmit high‑resolution images to a ground station, where sophisticated segmentation models (e.g., DeepLab, U‑Net) and spatial overlay analysis are performed. This workflow suffers from several limitations: (1) it demands stable and high‑bandwidth communication links, which are frequently unavailable in disaster‑stricken areas; (2) the pipeline latency—from image capture to final information—is often tens of seconds or even minutes, which is unacceptable for real‑time decision‑making; and (3) the reliance on full‑resolution pixel‑level segmentation makes the system computationally expensive and sensitive to segmentation errors caused by water reflections, shadows, and domain shifts.
To overcome these limitations, researchers have explored onboard processing using lightweight deep learning models. For example, Hernández et al. deployed a lightweight semantic segmentation network on an edge‑computing platform for real‑time flood detection from a UAV drone. Zhao et al. proposed a real‑time rescue target detection method using YOLO. Kyrkou et al. designed EmergencyNet for efficient aerial image classification. However, these methods typically operate at the region or bounding‑box level, which is insufficient for representing the precise geometry of buildings and for enabling reliable cross‑frame aggregation. Instance‑level representation—where each building is identified with a pixel‑wise mask—is far more valuable for downstream tasks such as counting, tracking, and damage level estimation.
Nevertheless, performing instance segmentation on a UAV drone in real‑time is challenging due to limited onboard computational resources (e.g., GPU, memory, power). Lightweight instance segmentation models like YOLOv8n‑seg and RT‑DETR Tiny can run on embedded platforms like Jetson Xavier NX, but they often produce fragmented boundaries and missing structures in complex flood scenes, leading to unreliable building delineation. The presence of water reflections, shadows, and texture‑degraded regions further exacerbates these issues.
In this paper, we introduce FloodSAM‑Duo, a novel framework that leverages the synergy between weak semantic reasoning and prompt‑guided fine segmentation to achieve high‑quality, object‑level flooded building detection directly onboard a UAV drone. The key contributions are:
- Dual‑scale weak semantic inference: A lightweight PSPNet extracts building and water responses at two complementary scales (large and small), enabling robust candidate region proposal without pixel‑level labels.
- Semantic‑stable prompt generation: Automatically generates trustworthy point prompts from the weak semantic responses within candidate regions, which guide FastSAM to perform local fine segmentation, resulting in structurally complete and boundary‑consistent masks.
- Onboard real‑time performance: The entire pipeline achieves 15–20 ms latency per frame on Jetson Xavier NX, making it suitable for near‑real‑time disaster monitoring.
- Structured output: Instance masks are converted into lightweight JSON‑based information (e.g., center coordinates, building counts), reducing the need for high‑resolution image transmission and enabling efficient multi‑frame fusion.
The remainder of the paper is organized as follows. Section 2 details the methodology. Section 3 presents experimental setup and results. Section 4 discusses the implications and limitations. Section 5 concludes the paper.
2. Methodology
FloodSAM‑Duo is designed as a coarse‑to‑fine inference pipeline that consists of three main modules: (i) dual‑scale weak semantic generation and candidate region inference, (ii) semantic‑stable prompt point generation and guided instance segmentation, and (iii) structured output generation and multi‑frame fusion. The entire workflow is optimized for low‑latency execution on a UAV drone’s embedded GPU.
2.1 Dual‑Scale Weak Semantic Generation and Candidate Flooded Region Inference
The first module aims to quickly identify potential flooded building regions from a single UAV drone image. We employ a lightweight Pyramid Scene Parsing Network (PSPNet) to generate weak semantic responses for two classes: “building” and “water”. These responses are not high‑fidelity segmentation maps but rather low‑resolution probability fields that capture the relative likelihood of each class. The key idea is to exploit the complementary nature of large‑scale and small‑scale features. Large‑scale features capture the overall spatial relationship between buildings and surrounding water bodies, while small‑scale features preserve local details such as building edges and corners.
Let the input UAV drone image be denoted as $$I \in \mathbb{R}^{H \times W \times 3}$$. The PSPNet encoder extracts a multi‑scale feature tensor $$F \in \mathbb{R}^{h \times w \times c}$$, where $$h \ll H$$ and $$w \ll W$$. From this tensor, we construct two weak semantic representations:
- Large‑scale semantic feature $$F^{(L)}$$ obtained via global pyramid pooling (e.g., 1×1 or 2×2 pooling): $$F^{(L)} = \mathcal{P}_{\text{global}}(F)$$. This feature models the overall morphology of buildings and their spatial association with water bodies, providing a stable estimate of the macro‑level inundation context.
- Small‑scale semantic feature $$F^{(S)}$$ obtained via local pyramid pooling (e.g., 4×4 or 8×8 pooling): $$F^{(S)} = \mathcal{P}_{\text{local}}(F)$$. This feature retains information about building boundaries, corners, and fine structural details, enabling robustness against water reflections and texture degradation.
These two features are then aligned in spatial resolution and concatenated along the channel dimension:
$$
F^{(D)} = \phi\left( \text{Concat}(F^{(L)}, F^{(S)}) \right)
$$
where $$\phi(\cdot)$$ is a lightweight 1×1 convolution that performs channel compression and cross‑scale information interaction. The resulting dual‑scale fused feature $$F^{(D)}$$ has the same spatial size as the encoder output but with richer semantic information.
Two separate prediction heads are applied to $$F^{(D)}$$ to generate building and water weak semantic response maps:
$$
S_b = \sigma(W_b \cdot F^{(D)}), \quad S_w = \sigma(W_w \cdot F^{(D)})
$$
where $$S_b, S_w \in [0,1]^{h \times w}$$ are the response maps, and $$\sigma(\cdot)$$ is the sigmoid activation function. These maps indicate, for each spatial location, the confidence of belonging to the building or water class.
Based on these weak semantic responses, we infer candidate flooded building regions. A pixel region $$\Omega_b \subset S_b$$ is considered a potential flooded building candidate if it satisfies any of the following structural‑spatial conditions:
- The building response region is surrounded by high‑intensity water response at the large scale.
- The building boundary is adjacent to a significant water response region at the small scale.
- There is a weak semantic “hole” within the building region where water response dominates.
- The water response in the local neighborhood of the building exceeds an adaptive threshold $$\tau_w$$.
This condition‑based inference is computationally inexpensive and does not require precise mask‑overlap calculations. It effectively screens the entire image and produces a set of candidate image patches (or bounding boxes) that are likely to contain flooded buildings. Only these candidate regions are forwarded to the next module, drastically reducing the computational load.
2.2 Semantic‑Stable Prompt Point Generation and Guided Instance Segmentation
The second module converts the coarse weak semantic responses into precise instance masks using a guided segmentation strategy. Instead of feeding the entire image or the candidate patch directly to a segmentation model, we first generate a sparse set of point prompts that serve as spatial anchors for the model.
Let the building weak semantic response map within a candidate region be $$S_b^{\text{crop}} \in [0,1]^{h_c \times w_c}$$. We perform local extrema detection to extract initial candidate prompt points:
$$
P_0 = \{ p_i \mid S_b^{\text{crop}}(p_i) > S_b^{\text{crop}}(q), \forall q \in \mathcal{N}(p_i) \}
$$
where $$\mathcal{N}(p_i)$$ is a local neighborhood around pixel $$p_i$$. These points correspond to the strongest semantic responses and are likely located inside building bodies.
To avoid excessive clustering of prompts in high‑response areas, we apply a spatial uniformity constraint: for any two candidate points $$p_i, p_j \in P_0$$, if $$\lVert p_i – p_j \rVert_2 < d_{\min}$$ (with $$d_{\min}$$ set to 5 pixels in our implementation), only the point with higher response value is retained. Furthermore, the candidate region is divided into a grid of subregions (e.g., 3×3), and in each subregion only the most confident point is kept. This yields a set of semantically stable prompt points $$P = \{p_k\}$$ distributed evenly across the building.
These point prompts are then input to FastSAM in a prompt‑guided inference mode. FastSAM is a lightweight variant of the Segment Anything Model (SAM) that can generate instance masks given point prompts. By confining the segmentation task to only the candidate regions and providing well‑placed prompts, FastSAM can produce high‑quality building masks with clean boundaries and complete structures, even under challenging conditions like water reflections and shadows. The output of this module is a set of binary instance masks $$\mathcal{M}_i$$ for each detected flooded building within the candidate region.
2.3 Structured Output and Multi‑Frame Fusion
The final module transforms instance masks into a compact, transmittable form. To facilitate onboard aggregation across consecutive frames (since a UAV drone often captures overlapping images), we represent each flooded building by its geometric center $$(x_i, y_i)$$ and attach a unique identifier. The set of buildings detected in a single frame is:
$$
B_t = \{ (x_i^t, y_i^t) \}_{i=1}^{N_t}
$$
where $$N_t$$ is the number of detected buildings in frame $$t$$.
When multiple frames are processed, we apply a spatial‑distance‑based fusion strategy to avoid double counting. For two buildings from different frames $$t$$ and $$t+1$$, if the Euclidean distance satisfies:
$$
d_{ij} = \sqrt{ (x_i^t – x_j^{t+1})^2 + (y_i^t – y_j^{t+1})^2 } < \tau
$$
with $$\tau = 3\text{–}5\,\text{m}$$ (corresponding to about 10–20 pixels at 0.2–0.5 m resolution), they are considered the same building and their positions are averaged. For clusters of many detections, we can further simplify by outputting the cluster center and the count of buildings within the cluster.
The final output is a JSON structure containing an array of flooded building objects, each with fields like id, longitude, latitude, and count (if aggregated). This structured information is extremely lightweight (e.g., a few hundred bytes per frame) and can be reliably downlinked even under poor communication conditions.
3. Experimental Setup and Results
3.1 Onboard Platform and Model Efficiency
We deployed FloodSAM‑Duo on an NVIDIA Jetson Xavier NX module operating in 15 W mode, a standard embedded platform for UAV drones. We compared our method with three representative lightweight approaches that are deployable on the same hardware: ENet (lightweight semantic segmentation), YOLOv8n‑seg (lightweight instance segmentation), and RT‑DETR Tiny (lightweight object detection with instance masks). Table 1 summarizes the model size, memory footprint, and single‑frame inference latency.
| Method | Parameters (M) | Model Size (MB) | GPU Memory (GB) | Latency (ms) |
|---|---|---|---|---|
| ENet (lightweight semantic segmentation) | 0.36 | 1.6 | 1.2 | 18–25 |
| YOLOv8n‑seg (lightweight instance segmentation) | 3.2 | 12 | 1.8 | 25–40 |
| RT‑DETR Tiny (lightweight object detection) | 4.7 | 19 | 2.3 | 40–60 |
| FloodSAM‑Duo (Ours) | 2.9 | 10 | 1.6 | 15–20 |
As shown, FloodSAM‑Duo achieves the lowest latency (15–20 ms) while keeping model size and memory consumption comparable to the baselines. The efficiency stems from the decoupled design: the lightweight PSPNet operates on low‑resolution feature maps for candidate screening, and FastSAM performs local segmentation only on candidate regions, avoiding full‑resolution dense inference. This makes our method particularly suitable for real‑time deployment on a UAV drone.
3.2 Flooded Building Detection Performance
We evaluated detection performance using two real‑world UAV drone datasets: (i) Suizhou flood dataset – containing about 3 000 image patches of size 512×512 pixels at 0.2–0.5 m resolution, with 70% for training, 15% for validation, and 15% for testing; (ii) Xinxiang flood dataset – used exclusively for cross‑region generalization testing, with no overlap in training. All images were captured by UAV drones during actual flood disasters, covering diverse scenarios including dense urban areas, rural settlements, water reflections, shadows, and heavily degraded textures.
We trained only the PSPNet weak‑semantic generator using supervised learning with building and water labels (coarse polygon annotations, not pixel‑wise instance masks). FastSAM was used in its off‑the‑shelf pre‑trained form without any fine‑tuning. The training loss was a weighted cross‑entropy. We employed Adam optimizer with initial learning rate 1e‑4, batch size 16, and standard data augmentation (random crop, flip, color jitter).
We report the following metrics: Precision, Recall (for flooded buildings), F1‑score, Average Precision for flooded class (AP_Flooded), instance‑level mean Intersection over Union (mIoU_building_inst), boundary F1 (BF1_boundary), and Miss Rate. Table 2 presents the quantitative results.
| Method | Precision | Recall_Flooded | F1_Flooded | AP_Flooded | mIoU_building_inst | BF1_boundary | Miss Rate |
|---|---|---|---|---|---|---|---|
| ENet | 0.63 | 0.55 | 0.59 | 0.48 | 0.52 | 0.58 | 0.45 |
| YOLOv8n‑seg | 0.74 | 0.68 | 0.71 | 0.62 | 0.65 | 0.72 | 0.32 |
| RT‑DETR Tiny | 0.70 | 0.61 | 0.65 | 0.56 | — | — | 0.38 |
| FloodSAM‑Duo (Ours) | 0.86 | 0.82 | 0.84 | 0.78 | 0.76 | 0.85 | 0.18 |
FloodSAM‑Duo significantly outperforms all baselines. Compared to YOLOv8n‑seg, the F1‑score improves by 13% (0.84 vs. 0.71), mIoU by 11% (0.76 vs. 0.65), and BF1 boundary quality by 13% (0.85 vs. 0.72). The miss rate is reduced to 0.18, indicating that our method is far less likely to overlook flooded buildings, even in challenging conditions such as small buildings partially covered by water or buildings with large shadows. The dual‑scale weak semantic features effectively guide the segmentation model to produce structurally complete masks, avoiding the fragmentation often seen in YOLOv8n‑seg.
3.3 Negative‑Sample Robustness
To verify that our method does not produce excessive false positives in non‑flooded scenarios, we constructed two negative‑sample test sets: (A) non‑flooded urban areas (buildings present but no water) containing 200 images, and (B) water‑only or farmland areas (water present but no buildings) containing 200 images. We measured the number of false positive flooded building detections (FP), false positive rate (FP Rate), and average false positives per image (Mean FP). Table 3 shows the results.
| Method | FP in Scene A (non‑flooded urban) | FP Rate in A (%) | FP in Scene B (no‑building water) | FP Rate in B (%) | Mean FP per Image |
|---|---|---|---|---|---|
| ENet | 42 | 21.0 | 57 | 28.5 | 2.48 |
| YOLOv8n‑seg | 27 | 13.5 | 34 | 17.0 | 1.52 |
| RT‑DETR Tiny | 31 | 15.5 | 29 | 14.5 | 1.50 |
| FloodSAM‑Duo (Ours) | 6 | 3.0 | 4 | 2.0 | 0.25 |
FloodSAM‑Duo achieves a mean false‑positive rate of only 0.25 per image, about six times lower than the next best baseline (1.50 for RT‑DETR Tiny). This robust behavior is attributed to the dual‑scale weak semantic screening: the candidate region inference module effectively suppresses activations in areas that lack the structural relationship between building and water, and the prompt‑guided segmentation further avoids producing mask outputs in the absence of a clear building signal.
3.4 Ablation Study
We conducted an ablation study to isolate the contributions of the dual‑scale design and the candidate region inference module. Four configurations were compared: (1) Single‑scale (Large) – only large‑scale weak semantics used; (2) Single‑scale (Small) – only small‑scale; (3) Dual‑scale without candidate region inference – both scales used, but the prompt generation is performed on the whole image (no candidate screening); (4) Full FloodSAM‑Duo. Table 4 summarizes the results.
| Configuration | Precision | Recall_Flooded | F1_Flooded | AP_Flooded | mIoU_building_inst | BF1_boundary | Miss Rate |
|---|---|---|---|---|---|---|---|
| Single‑scale (Large) | 0.75 | 0.64 | 0.69 | 0.60 | 0.62 | 0.68 | 0.36 |
| Single‑scale (Small) | 0.72 | 0.67 | 0.69 | 0.61 | 0.64 | 0.70 | 0.33 |
| Dual‑scale w/o Candidate Region | 0.80 | 0.74 | 0.77 | 0.69 | 0.70 | 0.78 | 0.26 |
| FloodSAM‑Duo (Full) | 0.86 | 0.82 | 0.84 | 0.78 | 0.76 | 0.85 | 0.18 |
The results confirm that both dual‑scale fusion and candidate region inference are essential for achieving top performance. Single‑scale features suffer from either missing small buildings (large‑scale only) or excessive sensitivity to noise (small‑scale only). The dual‑scale fusion without candidate region inference improves recall but still exhibits higher miss rate because FastSAM must search over larger areas, increasing the chance of missed prompts and scattered masks. The full model leverages candidate screening to reduce the search space, leading to more reliable prompt placement and better mask quality.
4. Discussion
FloodSAM‑Duo addresses a critical gap in real‑time disaster monitoring from UAV drones: the ability to produce instance‑level flooded building information directly onboard with low latency and high reliability. Our results demonstrate that the integration of weak semantic reasoning with prompt‑guided segmentation is a powerful paradigm that overcomes the limitations of both pure semantic segmentation (which lacks instance distinction) and pure instance segmentation (which is computationally heavy and sensitive to domain shifts).
One key advantage of our method is its robustness to resolution changes. While the experiments were conducted at 0.2–0.5 m resolution (typical for flood response UAV drones), the dual‑scale pooling parameters can be adjusted to adapt to other scales (e.g., centimeter‑level imagery). The framework is scale‑reparameterizable: by simply modifying the pooling kernel sizes and the downsampling ratios, the weak semantic responses can be tuned to capture building‑water structural relationships at the desired spatial granularity. This flexibility makes FloodSAM‑Duo potentially applicable to a wide range of UAV drone payloads.
Another important aspect is the lightweight nature of the output. Instead of streaming full‑resolution image masks, our system transmits a few kilobytes of JSON‑structured data per frame. In practice, a UAV drone flying at 15 m/s with 80% overlap might generate about 2–3 frames per second. Even with limited bandwidth (e.g., 50 kbps), the structured output can be reliably sent, enabling ground‑based situational awareness without demanding high‑capacity downlinks. This is especially valuable in post‑disaster environments where communication infrastructure is damaged.
Despite these strengths, there are limitations. First, the current candidate region inference relies on a set of manually defined heuristics (conditions 1–4 in Section 2.1). While these work well in the tested flood scenarios, they may need tuning for other disaster types (e.g., debris flow, tsunami). A future direction is to learn these conditions adaptively from data. Second, the prompt generation step assumes that the weak semantic responses within a candidate region are strong enough; in cases of very small buildings (e.g., less than 10 pixels in the low‑resolution feature map), the prompts may be missed. An alternative strategy such as multi‑scale detection or incorporation of attention mechanisms could alleviate this issue.
Third, our current multi‑frame fusion is simplistic, using only spatial distance. In dense urban areas with multiple buildings close together, this may lead to incorrect merging of distinct buildings. Future work could incorporate temporal tracking or geometric verification to improve fusion accuracy.
5. Conclusion
We have presented FloodSAM‑Duo, a novel framework for real‑time, object‑level detection of flooded buildings from UAV drone imagery. By combining dual‑scale weak semantic reasoning with prompt‑guided instance segmentation, our method enables a UAV drone to generate structured flood‑damage information directly onboard with a latency of only 15–20 ms per frame. Extensive experiments on two flood‑affected regions demonstrate that FloodSAM‑Duo significantly outperforms existing lightweight models in terms of detection accuracy, instance mask quality, boundary consistency, and false positive suppression. The framework’s lightweight architecture, efficient output format, and adaptability to different resolutions make it a practical solution for onboard flood disaster assessment. Future work will focus on adaptive candidate inference, improved fusion for temporal consistency, and integration with higher‑level decision support systems for autonomous UAV drone mission planning.