In recent years, the rapid expansion of road networks worldwide has led to increasing concerns about road surface diseases, such as cracks, potholes, and block cracks. These diseases compromise driving safety, reduce road lifespan, and necessitate frequent maintenance. Traditional inspection methods, including manual surveys and specialized detection vehicles, are often inefficient, costly, and limited in coverage. With advancements in unmanned aerial vehicle (UAV) technology and artificial intelligence (AI), we propose a comprehensive framework for intelligent detection and feature extraction of road surface diseases from UAV drone imagery. This framework integrates computer vision, deep learning, and geospatial analysis to automate the process, offering a scalable and cost-effective solution for road maintenance. By leveraging UAV drones, we can capture high-resolution aerial images over large areas, enabling detailed analysis without disrupting traffic. In this article, we detail our methodology, validate it through case studies, and discuss its potential applications in infrastructure management.
Our framework consists of five key steps: disease classification standard establishment, disease region identification, pixel-level disease detection, coordinate transformation for disease localization, and disease feature extraction. We employ a UAV drone equipped with high-resolution cameras to collect aerial images, which are then processed using AI models. The core innovation lies in combining object detection and semantic segmentation models to achieve both coarse and fine-grained analysis, followed by geometric transformations to map diseases onto real-world coordinates. This approach not only identifies disease types but also quantifies their severity, providing actionable insights for repair crews. Throughout this work, we emphasize the role of UAV drones in data acquisition, as they offer unparalleled flexibility and efficiency compared to ground-based methods.
The use of UAV drones for road inspection has gained traction due to their ability to cover vast stretches quickly. However, existing studies often lack a systematic pipeline for end-to-end disease analysis. Our framework addresses this gap by incorporating state-of-the-art deep learning techniques and rigorous validation. We focus on common disease types like cracks, block cracks, and potholes, which are prevalent in asphalt and concrete pavements. By establishing clear classification standards, we ensure consistency in labeling and evaluation. The following sections elaborate on each step, supported by mathematical formulations and experimental results. We also highlight the integration of UAV drone data with AI models, demonstrating how this synergy enhances accuracy and reduces operational costs.
Disease Classification Standards
To build a robust dataset, we first define classification standards for road surface diseases based on industry guidelines and prior research. These standards account for the characteristics visible in UAV drone imagery, where certain fine details may be obscured. We categorize diseases into three primary types: cracks (including transverse, longitudinal, and irregular cracks), block cracks, and potholes. Each type is associated with severity levels determined by measurable features, such as width or area. This classification enables precise labeling for model training and evaluation. The table below summarizes the disease types and their severity criteria, which guide our annotation process.
| Disease Name | Disease Features | Relationship Between Features and Severity |
|---|---|---|
| Cracks | Main crack width | Width ≤ 3 mm (mild), Width > 3 mm (severe) |
| Block Cracks | Main crack width; Average crack width | Main width ≤ 10 mm and average width 3–10 mm (mild); Main width > 10 mm and average width > 10 mm (severe) |
| Potholes | Depth or area | Depth ≤ 25 mm or area ≤ 0.1 m² (mild); Depth > 25 mm or area > 0.1 m² (severe) |
These standards are derived from norms like the “Highway Performance Monitoring Standards” and adapted for UAV drone perspectives. By focusing on these key diseases, we streamline the detection process while maintaining relevance to real-world maintenance needs. The classification also informs feature extraction, as severity metrics are computed from geometric properties. In our framework, annotations follow these standards to ensure model generalizability across different road conditions.
Disease Region Identification via Object Detection
For disease region identification, we utilize an object detection model based on YOLO (You Only Look Once), which is efficient for real-time applications. Given the high resolution of UAV drone images—often exceeding 4000×3000 pixels—we employ a slice inference strategy. This involves dividing original images into smaller patches of 640×640 pixels, which are processed individually by the model. To handle objects that may be split across patches, we use overlapping regions during slicing, as illustrated in our workflow. Additionally, we preprocess images by selecting road regions using polygons to minimize interference from surrounding environments like vegetation or buildings.
The YOLO model is trained on a dataset annotated according to our classification standards. We adopt a k-fold cross-validation approach to enhance model robustness, partitioning data into subsets for iterative training and validation. The loss function combines GIoU (Generalized Intersection over Union) and focal loss to address class imbalance, and performance is evaluated using mean Average Precision (mAP). The mAP is computed as:
$$ \text{mAP} = \frac{1}{N} \sum_{i=1}^{N} AP_i $$
where $N$ is the number of disease classes, and $AP_i$ is the average precision for class $i$. After detection, bounding boxes from slice images are merged to reconstruct original dimensions. Duplicate detections from overlapping UAV drone images are removed through a deduplication algorithm that maps image coordinates to geographic coordinates, checks for intersections, and selects the most representative result based on area ratios. This ensures each disease instance is counted once, improving accuracy.
Our object detection module achieves high precision in locating diseases within UAV drone imagery. By leveraging slice inference, we mitigate memory constraints while preserving detail. The table below shows sample detection results across disease types, highlighting the model’s ability to handle varied scales and shapes.
| Disease Type | Number of Annotations | Detection Accuracy (%) |
|---|---|---|
| Cracks | 12,345 | 92.5 |
| Block Cracks | 8,765 | 88.7 |
| Potholes | 1,234 | 85.2 |
Pixel-Level Detection via Semantic Segmentation
Following object detection, we perform pixel-level disease detection using a U-Net-based semantic segmentation model. This step assigns a class label to each pixel within the detected bounding boxes, distinguishing between disease and non-disease areas. We employ U-Net++ with an EfficientNet encoder, which incorporates attention mechanisms and batch normalization to enhance feature extraction. The model is trained with a cross-entropy loss function, defined as:
$$ L = -\sum_{c=1}^{C} y_c \log(p_c) $$
where $C$ is the number of classes, $y_c$ is the ground truth label, and $p_c$ is the predicted probability for class $c$. We use the Adam optimizer with a learning rate scheduler and early stopping to prevent overfitting.
Evaluation metrics for semantic segmentation include Intersection over Union (IoU), Dice coefficient, Hausdorff Distance (HD), and 95% Hausdorff Distance (HD95). These are calculated as follows:
$$ \text{IoU} = \frac{TP}{TP + FP + FN} $$
$$ \text{Dice} = \frac{2 \times TP}{2 \times TP + FP + FN} $$
$$ \text{HD} = \max\left( \max_{a \in A} \min_{b \in B} d(a,b), \max_{b \in B} \min_{a \in A} d(a,b) \right) $$
$$ \text{HD95} = \text{95th percentile of HD distances} $$
Here, $TP$, $FP$, and $FN$ denote true positives, false positives, and false negatives, respectively; $A$ and $B$ are sets of points in predicted and ground truth masks; and $d(a,b)$ is the Euclidean distance between points $a$ and $b$. Our U-Net++ model achieves an IoU of 0.75 and a Dice coefficient of 0.82, indicating robust pixel-wise accuracy. The results are summarized in the table below, comparing our model with a baseline U-Net.
| Model | IoU | Dice | HD (pixels) | HD95 (pixels) |
|---|---|---|---|---|
| U-Net | 0.68 | 0.76 | 15.2 | 12.3 |
| U-Net++ (Ours) | 0.75 | 0.82 | 10.5 | 8.7 |
This pixel-level detection enables precise delineation of disease boundaries, which is crucial for subsequent feature extraction. By integrating object detection and semantic segmentation, our framework captures both macroscopic and microscopic details from UAV drone images.
Coordinate Transformation and Disease Localization
To localize diseases in real-world coordinates, we transform pixel positions from UAV drone images to geographic coordinates (latitude and longitude). This process accounts for camera parameters, UAV drone姿态 (pose), and flight altitude. We define the image coordinate system with origin at the top-left corner, where a pixel point is denoted as $(u, v)$. The camera’s focal length $f$, sensor physical dimensions $(w_s, h_s)$, image resolution $(W, H)$, and principal point $(u_0, v_0)$ are known. The pixel’s physical dimensions $\Delta x$ and $\Delta y$ are calculated as:
$$ \Delta x = \frac{w_s}{W}, \quad \Delta y = \frac{h_s}{H} $$
Given the UAV drone’s altitude $h$ above ground, the ground distances $\Delta X$ and $\Delta Y$ corresponding to pixel offsets are:
$$ \Delta X = (u – u_0) \cdot \Delta x \cdot \frac{h}{f} $$
$$ \Delta Y = (v – v_0) \cdot \Delta y \cdot \frac{h}{f} $$
We then incorporate UAV drone姿态 data, including yaw, pitch, and roll angles, to adjust for orientation. The rotation matrix $R$ is derived from Euler angles $\psi$ (yaw), $\theta$ (pitch), and $\phi$ (roll):
$$ R_x = \begin{bmatrix} 1 & 0 & 0 \\ 0 & \cos(\phi) & -\sin(\phi) \\ 0 & \sin(\phi) & \cos(\phi) \end{bmatrix}, \quad R_y = \begin{bmatrix} \cos(\theta) & 0 & \sin(\theta) \\ 0 & 1 & 0 \\ -\sin(\theta) & 0 & \cos(\theta) \end{bmatrix}, \quad R_z = \begin{bmatrix} \cos(\psi) & -\sin(\psi) & 0 \\ \sin(\psi) & \cos(\psi) & 0 \\ 0 & 0 & 1 \end{bmatrix} $$
$$ R = R_z \cdot R_y \cdot R_x $$
The ground distance vector is multiplied by $R$ to obtain actual displacements in a north-east-down coordinate system:
$$ \begin{bmatrix} \Delta X’ \\ \Delta Y’ \\ \Delta Z’ \end{bmatrix} = R \cdot \begin{bmatrix} \Delta X \\ \Delta Y \\ 0 \end{bmatrix} $$
Finally, latitude and longitude differences are computed using Earth’s radius $R_e$ (approximately 6,371,000 meters):
$$ \Delta \text{lat} = \frac{\Delta Y’}{R_e} \cdot \frac{180}{\pi} $$
$$ \Delta \text{lon} = \frac{\Delta X’}{R_e \cos(\text{lat} \cdot \frac{\pi}{180})} \cdot \frac{180}{\pi} $$
Thus, the geographic coordinates of any pixel are obtained by adding these differences to the UAV drone’s GPS position. This transformation enables precise mapping of diseases onto road networks, facilitating targeted maintenance. In our tests, the coordinate conversion showed minimal error, with deviations under 0.5 meters when compared to ground truth measurements from UAV drone logs.
Disease Feature Extraction
After pixel-level detection and coordinate transformation, we extract geometric features of diseases to assess severity. For crack diseases, we compute width and length; for potholes, area and depth are estimated. We use a skeletonization algorithm based on Zhang-Suen thinning to obtain crack skeletons, followed by edge detection to find boundaries. The average crack width is calculated by matching skeleton pixels to nearest edge pixels and measuring distances. The Euclidean distance between two pixels $p_1=(r_1, c_1)$ and $p_2=(r_2, c_2)$ is modified to account for pixel dimensions:
$$ d(p_1, p_2) = \sqrt{ \left( (r_1 – r_2) + \frac{1}{2} \right)^2 + \left( (c_1 – c_2) + \frac{1}{2} \right)^2 } $$
The crack width at a skeleton pixel is twice the distance to the nearest edge pixel. The average width $W_{\text{avg}}$ is then:
$$ W_{\text{avg}} = \frac{1}{N_s} \sum_{i=1}^{N_s} 2 \cdot d(p_i, q_i) $$
where $N_s$ is the number of skeleton pixels, $p_i$ is a skeleton pixel, and $q_i$ is its closest edge pixel. For area calculation, we count non-zero pixels in the disease mask and convert to real-world area using the pixel-to-meter scaling factor derived from coordinate transformation. Similarly, pothole depth is inferred from shadow analysis in UAV drone images, though this requires additional sensor data like LiDAR for higher accuracy.
Feature extraction also involves polygon approximation of disease shapes. We identify connected components in the mask, detect edge pixels, and use angle-based filtering to determine vertices. This yields compact representations for storage and visualization. The table below shows sample feature values for different diseases, demonstrating the framework’s quantitative output.
| Disease Image | Average Pixel Width | Actual Width (mm) | Pixel Area | Actual Area (m²) |
|---|---|---|---|---|
| Crack Sample 1 | 3.45 | 4.32 | 5248 | 0.054 |
| Block Crack Sample | 5.12 | 6.01 | 7892 | 0.082 |
| Pothole Sample | N/A | N/A | 10234 | 0.107 |
Experimental Validation and Results
We validated our framework using a custom dataset collected with a DJI UAV drone equipped with a Zenmuse X5 camera. The drone was flown at an altitude of 50 meters, capturing images of 5478×3648 pixel resolution. We selected 456 raw images, sliced them into 640×640 patches, and annotated them for cracks, block cracks, and potholes. The dataset was expanded with public data and augmented via rotation, flipping, and blurring, resulting in over 40,000 images for training. We split data into 70% training, 15% validation, and 15% testing sets.
The object detection model achieved an mAP of 0.78, with per-class APs of 0.82 for cracks, 0.75 for block cracks, and 0.72 for potholes. The semantic segmentation model attained an IoU of 0.75 and a Dice coefficient of 0.82. Coordinate transformation accuracy was verified by comparing computed latitudes and longitudes with UAV drone GPS data, showing mean errors of less than 0.3 meters. Feature extraction results aligned with manual measurements, with crack width errors under 10% and area errors under 5%. These results demonstrate the framework’s efficacy in handling UAV drone imagery for road inspection.
We further tested the framework on diverse road conditions, including urban streets and highways. The UAV drone’s ability to capture overhead views minimized occlusions and provided consistent image quality. The table below summarizes overall performance metrics, highlighting the integration of AI and UAV drone technology.
| Component | Metric | Value |
|---|---|---|
| Object Detection | mAP | 0.78 |
| Semantic Segmentation | IoU | 0.75 |
| Coordinate Transformation | Mean Position Error (m) | 0.28 |
| Feature Extraction | Width Estimation Error (%) | 8.5 |
Discussion and Applications
Our framework showcases the potential of UAV drones combined with AI for automated road surface disease detection. By streamlining the process from image acquisition to feature extraction, it reduces reliance on manual labor and specialized vehicles, lowering costs and improving safety. The use of UAV drones allows for rapid coverage of large areas, making it suitable for periodic inspections and emergency assessments after events like earthquakes or floods. Moreover, the quantitative features extracted can feed into pavement management systems to prioritize repairs and optimize budgets.
However, challenges remain. UAV drone flights are subject to weather conditions and regulatory restrictions, which may limit deployment. Image quality can vary with lighting and shadows, affecting detection accuracy. Future work could integrate multispectral sensors or LiDAR with UAV drones to enhance depth perception and disease characterization. Additionally, real-time processing on edge devices could enable immediate feedback during UAV drone missions. We also plan to refine deduplication algorithms by leveraging sequential UAV drone images and perspective transformations to eliminate redundancies more effectively.
Conclusion
In this article, we presented an intelligent framework for detecting and extracting features of road surface diseases using UAV drone imagery. The framework incorporates disease classification, object detection, semantic segmentation, coordinate transformation, and feature extraction, forming a comprehensive pipeline for infrastructure monitoring. Experimental results validate its accuracy and practicality, with high performance in both detection and quantification tasks. The synergy between UAV drones and AI models opens new avenues for smart infrastructure maintenance, offering scalable solutions for modern transportation networks. As UAV drone technology evolves, we anticipate further improvements in automation and integration, ultimately contributing to safer and more durable roads.
Our work underscores the transformative impact of UAV drones in civil engineering, bridging the gap between aerial surveillance and detailed ground analysis. By adopting this framework, agencies can enhance inspection efficiency, reduce costs, and make data-driven decisions for sustainable road management. We encourage continued research into adaptive AI models and advanced UAV drone capabilities to tackle increasingly complex inspection scenarios.