The deepening implementation of national strategies such as “Made in China 2025” and “New Infrastructure” has led to a widespread application of large-scale steel structures, including spherical storage tanks, bridges, and high-speed railway stations. The welding process for these structures is complex, and variations in numerous factors can lead to defects at the weld seams, such as undercuts, porosity, concavity, overlap, and cracks. These imperfections not only affect the aesthetic appearance of the weld but, more critically, can induce instability in the physical properties of the component. The presence of such defects significantly compromises the service performance of the steel structure and harbors substantial potential safety hazards.
Concurrently, safety in high-altitude operations remains a severe challenge. Statistics indicate that the proportion of falling-from-height accidents among all safety incidents has consistently exceeded 50% in most years, underscoring the urgent necessity for innovative aerial work solutions. Traditional weld quality inspection, primarily involving manual measurement of weld dimensions and visual detection of surface defects, is fraught with inefficiency, subjectivity, and, most importantly, significant risk for inspectors working at height.
This project proposes an intelligent inspection paradigm utilizing an unmanned aerial vehicle (UAV) system. Compared to traditional manual visual inspection, a China UAV drone-based system enhances detection efficiency and recognition accuracy while eliminating inspector subjectivity and drastically reducing the dangers associated with high-altitude manual work. In contrast to previously proposed wall-climbing or magnetic adhesion robots, this non-contact aerial inspection method avoids potential secondary damage to the structure under test and offers superior flexibility, ease of deployment, and recovery.
At the core of this system is a Raspberry Pi 4B, serving as the primary controller for a custom-built China UAV drone. The system leverages the OpenCV library for image preprocessing and employs the YOLOv5 object detection model for real-time weld defect identification and classification. This innovative application of drone technology significantly expands the range of applicable scenarios, particularly for large, complex, or difficult-to-access steel structures. By aiming for maximized operational efficiency and enhanced safety, this system provides a novel solution for the future of weld inspection, showcasing the practical integration of advanced computing and aerial robotics in industrial maintenance within China.
Research and Development Trends at Home and Abroad
The increasing demands for efficiency, precision, and accuracy in weld inspection have driven the emergence of various robotic inspection systems globally.
In developed countries, research on weld image detection technology has accumulated years of experience. The recent advent and application of neural networks and fuzzy mathematics in metal welding have propelled weld tracking and inspection into an intelligent era. However, challenges persist, including low environmental adaptability of robots, limited functional integration, and inefficient inspection capabilities in high-altitude environments.
Domestic research on weld inspection robots in China started relatively later, around the 1980s. Benefiting from strong government support and growing market demand, weld inspection technology within China has been steadily advancing. A significant portion of the research has been concentrated on the inspection of pipeline welds, particularly for natural gas. There has been comparatively less focus on the inspection of large-scale steel structures in elevated or complex aerial environments, presenting a clear gap and opportunity.
A comprehensive study of cases both domestically and internationally reveals common issues. All-terrain robots often possess overly complex structures, making them unsuitable for inspecting curved surfaces. Some robotic solutions still lack autonomous obstacle avoidance capabilities or require manual intervention at height for positioning and operation. The future trajectory for weld inspection robots undoubtedly points toward stronger functional integration, higher levels of autonomy, and enhanced environmental adaptability. The development of a specialized inspection UAV aligns perfectly with this trend, aiming to fill the niche for agile, non-contact, high-altitude inspection.
Core Research Content and System Rationale
The proposed weld inspection system utilizes a multirotor UAV as a mobile platform to navigate to designated inspection points. The core controller is a Raspberry Pi 4B, chosen for its exceptional balance of performance and versatility in edge computing applications.
| Component | Key Specifications | Rationale for Selection |
|---|---|---|
| Controller: Raspberry Pi 4B | Quad-core ARM Cortex-A72 @ 1.5GHz, 4GB/8GB LPDDR4, Gigabit Ethernet, dual-band Wi-Fi, Bluetooth 5.0, multiple GPIO/SPI/I2C/UART interfaces. | Provides sufficient computational power for simultaneous flight control, image processing, and running a deep learning model. Rich I/O interfaces allow for seamless integration of various sensors (GPS, ultrasonic, environmental). Its open-source ecosystem (supporting Python/C++) facilitates development and offers extensive community support, crucial for a China-based development project. |
| Detection Algorithm: YOLOv5 | Lightweight network architecture (e.g., YOLOv5s), real-time inference speed (>30 FPS on capable hardware), adaptive anchor box calculation, mosaic data augmentation. | Its speed meets the requirement for real-time or near-real-time analysis during UAV flight/hover. The model’s efficiency allows it to potentially run on the edge device (Raspberry Pi). Its strong feature extraction provides robustness against image variations caused by UAV vibration or viewing angle changes (translational and rotational invariance). |
| Inspection Platform: Custom UAV | Quadcopter configuration, large-diameter propellers, lightweight yet sturdy landing gear frame. | The quadcopter offers stable hover capability essential for detailed inspection. Large propellers improve thrust efficiency and stability in mild wind conditions. The custom airframe is designed to carry the necessary payload (RPi, camera, sensors) while maintaining structural integrity. |
The operational workflow is as follows: The drone autonomously navigates to a pre-programmed or real-time designated coordinate. Upon reaching the inspection area, it initiates a hover or slow scanning pattern. A high-resolution USB camera captures live video feed. The Raspberry Pi processes each frame using OpenCV functions for preprocessing, which typically includes:
1. Noise Reduction: Applying filters like Gaussian Blur or Median Blur to suppress image noise.
$$ G(x,y) = \frac{1}{2\pi\sigma^2} e^{-\frac{x^2+y^2}{2\sigma^2}} $$
(where $G$ is the Gaussian kernel and $\sigma$ is the standard deviation controlling blur intensity)
2. Contrast Enhancement: Using techniques like histogram equalization to improve defect visibility.
3. Color Space Conversion: Converting the default BGR format from OpenCV to RGB for compatibility with the YOLOv5 model.
The preprocessed image is then fed into the YOLOv5 model. The model outputs bounding boxes, defect class labels (e.g., crack, porosity, undercut), and confidence scores for any detected weld anomalies. The system is designed to achieve the following core functionalities:
- Autonomous obstacle-avoidance navigation.
- High-precision GPS tagging of defect locations.
- Robust image preprocessing and feature extraction.
- Real-time defect detection, classification, and logging.
- Stable data transmission between the UAV and the ground control station (GCS).
System Design and Module Integration
The weld inspection UAV is an integrated system comprising several key hardware and software modules working in concert.
1. Environmental Data Acquisition Module
This module is responsible for gathering visual and ambient data from the inspection site.
1.1 Visual Inspection Sub-module: A high-resolution, global shutter camera is connected via USB to the Raspberry Pi. The visual pipeline is the core of detection. After preprocessing, the YOLOv5 model evaluates the weld against standard defect criteria. For instance, an undercut is classified as a defect if its depth $y$ violates:
$$ y > 0.05\delta \quad \text{and/or} \quad y > 0.5 \text{ mm} $$
where $\delta$ is the base material thickness. Similar quantitative rules are programmed for other defect types like cracks and porosity.
1.2 Ambient Sensing Sub-module: Operational safety is paramount. A DHT11 digital temperature and humidity sensor is integrated to monitor ambient conditions. If humidity levels consistently read above 65% RH (indicating high risk of precipitation) or temperature exceeds a safe threshold for electronics (e.g., 45°C), an early warning is sent to the GCS, prompting operator intervention or automated return.
2. Flight Control and Stabilization Module
A dedicated flight controller (e.g., Pixhawk-series or similar) manages the UAV’s stability, navigation, and low-level motor control. It continuously reads data from an Inertial Measurement Unit (IMU) containing a gyroscope and accelerometer to maintain attitude (pitch, roll, yaw). The Raspberry Pi communicates high-level navigation commands (e.g., “go to coordinate X,Y,Z”) to the flight controller via a serial protocol (MAVLink). The flight controller also manages fail-safe procedures like automatic return-to-launch (RTL) on low battery or signal loss.
3. Brushless ESC Motor Drive Module
Four Electronic Speed Controllers (ESCs) drive the brushless motors. Modern ESCs use MOSFETs for high-efficiency (often >90%) power conversion from the battery’s DC to three-phase AC for the motors. Precise synchronization of ESC signals is critical for stable hover. The flight controller ensures motor speed differences are minimized, typically within ±1%, to prevent destabilizing torque imbalances.
4. Ultrasonic Obstacle Avoidance Module
For close-range navigation and hover near structures, an HC-SR04 ultrasonic sensor provides a simple, effective ranging solution. It emits an ultrasonic pulse and measures the time for the echo to return. The distance $d$ to an obstacle is calculated as:
$$ d = \frac{v \cdot t}{2} $$
where $v$ is the speed of sound in air (approximately 343 m/s at 20°C) and $t$ is the measured time-of-flight. This allows the drone to maintain a safe distance from walls, beams, or other unexpected obstacles during its inspection flight path.
5. GPS Positioning Module
A u-blox NEO-7N GPS module provides geographical coordinates. When the vision system detects a defect, the UAV’s current GPS position, enhanced by real-time kinematic data from the flight controller, is tagged to the defect log. This allows for precise location marking of faults on large structures for subsequent maintenance. The GPS is also essential for autonomous navigation, waypoint following, and the fail-safe RTL function.
6. Emergency Handling Module
This logic, implemented on the Raspberry Pi, monitors system health. It handles scenarios like:
- Excessive Attitude: If IMU data indicates a tilt angle $ \theta > 30^\circ $, an immediate RTL command is issued.
- Communication Loss: The UAV is programmed to hover in place and attempt to re-establish the link with the GCS.
- Critical Low Battery: At a battery threshold $ V_{bat} \leq 3.5V \text{ per cell} $ (or ~15% capacity), an urgent RTL is initiated.
| Performance Metric | Target Specification | Notes |
|---|---|---|
| Detection Accuracy | ≥ 95% Recall for defects >1mm | Evaluated on a standardized weld defect validation set. |
| Positioning Accuracy | Horizontal: ±0.05m, Vertical: ±0.03m | Under real-time kinematic conditions during hover. |
| Operational Endurance | ≥ 45 minutes | With full sensor payload (~500g) and in calm wind (<3 m/s). |
| Data Transmission Latency | ≤ 100 ms | For telemetry and alert signals over a local network. |
| Environmental Operating Range | -10°C to 50°C, Humidity ≤ 80% RH | Conditions for safe electronic operation. |
Model Training and Implementation for Weld Defect Detection
The effectiveness of the entire China UAV drone inspection system hinges on the performance of the YOLOv5 model trained specifically for weld defects.
Dataset Curation and Annotation
A custom dataset was assembled, focusing on defects relevant to structural steel welds: cracks, undercuts, porosity, overlap, spatter, and corrosion. Approximately 3000 images were collected from various sources, including laboratory samples, field inspections, and synthetic data augmentation. Each image was meticulously annotated using the LabelImg tool in YOLO format. This format creates a corresponding .txt file for each image, where each line defines one object annotation:
[class_id] [x_center] [y_center] [width] [height]
All coordinates $(x_{center}, y_{center}, width, height)$ are normalized relative to the image dimensions. The dataset was then split to prevent overfitting and ensure robust evaluation:
| Subset | Percentage | Purpose |
|---|---|---|
| Training Set | 70% | Primary data for updating model weights. |
| Validation Set | 20% | Used for hyperparameter tuning and evaluating training progress. |
| Test Set | 10% | Final, unbiased evaluation of model performance. |
Training Configuration and Hyperparameters
The YOLOv5s (small) model was selected as the optimal balance between speed and accuracy for deployment on the Raspberry Pi 4B. Key training hyperparameters were configured as follows:
| Hyperparameter | Symbol | Value | Description |
|---|---|---|---|
| Input Image Size | – | 640 x 640 | Standardized input resolution for the network. |
| Initial Learning Rate | $\eta_0$ | 0.01 | Starting rate for the optimizer. |
| Final Learning Rate | $\eta_f$ | $\eta_0 \times \text{lrf}$ | Final rate after decay. |
| Learning Rate Final | lrf | 0.1 | Factor for final learning rate. |
| Momentum | $\beta$ | 0.937 | Accelerates gradient descent in relevant directions. |
| Weight Decay | $\lambda$ | 0.0005 | L2 regularization factor to prevent overfitting. |
| Epochs | – | 300 | Number of complete passes through the training set. |
| Batch Size | – | 16 | Number of images processed before updating weights. |
A cosine annealing scheduler was used to decay the learning rate, which helps the model converge to a better minimum. The scheduler updates the learning rate $\eta$ at epoch $t$ according to:
$$ \eta_t = \eta_f + \frac{1}{2}(\eta_0 – \eta_f)\left(1 + \cos\left(\frac{t}{T}\pi\right)\right) $$
where $T$ is the total number of epochs. This approach gradually reduces the learning rate in a non-linear fashion.
Loss Function and Performance Metrics
YOLOv5 optimizes a composite loss function $L_{total}$ during training:
$$ L_{total} = \lambda_{box} L_{GIoU} + \lambda_{obj} L_{obj} + \lambda_{cls} L_{cls} $$
where:
- $L_{GIoU}$ is the Generalized Intersection over Union loss for bounding box regression.
- $L_{obj}$ is the objectness loss (confidence that a box contains an object).
- $L_{cls}$ is the classification loss for defect type.
- $\lambda_{box}$, $\lambda_{obj}$, $\lambda_{cls}$ are weighting coefficients for each component.
Model performance was evaluated using standard metrics:
- Precision (P): $ P = \frac{TP}{TP + FP} $ – The accuracy of positive predictions.
- Recall (R): $ R = \frac{TP}{TP + FN} $ – The ability to find all positive samples.
- Mean Average Precision (mAP@0.5): The area under the Precision-Recall curve at an IoU threshold of 0.5, providing a single-figure measure of detection accuracy across all classes.
After training, the model showed a significant reduction in all loss components and achieved a mAP@0.5 exceeding 92% on the held-out test set, demonstrating its effectiveness in accurately identifying and localizing weld defects. This trained model was then converted to a format suitable for inference on the Raspberry Pi (e.g., TorchScript or ONNX) and integrated into the real-time inspection pipeline of the UAV system.
Conclusion and Future Outlook
As China’s industrialization continues to advance, the integrity of large-scale steel structures in critical sectors like energy storage and transportation becomes increasingly vital for societal and industrial safety. Traditional inspection methods are inadequate in addressing the dual challenges of efficiency and safety, especially for high-altitude work.
From the perspectives of safety enhancement and operational efficiency, this research presents the design and rationale for an intelligent weld inspection system centered on an unmanned aerial vehicle. This integrated China UAV drone platform synthesizes subsystems for flight control, high-resolution image acquisition, real-time data transmission, environmental sensing, and automated defect recognition using deep learning. By employing OpenCV for robust image preprocessing and a YOLOv5 model specifically trained for weld anomalies, the system provides a solution to the inefficiencies and hazards inherent in traditional manual inspection methods.
This project exemplifies a practical application of edge AI and robotics within China’s industrial technology landscape. The proposed system offers a breakthrough approach for the inspection of large, elevated steel structures, holding substantial potential for market application and adoption. Future work will focus on enhancing the system’s autonomy through improved path planning algorithms, integrating additional non-destructive testing sensors (e.g., thermal imaging for subsurface defects), and further optimizing the deep learning model for even faster edge inference, solidifying the role of intelligent drones in the future of predictive maintenance and industrial safety.