The safe operation of transportation systems is of paramount importance, with waterway transportation, as a vital component, receiving focused attention from regulatory bodies. Inland waterways, serving as public transportation infrastructure, underpin the safety and efficiency of waterborne logistics. Their effective management and maintenance are fundamental to ensuring unimpeded transport and supporting broader economic activities. However, prevailing maintenance methodologies remain largely traditional. Routine inspections of channel facilities—including revetments, navigation aids, bridges, and anchorages—are predominantly conducted manually at fixed intervals. Inspectors visually assess these structures to identify defects and subsequently formulate maintenance plans. This conventional approach suffers from significant drawbacks, including poor timeliness in identifying issues and inefficient allocation of resources, making it increasingly difficult to ensure the required navigational availability for growing traffic volumes. Consequently, the adoption of intelligent technologies to enhance inspection efficiency and reduce operational costs has become an imperative trend for the development of the waterway sector.
Unmanned Aerial Vehicle (UAV) technology, or UAV drones, has demonstrated substantial potential and value across numerous domains. Within the context of waterway management, UAV drones are revolutionizing traditional maintenance paradigms. They significantly enhance operational efficiency and safety while mitigating the risks and costs associated with manual, vessel-based inspections. Supported by technological innovation and favorable policies, UAV drones provide efficient solutions for channel patrolling, buoy maintenance, and emergency response. By integrating aerial survey data and real-time video feeds, these systems enable intelligent inspections, drastically reducing the need for manual site visits, lowering vehicle and vessel costs, and elevating the overall level of intelligent management. With the burgeoning “low-altitude economy,” the application of UAV drones in port and waterway sectors is rapidly evolving, expanding from demonstrations on major trunk waterways like the Yangtze and Pearl Rivers to broader inland networks.
This research focuses on the application technology for intelligent inspection using UAV drones in real-world operational scenarios and conducts systematic integration. We explore its effectiveness in multi-scenario applications within actual operational environments. The core objective is to achieve intelligent analysis and proactive warning for events such as revetment defects and abnormal status of ancillary facilities, based on automated UAV drones patrols. This aims to improve the quality and efficiency of waterway inspections, reduce associated costs, and ultimately better serve waterway maintenance operations.
Current State and Demand Analysis for Intelligent Lock and Channel Management
The intelligent management and maintenance of navigation locks and channels represent a persistent challenge and a critical bottleneck in the digital transformation of inland waterway systems, plagued by several thorny issues. Following institutional reforms, enforcement responsibilities have often been transferred to comprehensive transport law enforcement agencies, leading to staffing shortages at the grassroots level. This can result in untimely regulatory oversight, adversely affecting lock operational efficiency. For instance, newly constructed channels suffer from damage to slope greening and ecological revetments, while vessels frequently fail to moor correctly or overstay in waiting zones upstream and downstream of locks, occupying valuable water space and reducing throughput. Furthermore, while non-compliant placement of navigation aids (e.g., pipeline markers, bridge clearance gauges) can be identified, a lack of effective monitoring tools makes it difficult to supervise corrective actions by responsible entities, hindering the adjustment of navigation-affecting pipelines.
Moreover, the modernization level of lock and channel maintenance is generally inadequate. A scarcity of advanced digital and intelligent tools hampers maintenance efficiency and the timely detection of potential safety hazards. The industry is therefore actively seeking new technologies to foster development. UAV drones have shown unique advantages in the field of waterway maintenance. Leveraging their efficiency, precision, and safety, they are increasingly becoming a core means to enhance management efficiency and ensure navigational safety.
Currently, UAV drones serve as a crucial extension and supplement to traditional vessel and vehicle patrols. Their deployment offers irreplaceable auxiliary benefits: accessing areas unreachable by boats, vehicles, or personnel; enabling omnidirectional, multi-angle observation and video recording; and performing rapid, high-efficiency inspections over large areas in short timeframes. The use of UAV drones has achieved comprehensive coverage of inspection zones, already widely applied in the patrol of high-grade waterways. Empirical data suggests savings of at least two-thirds in both time and manpower compared to traditional methods, significantly reducing workload and safety risks for staff and establishing UAV drones as an indispensable tool.
However, the prevailing “UAV + manual vessel/vehicle” hybrid model still relies fundamentally on human visual examination of collected footage. This fails to fully exploit the inherent intelligent advantages of UAV drones technology. Building upon existing artificial intelligence and information technologies, this study investigates intelligent inspection application technologies for UAV drones tailored to authentic operational scenarios. Through systematic integration, we aim to implement automated patrols, intelligent analysis, and proactive warning, thereby elevating the informatization and standardization of waterway inspection work.
Core Application Scenarios of UAV Drones in Lock and Channel Management
Addressing the pain points and challenges in daily lock and channel management, this research conducts a comprehensive analysis of various machine vision solutions. Through the collection and training on real-world data, we construct intelligent video recognition models for UAV drones tailored to actual business scenarios. The goal is to achieve automated inspection, intelligent analysis, and proactive warning, facilitating a rapid transition towards digital and intelligent management.
1. Intelligent Early Warning for Revetment Defects in Approach Channels
Revetment defects constitute a critical maintenance task. Severe defects are a primary cause of revetment collapse, directly impacting navigational safety. For video-based event detection, several mature algorithms are available, such as the Region-based Convolutional Neural Network (R-CNN) series, Single Shot MultiBox Detector (SSD), and the You Only Look Once (YOLO) family of end-to-end detectors, each with distinct strengths and weaknesses. This study employs an object detection approach based on the YOLOv8 algorithm to automatically analyze, identify, and warn against events exhibiting significant anomalies like abnormal deformation.
The improved revetment defect target identification algorithm operates as follows: First, a dataset of revetment defect images is utilized. A Convolutional Neural Network (CNN) performs image segmentation to extract the revetment area and form connected domains. Fractured sections within these domains are flagged as potential damage. Subsequently, labeled images train a YOLOv8 model to recognize the state of these suspected damaged sections. Mutual validation between the segmentation and detection stages enhances the accuracy of defect identification. The training dataset is rich, containing numerous images of revetment defects captured under varying lighting and weather conditions. It covers a wide range of failure modes, including spalling/exposed aggregate on parapet/wave walls, large-area impact damage, disordered protective layers, slippage, block fracture, or erosion loss, providing robust support for model training.
The convolutional operation at the core of feature extraction can be represented as:
$$ (f * g)(t) = \int_{-\infty}^{\infty} f(\tau) g(t – \tau) d\tau $$
In the discrete, 2D context of image processing for a layer l, the feature map Flij at position (i, j) is computed from the previous layer’s activation Al-1 using kernel Kl:
$$ F^{l}_{ij} = \sum_{m} \sum_{n} A^{l-1}_{(i+m)(j+n)} \cdot K^{l}_{mn} + b^{l} $$
where bl is the bias term. The YOLOv8 model’s loss function L typically combines localization loss (Lbox), objectness confidence loss (Lobj), and classification loss (Lcls):
$$ L = \lambda_{box} L_{box} + \lambda_{obj} L_{obj} + \lambda_{cls} L_{cls} $$
where the λ terms are weighting coefficients balancing the contribution of each component.
| Algorithm | Core Principle | Advantages for UAV Application | Potential Challenges |
|---|---|---|---|
| Two-Stage Detectors (e.g., R-CNN, Fast R-CNN) | Generate region proposals first, then classify and refine. | High accuracy. | Slower processing speed, less suitable for real-time UAV drones video. |
| Single-Stage Detectors (e.g., SSD, YOLO variants) | Directly predict bounding boxes and class probabilities in one pass. | Very fast inference, enabling real-time analysis on UAV drones streams. | Historically slightly lower accuracy on small objects (improved in later versions). |
| Our Approach (YOLOv8-based with CNN pre-segmentation) | Combines segmentation for area isolation with fast detection for state classification. | Balances speed and accuracy, provides contextual validation. | Requires a two-model pipeline; dependency on segmentation quality. |
2. Intelligent Detection of Ancillary Facility Status
Ancillary facility status detection is divided into two categories based on operational needs. The first involves detecting whether navigation aids (buoys, beacons) are on-station and functioning correctly (e.g., light operational). Beyond data from electronic and positional sensors, methods based on YOLOv3-tiny for buoy recognition and machine vision for light quality inspection can be employed. The second category focuses on detecting compliance issues with shore-side signs and markers (e.g., pipeline crossing signs, bridge clearance boards), such as bending, obstruction, loss, or damage. Currently, monitoring the status of these facilities relies on manual visual patrols. This research explores a solution assisted by “UAV drones + machine vision” technology.
Sign and marker recognition primarily relies on object detection algorithms. Widely recognized algorithms include R-CNN, Fast R-CNN, algorithms utilizing Non-Maximum Suppression (NMS), and application-oriented lightweight versions like YOLOv5-lwa. After a comparative analysis, this study adopts a modified approach based on the CMF-YOLOv8s architecture, adjusted for the lock and channel management context. This method achieves recognition of shore-side markers by synchronizing Real-Time Kinematic (RTK) positioning data from UAV drones with video imagery.
The model’s logic is: a neural network is trained on a dataset of channel shore-side signs to form a sign detection module. Combined with UAV drone RTK data, the system can identify and record any new, previously unregistered signs. For detecting missing or damaged signs, the system compares the positions of identified signs during subsequent patrols against a baseline recorded during an initial comprehensive survey. Discrepancies indicate potential obstruction, displacement, or damage.
The detection process involves predicting bounding box coordinates (x, y, w, h), objectness score po, and class probabilities pc. For a detected sign, its geolocation (LatUAV, LonUAV) from the UAV drones is corrected using the camera’s bearing (θ) and the pixel offset (Δx, Δy) from the image center to the sign’s bounding box center, applying a transformation function T:
$$ (Lat_{sign}, Lon_{sign}) = T(Lat_{UAV}, Lon_{UAV}, \theta, \Delta x, \Delta y, \text{camera calibration params}) $$
A match with the baseline database is found if the calculated position falls within a threshold distance dthresh of a recorded sign’s position:
$$ \sqrt{(Lat_{sign} – Lat_{baseline})^2 + (Lon_{sign} – Lon_{baseline})^2} < d_{thresh} $$
Signs not meeting this condition are flagged for review.
| Facility Type | Primary Detection Target | Key Algorithmic Features | Data Integration Requirement |
|---|---|---|---|
| Navigation Aids (Buoys/Beacons) | Presence, Position, Light Operation | Small object detection, temporal analysis for light flashing patterns. | RTK/GPS coordinates, timestamped video frames. |
| Shore-side Regulatory Signs | Presence, Orientation, Legibility, Physical Integrity | Text/pattern recognition, pose estimation to detect tilting. | High-resolution imagery, accurate geotagging for baseline comparison. |
| Pipeline Crossing Markers | Presence, Correct Labeling | Symbol recognition, verification against pipeline registry data. | Spatial database of registered infrastructure. |
Technical Implementation and System Integration
The overall functionality of the intelligent inspection system includes modules for UAV video patrol input, intelligent event perception and recognition, intelligent auxiliary dispatch, inspection event reporting, and patrol report generation. These modules can be integrated to form a complete system or decomposed and incorporated into existing management systems as needed, enabling the digitization of the entire inspection workflow.
1. Overall Architecture
The intelligent inspection system is structured across five layers: Support Layer, Data Layer, Algorithm Layer, Application Layer, and User Layer. Corresponding interface specifications and information security safeguards ensure the system’s stable and secure operation.
| Layer | Components | Function Description |
|---|---|---|
| User Layer | Web Portal, Mobile App | Provides interfaces for dispatchers, maintenance crews, and managers to view alerts, reports, and dispatch tasks. |
| Application Layer | Patrol Management, Event Alerting, Report Generation, Dispatch Module | Orchestrates business logic: manages UAV drones missions, processes algorithm outputs, generates work orders. |
| Algorithm Layer | Revetment Defect Model, Facility Detection Model, Video Analysis Engine | The core intelligence. Hosts the trained machine vision models that analyze UAV drones video streams. |
| Data Layer | Video Database, Spatial (GIS) Database, Event Log, Baseline Facility Registry | Stores all raw video, processed metadata, historical events, and reference data for comparison. |
| Support Layer | UAV Fleet Mgmt. System, RTK/GNSS Service, Cloud/Edge Compute, Network | Provides the foundational hardware and services: controls the UAV drones, provides precise location, supplies computing power. |
2. System Integration
The integration of machine vision-based information systems is now commonplace. This study integrated the intelligent early-warning algorithm for revetment defects and the intelligent detection algorithm for ancillary facility status into the existing management system of the port and waterway authority. Building upon retained functionalities like buoy displacement warning and slope monitoring, the system was enhanced by ingesting video data from UAV drones revetment patrols. This enables the automatic processing and identification of anomalous events, with a multi-level alerting mechanism providing precise warnings.
2.1 Integration of Revetment Defect Intelligent Early Warning: The primary logic involves rapid UAV drones patrols along approach channel revetments. Video data is processed in near real-time, and identified defect events trigger proactive alerts.
- Performance Targets: Recognition accuracy for events like coping脱落 (spalling/displacement) and revetment collapse should be no less than 80%. The algorithms support self-learning and expansion, with accuracy expected to improve with increased sample size. The miss rate should not exceed 15%.
- Algorithm Prerequisites: Camera resolution不低于 (not less than) 4 megapixels; video bitrate of 4 Mbps (minimum 2 Mbps); moderate or better visibility; illumination不低于 3000 lux; and an unobstructed view free from trees or other occlusions.
2.2 Integration of Ancillary Facility Status Intelligent Detection: The focus here is on shore-side signs. The logic combines rapid UAV drones patrols with real-time processing of video against historical patrol data to warn of倾斜 (tilting),遮挡 (obstruction),倒塌 (collapse),丢失 (loss), or损毁 (damage).
- Performance Targets: Recognition accuracy不低于 85%, with support for self-learning. Miss rate should not exceed 15%.
- Algorithm Prerequisites: Same as for revetment defects: ≥4MP camera, 4 Mbps video, good visibility and illumination, no obstructions.
2.3 UAV Route Planning and Control: An innovative approach involved utilizing communication towers作为 (as) bases for the UAV drones. Based on the fixed location of these towers, coupled with precise route planning algorithms and efficient remote control technology, the system enables UAV drones to fly pre-defined routes, perform automated charging/battery swapping, and facilitate rapid video data transmission, significantly enhancing operational effectiveness.
3. Experimental Validation
This study validated the system’s algorithms using existing lightweight UAV drones to patrol a section of the waterway.
UAV Patrol Protocol: A DJI M30 drone, launched from a tower-based nest, was deployed along a 30 km test channel section. The drone flew at a constant speed, covering 2 × 7.5 km per sortie before returning to the nest for battery exchange, then continuing in the opposite direction. Two round trips were flown per planned route: the first with the camera angled directly at the revetment for defect inspection; the second with the camera aligned with the sailing direction for ancillary facility inspection. Upon return, video data was uploaded for algorithmic processing, with results returned to an administrator for secondary verification. The entire 30 km patrol, including battery swaps, was completed in approximately 90 minutes—a roughly tenfold efficiency increase over traditional methods.
Data Validation Strategy: Due to the low natural occurrence of anomalous events during the test patrol, a sandbox simulation technique was employed to augment the sample size. Realistic anomaly scenarios were recreated along the channel bank to provide more data for a conclusive performance comparison.
Under favorable conditions (good weather, no wind, no obstructions), and disregarding the accuracy of identifying “no anomaly” under normal conditions, the revetment defect detection accuracy reached 80.0%, and the ancillary facility missing/damage recognition accuracy was 89.3%, meeting the basic requirements for daily lock and channel patrols.
| Algorithm / Metric | Revetment Inspection | Ancillary Facility Inspection |
|---|---|---|
| Actual Number of Anomalies (Simulated) | 18 | 29 |
| Number of Anomalies Identified by Algorithm | 20 | 28 |
| Number Confirmed After Manual Verification | 16 | 25 |
| Accuracy (Confirmed/Actual) | 80.0% | 89.3% |
| Miss Rate ((Actual – Confirmed)/Actual) | 11.1% | 13.8% |
The number of anomalies confirmed after manual review closely approximated the actual number, though a certain miss rate persisted. Furthermore, factors such as high wind (causing camera shake), vegetation occlusion, and wave action adversely affected recognition reliability. Therefore, while the two algorithmic applications developed in this study cannot yet completely replace manual inspection, they serve as a highly effective supplement. They provide crucial support for the intelligent analysis and proactive warning of anomalous events, significantly improving the quality and efficiency of waterway patrols while reducing operational costs.
Conclusion
This research delves into the deepened application of UAV technology within the inspection and maintenance sector of the port and waterway industry. Addressing the issue of continued reliance on manual review for UAV drones footage in lock area inspections—a process that remains labor-intensive and costly—we demonstrate the combined application of UAV drones technology and machine vision. This integration provides port and waterway management authorities with intelligent detection and early-warning support for events such as approach channel revetment defects and abnormal status of ancillary facilities, powerfully promoting the transformation of inspection and maintenance business models.
The algorithms for UAV drones-based lock and channel inspection require further optimization in both breadth and depth. Future work will focus on achieving more detailed classification of defect types, improving the accuracy of defect identification, and reducing false positives. Expanding the library of detectable events and enhancing robustness under diverse environmental conditions (e.g., poor lighting, rain, foliage) are critical next steps. As the “low-altitude economy” matures and regulations evolve, the role of UAV drones as autonomous, intelligent agents in critical infrastructure management is poised to expand, driving forward the vision of truly smart and resilient waterways.