Transmission lines serve as the “expressways” for electrical energy transmission, forming a critical guarantee for the stable operation of the power system and the rapid development of the national economy. In recent years, with the in-depth exploration and research into ultra-high voltage and ultra-high voltage technologies in many regions, the construction scale of transmission lines has increased exponentially. Ensuring the continuous and reliable delivery of electricity to end-users necessitates routine inspection and maintenance of these lines, with the timely detection and removal of foreign objects being paramount.
For a long time, manual visual inspection has been the primary method for line patrols. However, this approach suffers from two significant shortcomings. Firstly, it is highly labor-intensive and time-consuming, with inspection efficiency often难以保证. Secondly, constrained by environmental factors such as terrain, climate, and land use, many transmission lines traverse complex areas like mountains, rivers, and gorges, introducing significant and uncontrollable risks for human inspectors.
To reduce the cost of manual patrols and further liberate the workforce, new inspection methods utilizing intelligent devices as carriers have gradually emerged in recent years. The transmission line inspection robot is a relatively common high-voltage line patrol device. This equipment typically operates on the ground wire of the transmission line, controlled via remote操控 by workers on the ground. As it moves along the line, it uses its onboard cameras to capture images of the line environment ahead. Personnel at the base station then screen these images or use existing object recognition algorithms to identify foreign objects. Although inspection robots can partially replace manual patrols, they face a major limitation: the presence of numerous line本体 obstacles such as jumper wires, vibration dampers, and insulators. These obstacles often block the robot’s path, preventing smooth operation along the line and severely impacting the efficiency of foreign object detection during inspections.
To overcome the limitations of inspection robots and further enhance the efficiency of line foreign object detection, Unmanned Aerial Vehicles (UAV drones) have gradually become vital tools for grid line inspection. UAV drone inspection typically involves storing captured images in an onboard memory system, which are later exported for manual analysis after the patrol is complete. This method still consumes considerable time and effort from line workers. With the continuous advancement of intelligent algorithms such as machine learning and deep learning models in recent years, the integration of deep learning-based object detection models with UAV drones has emerged as a key research focus for foreign object detection. However, high-performance detection models often come with a large number of parameters and high computational complexity, making it difficult to embed them directly into UAV drones with limited memory and computational resources. Instead, images often need to be transmitted back to a server for processing, which severely compromises the real-time performance and accuracy of foreign object detection during the flight.
Therefore, researching how to leverage UAV drone patrols to reduce labor costs and improve the real-time performance and accuracy of foreign object detection is essential. This work proposes a method for UAV drone-powered transmission line inspection and foreign object recognition based on a lightweight convolutional neural network. The method introduces hierarchical Dropout and an attention mechanism into the lightweight network GhostNet, using the YOLO v3 head detector to construct a novel lightweight foreign object recognition network (LW-FORNet) designed for embedding into inspection UAV drones. Furthermore, to meet the training requirements of LW-FORNet, a foreign object image dataset was collected and organized based on 10kV and above transmission lines in a designated area. Data augmentation techniques such as geometric transformations were applied, resulting in a final dataset of 35,000 images. Experimental results demonstrate that the proposed method can significantly improve the speed and accuracy of UAV drone patrols, enabling UAV drones to perform transmission line foreign object recognition quickly and accurately.
Problem Definition: UAV Drone Line Inspection Task
Transmission line inspection based on UAV drones represents a novel patrol method. This technology effectively brings the inspector’s vision closer to the transmission line, allowing for a close-up “physical examination” to promptly identify foreign objects present on the lines. The foreign object detection task in UAV drone line inspection typically involves seven steps: selecting an operator, pre-inspection preparation, UAV drone check, initiating the inspection, video/image capture, obstacle recognition, and information transmission. Specifically, before the inspection, the operator must first define the patrol route and take-off/landing locations, and survey the geographical environment and altitude of the target line. Next, checks are performed on the UAV drone’s airframe integrity, battery charge, remote communication, and data transmission status. Finally, the UAV drone follows the pre-planned route, tower by tower, using its high-definition camera to capture images of obstacles. A target recognition algorithm performs foreign object detection, and the results are transmitted in real-time to the ground control station, enabling the operator to monitor the inspection results promptly. The core of this workflow is the obstacle recognition step. This work focuses on this core component, proposing a lightweight convolutional neural network to achieve fast and accurate recognition of line foreign objects.
Dataset Collection and Processing for UAV Drone Inspection
A total of 3,000 original line images were collected, encompassing both line本体 obstacle images and non-body foreign object images that threaten line safety. This dataset was categorized into 7 classes.本体 obstacles include 4 categories: bird repellent spikes, vibration dampers, insulators, and line clamps. Non-body foreign objects include 3 categories: bird nests, plastic debris, and tree branches/foliage. To meet the training requirements of convolutional neural networks, data augmentation techniques including rotation and scale transformation were applied to the original dataset. Rotation angles were set to -30°, -15°, 15°, and 30°. Scale transformation ratios were set to multiples of 0.8, 0.9, 1.1, and 1.2. The augmented dataset totals 35,000 images, with 5,000 images per foreign object class. The LabelImg tool was used to annotate targets in the images, resulting in bounding boxes around the obstacles. A summary of the dataset categories and their distribution is provided in the table below.
| Object Category | Type | Number of Original Images | Number after Augmentation |
|---|---|---|---|
| Bird Repellent Spikes | Line本体 Obstacle | ~430 | 5,000 |
| Vibration Damper | Line本体 Obstacle | ~430 | 5,000 |
| Insulator | Line本体 Obstacle | ~430 | 5,000 |
| Line Clamp | Line本体 Obstacle | ~430 | 5,000 |
| Bird Nest | Non-body Foreign Object | ~430 | 5,000 |
| Plastic Debris | Non-body Foreign Object | ~430 | 5,000 |
| Tree Branches/Foliage | Non-body Foreign Object | ~430 | 5,000 |
| Total | 3,000 | 35,000 |
The Proposed LW-FORNet Algorithm for Foreign Object Recognition
This section details the proposed UAV drone line inspection foreign object recognition algorithm based on a lightweight convolutional neural network. First, the construction of the lightweight convolutional neural network LW-FORNet for foreign object recognition is introduced. Then, the training process of LW-FORNet on the collected foreign object dataset is described. Finally, the workflow of the UAV drone line foreign object recognition algorithm based on LW-FORNet is presented.
Construction of the LW-FORNet Lightweight Convolutional Neural Network
The constructed LW-FORNet network is a novel lightweight object detection network that incorporates design concepts from GhostNet, SENet, and YOLO v3, and introduces hierarchical Dropout to mitigate overfitting during training. The entire network can be divided into three parts: data processing, a feature extraction backbone network, and a head detector.
The feature extraction backbone network primarily uses depthwise separable convolutions and 1×1 convolutions for feature learning. Depthwise separable convolutions perform convolution operations only on部分 channels; feature fusion via 1×1 convolution can achieve results comparable to traditional convolution. The parameter count and computational cost of a standard convolution operation are given by:
$$P_1 = C_i \times k_w \times k_h \times C_o$$
$$F_1 = C_i \times k_w \times k_h \times W \times H \times C_o$$
where $P$ and $F$ represent the parameter count and computational cost (FLOPs) respectively, $C_i$ is the number of input channels, $C_o$ is the number of output channels (i.e., the number of filters in that layer), $k_w$ and $k_h$ are the width and height of the convolutional kernel, and $W$ and $H$ are the width and height of the feature map. The formulas for depthwise separable convolution are:
$$P_2 = k_w \times k_h \times C_o$$
$$F_2 = k_w \times k_h \times W \times H \times C_o$$
From these formulas, it is evident that the parameter count and computational cost of depthwise separable convolution are only $1/C_i$ of those of standard convolution. Therefore, using depthwise separable convolution significantly reduces the parameter count and computational complexity of LW-FORNet. Furthermore, to endow LW-FORNet with a lightweight structure, the design philosophy of GhostNet is leveraged within the feature extraction backbone to further reduce parameters and computational complexity. This structure主要分为 three steps. First, depthwise separable convolutions of varying scales are applied to a proportion of the input feature maps for convolution, where each original feature map corresponds to $s$ output feature maps. Second, the remaining feature maps from the first step are passed through an identity mapping to obtain new output feature maps. Finally, the two sets of feature maps are concatenated to produce the final output. Compared to a traditional convolutional structure, this approach yields a significant reduction in computational cost and parameter count:
$$f’ = \frac{C_i \times H \times W \times C_o \times k_w \times k_h}{\frac{C_o}{s} \times H \times W \times k_w \times k_h \times C_i + (s-1) \times \frac{C_o}{s} \times H \times W \times d \times d} \approx \frac{s \times C_i}{s + C_i – 1} \approx s$$
$$p’ = \frac{C_i \times C_o \times k_w \times k_h}{\frac{C_o}{s} \times k_w \times k_h \times C_i + (s-1) \times \frac{C_o}{s} \times d \times d} \approx \frac{s \times C_i}{s + C_i – 1} \approx s$$
Here, $C_o$ represents the number of feature maps generated from each original feature map via convolutional transformation, which is much smaller than the number of input feature maps $C_i$. The term $\frac{C_o}{s}$ represents the number of original input feature maps undergoing depthwise separable convolution. From these formulas, it can be concluded that the feature extraction structure of LW-FORNet can compress the network’s parameter count and computational cost by a factor of approximately $s$, enabling the goal of direct embedding into small UAV drones.
In addition to the aforementioned designs for reducing parameters and computation, an attention mechanism is incorporated between the second and third layers of LW-FORNet to better learn features of target regions, thereby improving target detection accuracy. This mechanism encourages the network to focus more on target areas during training. The implementation of the attention mechanism, based on SENet, involves three main steps. First, the feature map from the second layer undergoes a dimensionality transformation via global average pooling:
$$p_c = F_{sq}(v_c) = \frac{1}{H \times W} \sum_{i=1}^{H} \sum_{j=1}^{W} v_c(i, j), \quad p \in \mathbb{R}^{C_i}$$
where $p_c$ is the one-dimensional vector obtained after performing global average pooling on each input channel, $W$ and $H$ are the width and height of the feature map, and $v_c(i, j)$ is the feature value at row $i$, column $j$ in the feature map of channel $c$. Second, to capture non-mutually-exclusive nonlinear relationships among channels, ReLU and sigmoid activation functions are applied:
$$t = F_{ex}(p, W) = \sigma(g(p, W)) = \sigma(W_2 \cdot \text{ReLU}(W_1 \cdot p))$$
where the ReLU and sigmoid activation functions are defined as:
$$\text{ReLU}(x) = \max(0, x)$$
$$\sigma(x) = \frac{1}{1 + e^{-x}}$$
Finally, the activation values obtained for each channel from the previous step are multiplied element-wise with the original input feature map to produce the final output for this layer:
$$o_c = F_{scale}(v_c, t_c) = t_c \cdot v_c$$
The final part of the LW-FORNet network incorporates the YOLO v3 head detector for the final foreign object recognition output. YOLO v3 is a detector well-suited for small object detection, utilizing the K-means clustering algorithm and an anchor box mechanism for object localization. In the designed LW-FORNet network, three different scale anchor boxes (13×13, 26×26, 52×52) are set per pixel point, i.e., three candidate bounding boxes. Each candidate box contains five fundamental parameters: center coordinates $(x, y)$, box width $w$, height $h$, and class confidence $c$. Therefore, the number of filters in the final convolutional layer is:
$$n = m \times (x + c)$$
where $m$ represents the three different scales, $x$ is the number of parameters per candidate box (typically 5 for box coordinates and objectness), and $c$ is the number of classes in the dataset. With $x=5$ (4 for box offsets + 1 for objectness score) and $c=7$, we get $n = 3 \times (5 + 7) = 36$. Thus, the final convolutional layer has 36 filters. To obtain relatively accurate candidate boxes, Intersection over Union (IoU) is used to assess the positional reasonableness of a candidate box. This metric evaluates how well a candidate box covers the target region by calculating the area overlap ratio between the output box and the ground truth box:
$$\text{IoU} = \frac{DR \cap GT}{DR \cup GT}$$
where $DR$ represents the final candidate box output by LW-FORNet, and $GT$ is the ground truth box annotated in the original dataset.
Training the LW-FORNet Network
After constructing the lightweight convolutional neural network LW-FORNet, it is trained on the collected transmission line foreign object image dataset. To accelerate LW-FORNet training and prevent overfitting during the process, a hierarchical Dropout mechanism is introduced. This mechanism randomly skips certain convolutional layers during training, causing these skipped layers not to participate in the current iteration. The convolutional kernels that do not participate in training inherit their parameters from the most recent update. In the next iteration, the Dropout mechanism will again randomly skip certain layers, and this process continues until training is complete. Introducing the hierarchical Dropout mechanism significantly enhances the network’s robustness. Furthermore, to facilitate network training, the PyTorch deep learning framework was used for development, and a training environment was set up on a GPU. The configuration of the training platform is summarized below.
| Component | Model/Specification |
|---|---|
| Development Tool | Sublime Text |
| Programming Language | Python 3.8 |
| Deep Learning Framework | PyTorch |
| CPU | Intel Xeon E5-2609 v4 @ 1.70GHz |
| GPU | NVIDIA GeForce RTX 2080 Ti |
| Operating System | Ubuntu 18.04 LTS |
Workflow for LW-FORNet-based Line Foreign Object Recognition
After training is completed, the trained LW-FORNet model is embedded into the inspection UAV drone to perform the line foreign object recognition task. The core workflow of this task involves six main steps, with LW-FORNet being the core component enabling foreign object recognition on the UAV drone: exporting the trained LW-FORNet model, embedding it into the patrol UAV drone, capturing video/images via the onboard camera, feeding the images into the LW-FORNet recognition model, performing foreign object recognition and bounding box annotation, and transmitting the recognition results back to the ground station.
Experiments and Analysis
To validate the effectiveness of the proposed lightweight foreign object recognition network LW-FORNet for the UAV drone line inspection foreign object detection task, extensive quantitative and qualitative experiments were conducted. Comparisons were made with existing detection networks across two dimensions: recognition accuracy and the time required to process a single image. Furthermore, the proposed network was compared with state-of-the-art lightweight networks using metrics of parameter count and computational complexity. Additionally, the average IoU values computed by LW-FORNet for various foreign object categories were presented. Finally, visualization results of some foreign object detection outputs are provided. Due to the large parameter counts and high computational complexity of models like Faster R-CNN and YOLOF, which make them unsuitable for direct embedding into UAV drones, the accuracy and speed comparison experiments were conducted on a backend server for fairness. The visualization experiments were performed by directly embedding LW-FORNet into a UAV drone.
First, the accuracy and detection time of the proposed method on the line foreign object recognition task are presented. For fair comparison, all networks were trained and tested on the same number of training and testing samples (the code for all compared networks is publicly available). The results in the table are averages from five experimental runs. The comparison data shows that the LW-FORNet network significantly outperforms other compared models in terms of the mAP metric and the time required to process a single image. It achieves an mAP of 81.6% and requires only 29ms to complete foreign object detection on one image, surpassing currently well-performing models like YOLO v3 and Dynamic Head. The time comparison also indicates that the proposed method enables real-time foreign object detection for transmission lines. Furthermore, the comparison between LW-FORNet* and LW-FORNet results demonstrates that introducing the hierarchical Dropout mechanism helps reduce overfitting during training and improves the network’s generalization capability.
| Method | mAP (%) | Inference Time per Image (ms) |
|---|---|---|
| DarkNet + YOLO v3 | 77.6 | 36 |
| Faster R-CNN | 79.2 | 122 |
| ResNet + SSD | 72.3 | 46 |
| GhostNet + YOLO v3 | 78.8 | 29 |
| Sparse R-CNN | 79.4 | 33 |
| YOLOF | 80.1 | 32 |
| Dynamic Head | 81.2 | 33 |
| LW-FORNet* (w/o Hierarchical Dropout) | 80.4 | 29 |
| LW-FORNet (Proposed) | 81.6 | 29 |
Second, the parameter count and floating-point operations (FLOPs, i.e., multiply-add operations) of LW-FORNet are compared against other lightweight networks. These metrics直观反映 the complexity of the networks. ShuffleNet+YOLO v3 is a structure using ShuffleNet as the feature extractor fused with the YOLO v3 detector; MobileNet+YOLO v3 follows a similar principle. Mobile-former and MobileViT are recently proposed state-of-the-art lightweight object detection networks. The comparison data shows that the LW-FORNet network achieves the best results in both parameter count and FLOPs, at 2.9M and 356M respectively, significantly lower than lightweight structures like MobileNet+YOLO v3 and MobileViT. This proves that the network structure design of LW-FORNet fully incorporates lightweight design principles, enabling its embedding into mobile devices with limited memory and computational resources, such as inspection UAV drones.
| Method | Parameter Count (M) | FLOPs (M) |
|---|---|---|
| ShuffleNet + YOLO v3 | 4.6 | 502 |
| MobileNet + YOLO v3 | 3.5 | 418 |
| Mobile-former | 4.2 | 496 |
| MobileViT | 5.6 | 598 |
| LW-FORNet (Proposed) | 2.9 | 356 |
Finally, to further verify whether the proposed method can accurately localize target regions, for each foreign object category, 100 images were randomly selected, and the average IoU value for each category was computed across three different detection models, including the well-performing Faster R-CNN and YOLO v3 on authoritative datasets like ImageNet. The IoU value represents the overlap between the output region and the ground truth region; a higher value indicates more accurate localization. The designed LW-FORNet network achieved higher IoU values than both Faster R-CNN and YOLO v3 across all foreign object categories, demonstrating that the designed network structure is more suitable for transmission line foreign object recognition. Furthermore, there is significant variation in the average IoU values among the seven common foreign object categories encountered during UAV drone patrols. Bird repellent spikes have a relatively low IoU value, primarily because this type of foreign object shares similar material and color with the transmission tower, increasing the feature learning difficulty for the LW-FORNet network. Plastic debris, a common non-body foreign object, has the highest IoU value, demonstrating the effectiveness of the proposed method for the line foreign object recognition task.
Moreover, to visually demonstrate the effectiveness of the proposed UAV drone line inspection foreign object detection method based on the lightweight convolutional neural network LW-FORNet, sample detection results are visualized. The images show the detection of various foreign objects including insulators, tree branches, bird nests, bird repellent spikes, and plastic debris. Each visualized bounding box is labeled in the top-left corner with the object type and confidence score. A higher confidence score indicates more accurate recognition. The accurate annotation results are closely tied to both the feature extraction backbone network and the head detector. The feature extraction backbone network thoroughly learns the features of foreign objects, while the head detector calculates the bounding box localization based on the high-level semantic information learned by the backbone, ultimately producing the visualized image results. The visualization results更直观地证明 that inspection UAV drones equipped with the LW-FORNet network can accurately localize foreign objects and identify obstacles present on the transmission lines, providing significant assistance and support for transmission line inspection.
Conclusion
To enhance the accuracy and speed of foreign object detection during UAV drone line inspections and improve the efficiency of transmission line patrols, a method for UAV drone line inspection foreign object recognition based on LW-FORNet was proposed. LW-FORNet is a novel lightweight object detection network whose structure incorporates design concepts from GhostNet and YOLO v3. During the training of the LW-FORNet network, a hierarchical Dropout mechanism and an attention mechanism were introduced to improve training stability and the network’s local perceptual capability. Furthermore, to meet the training requirements of LW-FORNet, a foreign object image dataset was collected and organized based on 10kV and above transmission lines in a designated city. Data augmentation techniques such as geometric transformations were applied, resulting in a final dataset containing 35,000 images. To validate the method’s effectiveness, extensive comparative experiments were conducted across multiple dimensions: detection accuracy, speed, network complexity, IoU values, and visualization. The experimental results show that the proposed method significantly improves both the real-time performance and accuracy of foreign object detection. It requires only 29ms to process a single image and achieves an mAP of 81.6%, surpassing classic object detection models like Faster R-CNN. In terms of complexity, it clearly outperforms recently proposed lightweight networks such as MobileViT. The successful deployment of this lightweight model on a UAV drone platform demonstrates its practical utility for real-world power line inspection tasks, paving the way for more autonomous and efficient grid maintenance using UAV drone technology.