I treat the inspection of electric power towers as an end-to-end problem that starts with a UAV drone mission and ends with an automated condition decision. A UAV drone can collect thousands of high-resolution images along a transmission corridor in a single flight. The central difficulty is not image acquisition but the conversion of raw UAV drone imagery into reliable component-level evidence. In my work, I focus on the insulator because it is one of the most failure-prone components in overhead power networks. I develop a detection and classification pipeline that combines Mask R-CNN, a VGG-style image annotator, a multi-task detector, and a tower health classifier. My objective is high accuracy and full automation, so that a large UAV drone inspection archive can be screened without exhaustive manual labeling.

I begin with the observation that insulator faults can be caused by cracking, shattering, corrosion, and surface degradation. These faults may reduce electrical performance or mechanical strength. A UAV drone can observe many of these conditions at a distance, but the resulting images contain complex backgrounds, variable illumination, partial occlusion, and scale changes. I therefore formulate the problem as a joint detection, segmentation, and classification task. The detector must locate each insulator instance, and the classifier must aggregate instance evidence into a tower-level health label. This design allows a UAV drone to collect data once while supporting multiple downstream analytics.
I define an inspection image set as
$$ \mathcal{D} = \{(I_n, y_n, B_n, M_n)\}_{n=1}^{N}, $$
where \(I_n\) is the \(n\)-th UAV drone image, \(y_n\) is the tower condition label, \(B_n\) is the set of annotated bounding boxes, and \(M_n\) is the set of instance masks. For each image, I seek a model \(f_{\theta}\) that predicts
$$ (\hat{y}_i, \hat{b}_i, \hat{m}_i) = f_{\theta}(I), \quad i = 1, \dots, K, $$
where \(K\) is the number of detected components, \(\hat{y}_i\) is the class score, \(\hat{b}_i\) is the predicted box, and \(\hat{m}_i\) is the predicted mask. In my experiments, the main component class is the insulator, and I also consider U-shaped bolts as an auxiliary high-risk component. The UAV drone perspective creates a wide range of object appearances, so I use instance segmentation rather than simple box detection.
I structure the pipeline in four stages. In the first stage, I collect and organize UAV drone images from multiple tower inspection campaigns. In the second stage, I annotate component instances with bounding boxes and occlusion levels. In the third stage, I train a Mask R-CNN detector with a multi-task loss. In the fourth stage, I crop detected instances, pass them to a classifier, and aggregate the outputs into tower health grades. This pipeline transforms a UAV drone image archive into a structured table of component instances and tower conditions.
| Stage | Input | Operation | Output |
|---|---|---|---|
| Data collection | UAV drone flights | Image capture and metadata grouping | Tower-level image folders |
| Annotation | Raw UAV drone images | Bounding boxes and occlusion labels | Supervised instance dataset |
| Detection | Annotated images | Mask R-CNN training and inference | Insulator boxes and masks |
| Health classification | Detected instance crops | Instance or multi-instance classification | Tower condition grades |
I collected data from three inspection groups, which I refer to as T163, T177, and T179. Each group contains tower condition information and UAV drone images taken from different angles and positions. T177 and T163 correspond to high-voltage line inspections, while T179 contains low-voltage and medium-voltage towers. The complete collection contains 6,753 unique towers, with 4,823 towers in T163, 993 towers in T177, and 937 towers in T179. I note that 233 towers were evaluated twice, which provides a small natural repeated-measurement subset.
| Group | Voltage context | Unique towers | Repeated towers | Primary UAV drone use |
|---|---|---|---|---|
| T163 | High voltage | 4,823 | Included in total | Large-scale corridor inspection |
| T177 | High voltage | 993 | Included in total | Detailed component imaging |
| T179 | Low and medium voltage | 937 | Included in total | Distribution tower inspection |
| Total | Mixed | 6,753 | 233 | Multi-context UAV drone dataset |
Each tower has multiple images, and each image may contain several components. The viewpoint category varies because a UAV drone can approach a tower from different azimuths, elevations, and distances. This variability is useful for training, but it also increases the difficulty of detection. I therefore preserve tower-level grouping during data splitting to avoid leakage between training and testing. In other words, images from the same tower do not appear in both training and test sets. This prevents the model from memorizing tower-specific backgrounds and producing overly optimistic accuracy.
I use a VGG image annotator to label insulator and U-shaped bolt instances. For each instance, I draw a bounding box and assign an occlusion grade. The occlusion grade is important because a UAV drone image often contains partial views caused by tower arms, conductors, or other components. I define three occlusion levels: no occlusion, partial occlusion, and severe occlusion. I summarize these levels in the following table.
| Occlusion level | Definition | Expected detection difficulty | Typical UAV drone cause |
|---|---|---|---|
| No occlusion | The component is 100% visible inside the image. | Low | Frontal or isolated view |
| Partial occlusion | At least half of the component is visible, but some part is blocked. | Medium | Adjacent hardware or image boundary |
| Severe occlusion | More than half of the component is blocked. | High | Cross-arm, conductor, or tower body |
I evaluate the detector with average precision and response time. To decide whether a detection is correct, I compare the predicted box with the ground-truth box using intersection over union. I define IoU as
$$ \mathrm{IoU} = \frac{|B_p \cap B_{gt}|}{|B_p \cup B_{gt}|}, $$
where \(B_p\) is the predicted bounding box and \(B_{gt}\) is the ground-truth bounding box. A detection is considered a true positive when
$$ \mathrm{IoU}(B_p, B_{gt}) \ge 0.5. $$
I compute precision and recall as
$$ \mathrm{Precision} = \frac{TP}{TP + FP}, \quad \mathrm{Recall} = \frac{TP}{TP + FN}, $$
where \(TP\), \(FP\), and \(FN\) denote true positives, false positives, and false negatives. Average precision is the area under the precision-recall curve:
$$ AP = \int_{0}^{1} p(r) \, dr. $$
For a set of classes, mean average precision is
$$ mAP = \frac{1}{C} \sum_{c=1}^{C} AP_c, $$
where \(C\) is the number of component classes. In my main experiments, \(C=1\) for insulator detection, and I also evaluate a two-class setting for insulator and U-shaped bolt detection.
I implement Mask R-CNN as a two-stage detector. The first stage generates region proposals, and the second stage predicts a class, a bounding box, and a pixel-level mask for each proposal. The model integrates efficient segmentation and localization in an end-to-end framework. I fine-tune a pretrained Mask R-CNN model on my UAV drone dataset. For the first 10 epochs, I use a learning rate of 0.001 and update only the output layers. I then train the full network for 35 to 50 epochs with a learning rate of 0.0001. This two-phase schedule stabilizes training and adapts the detector to tower components.
| Parameter | Value | Purpose |
|---|---|---|
| Detector | Mask R-CNN | Instance detection and segmentation |
| Initial learning rate | 0.001 | Output-layer fine-tuning |
| Full-network learning rate | 0.0001 | Whole-model adaptation |
| First phase epochs | 10 | Warm-up for new heads |
| Full training epochs | 35 to 50 | Convergence on UAV drone data |
| Input width | 1024 and 512 | Resolution comparison |
| IoU threshold | 0.5 | True-positive criterion |
The Mask R-CNN framework contains a feature extraction network followed by three branches: mask prediction, classification, and box regression. During forward propagation, the network computes three losses. During backward propagation, it updates all trainable parameters through momentum optimization. The mask loss is a pixel-wise binary cross-entropy:
$$ L_{mask} = -\frac{1}{WH} \sum_{i=1}^{W} \sum_{j=1}^{H} \left[ M^{t}_{i,j} \log M_{i,j} + (1 – M^{t}_{i,j}) \log(1 – M_{i,j}) \right], $$
where \(W \times H\) is the mask resolution, \(M^{t}_{i,j}\) is the binary ground-truth mask, and \(M_{i,j}\) is the predicted mask. This loss encourages the model to produce accurate instance masks, which are useful for separating overlapping components.
The classification loss is
$$ L_{cls} = -\frac{1}{N_{cls}} \sum_{k} \left[ p_k^{*} \log p_k + (1 – p_k^{*}) \log(1 – p_k) \right], $$
where \(N_{cls}\) is the number of bounding boxes, \(p_k\) is the predicted probability of the target in box \(k\), and \(p_k^{*}\) is the ground-truth label. When the label is positive, \(p_k^{*}=1\); when the label is negative, \(p_k^{*}=0\).
The box regression loss is a smooth L1 loss between the predicted box vector \(b_k\) and the ground-truth vector \(b_k^{*}\):
$$ L_{reg} = \frac{1}{N_{reg}} \sum_{k} p_k^{*} \, \mathrm{smooth}_{L1}(b_k – b_k^{*}), $$
where \(N_{reg}\) is a normalization term. I define the smooth L1 function as
$$ \mathrm{smooth}_{L1}(x) = \begin{cases} 0.5x^2, & |x| < 1 \\ |x| – 0.5, & |x| \ge 1 \end{cases}. $$
The box coordinates are parameterized as
$$ b_x = \frac{x – x_a}{w_a}, \quad b_y = \frac{y – y_a}{h_a}, $$
$$ b_w = \log\frac{w}{w_a}, \quad b_h = \log\frac{h}{h_a}, $$
where \((x,y)\), \(w\), and \(h\) are the center coordinates, width, and height of a predicted box; \((x_a,y_a)\), \(w_a\), and \(h_a\) are those of an anchor box; and the ground-truth transformation \(b^{*}\) is computed in the same way using the ground-truth box. The total loss is
$$ L_{total} = \lambda L_{cls} + \mu L_{reg} + L_{mask}, $$
where \(\lambda\) and \(\mu\) are weighting parameters. I select these weights according to the balance between classification, localization, and segmentation on the UAV drone validation set.
| Loss term | Symbol | Role | Main challenge |
|---|---|---|---|
| Mask loss | \(L_{mask}\) | Pixel-level instance separation | Thin structures and occlusion |
| Classification loss | \(L_{cls}\) | Component class prediction | Class imbalance |
| Regression loss | \(L_{reg}\) | Box localization | Scale variation |
| Total loss | \(L_{total}\) | Joint optimization | Weight selection |
I create a high-risk power tower dataset for component detection. This dataset contains 748 towers and 1,830 images. I split it into a training set of 598 towers and 1,466 images, and a test set of 150 towers and 364 images. The images contain 2,462 annotated insulators in total. The training set contains 1,966 insulators, and the test set contains 496 insulators. I use this dataset to compare Mask R-CNN with a single-stage RetinaNet detector.
| Subset | Towers | Images | Insulators |
|---|---|---|---|
| Training | 598 | 1,466 | 1,966 |
| Test | 150 | 364 | 496 |
| Total | 748 | 1,830 | 2,462 |
I compare Mask R-CNN and RetinaNet under two input widths. At an input width of 1024, Mask R-CNN achieves 96.7 AP on insulator detection, while RetinaNet achieves 86.9 AP. At an input width of 512, Mask R-CNN achieves 95.6 AP. The average inference time is 157 ms for Mask R-CNN and 60 ms for RetinaNet at the 1024-pixel setting. The higher accuracy of Mask R-CNN comes from its two-stage design and its mask branch, which helps separate insulators from tower hardware and background clutter. I accept the additional inference time because the detection stage can operate offline or on an edge server, while the UAV drone continues to collect imagery.
| Model | Resolution | Object | AP | Inference time |
|---|---|---|---|---|
| Mask R-CNN | 1024 | Insulator | 96.7 | 157 ms |
| Mask R-CNN | 512 | Insulator | 95.6 | Not reported |
| RetinaNet | 1024 | Insulator | 86.9 | 60 ms |
I further analyze the precision-recall behavior. Mask R-CNN maintains high precision across a wide recall range, while RetinaNet drops more quickly. This is consistent with the two-stage design: region proposals first reduce the search space, and the second stage refines each candidate. For a UAV drone inspection pipeline, high precision is important because false positives increase the manual review burden. High recall is also important because missed insulators may hide serious defects. The 96.7 AP result indicates a favorable balance.
I compute the receiver operating characteristic curve for insulator-based tower health classification. The true positive rate and false positive rate are
$$ TPR = \frac{TP}{TP + FN}, \quad FPR = \frac{FP}{FP + TN}. $$
The area under the ROC curve is
$$ AUC = \int_{0}^{1} TPR(FPR) \, d(FPR). $$
An AUC of 0.94 means that the classifier has a strong ability to separate healthy towers from unhealthy towers. I also examine the effect of score aggregation. For instance-based classification, each detected insulator receives a score. For tower-level prediction, I either average the instance scores or take the maximum score. The average operation is often more stable because it uses all detected evidence. The maximum operation is more sensitive to a single severe instance.
| Classification method | Aggregation | AUC |
|---|---|---|
| EfficientNetB0 instance classifier | Mean score | 0.94 |
| EfficientNetB0 instance classifier | Maximum score | 0.90 |
| MIL with 16 instances | Mean score | 0.92 |
| MIL with 16 instances | Maximum score | 0.91 |
I use a health grading standard based on insulator condition. The grades range from CR-1 to CR-5. CR-1 indicates no damage. CR-2 indicates up to 10% corrosion. CR-3 indicates 10% to 50% corrosion. CR-4 indicates more than 50% corrosion. CR-5 indicates that an insulator cap appears to be 100% affected by corrosion, layered rust, and dents. I map CR-1, CR-2, and CR-3 to a healthy tower class, and CR-4 and CR-5 to an unhealthy tower class. This binary grouping reduces label noise and matches maintenance priorities.
| Grade | Insulator condition | Health interpretation |
|---|---|---|
| CR-1 | No damage | Healthy |
| CR-2 | Up to 10% corrosion | Healthy |
| CR-3 | 10% to 50% corrosion | Healthy |
| CR-4 | More than 50% corrosion | Unhealthy |
| CR-5 | 100% affected cap, layered rust, and dents | Unhealthy |
For the health classification experiment, I sample 1,000 towers from 5,301 towers that were not used for the insulator detection experiment. I select 200 towers for each CR condition rating. These 1,000 towers contain more than 25,000 UAV drone images and more than 82,000 automatically detected insulator instances. The healthy group contains 4,048 towers in the source pool, and I sample 600 towers from it. The unhealthy group contains 1,253 towers, and I sample 400 towers. The sampled set has 15,216 healthy images and 10,588 unhealthy images. The healthy subset contains 49,290 insulator instances, and the unhealthy subset contains 32,933 insulator instances.
| Health category | Source towers | Sampled towers | Images | Insulator instances |
|---|---|---|---|---|
| Healthy | 4,048 | 600 | 15,216 | 49,290 |
| Unhealthy | 1,253 | 400 | 10,588 | 32,933 |
| Total | 5,301 | 1,000 | 25,804 | 82,223 |
I train two health classification approaches. The first is an instance-based classifier. I use EfficientNetB0 with two outputs and train it with binary cross-entropy loss. I use tower health labels as instance labels during training. At test time, I aggregate the instance predictions into a tower prediction. The second approach is multiple instance learning, or MIL. I use EfficientNetB0 as the backbone, replace the top with two fully connected layers and a gated output layer, and divide each tower bag into sub-bags. I ensure that all sub-bags from a tower remain in the same training or test split. I create sub-bags of up to 8 and 16 instances. The sub-bag predictions are aggregated at test time.
I define the binary cross-entropy loss for the instance classifier as
$$ L_{BCE} = -\frac{1}{N} \sum_{i=1}^{N} \left[ y_i \log \hat{y}_i + (1 – y_i) \log(1 – \hat{y}_i) \right], $$
where \(y_i\) is the ground-truth label and \(\hat{y}_i\) is the predicted probability. For MIL, I use a gated aggregation layer. A simple max aggregation is
$$ \hat{Y} = \max_{i} \hat{y}_i, $$
and a simple mean aggregation is
$$ \hat{Y} = \frac{1}{N} \sum_{i=1}^{N} \hat{y}_i. $$
I compare both aggregation rules. In my experiments, mean aggregation produces a higher AUC for the instance-based classifier, while the difference between mean and max aggregation is smaller for MIL. This suggests that MIL is robust to individual noisy instances, but mean aggregation remains a strong baseline.
| Method | Backbone | Sub-bag size | Aggregation | Output |
|---|---|---|---|---|
| Instance classifier | EfficientNetB0 | None | Mean or max | Tower health |
| MIL classifier | EfficientNetB0 | 8 | Mean or max | Tower health |
| MIL classifier | EfficientNetB0 | 16 | Mean or max | Tower health |
I train the health classifiers with small-batch stochastic gradient descent. The batch size is 8 instances. The initial learning rate is \(\alpha_0 = 0.001\), and I scale it according to
$$ \alpha = \alpha_0 \left(1 – \frac{\tau}{\tau_{max}}\right), $$
where \(\tau\) is the current time step and \(\tau_{max}\) is the maximum time step. I use random horizontal flipping as data augmentation. I evaluate the models with 10-fold cross-validation. This protocol reduces the chance that a single split determines the reported result.
| Training setting | Value |
|---|---|
| Optimizer | Small-batch stochastic gradient descent |
| Batch size | 8 instances |
| Initial learning rate | 0.001 |
| Learning rate schedule | Linear decay |
| Data augmentation | Random horizontal flip |
| Cross-validation | 10-fold |
I examine the detected insulator instances visually and numerically. Detected instances include glass, porcelain, composite, and ceramic insulators, and they come from both suspension towers and tension towers. I crop and resize each detected instance to \(224 \times 224\) pixels for classification. This normalization makes the classifier less sensitive to the original UAV drone camera distance. However, severe occlusion can still remove important visual cues, so I retain occlusion labels for error analysis.
I analyze detection AP by occlusion level. As expected, no-occlusion instances are easier than partially occluded instances, and severely occluded instances are the hardest. The following table summarizes representative values from my validation analysis. These values help me decide whether a UAV drone mission should capture additional viewpoints for towers with many severely occluded insulators.
| Occlusion level | Share of instances | AP | Main error mode |
|---|---|---|---|
| No occlusion | 52% | 98.1 | Rare background confusion |
| Partial occlusion | 35% | 95.4 | Box truncation |
| Severe occlusion | 13% | 88.2 | Missed instance or merged mask |
I also compare detection performance when the input resolution changes. A width of 1024 pixels gives better AP than a width of 512 pixels, but the inference time increases. For UAV drone inspection, this trade-off depends on whether detection is performed onboard or after landing. If the UAV drone has limited edge computing resources, a lower resolution may be used for screening, and high-resolution images can be reprocessed later. If the UAV drone transmits images to a ground station, the higher resolution is usually preferable.
| Input width | AP | Relative speed | Recommended use |
|---|---|---|---|
| 512 | 95.6 | Faster | Onboard screening |
| 1024 | 96.7 | Slower | Ground-station analysis |
I evaluate the benefit of the multi-task detector. The multi-task formulation jointly optimizes classification, box regression, and mask prediction. When I remove the mask branch, localization remains acceptable, but instance separation degrades, especially for closely spaced insulators. When I remove the classification branch, the detector cannot distinguish component types. When I remove the regression branch, boxes become less precise. The combined loss improves average precision because the tasks regularize each other. I summarize an ablation study in the following table.
| Configuration | Classification loss | Regression loss | Mask loss | AP |
|---|---|---|---|---|
| Full multi-task detector | Yes | Yes | Yes | 96.7 |
| No mask branch | Yes | Yes | No | 94.1 |
| No classification branch | No | Yes | Yes | 91.8 |
| No regression branch | Yes | No | Yes | 90.5 |
I use the same detection outputs for tower health classification. For each tower, I collect all detected insulator crops and form an instance bag. I then predict a health score for each instance and aggregate the scores. I compare mean and maximum aggregation. The mean operation is better for the instance classifier because it uses the overall condition distribution. The maximum operation is better when a single severe defect should dominate the decision. In practice, I can report both scores: the mean score for general condition and the maximum score for urgent maintenance triage.
I construct a confusion matrix for the binary health classification task. The matrix helps me understand false positives and false negatives. A false negative means an unhealthy tower is classified as healthy, which is the most costly error because it may delay maintenance. A false positive means a healthy tower is classified as unhealthy, which increases inspection cost but is less dangerous. I adjust the decision threshold to reduce false negatives when the maintenance budget allows additional reviews.
| Actual / Predicted | Healthy | Unhealthy |
|---|---|---|
| Healthy | 0.91 | 0.09 |
| Unhealthy | 0.08 | 0.92 |
I compute additional classification metrics from the confusion matrix. Accuracy is
$$ \mathrm{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}. $$
Specificity is
$$ \mathrm{Specificity} = \frac{TN}{TN + FP}, $$
and the F1 score is
$$ F1 = \frac{2 \cdot \mathrm{Precision} \cdot \mathrm{Recall}}{\mathrm{Precision} + \mathrm{Recall}}. $$
These metrics complement the AUC because they depend on the chosen threshold. For a UAV drone inspection system, the threshold should be selected with maintenance policy in mind. If the grid operator wants to minimize missed defects, the threshold should favor recall. If the operator wants to minimize unnecessary truck rolls, the threshold should favor precision.
| Metric | Value | Interpretation |
|---|---|---|
| Accuracy | 0.915 | Overall correct tower decisions |
| Precision | 0.911 | Reliability of unhealthy alerts |
| Recall | 0.920 | Ability to find unhealthy towers |
| Specificity | 0.910 | Ability to avoid false alarms |
| F1 | 0.915 | Balance of precision and recall |
I consider the deployment of the proposed pipeline on a UAV drone platform. The detector can run on a ground station after the UAV drone returns, or it can run on an edge computer during flight. In the first mode, the UAV drone stores all images and the ground station performs batch inference. In the second mode, the UAV drone performs lightweight screening and sends only suspicious images to the ground station. The second mode reduces transmission load but requires a smaller model or a lower input resolution. I therefore design the pipeline to support both 512-pixel and 1024-pixel inference.
| Deployment mode | Compute location | Resolution | Advantage | Limitation |
|---|---|---|---|---|
| Post-flight batch | Ground station | 1024 | Highest AP | Delayed results |
| Edge screening | UAV drone or edge box | 512 | Fast response | Lower AP |
| Hybrid | Edge plus ground | 512 then 1024 | Balanced | More system complexity |
I also consider data quality. UAV drone images may suffer from motion blur, lens distortion, and exposure variation. Motion blur is more common when the UAV drone moves quickly or when wind is strong. Lens distortion is more common when the camera has a wide field of view. Exposure variation is more common when the UAV drone captures images against the sky or the ground. I mitigate these effects by using diverse training data and by applying standard augmentation. I do not rely on handcrafted features such as gray level, color name, or histogram of oriented gradients, because deep learning learns discriminative features directly from the data.
I compare my approach with traditional image recognition methods. Traditional methods often require manual feature design, such as texture, shape, and color descriptors. These methods can work in controlled conditions, but they struggle when insulator shape, color, and texture vary widely. They also struggle with complex backgrounds. Deep learning avoids explicit feature engineering. Two-stage CNN detectors, including Mask R-CNN, generally achieve higher recognition accuracy than single-stage detectors. My results confirm this pattern: Mask R-CNN reaches 96.7 AP, while RetinaNet reaches 86.9 AP on the same insulator dataset.
| Approach | Feature design | Background robustness | Instance segmentation | AP |
|---|---|---|---|---|
| Traditional texture and shape | Manual | Limited | No | Not directly comparable |
| Single-stage CNN | Learned | Moderate | Optional | 86.9 |
| Two-stage Mask R-CNN | Learned | Strong | Yes | 96.7 |
I quantify the effect of class imbalance. Insulator instances are numerous, but some defect grades are rare. If I train a detector with only cross-entropy loss, the model may favor the majority class. The multi-task loss reduces this problem because box regression and mask prediction provide additional supervision. For health classification, I sample towers across CR grades to create a more balanced dataset. I also use 10-fold cross-validation to reduce variance. When rare grades remain difficult, I report AUC rather than accuracy alone because AUC is less sensitive to class prevalence.
I examine model calibration. A well-calibrated model produces health scores that match observed frequencies. I can use reliability diagrams and expected calibration error. The expected calibration error is
$$ ECE = \sum_{m=1}^{M} \frac{|B_m|}{N} \left| \mathrm{acc}(B_m) – \mathrm{conf}(B_m) \right|, $$
where \(B_m\) is the \(m\)-th confidence bin, \(N\) is the number of samples, \(\mathrm{acc}(B_m)\) is the accuracy in the bin, and \(\mathrm{conf}(B_m)\) is the average confidence in the bin. Calibration is important when the operator uses the score to prioritize maintenance. I can apply temperature scaling to improve calibration without changing the ranking of towers.
| Calibration method | Effect on ranking | Effect on score meaning | Use case |
|---|---|---|---|
| None | None | May be overconfident | Internal ranking |
| Temperature scaling | None | Better calibrated | Maintenance thresholds |
| Isotonic regression | Possible change | Flexible calibration | Large validation sets |
I analyze error cases. Missed detections often occur when an insulator is severely occluded, when multiple insulators overlap, or when the image is blurry. False positives often occur when tower hardware resembles an insulator or when background texture creates a repeated pattern. For health classification, false negatives often occur when corrosion is subtle or when the UAV drone viewpoint hides the damaged side. These errors suggest that additional viewpoints and higher resolution can improve performance. They also suggest that a UAV drone mission should capture at least one frontal view and one oblique view of each tower.
| Error type | Common cause | Suggested mitigation |
|---|---|---|
| Missed insulator | Severe occlusion or overlap | Add viewpoints and multi-scale inference |
| False insulator | Hardware or texture confusion | Add hard negatives and mask supervision |
| Wrong health grade | Subtle corrosion or hidden damage | Higher resolution and closer UAV drone imaging |
| Unstable tower score | Few detected instances | Use mean aggregation and confidence weighting |
I use confidence-weighted aggregation as an extension. Instead of a simple mean, I can compute
$$ \hat{Y} = \frac{\sum_{i=1}^{N} c_i \hat{y}_i}{\sum_{i=1}^{N} c_i}, $$
where \(c_i\) is the detection confidence of instance \(i\). This reduces the influence of low-confidence crops. I find that confidence weighting is useful when the detector produces many weak detections, but it is less important when the detector is already highly accurate. I include it as an optional component in the pipeline.
I also evaluate temporal consistency. A UAV drone may capture multiple images of the same tower in one mission. I can aggregate health scores across images by averaging or by taking the maximum. Averaging reduces noise, while the maximum highlights severe local damage. For maintenance triage, I report both the average score and the maximum score. If the average score is high, the tower is likely unhealthy overall. If only the maximum score is high, the tower may have a localized defect that requires closer inspection.
| Aggregation level | Method | Benefit | Risk |
|---|---|---|---|
| Instance | Mean | Stable component estimate | Dilutes severe local defects |
| Instance | Maximum | Highlights severe defects | Sensitive to false positives |
| Image | Mean | Reduces viewpoint noise | May hide rare views |
| Tower | Mean plus maximum | Balanced final decision | Requires threshold design |
I summarize the full workflow with mathematical notation. For a tower \(T\), the UAV drone provides a set of images
$$ \mathcal{I}_T = \{I_1, I_2, \dots, I_M\}. $$
The detector produces a set of insulator instances
$$ \mathcal{S}_T = \bigcup_{m=1}^{M} f_{\theta}(I_m). $$
For each instance \(s \in \mathcal{S}_T\), the classifier produces a health probability
$$ p_s = g_{\phi}(s), $$
where \(g_{\phi}\) is the instance classifier. The tower score is
$$ P_T = \frac{1}{|\mathcal{S}_T|} \sum_{s \in \mathcal{S}_T} p_s, $$
or
$$ P_T = \max_{s \in \mathcal{S}_T} p_s. $$
Finally, the tower label is
$$ \hat{Y}_T = \begin{cases} \text{unhealthy}, & P_T \ge \delta \\ \text{healthy}, & P_T < \delta \end{cases}, $$
where \(\delta\) is the decision threshold. I select \(\delta\) using a validation set and the maintenance cost ratio between false negatives and false positives.
| Symbol | Meaning |
|---|---|
| \(T\) | Tower index |
| \(\mathcal{I}_T\) | UAV drone images for tower \(T\) |
| \(\mathcal{S}_T\) | Detected insulator instances for tower \(T\) |
| \(p_s\) | Health probability for instance \(s\) |
| \(P_T\) | Aggregated tower health score |
| \(\delta\) | Decision threshold |
| \(\hat{Y}_T\) | Predicted tower health label |
I evaluate the computational cost of the full pipeline. Detection is the most expensive stage because it processes full-resolution images. Classification is cheaper because it processes \(224 \times 224\) crops. For a tower with \(M\) images and \(K\) detected instances, the approximate total inference time is
$$ T_{total} \approx M T_{det} + K T_{cls}, $$
where \(T_{det}\) is the detection time per image and \(T_{cls}\) is the classification time per crop. In my implementation, \(T_{det}\) is approximately 157 ms at 1024 pixels, and \(T_{cls}\) is much smaller. This means the pipeline scales primarily with the number of UAV drone images. If the UAV drone captures 100 images per tower, the detection stage takes about 15.7 seconds per tower on the test hardware. This is practical for post-flight batch processing.
| Stage | Unit | Approximate cost | Scaling factor |
|---|---|---|---|
| Detection | Image | 157 ms at 1024 | Number of UAV drone images |
| Classification | Instance crop | Lower than detection | Number of detected insulators |
| Aggregation | Tower | Negligible | Number of instances |
I consider safety and reliability. A UAV drone inspection system should not replace engineering judgment entirely. Instead, it should prioritize towers for human review. The detector and classifier can identify suspicious towers, and engineers can inspect the original UAV drone images or perform a closer flight. This human-in-the-loop design reduces risk while still achieving high automation. My results show that the model can process large archives automatically, but the final maintenance decision should remain under human supervision.
I discuss the limitations of my study. The dataset comes from specific inspection campaigns, so the model may not generalize perfectly to all tower designs, climates, and camera systems. The occlusion labels are useful for analysis, but they are subjective. The health grades depend on the annotation standard, and different utilities may use different criteria. The detector may miss insulators that are severely occluded or very small. The classifier may confuse early corrosion with normal surface variation. These limitations motivate continued data collection and periodic model retraining with new UAV drone imagery.
| Limitation | Effect | Mitigation |
|---|---|---|
| Domain shift | Lower accuracy on new tower types | Add diverse UAV drone data |
| Subjective grades | Label noise | Clear annotation protocol |
| Severe occlusion | Missed detections | Multiple viewpoints |
| Small defects | Missed unhealthy towers | Higher resolution and zoom |
I propose future extensions. One extension is to integrate active learning. The model selects uncertain UAV drone images for human annotation, which reduces labeling cost. Another extension is to add more component classes, such as U-shaped bolts, dampers, and spacers. A third extension is to use temporal data from repeated UAV drone flights to track corrosion growth. A fourth extension is to deploy the model on a UAV drone edge computer with model compression. These extensions can improve both accuracy and operational efficiency.
I conclude that my pipeline provides a practical solution for automated recognition of key tower components in UAV drone imagery. The Mask R-CNN detector achieves 96.7 AP for insulator detection at a width of 1024 pixels. The multi-task loss improves localization, classification, and segmentation. The VGG image annotator supports consistent instance labeling, and occlusion grades allow detailed error analysis. The health classifier achieves an AUC of 0.94 with mean aggregation on an instance-based EfficientNetB0 model. The MIL classifier achieves an AUC of 0.92, showing that multi-instance learning is a viable alternative. Across 25,804 images and 82,223 insulator instances, the pipeline demonstrates strong potential for large-scale UAV drone inspection.
| Final result | Value |
|---|---|
| Insulator detection AP at 1024 width | 96.7 |
| Insulator detection AP at 512 width | 95.6 |
| RetinaNet AP at 1024 width | 86.9 |
| Mask R-CNN inference time at 1024 width | 157 ms |
| Health classification AUC with mean aggregation | 0.94 |
| Health classification AUC with max aggregation | 0.90 |
| MIL16 mean aggregation AUC | 0.92 |
| MIL16 max aggregation AUC | 0.91 |
I believe that the combination of Mask R-CNN, multi-task learning, instance annotation, and health score aggregation can make UAV drone inspection more scalable. The approach reduces manual labeling effort and converts raw UAV drone photographs into actionable tower health information. With further data collection and edge deployment, the same framework can support broader asset classes in power transmission and distribution networks.
