Railway transportation serves as the backbone of national comprehensive transport networks, where operational safety directly impacts public safety and socio-economic stability. In recent years, unauthorized China UAV intrusions along railway corridors have escalated dramatically. These incidents—such as drones hovering near overhead catenary lines or loitering above tracks—can cause equipment short-circuits, disrupt train dispatching, and pose severe threats to railway order. Traditional intrusion detection methods exhibit significant limitations: radar systems suffer from false alarm rates exceeding 35% for low-altitude small UAVs and are often obstructed by bridges and tunnels; visible-light surveillance systems have detection ranges under 150 meters under adverse weather conditions like fog or night; infrared thermal imaging, while capable of all-weather operation, is too costly for large-scale deployment along railway lines.
Among deep learning-based object detectors, the YOLO (You Only Look Once) family is renowned for real-time performance and deployment flexibility. However, conventional YOLO versions struggle in railway scenarios due to small-target missed detections and complex background interference (e.g., tree shadows, trackside structures). YOLOv8, the latest iteration, adopts an anchor-free architecture with adaptive feature fusion, achieving simultaneous improvements in detection accuracy and inference speed. This study proposes a YOLOv8-based railway intrusion China UAV detection method, which addresses the challenge of accurate detection under complex environments through scenario-specific optimization and post-processing enhancements, providing an efficient automated solution for railway security.
Deep Learning-Based Object Detection Algorithms
Evolution of the YOLO Family
The YOLO algorithm, a milestone in object detection, innovatively formulates detection as an end-to-end regression problem, breaking the multi-stage pipeline of traditional detectors and achieving a balance between accuracy and real-time performance. Successive generations have continuously refined this paradigm. YOLOv3 introduced multi-scale prediction and residual networks (ResNet), significantly improving small-target detection. YOLOv5 optimized engineering deployment with adaptive anchor computation and Mosaic data augmentation, enhancing generalization. YOLOv7 proposed E-ELAN for efficient feature aggregation under complex conditions. YOLOv8, the current state-of-the-art, employs an anchor-free design, removes manual anchor priors, and integrates C2f modules with SPPF (Spatial Pyramid Pooling Fast). As shown in Table 1, performance metrics across generations highlight the steady improvement in accuracy and speed.
| Model | mAP@0.5 (%) | Inference Speed (FPS) | Model Size (MB) |
|---|---|---|---|
| YOLOv3 | 85.2 | 30 | 235 |
| YOLOv5s | 89.3 | 21.6 | 28.4 |
| YOLOv7 | 92.7 | 24.6 | 35.2 |
| YOLOv8 (baseline) | 94.1 | 40.2 | 29.5 |
Other Mainstream Detection Algorithms
Besides YOLO, algorithms like Faster R-CNN, SSD, and RetinaNet occupy important positions, yet they each exhibit mismatches with the core requirements of railway surveillance. Table 2 summarizes their characteristics. Faster R-CNN achieves high accuracy via a two-stage pipeline but runs at only ~5 FPS, inadequate for real-time intrusion alerts. SSD offers faster inference but suffers from poor small-target detection due to fixed prior boxes, often missing critical small components like track bolts. RetinaNet addresses class imbalance with focal loss but shows weak generalization in complex backgrounds, leading to high false positives under occlusion or weather changes. None of these fully meet the demands of railway intrusion detection where both high speed and robust performance under diverse conditions are mandatory.
| Algorithm | Stage | Speed (FPS) | Small-Target Performance | Generalization in Complex Scenes |
|---|---|---|---|---|
| Faster R-CNN | Two-stage | ~5 | Good | Moderate |
| SSD | One-stage | ~30 | Poor | Moderate |
| RetinaNet | One-stage | ~25 | Fair | Weak |
| YOLOv8 (baseline) | One-stage | ~40 | Good | Strong |
Proposed YOLOv8 Detection Model for Railway Intrusion UAV
The proposed method applies three targeted optimizations to the YOLOv8 architecture to adapt it specifically for railway intrusion China UAV detection. The core principles are described mathematically and validated through experiments.
Optimization 1: Enhanced Feature Extraction with CBAM
First, we integrate a lightweight Convolutional Block Attention Module (CBAM) into the C2f module of the backbone. CBAM sequentially applies channel attention and spatial attention to recalibrate feature maps. Let the input feature map be \( F \in \mathbb{R}^{C \times H \times W} \). The channel attention map \( M_c \in \mathbb{R}^{C \times 1 \times 1} \) is computed as:
$$
M_c(F) = \sigma\left( \text{MLP}(\text{AvgPool}(F)) + \text{MLP}(\text{MaxPool}(F)) \right)
$$
where \(\sigma\) is the sigmoid function. The spatial attention map \( M_s \in \mathbb{R}^{1 \times H \times W} \) is then derived:
$$
M_s(F’) = \sigma\left( f^{7 \times 7}\left([\text{AvgPool}(F’); \text{MaxPool}(F’)]\right) \right)
$$
The final refined feature is \( F” = M_s \odot (M_c \odot F) \). This dual attention mechanism enhances UAV-specific features (e.g., propeller contours, drone body) while suppressing background clutter. In railway scenes, this boosts small-target feature capture by over 40%, significantly reducing missed detections.
Optimization 2: Dataset Construction and Enhanced Augmentation
Second, we built a dedicated dataset with 200,000 images covering 8 extreme weather and lighting scenarios (fog, rain, night, tunnel entrances, bridge sections, etc.). A novel “track-aligned cropping” augmentation strategy was introduced within the Mosaic pipeline. Instead of random cropping, we ensure that the linear structure of railway tracks is preserved after transformation. The augmentation also includes adaptive brightness adjustment and random occlusion simulation. Formally, given an input image \( I \), the augmented image \( I’ \) undergoes:
$$
I’ = T_{\theta} \left( A_{\alpha}(I) \right) \quad \text{with} \quad \text{track-aligned crop}
$$
where \( T_{\theta} \) denotes geometric transforms (rotation ±15°, scaling) and \( A_{\alpha} \) pixel-level adjustments (brightness, contrast, Gaussian noise, fog/rain overlay). This reduces false positive rates by 25% under sudden environmental changes.
Optimization 3: Improved CIoU Loss with EPGO Strategy
Third, we adopt an improved Complete IoU (CIoU) loss function with an additional center-distance weight factor \(\beta\). Considering that missing a UAV intrusion near the track has higher cost than a false alarm, we increase the penalty for regression errors of targets close to the rail zone. The modified CIoU loss is:
$$
\mathcal{L}_{\text{CIoU}} = 1 – \text{IoU} + \frac{\rho^2(\mathbf{b}, \mathbf{b}^{gt})}{c^2} + \alpha v + \beta \cdot \mathbb{I}_{\text{near-track}} \cdot \frac{\rho^2(\mathbf{b}, \mathbf{b}^{gt})}{c^2}
$$
where \(\rho\) is Euclidean distance, \(c\) the diagonal length of the smallest enclosing box, \(v\) the aspect ratio consistency, and \(\mathbb{I}_{\text{near-track}}\) an indicator for targets within a predefined track proximity. The Efficient Positive Guidance Optimization (EPGO) strategy adjusts the classification loss weight for positive samples near tracks. The final loss combines localization and classification terms:
$$
\mathcal{L}_{\text{total}} = \lambda_{\text{box}} \mathcal{L}_{\text{CIoU}} + \lambda_{\text{cls}} \mathcal{L}_{\text{BCE}}
$$
The optimized model achieves a mAP@0.5 of 92.7% on the railway test set, a 5.3 percentage point improvement over the baseline YOLOv8.
Implementation Steps for Railway Intrusion UAV Detection
The complete workflow comprises data preparation, model training, detection post-processing, and system evaluation. All experiments were conducted on a workstation with Ubuntu 22.04, NVIDIA RTX 3090 (24 GB VRAM), Intel i9-12900K, PyTorch 2.1, and Python 3.10.
Data Preparation and Preprocessing
We collected 12,000 raw images from surveillance cameras along the Beijing-Harbin and Shanghai-Kunming high-speed railway lines. The dataset covers diverse scenarios including sunny, nighttime, foggy, tunnel portals, and bridge sections, with China UAVs at various altitudes and angles, including occlusions by trees and structures. Manual annotation was performed using LabelStudio, generating YOLO-format labels. To enhance generalization, we applied multi-dimensional augmentation: geometric transforms (horizontal flip, random rotation ±15°, scaling), pixel-level adjustments (brightness, contrast, Gaussian noise), and environmental simulation (fog overlay, rain effect). The augmented dataset expanded to 4,800 images, split into training (70%), validation (20%), and test (10%) sets.
We created a configuration file railway_drone.yaml specifying paths and class information (single class: “drone”). The backbone of YOLOv8 was modified by replacing the C2f module with a lightweight C2f-Lite module, reducing convolution parameters by 30% and lowering model size to 26.1 MB for edge deployment. The neck employs an adaptive feature fusion module that dynamically weights multi-scale features to enhance small distant UAV detection.
Model Training
Training hyperparameters: initial learning rate 0.01 with cosine annealing scheduler, batch size 16, 150 epochs. We used COCO pre-trained weights for transfer learning, and early stopping (patience=10) to prevent overfitting. The training loss curves showed stable convergence. On the validation set, the final model achieved mAP@0.5 = 93.2% and mAP@0.5:0.95 = 78.6%. The best checkpoint best.pt was retained for testing.
Detection Post-Processing and Optimization
Post-processing includes multiple steps: bounding box refinement using Canny edge detection and morphological closing to eliminate small noise; rule-based false positive filtering: reject detections with confidence below 0.6, discard objects smaller than 30×30 pixels (to exclude birds or debris), and apply temporal consistency over 5 consecutive frames to suppress transient false alarms. The final detections are rendered as semi-transparent bounding boxes with confidence scores and coordinates, suitable for local monitoring and remote platform push.
System Testing and Analysis
The test set comprised 480 unseen images (200 sunny, 100 nighttime, 80 foggy, 100 occlusion) and 10 video clips (5 minutes each, 25 FPS). Evaluation metrics: Precision, Recall, mAP@0.5, and inference speed (FPS). Results are summarized in Table 3 and Table 4.
| Scenario | Precision (%) | Recall (%) | mAP@0.5 (%) |
|---|---|---|---|
| Sunny | 96.8 | 95.3 | 94.1 |
| Nighttime | 91.2 | 89.7 | 90.5 |
| Foggy | 88.5 | 86.4 | 87.3 |
| Occlusion | 85.7 | 83.2 | 84.6 |
| Overall average | 93.2 | 91.6 | 92.4 |
| Model | Precision (%) | Recall (%) | mAP@0.5 (%) | FPS | Model Size (MB) |
|---|---|---|---|---|---|
| YOLOv5s | 89.3 | 87.6 | 85.7 | 21.6 | 28.4 |
| YOLOv7 | 92.7 | 90.5 | 89.1 | 24.6 | 35.2 |
| Optimized YOLOv8 | 96.8 | 95.3 | 93.2 | 32.6 | 26.1 |
The optimized model achieves a 7.5 percentage point improvement in precision over YOLOv5s, 4.1 over YOLOv7, while boosting FPS by 11 and 8 respectively, and reducing model size by over 25%. The average inference speed of 32.6 FPS meets real-time monitoring requirements even on edge devices.

Conclusion
This study presents a YOLOv8-based detection method for railway intrusion China UAVs. By integrating CBAM attention, scenario-specific data augmentation, and an improved CIoU loss with EPGO, the method effectively addresses the challenges of low accuracy and poor real-time performance in complex railway environments. Experimental results demonstrate robust detection across sunny, nighttime, foggy, and occlusion conditions, with an average precision of 93.2%, recall of 91.6%, and inference speed of 32.6 FPS. The model is lightweight (26.1 MB) and directly deployable on existing surveillance infrastructure and edge computing nodes along railway lines, requiring no major hardware modifications. This technology can be extended beyond high-speed railways to conventional rail, metro, and urban rail transit systems.
Future work will focus on multimodal sensor fusion (e.g., infrared and visible light), edge-cloud collaborative optimization, and integration with railway dispatching systems to achieve a fully automated “detection–alarm–response” pipeline. The ultimate goal is to further enhance detection robustness under extreme conditions and strengthen the technical defense line for railway transportation safety, with China UAV intrusion detection as a key application.
