In recent years, the rapid expansion of the civilian drone market has highlighted the critical need for effective drone training programs. Traditional assessment methods in drone training often rely on performance metrics and behavioral observations, which fail to capture the cognitive states of operators. This limitation can hinder the development of adaptive training systems that respond to individual skill levels. To address this gap, our study explores the feasibility of using electroencephalogram (EEG) signals and deep learning techniques to classify drone operator proficiency levels. Specifically, we propose a bidirectional long short-term memory (Bi-LSTM) network to analyze EEG data collected during simulated drone flight tasks. Our approach aims to provide a novel method for evaluating drone training effectiveness, with potential applications in real-time adaptive systems for drone training.
The proliferation of drones in various sectors, from agriculture to surveillance, underscores the importance of standardized drone training protocols. However, current drone training evaluations primarily focus on outcome-based measures, such as task completion time or error rates, neglecting the cognitive load and mental states of trainees. Cognitive load assessment through physiological signals, like EEG, has shown promise in other domains, such as online learning and gaming. By leveraging EEG data, we can gain insights into the neural correlates of skill acquisition during drone training. This study investigates how EEG features, combined with advanced neural networks, can classify operators into different proficiency levels, thereby enhancing drone training methodologies.
Our research is motivated by the growing demand for personalized drone training systems. Military drone operations have historically incorporated cognitive evaluations, but civilian drone training lacks such sophisticated approaches due to differences in scale, complexity, and operator demographics. We aim to bridge this gap by developing a machine learning model that uses EEG signals to classify drone training performance. The core of our methodology is a Bi-LSTM network, which excels at processing sequential data like EEG time-series. By extracting power spectral density (PSD) features from EEG signals, we train the model to distinguish between beginner, intermediate, and advanced drone operators. This approach not only improves assessment accuracy but also paves the way for adaptive drone training interfaces.
The Bi-LSTM network is a variant of recurrent neural networks (RNNs) designed to capture long-term dependencies in sequential data. Unlike standard LSTM networks, Bi-LSTM processes data in both forward and backward directions, allowing it to integrate context from past and future time points. This bidirectional capability is particularly beneficial for EEG analysis, where brain activity patterns may evolve over time during drone training tasks. The LSTM unit consists of three gates: the forget gate, input gate, and output gate, which regulate the flow of information. Mathematically, these operations can be expressed as follows:
$$f_t = \sigma(W_{xf} x_t + W_{hf} h_{t-1} + b_f)$$
$$i_t = \sigma(W_{xi} x_t + W_{hi} h_{t-1} + b_i)$$
$$o_t = \sigma(W_{xo} x_t + W_{ho} h_{t-1} + b_o)$$
$$\tilde{c}_t = \tanh(W_{xc} x_t + W_{hc} h_{t-1} + b_c)$$
$$c_t = f_t \otimes c_{t-1} + i_t \otimes \tilde{c}_t$$
$$h_t = o_t \otimes \tanh(c_t)$$
where \( \sigma \) denotes the sigmoid function, \( \tanh \) is the hyperbolic tangent function, \( \otimes \) represents element-wise multiplication, \( W \) are weight matrices, \( b \) are bias vectors, \( x_t \) is the input at time \( t \), \( h_t \) is the hidden state, and \( c_t \) is the cell state. In the Bi-LSTM architecture, the hidden states from forward and backward passes are concatenated to form the final output, enhancing the model’s ability to capture complex EEG dynamics during drone training.
Our experimental design involved 11 participants who engaged in simulated drone flight tasks using a DJI Flight Simulator. The simulator replicated real-world drone操控 scenarios, requiring operators to navigate through tunnels and pass through rings within a time limit. This setup ensured safety while providing a realistic drone training environment. EEG signals were recorded using a 32-channel EMOTIVE EPOC Flex headset at a sampling rate of 128 Hz. Participants performed at least 10 trials each, and their performance was scored based on ring-crossing accuracy and error rates. To classify proficiency levels, we derived a performance metric that combined rewards (e.g., average rings passed) and penalties (e.g., missed rings or crashes), as shown in the formula below:
$$\text{Performance Score} = \frac{\text{Average Rings Passed} \times 0.7 + \text{Average Score} \times 0.3}{\text{Total Missed Rings} \times 0.3 + \text{Total Crashes} \times 0.7}$$
Based on this score, operators were categorized into three drone training levels: beginner (0–4 points), intermediate (4–6 points), and advanced (>6 points). Table 1 summarizes the participants’ performance statistics and assigned proficiency levels, demonstrating the variability in drone training outcomes.
| Participant ID | Average Rings Passed | Average Score | Total Missed Rings | Total Crashes | Performance Score | Proficiency Level |
|---|---|---|---|---|---|---|
| 01 | 5.00 | 7.33 | 1 | 2 | 3.35 | Beginner |
| 02 | 6.00 | 14.33 | 1 | 1 | 8.50 | Advanced |
| 03 | 4.00 | 3.88 | 1 | 1 | 3.96 | Beginner |
| 04 | 5.78 | 8.33 | 2 | 2 | 3.27 | Beginner |
| 05 | 7.13 | 11.38 | 1 | 2 | 4.94 | Intermediate |
| 06 | 4.00 | 8.00 | 1 | 2 | 3.06 | Beginner |
| 07 | 6.38 | 8.39 | 2 | 1 | 5.37 | Intermediate |
| 08 | 2.44 | 2.56 | 1 | 1 | 2.48 | Beginner |
| 09 | 22.46 | 75.69 | 5 | 3 | 10.68 | Advanced |
| 10 | 11.38 | 26.63 | 4 | 3 | 4.83 | Intermediate |
| 11 | 7.38 | 16.50 | 3 | 2 | 4.40 | Intermediate |
EEG signal preprocessing was crucial for removing noise and artifacts. We applied high-pass filtering at 0.16 Hz and low-pass filtering at 45 Hz using linear phase FIR filters, followed by a 50 Hz notch filter to eliminate power line interference. Independent component analysis (ICA) was employed to separate neural sources from artifacts like eye movements and muscle activity. The cleaned signals were then re-referenced to the average of all channels. For feature extraction, we computed the power spectral density (PSD) across five frequency bands: delta (0.5–4 Hz), theta (4–8 Hz), alpha (8–13 Hz), beta (13–30 Hz), and gamma (30–80 Hz). These PSD features serve as inputs to our Bi-LSTM model, capturing the spectral characteristics of brain activity during drone training tasks.
The Bi-LSTM model architecture comprises multiple layers designed to handle sequential EEG data effectively. As detailed in Table 2, the network includes bidirectional LSTM layers, batch normalization layers, dropout layers for regularization, and dense layers for classification. This structure enables the model to learn temporal dependencies in EEG signals, which are indicative of cognitive states in drone training. We implemented the model using a deep learning framework and trained it with a three-fold cross-validation strategy, where 70% of the data was used for training, 20% for validation, and 10% for testing. This approach ensures robust evaluation of the model’s performance in classifying drone training proficiency levels.
| Layer Number | Layer Name | Description |
|---|---|---|
| 0 | Input Layer | (32, 1) |
| 1 | Bidirectional LSTM Layer | (32, 256) |
| 2 | Batch Normalization Layer | (32, 256), ε=10⁻⁶ |
| 3 | Dropout Layer | (32, 256), dropout rate=0.1 |
| 4 | LSTM Layer | (32, 64) |
| 5 | Batch Normalization Layer | (32, 64), ε=10⁻⁶ |
| 6 | Dropout Layer | (32, 64), dropout rate=0.05 |
| 7 | LSTM Layer | (32, 32) |
| 8 | Batch Normalization Layer | (32, 32), ε=10⁻⁶ |
| 9 | Dropout Layer | (32, 32), dropout rate=0.01 |
| 10 | Dense Layer | (None, 3), activation=’softmax’ |
During the drone training simulations, participants operated the virtual drone in a controlled environment, which allowed for consistent data collection. The experimental setup included a flight simulator that mimicked real-world drone操控 challenges, such as obstacle avoidance and precision maneuvering.
This visual representation highlights the immersive nature of the drone training tasks, where operators must maintain focus and adapt to dynamic scenarios. The EEG data collected from these sessions reflect the cognitive demands of drone training, enabling our model to associate specific brain patterns with skill levels.
Results from the three-fold cross-validation demonstrate the effectiveness of our Bi-LSTM model in drone training classification. The training and validation accuracy curves show rapid convergence, with accuracy reaching approximately 80% within 50 epochs. In some folds, fluctuations occurred due to data noise, but overall, the model achieved stable learning dynamics. The final test accuracy on unseen data was 98.21%, with sensitivity and specificity values of 90% and 92%, respectively. These metrics indicate that the model reliably distinguishes between beginner, intermediate, and advanced operators in drone training contexts. The normalized confusion matrix reveals high precision for each class: 0.95 for advanced, 0.99 for beginner, and 0.96 for intermediate, with minimal misclassification errors.
We compared our Bi-LSTM model with other neural network architectures, including artificial neural networks (ANN), standard LSTM, and gated recurrent units (GRU). As shown in Table 3, the Bi-LSTM model outperformed all others in terms of validation accuracy (87.67%), test accuracy (98.21%), and specificity (0.92). This superiority stems from its bidirectional processing, which captures comprehensive temporal features in EEG data during drone training. The ANN model, while simpler, achieved lower accuracy (81.97%), highlighting the need for advanced sequential models in cognitive load assessment. These findings underscore the potential of Bi-LSTM networks to revolutionize drone training evaluations by providing accurate, real-time proficiency classifications.
| Model | Validation Accuracy (%) | Test Accuracy (%) | Sensitivity | Specificity |
|---|---|---|---|---|
| ANN | 82.06 | 81.97 | 0.99 | 0.64 |
| LSTM | 84.67 | 93.30 | 0.97 | 0.79 |
| GRU | 82.67 | 94.69 | 0.98 | 0.86 |
| Bi-LSTM | 87.67 | 98.21 | 0.90 | 0.92 |
The high classification accuracy of our model has significant implications for drone training programs. By integrating EEG-based assessments, instructors can obtain objective measures of trainee cognitive states, complementing traditional performance metrics. This approach enables personalized drone training, where the difficulty of tasks adjusts based on real-time proficiency detection. For instance, beginners might receive more guidance, while advanced operators face complex challenges to hone their skills. Such adaptive systems could reduce training times and improve safety outcomes in drone operations. Moreover, our methodology can be extended to other domains where cognitive load monitoring is essential, such as aviation or surgical training.
Future research should address limitations of this study, such as the small sample size of 11 participants. Expanding the participant pool will enhance the generalizability of our model across diverse populations in drone training. Additionally, incorporating other physiological signals, like eye-tracking or heart rate variability, could provide a multimodal assessment framework. Exploring different deep learning architectures, such as convolutional neural networks (CNNs) combined with LSTMs, may further improve classification performance. We also plan to investigate real-time implementation of our model in live drone training sessions, allowing for instantaneous feedback and adjustment. These advancements will strengthen the robustness and applicability of EEG-based proficiency classification in drone training.
In conclusion, our study presents a novel Bi-LSTM network for classifying drone training proficiency levels using EEG signals. The model achieves exceptional accuracy, sensitivity, and specificity, demonstrating its viability for cognitive assessment in drone training. By leveraging PSD features and bidirectional processing, we capture the nuanced brain activity patterns associated with different skill levels. This work contributes to the growing field of neuroergonomics, where physiological data inform human-machine interactions. As drone training evolves, our approach offers a scalable solution for enhancing evaluation methods and developing adaptive training systems. We envision a future where drone training is not only about technical skills but also about optimizing cognitive engagement for safer and more efficient operations.