In recent years, the rapid expansion of the drone industry has highlighted the need for effective training and evaluation methods for operators. Traditional approaches to assessing drone training often rely on behavioral metrics or process indicators, which fail to capture the cognitive aspects of learning. As a drone training evolves, it becomes crucial to incorporate psychophysiological signals, such as electroencephalogram (EEG), to understand mental workload and skill acquisition. This study explores the potential of using EEG signals combined with a deep learning model—specifically, a Bidirectional Long Short-Term Memory (Bi-LSTM) network—to classify operator proficiency levels in drone training. The primary goal is to develop a robust method for evaluating drone training effectiveness, which can adapt to individual skill levels and enhance future adaptive systems.
The global market for drones has seen substantial growth, with applications ranging from recreational to commercial sectors. However, standardized regulations and training protocols are still under development. Current drone training assessment methods often focus on performance outcomes, such as task completion time or error rates, but they overlook the cognitive dynamics during operation. EEG, as a non-invasive tool, provides insights into brain activity and has been used in various domains to measure cognitive load, stress, and learning efficiency. For instance, in online education or gaming environments, EEG has helped quantify mental states. In the context of drone training, assessing cognitive factors can offer a more comprehensive evaluation of operator competence.
Previous research has employed machine learning techniques for EEG classification, including Artificial Neural Networks (ANNs) and recurrent models like LSTM. However, these studies have largely focused on military drone operations or other high-stakes scenarios, which may not generalize to civilian drone training. Civilian drone training involves a broader range of operators with varying skill levels, making it essential to develop tailored approaches. The Bi-LSTM model, which processes sequential data in both forward and backward directions, is well-suited for capturing temporal dependencies in EEG signals. This study aims to bridge the gap by applying Bi-LSTM to classify drone training levels based on EEG-derived features, thereby contributing to the field of intelligent training evaluation.
The methodology of this study encompasses several stages: EEG data acquisition, preprocessing, feature extraction, and model development. We utilized a 32-channel EEG headset with a sampling rate of 128 Hz, following the international 10-20 system for electrode placement. Participants engaged in a drone simulation task using a DJI Flight Simulator, which replicates real-world flying conditions. The task required navigating through rings in a tunnel, with performance scored based on accuracy and errors. This setup ensures safety while providing a realistic drone training environment. To classify operators, we derived a performance metric that combines rewards (e.g., rings passed) and penalties (e.g., missed rings or crashes), categorizing individuals into three levels: basic, intermediate, and advanced.
EEG signals were preprocessed to remove artifacts and noise. We applied high-pass and low-pass filtering, along with notch filtering to eliminate powerline interference. Independent Component Analysis (ICA) was used to separate neural sources from ocular and muscular artifacts. Subsequently, the power spectral density (PSD) was extracted as a feature, representing the distribution of signal power across frequency bands such as delta, theta, alpha, beta, and gamma. PSD features were computed using Welch’s method, providing a compact representation of brain activity during drone training. These features served as input to the Bi-LSTM model for classification.
The Bi-LSTM network architecture consists of multiple layers designed to handle sequential EEG data. As shown in the model structure table, it includes bidirectional LSTM layers, batch normalization, dropout layers, and a dense output layer with softmax activation. The LSTM unit employs gates—forget, input, and output—to manage information flow, with equations defining their operations. The forget gate determines which information to discard, using a sigmoid function: $$f_t = \sigma(W_{xf} x_t + W_{hf} h_{t-1} + b_f)$$. The input gate updates the cell state: $$i_t = \sigma(W_{xi} x_t + W_{hi} h_{t-1} + b_i)$$ and $$\tilde{c}_t = \tanh(W_{xc} x_t + W_{hc} h_{t-1} + b_c)$$. The output gate controls the hidden state: $$o_t = \sigma(W_{xo} x_t + W_{ho} h_{t-1} + b_o)$$. The cell state and hidden state are computed as: $$c_t = f_t \odot c_{t-1} + i_t \odot \tilde{c}_t$$ and $$h_t = o_t \odot \tanh(c_t)$$, where \(\odot\) denotes element-wise multiplication. In the bidirectional setup, hidden states from forward and backward passes are concatenated: $$h_t = \sigma(W_h [\overrightarrow{h_t}, \overleftarrow{h_t}] + b_h)$$. This allows the model to capture contextual information from both past and future time steps, enhancing its ability to classify drone training levels.
For experimental validation, we recruited 11 participants who performed at least 10 trials each in the simulator. Their performance data was used to assign ability levels based on the derived metric. The table below summarizes the statistics and ratings for each participant, illustrating the distribution across basic, intermediate, and advanced categories. This data underpins the classification task, emphasizing the relevance of cognitive metrics in drone training.
| Participant ID | Average Rings Passed | Average Score | Total Missed Rings | Total Crashes | Performance Score | Ability Level |
|---|---|---|---|---|---|---|
| 01 | 5.00 | 7.33 | 1 | 2 | 3.35 | Basic |
| 02 | 6.00 | 14.33 | 1 | 1 | 8.50 | Advanced |
| 03 | 4.00 | 3.88 | 1 | 1 | 3.96 | Basic |
| 04 | 5.78 | 8.33 | 2 | 2 | 3.27 | Basic |
| 05 | 7.13 | 11.38 | 1 | 2 | 4.94 | Intermediate |
| 06 | 4.00 | 8.00 | 1 | 2 | 3.06 | Basic |
| 07 | 6.38 | 8.39 | 2 | 1 | 5.37 | Intermediate |
| 08 | 2.44 | 2.56 | 1 | 1 | 2.48 | Basic |
| 09 | 22.46 | 75.69 | 5 | 3 | 10.68 | Advanced |
| 10 | 11.38 | 26.63 | 4 | 3 | 4.83 | Intermediate |
| 11 | 7.38 | 16.50 | 3 | 2 | 4.40 | Intermediate |
The drone training simulation involved a challenging tunnel navigation task, where operators controlled a virtual drone to pass through rings. The visual below provides a glimpse into the experimental setup, highlighting the immersive nature of the simulator used for drone training. This environment allows for safe and repeatable assessments, crucial for developing cognitive evaluation methods.
After preprocessing, the PSD features were split into training, validation, and test sets for a three-fold cross-validation procedure. The Bi-LSTM model was trained over multiple epochs, with monitoring of accuracy metrics. The training curves demonstrated rapid convergence, with validation accuracy stabilizing around 80-90% across folds. This indicates that the model effectively learned patterns from EEG data associated with different drone training levels. To evaluate performance, we computed a confusion matrix and normalized it to assess classification precision. The normalized confusion matrix revealed high accuracy for each class: 0.95 for advanced, 0.99 for basic, and 0.96 for intermediate, with low error rates. These results underscore the model’s capability in distinguishing proficiency levels in drone training.
We compared the Bi-LSTM model against other neural network architectures, including ANN, LSTM, and Gated Recurrent Unit (GRU). The table below presents the average performance across three folds, showcasing the superiority of Bi-LSTM in terms of validation accuracy, test accuracy, sensitivity, and specificity. With a test accuracy of 98.21%, sensitivity of 0.90, and specificity of 0.92, Bi-LSTM outperformed the alternatives, highlighting its robustness for EEG-based classification in drone training contexts.
| Model | Validation Accuracy (%) | Test Accuracy (%) | Sensitivity | Specificity |
|---|---|---|---|---|
| ANN | 82.06 | 81.97 | 0.99 | 0.64 |
| LSTM | 84.67 | 93.30 | 0.97 | 0.79 |
| GRU | 82.67 | 94.69 | 0.98 | 0.86 |
| Bi-LSTM | 87.67 | 98.21 | 0.90 | 0.92 |
The model’s architecture is detailed in the following table, which outlines each layer’s specifications. This deep network leverages bidirectional processing to capture temporal dependencies in EEG sequences, making it ideal for analyzing brain activity during drone training tasks. The inclusion of dropout and batch normalization layers helps prevent overfitting and accelerates training, ensuring generalizability across different operators.
| Layer Number | Name | Description |
|---|---|---|
| 0 | Input Layer | (32, 1) |
| 1 | Bidirectional LSTM Layer | (32, 256) |
| 2 | Batch Normalization Layer | (32, 256), ε=10⁻⁶ |
| 3 | Dropout Layer | (32, 256), dropout rate=0.1 |
| 4 | LSTM Layer | (32, 64) |
| 5 | Batch Normalization Layer | (32, 64), ε=10⁻⁶ |
| 6 | Dropout Layer | (32, 64), dropout rate=0.05 |
| 7 | LSTM Layer | (32, 32) |
| 8 | Batch Normalization Layer | (32, 32), ε=10⁻⁶ |
| 9 | Dropout Layer | (32, 32), dropout rate=0.01 |
| 10 | Dense Layer | (None, 3), activation=’softmax’ |
The results affirm that EEG signals, when processed with advanced deep learning models like Bi-LSTM, can effectively classify operator skill levels in drone training. This approach offers a novel method for assessing training outcomes, moving beyond traditional metrics to incorporate cognitive dimensions. In practical terms, such systems could be integrated into drone training platforms to provide real-time feedback, adapting scenarios based on the operator’s proficiency. For instance, beginners might receive simplified tasks, while advanced users face more complex challenges, optimizing the learning curve in drone training programs.
However, this study has limitations, primarily the small sample size of 11 participants. Future work should involve larger and more diverse cohorts to validate the model’s generalizability and robustness. Additionally, exploring other EEG features or multimodal data (e.g., eye-tracking) could enhance classification accuracy. The integration of real-time adaptive systems in drone training holds promise for personalized education, potentially reducing accidents and improving operational efficiency. As drone technology continues to evolve, so too must the methods for training and evaluation, ensuring that operators are cognitively prepared for real-world demands.
In conclusion, this research demonstrates the feasibility of using Bi-LSTM networks to classify drone training levels based on EEG data. The model achieved high accuracy, sensitivity, and specificity, outperforming existing approaches. This contributes to the field of intelligent training assessment, offering a tool that can be deployed in civilian drone training programs to monitor cognitive progress. By leveraging psychophysiological signals, we can create more effective and adaptive training environments, ultimately enhancing safety and performance in the drone industry. Future studies should focus on scaling the method and integrating it into live training systems, paving the way for smarter drone training solutions.