Background: The accuracy of decoding fine motor imagery (MI) tasks remains relatively low due to the dense distribution of active areas in the cerebral cortex. Methods: To enhance the decoding of unilateral fine MI activity in the brain, a weight-optimized EEGNet model is introduced that recognizes six types of MI for the right upper limb, namely elbow flexion/extension, wrist pronation/supination and hand opening/grasping. The model is trained with augmented electroencephalography (EEG) data to learn deep features for MI classification. To address the sensitivity issue of the initial model weights to classification performance, a genetic algorithm (GA) is employed to determine the convolution kernel parameters for each layer of the EEGNet network, followed by optimization of the network weights through backpropagation. Results: The algorithm’s performance on the three joint classification is validated through experiment, achieving an average accuracy of 87.97%. The binary classification recognition rates for elbow joint, wrist joint, and hand joint are respectively 93.92%, 90.2%, and 94.64%. Thus, the product of the two-step accuracy value is obtained as the overall capability to distinguish the six types of MI, reaching an average accuracy of 81.74%. Compared to commonly used neural networks and traditional algorithms, the proposed method outperforms and significantly reduces the average error of different subjects. Conclusions: Overall, this algorithm effectively addresses the sensitivity of network parameters to initial weights, enhances algorithm robustness and improves the overall performance of MI task classification. Moreover, the method is applicable to other EEG classification tasks; for example, emotion and object recognition.