Bulletin of Electrical Engineering and Informatics

Yasir M. Abdal, Mohammed G. Ayoub, Mazin N. Farhan, Hasan A. Abdulla Department of Computer Technology Engineering, Technical Engineering College, Northern Technical University, Mosul, Iraq Department of Medical Instrumentation Technology Engineering, Technical Engineering College, Northern Technical University, Mosul, Iraq Department of Mechanical Power Technology Engineering, Technical Engineering College, Northern Technical University, Mosul, Iraq


INTRODUCTION
Technology advancement could play a key role for helping people in their daily life especially for those who are suffering from movement disorders, which occurs either genetically or through accidents. Such disorders cause partial or complete paralysis of the human body [1], [2]. For example, spinal cord injury prevents the communication from the brain to all areas of the human body. In such medical condition, the brain is correctly functional but the nerves responsible of movements are non-functioning [2], [3]. Biomedical signals such as electroencephalogram (EEG) and electromyography (EMG) have been used widely nowadays for human machine interaction (HMI), human computer interaction (HCI), and smart rehabilitation system [1], [4]- [7]. EEG is a non-invasive approach that is used to record brain activities as electrical signals using multiple electrodes on the scalp [8]. These signals can be used in various themes such as keyboard and mouse control, robot arm control and wheelchairs. Artificial intelligent (AI) has also been used for developing such systems such as convolutional neural network (CNN), long short-term memory (LSTM) and support vector machines (SVMs) [9]- [12]. Researchers have utilized AI in developing smart wheelchair systems such as voice-controlled wheelchairs, gesture-controlled wheelchair and eye and voicecontrolled system. According to the World Health Organization (WHO), there are seventy-five million disabled people need wheelchairs on their daily basis, which represents around 1% of the world's population [13]. Therefore, developing such systems could be the appropriate way to serve the life of those people. In recent years, there has been considerable interest in developing various kinds of smart wheelchairs using recent technologies such as cameras, sensors and wearable EEG headbands. These wheelchairs serve disabled people on their daily basis in different situations [14]- [16]. However, some points should be considered during developing smart wheelchairs in order to make them appropriate with the need of the wide range of disabled people. For example, some studies have tended to design wheelchairs that focus on the movement rather than the interaction with their environment. Different techniques have been investigated in developing and implementation such systems that serve disabled people. However, few researchers have addressed the issue of the cost while developing and implementing wheelchairs that based on EEG signal because of the high cost of the medical equipment. Numerous studies used joystick controllers to drive the wheelchairs easily such as visual joystick and electric joystick to assist people with different disabilities [17]- [19]. However, such technologies are not suitable for people with complete paralysis such as quadriplegia condition, which affects all four limbs.
In [9], [16] developed a voice recognition-based robotic wheelchair using CNNs. Regarding this method, it has some limitations for example; background noise affects the user commands. Moreover, this kind of systems can be only functioned with partial or incomplete paralyzed person. According to Tamplin et al. [20] quadriplegic patients have impaired vocal ability in comparison with the able-bodied. Therefore, these methods should be replaced with advanced technologies that can be used with completely paralyzed people. This can be solved in a variety of ways, for example using EEG or EOG headbands which can be worked with the aforementioned condition easily [21]- [23].
Eye-blink detection is an approach that is used in wide range of applications such as human computer interaction, alternative communication systems (ACS) and wheelchairs controlling. This approach uses different components for example, webcam and biomedical signals including EEG and EOG. Chaudhary et al. [24] proposed an eye-blink based predictive text system used CNN and SVM for eye-blink detection in real-time using webcam. Li et al. [25] have also used such approach for controlling wheelchair. They acquired eye-blink from a single-channel EOG signal that is used as an on/off switch commands.
The main goal of this study is to develop an easy-to-use HMI system that is capable to assist fully paralyzed people in their daily life. The system is based on a low-cost single channel EEG headband and could be used in motorized wheelchair easily. The paper is organized as follows: section 2 illustrates the hardware and software of the developed HMI system, which used in this study. Section 3 addresses the experimental procedure as well as the obtained results. Section 4 discusses the results of the study while section 5 concludes this study. Finally, section 6 shows future work.

METHOD
In this part, the permanently components, programming and designing process of the HMI proposed system Figure 1 are presented in details. The developed EEG system can be used in multiple scenarios regarding the time. Two different layouts have been developed for the proposed system graphical user interface (GUI) (cross-shaped and horizontal bar); these will be discussed in details later.

Hardware structure
The key motivation of this system is to help fully/partially paralyzed people to move by using their brainwaves (attention and meditation levels). This is done by acquiring the subjects' EEG signal using MindWave NeuroSky device to control the wheelchair navigation through a GUI Figures 2(a) and (b). Attention and meditation levels of the brain are difficulty controlled to set at specific level, thus, eye-blinking is used in this system as a metric to control directions and ignitions of the motorized wheelchair. The proposed system is composed of MindWave NeuroSky, MindWave RF dongle, Arduino Mega, TFT LCD adapter shield, TFT 320QDT_9341 and motorized wheelchair. MindWave NeuroSky is a device consists of eight parts; ear-clip, power switch, battery, ear-arm, flexible headband, sensor tip, sensor arm and ThinkGear chip. The principle of work of this device is based on two sensors (forehead sensor and the ear-clip ground reference) that detect and filter EEG power spectrum (α, β, δ, θ, and γ) from the frontal lobe of brain [23], [26]. These values (i.e., α, β, δ, θ, and γ) are analyzed through the ThinkGear chip to produce attention/meditation and blink levels. Then, these levels are wirelessly sent to the microcontroller board (Arduino Mega) as commands in short range via Bluetooth using MindWave radio frequency (RF) dongle. Thin-film-transistor-liquid-crystal-display (TFT LCD) adapter is attached to Arduino Mega board, so the TFT 320QDT_9341 display can easily be installed to the microcontroller board. Finally, the system could be wired easily to the motorized wheelchair controls so the subject can use and control it by his/her own brainwaves. Table 1 represents the approximate price of the used hardware in the proposed system that can be used with disabled people for different applications including wheelchairs.

GUI design
The smart wheelchair has a software part as well as the hardware in which the system GUI is represented. The aim of designing this GUI is to allow the user to interact with the system easily and comfortably. The average values of blinks have been acquired from the EEG headset raw data, which is sampled at the rate of 512 Hz for the duration of one second. The blink value depends on a specified threshold in the developed algorithm, which is more than the normal blink of human in order to distinguish between the user movement/stop commands and the ordinary eye blinks. The value will appear on the top of the GUI with a rectangular indicator that display a single blink or double blinks. A single blink represents movement/stop commands while the double blinks represent a stop without the need of choosing it from the GUI, which required more time for selection. The double blinks can be used for emergency stopping which is requires less time. The selected command will appear in a text box on the GUI, so the user can change it quickly in case of missing out the selection.

EXPERIMENTS AND RESULTS
A pilot study with five subjects was carried out to validate the proposed system and good results are achieved. The study includes four different eye-blinking based experiments with set of trials to evaluate the system performance. Five able-bodied subjects with the following rates agreed to participate in these experiments; (age: 34±4 years; height: 178±7.9 cm; body mass: 72.7±5.7 kg). The next section presents these experiments and their results. Figure 3 shows the proposed and implemented GUI design with two different scenarios (Figures 3(a) and (b)) in order to figure out the right one for the final design.

Experiment-1
In this experiment, the first designing scenario shown in Figure 3(a) was applied to the display. This scenario has five buttons (cross-shaped) referring to forward, backward, left, right and stop commands. In each trial, the subject was asked to select a specific button on the screen using eye-blinking. Four trials (30 seconds each) were conducted to evaluate the system scanning speed and its capability to offer selections in specific time. Table 2 shows the capability of each subject to select a specific direction in each trial.

Experiment-2
In this experiment, the second designing scenario which shown in Figure 3(b) was applied to the display. This scenario has five buttons (horizontal bar) referring to forward, backward, left, right, and stop. The experimental protocol is as same as experiment-1; in each trial, the subject was asked to select a specific button on the screen using eye-blinking. Table 3 shows the capability of each subject to select a specific direction in each trial.

Experiment-3
In this experiment, the first designing scenario shown in Figure 3(a) was applied to the display again. The experimental protocol includes four trials in which the subject was asked to follow a specific path (directions) on the ground using a motorized wheelchair (its speed 1 m/s). The display with scenario 1 is attached to this wheelchair. The path shown in Figure 4 is the one that subjects should follow. The length of the path is 21 meters, in which the subject expected to select button F=six times, L=twice, S=zero times, R=three times, B=zero times and double blink (D.B)=six times to complete the path. The ideal consumed time (when no selection mistakes are committed) for this scenario (cross-shaped) equal to 60 seconds. Subjects may commit to types of mistakes when using this system; missing specific selection and select wrong direction. Every missed selection will add five seconds (1-cycle scanning time) to the total consumed time. Whilst selecting a wrong button (different direction) will require additional time for multi selections until the direction is corrected. Table 4 shows the time consumed by each subject to complete the path in four trials. The period is theoretically calculated according to (1) which is based on the system scanning and selecting capabilities.

Experiment-4
In this experiment, the second designing scenario shown in Figure 3(b) was applied to the display again. The experimental protocol also includes four trials in which the subject was asked to follow a specific path (directions) on the ground using a motorized wheelchair (its speed 1 m/s). The display with scenario 2 is attached to this wheelchair. The path shown in Figure 4 is the same path that subjects should follow. The length of the path is 21 meters, in which the subject expected to select button B=zero times, F=six times, S=five times, R=three times, L=twice and double blink (D.B)=once only to complete the path. The ideal consumed time (when no selection mistakes are committed) for this scenario (horizontal bar) equal to 84 seconds. Subjects may commit to types of mistakes when using this system; missing specific selection and select wrong direction. Every missed selection will add five seconds (1-cycle scanning time) to the total consumed time. Whilst selecting a wrong button (different direction) will require additional time for multi selections until the direction is corrected. The period is theoretically calculated according to (2) which is based on the system scanning and selecting capabilities. Table 5

DISCUSSION
According to Table 2, results show that the average number of button selection falls between 3.8 and 4.6 while the mean standard deviation equals to 0.646. Trial 3 (for left button) shows the highest average number of selections (4.6 times/30 sec) whilst trail 2 (backward button) and trial 4 (right button) show the lowest average number (3.8 times/30 sec). Furthermore, the maximum recorded no. of selection in all trials is  Table 3 shows that that the average number of button selection falls between 3.6 and 4.2 while the mean standard deviation equals to 0.415. Trial 2 (backward button) shows the highest average number of selections (4.2 times/30 sec) whilst trial 4 (right button) show the lowest average number (3.6 times/30 sec). The mean standard deviation of second designing scenario conducted in experiment 2 is lower than the first scenario (done in experiment 1), thus, the second scenario considered more reliable than the first one in the process of button selecting through a scanning session.
In validating the system through navigation, Table 4 shows that the subjects average consumed time was between (72.5-86.25) sec to complete the set path using the first scenario (cross shape) which is approximately close to the theoretical calculation (60 sec/path). Whilst, Table 5 shows a dramatic increase in the consumed time for subjects using the second scenario (horizontal bar) which falls between (101.25-110) sec to complete the set path which is relatively high in comparison to the theoretical calculation (84 sec/path). Consequently, using the first scenario (cross-shaped) in navigating the wheelchair consuming less time than using the second scenario (horizontal bar) for the same path. Therefore, the first scenario is recommended for navigation but it needs training sessions for subjects to use it easily.

CONCLUSION
This paper proposed a human-machine interaction for motorized wheelchair based on singlechannel EEG headband. The system develops a simple HMI to assist fully/partially paralyzed people to move freely in their daily life using a motorized wheelchair. The proposed system was successfully validated through sets of practical experiments and two designed scenarios were applied. Five normal subjects were involved in the validation process and four trials were accomplished for each experiment. Four experiments were done to validate the designed scenarios by calculating the mean standard deviation and the average consumed time for each one of them. Choosing the proper scenario was based on the user's awareness capability in selecting on screen buttons and the system scanning time. The horizontal bar scenario considered the easier one in buttons selection process whilst the cross-shaped was the best one in the navigation. The system is cost effective and easy to use; however, a user training session is essential.

FUTURE WORK
The implemented system can be equipped with a global positioning system (GPS) technology. Adding this component has several benefits to the developed HMI system. The first benefit is the patient safety which means the wheelchair can be easily monitored remotely while another one is the navigation. In this case, patients can create their own path for movement. Sensors such as ultrasound and accelerometer could also be added to the system in order to increase the reliability. These components collect data and let patients to interact with their environment. Devices from the University of Strathclyde, Glasgow, UK in 2014. He has studied and worked with medical Instrumentation and have had the chance to get an in depth understanding of medical devices and their environment. He is looking to use his engineering skills in improving the current state of medical devices. His top research interest topics are medical image processing, biosignal processing and medical sensors. He can be contacted at email mazin.nadheer@ntu.edu.iq.

Hasan A. Abdulla
is an academic assistant lecturer of Computer Engineering at Northern Technical University, Mosul, Iraq. He received his bachelor's degree in Computer Technology Engineering in 2007 and master's degree in Computer Technology Engineering in 2021 from Technical Engineering College of Mosul. His current research interests include CNC software control system, deep learning, image processing, artificial intelligent, brain computer interface (BCI), EEG signal processing and machine learning technique. He can be contacted at email: hasan.alsarraf@ntu.edu.iq.