Published on Apr 02, 2024
In this paper, we describe a human-computer interface (HCI) system that includes an enabler for controlling gadgets based on signal analysis of brain activities transmitted from the enabler to the gadgets. The enabler is insertable in a user’s ear and includes a recorder that records brain signals. A processing unit of the system, which is inserted in a gadget, commands the gadget based on decoding the recorded brain signals. The proposed device and system could facilitate a brain-machine interface to control the gadget from electroencephalography signals in the user’s brain.
HCI has been primarily implemented by monitoring direct manipulation of devices such as mice, keyboards, pens, touch surfaces, etc. However, as digital information becomes more integrated into everyday life, situations arise where it may be inconvenient to use hands to directly manipulate a gadget. For example, a driver might find it useful to interact with a vehicle navigation system without removing hands from the steering wheel. Further, a person in a meeting may wish to invisibly interact with a communication device. Accordingly, in the past few years there have been significant activities in the field of hands-free human-machine interface [1]. It is predicted that the future of HCI is moving toward compact and convenient hands-free devices.
Notably, in a recent report [2], IBM has predicted that at least in the next five years, mind-reading technologies for controlling gadgets would be available in the communication market. In the IBM report it is predicted that "if you just need to think about calling someone, it happens…or you can control the cursor on a computer screen just by thinking about where you want to move it." Accordingly, there is a need to make such enablers that could capture, analyze, process, and transfer the brain signals, and command a gadget based on the instructions that a user has in mind. This paper discusses an enabler that is insertable in a user’s ear to record an electroencephalography in the brain as brain signals while the user imagines various commands for controlling a gadget. The ear could provide a relatively inconspicuous location. Indeed, ear is known as a site where brain wave activity is detectable.
Certain areas of the ear, such as the area of the ear canal have proven to be better locations for detecting brain wave activity. Particularly, the area of the upper part of the ear, called the triangular fossa has high brain wave activity, especially near the skull. It is considered that the thinness of the skull at this area could facilitate higher reading of the brain wave activities. The proposed enabler of this paper could transmit, for example wirelessly, the brain signals to a processing unit inserted in the gadget. The processing unit decodes the received brain signals by a pattern recognition technique. Based on the decoded brain signals, the processing unit could control applications that are installed in the gadget. The details of the device and system that could facilitate such brain-machine interface are discussed in this paper. This paper addresses the current technologies in mind-reading systems, the deficiencies and limits of the existing technologies, along with possible solutions to have a practical device for brain-computer interaction, and the future plans to achieve such cutting-edge technology.
This paper discusses an enabler for controlling a gadget based on signal analysis of brain activities transmitted from the enabler to the gadget in a system, which could overcome the issues set forth above in the conventional devices. Therefore, it is an advantage of the system disclosed in this paper to provide an improved human-computer interface system, having many of the same capabilities as conventional input devices, but which is hands-free and does not require hand operated electromechanical controls, or microphone-based speech processing methods, and is easy to insert to provide comfort for a user of the enabler to enable easily controlling gadgets such as mobile phones, personal digital assistant devices, media players, etc. With the proposed enabler and system of this paper, these gadgets can be controlled without a need for an additional hardware, particularly without additional electrodes outside the enabler.
The enabler includes a recorder that is insertable in an outer ear area of the user. The recorder records electroencephalography signals generated in the brain. The recorded signals are transferred to a processing unit inserted in the gadget for converting the signals to command applications in the gadget. The proposed system is illustrated in Fig. 1, in which signals derived from the user’s ear are used for decoding the brain activities to enable mental controlling of a gadget. As shown in the figure, in the proposed system, an HCI enabler is inserted in the ear of the user. The enabler uses electroencephalography recordings from the canal of the external ear to obtain brain activities in a way that is used as a brain-computer interface using signals of complex cognitive.
A recorder that is inserted in the enabler records the brain signals. The recorder has an electrode that is located at the entrance of the ear, and could be mounted with an earplug. Signals can be amplified and digitized for transmitting from the enabler. The enabler wirelessly transmits the recorded brain signals to the processing unit that includes a decoder. A transmitting device installed in the enabler produces a radio frequency signal corresponding to voltages sensed by the recorder and transmits the radio frequency signal by radio frequency telemetry through a transmitting antenna. The transmitting device could include the transmitting antenna, a transmitter, an amplifying device, a controller, and a power supply unit, such as a battery. The amplifying device could include an input amplifier and a bandpass filter. The amplifying device receives an electrode signal from the recorder.
The electrode signal is a response to changes in the brain electrical activities of the user. The input amplifier could provide an initial gain to the electrode signal, and the bandpass filter could provide an additional gain to the electrode signal resulting in an output signal with an overall gain of much higher than the electrode signal. The controller is electrically connected to the bandpass filter. The output signal from the bandpass filter is inputted to the controller. The controller provides signal conditioning to the output signal to provide telemetry transmission. Such signal conditioning includes analog to digital conversion. The controller also controls the transmitter channel frequency thereby controlling the frequency of the radio frequency signal to be transmitted.
A receiving device in the processing unit, through a receiving antenna, receives the radio frequency signal generated and transmitted by the transmitting device. The receiving device produces a data output corresponding to the received radio frequency signal. The receiving device could include any radio frequency receiving means with a plurality of channels. A desired channel is selected via processor control of the frequency of an oscillator. The receiving device could employ a frequency shift keyed demodulation format. The receiving could also include a microcontroller incorporated to program the oscillator. The receiving device outputs data and error correction bits to the microcontroller which removes error correction bits and outputs corrected data as the data output to an operator interface. The data output corresponds to the radio frequency signal received by the receiving device. The receiving device outputs the data output to the operator interface. The operator interface could include software which allows automatic synchronization of the stimulus with the data output.
The decoder decodes the data output using a common algorithm such as pattern classifier. By evaluating frequencies in a wide range from theta to gamma brain signals recorded by the recorder, complex cognitive signals are decodable and are used for controlling the gadget. The processing unit converts the decoded signals to command signals for running an application inserted in the gadget. The above-mentioned pattern classifier can utilize conventional algorithms that apply classifier-directed pattern recognition techniques to identify and measure specific changes in each input signal and derive an index of the relative strength of the change [3]. In one method, a rule-based hierarchical database structure, describes the relevant features within each signal and a weighting function for each feature.
A self-learning heuristic algorithm governs the use and reweighting criteria for each feature, maintains the database of the feature indexes, and regulates feedback from a Feedback Control Interface. The output vectors are sent through cascades of classifiers, which select the most appropriate combination of the features necessary to generate a control signal to match an application in the gadget. Calibration, training, and feedback adjustment could be performed at the classifier stage prior to characterization of the control signal to match the control interface requirements. In sum, the proposed method to implement the enabler of this paper could include receiving a signal indicative of a mental activity of a user, decoding the signal, using pattern recognition to identify and quantify a change in the signal, classifying the signal according to a response index to obtain a classified signal, comparing the classified signal to data in a response cache to identify a response that corresponds to the classified signal, and delivering a command to the gadget to implement the response.
Other pattern recognizing algorithms such as wavelet, Hilbert, Fourier or other transformation can also be applied to single trails of the electroencephalograph from the recorder to perform pattern recognition. After a program for executing the above method is loaded into a memory of the processing unit, the program enables the processing unit to carry out the method for controlling the gadget, in which the processing unit coordinates with a recording electrode, and the detector for detecting signals of an electroencephalogram is placed in the canal of the outer ear of the user. Accordingly, the brain activities are decoded from the signals of the electroencephalogram and the gadget is controlled based on the results of the decoding.
In conclusion, based on the discussion presented in this paper, it appears that the trend in the future of the HCI devices and systems is moving toward providing systems and devices that could efficiently convey the brain signals to command gadgets, while a user is thinking about commanding the gadgets. Recently, among researchers in industry and academia, several attempts have been made to enhance the brain-reading interface technologies. However, as set forth in this paper, each of these devices and systems suffers from deficiencies that refrains the field from achieving maturity. Indeed, it appears that much more research is needed to achieve to a point of commercializing these systems and devices that could be affordable and comfortable for the users.
One of the major obstacles in this journey appears to be in the area of pattern recognition of the mind signals, considering the limited understanding of a user’s brain and its electrical activities, since the accuracy of a mind signal detection could be degraded as the number of mind states increases, such as when the user thinks about a series of words to implement a task. In this paper a system was proposed that includes an enabler for controlling gadgets based on signal analysis of brain activities transmitted from the enabler to the gadget. The enabler could be inserted in the user’s ear and includes a recorder that records brain signals. A processing unit of the system commands the device based on decoding the recorded brain signals. The proposed enabler could provide a compact, convenient, and hands-free device to facilitate a brain-machine interface to control the gadget from electroencephalography signals in the user’s brain
1] Tian, J., Ahmaniemi, T., Boda, P., Vetek, K., Apparatus, method and computer program product providing a hierarchical approach to command-control tasks using a brain-computer interface, U.S. Patent No. 8,005,766, August 23, 2011.
2] Molina, G., Nelson G., Bruekers, A., Damstra, M., Weda, J., Relating to computer brain interfaces, U.S. Patent Application Publication No. 2011/0238685, September 29, 2011.
3] Liao, X., Yao, D., Li, C. 2007. Transductive SVM for reducing the training effort in BCI. Journal of Neural Engineering. 4 (Dec. 2007), 246-254.
4] Tan, D., Saponas, T., Morris, D., Turner, J., Wearable electromyography-based for human-computer interface, U.S. Patent Application Publication No. 2009/0326406, December 31, 2009.
| Are you interested in this topic.Then mail to us immediately to get the full report.
email :- contactv2@gmail.com |
