Rohan Sagar
SFSU - English 670
November 20, 2017
Literature Review (Final Draft)
Instructor: Sofia de Almeida
The evolution of myoelectric control for use in robotic prosthetics.
Abstract:
Myo-electric control is one of the technologies that has the potential to change the way we interact with machines. Research on myo-electric control first began in the 1940s. However, due to the limitations in computational power, significant progress wasn’t made until the 1970’s. In the present day, the research on myoelectric control has seen advancement to a point, where researchers use techniques such as machine learning through classification and complex data acquisition algorithms. By doing so, they were able to elevate the standards of myoelectric control so that they can be used in applications such as advanced robotic prosthetics, that can mimic the function of a human hand with high accuracy and less latency. The following article describes the evolution of the research in myoelectric control since the early 1940s and gives an account of all the major breakthroughs in the field. It gives a brief introduction to the research and moves on to explain various aspects of the research such as market review, connection between myoelectric data and neural drive, methodology, data acquisition and implementation.
Introduction:
It is a bitter truth that accidents keep happening every day and some of them even lead to the victims losing their limbs. Hand amputations in particular, can drastically affect the victim’s capabilities. There is a wide variety of prosthetics available in the market for victims with such disabilities. However, most of the victims prefer a cosmetic alternative over a functional robotic prosthetic. This is because the robotic prosthetics that are available in the market today, are either too expensive, or too complicated to use (Atzori & Muller, 2015). In order to make them inexpensive and intuitive to the user, a technique called Myo-electric Control can be used.
Electromyography (EMG) is a technique used for evaluating and recording the electrical activity of skeletal muscles. Whenever a muscle is contracted, electrical activity occurs in the muscle, which can then be recorded by using a device known as electromyograph. Myo-Electric control utilizes electromyography to extract the data from muscles and uses it to control the desired object.
The researchers in this field have distinguished opinions on how the prosthetics can be made better. While some believe that the key to advancement lies in increasing the amount of data collected by using multiple degrees of freedom (Atzori & Muller, 2015), others say that embedding electrodes inside the body that are connected directly to the nervous system, can allow the system to extract data that contains accurate information about the intended action. In addition to that they can also be used to induce an electrical impulse that allows the user to have the sense of touch (Farina, et al., 2014).
There is a considerable number of researchers that agree with Farina, et al., (2014). However, they also say that such implants would need a surgery that can be expensive and uneasy. Hence, they base their argument of myoelectric control being the future of prosthetics, on an assumption that cost and ease of use are some of the desired necessities of robotic prosthetics. This could be the reason why they are working towards building robust and accurate robotic prosthetics that harness the capabilities of myo-electric control. (Englehart & Hudgins, 2003; Jiang, Dosen, Muller & Farina, 2012; Zhang, Huang & Yang, 2012; Donovan, et al., 2016).
The research on myoelectric control is branched into several sub fields. Mentioned below are a subset of them: algorithms for data acquisition and processing, hardware implementations of the system in multiple configurations, in an attempt to find the best hardware architecture that is both small in size and powerful enough to handle the demanding computational needs of machine learning, the scope of cloud computing in myoelectric control.
The following article goes into detail about the research that is being done in all the sub-fields mentioned above.
Market Review:
Before we dive deep into the research it is important for us to understand the need for it. The following section talks about the need for the robotic prosthetics and their current market status.
In 2005 about 41,000 victims were found with loss of an upper limb, therefore there exists an increasing demand for robotic prosthetics (Ziegler-Graham et al., 2008; Atzori & Muller, 2015). To capitalize this demand, numerous companies have come up with commercial robotic prosthetics. Atzori and Muller (2015) mentioned that most of the prosthetics available in the market today, are based on myoelectric control. They harness the power of electromyography to extract electrical impulses from the user’s arm/leg which it converts to movement. However, according to the authors, they still have imperfections and are subject to errors. In addition to that they are expensive, which make these prosthetics approachable only to a small customer base.
Another problem with commercial prosthetics that exist today is that their research methodology, data and techniques are not accessible to academic researchers, as they are usually considered as the intellectual property of the firm. The reason behind this is that companies spend millions of dollars for developing a technology so that they can make money by using it in their products. Occasionally they even fund the research that is being done in educational institutions so that they can get a fresh look on their research. Making their research methodology and data available to academic researchers exposes them to potential intellectual property theft, therefore, they are usually protective about it and even go as far as getting patents for their technology.
Connection between myoelectric data and neural drive:
Neural drive is the way in which the human brain communicates with the rest of the body. Our main goal while obtaining myoelectric data is to extract neural information by using it. According to Farina, et al. (2014), it would be easy to get neural data with the help of an implant that is connected to the spinal cord. However, the use of implants is not practical in the current day as the cost involved in the surgery in high. The authors found that the myoelectric data collected from skeletal muscles has critical information regarding the neural drive. By extracting this data, we can map it to the action intended thus being able to replicate it. This served as a foundation to myoelectric control. Farina et al. (2014) proposed mathematical expressions to explain the relationship between the myoelectric data and the neural drive. They also give brief information about a few novel views in academic myoelectric control such as pattern recognition, EMG data mapping, extraction of neural code from surface EMG and multi-nodal approach.
Methodology:
Jiang, Dosen, Muller, & Farina (2012) explain the methodology of myoelectric control in a way that it can be used by new researches to understand the basics of myoelectric control. Jiang et al., explain the methodology as follows.
First the myoelectric data is obtained by using EMG sensors and the data is stored. This data collected from the arm/leg of a person, should be processed and analyzed so that the neural information can be extracted and used by the prosthetic for control purposes. This is done by sending the collected data to a program called as a classifier. The classifier is a machine learning algorithm that needs to be trained with data and the results corresponding to it. After the training stage is completed it can be given a new set of data to process and analyze. The classifier processes the new data based on the data acquired during the training phase and predicts an outcome that is used by the robotic prosthetic control unit to trigger actuation.
Data Acquisition:
Various algorithms have been developed to acquire the data efficiently. The sliding window algorithm as illustrated by Englehart and Hudgins (2003) takes a small window of myoelectric data at once and then performs calculations on it before going to next set of data. The following figure illustrates the sliding window technique.
Fig 1: Sliding Window algorithm
In this figure W1, W2, W3 are the data windows that are being analyzed by the classifier. T is the processing delay (The time taken by the classifier to process the data acquired by the window and give a result). D1, D2 and D3 are the decisions given by the classifier. The window length as illustrated in the figure us 256 milliseconds. This is because 300 milliseconds, is considered as an acceptable delay when it comes to robotic prosthetics. Since the classifier takes a short amount of time to process the data after 256 milliseconds of data acquisition it adds up to around 300 milliseconds.
According to Englehart and Hudgins (2003), this technique yields good accuracy and reduces latency. However, it is not efficient in terms of processor utilization. It only uses the processor of the device that it runs on, for a small amount of time when the data that has been collected is being processed, which means that the processor is idle while the data is being collected which is not efficient. Therefore, this technique was modified by Englehart and Hudgins to yield better efficiency. The figure below illustrates the modified version of the sliding window technique.
Fig 2: Sliding Window Technique with Overlapping Windows
In the figure R1, R2, R3 are the windows, T is the time taken for processing and D1, D2, D3 are the results of the classifier.
Instead of using one window at a time multiple windows of data are collected and analyzed synchronously to get a larger number of decisions and thus increase the efficiency of the algorithm. However, the research in this field has advanced to a point where it uses accelerometer and gyroscope readings to complement the myoelectric data, thus creating a need for new and advanced algorithms for data acquisition.
Implementation:
Implementing a myoelectric control system poses many challenges, including, but not limited to, latency and efficient processor utilization. Researchers at San Francisco State University have been implementing myoelectric control for various applications, such as robotic prosthetics and virtual reality (Zhang, Huang & Yang, 2012; Zhang, et.al., 2012; Donovan, et al., 2016)
They implemented a myoelectrically controlled robotic leg in 2012 using a system that comprises of a microcontroller and an FPGA (Field Programmable Gate Array). The microcontroller system uses its ADC’s (Analog to Digital Converters) to get the analog data from the electromyographic (EMG) sensors and convert it to digital data. The digital data is sent to an FPGA which does all the processing and pattern recognition using an SPI (Serial Peripheral Interface) bus. They presented the data pertaining to the system performance by providing the delays for systems using 7 EMG sensors and 12 EMG sensors which are 0.23 and 0.25ms respectively. (Zhang, Huang & Yang, 2012)
In 2016, they used an open-source platform for myoelectric data extraction called as myo band to build a 3D printed, myo-electrically controlled, robotic arm and virtual reality programs for psychological therapy for amputees. The myo band is a commercially available product that is equipped with 6 myoelectric sensors, accelerometer and a gyroscope. It can be used for a wide variety of applications. It extracts the myoelectric, accelerometer and gyroscopic data from the arm of the person that is wearing it and is equipped with communication capabilities so that it can send that data to a computer for further use (Donovan, et al., 2016).
Researchers at ICE (Intelligent computing and embedded systems) lab in San Francisco state university harnessed this feature of the myo-band and made a robotic arm that can replicate the movement of a natural human hand upon mental command. By using the data that they obtained from the myo band, they made a virtual reality program that simulates a virtual arm which can be used for psychological therapy for victims with limb amputations.
The scope of cloud computing in myoelectric control:
Currently the research is at a point where it can be used to make cheap, accurate and efficient robotic prosthetics (Atzori & Muller, 2015). However, since technology keeps evolving every day, there is always scope for improvement. The recent invention of cloud computing helps take the computing out of the physical device itself and sends it to the cloud via the internet to process. Researchers are trying to integrate cloud computing into myoelectrically controlled robotic prosthetics so that they can make it smaller yet more powerful than the current versions. However, if the internet link is slow, it can cause latency issues thus current day research is being done on how to integrate the systems with cloud computing with minimum latency (Zhang, et al., 2017).
The research on myoelectric control has led to considerable improvements in robotic prosthetics. Today robotic prosthetics that are implemented using this research serve thousands of satisfied amputees. However, some researchers believe that there is still scope for improvement (Mitra, & Acharya, 2007; Atzori and Muller 2015). The robotic prosthetics that are currently available in the market are equipped with a wide variety of sensors, but they lack sensorial feedback mechanisms. As a result, actuation information can be extracted from the arm of the user but touch information cannot be fed back. Farina, et al. (2014) suggest that this feature can be enabled by the use of electrodes that are inserted into the user’s arm. However, using electrodes makes it difficult for the user to remove the prosthetic arm for when it is needed. Using a wireless implant may make it easier for the user to detach the prosthetic but a surgery will still be needed.
Since the research in this field was mostly geared towards data analysis by machine learning and myo-electric control enhancement, less progress was made in finding alternative methods for delivering sensory feedback to the user. Perhaps it is time for the researchers to look into techniques such as using haptic engines (mechanisms that use vibrations to provide feedback) similar to the touch pad that apple invented, for use in the Mac Book Pro 2017 to give the feel of a click without actually having to mechanically click, in robotic prosthetics. Such mechanisms hold the key to the future of sensory feedback in robotic prosthetics.
References:
Englehart, K., & Hudgins, B. (2003). A robust, real-time control scheme for multifunction myoelectric control. IEEE transactions on biomedical engineering, 50(7), 848-854.
Atzori, M., & Müller, H. (2015). Control capabilities of myoelectric robotic prostheses by hand amputees: a scientific research and market overview. Frontiers in systems neuroscience, 9.
Zhang, X., Huang, H., & Yang, Q. (2012, June). Implementing an FPGA system for real-time intent recognition for prosthetic legs. In Design Automation Conference (DAC), 2012 49th ACM/EDAC/IEEE (pp. 169-175). IEEE.
Donovan, I., Valenzuela, K., Ortiz, A., Dusheyko, S., Jiang, H., Okada, K., & Zhang, X. (2016, October). MyoHMI: A low-cost and flexible platform for developing real-time human machine interface for myoelectric controlled applications. In Systems, Man, and Cybernetics (SMC), 2016 IEEE International Conference on (pp. 004495-004500). IEEE.
Oskoei, M. A., & Hu, H. (2007). Myoelectric control systems—A survey. Biomedical Signal Processing and Control, 2(4), 275-294.
Jiang, N., Dosen, S., Muller, K. R., & Farina, D. (2012). Myoelectric control of artificial limbs—is there a need to change focus?[In the spotlight]. IEEE Signal Processing Magazine, 29(5), 152-150.
Mitra, S., & Acharya, T. (2007). Gesture recognition: A survey. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 37(3), 311-324.
Farina, D., Jiang, N., Rehbaum, H., Holobar, A., Graimann, B., Dietl, H., & Aszmann, O. C. (2014). The extraction of neural information from the surface EMG for the control of upper-limb prostheses: emerging avenues and challenges. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 22(4), 797-809.
Zhang, X., Liu, Y., Zhang, F., Ren, J., Sun, Y. L., Yang, Q., & Huang, H. (2012). On design and implementation of neural-machine interface for artificial legs. IEEE transactions on industrial informatics, 8(2), 418-429.
