Artificial Intelligence and Machine Learning for Sensor-based Embedded Systems

With microcontrollers now integral components of most technical devices from washing machines to wearables, the need for machine learning for microcontrollers and embedded systems that’s platform-independent and not cloud-based has grown.

Software solutions currently exist for machine learning, but they’re primarily available for the PC and Python-based. There is still no solution that makes it possible to execute and train neural networks on embedded systems such as microcontrollers.

Conducting training directly in the embedded system, for example when an implanted sensor is to calibrate itself, means creating sensor-related AI that can be directly integrated in a sensor system.

Recently, a team of researchers at Fraunhofer IMS has created Artificial Intelligence for Embedded Systems (AIfES), a machine learning library programmed in C that can run on microcontrollers as well as other platforms: PCs, Raspberry PI and Android. The library currently contains a completely configurable artificial neural network (ANN), which can also generate deep networks for deep learning when necessary. An ANN is an attempt to mathematically simulate the human brain using algorithms in order to make functional contexts learnable for the algorithms. AIfES has been optimized specifically for embedded systems.

AIfES is a platform-independent machine learning library that can be used to realize self-learning microelectronics requiring no connection to a cloud or to high-performance computers. The sensor-related AI system recognizes handwriting and gestures, enabling gesture control of input when the library is running on a wearable.

“We’ve reduced the source code to a minimum, which means the ANN can be trained directly on the microcontroller or the sensor, i.e. the embedded system. In addition the source code is universally valid and can be compiled for almost any platform. Because the same algorithms are always used, an ANN generated for example on a PC can easily be ported to a microcontroller. Until now this has been impossible in this form with commercially available software solutions,” says Dr. Pierre Gembaczka, research associate at Fraunhofer IMS.

Another uniquely qualifying feature of the sensor-related AI from Fraunhofer IMS: until now artificial intelligence and neural networks have been used primarily for image processing and speech recognition, sometimes with the data leaving the local systems. For example, voice profiles are processed in the cloud on external servers, since the computing power of the local system is not always adequate. “It’s difficult to protect privacy in this process, and enormous amounts of data are transmitted. That’s why we’ve chosen a different approach and are turning away from machine learning processes in the cloud in favor of machine learning directly in the embedded system. Since no sensitive data leave the system, data protection can be guaranteed and the amounts of data to be transferred are significantly reduced,” says Burkhard Heidemann, “Embedded Systems” group manager at Fraunhofer IMS. “Of course it’s not possible to implement giant deep learning models on an embedded system, so we’re increasing our efforts toward making an elegant feature extraction to reduce input signals.” By embedding the AI directly in the microcontroller, the researchers make it possible to equip a device with additional functions without the need for expensive hardware modifications.

AIfES doesn’t focus on processing large amounts of data, instead transferring only the data needed to build very small neural networks. “We’re not following the trend toward processing big data; we’re sticking with the absolutely necessary data and are creating a kind of micro-intelligence in the embedded system that can resolve the task in question. We develop new feature extractions and new data pre-processing strategies for each problem so that we can realize the smallest possible ANN. This enables subsequent learning on the controller itself,” Gembaczka explains.

The approach has already been put into practice in the form of several demonstrators. The research team implemented the recognition of handwritten numbers on an inexpensive 8-bit microcontroller (Arduino Uno). This was made technically possible by developing an innovative feature extraction method. Another demonstrator is capable of recognizing complex gestures made in the air. Here the IMS scientists have developed a system consisting of a microcontroller and an absolute orientation sensor that recognizes numbers written in the air. “One possible application here would be operation of a wearable,” the researchers point out. “In order for this type of communication to work, various persons write the numbers one through nine several times. The neural network receives this training data, learns from it and in the next step identifies the numbers independently. And almost any figure can be trained, not only numbers.“ This eliminates the need to control the device using speech recognition: the wearable can be controlled with gestures and the user’s privacy remains protected.

There are practically no limits to the potential applications of AIfES: For example, a wristband with integrated gesture recognition could be used to control indoor lighting. And not only can AIfES recognize gestures, it can also monitor how well the gestures have been made. Exercises and movements in physical therapy and fitness can be evaluated without the need for a coach or therapist. Privacy is maintained since no camera or cloud is used. AIfES can be used in a variety of fields such as automotive, medicine, Smart Home and Industrie 4.0.

The AIfES library makes it possible to decentralize computing power for example by allowing small embedded systems to receive data before processing and pass on the results to a superordinate system. This dramatically reduces the amount of data to be transferred. In addition, it’s possible to implement a network of small learning-capable systems which distribute tasks among themselves.

AIfES currently contains a neural network with a feedforward structure that also supports deep neural networks. “We programmed our solution so that we can describe a complete network with one single function,” says Gembaczka. The integration of additional network forms and structures is currently in development. Furthermore the researcher and his colleagues are developing hardware components for neural networks in addition to other learning algorithms and demonstrators. Fraunhofer IMS is currently working on a RISC-V microprocessor which will have a hardware accelerator specifically for neural networks. A special version of AIfES is being optimized for this hardware in order to optimally exploit the resource.

The research team will present an energy-independent current sensor in integrated AI at the Sensor & Test fair in Nuremberg from June 25 to 27, 2019.

Fraunhofer Institute for Microelectronic Circuits and Systems IMSFraunhofer Institute for

Source:  Fraunhofer-Gesellschaft