Specialized Hardware Needed For AI To Develop In Gadgets

Computers are now able to beat humans at the Go game, identify melanomas as accurately as dermatologists do, and help autonomous vehicles navigate the world, and this is all thanks to an artificial intelligence technique called deep learning.

Now, circuit designers are working on hardware they hope will lead to the democratization of deep learning, bringing the powerful method to the chips inside smart phones, wearables, and other consumer electronics.

Mobile phones, for example, will do a better job of understanding our individual accents and linguistic quirks. (This will save many of us from being constantly upset with a daft digital assistant. Right,Siri?) And home security systems will respond to the sound of a burglar breaking a window, but know not to alert the police when someone is clumsily emptying the dishwasher.

To that end, at the recent IEEE International Solid-State Circuits Conference (ISSCC) in San Francisco, academic and industry engineers showed how they have built on work presented at last year’s conference to produce specialized, energy efficient deep-learning processors.

This dedicated hardware will give electronic devices a new level of smarts because, unlike traditional software, it relies on high-level abstraction like the human brain. What’s more, it won’t drain the gadgets’ batteries.

Mahesh Mehendale, TI Fellow at Texas Instruments in Bangalore, commented: “We’re beginning to see that there is a need to develop more specialized hardware to get both performance and energy efficiency.”

He co-chaired the conference session with Takashi Hashimoto, chief engineer in the technology development laboratory at Panasonic.

The first step to widespread adoption of deep learning is for companies to start marketing dedicated, low-power chips. For that reason, said Mehendale, the session’s entry from STMicroelectronics is significant. Like many projects of this sort, the company’s chip uses an architecture that brings memory and processing closer together.

Compared to other algorithms, neural networks require frequent fetching of data; shortening the distance this data has to travel saves energy. Guiseppi Desoli, a researcher at STM’s Cornaredo, Italy, outpost, presented a neural network processor that can perform 2.9 trillion operations per second (teraOPS) p/W.

STMicroelectronics’ processor is designed to run algorithms called convolutional neural networks, which are used for image recognition. During his presentation, Desoli said the company believes neural networks can be incorporated into the Internet of Things – if designers can get power use down.

He said: “A normal battery will only last a few hours,” when powering a deep-learning processor that can perform only a few teraOPS p/W.

Hoi-Jun Yoo’s group at the Korea Advanced Institute of Science and Technology (KAIST) in Daejeon, described a chip that pulls off a different feat. Not only is it more energy efficient than the one from STMicroelectronics (performing 8.1teraOPS p/W), it can also run two kinds of neural networks.

One, a convolutional neural network, is best for image recognition, because such networks excel at understanding information in photographs and other static images. The other, a recurrent neural network, can grapple with a sequence of information, because it remembers the previous input.

Such networks are used for tasks like decoding speech; whether you’re listening or talking, you have to remember what was said a few seconds ago for a conversation to make sense.

Yoo’s group demonstrated a second deep-learning processor paired with an image sensor. The resulting gadget: a low-power, wearable badge that recognizes faces. This device relies on a specialized architecture that runs a convolutional neural network at 620µWs. That trickle of power is small enough for a coin cell battery to keep it running for more than 10 hours. In one demo, the KAIST system labeled photos with ‘Julia Roberts’ and ‘pizza.’ It can also spot Matt Damon, should the wearer ever come across him in person.

Another issue engineers delved into at ISSC was systems-level design. One way to save energy is to use low-power circuits to make initial decisions, then, when necessary, wake up relatively more power-hungry neural networks to do the heavy lifting.

Anantha Chandrakasan’s lab at MIT presented a chip that uses a circuit to distinguish speech from other sounds. This circuit gates a neural network that can then recognize words. The MIT chip can perform tasks requiring a vocabulary of up to 145,000 words. It makes about one-fourth the number of word errors on a standardized test compared with the previous system, while using about just one-third of the power of its predecessor.

The new chips presented last week, said Mehendale, show that customized hardware is more efficient for running neural networks. Training neural networks is another matter.

Today, it must still be done on powerful computers. In coming years, perhaps next year, researchers will develop dedicated hardware for the deep learning training process, which will make that energy intensive process more efficient, said Panasonic’s Hashimoto.

More information: IEEE Spectrum