AI Inference Processor Architecture for Edge Devices with Co-processing

CEVA has announced NeuPro-S, its second-generation AI processor architecture for deep neural network inferencing at the edge. In conjunction with NeuPro-S, CEVA also introduced the CDNN-Invite API, a deep neural network compiler technology that supports heterogeneous co-processing of NeuPro-S cores together with custom neural network engines, in a unified neural network optimizing run-time firmware.

NeuPro-S, along with CDNN-Invite API, is ideal for any vision-based device with the need for edge AI processing, including autonomous cars, smartphones, surveillance cameras, consumer cameras and the emerging use cases in AR/VR headsets, robots and industrial applications.

Designed to optimally process neural networks for segmentation, detection and classification of objects within videos and images in edge devices, NeuPro-S includes system-aware enhancements that deliver significant performance improvements.

These include support for multi-level memory systems to reduce costly transfers with external SDRAM, multiple weight compression options, and heterogeneous scalability that enables various combinations of CEVA-XM6 vision DSPs, NeuPro-S cores and custom AI engines in a single, unified architecture. This enables NeuPro-S to achieve on average, 50% higher performance, 40% lower memory bandwidth and 30% lower power consumption than CEVA’s first-generation AI processor.

The NeuPro-S family includes NPS1000, NPS2000 and NPS4000, pre-configured processors with 1000, 2000 and 4000 8-bit MACs respectively per cycle. The NPS4000 offers the highest CNN performance per core with up to 12.5 Tera Operations Per Second (TOPS) @ 1.5GHz and is fully scalable to reach up to 100 TOPS.

Addressing the growing diversity of application-specific neural networks and processors, the CDNN-Invite API allows the seamless incorporation of customer’s designed neural network engines into CEVA’s award-winning Deep Neural Network (CDNN) framework.

CDNN will then holistically optimize and enhance networks and layers to take advantage of the performance excellence of each of the CEVA-XM6 vision DSP, NeuPro-S and custom neural network processors. The CDNN-Invite API is already being adopted by lead customers who are working closely with CEVA engineers to deploy it in commercial products.

Ilan Yona, Vice President and General Manager of the Vision Business Unit at CEVA, commented: “CEVA continues to set the pace in proliferating deep neural networks processors and optimization technologies for edge devices. The NeuPro-S architecture addresses the root causes of the growing challenges in data bandwidth and power consumption in these devices.

“With our CDNN-Invite API, we have further extended our indisputable competitive edge in neural network compiler technologies by reducing the entry barriers for the growing community of neural network innovators, allowing them to benefit from the breadth of support and ease of use our CDNN compiler technology offers.”

CEVA’s vast experience and holistic strategy to consolidate processing, acceleration, software and tools delivers an open, power-efficient and programmable platform for deep learning. The fully programmable CEVA-XM6 vision DSP incorporated in the NeuPro-S architecture facilitates simultaneous processing of imaging, computer vision and general DSP workloads in addition to AI runtime processing.

This also allows customers and algorithm developers to take advantage of CEVA’s extensive imaging and vision software and libraries, including the CEVA-SLAM software development kit for 3D mapping, CEVA-CV and CEVA-VX software libraries for computer vision development, and its recently acquired wide-angle imaging software suite including dewarp, video stitching and Data-in-Picture sensor fusion technology.

The NeuPro-S further builds upon CEVA’s success in automotive use cases by providing solutions that meet safety requirements including quality assurance standards IATF 16949 and automotive standards including ISO 26262 and A-Spice.

NeuPro-S is available today and has already been licensed to lead customers for automotive and consumer camera applications. CDNN-Invite API is available for lead customers today and for general licensing by the end of 2019.