AI chips for connected camera technologies

At the CES 2021 (Consumer Electronics Show) digital exhibition, Ambarella unveiled CV5, an artificial intelligence (AI) vision processor capable of recording 8 000 video or four 4 000 video streams. It comes with two CAN FD interfaces.

The system on chip (SoC) enables the development of automotive camera systems, consumer cameras (drone, action, and 360°), and robotic cameras. It combines the company’s CVflow AI engine with dual Arm A76 CPUs (central processing unit) to provide the performance necessary for a range of AI-based algorithms. Its image signal processor (ISP) can simultaneously process images for both human viewing and machine processing. Fabricated in 5-Nm process technology, the chip consumes under 2 W of power while encoding full 8 000 video at 30 frames per second. As peripheral interface, the product supports besides others two CAN FD interfaces.

In automotive video telematics applications, the chip provides the performance necessary to encode multiple video streams from front ADAS (advanced driver assistance systems), driver monitoring, cabin monitoring, and side-view cameras. Its CVflow AI engine can run ADAS algorithms such as lane departure, and forward collision warning as well as driver monitoring algorithms such as drowsy driver detection. The combination of high-resolution video capture with AI processing enables ADAS cameras to recognize images over distances and with accuracy.

The power consumption of the SoC also makes it a solution for the next generation of high resolution and high frame-rate action, 360°, and virtual reality (VR) cameras, offering 8 000 recording and playback in small form factor designs, said the company.

In robotic and drone applications, CV5’s CVflow AI engine can accelerate localization and mapping (Slam), path planning, and obstacle detection, and avoidance algorithms for navigation and autonomous operation. For drone-based aerial videography and cinematography, the product can perform flight control and navigation functions while recording up to 8 000 resolution video recording at 60 frames per second.

The SoC shares common SDK (software development kit) and computer vision (CV) tools with the company’s other CVflow SoC families. A set of CV tools supports customers port their own neural networks onto CV5, including a compiler, debugger, and support for industry-standard machine learning frameworks, such as Pytorch, ONNX, Caffe, and Tensorflow, as well as guidelines for convolutional neural network (CNN) performance optimization.

“With the introduction of CV5, Ambarella is defining the next generation of automotive, consumer, and robotic cameras,” said Fermi Wang, CEO of Ambarella. “By combining 8 000 single-channel and 4 000 multi-channel recording with the high performance of our CVflow AI engine, we are enabling cameras with the highest-quality imaging and innovative new AI features.”

cw