This demo showcases real-time Human Pose Estimation, based on the Open Pose library, ported onto the camera platform, and designed by Rhonda’s Activity Recognition neural network for human behavior recognition. The two Deep Learning Neural Networks (DNN), along with the video pipeline, run on the Rhonda Software CV22 System on a Module (CV22 SoM).

CV22 SoM – designed in-house as a low-power camera platform, is capable of running multiple neural networks, in addition to providing superior image quality. The core of the SoM platform is an Ambarella CV22 System on a Chip – an ARM-based Image Signal Processor with a DNN inference acceleration engine, implemented on a single crystal.

Both CV applications run simultaneously. The Pose Estimation network performs human body detection in a full 4K frame, and people’s figures recognized in the camera’s field of view are visualized with “skeleton-like” pose markups. A blob of pixels around a foreground skeleton selected within the region of interest is passed to the Activity recognition DNN.

The activity recognition algorithm is a simple, yet robust demo built by Rhonda’s CV team from scratch, and trained to identify several activity types: walking, standing, welcome hand gestures (high-five), jumping jacks, body-weight squats. Recognized Activity for the foreground body is displayed in the upper- left corner of the screen.

After the initial port onto the CV22 platform, Open Pose algorithm delivered a frame rate of 1 frame per second. It took a number of optimization procedures performed by Rhonda’s CV experts (such as pruning, quantization, and dedicated retraining) to achieve a fifteen fold acceleration in performance.

The system can be trained for different use cases, such as security, elderly care, production automation, sports activity analysis, and more.

For demo and testing purposes we’ve deployed a setup with HDMI video injection to show platforms’ recognition capabilities with additional activities.

As a road-safety application example, Rhonda Software has assembled the Pedestrian detection demo, based on the same optimized port of the Open Pose library. The algorithm is applied to automotive conditions to detect pedestrians as participants of road traffic.