Industrial smart security camera application using Renesas RZ/V2MA

Introduction to RZ/V2MA

Industrial environments require robust and intelligent security solutions to ensure the safety, efficiency, and security of operations. Renesas RZ/V2MA, with its advanced DRP-AI technology, offers a powerful solution for industrial smart security cameras, addressing a variety of challenges unique to these settings. Key functionalities include detecting unauthorized access, ensuring continuous monitoring, dynamic threat assessment, monitoring equipment and personnel activities, and implementing rapid response systems. This technology also provides advanced features for environmental monitoring and asset protection.

Comprehensive Monitoring
Deploy multiple RZ/V2MA-powered cameras across critical areas in industrial facilities to capture real-time footage. Utilize AI and machine learning algorithms to conduct in-depth analysis, detect intrusions, and monitor operational compliance, ensuring a safe and secure environment.

Anomaly Detection and Response
Process video streams to identify abnormal activities such as unauthorized access, equipment malfunctions, or safety violations. Enhance anomaly detection by integrating data from multiple cameras and automate alerts to security personnel for immediate action, minimizing potential risks and downtime.

Operational Efficiency
Leverage Edge AI capabilities of the RZ/V2MA to optimize operations by monitoring equipment and personnel activities. Detect inefficiencies, track operational metrics, and provide actionable insights to improve productivity and safety protocols within the industrial setting.

Environmental Monitoring
Monitor environmental conditions within the facility, such as temperature, humidity, and air quality. Use AI to analyze data and detect deviations from set parameters, triggering alerts and ensuring a safe working environment for employees and safeguarding sensitive equipment.

Asset Protection
Enhance asset security by employing advanced AI-driven surveillance to monitor high-value or critical assets. Implement tracking mechanisms to detect unauthorized movements or tampering, ensuring the protection of valuable resources and reducing the risk of theft or damage.

Data Integration and Reporting
Generate detailed analytics and reports from processed video data, providing valuable insights for facility management and security teams. Utilize comprehensive data analysis to optimize security protocols, improve operational efficiency, and plan for future security enhancements.

Redundancy and System Resilience
Ensure system reliability and resilience by processing data from multiple cameras to provide redundancy. In the event of a camera failure or obstruction, maintain continuous monitoring by relying on data from other cameras, ensuring uninterrupted surveillance and security operations.

By implementing industrial smart security cameras using RZ/V2MA,  facilities can achieve enhanced security, operational efficiency, and environmental monitoring. This advanced solution not only addresses the unique challenges of industrial environments but also provides a scalable and resilient approach to modern security management.

Why RZ/V2MA?

Features and Functionality

AI Acceleration

• Feature: Equipped with DRP-AI providing 1.0 TOPS/W class power performance and an OpenCV accelerator.
• Functionality: Executes deep learning algorithms and high-performance image processing with minimal latency, enhancing tasks like object detection and activity monitoring.

High-Speed Interfaces

• Feature: Includes Gigabit Ethernet, USB3.1 Gen1, PCIe® Gen 2, SDIO 3.0, and eMMC™ 4.5.1.
• Functionality: Ensures seamless connectivity and rapid data transfer, supporting real-time video streaming and efficient system communication.

Video and Graphics Processing

• Feature: Supports H.265/H.264 encoding up to 2160p for H.265 and 1080p for H.264.
• Functionality: Provides high-quality video processing for clear and detailed footage, essential for accurate monitoring and analysis.

Memory and CPU

• Feature: Dual Cortex®-A53 CPUs up to 996 MHz and 32-bit LPDDR4-3200 memory interface.
• Functionality: Handles intensive computational tasks and large data volumes efficiently, ensuring smooth operation and quick response times.

Compact Package

• Feature: FCBGA package with a 15×15 mm size and 0.5-mm pitch.
• Functionality: Suitable for space-constrained applications, allowing flexible deployment in various industrial setups.

Why is DRP-AI a game changer in RZ/V2MA?

The advantages of DRP-AI can be viewed from the hardware and software perspective.

Hardware perspective 

The DRP-AI hardware primarily features:

• DRP (Dynamically Reconfigurable Processor): Programable hardware
• AI-MAC (multiply-and-accumulate): Hardware dedicated for MAC computing
• DMAC (Direct Memory Access Controller)

Figure 1: DRP-AI architecture

The DRP is optimized for applications like Image preprocessing and pooling applications by dynamically changing the hardware configuration, while the AI-MAC is proficient in convolution application.

Software perspective

The DRP-AI translator automatically allocates each process of the AI model to the AI-MAC and DRP, thus allowing the user to easily use

DRP-AI without being aware of the hardware.

The primary functions of DRP-AI translator includes:

• Generating DRP-AI optimized executables from trained models based on the ONNX format, which is independent of various AI frameworks.

• Scheduling of each operation to process the AI model

• Hiding the overhead such as memory access time that occurs during the transition of each operation in the schedule
  
• Optimization of graph structure in the network (Layer fusion, DRP & AI-MAC processing allocation) to minimize memory access and improve computing efficiency. 

Figure 2： AI model implementation flow by DRP-AI

How DRP-AI attains a high power efficiency?

Reduction of external memory communication volume by data reuse technique: 

In traditional matrix operations, such as those performed on GPUs, data is often expanded and replicated multiple times, leading to increased memory access and power consumption. DRP-AI addresses this by reusing data within its architecture.

Here’s how it works:

Data Loading: Data is initially loaded from external memory into the DRP-AI buffer.

Data Transfer and Reuse: Instead of reloading data for each operation, DRP-AI transfers the data to internal registers and reuses it for multiple calculations within the AI-MAC unit.

Shifting: After each operation, the data is shifted within the register, enabling further reuse without accessing external memory again.

This data reuse technique significantly reduces the number of memory accesses required for matrix operations.

For a convolution operation on a (N X N) image matrix, while a conventional GPU, in the preprocessing step, each pixel is used N² times, hence increasing the amount of processed data N² times. This causes an increase in power consumption and communication bandwidth. On the other hand, AI-MAC can reuse the data by shifting the data taken into the register corresponding to the MAC arithmetic unit to the adjacent register. In addition, AI-MAC can reuse data not only for input data, but also for output and weight information, reducing access to external memory by more than one order of magnitude.

 Figure 3： Data Reuse Diagram

Low power control using inputted zero data:

One of the characteristics in AI model computation is the high ratio of “zero” values in the weight data and input/output data of each layer (it is called sparsification).

In DRP-AI, unnecessary computational power is reduced by introducing a switching technology that detects in advance when zero is entered in the input for each element of the operation and

prevents unnecessary operations.

Scheduling of operation flow:

Optimization of the order and timing of operations such as external data access or MAC operation processing, etc. is essential for efficient AI execution. Scheduling can avoid unnecessary waiting time and power generation between each process. Since the DRP-AI translator

automatically generates this optimized scheduling, the user can easily handle the DRP-AI.

In the white paper, titled “Embedded AI-Accelerator DRP-AI”, by Hideaki Abe et al, it has been observed that for the same machine learning task, while a conventional GPU was functioning at around 80⁰ celsius, the DRP-AI accelerated board managed to efficiently keep the temperature around 41⁰ Centigrade while delivering the same performance at an even better frame rate.

Sounds too good to be true, right? Read on for real-world insights and test results 

LiveBench Test Results

What is LiveBench?

The LiveBench environment provided by TenXer Labs, lets the user evaluate real hardware over the cloud.
Users can run different tests on the real hardware, live, from anywhere and get a thorough understanding. LiveBench also provides detailed documentation and technical know-how to accelerate IC adoption for the system engineers. 

LiveBench Capabilities

LiveBench specializes on the following:

• Remote access to real hardware in real time
• Instrument (Regulated power supply, Electronic load, DSOs, measuring instruments etc.) interfacing with cloud
• Sensors and actuators integration
• Live data acquisition and plotting
• Data analytics 
• Technical documentation and bring up know-hows 
• Hosting softwares and IDEs
• User space and data management

And many more…

LiveBench has been used to perform a 360° evaluation of the RZ/V2MA SoM, which includes testing for:

• Running live Edge AI algorithms evaluation on board
• Electrical, thermal and system profiling
• Benchmarking

The following results are accumulated from the LiveBench evaluation.

Low power and current consumption

The board demonstrated exceptional power efficiency during inference, regardless of the algorithm. When tested with YOLOv3 and ResNet50, the total input current ranged from 0.2 to 0.5 amps, with power consumption consistently below 2.5 watts. making it a highly power efficient SoM.

     Figure 4: Power profiling graph from LiveBench evaluation

Voltage Stability and Regulation

When connected to a 5-volt supply, the board maintains a stable 5-volt input terminal voltage after boot-up. During inference, it exhibits a maximum voltage regulation of less than 1%, regardless of the algorithms and input types. Given the maximum allowable voltage regulation of 5%, the voltage stability of this board is exceptional.

Figure 5: Voltage regulation graph from LiveBench evaluation

Thermal profile

This aspect was of particular interest. During 10-minute runs of various algorithms (YOLOv3, ResNet50) on the LiveBench, thermal profiling conducted with an MLX 90640 sensor showed the temperature never exceeded 45°C. This performance is noteworthy for any Edge-AI application.

 Figure 6: Thermal profile from LiveBench evaluation

Memory Consumption

During the tests, the application always manages to maintain a below 10% memory consumption showing an efficient memory management on the application part. 

Figure 7: App memory consumption from LiveBench evaluation

Inference Statistics

While running the application on live hardware, the key inference parameters (e.g., frame rate, inference time, post-processing time, etc.) were recorded and summarized. The LiveBench platform provided the facilities for these detailed analyses and many more. The results are tabulated below**:

**The parameter values may vary over time and from test to test. 

 Figure 8: Edge AI app

Power Cycle Test of RZ/V2MA

Power cycle tests will assess the board’s initial transient conditions, voltage stability, and boot-up current draw. The results indicate that both the voltage profile and initial inrush current are well within safe limits. During boot-up, a minor voltage increase to 5.0072 volts was observed, resulting in a voltage regulation of -0.14%, which is also within a safe range. An initial transient current of approximately 0.2 amps was recorded, leading to a boot time power consumption of less than one watt.

Figure 9: Power cycle test

CLI Benchmarking in RZ/V2MA

CLI benchmarking on the LiveBench has been utilized to evaluate the board’s capabilities. Users can run various programs, including mathematical operations and file handling, to assess the board’s raw computation power. The board has demonstrated strong performance in this area. For instance, it completed 1000 addition operations in just 0.000746 seconds. In file system operations, it took approximately 3.2 seconds to create and delete 10,000 files.

                             Figure 10: CLI Benchmarking results

Conclusion – RZ/V2MA SMARC SoM Promise

The RZ/V2MA SMARC SoM shows promise as a foundation for advanced industrial security camera applications. The test results from TenXer Labs’ LiveBench indicate potential for enhanced object detection, anomaly identification, and real-time response through AI integration. While further testing under rigorous industrial conditions is warranted, the SoM demonstrates a solid starting point for developing intelligent industrial surveillance solutions.

Reference

White Paper: Embedded AI-Accelerator DRP-AI

Authors: Hideaki Abe, Koichi Nose, Kazutaka Kikuchi

Link: https://www.renesas.com/us/en/document/whp/embedded-ai-accelerator-drp-ai?language=en

Fig 1, Fig 2 and Fig 3: Image source: White Paper : Embedded AI-Accelerator DRP-AI