SPRADA8 may   2023 AM68A , TDA4VL-Q1

 

  1.   1
  2.   Abstract
  3.   Trademarks
  4. 1Introduction
  5. 2AM68A Processor
  6. 3Edge AI Use Cases on AM68A
    1. 3.1 AI Box
    2. 3.2 Machine Vision
    3. 3.3 Multi-Camera AI
  7. 4Software Tools and Support
    1. 4.1 Edge AI Software Development Kit (SDK)
    2. 4.2 Edge AI SDK Demonstrations
    3. 4.3 Edge AI Model Zoo
    4. 4.4 Edge AI Studio
  8. 5Conclusion
  9. 6Reference

Multi-Camera AI

Lots of existing and emerging markets for edge AI technology and edge AI processors fall into the multi-camera AI category. Multi-camera AI is similar to AI Box, but multiple cameras are directly connected to the system through MIPI CSI-2 in multi-camera AI while encoded bitstreams from remote cameras are streamed into the system through Ethernet in AI Box. Multi-camera AI use cases include the following:

  • Smart shopping cart is an emerging end equipment for enhanced shopping experiences. Edge analytics on 2 to 6+ multiple cameras mounted on the shopping cart automatically detects the items placed in the cart, read barcodes on them, calculate order totals, and allow consumers to pay for groceries, bypassing long checkout lines. With the localization of a shopping cart, personalized shopping experience can be provided by identifying the locations of items on the shopping list and recommending new products to customers.
  • Functional safe 3D perception provides advance assistance to machine operators for preventing collision and protecting the workers and pedestrians around. Multiple cameras are mounted around vehicles and mobile machines such as the ones used for construction, agriculture and mining and AI based 3D perception with functional safety is enabled. When combined with localization and navigation, the technology is enabling mobile machines and cobots to operate fully autonomously yet co-exist alongside humans and properties increasing the task efficiency.
  • In smart farming, the multi-camera AI systems enable 24 hours per day, 7 days per week cattle identification, behavior monitoring and video data analysis to improve farming operations, discover health and feeding patterns, and assess how farming practices impact livestock. The camera system delivers daily event notifications to farmers through their phones, while also providing remote access to detailed analytics about their herd and farm operations, which is helpful for farmers to turn visual information into actionable insights and make data-driven decisions to maximize productivity and profitability.
  • In smart agriculture, camera mounted on tractor, robots, and drones maximizes the efficiency in planting, fertilizing, and harvesting of the crops with the help of edge analytics.
GUID-20230503-SS0I-QXH5-DMJJ-4JXTDJSCXQJB-low.svgFigure 3-3 Multi-Camera AI Block Diagram With Data Flow on AM68A

The data flow for a multi-camera AI use case example on AM68A is depicted in Figure 3-3. In this example, six 2MP cameras are combined using two MIPI CSI-2 aggregators and captured at 30 fps through two MIPI CSI-2 RX ports, which is the main difference from AI Box. The images through de-mosaic and lens distortion correction is scaled to smaller resolution by MSC. The outputs obtained through DL preprocessing, DL network on MMA and DL post-processing are encoded by the HW accelerated H.264, H.265 encoder, and streamed out or saved in storage. Table 3-3 shows the resource utilization and estimated power consumption for the multi-camera AI use case with six and four channels of 2MP camera.

Table 3-3 AM68A Resource Utilization and Power Consumption for the Multi-Camera AI Use Case
Main IPUtilization (6 × 2MP at 30 fps)Utilization (4 × 2MP at 30 fps)
2 × CSI-2 RX6 × 2MP at 30fps = 5.76 Gbps (29%)4 × 2MP at 30fps = 3.84 Gbps (19%)
VPAC (VISS, MSC, LDC)6 × 2MP at 30fps = 360 MP/s (60%)4 × 2MP at 30fps = 240 MP/s (40%)
MMA8 TOPS (100%)6 TOPS (75%)
Encoder6 × 2MP at 30 fps = 360 MP/s (75%)4 × 2MP at 30fps = 240 MP/s (50%)
2 × A72DL pre- and post-processing, and so forth (50%)DL pre- and post-processing, and so forth (40%)
DDR Bandwidth9.7 GBps (29%)7.2 GBps (21%)
Power Consumption (85°C)7.2 W6.5 W