SPRADH2 February   2024 AM62A3 , AM62A3-Q1 , AM62A7 , AM62A7-Q1 , AM62P , AM62P-Q1 , DS90UB953A-Q1 , DS90UB960-Q1 , TDES960 , TSER953

 

  1.   1
  2.   Abstract
  3.   Trademarks
  4. 1Introduction
  5. 2Connecting Multiple CSI-2 Cameras to the SoC
    1. 2.1 CSI-2 Aggregator Using SerDes
    2. 2.2 CSI-2 Aggregator without Using SerDes
  6. 3Enabling Multiple Cameras in Software
    1. 3.1 Camera Subsystem Software Architecture
    2. 3.2 Image Pipeline Software Architecture
  7. 4Reference Design
    1. 4.1 Supported Cameras
    2. 4.2 Setting up Four IMX219 Cameras
    3. 4.3 Configuring Cameras and CSI-2 RX Interface
    4. 4.4 Streaming from Four Cameras
      1. 4.4.1 Streaming Camera Data to Display
      2. 4.4.2 Streaming Camera Data through Ethernet
      3. 4.4.3 Storing Camera Data to Files
    5. 4.5 Multicamera Deep Learning Inference
      1. 4.5.1 Model Selection
      2. 4.5.2 Pipeline Setup
  8. 5Performance Analysis
  9. 6Summary
  10. 7References

1 Introduction

Embedded cameras play an important role in modern vision systems. Using multiple cameras in a system expands the capabilities of these systems and enables capabilities that are not possible with a single camera. Below are some examples of the applications using multiple embedded cameras:

Security Surveillance: Multiple cameras placed strategically provide comprehensive surveillance coverage. They enable panoramic views, reduce blind spots, and enhance the accuracy of object tracking and recognition, improving overall security measures.

Surround View: Multiple cameras are used to create a stereo vision setup, enabling three-dimensional information and the estimation of depth. This is crucial for tasks such as obstacle detection in autonomous vehicles, precise object manipulation in robotics, and enhanced realism of augmented reality experiences.

Cabin Recorder and Camera Mirror System: A car cabin recorder with multiple cameras can provide more coverage using a single processor. Similarly, a camera mirror system with two or more cameras can expand the driver’s field of view and eliminate blind spots from all sides of a car.

Medical Imaging: Multiple cameras can be used in medical imaging for tasks like surgical navigation, providing surgeons with multiple perspectives for enhanced precision. In endoscopy, multiple cameras enable a thorough examination of internal organs.

Drones and Aerial Imaging: Drones often come equipped with multiple cameras to capture high-resolution images or videos from different angles. This is useful in applications like aerial photography, agriculture monitoring, and land surveying.

With the advancement of microprocessors, multiple cameras can be integrated into a single System-on-Chip (SoC) to provide compact and efficient solutions. The AM62Ax SoC, with high-performance video/vision processing and deep learning acceleration, is an ideal device for the above-mentioned use cases. Another AM6x device, the AM62P, is built for high-performance embedded 3D display applications. Equipped with 3D graphics acceleration, the AM62P can easily stitch together the images from multiple cameras and produce a high-resolution panoramic view. The innovative features of the AM62A/AM62P SoC have been presented in various publications, such as [4], [5], [6], etc. This application note will not repeat those feature descriptions but instead focuses on integrating multiple CSI-2 cameras into embedded vision applications on AM62A/AM62P.

Table 1-1 shows the main differences between AM62A and AM62P as far as image processing is concerned.

Table 1-1 Differences Between AM62A and AM62P in Image Processing
SoC AM62A AM62P
Supported Camera Type With or without built-in ISP With Built-in ISP
Camera Output Data Raw/YUV/RGB YUV/RGB
ISP HWA Yes No
Deep Learning HWA Yes No
3-D Graphics HWA No Yes