SPRADP7A February   2025  – March 2025 AM62A3 , AM62A3-Q1 , AM62A7 , AM62A7-Q1 , AM67A , TDA4AEN-Q1

 

  1.   1
  2.   Abstract
  3.   Trademarks
  4. 1Introduction
  5. 2Building Blocks of an RGB-IR Vision Pipeline
    1. 2.1 CSI Receiver
    2. 2.2 Image Signal Processor
    3. 2.3 Video Processing Unit
    4. 2.4 TI Deep Learning Acceleration
    5. 2.5 GStreamer and TIOVX Frameworks
  6. 3Performance Considerations and Benchmarking Tools
  7. 4Reference Design
    1. 4.1 Camera Module
    2. 4.2 Sensor Driver
    3. 4.3 CSI-2 Rx Driver
    4. 4.4 Image Processing
    5. 4.5 Deep Learning for Driver and Occupancy Monitoring
    6. 4.6 Reference Code and Applications
  8. 5Application Examples and Benchmarking
    1. 5.1 Application 1: Single-stream Capture and Visualization with GST
    2. 5.2 Application 2: Dual-stream Capture and Visualization with GST and TIOVX Frameworks
    3. 5.3 Application 3: Representative OMS-DMS + Video Telephony Pipeline in GStreamer
  9. 6Summary
  10. 7References
  11. 8Revision History

CSI Receiver

The camera serial interface (CSI) receiver (Rx) on the AM62A complies with MIPI CSI v1.2 and supports up to 16 virtual channels. This receiver contains a DMA wrapper that transfers the captured image data to memory via DMA. This wrapper creates multiple DMA contexts, each dedicated to storing data from one virtual channel to the memory.

The CSI Rx driver in the Processor SDK Linux (or EdgeAI SDK) for AM62A [2] is compliant to the V4L2 framework. The driver creates a V4L2 video device node for each DMA context which is used to capture image data from each virtual channel and store to the memory. User-space applications can then access the captured image data via the video node associated with each virtual channel.

An RGB-IR sensor typically uses a 4x4 mosaic pattern containing both color and IR pixels. An external IR illuminator can be used to provide adequate IR lighting and can be periodically turned on and off, allowing the sensor to capture either color (RGB) dominant or IR dominant images. The sensor can be synchronized with the illuminator to send RGB dominant images through one virtual channel when the illuminator is off and IR dominant images through another channel when the illuminator is on. To handle these images separately, the CSI Rx driver creates two video device nodes: one for receiving RGB data (when the IR illuminator is off) and another for receiving IR data (when the IR illuminator is on).