SPRADP7A February   2025  – March 2025 AM62A3 , AM62A3-Q1 , AM62A7 , AM62A7-Q1 , AM67A , TDA4AEN-Q1

 

  1.   1
  2.   Abstract
  3.   Trademarks
  4. 1Introduction
  5. 2Building Blocks of an RGB-IR Vision Pipeline
    1. 2.1 CSI Receiver
    2. 2.2 Image Signal Processor
    3. 2.3 Video Processing Unit
    4. 2.4 TI Deep Learning Acceleration
    5. 2.5 GStreamer and TIOVX Frameworks
  6. 3Performance Considerations and Benchmarking Tools
  7. 4Reference Design
    1. 4.1 Camera Module
    2. 4.2 Sensor Driver
    3. 4.3 CSI-2 Rx Driver
    4. 4.4 Image Processing
    5. 4.5 Deep Learning for Driver and Occupancy Monitoring
    6. 4.6 Reference Code and Applications
  8. 5Application Examples and Benchmarking
    1. 5.1 Application 1: Single-stream Capture and Visualization with GST
    2. 5.2 Application 2: Dual-stream Capture and Visualization with GST and TIOVX Frameworks
    3. 5.3 Application 3: Representative OMS-DMS + Video Telephony Pipeline in GStreamer
  9. 6Summary
  10. 7References
  11. 8Revision History

1 Introduction

With the growing demand for image sensing in both visible and infrared light, RGB-IR sensors, which capture both RGB and IR images with a single camera, are becoming increasingly popular. Effectively processing and leveraging RGB-IR data is essential for various artificial intelligence applications, including driver monitoring and occupancy monitoring (also known as cabin monitoring), robotics, security surveillance, and smart home systems. The AM62A SoC is an excellent choice for building such intelligent systems, as detailed in [1].

This application note focuses on implementing a vision processing pipeline on the AM62A SoC, from an RGB-IR camera to the AI engine, for a driver and occupancy monitoring system (DMS or OMS) with support for video calls or video recording. This document begins by outlining the essential hardware and software components, followed by a reference design and benchmarks.