SPRADP7A February   2025  – March 2025 AM62A3 , AM62A3-Q1 , AM62A7 , AM62A7-Q1 , AM67A , TDA4AEN-Q1

 

  1.   1
  2.   Abstract
  3.   Trademarks
  4. 1Introduction
  5. 2Building Blocks of an RGB-IR Vision Pipeline
    1. 2.1 CSI Receiver
    2. 2.2 Image Signal Processor
    3. 2.3 Video Processing Unit
    4. 2.4 TI Deep Learning Acceleration
    5. 2.5 GStreamer and TIOVX Frameworks
  6. 3Performance Considerations and Benchmarking Tools
  7. 4Reference Design
    1. 4.1 Camera Module
    2. 4.2 Sensor Driver
    3. 4.3 CSI-2 Rx Driver
    4. 4.4 Image Processing
    5. 4.5 Deep Learning for Driver and Occupancy Monitoring
    6. 4.6 Reference Code and Applications
  8. 5Application Examples and Benchmarking
    1. 5.1 Application 1: Single-stream Capture and Visualization with GST
    2. 5.2 Application 2: Dual-stream Capture and Visualization with GST and TIOVX Frameworks
    3. 5.3 Application 3: Representative OMS-DMS + Video Telephony Pipeline in GStreamer
  9. 6Summary
  10. 7References
  11. 8Revision History

4.4 Image Processing

The OX05B1S is a raw sensor, and the captured raw images are processed by the ISP. To achieve the best image quality for RGB applications, the ISP must be tuned. For details on how to tune the ISP for a specific sensor, refer to the AM6xA ISP Tuning Guide [4]. Pre-tuned ISP configuration binaries for the OX05B1S are provided by the SDK under /opt/imaging/ox05b1s/linear on the target.

The AM62A ISP produces two output frames for each RGB-IR input frame: one YUV frame and one IR frame. In this reference design, the OX05B1S is configured to alternate between RGB-dominant and IR-dominant streams, each at 30 fps. In the vision pipeline implemented by GStreamer or TIOVX, only the RGB output frames are used for the RGB-dominant stream, while the IR frames are discarded, and vice versa for the IR-dominant stream.

While processing the RGB-dominant stream, the ISP also generates statistics that can be used for auto exposure and gain control (AE) and auto white balancing (AWB). The Processor SDK provides AE and AWB algorithms (also referred to as "2A") that are used in this reference design to adjust the sensor exposure and gain and white balancing gains. The exposure and gain adjustments are sent to the sensor, while the white balancing gains are provided to the ISP.