SPRAD86A March   2023  – May 2024 AM62A3 , AM62A3-Q1 , AM62A7 , AM62A7-Q1 , AM68A , AM69A

 

  1.   1
  2.   Abstract
  3.   Trademarks
  4. 1 Introduction
  5. 2 Tuning Overview
  6. 3 Hardware Requirement
  7. 4 Software Requirement
    1. 4.1 Processor SDK Linux
    2. 4.2 TI's Reference Imaging Software
    3. 4.3 ISP Tuning Tool
  8. 5 Sensor Software Integration
    1. 5.1 Overview of Image Pipeline Software Architecture
    2. 5.2 Adding Sensor Driver to SDK
    3. 5.3 Updating TIOVX Modules
      1. 5.3.1 Source Code Change
      2. 5.3.2 Rebuild Modules
    4. 5.4 Update GStreamer Plug-in for VISS
      1. 5.4.1 Update VISS Plug-in Property
      2. 5.4.2 Add Exposure Setting for 2A Algorithm
        1. 5.4.2.1 Gain
        2. 5.4.2.2 Exposure Time
        3. 5.4.2.3 Other Parameters
      3. 5.4.3 Rebuild Plug-ins
      4. 5.4.4 Verify New Sensor in GStreamer Plug-in
  9. 6 Tuning Procedure
    1. 6.1 Verify Functional Operation of Camera Capturing
    2. 6.2 Enable Camera Streaming With Initial VPAC Configuration
      1. 6.2.1 Generate Configuration Files
      2. 6.2.2 Generate DCC Binary Files
      3. 6.2.3 Stream Video With the Initial Configuration
    3. 6.3 Adjust Camera Mounting
  10. 7 Perform Basic Tuning
    1. 7.1 Launch the Tuning Tool and Create a Project
    2. 7.2 Tuning Order
    3. 7.3 Black Level Subtraction
    4. 7.4 Hardware 3A (H3A)
    5. 7.5 PCID
    6. 7.6 Auto White Balance (AWB)
      1. 7.6.1 Capture Raw Images for Different Lighting Conditions
      2. 7.6.2 Tuning AWB
    7. 7.7 Color Correction
  11. 8 Perform Fine Tuning
    1. 8.1 Edge Enhancement (EE)
    2. 8.2 Noise Filter 4 (NSF4)
  12. 9 Live Tuning
    1. 9.1 Requirements
    2. 9.2 Supported Features
      1. 9.2.1 RAW Capture
      2. 9.2.2 YUV Capture
      3. 9.2.3 Live DCC Update
      4. 9.2.4 Exposure Control
      5. 9.2.5 White Balance Control
      6. 9.2.6 Sensor Register Read/Write
  13. 10Summary
  14. 11Revision History

5.1 Overview of Image Pipeline Software Architecture

This section provides an overview of the ISP and the image pipeline software architecture on AM6xA, which helps understand the tuning procedure described later.

The ISP on the AM6xA devices provides common vision primitive functions for image data processing at the pixel level. There are three sub-modules in the ISP: Vision Imaging Subsystem (VISS), Lens Distortion Correction (LDC), and Multi-Scalar (MSC).

  • Vision Imaging Sub-System (VISS): The VISS performs image processing on raw data, which includes wide dynamic range (WDR) merge, defect pixel correction (DPC), lens shading correction (LSC), global and local brightness and contrast enhancement (GLBCE), demosaicing, color conversion, and edge enhancement (EE). This block takes the raw image as input and produces YUV output.
  • Lens Distortion Correction (LDC): The LDC engine is a YUV domain processor designed to perform perspective and geometric transforms. This can be used for creating several effects, such as lens distortion correction, epipolar rectification, and generic perspective transformation.
  • Multi-Scalar (MSC): The MSC can generate up to 10 scaled outputs from a given input with various scaling ratios (between 1 × and 0.25 ×). Each of the 10 scaling operations can be configured to perform pyramid scale or inter-octave scale generation.

Out of these three sub-modules, VISS and LDC need to be tuned, and the VISS contains multiple processing blocks that need to be tuned individually.

Figure 5-1 shows the image processing pipeline software architecture for AM6xA, including the camera capture subsystem and ISP. The camera capture driver runs in Linux on the A53 or A72 core, and the ISP driver runs in RTOS on the R5 core. Though running on different CPU cores, camera capture and ISP can be interfaced with and integrated by the same framework of GStreamer or TIOVX.

AM6xA Image Processing Pipeline Figure 5-1 AM6xA Image Processing Pipeline

In this report, GStreamer examples are provided for integrating the image pipeline components. The camera capture subsystem is accessed through the GStreamer plug-in v4l2src. Each of the three ISP sub-modules is accessed through the corresponding GStreamer plug-in: tiovxisp, tiovxldc, and tiovxmsc.