SPRADP7A February   2025  – March 2025 AM62A3 , AM62A3-Q1 , AM62A7 , AM62A7-Q1 , AM67A , TDA4AEN-Q1

 

  1.   1
  2.   Abstract
  3.   Trademarks
  4. 1Introduction
  5. 2Building Blocks of an RGB-IR Vision Pipeline
    1. 2.1 CSI Receiver
    2. 2.2 Image Signal Processor
    3. 2.3 Video Processing Unit
    4. 2.4 TI Deep Learning Acceleration
    5. 2.5 GStreamer and TIOVX Frameworks
  6. 3Performance Considerations and Benchmarking Tools
  7. 4Reference Design
    1. 4.1 Camera Module
    2. 4.2 Sensor Driver
    3. 4.3 CSI-2 Rx Driver
    4. 4.4 Image Processing
    5. 4.5 Deep Learning for Driver and Occupancy Monitoring
    6. 4.6 Reference Code and Applications
  8. 5Application Examples and Benchmarking
    1. 5.1 Application 1: Single-stream Capture and Visualization with GST
    2. 5.2 Application 2: Dual-stream Capture and Visualization with GST and TIOVX Frameworks
    3. 5.3 Application 3: Representative OMS-DMS + Video Telephony Pipeline in GStreamer
  9. 6Summary
  10. 7References
  11. 8Revision History

3 Performance Considerations and Benchmarking Tools

When building a vision application, the following performance metrics need to be considered and benchmarked:

  • End-to-end latency: The latency between capturing an image and generating the analytics result must be as low as possible to allow for timely decision-making and responsive actions.
  • Video throughput (frames per second): Images must be captured and processed at the desired frame rate without frame drops.
  • CPU load: The load on general-purpose CPU cores (A53 in the case of AM62A) due to the vision pipeline must be minimal, as all image processing is done on hardware accelerators.
  • DDR utilization: The DDR read and write operations by the vision pipeline must leave enough bandwidth for other system tasks.
  • Hardware accelerators (HWA) load (ISP, VPU, C7x/MMA): HWAs are dedicated to specific functionalities and cannot be used for other purposes. The HWAs can be utilized up to 100% by the vision pipeline with some margin.

The EdgeAI SDK for AM62A provides several tools to benchmark these performance metrics:

  • Perf_stats tool [5]: Measures the load on CPU cores and HWAs, as well as DDR utilization.
  • GStreamer debug trace: By setting the environment variable GST_DEBUG_FILE, GStreamer debug messages can be redirected to a file. An EdgeAI SDK script (/opt/edgeai-gst-apps/scripts/gst_tracers/parse_gst_tracers.py) can process these messages and estimate the processing time for each element in the GStreamer pipeline.
  • GStreamer plugin fpsdisplaysink: Displays the throughput of the pipeline in frames per second (fps).
  • Custom GStreamer plugin tiperfoverlay: Projects the CPU loads, DDR utilization, HWA loads, and fps on the display or prints them on the terminal console.