5 Camera Mirror System Data Flow and Latency

In a camera mirror system, the image data from the camera typically goes through the CSI-2 RX interface, ISP, deep learning engine, and finally to the display. Figure 5-1shows the data flow on the AM62A device.

Figure 5-1 Typical Data Flow of a Camera Mirror System

For camera mirror applications, the latency from camera to display (that is, the so-called glass-to-glass latency) must be as small as possible. Every block in the data flow contributes to the latency. The AM62A SoC has the following differentiating features that can achieve optimal latency:

• High throughput ISP: each of the three blocks of VPAC3L can process 1 pixel per clock cycle, up to 300MP/s after accounting for overhead.
• ISP on-the-fly mode: when operating in this mode, VISS processes the camera data on the fly, without waiting for a full frame of data to be available.
• High performance deep learning accelerator: the C7x, MMA deep learning accelerator has 2 TOPS processing capability.
• DDR subsystem supports up to 3733 MT/s.

Table 5-1 shows an exemplary analysis of the latency accumulated block after block in the data path shown in Figure 5-1. A 2.1MP (1936 × 1100) sensor running at 60 fps is used for the analysis. The latency introduced by each block is estimated as the following:

• At 60 fps, the frame duration is 16.67 ms.
• This analysis assumes VISS running in memory-to-memory mode instead of on-the-fly mode. Therefore, VISS has to wait until a whole frame is available before it starts processing.
• The processing time for each of VISS, LDC, and MSC is estimated as 1936 × 1100 Pixels / 300MP/s = 7 ms. Configuration time for each block is considered as 1 ms to be conservative. Therefore, total latency is about 8 ms for each block.
• Deep learning latency is estimated to be 8 ms according to Section 4.

As shown in this analysis, the total latency of the data flow shown in Figure 5-1 is about 65ms, which is adequate for a typical camera mirror system.