The DSP compute cluster integrates a TI C7000 vector DSP core, coupled with a matrix multiply accelerator (MMA). For simplicity, the combination of C7000™ digital signal processor (DSP) and MMA is hereafter referred to as the C7™ neural processing unit (NPU). There are four instances of the C7 NPU compute engine available in the TDA4VH-Q1 DSP compute cluster, as shown in Figure 2-1.

The 512-bit data width and efficient data-streaming architecture enable the MMA to sustain high throughput for matrix operations during machine learning inference. For instance, the MMA achieves a throughput of one 64-element inner product per clock cycle using 8-bit operands. This involves 4,096 multiply-accumulate (MAC) operations per clock cycle, totaling 8,192 operations. Because the MMA can sustain this calculation indefinitely and operates at a clock rate up to 1GHz, the MMA is capable of 8 tera operations per second (TOPS).

A separate neural network model can be computed on each of the four C7 NPU subsystems, providing a total 32 TOPS of machine learning inference performance. Alternatively, it is also possible to run a single large model spanning multiple C7 NPUs. The C7 NPU memory system is designed to provide maximal utilization of this performance; each of the four C7 NPU compute engines is tightly coupled with level 1 (L1) instruction and data caches, level 2 (L2) cache RAM, and 3Mbytes of extended L2 RAM for model storage and execution.

When not performing AI computations, the programming flexibility of the C7 DSP core enables developers to leverage the native vector DSP instructions for traditional signal processing tasks, such as Fast Fourier Transforms (FFTs), or complex matrix operations that are commonly required for SLAM computations. The C7000 Very Long Instruction Word (VLIW) architecture provides a high degree of parallelism to enable high-throughput Single Instruction Multiple Data (SIMD) operations. The Texas Instruments optimized FFT library [3] provides highly optimized kernels for computing FFTs as shown in Table 2-1.

For example, in a range calculation for 16 data channels, a single C7 DSP can concurrently compute up to 16 1024-point floating point FFTs within 17,988 instruction cycles. When operating at the full 1GHz clock rate these sixteen FFT computations complete in under 18uS. However performing such calculations continuously incurs an additional cycle penalty of 12 to 17% for background data movements. Thus, factoring in a 16% overhead for memory transfers yields a sustained throughput is 767,000 1024-point FFTs per second.