Many backbone networks use residual structures like the one shown, creating localized parallel paths called "Residuals" that are beneficial during training. Residuals avoid the the vanishing gradient problem.

During compilation, TIDL simulates DDR bandwidth for different computation orders (left branch first, right first, interleaved), and will select the most efficient. It also decides whether the first Conv layer’s output stays in L3/MSMC until the add operation or is written to DDR immediately. Storing in DDR causes direct bandwidth cost, while holding in L3 may occupy memory during left branch computation, potentially forcing parts of the left branch to use DDR instead.

TIDL will select a strategy that maximizes L3 occupancy, but large intermediate feature maps may require DDR be used. In this case, it is recommeded to prioritize optimizing the size of the longer path (left-side in the figure) to avoid multiple feature-maps going to DDR as opposed to the single feature map along the skip connection.

Figure 4-2 Residual structure in a neural network. The right-side path’s “skip” connection has a feature map that must be stored until the left-side path completes