Home Microcontrollers (MCUs) & processors Processors Digital signal processors (DSPs)

Microcontrollers (MCUs) & processors

Digital signal processors (DSPs) – Deep learning

TI Deep Learning (TIDL) software

TIDL software brings deep learning inference to the edge by enabling applications to leverage TI’s proprietary, highly optimized Convolutional Neural Network (CNN)/Deep Neural Network (DNN) implementation on TI’s Sitara AM57x processors. TIDL is a set of open-source Linux software packages and tools that enables offloading of deep learning inference to either the Embedded Vision Engine (EVE) subsystems, C66x DSPs, or both.

Benefits of TIDL

  • Low power deep learning inference for embedded applications at the edge
  • Hardware acceleration for deep learning: runs on scalable platform leveraging C66x DSP cores and/or EVE subsystems on AM57x Sitara™ Processors
  • Can run multiple DNNs simultaneously on different cores
  • Improved performance by leveraging embedded techniques of sparsity, efficient CNN, configurations and dynamic quantization
  • Supports importing from Caffe and Tensorflow
  • Provides Caffe-Jacinto framework optimized for embedded applications
  • Optimized example CNN models that demonstrate real-time processing of applications involving image classification, object detection and pixel-level semantic segmentation

 

TIDL software is available as part of TI’s free Processors (SDK) for Linux. The initial release of TIDL supportsCNN layer types with most of the popular CNN layers present in frameworks such as Caffe and TensorFlow.  Follow Processor SDK Linux TIDL documentation to learn the latest list of TIDL supported
layers. 

New to deep learning?

AM57x Sitara Processors run the deep learning inference. The inference is the part that is deployed in an end application to perform a specific task, i.e. inspecting a bottle on an assembly line, counting and tracking people within a room, or determining whether a bill is counterfeit. The neural network must still be trained on a desktop or cloud environment and then the trained network must be imported into the AM57x Sitara Processors using TI’s device translation tool.

See the figure below for the TIDL development flow and components of TIDL.

placeholder

TIDL development flow

The TIDL development flow consists of three main parts.

  1. Training - Teaches the network model to perform a specific task. Usually occurs offline on desktops or in the cloud and entails feeding large labeled data sets into a DNN. This is where the ‘learning’ part of deep learning. Result of the training phase is a trained network model.
  2. Import (Format Conversion) - Consists of using the TIDL device translator tool to convert trained network models into an internal format best suited for use inside the TIDL library running on AM57x Processor. Command line parameters specify if the inference will run on C66x DSP cores, EVE subsystem or both.
  3. Deployment (Inference) - Run the converted network model on an AM57x Processor using TIDL APIs. 

 

More details on each of these are provided further in the TI Design Guide for deep learning inference for embedded applications reference design and in the Getting Started Guide (GSG) user’s guide section on deep learning.

TIDL example applications

 

Spatial 2D data

(image, video, ToF, radar/mmWave, etc.)

placeholder

Time series data

(speech, audio, machine data)

placeholder
Example applications
Role of deep learning
Vision computers, optical inspection,
semiconductor test and manufacturing
Classify products as good or defective. Deep learning can improve accuracy and flexibility
Building automation Can track/identify/count people and objects
Automated Sorting Equipment, industrial
robots
Ability to recognize objects and guide movement/placement
ATMs, currency counters Determine valid/counterfeit currency
Vacuum robots, robotic lawn mowers Improve ability to recognize obstacles and things like power cords/wires
Smart appliances Recognize objects in refrigerator, determine food and auto cook
Building automation/security Classify and recognize specific sounds/audio patterns
Industrial Equipment Identify wear and expected lifetime of equipment

Examples

Processor SDK includes several examples that are provided in source, including image classification, pixel segmentation and a single shot multi-box detection demo. 

There is also a Matrix-GUI example that is based on ImageNet. This demo can recognize up to 1,000 different classes of object and display the label on the screen along with the image. This can run from either a live camera or from a pre-recorded video clip. A subset of “acceptable” classes are used for classification filtering to tune demo performance and can be listed in the source file “findclasses.cpp.”

placeholder

More information on these demos can be found here.

Sitara™ AM57x

Single/dual Cortex®-A15 and C66x DSPs