SPRADB0 may   2023 AM62A3 , AM62A3-Q1 , AM62A7 , AM62A7-Q1 , AM68A , AM69A

 

  1.   1
  2.   Abstract
  3.   Trademarks
  4. 1Introduction
    1. 1.1 Intended Audience
    2. 1.2 Host Machine Information
  5. 2Creating the Dataset
    1. 2.1 Collecting Images
    2. 2.2 Labelling Images
    3. 2.3 Augmenting the Dataset (Optional)
  6. 3Selecting a Model
  7. 4Training the Model
    1. 4.1 Input Optimization (Optional)
  8. 5Compiling the Model
  9. 6Using the Model
  10. 7Building the End Application
    1. 7.1 Optimizing the Application With TI’s Gstreamer Plugins
    2. 7.2 Using a Raw MIPI-CSI2 Camera
  11. 8Summary
  12. 9References

5 Compiling the Model

As mentioned in the previous section, compilation can be handled automatically if using edgeai-modelmaker for training. If a model needs compilation on its own, or if the user wishes to be more involved in the compilation and calibration then they can refer to this section. Edgeai-tidl-tools hosts code that serves this purpose. Model compilation must be done on an x86 host machine.

The edgeai-tidl-tools repository offers several options for model compilation, such as Jupyter notebooks and python3 scripts. There are instructions in that repository for compiling custom models. The process using Jupyter notebook can be done on Edge AI Cloud as well using the tools for custom models. Jupyter [7] notebooks are a great tool to show and develop code step-by-step in a modularized form, but the Jupyter kernel can be unstable on long-running or memory-intensive tasks. The Python3 scripts were used instead for the mobilenetv2SSD model compilation due to better runtime stability.

To compile a model, a set of images is passed through the runtime of choice for the model (ONNX for the food-recognition model), during which a TI Deep Learning compilation backend analyzes the intermediate values and formats. As this script runs, "artifacts" are generated in the form of compiled binary files for running the model on the C7xMMA. This includes an "io" file for how CPUs pass information to/from the C7xMMA and a "net" file which encodes the network layers itself.

In this process, the set of images are used to calibrate the model for quantization from native 32-bit floating points to fixed-point integers (8 or 16-bit). The tools analyze the range of intermediate values within the model during execution and choose a quantization scheme the best matches that range. Using a larger set of images that better characterizes the breadth of the input scenes are encountered in practice can improve the quality of quantization, but takes longer.

The python scripts underedgeai-tidl-tools/examples/osrt_python/ can handle all steps of model compilation, but are less clear on what is happening with preprocessing, configuration, etc. than the more interactive Jupyter notebooks. The onnxrt_ep.py and model_configs.py files required a few changes for compilation to complete:

  • There is a list of models to compile. For the food-recognition model, this included 'od-8020_onnxrt_coco_edgeai-mmdet_ssd_mobilenetv2_lite_512x512_20201214_model_onnx'.
  • The configuration for the model is held in a large python dictionary from model_configs.py in the above directory. The key is the string from the previous bullet.
    • "model_path" must point to the trained model from the custom dataset.
    • “meta_layers_names_list” must point to the PROTOTXT file path that describes the model architecture.
    • If the two above files are not present, they are downloaded, assuming it is a model architecture that is already supported and has a download at the provided URL.

The TIDL_TOOLS_PATH environment variable in the calling environment should be pointing to the tidl_tools for the target processor. By default, it points to the tools downloaded when setting up edgeai-tidl-tools, during which an SOC variable should have been set for the device to compile to for, e.g. SOC=am62a.

Once complete, compiled artifacts are in a directory with the model's full name under edgeai-tidl-tools/model-artifacts.

One last (optional) item before continuing to using the model in practice: getting class labels aligned between model output and text name. This is only necessary if using edgeai-gst-apps or the tidlpostprocess GST plugin, as they attempt to write text for the recognized object(s) on the image. When modelmaker attempts to compile a model, it creates a dataset.yaml file to describe the dataset, which includes mapping the output class integer to a text label.. This is retrieved from the COCO JSON file that holds labels. The param.yaml file needs to also provide a mapping (key [metric][label_offset_pred]) from the output of the model to the index in dataset.yaml. For the food-recognition model, it is one-to-one since all classes trained on are used. The dataset.yaml is automatically generated by TI’s training frameworks, but the param.yaml may require manual update.