Skip to main content
Version: kirkstone_1-06-00

NXP eIQ

Introduction​

In this page, we will show how to integrate to the Clea OS the following AI runtimes:

  • TensorFlow v 2.9.1
  • OpenCV 4.6.0

NXP eIQ​

NXP eIQ software provides the basis for Machine Learning application optimized for i.MX SoCs, enabling Neural Network acceleration on NXP SoCs on the GPU or NPU through the OpenVX backend.

When executing inference on Cortex-A cores, NXP eIQ inference engines support multi-threaded execution. eIQ based on NXP BSP L5.15.71_2.2.x also supports DeepViewRT (install guide/documentation).

You can find more detailed information on the features of eIQ for each specific version on the i.MX Machine Learning User's Guide available on the NXP's Embedded Linux Documentation.

OpenVX provides NPU/GPU acceleration for TensorFlow Lite, but not OpenCV (as stated on the i.MX Machine Learning User's Guide).

TensorFlow Lite​

As stated in the TensorFlow Lite Documentation:

TensorFlow Lite is a set of tools that enables on-device machine learning by helping developers run their models on mobile, embedded, and IoT devices.

In order to execute TensorFlow models with TensorFlow Lite, you need to use the TensorFlow Lite Converter. The TensorFlow Lite version needs to match the TensorFlow version used to design the model.

Not every TensorFlow model is directly convertible to TensorFlow Lite, because some TensorFlow operators (ops) do not have a TensorFlow Lite equivalent. However, in some situations, you can use a mix of TensorFlow and TensorFlow Lite ops by enabling the Select TensorFlow Ops feature. Please, see the TensorFlow Lite Documentation for more information about this feature and how to enable it.

Clea OS​

Pre-Requisites​

Adding eIQ recipes to Reference Images for Yocto Project​

Cloning the Clea OS BSP repository​

In an empty directory, use git-repo to obtain the BSP on the last version:

Create a folder for the project

$ mkdir -p ~/projects/clea-os
$ cd ~/projects/clea-os

Initialize the manifest environment

$ repo init -u https://git.seco.com/clea-os/seco-manifest.git -b kirkstone
$ repo sync -j$(nproc) --fetch-submodules --no-clone-bundle

Define the configuration that you need to build (e.g. 'seco_smarc_d18_clea_os_embedded_wayland' which refers to Clea OS Embedded image for D18 board):

$ . ./seco-setup.sh -d seco_smarc_d18_clea_os_embedded_wayland
$ . ./seco-setup.sh -c

Getting eIQ​

eIQ is provided on a Yocto layer called meta-imx/meta-ml.

The next steps expect the current directory to be ~/projects/clea-os/layers.

Clone the meta-imx Git repository into your project layers directory:

$ git clone -b kirkstone-5.15.71-2.2.2 https://github.com/nxp-imx/meta-imx.git

Adding the recipes to distribution​

Add meta-imx/meta-ml to layer set to be included in the project:

$ cd ~/projects/clea-os/build_d18_embedded_wayland
$ echo 'BBLAYERS:append = " ${BSPDIR}/layers/meta-imx/meta-ml "' >> conf/bblayers.conf

Add the meta-ml recipes and some image processing libraries to your image:

$ echo 'IMAGE_INSTALL:append = " tensorflow-lite tensorflow-lite-vx-delegate opencv python3-pillow adwaita-icon-theme "' >> conf/local.conf

Configuring the Machine​

If you want to build for a machine based on an NXP SoM, some downloads require you to read and accept the NXP/Freescale EULA available in ~/projects/clea-os/layers/meta-freescale/EULA.

You have to state your acceptance by appending the following line to your ~/projects/clea-os/build_d18_embedded_wayland/conf/local.conf file:

ACCEPT_FSL_EULA = "1"

Building​

Build the seco-image-clea-os-full image for your target SoM:

$ bitbake seco-image-clea-os-full

Flashing the image​

To flash your image to the board, see the Installation Guide for your SoM.

Executing Demos​

NXP provides an inference example, supporting CPU, GPU, and NPU.

To execute it, cd to the example's directory:

# cd /usr/bin/tensorflow-lite-2.9.1/examples/

This demo will take an arbitrary picture (grace_hopper.bmp) as an input of an image classification neural network based on Mobilenet V1 (224x224 input size). See more information about this demo on the NXP's i.MX Machine Learning User's Guide.

To run the demo:

  • NPU
# USE_GPU_INFERENCE=0 ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt --external_delegate_path=/usr/lib/libvx_delegate.so
  • GPU
# USE_GPU_INFERENCE=1 ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt --external_delegate_path=/usr/lib/libvx_delegate.so
  • CPU
# ./label_image -m mobilenet_v1_1.0_224_quant.tflite -i grace_hopper.bmp -l labels.txt

See below a comparison of Inference Time executing this demo:

  Som                                    Inference Time   FPS (1/Inference Time)
  -------------------------------------- ---------------- ------------------------
  Seco D18 iMX8M Plus - CPU only         40.83  ms        24.49 fps
  Seco D18 iMX8M Plus with GPU Support   163.97 ms        6.09  fps
  Seco D18 iMX8M Plus with NPU Support   2.54   ms        393.70 fps

Alternatively, you can run the same example using a Python implementation:

  • NPU
# USE_GPU_INFERENCE=0 python3 label_image.py -e /usr/lib/libvx_delegate.so
  • GPU
# USE_GPU_INFERENCE=1 python3 label_image.py -e /usr/lib/libvx_delegate.so
  • CPU
# USE_GPU_INFERENCE=0 python3 label_image.py

As explained on the NXP's Application Note AN12964, the i.MX 8M Plus SoC requires an Warmup Time of about 7 seconds to initiate before delivering its expected high performance. You will observe this extra time when starting an application with NPU support.

Additional Resources​

See the version-specific NXP's i.MX Machine Learning User's Guide for more information about eIQ.