Skip to main content
Version: kirkstone_1-07-00

Rockchip NPU

Introduction​

Many Rockchip SOC have a NPU(Neural Process Unit) that Neural network acceleration engine with processing performance up to 1 TOPS.

Supported Platform​

These are the model supported:

  • RK3566/RK3568 Series
  • RK3588 Series
  • RK3576 Series
  • RK3562 Series
  • RV1103/RV1106
  • RK2118

In Clea OS Embedded, the Rockchip SDK will soon be included to enable the NPU. By following the steps in this guide, it will be possible to run a demo model and set up the environment to convert models for running on Rockchip boards.

Clone the SDK​

Start with cloning the Rockchip SDK rknn-toolkit2

$ git clone https://github.com/airockchip/rknn-toolkit2/tree/master

The contents of what you will get:

.
├── autosparsity
├── CHANGELOG.md
├── doc
├── LICENSE
├── README.md
├── res
├── rknn-toolkit2
├── rknn-toolkit-lite2
└── rknpu2

Running Demo​

Copy the folder rknn-toolkit2/examples in the board folder Home, then run:

# cd    
# cd examples/resnet18/
# python3 test.py 
--> Load RKNN model
done
--> Init runtime environment
I RKNN: [10:41:42.758] RKNN Runtime Information, librknnrt version: 2.0.0b0 (35a6907d79@2024-03-24T10:31:14)
I RKNN: [10:41:42.765] RKNN Driver Information, version: 0.8.2
I RKNN: [10:41:42.776] RKNN Model Information, version: 6, toolkit version: 2.0.0b0+9bab5682(compiler version: 2.0.0b0 
(35a6907d79@2024-03-24T02:34:11)), target: RKNPU lite, target platform: rk3566, framework name: PyTorch, framework layo
ut: NCHW, model inference type: static_shape
done
--> Running model
resnet18
-----TOP 5-----
[812] score:0.999680 class:"space shuttle"
[404] score:0.000249 class:"airliner"
[657] score:0.000013 class:"missile"
[466] score:0.000009 class:"bullet train, bullet"
[895] score:0.000008 class:"warplane, military plane"

done

Non-RKNN Model​

For other models like Caffe, TensorFlow, etc, to run on Rockchip platform, conversions are needed. Use RKNN-Toolkit2 to convert other model into RKNN model.

RKNN-Toolkit2​

RKNN-Toolkit2 is a development kit that provides users with model conversion, inference and performance evaluation on PC and Rockchip NPU platforms. Users can easily complete the following functions through the Python interface provided by the tool:

  • Model conversion: support to convert Caffe / TensorFlow / TensorFlow Lite / ONNX / Darknet / PyTorch model to RKNN model, support RKNN model import/export, which can be used on Rockchip NPU platform later

  • Quantization: support to convert float model to quantization model, currently support quantized methods including asymmetric quantization(asymmetric_quantized-8, asymmetric_quantized-16). and support hybrid quantization. Asymmetric_quantized-16 not supported yet

  • Model inference: Able to simulate Rockchip NPU to run RKNN model on PC and get the inference result. This tool can also distribute the RKNN model to the specified NPU device to run, and get the inference results

  • Performance evaluation: distribute the RKNN model to the specified NPU device to run, and evaluate the model performance in the actual device

  • Memory evaluation: Evaluate memory consumption at runtime of the model. When using this function, the RKNN model must be distributed to the NPU device to run, and then call the relevant interface to obtain memory information

  • Quantitative error analysis: This function will give the Euclidean or cosine distance of each layer of inference results before and after the model is quantized. This can be used to analyze how quantitative error occurs, and provide ideas for improving the accuracy of quantitative models

RKNN-Toolkit2 installation​

It is recommended to use virtualenv to manage the python environment because there may be multiple versions of the python environment in the system at the same time. Let's start with Python 3.6 as an example:

Install virtualenv, Python3.6 and pip3

$ sudo apt install virtualenv python3 python3-dev python3-pip

Install dependent libraries

$ sudo apt install libxslt1-dev zlib1g zlib1g-dev libglib2.0-0 libsm6 \
libgl1-mesa-glx libprotobuf-dev gcc

Use virtualenv and install Python dependency, such as requirements_cp36-1.3.0.txt

$ virtualenv -p /usr/bin/python3 venv
$ source venv/bin/activate
$ pip3 install -r doc/requirements_cp36-*.txt

Install RKNN-Toolkit2, such as rknn_toolkit2-1.3.0_11912b58-cp36-cp36m-linux_x86_64.whl

$ sudo pip3 install packages/rknn_toolkit2*cp36*.whl

Check if RKNN-Toolkit2 is installed successfully or not, press key ctrl+d to exit

(venv)$ python3
>>> from rknn.api import RKNN
>>>

Model conversion​

To convert a model to the RKNN model follow the README.md in rknn-toolkit2/rknn-toolkit2/examples/[library]/[model_type] as each model need different steps

Detailed Development Documents​

Please refer to pdf files under docs/ directory in RKNN and Toolkit SDK for development repo.