主要内容

Code Generation for Quantized Deep Learning Networks

Deep learning uses neural network architectures that contain many processing layers, including convolutional layers. Deep learning models typically work on large sets of labeled data. Performing inference on these models is computationally intensive, consuming significant amount of memory. Neural networks use memory to store input data, parameters (weights), and activations from each layer as the input propagates through the network. Deep Neural networks trained in MATLAB®use single-precision floating point data types. Even networks that are small in size require a considerable amount of memory and hardware to perform these floating-point arithmetic operations. These restrictions can inhibit deployment of deep learning models to devices that have low computational power and smaller memory resources. By using a lower precision to store the weights and activations, you can reduce the memory requirements of the network.

You can use Deep Learning Toolbox™ in tandem with the Deep Learning Toolbox Model Quantization Library support package to reduce the memory footprint of a deep neural network by quantizing the weights, biases, and activations of convolution layers to 8-bit scaled integer data types. Then, you can useMATLAB Coder™to generate optimized code for the quantized network.

ARMCortex-A Processors

The generated code takes advantage of ARM®使用ARM COMPUTE库的处理器SIMD。生成的代码可以作为源代码，静态或动态库或可执行文件集成到您的项目中，您可以将其部署到各种ARM Cortex-A CPU平台（例如Raspberry Pi™）中。要在ARM Cortex-A处理器上部署量化的网络，您必须使用ARM Compute Library版本20.02.1。

金宝app支持的层和课程

You can generate C++ code for these layers that uses the ARM Compute Library and performs inference computations in 8-bit integers:

C++ code generation for such quantized deep learning networks supportsDAGNetwork(Deep Learning Toolbox)和SeriesNetwork(Deep Learning Toolbox)objects.

生成代码

要生成在8位整数中执行推理计算的代码，coder.ARMNEONConfigobjectdlcfg，设置se additional properties:

dlcfg.CalibrationResultFile =“ dlquantizerObjectMatfile”; dlcfg.DataType ='int8';

或者，在MATLAB Coder应用程序，在深度学习tab, setTarget librarytoC臂ompute。然后设置数据类型和校准结果文件路径parameters.

Here“ dlquantizerObjectMatfile”is the name of the MAT-file thatdlquantizer(Deep Learning Toolbox)generates for specific calibration data. For the purpose of calibration, set the执行环境属性dlquantizer反对'CPU'。

Otherwise, follow the steps described in使用ARM计算库的深度学习网络代码生成。

例如，请参阅Code Generation for Quantized Deep Learning Network on Raspberry Pi。

ARMCortex-M Processors

The generated code takes advantage of the CMSIS-NN library version 5.7.0 and can be integrated into your project as a static library that you can deploy to a variety of ARM Cortex-M CPU platforms.

金宝app支持的层和课程

The code generated for thefullyConnectedLayer(Deep Learning Toolbox)代表完全连接的层的对象使用CMSIS-NN库，并在8位整数中执行推理计算。

Your deep learning network can also contain the following layers. The generated code performs computations for these layers in 32-bit floating point type.

  • lstmLayer(Deep Learning Toolbox)对象，代表长期的短期内存层。的价值SequenceLengththat you pass to预测must be a compile-time constant.

  • softmaxLayer(Deep Learning Toolbox)object, which represents a softmax layer.

  • 输入和输出层。

C code generation for such quantized deep learning networks supportsSeriesNetwork(Deep Learning Toolbox)objects andDAGNetwork(Deep Learning Toolbox)objects that can be converted toSeriesNetworkobjects.

生成代码

要通过使用CMSIS-NN库在8位整数中执行推理计算的代码，coder.CMSISNNConfigobjectdlcfg，设置CalibrationResultFile财产：

dlcfg.CalibrationResultFile =“ dlquantizerObjectMatfile”;

或者，在MATLAB Coder应用程序，在深度学习tab, setTarget librarytoCMSIS-NN。然后设置校准结果文件路径parameter.

Here“ dlquantizerObjectMatfile”is the name of the MAT-file thatdlquantizer(Deep Learning Toolbox)generates for specific calibration data. For the purpose of calibration, set the执行环境属性dlquantizer反对'CPU'。

例如，请参阅在Cortex-M目标上量化深度学习网络的代码生成。

See Also

Apps

Functions

Objects

相关话题