This post is from guest blogger Kishen Mahadevan, Product Marketing. Kishen helps customers understand AI, deep learning and reinforcement learning concepts and technologies. In this post, Kishen explains how deep learning can be integrated into an engineering system designed in Simulink.

背景

推动Arti深度学习是一个关键技术ficial Intelligence (AI) megatrend. Popular applications of deep learning include autonomous driving, speech recognition, and defect detection. When deep learning is used in complex systems it is important to note that a trained deep learning model is only a small component of a larger system. For example, embedded software for self-driving cars has components such as adaptive cruise control, lane keep assist, sensor fusion, and lidar processing in addition to a deep learning model that performs a specific task, say lane detection. How do you then integrate, implement, and test all these different components together while minimizing expensive testing with the actual vehicle? This is where Model-Based Design with MATLAB and Simulink fits in.

Introduction

When you create a Simulink model for any complex system, you typically have two main components, as shown in Figure 1. The first component represents a collection of algorithms that will be implemented in the embedded system and includes controls, computer vision, and sensor fusion. The second component represents the dynamics of the machine or process we want to develop embedded software for. This component can be a vehicle dynamics model, dynamics of Li-Ion battery, or a model of a hydraulic valve.

Having both of these components in the same Simulink model allows you to run simulations to verify and validate embedded algorithms before implementing them on target hardware. Trained deep learning models can be used in both of these components. Examples of using deep learning for algorithm development include use of deep learning for object detection and for soft, or virtual sensing. In the latter scenario deep learning model is used to compute a signal that cannot be measured directly, for example a state-of-charge for a Li-Ion battery.

深度学习模型也可以用于环境建模。有时将其称为减少订单建模。可以用更快的基于AI的模型代替机器或工艺的详细高保真模型，该模型经过训练以捕获原始模型的基本动力学。

Figure 1: Integrating deep learning models into Simulink

In this blog, we will focus on an example that illustrates the use of deep learning for algorithm development. The example shows how you can integrate a trained deep learning model into Simulink for system-level simulation and code generation.

笔记: The features and capabilities showcased in this blog can be applied for algorithm development as well as reduced-order modeling.

Simulink示例中的深度学习金宝app

Deep learning workflow involves four main stages:

1. Data preparation
2. AI modeling
3. Simulation and Testing
4. 部署

让我们以电池管理系统的示例为例，其中使用深度学习来估算电池的收费（SOC）。SOC是电池管理系统的重要信号，但在操作过程中不能直接测量它。但是，借助实验室中收集了足够的数据，可以培训深度学习模型，以使用常用的测量来准确预测电池SOC。

Let’s start by looking at the data predictors or the observations required for the deep learning network. These predictors include measurements of voltage, current, temperature, the calculated moving average values of voltage, and current. The data needed to train a deep learning network also includes the response - battery SOC associated with each set of those measurements.

图2：训练深度学习网络（顶部），深度学习网络输入和输出（底部）的数据

With this data, the deep learning model is configured to receive five inputs and provide state-of-charge (SOC) of the battery as the predicted output.

Once the data has been preprocessed, you can train a deep learning model usingDeep Learning Toolbox。有时，您可能已经在Tensorflow或其他深度学习框架中开发了AI模型。使用深度学习工具箱，您可以将这些模型导入MATLAB，以进行系统级仿真和代码生成。在此示例中，我们使用已在TensorFlow培训的现有深度学习模型。

图3：深度学习工作流程

步骤1：数据准备

对于工作流程的这一步骤，我们使用从实验室收集的已经可用的预处理实验数据。该数据包括图2中突出显示的所有预测因素和响应。麦克马斯特大学向我们提供了此数据（数据源).

Step 2: AI Modeling

As pointed out earlier, the deep learning model can be trained in MATLAB using Deep Learning Toolbox. Refer to the learn more about how to train a deep learning network to predict SOC in MATLAB.

如本示例中已经提到的那样，我们已经为已经在TensorFlow中培训的深度学习模型提供了。

To import this trained network into MATLAB, we use theimportTensorFlowNetworkfunction.

Figure 4: Direct network import from TensorFlow into MATLAB

We then analyze the imported network architecture usinganalyzeNetworkto check for warning or errors and observe that all the imported layers are supported.

Figure 5: Analyzing the imported network using Deep Learning Network Analyzer

We then load the test data and verify performance of the imported network in MATLAB.

图6:MATLAB代码来加载和情节的预测results

Figure 7: Comparing deep learning SOC prediction with true observed SOC value.

We see that the deep learning predicted SOC of the battery is in alignment with the experimentally observed values.

步骤3：仿真和测试

To be able to simulate and test this deep learning SOC estimator with all other components of a Battery Management System, we first need to bring this component into Simulink. To accomplish this, we use预测block from Deep Learning Toolbox block library to add the deep learning model into a Simulink model.

Figure 8: Deep Learning Toolbox library to bring trained deep learning models into Simulink

Figure 9 shows the open-loop Simulink model. The Predict block loads our trained deep learning model into Simulink from a .MAT file. The block receives the preprocessed data as the input and estimates SOC of the battery.

图9：将训练训练的深度学习模型集成到Simulink金宝app

We then simulate this model and observe that prediction from our deep learning network in Simulink is identical to the true measured data as shown in Figure 10.

图10：比较深度学习网络和真实价值的SOC预测的仿真结果

Now that we have tested the component in Simulink, we can integrate it into a larger model and simulate the complete system. This is shown in Figure 11.

图11：电池管理系统和电池厂的系统级仿真模型金宝app

金宝app图11中所示的Simulink模型包含一个电池管理系统，该系统负责监视电池状态并确保安全操作，以及一个对电池和负载动力学建模的电池厂。

Our deep learning SOC predictor resides as one of the components under the battery management system along with the logic for cell balancing, prevention of overcharging and over-discharging, and other components.

Figure 12: Components of battery management system

现在，我们模拟了这个闭环系统并观察SOC预测。

Figure 13: System-level simulation results comparing SOC prediction from deep learning network with the true value

We can see that SOC predictions from our deep learning model is very similar to predictions from the true measured value.

Step 4: Deployment

为了强调在此示例中部署我们深度学习网络的能力，我们使用的开环模型仅包含深度学习SOC预测器。但是，对于系统级模型，工作流程步骤保持不变。

我们首先从Simulink中训练的深度学习模型中生成C代码。金宝app

Figure 14: C code generation for the deep learning network in Simulink

We can see that the generated code contains calls to deep learning step functions that perform SOC prediction.

接下来，我们将生成的代码部署到NXP板上进行处理器（PIL）仿真。在PIL模拟中，我们仅针对正在开发的算法生成生产代码，在这种情况下为深度学习SOC组件，并在目标硬件板NXP S32K3上执行该代码。这使我们能够验证嵌入式目标上的代码行为。

We now add driver blocks to the model to allow us to interface to and from the NXP board and simulate the model.

图15：NXP板上深度学习SOC组件的循环仿真

We see that the behavior of the generated code on the NXP target is identical to the true measured SOC value.

关键要点

• Integrate deep learning models into system-level Simulink models
• 通过深度学习的测试系统级设计
• Generate code and deploy your application, including deep learning component, to embedded target
• Train deep learning models in MATLAB or import pretrained TensorFlow and ONNX models

Resources to learn more

• ai for 金宝appsimulink用户
• 使用深度学习的电池状态估算