Assume that you have an existing trained reinforcement learning agent. For this example, load the trained agent fromTrain DDPG Agent to Control Double Integrator System。

load('doubleintegddpg.mat','agent')

Obtain the critic representation from the agent.

critic = getCritic(agent);

Obtain the learnable parameters from the critic.

params = getLearnableParameters(critic);

Modify the parameter values. For this example, simply multiply all of the parameters by2。

modifiedParams = cellfun(@(x) x*2,params,'UniformOutput'，错误的）;

Set the parameter values of the critic to the new modified values.

critic = setLearnableParameters(critic,modifiedParams);

将代理商的评论家定为新的修改者。

agent = setCritic(agent,critic);

创建一个具有连续动作空间的环境，并获得其观察和行动规范。对于此示例，加载示例中使用的环境Train DDPG Agent to Control Double Integrator System。

Load the predefined environment.

env = rlPredefinedEnv(“双整合器连续”)

env = DoubleIntegratorContinuousAction with properties: Gain: 1 Ts: 0.1000 MaxDistance: 5 GoalThreshold: 0.0100 Q: [2x2 double] R: 0.0100 MaxForce: Inf State: [2x1 double]

获得观察和行动规格。

obsinfo = getObservationinfo（env）;actinfo = getActionInfo（env）;

Create a PPO agent from the environment observation and action specifications.

agent = rlPPOAgent(obsInfo,actInfo);

为了修改加强学习代理中的深层神经网络，您必须首先提取演员和评论家的表现。

actor = getActor(agent); critic = getCritic(agent);

Extract the deep neural networks from both the actor and critic representations.

actornet = getModel（Actor）;评论家= getModel（评论家）;

The networks aredlnetworkobjects. To view them using theplotfunction, you must convert them toLayerGraphobjects.

例如，查看演员网络。

plot(layerGraph(actorNet))

To validate a network, useanalyzeNetwork。例如，验证评论家网络。

analyzeNetwork(criticNet)

You can modify the actor and critic networks and save them back to the agent. To modify the networks, you can use the深网设计师app. To open the app for each network, use the following commands.

DeepNetworkDesigner（layergraph（Crivernet））deepnetworkDesigner（layergraph（actornet））

在深网设计师, modify the networks. For example, you can add additional layers to your network. When you modify the networks, do not change the input and output layers of the networks returned byGetModel。有关构建网络的更多信息，请参阅Build Networks with Deep Network Designer。

验证修改后的网络深网设计师，您必须点击分析DLNETWORK, under theAnalysis部分。要将修改后的网络结构导出到MATLAB®工作区，请生成用于创建新网络的代码，并从命令行运行此代码。请勿在深网设计师。For an example that shows how to generate and run code, see使用深网设计师创建代理，并使用图像观测来训练。

For this example, the code for creating the modified actor and critic networks is increateModifiedNetworks.m。

createModifiedNetworks

每个修改的网络都包括一个附加的fullyConnectedLayer和reluLayerin their output path. View the modified actor network.

情节（修改Actercornet）

导出网络后，将网络插入演员和评论家的代表。

Actor = SetModel（Actor，ModifiedActornet）;评论家= setModel（评论家，修改Criticnet）；

最后，将改良的演员和评论家的代表插入演员和评论家的对象。

agent = setActor(agent,actor); agent = setCritic(agent,critic);