Main Content

rlSimulinkEnv

Create reinforcement learning environment using dynamic model implemented in金宝app

所有的页面崩溃

Syntax

env = rl金宝appSimulinkEnv (mdl代理Blocks)
env = rl金宝appSimulinkEnv (mdl代理Blocks,obsInfo,actInfo)
env = rlSimulinkEnv(___,'UseFastRestart',fastRestartToggle)

Description

TherlSimulinkEnvfunction creates a reinforcement learning environment object from a Simulink®model. The environment object acts an interface so that when you callsimortrain, these functions in turn call the Simulink model to generate experiences for the agents.

example

env= rlSimulinkEnv(mdl,agentBlocks)creates the reinforcement learning environment objectenvfor the Simulink modelmdl.agentBlockscontains the paths to one or more reinforcement learning agent blocks inmdl. If you use this syntax, each agent block must reference an agent object already in the MATLAB®workspace.

example

env= rlSimulinkEnv(mdl,agentBlocks,obsInfo,actInfo)creates the reinforcement learning environment objectenvfor the modelmdl. The two cell arraysobsInfoandactInfomust contain the observation and action specifications for each agent block inmdl, in the same order as they appear inagentBlocks.

env= rlSimulinkEnv(___,'UseFastRestart',fastRestartToggle)creates a reinforcement learning environment objectenvand additionally enables fast restart. Use this syntax after any of the input arguments in the previous syntaxes.

Examples

collapse all

Open Live Script

Create a Simulink environment using the trained agent and corresponding Simulink model from theCreate Simulink Environment and Train Agentexample.

Load the agent in the MATLAB® workspace.

loadrlWaterTankDDPGAgent

Create an environment for therlwatertankmodel, which contains an RL Agent block. Since the agent used by the block is already in the workspace, you do not need to pass the observation and action specifications to create the environment.

env = rlSimulinkEnv('rlwatertank','rlwatertank/RL Agent')
env = SimulinkEnvWithAgent with properties: Model : rlwatertank AgentBlock : rlwatertank/RL Agent ResetFcn : [] UseFastRestart : on

Validate the environment by performing a short simulation for two sample times.

validateEnvironment(env)

You can now train and simulate the agent within the environment by usingtrainandsim, respectively.

This example uses:

Open Live Script

For this example, consider therlSimplePendulumModelSimulink model. The model is a simple frictionless pendulum that initially hangs in a downward position.

Open the model.

mdl ='rlSimplePendulumModel'; open_system(mdl)

CreaterlNumericSpecandrlFiniteSetSpecobjects for the observation and action information, respectively.

obsInfo = rlNumericSpec([3 1])% vector of 3 observations: sin(theta), cos(theta), d(theta)/dt
obsInfo = rlNumericSpec with properties: LowerLimit: -Inf UpperLimit: Inf Name: [0x0 string] Description: [0x0 string] Dimension: [3 1] DataType: "double"
actInfo = rlFiniteSetSpec([-2 0 2])% 3 possible values for torque: -2 Nm, 0 Nm and 2 Nm
actInfo = rlFiniteSetSpec with properties: Elements: [3x1 double] Name: [0x0 string] Description: [0x0 string] Dimension: [1 1] DataType: "double"

You can use dot notation to assign property values for therlNumericSpecandrlFiniteSetSpecobjects.

obsInfo.Name ='observations'; actInfo.Name ='torque';

Assign the agent block path information, and create the reinforcement learning environment for the Simulink model using the information extracted in the previous steps.

agentBlk = [mdl'/RL Agent']; env = rlSimulinkEnv(mdl,agentBlk,obsInfo,actInfo)
env = SimulinkEnvWithAgent with properties: Model : rlSimplePendulumModel AgentBlock : rlSimplePendulumModel/RL Agent ResetFcn : [] UseFastRestart : on

You can also include a reset function using dot notation. For this example, randomly initializetheta0in the model workspace.

env.ResetFcn = @(in) setVariable(in,'theta0',randn,'Workspace',mdl)
env = SimulinkEnvWithAgent with properties: Model : rlSimplePendulumModel AgentBlock : rlSimplePendulumModel/RL Agent ResetFcn : @(in)setVariable(in,'theta0',randn,'Workspace',mdl) UseFastRestart : on
Open Live Script

Create an environment for the Simulink model from the exampleTrain Multiple Agents to Perform Collaborative Task.

MATLAB的工作区中加载代理。

loadrlCollaborativeTaskAgents

Create an environment for therlCollaborativeTaskmodel, which has two agent blocks. Since the agents used by the two blocks (agentAandagentB) are already in the workspace, you do not need to pass their observation and action specifications to create the environment.

env = rlSimulinkEnv('rlCollaborativeTask',["rlCollaborativeTask/Agent A","rlCollaborativeTask/Agent B"])
env = SimulinkEnvWithAgent with properties: Model : rlCollaborativeTask AgentBlock : [ rlCollaborativeTask/Agent A rlCollaborativeTask/Agent B ] ResetFcn : [] UseFastRestart : on

You can now simulate or train the agents within the environment usingsimortrain, respectively.

Input Arguments

collapse all

Simulink model name, specified as a string or character vector. The model must contain at least oneRL Agentblock.

Agent block paths, specified as a string, character vector, or string array.

Ifmdlcontains a singleRL Agentblock, specifyagentBlocksas a string or character vector containing the block path.

Ifmdlcontains multipleRL Agentblocks, specifyagentBlocksas a string array, where each element contains the path of one agent block.

mdlcan containRL Agentblocks whose path is not included inagentBlocks. Such agent blocks behave as part of the environment, selecting actions based on their current policies. When you callsimortrain, the experiences of these agents are not returned and their policies are not updated.

Multi-agent simulation is not supported for MATLAB environments.

The agent blocks can be inside of a model reference. For more information on configuring an agent block for reinforcement learning, seeRL Agent.

Observation information, specified as a specification object, an array of specification objects, or a cell array.

Ifmdlcontains a single agent block, specifyobsInfoas anrlNumericSpecobject, anrlFiniteSetSpecobject, or an array containing a mix of such objects.

Ifmdlcontains multiple agent blocks, specifyobsInfoas a cell array, where each cell contains a specification object or array of specification objects for the corresponding block inagentBlocks.

For more information, seegetObservationInfo.

Action information, specified as a specification object or a cell array.

Ifmdlcontains a single agent block, specifyactInfoas anrlNumericSpecorrlFiniteSetSpecobject.

Ifmdlcontains multiple agent blocks, specifyactInfoas a cell array, where each cell contains a specification object for the corresponding block inagentBlocks.

For more information, seegetActionInfo.

Option to toggle fast restart, specified as either'on'or'off'. Fast restart allows you to perform iterative simulations without compiling a model or terminating the simulation each time.

For more information on fast restart, seeHow Fast Restart Improves Iterative Simulations(Simulink).

Output Arguments

collapse all

Reinforcement learning environment, returned as aSimulinkEnvWithAgentobject.

For more information on reinforcement learning environments, seeCreate Simulink Reinforcement Learning Environments.

Version History

Introduced in R2019a

See Also

Functions

Blocks

Topics