Main Content

Create金宝appReinforcement Learning Environments

在强化学习的情况下,train an agent to complete a task, the environment models the dynamics with which the agent interacts. As shown in the following figure, the environment:

  1. Receives actions from the agent.

  2. Outputs observations in response to the actions.

  3. Generates a reward measuring how well the action contributes to achieving the task.

Creating an environment model includes defining the following:

  • Action and observation signals that the agent uses to interact with the environment.

  • Reward signal that the agent uses to measure its success. For more information, seeDefine Reward Signals.

  • Environment dynamic behavior.

Action and Observation Signals

When you create an environment object, you must specify the action and observation signals that the agent uses to interact with the environment. You can create both discrete and continuous action spaces. For more information, seerlNumericSpecandrlFiniteSetSpec, respectively.

What signals you select as actions and observations depends on your application. For example, for control system applications, the integrals (and sometimes derivatives) of error signals are often useful observations. Also, for reference-tracking applications, having a time-varying reference signal as an observation is helpful.

When you define your observation signals, ensure that all the system states are observable through the observations. For example, an image observation of a swinging pendulum has position information but does not have enough information to determine the pendulum velocity. In this case, you can specify the pendulum velocity as a separate observation.

Predefined金宝appEnvironments

Reinforcement Learning Toolbox™ software provides predefined Simulink®environments for which the actions, observations, rewards, and dynamics are already defined. You can use these environments to:

  • Learn reinforcement learning concepts.

  • Gain familiarity with Reinforcement Learning Toolbox software features.

  • Test your own reinforcement learning agents.

For more information, seeLoad Predefined Simulink Environments.

Custom金宝appEnvironments

To specify your own custom reinforcement learning environment, create a Simulink model with anRL Agentblock. In this model, connect the action, observation, and reward signals to theRL Agentblock. For an example, seeWater Tank Reinforcement Learning Environment Model.

For the action and observation signals, you must create specification objects usingrlNumericSpecfor continuous signals andrlFiniteSetSpecfor discrete signals. For bus signals, create specifications usingbus2RLSpec.

奖励的信号,构造一个标量信号n the model and connect this signal to theRL Agentblock. For more information, seeDefine Reward Signals.

After configuring the Simulink model, create an environment object for the model using therlSimulinkEnvfunction.

If you have a reference model with an appropriate action input port, observation output port, and scalar reward output port, you can automatically create a Simulink model that includes this reference model and anRL Agentblock. For more information, seecreateIntegratedEnv. This function returns the environment object, action specifications, and observation specifications for the model.

Your environment can include third-party functionality. For more information, seeIntegrate with Existing Simulation or Environment(Simulink).

See Also

||

Related Topics