Create an observation specification object (or alternatively usegetObservationInfoto extract the specification object from an environment). For this example, define the observation space as a continuous four-dimensional space, so that a single observation is a column vector containing four doubles.

obsInfo = rlNumericSpec([4 1]);

Create an action specification object (or alternatively usegetActionInfoto extract the specification object from an environment). For this example, define the action space as a continuous two-dimensional space, so that a single action is a column vector containing two doubles.

2 actInfo = rlNumericSpec ([1]);

Create a deep neural network approximator for the actor. The input of the network (here calledmyobs) must accept a four-element vector (the observation vector just defined byobsInfo), and its output must be the action (here calledmyact) and be a two-element vector, as defined byactInfo.

net = [featureInputLayer(4,'Normalization','none','Name','myobs')富尔语lyConnectedLayer(2,'Name','myact')];

Create the critic withrlQValueRepresentation, using the network, the observations and action specification objects, as well as the names of the network input and output layers.

actor = rlDeterministicActorRepresentation(net,obsInfo,actInfo,...'Observation',{'myobs'},'Action',{'myact'})

actor = rlDeterministicActorRepresentation with properties: ActionInfo: [1x1 rl.util.rlNumericSpec] ObservationInfo: [1x1 rl.util.rlNumericSpec] Options: [1x1 rl.option.rlRepresentationOptions]

To check your actor, usegetActionto return the action from a random observation, using the current network weights.

act = getAction(actor,{rand(4,1)}); act{1}

ans =2x1 single column vector-0.5054 1.5390

You can now use the actor to create a suitable agent (such as anrlACAgent,rlPGAgent, orrlDDPGAgentagent).

Create an observation specification object (or alternatively usegetObservationInfoto extract the specification object from an environment). For this example, define the observation space as a continuous four-dimensional space, so that a single observation is a column vector containing 3 doubles.

obsInfo = rlNumericSpec([3 1]);

The deterministic actor does not support discrete action spaces. Therefore, create acontinuous action spacespecification object (or alternatively usegetActionInfoto extract the specification object from an environment). For this example, define the action space as a continuous two-dimensional space, so that a single action is a column vector containing 2 doubles.

2 actInfo = rlNumericSpec ([1]);

Create a custom basis function. Each element is a function of the observations defined byobsInfo.

myBasisFcn = @(myobs) [myobs(2)^2; myobs(1); 2*myobs(2)+myobs(1); -myobs(3)]

myBasisFcn =function_handle with value:@(myobs)[myobs(2)^2;myobs(1);2*myobs(2)+myobs(1);-myobs(3)]

The output of the actor is the vectorW'*myBasisFcn(myobs), which is the action taken as a result of the given observation. The weight matrixWcontains the learnable parameters and must have as many rows as the length of the basis function output and as many columns as the dimension of the action space.

Define an initial parameter matrix.

W0 = rand(4,2);

Create the actor. The first argument is a two-element cell containing both the handle to the custom function and the initial weight matrix. The second and third arguments are, respectively, the observation and action specification objects.

actor = rlDeterministicActorRepresentation({myBasisFcn,W0},obsInfo,actInfo)

actor = rlDeterministicActorRepresentation with properties: ActionInfo: [1x1 rl.util.rlNumericSpec] ObservationInfo: [1x1 rl.util.rlNumericSpec] Options: [1x1 rl.option.rlRepresentationOptions]

To check your actor, use thegetActionfunction to return the action from a given observation, using the current parameter matrix.

a = getAction(actor,{[1 2 3]'}); a{1}

ans = 2x1 dlarray 2.0595 2.3788

You can now use the actor (along with an critic) to create a suitable continuous action space agent.

Create observation and action information. You can also obtain these specifications from an environment.

obsinfo = rlNumericSpec([4 1]); actinfo = rlNumericSpec([2 1]); numObs = obsinfo.Dimension(1); numAct = actinfo.Dimension(1);

Create a recurrent deep neural network for the actor. To create a recurrent neural network, use asequenceInputLayeras the input layer and include at least onelstmLayer.

net = [sequenceInputLayer(numObs,'Normalization','none','Name','state')富尔语lyConnectedLayer(10,'Name','fc1') reluLayer('Name','relu1') lstmLayer(8,'OutputMode','sequence','Name','ActorLSTM')富尔语lyConnectedLayer(20,'Name','CriticStateFC2')富尔语lyConnectedLayer(numAct,'Name','action') tanhLayer('Name','tanh1')];

Create a deterministic actor representation for the network.

actorOptions = rlRepresentationOptions(“LearnRate”,1e-3,'GradientThreshold',1); actor = rlDeterministicActorRepresentation(net,obsinfo,actinfo,...'Observation',{'state'},'Action',{'tanh1'});