lstm

Long short-term memory

Since R2019b

collapse all in page

Syntax

Y = lstm(X,H0,C0,weights,recurrentWeights,bias)

[Y,hiddenState,cellState] = lstm(X,H0,C0,weights,recurrentWeights,bias)

[___] = lstm(___,'DataFormat',FMT)

Description

The long short-term memory (LSTM) operation allows a network to learn long-term dependencies between time steps in time series and sequence data.

Note

This function applies the deep learning LSTM operation todlarraydata. If you want to apply an LSTM operation within alayerGraphobject orLayerarray, use the following layer:

lstmLayer

example

Y= lstm(X,H0,C0,weights,recurrentWeights,bias)applies a long short-term memory (LSTM) calculation to inputXusing the initial hidden stateH0, initial cell stateC0, and parametersweights,recurrentWeights, andbias. The inputXmust be a formatteddlarray. The outputYis a formatteddlarraywith the same dimension format asX, except for any'S'dimensions.

Thelstmfunction updates the cell and hidden states using the hyperbolic tangent function (tanh) as the state activation function. Thelstmfunction uses the sigmoid function given by $σ (x) = {(1 + e^{- x})}^{- 1}$ as the gate activation function.

[Y,hiddenState,cellState] = lstm(X,H0,C0,weights,recurrentWeights,bias)also returns the hidden state and cell state after the LSTM operation.

[___] = lstm(___,'DataFormat',FMT)also specifies the dimension formatFMTwhenXis not a formatteddlarray. The outputYis an unformatteddlarraywith the same dimension order asX, except for any'S'dimensions.

Examples

collapse all

Apply LSTM Operation to Sequence Data

Open Live Script

Perform an LSTM operation using three hidden units.

Create the input sequence data as 32 observations with 10 channels and a sequence length of 64

numFeatures = 10; numObservations = 32; sequenceLength = 64; X = randn(numFeatures,numObservations,sequenceLength); dlX = dlarray(X,'CBT');

Create the initial hidden and cell states with three hidden units. Use the same initial hidden state and cell state for all observations.

numHiddenUnits = 3; H0 = zeros(numHiddenUnits,1); C0 = zeros(numHiddenUnits,1);

Create the learnable parameters for the LSTM operation.

weights = dlarray(randn(4*numHiddenUnits,numFeatures),'CU'); recurrentWeights = dlarray(randn(4*numHiddenUnits,numHiddenUnits),'CU'); bias = dlarray(randn(4*numHiddenUnits,1),'C');

Perform the LSTM calculation

[dlY,hiddenState,cellState] = lstm(dlX,H0,C0,weights,recurrentWeights,bias);

View the size and dimensions ofdlY.

size(dlY)

ans =1×33 32 64

dlY.dims

ans = 'CBT'

View the size ofhiddenStateandcellState.

size(hiddenState)

ans =1×23 32

size(cellState)

ans =1×23 32

Check that the outputhiddenStateis the same as the last time step of outputdlY.

ifextractdata(dlY(:,:,end)) == hiddenState disp("The hidden state and the last time step are equal.");elsedisp("The hidden state and the last time step are not equal.")end

The hidden state and the last time step are equal.

You can use the hidden state and cell state to keep track of the state of the LSTM operation and input further sequential data.

Input Arguments

collapse all

`X`—Input data
`dlarray`|numeric array

Input data, specified as a formatteddlarray, an unformatteddlarray,或者一个号码ic array. WhenXis not a formatteddlarray, you must specify the dimension label format using'DataFormat',FMT. IfXis a numeric array, at least one ofH0,C0,weights,recurrentWeights, orbiasmust be adlarray.

Xmust contain a sequence dimension labeled"T". IfXhas any spatial dimensions labeled"S", they are flattened into the"C"channel dimension. IfXdoes not have a channel dimension, then one is added. IfXhas any unspecified dimensions labeled"U", they must be singleton.