장단기기억신경망- MATLAB和Simul金宝appink MathWorks한국gydF4y2Ba - 金宝app,下载188bet金宝搏,金宝搏官方网站

장단기기억신경망gydF4y2Ba

이항목에서는장단기기억(LSTM)신경망을사용하여분류및회귀작업에대해시퀀스및시계열데이터로작업하는방법을설명합니다。LSTM신경망을사용하여시퀀스데이터를분류하는방법을보여주는예제는gydF4y2Ba딥러닝을사용한시퀀스분류gydF4y2Ba항목을참조하십시오。gydF4y2Ba

LSTM신경망은시퀀스데이터의시간스텝간의장기적인종속성을학습할수있는순환신경망(RNN)의일종입니다。gydF4y2Ba

LSTM신경망아키텍처gydF4y2Ba

LSTM신경망의핵심구성요소는시퀀스입력계층과LSTM계층입니다。gydF4y2Ba시퀀스입력계층gydF4y2Ba은신경망에시퀀스또는시계열데이터를입력합니다。gydF4y2BaLSTM계층gydF4y2Ba은시퀀스데이터의시간스텝간의장기종속성을학습합니다。gydF4y2Ba

다음도식은분류를위한간단한LSTM신경망의아키텍처를보여줍니다。이신경망은시퀀스입력계층으로시작하고,그뒤에LSTM계층이옵니다。클래스레이블을예측하기위해,신경망의끝부분에는완전연결계층,소프트맥스계층,분류출력계층이옵니다。gydF4y2Ba

다음도식은회귀를위한간단한LSTM신경망의아키텍처를보여줍니다。이신경망은시퀀스입력계층으로시작하고,그뒤에LSTM계층이옵니다。신경망의끝부분에는완전연결계층과회귀출력계층이옵니다。gydF4y2Ba

다음도식은비디오분류를위한신경망의아키텍처를보여줍니다。신경망에영상시퀀스를입력하려면시퀀스입력계층을사용하십시오。특징추출을위해컨벌루션계층을사용하려면,즉비디오의각프레임에독립적으로컨벌루션연산을적용하려면,시퀀스접기계층뒤에컨벌루션계층을사용하고,그뒤에시퀀스펼치기계층을사용하십시오。벡터로구성된시퀀스에서학습하도록LSTM계층을사용하려면평탄화계층뒤에LSTM계층과출력계층을사용하십시오。gydF4y2Ba

분류LSTM신경망gydF4y2Ba

라이브스크립트열기gydF4y2Ba

sequence-to-label분류를위한LSTM신경망을만들려면시퀀스입력계층,LSTM계층,완전연결계층,소프트맥스계층,분류출력계층을포함하는계층배열을만듭니다。gydF4y2Ba

시퀀스입력계층의크기를입력데이터의특징개수로설정합니다。완전연결계층의크기를클래스개수로설정합니다。시퀀스길이는지정할필요가없습니다。gydF4y2Ba

LSTM계층의경우,은닉유닛의개수와출력모드gydF4y2Ba“最后一次”gydF4y2Ba를지정합니다。gydF4y2Ba

numFeatures = 12;numHiddenUnits = 100;numClasses = 9;层= [gydF4y2Ba．．.gydF4y2BasequenceInputLayer numFeatures lstmLayer (numHiddenUnits,gydF4y2Ba“OutputMode”gydF4y2Ba，gydF4y2Ba“最后一次”gydF4y2Ba) fulllyconnectedlayer (numClasses) softmaxLayer classificationLayer;gydF4y2Ba

sequence-to-label분류를위한LSTM신경망을훈련시키고새데이터를분류하는방법을보여주는예제는gydF4y2Ba딥러닝을사용한시퀀스분류gydF4y2Ba항목을참조하십시오。gydF4y2Ba

sequence-to-sequence분류를위한LSTM신경망을만들려면sequence-to-label분류와동일한아키텍처를사용하되LSTM계층의출력모드를gydF4y2Ba“序列”gydF4y2Ba로설정합니다。gydF4y2Ba

numFeatures = 12;numHiddenUnits = 100;numClasses = 9;层= [gydF4y2Ba．．.gydF4y2BasequenceInputLayer numFeatures lstmLayer (numHiddenUnits,gydF4y2Ba“OutputMode”gydF4y2Ba，gydF4y2Ba“序列”gydF4y2Ba) fulllyconnectedlayer (numClasses) softmaxLayer classificationLayer;gydF4y2Ba

회귀LSTM신경망gydF4y2Ba

라이브스크립트열기gydF4y2Ba

sequence-to-one회귀를위한LSTM신경망을만들려면시퀀스입력계층,LSTM계층,완전연결계층,회귀출력계층을포함하는계층배열을만듭니다。gydF4y2Ba

시퀀스입력계층의크기를입력데이터의특징개수로설정합니다。완전연결계층의크기를응답변수의개수로설정합니다。시퀀스길이는지정할필요가없습니다。gydF4y2Ba

LSTM계층의경우,은닉유닛의개수와출력모드gydF4y2Ba“最后一次”gydF4y2Ba를지정합니다。gydF4y2Ba

numFeatures = 12;numHiddenUnits = 125;numResponses = 1;层= [gydF4y2Ba．．.gydF4y2BasequenceInputLayer numFeatures lstmLayer (numHiddenUnits,gydF4y2Ba“OutputMode”gydF4y2Ba，gydF4y2Ba“最后一次”gydF4y2Ba) fullyConnectedLayer (numResponses) regressionLayer];gydF4y2Ba

sequence-to-sequence회귀를위한LSTM신경망을만들려면sequence-to-one회귀와동일한아키텍처를사용하되LSTM계층의출력모드를gydF4y2Ba“序列”gydF4y2Ba로설정합니다。gydF4y2Ba

numFeatures = 12;numHiddenUnits = 125;numResponses = 1;层= [gydF4y2Ba．．.gydF4y2BasequenceInputLayer numFeatures lstmLayer (numHiddenUnits,gydF4y2Ba“OutputMode”gydF4y2Ba，gydF4y2Ba“序列”gydF4y2Ba) fullyConnectedLayer (numResponses) regressionLayer];gydF4y2Ba

sequence-to-sequence회귀를위한LSTM신경망을훈련시키고새데이터에대해예측하는방법을보여주는예제는gydF4y2Ba딥러닝을사용한sequence-to-sequence회귀gydF4y2Ba항목을참조하십시오。gydF4y2Ba

비디오분류신경망gydF4y2Ba

라이브스크립트열기gydF4y2Ba

비디오데이터나의료영상과같이영상으로구성된시퀀스를포함하는데이터를위한딥러닝신경망을만들려면시퀀스입력계층을사용하여영상시퀀스입력값을지정합니다。gydF4y2Ba

특징추출을위해컨벌루션계층을사용하려면,즉비디오의각프레임에독립적으로컨벌루션연산을적용하려면,시퀀스접기계층뒤에컨벌루션계층을사용하고,그뒤에시퀀스펼치기계층을사용하십시오。벡터로구성된시퀀스에서학습하도록LSTM계층을사용하려면평탄화계층뒤에LSTM계층과출력계층을사용하십시오。gydF4y2Ba

inputSize = [28 28 1];filterSize = 5;numFilters = 20;numHiddenUnits = 200;numClasses = 10;层= [gydF4y2Ba．．.gydF4y2BasequenceInputLayer (inputSizegydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“输入”gydF4y2Ba) sequenceFoldingLayer (gydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“折”gydF4y2Ba) convolution2dLayer (filterSize numFilters,gydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“conv”gydF4y2Ba) batchNormalizationLayer (gydF4y2Ba“名字”gydF4y2Ba，gydF4y2Babn的gydF4y2Ba) reluLayer (gydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“relu”gydF4y2Ba) sequenceUnfoldingLayer (gydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“展开”gydF4y2Ba) flattenLayer (gydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“平”gydF4y2Ba) lstmLayer (numHiddenUnitsgydF4y2Ba“OutputMode”gydF4y2Ba，gydF4y2Ba“最后一次”gydF4y2Ba，gydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“lstm”gydF4y2Ba) fullyConnectedLayer (numClassesgydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“俱乐部”gydF4y2Ba) softmaxLayer (gydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“softmax”gydF4y2Ba) classificationLayer (gydF4y2Ba“名字”gydF4y2Ba，gydF4y2Ba“分类”gydF4y2Ba));gydF4y2Ba

계층을계층그래프로변환하고,시퀀스접기계층의gydF4y2BaminiBatchSizegydF4y2Ba출력값을시퀀스펼치기계층의대응되는입력값에연결합니다。gydF4y2Ba

lgraph = layerGraph(层);lgraph = connectLayers (lgraph,gydF4y2Ba“折/ miniBatchSize”gydF4y2Ba，gydF4y2Ba“展开/ miniBatchSize”gydF4y2Ba)；gydF4y2Ba

딥러닝신경망을비디오분류를위해훈련시키는방법을보여주는예제는gydF4y2Ba딥러닝을사용하여비디오분류하기gydF4y2Ba항목을참조하십시오。gydF4y2Ba

심층LSTM신경망gydF4y2Ba

라이브스크립트열기gydF4y2Ba

LSTM계층앞에출력모드가gydF4y2Ba“序列”gydF4y2Ba인LSTM계층을추가로삽입하여LSTM신경망의심도를높일수있습니다。과적합을방지하기위해LSTM계층뒤에드롭아웃계층을삽입할수있습니다。gydF4y2Ba

sequence-to-label분류신경망의경우,마지막LSTM계층의출력모드가gydF4y2Ba“最后一次”gydF4y2Ba가되어야합니다。gydF4y2Ba

numFeatures = 12;numHiddenUnits1 = 125;numHiddenUnits2 = 100;numClasses = 9;层= [gydF4y2Ba．．.gydF4y2BasequenceInputLayer numFeatures lstmLayer (numHiddenUnits1,gydF4y2Ba“OutputMode”gydF4y2Ba，gydF4y2Ba“序列”gydF4y2Ba) dropoutLayer (0.2) lstmLayer (numHiddenUnits2gydF4y2Ba“OutputMode”gydF4y2Ba，gydF4y2Ba“最后一次”gydF4y2Ba) dropoutLayer(0.2) fulllyconnectedlayer (numClasses) softmaxLayer classificationLayer];gydF4y2Ba

sequence-to-sequence분류신경망의경우,마지막LSTM계층의출력모드가gydF4y2Ba“序列”gydF4y2Ba가되어야합니다。gydF4y2Ba

numFeatures = 12;numHiddenUnits1 = 125;numHiddenUnits2 = 100;numClasses = 9;层= [gydF4y2Ba．．.gydF4y2BasequenceInputLayer numFeatures lstmLayer (numHiddenUnits1,gydF4y2Ba“OutputMode”gydF4y2Ba，gydF4y2Ba“序列”gydF4y2Ba) dropoutLayer (0.2) lstmLayer (numHiddenUnits2gydF4y2Ba“OutputMode”gydF4y2Ba，gydF4y2Ba“序列”gydF4y2Ba) dropoutLayer(0.2) fulllyconnectedlayer (numClasses) softmaxLayer classificationLayer];gydF4y2Ba

계층gydF4y2Ba

계층gydF4y2Ba	설명gydF4y2Ba
`sequenceInputLayergydF4y2Ba`	시퀀스입력계층은신경망에시퀀스데이터를입력합니다。gydF4y2Ba
`lstmLayergydF4y2Ba`	LSTM계층은시계열및시퀀스데이터에서시간스텝간의장기종속성을학습합니다。gydF4y2Ba
`bilstmLayergydF4y2Ba`	BiLSTM(양방향장단기기억)계층은시계열또는시퀀스데이터의시간스텝간의양방향장기종속성을학습합니다。이러한종속성은신경망이각시간스텝에서전체시계열로부터학습하도록하려는경우에유용할수있습니다。gydF4y2Ba
`gruLayergydF4y2Ba`	格勒乌계층은시계열및시퀀스데이터에서시간스텝간의종속성을학습합니다。gydF4y2Ba
`sequenceFoldingLayergydF4y2Ba`	시퀀스접기계층은영상시퀀스배치를영상배치로변환합니다。영상시퀀스의시간스텝에대해독립적으로컨벌루션연산을수행하려는경우시퀀스접기계층을사용하십시오。gydF4y2Ba
`sequenceUnfoldingLayergydF4y2Ba`	시퀀스펼치기계층은시퀀스접기후에입력데이터의시퀀스구조를복원합니다。gydF4y2Ba
`flattenLayergydF4y2Ba`	평탄화계층은입력값의공간차원을채널차원으로축소합니다。gydF4y2Ba
`wordEmbeddingLayergydF4y2Ba`(文本分析工具箱)gydF4y2Ba	단어임베딩계층은단어인덱스를벡터로매핑합니다。gydF4y2Ba

분류,예측및전망gydF4y2Ba

새로운데이터를분류하거나데이터에대해예측을수행하려면gydF4y2Ba分类gydF4y2Ba와gydF4y2Ba预测gydF4y2Ba를사용하십시오。gydF4y2Ba

LSTM신경망은각예측사이의신경망상태를기억할수있습니다。신경망상태는전체시계열을미리갖고있지않거나긴시계열에대해여러예측을수행하려는경우에유용합니다。gydF4y2Ba

시계열의각부분을예측및분류하고신경망상태를업데이트하려면gydF4y2BapredictAndUpdateStategydF4y2Ba와gydF4y2BaclassifyAndUpdateStategydF4y2Ba를사용하십시오。각예측사이의신경망상태를재설정하려면gydF4y2BaresetStategydF4y2Ba를사용하십시오。gydF4y2Ba

시퀀스의미래의시간스텝을전망하는방법을보여주는예제는gydF4y2Ba딥러닝을사용한시계열전망gydF4y2Ba항목을참조하십시오。gydF4y2Ba

시퀀스채우기자,르기및분할gydF4y2Ba

LSTM신경망은다양한시퀀스길이를갖는입력데이터를지원합니다。신경망에데이터를통과시키면각미니배치의모든시퀀스가지정된길이를갖도록시퀀스가채워지거나,잘리거나,분할됩니다。gydF4y2BatrainingOptionsgydF4y2Ba의gydF4y2BaSequenceLengthgydF4y2Ba및gydF4y2BaSequencePaddingValuegydF4y2Ba이름——값쌍인수를사용하여시퀀스길이와시퀀스를채우는데사용되는값을지정할수있습니다。gydF4y2Ba

신경망을훈련시킨후에는gydF4y2Ba分类gydF4y2Ba，gydF4y2Ba预测gydF4y2Ba，gydF4y2BaclassifyAndUpdateStategydF4y2Ba，gydF4y2BapredictAndUpdateStategydF4y2Ba및gydF4y2Ba激活gydF4y2Ba함수를사용할때동일한미니배치크기와채우기옵션을사용하십시오。gydF4y2Ba

시퀀스를길이로정렬하기gydF4y2Ba

시퀀스를채우거나자를때사용되는채우기양이나버려지는데이터를줄이려면시퀀스길이를기준으로데이터를정렬해보십시오。시퀀스길이를기준으로데이터를정렬하려면먼저gydF4y2BacellfungydF4y2Ba을사용하여각시퀀스에gydF4y2Ba大小(X, 2)gydF4y2Ba를적용하여각시퀀스의열개수를가져옵니다。그런다음gydF4y2Ba排序gydF4y2Ba를사용하여시퀀스길이를정렬하고,두번째출력값을사용하여원래시퀀스를재정렬합니다。gydF4y2Ba

sequenceLengths = cellfun(@(X) size(X,2)， XTrain);[sequenceLengthsSorted, idx] = (sequenceLengths)进行排序;XTrain = XTrain (idx);gydF4y2Ba

다음그림은정렬된데이터와정렬되지않은데이터의시퀀스길이를막대차트로보여줍니다。gydF4y2Ba

시퀀스채우기gydF4y2Ba

시퀀스길이를gydF4y2Ba“最长”gydF4y2Ba로지정하면미니배치의모든시퀀스가미니배치에있는가장긴시퀀스와동일한길이를갖도록소프트웨어가시퀀스를채웁니다。이는디폴트옵션입니다。gydF4y2Ba

다음그림은gydF4y2Ba“SequenceLength”gydF4y2Ba를gydF4y2Ba“最长”gydF4y2Ba로설정하는효과를보여줍니다。gydF4y2Ba

시퀀스자르기gydF4y2Ba

시퀀스길이를gydF4y2Ba“最短”gydF4y2Ba로지정하면미니배치의모든시퀀스가미니배치에있는가장짧은시퀀스와동일한길이를갖도록소프트웨어가시퀀스를자릅니다。시퀀스의나머지데이터는버려집니다。gydF4y2Ba

다음그림은gydF4y2Ba“SequenceLength”gydF4y2Ba를gydF4y2Ba“最短”gydF4y2Ba로설정하는효과를보여줍니다。gydF4y2Ba

시퀀스분할하기gydF4y2Ba

시퀀스길이를정수값으로설정하면소프트웨어가미니배치의모든시퀀스를미니배치에있는가장긴시퀀스길이보다큰,지정된길이의가장가까운배수로채웁니다。그런다음소프트웨어가각시퀀스를지정된길이의더작은시퀀스로분할합니다。분할이이루어지면소프트웨어가미니배치를추가로만듭니다。gydF4y2Ba

전체시퀀스가메모리에맞지않으면이옵션을사용하십시오。또는gydF4y2BatrainingOptionsgydF4y2Ba의gydF4y2Ba“MiniBatchSize”gydF4y2Ba옵션을더낮은값으로설정하여미니배치당시퀀스개수를줄여보십시오。gydF4y2Ba

시퀀스길이를양의정수로지정하면소프트웨어가이작은크기의시퀀스를연속적인반복을통해처리합니다。신경망은분할된시퀀스사이에서신경망상태를업데이트합니다。gydF4y2Ba

다음그림은gydF4y2Ba“SequenceLength”gydF4y2Ba5로를설정하는효과를보여줍니다。gydF4y2Ba

채우기방향지정하기gydF4y2Ba

채우기와자르기의위치는훈련,분류및예측정확도에영향을줄수있습니다。gydF4y2BatrainingOptionsgydF4y2Ba의gydF4y2Ba“SequencePaddingDirection”gydF4y2Ba옵션을gydF4y2Ba“左”gydF4y2Ba또는gydF4y2Ba“对”gydF4y2Ba로설정해보고무엇이데이터에가장적합한지살펴보십시오。gydF4y2Ba

LSTM계층은시퀀스데이터를한번에하나의시간스텝씩처리하기때문에계층의gydF4y2BaOutputModegydF4y2Ba속성이gydF4y2Ba“最后一次”gydF4y2Ba인경우마지막시간스텝에서의채우기가계층출력에부정적인영향을줄수있습니다。시퀀스데이터를왼쪽에서채우거나자르려면gydF4y2Ba“SequencePaddingDirection”gydF4y2Ba옵션을gydF4y2Ba“左”gydF4y2Ba로설정하십시오。gydF4y2Ba

sequence-to-sequence신경망의경우(각LSTM계층의gydF4y2BaOutputModegydF4y2Ba속성이gydF4y2Ba“序列”gydF4y2Ba인경우),첫시간스텝들에서의채우기가앞쪽에있는시간스텝들의예측에부정적인영향을줄수있습니다。시퀀스데이터를오른쪽에서채우거나자르려면gydF4y2Ba“SequencePaddingDirection”gydF4y2Ba옵션을gydF4y2Ba“对”gydF4y2Ba로설정하십시오。gydF4y2Ba

다음그림은시퀀스데이터를각각왼쪽과오른쪽에서채운경우를보여줍니다。gydF4y2Ba

다음그림은시퀀스데이터를각각왼쪽과오른쪽에서자른경우를보여줍니다。gydF4y2Ba

시퀀스데이터정규화하기gydF4y2Ba

0이중심이되는정규화를사용하여훈련시점에자동으로훈련데이터의중심을조정하려면gydF4y2BasequenceInputLayergydF4y2Ba의gydF4y2Ba归一化gydF4y2Ba옵션을gydF4y2Ba“zerocenter”gydF4y2Ba로설정하십시오。또는먼저모든시퀀스의특징별평균값과표준편차를계산하여시퀀스데이터를정규화할수도있습니다。그런다음각훈련관측값에서평균값을빼고표준편차로나눕니다。gydF4y2Ba

μ=意味着([XTrain {:}), 2);σ=性病([XTrain {}):, 0, 2);XTrain = cellfun(@(X) (X-mu)./sigma，gydF4y2Ba“UniformOutput”gydF4y2Ba、假);gydF4y2Ba

메모리에담을수없는큰데이터gydF4y2Ba

데이터가너무커서메모리에담을수없거나데이터배치를읽을때특정연산을수행하려면시퀀스,시계열및신호데이터를위한데이터저장소를사용합니다。gydF4y2Ba

자세한내용은gydF4y2Ba메모리에다담을수없는큰시퀀스데이터를사용하여신경망훈련시키기gydF4y2Ba및gydF4y2Ba使用深度学习对内存不足的文本数据进行分类gydF4y2Ba항목을참조하십시오。gydF4y2Ba

시각화gydF4y2Ba

激活gydF4y2Ba함수를사용하여활성화를추출하여,LSTM신경망이시퀀스데이터와시계열데이터에서학습한특징을검사하고시각화합니다。자세한내용은gydF4y2Ba可视化LSTM网络的激活gydF4y2Ba항목을참조하십시오。gydF4y2Ba

LSTM계층아키텍처gydF4y2Ba

다음도식은길이가年代인특징(채널)C개를갖는시계열X가LSTM계층을통과하는흐름을보여줍니다。이도식에서gydF4y2Ba ${hgydF4y2Ba}_{tgydF4y2Ba}$ 와gydF4y2Ba ${cgydF4y2Ba}_{tgydF4y2Ba}$ 는각각시간스텝t에서의출력값(gydF4y2Ba은닉상태gydF4y2Ba라고도과함)gydF4y2Ba셀상태gydF4y2Ba를나타냅니다。gydF4y2Ba

첫번째LSTM블록은신경망의초기상태와시퀀스의첫번째시간스텝을사용하여첫번째출력값과업데이트된셀상태를계산합니다。시간스텝t에서,이블록은신경망gydF4y2Ba $（gydF4y2Ba {cgydF4y2Ba}_{tgydF4y2Ba -gydF4y2Ba 1gydF4y2Ba} ，gydF4y2Ba {hgydF4y2Ba}_{tgydF4y2Ba -gydF4y2Ba 1gydF4y2Ba} ）gydF4y2Ba$ 의현재상태와시퀀스의다음시간스텝을사용하여출력값과업데이트된셀상태gydF4y2Ba ${cgydF4y2Ba}_{tgydF4y2Ba}$ 를계산합니다。gydF4y2Ba

계층의상태는gydF4y2Ba은닉상태gydF4y2Ba（gydF4y2Ba출력상태gydF4y2Ba)와gydF4y2Ba셀상태gydF4y2Ba로구성됩니다。시간스텝t에서의은닉상태는이시간스텝에대한LSTM계층의출력값을포함합니다。셀상태는이전시간스텝에서학습한정보를포함합니다。이계층은각시간스텝에서셀상태에정보를추가하거나셀상태로부터정보를제거합니다。계층은gydF4y2Ba게이트gydF4y2Ba를사용하여이러한업데이트를제어합니다。gydF4y2Ba

다음구성요소는계층의셀상태와은닉상태를제어합니다。gydF4y2Ba

구성요소gydF4y2Ba	목적gydF4y2Ba
입력게이트(我)gydF4y2Ba	셀상태업데이트의수준제어gydF4y2Ba
망각게이트(f)gydF4y2Ba	셀상태재설정(망각)의수준제어gydF4y2Ba
셀후보(g)gydF4y2Ba	셀상태에정보추가gydF4y2Ba
출력게이트(o)gydF4y2Ba	은닉상태에추가되는셀상태의수준제어gydF4y2Ba

다음도식은시간스텝t에서의데이터흐름을보여줍니다。이도식은게이트가셀상태와은닉상태를망각,업데이트,출력하는방식을보여줍니다。gydF4y2Ba

LSTM계층의학습가능한가중치는입력가중치W (gydF4y2BaInputWeightsgydF4y2Ba),순환가중치R (gydF4y2BaRecurrentWeightsgydF4y2Ba),편향b (gydF4y2Ba偏见gydF4y2Ba)입니다。행렬W, R、b는각각각구성요소의입력가중치결합,순환가중치결합,편향결합입니다。이러한행렬은다음과같이결합됩니다。gydF4y2Ba

$WgydF4y2Ba ＝gydF4y2Ba ［gydF4y2Ba \begin{matrix} {WgydF4y2Ba}_{我gydF4y2Ba} \\ {WgydF4y2Ba}_{fgydF4y2Ba} \\ {WgydF4y2Ba}_{ggydF4y2Ba} \\ {WgydF4y2Ba}_{ogydF4y2Ba} \end{matrix} ］gydF4y2Ba ，gydF4y2Ba RgydF4y2Ba ＝gydF4y2Ba ［gydF4y2Ba \begin{matrix} {RgydF4y2Ba}_{我gydF4y2Ba} \\ {RgydF4y2Ba}_{fgydF4y2Ba} \\ {RgydF4y2Ba}_{ggydF4y2Ba} \\ {RgydF4y2Ba}_{ogydF4y2Ba} \end{matrix} ］gydF4y2Ba ，gydF4y2Ba bgydF4y2Ba ＝gydF4y2Ba ［gydF4y2Ba \begin{matrix} {bgydF4y2Ba}_{我gydF4y2Ba} \\ {bgydF4y2Ba}_{fgydF4y2Ba} \\ {bgydF4y2Ba}_{ggydF4y2Ba} \\ {bgydF4y2Ba}_{ogydF4y2Ba} \end{matrix} ］gydF4y2Ba ，gydF4y2Ba$

여기서,f, g, o는입력게이트,망각게이트,셀후보,출력게이트를나타냅니다。gydF4y2Ba

시간스텝t에서의셀상태는다음과같이표현됩니다。gydF4y2Ba

${cgydF4y2Ba}_{tgydF4y2Ba} ＝gydF4y2Ba {fgydF4y2Ba}_{tgydF4y2Ba} ⊙gydF4y2Ba {cgydF4y2Ba}_{tgydF4y2Ba -gydF4y2Ba 1gydF4y2Ba} +gydF4y2Ba {我gydF4y2Ba}_{tgydF4y2Ba} ⊙gydF4y2Ba {ggydF4y2Ba}_{tgydF4y2Ba} ，gydF4y2Ba$

여기서gydF4y2Ba $⊙gydF4y2Ba$ 는아다마르곱(벡터의요소별곱셈)을나타냅니다。gydF4y2Ba

시간스텝t에서의은닉상태는다음과같이표현됩니다。gydF4y2Ba

${hgydF4y2Ba}_{tgydF4y2Ba} ＝gydF4y2Ba {ogydF4y2Ba}_{tgydF4y2Ba} ⊙gydF4y2Ba {σgydF4y2Ba}_{cgydF4y2Ba} （gydF4y2Ba {cgydF4y2Ba}_{tgydF4y2Ba} ）gydF4y2Ba ，gydF4y2Ba$

여기서gydF4y2Ba ${σgydF4y2Ba}_{cgydF4y2Ba}$ 는상태활성화함수를나타냅니다。gydF4y2BalstmLayergydF4y2Ba함수는기본적으로쌍곡탄젠트함수(双曲正切)를사용하여상태활성화함수를계산합니다。gydF4y2Ba

다음수식은시간스텝t에서의구성요소를설명합니다。gydF4y2Ba

구성요소gydF4y2Ba	식gydF4y2Ba
입력게이트gydF4y2Ba	${我gydF4y2Ba}_{tgydF4y2Ba} ＝gydF4y2Ba {σgydF4y2Ba}_{ggydF4y2Ba} （gydF4y2Ba {WgydF4y2Ba}_{我gydF4y2Ba} {xgydF4y2Ba}_{tgydF4y2Ba} +gydF4y2Ba {RgydF4y2Ba}_{我gydF4y2Ba} {hgydF4y2Ba}_{tgydF4y2Ba -gydF4y2Ba 1gydF4y2Ba} +gydF4y2Ba {bgydF4y2Ba}_{我gydF4y2Ba} ）gydF4y2Ba$
망각게이트gydF4y2Ba	${fgydF4y2Ba}_{tgydF4y2Ba} ＝gydF4y2Ba {σgydF4y2Ba}_{ggydF4y2Ba} （gydF4y2Ba {WgydF4y2Ba}_{fgydF4y2Ba} {xgydF4y2Ba}_{tgydF4y2Ba} +gydF4y2Ba {RgydF4y2Ba}_{fgydF4y2Ba} {hgydF4y2Ba}_{tgydF4y2Ba -gydF4y2Ba 1gydF4y2Ba} +gydF4y2Ba {bgydF4y2Ba}_{fgydF4y2Ba} ）gydF4y2Ba$
셀후보gydF4y2Ba	${ggydF4y2Ba}_{tgydF4y2Ba} ＝gydF4y2Ba {σgydF4y2Ba}_{cgydF4y2Ba} （gydF4y2Ba {WgydF4y2Ba}_{ggydF4y2Ba} {xgydF4y2Ba}_{tgydF4y2Ba} +gydF4y2Ba {RgydF4y2Ba}_{ggydF4y2Ba} {hgydF4y2Ba}_{tgydF4y2Ba -gydF4y2Ba 1gydF4y2Ba} +gydF4y2Ba {bgydF4y2Ba}_{ggydF4y2Ba} ）gydF4y2Ba$
출력게이트gydF4y2Ba	${ogydF4y2Ba}_{tgydF4y2Ba} ＝gydF4y2Ba {σgydF4y2Ba}_{ggydF4y2Ba} （gydF4y2Ba {WgydF4y2Ba}_{ogydF4y2Ba} {xgydF4y2Ba}_{tgydF4y2Ba} +gydF4y2Ba {RgydF4y2Ba}_{ogydF4y2Ba} {hgydF4y2Ba}_{tgydF4y2Ba -gydF4y2Ba 1gydF4y2Ba} +gydF4y2Ba {bgydF4y2Ba}_{ogydF4y2Ba} ）gydF4y2Ba$

위식에서gydF4y2Ba ${σgydF4y2Ba}_{ggydF4y2Ba}$ 는게이트활성화함수를나타냅니다。gydF4y2BalstmLayergydF4y2Ba함수는기본적으로gydF4y2Ba $σgydF4y2Ba （gydF4y2Ba xgydF4y2Ba ）gydF4y2Ba ＝gydF4y2Ba {（gydF4y2Ba 1gydF4y2Ba +gydF4y2Ba {egydF4y2Ba}^{-gydF4y2Ba xgydF4y2Ba} ）gydF4y2Ba}^{-gydF4y2Ba 1gydF4y2Ba}$ 으로표현되는시그모이드함수를사용하여게이트활성화함수를계산합니다。gydF4y2Ba

참고문헌gydF4y2Ba

S. Hochreiter和J. Schmidhuber。“长短期记忆。”神经计算。1997年第9卷第8期，第1735 - 1780页。gydF4y2Ba

참고항목gydF4y2Ba

장단기기억신경망gydF4y2Ba

LSTM신경망아키텍처gydF4y2Ba

분류LSTM신경망gydF4y2Ba

회귀LSTM신경망gydF4y2Ba

비디오분류신경망gydF4y2Ba

심층LSTM신경망gydF4y2Ba

계층gydF4y2Ba

분류,예측및전망gydF4y2Ba

시퀀스채우기자,르기및분할gydF4y2Ba

시퀀스를길이로정렬하기gydF4y2Ba

시퀀스채우기gydF4y2Ba

시퀀스자르기gydF4y2Ba

시퀀스분할하기gydF4y2Ba

채우기방향지정하기gydF4y2Ba

시퀀스데이터정규화하기gydF4y2Ba

메모리에담을수없는큰데이터gydF4y2Ba

시각화gydF4y2Ba

LSTM계층아키텍처gydF4y2Ba

참고문헌gydF4y2Ba

참고항목gydF4y2Ba

관련항목gydF4y2Ba

深度学习工具箱문서gydF4y2Ba

지원gydF4y2Ba

介绍基于MATLAB的深度学习gydF4y2Ba