WordembeddingLayer.

用于深度学习网络的单词嵌入层

展开全部in page

描述

嵌入层映射到向量中的Word Indices。

使用一个词嵌入层深度学习长short-term memory (LSTM) network. An LSTM network is a type of recurrent neural network (RNN) that can learn long-term dependencies between time steps of sequence data. A word embedding layer maps a sequence of word indices to embedding vectors and learns the word embedding during training.

This layer requires Deep Learning Toolbox™.

Creation

Syntax

tillay = wordembeddinglayer（维度，numwords）
tillay = wordembeddinglayer（维度，num字，名称，值）

描述

example

层= wordEmbeddingLayer(尺寸,numWords)创建一个单词嵌入层，并指定嵌入的维度和词汇量大小。

example

层= wordEmbeddingLayer(尺寸,numWords,名称，价值)sets optionalpropertiesusing one or more name-value pairs. Enclose each property name in single quotes.

Properties

展开全部

单词嵌入

尺寸of the word embedding, specified as a positive integer.

例：300

模型中的单词数指定为正整数。如果培训数据中的唯一单词的数量大于NumWords然后，该图层将失控的单词映射到同一矢量。

Parameters and Initialization

初始化权重的功能，指定为以下内容之一：

  • '窄正常'- 通过独立从正常分布采样，零平均值和标准偏差0.01来初始化权重。

  • 'glorot'– Initialize the weights with the Glorot initializer[1](also known as Xavier initializer). The Glorot initializer independently samples from a uniform distribution with zero mean and variance2/(numIn + numOut)，在哪里numin = numwords + 1和numOut = Dimension。

  • '他'– Initialize the weights with the He initializer[2]。The He initializer samples from a normal distribution with zero mean and variance2/numIn，在哪里numin = numwords + 1。

  • 'orthogonal'– Initialize the input weights withQ, the orthogonal matrix given by the QR decomposition ofZ=QR对于随机矩阵Z从单位正态分布中取样。[3]

  • 'zeros'- 用零初始化权重。

  • '那些'– Initialize the weights with ones.

  • 功能句柄 - 使用自定义功能初始化权重。如果指定函数句柄，则函数必须是表单weights = func(sz)，在哪里sz是重量的大小。

该图层仅初始化权重Weightsproperty is empty.

Data Types:char|string|function_handle

层重量, specified as a尺寸-通过-（NumWords+1）阵列。

For input integersi小于或等于NumWords，该图层输出向量Weights(:,i)。否则，图层映射输出向量Weights(:,NumWords+1)。

Learn Rate and Regularization

学习权重的因素，指定为非负标量。

该软件通过全局学习速率乘以此因素来确定该层中权重的学习率。例如，如果WeightLearnRateFactor是2，那么该层中权重的学习率是当前全球学习率的两倍。该软件根据指定的设置确定全局学习速率培训选项(Deep Learning Toolbox)function.

例：2

L2 regularization factor for the weights, specified as a nonnegative scalar.

The software multiplies this factor by the global L2 regularization factor to determine the L2 regularization for the weights in this layer. For example, if掌权2factor.is 2, then the L2 regularization for the weights in this layer is twice the global L2 regularization factor. You can specify the global L2 regularization factor using the培训选项(Deep Learning Toolbox)function.

例：2

层

图层名称，指定为字符向量或字符串标量。如果Name被设置为'', then the software automatically assigns a name at training time.

Data Types:char|string

图层的输入数。此图层仅接受单个输入。

Data Types:双

图层的输入名称。此图层仅接受单个输入。

Data Types:细胞

图层的输出数。此图层仅具有单个输出。

Data Types:双

图层的输出名称。此图层仅具有单个输出。

Data Types:细胞

例子

全部收缩

此示例使用：

打开直播脚本

Create a word embedding layer with embedding dimension 300 and 5000 words.

第= WordEmbeddingLayer（300,5000）
层= WordEmbeddingLayer with properties: Name: '' Hyperparameters Dimension: 300 NumWords: 5000 Learnable Parameters Weights: [] Show all properties

在clude a word embedding layer in an LSTM network.

inputSize = 1; embeddingDimension = 300; numWords = 5000; numHiddenUnits = 200; numClasses = 10; layers = [ sequenceInputLayer(inputSize) wordEmbeddingLayer(embeddingDimension,numWords) lstmLayer(numHiddenUnits,'OutputMode','last') fullyConnectedLayer(numClasses) softmaxLayer classificationLayer]
图层= 6x1层阵列，图层：1''序列输入序列输入用1尺寸2'字嵌入层字嵌入层，具有300尺寸和5000个独特单词3'LSTM LSTM，具有200个隐藏单元4''完全连接的10个完全连接的10连接的第5层'Softmax Softmax 6''分类输出crossentropyex

此示例使用：

打开直播脚本

To initialize a word embedding layer in a deep learning network with the weights from a pretrained word embedding, use theWord2vec.提取层权重的功能并设置'Weights'名称 - 值对WordembeddingLayer.function. The word embedding layer expects columns of word vectors, so you must transpose the output of theWord2vec.function.

emb = fasttextwordembedding;单词= emb.vocabulary;尺寸= emb.dimension;numwords = numel（单词）;tallay = wordembeddinglayer（维度，numword，。。。'Weights',word2vec(emb,words)')
Layer = WordembeddingLayer具有属性：名称：''''''''QuideParameters维数：300 NUM字：999994学习参数权重：[300×999994单个]显示所有属性

To create the corresponding word encoding from the word embedding, input the word embedding vocabulary to thewordEncoding用作单词列表。

enc = wordEncoding(words)
enc = wordencoding属性：numwords：999994词汇表：[1×999994字符串]

参考资料

[1] Glorot，Xavier和Yoshua Bengio。“了解训练深馈神经网络的难度。”在第十三国际人工智能和统计国际会议的诉讼程序, pp. 249-256. 2010.

[2] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification." InIEEE计算机愿景国际会议的诉讼程序, pp. 1026-1034. 2015.

[3]萨克斯,安德鲁·M。詹姆斯·l·麦克勒兰德和Surya Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks."arXiv preprint arXiv:1312.6120(2013).

Extended Capabilities

GPU Code Generation
Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

See Also

||||||(Deep Learning Toolbox)|(Deep Learning Toolbox)|(Deep Learning Toolbox)

Topics

在troduced in R2018b

文本分析工具箱Documentation

金宝app

Getting Started with Text Analytics in MATLAB

Getting Started with Text Analytics in MATLAB

Download now