crossChannelNormalizationLayer

Channel-wise local response normalization layer

expand all in page

Description

A channel-wise local response (cross-channel) normalization layer carries out channel-wise normalization.

Creation

Syntax

layer = crossChannelNormalizationLayer(windowChannelSize)

layer = crossChannelNormalizationLayer(windowChannelSize,Name,Value)

Description

layer= crossChannelNormalizationLayer(windowChannelSize)creates a channel-wise local response normalization layer and sets theWindowChannelSizeproperty.

example

layer= crossChannelNormalizationLayer(windowChannelSize,Name,Value)sets the optional propertiesWindowChannelSize,Alpha,Beta,K, andNameusing name-value pairs. For example,crossChannelNormalizationLayer(5,'K',1)creates a local response normalization layer for channel-wise normalization with a window size of 5 andKhyperparameter 1. You can specify multiple name-value pairs. Enclose each property name in single quotes.

Properties

expand all

Cross-Channel Normalization

`WindowChannelSize`—Size of the channel window
positive integer

Size of the channel window, which controls the number of channels that are used for the normalization of each element, specified as a positive integer.

IfWindowChannelSizeis even, then the window is asymmetric. The software looks at the previousfloor((w-1)/2)channels and the followingfloor(w/2)channels. For example, ifWindowChannelSizeis 4, then the layer normalizes each element by its neighbor in the previous channel and by its neighbors in the next two channels.

Example:5

`Alpha`—αhyperparameter in normalization
0.0001(default) |numeric scalar

αhyperparameter in the normalization (the multiplier term), specified as a numeric scalar.

Example:0.0002

`Beta`—βhyperparameter in normalization
0.75(default) |numeric scalar

βhyperparameter in the normalization, specified as a numeric scalar. The value ofBetamust be greater than or equal to 0.01.

Example:0.8

`K`—Khyperparameter in the normalization
2(default) |numeric scalar

Khyperparameter in the normalization, specified as a numeric scalar. The value ofKmust be greater than or equal to 10^-5.

Example:2.5

Layer

`Name`—Layer name
`''`(default) |character vector|string scalar

Layer name, specified as a character vector or a string scalar. ForLayerarray input, thetrainNetwork,assembleNetwork,layerGraph, anddlnetworkfunctions automatically assign names to layers with name''.

Data Types:char|string

`全国矿工工会Inputs`—全国矿工工会ber of inputs
`1`(default)

This property is read-only.

全国矿工工会ber of inputs of the layer. This layer accepts a single input only.

Data Types:double

`InputNames`—Input names
`{'in'}`(default)

This property is read-only.

Input names of the layer. This layer accepts a single input only.

Data Types:cell

`全国矿工工会Outputs`—全国矿工工会ber of outputs
`1`(default)

This property is read-only.

全国矿工工会ber of outputs of the layer. This layer has a single output only.

Data Types:double

`OutputNames`—Output names
`{'out'}`(default)

This property is read-only.

Output names of the layer. This layer has a single output only.

Data Types:cell

Examples

collapse all

创建本地反应正常化层

Open Live Script

Create a local response normalization layer for channel-wise normalization, where a window of five channels normalizes each element, and the additive constant for the normalizer $K$ is 1.

layer = crossChannelNormalizationLayer(5,'K',1)

layer = CrossChannelNormalizationLayer with properties: Name: '' Hyperparameters WindowChannelSize: 5 Alpha: 1.0000e-04 Beta: 0.7500 K: 1

Include a local response normalization layer in aLayerarray.

layers = [...imageInputLayer([28 28 1]) convolution2dLayer(5,20) reluLayer crossChannelNormalizationLayer(3) fullyConnectedLayer(10) softmaxLayer classificationLayer]

layers = 7x1 Layer array with layers: 1 '' Image Input 28x28x1 images with 'zerocenter' normalization 2 '' Convolution 20 5x5 convolutions with stride [1 1] and padding [0 0 0 0] 3 '' ReLU ReLU 4 '' Cross Channel Normalization cross channel normalization with 3 channels per element 5 '' Fully Connected 10 fully connected layer 6 '' Softmax softmax 7 '' Classification Output crossentropyex

Limitations

This layer does not support 3-D image inputs or vector sequence inputs.

More About

expand all

Local Response Normalization

A channel-wise local response (cross-channel) normalization layer carries out channel-wise normalization.

This layer performs a channel-wise local response normalization. It usually follows the ReLU activation layer. This layer replaces each element with a normalized value it obtains using the elements from a certain number of neighboring channels (elements in the normalization window). That is, for each element $x$ in the input,trainNetworkcomputes a normalized value $x^{'}$ using

$x^{'} = \frac{x}{{(K + \frac{α * s s}{w i n d o w C h a n n e l S i z e})}^{β}},$

whereK,α, andβare the hyperparameters in the normalization, andssis the sum of squares of the elements in the normalization window[1]. You must specify the size of the normalization window using thewindowChannelSizeargument of thecrossChannelNormalizationLayerfunction. You can also specify the hyperparameters using theAlpha,Beta, andKname-value pair arguments.

The previous normalization formula is slightly different than what is presented in[1]. You can obtain the equivalent formula by multiplying thealphavalue by thewindowChannelSize.

References

[1] Krizhevsky, A., I. Sutskever, and G. E. Hinton. "ImageNet Classification with Deep Convolutional Neural Networks."Advances in Neural Information Processing Systems. Vol 25, 2012.

crossChannelNormalizationLayer

Description

Creation

Syntax

Description

Properties

Cross-Channel Normalization

`WindowChannelSize`—Size of the channel window
positive integer

`Alpha`—αhyperparameter in normalization
0.0001(default) |numeric scalar

`Beta`—βhyperparameter in normalization
0.75(default) |numeric scalar

`K`—Khyperparameter in the normalization
2(default) |numeric scalar

Layer

`Name`—Layer name
`''`(default) |character vector|string scalar

`全国矿工工会Inputs`—全国矿工工会ber of inputs
`1`(default)

`InputNames`—Input names
`{'in'}`(default)

`全国矿工工会Outputs`—全国矿工工会ber of outputs
`1`(default)

`OutputNames`—Output names
`{'out'}`(default)

Examples

创建本地反应正常化层

Limitations

More About

Local Response Normalization

References

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

GPU Code Generation
Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Version History

See Also

Topics

crossChannelNormalizationLayer

Description

Creation

Syntax

Description

Properties

Cross-Channel Normalization

WindowChannelSize—Size of the channel windowpositive integer

Alpha—αhyperparameter in normalization0.0001(default) |numeric scalar

Beta—βhyperparameter in normalization0.75(default) |numeric scalar

K—Khyperparameter in the normalization2(default) |numeric scalar

Layer

Name—Layer name''(default) |character vector|string scalar

全国矿工工会Inputs—全国矿工工会ber of inputs1(default)

InputNames—Input names{'in'}(default)

全国矿工工会Outputs—全国矿工工会ber of outputs1(default)

OutputNames—Output names{'out'}(default)

Examples

创建本地反应正常化层

Limitations

More About

Local Response Normalization

References

Extended Capabilities

C/C++ Code GenerationGenerate C and C++ code using MATLAB® Coder™.

GPU Code GenerationGenerate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Version History

See Also

Topics

`WindowChannelSize`—Size of the channel window
positive integer

`Alpha`—αhyperparameter in normalization
0.0001(default) |numeric scalar

`Beta`—βhyperparameter in normalization
0.75(default) |numeric scalar

`K`—Khyperparameter in the normalization
2(default) |numeric scalar

`Name`—Layer name
`''`(default) |character vector|string scalar

`全国矿工工会Inputs`—全国矿工工会ber of inputs
`1`(default)

`InputNames`—Input names
`{'in'}`(default)

`全国矿工工会Outputs`—全国矿工工会ber of outputs
`1`(default)

`OutputNames`—Output names
`{'out'}`(default)

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

GPU Code Generation
Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.