Check Custom Layer Validity

If you create a custom deep learning layer, then you can use thecheckLayerfunction to check that the layer is valid. The function checks layers for validity, GPU compatibility, correctly defined gradients, and code generation compatibility. To check that a layer is valid, run the following command:

checkLayer(layer,validInputSize)

wherelayeris an instance of the layer,validInputSizeis a vector or cell array specifying the valid input sizes to the layer. To check with multiple observations, use theObservationDimension选择。To check for code generation compatibility, set theCheckCodegenCompatibilityoption to1(true).For large input sizes, the gradient checks take longer to run. To speed up the tests, specify a smaller valid input size.

Check Custom Layer Validity

Open Live Script

Check the validity of the example custom layerpreluLayer.

The custom layerpreluLayer, attached to this is example as a supporting file, applies the PReLU operation to the input data. To access this layer, open this example as a live script.

Create an instance of the layer and check that it is valid usingcheckLayer. Set the valid input size to the typical size of a single observation input to the layer. For a single input, the layer expects observations of sizeh-by-w-by-c, whereh,w, andcare the height, width, and number of channels of the previous layer output, respectively.

SpecifyvalidInputSizeas the typical size of an input array.

layer = preluLayer(20); validInputSize = [5 5 20]; checkLayer(layer,validInputSize)

Skipping multi-observation tests. To enable tests with multiple observations, specify the 'ObservationDimension' option. For 2-D image data, set 'ObservationDimension' to 4. For 3-D image data, set 'ObservationDimension' to 5. For sequence data, set 'ObservationDimension' to 2. Skipping GPU tests. No compatible GPU device found. Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options. Running nnet.checklayer.TestLayerWithoutBackward .......... .. Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 12 Passed, 0 Failed, 0 Incomplete, 16 Skipped. Time elapsed: 0.18741 seconds.

The results show the number of passed, failed, and skipped tests. If you do not specify theObservationsDimensionoption, or do not have a GPU, then the function skips the corresponding tests.

Check Multiple Observations

For multi-observation input, the layer expects an array of observations of sizeh-by-w-by-c-by-N, whereh,w, andcare the height, width, and number of channels, respectively, andNis the number of observations.

To check the layer validity for multiple observations, specify the typical size of an observation and set theObservationDimensionoption to 4.

layer = preluLayer(20); validInputSize = [5 5 20]; checkLayer(layer,validInputSize,ObservationDimension=4)

Skipping GPU tests. No compatible GPU device found. Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options. Running nnet.checklayer.TestLayerWithoutBackward .......... ........ Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 18 Passed, 0 Failed, 0 Incomplete, 10 Skipped. Time elapsed: 0.19972 seconds.

In this case, the function does not detect any issues with the layer.

List of Tests

ThecheckLayerfunction checks the validity of a custom layer by performing a series of tests.

Intermediate Layers

ThecheckLayerfunction uses these tests to check the validity of custom intermediate layers (layers of typennet.layer.Layer).

Test	Description
`functionSyntaxesAreCorrect`	The syntaxes of the layer functions are correctly defined.
`predictDoesNotError`	`predict`function does not error.
`forwardDoesNotError`	When specified, the`forward`function does not error.
`forwardPredictAreConsistentInSize`	When`forward`is specified,`forward`and`predict`output values of the same size.
`backwardDoesNotError`	When specified,`backward`does not error.
`backwardIsConsistentInSize`	When`backward`is specified, the outputs of`backward`are consistent in size: The derivatives with respect to each input are the same size as the corresponding input. The derivatives with respect to each learnable parameter are the same size as the corresponding learnable parameter.
`predictIsConsistentInType`	The outputs of`predict`are consistent in type with the inputs.
`forwardIsConsistentInType`	When`forward`is specified, the outputs of`forward`are consistent in type with the inputs.
`backwardIsConsistentInType`	When`backward`is specified, the outputs of`backward`are consistent in type with the inputs.
`gradientsAreNumericallyCorrect`	When`backward`is specified, the gradients computed in`backward`are consistent with the numerical gradients.
`backwardPropagationDoesNotError`	When`backward`is not specified, the derivatives can be computed using automatic differentiation.
`predictReturnsValidStates`	For layers with state properties, the`predict`function returns valid states.
`forwardReturnsValidStates`	For layers with state properties, the`forward`function, if specified, returns valid states.
`resetStateDoesNotError`	For layers with state properties, the`resetState`function, if specified, does not error and resets the states to valid states.
`codegenPragmaDefinedInClassDef`	The pragma`"%#codegen"`for code generation is specified in class file.
`checkForSupportedLayerPropertiesForCodegen`	The layer properties support code generation.
`predictIsValidForCodeGeneration`	`predict`is valid for code generation.
`doesNotHaveStateProperties`	For code generation, the layer does not have state properties.
`supportedFunctionLayer`	For code generation, the layer is not a`FunctionLayer`object.

Some tests run multiple times. These tests also check different data types and for GPU compatibility:

predictIsConsistentInType
forwardIsConsistentInType
backwardIsConsistentInType

To execute the layer functions on a GPU, the functions must support inputs and outputs of typegpuArray与底层数据类型single.

Output Layers

ThecheckLayerfunction uses these tests to check the validity of custom output layers (layers of typennet.layer.ClassificationLayerornnet.layer.RegressionLayer).

Test	Description
`forwardLossDoesNotError`	`forwardLoss`does not error.
`backwardLossDoesNotError`	`backwardLoss`does not error.
`forwardLossIsScalar`	The output of`forwardLoss`is scalar.
`backwardLossIsConsistentInSize`	When`backwardLoss`is specified, the output of`backwardLoss`is consistent in size:`dLdY`is the same size as the predictions`Y`.
`forwardLossIsConsistentInType`	The output of`forwardLoss`is consistent in type:`loss`is the same type as the predictions`Y`.
`backwardLossIsConsistentInType`	When`backwardLoss`is specified, the output of`backwardLoss`is consistent in type:`dLdY`must be the same type as the predictions`Y`.
`gradientsAreNumericallyCorrect`	When`backwardLoss`is specified, the gradients computed in`backwardLoss`are numerically correct.
`backwardPropagationDoesNotError`	When`backwardLoss`is not specified, the derivatives can be computed using automatic differentiation.

TheforwardLossIsConsistentInTypeandbackwardLossIsConsistentInTypetests also check for GPU compatibility. To execute the layer functions on a GPU, the functions must support inputs and outputs of typegpuArray与底层数据类型single.

Generated Data

To check the layer validity, thecheckLayerfunction generates data depending on the type of layer:

Layer Type Description of Generated Data

Intermediate Values in the range [-1,1]

回归输出 Predictions and targets with values in the range [-1,1]

Classification output

Layer Type	Description of Generated Data
Intermediate	Values in the range [-1,1]
回归输出	Predictions and targets with values in the range [-1,1]
Classification output	Predictions with values in the range [0,1]. 如果你年代pecify the`ObservationDimension`option, then the targets are one-hot encoded vectors (vectors containing a single 1, and 0 elsewhere). If you do not specify the`ObservationDimension`option, then the targets are values in the range [0,1].

Predictions with values in the range [0,1].

如果你年代pecify theObservationDimensionoption, then the targets are one-hot encoded vectors (vectors containing a single 1, and 0 elsewhere).

If you do not specify theObservationDimensionoption, then the targets are values in the range [0,1].

To check for multiple observations, specify the observation dimension using theObservationDimension选择。如果你年代pecify the observation dimension, then thecheckLayerfunction checks that the layer functions are valid using generated data with mini-batches of size 1 and 2. If you do not specify this name-value pair, then the function skips the tests that check that the layer functions are valid for multiple observations.

Diagnostics

If a test fails when you usecheckLayer, then the function provides a test diagnostic and a framework diagnostic. The test diagnostic highlights any issues found with the layer. The framework diagnostic provides more detailed information.

Function Syntaxes

The testfunctionSyntaxesAreCorrectchecks that the layer functions have correctly defined syntaxes.

Test Diagnostic	Description	Possible Solution
`Incorrect number of input arguments for 'predict' in Layer`.	The syntax for the`predict`function is not consistent with the number of layer inputs.	Specify the correct number of input and output arguments in`predict`. The`predict`function syntax depends on the type of layer. `Z = predict(layer,X)`forwards the input data`X`through the layer and outputs the result`Z`, where`layer`has a single input, a single output. `[Z,state] = predict(layer,X)`also outputs the updated state parameter`state`, where`layer`has a single state parameter. You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters: For layers with multiple inputs, replace`X`with`X1,...,XN`, where`N`is the number of inputs. The`NumInputs`property must match`N`. For layers with multiple outputs, replace`Z`with`Z1,...,ZM`, where`M`is the number of outputs. The`NumOutputs`property must match`M`. For layers with multiple state parameters, replace`state`with`state1,...,stateK`, where`K`is the number of state parameters. Tip If the number of inputs to the layer can vary, then use`varargin`instead of`X1,…,XN`. In this case,`varargin`是一个单元阵列的输入,在哪里`varargin{i}`corresponds to`Xi`. If the number of outputs can vary, then use`varargout`instead of`Z1,…,ZN`. In this case,`varargout`is a cell array of the outputs, where`varargout{j}`corresponds to`Zj`. Tip If the custom layer has a`dlnetwork`object for a learnable parameter, then in the`predict`function of the custom layer, use the`predict`function for the`dlnetwork`. Using the`dlnetwork`object`predict`function ensures that the software uses the correct layer operations for prediction.
`Incorrect number of output arguments for 'predict' in Layer`	The syntax for the`predict`function is not consistent with the number of layer outputs.
`Incorrect number of input arguments for 'forward' in Layer`	The syntax for the optional`forward`function is not consistent with the number of layer inputs.	Specify the correct number of input and output arguments in`forward`. The`forward`function syntax depends on the type of layer: `Z = forward(layer,X)`forwards the input data`X`through the layer and outputs the result`Z`, where`layer`has a single input, a single output. `[Z,state] = forward(layer,X)`also outputs the updated state parameter`state`, where`layer`has a single state parameter. `[__,memory] = forward(layer,X)`also returns a memory value for a custom`backward`function using any of the previous syntaxes. If the layer has both a custom`forward`function and a custom`backward`function, then the forward function must return a memory value. You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters: For layers with multiple inputs, replace`X`with`X1,...,XN`, where`N`is the number of inputs. The`NumInputs`property must match`N`. For layers with multiple outputs, replace`Z`with`Z1,...,ZM`, where`M`is the number of outputs. The`NumOutputs`property must match`M`. For layers with multiple state parameters, replace`state`with`state1,...,stateK`, where`K`is the number of state parameters. Tip If the number of inputs to the layer can vary, then use`varargin`instead of`X1,…,XN`. In this case,`varargin`是一个单元阵列的输入,在哪里`varargin{i}`corresponds to`Xi`. If the number of outputs can vary, then use`varargout`instead of`Z1,…,ZN`. In this case,`varargout`is a cell array of the outputs, where`varargout{j}`corresponds to`Zj`. Tip If the custom layer has a`dlnetwork`object for a learnable parameter, then in the`forward`function of the custom layer, use the`forward`function of the`dlnetwork`object. Using the`dlnetwork`object`forward`function ensures that the software uses the correct layer operations for training.
`Incorrect number of output arguments for 'forward' in Layer`	The syntax for the optional`forward`function is not consistent with the number of layer outputs.
`Incorrect number of input arguments for 'backward' in Layer`	The syntax for the optional`backward`function is not consistent with the number of layer inputs and outputs.	Specify the correct number of input and output arguments in`backward`. The`backward`function syntax depends on the type of layer. `dLdX = backward(layer,X,Z,dLdZ,memory)`returns the derivatives`dLdX`of the loss with respect to the layer input, where`layer`has a single input and a single output.`Z`corresponds to the forward function output and`dLdZ`对应的导数与res损失pect to`Z`. The function input`memory`corresponds to the memory output of the forward function. `[dLdX,dLdW] = backward(layer,X,Z,dLdZ,memory)`also returns the derivative`dLdW`of the loss with respect to the learnable parameter, where`layer`has a single learnable parameter. `[dLdX,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory)`also returns the derivative`dLdSin`of the loss with respect to the state input using any of the previous syntaxes, where`layer`has a single state parameter and`dLdSout`对应的导数与res损失pect to the layer state output. `[dLdX,dLdW,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory)`also returns the derivative`dLdW`of the loss with respect to the learnable parameter and returns the derivative`dLdSin`of the loss with respect to the layer state input using any of the previous syntaxes, where`layer`has a single state parameter and single learnable parameter. You can adjust the syntaxes for layers with multiple inputs, multiple outputs, multiple learnable parameters, or multiple state parameters: For layers with multiple inputs, replace`X`and`dLdX`with`X1,...,XN`and`dLdX1,...,dLdXN`, respectively, where`N`is the number of inputs. For layers with multiple outputs, replace`Z`and`dLdZ`with`Z1,...,ZM`and`dLdZ1,...,dLdZM`, respectively, where`M`is the number of outputs. For layers with multiple learnable parameters, replace`dLdW`with`dLdW1,...,dLdWP`, where`P`is the number of learnable parameters. For layers with multiple state parameters, replace`dLdSin`and`dLdSout`with`dLdSin1,...,dLdSinK`and`dLdSout1,...,dLdSoutK`, respectively, where`K`is the number of state parameters. To reduce memory usage by preventing unused variables being saved between the forward and backward pass, replace the corresponding input arguments with`~`. Tip If the number of inputs to`backward`可以各有不同,n use`varargin`instead of the input arguments after`layer`. In this case,`varargin`是一个单元阵列的输入,在哪里the first`N`elements correspond to the`N`layer inputs, the next`M`elements correspond to the`M`layer outputs, the next`M`elements correspond to the derivatives of the loss with respect to the`M`layer outputs, the next`K`elements correspond to the`K`derivatives of the loss with respect to the`K`states outputs, and the last element corresponds to`memory`. If the number of outputs can vary, then use`varargout`instead of the output arguments. In this case,`varargout`is a cell array of the outputs, where the first`N`elements correspond to the`N`the derivatives of the loss with respect to the`N`layer inputs, the next`P`elements correspond to the derivatives of the loss with respect to the`P`learnable parameters, and the next`K`elements correspond to the derivatives of the loss with respect to the`K`state inputs. Tip If the layer forward functions support`dlarray`objects, then the software automatically determines the backward function and you do not need to specify the`backward`function. For a list of functions that support`dlarray`objects, seeList of Functions with dlarray Support.
`Incorrect number of output arguments for 'backward' in Layer`	The syntax for the optional`backward`function is not consistent with the number of layer outputs.

For layers with multiple inputs or outputs, you must set the values of the layer propertiesNumInputs(or alternatively,InputNames) andNumOutputs(or alternatively,OutputNames) in the layer constructor function, respectively.

Multiple Observations

ThecheckLayerfunction checks that the layer functions are valid for single and multiple observations.To check for multiple observations, specify the observation dimension using theObservationDimension选择。如果你年代pecify the observation dimension, then thecheckLayerfunction checks that the layer functions are valid using generated data with mini-batches of size 1 and 2. If you do not specify this name-value pair, then the function skips the tests that check that the layer functions are valid for multiple observations.

Test Diagnostic Description Possible Solution

Test Diagnostic	Description	Possible Solution
`Skipping multi-observation tests. To enable checks with multiple observations, specify the 'ObservationDimension' parameter in checkLayer`.	If you do not specify the`'ObservationDimension'`parameter in`checkLayer`, then the function skips the tests that check data with multiple observations.	Use the command`checkLayer(layer,validInputSize,'ObservationDimension',dim)`, where`layer`is an instance of the custom layer,`validInputSize`is a vector specifying the valid input size to the layer, and`dim`specifies the dimension of the observations in the layer input. For more information, seeLayer Input Sizes.

Skipping multi-observation tests. To enable checks with multiple observations, specify the 'ObservationDimension' parameter in checkLayer

If you do not specify the'ObservationDimension'parameter incheckLayer, then the function skips the tests that check data with multiple observations.

Use the commandcheckLayer(layer,validInputSize,'ObservationDimension',dim), wherelayeris an instance of the custom layer,validInputSizeis a vector specifying the valid input size to the layer, anddimspecifies the dimension of the observations in the layer input.

For more information, seeLayer Input Sizes.

Functions Do Not Error

These tests check that the layers do not error when passed input data of valid size.

Intermediate Layers.The testspredictDoesNotError,forwardDoesNotError, andbackwardDoesNotErrorcheck that the layer functions do not error when passed inputs of valid size. If you specify an observation dimension, then the function checks the layer for both a single observation and multiple observations.

Test Diagnostic	Description	Possible Solution
`The function 'predict' threw an error:`	The`predict`function errors when passed data of size`validInputSize`.	Address the error described in the`Framework Diagnostic`section. Tip If the layer forward functions support`dlarray`objects, then the software automatically determines the backward function and you do not need to specify the`backward`function. For a list of functions that support`dlarray`objects, seeList of Functions with dlarray Support.
`The function 'forward' threw an error:`	The optional`forward`function errors when passed data of size`validInputSize`.
`The function 'backward' threw an error:`	The optional`backward`function errors when passed the output of`predict`.

Output Layers.The testsforwardLossDoesNotErrorandbackwardLossDoesNotErrorcheck that the layer functions do not error when passed inputs of valid size. If you specify an observation dimension, then the function checks the layer for both a single observation and multiple observations.

Test Diagnostic	Description	Possible Solution
`The function 'forwardLoss' threw an error:`	The`forwardLoss`function errors when passed data of size`validInputSize`.	Address the error described in the`Framework Diagnostic`section. Tip If the`forwardLoss`function supports`dlarray`objects, then the software automatically determines the backward loss function and you do not need to specify the`backwardLoss`function. For a list of functions that support`dlarray`objects, seeList of Functions with dlarray Support.
`The function 'backwardLoss' threw an error:`	The optional`backwardLoss`function errors when passed data of size`validInputSize`.

Outputs Are Consistent in Size

These tests check that the layer function outputs are consistent in size.

Intermediate Layers.The testbackwardIsConsistentInSizechecks that thebackwardfunction outputs derivatives of the correct size.

Thebackwardfunction syntax depends on the type of layer.

dLdX = backward(layer,X,Z,dLdZ,memory)returns the derivativesdLdXof the loss with respect to the layer input, wherelayerhas a single input and a single output.Zcorresponds to the forward function output anddLdZ对应的导数与res损失pect toZ. The function inputmemorycorresponds to the memory output of the forward function.
[dLdX,dLdW] = backward(layer,X,Z,dLdZ,memory)also returns the derivativedLdWof the loss with respect to the learnable parameter, wherelayerhas a single learnable parameter.
[dLdX,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory)also returns the derivativedLdSinof the loss with respect to the state input using any of the previous syntaxes, wherelayerhas a single state parameter anddLdSout对应的导数与res损失pect to the layer state output.
[dLdX,dLdW,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory)also returns the derivativedLdWof the loss with respect to the learnable parameter and returns the derivativedLdSinof the loss with respect to the layer state input using any of the previous syntaxes, wherelayerhas a single state parameter and single learnable parameter.

You can adjust the syntaxes for layers with multiple inputs, multiple outputs, multiple learnable parameters, or multiple state parameters:

For layers with multiple inputs, replaceXanddLdXwithX1,...,XNanddLdX1,...,dLdXN, respectively, whereNis the number of inputs.
For layers with multiple outputs, replaceZanddLdZwithZ1,...,ZManddLdZ1,...,dLdZM, respectively, whereMis the number of outputs.
For layers with multiple learnable parameters, replacedLdWwithdLdW1,...,dLdWP, wherePis the number of learnable parameters.
For layers with multiple state parameters, replacedLdSinanddLdSoutwithdLdSin1,...,dLdSinKanddLdSout1,...,dLdSoutK, respectively, whereKis the number of state parameters.

To reduce memory usage by preventing unused variables being saved between the forward and backward pass, replace the corresponding input arguments with~.

Tip

If the number of inputs tobackward可以各有不同,n usevarargininstead of the input arguments afterlayer. In this case,varargin是一个单元阵列的输入,在哪里the firstNelements correspond to theNlayer inputs, the nextMelements correspond to theMlayer outputs, the nextMelements correspond to the derivatives of the loss with respect to theMlayer outputs, the nextKelements correspond to theKderivatives of the loss with respect to theKstates outputs, and the last element corresponds tomemory.

If the number of outputs can vary, then usevarargoutinstead of the output arguments. In this case,varargoutis a cell array of the outputs, where the firstNelements correspond to theNthe derivatives of the loss with respect to theNlayer inputs, the nextPelements correspond to the derivatives of the loss with respect to thePlearnable parameters, and the nextKelements correspond to the derivatives of the loss with respect to theKstate inputs.

The derivativesdLdX1, …,dLdXnmust be the same size as the corresponding layer inputs, anddLdW1,…,dLdWkmust be the same size as the corresponding learnable parameters. The sizes must be consistent for input data with single and multiple observations.

Test Diagnostic	Description	Possible Solution
`Incorrect size of 'dLdX' for 'backward'`.	The derivatives of the loss with respect to the layer inputs must be the same size as the corresponding layer input.	Return the derivatives`dLdX1,…,dLdXn`with the same size as the corresponding layer inputs`X1,…,Xn`.
`Incorrect size of the derivative of the loss with respect to the input 'in1' for 'backward'`
`The size of 'Z' returned from 'forward' must be the same as for 'predict'`.	The outputs of`predict`must be the same size as the corresponding outputs of`forward`.	Return the outputs`Z1,…,Zm`of`predict`with the same size as the corresponding outputs`Z1,…,Zm`of`forward`.
`Incorrect size of the derivative of the loss with respect to 'W' for 'backward'`.	The derivatives of the loss with respect to the learnable parameters must be the same size as the corresponding learnable parameters.	Return the derivatives`dLdW1,…,dLdWk`with the same size as the corresponding learnable parameters`W1,…,Wk`.

Tip

If the layer forward functions supportdlarrayobjects, then the software automatically determines the backward function and you do not need to specify thebackwardfunction. For a list of functions that supportdlarrayobjects, seeList of Functions with dlarray Support.

Output Layers.The testforwardLossIsScalarchecks that the output of theforwardLossfunction is scalar. When thebackwardLossfunction is specified, the testbackwardLossIsConsistentInSizechecks that the outputs offorwardLossandbackwardLoss是正确的大小。

The syntax forforwardLossisloss = forwardLoss(layer,Y,T). The inputYcorresponds to the predictions made by the network. These predictions are the output of the previous layer. The inputTcorresponds to the training targets. The outputlossis the loss betweenYandTaccording to the specified loss function. The outputlossmust be scalar.

If theforwardLossfunction supportsdlarrayobjects, then the software automatically determines the backward loss function and you do not need to specify thebackwardLossfunction. For a list of functions that supportdlarrayobjects, seeList of Functions with dlarray Support.

The syntax forbackwardLossisdLdY = backwardLoss(layer,Y,T). The inputYcontains the predictions made by the network andTcontains the training targets. The outputdLdYis the derivative of the loss with respect to the predictionsY. The outputdLdYmust be the same size as the layer inputY.

Test Diagnostic Description Possible Solution

Test Diagnostic	Description	Possible Solution
`Incorrect size of 'loss' for 'forwardLoss'`.	The output`loss`of`forwardLoss`must be a scalar.	Return the output`loss`as a scalar. For example, if you have multiple values of the loss, then you can use`mean`or`sum`.
`Incorrect size of the derivative of loss 'dLdY' for 'backwardLoss'`.	When`backwardLoss`is specified, the derivatives of the loss with respect to the layer input must be the same size as the layer input.	Return derivative`dLdY`with the same size as the layer input`Y`. If the`forwardLoss`function supports`dlarray`objects, then the software automatically determines the backward loss function and you do not need to specify the`backwardLoss`function. For a list of functions that support`dlarray`objects, seeList of Functions with dlarray Support.

Incorrect size of 'loss' for 'forwardLoss'.

The outputlossofforwardLossmust be a scalar.

Return the outputlossas a scalar. For example, if you have multiple values of the loss, then you can usemeanorsum.

Incorrect size of the derivative of loss 'dLdY' for 'backwardLoss'.

WhenbackwardLossis specified, the derivatives of the loss with respect to the layer input must be the same size as the layer input.

Return derivativedLdYwith the same size as the layer inputY.

Consistent Data Types and GPU Compatibility

These tests check that the layer function outputs are consistent in type and that the layer functions are GPU compatible.

If the layer forward functions fully supportdlarrayobjects, then the layer is GPU compatible. Otherwise, to be GPU compatible, the layer functions must support inputs and return outputs of typegpuArray(Parallel Computing Toolbox).

Many MATLAB^®built-in functions supportgpuArray(Parallel Computing Toolbox)anddlarrayinput arguments. For a list of functions that supportdlarrayobjects, seeList of Functions with dlarray Support. For a list of functions that execute on a GPU, seeRun MATLAB Functions on a GPU(Parallel Computing Toolbox).To use a GPU for deep learning, you must also have a supported GPU device. For information on supported devices, seeGPU Support by Release(Parallel Computing Toolbox).For more information on working with GPUs in MATLAB, seeGPU Computing in MATLAB(Parallel Computing Toolbox).

Intermediate Layers.The testspredictIsConsistentInType,forwardIsConsistentInType, andbackwardIsConsistentInTypecheck that the layer functions output variables of the correct data type. The tests check that the layer functions return consistent data types when given inputs of the data typessingle,double, andgpuArraywith the underlying typessingleordouble.

Tip

If you preallocate arrays using functions such aszeros, then you must ensure that the data types of these arrays are consistent with the layer function inputs. To create an array of zeros of the same data type as another array, use the"like"option ofzeros. For example, to initialize an array of zeros of sizeszwith the same data type as the arrayX, useZ = zeros(sz,"like",X).

Test Diagnostic	Description	Possible Solution
`Incorrect type of 'Z' for 'predict'`.	The types of the outputs`Z1,…,Zm`of the`predict`function must be consistent with the inputs`X1,…,Xn`.	Return the outputs`Z1,…,Zm`with the same type as the inputs`X1,…,Xn`.
`Incorrect type of output 'out1' for 'predict'`.
`Incorrect type of 'Z' for 'forward'`.	The types of the outputs`Z1,…,Zm`of the optional`forward`function must be consistent with the inputs`X1,…,Xn`.
`Incorrect type of output 'out1' for 'forward'`.
`Incorrect type of 'dLdX' for 'backward'`.	The types of the derivatives`dLdX1,…,dLdXn`of the optional`backward`function must be consistent with the inputs`X1,…,Xn`.	Return the derivatives`dLdX1,…,dLdXn`with the same type as the inputs`X1,…,Xn`.
`Incorrect type of the derivative of the loss with respect to the input 'in1' for 'backward'`.
`Incorrect type of the derivative of loss with respect to 'W' for 'backward'`.	The type of the derivative of the loss of the learnable parameters must be consistent with the corresponding learnable parameters.	For each learnable parameter, return the derivative with the same type as the corresponding learnable parameter.

Tip

Output Layers.The testsforwardLossIsConsistentInTypeandbackwardLossIsConsistentInTypecheck that the layer functions output variables of the correct data type. The tests check that the layers return consistent data types when given inputs of the data typessingle,double, andgpuArraywith the underlying typessingleordouble.

Test Diagnostic	Description	Possible Solution
`Incorrect type of 'loss' for 'forwardLoss'`.	The type of the output`loss`of the`forwardLoss`function must be consistent with the input`Y`.	Return`loss`with the same type as the input`Y`.
`Incorrect type of the derivative of loss 'dLdY' for 'backwardLoss'`.	The type of the output`dLdY`of the optional`backwardLoss`function must be consistent with the input`Y`.	Return`dLdY`with the same type as the input`Y`.

Tip

Correct Gradients

The testgradientsAreNumericallyCorrectchecks that the gradients computed by the layer functions are numerically correct. The testbackwardPropagationDoesNotErrorchecks that the derivatives can be computed using automatic differentiation.

Intermediate Layers.When the optionalbackwardfunction is not specified, the testbackwardPropagationDoesNotErrorchecks that the derivatives can be computed using automatic differentiation. When the optionalbackwardfunction is specified, the testgradientsAreNumericallyCorrecttests that the gradients computed inbackwardare numerically correct.

Test Diagnostic	Description	Possible Solution
`Expected a dlarray with no dimension labels, but instead found labels`.	When the optional`backward`function is not specified, the layer forward functions must output`dlarray`objects without dimension labels.	Ensure that any`dlarray`objects created in the layer forward functions do not contain dimension labels.
`通过层无法向后传播。Check that the 'forward' function fully supports automatic differentiation. Alternatively, implement the 'backward' function manually`.	One or more of the following: When the optional`backward`function is not specified, the layer forward functions do not support`dlarray`objects. When the optional`backward`function is not specified, the tracing of the input`dlarray`objects in the forward functions have been broken. For example, by using the`extractdata`function.	Check that the forward functions support`dlarray`objects. For a list of functions that support`dlarray`objects, seeList of Functions with dlarray Support. Check that the derivatives of the input`dlarray`objects can be traced. To learn more about the derivative trace of`dlarray`objects, seeDerivative Trace. Alternatively, define a custom backward function by creating a function named`backward`. To learn more, see.
`通过层无法向后传播。Check that the 'predict' function fully supports automatic differentiation. Alternatively, implement the 'backward' function manually`.
`The derivative 'dLdX' for 'backward' is inconsistent with the numerical gradient`.	One or more of the following: When the optional`backward`function is specified, the derivative is incorrectly computed The forward functions are non-differentiable at some input points Error tolerance is too small	If the layer forward functions support`dlarray`objects, then the software automatically determines the backward function and you can omit the backward function. For a list of functions that support`dlarray`objects, seeList of Functions with dlarray Support. Check that the derivatives in`backward`are correctly computed. If the derivatives are correctly computed, then in the`Framework Diagnostic`section, manually check the absolute and relative error between the actual and expected values of the derivative. If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic.
`The derivative of the loss with respect to the input 'in1' for 'backward' is inconsistent with the numerical gradient`.
`The derivative of loss with respect to 'W' for 'backward' is inconsistent with the numerical gradient`.

Tip

Output Layers.When the optionalbackwardLossfunction is not specified, the testbackwardPropagationDoesNotErrorchecks that the derivatives can be computed using automatic differentiation. When the optionalbackwardLossfunction is specified, the testgradientsAreNumericallyCorrecttests that the gradients computed inbackwardLossare numerically correct.

Test Diagnostic Description Possible Solution

Expected a dlarray with no dimension labels, but instead found labels When the optionalbackwardLossfunction is not specified, theforwardLossfunction must outputdlarrayobjects without dimension labels. Ensure that anydlarrayobjects created in theforwardLossfunction does not contain dimension labels.

Test Diagnostic	Description	Possible Solution
`Expected a dlarray with no dimension labels, but instead found labels`	When the optional`backwardLoss`function is not specified, the`forwardLoss`function must output`dlarray`objects without dimension labels.	Ensure that any`dlarray`objects created in the`forwardLoss`function does not contain dimension labels.
`通过层无法向后传播。Check that the 'forwardLoss' function fully supports automatic differentiation. Alternatively, implement the 'backwardLoss' function manually`	One or more of the following: When the optional`backwardLoss`function is not specified, the layer`forwardLoss`function does not support`dlarray`objects. When the optional`backwardLoss`function is not specified, the tracing of the input`dlarray`objects in the`forwardLoss`function has been broken. For example, by using the`extractdata`function.	Check that the`forwardLoss`function supports`dlarray`objects. For a list of functions that support`dlarray`objects, seeList of Functions with dlarray Support. Check that the derivatives of the input`dlarray`objects can be traced. To learn more about the derivative trace of`dlarray`objects, seeDerivative Trace. Alternatively, define a custom backward loss function by creating a function named`backwardLoss`. To learn more, see.
`The derivative 'dLdY' for 'backwardLoss' is inconsistent with the numerical gradient`.	One or more of the following: The derivative with respect to the predictions`Y`is incorrectly computed Function is non-differentiable at some input points Error tolerance is too small	Check that the derivatives in`backwardLoss`are correctly computed. If the derivatives are correctly computed, then in the`Framework Diagnostic`section, manually check the absolute and relative error between the actual and expected values of the derivative. If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic.

通过层无法向后传播。Check that the 'forwardLoss' function fully supports automatic differentiation. Alternatively, implement the 'backwardLoss' function manually

One or more of the following:

When the optionalbackwardLossfunction is not specified, the layerforwardLossfunction does not supportdlarrayobjects.
When the optionalbackwardLossfunction is not specified, the tracing of the inputdlarrayobjects in theforwardLossfunction has been broken. For example, by using theextractdatafunction.

Check that theforwardLossfunction supportsdlarrayobjects. For a list of functions that supportdlarrayobjects, seeList of Functions with dlarray Support.

Check that the derivatives of the inputdlarrayobjects can be traced. To learn more about the derivative trace ofdlarrayobjects, seeDerivative Trace.

Alternatively, define a custom backward loss function by creating a function namedbackwardLoss. To learn more, see.

The derivative 'dLdY' for 'backwardLoss' is inconsistent with the numerical gradient.

One or more of the following:

The derivative with respect to the predictionsYis incorrectly computed
Function is non-differentiable at some input points
Error tolerance is too small

Check that the derivatives inbackwardLossare correctly computed.

If the derivatives are correctly computed, then in theFramework Diagnosticsection, manually check the absolute and relative error between the actual and expected values of the derivative.

If the absolute and relative errors are within an acceptable margin of the tolerance, then you can ignore this test diagnostic.

Tip

Valid States

For layers with state properties, the testpredictReturnsValidStateschecks that the predict function returns valid states. Whenforwardis specified, the testforwardReturnsValidStateschecks that the forward function returns valid states. The testresetStateDoesNotErrorchecks that theresetStatefunction returns a layer with valid state properties.

Test Diagnostic	Description	Possible Solution
`Error using 'predict' in Layer. 'State' must be real-values numeric array or unformatted dlarray object`.	圣ate outputs must be real-valued numeric arrays or unformatted`dlarray`objects.	Ensure that the states identified in the`Framework Diagnostic`are real-valued numeric arrays or unformatted`dlarray`objects.
`Error using 'resetState' in Layer. 'State' must be real-values numeric array or unformatted dlarray object`	圣ate properties of returned layer must be real-valued numeric arrays or unformatted`dlarray`objects.

Code Generation Compatibility

如果你年代et theCheckCodegenCompatibilityoption to1(true), then thecheckLayerfunction checks the layer for code generation compatibility.

The testcodegenPragmaDefinedInClassDefchecks that the layer definition contains the code generation pragma%#codegen. The testcheckForSupportedLayerPropertiesForCodegenchecks that the layer properties support code generation. The testpredictIsValidForCodegenerationchecks that the outputs ofpredictare consistent in dimension and batch size.

Code generation supports intermediate layers with 2-D image or feature input only. Code generation does not support layers with state properties (properties with attribute圣ate).

ThecheckLayerfunction does not check that functions used by the layer are compatible with code generation. To check that functions used by the custom layer also support code generation, first use theCode Generation Readinessapp. For more information, seeCheck Code by Using the Code Generation Readiness Tool(MATLAB Coder).

Test Diagnostic	Description	Possible Solution
`Specify '%#codegen' in the class definition of custom layer`	The layer definition does not include the pragma`"%#codegen"`for code generation.	Add the`%#codegen`directive (or pragma) to your layer definition to indicate that you intend to generate code for this layer. Adding this directive instructs the MATLAB Code Analyzer to help you diagnose and fix violations that result in errors during code generation.
`Nonscalar layer properties must be type single or double or character array for custom layer`	The layer contains non-scalar properties of type other than single, double, or character array.	Convert non-scalar properties to use a representation of type single, double, or character array. For example, convert a categorical array to an array of integers of type`double`representing the categories.
`Scalar layer properties must be numeric, logical, or string for custom layer`	The layer contains scalar properties of type other than numeric, logical, or string.	Convert scalar properties to use a numeric representation, or a representation of type logical or string. For example, convert a categorical scalar to an integer of type`double`representing the category.
`For code generation, 'Z' must have the same number of dimensions as the layer input`.	The number of dimensions of the output`Z`of`predict`does not match the number of dimensions of the layer inputs.	In the`predict`function, return the outputs with the same number of dimensions as the layer inputs.
`For code generation, 'Z' must have the same batch size as the layer input`.	The size of the batch size of the output`Z`of`predict`does not match the size of the batch size of the layer inputs.	In the`predict`function, return the outputs with the batch size as the layer inputs.