Main Content

dlfeval

Evaluate deep learning model for custom training loops

Description

Usedlfevalto evaluate custom deep learning models for custom training loops.

Tip

For most deep learning tasks, you can use a pretrained network and adapt it to your own data. For an example showing how to use transfer learning to retrain a convolutional neural network to classify a new set of images, seeTrain Deep Learning Network to Classify New Images. Alternatively, you can create and train networks from scratch usinglayerGraphobjects with thetrainNetworkandtrainingOptionsfunctions.

If thetrainingOptionsfunction does not provide the training options that you need for your task, then you can create a custom training loop using automatic differentiation. To learn more, seeDefine Deep Learning Network for Custom Training Loops.

example

[y1,...,yk] = dlfeval(fun,x1,...,xn)evaluates the deep learning array functionfunat the input argumentsx1,...,xn. Functions passed todlfevalcan contain calls todlgradient, which compute gradients from the inputsx1,...,xnby using automatic differentiation.

Examples

collapse all

Rosenbrock's function is a standard test function for optimization. Therosenbrock.mhelper function computes the function value and uses automatic differentiation to compute its gradient.

类型rosenbrock.m
function [y,dydx] = rosenbrock(x) y = 100*(x(2) - x(1).^2).^2 + (1 - x(1)).^2; dydx = dlgradient(y,x); end

To evaluate Rosenbrock's function and its gradient at the point[–1,2], create adlarrayof the point and then calldlfevalon the function handle@rosenbrock.

x0 = dlarray([-1,2]); [fval,gradval] = dlfeval(@rosenbrock,x0)
fval = 1x1 dlarray 104
gradval = 1x2 dlarray 396 200

Alternatively, define Rosenbrock's function as a function of two inputs,x1and x2.

类型rosenbrock2.m
function [y,dydx1,dydx2] = rosenbrock2(x1,x2) y = 100*(x2 - x1.^2).^2 + (1 - x1).^2; [dydx1,dydx2] = dlgradient(y,x1,x2); end

Calldlfevalto evaluaterosenbrock2on twodlarrayarguments representing the inputs–1and2.

x1 = dlarray(-1); x2 = dlarray(2); [fval,dydx1,dydx2] = dlfeval(@rosenbrock2,x1,x2)
fval = 1x1 dlarray 104
dydx1 = 1x1 dlarray 396
dydx2 = 1x1 dlarray 200

情节Rosenbro的梯度ck's function for several points in the unit square. First, initialize the arrays representing the evaluation points and the output of the function.

[X1 X2] = meshgrid(linspace(0,1,10)); X1 = dlarray(X1(:)); X2 = dlarray(X2(:)); Y = dlarray(zeros(size(X1))); DYDX1 = Y; DYDX2 = Y;

Evaluate the function in a loop. Plot the result usingquiver.

fori = 1:length(X1) [Y(i),DYDX1(i),DYDX2(i)] = dlfeval(@rosenbrock2,X1(i),X2(i));endquiver(extractdata(X1),extractdata(X2),extractdata(DYDX1),extractdata(DYDX2)) xlabel('x1') ylabel('x2')

Figure contains an axes object. The axes object contains an object of type quiver.

Usedlgradientanddlfevalto compute the value and gradient of a function that involves complex numbers. You can compute complex gradients, or restrict the gradients to real numbers only.

Define the functioncomplexFun, listed at the end of this example. This function implements the following complex formula:

f ( x ) = ( 2 + 3 i ) x

Define the functiongradFun, listed at the end of this example. This function callscomplexFunand usesdlgradientto calculate the gradient of the result with respect to the input. For automatic differentiation, the value to differentiate — i.e., the value of the function calculated from the input — must be a real scalar, so the function takes the sum of the real part of the result before calculating the gradient. The function returns the real part of the function value and the gradient, which can be complex.

Define the sample points over the complex plane between -2 and 2 and -2 i and 2 i and convert todlarray.

functionRes = linspace(-2,2,100); x = functionRes + 1i*functionRes.'; x = dlarray(x);

Calculate the function value and gradient at each sample point.

[y, grad] = dlfeval(@gradFun,x); y = extractdata(y);

Define the sample points at which to display the gradient.

gradientRes = linspace(-2,2,11); xGrad = gradientRes + 1i*gradientRes.';

Extract the gradient values at these sample points.

[~,gradPlot] = dlfeval(@gradFun,dlarray(xGrad)); gradPlot = extractdata(gradPlot);

Plot the results. Useimagescto show the value of the function over the complex plane. Usequiverto show the direction and magnitude of the gradient.

imagesc([-2,2],[-2,2],y); axisxycolorbar holdonquiver(real(xGrad),imag(xGrad),real(gradPlot),imag(gradPlot),"k"); xlabel("Real") ylabel("Imaginary") title("Real Value and Gradient","Re$(f(x)) = $ Re$((2+3i)x)$","interpreter","latex")

The gradient of the function is the same across the entire complex plane. Extract the value of the gradient calculated by automatic differentiation.

grad(1,1)
ans = 1×1 dlarray 2.0000 - 3.0000i

By inspection, the complex derivative of the function has the value

df ( x ) dx = 2 + 3 i

However, the function Re( f ( x ) ) is not analytic, and therefore no complex derivative is defined. For automatic differentiation in MATLAB, the value to differentiate must always be real, and therefore the function can never be complex analytic. Instead, the derivative is computed such that the returned gradient points in the direction of steepest ascent, as seen in the plot. This is done by interpreting the function Re ( f ( x ) ) :C Ras a function Re ( f ( x R + i x I ) ) :R × R R.

functiony = complexFun(x) y = (2+3i)*x;endfunction[y,grad] = gradFun(x) y = complexFun(x); y = real(y); grad = dlgradient(sum(y,"all"),x);end

Input Arguments

collapse all

Function to evaluate, specified as a function handle. Iffunincludes adlgradientcall, thendlfevalevaluates the gradient by using automatic differentiation. In this gradient evaluation, each argument of thedlgradientcall must be adlarrayor a cell array, structure, or table containing adlarray. The number of input arguments todlfevalmust be the same as the number of input arguments tofun.

Example:@rosenbrock

Data Types:function_handle

Function arguments, specified as any MATLAB data type or adlnetworkobject.

An input argumentxjthat is a variable of differentiation in adlgradientcall must be a traceddlarrayor a cell array, structure, or table containing a traceddlarray. An extra variable such as a hyperparameter or constant data array does not have to be adlarray.

To evaluate gradients for deep learning, you can provide adlnetworkobject as a function argument and evaluate the forward pass of the network insidefun.

Example:dlarray([1 2;3 4])

Data Types:single|double|int8|int16|int32|int64|uint8|uint16|uint32|uint64|logical|char|string|struct|table|cell|function_handle|categorical|datetime|duration|calendarDuration|fi
Complex Number Support:Yes

Output Arguments

collapse all

Function outputs, returned as any data type. If the output results from adlgradientcall, the output is adlarray.

Tips

Extended Capabilities

Version History

Introduced in R2019b