Main Content

Create a Deep Learning Experiment for Regression

This example uses:

Open Example

This example shows how to train a deep learning network for regression by usingExperiment Manager. In this example, you use a regression model to predict the angles of rotation of handwritten digits. A custom metric function determines the fraction of angle predictions within an acceptable error margin from the true angles. For more information on using a regression model, seeTrain Convolutional Neural Network for Regression.

Open Experiment

First, open the example. Experiment Manager loads a project with a preconfigured experiment that you can inspect and run. To open the experiment, in theExperiment Browserpane, double-click the name of the experiment (RegressionExperiment).

Built-in training experiments consist of a description, a table of hyperparameters, a setup function, and a collection of metric functions to evaluate the results of the experiment. For more information, seeConfigure Built-In Training Experiment.

TheDescriptionfield contains a textual description of the experiment. For this example, the description is:

Regression model to predict angles of rotation of digits, using hyperparameters to specify: * the number of filters used by the convolution layers * the probability of the dropout layer in the network

TheHyperparameterssection specifies the strategy (Exhaustive Sweep) and hyperparameter values to use for the experiment. When you run the experiment, Experiment Manager trains the network using every combination of hyperparameter values specified in the hyperparameter table. This example uses two hyperparameters:

  • Probabilitysets the probability of the dropout layer in the neural network. By default, the values for this hyperparameter are specified as[0.1 0.2].

  • Filters显示过滤器使用的数量convolution layer in the neural network. In the subsequent convolution layers, the number of filters is a multiple of this value. By default, the values of this hyperparameter are specified as[4 6 8].

TheSetup Functionconfigures the training data, network architecture, and training options for the experiment. The input to the setup function is a structure with fields from the hyperparameter table. The setup function returns four outputs that you use to train a network for image regression problems. The setup function has three sections.

  • Load Training Datadefines the training and validation data for the experiment as 4-D arrays. The training and validation data sets each contain 5000 images of digits from 0 to 9. The regression values correspond to the angles of rotation of the digits.

[XTrain,~,YTrain] = digitTrain4DArrayData; [XValidation,~,YValidation] = digitTest4DArrayData;

  • Define Network Architecturedefines the architecture for a convolutional neural network for regression.

inputSize = [28 28 1]; numFilters = params.Filters; layers = [ imageInputLayer(inputSize) convolution2dLayer(3,numFilters,Padding="same") batchNormalizationLayer reluLayer averagePooling2dLayer(2,Stride=2) convolution2dLayer(3,2*numFilters,Padding="same") batchNormalizationLayer reluLayer averagePooling2dLayer(2,Stride=2) convolution2dLayer(3,4*numFilters,Padding="same") batchNormalizationLayer reluLayer convolution2dLayer(3,4*numFilters,Padding="same") batchNormalizationLayer reluLayer dropoutLayer(params.Probability) fullyConnectedLayer(1) regressionLayer];

  • Specify Training Optionsdefines atrainingOptionsobject for the experiment. The example trains the network for 30 epochs. The learning rate is initially 0.001 and drops by a factor of 0.1 after 20 epochs. The software trains the network on the training data and calculates the root mean squared error (RMSE) and loss on the validation data at regular intervals during training. The validation data is not used to update the network weights.

miniBatchSize = 128; validationFrequency = floor(numel(YTrain)/miniBatchSize); options = trainingOptions("sgdm",...MiniBatchSize=miniBatchSize,...MaxEpochs=30,...InitialLearnRate=1e-3,...LearnRateSchedule="piecewise",...LearnRateDropFactor=0.1,...LearnRateDropPeriod=20,...Shuffle="every-epoch",...ValidationData={XValidation,YValidation},...ValidationFrequency=validationFrequency,...Verbose=false);

To inspect the setup function, underSetup Function, clickEdit. The setup function opens in MATLAB® Editor. In addition, the code for the setup function appears inAppendix 1at the end of this example.

TheMetricssection specifies optional functions that evaluate the results of the experiment. Experiment Manager evaluates these functions each time it finishes training the network. To inspect a metric function, select the name of the metric function and clickEdit. The metric function opens in MATLAB Editor.

This example includes a metric functionAccuracythat determines the percentage of angle predictions within an acceptable error margin from the true angles. By default, the function uses a threshold of 10 degrees. The code for the metric function appears inAppendix 2at the end of this example.

Run Experiment

When you run the experiment, Experiment Manager trains the network defined by the setup function six times. Each trial uses a different combination of hyperparameter values. By default, Experiment Manager runs one trial at a time. If you have Parallel Computing Toolbox™, you can run multiple trials at the same time or offload your experiment as a batch job in a cluster.

  • To run one trial of the experiment at a time, on theExperiment Managertoolstrip, underMode, selectSequentialand clickRun.

  • To run multiple trials at the same time, underMode, selectSimultaneousand clickRun. If there is no current parallel pool, Experiment Manager starts one using the default cluster profile. Experiment Manager then runs as many simultaneous trials as there are workers in your parallel pool. For best results, before you run your experiment, start a parallel pool with as many workers as GPUs. For more information, seeUse Experiment Manager to Train Networks in ParallelandGPU Computing Requirements(Parallel Computing Toolbox).

  • To offload the experiment as a batch job, underMode, selectBatch SequentialorBatch Simultaneous, specify yourClusterandPool Size, and clickRun. For more information, seeOffload Experiments as Batch Jobs to Cluster.

A table of results displays the RMSE and loss for each trial. The table also displays the accuracy of the trial, as determined by the custom metric functionAccuracy.

To display the training plot and track the progress of each trial while the experiment is running, underReview Results, clickTraining Plot.

Evaluate Results

To find the best result for your experiment, sort the table of results by accuracy.

  1. Point to theAccuracycolumn.

  2. Click the triangle icon.

  3. SelectSort in Descending Order.

The trial with the highest accuracy appears at the top of the results table.

To test the performance of an individual trial, export the trained network and display a box plot of the residuals for each digit class.

  1. Select the trial with the highest accuracy.

  2. On theExperiment Managertoolstrip, clickExport>Trained Network.

  3. In the dialog window, enter the name of a workspace variable for the exported network. The default name istrainedNetwork.

  4. Use the exported network as the input to the functionplotResiduals, which is listed inAppendix 3at the end of this example. For instance, in the MATLAB Command Window, enter:

plotResiduals(trainedNetwork)

The function creates a residual box plot for each digit. The digit classes with highest accuracy have a mean close to zero and little variance.

To record observations about the results of your experiment, add an annotation.

  1. In the results table, right-click theAccuracycell of the best trial.

  2. SelectAdd Annotation.

  3. In theAnnotationspane, enter your observations in the text box.

For more information, seeSort, Filter, and Annotate Experiment Results.

Close Experiment

In theExperiment Browserpane, right-click the name of the project and selectClose Project. Experiment Manager closes all of the experiments and results contained in the project.

Appendix 1: Setup Function

This function configures the training data, network architecture, and training options for the experiment.

Input

  • paramsis a structure with fields from the Experiment Manager hyperparameter table.

Output

  • XTrainis a 4-D array containing the training data.

  • YTrainis a 1-D array containing the regression values for training,

  • layersis a layer graph that defines the neural network architecture.

  • optionsis atrainingOptionsobject.

function[XTrain,YTrain,layers,options] = RegressionExperiment_setup1(params) [XTrain,~,YTrain] = digitTrain4DArrayData; [XValidation,~,YValidation] = digitTest4DArrayData; inputSize = [28 28 1]; numFilters = params.Filters; layers = [ imageInputLayer(inputSize) convolution2dLayer(3,numFilters,Padding="same") batchNormalizationLayer reluLayer averagePooling2dLayer(2,Stride=2) convolution2dLayer(3,2*numFilters,Padding="same") batchNormalizationLayer reluLayer averagePooling2dLayer(2,Stride=2) convolution2dLayer(3,4*numFilters,Padding="same") batchNormalizationLayer reluLayer convolution2dLayer(3,4*numFilters,Padding="same") batchNormalizationLayer reluLayer dropoutLayer(params.Probability) fullyConnectedLayer(1) regressionLayer]; miniBatchSize = 128; validationFrequency = floor(numel(YTrain)/miniBatchSize); options = trainingOptions("sgdm",...MiniBatchSize=miniBatchSize,...MaxEpochs=30,...InitialLearnRate=1e-3,...LearnRateSchedule="piecewise",...LearnRateDropFactor=0.1,...LearnRateDropPeriod=20,...Shuffle="every-epoch",...ValidationData={XValidation,YValidation},...ValidationFrequency=validationFrequency,...Verbose=false);end

Appendix 2: Compute Accuracy of Regression Model

This function calculates the number of predictions within an acceptable error margin from the true angles.

functionmetricOutput = Accuracy(trialInfo) [XValidation,~,YValidation] = digitTest4DArrayData; YPredicted = predict(trialInfo.trainedNetwork,XValidation); predictionError = YValidation - YPredicted; thr = 10; numCorrect = sum(abs(predictionError) < thr); numValidationImages = numel(YValidation); metricOutput = 100*numCorrect/numValidationImages;end

Appendix 3: Display Box Plot of Residuals for Each Digit

This function creates a residual box plot for each digit.

function~, plotResiduals(净)[XValidation YValidation] = digitTest4DArrayData; YPredicted = predict(net,XValidation); predictionError = YValidation - YPredicted; residualMatrix = reshape(predictionError,500,10); figure boxplot(residualMatrix,..."Labels",["0","1","2","3","4","5","6","7","8","9"]) xlabel("Digit Class") ylabel("Degrees Error") title("Residuals")end

See Also

Apps

Functions

Related Topics