Main Content

kfoldfun

Cross-validate function using cross-validated ECOC model

Description

example

vals= kfoldfun(CVMdl,fun)cross-validates the functionfunby applyingfunto the data stored in the cross-validated ECOC modelCVMdl. You must passfunas a function handle.

Examples

collapse all

Train a multiclass ECOC classifier, and then cross-validate the model using a customk-fold loss function.

Load Fisher’s iris data set. Specify the predictor dataX, the response dataY, and the order of the classes inY.

loadfisheririsX = meas; Y = categorical(species); classOrder = unique(Y);% Class orderrng(1);% For reproducibility

Train and cross-validate an ECOC model using support vector machine (SVM) binary classifiers. Standardize the predictors using an SVM template, and specify the class order.

t = templateSVM('Standardize',1); CVMdl = fitcecoc(X,Y,'CrossVal','on',“学习者”,t,...'ClassNames',classOrder);

CVMdlis aClassificationPartitionedECOCmodel. By default, the software implements 10-fold cross-validation.

Compute the classification error (proportion of misclassified observations) for the validation-fold observations.

L = kfoldLoss(CVMdl)
L = 0.0400

Examine the result when the cost of misclassifying a flower asversicoloris10and the cost of any other error is1. Write a function namednoversicolorthat assigns a cost of1for general misclassification and a cost of10for misclassifying a flower asversicolor.

If you use the live script file for this example, thenoversicolorfunction is already included at the end of the file. Otherwise, you need to create this function at the end of your .m file or add it as a file on the MATLAB path.

Compute the mean misclassification error with thenoversicolorcost.

foldLoss = kfoldfun(CVMdl,@noversicolor); mean(foldLoss)
ans =single0.0667

This code creates the functionnoversicolor.

functionaverageCost = noversicolor(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)% noversicolor: Example custom cross-validation function that assigns a cost of% 10 for misclassifying versicolor irises and a cost of 1 for misclassifying% the other irises. This example function requires the fisheriris data% set.Ypredict = predict(CMP,Xtest); misclassified = not(strcmp(Ypredict,Ytest));% Different resultclassifiedAsVersicolor = strcmp(Ypredict,'versicolor');% Index of bad decisionscost = sum(misclassified) +...9*sum(misclassified & classifiedAsVersicolor);% Total differencesaverageCost = single(cost/numel(Ytest));% Average errorend

Input Arguments

collapse all

Cross-validated ECOC model, specified as aClassificationPartitionedECOCmodel.

Cross-validated function, specified as a function handle.funhas this syntax:

testvals = fun(CMP,Xtrain,Ytrain,Wtrain,Xtest,Ytest,Wtest)
  • CMPis a compact model stored in one element of theCVMdl.Trainedproperty.

  • Xtrainis the training matrix of predictor values.

  • Ytrainis the training array of response values.

  • Wtrainis the set of training weights for observations.

  • XtestandYtestare the validation data, with associated weightsWtest.

  • The returned valuetestvalsmust have the same size across all folds.

Data Types:function_handle

Output Arguments

collapse all

Cross-validation results, returned as a numeric matrix.valscorresponds to the arrays of thetestvalsoutput, concatenated vertically over all the folds. For example, iftestvalsfrom every fold is a numeric vector of lengthn,kfoldfun返回一个KFold-by-nnumeric matrix with one row per fold.

Extended Capabilities

Introduced in R2014b