使用神经图像量化图像质量评估

dataDir = fullfile (tempdir,“LIVEInTheWild”);如果~存在(dataDir“dir”mkdir (dataDir);结束

trainedNet_url =“https://ssd.mathworks.com/金宝appsupportfiles/image/data/trainedNIMA.zip”;downloadTrainedNetwork (trainedNet_url dataDir);负载(fullfile (dataDir“trainedNIMA.mat”));

imOriginal = imread (“kobi.png”);

imBlur = imgaussfilt (imOriginal 5);蒙太奇({imOriginal, imBlur})

[meanOriginal, stdOriginal] = predictNIMAScore (dlnet imOriginal);[meanBlur, stdBlur] = predictNIMAScore (dlnet imBlur);

图t = tiledlayout (1、2);displayImageAndScoresForNIMA (t, imOriginal meanOriginal stdOriginal,的“原始图像”)displayImageAndScoresForNIMA (t imBlur meanBlur stdBlur,“模糊图像”)

imageData =负载(fullfile (dataDir“数据”,“AllImages_release.mat”));imageData = imageData.AllImages_release;nImg =长度(imageData);imageList (1:7) = fullfile (dataDir“图片”,“trainingImages”imageData (1:7));imageList (8: nImg) = fullfile (dataDir,“图片”imageData(8:结束));

imd = imageDatastore (imageList);

meanData =负载(fullfile (dataDir“数据”,“AllMOS_release.mat”));meanData = meanData.AllMOS_release;stdData =负载(fullfile (dataDir“数据”,“AllStdDev_release.mat”));stdData = stdData.AllStdDev_release;

图t = tiledlayout (1、3);idx1 = 785;idx1 displayImageAndScoresForNIMA (t, readimage (imd),…meanData (idx1) stdData (idx1),“图像”+ imageData (idx1)) idx2 = 203;idx2 displayImageAndScoresForNIMA (t, readimage (imd),…meanData (idx2) stdData (idx2),“图像”+ imageData (idx2)) idx3 = 777;idx3 displayImageAndScoresForNIMA (t, readimage (imd),…meanData (idx3) stdData (idx3),“图像”+ imageData (idx3))

rescaleSize = (256 - 256);imd =变换(imd, @ (x) imresize (x, rescaleSize));

newMaxScore = 10;概率= createNIMAScoreDistribution (meanData stdData);cumProb = cumsum(问题2);

probDS = arrayDatastore (cumProb ', IterationDimension = 2);

probDS dsCombined =结合(imd);

sampleRead =预览(dsCombined)

图tiledlayout (1、2) nexttile imshow (sampleRead{1})标题(“从数据集样本图像”)nexttile情节(sampleRead{2})标题(“累积分数分配”)

numTrain =地板(0.70 * nImg);numVal =地板(0.15 * nImg);Idx = randperm (nImg);idxTrain = Idx (1: numTrain);idxVal = Idx (numTrain + 1: numTrain + numVal);idxTest = Idx (numTrain + numVal + 1: nImg);dsTrain =子集(dsCombined idxTrain);dsVal =子集(dsCombined idxVal);dst =子集(dsCombined idxTest);

inputSize = (224 - 224);dsTrain =变换(dsTrain @ (x) augmentDataForNIMA (x, inputSize));

meanImage = 0 ([inputSize 3]);meanImageSq = 0 ([inputSize 3]);而hasdata (dsTrain) dat =阅读(dsTrain);img =双(dat {1});meanImage = meanImage +编码器;meanImageSq = meanImageSq + img。^ 2;结束meanImage = meanImage / numTrain;meanImageSq = meanImageSq / numTrain;varImage = meanImageSq - meanImage。^ 2;stdImage =√varImage);

重置(dsTrain);

miniBatchSize = 128;mbqTrain = minibatchqueue (dsTrain MiniBatchSize = MiniBatchSize,…PartialMiniBatch =“丢弃”MiniBatchFormat = (“SSCB”,”“),…DispatchInBackground = canUseGPU);mbqVal = minibatchqueue (dsVal MiniBatchSize = MiniBatchSize,…MiniBatchFormat = [“SSCB”,”“),DispatchInBackground = canUseGPU);

网= mobilenetv2;

lgraph = layerGraph(净);

镶嵌者= imageInputLayer ([inputSize 3], Name =“输入”,…归一化=“zscore”意味着= meanImage StandardDeviation = stdImage);lgraph = replaceLayer (lgraph,“input_1”、镶嵌者);

lgraph = removeLayers (lgraph, (“ClassificationLayer_Logits”,“Logits_softmax”,“分对数”]);newFinalLayers = [dropoutLayer (0.75, Name =“下降”)fullyConnectedLayer (newMaxScore Name =“俱乐部”WeightLearnRateFactor = 10, BiasLearnRateFactor = 10) softmaxLayer (Name =“概率”));lgraph = addLayers (lgraph newFinalLayers);lgraph = connectLayers (lgraph,“global_average_pooling2d_1”,“下降”);dlnet = dlnetwork (lgraph);

deepNetworkDesigner (lgraph)

numEpochs = 150;动量= 0.9;initialLearnRate = 3 e - 3;衰变= 0.95;

doTraining = false;如果doTraining迭代= 0;速度= [];开始=抽搐;[hFig, lineLossTrain lineLossVal] = initializeTrainingPlotNIMA;为时代= 1:numEpochs shuffle (mbqTrain);learnRate = initialLearnRate /(1 +衰变*地板(时代/ 10));而hasdata (mbqTrain)迭代=迭代+ 1;[dlX, cdfY] =下一个(mbqTrain);(研究生,亏损)= dlfeval (@modelGradients、dlnet dlX, cdfY);[dlnet,速度]= sgdmupdate (dlnet、毕业生、速度、learnRate动量);updateTrainingPlotNIMA (lineLossTrain损失、时代、迭代,开始)结束%绘制添加验证数据[lossVal, ~ ~] = modelPredictions (dlnet mbqVal);时代,updateTrainingPlotNIMA (lineLossVal lossVal迭代,开始)结束%保存训练网络modelDateTime =字符串(datetime (“现在”格式=“yyyy-MM-dd-HH-mm-ss”));保存(fullfile (dataDir“trainedNIMA——”+ modelDateTime +“.mat”),“dlnet”);其他的负载(fullfile (dataDir“trainedNIMA.mat”));结束

mbqTest = minibatchqueue (dst MiniBatchSize = MiniBatchSize MiniBatchFormat = [“SSCB”,”“]);

[YPredTest, ~, cdfYTest] = modelPredictions (dlnet mbqTest);

meanPred = extractdata (YPredTest) * (1:10)”;stdPred =√extractdata YPredTest) * ((1:10) ^ 2)”——meanPred。^ 2);origCdf = extractdata (cdfYTest);origPdf = [origCdf (1:);diff (origCdf)];meanOrig = origPdf ' * (1:10) ';stdOrig =√origPdf‘* ((1:10) ^ 2)”——meanOrig。^ 2);

emdt = earthMoverDistance (YPredTest cdfYTest 1)

qualityThreshold = 5;binaryPred = meanPred > qualityThreshold;binaryOrig = meanOrig > qualityThreshold;

binaryAccuracy = 100 * (binaryPred = = binaryOrig) /长度总和(binaryPred)

meanLCC = corr (meanOrig meanPred)

meanSRCC = corr (meanOrig、meanPred类型=“枪兵”)

函数(渐变,亏损)= modelGradients (dlnet、dlX cdfY) dlYPred =前进(dlnet dlX);损失= earthMoverDistance (dlYPred cdfY 2);梯度= dlgradient(损失、dlnet.Learnables);结束

函数损失= earthMoverDistance (YPred cdfY, r) N =大小(cdfY, 1);cdfYPred = computeCDF (YPred);cdfDiff = (1 / N) * (abs (cdfY - cdfYPred)。^ r);lossArray = (cdfDiff, 1)总和。^ (1 / r);损失=意味着(lossArray);结束函数cdfY = computeCDF (Y)%概率质量函数Y,计算累积概率[N, miniBatchSize] =大小(Y);L = repmat (triu ((N)), 1, 1, miniBatchSize);L3d =排列(L, [1 3 2]);刺激= y * L3d;prodSum =总和(prod 1);cdfY =重塑(prodSum (:)“miniBatchSize, N)”;结束

函数[dlYPred,失去,cdfYOrig] = modelPredictions (dlnet兆贝可)重置(兆贝可);损失= 0;numObservations = 0;dlYPred = [];cdfYOrig = [];而hasdata(兆贝可)[dlX cdfY] =(兆贝可);miniBatchSize =大小(dlX 4);海底=预测(dlnet dlX);损失=损失+ earthMoverDistance(海底cdfY 2) * miniBatchSize;dlYPred = (dlYPred海底);cdfYOrig = [cdfYOrig cdfY];numObservations = numObservations + miniBatchSize;结束损失= / numObservations损失;结束