Main Content

evaluateSemanticSegmentation

Evaluate semantic segmentation data set against ground truth

在页面中崩溃

Syntax

SSM =评估emantancistementation（DSResults，dstruth）
SSM= evaluateSemanticSegmentation(imageSetConfusion,classNames)
[ssm,blockMetrics] = evaluateSemanticSegmentation(blockSetConfusion,classNames)
[___] = evaluateSemanticSegmentation(___,Name,Value)

Description

example

SSM= evaluateSemanticSegmentation(dsresults,dsTruth)computes various metrics to evaluate the quality of the semantic segmentation results,dsresults，反对地面真理细分，dsTruth。

SSM= evaluateSemanticSegmentation(imageSetConfusion,classNames)computes various metrics to evaluate the quality of the semantic segmentation results from confusion matrices,imageSetConfusion, with segmentation classesclassNames。

[SSM,blockMetrics] = evaluateSemanticSegmentation(BlocksetConfusion,classNames)computes various metrics to evaluate the quality of the block-based semantic segmentation results from confusion matrices,BlocksetConfusionwith classesclassNames。

[___] = evaluateSemanticSegmentation(___,姓名,Value)computes semantic segmentation metrics using one or more姓名,Valuepair arguments to control the evaluation.

Examples

collapse all

This example uses:

打开实时脚本

ThetriangleImagesdata set has 100 test images with ground truth labels. Define the location of the data set.

dataSetDir = fullfile(toolboxdir('vision'),'VisionData','triangleImages');

Define the location of the test images.

testImagesDir = fullfile(dataSetDir,'testImages');

Define the location of the ground truth labels.

testLabelsDir = fullfile（datasetDir，'testLabels');

Create an imageDatastore holding the test images.

imds = imageDatastore(testImagesDir);

Define the class names and their associated label IDs.

classNames = [“三角形”,"background"]; labelIDs = [255 0];

为测试图像创建一个拿着地面真实像素标签的Pixellabeldatastore。

pxdsTruth = pixelLabelDatastore(testLabelsDir,classNames,labelIDs);

加载语义分割网络，该网络已接受了训练的训练图像triangleImages。

net = load(“三角段网络”); net = net.net;

Run the network on the test images. Predicted labels are written to disk in a temporary directory and returned as a pixelLabelDatastore.

pxdsResults = semanticseg(imds,net,“写入”，tempdir）;
Running semantic segmentation network ------------------------------------- * Processed 100 images.

评估针对地面真理的预测结果。

指标=评估仪分割（PXDSRESULTS，PXDSTRUTH）;
Evaluating semantic segmentation results ---------------------------------------- * Selected metrics: global accuracy, class accuracy, IoU, weighted IoU, BF score. * Processed 100 images. * Finalizing... Done. * Data set metrics: GlobalAccuracy MeanAccuracy MeanIoU WeightedIoU MeanBFScore ______________ ____________ _______ ___________ ___________ 0.90624 0.95085 0.61588 0.87529 0.40652

Display the properties of thesemanticSegmentationMetrics目的。

metrics
指标=具有属性的smanticanticsementicationmetrics：ConfusionMatrix：[2x2表]归一化confusionMatrix：[2x2表] dataSetmetrics：[1x5表] classmetrics：[2x3 Table] ImageMetrics：[100x5表]：[100x5表]

显示分类精度，联合的交叉点以及每个类别的边界F-1分数。这些值存储在ClassMetricsproperty.

指标
ans=2×3桌Accuracy IoU MeanBFScore ________ _______ ___________ triangle 1 0.33005 0.028664 background 0.9017 0.9017 0.78438

显示存储在NormalizedConfusionMatrixproperty.

指标。注册matrix
ans=2×2桌triangle background ________ __________ triangle 4730 0 background 9601 88069

Input Arguments

collapse all

Predicted pixel labels resulting from semantic segmentation, specified as a datastore or a cell array of datastore objects.dsresultscan be any datastore that returns categorical images, such asPixelLabelDatastore或者pixelLabelImageDatastore。The读(dsresults) must return a categorical array, a cell array, or a table. If the读function returns a multicolumn cell array or table, the second column must contain categorical arrays.

Ground truth pixel labels, specified as a datastore or a cell array of datastore objects.dsresultscan be any datastore that returns categorical images, such asPixelLabelDatastore或者pixelLabelImageDatastore。Using读(dsTruth) must return a categorical array, a cell array, or a table. If the读function returns a multicolumn cell array or table, the second column must contain categorical arrays.

Confusion matrix for the classes in the segmented images, specified as one of the following, whereF是数据集中的图像数量。

  • Table withFrows and one variable with the nameConfusionMatrix。表中的每一行都包含一个带有相应图像的混淆矩阵的单元格数组。

  • F- 通过一个单元格数组。Each element of the cell array contains the confusion matrix for the corresponding image.

分段块的混淆矩阵，指定为表格Brows and three columns, whereBis the total number of blocks in all images in the data set. The three columns are the variablesImageNumber,ConfusionMatrix， 和Blockinfo。您可以使用segmentationConfusionMatrix在调用基于块的呼叫中功能applyfunction. For an example, see计算基于块的工作流程中的分割指标。

班级名称, specified as an array of strings or a cell array of character vectors.

Example:["sky" "grass" "building" "sidewalk"]

姓名-Value Arguments

将可选的参数对name1 = value1，...，namen = valuen, where姓名is the argument name and价值is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

在R2021a之前，请使用逗号分隔每个名称和值，并附上姓名用引号。

Example:metrics = evaluateSemanticSegmentation(pxdsResults,pxdsTruth,'Metrics',"bfscore")computes only the mean BF score of each class, each image, and the entire data set.

细分指标insemanticSegmentationMetrics至compute, specified as the comma-separated pair consisting of'Metrics'和一个向量的字符串。这个参数指定了which variables in theDataSetMetrics,ClassMetrics， 和ImageMetricstables to compute.ConfusionMatrix和NormalizedConfusionMatrixare computed regardless of the value of'Metric'。

价值 Description Aggregate Data Set Metric Image Metric Class Metric
"all"

评估所有语义分割指标。

The function excludesMeanBFScorefrom the semantic segmentation metrics when you specify a confusion matrix (imageSetConfusion或者BlocksetConfusion）作为功能的输入。

 All aggregate data set metrics All image metrics 所有班级指标
“准确性”

准确性indicates the percentage of correctly identified pixels for each class. Use the accuracy metric if you want to know how well each class correctly identifies pixels.

  • 对于每个班级，准确性根据地面真相，正确分类的像素与该类别的像素总数的比率。换句话说，

    准确性score = TP / (TP + FN)

    TP是真实阳性的数量，FN是假否定的数量。

  • 对于聚合数据集，MeanAccuracyis the average准确性of all classes in all images.

  • For each image,MeanAccuracyis the average准确性of all classes in that particular image.

类的准确性是类似于全球精度的简单度量，但可能会产生误导。例如，标记所有像素“汽车”为“汽车”类给出了完美的分数（尽管不是其他类别）。将类准确性与IOU结合使用，以更完整地评估分割结果。

 MeanAccuracy MeanAccuracy 准确性
“ BFSCORE”

The boundary F1 (BF) contour matching score indicates how well the predicted boundary of each class aligns with the true boundary. Use the BF score if you want a metric that tends to correlate better with human qualitative assessment than the IoU metric.

  • 对于每个班级，MeanBFScore是所有图像上该类别的平均BF分数。

  • For each image,MeanBFScore是该特定图像中所有类的平均BF分数。

  • 对于聚合数据集，MeanBFScore是所有图像中所有类的平均BF分数。

For more information, seebfscore。

This metric is not available when you specify a confusion matrix (imageSetConfusion或者BlocksetConfusion）作为功能的输入。

 MeanBFScore MeanBFScore MeanBFScore
“全球准确性”

GlobalAccuracyis the ratio of correctly classified pixels, regardless of class, to the total number of pixels. Use the global accuracy metric if you want a quick and computationally inexpensive estimate of the percentage of correctly classified pixels.

 GlobalAccuracy GlobalAccuracy none
"iou"

联合（IOU）的交叉点，也称为Jaccard相似系数，是最常用的度量标准。如果您想要惩罚误报的统计精度测量，请使用IOU指标。

  • 对于每个班级，IoUis the ratio of correctly classified pixels to the total number of ground truth and predicted pixels in that class. In other words,

    IoU score = TP / (TP + FP + FN)

    The image describes the true positives (TP), false positives (FP), and false negatives (FN).

  • For each image,MeanIoUis the average IoU score of all classes in that particular image.

  • 对于聚合数据集，MeanIoUis the average IoU score of all classes in all images.

For more information, seejaccard。

 MeanIoU MeanIoU IoU
"weighted-iou" 每个班级的平均值，由该类中的像素数量加权。如果图像的尺寸不成比例，请使用此度量标准，以减少小类错误对汇总质量评分的影响。 WeightedIoU WeightedIoU none

Example:metrics = evaluateSemanticSegmentation(pxdsResults,pxdsTruth,'Metrics',["global-accuracy","iou"])calculates the global accuracy and IoU metrics across the data set, images, and classes.

Data Types:细绳

标志以在命令窗口中显示评估进度信息，指定为逗号分隔对'Verbose'和either1(真的) or0(false).

The displayed information includes a progress bar, elapsed time, estimated time remaining, and data set metrics.

Example:metrics = evaluateSemanticSegmentation(pxdsResults, pxdsTruth,'Verbose',0)calculates segmentation metrics without displaying progress information.

Data Types:logical

输出参数

collapse all

语义分割指标，返回semanticSegmentationMetrics目的。

基于块的语义分割指标，作为一个F- 通过一个单元阵列，其中F是数据集中的图像数量。单元阵列中的每个元素都包含有关相应图像中所有块的所有指标的信息，该指标格式为表格。

Each table hasK(f) rows, whereK(f) is the number of blocks in thef数据集中的图像。该表最多有五个变量：

  • 表总是包括Blockinfovariable. This table data in this variable are structs that provide spatial information about the block. The four fields of the struct areBlockStartWorld,BlockEndWorld,datastartworld， 和DataEndWorld。有关这些领域的更多信息，请参阅IncludeBlockInfo名称值对参数applyfunction.

  • The table includes the metrics in theDataSetMetricsproperty of theSSMoutput argument. By default, the metrics areGlobalAccuracy,MeanAccuracy,MeanIoU， 和WeightedIoU。However, if you create theSSM并指定指标的子集，以使用Metricsname-value pair argument, then the table includes only the specified metrics.

Tips

  • A value ofNaNin the dataset, class, or image metrics, indicates that one or more classes were missing during the computation of the metrics when using theevaluateSemanticSegmentationfunction. In this case, the software was unable to accurately compute the metrics.

    可以通过查看丢失的课程来找到ClassMetrics属性，为每个类提供指标。要更准确地评估您的网络，请使用包括缺失类别的更多数据来增强地面真相。

References

[1] Csurka, G., D. Larlus, and F. Perronnin. "What is a good evaluation measure for semantic segmentation?"Proceedings of the British Machine Vision Conference, 2013, pp. 32.1–32.11.

扩展功能

Version History

Introduced in R2017b

See Also

Functions

Objects

Topics