Main Content

groundTruthMultisignal

Ground truth label data for multiple signals

Since R2020a

expand all in page

Description

ThegroundTruthMultisignalobject contains information about the ground truth data source, label definitions, and marked label annotations for multiple signals. The source of the signals can be a video, image sequence, lidar point cloud, or any other custom format containing multiple signals. You can export or import agroundTruthMultisignalobject from theGround Truth Labelerapp.

To create training data for deep learning applications from arrays ofgroundTruthMultisignal对象, use thegatherLabelDatafunction.

创建

To export agroundTruthMultisignalobject from theGround Truth Labelerapp, on the app toolstrip, selectExport Labels>To Workspace。The app exports the object to the MATLAB®workspace. To create agroundTruthMultisignalobject programmatically, use thegroundTruthMultisignalfunction (described here).

Syntax

gTruth = groundTruthMultisignal(dataSources,labelDefs,roiData,sceneData)

Description

example

gTruth= groundTruthMultisignal (dataSources,labelDefs,roiData,sceneData)returns an object containing ground truth labels that can be imported into theGround Truth Labelerapp.

  • dataSourcesspecifies the sources of the ground truth data and sets theDataSourceproperty.

  • labelDefsspecifies the label, sublabel, and attribute definitions of the ground truth data and sets theLabelDefinitionsproperty.

  • roiDataspecifies the identifying information, position, and timestamps for the marked region of interest (ROI) labels and sets theROILabelDataproperty.

  • sceneDataspecifies the identifying information and timestamps for marked scene labels and sets theSceneLabelDataproperty.

Properties

expand all

Sources of ground truth data, specified as a vector ofMultiSignalSource对象。These objects contain information that describe the sources from which ground truth data was labeled. This table describes the type ofMultiSignalSource对象that you can specify in this vector.

MultiSignalSourceObject Type Data Source Class Reference
VideoSource Video file vision.labeler.loading.VideoSource
ImageSequenceSource Image sequence folder vision.labeler.loading.ImageSequenceSource
VelodyneLidarSource Velodyne®packet capture (PCAP) file vision.labeler.loading.VelodyneLidarSource
RosbagSource Rosbag file vision.labeler.loading.RosbagSource
PointCloudSequenceSource Point cloud sequence folder vision.labeler.loading.PointCloudSequenceSource
CustomImageSource Custom image format vision.labeler.loading.CustomImageSource

To specify additional data sources, create a new type ofMultiSignalSourceobject by using thevision.labeler.loading.MultiSignalSourceclass.

Label definitions, specified as a table. To create this table, use one of these options.

  • 在Ground Truth Labelerapp, create label definitions, and then export them as part of agroundTruthMultisignalobject.

  • Use alabelDefinitionCreatorMultisignalobject to generate a label definitions table. If you save this table to a MAT-file, you can then load the label definitions into aGround Truth Labelerapp session by selectingOpen>Label Definitionsfrom the app toolstrip.

  • Create the label definitions table at the MATLAB command line.

This table describes the required and optional columns of the table specified in theLabelDefinitionsproperty.

Column Description Required or Optional
Name 字符串或字符vectors specifying the name of each label definition.

Required
SignalType

SignalTypeenumerations that specify the signal type supported for each label definition. Valid values areImagefor image signals such as videos or image sequences,PointCloudfor lidar signals, orTime对现场标签定义。

If a label definition supports multiple signal types, then the label definition has a separate row for each signal type. For example, consider a label definition namedcar。在Ground Truth Labelerapp, you draw this label as a rectangle in image signals and a cuboid in lidar point cloud signals. In theLabelDefinitionstable,carappears twice and has theseName,SignalType, andLabelTypevalues.

Name SignalType LabelType
'car' Image Rectangle
'car' PointCloud Cuboid

Required
LabelType

labelTypeenumerations that specify the type of each label definition.

For ROI label definitions with aSignalTypeofImage, validLabelTypeenumerations are:

  • labelType.Rectangle

  • labelType.Line

  • labelType.PixelLabel

  • labelType.Polygon

  • labelType.ProjectedCuboid

For ROI label definitions with aSignalTypeofPointCloud, the only validLabelTypeenumeration islabelType.Cuboid。

For scene label definitions, the only validLabelTypeenumeration islabelType.Scene。

You can also add ground truth data that is not an ROI or scene label to agroundTruthMultisignalobject. In the label definitions table, specify a label definition whoselabelTypeisCustom。

Required
Group 字符串或字符vectors specifying the group to which each label definition belongs.

Optional

If you create label definitions at the MATLAB command line, you do not need to include aGroupcolumn.

If you export label definitions from theGround Truth Labelerapp or create them using alabelDefinitionCreatorMultisignalobject, the label definitions table includes this column, even if you did not specify groups. The app assigns each label definition aGroupvalue of'None'。
Description 字符串或字符vectors that describe each label definition.

Optional

If you create label definitions at the MATLAB command line, you do not need to include aDescriptioncolumn.

If you export label definitions from theGround Truth Labelerapp or create them using alabelDefinitionCreatorMultisignalobject, the label definitions table includes this column, even if you did not specify descriptions. TheDescriptionfor these label definitions is an empty character vector.
LabelColor 1-by-3 row vectors of RGB triplets that specify the colors of the label definitions. Values are in the range [0, 1]. The color yellow (RGB triplet [1 1 0]) is reserved for the color of selected labels in theGround Truth Labelerapp.

Optional

When you define labels in theGround Truth Labelerapp, you must specify a color. Therefore, an exported label definitions table always includes this column.

When you create label definitions using thelabelDefinitionCreatorMultisignalobject without specifying colors, the returned label definition table includes this column, but all column values are empty.
PixelLabelID Scalars, column vectors,M-by-3 matrices of integer-valued label IDs.PixelLabelIDspecifies the pixel label values used to represent a label definition. Pixel label ID values must be between 0 and 255.

Optional

When you define pixel labels in theGround Truth Labelerapp or thelabelDefinitionCreatorMultisignalobject, the generated label definitions table includes this column.

When creating a label definitions table at the MATLAB command line, if you setLabelTypetolabelType.PixelLabelfor any label, then this column is required.
Hierarchy Structures containing sublabel and attribute data for each label definition. For an example of theHierarchyformat, seeExport and Explore Ground Truth Labels for Multiple Signals。

Optional

When you define sublabels or attributes in theGround Truth Labelerapp or thelabelDefinitionCreatorMultisignalobject, the generated label definitions table includes this column.

ROI label data across all signals, specified as anROILabelDataobject.

ForRectangle,Cuboid,ProjectedCuboid,Polygon, andLinelabel types, ground truth data that is not a floating-point array has a data type ofsingle。

Scene label data across all signals, specified as aSceneLabelDataobject.

Object Functions

selectLabelsByLabelName Select multisignal ground truth by label name
selectLabelsByLabelType Select multisignal ground truth by label type
selectLabelsByGroupName Select multisignal ground truth by label group name
selectLabelsBySignalName Select multisignal ground truth by signal name
selectLabelsBySignalType Select multisignal ground truth labels by signal type
gatherLabelData Gather synchronized label data from ground truth
writeFrames Write signal frames for ground truth data to disk
changeFilePaths Change file paths in multisignal ground truth data

Examples

collapse all

Open Script

Create ground truth data for a video signal and a lidar point cloud sequence signal that captures the same driving scene. Specify the signal sources, label definitions, and ROI and scene label data.

Create the video data source from an MP4 file.

sourceName ='01_city_c2s_fcw_10s.mp4'; sourceParams = []; vidSource = vision.labeler.loading.VideoSource; vidSource.loadSource(sourceName,sourceParams);

Create the point cloud sequence source from a folder of point cloud data (PCD) files.

pcSeqFolder =全file(toolboxdir('driving'),'drivingdata','lidarSequence'); addpath(pcSeqFolder) loadtimestamps.matrmpath(pcSeqFolder) lidarSourceData = load(fullfile(pcSeqFolder,'timestamps.mat')); sourceName = pcSeqFolder; sourceParams = struct; sourceParams.Timestamps = timestamps; pcseqSource = vision.labeler.loading.PointCloudSequenceSource; pcseqSource.loadSource(sourceName,sourceParams);

Combine the signal sources into an array.

dataSource = [vidSource pcseqSource]
dataSource = 1x2 heterogeneous MultiSignalSource (VideoSource, PointCloudSequenceSource) array with properties: SourceName SourceParams SignalName SignalType Timestamp NumSignals

Create a table of label definitions for the ground truth data by using alabelDefinitionCreatorMultisignalobject.

  • TheCarlabel definition appears twice. Even thoughCaris defined as a rectangle, you can draw rectangles only for image signals, such as videos. ThelabelDefinitionCreatorMultisignalobject creates an additional row for lidar point cloud signals. In these signal types, you can drawCarlabels as cuboids only.

  • 洛杉矶bel definitions have no descriptions and no assigned colors, so theDescriptionandLabelColorcolumns are empty.

  • 洛杉矶bel definitions have no assigned groups, so for all label definitions, the corresponding cell in theGroupcolumn is set to'None'。

  • Roadis a pixel label definition, so the table includes aPixelLabelIDcolumn.

  • 没有标签定义sublabels或属性,so the table does not include aHierarchycolumn for storing such information.

ldc = labelDefinitionCreatorMultisignal; addLabel(ldc,'Car','Rectangle'); addLabel(ldc,'Truck','ProjectedCuboid'); addLabel(ldc,'Lane','Line'); addLabel(ldc,'Road','PixelLabel'); addLabel(ldc,'Sunny','Scene'); labelDefs = create(ldc)
labelDefs = 7x7 table Name SignalType LabelType Group Description LabelColor PixelLabelID _________ __________ _______________ ________ ___________ __________ ____________ {'Car' } Image Rectangle {'None'} {' '} {0x0 char} {0x0 double} {'Car' } PointCloud Cuboid {'None'} {' '} {0x0 char} {0x0 double} {'Truck'} Image ProjectedCuboid {'None'} {' '} {0x0 char} {0x0 double} {'Lane' } Image Line {'None'} {' '} {0x0 char} {0x0 double} {'Lane' } PointCloud Line {'None'} {' '} {0x0 char} {0x0 double} {'Road' } Image PixelLabel {'None'} {' '} {0x0 char} {[ 1]} {'Sunny'} Time Scene {'None'} {' '} {0x0 char} {0x0 double}

Create ROI label data for the first frame of the video.

numVideoFrames = numel(vidSource.Timestamp{1}); carData = cell(numVideoFrames,1); laneData = cell(numVideoFrames,1); truckData = cell(numVideoFrames,1); carData{1} = [304 212 37 33]; laneData{1} = [70 458; 311 261]; truckData{1} = [309,215,33,24,330,211,33,24]; videoData = timetable(vidSource.Timestamp{1},carData,laneData,。..'VariableNames',{'Car','Lane'});

Create ROI label data for the first point cloud in the sequence.

numPCFrames = numel(pcseqSource.Timestamp{1}); carData = cell(numPCFrames, 1); carData{1} = [27.35 18.32 -0.11 4.25 4.75 3.45 0 0 0]; lidarData = timetable(pcseqSource.Timestamp{1},carData,'VariableNames',{'Car'});

Combine the ROI label data for both sources.

signalNames = [dataSource.SignalName]; roiData = vision.labeler.labeldata.ROILabelData(signalNames,{videoData,lidarData})
roiData = ROILabelData with properties: video_01_city_c2s_fcw_10s: [204x2 timetable] lidarSequence: [34x1 timetable]

Create scene label data for the first 10 seconds of the driving scene.

sunnyData = seconds([0 10]); labelNames = ["Sunny"]; sceneData = vision.labeler.labeldata.SceneLabelData(labelNames,{sunnyData})
sceneData = SceneLabelData with properties: Sunny: [0 sec 10 sec]

Create a ground truth object from the signal sources, label definitions, and ROI and scene label data. You can import this object into theGround Truth Labelerapp for manual labeling or to run a labeling automation algorithm on it. You can also extract training data from this object for deep learning models by using thegatherLabelDatafunction.

gTruth = groundTruthMultisignal(dataSource,labelDefs,roiData,sceneData)
gTruth = groundTruthMultisignal with properties: DataSource: [1x2 vision.labeler.loading.MultiSignalSource] LabelDefinitions: [7x7 table] ROILabelData: [1x1 vision.labeler.labeldata.ROILabelData] SceneLabelData: [1x1 vision.labeler.labeldata.SceneLabelData]

Tips

  • groundTruthMultisignal对象with video-based data sources rely on the video reading capabilities of your operating system. AgroundTruthMultisignalobject created using video data sources remains consistent only for the same platform that was used to create it. To create a platform-independentgroundTruthMultisignalobject, convert the videos into sequences of images and include the associated timestamps with the image sequences.

  • To create agroundTruthMultisignalobject containing ROI label data but no scene label data, specify theSceneLabelDataproperty as an empty array. To create this array, at the MATLAB command prompt, enter this code.

    sceneData = vision.labeler.labeldata.SceneLabelData.empty

Version History

Introduced in R2020a

See Also

Apps

Objects

Topics