Main Content

gatherLabelData

Gather synchronized label data from ground truth

Description

labelData= gatherLabelData(gTruth,signalNames,labelTypes)returns synchronized label data gathered from multisignal ground truth data,gTruth。The function returns label data for the signals specified bysignalNamesand the label types specified bylabelTypes

example

[labelData,timestamps] = gatherLabelData(___)additionally returns the signal timestamps associated with the gathered label data, using the arguments from the previous syntax.

Usetimestampswith thewriteFramesfunction to write the associated signal frames from thegroundTruthMultisignal对象to disk. Use these frames and the associated labels as training data for machine learning or deep learning models.

[___] = gatherLabelData(___,'SampleFactor',sampleFactor)specifies the sample factor used to subsample label data.

Examples

全部折叠

Gather label data for a video signal and a lidar point cloud sequence signal from agroundTruthMultisignalobject. Write the signal frames associated with that label data to disk and visualize the frames.

添加的点云序列文件夹路径TLAB® search path. The video is already on the MATLAB search path.

pcSeqDir =局部e(toolboxdir('driving'),'drivingdata',。..'lidarSequence'); addpath(pcSeqDir);

Load agroundTruthMultisignalobject that contains label data for the video and the lidar point cloud sequence.

data = load('MultisignalGTruth.mat'); gTruth = data.gTruth;

Specify the signals from which to gather label data.

signalNames = ["video_01_city_c2s_fcw_10s""lidarSequence"];

The video contains rectangle labels, whereas the lidar point cloud sequence contains cuboid labels. Gather the rectangle labels from the video and the cuboid labels from the lidar point cloud sequence.

labelTypes = [labelType.Rectangle labelType.Cuboid]; [labelData,timestamps] = gatherLabelData(gTruth,signalNames,labelTypes);

Display the first eight rows of label data from the two signals. Both signals contain data for theCarlabel. In the video, theCarlabel is drawn as a rectangle bounding box. In the lidar point cloud sequence, theCarlabel is drawn as a cuboid bounding box.

videoLabelSample = head(labelData{1}) lidarLabelSample = head(labelData{2})
videoLabelSample = table Car _________________ {[299 213 42 33]} lidarLabelSample = table Car ____________________________________________________ {[17.7444 6.7386 3.3291 3.6109 3.2214 3.5583 0 0 0]}

Write signal frames associated with the gathered label data to temporary folder locations, with one folder per signal. Use the timestamps returned by thegatherLabelDatafunction to indicate which signal frames to write.

outputFolder = fullfile(tempdir,["videoFrames""lidarFrames"]); fileNames = writeFrames(gTruth,signalNames,outputFolder,timestamps);
Writing 2 frames from the following signals: * video_01_city_c2s_fcw_10s * lidarSequence

Load the written video signal frames by using animageDatastoreobject. Load the associated rectangle label data by using aboxLabelDatastoreobject.

imds = imageDatastore(fileNames{1}); blds = boxLabelDatastore(labelData{1});

Load the written lidar signal frames by using afileDatastoreobject. Load the associated cuboid label data by using aboxLabelDatastoreobject.

fds = fileDatastore(fileNames{2},'ReadFcn',@pcread); clds = boxLabelDatastore(labelData{2});

Visualize the written video frames by using avision.VideoPlayerobject. Visualize the written lidar frames by using apcplayerobject.

videoPlayer = vision.VideoPlayer; ptCloud = preview(fds); ptCloudPlayer = pcplayer(ptCloud.XLimits,ptCloud.YLimits,ptCloud.ZLimits);whilehasdata(imds)% Read video and lidar frames.I = read(imds); ptCloud = read(fds);% Visualize video and lidar frames.videoPlayer(I); view(ptCloudPlayer,ptCloud);end

Remove the path to the point cloud sequence folder.

rmpath(pcSeqDir);

Input Arguments

全部折叠

Multisignal ground truth data, specified as agroundTruthMultisignalobject or vector ofgroundTruthMultisignal对象。

EachgroundTruthMultisignalobject ingTruthmust include all the signals specified in thesignalNamesinput.

In addition, each object must include at least one marked label per gathered label definition. SupposegTruthis agroundTruthMultisignal对象包含标签数据or a single video signal namedvideo_front_camera。The object contains marked rectangle region of interest (ROI) labels for thecarlabel definition but not for thetrucklabel definition. If you use this syntax to gather labels of typeRectanglefrom this object, then thegatherLabelDatafunction returns an error.

labelData = gatherLabelData(gTruth,"video_front_camera",labelType.Rectangle);

Names of the signals from which to gather label data, specified as a character vector, string scalar, cell array of character vectors, or string vector. The signal names must be valid signal names stored in the input multisignal ground truth data,gTruth

To obtain the signal names from agroundTruthMultisignalobject, use this syntax, wheregTruthis the variable name of the object:

gTruth.DataSource.SignalName

Example:'video_01_city_c2s_fcw_10s'

Example:"video_01_city_c2s_fcw_10s"

Example:{'video_01_city_c2s_fcw_10s','lidarSequence'}

Example:["video_01_city_c2s_fcw_10s" "lidarSequence"]

Label types from which to gather label data, specified as alabelTypeenumeration scalar,labelTypeenumeration vector, or a cell array oflabelTypeenumeration scalars and vectors. ThegatherLabelDatafunction gathers label data for each signal specified by inputsignalNamesand eachgroundTruthMultisignalobject specified by inputgTruth。The number of elements inlabelTypesmust match the number of signals insignalNames

Gather Label Data for Single Label Type per Signal

To gather label data for a single label type per signal, specifylabelTypesas alabelTypeenumeration scalar or vector. Across allgroundTruthMultisignal对象ingTruth,gatherLabelDatafunction gatherslabelTypes(n)label data fromsignalName(n), wherenis the index of the label type and the corresponding signal name whose label data is to be gathered. Each returned table in the outputlabelDatacell array contains data for only one label type per signal.

In this code sample, thegatherLabelDatafunction gathers labels of typeRectanglefrom a video signal namedvideo_front_camera。The function also gathers labels of typeCuboidfrom a lidar point cloud sequence signal stored in a folder namedlidarData。ThegTruthinput contains thegroundTruthMultisignal对象from which this data is to be gathered.

labelData = gatherLabelData(gTruth,。..["video_front_camera","lidarData"],。..[labelType.Rectangle,labelType.Cuboid];

To gather label data for a single label type from separate signals, you must repeat the label type for each signal. In this code sample, thegatherLabelDatafunction gathers labels of typeRectanglefrom thevideo_left_cameraandvideo_right_cameravideo signals.

labelData = gatherLabelData(gTruth,。..["video_left_camera","video_right_camera"],。..[labelType.Rectangle,labelType.Rectangle];

Gather Label Data for Multiple Label Types per Signal

To gather label data for multiple label types per signal, specifylabelTypesas a cell array oflabelTypeenumeration scalars and vectors. Across allgroundTruthMultisignal对象ingTruth,gatherLabelDatafunction gatherslabelTypes{n}label data fromsignalName(n), wherenis the index of the label types and the corresponding signal name whose label data is to be gathered. The function groups the data for these label types into one table per signal pergroundTruthMultisignalobject.

In this code sample, thegatherLabelDatafunction gathers labels of typeRectangleandLinefrom thevideo_front_cameravideo signal. The function also gathers labels of typeCuboidfrom a lidar point cloud sequence signal stored in a folder namedlidarData。ThegTruthinput contains thegroundTruthMultisignal对象from which this data is to be gathered.

labelData = gatherLabelData(gTruth,。..["video_front_camera",。.."lidarData"],。..{[labelType.Rectangle labelType.Line],。..labelType.Cuboid};

Valid Enumeration Types

You can specify one or more of these enumeration types.

  • labelType.Rectangle— Rectangle ROI labels

  • labelType.Cuboid— Cuboid ROI labels (point clouds)

  • labelType.ProjectedCuboid— Projected cuboid ROI labels (images and video data)

  • labelType.Line— Line ROI labels

  • labelType.PixelLabel— Pixel ROI labels

  • labelType.Polygon— Pixel ROI labels

  • labelType.Scene— Scene labels

To gather label data for scenes, you must specifylabelTypesas thelabelType.Sceneenumeration scalar. You cannot specify any other label types withlabelType.Scene

Sample factor used to subsample label data, specified as a positive integer. A sample factor ofKincludes everyKth signal frame. Increase the sample factor to drop redundant frames from signals with high sample rates, such as videos.

Example:'SampleFactor',5

Output Arguments

全部折叠

Label data, returned as anM-by-Ncell array of tables, where:

  • Mis the number ofgroundTruthMultisignal对象ingTruth

  • WhenlabelTypescontains ROIlabelTypeenumerations,Nis the number of signals insignalNamesand the number of elements inlabelTypes。In this case,labelData{m,n}contains a table of label data for thenth signal ofsignalNamesthat is in themthgroundTruthMultisignalobject ofgTruth。The table contains label data for only the label types in thenth position oflabelTypes

  • WhenlabelTypescontains only thelabelType.Sceneenumeration,Nis equal to1。In this case,labelData{m}contains a table of scene label data across all signals in themthgroundTruthMultisignalobject ofgTruth

For a given label data table,tbl,table is of sizeT-by-L, where:

  • Tis the number of timestamps in the signal for which label data exists.

  • Lis the number of label definitions that are of the label types gathered for that signal.

  • tbl(t,l)contains the label data gathered for thelth label at thetth timestamp.

If one of the signals has no label data at a timestamp, then the corresponding label data table does not include a row for that timestamp.

For each cell in the table, the format of the returned label data depends on the type of label.

Label Type Storage Format for Labels at Each Timestamp
labelType.Rectangle

M-by-4 numeric matrix of the form[x, y, w, h], where:

  • Mis the number of labels in the frame.

  • xandyspecify the upper-left corner of the rectangle.

  • wspecifies the width of the rectangle, which is its length along thex-axis.

  • hspecifies the height of the rectangle, which is its length along they-axis.

labelType.Cuboid

M-by-9 numeric matrix with rows of the form[xctr, yctr, zctr, xlen, ylen, zlen, xrot, yrot, zrot], where:

  • Mis the number of labels in the frame.

  • xctr,yctr, andzctrspecify the center of the cuboid.

  • xlen,ylen, andzlenspecify the length of the cuboid along thex-axis,y-axis, andz-axis, respectively, before rotation has been applied.

  • xrot,yrot, andzrotspecify the rotation angles for the cuboid along thex-axis,y-axis, andz-axis, respectively. These angles are clockwise-positive when looking in the forward direction of their corresponding axes.

The figure shows how these values determine the position of a cuboid.

labelType.ProjectedCuboid

M-by-8 vector of the form[x1, y1, w1, h1, x2, y2, w2, h2], where:

  • Mis the number of labels in the frame.

  • x1, y1specifies thex,ycoordinates for the upper-left location of the front-face of the projected cuboid

  • w1specifies the width for the front-face of the projected cuboid.

  • h1specifies the height for the front-face of the projected cuboid.

  • x2, y2specifies thex,ycoordinates for the upper-left location of the back-face of the projected cuboid.

  • w2specifies the width for the back-face of the projected cuboid.

  • h2specifies the height for the back-face of the projected cuboid.

The figure shows how these values determine the position of a cuboid.

Labeled projected cuboid

labelType.Line

M-by-1 vector of cell arrays, whereMis the number of labels in the frame. Each cell array contains anN-by-2 numeric matrix of the form[x1 y1; x2 y2; ... ; xN yN]forNpoints in the polyline.

labelType.PixelLabel

Label data for all pixel label definitions is stored in a singleM-by-1PixelLabelDatacolumn forMimages or frames. Each element contains a filename for a pixel label image. A pixel label image describes the label or labels contained in the corresponding image. The labels can be described as a 1- or 3- channel label matrix. To usePixelLabelDatawith any of the labeler apps, you must use a single-channel label matrix, where the values are of typeuint8。你可以一个三路像素标签数据转换垫rix to a single-channel label matrix programmatically to use with the labeler apps.

labelType.Polygon

M-by-1 vector of cell arrays, whereMis the number of labels. Each cell array contains anN-by-2 numeric matrix of the form[x1 y1; x2 y2; ... ; xN yN]forNpoints in the polygon.

labelType.Scene Logical1(true) if the scene label is applied, otherwise logical0(false)

Label Data Format

Consider a cell array of label data gathered by using thegatherLabelDatafunction. The function gathers labels from threegroundTruthMultisignal对象with variable namesgTruth1,gTruth2, andgTruth3

  • For a video signal namedvideo_front_camera,function gathers labels of typeRectangleandLine

  • For a lidar point cloud sequence signal stored in a folder namedlidarData,function gathers labels of typeCuboid

This code shows the call to thegatherLabelDatafunction.

labelData = gatherLabelData([gTruth1 gTruth2 gTruth3],。..["video_front_camera",。.."lidarData"],。..{[labelType.Rectangle labelType.Line],。..labelType.Cuboid};
ThelabelDataoutput is a 3-by-2 cell array of tables. Each row of the cell array contains label data for one of thegroundTruthMultisignal对象。The first column contains the label data for the video signal,video_front_camera。The second column contains the label data for the point cloud sequence signal,lidarData。This figure shows thelabelDatacell array.

This figure shows the label data table for the video signal in the thirdgroundTruthMultisignalobject. ThegatherLabelDatafunction gathered data for aRectanglelabel namedcarand aLinelabel namedlane。The table contains label data at four timestamps in the signal.

This figure shows the label data table for the lidar signal in the thirdgroundTruthMultisignalobject. ThegatherLabelDatafunction gathered data for aCuboidlabel, also namedcar。Thecarlabel appears in both signal types because it is marked as aRectanglelabel for video signals and aCuboidlabel for lidar signals. The table contains label data at four timestamps in the signal.

Signal timestamps, returned as anM-by-Ncell array ofdurationvectors, where:

  • Mis the number ofgroundTruthMultisignal对象ingTruth

  • Nis the number of signals insignalNames

  • labelData{m,n}contains the timestamps for thenth signal ofsignalNamesthat is in themthgroundTruthMultisignalobject ofgTruth

If you gather label data from multiple signals, the signal timestamps are synchronized to the timestamps of the first signal specified bysignalNames

Limitations

  • ThegatherLabelDatafunction does not gather label data for sublabels or attributes. If a label contains sublabels or attributes, in thelabelDataoutput, the function returns the position of the parent label only.

Introduced in R2020a