Model Setup

The model used in this example simulates a vehicle driving in a city scene.

Depth Visualization

A depth map is a grayscale representation of camera sensor output. These maps visualize camera images in grayscale, with brighter pixels indicating objects that are farther away from the sensor. You can use depth maps to validate depth estimation algorithms for your sensors.

TheDepth港模拟3 d相机块输出depth map of values in the range of 0 to 1000 meters. In this model, for better visibility, a Saturation block saturates the depth output to a maximum of 150 meters. Then, a Gain block scales the depth map to the range [0, 1] so that the To Video Display block can visualize the depth map in grayscale.

Semantic Segmentation Visualization

Semantic segmentationdescribes the process of associating each pixel of an image with a class label, such asroad,building, ortraffic sign. In the 3D simulation environment, you generate synthetic semantic segmentation data according to a label classification scheme. You can then use these labels to train a neural network for UAV flight applications, such as landing zone identification. By visualizing the semantic segmentation data, you can verify your classification scheme.

TheLabelsport of the Simulation 3D Camera block outputs a set of labels for each pixel in the output camera image. Each label corresponds to an object class. For example, in the default classification scheme used by the block,1corresponds to buildings. A label of0refers to objects of an unknown class and appears as black. For a complete list of label IDs and their corresponding object descriptions, see theLabelsport description on theSimulation 3D Camerablock reference page.

The MATLAB® Function block uses thelabel2rgb(Image Processing Toolbox)function to convert the labels to a matrix of RGB triplets for visualization. The colormap is based on the colors used in the CamVid dataset, as shown in theSemantic Segmentation Using Deep Learning(计算机视觉工具箱)example. The colors are mapped to the predefined label IDs used in the default Unreal Engine simulation scenes. The helper functionsim3dColormapdefines the colormap. Inspect these colormap values.

opensim3dColormap.m

Model Simulation

Run the model.

sim卡('uav_ue4_depth_imaging.slx');

When the simulation begins, it can take a few seconds for the visualization engine to initialize, especially when you are running it for the first time. TheMathWorks_Aerospacewindow displays the scene from the scene origin. In this scene, the quadrotor UAV flies a short distance down one city block.

The Camera Display, Depth Display, and Semantic Segmentation Display blocks display the outputs from the camera sensor.

To change the visualization range of the output depth data, try updating the values in the Saturation and Gain blocks.

To change the semantic segmentation colors, try modifying the color values defined in thesim3dColormapfunction. Alternatively, in the uavlabel2rgbMATLAB Function block, try replacing the input colormap with your own colormap or a predefined colormap. Seecolormap.