Introduction

A highway lane-following system steers a vehicle to travel within a marked lane. It also maintains a set velocity or safe distance from a preceding vehicle in the same lane. The system typically includes lane detection, sensor fusion, decision logic, and controls components. System-level simulation is a common technique for assessing functionality of the integrated components. Simulations are configured to test scenarios based on system requirements. Automatically running these simulations enables regression testing to verify system-level functionality.

TheHighway Lane Following(Automated Driving Toolbox)example showed how to simulate a system-level model for lane-following. This example shows how to automate testing that model against multiple scenarios using Simulink Test™. The scenarios are based on system-level requirements. In this example, you will:

Testing the system-level model requires a photorealistic simulation environment. In this example, you enable system-level simulation through integration with the Unreal Engine from Epic Games®. The 3D simulation environment requires a Windows® 64-bit platform.

if~ispc error("The 3D simulation environment requires a Windows 64-bit platform");end

To ensure reproducibility of the simulation results, set the random seed.

rng(0);

Review Requirements

Simulink Requirements™ lets you author, analyze, and manage requirements within Simulink. This example contains ten test scenarios, with high-level testing requirements defined for each scenario. Open the requirement set.

To explore the test requirements and test bench model, open a working copy of the project example files. MATLAB copies the files to an example folder so that you can edit them. TheTestAutomationfolder contains the files that enables the automate testing.

addpath(fullfile(matlabroot,'toolbox','driving','drivingdemos')); helperDrivingProjectSetup('HighwayLaneFollowing.zip','workDir', pwd);

open('HighwayLaneFollowingTestRequirements.slreqx')

Alternatively, you can also open the file from theRequirementstab of the Requirements Manager app in Simulink.

Each row in this file specifies the requirements in textual and graphical formats for testing the lane-following system for a test scenario. The scenarios with thescenario_LF_prefix enable you to test lane-detection and lane-following algorithms without obstruction by other vehicles. The scenarios with thescenario_LFACC_prefix enable you to test lane-detection, lane-following, and ACC behavior with other vehicles on the road.

These requirements are implemented as test scenarios with the same names as the scenarios used in theHighwayLaneFollowingTestBenchmodel.

Review Test Bench Model

This example reuses theHighwayLaneFollowingTestBenchmodel from theHighway Lane Following(Automated Driving Toolbox)example. Open the test bench model.

open_system("HighwayLaneFollowingTestBench");

This test bench model hasSimulation 3D Scenario,Lane Marker Detector,Vehicle Detector,Forward Vehicle Sensor Fusion,Lane Following Decision Logic和Lane Following Controller和Vehicle Dynamicscomponents.

This test bench model is configured using thehelperSLHighwayLaneFollowingSetupscript. This setup script takesscenarioNameas input.scenarioNamecan be any one of the previously described test scenarios. To run the setup script, use code:

scenarioName ="scenario_LFACC_03_Curve_StopnGo"; helperSLHighwayLaneFollowingSetup("scenarioFcnName",scenarioName);

You can now simulate the model and visualize the results. For more details on the analysis of the simulation results and the design of individual components in the test bench model, see theHighway Lane Following(Automated Driving Toolbox)example.

在这个例子中,更多的关注于自动化the simulation runs for this test bench model using Simulink Test for the different test scenarios. TheMetrics Assessmentsubsystem enables integration of system-level metric evaluations with Simulink Test. This subsystem usesCheck Static Rangeblocks for this integration. Open theMetrics Assessmentsubsystem.

open_system("HighwayLaneFollowingTestBench/Metrics Assessment");

In this example, four metrics are used to assess the lane-following system.

Disable Runtime Visualizations

The system-level test bench model visualizes intermediate outputs during the simulation for the analysis of different components in the model. These visualizations are not required when the tests are automated. You can reduce execution time for the automated testing by disabling them.

Disable runtime visualizations for theLane Marker Detectorsubsystem.

load_system('LaneMarkerDetector'); blk ='LaneMarkerDetector/Lane Marker Detector'; set_param(blk,'EnableDisplays','off');

Disable runtime visualizations for theVehicle Detectorsubsystem.

load_system('VisionVehicleDetector'); blk ='VisionVehicleDetector/Vision Vehicle Detector/ACF/ACF'; set_param(blk,'EnableDisplay','off');

Configure theSimulation 3D Scene Configuration(Automated Driving Toolbox)block to run the Unreal Engine in headless mode, where the 3D simulation window is disabled.

blk = ['HighwayLaneFollowingTestBench/Simulation 3D Scenario/',...'Simulation 3D Scene Configuration']; set_param(blk,'EnableWindow','off');

Automate Testing

The Test Manager is configured to automate the testing of the lane-following application. Open theHighwayLaneFollowingTestAssessments.mldatxtest file in the Test Manager.

sltestmgr; testFile = sltest.testmanager.load('HighwayLaneFollowingTestAssessments.mldatx');

Observe the populated test cases that were authored previously in this file. Each test case is linked to the corresponding requirement in the Requirements Editor for traceability. Each test case uses thePOST-LOADcallback to run the setup script with appropriate inputs and to configure the output video file name. After the simulation of the test case, it invokeshelperGenerateFilesForLaneFollowingReportfrom theCLEAN-UPcallback to generate the plots explained in theHighway Lane Following(Automated Driving Toolbox)example.

Run and explore results for a single test scenario:

To reduce command-window output, turn off the MPC update messages.

mpcverbosity('off');

To test the system-level model with thescenario_LFACC_03_Curve_StopnGotest scenario from Simulink Test, use this code:

testSuite = getTestSuiteByName(testFile,'Test Scenarios'); testCase = getTestCaseByName(testSuite,'scenario_LFACC_03_Curve_StopnGo'); resultObj = run(testCase);

To generate a report after the simulation, use this code:

sltest.testmanager.report(resultObj,'Report.pdf',...,'Title','Highway Lane Following',...“IncludeMATLABFigures”,true,...'IncludeErrorMessages',true,...'IncludeTestResults',0,'LaunchReport',true);

Examine the reportReport.pdf. Observe that theTest environmentsection shows the platform on which the test is run and the MATLAB® version used for testing. TheSummarysection shows the outcome of the test and duration of the simulation in seconds. TheResultssection shows pass/fail results based on the assessment criteria. This section also shows the plots logged from thehelperGenerateFilesForLaneFollowingReportfunction.

Run and explore results for all test scenarios:

You can simulate the system for all the tests by usingsltest.testmanager.run. Alternatively, you can simulate the system by clickingPlayin the Test Manager app.

After completion of the test simulations, the results for all the tests can be viewed in theResults and Artifactstab of the Test Manager. For each test case, theCheck Static Rangeblocks in the model are associated with the Test Manager to visualize overall pass/fail results.

You can find the generated report in current working directory. This report contains a detailed summary of pass/fail statuses and plots for each test case.

Verify test status in Requirements Editor:

Open the Requirements Editor and selectDisplay. Then, selectVerification Statusto see a verification status summary for each requirement. Green and red bars indicate the pass/fail status of simulation results for each test.

Automate Testing with Generated Code

TheHighwayLaneFollowingTestBenchmodel enables integrated testing ofLane Marker Detector,Vehicle Detector,Forward Vehicle Sensor Fusion,Lane Following Decision Logic, andLane Following Controllercomponents. It is often helpful to perform regression testing of these components through software-in-the-loop (SIL) verification. If you have Embedded Coder™ Simulink Coder™ license, then you can generate code for these components. This workflow lets you verify that the generated code produces expected results that match the system-level requirements throughout simulation.

SetLane Marker Detectorto run in Software-in-the-loop mode.

model ='HighwayLaneFollowingTestBench/Lane Marker Detector'; set_param(model,“SimulationMode”,'Software-in-the-loop');

SetVehicle Detectorto run in Software-in-the-loop mode.

model ='HighwayLaneFollowingTestBench/Vehicle Detector'; set_param(model,“SimulationMode”,'Software-in-the-loop');

SetForward Vehicle Sensor Fusionto run in Software-in-the-loop mode.

model ='HighwayLaneFollowingTestBench/Forward Vehicle Sensor Fusion'; set_param(model,“SimulationMode”,'Software-in-the-loop');

SetLane Following Decision Logicto run in Software-in-the-loop mode.

model ='HighwayLaneFollowingTestBench/Lane Following Decision Logic'; set_param(model,“SimulationMode”,'Software-in-the-loop');

SetLane Following Controllerto run in Software-in-the-loop mode.

model ='HighwayLaneFollowingTestBench/Lane Following Controller'; set_param(model,“SimulationMode”,'Software-in-the-loop');

Now, runsltest.testmanager.runto simulate the system for all the test scenarios. After the completion of tests, review the plots and results in the generated report.

Enable the MPC update messages again.

mpcverbosity('on');

Automate Testing in Parallel

If you have a Parallel Computing Toolbox™ license, then you can configure Test Manager to execute tests in parallel using a parallel pool. To run tests in parallel, save the models after disabling the runtime visualizations usingsave_system('LaneMarkerDetector'),save_system('VisionVehicleDetector')和save_system('HighwayLaneFollowingTestBench'). Test Manager uses the default Parallel Computing Toolbox cluster and executes tests only on the local machine. Running tests in parallel can speed up execution and decrease the amount of time it takes to get test results. For more information on how to configure tests in parallel from the Test Manager, seeRun Tests Using Parallel Execution.