Main Content

batch

RunMATLABscript or function on worker

collapse all in page

Syntax

j = batch(script)
j = batch(expression)
j = batch(myCluster,script)
j = batch(myCluster,expression)
j = batch(fcn,N,{x1,...,xn})
j = batch(myCluster,fcn,N,{x1,...,xn})
j = batch(___,Name,Value)

Description

example

j= batch(script)runs the script filescripton a worker in the cluster specified by the default cluster profile. (Note: Do not include the.mfile extension with the script name.) The function returnsj, a handle to the job object that runs the script. The script filescriptis copied to the worker.

By default, workspace variables are copied from the client to workers when you runbatch(script). Job and task objects are not copied to workers.

j= batch(expression)runsexpressionas an expression on a worker in the cluster specified by the default cluster profile. The function returnsj, a handle to the job object that runs the expression.

By default, workspace variables are copied from the client to workers when you runbatch(expression). Job and task objects are not copied to workers.

j= batch(myCluster,script)is identical tobatch(script)except that the script runs on a worker in the cluster specified by the cluster objectmyCluster.

j= batch(myCluster,expression)is identical tobatch(expression)except that the expression runs on a worker in the cluster specified by the cluster objectmyCluster.

example

j= batch(fcn,N,{x1,...,xn})runs the functionfcnon a worker in the cluster specified by the default cluster profile. The function returnsj, a handle to the job object that runs the function. The function is evaluated with the given arguments,x1,...,xn, and returnsNoutput arguments. The function file forfcnis copied to the worker. (Note: Do not include the.mfile extension with the function name argument.)

example

j= batch(myCluster,fcn,N,{x1,...,xn})is identical tobatch(fcn,N,{x1,...,xn})except that the function runs on a worker in the cluster specified by the cluster objectmyCluster.

example

j= batch(___,Name,Value)specifies options that modify the behavior of a job using one or more name-value pair arguments. These options support batch for functions and scripts, unless otherwise indicated. Use this syntax in addition to any of the input argument combinations in previous syntaxes.

Examples

collapse all

Open Live Script

This example shows how to usebatchto offload work to a MATLAB worker session that runs in the background.

You can continue using MATLAB while computations take place.

Run a script as a batch job by using thebatchfunction. By default,batchuses your default cluster profile. Check your default cluster profile on the MATLABHometab, in theEnvironmentsection, inParallel>Select Parallel Environment.. Alternatively, you can specify a cluster profile with the'Profile'name-value pair argument.

工作=批('myScript');

batchdoes not block MATLAB and you can continue working while computations take place.

If you want to block MATLAB until the job finishes, use thewaitfunction on the job object.

wait(job);

By default, MATLAB saves the Command Window output from the batch job to the diary of the job. To retrieve it, use thediaryfunction.

diary(job)
--- Start Diary --- n = 100 --- End Diary ---

After the job finishes, fetch the results by using theloadfunction.

load(job,'x'); plot(x)

If you want to load all the variables in the batch job, useload(job)instead.

When you have loaded all the required variables, delete the job object to clean up its data and avoid consuming resources unnecessarily.

delete(job); clearjob

Note that if you send a script file usingbatch, MATLAB transfers all the workspace variables to the cluster, even if your script does not use them. The data transfer time for a large workspace can be substantial. As a best practice, convert your script to a function file to avoid this communication overhead. For an example that uses a function, seeRun Batch Job and Access Files from Workers.

For more advanced options withbatch, seeRun Batch Job and Access Files from Workers.

Open Live Script

You can offload your computations to run in the background by usingbatch. If your code needs access to files, you can use additional options, such as'AttachedFiles'or“AdditionalPaths', to make the data accessible. You can close or continue working in MATLAB while computations take place and recover the results later.

Prepare Example

Use the supporting functionprepareSupportingFilesto copy the required data for this example to your current working folder.

prepareSupportingFiles;

Your current working folder now contains 4 files:A.dat,B1.dat,B2.dat, andB3.dat.

Run Batch Job

Create a cluster object usingparcluster. By default,parclusteruses your default cluster profile. Check your default cluster profile on the MATLABHometab, in theEnvironmentsection, inParallel>Select a Default Cluster.

c = parcluster();

Place your code inside a function and submit it as a batch job by usingbatch. For an example of a custom function, see the supporting functiondivideData. Specify the expected number of output arguments and a cell array with inputs to the function.

Note that if you send a script file using batch, MATLAB transfers all the workspace variables to the cluster, even if your script does not use them. If you have a large workspace, it impacts negatively the data transfer time. As a best practice, convert your script to a function file to avoid this communication overhead. You can do this by simply adding a function line at the beginning of your script. To reduce overhead in this example,divideDatais defined in a file outside of this live script.

If your code uses a parallel pool, use the'Pool'name-value pair argument to create a parallel pool with the number of workers that you specify.batchuses an additional worker to run the function itself.

By default,batchchanges the initial working folder of the workers to the current folder of the MATLAB client. It can be useful to control the initial working folder in the workers. For example, you might want to control it if your cluster uses a different filesystem, and therefore the paths are different, such as when you submit from a Windows client machine to a Linux cluster.

  • To keep the initial working folder of the workers and use their default, set'CurrentFolder'to'.'.

  • To change the initial working folder, set'CurrentFolder'to a folder of your choice.

This example uses a parallel pool with three workers and chooses a temporary location for the initial working folder. Usebatchto offload the computations individeData.

工作=批(c,@divideData,1,{},...'Pool',3,...'CurrentFolder',tempdir);

batchrunsdivideDataon a parallel worker, so you can continue working in MATLAB while computations take place.

If you want to block MATLAB until the job completes, use thewaitfunction on the job object.

wait(job);

To retrieve the results, usefetchOutputson the job object. AsdivideDatadepends on a file that the workers cannot find,fetchOutputsthrows an error. You can access error information by usinggetReporton theErrorproperty ofTaskobjects in the job. In this example, the code depends on a file that the workers cannot find.

getReport(job.Tasks(1).Error)
ans = 'Error using divideData (line 4) Unable to read file 'B2.dat'. No such file or directory.'

Access Files from Workers

By default,batchautomatically analyzes your code and transfers required files to the workers. In some cases, you must explicitly transfer those files -- for example, when you determine the name of a file at runtime.

In this example,divideDataaccesses the supporting fileA.dat, whichbatchautomatically detects and transfers. The function also accessesB1.dat, but it resolves the name of the file at runtime, so the automatic dependency analysis does not detect it.

typedivideData.m
function X = divideData() A = load("A.dat"); X = zeros(flip(size(A))); parfor i = 1:3 B = load("B" + i + ".dat"); X = X + A\B; end end

If the data is in a location that the workers can access, you can use the name-value pair argument“AdditionalPaths'to specify the location.“AdditionalPaths' adds this path to the MATLAB search path of the workers and makes the data visible to them.

pathToData = pwd;工作(2)=批(c @divideData 1, {},...'Pool',3,...'CurrentFolder',tempdir,...“AdditionalPaths',pathToData); wait(job(2));

If the data is in a location that the workers cannot access, you can transfer files to the workers by using the'AttachedFiles'name-value pair argument. You need to transfer files if the client and workers do not share the same file system, or if your cluster uses the generic scheduler interface in nonshared mode. For more information, seeConfigure Using the Generic Scheduler Interface(MATLAB Parallel Server).

filenames ="B"+ string(1:3) +".dat"; job(3) = batch(c,@divideData,1,{},...'Pool',3,...'CurrentFolder',tempdir,...'AttachedFiles',filenames);

Find Existing Job

You can close MATLAB after job submission and retrieve the results later. Before you close MATLAB, make a note of the job ID.

job3ID = job(3).ID
job3ID = 25

When you open MATLAB again, you can find the job by using thefindJobfunction.

job(3) = findJob(c,'ID',job3ID); wait(job(3));

Alternatively, you can use the Job Monitor to track your job. You can open it from the MATLABHometab, in theEnvironmentsection, inParallel>Monitor Jobs.

Retrieve Results and Clean Up Data

To retrieve the results of a batch job, use thefetchOutputsfunction.fetchOutputsreturns a cell array with the outputs of the function run withbatch.

X = fetchOutputs(job(3))
X =1×1 cell array{40×207 double}

When you have retrieved all the required outputs and do not need the job object anymore, delete it to clean up its data and avoid consuming resources unnecessarily.

delete(job) clearjob

Input Arguments

collapse all

MATLAB script, specified as a character vector or string scalar.

By default, workspace variables are copied from the client to workers when you specify this argument. Job and task objects are not copied to workers.

Example:batch('aScript');

Data Types:char|string

Expression to evaluate, specified as a character vector or string scalar.

By default, workspace variables are copied from the client to workers when you specify this argument. Job and task objects are not copied to workers.

Example:batch('y = magic(3)');

Data Types:char|string

Cluster, specified as aparallel.Cluster对象,该对象代表了集群计算资源。To create the object, use theparclusterfunction.

Example:cluster = parcluster; batch(cluster,'aScript');

Data Types:parallel.Cluster

Function to be evaluated by the worker, specified as a function handle or function name.

Example:batch(@myFunction,1,{x,y});

Data Types:char|string|function_handle

Number of outputs expected from the evaluated functionfcn, specified as a nonnegative integer.

Example:batch(@myFunction,1,{x,y});

Data Types:single|double|int8|int16|int32|int64|uint8|uint16|uint32|uint64

Input arguments to the functionfcn, specified as a cell array.

Example:batch(@myFunction,1,{x,y});

Data Types:cell

Name-Value Arguments

Specify optional pairs of arguments asName1=Value1,...,NameN=ValueN, whereNameis the argument name andValueis the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and encloseNamein quotes.

Example:j = batch(@myFunction,1,{x,y},'Pool',3);

Variables to copy to workers, specified as the comma-separated pair consisting of'Workspace'and a structure scalar.

默认值是一个结构标量场s corresponding to variables in the client workspace. Specify variables as fields in the structure scalar.

Workspace variables are only copied from the client to workers if you specifyscriptorexpression. Job and task objects are not copied to workers.

Example:workspace.myVar = 5; j = batch('aScript','Workspace',workspace);

Data Types:struct

Cluster profile used to identify the cluster, specified as the comma-separated pair consisting of'Profile'and a character vector or string. If this option is omitted, the default profile is used to identify the cluster and is applied to the job and task properties.

Example:j = batch('aScript','Profile','Processes');

Data Types:char|string

Paths to add to the MATLAB search path of the workers before the script or function executes, specified as the comma-separated pair consisting of“AdditionalPaths'and a character vector, string array, or cell array of character vectors.

The default search path might not be the same on the workers as it is on the client; the path difference could be the result of different current working folders (cwd), platforms, or network file system access. Specifying the“AdditionalPaths'name-value pair argument helps ensure that workers look for files, such as code files, data files, or model files, in the correct locations.

You can use“AdditionalPaths'to access files in a shared file system. Note that path representations can vary depending on the target machines.“AdditionalPaths'must be the paths as seen by the machines in the cluster. For example, ifZ:\dataon your local Windows®machine is/network/datato your Linux®cluster, then add the latter to“AdditionalPaths'. If you use a datastore, use'AlternateFileSystemRoots'instead to deal with other representations. For more information, seeSet Up Datastore for Processing on Different Machines or Clusters.

Note thatAdditionalPathsonly helps to find files when you refer to them using a relative path or file name, and not an absolute path.

Example:j = batch(@myFunction,1,{x,y},'AdditionalPaths','/network/data/');

Data Types:char|string|cell

Files or folders to transfer to the workers, specified as the comma-separated pair consisting of'AttachedFiles'and a character vector, string array, or cell array of character vectors.

Example:j = batch(@myFunction,1,{x,y},'AttachedFiles','myData.dat');

Data Types:char|string|cell

Flag to add user-added entries on the client path to worker paths, specified as the comma-separated pair consisting of'AutoAddClientPath'and a logical value.

Example:j = batch(@myFunction,1,{x,y},'AutoAddClientPath',false);

Data Types:logical

Flag to enable dependency analysis and automatically attach code files to the job, specified as the comma-separated pair consisting of'AutoAttachFiles'and a logical value. If you set the value totrue, the batch script or function is analyzed and the code files that it depends on are automatically transferred to the workers.

Example:j = batch(@myFunction,1,{x,y},'AutoAttachFiles',true);

Data Types:logical

Folder in which the script or function executes, specified as the comma-separated pair consisting of'CurrentFolder'and a character vector or string. There is no guarantee that this folder exists on the worker. The default value for this property is the current directory of MATLAB when thebatchcommand is executed. If the argument is'.', there is no change in folder before batch execution.

Example:j = batch(@myFunction,1,{x,y},'CurrentFolder','.');

Data Types:char|string

Flag to collect the diary from the function call, specified as the comma-separated pair consisting of'CaptureDiary'and a logical value. For information on the collected data, seediary.

Example:j = batch('aScript','CaptureDiary',false);

Data Types:logical

Environment variables to copy from the client session to the workers, specified as the comma-separated pair consisting of'EnvironmentVariables'and a character vector, string array, or cell array of character vectors. The names specified here are appended to theEnvironmentVariablesproperty specified in the applicable parallel profile to form the complete list of environment variables. Listed variables that are not set are not copied to the workers. These environment variables are set on the workers for the duration of the batch job.

Example:j = batch('aScript','EnvironmentVariables',"MY_ENV_VAR");

Data Types:char|string|cell

Number of workers to make into a parallel pool, specified as the comma-separated pair consisting of'Pool'and either:

  • A nonnegative integer.

  • A 2-element vector of nonnegative integers, which is interpreted as a range. The size of the resulting parallel pool is as large as possible in the range requested.

In addition, note thatbatchuses another worker to run the batch job itself.

The script or function uses this pool to execution statements such asparforandspmdthat are inside the batch code. Because the pool requiresNworkers in addition to the worker running the batch, the cluster must have at leastN+1workers available. You do not need a parallel pool already running to executebatch,新池批量创建is not related to a pool you might already have open. For more information, seeRun a Batch Job with a Parallel Pool.

If you use the default value,0, the script or function runs on only a single worker and not on a parallel pool.

Example:j = batch(@myFunction,1,{x,y},'Pool',4);

Example:j = batch(@myFunction,1,{x,y},'Pool',[2,6]);

Data Types:single|double|int8|int16|int32|int64|uint8|uint16|uint32|uint64

Output Arguments

collapse all

Job, returned as aparallel.Jobobject.

Example:j = batch('aScript');

Data Types:parallel.Job

Tips

To view the status or track the progress of a batch job, use the Job Monitor, as described inJob Monitor. You can also use the Job Monitor to retrieve a job object for a batch job that was created in a different session, or for a batch job that was created without returning a job object from thebatchcall.

Delete any batch jobs you no longer need to avoid consuming cluster storage resources unnecessarily.

Version History

Introduced in R2008a

expand all

See Also

|||||

Topics