主要内容

从桌面到群集扩展

此示例显示了如何在本地机器上开发并行MATLAB®代码,并扩展到群集。集群提供更多的计算资源来加快和分发您的计算。您可以在本地计算机上并行交互方式运行代码,然后在群集上,而无需更改代码。完成本地计算机上的代码原型后,您可以使用批处理作业将计算卸载到集群中。因此,您可以关闭MATLAB并稍后检索结果。

Develop Your Algorithm

首先在本地计算机上原型制作算法。该示例使用整数分解作为样本问题。这是一个计算密集的问题,其中分解的复杂性随数量的大小而增加。您使用简单的算法来分解一系列整数编号。

以64位的精度创建质数的矢量,然后随机乘以成对的素数以获得大型的复合数。创建一个数组来存储每个分解的结果。以下每个示例中的每个部分中的代码可能需要超过20分钟。为了使其更快,请使用较少的质数减少工作量,例如2^19。运行2^21查看最佳最终图。

primenumbers = primes(uint64(2^21));compositenumbers = Primenumbers。因子=零(Numel(Primenumbers),2);

使用循环来考虑每个复合数,并测量计算所需的时间。

tic;foridx = 1:numel(compositeNumbers)因子(idx,:) = factor(compositenumbers(idx));endtoc
经过的时间为684.464556秒。

在本地平行池上运行代码

并行计算工具箱™使您可以通过在并行池中的多个工人上运行来扩展工作流程。上一个迭代forloop are independent, and so you can use aparfor循环以将迭代分配给多个工人。只需改变您的forloop into aparfor环形。然后,运行代码并测量整体计算时间。该代码在平行池中运行,没有进一步的更改,工人将您的计算发送回本地工作区。由于工作负载分布在几个工人之间,因此计算时间较低。

tic;parforidx = 1:numel(compositeNumbers)因子(idx,:) = factor(compositenumbers(idx));endtoc
经过的时间为144.550358秒。

When you useparfor和you have Parallel Computing Toolbox, MATLAB automatically starts a parallel pool of workers. The parallel pool takes some time to start. This example shows a second run with the pool already started.

默认集群配置文件是'local'。You can check that this profile is set as default on the MATLABHome选项卡,in平行>选择默认集群。启用了此配置文件,MATLAB将在计算机上为并行池创建工人。当您使用'local'默认情况下,个人资料MATLAB启动了与机器中的物理内核一样多的工人,直到您首选的工人数量。您可以使用并行首选项控制并行行为。在matlab上Hometab, select平行>并行偏好

To measure the speedup with the number of workers, run the same code several times, limiting the maximum number of workers. First, define the number of workers for each run, up to the number of workers in the pool, and create an array to store the result of each test.

numworkers = [1 2 4 6];tlocal =零(size(numworkers));

使用循环通过最大数量的工人进行迭代,并运行先前的代码。要限制工人的数量,请使用第二个输入参数parfor

forw = 1:numel(numworkers)tic;parfor(idx = 1:numel(compositeNumbers),numworkers(w))因子(idx,:) = factor(compositeNumbers(idx));endtlocal(w)= toc;end

Calculate the speedup by computing the ratio between the computation time of a single worker and the computation time of each maximum number of workers. To visualize how the computations scale up with the number of workers, plot the speedup against the number of workers. Observe that the speedup increases with the number of workers. However, the scaling is not perfect due to overhead associated with parallelization.

f =图;加速= tlocal(1)./ tlocal;情节(Numworkers,Speedup);标题('Speedup with the number of workers');Xlabel(“工人人数”);Xticks(Numworkers);ylabel('加速');

完成计算后,请删除当前的并行池,以便为群集创建一个新的。您可以使用gcpfunction.

delete(gcp);

设置群集

If your computing task is too big or too slow for your local computer, you can offload your calculation to a cluster onsite or in the cloud. Before you can run the next sections, you must get access to a cluster. On the MATLABHome标签,转到平行>发现集群找出您是否已经可以使用MATLAB Parallel Server™访问群集。有关更多信息,请参阅发现集群

如果您无法访问群集,则必须在运行下一个部分之前配置对一个群集的访问。在MATLAB中,您可以直接从MATLAB桌面中创建云服务中的簇。在Home标签,在平行menu, selectCreate and Manage Clusters。在集群配置文件管理器中,单击创建云集群。要了解有关扩展到云的更多信息,请参阅Getting Started with Cloud Center。要了解有关您将缩放到网络集群扩展的选择的更多信息,请参见Get Started with MATLAB Parallel Server(MATLAB并行服务器)

设置集群配置文件后,您可以在平行>Create and Manage Clusters。有关更多信息,请参阅发现集群并使用群集配置文件。下图显示了一个集群配置文件集群配置文件管理器

在集群并行池上运行代码

如果要默认在集群中运行并行功能,请将群集配置文件设置为默认平行>选择默认集群

You can also use a programmatic approach to specify your cluster. To do so, start a parallel pool in the cluster by specifying the name of your cluster profile in theParpool命令。在以下代码中,替换mycluster带有群集配置文件的名称。还可以用第二个输入参数指定工人数量。

Parpool(“ mycluster',64);
使用“ mycluster”配置文件...连接到64名工人。

和以前一样,通过几次运行相同的代码来测量工人数量的加速,并限制最大工人数量。因为此示例中的群集允许比本地设置更多的工人,所以数字工人can hold more values. If you run this code, theparfor循环现在在集群中运行。

numworkers = [1 2 4 6 16 32 64];tcluster = zeros(size(numworkers));forw = 1:numel(numworkers)tic;parfor(idx = 1:numel(compositeNumbers),numworkers(w))因子(idx,:) = factor(compositeNumbers(idx));endtcluster(w)= toc;end

Calculate the speedup, and plot it against the number of workers to visualize how the computations scale up with the number of workers. Compare the results with those of the local setup. Observe that the speedup increases with the number of workers. However, the scaling is not perfect due to overhead associated with parallelization.

图(f);抓住速度= tcluster(1)./ tcluster;情节(Numworkers,Speedup);标题('Speedup with the number of workers');Xlabel(“工人人数”);xticks(numWorkers(2:end)); ylabel('加速');

完成计算后,删除当前的并行池。

delete(gcp);

卸载和扩展您的计算batch

After you are done prototyping and running interactively, you can use batch jobs to offload the execution of long-running computations in the background with batch processing. The computation happens in the cluster, and you can close MATLAB and retrieve the results later.

使用batch功能以将批处理作业提交给您的群集。您可以将算法的内容放在脚本中,并使用batch提交它的功能。例如,脚本myParallelAlgorithm根据本示例中显示的整数分解问题执行简单的基准测试。该脚本测量具有不同工人数量的几个问题复杂性的计算时间。

请注意,如果您使用脚本文件batchMATLAB转移所有工作空间变量the cluster, even if your script does not use them. If you have a large workspace, it impacts negatively the data transfer time. As a best practice, convert your script to a function file to avoid this communication overhead. You can do this by simply adding a function line at the beginning of your script. To learn how to convertmyParallelAlgorithmto a function, seemyParallelAlgorithmFcn

The following code submitsmyParallelAlgorithmFcnas a batch job.myParallelAlgorithmFcnreturns two output arguments,数字工人time,,,,和you must specify2as the number of outputs input argument. Because the code needs a parallel pool for theparfor循环,使用'水池'名称值对batch指定工人人数。群集使用额外的工人运行该功能本身。默认,batch将集群中工人的当前文件夹更改为MATLAB客户端的当前文件夹。控制当前文件夹可能很有用。例如,如果您的群集使用不同的文件系统,因此路径是不同的,例如,当您从Windows客户端计算机提交到Linux群集时。设置名称值对'CurrentFolder'到您选择的文件夹,或'。'to avoid changing the folder of the workers.

TotalNumberOfWorkers = 65;群集= Parcluster(“ mycluster');job = batch(cluster,'myParallelAlgorithmFcn',,,,2,,,,'水池',,,,totalNumberOfWorkers-1,'CurrentFolder',,,,'。');

要在提交后监视工作状态,请打开工作监视器平行>监视工作。当计算从集群开始时,作业状态变为跑步

You can close MATLAB after the job has been submitted. When you open MATLAB again, the Job Monitor keeps track of the job for you, and you can interact with it if you right-click it. For example, to retrieve the job object, select显示详细资料,,,,和to transfer the outputs of the batch job into the workspace, select提取输出

Alternatively, if you want to block MATLAB until the job completes, use thewaitfunction on the job object.

等待(工作);

要从群集传输函数的输出,请使用fetchOutputsfunction.

outputs = fetchOutputs(job); numWorkers = outputs{1}; time = outputs{2};

检索结果后,您可以在本地计算机上使用它们进行计算。计算加速度,并将其绘制在工人数量上。由于代码对不同的问题复杂性运行因素化,因此您可以获得每个级别的绘图。您可以看到,对于每个问题的复杂性,加速随着工人的数量而增加,直到其他工人的开销大于并行化的绩效增长。随着您提高问题的复杂性,您可以在大量工人中获得更好的速度,因为与并行化相关的开销不那么重要。

figure speedup = time(1,:)./time; plot(numWorkers,speedup); legend(问题复杂性1',,,,“问题复杂性2”,,,,问题复杂性3',,,,问题复杂性4',,,,'地点',,,,'northwest');标题(“加速与复杂性”);Xlabel(“工人人数”);xticks(numWorkers(2:end)); ylabel('加速');

也可以看看

|||

相关示例

更多关于