The addition of the large array handling API may require you to update your MEX-files. The reason that code changes are required is that in order to enable large sized data, the types of inputs and outputs to the MATLAB API had to be changed. We have implemented the changes to our API in a way that reduces the impact on existing users. We adopted an "opt-in" strategy, where if you want to take advantage of the new large array handling feature, you can opt-in using the MEX flag "-largeArrayDims". If you chose to opt-in you will need to make the updates described in this solution. If you chose to not "opt-in", you do not need to do anything now. Beware, though, in the future the new large array handling API will become the default, and you will then need to update your code or take other action.
This solution walks you through a step-by-step process for identifying the changes you need to make. The procedure suggests that you first identify and list variables that are candidates for changing before modifying your code. We recommend that you don't edit your code until you have completed the identification steps.
首先，考虑高级技术解释，了解需要进行的变化。随着MEX API使用的类型的变化，您的现有代码会将输入和输出传递给MATLAB，错误的类型是有可能的。具体地，改变的参数类型是指数和大小。两者都从32位类型变为64位兼容类型。
The following procedures will help you identify the type mismatches that need to be corrected. It follows a methodical process which includes first testing the original code, then identifying changes, then updating a copy of the code, building the new code, and finally retesting. There are suggestions along the way for how to deal with common code patterns, build failures and warnings, and runtime issues.
Note: This document uses primarily C/C++ terminology and example code. Fortran MEX-files share the same issues; you can find additional Fortran specifics in Section 8.8.
In more detail:
1. Test existing code
适应你的代码来处理大数组之前,you should first verify that it works with the traditional 32-bit array dimensions. You may want to build a list of expected inputs and outputs, or even a full test suite. Back up your source code and binaries for safekeeping and so that you can compare the results with your updated source code.
2. Identify variables that contain 64-bit index or size values
为了处理非常大的数组，您需要转换包含数组指标或大小的所有变量，以使用MWSIZE / MWindex数据类型而不是32位“int”。在64位体系结构上使用大型数组尺寸时，这些数据类型是64位整数，并实现为预处理器宏。使用-LargeArrayDims时，这两种类型都与C / C ++中的Size_t类型相同。
We suggest looking at three classes of variables: those used directly by the API functions, intermediate variables, and variables that are used as both sizes / indices and 32-bit integers.
2.1。搜索64位API函数直接使用的变量
要识别这些变量，请查找将MWSIZE / MWINDEX值的MX *和MEX *函数作为输入或输出值。
有关准确列表，请检查MATLAB版本中的文档。您可以在此找到此列表：
You can find these functions by using your editor's Find function or a utility such as GREP.
For example, in older versions of MATLAB (R2006a and earlier), this signature was:
mxArray*mxCreateDoubleMatrix(int m, int n, mxComplexity ComplexFlag);
MXCreatedouBlematrix在大型阵列维度API中的签名是：
mxArray*mxCreateDoubleMatrix(mwSize m, mwSize n, mxComplexity ComplexFlag);
注意前两个输入参数已从“int”更改为“mwsize”。作为第一个或第二个输入传递的变量必须声明为mwsize。
您可以通过帮助浏览器或DOC命令可以找到每个函数参考页面上的新签名：
docmxCreateDoubleMatrix
Because mwSize is the same as "int" when using 32-bit array dimensions, your code may still be written to this older signature. In order to take advantage of large array handling, you must update your code to use mwSize/mwIndex to allow for 64-bit size and index values.
通过您的代码进行搜索，并注意任何定义为“int”或类似类型的变量。请勿在此时进行任何编辑，只需注意哪些变量需要更新。
如果您的代码是：
intm，n;
...
YP_OUT = mxCreateDoubleMatrix(m, n, mxREAL);
both m and n should be declared mwSize:
mwSizem，n;
...
YP_OUT = mxCreateDoubleMatrix(m, n, mxREAL);
2.2。找到中间变量
It is possible that your code uses intermediate variables to calculate sizes and indices. If that is the case, you will need to ensure that those variables are also declared as the appropriate type. Consider this example:
mwSizem，n;
intnumDataPoints;
m = 3;
numDataPoints = m * 2;
n = numDataPoints + 1;
...
YP_OUT = mxCreateDoubleMatrix(m, n, mxREAL);
The first search in Section 2.1 will miss intermediate variables that are not directly used in mx* function calls, such as numDataPoints in this example. These variables will not be large enough to handle 64-bit indices coming from mwSize variables. Storing 64-bit indices in these 32-bit intermediate variables will truncate the indices, leading to incorrect results or crashes. In this example, numDataPoints should be changed to be mwSize, since "m" is mwSize:
mwSizem，n;
mwSizenumDataPoints;
检查您的代码，并将所有此类中间体添加到要转换的变量列表中。
同样，在这一点上不要做任何编辑。
2.3. Replace variables that serve multiple purposes
It is possible that your code uses the same variable for both indices (which we are converting to 64-bit) and STRUCT field numbers or status codes (both of which stay 32-bit).
需要识别用于多种目的的变量，并用两个变量替换---一个mwsize / mwindex和一个32位整数。对于两个阵列大小和结构字段的数量使用一个变量，这种情况尤其常见。另一个案例是状态代码/成功标志也用作索引。
For example, mxCreateDoubleMatrix expects mwSize inputs, but mxCreateStructMatrix requires an int:
mxArray*mxCreateDoubleMatrix(mwSize m, mwSize n, mxComplexity ComplexFlag);
mxArray*mxCreateStructMatrix(mwSize m, mwSize n, int nfields, const char **fieldnames);
Now consider the numDataPoints variable:
mxArray*myNumeric, *myStruct;
intnumSensors;
mwSizem, n;
char**fieldnames;
intNumfields.;
...
myNumeric = mxCreateDoubleMatrix(numSensors, n, mxREAL);
mystruct = mxcreatestructmatrix（m，n，numsensors，fieldnames）;
在此示例中，您需要两个新的变量来替换NumSensors以正确处理两个功能：
mxArray*myNumeric, *myStruct;
mwSizenumSensorSize;
intnumSensorFields;
mwSizem, n;
char**fieldnames;
...
myNumeric = mxCreateDoubleMatrix(numSensorSize, n, mxREAL);
myStruct = mxCreateStructMatrix(m, n, numSensorFields, fieldnames);
注意需要以这种方式替换哪些变量。同样，在这一点上不要做任何编辑。
3. 3rd party libraries
哟你MEX-file可能涉及的部分代码u did not write, and to which you do not have access. These may be entire MEX-files, numerical routines, or device drivers. There are several ways to approach this issue:
3.1. If you do not have access to the source code, contact the vendor. Refer them to this document and discuss options to handle large array dimensions with them.
3.2. If you share code with others, such as publicly available source code, check with the relevant author or user group to see if someone has converted the code to use large array dimensions. If not, you may want to convert the code yourself. Again, apply the suggestions in this document.
4.克雷亚te a working copy, make edits, and test with 32-bit dimensions.
此时，您知道需要编辑哪种类型的声明。制作源代码的副本并更改相关声明。像往常一样编译代码：
mexmyMexFile.c
这使用传统的32位维度。这与使用-compatiblearraydims标志相同：
mex-compatibleArrayDims myMexFile.c
与原始二进制文件进行比较;两者都应该返回相同的结果。
如果没有，调试和解决任何差异。现在，这些将更容易地解决（使用相同的32位大小和指数）而不是下一步。
5.使用-largeArraydims构建并解决构建故障和警告。
您现在准备使用大型数组处理API编译MEX文件。只需将-largeArrayDims标志添加到您的编译;例如，而不是
mexmyMexFile.c
use
mex-largeArrayDims myMexFile.c
使用-LargeArrayDim时，您的编译器可以将MWSIZE / MWINDEX称为“size_t”，“无符号
_
_int64", or other similar names.
此时大多数构建问题与32位和64位类型之间的类型不匹配有关。您可能遇到的一些常见问题是：
5.1. Type mismatch in assignment
In some instances --- especially simple assignments --- your C/C++ compiler can warn about type mismatches. For example:
mwSizem，n;
intnumDataPoints;
...
numDataPoints = m * 2;
When you compile this file using "mex -largeArrayDims buggyMexFile.c", you may receive a type-mismatch warning.
From Microsoft Visual Studio on 64-bit Windows: ERROR: C:\Work\buggyMexFile.c(31) : warning C4267: '=' : conversion from 'size_t' to 'int', possible loss of data
This example can be fixed using Section 2.2 to convert the intermediate variable numDataPoints to mwSize.
5.2. Type mismatch in pointer type
When using mxCreate*Array functions that take arrays of dimensions, you will need to convert the pointer from int* to mwSize*:
mwSizendim;
int*myDims;
...
pa1 = mxCreateLogicalArray(ndim, myDims);
From gcc on 64-bit Linux: ERROR: buggyMexFile.c: In function 'mexFunction': buggyMexFile.c:35: warning: passing argument 2 of 'mxCreateLogicalArray' from incompatible pointer type
In this case, myDims must be mwSize*, as described in Section 2.1.
6. Execute MEX-file, test, and debug
Compare the results of running your MEX-file compiled with -largeArrayDims with results from your original binary. If there are any differences or failures, use a debugger to investigate the cause.
Debuggers are key tool for investigating programming issues in your code. For more instructions on debugging MEX files, please use the links provided below:
For more information on the capabilities of your debugger, refer to your compiler documentation.
After resolving any issues with the -largeArrayDims version, your converted MEX-file now replicates the functionality of your original code while using the large array handling API.
以下是运行MEX文件时可能遇到的常见问题列表。
6.1。出现内存错误
One potential error mode you might run into is the "Out of Memory" error. This is one of the most common failure modes when converting code to use large array dimensions. It is most often caused by the use of mismatching types with functions that allocate memory, such as mxCreateDoubleMatrix. The type mismatch corrupts the requested array size, causing MATLAB to try to allocate much more memory than you expect. This value can far exceed the available memory (RAM) of the machine you are using.
The function expects 8 bytes, so it grabs the 4 bytes adjacent to the ones passed in by the 4 byte variable. The extra bytes have random information in them and as a consequence, the function interprets the input as a very different value than what was passed in.
For example, mxCreateDoubleMatrix expects the first argument to be mwSize, which is 64 bits when using -largeArrayDims on a 64-bit machine. Suppose the variable passed as the first argument is an int, which is 32 bits:
intm，n;
m = 10;
...
YP_OUT = mxCreateDoubleMatrix(m, n, mxREAL);
The value that mxCreateDoubleMatrix receives may contain 32 bits of random information. For example, the value could be interpreted as 85899345930 (10 + 20*2^32). This far exceeds the memory capacity of most machines, and consequently MATLAB reports an Out of Memory error:
??? Error using ==> buggyMexFile
Out of memory. Type HELP MEMORY for your options.
If you encounter this issue, change the type of the variable to mwSize or mwIndex as suggested in section 2.1:
mwSizem，n;
m = 10;
...
YP_OUT = mxCreateDoubleMatrix(m, n, mxREAL);
This case is difficult to debug because "m" and "n" both contain the correct value, but are the wrong storage class. Your debugger can help identify this. For example, the Microsoft Visual Studio debugger displays a "Type" column that will show "int" for the variables above, and "size_t", "unsigned
_
_int64", or similar names for the correct type.
6.2. Segmentation Violation, Assertion, or other crash
Use your debugger to step through both your original and modified code, looking for the first difference between the two, especially in any variables you modified. This root cause can happen well before the actual crash.
Again, your debugger can identify the data types for each of your values.
6.3. Different results
Use your debugger to step through both your original and modified code, looking for the first difference between the two, especially in any variables you modified. This root cause can happen well before returning results to MATLAB.
7. Call your MEX-file with gigantic arrays
If you have access to a machine with large amounts of memory, you can now experiment with arrays with more than 2^32-1 elements. Working with these arrays takes large amounts of memory --- an array of double-precision floating point numbers (the default in MATLAB) with 2^32 elements takes ~32 GB of memory.
You can find more information on 64-bit mxArrays in MEX-files, and an example arraySize.c MEX-file that demonstrates the use of large arrays, here:
8. Frequently Asked Questions (answered in the attached largeArrayDimsFaq.txt):
8.1。如果我只使用32位MATLAB ?
8.2. What if I already made the recommended changes to support sparse arrays on 64-bit systems?
8.3。如果我已经提出了建议的更改以支持64位MXArrays，该怎么办？金宝app
8.4. What if I already made the recommended changes to my Fortran source files?
8.5. What if I want to opt out (don't want to upgrade)?
8.6。如果我什么都不做了什么？
8.7。如果我没有源mex文件怎么办？
8.8. What if I'm using Fortran?
8.9. What if I find deprecated functions such as mexPutFull in my code?
