Select Datastore for File Format or Application
A datastore is a repository for collections of data that are too large to fit in memory. Each file format and application uses a different type of datastore, which contains properties pertinent to the type of data or application that it supports. MATLAB®provides datastores for standard file formats such as Excel®files and datastores for specific applications such as Deep Learning. In addition to the existing datastores, if your data is in a proprietary format, then you can develop a customized datastore using the custom datastore framework.
Datastores for Standard File Formats
For a collection of data in standard file format use one of these options.
Datastore | Description |
---|---|
TabularTextDatastore |
Text files containing column-oriented data, including CSV files |
SpreadsheetDatastore |
Spreadsheet files with a supported Excel format such as |
ImageDatastore |
Image files, including formats that are supported by |
ParquetDatastore |
Parquet files containing column-oriented data |
FileDatastore |
Files with nonstandard file format Requires a custom file reading function |
Transform or combine existing datastores.
Datastore | Description |
---|---|
CombinedDatastore |
Datastore to combine data read from multiple underlying datastores |
TransformedDatastore |
Datastore to transform underlying datastore |
Datastores to integrate with MapReduce and tall arrays.
Datastore | Description |
---|---|
KeyValueDatastore |
Key-value pair data that are inputs to or outputs of |
TallDatastore |
Datastore for checkpointing |
Datastores for Specific Applications
Based on your application use one of these datastores.
Application | Datastore | Description | Toolbox Required |
---|---|---|---|
Simulink Model Data |
SimulationDatastore (Simulink) |
Datastore for simulation input and output data that you use with a Simulink®model |
金宝app |
Simulation Ensemble and Predictive Maintenance Data |
simulationEnsembleDatastore (Predictive Maintenance Toolbox) |
Datastore to manage simulation ensemble data |
Predictive Maintenance Toolbox™ |
fileEnsembleDatastore (Predictive Maintenance Toolbox) |
Datastore to manage ensemble data in custom file format |
Predictive Maintenance Toolbox |
|
Measurement Data Format (MDF) Files |
mdfDatastore (Vehicle Network Toolbox) |
Datastore for collection of MDF files |
Vehicle Network Toolbox™ |
mdfDatastore (Powertrain Blockset) |
Datastore for collection of MDF files |
Powertrain Blockset™ |
|
Deep Learning Datastores for preprocessing image or sequence data |
pixelLabelDatastore (Computer Vision Toolbox) |
Datastore for pixel label data |
Computer Vision Toolbox™ and Deep Learning Toolbox™ |
pixelLabelImageDatastore (Computer Vision Toolbox) |
Datastore for training semantic segmentation networks Datastore isnondeterministic |
Computer Vision Toolbox and Deep Learning Toolbox |
|
boxLabelDatastore (Computer Vision Toolbox) |
Datastore for bounding box label data |
Computer Vision Toolbox and Deep Learning Toolbox |
|
signalDatastore (Signal Processing Toolbox) |
Datastore for collection of signal files | Signal Processing Toolbox™ and Deep Learning Toolbox |
|
randomPatchExtractionDatastore (图像处理工具箱) |
Datastore for extracting random patches from images or pixel label images Datastore isnondeterministic |
Image Processing Toolbox™ and Deep Learning Toolbox |
|
denoisingImageDatastore (图像处理工具箱) |
Datastore to train an image denoising deep neural network Datastore isnondeterministic |
Image Processing Toolbox and Deep Learning Toolbox |
|
augmentedImageDatastore (Deep Learning Toolbox) |
Datastore for resizing and augmenting training images Datastore isnondeterministic |
Deep Learning Toolbox |
|
Audio Data | audioDatastore (Audio Toolbox) |
Datastore for collection of audio files |
Audio Toolbox™ |
Out-of-Memory Image Data | blockedImageDatastore (图像处理工具箱) |
Datastore to manage blocks of a single image that is too large to fit in memory | Image Processing Toolbox |
Database Data | databaseDatastore (Database Toolbox) |
Datastore for collections of data in a relational database |
Database Toolbox™ |
Custom File Formats
For a collection of data in a custom file format, if each individual file fits in memory, useFileDatastore
along with your custom file reading function. Otherwise, develop your own fully customized datastore for custom or proprietary data using thematlab.io.Datastore
class. SeeDevelop Custom Datastore.
Nondeterministic Datastores
Datastores that do not return the exact same data for a call to theread
function after a call to thereset
function are nondeterministic datastores. Do not use nondeterministic datastores withtall
arrays,mapreduce
, or any other code that requires reading the data more than once.
Some applications require data that is randomly augmented or transformed. For example, theaugmentedImageDatastore
(Deep Learning Toolbox)datastore, from the deep learning application augments training image data with randomized preprocessing operations to help prevent the network from overfitting and memorizing the exact details of the training images. The output of this datastore is different every time you perform aread
operation after a call toreset
.
See Also
TabularTextDatastore
|SpreadsheetDatastore
|ImageDatastore
|FileDatastore
|TallDatastore
|tall