Constructing a memmapfile Object :: Data Import and Export (Programming)

Programming

Constructing a memmapfile Object

The first step in mapping to any file is to construct an instance of the memmapfile class using the class constructor function. You can have MATLAB assign default values to each of the new object's properties, or you can specify property values yourself in the call to the memmapfile constructor.

For information on how to set these values, see the sections that cover

Constructing the Object with Default Property Values

The simplest and most general way to call the constructor is with one input argument that specifies the name of the file you want to map. All other properties are optional and are given their default values. Use the syntax shown here:

```
objname = memmapfile(filename)
```

To construct a map for the file records.dat that resides in your current working directory, type the following:

```
m = memmapfile('records.dat');
```

MATLAB constructs an instance of the memmapfile class, assigns it to the variable m, and maps the entire records.dat file to memory, setting all properties of the object to their default values. In this example, the command maps the entire file as a sequence of unsigned 8-bit integers, and gives the caller read-only access to its contents.

You can set any property to a new value at any time after the object has been constructed. Use the objname.property syntax to assign the new value. For example, to set a new offset value for memory map object m, type

```
m.offset = 2048;
```

Changing Property Values

You can make the memory map more specific to your needs by including more information when calling the constructor. In addition to the filename argument, there are four other parameters that you can pass to the constructor. Each of these parameters represents a property of the object, and each requires an accompanying value to be passed as well:

objname = memmapfile(filename, prop1, value1, prop2, value2, ...)

For example, to construct a map using nondefault values for the offset, format, and writable properties, type the following, enclosing all property names in quotes:

m = memmapfile('records.dat',         ...
               'offset', 1024,        ...
               'format', 'uint32',    ...
               'writable', true);

Type the object name to see the current settings for all properties:

m

m =
    Filename: 'd:\matlab\mfiles\records.dat'
    Writable: true
      Offset: 1024
      Format: 'uint32'
      Repeat: Inf
        Data: 4778x1 uint32 array

You can also change the value any property after the object has been constructed. Use the syntax

```
objname.property = newvalue;
```

For example, to change the format value for object m from uint32 to uint16, type the following. (Property names are not case sensitive.)

```
m.format = 'uint16';
```

Further read and write operations to the region mapped by m will now treat the data in the file as a sequence of unsigned 16-bit integers. Whenever you change the value of a memmapfile property, MATLAB remaps the file to memory.

Selecting the File to Map

filename is the only required argument when you call the memmapfile constructor. When you call the memmapfile constructor, MATLAB assigns the filename that you specify to the filename property of the new object instance.

Specify the filename as a quoted string, (e.g., 'records.dat'). It must be first in the argument list and not specified as a parameter-value pair. filename must include a filename extension if the name of the file being mapped has an extension. The filename argument cannot include any wildcard characters (e.g., * or ?), and is not case sensitive.

Note Unlike the other memmapfile constructor arguments, you must specify filename as a single string, and not as a parameter-value pair.

If the file to be mapped resides somewhere on the MATLAB path, you can use a partial pathname. If the path to the file is not fully specified, MATLAB searches for the file in your current working directory first, and then on the MATLAB path.

Once memmapfile locates the file, MATLAB stores the absolute pathname for the file internally, and then uses this stored path to locate the file from that point on. This enables you to work in other directories outside your current work directory and retain access to the mapped file.

You can change the value of the filename property at any time after constructing the memmapfile object. You might want to do this if

You want to use the same memmapfile object on more than one file.
You save your memmapfile object to a MAT-file, and then later load it back into MATLAB in an environment where the mapped file has been moved to a different location. This requires that you modify the path segment of the filename string to represent the new location.

For example, save memmapfile object m to file mymap.mat:

disp(m.filename)
    d:\matlab\mfiles\records.dat

save mymat m

Now move the file to another location, load the object back into MATLAB, and update the path in the filename property:

load mymat m
m.filename = 'f:\testfiles\oct1\records.dat'

Note You can only map an existing file. You cannot create a new file and map that file to memory in one operation. Use the MATLAB file I/O functions to create the file before attempting to map it to memory.

Setting the Start of the Mapped Region

By default, MATLAB begins a memory map at the start of the file. To begin the mapped region at some point beyond the start of the file, specify an offset parameter in the call to the memmapfile constructor:

objname = memmapfile(filename, 'offset', bytecount)

The bytecount value is the number of bytes from the beginning of the file to the point in the file where you want the memory map to start (a zero-based offset). To map the file records.dat from a point 1024 bytes from the start and extending to the end of the file, type

m = memmapfile('records.dat', 'offset', 1024);

You can change the starting position of an existing memory map by setting the offset property for the associated object to a new value. The following command sets the offset of memmapfile object m to be 2,048 bytes from the start of the mapped file:

```
m.offset = 2048;
```

Note The Sol2 and HP-UX platforms support aligned data access only. If you attempt to use a memmapfile offset on Sol2 or HP-UX that does not take the necessary alignment considerations into account, MATLAB generates an error. (See Aligned Access on Sol2 and HP-UX).

Identifying the Contents of the Mapped Region

By default, MATLAB considers all the data in a mapped file to be a sequence of unsigned 8-bit integers. To have the data interpreted otherwise as it is read or written to in the mapped file, specify a format parameter and value in your call to the constructor:

objname = memmapfile(filename, 'format', formatspec)

The formatspec argument can either be a character string that identifies a single data type used throughout the mapped region, or a cell array that specifies more than one data type.

For example, say that you map a file that is 12 kilobytes in length. Data read from this file could be treated as a sequence of 6,000 16-bit (2-byte) integers, or as 1,500 8-byte double-precision floating-point numbers, to name just a couple of possibilities. Or you could read this data in as a combination of different types: for example, as 4,000 8-bit (1-byte) integers followed by 1,000 64-bit (8-byte) integers. You determine how MATLAB will interpret the mapped data by setting the format property of the memmapfile object when you call its constructor.

Note MATLAB arrays are stored on disk in column-major order. (The sequence of array elements is column 1, row 1; column 1, row 2; ..., column 1, last row; column 2, row 1, and so on.) You might need to transpose or rearrange the order of array elements when reading or writing via a memory map.

Note The Sol2 and HP-UX platforms support aligned data access only. If you attempt to use a memmapfile format on Sol2 or HP-UX that does not take the necessary alignment considerations into account, MATLAB generates an error. (See Aligned Access on Sol2 and HP-UX).

Data types supported for the format property are shown at the end of this section. See Supported Data Types for the format Property.

For more information on format options see

Mapping a Single Data Type. If the file region being mapped contains data of only one type, specify the format value as a character string identifying that type:

objname = memmapfile(filename, 'format', datatype)

The following command constructs a memmapfile object for the entire file records.dat, and sets the format property for that object to uint64. Any read or write operations made via the memory map will read and write the file contents as a sequence of unsigned 64-bit integers:

m = memmapfile('records.dat', 'format', 'uint64');

You can change the value of the format property at any time after the memmapfile object is constructed. Use the object.property syntax shown here in assigning the new value:

```
m.format = 'int32';
```

Further read and write operations to the region mapped by m will now treat the data in the file as a sequence of signed 32-bit integers.

Property names, like format, are not case sensitive.

Formatting the Mapped Data to an Array. You can also specify an array shape to be applied to the data read or written to the mapped file, and a field name to be used in referencing this array. Use a cell array to hold these values either when calling the memmapfile constructor or when modifying m.format after the object has been constructed. The cell array contains three elements: the data type to be applied to the mapped region, the dimensions of the array shape that is applied to the region, and a field name to use in referencing the data:

objname = memmapfile(filename, 'format', ...
                     {datatype, dimensions, varname})

The command below constructs a memmapfile object for a region of records.dat such that the contents of the region are handled by MATLAB as a 4-by-10-by-18 array of unsigned 32-bit integers, and can be referenced in the structure of the returned object using the field name x:

m = memmapfile('records.dat',        ...
               'offset', 1024,       ...
               'format', {'uint32' [4 10 18] 'x'});

A = m.data.x;

whos A
  Name      Size                           Bytes  Class

  A        4x10x18                         2880   uint32 array

Grand total is 720 elements using 2880 bytes

You can change the data type, array shape, or field name that MATLAB applies to the mapped region at any time by setting a new value for the format property of the object:

```
m.format = {'uint64' [90 4 10] 'x'};
```

Mapping Multiple Data Types and Arrays. If the region being mapped is composed of segments of varying data types or array shapes, you can specify an individual format for each segment using an N-by-3 cell array, where N is the number of segments. The cells of each cell array row identify the data type for that segment, the array dimensions to map the data to, and a field name by which to reference that segment:

objname = memmapfile(filename, 'format', {                 ...
                     datatype1, dimensions1, fieldname1;   ...
                     datatype2, dimensions2, fieldname2;   ...
                         :          :            :         ...
                     datatypeN, dimensionsN, fieldnameN})

The following command maps a 24 kilobyte file containing data of three different data types: int16, uint32, and single. The int16 data is mapped as a 2-by-2 matrix that can be accessed using the field name model. The uint32 data is a scalar value accessed as field serialno. The single data is a 1-by-3 matrix named expenses.

Each of these fields belongs to the 800-by-1 structure array m.data:

m = memmapfile('records.dat', 'offset', 2048, 'format', {  ...
               'int16'  [2 2] 'model';                     ...
               'uint32' [1 1] 'serialno';                  ...
               'single' [1 3] 'expenses'});

Mapping of the Example File

The figure below shows the ordering of the array elements more closely. In particular, it illustrates that MATLAB arrays are stored on the disk in column-major order. The sequence of array elements in the mapped file is row 1, column 1; row 2, column 1; row 1, column 2; and row 2, column 2.

If the data in your file is not stored in this order, you might need to transpose or rearrange the order of array elements when reading or writing via a memory map.

Supported Data Types for the format Property . Any of the following data types can be used when you specify a format value. The default type is uint8.

format String
Data Type Description

'int8'
Signed 8-bit integers

'int16'
Signed 16-bit integers

'int32'
Signed 32-bit integers

'int64'
Signed 64-bit integers

'uint8'
Unsigned 8-bit integers

'uint16'
Unsigned 16-bit integers

'uint32'
Unsigned 32-bit integers

'uint64'
Unsigned 64-bit integers

'single'
32-bit floating-point

'double'
64-bit floating-point

format String	Data Type Description
`'int8'`	Signed 8-bit integers
`'int16'`	Signed 16-bit integers
`'int32'`	Signed 32-bit integers
`'int64'`	Signed 64-bit integers
`'uint8'`	Unsigned 8-bit integers
`'uint16'`	Unsigned 16-bit integers
`'uint32'`	Unsigned 32-bit integers
`'uint64'`	Unsigned 64-bit integers
`'single'`	32-bit floating-point
`'double'`	64-bit floating-point

Repeating a Format Scheme

Once you have set a format value for the memmapfile object, you can have MATLAB apply that format to the file data multiple times by specifying a repeat value when you call the memmapfile constructor:

objname = memmapfile(filename, 'format', formatspec, ....
                     'repeat', count)

The repeat value applies to the whole format specifier, whether that specifier describes just a single data type that repeats, or a more complex format that includes various data types and array shapes. The default repeat value is infinity (inf), which means that the full extent of the format specifier repeats as many times as possible within the mapped region.

The next example maps a file region identical to that of the previous example, except the pattern of int16, uint32, and single data types is repeated only three times within the mapped region of the file:

m = memmapfile('records.dat', 'offset', 2048, 'format', {  ...
               'int16'  [2 2] 'model';                     ...
               'uint32' [1 1] 'serialno';                  ...
               'single' [1 3] 'expenses'},                 ...
               'repeat', 3);

You can change the value of the repeat property at any time. To change the repeat value to 5, type

```
m.repeat = 5;
```

Property names, like repeat, are not case sensitive.

Keeping the Repeated Format Within the Mapped Region. MATLAB maps only the full pattern specified by the format property. If you repeat a format such that it would cause the map to extend beyond the end of the file, then either of two things can happen:

If you specify a repeat value of Inf, then only those repeated segments that fit within the file in their entirety are applied to the map.
If you specify a repeat value other than Inf, and that value would cause the map to extend beyond the end of the file, then MATLAB generates an error.

Considering the last example, if the part of the file from m.offset to the end were 70 bytes (instead of the 72 bytes required to repeat m.format three times) and you used a repeat value of Inf, then only two full repetitions of the specified format would have been mapped. The end result would be as if you had constructed the map with this command:

m = memmapfile('records.dat', 'offset', 2048, 'format', {  ...
               'int16'  [2 2] 'model';                     ...
               'uint32' [1 1] 'serialno';                  ...
               'single' [1 3] 'expenses'},                 ...
               'repeat', 2);

If repeat were set to 3 and you had only 70 bytes to the end of the file, you would get an error.

Note memmapfile does not expand or append to a mapped file. Use standard file I/O functions like fopen and fwrite to do this.

Setting the Type of Access

You can map a file region to allow either read-only or read and write access to its contents. Pass a writable parameter and value in the memmapfile constructor, or set m.writable on an existing object to set the type of access allowed:

objname = memmapfile(filename, 'writable', trueorfalse)

The value passed can be either true (equal to logical(1)) or false (equal to logical(0)). By default, it is false, meaning that the mapped region is read only.

To map a read and write region of the file records.dat in memory, type

m = memmapfile('records.dat', 'writable', true);

Note To successfully modify the file you are mapping to, you must have write permission for that file. If you do not have write permission, you can still set the writable property to true, but attempting to write to the file generates an error.

You can change the value of the writable property at any time. To make the memory map to records.dat read only, type

```
m.writable = false;
```

Property names, like writable, are not case sensitive.

The memmapfile Class Reading a Mapped File