Mamba Abstract Memory Model¶

Mamba contains a memory management component, the Mamba Resource Manager (MRM), which defines an abstract memory model to represent the available system memory and serves as an abstraction layer and uniform interface to interact with heterogeneous memories. The MRM provides a public API to construct and manage a set of memory spaces in a user application, along with a low level interface to allocate, access, transport, and free simple memory blocks in a heterogeneous memory hierarchy.

Memory Model Design¶

The Mamba library is built on the assumption there are a variety of types of memory on a single node, with different characteristics. The memory model of Mamba allows heterogeneous memory systems to be described in a uniform manner, to simplify the process of allocating and moving data across such systems. Conceptually, as illustrated in the following figure, Mamba considers all the available memory as a single abstract memory space, consisting of a group of memory sub-spaces each with a set of defined characteristics, from which data may be allocated.

A conceptual illustration of memory abstraction in the Mamba library.

This concept is implemented in the Mamba library by five key interacting data structures, illustrated below. Two memory layers exist, each with a size-limited memory space. Each space has a corresponding memory interface, from which an allocation object may be obtained and used within a specific execution context.

Memory Layer¶

A Memory Layer, defined in the Mamba API as mmbMemLayer, represents a particular type of memory with a defined set of characteristics. The memory layers available to a user are defined by the available hardware in the system, and may have characteristics such as high bandwidth or low latency. Such layers may be defined by the user, or discovered automatically where possible during library initialisation.

Memory Space¶

A Memory Space, defined in the Mamba API as mmbMemSpace, represents a size-limited, addressable, instantiation of space corresponding to a specific memory layer, from which blocks of memory may be allocated. The size may be artificially limited by the user, or hardware-limited. A memory space has an associated execution context, mmbExecutionContext, that determines how memory will be allocated and made available in that space. The execution context provides the user a means to choose the programming model through which memory must be made available. For example, NVIDIA GPU device memory could be provided within a CUDA, HIP, or OpenACC execution context.

Memory Interface¶

A Memory Interface, defined in the Mamba API as mmbMemInterface, acts as an interface to a specific memory space, providing a mechanism to allocate, move, copy, and free memory. A memory interface may have a specific strategy, that defines the behaviour of the interface. For example, this may enforce thread safety during allocation, or define the type of memory allocator used (e.g. pooled vs. slab).

Allocation¶

An Allocation object, defined in the Mamba API as mmbAllocation, provides an abstract container for a memory allocation in a specific memory space. Allocation objects are provided by memory interfaces, and passed into generic allocation, copy, and free routines. Allocation objects contain, if feasible, a real data pointer, along with provenance information such as the interface through which it was allocated, and whether the un- derlying data is owned by the allocation object or not. The concept of ownership allows, for example, sub-allocations to be created as slices of existing allocations.

Execution Context¶

An Execution Context, defined in the Mamba API as mmbExecutionContext, provides a mechanism to define the context within which an allocation object will be used during program execution. The execution context captures a variety of hardware execution targets and accompanying programming models to ensure the MRM can create allocation objects that are compatible with the intended use case for the allocation. For example, a variety of memory allocators are available for GPU; the correct choice depends on the type of GPU and the programming model used by the application developer. The execution context allows the developer to specify, for example, to use an OpenCL based memory allocator rather than a CUDA-based allocator for a GPU allocation in order to be compatible with application code written using the OpenCL programming model.

Memory Model API¶

The Mamba library provides two mechanisms for constructing a topology view of the underlying memory system. Topology Discovery, where the available memories are discovered during Mamba initialisation via integration with the hwloc library for hardware locality; or topology construction, where the application developer constructs their own view of the memory system via the Mamba API.

Topology Discovery or Construction¶

The discovery feature enables Mamba to automatically discover the available memories on a node via the hwloc library for hardware locality. This mode is enabled via build configuration argument --enable-discovery; when enabled, discovery will occur during initialisation of the Mamba library.

During discovery, Mamba will store internally a memory topology that includes a list of available memory layers and a default memory space for each layer. In the case discovery is not enabled, the Mamba library relies on the user to provide a view of the available memory system. The user may do this via a series of API calls to register available memory layers. Once a memory layer is registered, a default space is created and the user may further configure or construct spaces. As a convenience, the user may combine these steps to construct a new memory space corresponding to the layer upon registration.

mmbError mmb_register_memory(const mmbMemLayer layer,
                   const mmbExecutionContext ex_context,
                   const mmbMemSpaceConfig *config_opts,
                   mmbMemSpace **new_space);

Space Construction¶

Once the system memory topology has been discovered or constructed, the user may then request a pre-constructed space and further configure if necessary, or create their own space with custom configuration

mmbError mmb_create_space(const mmbMemLayer layer,
                         const mmbExecutionContext ex_con,
                         const mmbMemSpaceConfig *config_opts,
                         mmbMemSpace **out_space);

mmbError mmb_request_space(const mmbMemLayer layer,
                         const mmbExecutionContext ex_con,
                         const mmbMemSpaceConfig *config_opts,
                         mmbMemSpace **out_space);

mmbError mmb_memspace_configure(const mmbMemSpaceConfig *config_opts,
                         mmbMemSpace *space);

The custom configuration support includes providing default options for any memory interfaces constructed in this space, and configuring a set of sub-domains within the memory space, for example to support a space spanning multiple NUMA regions or spanning multiple devices.

Interface Construction¶

Once a suitable set of memory spaces have been discovered, or constructed, the user may acquire one or more memory interfaces. As shown in the code snippet below, upon provision of a valid memory space and set of configuration options, the user may construct a new, or request an existing, memory interface.

mmbError mmb_create_interface(mmbMemSpace *space,
                             const mmbMemInterfaceConfig *config_opts,
                             mmbMemInterface **out_interface);

mmbError mmb_request_interface(mmbMemSpace *space,
                             const mmbMemInterfaceConfig *config_opts,
                             mmbMemInterface **out_interface);

mmbError mmb_destroy_interface(mmbMemInterface *interface);

Once a valid memory interface is acquired, it may be used to either construct a Mamba Array or allocate blocks of memory using the resource management API.