Mamba Resource Manager¶
The Mamba Resource Manager (MRM) implements the memory model, providing the required data structures and API to build the higher level Mamba data containers. The broad structure of the MRM is illustrated in the figure, showing the key software layers through which a user request to allocate memory will pass. In the following we detail the Interface and Provider layers, along with the two primary supported providers.
The structure of the Mamba Resource Manager, consisting of an allocation API supported by an interface layer and a provider layer that provide stackable allocation strategies and per layer and execution context allocator implementations.¶
Interface Layer¶
The interface layer of the MRM provides an abstraction layer between the user allocation API, and the underlying memory providers. One or more interfaces may be created per memory space, separating user access to different spaces, whilst sharing access to the underlying provider layer that may serve multiple different spaces.
General interface construction as part of the memory API utilisation is demonstrated in Mamba Abstract Memory Model, in this section we provide further details on configuration of interfaces. During construction, an interface may be provided a set of options for configuration, for example which underlying provider to use.
As Mamba abstracts data management and not compute, it can be necessary for users to pass in provider-specific data structures for effective cooperation between user-managed compute and Mamba-managed data when a shared compute/data programming model is utilised. The following code snippet shows an example of such data structures being passed in during interface construction. A provider-specific options structure contains opaque pointers for each underlying memory implementation, which may be used via explicit include of an interoperability header file.
#include "mmb_provider_opencl.h"
// Omitted user-managed OpenCL setup, creating a cl_context context, a cl_command queue queue, and a_cl device id list devices[].
// Omitted Mamba memory space setup, as shown in previous sections, creating an mmbMemorySpace gpu_space.
cl_device_id d = devices[device_index];
cl_context c = context;
cl_command_queue q = queue;
struct mmb_opencl_t oclopt = { .device = d, .context = c, .queue = q};
mmbMemInterfaceConfig gpu_opencl_conf = (mmbMemInterfaceConfig){
.provider = MMB_PROVIDER_ANY,
.strategy = MMB_STRATEGY_ANY,
.name = "",
.provider_opts = {.ocl = &oclopt}
};
mmbMemInterface *gpu_interface;
mmb_create_interface(gpu_space, &gpu_opencl_conf, &gpu_interface);
Each interface may apply one of a set of available strategies. A strategy provides a set of common functionality on top of individual providers, for example adding thread-safety or memory pooling support to a specific interface without requiring such support to be implemented in every hardware-specific provider implementation. The chosen strategy may be applied at interface configuration time as in the code above, strategies may also be stacked in the implementation, for example to provide thread-safe and pooled support in a single strategy. The currently available strategies are:
MMB_STRATEGY_ANYMMB_STRATEGY_NONEMMB_POOLEDMMB_THREAD_SAFEMMB_STATISTICSMMB_POOLED_STATISTICS
The choice of MMB_STRATEGY_ANY results in the interface utilising a default strategy for the memory space, which if not specified during space construction is MMB_STRATEGY_NONE.
Each configuration option that offers a default setting is configurable via compile-time or environment variables.
Compile-time Variables¶
The following variables can be set at compile time (or during the call to configure when provided to CPPFLAGS). In order to set their value, use the format -D<name>=<value>.
MMB_CONFIG_PROVIDER_DEFAULT: Default memory provider to use to allocate mem- ory when none is requested. Default:MMB_NATIVE.MMB_CONFIG_STRATEGY_DEFAULT: Default memory allocation strategy to use when none is requested. Default:MMB_STRATEGY_NONE.MMB_CONFIG_EXECUTION_CONTEXT_GPU_DEFAULT: Default execution context to use when allocating and copying memory to/from GPUs. Default:MMB_GPU_CUDA.MMB_CONFIG_PROVIDER_DEFAULT_ENV_NAME: Environment variable’s name to look for when setting default provider. Default:MMB_CONFIG_PROVIDER_DEFAULT.MMB_CONFIG_STRATEGY_DEFAULT_ENV_NAME: Environment variable’s name to look for when setting default strategy. Default:MMB_CONFIG_STRATEGY_DEFAULT.MMB_CONFIG_INTERFACE_NAME_DEFAULT_ENV_NAME: Environment variable’s name to look for when setting default strategy. Default:MMB_CONFIG_INTERFACE_NAME_DEFAULT.MMB_CONFIG_EXECUTION_CONTEXT_GPU_DEFAULT_ENV_NAME: Environment variable’s name to look for when setting default execution context for the GPU. Default:MMB_CONFIG_EXECUTION_CONTEXT_GPU_DEFAULT.
Run-time variables¶
The following variables can be set in the environment at run-time in order to modify some of the compile-time defined behaviors. These variable are read only once, during the library initialization.
MMB_CONFIG_PROVIDER_DEFAULTMMB_CONFIG_STRATEGY_DEFAULTMMB_CONFIG_INTERFACE_NAME DEFAULTMMB_CONFIG_EXECUTION_CONTEXT_GPU_DEFAULT
The variables defaults to the compile-time values. The name of these of these variable can be changed at compile time using the available settings above. For simplicity, MMB_CONFIG_EXECUTION_CONTEXT_GPU_DEFAULT also accepts NONE as a valid choice.
Statistics¶
The statistics strategy will record telemetry data during utilisation of the interface, this currently includes the number of allocations and total allocated size. Work is on- going to extend this to include telemetry related to data transport, and extend the user-interface to interact with telemetry data.
Provider Layer¶
The provider layer provides the lowest level abstraction of the underlying memory hardware. Multiple providers are supported, and each provider implements the hardware and/or programming-model specific allocation API for each memory layer and execution context pairing supported by that provider.
Primary support in Mamba is for the System and uMMAP-IO providers, however experimental support is included for the SICM, UMPIRE, and JEMALLOC memory management libraries.
Provider: System¶
The System provider is the default provider in Mamba, and is built into the library. This provider has support for the following memory layers:
MMB_DRAMMMB_GDRAMMMB_HBMMMB_NVDIMM
As well as the following execution contexts:
MMB_CPUMMB_GPU_CUDAMMB_GPU_HIPMMB_GPU_OPENCL
For each valid pairing of memory layer and execution context, the system provider implements a set of memory management functions to be utilised by the Mamba resource manager.