Lumberyard
User Guide (Version 1.16)

Using Memory Allocators in Lumberyard

Lumberyard’s memory management system determines how memory is allocated. In Lumberyard version 1.16, the memory management system has been refactored. All memory allocations go through one pipeline, and memory allocation can be tracked. This makes it easier and quicker to pinpoint memory leaks or optimize memory usage to improve game performance. This improvement is especially important for mobile and console applications, where memory resources are usually more constrained than in PC environments.

Lumberyard supports all the best known memory allocation schemes. You can use Lumberyard's allocators to categorize allocations or keep similar allocations together to improve locality or reduce fragmentation.

Manually Allocating Memory

Lumberyard uses the following memory allocation functions. You can find the source code in the lumberyard_version\dev\Code\Framework\AzCore\AzCore\Memory\ directory.

Lumberyard Function Replaces Overloads and Descriptions
aznew operator new

Use instead of new T(). If AZCORE_ENABLE_MEMORY_TRACKING is defined, uses the bound allocator and record tracking information for T.

Note

There is no azdelete function. The built-in operator delete is sufficient.

azmalloc malloc
azmalloc(size, [alignment])

Uses the SystemAllocator to allocate memory.

azmalloc(size, alignment, Allocator, [allocationName])

Uses the specified allocator to allocate memory. allocationName specifies a optional name for the allocation.

azcalloc calloc
azcalloc(size, [alignment])

Uses SystemAllocator to allocate memory and fills the memory with zeroes.

azcalloc(size, alignment, Allocator, [allocationName])

Uses the specified allocator to allocate memory and fills the memory with zeroes. Optionally specifies a name for the allocation.

azrealloc realloc
azrealloc(ptr, size)

Reallocates from SystemAllocator.

azrealloc(ptr, size, Allocator, [allocationName])

Reallocates from the specified allocator. Optionally specifies a name for the allocation.

azfree free
azfree(ptr)

Frees memory and returns it to the SystemAllocator.

azfree(ptr, Allocator)

Frees memory and returns it to the specified allocator.

azcreate new with allocator
azcreate(T, [params])

Uses the SystemAllocator to allocate memory and passes optional parameters wrapped in () or {} to the constructor.

azcreate(T, params, Allocator, [allocationName])

Uses the specified allocator to allocate memory. Optionally specifies a name for the allocation.

azdestroy operator delete with allocator

Destroys memory. azdestroy should be used only with a corresponding azcreate function.

azdestroy(T*)

Calls the destructor, frees memory, and returns the memory to the SystemAllocator.

azdestroy(T*, Allocator)

Calls the destructor, frees memory, returns the memory to the specified allocator.

AZ Memory Allocators

The following diagram illustrates the hierarchy of AZ memory allocators.


        AZ memory allocator hierarchy
  • OSAllocator – Acts as the interface to operating system memory and should be used for direct operating system allocations on the C heap. OSAllocator is booted as early as possible in main(), and removed last, right before returning. If you don't create OSAllocator, the SystemAllocator creates it when needed.

    OSAllocator uses system calls to allocate memory. The calls are not recorded or tracked. Other allocators use OSAllocator to obtain memory from the operating system. Drillers and memory tracking tools can use OSAllocator for data debugging.

  • BestFitExternalMapAllocator – Uses external maps to store memory tracking information for uncached memory.

  • SystemAllocator – The system allocator is the general purpose allocator for the AZ memory library. Like all other allocators, SystemAllocator is a singleton, but it must be initialized first and destroyed last. All other allocators use SystemAllocator for internal allocations.

  • LegacyAllocator – Handles legacy memory allocations. For more information, see Legacy Memory Management.

  • PoolAllocator – Performs extremely fast small object memory allocations. PoolAllocator can allocate sizes in a range specified by m_minAllocationSize to m_maxPoolSize.

    Note

    PoolAllocator is not thread safe. If you need a thread-safe version, use ThreadPoolAllocator, or inherit from ThreadPoolBase and then write custom code to handle the synchronization.

  • ThreadPoolAllocator – Thread safe pool allocator. If you want to create your own thread pool heap, inherit from ThreadPoolBase, as Lumberyard requires a unique static variable for the allocator type.

Applying Allocators to Your Classes

To apply an allocator to your class, use the AZ_CLASS_ALLOCATOR macro in your class or directly call AZ::AllocatorInstance<some_allocator>.

AZCore relies on AZ_CLASS_ALLOCATOR to specify the default allocator for the class or on explicit azcreate and azdestroy calls that specify the allocator in their signature.

  • If your class does not implement AZ_CLASS_ALLOCATOR and you call new or delete, new or delete calls use the global operator new or operator delete.

  • If your class does not implement AZ_CLASS_ALLOCATOR and you call aznew, you must implement a new operator that uses the aznew call signature.

AZ Allocator Schemas

Each allocator commonly implements the IAllocator interface and uses a schema to implement the allocation algorithms and bookkeeping. This strategy enables the same schema to be used in multiple allocators.

Allocator Schemas

Schema Description
AZ::HphaSchema

This is the preferred schema. It combines a small block allocator for small allocations and a red-black tree for large allocations. This provides good general purpose performance. Use this schema if you're not sure which one to use.

Note

HphaSchema is based on Dimitar Lazarov's "High Performance Heap Allocator" (Game Programming Gems 7, Charles River Media, 2008, pp. 15–23).

AZ::HeapSchema

Uses nedmalloc internally. Because nedmalloc uses thread caches to accelerate the re-use of memory, HeapSchema can be useful for intensive allocation processing across multiple threads.

AZ::BestFitExternalSchema

A best-fit allocation scheme that uses an external map to store bookkeeping outside the memory being managed. Because the tracking node is stored outside the main chunk, Lumberyard can use this allocator with uncached memory. This is most useful for GPU resource management (for example, for textures, constant buffers, and compute buffers).

AZ::ChildAllocatorSchema

Acts as a pass-through schema to another allocator. Use this schema to create a new allocator based on an existing allocator like SystemAllocator. To properly tag the memory that each gem or logical subsystem allocates, each gem or subsystem can create its own child allocator. For more information, see Creating an Allocator.

AZ::PoolSchema

A specialized schema that implements a small block allocator for managing small, high-throughput allocations. Objects are typically pooled at the cost of using more memory.

Note

PoolSchema is not thread safe. If you need a thread-safe version, use ThreadPoolSchema or write custom code to handle the synchronization.

AZ::ThreadPoolSchema

A thread-safe pool schema that uses thread local storage to implement a small block allocator for each thread.

Important

Because the thread pool allocator creates separate pools for each thread, it uses somewhat more memory, especially for fixed pool sizes.

Creating an Allocator

We recommend that each Lumberyard gem or logical subsystem create a ChildAllocator to properly tag the memory that it allocates. This practice makes it easier to budget resource usage and get a holistic view of it.

If you choose to write your own schema, be aware that caching significant chunks of memory can be problematic. Such caching can hamper the ability of other systems to evolve to fit the content in your game. Unless you have specific requirements, we recommend that you create a ChildAllocator that eventually uses the SystemAllocator. Using a ChildAllocator ensures that your memory is as recoverable and reusable as possible.

Prior to Lumberyard version 1.16, the most common mechanism for creating a new allocator was to inherit from SystemAllocator. This practice, which creates a completely separate free list of memory usage, results in the problem of memory being spread among disparate caches. In most cases, it is better to use the inheritance ChildAllocator<SystemAllocator>, which also makes it trivial to swap the base class of your custom allocator.

To create an allocator

  1. Choose a schema to use, write a custom schema, or choose an existing allocator that you want to modify. For more information, see AZ Allocator Schemas.

  2. Inherit from AllocatorBase<your_schema> to create your Allocator class.

  3. Add AZ_TYPE_INFO so that AllocatorInstance<> can properly manage your type.

Using Your Own Allocators from Containers

To use your own allocator from a container, wrap your allocator in AZ::AZStdAlloc, like the following example.

AZStd::vector<MyClass, AZ::AZStdAlloc<CustomAllocator>>

Static Initialization

In a monolithic build, at static initialization time (before the allocators are bootstrapped), allocations are routed directly to the underlying operating system. These static allocations are tracked in a fixed size set and sent back to the OS when they are freed. They are also reported separately to memory tracking in the Global category. To discover the memory that is being allocated globally, set a breakpoint in AZ::Internal::GlobalAlloc.

Legacy Memory Management

Starting in Lumberyard version 1.16, all Cry* allocation routines route to AZ::LegacyAllocator, which you can find in the lumberyard_version\dev\Code\CryEngine\CryCommon\LegacyAllocator.h file. LegacyAllocator has the same lifetime as OSAllocator and obtains its memory from OSAllocator.

Cry dynamic-link libraries override the new and delete functions instead of tagging their classes with allocators. This behavior is controlled with the USE_CRY_NEW_AND_DELETE macro (lumberyard_version\dev\Code\CryEngine\CryCommon\CryMemoryManager_impl.h). This practice should not be used outside Cry DLLs. When compiled monolithically, the LegacyAllocator catches any uses of global new or delete. This allows all allocations to be tracked and managed.

Note

As of Lumberyard version 1.16, operator new and operator delete overrides are restricted to Cry DLLs.

All Cry static functions that allocate memory have been removed or wrapped in StaticInstance<T>, which creates the functions only when the functions are first accessed. StaticInstance<T> can be used in any DLL which depends on CryCommon and includes the lumberyard_version\dev\Code\CryEngine\CryCommon\platform_impl.h file.

Note

Within AZ code, there is a NewAndDelete.inl file in the lumberyard_version\dev\Code\Framework\AzCore\AzCore\Memory\ directory. You can use this code within a gem DLL, but only in nonmonolithic mode. We do not recommended using this code except to track down untagged classes or allocations.