Parallel dynamic memory allocation using a nested hierarchical heap
One embodiment of the present invention sets forth a technique for dynamically allocating memory using a nested hierarchical heap. A lock-free mechanism is used to access to a hierarchical heap data structure for allocating and deallocating memory from the heap. The heap is organized as a series of levels of fixed-size blocks, where all blocks at given level are the same size. At each lower level of the hierarchy, a collection of N blocks in the lower level equals the size of a single block at the level above. When a thread requests an allocation, one or more blocks at only one level are allocated to the thread. When threads are finished using an allocation, each thread deallocates the respective allocated blocks. When all of the blocks for a level have been deallocated, defragmentation is performed at that level.
1. A method of allocating memory from a nested hierarchical heap, the method comprising:
receiving a memory allocation request specifying an amount of memory;
identifying a heap level based on the amount of memory and a block size of a plurality of heap levels of the nested hierarchical heap, wherein each of the plurality of heap levels of the nested hierarchical heap is associated with a different block collection state;
computing a number of blocks needed to satisfy the memory allocation request;
determining that the number of blocks is available at the heap level by reading the block collection state associated with the heap level, wherein each bit included in the block collection state is associated with a different block of the heap level and indicates whether the block is available; and
allocating the number of blocks using an atomic operation.
2. The method of claim 1 , further comprising:
determining that the number of blocks is not available at the heap level; and
generating a new block collection at the heap level from a first block at an immediately higher heap level of the nested hierarchical heap.
3. The method of claim 2 , further comprising, prior to generating the new block collection, setting a flag by a first thread to prevent other threads from generating another new block collection at the heap level.
4. The method of claim 3 , further comprising, after generating the new block collection, clearing the flag by the first thread to allow other threads to generate another new block collection at the heap level.
5. The method of claim 1 , further comprising:
receiving a memory free request specifying the amount of memory; and
releasing the number of blocks at the heap level using another atomic operation.
6. The method of claim 5 , further comprising:
determining that defragmentation should occur; and
releasing a parent block from which a block collection including the number of blocks was generated.
7. The method of claim 1 , wherein each heap level includes one or more block collections that each include N blocks, and a single block collection at a first heap level of the nested hierarchical heap has a storage capacity of a single block at an immediately higher heap level of the nested hierarchical heap.
8. The method of claim 1 , wherein the allocating of the number of blocks comprises executing an atomic compare-and-swap operation.
9. The method of claim 1 , wherein each block collection state further includes a status bit configured to lock access to the corresponding heap level during block collection generation and defragmentation of the heap level.
10. A system for allocating memory from a nested hierarchical heap, the system comprising:
a memory that is configured to store the nested hierarchical heap; and
a processor that is configured to:
receive a memory allocation request specifying an amount of memory;
identify a heap level based on the amount of memory and a block size of a plurality of heap levels of the nested hierarchical heap, wherein each of the plurality of heap levels of the nested hierarchical heap is associated with a different block collection state;
compute a number of blocks needed to satisfy the memory allocation request;
determine that the number of blocks is available at the heap level by reading the block collection state associated with the heap level, wherein each bit included in the block collection state is associated with a different block of the heap level and indicates whether the block is available; and
allocate the number of blocks using an atomic operation.
11. The system of claim 10 , wherein the processor is configured to:
determine that the number of blocks is not available at the heap level; and
generate a new block collection at the heap level from a first block at an immediately higher heap level of the nested hierarchical heap.
12. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to allocate memory from a nested hierarchical heap, by performing the steps of:
receiving a memory allocation request specifying an amount of memory;
identifying a heap level based on the amount of memory and a block size of a plurality of heap levels of the nested hierarchical heap, wherein each of the plurality of heap levels of the nested hierarchical heap is associated with a different block collection state;
computing a number of blocks needed to satisfy the memory allocation request;
determining that the number of blocks is available at the heap level by reading the block collection state associated with the heap level, wherein each bit included in the block collection state is associated with a different block of the heap level and indicates whether the block is available; and
allocating the number of blocks using an atomic operation.
13. The non-transitory computer-readable storage medium of claim 12 , further comprising:
determining that the number of blocks is not available at the heap level; and
generating a new block collection at the heap level from a first block at an immediately higher heap level of the nested hierarchical heap.
14. The non-transitory computer-readable storage medium of claim 13 , further comprising, prior to generating the new block collection, setting a flag by a first thread to prevent other threads from generating another new block collection at the heap level.
15. The non-transitory computer-readable storage medium of claim 14 , further comprising, after generating the new block collection, clearing the flag by the first thread to allow other threads to generate another new block collection at the heap level.
16. The non-transitory computer-readable storage medium of claim 12 , further comprising:
receiving a memory free request specifying the amount of memory; and
releasing the number of blocks at the heap level using another atomic operation.
17. The non-transitory computer-readable storage medium of claim 16 , further comprising:
determining that defragmentation should occur; and
releasing a parent block from which a block collection including the number of blocks was generated.
18. The non-transitory computer-readable storage medium of claim 12 , wherein each heap level includes one or more block collections that each include N blocks, and a single block collection at a first heap level of the nested hierarchical heap has a storage capacity of a single block at an immediately higher heap level of the nested hierarchical heap.
19. The non-transitory computer-readable storage medium of claim 12 , wherein the allocating of the number of blocks comprises executing an atomic compare-and-swap operation.