Lock free container packing
View Patent ↗Systems and methods for writing data are provided. A lock-free container and methods of writing to the lock-free container are disclosed. The container is associated with a tail pointer that identifies free space in the container. Threads writing to the container access the tail pointer and update an offset in the tail pointer to account for a size of a write to the container. Multiple threads can write to the same container without having to contend for a container lock.
1. In a system that includes a memory and a storage system, a method for writing to a container stored in the memory, the method comprising:
associating a container with a tail pointer that is stored in the memory, wherein the tail pointer identifies a location in the container associated with free space and wherein the container is configured such that multiple threads can write to the container at the same time;
configuring the tail pointer such that the tail pointer can only be accessed and updated by one thread at a time;
when a writing thread obtains access to the tail pointer, allocating a portion of the free space in the container to the writing thread, wherein the thread updates the tail pointer such that the portion of the free space allocated to the writing thread is no longer available to other threads;
writing a segment to the portion of the free space by the writing segment; and
incrementing a segment counter and retrieving a segment index for the segment as an index into a header array, wherein the writing thread inserts meta-data about the segment into the header array at the segment count.
2. The method of claim 1 , further comprising guaranteeing that the segment index corresponds to an ordinal position of the segment in the container.
3. The method of claim 1 , further comprising writing, by a last thread to write to the container, the container to the storage system.
4. The method of claim 1 , wherein multiple threads write to the container simultaneously.
5. The method of claim 1 , wherein the memory contains multiple containers and each container is associated with a corresponding tail pointer, wherein multiple threads write to multiple containers simultaneously.