Buffer manager
Systems, methods, apparatus and computer-readable medium are described for improving efficiency and robustness for sharing data across different users, agents, processes, drivers, and applications in a system. A shared buffer between multiple entities may be used for sharing data such that the data is only stored once and accessed by multiple entities without copying the data over and over again. An example system may be a network device. For a received packet at a network interface of a network device, the packet may be stored directly in memory. The application or process responsible for accessing and/or manipulating the packet can directly do so by simply using a buffer pointer provided by the buffer manager.
1. A device comprising:
one or more processors configured to execute a buffer manager in kernel mode, a first process in kernel mode and a second process in user mode, wherein the buffer manager manages a plurality of buffers;
the first process is configured to:
request from the buffer manager access to a first buffer from the plurality of buffers, a base address for the plurality of buffers being mapped to a fixed virtual address wherein the buffer manager assigns the fixed virtual address to each process requesting access to the plurality of buffers for all buffers in the plurality of buffers mapping to a same physical memory space, wherein fixing the virtual and physical address translation allows transfer of control for a buffer by transferring a buffer offset, wherein each process calculates the virtual address for the buffer based on the base address for each process and the buffer offset;
receive a first address in response to the request for access to the first buffer;
use the first address from the first process to access the first buffer; and
send the first address from the first process to a second process;
the second process is configured to access the first buffer using the first address received from the first process; and
a third process, using the first address, de-allocates use of the first buffer.
2. The device of claim 1 , comprising memory configured to store
the plurality of buffers comprising the first buffer; and
a metadata list comprising a plurality of entries, wherein a first entry from the plurality of entries is accessed using the first address.
3. The device of claim 2 , wherein the first entry includes the first address for accessing the first buffer.
4. The device of claim 2 , wherein the first entry includes one or more of process identifier, timestamp, and validity indicator.
5. The device of claim 2 , wherein each entry of the plurality of entries of the metadata list is associated with a buffer from the plurality of buffers and metadata stored at each entry of the plurality of entries provides information associated with using of the respective buffer.
6. The device of claim 2 , wherein the buffer manager is further configured to provide an interface for accessing the first buffer from the plurality of buffers, wherein the buffer manager manages information associated with the first buffer by updating the first entry associated with the first buffer.
7. The device of claim 1 , wherein a contiguous physical memory block is allocated for the plurality of buffers and the metadata list.
8. The device of claim 1 , wherein the device is a network device and the network device further comprises:
a network interface configured to receive a network packet at the network interface;
a direct memory access engine for transferring a portion of the network packet to the first buffer from the plurality of buffers in physical memory using a direct memory access transfer; and
the first process and the second process to access the portion of the network packet stored in the first buffer using the first address.
9. The device of claim 1 , wherein the plurality of buffers comprises multiple pools of buffers, wherein
a first pool of buffers from the plurality of buffers comprises buffers of a first size; and
a second pool of buffers from the plurality of buffers comprises buffers of a second size.
10. The device of claim 1 , wherein a plurality of users is configured to access a buffer using an interface provided by the buffer manager.
11. The device of claim 1 , wherein each of a plurality of users is executing in a different process.
12. A method comprising:
executing, on one or more processors, a buffer manager in kernel mode, a first process in kernel mode and a second process in user mode, wherein the buffer manager manages a plurality of buffers;
requesting, from the first process, access to a first buffer from the plurality of buffers, a base address for the plurality of buffers being mapped to a fixed virtual address wherein the buffer manager assigns the fixed virtual address to each process requesting access to the plurality of buffers for all buffers in the plurality of buffers mapping to a same physical memory space, wherein fixing the virtual and physical address translation allows transfer of control for a buffer by transferring a buffer offset, wherein each process calculates the virtual address for the buffer based on the fixed virtual address for each process and the buffer offset;
receiving a first address at the first process for accessing the first process;
using, by the first process, the first address to access the first buffer;
sending, by the first process, the first address from the first process to a second process;
receiving, by the second process, the first address to access the first buffer using the first address received from the first process; and
by a third process, using the first address, de-allocating use of the first buffer using the first address.
13. The method of claim 12 , comprising accessing a metadata list comprising a plurality of entries, wherein a first entry from the plurality of entries is accessed using the first address.
14. The method of claim 13 , wherein the first entry includes one or more of the first address, process identifier, timestamp, and validity indicator.
15. The method of claim 13 , wherein each entry of the plurality of entries of the metadata list is associated with a buffer from the plurality of buffers and metadata stored at each entry of the plurality of entries provides information associated with using of the respective buffer.
16. The method of claim 13 , wherein the buffer manager is further configured to provide an interface for accessing the first buffer from the plurality of buffers, wherein the buffer manager manages information associated with the first buffer by updating the first entry associated with the first buffer.
17. The device of claim 1 , wherein the buffer manager and users operating in the user mode or kernel mode transfer control for a buffer by transferring the buffer offset from one user to another.
18. The method of claim 12 , wherein the buffer manager and users operating in the user mode or kernel mode transfer control for a buffer by transferring the buffer offset from one user to another.
19. The device of claim 1 , wherein the one or more processors are further configured to allocate a buffer from the plurality of buffers, wherein the one or more processors for allocating the buffer from the plurality of buffers are further configured to
determine if a buffer size requested is valid,
select a buffer with a valid size from the plurality of buffers,
check whether the plurality of buffers is running out of space, and
based on whether valid pointers are retrieved, determine whether the plurality of buffers are out of space or running out of space.
20. The method of claim 12 , further comprising:
allocating a buffer from the plurality of buffers, wherein allocating a buffer from the plurality of buffers includes
determining if a buffer size requested is valid;
selecting a buffer with a valid size from the plurality of buffers;
checking whether the plurality of buffers is running out of space; and
based on whether valid pointers are retrieved, determining whether the plurality of buffers are out of space or running out of space.