Methods and systems for optimizing file system usage
A method for generating a thread queue, that includes obtaining, by a user space file system, central processing unit (CPU) socket data, and based on the CPU socket data, generating a plurality of thread handles for a plurality of cores, ordering the plurality of thread handles, in the thread queue, for a first core of the plurality of cores, and saving the thread queue to a region of shared memory.
1. A method for generating thread queues, comprising:
obtaining, by a user space file system, central processing unit (CPU) socket data, the CPU socket data comprising: identities of processors, identities of a plurality of cores of the processors, and in which sockets the processors reside; and
for each core in the plurality of cores:
generating at least one thread handle to associate with the core;
maintaining, to associate with the core, a thread queue comprising a plurality of thread handles, the plurality of thread handles comprising the at least one thread handle generated for each core in the plurality of cores; and
ordering, within the thread queue, the plurality of thread handles based on a latency and a cache locality between the core associated with the thread queue and other cores in the plurality of cores;
saving the thread queues to a region of shared memory,
wherein the region of the shared memory is shared by an application and the user space file system;
receiving, by a process coordinator, a file request specifying a core identifier corresponding to the core;
identifying the thread queue associated with the core identifier;
selecting a first thread handle of the plurality of thread handles ordered within the thread queue; and
directing the file request to be executed on a thread associated with the first thread handle,
wherein the file request is subsequently executed on the thread associated with the first thread handle.
2. The method of claim 1 , wherein a first subset of the plurality of thread handles comprises the at least one thread handle generated for the core, wherein the first subset of the plurality of thread handles ordered within the thread queue associated with the core is positioned first in the thread queue.
3. The method of claim 2 , wherein a second subset of the plurality of thread handles comprises the at least one thread handle generated for a second core in the plurality of cores, wherein the second subset of the plurality of thread handles ordered within the thread queue associated with the core is positioned last in the thread queue, wherein the core has a greater latency with respect to the second core as compared with the other cores in the plurality of cores.
4. The method of claim 1 , wherein the method further comprises:
receiving, by the user space file system, a second file request comprising a file identifier;
locating file data associated with the file identifier in persistent storage; and
copying the file data from the persistent storage to a memory region based on a user space file system virtual address space.
5. The method of claim 4 , wherein the second file request was generated by the application, wherein the application comprises an application virtual address space mapped to the memory region.
6. The method of claim 4 , wherein the second file request was written to a second memory region associated with a second thread handle of the plurality of thread handles ordered within the thread queue associated with the core.
7. The method of claim 6 , wherein the second file request is associated with the core.
8. The method of claim 7 , wherein the user space file system virtual address space is uniquely associated with a second thread, and wherein copying of the file data comprises updating a page table entry of a memory management unit (MMU), uniquely associated with the user space file system virtual address space, to include the memory region.
9. The method of claim 7 , wherein the copying of the file data is processed by a second core in the plurality of cores, wherein the thread queue associated with the core no longer comprises the at least one thread handle generated for the core, wherein a new first thread handle in the thread queue had been generated for the second core.
10. A non-transitory computer readable medium comprising instructions which, when executed by a computer processor, enables the computer processor to perform a method for generating thread queues, comprising:
obtaining, by a user space file system, central processing unit (CPU) socket data, the CPU socket data comprising: identities of processors, identities of a plurality of cores of the processors, and in which sockets the processors reside; and
for each core in the plurality of cores:
generating at least one thread handle to associate with the core;
maintaining, to associate with the core, a thread queue comprising a plurality of thread handles, the plurality of thread handles comprising the at least one thread handle generated for each core in the plurality of cores; and
ordering, within the thread queue, the plurality of thread handles based on a latency and a cache locality between the core associated with the thread queue and other cores in the plurality of cores;
saving the thread queues to a region of shared memory,
wherein the region of the shared memory is shared by an application and the user space file system;
receiving, by a process coordinator, a file request specifying a core identifier corresponding to the core;
identifying the thread queue associated with the core identifier;
selecting a first thread handle of the plurality of thread handles ordered within the thread queue; and
directing the file request to be executed on a thread associated with the first thread handle,
wherein the file request is subsequently executed on the thread associated with the first thread handle.
11. The non-transitory computer readable medium of claim 10 , wherein a first subset of the plurality of thread handles comprises the at least one thread handle generated for the core, wherein the first subset of the plurality of thread handles ordered within the thread queue associated with the core is positioned first in the thread queue.
12. The non-transitory computer readable medium of claim 11 , wherein a second subset of the plurality of thread handles comprises the at least one thread handle generated for a second core in the plurality of cores, wherein the second subset of the plurality of thread handles ordered within the thread queue associated with the core is positioned last in the thread queue, wherein the core has a greater latency with respect to the second core as compared with the other cores in the plurality of cores.
13. The non-transitory computer readable medium of claim 10 , wherein the method further comprises:
receiving, by the user space file system, a second file request comprising a file identifier;
locating file data associated with the file identifier in persistent storage; and
copying the file data from the persistent storage to a memory region based on a user space file system virtual address space.
14. The non-transitory computer readable medium of claim 13 , wherein the second file request was generated by the application, wherein the application comprises an application virtual address space mapped to the memory region.
15. The non-transitory computer readable medium of claim 13 , wherein the second file request was written to a second memory region associated with a second thread handle of the plurality of thread handles ordered within the thread queue associated with the core.
16. The non-transitory computer readable medium of claim 15 , wherein the second file request is associated with the core.
17. The non-transitory computer readable medium of claim 16 , the user space file system virtual address space is uniquely associated with a second thread, and wherein copying of the file data comprises updating a page table entry of a memory management unit (MMU), uniquely associated with the user space file system virtual address space, to include the memory region.
18. A node, comprising:
user space file system;
an application;
memory;
a plurality of cores; and
a processor, wherein the processor is configured to perform a method for generating thread queues, comprising:
obtaining, by the user space file system, central processing unit (CPU) socket data, the CPU socket data comprising:
identities of processors, identities of a plurality of cores of the processors, and in which sockets the processors reside; and
for each core in the plurality of cores:
generating at least one thread handle to associate with the core;
maintaining, to associate with the core, a thread queue comprising a plurality of thread handles, the plurality of thread handles comprising the at least one thread handle generated for each core in the plurality of cores; and
ordering, within the thread queue, the plurality of thread handles based on a latency and a cache locality between the core associated with the thread queue and other cores in the plurality of cores;
saving the thread queues to a region of the memory,
wherein the region of the memory is shared by the application and the user space file system;
receiving, by a process coordinator, a file request specifying a core identifier corresponding to the core;
identifying the thread queue associated with the core identifier;
selecting a first thread handle of the plurality of thread handles ordered within the thread queue; and
directing the file request to be executed on a thread associated with the first thread handle,
wherein the file request is subsequently executed on the thread associated with the first thread handle.