Method and system for facilitating atomicity assurance on metadata and data bundled storage

1. A computer-implemented method for storing data and metadata in a data stream, the method comprising:

receiving a first stream of data to be written to a storage device, wherein the first stream includes a plurality of I/O requests associated with data and corresponding metadata;

in response to determining that residual data associated with a preceding I/O request of the first stream exists in a data buffer:

appending, to the residual data, a first portion of data from a current I/O request to obtain a first page of data;

writing a remainder portion of the current I/O request to the data buffer to obtain current residual data; and

writing the first page of data to the storage device at a current physical page address, wherein the current physical page address is retrieved from a data structure based on a logical page address associated with the first page; and

determining whether the current residual data corresponds to an end of the first stream, which comprises:

retrieving, from the data structure, a second physical page address corresponding to a next logical page address associated with a next I/O request;

responsive to determining that the second physical page address is null, setting the second physical page address to an invalid and not null physical page address;

responsive to waiting a predetermined period of time, retrieving the second physical page address from the data structure; and

responsive to determining that the second physical page address has been updated and is null, determining that the second physical page address has been trimmed and that the current residual data corresponds to the end of the first stream.

2. The method of claim 1 , wherein the first portion begins from a first logical page address associated with a current write pointer in the current I/O request, and wherein the method further comprises:

setting the current write pointer to an end of the remainder portion; and

updating, in a data structure, an entry which maps the first logical page address to the current physical page address.

3. The method of claim 1 , wherein in response to determining that residual data associated with the first stream does not exist in the data buffer, the method further comprises:

writing, to the data buffer, a second portion of data from the current I/O request to obtain a second page of data,

wherein the second portion begins from a second logical page address associated with a current write pointer in the current I/O request;

writing the remainder portion of the current I/O request to the data buffer to obtain current residual data;

setting the current write pointer to an end of the remainder portion; and

updating, in a data structure, an entry which maps the second logical page address to the current physical page address.

4. The method of claim 1 , wherein subsequent to writing the remainder portion of the current I/O request to the data buffer, the method further comprises:

sending, to a host computing device from which the first stream of data is received, an acknowledgement that the current I/O request is successful.

5. The method of claim 1 , wherein a beginning of the remainder portion in the current I/O request corresponds to a third logical page address, and wherein the method further comprises:

updating, in a data structure, an entry which maps the third logical page address to a next physical page address sequential to the current physical page address.

6. The method of claim 1 , wherein in response to determining that the current residual data corresponds to the end of the first stream, the method further comprises:

appending zeros to the current residual data to obtain a third page of data;

writing the third page of data to the storage device at a next physical page address sequential to the current physical page address;

updating, in the data structure, an entry which maps a fourth logical page address associated with the current residual data to the next physical page address; and

clearing the current residual data in the data buffer.

7. The method of claim 1 , wherein determining whether the current residual data corresponds to the end of the first stream further comprises:

receiving, by a first microprocessor of the storage device, a command to write the first page of data to the storage device, wherein the first microprocessor retrieves the current physical page address from the data structure;

executing the command by writing the first page to the storage device at the current physical page address;

responsive to determining that the second physical page address has been updated and is not null, determining that the current residual data does not correspond to the end of the first stream; and

responsive to determining that the second physical page address has not been updated, determining that the current residual data corresponds to the end of the first stream.

8. The method of claim 7 , further comprising:

reporting whether the residual data corresponds to the end of the first stream.

9. A computer system for storing data and metadata in a data stream, the system comprising:

a processor; and

a memory coupled to the processor and storing instructions, which when executed by the processor cause the processor to perform a method, wherein the computer system comprises a storage device, the method comprising:

receiving a first stream of data to be written to the storage device, wherein the first stream includes a plurality of I/O requests associated with data and corresponding metadata;

in response to determining that residual data associated with a preceding I/O request of the first stream exists in a data buffer:

appending, to the residual data, a first portion of data from a current I/O request to obtain a first page of data;

writing a remainder portion of the current I/O request to the data buffer to obtain current residual data; and

writing the first page of data to the storage device at a current physical page address, wherein the current physical page address is retrieved from a data structure based on a logical page address associated with the first page; and

determining whether the current residual data corresponds to an end of the first stream, which comprises:

retrieving, from the data structure, a second physical page address corresponding to a next logical page address associated with a next I/O request;

responsive to determining that the second physical page address is null, setting the second physical page address to an invalid and not null physical page address;

responsive to waiting a predetermined period of time, retrieving the second physical page address from the data structure; and

responsive to determining that the second physical page address has been updated and is null, determining that the second physical page address has been trimmed and that the current residual data corresponds to the end of the first stream.

10. The computer system of claim 9 , wherein the first portion begins from a first logical page address associated with a current write pointer in the current I/O request, and wherein the method further comprises:

setting the current write pointer to an end of the remainder portion; and

updating, in a data structure, an entry which maps the first logical page address to the current physical page address.

11. The computer system of claim 9 , wherein in response to determining that residual data associated with the first stream does not exist in the data buffer, the method further comprises:

writing, to the data buffer, a second portion of data from the current I/O request to obtain a second page of data,

wherein the second portion begins from a second logical page address associated with a current write pointer in the current I/O request;

writing the remainder portion of the current I/O request to the data buffer to obtain current residual data;

setting the current write pointer to an end of the remainder portion; and

updating, in a data structure, an entry which maps the second logical page address to the current physical page address.

12. The computer system of claim 9 , wherein subsequent to writing the remainder portion of the current I/O request to the data buffer, the method further comprises:

sending, to a host computing device from which the first stream of data is received, an acknowledgement that the current I/O request is successful.

13. The computer system of claim 9 , wherein a beginning of the remainder portion in the current I/O request corresponds to a third logical page address, and wherein the method further comprises:

updating, in a data structure, an entry which maps the third logical page address to a next physical page address sequential to the current physical page address.

14. The computer system of claim 9 , wherein in response to determining that the current residual data corresponds to the end of the first stream, the method further comprises:

appending zeros to the current residual data to obtain a third page of data;

writing the third page of data to the storage device at a next physical page address sequential to the current physical page address;

updating, in the data structure, an entry which maps a fourth logical page address associated with the current residual data to the next physical page address; and

clearing the current residual data in the data buffer.

15. The computer system of claim 9 , wherein determining whether the current residual data corresponds to the end of the first stream further comprises:

receiving, by a first microprocessor of the storage device, a command to write the first page of data to the storage device, wherein the first microprocessor retrieves the current physical page address from the data structure;

executing the command by writing the first page to the storage device at the current physical page address;

responsive to determining that the second physical page address has been updated and is not null, determining that the current residual data does not correspond to the end of the first stream; and

responsive to determining that the second physical page address has not been updated, determining that the current residual data corresponds to the end of the first stream.

16. The computer system of claim 15 , further comprising:

reporting whether the residual data corresponds to the end of the first stream.

17. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:

receiving a first stream of data to be written to a storage device, wherein the first stream includes a plurality of I/O requests associated with data and corresponding metadata;

in response to determining that residual data associated with a preceding I/O request of the first stream exists in a data buffer:

appending, to the residual data, a first portion of data from a current I/O request to obtain a first page of data;

writing a remainder portion of the current I/O request to the data buffer to obtain current residual data; and

writing the first page of data to the storage device at a current physical page address, wherein the current physical page address is retrieved from a data structure based on a logical page address associated with the first page; and

determining whether the current residual data corresponds to an end of the first stream, which comprises:

retrieving, from the data structure, a second physical page address corresponding to a next logical page address associated with a next I/O request;

responsive to determining that the second physical page address is null, setting the second physical page address to an invalid and not null physical page address;

responsive to waiting a predetermined period of time, retrieving the second physical page address from the data structure; and

responsive to determining that the second physical page address has been updated and is null, determining that the second physical page address has been trimmed and that the current residual data corresponds to the end of the first stream.

18. The storage medium of claim 17 , wherein in response to determining that residual data associated with the first stream does not exist in the data buffer, the method further comprises:

writing, to the data buffer, a second portion of data from the current I/O request to obtain a second page of data,

wherein the second portion begins from a second logical page address associated with a current write pointer in the current I/O request;

writing the remainder portion of the current I/O request to the data buffer to obtain current residual data;

setting the current write pointer to an end of the remainder portion; and

updating, in a data structure, an entry which maps the second logical page address to the current physical page address.

19. The storage medium of claim 17 , wherein in response to determining that the current residual data corresponds to the end of the first stream, the method further comprises:

appending zeros to the current residual data to obtain a third page of data;

writing the third page of data to the storage device at a next physical page address sequential to the current physical page address;

updating, in the data structure, an entry which maps a fourth logical page address associated with the current residual data to the next physical page address; and

clearing the current residual data in the data buffer.

20. The storage medium of claim 17 , wherein determining whether the current residual data corresponds to the end of the first stream further comprises:

receiving, by a first microprocessor of the storage device, a command to write the first page of data to the storage device, wherein the first microprocessor retrieves the current physical page address from the data structure;

executing the command by writing the first page to the storage device at the current physical page address;

responsive to determining that the second physical page address has been updated and is not null, determining that the current residual data does not correspond to the end of the first stream; and

responsive to determining that the second physical page address has not been updated, determining that the current residual data corresponds to the end of the first stream.