Performing data reduction during host data ingest
A technique performs data reduction on host data of a write request during ingest under certain circumstances. Therein, raw host data of a write request is placed from the host into a data cache. Further, a data reducing ingest operation is performed that reduces the raw host data from the data cache into reduced host data (e.g., via deduplication, compression, combinations thereof, etc.). After completion of the data reducing ingest operation, a late-binding operation is performed that updates a mapper with ability to access the reduced host data from secondary storage. Such ingest-time data reduction may be enabled/disabled (e.g., turned on or off) per input/output (I/O) operation (e.g., used only for relatively large asynchronous I/O operations) and/or activated in situations in which the ingest bandwidth is becoming a bottleneck.
1. Within data storage equipment, a method of processing a set of write requests from a host, the method comprising:
placing raw host data of the set of write requests from the host into a data cache;
performing a data reducing ingest operation that reduces the raw host data from the data cache into reduced host data; and
after completion of the data reducing ingest operation, performing a late-binding operation that updates a mapper with ability to access the reduced host data from secondary storage;
wherein the raw host data includes first host data from a first host write operation and second host data from a second host write operation; and
wherein performing the data reducing ingest operation includes:
reducing the first host data into first reduced data based on a first latency indication of the first host write operation indicating that the first host write operation is not latency sensitive, and storing the first reduced data in a logger of the data storage equipment, and
not reducing the second host data based on a second latency indication of the second host write operation indicating that the second host write operation is latency sensitive, and storing the second host data that is not reduced based on the second latency indication in the logger of the data storage equipment.
2. A method as in claim 1 wherein the data storage equipment includes (i) storage processing circuitry having the data cache, (ii) a first storage tier of non-volatile random access memory (NVRAM), at least a portion of which forms the logger, and (iii) a second storage tier of solid state drive (SSD) memory; and
wherein performing the data reducing ingest operation includes:
storing the reduced host data in at least one of the first storage tier and the second storage tier and acknowledging completion of the write request to the host.
3. A method as in claim 2 wherein performing the late-binding operation includes:
transitioning host data access control to the mapper to enable the mapper to access the reduced host data from the second storage tier in response to a subsequent input/output (I/O) request.
4. A method as in claim 3 wherein performing the data reducing ingest operation further includes:
prior to storing the reduced host data, performing a set of data reduction operations on the raw host data to generate the reduced host data, the reduced host data consuming less storage space than the raw host data.
5. A method as in claim 4 wherein performing the set of data reduction operations includes:
performing a set of deduplication operations on the raw host data to generate the reduced host data from the raw host data.
6. A method as in claim 4 wherein performing the set of data reduction operations includes:
performing a set of compression operations on the raw host data to generate the reduced host data from the raw host data.
7. A method as in claim 4 wherein performing the set of data reduction operations includes:
performing a set of deduplication operations on the raw host data to generate partially-reduced host data from the raw host data, and
performing a set of compression operations on the partially-reduced host data to generate the reduced host data from the partially-reduced host data.
8. A method as in claim 3 wherein storing the reduced host data includes:
writing the reduced host data from the data cache directly into the first storage tier of NVRAM when a current ingest bandwidth provided by the data storage equipment is below a predefined bandwidth threshold; and
writing the reduced host data from the data cache directly into the second storage tier of SSD memory to circumvent the first storage tier when the current ingest bandwidth provided by the data storage equipment is above the predefined bandwidth threshold.
9. A method as in claim 3 wherein storing the reduced host data includes:
writing the reduced host data from the data cache directly into the first storage tier of NVRAM when a size of the raw host data of the write request is less than a predefined size threshold; and
writing the reduced host data from the data cache directly into the second storage tier of SSD memory to circumvent the first storage tier when a size of the raw host data of the write request exceeds the predefined size threshold.
10. A method as in claim 3 wherein storing the reduced host data includes:
writing the reduced host data from the data cache directly into the first storage tier of NVRAM when the write request identifies a synchronous host write operation; and
writing the reduced host data from the data cache directly into the second storage tier of SSD memory to circumvent the first storage tier when the write request identifies an asynchronous host write operation.
11. A method as in claim 3 wherein the reduced host data includes a set of compressed blocks generated by the data reducing ingest operation; and
wherein performing the late-binding operation further includes:
updating a mapping tree of the mapper to point to a set of locations within the second storage tier that contains the set of compressed blocks.
12. A method as in claim 3 wherein the reduced host data includes provisional deduplication metadata generated by the data reducing ingest operation, the provisional deduplication metadata identifying a set of blocks of raw host data that has been deduplicated; and
wherein performing the late-binding operation further includes:
updating mapper deduplication metadata based on the provisional deduplication metadata to enable the mapper to access the set of blocks of raw host data that has been deduplicated.
13. A method as in claim 3 wherein performing the late-binding operation further includes:
flushing a set of pages of the reduced host data from the first storage tier into the second storage tier.
14. A method as in claim 3 wherein performing the data reducing ingest operation includes:
storing the reduced host data in the first storage tier, the reduced host data being subsequently flushed from the first storage tier into the second storage tier during the late-binding operation; and
wherein the method further comprises:
placing other raw host data of another write request from the host into the data cache;
performing another data reducing ingest operation that reduces the other raw host data from the data cache into other reduced host data, the other reduced host data being stored directly into the second storage tier circumventing the first storage tier; and
after completion of the other data reducing ingest operation, performing another late-binding operation that updates the mapper with ability to access the other reduced host data from the second storage tier.
15. A method as in claim 3 wherein performing the data reducing ingest operation includes:
storing the reduced host data directly into the second storage tier circumventing the first storage tier; and
wherein the method further comprises:
placing other raw host data of another write request from the host into the data cache;
performing another data reducing ingest operation that reduces the other raw host data from the data cache into other reduced host data, the other reduced host data being stored in the first storage tier; and
after completion of the other data reducing ingest operation, performing another late-binding operation that subsequently flushes the other reduced host data from the first storage tier into the second storage tier and updates the mapper with ability to access the other reduced host data from the second storage tier.
16. A method as in claim 3 , further comprising:
placing other raw host data of another write request from the host into the data cache;
transferring the other raw host data from the data cache into the first storage tier and acknowledging completion of the other write request to the host; and
performing a flushing operation that (i) reduces the other raw host data from the first storage tier into other reduced host data using a set of data reduction operations, (ii) stores the other reduced host data in the second storage tier, and (iii) updates the mapper with ability to access the other reduced host data from the second storage tier.
17. A method as in claim 1 wherein, after completion of the data reducing ingest operation, performing the late-binding operation includes:
writing the reduced host data from the data cache directly into the secondary storage circumventing a storage tier of non-volatile random access memory (NVRAM).
18. A method as in claim 1 , further comprising:
prior to performing the data reducing ingest operation, performing a set of latency indication detection operations that detect the first latency indication of the first host write operation and the second latency indication of the second host write operation.
19. Data storage equipment, comprising:
memory; and
control circuitry coupled to the memory, the memory storing instructions which, when carried out by the control circuitry, cause the control circuitry to perform a method of:
placing raw host data of the set of write requests from the host into a data cache,
performing a data reducing ingest operation that reduces the raw host data from the data cache into reduced host data, and
after completion of the data reducing ingest operation, performing a late-binding operation that updates a mapper with ability to access the reduced host data from secondary storage:
wherein the raw host data includes first host data from a first host write operation and second host data from a second host write operation; and
wherein performing the data reducing ingest operation includes:
reducing the first host data into first reduced data based on a first latency indication of the first host write operation indicating that the first host write operation is not latency sensitive, and storing the first reduced data in a logger of the data storage equipment, and
not reducing the second host data based on a second latency indication of the second host write operation indicating that the second host write operation is latency sensitive, and storing the second host data that is not reduced based on the second latency indication in the logger of the data storage equipment.
20. A computer program product having a non-transitory computer readable medium which stores a set of instructions to process a write request from a host; the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of:
placing raw host data of the set of write requests from the host into a data cache;
performing a data reducing ingest operation that reduces the raw host data from the data cache into reduced host data; and
after completion of the data reducing ingest operation, performing a late-binding operation that updates a mapper with ability to access the reduced host data from secondary storage;
wherein, after completion of the data reducing ingest operation, performing the late-binding operation includes:
writing the reduced host data from the data cache directly into the secondary storage circumventing a storage tier of non-volatile random access memory (NVRAM).