System and method using metadata to directly read target data in storage nodes
A data storage system, an intelligent network interface card, and a compute node are provided, and belong to the field of data storage. The data storage system includes a compute node and a plurality of storage nodes. The compute node stores metadata, and the metadata may indicate an address at which target data is stored in the plurality of storage nodes. According to this application, the compute node may directly read the target data from a corresponding storage node based on the address indicated by the metadata stored in the compute node, does not need to perform cross-node data forwarding by using the plurality of storage nodes, and does not need to perform a plurality of times of metadata searching. This effectively reduces a data reading delay and improves a data reading speed.
1 . A data storage system comprising:
a compute circuit configured to obtain a write request comprising a logical address and to-be-written data, determine fingerprint information of the to-be-written data, obtain metadata, and read target data in a storage node based on the metadata comprising a fingerprint information set,
wherein the fingerprint information set comprises fingerprint information of deduplication data stored in the plurality of storage nodes,
wherein the storage node is among a plurality of storage nodes configured to persistently store the target data, and
wherein the metadata indicates an address at which the target data is stored in the plurality of storage nodes;
a compute node configured to store the logical address and the fingerprint information of the to-be-written data,
wherein the compute node comprises an intelligent network interface card,
wherein the intelligent network interface card comprises a storage circuit, a compute circuit and an input/output (I/O) circuit,
wherein the storage circuit is configured to store the metadata, and
the compute circuit is further configured to, based on finding the fingerprint information of the to-be-written data is in the fingerprint information set,
store the logical address and the fingerprint information of the to-be-written data in the storage circuit,
delete the to-be-written data, and
control the I/O circuit to send the fingerprint information of the to-be-written data and the logical address to the storage node.
2 . The data storage system according to claim 1 , wherein the logical address is a second logical address, and wherein the metadata comprises a correspondence between a first logical address and physical address information, and the compute circuit is further configured to:
obtain a read request comprising the first logical address;
search the correspondence for physical address information corresponding to the first logical address; and
read the target data in the storage node based on the found first physical address information.
3 . The data storage system according to claim 2 , wherein the intelligent network interface card further comprises an input/output (I/O) circuit;
the compute circuit is further configured to, based on the first physical address information not being found, control the I/O circuit to send the read request to the storage node;
the storage node is configured to obtain the first physical address information, obtain the target data in the storage node based on the first physical address information, and send the obtained target data and the physical address information to the I/O circuit; and
the I/O circuit is configured to store the first logical address and the first physical address information in the storage circuit.
4 . The data storage system according to claim 2 , wherein the physical address information comprises the fingerprint information of deduplication data or a physical address of snapshot data.
5 . The data storage system according to claim 1 , wherein
the compute circuit is further configured to, based on not finding the fingerprint information of the to-be-written data in the fingerprint information set, control the I/O circuit to send the write request to the storage node; and
the storage node is configured to store the to-be-written data based on the write request.
6 . The data storage system according to claim 5 , wherein the compute circuit is further configured to:
after the storage node stores the to-be-written data, search the correspondence for physical address information associated with the logical address; and
based on finding the physical address information associated with the logical address, update the physical address information associated with the logical address to the fingerprint information of the to-be-written data.
7 . The data storage system according to claim 1 , wherein the intelligent network interface card is a data processing unit (DPU).
8 . An intelligent network interface card that stores metadata indicating an address at which target data is stored in a plurality of storage nodes, the intelligent network interface card comprising: and
an input/output (I/O) circuit, a storage circuit configured to store the metadata, and a compute circuit configured to obtain the metadata, and read the target data in the storage node based on the metadata,
wherein the intelligent network interface card is configured to obtain the metadata, and read the target data in the storage node based on the metadata,
wherein the metadata further comprises a fingerprint information set and
wherein the fingerprint information set comprises fingerprint information of deduplication data stored in the plurality of storage nodes; and
the compute circuit is further configured to obtain a write request comprising a logical address and to-be-written data, determine fingerprint information of the to-be-written data and, in response to determining the fingerprint information of the to-be-written data is in the fingerprint information set,
i. store in the storage circuit the logical address and the fingerprint information of the to-be-written data,
ii. delete the to-be-written data, and
iii. control the I/O circuit to send to the storage node the fingerprint information of the to-be-written data and the logical address, so that the storage node stores the logical address and the fingerprint information of the to-be-written data.
9 . The intelligent network interface card according to claim 8 , wherein the logical address is a second logical address, and wherein the metadata comprises a correspondence between a first logical address and physical address information, and the compute circuit is configured to:
obtain a read request comprising the first logical address;
search the correspondence for first physical address information associated with the first logical address; and
read the target data in the storage node based on the found first physical address information.
10 . The intelligent network interface card according to claim 9 , further comprising an input/output (I/O) circuit;
wherein the compute circuit is further configured to, based on not finding the first physical address information, control the I/O circuit to send the read request to the storage node; and
the I/O circuit is configured to receive the target data and the first physical address information obtained by the storage node based on the read request, and store the first logical address and the first physical address information in the storage circuit.
11 . The intelligent network interface card according to claim 10 , wherein the physical address information comprises fingerprint information of deduplication data or a physical address of snapshot data.
12 . The intelligent network interface card according to claim 9 , wherein the physical address information comprises the fingerprint information of deduplication data or a physical address of snapshot data.
13 . The intelligent network interface card according to claim 8 , wherein
the compute circuit is further configured to: based on the fingerprint information of the to-be-written data not being found in the fingerprint information set, control the I/O circuit to send the write request to the storage node, so that the storage node stores the to-be-written data based on the write request.
14 . The intelligent network interface card according to claim 13 , wherein
the compute circuit is further configured to: after the storage node stores the to-be-written data, search the correspondence for physical address information associated with the logical address; and based on the physical address information associated with the logical address being found, update the physical address information associated with the logical address to the fingerprint information of the to-be-written data.
15 . The intelligent network interface card according to claim 8 , wherein the intelligent network interface card is a data processing unit (DPU).
16 . An intelligent network interface card comprising an input/output (I/O) circuit, a storage circuit configured to store metadata and a compute circuit configured to obtain the metadata, and read target data in a storage node based on the metadata,
wherein the metadata indicates an address at which the target data is stored in a plurality of storage nodes,
wherein information in the metadata includes a correspondence between a logical address and physical address information, and
wherein the metadata further comprises a fingerprint information set which comprises fingerprint information of deduplication data stored in the plurality of storage nodes; and
the compute circuit is configured to:
obtain a read request comprising a first logical address:
search the correspondence for physical address information associated with the first logical address;
obtain a write request comprising a second logical address and to-be-written data;
determine fingerprint information of the to-be-written data; and
based on the fingerprint information of the to-be-written data being found in the fingerprint information set, store the second logical address and the fingerprint information of the to-be-written data in the storage circuit, delete the to-be-written data, and control the I/O circuit to send the fingerprint information of the to-be-written data and the second logical address to the storage node, so that the storage node stores the second logical address and the fingerprint information of the to-be-written data.
17 . The intelligent network interface card according to claim 16 , further comprising an input/output (I/O) circuit;
the compute circuit is further configured to, based on not finding the physical address information, control the I/O circuit to send the read request to the storage node; and
the I/O circuit is configured to receive the target data and the physical address information obtained by the storage node based on the read request, and store the logical address and the physical address information in the storage circuit.
18 . The intelligent network interface card according to claim 16 , wherein the physical address information comprises fingerprint information of deduplication data or a physical address of snapshot data.
19 . The intelligent network interface card according to claim 16 , wherein the compute circuit is further configured to:
based on the fingerprint information of the to-be-written data not being found in the fingerprint information set, control the I/O circuit to send the write request to the storage node, so that the storage node stores the to-be-written data based on the write request.
20 . The intelligent network interface card according to claim 16 , wherein the intelligent network interface card is a data processing unit (DPU).