Optimized vector data storage
An embodiment includes responsive to receiving a request by a system, determining if the request is a write operation of a vector embedding to a block of a data store of the system. The embodiment includes performing the write operation of the vector embedding to the block of the data store wherein the block is determined according to a rule. The embodiment also includes extracting an identifier of the block and modifying a metadata value of the data store with the identifier and the rule.
1 . A computer-implemented method comprising:
responsive to receiving a request by a system, determining the request is a write operation of a vector embedding to a block of a data store of the system;
performing the write operation of the vector embedding to the block of the data store wherein the block is determined according to a rule, the rule comprising a characteristic of the vector embedding wherein performing the write operation according to the rule reduces computer load and enhances performance of the data store of the system; and
extracting an identifier of the block and modifying a metadata value of the data store with the identifier and the rule.
2 . The computer-implemented method of claim 1 , performing a read operation of the vector embedding from the block of the data store wherein the block is determined according to a comparison between the request and the rule in the metadata value of the data store wherein responsive to a distance between the request and the rule below a threshold, the read operation reads all vector embeddings stored in the block, wherein all the vector embeddings are in accordance with the rule.
3 . The computer-implemented method of claim 1 , wherein performing the write operation comprises creating a new block based on a new rule.
4 . The computer-implemented method of claim 1 , wherein the rule is based on a vector distance between two or more vector embeddings.
5 . The computer-implemented method of claim 1 , wherein the block is determined based on a similarity learning between the rule and the vector embedding.
6 . The computer-implemented method of claim 1 , wherein updating the rule causes a modification of a metadata value.
7 . The computer-implemented method of claim 1 , performing a delete operation of the vector embedding from the block of the data store wherein the block is determined according to a comparison between the request and the rule and the vector embedding is deleted from the metadata value.
8 . A computer program product comprising one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by a processor to cause the processor to perform operations comprising:
responsive to receiving a request by a system, determining the request is a write operation of a vector embedding to a block of a data store of the system;
performing the write operation of the vector embedding to the block of the data store wherein the block is determined according to a rule, the rule comprising a characteristic of the vector embedding wherein performing the write operation according to the rule reduces computer load and enhances performance of the data store of the system; and
extracting an identifier of the block and modifying a metadata value of the data store with the identifier and the rule.
9 . The computer program product of claim 8 , performing a read operation of the vector embedding from the block of the data store wherein the block is determined according to a comparison between the request and the rule in the metadata value of the data store wherein responsive to a distance between the request and the rule below a threshold, the read operation reads all vector embeddings stored in the block, wherein all the vector embeddings are in accordance with the rule.
10 . The computer program product of claim 8 , wherein performing the write operation comprises creating a new block based on a new rule.
11 . The computer program product of claim 8 , wherein the rule is based on a vector distance between two or more vector embeddings.
12 . The computer program product of claim 8 , wherein the block is determined based on a similarity learning between the rule and the vector embedding.
13 . The computer program product of claim 8 , wherein updating the rule causes a modification of a metadata value.
14 . The computer program product of claim 8 , performing a delete operation of the vector embedding from the block of the data store wherein the block is determined according to a comparison between the request and the rule and the vector embedding is deleted from the metadata value.
15 . A computer system comprising a processor and one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable by the processor to cause the processor to perform operations comprising:
responsive to receiving a request by a system, determining by the processor the request is a write operation of a vector embedding to a block of a data store of the system;
performing by the processor the write operation of the vector embedding to the block of the data store wherein the block is determined according to a rule, the rule comprising a characteristic of the vector embedding wherein performing the write operation according to the rule reduces computer load and enhances performance of the data store of the system; and
extracting by the processor an identifier of the block and modifying a metadata value of the data store with the identifier and the rule.
16 . The computer system of claim 15 , performing a read operation of the vector embedding from the block of the data store wherein the block is determined according to a comparison between the request and the rule in the metadata value of the data store wherein responsive to a distance between the request and the rule below a threshold, the read operation reads all vector embeddings stored in the block, wherein all the vector embeddings are in accordance with the rule.
17 . The computer system of claim 15 , wherein performing the write operation comprises creating a new block based on a new rule.
18 . The computer system of claim 15 , wherein the rule is based on a vector distance between two or more vector embeddings.
19 . The computer system of claim 15 , wherein the block is determined based on a similarity learning between the rule and the vector embedding.
20 . The computer system of claim 15 , performing a delete operation of the vector embedding from the block of the data store wherein the block is determined according to a comparison between the request and the rule and the vector embedding is deleted from the metadata value.