Method to organize file storage based on semantic search history
A storage device organizes data on different storage media based on access history and user behavior to accelerate data retrieval. The storage device includes a first storage media and a second storage media to store the data. The second storage media may be accessed faster than the first storage media. A controller on the storage device executes a first data retrieval to identify hot blocks in the first storage media based on the access history. The controller also executes a second data retrieval to identify the hot blocks in the first storage media based on a semantic search. The controller may prefetch the hot blocks from the first storage media, cache the prefetched hot blocks in the second storage media and/or move the hot blocks from a slower storage medium to a faster storage medium.
1 . A storage device to organize data on different storage media to accelerate data retrieval on the storage device, the storage device comprises:
a first storage media to store data in a persistent state, the first storage media being at least a NAND or NOR memory device;
a second storage media to store data, the second storage media being a storage media that is accessed faster than the first storage media and the second storage media being a random-access memory; and
a controller to execute a first data retrieval to identify hot blocks in the first storage media based on an access history, execute a second data retrieval to identify the hot blocks in the first storage media based on a semantic search, prefetch the hot blocks identified using the first data retrieval and the second data retrieval from the first storage media, and at least one of cache prefetched blocks in the second storage media and move data retrieved using the first data retrieval and the second data retrieval from a slower storage medium to a faster storage medium.
2 . The storage device of claim 1 , further comprising a vector database to store vectors, wherein a vector represents one of a specific attribute of data stored on the first storage media and a semantic query.
3 . The storage device of claim 1 , further comprising a search history database to store queries executed with via semantic searches.
4 . The storage device of claim 1 , wherein the first data retrieval includes identifying at least one of frequently accessed blocks and recently accessed blocks to be moved from a slower storage media to a faster storage media.
5 . The storage device of claim 1 , wherein the first data retrieval includes using a file access hint to prefetch data from the first storage media to be cached on the second storage media.
6 . The storage device of claim 1 , wherein the second data retrieval includes using semantic search history information and results from the semantic search to identify the hot blocks in the first storage media.
7 . The storage device of claim 1 , wherein the second data retrieval includes using semantic search history information and results from the semantic search to identify data with a specific attribute in a file.
8 . The storage device of claim 1 , wherein the controller uses at least one of a time stamp, a user identification, and search query as input to execute the first data retrieval and the second data retrieval.
9 . The storage device of claim 1 , wherein as output for the second data retrieval, the controller identifies a requested data and associated relevant data for a semantic query.
10 . The storage device of claim 1 , wherein the controller caches a first amount of data from the first data retrieval and a second amount of data from the second data retrieval into the second storage media and dynamically modifies the first amount of data and the second amount of data.
11 . A method in a storage device for organizing data on different storage media to accelerate data retrieval on the storage device, the storage device comprises a controller to execute the method comprising:
executing a first data retrieval to identify hot blocks in a first storage media based on an access history;
executing a second data retrieval to identify the hot blocks in the first storage media based on a semantic search;
prefetching the hot blocks identified using the first data retrieval and the second data retrieval from the first storage media; and
at least one of caching prefetched hot blocks in a second storage media that is accessed faster than the first storage media and moving data retrieved using the first data retrieval and the second data retrieval from a slower storage medium to a faster storage medium.
12 . The method of claim 11 , further comprising using a search history database to store queries executed via semantic searches.
13 . The method of claim 11 , wherein executing the first data retrieval further comprises at least one of:
identifying at least one of frequently accessed blocks and recently accessed blocks to be moved from a slower storage media to a faster storage media, and
using a file access hint to identify data stored on the first storage media to be cached on the second storage media.
14 . The method of claim 11 , wherein executing the second data retrieval includes using semantic search history information and results from the semantic search to identify the hot blocks in the first storage media.
15 . The method of claim 11 , wherein executing the second data retrieval includes using semantic search history information and results from the semantic search to identify data with a specific attribute in a file.
16 . The method of claim 11 , further comprising using at least one of a time stamp, a user identification, and search query as input to execute the first data retrieval and the second data retrieval.
17 . The method of claim 11 , further comprising identifying requested data and associated relevant data for a semantic query as output for the second data retrieval.
18 . The method of claim 11 , further comprising caching a first amount of data from the first data retrieval and a second amount of data from the second data retrieval into the second storage media and dynamically modifying the first amount of data and the second amount of data.
19 . A method in a storage device for organizing data on different storage media to accelerate data retrieval on the storage device, the storage device comprises a controller to execute the method comprising:
executing a first data retrieval to identify hot blocks in a first storage media based on an access history;
executing a second data retrieval to identify the hot blocks in the first storage media based on a semantic search;
prefetching the hot blocks identified using the first data retrieval and the second data retrieval from the first storage media;
at least one of caching a first amount of data from the first data retrieval and a second amount of data from the second data retrieval in a second storage media that is accessed faster than the first storage media and moving data retrieved using the first data retrieval and the second data retrieval from a slower storage medium to a faster storage medium; and
dynamically adjusting the first amount of data and the second amount of data being cached in the second storage media.
20 . The method of claim 19 , further comprising at least one of caching the second amount of data within a history window and moving the second amount of data within the history window from a slower storage medium to a faster storage medium.