Method and System for Data Storage and Management
Exemplary memory storage system and methods for distributive storage of data. Exemplary embodiments provide methods and systems including a plurality of nodes where each node has memory for storing data. The nodes may be configured to receive data and store the data at the node if the data is intended for the node or pass the data to another node if the data is not intended for the node. The nodes may manage memory and allocation of specific memory addresses locally, while the system of nodes manages memory based on a naming convention to indicate the nodes and not the individual memory addresses within a node.
1 . A distributive memory storage system, comprising:
a data generating source configured to generate a data streams;
a plurality of nodes in which each of the nodes of the plurality of nodes includes a processor and memory, and is configured to receive the generated data stream.
2 . The distributive memory storage system of claim 1 , wherein the system is configured to fragment the data stream, and store different fragments on different nodes of the plurality of nodes.
3 . The distributive memory storage system of claim 2 , wherein the system is configured to associate metadata to a fragment of the data stream providing instructions on how to recreate the data stream from the fragment.
4 . The distributive memory storage system of claim 3 , wherein the metadata indicates a node to store the fragment but not a specific address of a memory location being used for the fragment.
5 . The distributed memory storage system of claim 4 , wherein a node is configured to receive the fragment and store the fragment at the memory of the node if the metadata indicates the node or pass the fragment to another node if the metadata indicates the fragment is not for the node.
6 . The distributed memory storage system of claim 2 , wherein the system is configured to save the fragments in file formats in which a memory location corresponding to a fragment is unknown to other nodes of the plurality of nodes except the node in which the fragment is stored.
7 . The distributed memory storage system of claim 2 , wherein a mapping of specific memory locations is managed locally on a local node providing the memory for a specific fragment of the data stream.
8 . The distributive memory storage system of claim 1 , wherein the system does not include a centralized data storage location.
9 . The distributive memory storage system of claim 1 , wherein the plurality of nodes are configured to store portions of the data stream to define a decentralized storage network.
10 . The distributive memory storage system of claim 1 , further comprising a memory controller associated with each node of the plurality of nodes, wherein the memory controller is configured with a naming convention that identifies available memory for each of the plurality of nodes without knowing a specific memory location within the node.
11 . The distributive memory storage system of claim 10 , wherein the memory controller is configured to have access to the memory and specific memory locations available to the node associated with the memory controller.
12 . A method of storing data in a distributive memory storage system, comprising:
receiving a data stream;
providing a plurality of nodes in which each of the nodes of the plurality of nodes includes a processor and memory, and is configured to receive data; and
storing at least a portion of the data stream at a node of the plurality of nodes.
13 . The method of claim 12 , further comprising fragmenting the data stream; and storing different fragments on different nodes of the plurality of nodes.
14 . The method of claim 13 , further comprising associating metadata to a fragment of the data stream providing instructions on how to recreate the data stream from the fragment.
15 . The method of claim 14 , wherein the metadata indicates a node to store the fragment but not a specific address of a memory location being used for the fragment.
16 . The method of claim 4 , further comprising receiving a first fragment of the data stream at a first node; determining at the first node whether the metadata of the first fragment indicates the first node as a storage location of the first fragment, and storing the fragment at the memory of the node if the metadata indicates the first node or passing the fragment to another node if the metadata does not indicate the first node.