Data management system
Disclosed herein is a data management system for storing a plurality of incoming data streams. The data management system utilizes a high speed storage device in combination with an intelligent FIFO process to reliably store the incoming data streams to a storage device without fragmentation. The data management system further includes one or more external storage devices that can be used for archival purposes.
1. A data management device comprising:
a processor for receiving a plurality of video streams from a plurality of video sources,
wherein the processor packetizes the plurality of video streams into a plurality of data packets;
a first storage device for storing the plurality of data packets,
wherein the first storage device is a solid state drive (SSD)
wherein the processor transfers the plurality of data packets to the first storage device over a high speed data bus; and
a second storage device interfaced with the first storage device using a storage device interface,
wherein the second storage device is a hard disk drive (HDD);
wherein the first storage device transfers a subset of the plurality of data packets to the second storage device using a First-In-First-Out (FIFO) process continuously, and
wherein the subset of the plurality of data packets includes an oldest data packet currently located on the first storage device.
2. The data management device according to claim 1 , further comprising;
an external HDD device interfaced with the data management system;
wherein the second storage device continuously transfers an oldest data packet subset currently located on the second storage device to the external HDD device using the FIFO process.
3. The data management device according to claim 2 , wherein the second storage device starts the transfer to the external storage device after the second storage device has reached a predetermined capacity.
4. The data management device according to claim 3 , wherein the predetermined capacity is 90%.
5. The data management device according to claim 1 , wherein the first storage device deletes the subset of the plurality of data packets after the transfer to the second storage device.
6. The data management device according to claim 1 , wherein each of the plurality of video streams contains a higher resolution video and a lower resolution video, and wherein the processer optimizes the subset of the plurality of data packets according to a predetermined rule before the first storage device transfer the subset to the second storage device.
7. The data management device according to claim 6 , wherein the predetermined rule comprises deleting the higher resolution video from the subset so that the higher resolution video is not transferred to the second storage device.
8. The data management device according to claim 1 , wherein the FIFO process writes the subset to the second storage device on continuous sectors to prevent fragmentation on the second storage device.
9. A data management process comprising:
receiving, at a data management device having a processor, a plurality of video streams from a plurality of video sources;
packetizing, using the processor, the plurality of video streams into a plurality of data packets;
storing the plurality of data packets on a first storage device using a high speed data bus;
transferring a subset of the plurality of data packets to a second storage device from the first storage device in a continuous process,
wherein the subset of the plurality of data packets always contains an oldest data packet currently stored on the first storage device,
wherein the continuous process is a First-In-First-Out (FIFO) process, and
wherein the first storage device is a solid-state drive (SSD) and the second storage device is a hard disk drive (HDD).
10. The data management process according to claim 9 , further comprising:
transferring an oldest subset currently located on the second storage device to an external storage device using the FIFO process; and
deleting the oldest subset from the second storage device after the transfer.
11. The data management process according to claim 9 , wherein the processor sends an alert to a user when the external storage device is full,
wherein the alert informs the user that the external storage device is at full capacity and can be swapped.
12. The data management process according to claim 9 , wherein, when the external storage device is full, an oldest subset currently located on the external storage device is deleted whenever a new transfer is received from the second storage device.
13. The data management process according to claim 9 , wherein the second storage device starts the transfer to the external storage device after the second storage device has reached a predetermined capacity.
14. The data management process according to claim 9 , further comprising;
replacing the external storage device with an additional storage device when the external storage device is full; and
cataloging the external storage device.
15. The data management process according to claim 9 , wherein the external storage device is reused as the additional storage device after a predetermined period of time.
16. The data management process according to claim 9 , further comprising;
optimizing the subset according to a predetermined rule before transferring the subset to the second storage device.
17. The data management process according to claim 9 , wherein the first storage device has a higher write speed than the second storage device, and wherein the second storage device has a higher capacity than the first storage device.