Format identification for fragmented image data
View Patent ↗Format identification for fragmented data is disclosed. In some embodiments, an input stream of information that includes a continuity property is received. A format identifier of at least a portion of the stream is determined, wherein the format identifier includes a data representation size, a group size, and an alignment that is consistent with the continuity property. The stream of information is compressed using a compression technique selected based on the format identifier to produce a compressed stream, and the compressed stream is stored.
1. A system for storing information, comprising:
an interface that receives an uncompressed input stream of information;
a processor configured to:
determine that at least a portion of the uncompressed input stream satisfies a continuity property by determining a format identifier that at least the portion of the uncompressed input stream satisfies, wherein the determined format identifier comprises a tuple including a data representation size corresponding to a number of bytes used to store values, a group size corresponding to a number of channels, and a number of bytes for alignment; and
compress the at least portion of the uncompressed input stream of information using a compression technique selected based on the determined continuity property and format identifier to produce a compressed stream; and
a memory that stores the compressed stream.
2. The system of claim 1 , wherein the compression technique takes advantage of the continuity property.
3. The system of claim 1 , wherein the input stream of information is fragmented.
4. The system of claim 1 , wherein the processor is further configured to de-duplicate the input stream of information.
5. The system of claim 1 , wherein the format identifier indicates whether the input stream of information corresponds to an image.
6. The system of claim 1 , wherein the uncompressed input stream of information includes segments and the processor is further configured to identify breakpoints between segments.
7. The system of claim 1 , wherein the uncompressed input stream of information includes segments and the processor is further configured to identify breakpoints between segments and assign different format identifiers to each segment.
8. The system of claim 1 , wherein the uncompressed input stream of information includes segments and the processor is further configured to identify breakpoints between segments having the same format identifier based on statistical characteristics of data in the segments.
9. The system of claim 1 , wherein the group size corresponds to a number of color values for a pixel.
10. The system of claim 1 , wherein the processor is further configured to determine a fitting penalty based on a difference between values for a candidate format identifier.
11. The system of claim 1 , wherein the processor is further configured to assign a fitting penalty to each of a plurality of candidate format identifiers for a portion of the input stream of information.
12. The system of claim 1 , wherein the processor is further configured to assign a fitting penalty to each of a plurality of candidate format identifiers for a portion of the input stream of information and wherein the processor penalizes a change in format identifier.
13. A method for storing information, comprising:
receiving an uncompressed input stream of information;
determining that at least a portion of the uncompressed input stream satisfies a continuity property by determining a format identifier that at least the portion of the uncompressed input stream satisfies, wherein the determined format identifier comprises a tuple including a data representation size corresponding to a number of bytes used to store values, a group size corresponding to a number of channels, and a number of bytes for alignment;
compressing the at least portion of the uncompressed input stream of information using a compression technique selected based on the determined continuity property and format identifier to produce a compressed stream; and
storing the compressed stream.
14. The method of claim 13 , wherein the compression technique takes advantage of the continuity property.
15. The method of claim 13 , wherein the input stream of information is fragmented.
16. The method of claim 13 , wherein the format identifier indicates whether the input stream of information corresponds to an image.
17. The method of claim 13 , wherein the input stream of information includes segments and further comprising identifying breakpoints between segments.
18. The method of claim 13 , further comprising determining a fitting penalty based on a difference between values for a candidate format identifier.
19. The method of claim 13 , further comprising de-duplicating the input stream of information.
20. The method of claim 13 , wherein the uncompressed input stream of information includes segments and further comprising identifying breakpoints between segments and assigning different format identifiers to each segment.
21. The method of claim 13 , wherein the uncompressed input stream of information includes segments and further comprising identifying breakpoints between segments having the same format identifier based on statistical characteristics of data in the segments.
22. The method of claim 13 , wherein the group size corresponds to a number of color values for a pixel.
23. The method of claim 13 , further comprising assigning a fitting penalty to each of a plurality of candidate format identifiers for a portion of the input stream of information.
24. The method of claim 13 , further comprising assigning a fitting penalty to each of a plurality of candidate format identifiers for a portion of the input stream of information and penalizing a change in format identifier.
25. A computer program product embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving an uncompressed input stream of information;
determining that at least a portion of the uncompressed input stream satisfies a continuity property by determining a format identifier that at least the portion of the uncompressed input stream satisfies, wherein the determined format identifier comprises a tuple including a data representation size corresponding to a number of bytes used to store values, a group size corresponding to a number of channels, and a number of bytes for alignment;
compressing the at least portion of the uncompressed input stream of information using a compression technique selected based on the determined continuity property and format identifier to produce a compressed stream; and
storing the compressed stream.
26. The computer program product of claim 25 , wherein the compression technique takes advantage of the continuity property.
27. The computer program product of claim 25 , wherein the input stream of information is fragmented.
28. The computer program product of claim 25 , further comprising computer instructions for de-duplicating the input stream of information.
29. The computer program product of claim 25 , wherein the format identifier indicates whether the input stream of information corresponds to an image.
30. The computer program product of claim 25 , wherein the uncompressed input stream of information includes segments and further comprising computer instructions for identifying breakpoints between segments.
31. The computer program product of claim 25 , wherein the uncompressed input stream of information includes segments and further comprising computer instructions for identifying breakpoints between segments and assigning different format identifiers to each segment.
32. The computer program product of claim 25 , wherein the uncompressed input stream of information includes segments and further comprising computer instructions for identifying breakpoints between segments having the same format identifier based on statistical characteristics of data in the segments.
33. The computer program product of claim 25 , wherein the group size corresponds to a number of color values for a pixel.
34. The computer program product of claim 25 , further comprising computer instructions for determining a fitting penalty based on a difference between values for a candidate format identifier.
35. The computer program product of claim 25 , further comprising computer instructions for assigning a fitting penalty to each of a plurality of candidate format identifiers for a portion of the input stream of information.
36. The computer program product of claim 25 , further comprising computer instructions for assigning a fitting penalty to each of a plurality of candidate format identifiers for a portion of the input stream of information and penalizing a change in format identifier.