Method and system for efficient data transmission with server side de-duplication

1. A method comprising:

encoding a first data block having a first length into a bitstream having a second length;

transmitting the bitstream to a server; and

reducing redundant data blocks by decoding the first data block from a first plurality of data blocks and the bitstream where each block in the first plurality of data blocks has a length equal to the first length.

2. The method of claim 1 , wherein the decoding is performed with a Slepian-Wolf decoder.

3. The method of claim 1 , wherein upon decoding being successful for the complete first length, de-duplication is performed on the first data block.

4. The method of claim 1 , wherein upon the decoding being unsuccessful for the complete first length, requesting further information of the first data block from a client.

5. The method of claim 4 , further comprising:

encoding the first data block having the first length into another bitstream having one of the second length and a third length;

transmitting the other bitstream to the server; and

reducing redundant data blocks by decoding the first data block from a second plurality of data blocks and the other bitstream, where each block in the second plurality of data blocks has a length equal to the first length.

6. The method of claim 1 , wherein de-duplication is performed by the decoding.

7. The method of claim 1 , wherein the decoding is performed with a variable length for a predetermined collision rate.

8. The method of claim 1 , wherein a transmission rate for the transmitting is variable to meet a predetermined collision rate.

9. A system for reducing redundant data blocks, comprising:

a client device including an encoder module configured to encode a data block into a bitstream;

a server device including a decoder module configured to decode the bitstream using a plurality of previously stored data blocks on the server device; and

a de-duplication module coupled to the decoder module configured to deduplicate successful decoded portions of the data block.

10. The system of claim 9 , further comprising a data storage device coupled to the server device.

11. The system of claim 9 , further comprising at least another client device.

12. The system of claim 9 , wherein the encoder module performs a Slepian-Wolf encoding.

13. The system of claim 9 , wherein the decoder module performs a Slepian-Wolf decoding.

14. A computer program product comprising a computer usable medium including a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:

encode a first data block having a first length into a bitstream having a second length;

transmit the bitstream to a server; and

reduce redundant data blocks by decoding the first data block from a first plurality of data blocks and the bitstream where each block in the first plurality of data blocks has a length equal to the first length.

15. The computer program product of claim 14 , wherein the decoding is performed with a Slepian-Wolf decoder.

16. The computer program product of claim 14 , wherein upon decoding being successful for the complete first length, de-duplication is performed on the first data block.

17. The computer program product of claim 14 , wherein upon decoding being unsuccessful for the complete first length, requesting further information of the first data block from a client.

18. The computer program product of claim 14 , further causing the computer to:

encode the first data block having the first length into another bitstream having one of the second length and a third length;

transmit the other bitstream to the server; and

reduce redundant data blocks by decoding the first data block from a second plurality of data blocks and the other bitstream, where each block in the second plurality of data blocks has a length equal to the first length.

19. The computer program product of claim 14 , wherein a transmission rate for the transmitting is variable to meet a predetermined collision rate.

20. A method comprising:

encoding a first data block having a first length into a bitstream having a second length using a Slepian-Wolf encoding process;

transmitting the bitstream to a server device; and

reducing redundant data blocks before de-duplication using a Slepian-Wolf decoding process by decoding the first data block from a first plurality of data blocks and the bitstream, where each block in the first plurality of data blocks has a length equal to the first length.

21. The method of claim 20 , wherein upon the decoding being unsuccessful for the complete first length, requesting further information of the first data block from a client.

22. The method of claim 21 , further comprising:

encoding the first data block having the first length into another bitstream having one of the second length and a third length;

transmitting the other bitstream to the server device; and

reducing redundant data blocks by decoding the first data block from a second plurality of data blocks and the other bitstream, where each block in the second plurality of data blocks has a length equal to the first length.

23. A system for reducing transmission of redundant data blocks, comprising:

a client device including a Slepian-Wolf encoder module configured to encode a data block into a bitstream;

a server device device including a Slepian-Wolf decoder module configured to decode the bitstream using a plurality of previously stored data blocks on the server device; and

a de-duplication module coupled to the decoder module configured to deduplicate successful decoded portions of the data block.

24. The system of claim 23 , wherein the Slepian-Wolf decoder is configured to reduce redundant data blocks before de-duplication.