System and method for video error concealment
View Patent ↗The present invention provides, in one embodiment, a system and method for concealing video errors. The system encodes, reorders, and packetizes video information into video data packets for transmission over a communication network such that the system conceals errors caused by lost video data packets when the system receives, depacketizes, orders, and decodes the data packets. In one embodiment, the system and method encodes and packetizes video information, such that adjacent macroblocks are not placed in the same video data packets. Additionally, the system and method may provide information accompanying the video data packets to facilitate the decoding process. An advantage to such a scheme is that errors due to video data packet loss are spatially distributed over a video frame. Thus, if regions of data surrounding a lost macroblock are successfully decoded, the decoder may predict motion vectors and spatial content with a higher degree of accuracy, which leads to higher video quality.
1. A system for processing video data, comprising:
a coding engine for processing each frame of a video signal to generate macroblocks and to encode the macroblocks;
a macroblock reordering engine for assigning a macroblock group identifier (MBGID) from a plurality of MBGIDs to each encoded macroblock; and
a macroblock packetization engine for placing each of the encoded macroblocks into a particular data packet according to the MBGID, such that each packet contains only macroblocks having a common MBGID.
2. The system of claim 1 , wherein the macroblock reordering engine assigns a different MBGID to adjacent encoded macroblocks.
3. The system of claim 1 , wherein the coding engine generates non-differentially encoded motion vectors for each of the encoded macroblocks which is an inter-coded macroblock.
4. The system of claim 1 , wherein the macroblock packetization engine places the encoded macroblocks with different assigned MBGIDs into different data packets.
5. The system of claim 1 , wherein the MBGID has a value from 1 to a maximum group identifier (MGID).
6. The system of claim 5 , wherein the macroblock reordering engine determines the MGID based on video data rates.
7. The system of claim 5 , wherein the macroblock reordering engine determines the MGID based on video format.
8. The system of claim 1 , wherein the coding engine encodes a given macroblock of a current frame as an intra-coded macroblock if significant differences exist between data of the given macroblock of the current frame and data of a closest matching offset macroblock of a previous frame.
9. The system of claim 1 , wherein the coding engine encodes a given macroblock of a current frame as an inter-coded macroblock if significant similarities exist between data of the given macroblock of the current frame and data of a closest matching offset macroblock of a previous frame.
10. The system of claim 1 , wherein the coding engine further generates an intra-macroblock map identifying intra-coded macroblocks in a given frame.
11. The system of claim 10 , wherein the coding engine encodes the intra-macroblock map by selecting an intra-macroblock map encoding method that generates a fewest number of bits.
12. The system of claim 10 , wherein the coding engine uses run-length encoding to encode the intra-macroblock map.
13. The system of claim 10 , wherein the coding engine uses a bitmap to encode the intra-macroblock map.
14. The system of claim 1 , wherein the macroblock packetization engine of a receiving device de-packetizes the encoded macroblocks.
15. The system of claim 1 , wherein the macroblock reordering engine of a receiving device orders the encoded macroblocks.
16. The system of claim 1 , wherein the coding engine of a receiving device decodes the encoded macroblocks and detects missing decoded macroblocks.
17. The system of claim 16 , wherein the coding engine of the receiving device spatially interpolates data of a given missing decoded macroblock from data of adjacent decoded macroblocks for concealing effects of video error, if the given missing decoded macroblock was encoded as an intra-coded macroblock.
18. The system of claim 16 , wherein the coding engine of the receiving device estimates a motion vector of a given missing decoded macroblock based on motion vectors of adjacent decoded macroblocks to reconstruct data content of the given missing decoded macroblock via motion compensation for concealing effects of video error, if the given missing decoded macroblock was encoded as an inter-coded macroblock.
19. The system of claim 1 , wherein the macroblock packetization engine is a transport engine for placing each of the encoded macroblocks according to the MBGID into a particular format for transport on a circuit-switched network.
20. A method for processing video data, comprising the steps of:
processing each frame of a video signal to generate macroblocks; encoding the macroblocks;
assigning a macroblock group identifier (MBGID) from a plurality of MBGIDs to each encoded macroblock;
reordering the encoded macroblocks according to the MBGID; and
placing each of the encoded macroblocks into a particular data packet, such that each packet contains only macroblocks having a common MBGID.
21. The method of claim 20 , wherein the step of assigning further comprises the step of assigning adjacent encoded macroblocks a different MBGID.
22. The method of claim 20 , further comprising the step of generating non-differentially encoded motion vectors for each of the encoded macroblocks which is an inter-coded macroblock.
23. The method of claim 20 , wherein the step of placing further comprises the step of placing the encoded macroblocks with different assigned MBGIDs into different data packets.
24. The method of claim 20 , wherein the MBGID has a value from 1 to a maximum group identifier (MGID).
25. The method of claim 23 , further comprising the step of determining the MGID based on video data rates.
26. The method of claim 23 , further comprising the step of determining the MGID based on video format.
27. The method of claim 20 , wherein the step of encoding further comprises the step of encoding a given macroblock of a current frame as an intra-coded macroblock if significant differences exist between data of the given macroblock of the current frame and data of a closest matching offset macroblock of a previous frame.
28. The method of claim 20 , wherein the step of encoding further comprises the step of encoding a given macroblock of a current frame as an inter-coded macroblock if significant similarities exist between data of the given macroblock of the current frame and data of a closet matching offset macroblock of a previous frame.
29. The method of claim 20 , further comprising the step of generating an intra-macroblock map identifying intra-coded macroblocks in a given frame.
30. The method of claim 29 , further comprising the step of selecting an intra-macroblock map encoding method that generates a fewest number of bits.
31. The method of claim 29 , further comprising the step of using run-length encoding to encode the intra-macroblock map.
32. The method of claim 29 , further comprising the step of using a bitmap to encode the intra-macroblock map.
33. The method of claim 20 , further comprising the step of de-packetizing the encoded macroblocks.
34. The method of claim 20 , further comprising the step of ordering the encoded macroblocks.
35. The method of claim 20 , further comprising the steps of decoding the encoded macroblocks and detecting missing decoded macroblocks.
36. The method of claim 35 , further comprising the step of spatially interpolating data of a given missing decoded macroblock from data of adjacent decoded macroblocks for concealing effects of video error, if the given missing decoded macroblock was encoded as an intra-coded macroblock.
37. The method of claim 35 , further comprising the step of estimating a motion vector of a given missing decoded macroblock based on motion vectors of adjacent decoded macroblocks to reconstruct data content of the given missing decoded macroblock via motion compensation for concealing effects of video error, if the given missing decoded macroblock was encoded as an inter-coded macroblock.
38. The method of 20 , wherein the step of placing further comprises placing each of the encoded macroblocks according to the MBGID into a particular format for transport on circuit-switched network.
39. A computer-readable medium encoded with a computer program, when executed by a computer, to perform a method for processing video data, the method comprising:
processing each frame of a video signal to generate macroblocks;
encoding the macroblocks;
assigning a macroblock group identifier (MBGID) from a plurality of MBGIDs to each encoded macroblock;
reordering the encoded macroblocks according to the MBGID; and placing each of the reordered macroblocks into a particular data packet, such that each packet contains only macroblocks having a common MBGID.
40. The computer-readable medium of claim 39 , the method further comprising assigning adjacent encoded macroblocks a different MBGID.
41. The computer readable medium of claim 39 , the method further comprising generating non-differentially encoded motion vectors for each of the encoded macroblocks which is an inter-coded macroblock.
42. The computer readable medium of claim 39 , wherein placing further comprises placing the encoded macroblocks with different assigned MBGIDs into different data packets.
43. The computer readable medium of claim 39 , wherein encoding further comprises encoding a given macroblock of a current frame as an intra-coded macroblock if significant differences exist between data of the given macroblock of the current frame and data of a closest matching offset macroblock of a previous frame.
44. The computer readable medium of claim 39 , wherein encoding further comprises encoding a given macroblock of a current frame as an inter-coded macroblock if significant similarities exist between data of the given macroblock of the current frame and data of a closest matching offset macroblock of a previous frame.
45. The computer readable medium of claim 39 , the method further comprising generating an intra-macroblock map identifying intra-coded macroblocks in a given frame.
46. A method for processing video data, comprising the steps of:
generating macroblocks for each frame of a video signal;
encoding the macroblocks;
assigning a macroblock group identifier (MBGID) from a plurality of MBGIDs to each encoded macroblock;
reordering the encoded macroblocks according to the MBGID;
placing the reordered macroblocks into different data packets, such that each packet contains only macroblocks having a common MBGID;
transmitting the different data packets to a remote device;
decoding the encoded macroblocks of the different data packets received at the remote device;
detecting missing decoded macroblocks; and
reconstructing data of the missing decoded macroblocks.
47. The method of claim 46 , wherein the step of reconstructing data further comprises the step of spatially interpolating data of a given missing decoded macroblock from data of adjacent decoded macroblocks for concealing effects of video error, if the given missing decoded macroblock was encoded as an intra-coded macroblock.
48. The method of claim 46 , wherein the step of reconstructing data further comprises the step of estimating a motion vector of a given missing decoded macroblock based on motion vectors of adjacent decoded macroblocks to reconstruct data content of the given missing decoded macroblock via motion compensation for concealing effects of video error, if the given missing decoded macroblock was encoded as an inter-coded macroblock.
49. A system for processing video data, comprising:
means for processing each frame of a video signal to generate macroblocks;
means for encoding the macroblocks;
means for assigning a macroblock group identifier (MBGID) from a plurality of MBGIDs to each encoded macroblock;
means for reordering the encoded macroblocks according to the MBGID; and
means for placing each of the reordered macroblocks into a particular data packet, such that each packet contains only macroblocks having a common MBGID.
50. A system for processing video data, comprising:
a macroblock de-packetization engine for receiving a plurality of data packets, each packet containing a plurality of macroblocks wherein each of the plurality of macroblocks in a given packet has a common macroblock group identifier (MBGID);
a macroblock reordering engine for reordering the received macroblocks from the plurality of data packets into a predetermined order; and
a decoding engine for processing the reordered macroblocks to generate a video signal.
51. The system of claim 50 , wherein the predetermined order is a raster scan order.
52. The system of claim 50 , wherein the plurality of data packets include non-differentially encoded motion vectors for each of the encoded macroblocks that is an inter-coded macroblock
53. The system of claim 50 , wherein different data packets contain macroblocks with different assigned MBGIDs.
54. The system of claim 1 , wherein the MBGIDs range in value from 1 to a maximum group identifier (MGID).
55. The system of claim 54 , wherein MGID corresponds to a video data rate.
56. The system of claim 54 , wherein the MGID corresponds to a video format.
57. The system of claim 50 , wherein the plurality of data packets includes an intra-macroblock map identifying intra-coded macroblocks in a given frame.
58. The system of claim 57 , wherein the intra-macroblock map is encoded using run-length encoding.
59. The system of claim 57 , wherein the intra-macroblock map is encoded using a bitmap.
60. The system of claim 57 wherein the decoding engine detects whether the intra-macroblock map is encoded using run-length coding or a bitmap and processes the macroblocks accordingly.
61. The system of claim 50 , wherein the decoding engine detects macroblocks missing from the reordered macroblocks.
62. The system of claim 61 , wherein the decoding engine spatially interpolates data of a given missing macroblock from data of adjacent decoded macroblocks, if the given missing macroblock was encoded as an intra-coded macroblock.
63. The system of claim 61 , wherein the decoding engine estimates a motion vector of a given missing macroblock based on motion vectors of adjacent decoded macroblocks to reconstruct data content of the given missing macroblock via motion compensation, if the given missing decoded macroblock was encoded as an inter-coded macroblock.
64. A method for processing video data, comprising the steps of:
receiving a plurality of data packets, each data packet containing a plurality of macroblocks, each macroblock within a particular packet having a common macroblock group identifier (MBGID);
de-packetizing the encoded macroblocks;
reordering the received macroblocks into a predetermined order different from the order in which they were received; and
decoding the reordered macroblocks to produce a video signal.
65. The method of claim 64 , wherein the predetermined order is a raster scan order
66. The method of claim 64 , wherein the plurality of data packets contains non-differentially encoded motion vectors for each of the encoded macroblocks which is an inter-coded macroblock.
67. The method of claim 64 , wherein different data packets contain macroblocks with different assigned MBGIDs.
68. The method of claim 64 , wherein the MBGIDs range in value from 1 to a maximum group identifier (MGID).
69. The method of claim 68 , wherein the MGID corresponds to a video data rate.
70. The method of claim 68 , wherein the MGID corresponds to a video format.
71. The method of claim 64 , wherein the plurality of data packets contains an intra-macroblock map identifying intra-coded macroblocks in a given frame.
72. The method of claim 71 , wherein the intra-macroblock map is run-length encoded.
73. The method of claim 71 , wherein the intra-macroblock map is encoded using a bitmap.
74. The method of claim 64 , further comprising the step of detecting missing macroblocks.
75. The method of claim 74 , further comprising the step of spatially interpolating data of a given missing macroblock from data of adjacent decoded macroblocks, if the given missing macroblock was encoded as an intra-coded macroblock.
76. The method of claim 74 , further comprising the step of estimating a motion vector of a given missing macroblock based on motion vectors of adjacent macroblocks to reconstruct data content of the given missing macroblock via motion compensation, if the given missing macroblock was encoded as an inter-coded macroblock.
77. A computer-readable medium encoded with a computer program, when executed by a computer, to perform a method for processing video data, the method comprising:
receiving a plurality of data packets, each data packet containing a plurality of macroblocks, each macroblock within a particular data packet having a common macroblock group identifier (MBGID);
de-packetizing the encoded macroblocks;
reordering the received macroblocks into a predetermined order different from the order in which they were received; and
decoding the reordered macroblocks to produce a video signal.
78. The computer-readable medium of claim 77 , wherein the predetermined order is a raster scan order
79. The computer-readable medium of claim 77 , wherein the plurality of data packets contains non-differentially encoded motion vectors for each of the encoded macroblocks which is an inter-coded macroblock.
80. The computer-readable medium of claim 77 , wherein different data packets contain macroblocks with different assigned MBGIDs.
81. The computer-readable medium of claim 77 , wherein the MBGIDs range in value from 1 to a maximum group identifier (MGID).
82. The computer-readable medium of claim 81 , wherein the MGID corresponds to a video data rate.
83. The computer-readable medium of claim 81 , wherein the MGID corresponds to a video format.
84. The computer-readable medium of claim 77 , wherein the plurality of data packets contains an intra-macroblock map identifying intra-coded macroblocks in a given frame.
85. The computer-readable medium of claim 84 , wherein the intra-macroblock map is run-length encoded.
86. The computer-readable medium of claim 84 , wherein the intra-macroblock map is encoded using a bitmap.
87. The computer-readable medium of claim 77 , the method further comprising detecting missing macroblocks.
88. The computer-readable medium of claim 87 , to the method further comprising spatially interpolating data of a given missing macroblock from data of adjacent decoded macroblocks, if the given missing macroblock was encoded as an intra-coded macroblock.
89. The computer-readable medium of claim 87 , the method further comprising estimating a motion vector of a given missing macroblock based on motion vectors of adjacent macroblocks to reconstruct data content of the given missing macroblock via motion compensation, if the given missing macroblock was encoded as an inter-coded macroblock.