System and method for embedding a two dimensional code in video images

1. A method of producing a set of QR codes with visual image information in a computing means, said method including the steps of:

receiving data to encode, stored in a first memory of the computing means, and a set of images comprising a three dimensional array stored in a second memory of the computing means;

generating a three dimensional array comprising a set of QR codes from the data stored in the first memory such that each QR code is represented as a matrix with M rows and N columns and a time index, wherein each element in each matrix is one of two unique values (binary);

generating a second three dimensional array comprising a set of K matrices with M rows and N columns wherein each K matrix is related to a time index t and has L layers, each element in the second three dimensional array has coordinates row r, column c, time index t and each coordinate element is defined as a function of the set of images stored in the second memory;

divide each of the K, M by N matrices into three dimensional sub-blocks of elements with m rows, n columns and depth equal to k elements, wherein m≤M, n≤N and k≤L;

modify a sub-set of the element values of each image sub-block based upon the values of the corresponding QR code elements in the sub-block with the same element rows, columns and time index t;

inside each sub-block, define a subset of elements from each image sub block and replace them with the elements in the sub-set of each image sub-block modified based on the QR code sub block with the same row, column and time index coordinates; and

wherein the modification of element values of the images stored in the second memory is derived by minimizing a visual cost function f(r,c,t) and by imposing a limit on a probability of decoding error, where r and c represent matrix row and column coordinates surrounding the subject element under consideration, and t represents the time index of the image within the set of images being considered,

wherein the limit on the probability of decoding error is derived from a three dimensional array comprising a set of K priority matrices of M rows and N columns, and

wherein the probability of decoding error is derived by: dividing the set of QR code matrices, the set of image matrices and the set of priority matrices in overlapping windows of P<M rows and Q<N columns and R<=k depth elements centered on the sub-blocks, and applying a function, representing the probability of decoding error, of the overlapping windows and n parameters p 1 , p 2 , p 3 , p 4 , . . . , p n , wherein application of the function uniquely determines the location and values of modified elements in the three dimensional image array.

2. The method of claim 1 , wherein the three dimensional array stored in the second memory is a gray-scale image having a single scalar value for each element in the M by N matrix, wherein L=1.

3. The method of claim 1 , wherein the three dimensional array stored in the second memory is a color image composed of a red, green, and blue scalar value for each element in the M by N by K array, wherein L=3.

4. The method of claim 1 , wherein the three dimensional arrays stored in the second memory includes an alpha transparency layer of each graphic representation in a set of graphic representations.

5. The method of claim 1 , wherein the sub-set of elements inside the sub-block of m rows and n columns and k depth is a set of K square matrices of d rows and d columns where d<m and d<n.

6. The method of claim 1 , wherein the subset of elements adjusted in the three dimensional image matrix is distributed within the sub-block of m rows and n columns and depth k and whose location is determined by a predetermined ordering stored in a third memory.

7. A method of producing a set of QR codes with visual image information in a computing device, said method including the steps of:

receiving data to encode, stored in a first memory of the computing device, along with a set of images comprising a three dimensional array stored in a second memory of the computing device;

generating a three dimensional array comprising a set of QR codes from the data stored in the first memory such that each QR code is represented as a matrix with M rows and N columns and a time index and where each element is one of two unique values;

generating a second three dimensional array comprising a set of K matrices with M rows and N columns where each element in the set of K matrices is defined as a function of the set of images stored in the second memory and each K matrix is related to a time index t and has L layers;

divide each of the K, M by N matrices into three dimensional sub-blocks of elements with m rows, n columns and depth equal to k elements, wherein m≤M, n≤N, and k≤L;

converting the image elements of each image stored in the second memory to a scalar value that measures each element's visual appearance according to a visual cost function f(r,c,t) where r and c represent matrix row and column coordinates surrounding the subject element under consideration, and t represents the time index of the image within the set of images being considered;

modifying a subset of element values of each image matrix inside a sub-block based upon the values of the corresponding QR code elements in the sub-block with the same element rows, columns and time index;

inside each sub-block, define a subset of elements from each image sub-block and replace each sub-block with the elements in the sub-set of each image sub-block modified based on the QR code sub-block with the same row, column, and time index coordinates; and

wherein the modification of element values of the images stored in the second memory is derived by minimizing the visual cost function f(r,c,t) and by imposing a limit on a probability of decoding error,

wherein the limit on the probability of decoding error is derived from a three dimensional array comprising a set of K priority matrices of M rows and N columns, and

wherein the probability of decoding error is derived by: dividing the set of QR code matrices, the set of image matrices and the set of priority matrices in overlapping windows of P<M rows and Q<N columns and R<=k depth elements centered on the sub-blocks, and applying a function, representing the probability of decoding error, of the overlapping windows and n parameters p 1 , p 2 , p 3 , p 4 , . . . , p n , wherein application of the function uniquely determines the location and values of modified elements in the three dimensional image array.

8. The method of claim 7 , wherein the set of priority matrices is provided by a user.

9. The method of claim 7 , wherein the set of priority matrices is derived from the image stored in the second memory.

10. The method of claim 7 , wherein the parameters p 1 , p 2 correspond to concentration of adjusted elements in the sub-blocks.

11. The method of claim 7 , wherein the parameters p 3 , p 4 , . . . , p n correspond to intensity parameters that determine how the intensity of modified elements is transformed.

12. The method of claim 7 , wherein the probability of decoding error is composed as a linear combination of two components:

a probability of detection error defined as a probability of incorrectly sampling the center of the sub-blocks at a decoder; and

a probability of binarization error defined as a probability of incorrectly detecting any element of the QR code matrix at the decoder.

13. The method of claim 7 , wherein the visual cost function f(r,c,t) is derived as a function of the overlapping windows and the set of n parameters p 1 , p 2 , p 3 , p 4 , . . . , p n .

14. The method of claim 13 , wherein two additional multivariate functions:

g(p 1 , p 2 , p 3 , p 4 , . . . , p n ) and h(p 1 , p 2 , p 3 , p 4 , . . . , p n ) enforce an inequality g(p 1 , p 2 , p 3 , p 4 , . . . , p n )<0 and an equality h(p 1 , p 2 , p 3 , p 4 , . . . , p n )=0 constraints.

15. The method of claim 7 , wherein the visual cost function is obtained via a visual perception metric for video between the modified set of images and the original set of images.

16. The method of claim 7 , wherein the visual cost function includes a distance metric in a perceptually uniform color space in order to determine a color difference between sub regions of the original set of image matrices and the adjusted set of image matrices.

17. A system, comprising:

a memory;

a processor, when executing programming code stored in the memory, configured to perform functions for combining image data with two dimensional (2D) code data, the functions including:

receiving data to encode, stored in a first memory of the system, along with a set of images comprising a three dimensional array stored in a second memory of the system;

generating a three dimensional array comprising a set of QR Codes from the data stored in the first memory such that each QR code is represented as a matrix with M rows and N columns and a time index and where each element is one of two unique values;

generating a second three dimensional array comprising a set of K matrices with M rows and N columns where each element is defined as a function of the set of images stored in the second memory and each K matrix is related to a time index t and has L layers;

dividing each of the K, M by N matrices into three dimensional sub-blocks of elements with m rows, n columns and depth equal to k elements;

converting the image elements of each image stored in the second memory to a scalar value that measure each element's visual appearance according to a visual cost function f(r,c,t) where r and c represent matrix row and column coordinates surrounding the subject element under consideration, and t represents a time index of the image within the set of images being considered;

modifying a subset of element values of each image matrix inside a subblock based upon the values of the corresponding QR code elements in the subblock with the same element rows, columns and time index;

inside each sub-block, define a subset of elements from each image sub-block and replace each sub-block with the elements in the sub-set of each image sub-block modified based on the QR code sub-block with the same row, column, and time index coordinates to provide an embedded time-based image; and

wherein the modification of element values of the images stored in the second memory is derived by minimizing the visual cost function f(r,c,t) and by imposing a limit on a probability of decoding error,

wherein the limit on the probability of decoding error is derived from a three dimensional array comprising a set of K priority matrices of M rows and N columns, and

wherein the probability of decoding error is derived by: dividing the set of QR code matrices, the set of image matrices and the set of priority matrices in overlapping windows of P<M rows and Q<N columns and R<=k depth elements centered on the sub-blocks, and applying a function, representing the probability of decoding error, of the overlapping windows and n parameters p 1 , p 2 , p 3 , p 4 , . . . , p n , wherein application of the function uniquely determines the location and values of modified elements in the three dimensional image array.

18. The system of claim 17 , further comprising:

a display device;

wherein the processor upon execution of programming code stored in the memory is further configured to perform functions, including functions to:

present the embedded time-based image on the display device.