IP Library Granted Patent US 9,996,891
Granted Patent B2
US 9,996,891 · App. 15/145,565 · Granted Jun 12, 2018

System and method for digital watermarking

Inventors: Ruopeng Wang (Santa Clarita, CA); Joel Bigley (Valencia, CA)
Assignee: Deluxe Media Inc.
G06T1/0078G06T1/0071H04N1/3232H04N1/32245H04N1/32267H04N1/32288H04N21/8358G06T2201/0051G06T2201/0063
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,996,891
App. No.
15/145,565
Granted
Jun 12, 2018
Kind
B2
Abstract

A digital watermarking system and method are disclosed. In one respect, the disclosed digital watermarking includes generating an extracted signal by applying a watermark extractor to an original image, generating a mixed signal by mixing the first signal with a periodic watermark signal using a local weighting factor for the periodic watermark signal that attenuates a strength of the watermark signal in proportion to a pixel luminance level, and replacing the extracted signal in the original image with the mixed signal to generate a marked image, wherein the watermark signal is extractable from the marked image using the watermark extractor.

Claims (66)

1. A digital watermarking method, comprising

generating an extracted signal by applying a watermark extractor to an original image;

generating a mixed signal by mixing the extracted signal with a periodic watermark signal using a local weighting factor for the periodic watermark signal that attenuates a strength of the periodic watermark signal in proportion to a pixel luminance level; and

replacing the extracted signal in the original image with the mixed signal to generate a marked image.

2. The method of claim 1 , comprising generating the local weighting factor using a noise visibility function having a gradient removal factor that prevents the noise visibility function from significantly incrementing at locations of spatial gradient in the original image.

3. The method of claim 1 , comprising generating the local weighting factor using a noise visibility function having a density filter that prevents the periodic watermark signal from becoming visible in the marked image at spatial locations with highly dense textures.

4. The method of claim 1 , wherein mixing the extracted signal with the periodic watermark signal includes applying a watermark block of size M×M to corresponding M×M sections of the extracted signal.

5. The method of claim 4 , wherein the watermark block is generated by operations including

generating a plurality of pseudo random numbers using a unique input identification as a seed;

generating a N(0,1) normal distributed random number in double precision using one of the plurality of pseudo random numbers as a seed;

using the normal distributed random number to populate a watermark pattern sequentially; and

applying low-pass filtering of kernel size 3 on the watermark block to limit the frequency band.

6. The method of claim 5 , wherein

the watermark block is of size 64×64; and

the operation of generating a plurality of pseudo random numbers includes generating 4096 random numbers.

7. The method of claim 5 , wherein the unique input identification is 32 bits in length.

8. The method of claim 1 , wherein the periodic watermark signal is a first periodic watermark signal, the method comprising

generating a third signal by applying the watermark extractor to the marked image;

generating a fourth signal by mixing the third signal with a second periodic watermark signal using a local weighting factor for the second periodic watermark signal that attenuates a strength of the second periodic watermark signal in proportion to a pixel luminance level; and

replacing the third signal in the marked image with the fourth signal to generate an additional marked image.

9. The method of claim 8 , wherein the operation of replacing the extracted signal in the original image with the mixed signal and the operation of replacing the third signal in the marked image with the fourth signal occur at different stages of distribution of the original images.

10. The method of claim 1 , wherein the periodic watermark signal is extractable from the marked image using the watermark extractor.

11. The method of claim 1 , wherein the original image is an individual frame of a digital video.

12. The method of claim 1 , wherein the original image is an individual frame of a streaming video and the method is selectively applied on either I frames or subsequent P frames of a streaming video.

13. The method of claim 1 , wherein the watermark extractor uses a Wiener filter.

14. The method of claim 1 , wherein the periodic watermark signal is normalized.

15. The method of claim 8 , wherein either or both of the first and second periodic watermark signals are extractable from the additional marked images using the watermark extractor.

16. A digital watermarking method, comprising

applying an electro-optical transfer function to an original image to generate an EOTF image;

generating an extracted signal by applying a watermark extractor to the EOTF image;

generating a mixed signal by mixing the extracted signal with a periodic watermark signal using a local weighting factor for the periodic watermark signal that attenuates a strength of the watermark signal in proportion to a pixel luminance level;

replacing the extracted signal in the EOTF image with the mixed signal to generate a marked image; and

applying an optical-electro transfer function to the marked image to generate a final marked image, wherein the optical-electro transfer function utilized performs an inverse operation of the electro-optical transfer function, and wherein the periodic watermark signal is extractable from the final marked image.

17. The method of claim 16 , wherein the periodic watermark signal is normalized.

18. The method of claim 16 , wherein the electro-optical transfer function applies a constant gamma correction to the original image.

19. The method of claim 16 , wherein the electro-optical transfer function applies a SMPTE 2084 standard compliant transfer function to the original image.

20. The method of claim 16 , wherein the electro-optical transfer function applies transfer function compliant with a Dolby Perceptual Quantizer to the original image.

21. The method of claim 20 , wherein the electro-optical transfer function is compliant with ITU-R Recommendation BT.2020.

22. The method of claim 16 , wherein the original image has a high dynamic range.

23. The method of claim 16 , wherein the original image has highly varying ranges of luminance.

24. The method of claim 16 , comprising generating the local weighting factor using a noise visibility function having a gradient removal factor that prevents the noise visibility function from significantly incrementing at locations of spatial gradient in the EOTF image.

25. The method of claim 16 , comprising generating the local weighting factor using a noise visibility function having a density filter that prevents the watermark from becoming visible at spatial locations with highly dense textures.

26. The method of claim 16 , wherein the periodic watermark signal comprises a watermark block having a size of M×M sections.

27. The method of claim 26 , wherein the extracted signal comprises a block of M×M sections and each of the M×M sections of the watermark block correspond to the M×M sections of the extracted signal.

28. The method of claim 27 , further comprising generating the watermark block from a plurality of pseudo random numbers.

29. The method of claim 28 , wherein the plurality of pseudo random numbers is generated using a unique input identification.

30. The method of claim 29 , wherein the unique input identification is 32 bits in length.

31. The method of claim 28 , wherein the generation of the watermark block comprises generating a N(0,1) normal distributed random number in double precision using at least one of the plurality of pseudo random numbers as a seed.

32. The method of claim 31 , wherein the generation of the watermark block comprises using the normal distributed random number to populate a watermark pattern sequentially.

33. The method of claim 26 , comprising

generating the watermark block; and

limiting a frequency band used to mix the watermark block with the extracted signal by applying low-pass filtering to the watermark block.

34. The method of claim 33 , wherein the low-pass filtering utilizes a kernel size of 3.

35. The method of claim 16 , wherein the original image is an individual frame of a digital video.

36. The method of claim 16 , wherein the original image is an individual frame of a streaming video.

37. The method of claim 36 , wherein the streaming video includes a sequence of I frames and the method is applied to two or more of the I frames.

38. The method of claim 16 , wherein the watermark extractor uses an average of two or more local noise variances for the original image to generate the extracted signal.

39. The method of claim 16 , wherein the watermark extractor comprises a Helicon filter.

40. The method of claim 16 , wherein the final image is humanly perceptible and includes a watermark having a noise pattern that resembles film grain.

41. The method of claim 16 , wherein the periodic watermark signal uniquely identifies a viewer and a time at which the final image is communicated to the viewer.

42. The method of claim 41 , wherein the original image is a frame of a streaming video to be communicated to the viewer over a telecommunications network.

43. The method of claim 42 , wherein the telecommunications network comprises the Internet.

44. The method of claim 42 , wherein the periodic watermark signal is dynamically generated and applied to the EOTF image of the frame of the streaming video immediately prior to a time of communication of the final image in the streaming video to the viewer.

45. The method of claim 16 , wherein the mixed signal is generated based on an embedding interval.

46. The method of claim 45 , wherein the final image is communicated to a receiving device over a network connection and the embedding interval is determined based on a target CPU load for a streaming server utilized to communicate the final image.

47. The method of claim 46 , wherein the receiving device is a home set-top box.

Assignments (30)
RELEASE OF SECURITY INTEREST Recorded Apr 4, 2023
From: PNC BANK, NATIONAL ASSOCIATION
To: DELUXE MEDIA INC.; SOFTITLER NET, INC.; DELUXE LABORATORIES LLC; DELUXE ONE LLC
Reel/Frame 063213/0909 →
RELEASE OF SECURITY INTEREST IN PATENTS Recorded Apr 2, 2023
From: LOAN ADMIN CO LLC, AS COLLATERAL AGENT
To: DELUXE MEDIA INC.; SOFTITLER NET, INC.; DELUXE ONE LLC
Reel/Frame 063237/0243 →
GRANT OF SECURITY INTEREST IN UNTED STATES PATENTS Recorded Aug 21, 2020
From: DELUXE MEDIA INC.; SOFTITLER NET, INC.; DELUXE ONE LLC
To: LOAN ADMIN CO LLC
Reel/Frame 054153/0001 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Jul 21, 2020
From: DELUXE MEDIA INC.; SOFTITLER NET, INC.; DELUXE LABORATORIES LLC; DELUXE ONE LLC
To: PNC BANK, NATIONAL ASSOCIATION
Reel/Frame 053265/0329 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Jul 1, 2020
From: CORTLAND CAPITAL MARKET SERVICES LLC
To: DELUXE ENTERTAINMENT SERVICES INC.; DELUXE MEDIA INC. (F/K/A DELUXE DIGITAL STUDIOS, INC.); SOFTITLER NET, INC.; DELUXE ONE LLC (F/K/A DELUXE MEDIUSONE LLC); DELUXE LABORATORIES LLC
Reel/Frame 053117/0015 →
RELEASE OF SECURITY INTEREST Recorded Jul 1, 2020
From: PNC BANK, NATIONAL ASSOCIATION
To: DELUXE MEDIA INC.; DELUXE LABORATORIES LLC; DELUXE ONE LLC; SOFTITLER NET, INC.
Reel/Frame 053102/0558 →
PARTIAL RELEASE OF SECURITY INTEREST Recorded Jul 1, 2020
From: CORTLAND CAPITAL MARKET SERVICES LLC
To: DELUXE ENTERTAINMENT SERVICES INC.; DELUXE MEDIA INC. (F/K/A DELUXE DIGITAL STUDIOS, INC.); SOFTITLER NET, INC.; DELUXE ONE LLC (F/K/A DELUXE MEDIUSONE LLC); DELUXE LABORATORIES LLC
Reel/Frame 053116/0566 →
RELEASE OF SECURITY INTEREST Recorded Jun 29, 2020
From: CORTLAND CAPITAL MARKET SERVICES LLC (AS SUCCESSOR COLLATERAL AGENT TO CREDIT SUISSE AG) (REEL/FRAME 50609/0590)
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES INC. (AS SUCCESSOR BY MERGER TO DELUXE ENTERTAINMENT SERVICES GROUP INC.); DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SOFTITLER NET, INC. (AS SUCCESSOR BY MERGER TO SFERA LABS, LLC)
Reel/Frame 053074/0274 →
COURT ORDER Recorded Jun 29, 2020
From: CREDIT SUISSE AG (RELEASING REEL/FRAME 50609/0083)
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES INC. (AS SUCCESSOR BY MERGER TO DELUXE ENTERTAINMENT SERVICES GROUP INC.); DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SOFTITLER NET, INC. (AS SUCCESSOR BY MERGER TO SFERA LABS, LLC)
Reel/Frame 053081/0672 →
RELEASE OF SECURITY INTEREST Recorded Jun 29, 2020
From: CORTLAND CAPITAL MARKET SERVICES LLC (AS SUCCESSOR COLLATERAL AGENT TO CREDIT SUISSE AG) (REEL/FRAME 49924/0706)
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES INC. (AS SUCCESSOR BY MERGER TO DELUXE ENTERTAINMENT SERVICES GROUP INC.); DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SOFTITLER NET, INC. (AS SUCCESSOR BY MERGER TO SFERA LABS, LLC)
Reel/Frame 053074/0294 →
COURT ORDER Recorded Jun 29, 2020
From: CREDIT SUISSE AG (RELEASING REEL/FRAME 49924/0596)
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES INC. (AS SUCCESSOR BY MERGER TO DELUXE ENTERTAINMENT SERVICES GROUP INC.); DELUXE MEDIA INC.; DELUXE ONE LLC; SOFTITLER NET, INC. (AS SUCCESSOR BY MERGER TO SFERA LABS, LLC)
Reel/Frame 053083/0373 →
SECURITY INTEREST Recorded Jan 8, 2020
From: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE MEDIA INC.; DELUXE ENTERTAINMENT SERVICES INC.; DELUXE LABORATORIES LLC; DELUXE ONE LLC; DELUXE BROADCAST SERVICES LTD; DELUXE UK HOLDINGS LIMITED
To: PNC BANK, NATIONAL ASSOCIATION
Reel/Frame 051521/0411 →
RELEASE OF SECURITY AGREEMENT Recorded Jan 7, 2020
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE ONE LLC; DELUXE MEDIA INC.; DELUXE ENTERTAINMENT SERVICES INC.; DELUXE LABORATORIES LLC; SOFTITLER NET, INC.
Reel/Frame 051541/0396 →
SECURITY INTEREST Recorded Nov 13, 2019
From: DELUXE 3D LLC; DELUXE ENTERTAINMENT SERVICES INC.; DELUXE MEDIA INC. (F/K/A DELUXE DIGITAL STUDIOS, INC.); DELUXE CREATIVE SERVICES INC. (F/K/A DELUXE MEDIA CREATIVE SERVICES INC.); SOFTITLER NET, INC.; DELUXE ONE LLC (F/K/A DELUXE MEDIUSONE LLC); DELUXE LABORATORIES LLC
To: CORTLAND CAPITAL MARKET SERVICES LLC
Reel/Frame 050994/0428 →
SECURITY INTEREST Recorded Nov 13, 2019
From: DELUXE 3D LLC; DELUXE ENTERTAINMENT SERVICES INC.; DELUXE MEDIA INC. (F/K/A DELUXE DIGITAL STUDIOS, INC.); DELUXE CREATIVE SERVICES INC. (F/K/A DELUXE MEDIA CREATIVE SERVICES INC.); SOFTITLER NET, INC.; DELUXE ONE LLC (F/K/A DELUXE MEDIUSONE LLC); DELUXE LABORATORIES LLC
To: CORTLAND CAPITAL MARKET SERVICES LLC
Reel/Frame 050994/0293 →
RELEASE OF SECURITY INTEREST Recorded Nov 12, 2019
From: CORTLAND CAPITAL MARKET SERVICES LLC (AS SUCCESSOR COLLATERAL AGENT TO CREDIT SUISSE AG)
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE ENTERTAINMENT SERVICES INC. (AS SUCCESSOR BY MERGER TO DELUXE ENTERTAINMENT SERVICES GROIUP INC.); DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SOFTITLER NET, INC. (AS SUCCESSOR BY MERGER TO SFERA LABS, LLC)
Reel/Frame 050983/0730 →
RELEASE OF SECURITY INTEREST Recorded Nov 11, 2019
From: CORTLAND CAPITAL MARKET SERVICES LLC (AS SUCCESSOR COLLATERAL AGENT TO CREDIT SUISSE AG)
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE ONE LLC (AS SUCCESSOR BY MERGER TO DELUXE DIGITAL DISTRIBUTION INC.); DELUXE MEDIA INC.; SOFTITLER NET, INC. (AS SUCCESSOR BY MERGER TO SFERA LABS, LLC); DELUXE ENTERTAINMENT SERVICES INC.
Reel/Frame 050974/0439 →
RELEASE OF SECURITY AGREEMENT Recorded Nov 7, 2019
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
Reel/Frame 050965/0381 →
RELEASE OF SECURITY AGREEMENT Recorded Nov 7, 2019
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
Reel/Frame 050965/0459 →
SECURITY INTEREST Recorded Nov 7, 2019
From: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE ONE LLC; DELUXE MEDIA INC.; DELUXE ENTERTAINMENT SERVICES INC.; DELUXE LABORATORIES LLC; SOFTITLER NET, INC.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 050949/0192 →
RELEASE OF SECURITY AGREEMENT Recorded Oct 9, 2019
From: CREDIT SUISSE AG, AS COLLATERAL AGENT
To: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
Reel/Frame 050673/0247 →
SECURITY INTEREST Recorded Oct 2, 2019
From: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
To: CREDIT SUISSE AG, AS COLLATERAL AGENT
Reel/Frame 050609/0083 →
SECURITY INTEREST Recorded Oct 2, 2019
From: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 050609/0255 →
SECURITY INTEREST Recorded Oct 2, 2019
From: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
To: CREDIT SUISSE AG, AS COLLATERAL AGENT
Reel/Frame 050609/0590 →
SECURITY INTEREST Recorded Sep 20, 2019
From: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE LABORATORIES LLC; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
To: CREDIT SUISSE AG, AS COLLATERAL AGENT
Reel/Frame 050451/0712 →
SECURITY INTEREST Recorded Jul 31, 2019
From: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
To: CREDIT SUISSE AG, AS COLLATERAL AGENT
Reel/Frame 049924/0706 →
SECURITY INTEREST Recorded Jul 31, 2019
From: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 049924/0645 →
SECURITY INTEREST Recorded Jul 31, 2019
From: DELUXE 3D LLC; DELUXE CREATIVE SERVICES INC.; DELUXE DIGITAL DISTRIBUTION INC.; DELUXE DIGITAL STUDIOS, INC.; DELUXE ENTERTAINMENT SERVICES GROUP INC.; DELUXE MEDIA INC.; DELUXE ONE LLC; SFERA LABS, LLC
To: CREDIT SUISSE AG, AS COLLATERAL AGENT
Reel/Frame 049924/0596 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 4, 2016
From: BIGLEY, JOEL
To: DELUXE MEDIA INC.
Reel/Frame 039347/0288 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2016
From: WANG, RUOPENG
To: DELUXE MEDIA INC.
Reel/Frame 038615/0734 →
Continuity (2)
Provisional Application 62173835 · Jun 10, 2015
Related Publication 20160364826A1 · Dec 15, 2016