IP Library Granted Patent US 12,654,374
Granted Patent B1
US 12,654,374 · App. 18/303,442 · Granted Jun 16, 2026

Neural network stylization of watermark signals

Inventors: Tomas Filler (Beaverton, OR); Nicholas Anderson (Colorado Springs, CO); Ajith M Kamath (Beaverton, OR)
Assignee: Digimarc Corporation
B29C45/372B23K26/362B65D1/0207B29L2031/7158B65D2203/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,654,374
App. No.
18/303,442
Granted
Jun 16, 2026
Kind
B1
Abstract

A watermark image is stylized using a neural network. Stylization can employ a composite loss function in which a content loss term is based on image similarity to the watermark image itself, rather than based on similarity to layer activations produced by the neural network. The composite loss function may further avoid use of mean-squared error in connection with content loss, to allow the network freedom to produce a stylized image having a mean luma value that is remote from a mean luma value of the watermark image. An exemplary application generates texture patterns that simulate leather, yet convey plural-symbol payloads. Many other features and arrangements are also detailed.

Claims (15)

1 . A watermarked image embedded in a physical surface or stored on a non-transitory computer readable medium, previously derived from an original image, the watermarked image having a mean value that is not between 124 and 131, wherein at least one of the following conditions is satisfied:

(a) a gradient between a pair of adjoining pixels in the watermarked image differs, by more than 30, from a gradient between a spatially-corresponding pair of pixels in the original image; or

(b) a pixel value in the watermarked image differs, by more than 64, from a pixel value in the original image;

wherein the watermarked image is structurally different than the original image, with edges having been both added to and removed from the original image.

2 . The watermarked image of claim 1 in which condition (a) is satisfied, and in which a gradient between a pair of adjoining pixels in the watermarked image differs, by more than 50, 80 or 120 from a gradient between a spatially-corresponding pair of pixels in the original image.

3 . The watermarked image of claim 1 in which condition (b) is satisfied, and in which a pixel value in the watermarked image differs, by more than 128 or 192, from a pixel value in the original image.

4 . The watermarked image of claim 1 in which both conditions (a) and (b) are satisfied.

5 . The watermarked image of claim 1 having a mean value that is not between 117 and 139.

6 . The watermarked image of claim 1 in which the watermarked image encodes a plural-symbol payload, said payload being operative, when sensed by a compliant reader component of a system, to control an operation of said system.

7 . The watermarked image of claim 1 in which said numeric parameters are stated in context of an 8-bit representation, but can be adjusted proportionately for use in other bit representations.

8 . The watermarked image of claim 1 in which satisfaction of said condition(s) is not due to shifting of an edge in the original image, or modulation of a line width in the original image.

9 . The watermarked image of claim 1 in which the watermarked image includes interpixel gradients that are stronger, by at least 50, than corresponding interpixel gradients in the original image.

10 . The watermarked image of claim 1 in which the watermarked image is characterized by not being simply composed as a weighted combination of a watermark image and a style image, such that there is no scaled version of the style image that can be subtracted from the watermarked image to leave just a scaled version of a watermark image as residue.

11 . Retail product packaging or container that conveys the watermarked image of claim 1 , in which the watermarked image encodes a plural-symbol payload, said payload being operative, when sensed by a compliant reader component of: (a) a waste sorting system, to control an ejection operation of said waste sorting system, or (b) a retail checkout system, to add a price of said good to a shopper's checkout tally.

12 . A molded item having a surface level that varies in accordance with pixel values of the watermarked image of claim 1 .

Assignments (3)
ARTICLES OF CONVERSION Recorded Jun 19, 2026
From: DIGIMARC CORPORATION
To: DIGIMARC LLC
Reel/Frame 075863/0211 →
ARTICLES OF AMENDMENT OFTHE ARTICLES OF ORGANIZATION OF DIGIMARC LLC Recorded Jun 19, 2026
From: DIGIMARC LLC
To: DMRC LLC
Reel/Frame 075863/0266 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 7, 2026
From: FILLER, TOMAS; ANDERSON, NICHOLAS; KAMATH, AJITH M.
To: DIGIMARC CORPORATION
Reel/Frame 074588/0499 →
Continuity (4)
Continuation In Part 17835775 · Jun 8, 2022
Continuation 17681262 · Feb 25, 2022
Provisional Application 63337000 · Apr 29, 2022
Provisional Application 63333117 · Apr 20, 2022
References Cited (140)
US 5636292A · Rhoads · 1997 [cited by applicant]
US 5832119A · Rhoads · 1998 [cited by applicant]
US 5862260A · Rhoads · 1999 [cited by applicant]
US 6060428A · Chang · 2000 [cited by applicant]
US 6345104B1 · Rhoads · 2002 [cited by applicant]
US 6483927B2 · Brunk · 2002 [cited by applicant]
US 6567534B1 · Rhoads · 2003 [cited by applicant]
US 6580809B2 · Stach · 2003 [cited by applicant]
US 6590996B1 · Reed · 2003 [cited by applicant]
US 6614914B1 · Rhoads · 2003 [cited by applicant]
US 6625297B1 · Bradley · 2003 [cited by applicant]
US 6724914B2 · Brundage · 2004 [cited by applicant]
US 6760464B2 · Brunk · 2004 [cited by applicant]
US 6987861B2 · Rhoads · 2006 [cited by applicant]
US 6993152B2 · Patterson · 2006 [cited by applicant]
US 7072490B2 · Stach · 2006 [cited by applicant]
US 7231061B2 · Bradley · 2007 [cited by applicant]
US 7340076B2 · Stach · 2008 [cited by applicant]
US 7532741B2 · Stach · 2009 [cited by applicant]
US 7555139B2 · Rhoads · 2009 [cited by applicant]
US 7831062B2 · Stach · 2010 [cited by applicant]
US 7856143B2 · Abe · 2010 [cited by applicant]
US 8009893B2 · Rhoads · 2011 [cited by applicant]
US 8144368B2 · Rodriguez · 2012 [cited by applicant]
US 8515121B2 · Stach · 2013 [cited by applicant]
US 9635378B2 · Holub · 2017 [cited by applicant]
US 9747656B2 · Stach · 2017 [cited by applicant]
US 10657676B1 · Rehfeld · 2020 [cited by applicant]
US 10664722B1 · Sharma · 2020 [cited by applicant]
US 10748232B2 · Kamath · 2020 [cited by applicant]
US 11062108B2 · Bradley · 2021 [cited by applicant]
US 11276133B2 · Kamath · 2022 [cited by applicant]
US 20020136429A1 · Stach · 2002 [cited by applicant]
US 20030039376A1 · Stach · 2003 [cited by applicant]
US 20030053654A1 · Patterson · 2003 [cited by applicant]
US 20040032972A1 · Stach · 2004 [cited by applicant]
US 20050207615A1 · Stach · 2005 [cited by applicant]
US 20060071081A1 · Wang · 2006 [cited by applicant]
US 20060115110A1 · Rodriguez · 2006 [cited by applicant]
US 20070071278A1 · Cheng · 2007 [cited by applicant]
US 20080112590A1 · Stach · 2008 [cited by applicant]
US 20080264824A1 · Alejandra · 2008 [cited by applicant]
US 20090018996A1 · Hunt · 2009 [cited by applicant]
US 20090129592A1 · Swiegers · 2009 [cited by applicant]
US 20090220121A1 · Stach · 2009 [cited by applicant]
US 20100119108A1 · Rhoads · 2010 [cited by applicant]
US 20100165158A1 · Rhoads · 2010 [cited by applicant]
US 20100303349A1 · Bechtel · 2010 [cited by applicant]
US 20100325117A1 · Sharma · 2010 [cited by applicant]
US 20110110555A1 · Stach · 2011 [cited by applicant]
US 20110212717A1 · Rhoads · 2011 [cited by applicant]
US 20110214044A1 · Davis · 2011 [cited by applicant]
US 20110276663A1 · Rhoads · 2011 [cited by applicant]
US 20120133954A1 · Takabayashi · 2012 [cited by applicant]
US 20120229467A1 · Czerwinski · 2012 [cited by applicant]
US 20130001313A1 · Denniston, Jr. · 2013 [cited by applicant]
US 20140029809A1 · Rhoads · 2014 [cited by applicant]
US 20140052555A1 · Macintosh · 2014 [cited by applicant]
US 20140210780A1 · Lee · 2014 [cited by applicant]
US 20140366052A1 · Ives · 2014 [cited by applicant]
US 20150055837A1 · Rhoads · 2015 [cited by applicant]
US 20150262347A1 · Duerksen · 2015 [cited by applicant]
US 20150269617A1 · Mikurak · 2015 [cited by applicant]
US 20160026853A1 · Wexler · 2016 [cited by applicant]
US 20160189381A1 · Rhoads · 2016 [cited by applicant]
US 20160275326A1 · Falkenstern · 2016 [cited by applicant]
US 20170004597A1 · Boles · 2017 [cited by applicant]
US 20170024840A1 · Holub · 2017 [cited by applicant]
US 20170024845A1 · Filler · 2017 [cited by applicant]
US 20170193628A1 · Sharma · 2017 [cited by applicant]
US 20180005343A1 · Rhoads · 2018 [cited by applicant]
US 20180068463A1 · Risser · 2018 [cited by applicant]
US 20180082407A1 · Rymkowski · 2018 [cited by applicant]
US 20180082715A1 · Rymkowski · 2018 [cited by applicant]
US 20180150947A1 · Lu · 2018 [cited by applicant]
US 20180158224A1 · Bethge · 2018 [cited by applicant]
US 20180182116A1 · Rhoads · 2018 [cited by applicant]
US 20180211157A1 · Liu · 2018 [cited by applicant]
US 20180285679A1 · Amitay · 2018 [cited by applicant]
US 20180357800A1 · Oxholm · 2018 [cited by applicant]
US 20180373999A1 · Xu · 2018 [cited by applicant]
US 20190139176A1 · Stach · 2019 [cited by applicant]
US 20190213705A1 · Kamath · 2019 [cited by applicant]
US 20190266749A1 · Rhoads · 2019 [cited by applicant]
US 20190289330A1 · Alakuijala · 2019 [cited by applicant]
US 20190332840A1 · Sharma · 2019 [cited by applicant]
US 20200082249A1 · Hua · 2020 [cited by applicant]
US 20220388213A1 · Filler · 2022 [cited by applicant]
WO 2006048368 · 2006 [cited by applicant]
WO 2011029845A2 · 2011 [cited by applicant]
WO 2016153911A1 · 2016 [cited by applicant]
WO 2018111786 · 2018 [cited by applicant]
WO 2019113471 · 2019 [cited by applicant]
WO 2019165364 · 2019 [cited by applicant]
Crowson, Web Archive of github<dot>com/crowsonkb/style-transfer-pytorch, May 13, 2021. 7 pages. [cited by applicant]
Deng et al, Arbitrary Video Style Transfer via Multi-Channel Correlation, Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, No. 2, pp. 1210-1217, 2021. [cited by applicant]
Gatys et al., “Image Style Transfer Using Convolutional Neural Networks”, Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2414-2423, 2016. [cited by applicant]
Kingma, et al, Adam: a method for stochastic optimization, arXiv preprint arXiv:1412.6980, 2017. [cited by applicant]
Li, et al, Demystifying neural style transfer, arXiv preprint arXiv:1701.01036, 2017. 7 pages. [cited by applicant]
Liu et al, CS 229 Project Final Report: Neural Style Transfer, Stanford University, 2019. 5 pages. [cited by applicant]
Risser, et al, Stable and controllable neural texture synthesis and style transfer using histogram losses, arXiv preprint arXiv:1701.08893, 2017. 14 pages. [cited by applicant]
Sanakoyeu, et al, A style-aware content loss for real-time hd style transfer, Proceedings of the European conference on computer vision (ECCV), pp. 698-714, 2018. [cited by applicant]
Tuyen, et al, Deep correlation multimodal neural style transfer, IEEE Access, 9, 141329-141338, 2021. [cited by applicant]
Yeh, et al, Improved style transfer by respecting inter-layer correlations, arXiv preprint arXiv:1801.01933, 2018. 11 pages. [cited by applicant]
12 Creative Barcode Designs that (Amazingly) Work, Kongkiat blog from Web Archive, copy dated Apr. 23, 2015. [cited by applicant]
40 Gorgeous QR Code Artworks That Rock, Kongkiat blog from Web Archive, copy dated Apr. 9, 2015. [cited by applicant]
A. Secord, “Weighted Voronoi Stippling,” Proc. 2nd Ann. Symp. Non-Photorealistic Animation and Rendering (NPAR 2002), ACM Press, 2002, pp. 27-43. [cited by applicant]
Chu, et al, Halftone QR codes, ACM Transactions on Graphics, vol. 32, No. 6, Nov. 1, 2013, p. 217. (8 pages). [cited by applicant]
Davis B, Signal rich art: enabling the vision of ubiquitous computing. In Media Watermarking, Security, and Forensics III Feb. 8, 2011 (vol. 7880, p. 788002). International Society for Optics and Photonics. (11 pages). [cited by applicant]
Davis, Bruce ED—Memon Nasir D. et al: “Signal rich art: enabling the vision of ubiquitous computing”, Media Watermarking, Security, and Forensics III, SPIE, 1000 20th St. Bellingham WA 98225-6705 USA, vol. 7880, No. 1, … [cited by applicant]
Excerpts from file of corresponding EPO application 18836559.7 including related PCT documents concerning application PCT/US18/64516 (published as WO2019113471). [cited by applicant]
Extended European Search Report for App. No. EP19815064.1, dated Feb. 28, 2022, 13 pages. [cited by applicant]
Gatys, et al, A Neural Algorithm of Artistic Style. arXiv preprint arXiv:1508.06576, Aug. 26, 2015. 16 pages. [cited by applicant]
Google Scholar Search Results. [cited by applicant]
Grinchuk, et al, Learnable visual markers, Advances in Neural Information Processing Systems 29 (2016). [cited by applicant]
Hayes et al., “Generating Steganographic Images via Adversarial Training”, Proceedings of the 31st annual conference on advances in Neural Information Processing Systems, Mar. 2017, pp. 1951-1960, XP055573249. [cited by applicant]
International Preliminary Report on Patentability for PCT/US2019/036126, dated May 22, 2020. (8 pages). [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US19/36126, date of mailing Oct. 9, 2019, 12 pages. [cited by applicant]
Invitation to Pay Additional Fees including Communication Relating to the Results of the Partial International Search in PCT/US2018/064516, mailed Apr. 5, 2019. 17 pages. [cited by applicant]
Johnson, et al, “Perceptual losses for real-time style transfer and super-resolution”, European Conference on Computer Vision, Oct. 8, 2016, pp. 694-711. [cited by applicant]
Johnson, excerpts from github web site “Fast-neural-style,” Nov. 19, 2017. 19 pages. [cited by applicant]
Kaplan, Craig S. et al: “TSP Art”, Bridges 2005: Renaissance Banff: Mathematics, Music, Art, Culture, Jul. 31, 2005 (Jul. 31, 2005), pp. 301-308, XP055893366, ISBN: 978-0-9665201-6-3 Retrieved from the Internet: URL:htt… [cited by applicant]
Ke et al., “Kernel Target Alignment for Feature Kernel Selection in Universal Steganographic Detection based on Multiple Kernel SVM”, International Symposium on Instrumentation & Measurement, Sensor Network and Automati… [cited by applicant]
Lengstrom, excerpts from github web site, “Fast Style Transfer in TensorFlow,” Oct. 3, 2017. 24 pages. [cited by applicant]
Li Hai-Sheng et al., “Style transfer for QR code”, Multimedia Tools and Applications, vol. 79, No. 45-46, Dec. 1, 2020, pp. 33839-33852, XP037308023. [cited by applicant]
Lin, et al., Artistic QR code embellishment. Computer Graphics Forum, Oct. 1, 2013, vol. 32, No. 7, pp. 137-146. [cited by applicant]
Lin, et al., Efficient QR code beautification with high quality visual content, IEEE Transactions on Multimedia, vol. 17, No. 9, Sep. 2015, pp. 1515-1524. [cited by applicant]
Liu, et al., Line-based cubism-like image—a new type of art image and its application to lossless data hiding, IEEE Transactions on Information Forensics and Security, vol. 7, No. 5, Oct. 2012, pp. 1448-1458. [cited by applicant]
Nikulin, Exploring the neural algorithm of artistic style, arXiv preprint arXiv:1602.07188, Feb. 23, 2016. 15 pages. [cited by applicant]
Photoshop Elements Help—Patterns, Web Archive, Mar. 13, 2014. 2 pages. [cited by applicant]
Preston, et al, Enabling hand-crafted visual markers at scale, Proceedings of the 2017 ACM Conference on Designing Interactive Systems, Jun. 10, 2017, pp. 1227-1237. [cited by applicant]
Puyang, et al, Style Transferring Based Data Hiding for Color Images, International Conference on Cloud Computing and Security, Jun. 8, 2018, pp. 440-449. [cited by applicant]
Raval, excerpts from github web site, Style Transfer Using VGG-16 Model, Mar. 8, 2017. 15 pages. [cited by applicant]
Rosebrock, excerpts from web page Neural Style Transfer with OpenCV, Aug. 27, 2018. 23 pages. [cited by applicant]
Russian and Japanese Barcodes: a New Venue for Artistic Expression, Inventorspot web page from Web Archive, copy dated Mar. 20, 2015. [cited by applicant]
Simonyan et al, Very Deep Convolutional Networks for Large-Scale Image Recognition, arXiv preprint 1409.1556v6, Apr. 10, 2015. 14 pages. [cited by applicant]
Ulyanov, et al, Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis, Proc. 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 6924-6932. [cited by applicant]
Wong, Fernando J. et al: “Abstracting images into continuous-line artistic styles”, Visual Computer, Springer, Berlin, DE, vol. 29, No. 6, Apr. 23, 2013 (Apr. 23, 2013), pp. 729-738, XP035366252, ISSN: 0178-2789, DOI: 1… [cited by applicant]
Yang, et al, ARTcode: Preserve art and code in any image, Proc. 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp. 904-915. [cited by applicant]
Zhu et al., “Unpaired Image to Image Translation using Cycle-Consistent Adversarial Networks”, 2017 IEEE International Conference on Computer Vision, Mar. 2017, pp. 2242-2251. [cited by applicant]