IP Library Granted Patent US 12,347,151
Granted Patent B2
US 12,347,151 · App. 18/382,729 · Granted Jul 1, 2025

Encoding and decoding a stylized custom graphic

Inventors: Drake Austin Rehfeld (Glendora, CA); Rahul Bhupendra Sheth (Los Angeles, CA); Ning Zhang (Los Angeles, CA)
Assignee: Snap Inc.
G06T9/002G06N3/047H04L51/52
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,347,151
App. No.
18/382,729
Granted
Jul 1, 2025
Kind
B2
Abstract

An example system includes an encoder configured to receive a bit string and encode the bit string into a visual representation, and a decoder configured to receive an image including the visual representation and decode the bit string from the visual representation. In some examples, the encoder and decoder are trained as a pair by obtaining a training bit string, encoding the training bit string into a training visual representation using the encoder, decoding the training visual representation using the decoder to generate a decoded bit string, determining an error between the training bit string and the decoded bit string, and updating parameters of the encoder and decoder to reduce the error.

Claims (30)

1. A system comprising:

an encoder configured to receive a bit string and encode the bit string into a visual representation; and

a decoder configured to receive an image including the visual representation and decode the bit string from the visual representation; wherein the encoder and decoder are trained as a pair by:

obtaining a training bit string;

encoding the training bit string into a training visual representation using the encoder;

decoding the training visual representation using the decoder to generate a decoded bit string;

determining an error between the training bit string and the decoded bit string; and

updating parameters of the encoder and decoder to reduce the error.

2. The system of claim 1 , wherein the encoder and decoder comprise convolutional neural networks.

3. The system of claim 1 , wherein updating the parameters of the encoder and decoder comprises backpropagating the error and updating weights of the encoder and decoder.

4. The system of claim 1 , wherein encoding the bit string comprises converting the bit string to image data.

5. The system of claim 1 , wherein decoding the visual representation comprises extracting content information while ignoring style information.

6. The system of claim 1 , wherein the error is a loss function determined based on a difference between each bit in the training bit string and each corresponding bit in the decoded bit string.

7. The system of claim 6 , wherein the loss function is backpropagated to update the parameters.

8. The system of claim 1 , wherein updating the parameters minimizes the error.

9. The system of claim 1 , wherein the encoder is configured to encode the bit string into image data.

10. The system of claim 1 , wherein the visual representation is a grayscale image.

11. A method comprising:

obtaining a training bit string; encoding the training bit string into a training visual representation using an encoder; decoding the training visual representation using a decoder to generate a decoded bit string; determining an error between the training bit string and the decoded bit string; and

updating parameters of the encoder and decoder to reduce the error.

12. The method of claim 11 , wherein the encoder and decoder comprise convolutional neural networks.

13. The method of claim 11 , wherein updating the parameters comprises backpropagating the error and updating weights of the encoder and decoder.

14. The method of claim 11 , wherein encoding the bit string comprises converting the bit string to image data.

15. The method of claim 11 , wherein the error is a loss function determined based on a difference between each bit in the training bit string and each corresponding bit in the decoded bit string.

16. The method of claim 15 , wherein updating the parameters comprises backpropagating the loss function.

17. The method of claim 11 , wherein updating the parameters minimizes the error.

18. The method of claim 11 , wherein the visual representation is a grayscale image.

19. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: obtaining a training bit string; encoding the training bit string into a training visual representation using an encoder; decoding the training visual representation using a decoder to generate a decoded bit string; determining an error between the training bit string and the decoded bit string; and

updating parameters of the encoder and decoder to reduce the error.

20. The non-transitory machine-readable medium of claim 19 , wherein the encoder and decoder comprise convolutional neural networks trained as a pair.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 15, 2025
From: REHFELD, DRAKE AUSTIN; SHETH, RAHUL BHUPENDRA; ZHANG, NING
To: SNAP INC.
Reel/Frame 071127/0223 →
Continuity (5)
Continuation 18128128 · Mar 29, 2023
Continuation 17302361 · Apr 30, 2021
Continuation 16846949 · Apr 13, 2020
Continuation 16022536 · Jun 28, 2018
Related Publication 20240054687A1 · Feb 15, 2024
References Cited (37)
US 7593961B2 · Eguchi et al. · 2009 [cited by applicant]
US 10657676B1 · Rehfeld et al. · 2020 [cited by applicant]
US 11024058B2 · Rehfeld et al. · 2021 [cited by applicant]
US 11670012B2 · Rehfeld et al. · 2023 [cited by applicant]
US 11887344B2 · Rehfeld et al. · 2024 [cited by applicant]
US 20010016051A1 · Rhoads · 2001 [cited by applicant]
US 20160328630A1 · Han et al. · 2016 [cited by applicant]
US 20180091819A1 · Cook et al. · 2018 [cited by applicant]
US 20190108651A1 · Gu · 2019 [cited by examiner]
US 20190138860A1 · Liu et al. · 2019 [cited by applicant]
US 20190370638A1 · Green · 2019 [cited by examiner]
US 20200242812A1 · Rehfeld et al. · 2020 [cited by applicant]
US 20210256736A1 · Rehfeld et al. · 2021 [cited by applicant]
US 20230237706A1 · Rehfeld et al. · 2023 [cited by applicant]
“U.S. Appl. No. 16/022,536, Corrected Notice of Allowability mailed Feb. 28, 2020”, 6 pgs. [cited by applicant]
“U.S. Appl. No. 16/022,536, Notice of Allowance mailed Jan. 14, 2020”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 16/846,949, Non Final Office Action mailed Sep. 24, 2020”, 14 pgs. [cited by applicant]
“U.S. Appl. No. 16/846,949, Notice of Allowance mailed Feb. 3, 2021”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 16/846,949, Response filed Dec. 17, 20 to Non Final Office Action mailed Sep. 24, 2020”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 17/302,361, Examiner Interview Summary mailed Jan. 9, 2023”, 2 pgs. [cited by applicant]
“U.S. Appl. No. 17/302,361, Final Office Action mailed Nov. 25, 2022”, 17 pgs. [cited by applicant]
“U.S. Appl. No. 17/302,361, Non Final Office Action mailed Sep. 20, 2022”, 16 pgs. [cited by applicant]
“U.S. Appl. No. 17/302,361, Notice of Allowance mailed Jan. 24, 2023”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 17/302,361, Response filed Jan. 6, 2023 to Final Office Action mailed Nov. 25, 2022”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 17/302,361, Response filed Nov. 11, 2022 to Non Final Office Action mailed Sep. 20, 2022”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 18/128,128, Corrected Notice of Allowability mailed Sep. 21, 2023”, 2 pgs. [cited by applicant]
“U.S. Appl. No. 18/128,128, Non Final Office Action mailed Aug. 17, 2023”, 22 pgs. [cited by applicant]
“U.S. Appl. No. 18/128,128, Notice of Allowance mailed Sep. 13, 2023”, 10 pgs. [cited by applicant]
“U.S. Appl. No. 18/128,128, Response filed Aug. 28, 2023 to Non Final Office Action mailed Aug. 17, 2023”, 10 pgs. [cited by applicant]
Dosovitskuy, Alexey, et al., “Learning to Generate Chairs with Convolutional Neural Networks”, Conference on Computer Vision and Pattern Recognition (CVPR), (2015), 9 pgs. [cited by applicant]
Gatys, Leon A., et al., “Image Style Transfer Using Convolutional Neural Networks”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, (Jun. 27-30, 2016), 2414-2423. [cited by applicant]
Grinchuk, Oleg, et al., “Learnable Visual Markers”, Skolkovo Institute of Science and Technology, Moscow, Russia, (2016), 9 pgs. [cited by applicant]
Johnson, Justin, et al., “Perceptual Losses for Real-Time Style Transfer and Super-Resolution”, arXiv preprint, arXiv:1603.08155v1, (Mar. 27, 2016), 18 pgs. [cited by applicant]
Kingma, Diederik P, et al., “Auto-Encoding Variational Bayes”, arXiv:1312.6114v10, (2014), 14 pgs. [cited by applicant]
Lecun, Y., et al., “Backpropagation Applied to Handwritten Zip Code Recognition”, Neural Computation, 1(4), (1989), 541-555. [cited by applicant]
Zeiler, Matthew D., et al., “Adaptive Deconvolutional Networks for Mid and High Level Feature Learning”, International Conference on Computer Vision (ICCV), (2011), 8 pgs. [cited by applicant]
“U.S. Appl. No. 18/128,128, 312 Amendment filed Dec. 12, 2023”, 3 pgs. [cited by applicant]