IP Library › Granted Patent US 10,387,730
Granted Patent B1
US 10,387,730 · App. 15/492,089 · Granted Aug 20, 2019

Augmented reality typography personalization system

Inventors: Piers Cowburn (London, GB); Qi Pan (London, GB); Eitan Pilipski (Los Angeles, CA)
Assignee: Snap Inc.
G06K9/00671G06T7/11G06K2209/01
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,387,730
App. No.
15/492,089
Filed
Apr 20, 2017
Granted
Aug 20, 2019
Kind
B1
Art Unit
2669
USPC
382/176
Abstract

Disclosed are augmented reality (AR) personalization systems to enable a user to edit and personalize presentations of real-world typography in real-time. The AR personalization system captures an image depicting a physical location via a camera coupled to a client device. For example, the client device may include a mobile device that includes a camera configured to record and display images (e.g., photos, videos) in real-time. The AR personalization system causes display of the image at the client device, and scans the image to detect occurrences of typography within the image (e.g., signs, billboards, posters, graffiti).

Claims (105)

1. A method comprising:

capturing an image at a client device, the image depicting a physical location;

analyzing the image depicting the physical location to identify an occurrence of typography at a location within the image depicting the physical location, the occurrence of the typography comprising at least a text string, and a background pattern;

detecting properties of the text string, the properties of the text string including a character-set, a length, and a height;

identifying a first character and a second character that make up the character-set, the first character at a first position in the text string, and the second character at a second position in the text string;

receiving a request to remove the first character from the text string;

generating an updated text string based on the request to remove the first character from the text string, the updated text string including at least the second character; and

generating a presentation of the image at the client device, the presentation of the image including a display of the updated text string at the location of the occurrence of typography within the image depicting the physical location, the display of the updated text string comprising the background pattern at the first position of the text string previously occupied by the first character, and the second character at the second position of the text string.

2. The method of claim 1 , wherein the properties of the text string further include a typeface, and the method further comprises:

receiving a request to add a third character to the text string at a third position in the text string; and

updating the display of the updated text string in the presentation of the image at the client device to include the third character at the third position.

3. The method of claim 1 , wherein the properties of the text string further include a typeface, and the generating the presentation of the image at the client device includes:

comparing the text string against a catalogue of characters based on the typeface of the text string;

identifying a distortion of the text string based on the comparing;

determining a shape of a surface of the text string based on the distortion; and

wherein the generating the presentation of the image is based on the shape of the surface.

4. The method of claim 1 , wherein the client device is a first client device, and the method further comprises:

retrieving location data of the physical location depicted by the image;

geo-tagging the request to remove the first character from the text string based on the location data;

detecting a second client device at the physical location;

causing display of the presentation of the image at the second client device in response to the detecting the second client device at the physical location, wherein the presentation of the image includes the background pattern at the first position of the text string in the presentation of the image, and the second character at the second position in the text string of the presentation of the image.

5. The method of claim 1 , wherein the character-set is a first character-set, and the method further comprises:

receiving a request to replace the first character-set with a second character-set;

removing the first character-set from the presentation of the image in response to the receiving the request to replace the first character-set with the second character-set; and

updating the presentation of the image to include the second character-set.

6. The method of claim 1 , wherein the text string is a first text string, the character-set is a first character-set, and the receiving the request to remove the first character from the text string includes:

detecting the client device at the physical location;

accessing a user profile associated with a user of the client device in response to the detecting the client device at the physical location;

retrieving a second character-set from the user profile of the user, the second character-set including a username of the user;

generating a second text string based on the second character set and the properties of the first text string; and

wherein the updating the presentation of the image includes replacing the first text string at the location within the image with the second text string.

7. The method of claim 1 , wherein the method further comprises:

overlaying a text field over the text string within the portion of the presentation, the text field having dimensions based on at least the properties of the text string; and

wherein the receiving the request to remove the first character from the text string includes receiving a user input to delete the first character from the text string into the text field.

8. The method of claim 1 , wherein the client device is a first client device, the character-set is a first character-set, and the method further comprises:

receiving a second character-set from a second client device, the second character-set to replace the first character-set in the presentation of the image;

assigning the second character-set location coordinates of the physical location; and

wherein the causing display of the presentation of the image at the client device includes:

detecting the first client device at the physical location;

retrieving the second character-set in response to the detecting the first client device at the physical location; and

causing display of the presentation of the image at the first client device, wherein the presentation of the image includes the second character-set.

9. The method of claim 1 , wherein the client device is a first client device, and the method further comprises:

generating a message that includes the updated presentation of the image; and

transmitting the message to a second client device.

10. The method of claim 1 , wherein the image is a first image, the physical location is a first physical location, the client device is a first client device, and the method further comprises:

assigning the request to remove the first character from the text string to the text string;

capturing a second image at a second client device, the second image depicting a second physical location, wherein the second physical location includes an occurrence of the text string;

identifying the text string within the second image depicting the second physical location in response to the capturing the second image;

retrieving the request to remove the first character from the text string in response to the identifying the text string; and

causing display of a presentation of the second image at the second client device based on the request to remove the first character from the text string.

11. A system comprising:

a memory; and

at least one hardware processor couple to the memory and comprising instructions that cause the system to perform operations comprising:

capturing an image at a client device, the image depicting a physical location;

analyzing the image depicting the physical location to identify an occurrence of typography at a location within the image depicting the physical location, the occurrence of the typography comprising at least a text string, and a background pattern;

detecting properties of the text string, the properties of the text string including a character-set, a length, and a height;

identifying a first character and a second character that make up the character-set, the first character at a first position in the text string, and the second character at a second position in the text string;

receiving a request to remove the first character from the text string;

generating an updated text string based on the request to remove the first character from the text string, the updated text string including at least the second character; and

generating a presentation of the image at the client device, the presentation of the image including a display of the updated text string at the location of the occurrence of typography within the image depicting the physical location, the display of the updated text string comprising the background pattern at the first position of the text string previously occupied by the first character, and the second character at the second position of the text string.

12. The system of claim 11 , wherein the properties of the text string further include a typeface, and the instructions cause the system to perform operations further comprising:

receiving a request to add a third character to the text string at a third position in the text string; and

updating the display of the updated text string in the presentation of the image at the client device to include the third character at the third position.

13. The system of claim 11 , wherein the client device is a first client device, and the instructions cause the system to perform operations further comprising:

retrieving location data of the physical location depicted by the image;

geo-tagging the request to remove the first character from the text string based on the location data;

detecting a second client device at the physical location;

causing display of the presentation of the image at the second client device in response to the detecting the second client device at the physical location, wherein the presentation of the image includes the background pattern at the first position of the text string and the second character at the second position of the text string.

14. The system of claim 11 , wherein the character-set is a first character-set, and the instructions cause the system to perform operations further comprising:

receiving a request to replace the first character-set with a second character-set;

removing the first character-set from the presentation of the image in response to the receiving the request to replace the first character-set with the second character-set; and

updating the presentation of the image to include the second character-set.

15. The system of claim 11 , wherein the text string is a first text string, the character-set is a first character-set, and the receiving the request to remove the first character from the text string includes:

detecting the client device at the physical location;

accessing a user profile associated with a user of the client device in response to the detecting the client device at the physical location;

retrieving a second character-set from the user profile of the user, the second character-set including a username of the user;

generating a second text string based on the second character set and the properties of the first text string; and

wherein the updating the presentation of the image includes replacing the first text string with the second text string.

16. The system of claim 11 , wherein the instructions cause the system to perform operations further comprising:

overlaying a text field over the text string within the portion of the presentation, the text field having dimensions based on at least the properties of the text string; and

wherein the receiving the request to remove the first character from the text string includes receiving a user input to delete the first character from the text string into the text field.

17. The system of claim 11 , wherein the client device is a first client device, the character-set is a first character-set, and the instructions cause the system to perform operations further comprising:

receiving a second character-set from a second client device, the second character-set to replace the first character-set in the presentation of the image;

assigning the second character-set location coordinates of the physical location; and

wherein the causing display of the presentation of the image at the client device includes:

detecting the first client device at the physical location;

retrieving the second character-set in response to the detecting the first client device at the physical location; and

causing display of the presentation of the image at the first client device, wherein the presentation of the image includes the second character-set.

18. The system of claim 11 , wherein the client device is a first client device, and the instructions cause the system to perform operations further comprising:

generating a message that includes the updated presentation of the image; and

transmitting the message to a second client device.

19. The system of claim 11 , wherein the image is a first image, the physical location is a first physical location, the client device is a first client device, and the instructions cause the system to perform operations further comprising:

assigning the request to remove the first character from the text string to the text string;

capturing a second image at a second client device, the second image depicting a second physical location, wherein the second physical location includes an occurrence of the text string;

identifying the text string within the second image depicting the second physical location in response to the capturing the second image;

retrieving the request to remove the first character from the text string in response to the identifying the text string; and

causing display of a presentation of the second image at the second client device based on the request to remove the first character from the text string.

20. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations including:

capturing an image at a client device, the image depicting a physical location;

analyzing the image depicting the physical location to identify an occurrence of typography at a location within the image depicting the physical location the occurrence of the typography comprising at least a text string, and a background pattern;

detecting properties of the text string, the properties of the text string including a character-set, a length, and a height;

identifying a first character and a second character that make up the character-set, the first character at a first position in the text string, and the second character at a second position in the text string;

receiving a request to remove the first character from the text string;

generating an updated text string based on the request to remove the first character from the text string, the updated text string including at least the second character; and

generating a presentation of the image at the client device, the presentation of the image including a display of the updated text string at the location of the occurrence of typography within the image depicting the physical location, the display of the updated text string comprising the background pattern at the first position of the text string previously occupied by the first character, and the second character at the second position of the text string.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 1, 2019
From: COWBURN, PIERS; PAN, QI; PILIPSKI, EITAN
To: SNAP INC.
Reel/Frame 049644/0045 →
Cited By (16)
US 12,192,426 US 12,197,884 US 12,211,159 US 12,217,374 US 12,266,062 US 12,292,299 US 12,340,475 US 12,347,045 US 12,361,652 US 12,387,436 US 12,394,127 US 12,443,325 US 12,444,138 US 12,488,548 US 12,513,254 US 12,541,929