IP Library › Granted Patent US 10,509,533
Granted Patent B2
US 10,509,533 · App. 14/275,523 · Granted Dec 17, 2019

Systems and methods of generating augmented reality (AR) objects

Inventors: Raphael Grasset (Vienna, AT); Hartmut Seichter (Erfurt, DE)
Assignee: QUALCOMM Incorporated
G06F3/04815G06F3/04883
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,509,533
App. No.
14/275,523
Granted
Dec 17, 2019
Kind
B2
Abstract

Methods, systems, computer-readable media, and apparatuses for generating an Augmented Reality (AR) object are presented. The method may include capturing an image of one or more target objects, wherein the one or more target objects are positioned on a pre-defined background. The method may also include segmenting the image into one or more areas corresponding to the one or more target objects and one or more areas corresponding to the pre-defined background. The method may additionally include converting the one or more areas corresponding to the one or more target objects to a digital image. The method may further include generating one or more AR objects corresponding to the one or more target objects, based at least in part on the digital image.

Claims (51)

1. A method for generating one or more augmented reality (AR) objects by a mobile device, the method comprising:

capturing, by a camera of the mobile device based on instructions from an application executed on the mobile device, a digital image of one or more physical objects;

segmenting the digital image into one or more areas corresponding to the one or more physical objects and one or more areas corresponding to a background;

generating, from the digital image, one or more digital representations of the one or more physical objects in the digital image, the one or more digital representations of the one or more physical objects being generated using data from the digital image in the one or more areas corresponding to the one or more physical objects;

converting, by a processor of the mobile device based on instructions from the application, the one or more digital representations of the one or more physical objects in the digital image to one or more two-dimensional augmented reality (AR) objects, wherein converting the one or more digital representations of the one or more physical objects to the one or more two-dimensional AR objects causes the one or more digital representations to be editable versions of the one or more physical objects;

dividing a two-dimensional AR object of the one or more two-dimensional AR objects into a plurality of parts;

editing a geometry of the two-dimensional AR object based on a user input, the editing including at least one of adding a depth effect to at least one part of the two-dimensional AR object and manipulating a designated part of the plurality of parts based on the user input, the added depth effect causing the at least one part to appear to have more than two dimensions when displayed;

augmenting a target image using the one or more two-dimensional AR objects; and

displaying the augmented target image on the mobile device based on instructions from the application.

2. The method of claim 1 , wherein segmenting the digital image further comprises subtracting the background from the digital image.

3. The method of claim 1 , wherein an amount of the depth effect added to the at least one part of the two-dimensional AR object is based on a determined finger pressure on a display of the mobile device.

4. The method of claim 1 , wherein one of the one or more physical objects comprises a two-dimensional sketch.

5. The method of claim 1 , wherein one of the one or more physical objects comprises a three-dimensional object.

6. The method of claim 1 , wherein at least one of the one or more physical objects is coplanar with the background.

7. The method of claim 1 , wherein the background is a feature tracking target.

8. The method of claim 1 , further comprising refining one of the one or more areas corresponding to the one or more physical objects.

9. The method of claim 1 , wherein the one or more two-dimensional AR objects are generated based on detection of a two-dimensional blob within the captured digital image, the detection of the two-dimensional blob being based on a surface reflectance associated with the one or more physical objects positioned on the background.

10. The method of claim 1 , wherein the background includes a horizontal tracking area and a vertical tracking area.

11. The method of claim 1 , further comprising displaying, on a display of the mobile device, the one or more two-dimensional AR objects in real-time.

12. An apparatus for generating one or more Augmented Reality (AR) objects, comprising:

memory;

an image capturing module including an image sensor, the image capturing module configured to capture, based on instructions from an application executed on the apparatus, a digital image of one or more physical objects; and

one or more processors coupled to the memory and the image capturing module, the one or more processors configured to:

segment the digital image into one or more areas corresponding to the one or more physical objects and one or more areas corresponding to a background;

generate, from the digital image, one or more digital representations of the one or more physical objects in the digital image, the one or more digital representations of the one or more physical objects being generated using data from the digital image in the one or more areas corresponding to the one or more physical objects;

convert, based on instructions from the application, the one or more digital representations of the one or more physical objects in the digital image to one or more two-dimensional augmented reality (AR) objects, wherein converting the one or more digital representations of the one or more physical objects to the one or more two-dimensional AR objects causes the one or more digital representations to be editable versions of the one or more physical objects;

divide a two-dimensional AR object of the one or more two-dimensional AR objects into a plurality of parts;

edit a geometry of the two-dimensional AR object based on a user input, the editing including at least one of adding a depth effect to at least one part of the two-dimensional AR object and manipulating a designated part of the plurality of parts based on the user input, the added depth effect causing the at least one part to appear to have more than two dimensions when displayed;

augment a target image using the one or more two-dimensional AR objects; and

display the augmented target image on the apparatus based on instructions from the application.

13. The apparatus of claim 12 , wherein segmenting the digital image further comprises subtracting the background from the digital image.

14. The apparatus of claim 12 , wherein an amount of the depth effect added to the at least one part of the two-dimensional AR object is based on a determined finger pressure on a display.

15. The apparatus of claim 12 , wherein one of the one or more physical objects comprises at least one or more of a two-dimensional sketch or a three-dimensional object.

16. The apparatus of claim 12 , wherein at least one of the one or more physical objects is coplanar with the background.

17. The apparatus of claim 12 , wherein the background is a feature tracking target.

18. The apparatus of claim 12 , wherein the one or more processors are further configured to refine one of the one or more areas corresponding to the one or more physical objects.

19. The apparatus of claim 12 , wherein the one or more two-dimensional AR objects are generated based on detection of a two-dimensional blob within the captured digital image, the detection of the two-dimensional blob being based on a surface reflectance associated with the one or more physical objects positioned on the background.

20. The apparatus of claim 12 , wherein the background includes a horizontal tracking area and a vertical tracking area.

21. The apparatus of claim 12 , further comprising a display device configured to display the one or more two-dimensional AR objects in real-time.

22. One or more non-transitory computer-readable media storing computer-executable instructions for generating one or more Augmented Reality (AR) objects that, when executed, cause one or more computing devices included in a mobile device to:

capture, by a camera of the mobile device based on the computer-executable instructions being executed on the mobile device, a digital image of one or more physical objects;

segment the digital image into one or more areas corresponding to the one or more physical objects and one or more areas corresponding to a background;

generate, from the digital image, one or more digital representations of the one or more physical objects in the digital image, the one or more digital representations of the one or more physical objects being generated using data from the digital image in the one or more areas corresponding to the one or more physical objects; and

convert, by the one or more computing devices based on the computer-executable instructions being executed on the mobile device, the one or more digital representations of the one or more physical objects in the digital image to one or more two-dimensional augmented reality (AR) objects, wherein converting the one or more digital representations of the one or more physical objects to the one or more two-dimensional AR objects causes the one or more digital representations to be editable versions of the one or more physical objects;

divide a two-dimensional AR object of the one or more two-dimensional AR objects into a plurality of parts;

edit a geometry of the two-dimensional AR object based on a user input, the editing including at least one of adding a depth effect to at least one part of the two-dimensional AR object and manipulating a designated part of the plurality of parts based on the user input, the added depth effect causing the at least one part to appear to have more than two dimensions when displayed;

augment a target image using the one or more two-dimensional AR objects; and

display the augmented target image on the mobile device based on the computer-executable instructions being executed on the mobile device.

23. The method of claim 1 , wherein the added depth effect causes the part to appear to have two and a half dimensions.

24. The method of claim 1 , wherein the one or more AR objects are generated in real-time by the mobile device.

25. The method of claim 1 , wherein the one or more AR objects are generated in real-time by the mobile device without using an electronic device other than the mobile device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 19, 2014
From: GRASSET, RAPHAEL; SEICHTER, HARTMUT
To: QUALCOMM INCORPORATED
Reel/Frame 033142/0312 →
Continuity (2)
Provisional Application 61823295 · May 14, 2013
Related Publication 20140344762A1 · Nov 20, 2014
Cited By (3)
US 12,469,229 US 12,483,403 US 12,718,506