IP Library › Granted Patent US 11,265,531
Granted Patent B2
US 11,265,531 · App. 17/106,262 · Granted Mar 1, 2022

Systems and methods of creating a three-dimensional virtual image

Inventor: Marc Saunders (Aurora, CO)
Assignee: HashD, Inc.
H04N13/243H04N13/117H04N13/15H04N13/207H04N13/246G06F3/017H04N2013/0077
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,265,531
App. No.
17/106,262
Granted
Mar 1, 2022
Kind
B2
Abstract

Aspects are generally directed toward creating a three-dimensional virtual model by a user identifying a three-dimensional object. Next, the system captures a plurality of two-dimensional images of the object in succession, the plurality of images being captured from different orientations, and recording the plurality of images on a storage medium. Then the system determines the relative change in position of the plurality of images by comparing two subsequent images, wherein the relative change is determined by a difference in color intensity values between the pixels of one image and another image, generates a plurality of arrays from the difference determined and generates a computer image from the plurality of arrays, wherein the computer image represents the three-dimensional object.

Claims (61)

1. A method of generating a model of an object comprising:

obtaining, sequentially, a plurality of two-dimensional images of the object with a movable camera, the plurality of two-dimensional images being captured from different orientations of the camera relative to the object;

recording, while obtaining the plurality of two-dimensional images of the object, positional changes of the camera for each of the plurality of two-dimensional images being captured from different orientations, the positional changes including: an angle between each of the plurality of two-dimensional images and a distance the camera moves between each of the plurality of two-dimensional images;

storing the plurality of two-dimensional images and the positional changes into memory;

determining a relative change in position of the object within the plurality of two-dimensional images by comparing two images of the plurality of two-dimensional images, wherein the relative change is determined by evaluating a difference in color intensity values between pixels of one of the two-dimensional images and another of the two-dimensional images, the comparison including performing repeated shifting of one of the two-dimensional images relative to another, the relative change used to determine a distance from the camera to the object;

generating a three-dimensional coordinate based on the relative change in position; and

outputting a computer image including the three-dimensional coordinate, wherein the computer image represents the object.

2. The method according to claim 1 , wherein the plurality of two-dimensional images of the object are obtained using a single camera.

3. The method according to claim 2 , further comprising:

receiving orientation information for each two-dimensional image of the plurality of two-dimensional images; and

generating the three-dimensional coordinate based on the relative change in position and the received orientation information.

4. The method according to claim 3 , wherein the orientation information is received from a gyroscope.

5. The method according to claim 4 , further comprising:

subdividing the two-dimensional images of the plurality of two-dimensional images into a plurality of sub-images; and

for each sub-image of the plurality of sub-images:

determining a relative change in position of a first sub-image from a first of the two-dimensional images with respect to a first sub-image from a second of the two-dimensional images,

wherein the relative change is determined by the difference in color intensity values between pixels of the first sub-image from the first of the two-dimensional images and the pixels of the first sub-image from the second of the two-dimensional images, and

generating the three-dimensional coordinate based on the relative change in position of the first sub-image from the first two-dimensional image and the first sub-image from the second two-dimensional image.

6. The method according to claim 5 , wherein each of the two-dimensional images are subdivided into four sub-images.

7. The method according to claim 4 , further comprising:

determining a similarity between the one of the two-dimensional images and the other the two-dimensional image; and

maximizing the similarity between the one of the two-dimensional images and the other two-dimensional images by shifting in at least one of a horizontal or vertical direction, the one of the two-dimensional images with respect to the other image, wherein the relative change is determined by creating a difference array, the difference array including differences in color intensity values between pixels of the one of the two-dimensional images and the other image, wherein the one of the two-dimensional images is shifted in at least one of a horizontal or vertical direction with respect to the other image.

8. A device comprising:

a display;

a camera;

a processor configured to communicate with the display and camera; and

storage configured to store instructions that when executed by the processor, cause the processor to:

obtain, sequentially, a plurality of two-dimensional images of the object with a movable camera, the plurality of two-dimensional images being captured from different orientations of the camera relative to the object;

record, while obtaining the plurality of two-dimensional images of the object, positional changes of the camera for each of the plurality of two-dimensional images being captured from different orientations, the positional changes including: an angle between each of the plurality of two-dimensional images and a distance the camera moves between each of the plurality of two-dimensional images;

store the plurality of two-dimensional images and the positional changes in the storage;

determine a relative change in position of the object within the plurality of two-dimensional images by comparing two images of the plurality of two-dimensional images, wherein the relative change is determined by evaluating a difference in color intensity values between pixels of one of the two-dimensional images and another of the two-dimensional images, the comparison including performing repeated shifting of one of the two-dimensional images relative to another, the relative change used to determine a distance from the camera to the object;

generate a three-dimensional coordinate based on the relative change in position; and

output a computer image including the three-dimensional coordinate, wherein the computer image represents a virtual three-dimensional model of the object.

9. The device according to claim 8 , wherein the plurality of two-dimensional images of the object are obtained using a single camera of the mobile device.

10. The device according to claim 9 , wherein the one or more memories include one or more instructions that when executed by the processor, cause the processor to:

receive orientation information for each two-dimensional image of the plurality of two-dimensional images; and

generate the three-dimensional coordinate based on the relative change in position and the received orientation information.

11. The device according to claim 10 , wherein the orientation information is received from a gyroscope.

12. The mobile device according to claim 11 , wherein the one or more memories include one or more instructions that when executed by the processor, cause the processor to:

subdivide the two-dimensional images of the plurality of two-dimensional images into a plurality of sub-images; and

for each sub-image of the plurality of sub-images, determine a relative change in position of a first sub-image from a first of the two-dimensional images with respect to a first sub-image from a second of the two-dimensional images, wherein the relative change is determined by the difference in color intensity values between pixels of the first sub-image from the first two-dimensional image and the pixels of the first sub-image from the second of the two-dimensional images, and generate the three-dimensional coordinate based on the relative change in position of the first sub-image from the first two-dimensional image and the first sub-image from the second two-dimensional image.

13. The mobile device according to claim 12 , wherein each of the two-dimensional images of the plurality of two-dimensional images are each subdivided into four sub-images.

14. The mobile device according to claim 11 , wherein the one or more memories include one or more instructions that when executed by the processor, cause the processor to:

determine a similarity between the one of the two-dimensional images and the other two-dimensional image; and

maximize the similarity between the one of the two-dimensional images and the other two-dimensional image by shifting in at least one of a horizontal or vertical direction, the one of the two-dimensional images with respect to the other two-dimensional image, wherein the relative change is determined by creating a difference array, the difference array including differences in color intensity values between pixels of the one two-dimensional image and the other two-dimensional image, wherein the one of the two-dimensional images is shifted in at least one of a horizontal or vertical direction with respect to the other two-dimensional images.

15. A non-transitory computer-readable information storage media having stored thereon instructions that cause a processor to execute a method, the instructions comprising:

instructions to obtain, sequentially, a plurality of two-dimensional images of the object with a movable camera, the plurality of two-dimensional images being captured from different orientations of the camera relative to the object;

instructions to record, while obtaining the plurality of two-dimensional images of the object, positional changes of the camera for each of the plurality of two-dimensional images being captured from different orientations, the positional changes including: an angle between each of the plurality of two-dimensional images and a distance the camera moves between each of the plurality of two-dimensional images;

instructions to store the plurality of two-dimensional images and the positional changes in the storage;

instructions to determine a relative change in position of the object within the plurality of two-dimensional images by comparing two images of the plurality of two-dimensional images, wherein the relative change is determined by evaluating a difference in color intensity values between pixels of one of the two-dimensional images and another of the two-dimensional images, the comparison including performing repeated shifting of one of the two-dimensional images relative to another, the relative change used to determine a distance from the camera to the object;

instructions to generate a three-dimensional coordinate based on the relative change in position; and

instructions to output a computer image including the three-dimensional coordinate, wherein the computer image represents a virtual three-dimensional model of the object.

16. The non-transitory computer-readable information storage media of claim 15 , wherein the plurality of two-dimensional images of the are obtained using a single camera of a device.

17. The non-transitory computer-readable information storage media of claim 16 , further comprising:

instructions to receive orientation information for each two-dimensional image of the plurality of two-dimensional images; and

instructions to generate the three-dimensional coordinate based on the relative change in position and the received orientation information.

18. The non-transitory computer-readable information storage media of claim 17 , wherein the orientation information is received from a gyroscope of a mobile device.

19. The non-transitory computer-readable information storage media of claim 17 , further comprising:

instructions to subdivide the two-dimensional images of the plurality of two-dimensional images into a plurality of sub-images; and

instructions to determine a relative change in position of a first sub-image from a first of the two-dimensional images with respect to a first sub-image from a second of the two-dimensional images, wherein the relative change is determined by the difference in color intensity values between pixels of the first sub-image from the first of the two-dimensional images and the pixels of the first sub-image from the second of the two-dimensional images, and generate the three-dimensional coordinate based on the relative change in position of the first sub-image from the first of the two-dimensional images and the first sub-image from the second of the two-dimensional images.

20. The non-transitory computer-readable information storage media of claim 19 , wherein the two-dimensional images of the plurality of two-dimensional images are each subdivided into four sub-images.

Continuity (3)
Continuation 15268526 · Sep 16, 2016
Provisional Application 62219583 · Sep 16, 2015
Related Publication 20210084285A1 · Mar 18, 2021