Granted Patent B1

US 12,189,130 · App. 18/385,740 · Granted Jan 7, 2025

Inventors: Aldis Sipolins (Somerville, MA); Ning Xu (Irvine, CA)

Assignee: Adeia Imaging LLC

G02B27/0172G02B26/0833G02B27/0179H04N23/695H04N23/698G02B2027/0138

Abstract

Claims (76)

1. A method comprising:

receiving an indication to place an augmented reality (AR) object on a screen of a device comprising a static camera having a static view direction and at least one adjustable camera having an adjustable view direction;

capturing a first image using the static camera;

in response to determining that the first image lacks sufficient positioning cues for placement of the AR object:

moving the adjustable view direction of the at least one adjustable camera from a first direction to a second direction; and

capturing a second image using the at least one adjustable camera in the second direction; and

generating for display on the screen of the device the AR object in a position on the screen of the device determined at least in part based on positioning cues from the second image.

2. The method of claim 1 , wherein the adjustable view direction of the at least one adjustable camera is based on an orientation of a micro-electro-mechanical system (MEMS) mirror, and wherein moving the adjustable view direction of the at least one adjustable camera comprises:

controlling the MEMS mirror to change from a first orientation to a second orientation.

3. The method of claim 1 , further comprising determining the second direction of the at least one adjustable camera by moving the at least one adjustable camera in a random direction.

4. The method of claim 1 , further comprising:

determining that a combination of (i) the first image captured by the static camera and (ii) the second image captured by the at least one adjustable camera in the second direction includes sufficient positioning cues for placement of the AR object; and

storing the second direction.

5. The method of claim 4 , further comprising:

in response to receiving a second indication to place a second AR object on the screen of the device:

moving the adjustable view direction of the at least one adjustable camera to the stored second direction;

capturing a third image using the at least one adjustable camera directed in the stored second direction; and

generating for display the second AR object in a second position on the screen of the device determined at least in part based on positioning cues from the third image.

6. The method of claim 4 , further comprising:

moving the device from a first location to a second location;

determining a difference between the first location and the second location using dead reckoning based on one of inertial sensors or GPS; and

in response to receiving a second indication to place a second AR object on the screen of the device located in the second location:

moving the adjustable view direction of the at least one adjustable camera to a third direction, wherein the third direction is based on the stored second direction and the difference between the first location and the second location; and

capturing a third image using the at least one adjustable camera in the third direction; and

generating for display the second AR object in a second position on the screen of the device determined at least in part based on positioning cues from the third image.

7. The method of claim 1 , wherein moving the adjustable view direction of the at least one adjustable camera comprises:

sampling a plurality of different view directions of the at least one adjustable camera; and

storing the plurality of different view directions in a ranked list, wherein the plurality of different view directions are ranked based on a number of positioning cues detectable in images captured by the at least one adjustable camera when positioned in each respective view direction of the plurality of different view directions.

8. The method of claim 7 , wherein moving the adjustable view direction of the at least one adjustable camera further comprises:

selecting a best ranked view direction of the plurality of different view directions stored in the ranked list; and

moving the adjustable view direction of the at least one adjustable camera to the best ranked view direction.

9. The method of claim 1 , wherein moving the adjustable view direction of the at least one adjustable camera comprises:

rotating the adjustable view direction of the at least one adjustable camera between the second direction and a plurality of additional view directions;

capturing images using the at least one adjustable camera in each of the second direction and the plurality of additional view directions; and

in response to determining that a third direction of the plurality of additional view directions provides a greater number of positioning cues than the second direction, capturing the second image using the at least one adjustable camera in the third direction.

10. The method of claim 1 , further comprising:

determining that the first image captured by the static camera includes sufficient positional cues for positioning the AR object on the screen of the device; and

moving the adjustable view direction of the at least one adjustable camera such that a field of view of the at least one adjustable camera is outside the field of view of the static camera.

11. A system comprising:

input/output circuitry configured to receive an indication to place an augmented reality (AR) object on a screen of a device comprising a static camera having a static view direction and at least one adjustable camera having an adjustable view direction; and

control circuitry configured to:

capture a first image using the static camera;

in response to determining that the first image lacks sufficient positioning cues for placement of the AR object:

move the adjustable view direction of the at least one adjustable camera from a first direction to a second direction; and

capture a second image using the at least one adjustable camera in the second direction; and

generate for display on the screen of the device the AR object in a position on the screen of the device determined at least in part based on positioning cues from the second image.

12. The system of claim 11 , wherein the adjustable view direction of the at least one adjustable camera is based on an orientation of a micro-electro-mechanical system (MEMS) mirror, and wherein the control circuitry is further configured to move the adjustable view direction of the at least one adjustable camera by controlling the MEMS mirror to change from a first orientation to a second orientation.

13. The system of claim 11 , wherein the control circuitry is further configured to determine the second direction of the at least one adjustable camera by moving the at least one adjustable camera in a random direction.

14. The system of claim 11 , wherein the control circuitry is further configured to:

determine that a combination of (i) the first image captured by the static camera and (ii) the second image captured by the at least one adjustable camera in the second direction includes sufficient positioning cues for placement of the AR object; and

store the second direction.

15. The system of claim 14 , wherein the control circuitry is further configured to:

in response to receiving a second indication to place a second AR object on the screen of the device:

move the adjustable view direction of the at least one adjustable camera to the stored second direction;

capture a third image using the at least one adjustable camera directed in the stored second direction; and

generate for display the second AR object in a second position on the screen of the device determined at least in part based on positioning cues from the third image.

16. The system of claim 14 , wherein the control circuitry is further configured to:

move the device from a first location to a second location;

determine a difference between the first location and the second location using dead reckoning based on one of inertial sensors or GPS; and

in response to receiving a second indication to place a second AR object on the screen of the device located in the second location:

move the adjustable view direction of the at least one adjustable camera to a third direction, wherein the third direction is based on the stored second direction and the difference between the first location and the second location; and

capture a third image using the at least one adjustable camera in the third direction; and

generate for display the second AR object in a second position on the screen of the device determined at least in part based on positioning cues from the third image.

17. The system of claim 11 , wherein the control circuitry is further configured to move the adjustable view direction of the at least one adjustable camera by:

sampling a plurality of different view directions of the at least one adjustable camera; and

storing the plurality of different view directions in a ranked list, wherein the plurality of different view directions are ranked based on a number of positioning cues detectable in images captured by the at least one adjustable camera when positioned in each respective view direction of the plurality of different view directions.

18. The system of claim 17 , wherein the control circuitry is further configured to move the adjustable view direction of the at least one adjustable camera by:

selecting a best ranked view direction of the plurality of different view directions stored in the ranked list; and

moving the adjustable view direction of the at least one adjustable camera to the best ranked view direction.

19. The system of claim 11 , wherein the control circuitry is further configured to move the adjustable view direction of the at least one adjustable camera by:

rotating the adjustable view direction of the at least one adjustable camera between the second direction and a plurality of additional view directions;

capturing images using the at least one adjustable camera in each of the second direction and the plurality of additional view directions; and

in response to determining that a third direction of the plurality of additional view directions provides a greater number of positioning cues than the second direction, capturing the second image using the at least one adjustable camera in the third direction.

20. The system of claim 11 , wherein the control circuitry is further configured to:

determine that the first image captured by the static camera includes sufficient positional cues for positioning the AR object on the screen of the device; and

move the adjustable view direction of the at least one adjustable camera such that a field of view of the at least one adjustable camera is outside the field of view of the static camera.

Assignments (1)

References Cited (25)