IP Library Granted Patent US 12,096,112
Granted Patent B2
US 12,096,112 · App. 17/831,305 · Granted Sep 17, 2024

User interface to select field of view of a camera in a smart glass

Inventors: Sapna Shroff (Burlingame, CA); Sebastian Sztuk (Menlo Park, CA); Johana Gabriela Coyoc Escudero (Union City, CA); Jun Hu (San Jose, CA)
Assignee: Meta Platforms Technologies, LLC
H04N23/64G02B27/0172G02C7/101G02C7/16G06F3/013G06F3/016G06F3/017G06F3/044G06F3/167G06F40/205G06V10/25G06V10/759G06V20/50G10L17/06G10L17/22H04N23/56H04N23/62H04N23/63H04N23/66H04N23/695H04N23/90G02B2027/0138G02B2027/0141G02B2027/0178
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,096,112
App. No.
17/831,305
Granted
Sep 17, 2024
Kind
B2
Abstract

A wearable device for use in immersive reality applications is provided. The wearable device has a frame including an eyepiece to provide a forward-image to a user, a first forward-looking camera mounted on the frame, the first forward-looking camera having a field of view within the forward-image, a sensor configured to receive a command from the user, the command indicative of a region of interest within the field of view, and an interface device to indicate to the user that a field of view of the first forward-looking camera is aligned with the region of interest. Methods of use of the device, a memory storing instructions and a processor to execute the instructions to cause the device to perform the methods of use, are also provided.

Claims (31)

1. A device, comprising:

a frame including an eyepiece to provide a forward-image to a user;

a first forward-looking camera mounted on the frame, the first forward-looking camera having a first field of view within the forward-image;

a sensor configured to receive a command from the user, the command indicative of a region of interest within the forward-image;

a second forward-looking camera mounted on the frame, wherein the second forward-looking camera is activated when the first field of view of the first forward-looking camera is misaligned with the region of interest;

a first interface device to indicate to the user to alter a position of the user to cause an adjustment to an alignment between the region of interest and at least one of the first field of view of the first forward-looking camera and a second field of view of the second forward-looking camera; and

a second interface device to indicate to the user that the region of interest is aligned with at least one of the first field of view and the second field of view, wherein the second interface device is different from the first interface device.

2. The device of claim 1 , wherein the sensor includes a microphone configured to receive a voice command from the user.

3. The device of claim 1 , wherein the sensor includes a capacitive sensor configured to receive a touch gesture from the user.

4. The device of claim 1 , wherein the sensor is an eye tracking sensor, further comprising a processor configured to determine a gaze direction of the user based on a signal from the eye tracking sensor, and to identify the region of interest as a portion of the forward-image centered along the gaze direction.

5. The device of claim 1 , wherein the sensor is the first forward-looking camera and the command from the user is a hand gesture.

6. The device of claim 1 , further comprising a processor configured to receive a hand gesture from the user and identify the command indicative of the region of interest within the first field of view based on the hand gesture.

7. The device of claim 1 , further comprising a processor configured to identify, in an image from the first forward-looking camera, an object of interest based on the command from the user.

8. The device of claim 1 , wherein the command from the user includes activating the second forward-looking camera when the first field of view of the first forward-looking camera is misaligned with the region of interest.

9. The device of claim 1 , wherein the first forward-looking camera includes a motion actuator configured to adjust the first field of view of the first forward-looking camera based on the command from the user.

10. The device of claim 1 , wherein the eyepiece comprises a display configured to show the user the first field of view of the first forward-looking camera.

11. A computer-implemented method, comprising:

receiving, in a first camera device mounted on a smart glass system, a command from a user, the command being indicative of an area of interest in a forward-image viewed by the user;

determining a boundary of the area of interest based on the command;

activating a second camera device mounted on the smart glass system when a first field of view of the first forward-looking camera is misaligned with the area of interest;

providing, by a first interface device, a first feedback to indicate to the user to alter a position of the user to cause an adjustment to an alignment between the area of interest and at least one of the first field of view of the first camera device and a second field of view of the second camera device; and

providing, by a second interface device, a second feedback to the user based on an overlap between the area of interest and at least one of the first field of view and the second field of view, wherein the second interface device is different from the first interface device.

12. The computer-implemented method of claim 11 , wherein the command is a voice command and receiving the command comprises converting the voice command to a text command and parsing the text command for a keyword indicative of a user intention.

13. The computer-implemented method of claim 11 , wherein the command is a voice command, further comprising identifying the user based on a voice signature, and authenticating the voice command when the user is identified.

14. The computer-implemented method of claim 11 , wherein the command is a voice command indicating an object of interest in the forward-image displayed to the user, further comprising centering the first field of view of the first camera device around the object of interest.

15. The computer-implemented method of claim 11 , further comprising receiving a second command from the user responsive to the first feedback or the second feedback, the second command directed to activating the second camera device.

16. The computer-implemented method of claim 11 , wherein receiving the command from the user comprises identifying a hand gesture from the user with a hand gesture recognition model based on a learned history of recorded hand gestures from the user.

17. The computer-implemented method of claim 11 , wherein receiving the command from the user comprises receiving one of: a two-hand gesture marking opposite corners of the area of interest, a finger gesture delineating a border of the area of interest, a two-finger gesture forming a cross-hair indicating a center of the area of interest, or a circle gesture including the center of the area of interest.

18. The computer-implemented method of claim 11 , wherein providing the first feedback comprises activating an actuator to indicate to the user to move a head position to improve the alignment between the area of interest and the first field of view of the first camera device.

19. The computer-implemented method of claim 11 , wherein the command is a hand gesture, further comprising providing a feedback to the user when the hand gesture is not recognized by a hand gesture recognition model.

20. The computer-implemented method of claim 11 , wherein the command is a hand gesture from the user, further comprising providing a feedback to the user that the hand gesture is incomplete within the first field of view of the first camera device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 20, 2023
From: SHROFF, SAPNA; SZTUK, SEBASTIAN; ESCUDERO, JOHANA GABRIELA COYOC; HU, JUN
To: META PLATFORMS TECHNOLOGIES, LLC
Reel/Frame 062441/0631 →
Continuity (4)
Provisional Application 63227240 · Jul 29, 2021
Provisional Application 63227238 · Jul 29, 2021
Provisional Application 63227231 · Jul 29, 2021
Related Publication 20230031871A1 · Feb 2, 2023