IP Library Granted Patent US 11,765,460
Granted Patent B2
US 11,765,460 · App. 16/909,387 · Granted Sep 19, 2023

Digital zoom based on remote user instructions of live video

Inventors: Sanjay Jhawar (Menlo Park, CA); Ken Lustig (Mercer Island, WA)
Assignee: RealWear, Inc.
H04N23/69G02B27/017H04N1/00129H04N7/185H04N23/635H04N23/661H04N25/443H04N1/00408H04N23/631H04N25/46H04N25/61H04N2101/00H04N2201/0084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,765,460
App. No.
16/909,387
Granted
Sep 19, 2023
Kind
B2
Abstract

Systems and methods facilitate digital zoom of live video based on remote user instructions. A user of a remote device may desire to zoom in on a particular aspect of a copy of a full image captured by a sensor at a first resolution and communicated to the remote device at a second resolution less than the first resolution. A selection that corresponds to a user-selected area of the copy and corresponding to a portion of the sensor is received by the sensor. Based on the selection, an instruction is generated to capture a second electronic image utilizing the portion of the sensor with the first resolution. Thus, rather than zooming into the first electronic image at the second resolution, the second electronic image corresponding to the portion of the sensor is captured at the first resolution and then converted to the second resolution, resulting in a higher resolution zoom.

Claims (51)

1. A computer-implemented method comprising:

capturing, by a head-mounted computing device, a first electronic image via a sensor with a first resolution;

compressing, by the head-mounted computing device, the first electronic image to create a copy of the first electronic image with a second resolution less than the first resolution;

communicating, by the head-mounted computing device, the copy of a first electronic image to a remote device over a communication session established with the remote device via a network;

receiving, by the head-mounted computing device, a selected portion of the communicated copy from the remote device, wherein the selected portion corresponds to a user-selected area of the communicated copy;

based on the selected portion, generating, by the head-mounted computing device, an instruction to capture a second electronic image utilizing a portion of the sensor that corresponds to the user-selected area;

causing, by the head-mounted computing device, the portion of the sensor to capture the second electronic image with the first resolution in response to the generated instruction;

compressing, by the head-mounted computing device, the second electronic image to create a copy of the second electronic image with the second resolution;

providing for display, by the head-mounted computing device, the copy of the second electronic image to the remote device over the established communication session.

2. The method of claim 1 , wherein the user-selected area is off-center.

3. The method of claim 1 , wherein the selection is communicated to the sensor via a wearable device.

4. The method of claim 1 , further comprising enabling a user of the remote device to activate a laser pointer of a wearable device.

5. The method of claim 4 , wherein the laser pointer is visible to a user of the wearable device via a display of the wearable device.

6. The method of claim 1 , further comprising detecting low light for the portion of the sensor.

7. The method of claim 6 , further comprising generating a binning instruction to perform variable pixel binning for the portion of the sensor.

8. The method of claim 7 , wherein the variable pixel binning for the portion of the sensor is performed independently from pixel binning of the sensor as a whole.

9. The method of claim 1 , further comprising detecting distortion at a border of the second electronic image.

10. The method of claim 9 , further comprising generating a correction instruction to apply image correction to the border of the second electronic image.

11. The method of claim 10 , further comprising providing for display, to the remote device, the second electronic image with image correction.

12. The method of claim 1 , further comprising compressing only a video frame corresponding to the portion of the sensor to create the second electronic image at the second resolution.

13. The method of claim 1 wherein the second electronic image is captured without suspending a video mode and activating a camera mode of the sensor.

14. At least one computer storage media, having instructions thereon that, when executed by at least one processor of a computing system, cause the computing system to:

capture, by a head-mounted computing device, a first electronic image via a sensor with a first resolution;

compress, by the head-mounted computing device, the first electronic image to create a copy of the first electronic image with a second resolution less than the first resolution;

communicate, by the head-mounted computing device, the copy of a first electronic image to a remote device over a communication session established with the remote device via a network;

generate, by the head-mounted computing device, an instruction to capture a second electronic image utilizing a portion of the sensor that corresponds to a user selected area;

cause, by the head-mounted computing device, the sensor to capture the second electronic image with the first resolution in response to the generated instruction;

compressing, by the head-mounted computing device, the second electronic image to create a copy of the second electronic image with the second resolution; and

provide for display, by the head-mounted computing device to the remote device, the copy of the second electronic image to the remote device.

15. The computer storage media of claim 14 , further comprising:

detecting low light for the portion of the sensor;

generating a binning instruction to perform variable pixel binning for the portion of the sensor; and

performing variable pixel binning for the portion of the sensor, wherein the variable pixel binning for the portion of the sensor is performed independently from pixel binning of the sensor as a whole.

16. The computer storage media of claim 14 , further comprising:

detecting distortion at a border of the second electronic image;

generating a correction instruction to apply image correction to the border of the second electronic image; and

providing for display, to the remote device, the second electronic image with image correction.

17. The computer storage media of claim 14 , further comprising enabling a user of the remote device to activate a laser pointer of a head-mounted display (HMD), wherein the laser pointer is visible to a HMD user via the HMD.

18. A computerized system comprising:

at least one processor; and

at least one computer storage media storing computer-useable instructions that, when executed by the at least one processor, causes the at least one processor to:

capture, by a head-mounted computing device, a first electronic image via a sensor with a first resolution;

compress, by the head-mounted computing device, the first electronic image to create a copy of the first electronic image with a second resolution less than the first resolution;

communicate, by the head-mounted computing device, the copy of a first electronic image to a remote device over a communication session established with the remote device by the head-mounted communication device;

receive, by the head-mounted computing device, a selected portion of the communicated copy from the remote device, wherein the selected portion corresponds to a user-selected area of the communicated copy;

based on the selected portion, generate, by the head-mounted computing device, an instruction to capture a second electronic image utilizing a portion of the sensor that corresponds to the user-selected area;

cause, by the head-mounted computing device, the portion of the sensor to capture the second electronic image with the first resolution in response to the generated instruction;

compress, by the head-mounted computing device, the second electronic image to create a copy of the second electronic image with the second resolution; and

provide for display, by the head-mounted computing device, the copy of the second electronic image to the remote device over the established communication session.

19. The computerized system of claim 18 , further comprising compressing only a video frame corresponding to the portion of the sensor to create the second electronic image at the second resolution.

20. The computerized system of claim 18 , wherein the second electronic image is captured without suspending a video mode and activating a camera mode of the sensor.

Assignments (2)
SECURITY INTEREST Recorded Jun 6, 2024
From: REALWEAR, INC.
To: WESTERN ALLIANCE BANK
Reel/Frame 067646/0492 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 6, 2020
From: JHAWAR, SANJAY; LUSTIG, KEN
To: REALWEAR, INC.
Reel/Frame 053124/0228 →
Continuity (1)
Related Publication 20210400198A1 · Dec 23, 2021