IP Library Granted Patent US 11,023,769
Granted Patent B2
US 11,023,769 · App. 16/249,664 · Granted Jun 1, 2021

Modifying an image based on identifying a feature

Inventors: Ming Qian (Cary, NC); Sujin Jang (Rolling Meadows, IL); John Weldon Nicholson (Cary, NC); Song Wang (Cary, NC)
Assignee: Lenovo (Singapore) PTE. LTD.
G06K9/4604G06K9/00375G06K9/6256G06N3/0445G06T7/246G06T7/73
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,023,769
App. No.
16/249,664
Granted
Jun 1, 2021
Kind
B2
Abstract

For modifying an image, a processor detects a feature in the image using a convolutional neural network trained on a feature training set. The processor further places the feature within the displayed image. The processor determines an intent for the image. In addition, the processor modifies the image based on the intent.

Claims (24)

1. An apparatus comprising:

a camera that captures an image;

a processor;

a memory that stores code executable by the processor to:

detect a feature in the image using a convolutional neural network trained on a feature training set, wherein the feature is selected from the group consisting of one or more fingers, one or more fingertips, one or more gloved fingers, one or more gloved fingertips, a hand, a gloved hand, an instrument, and a tool;

place the feature at a center of a displayed image;

determine an intent from a voice command using a recurrent neural network trained on motions of the feature for the image, wherein intent is selected from the group comprising a specified zoom, a maximum zoom, a zoom in, a zoom out, following the feature, a pan left, a pan right, a pan up, and a pan down, and the recurrent neural network is a Long Short Term Memory (LSTM) neural network that receives a plurality of temporal instances of the feature; and

modify the displayed image based on the intent, wherein the plurality of temporal instances of the feature are each scaled to one of two or more window sizes and each window size is a one dimensional kernel of a specified length.

2. A method comprising:

detecting, by use of a processor, a feature in an image using a convolutional neural network trained on a feature training set, wherein the feature is selected from the group consisting of one or more fingers, one or more fingertips, one or more gloved fingers, one or more gloved fingertips, a hand, a gloved hand, an instrument, and a tool;

placing the feature at a center of a displayed image;

determining an intent from a voice command using a recurrent neural network trained on motions of the feature for the image, wherein intent is selected from the group comprising a specified zoom, a maximum zoom, a zoom in, a zoom out, following the feature, a pan left, a pan right, a pan up, and a pan down, and the recurrent neural network is a Long Short Term Memory (LSTM) neural network that receives a plurality of temporal instances of the feature; and

modifying the displayed image based on the intent, wherein the plurality of temporal instances of the feature are each scaled to one of two or more window sizes and each window size is a one dimensional kernel of a specified length.

3. A program product comprising a computer readable storage medium that stores code executable by a processor, the executable code comprising code to:

detect a feature in an image using a convolutional neural network trained on a feature training set, wherein the feature is selected from the group consisting of one or more fingers, one or more fingertips, one or more gloved fingers, one or more gloved fingertips, a hand, a gloved hand, an instrument, and a tool;

place the feature at a center of a displayed image;

determine an intent from a voice command using a recurrent neural network trained on motions of the feature for the image, wherein intent is selected from the group comprising a specified zoom, a maximum zoom, a zoom in, a zoom out, following the feature, a pan left, a pan right, a pan up, and a pan down, and the recurrent neural network is a Long Short Term Memory (LSTM) neural network that receives a plurality of temporal instances of the feature; and

modify the displayed image based on the intent, wherein the plurality of temporal instances of the feature are each scaled to one of two or more window sizes and each window size is a one dimensional kernel of a specified length.

4. The apparatus of claim 1 , wherein the convolutional neural network generates a temporal slice that is flattened across a time interval and the flattened temporal slice is input into the recurrent neural network.

5. The apparatus of claim 1 , wherein the camera captures a wide field-of-view and the displayed image is modified by selecting a portion of the field-of-view to be presented.

6. The method of claim 2 , wherein the convolutional neural network generates a temporal slice that is flattened across a time interval and the flattened temporal slice is input into the recurrent neural network.

7. The method of claim 2 , wherein a camera captures a wide field-of-view and the displayed image is modified by selecting a portion of the field-of-view to be presented.

8. The program product of claim 3 , wherein the convolutional neural network generates a temporal slice that is flattened across a time interval and the flattened temporal slice is input into the recurrent neural network.

9. The program product of claim 3 , wherein a camera captures a wide field-of-view and the displayed image is modified by selecting a portion of the field-of-view to be presented.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2025
From: LENOVO PC INTERNATIONAL LIMITED
To: LENOVO SWITZERLAND INTERNATIONAL GMBH
Reel/Frame 069870/0670 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2022
From: LENOVO (SINGAPORE) PTE LTD
To: LENOVO PC INTERNATIONAL LIMITED
Reel/Frame 060638/0160 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 16, 2019
From: QIAN, MING; JANG, SUJIN; NICHOLSON, JOHN WELDON; WANG, SONG
To: LENOVO (SINGAPORE) PTE. LTD.
Reel/Frame 048037/0371 →