IP Library Granted Patent US 11,558,543
Granted Patent B2
US 11,558,543 · App. 17/096,356 · Granted Jan 17, 2023

Modifying capture of video data by an image capture device based on video data previously captured by the image capture device

Inventor: Vincent Charles Cheung (San Carlos, CA)
Assignee: Meta Platforms, Inc.
H04N5/23219G06F16/24578G06T7/90G06V10/22G06V40/10G06V40/103G06V40/107H04N5/23296H04N5/232061H04N5/232127H04N5/2628H04N7/141
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,558,543
App. No.
17/096,356
Granted
Jan 17, 2023
Kind
B2
Abstract

Various client devices include displays and one or more image capture devices configured to capture video data. Different users of an online system may authorize client devices to exchange information captured by their respective image capture devices. Additionally, a client device modifies captured video data based on users identified in the video data. For example, the client device changes parameters of the image capture device to more prominently display a user identified in the video data and may further change parameters of the image capture device based on gestures or movement of the user identified in the video data. The client device may apply multiple models to captured video data to modify the captured video data or subsequent capturing of video data by the image capture device.

Claims (58)

1. A method comprising:

capturing, by an image capture device of a client device, video data of a local area within a field of view of the image capture device;

applying one or more models maintained by the client device to the captured video data, each of the one or more models applying one or more rules to locate users in the captured video data;

in response to applying the one or more models, locating a first user in the captured video data;

comparing the first user within the captured video data to stored data describing users included in the captured video data who have previously been presented with at least a threshold set of dimensions by the captured video data;

modifying the captured video data to prominently present the first user in the captured video data with at least the threshold set of dimensions in response to the comparison determining the first user does not match a user included in the captured video data who had previously been presented with at least a threshold set of dimensions by the captured video data, wherein modifying the captured video data to prominently present the first user comprises:

determining that the first user is holding an object in the local area from the application of the one or more models and,

changing a focal point of the image capture device to the object and increasing a magnification of the image capture device; and

transmitting the modified video data including the captured video data with the first user displayed with at least the threshold set of dimensions from the client device to an online system.

2. The method of claim 1 , wherein modifying the captured video data to prominently present the first user comprises:

modifying one or more parameters of the image capture device based on application of the one or more of the maintained models; and

capturing additional video data of the local area around the first user using the modified one or more parameters of the image capture device.

3. The method of claim 1 , wherein modifying the captured video data to prominently present the first user comprises:

identifying movement of the first user from application of one or more of the maintained models; and

modifying a field of view of the image capture device based on identified movement of the first user.

4. The method of claim 1 , wherein modifying the captured video data to prominently present the first user comprises:

determining that the first user is gesturing towards an object in the local area from application of the one or more models; and

decreasing a magnification of the image capture device so additional video data captured by the image capture device includes the first user and the object towards which the first user is gesturing.

5. The method of claim 1 , wherein modifying the captured video data to prominently present the first user comprises:

modifying one or more parameters of the image capture device affecting a field of view of the image capture device based on application of the one or more models.

6. The method of claim 1 , wherein the modifying the captured video data to prominently present the first user in response to the comparison determining the first user does not match the user included in the captured video data who had previously been presented with at least the threshold set of dimension by the captured video data comprises modifying the captured video data to crop or to zoom a portion of the video data including one or more portions of the first user based on application of the one or more models.

7. The method of claim 6 , wherein modifying the captured video data to crop or to zoom the portion of the video data including one or more portions of the first user based on application of the one or more models comprises:

modifying the video data to include a torso of the first user based on the application of one or more rules included in the one or more models,

wherein the one or more rules are based on one or more selected from a group consisting of: content of video data previously captured by the image capture device, movement of the user in video data previously captured by the image capture device, and any combination thereof.

8. The method of claim 1 , wherein modifying the captured video data to prominently present the first user in response to the comparison determining the first user does not match the user included in the captured video data who had previously been presented with at least the threshold set of dimension by the captured video data comprises removing one or more frames from the captured video data in response to determining there is less than a threshold amount of movement of objects or of one or more users located within the captured video.

9. The method of claim 1 , wherein locating the first user in the captured video data comprises receiving a privacy setting from the user of the client device authorizing location of the first user in the captured video data.

10. A non-transitory computer program product comprising a computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

capture, by an image capture device of a client device, video data of a local area within a field of view of the image capture device;

apply one or more models maintained by the client device to the captured video data, each of the one or more models applying one or more rules to locate users in the captured video data;

in response to applying the one or more models, locate a first user in the captured video data;

compare the first user within the captured video data to stored data describing users included in the captured video data who have previously been presented with at least a threshold set of dimensions by the captured video data;

modify the captured video data to prominently present the first user in the captured video data with at least the threshold set of dimensions in response to the comparison determining the first user does not match a user included in the captured video data who had previously been presented with at least a threshold set of dimensions by the captured video data, wherein modifying the captured video data to prominently present the first user comprises:

determining that the first user is holding an object in the local area from the application of the one or more models and,

changing a focal point of the image capture device to the object and increasing a magnification of the image capture device; and

transmit the modified video data including the captured video data with the first user displayed with at least the threshold set of dimensions from the client device to an online system.

11. The computer program product of claim 10 , wherein modify the captured video data to prominently present the first user in response to the comparison determining the first user does not match the user included in the captured video data who had previously been presented with at least the threshold set of dimension by the captured video data comprises modify the captured video data to crop or to zoom a portion of the video data including one or more portions of the first user based on application of the one or more models.

12. The computer program product of claim 10 , wherein modifying the captured video data to prominently present the first user in response to the comparison determining the first user does not match the user included in the captured video data who had previously been presented with at least the threshold set of dimension by the captured video data comprises removing one or more frames from the captured video data in response to determining there is less than a threshold amount of movement of objects or of one or more users located within the captured video.

13. The computer program product of claim 10 , wherein locate the first user in the captured video data comprises receiving a privacy setting from the first user of the client device authorizing location of the first user in the captured video data.

14. The computer program product of claim 10 , wherein modify the captured video data to prominently present the first user comprises:

identifying movement of the first user from application of one or more of the maintained models; and

modifying a field of view of the image capture device based on identified movement of the first user.

15. The computer program product of claim 10 , wherein modify the captured video data to prominently present the first user comprises:

determining that the first user is gesturing towards an object in the local area from application of the one or more models; and

decreasing a magnification of the image capture device so additional video data captured by the image capture device includes the first user and the object towards which the first user is gesturing.

16. A device comprising:

a processor; and

a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

capture, by an image capture device of a client device, video data of a local area within a field of view of the image capture device;

apply one or more models maintained by the client device to the captured video data, each of the one or more models applying one or more rules to locate users in the captured video data;

in response to applying the one or more models, locate a first user in the captured video data;

compare the first user within the captured video data to stored data describing users included in the captured video data who have previously been presented with at least a threshold set of dimensions by the captured video data;

modify the captured video data to prominently present the first user in the captured video data with at least the threshold set of dimensions in response to the comparison determining the first user does not match a user included in the captured video data who had previously been presented with at least a threshold set of dimensions by the captured video data, wherein modifying the captured video data to prominently present the first user comprises:

determining that the first user is holding an object in the local area from the application of the one or more models and,

changing a focal point of the image capture device to the object and increasing a magnification of the image capture device; and

transmit the modified video data including the captured video data with the first user displayed with at least the threshold set of dimensions from the client device to an online system.

17. The device of claim 16 , wherein modify the captured video data to prominently present the first user in response to the comparison determining the first user does not match the user included in the captured video data who had previously been presented with at least the threshold set of dimension by the captured video data comprises modify the captured video data to crop or to zoom a portion of the video data including one or more portions of the first user based on application of the one or more models.

18. The device of claim 16 , wherein modify the captured video data to prominently present the first user in response to the comparison determining the first user does not match the user included in the captured video data who had previously been presented with at least the threshold set of dimension by the captured video data comprises removing one or more frames from the captured video data in response to determining there is less than a threshold amount of movement of objects or of one or more users located within the captured video.

19. The device of claim 16 , wherein locate the first user in the captured video data comprises receiving a privacy setting from the first user of the client device authorizing location of the first user in the captured video data.

Assignments (1)
CHANGE OF NAME Recorded Nov 18, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058897/0824 →