IP Library › Granted Patent US 11,182,634
Granted Patent B2
US 11,182,634 · App. 16/268,443 · Granted Nov 23, 2021

Systems and methods for modifying labeled content

Inventors: Kenneth Mitchell (Burbank, CA); Caio Jose dos Santos Brito (Recife, BR)
Assignee: Disney Enterprises, Inc.
G06K9/342G06K9/0061G06K9/00281G06K9/6206
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,182,634
App. No.
16/268,443
Granted
Nov 23, 2021
Kind
B2
Abstract

Systems and methods are disclosed for modifying labeled target content for a capture device. A computer-implemented method may use a computer system that includes non-transient electronic storage, a graphical user interface, and one or more physical computer processors. The computer-implemented method may include: obtaining labeled target content, the labeled target content including one or more facial features that have been labeled; modifying the labeled target content to match dynamically captured content from a first capture device to generate modified target content; and storing the modified target content. The dynamically captured content may include the one or more facial features.

Claims (82)

1. A computer-implemented method for modifying labeled target content, the method being implemented in a computer system that comprises a storage, one or more physical computer processors, and a graphical user interface, the method comprising:

obtaining, from the storage, labeled target content from a first capture device, the labeled target content comprising one or more first facial features that have been labeled;

modifying, with the one or more physical computer processors, the labeled target content to match dynamically captured content from a second capture device to generate modified target content, wherein the second capture device is different from the first capture device;

estimating, with the one or more physical computer processors, one or more bounding boxes in the dynamically captured content based on the modified target content; and

generating one or more second facial features for the dynamically captured content based on the one or more bounding boxes.

2. The computer-implemented method of claim 1 , wherein the one or more first facial features comprise one or more mouth features.

3. The computer-implemented method of claim 2 , wherein modifying the labeled target content comprises:

cropping, with the one or more physical computer processors, the labeled target content such that the one or more mouth features are in cropped target content; and

warping, with the one or more physical computer processors, the cropped target content to match one or more parts of the dynamically captured content that correspond to the one or more mouth features.

4. The computer-implemented method of claim 1 , wherein the one or more first facial features comprises one or more eye features.

5. The computer-implemented method of claim 4 , wherein modifying the labeled target content comprises:

rotating, with the one or more physical computer processors, the labeled target content based on an angle of the dynamically captured content to generate rotated target content;

cropping, with the one or more physical computer processors, the rotated target content such that the one or more eye features are in cropped target content; and

warping, with the one or more physical computer processors, the cropped target content to match one or more parts of the dynamically captured content that correspond to the one or more eye features.

6. The computer-implemented method of claim 1 ,

wherein the one or more first facial features comprise one or more eye features, and

wherein generating the one or more second facial features comprises:

estimating, with the one or more physical computer processors, the one or more bounding boxes around parts of a face in the dynamically captured content using one or more facial parameters in the dynamically captured content to generate the one or more second facial features from the dynamically captured content, wherein the one or more facial parameters comprise one or more of a color, a curve, or a reflected light intensity;

generating, with the one or more physical computer processors, the one or more second facial features in the dynamically captured content using the one or more bounding boxes;

identifying, with the one or more physical computer processors, a closed eye when a given image in converted captured content is within a first threshold range, wherein the converted captured content is derived from the dynamically captured content; and

generating, with the one or more physical computer processors, a position of a pupil when a portion of the given image in the converted captured content is within a second threshold range.

7. The computer-implemented method of claim 1 , further comprising:

dynamically generating, with the one or more physical computer processors, a representation of a face using visual effects to depict the one or more second facial features; and

displaying, via the graphical user interface, the representation.

8. The computer-implemented method of claim 1 , wherein modifying the labeled target content comprises converting, with the one or more physical computer processors, the labeled target content into converted content, wherein the converted content uses a different color format than the labeled target content.

9. The computer-implemented method of claim 1 ,

wherein the second capture device comprises a

red-green-blue camera, an infrared camera, an infrared illuminator, or a combination thereof.

10. The computer-implemented method of claim 1 , further comprising:

dynamically generating, with the one or more physical computer processors, a representation of a face using visual effects to depict the one or more second facial features or changes to the one or more second facial features.

11. The computer-implemented method of claim 10 ,

wherein the one or more first facial features comprise one or more mouth features, and

wherein modifying the labeled target content comprises:

cropping, with the one or more physical computer processors, the labeled target content such that the one or more mouth features are in cropped target content; and

warping, with the one or more physical computer processors, the cropped target content to match one or more parts of the dynamically captured content that correspond to the one or more mouth features.

12. The computer-implemented method of claim 10 ,

wherein the one or more first facial features comprise one or more eye features, and

wherein modifying the labeled target content comprises:

rotating, with the one or more physical computer processors, the labeled target content based on an angle of the dynamically captured content to generate rotated target content;

cropping, with the one or more physical computer processors, the rotated target content such that the one or more eye features are in cropped target content; and

warping, with the one or more physical computer processors, the cropped target content to match one or more parts of the dynamically captured content that correspond to the one or more eye features.

13. The computer-implemented method of claim 10 ,

wherein the one or more first facial features comprise one or more eye features, and

wherein generating the one or more second facial features comprises:

estimating, with the one or more physical computer processors, the one or more bounding boxes around parts of a face in the dynamically captured content using one or more facial parameters in the dynamically captured content, wherein the one or more facial parameters comprise one or more of a color, a curve, or a reflected light intensity;

generating, with the one or more physical computer processors, the one or more second facial features in the dynamically captured content using the one or more bounding boxes;

identifying, with the one or more physical computer processors, a closed eye when a given image in converted captured content is within a first threshold range, wherein the converted captured content is derived from the dynamically captured content; and

generating, with the one or more physical computer processors, a position of a pupil when a portion of the given image in the converted captured content is within a second threshold range.

14. The computer-implemented method of claim 10 ,

wherein the second capture device comprises a head mounted display, and

wherein the head mounted display comprises a red-green-blue camera, an infrared camera, an infrared illuminator, or a combination thereof.

15. A computer-implemented method for modifying labeled target content for a capture device, the method being implemented in a computer system that comprises a storage, one or more physical computer processors, and a graphical user interface, the method comprising:

obtaining, from the storage, labeled target content, the labeled target content comprising one or more first facial features that have been labeled;

modifying, with the one or more physical computer processors, the labeled target content to match dynamically captured content from a first capture device to generate modified target content;

estimating, with the one or more physical computer processors, one or more bounding boxes in the dynamically captured content based on the modified target content; and

generating one or more second facial features for the dynamically captured content based on the one or more bounding boxes,

wherein the dynamically captured content includes a first distortion associated with the first capture device and the labeled target content includes a second distortion associated with a second capture device used to capture the labeled target content.

16. A system to modify labeled target content, the system comprising:

a storage;

a graphical user interface; and

one or more physical computer processors configured by machine-readable instructions to:

obtain, from the storage, labeled target content from a first capture device, the labeled target content comprising one or more first facial features that have been labeled;

modify, with the one or more physical computer processors, the labeled target content to match dynamically captured content from a second capture device to generate modified target content, wherein the second capture device is different from the first capture device;

estimate one or more bounding boxes in the dynamically captured content based on the modified target content; and

generate one or more second facial features for the dynamically captured content based on the one or more bounding boxes.

17. The system of claim 16 ,

wherein the one or more first facial features comprise one or more mouth features, and

wherein modifying the labeled target content comprises:

cropping, with the one or more physical computer processors, the labeled target content such that the one or more mouth features are in cropped target content; and

warping, with the one or more physical computer processors, the cropped target content to match one or more parts of the dynamically captured content that correspond to the one or more mouth features.

18. The system of claim 16 ,

wherein the one or more first facial features comprise one or more eye features, and

wherein modifying the labeled target content comprises:

rotating, with the one or more physical computer processors, the labeled target content based on an angle of the dynamically captured content to generate rotated target content;

cropping, with the one or more physical computer processors, the rotated target content such that the one or more eye features are in cropped target content; and

warping, with the one or more physical computer processors, the cropped target content to match one or more parts of the dynamically captured content that correspond to the one or more eye features.

19. The system of claim 16 ,

wherein the one or more first facial features comprise one or more eye features, and

wherein the one or more physical computer processors are further configured by the machine-readable instructions to:

estimate the one or more bounding boxes around parts of a face in the dynamically captured content using one or more facial parameters in the dynamically captured content, wherein the one or more facial parameters comprise one or more of a color, a curve, or a reflected light intensity;

identify a closed eye when a given image in converted captured content is within a first threshold range, wherein the converted captured content is derived from the dynamically captured content; and

generate a position of a pupil when a portion of the given image in the converted captured content is within a second threshold range.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2019
From: MITCHELL, KENNETH; BRITO, CAIO JOSE DOS SANTOS
To: DISNEY ENTERPRISES, INC.
Reel/Frame 048278/0991 →
Continuity (1)
Related Publication 20200250457A1 · Aug 6, 2020