IP Library Granted Patent US 9,740,916
Granted Patent B2
US 9,740,916 · App. 15/231,296 · Granted Aug 22, 2017

Systems and methods for persona identification using combined probability maps

Inventors: Dennis Lin (Chicago, IL); Glenn Francisco (Chicago, IL); Quang Nguyen (Ho Chi Minh, VN); Long Dang (Ho Chi Minh, VN)
Assignee: Personify Inc.
G06K9/00281G06K9/00228G06K9/4642G06K9/4652G06K9/6207G06K9/6218G06K9/6289G06T7/11G06T7/143G06T7/149G06T7/194H04N7/147H04N7/15G06T2207/10016G06T2207/10024G06T2207/10028G06T2207/20076G06T2207/20228G06T2207/30196G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,740,916
App. No.
15/231,296
Granted
Aug 22, 2017
Kind
B2
Abstract

Disclosed herein are systems and methods for persona identification using combined probability maps. An embodiment takes the form of a method that includes obtaining at least one frame of pixel data; processing the at least one frame of pixel data to generate a hair-identification probability map; and generating a persona image by extracting pixels from the at least one frame of pixel data based at least in part on the generated hair-identification probability map.

Claims (45)

1. A method comprising:

obtaining at least one frame of pixel data;

processing the at least one frame of pixel data to generate a hair-identification probability map at least in part by:

identifying a plurality of pixel columns that cross an identified head contour; and

for each pixel column in the plurality of pixel columns:

performing a color-based segmentation of the pixels in the pixel column into a foreground segment, a hair segment, and a background segment; and

assigning the pixels in the hair segment an increased foreground-probability value in the hair-identification probability map; and

generating a persona image by extracting pixels from the at least one frame of pixel data based at least in part on the generated hair-identification probability map.

2. The method of claim 1 , further comprising converting the head contour into a multi-segment polygon that approximates the head contour, the multi-segment polygon being formed of multiple head-contour segments, wherein identifying the plurality of pixel columns that cross the identified head contour comprises identifying pixel columns that cross one of the head-contour segments.

3. The method of claim 1 , wherein performing a color-based segmentation comprises performing a color-based segmentation using a clustering algorithm.

4. The method of claim 3 , wherein the clustering algorithm is a k-means algorithm with k=3.

5. The method of claim 1 , wherein performing the color-based segmentation of the pixels in a given pixel column into the foreground segment, the hair segment, and the background segment of the given pixel column comprises:

identifying an average foreground-pixel color, an average hair-pixel color, and an average background-pixel color for the given pixel column; and

identifying the foreground segment, the hair segment, and the background segment of the given pixel column using a clustering algorithm to cluster the pixels in the given pixel column around the identified average foreground-pixel color, the identified average hair-pixel color, and the identified average background-pixel color for the given pixel column, respectively.

6. The method of claim 5 , wherein:

identifying the average foreground-pixel color for the given pixel column comprises identifying the average foreground-pixel color for the given pixel column based on a first set of pixels at an innermost end of the given pixel column;

identifying the average hair-pixel color for the given pixel column comprises identifying the average hair-pixel color for the given pixel column based on a second set of pixels that includes a point where the given pixel column crosses the identified head contour; and

identifying the average background-pixel color for the given pixel column comprises identifying the average background-pixel color for the given pixel column based on a third set of pixels at an outermost end of the given pixel column.

7. The method of claim 1 , further comprising, for each pixel column in the plurality of pixel columns:

assigning the pixels in the foreground and background segments an equal probability of being in the foreground and being in the background in the hair-identification probability map.

8. The method of claim 1 , wherein assigning the pixels in the hair segment an increased foreground-probability value in the hair-identification probability map comprises:

assigning a first value to the pixels in the hair segment in the hair-identification probability map; and

assigning a second value to the pixels in the foreground and background segments in the hair-identification probability map, wherein the first value corresponds to a higher probability of being a foreground pixel than does the second value.

9. The method of claim 1 , further comprising processing the at least one frame of pixel data to generate at least one additional probability map, wherein generating the persona image by extracting pixels from the at least one frame of pixel data is further based on the generated at least one additional probability map.

10. The method of claim 9 , wherein:

obtaining the at least one frame of pixel data comprises obtaining the at least one frame of pixel data and corresponding image depth data; and

processing the at least one frame of pixel data to generate the at least one additional probability map comprises processing the at least one frame of pixel data and the corresponding image depth data to generate at least one of the at least one additional probability maps.

11. The method of claim 9 , further comprising combining the hair-identification probability map and the at least one additional probability map to obtain an aggregate persona probability map, wherein generating the persona image by extracting pixels from the at least one frame of pixel data based at least in part on the generated hair-identification probability map and at least in part on the generated at least one additional probability map comprises generating the persona image by extracting pixels from the at least one frame of pixel data based on the aggregate persona probability map.

12. The method of claim 1 , further comprising processing the at least one frame of pixel data to generate at least one additional probability map, wherein generating the persona image by extracting pixels from the at least one frame of pixel data is further based at least in part on the generated at least one additional probability map.

13. The method of claim 12 , wherein:

obtaining the at least one frame of pixel data comprises obtaining the at least one frame of pixel data and corresponding image depth data; and

processing the at least one frame of pixel data to generate the at least one additional probability map comprises processing the at least one frame of pixel data and the corresponding image depth data to generate at least one of the at least one additional probability maps.

14. The method of claim 12 , further comprising combining the hair-identification probability map and the at least one additional probability map to obtain an aggregate persona probability map, wherein generating the persona image by extracting pixels from the at least one frame of pixel data based at least in part on the generated hair-identification probability map and at least in part on the generated at least one additional probability map comprises generating the persona image by extracting pixels from the at least one frame of pixel data based on the aggregate persona probability map.

15. An apparatus comprising:

a hair-identification module that is configured to generate a hair-identification probability map based on at least one frame of pixel data at least in part by:

identifying a plurality of pixel columns that cross an identified head contour; and

for each pixel column in the plurality of pixel columns:

performing a color-based segmentation of the pixels in the pixel column into a foreground segment, a hair segment, and a background segment; and

assigning the pixels in the hair segment an increased foreground-probability value in the hair-identification probability map; and

a persona extraction module configured to generate a persona image by extracting pixels from at least one frame of pixel data based at least in part on the generated hair-identification probability map.

16. The apparatus of claim 15 , further comprising a foreground-background module configured to generate a foreground-background map based on image depth data corresponding to the at least one frame of pixel data, wherein the persona extraction module is configured to generate the persona image by extracting pixels from the at least one frame of pixel data based also on the generated foreground-background map.

17. The apparatus of claim 15 , further comprising:

a plurality of additional persona identification modules configured to generate a corresponding plurality of additional persona probability maps based on the at least one frame of pixel data; and

a combiner module configured to generate an aggregate persona probability map based on the hair-identification probability map and the plurality of additional persona probability maps,

wherein the persona extraction module being configured to generate the persona image by extracting pixels from the at least one frame of pixel data based at least in part on the generated hair-identification probability map comprises the persona extraction module being configured to generate the persona image by extracting pixels from the at least one frame of pixel data based on the aggregate persona probability map.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 13, 2021
From: HONGFUJIN PRECISION INDUSTRY WUHAN
To: PERSONIFY, INC.
Reel/Frame 057467/0738 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 26, 2019
From: PERSONIFY, INC.
To: HONGFUJIN PRECISION INDUSTRY (WUHAN) CO. LTD.
Reel/Frame 051367/0920 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 20, 2017
From: LIN, DENNIS J.; FRANCISCO, GLENN B.; NGUYEN, QUANG H.; DANG, LONG
To: PERSONIFY, INC.
Reel/Frame 042080/0107 →
Continuity (2)
Continuation 14145874 · Dec 31, 2013
Related Publication 20160350585A1 · Dec 1, 2016