IP Library › Granted Patent US 10,930,032
Granted Patent B1
US 10,930,032 · App. 16/548,388 · Granted Feb 23, 2021

Generating concept images of human poses using machine learning models

Inventors: Samarth Bharadwaj (Bangalore, IN); Saneem Chemmengath (Bangalore, IN); Suranjana Samanta (Bangalore, IN); Karthik Sankaranarayanan (Bangalore, IN)
Assignee: International Business Machines Corporation
G06T11/20G06K9/00335G06K9/6256G06K9/6263G06N20/00G06T7/70G06T2207/20081G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,930,032
App. No.
16/548,388
Granted
Feb 23, 2021
Kind
B1
Abstract

Methods, systems, and computer program products for generating concept images of human poses using machine learning models are provided herein. A computer-implemented method includes identifying one or more events from input data by applying a machine learning recognition model to the input data, wherein the identifying comprises (i) detecting multiple entities from the input data and (ii) determining one or more behavioral relationships among the multiple entities in the input data; generating, using a machine learning interpretability model and the identified events, one or more images illustrating one or more human poses related to the identified events; outputting the one or more generated images to at least one user; and updating the machine learning recognition model based at least in part on (i) the one or more generated images and (ii) input from the at least one user.

Claims (37)

1. A computer-implemented method comprising:

identifying one or more events from input data by applying a machine learning recognition model to the input data, wherein said identifying comprises (i) detecting multiple entities from the input data, (ii) determining one or more behavioral relationships among the multiple entities in the input data, and (iii) generating, via the machine learning recognition model, a vector representation of one or more human poses related to the identified events;

generating, using a machine learning interpretability model and the identified events, one or more images illustrating one or more human poses related to the identified events, wherein said generating the one or more images comprises converting the vector representation to one or more human pose representations over one or more stick figure images;

outputting the one or more generated images to at least one user; and

updating the machine learning recognition model based at least in part on (i) the one or more generated images and (ii) input from the at least one user;

wherein the method is carried out by at least one computing device.

2. The computer-implemented method of claim 1 , comprising:

training the machine learning recognition model using data pertaining to human pose structures and image texture information.

3. The computer-implemented method of claim 1 , wherein the input data comprise image data.

4. The computer-implemented method of claim 1 , wherein the input data comprise text data.

5. The computer-implemented method of claim 1 , wherein the input data comprise multimodal data.

6. The computer-implemented method of claim 1 , wherein the one or more images comprise one or more stick figure images.

7. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

identify one or more events from input data by applying a machine learning recognition model to the input data, wherein said identifying comprises (i) detecting multiple entities from the input data, (ii) determining one or more behavioral relationships among the multiple entities in the input data, and (iii) generating, via the machine learning recognition model, a vector representation of one or more human poses related to the identified events;

generate, using a machine learning interpretability model and the identified events, one or more images illustrating one or more human poses related to the identified events, wherein said generating the one or more images comprises converting the vector representation to one or more human pose representations over one or more stick figure images;

output the one or more generated images to at least one user; and

update the machine learning recognition model based at least in part on (i) the one or more generated images and (ii) input from the at least one user.

8. The computer program product of claim 7 , wherein the input data comprise one or more of image data, text data, and multimodal data.

9. A system comprising:

a memory; and

at least one processor operably coupled to the memory and configured for:

identifying one or more events from input data by applying a machine learning recognition model to the input data, wherein said identifying comprises (i) detecting multiple entities from the input data, (ii) determining one or more behavioral relationships among the multiple entities in the input data, and (iii) generating, via the machine learning recognition model, a vector representation of one or more human poses related to the identified events;

generating, using a machine learning interpretability model and the identified events, one or more images illustrating one or more human poses related to the identified events, wherein said generating the one or more images comprises converting the vector representation to one or more human pose representations over one or more stick figure images;

outputting the one or more generated images to at least one user; and

updating the machine learning recognition model based at least in part on (i) the one or more generated images and (ii) input from the at least one user.

10. A computer-implemented method comprising:

training a machine learning recognition model using data pertaining to human pose structures and image texture information;

identifying one or more events from input data by applying the machine learning recognition model to the input data, wherein said identifying comprises (i) detecting multiple entities from the input data, (ii) determining one or more behavioral relationships among the multiple entities in the input data, and (iii) generating, via the machine learning recognition model, a vector representation of one or more human poses related to the identified events;

generating, using a machine learning interpretability model and the identified events, one or more images illustrating one or more human poses related to the identified events, wherein said generating the one or more images comprises converting the vector representation to one or more human pose representations over one or more stick figure images;

outputting, via a dialog system, the one or more generated images to at least one human user;

automatically producing at least one machine-originated image representative of the one or more generated images based at least in part on feedback from the at least one human user;

updating the machine learning recognition model based at least in part on the at least one machine-originated image;

wherein the method is carried out by at least one computing device.

11. The computer-implemented method of claim 10 , wherein the input data comprise image data.

12. The computer-implemented method of claim 10 , wherein the input data comprise text data.

13. The computer-implemented method of claim 10 , wherein the input data comprise multimodal data.

14. The computer-implemented method of claim 10 , wherein the one or more images comprise one or more stick figure images.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 22, 2019
From: BHARADWAJ, SAMARTH; CHEMMENGATH, SANEEM; SAMANTA, SURANJANA; SANKARANARAYANAN, KARTHIK
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 050137/0820 →
Cited By (2)
US 12,318,661 US 12,364,905