IP Library Granted Patent US 10,949,463
Granted Patent B2
US 10,949,463 · App. 16/588,653 · Granted Mar 16, 2021

Behavioral measurements in a video stream focalized on keywords

Inventors: Ramya Narasimha (Palo Alto, CA); Hector H. Gonzalez-Banos (Mountain View, CA)
Assignee: Ricoh Company, Ltd.
G06F16/743G06F16/71G06F16/735G06F16/738G06F16/7834G06F16/7867
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,949,463
App. No.
16/588,653
Granted
Mar 16, 2021
Kind
B2
Abstract

A system and method for analyzing behavior in a video is described. The method includes extracting a plurality of salient fragments of a video; associating a time anchor with an utterance of a first keyword in an audio track associated with the video; generating a focalized visualization, based on the time anchor, from one or more of the plurality of salient fragments of the video; tagging a human subject in the focalized visualization with a unique identifier; and analyzing behavior of the human subject, using the focalized visualization, to generate a behavior score associated with the unique identifier and the first keyword.

Claims (51)

1. A computer-implemented method comprising:

extracting a plurality of salient fragments from a video, a salient fragment being a video sequence tracking a salient object through a subset of a series of frames in the video;

associating a time anchor with an utterance of a first keyword in an audio track associated with the video;

generating a video segment focalized on the time anchor associated with the utterance of the first keyword from one or more of the plurality of salient fragments of the video;

tagging a human subject in the video segment with a unique identifier; and

analyzing behavior of the human subject, using the video segment, to generate a behavior score associated with the unique identifier and the first keyword.

2. The computer-implemented method of claim 1 , further comprising:

storing the behavior score as a record in a database using the unique identifier and the first keyword as attributes.

3. The computer-implemented method of claim 2 , further comprising:

performing a query on the database based on selection criteria that is selected from the group consisting of a date of record, the unique identifier, the time anchor, the first keyword, the behavior score, a minimum behavior score, a maximum behavior score, and an average behavior score.

4. The computer-implemented method of claim 1 , further comprising:

generating a baseline behavior for a behavioral attribute; and

producing a contrastive behavior score by comparing the behavior of the human subject to the baseline behavior.

5. The computer-implemented method of claim 4 , wherein generating the baseline behavior comprises analyzing behavior of the human subject at a time anchor associated with an utterance of a second keyword in the audio track associated with the video.

6. The computer-implemented method of claim 1 , wherein the first keyword is a slogan.

7. The computer-implemented method of claim 1 , wherein the first keyword is included in a set of keywords and the behavior score is associated with the unique identifier and the set of keywords.

8. The computer-implemented method of claim 7 , wherein the set of keywords is extracted from a syllabus describing a course of study.

9. The computer-implemented method of claim 7 , wherein the set of keywords includes safety terms included in a safety training program.

10. The computer-implemented method of claim 7 , wherein the set of keywords is extracted from a transcript of a meeting.

11. A computer program product comprising a non-transitory computer readable medium storing a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:

extract a plurality of salient fragments from a video, a salient fragment being a video sequence tracking a salient object through a subset of a series of frames in the video;

associate a time anchor with an utterance of a first keyword in an audio track associated with the video;

generate a video segment focalized on the time anchor associated with the utterance of the first keyword from one or more of the plurality of salient fragments of the video;

tag a human subject in the video segment with a unique identifier; and

analyze behavior of the human subject, using the video segment, to generate a behavior score associated with the unique identifier and the first keyword.

12. The computer program product of claim 11 , wherein the computer readable program when executed on the computer further causes the computer to:

store the behavior score as a record in a database using the unique identifier and the first keyword as attributes.

13. The computer program product of claim 12 , wherein the computer readable program when executed on the computer further causes the computer to:

perform a query on the database based on selection criteria that is selected from the group consisting of a date of record, the unique identifier, the time anchor, the first keyword, the behavior score, a minimum behavior score, a maximum behavior score, and an average behavior score.

14. The computer program product of claim 11 , wherein the computer readable program when executed on the computer further causes the computer to:

generate a baseline behavior for a behavioral attribute; and

produce a contrastive behavior score by comparing the behavior of the human subject to the baseline behavior.

15. The computer program product of claim 14 , wherein to generate the baseline behavior, the computer readable program when executed on the computer further causes the computer to:

analyze behavior of the human subject at a time anchor associated with an utterance of a second keyword in the audio track associated with the video.

16. A system comprising:

one or more processors; and

a memory, the memory storing instructions which when executed cause the one or more processors to:

extract a plurality of salient fragments from a video, a salient fragment being a video sequence tracking a salient object through a subset of a series of frames in the video;

associate a time anchor with an utterance of a first keyword in an audio track associated with the video;

generate a video segment focalized on the time anchor associated with the utterance of the first keyword from one or more of the plurality of salient fragments of the video;

tag a human subject in the video segment with a unique identifier; and

analyze behavior of the human subject, using the video segment, to generate a behavior score associated with the unique identifier and the first keyword.

17. The system of claim 16 , wherein the instructions further cause the one or more processors to:

store the behavior score as a record in a database using the unique identifier and the first keyword as attributes.

18. The system of claim 17 , wherein the instructions further cause the one or more processors to:

perform a query on the database based on selection criteria that is selected from the group consisting of a date of record, the unique identifier, the time anchor, the first keyword, the behavior score, a minimum behavior score, a maximum behavior score, and an average behavior score.

19. The system of claim 16 , wherein the instructions further cause the one or more processors to:

generate a baseline behavior for a behavioral attribute; and

produce a contrastive behavior score by comparing the behavior of the human subject to the baseline behavior.

20. The system of claim 19 , wherein to generate the baseline behavior, the instructions further cause the one or more processors to:

analyze behavior of the human subject at a time anchor associated with an utterance of a second keyword in the audio track associated with the video.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 25, 2019
From: NARASIMHA, RAMYA; GONZALEZ-BANOS, HECTOR H.
To: RICOH COMPANY, LTD.
Reel/Frame 050826/0942 →
Continuity (5)
Continuation In Part 16386241 · Apr 16, 2019
Continuation 15916997 · Mar 9, 2018
Continuation 15453722 · Mar 8, 2017
Continuation 15447416 · Mar 2, 2017
Related Publication 20200026729A1 · Jan 23, 2020