Granted Patent B2

US 12,307,817 · App. 17/885,907 · Granted May 20, 2025

Inventors: Aditya Kumar (Noida, IN); Natasha Meena (Noida, IN)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

G06V40/174G06T7/254G06T7/70G06V40/20G06T2207/20224

Abstract

Claims (49)

1. A method for automatically capturing and processing an image of a user watching, reading, or listening to a multimedia content, the method comprising:

determining a level of an emotion identified from the multimedia content;

determining an adjusted emotion level of the multimedia content by adjusting the level of the emotion based on user information;

capturing a plurality of images of the user over a period of time from when the adjusted emotion level of the multimedia content is greater than a first threshold to when a reaction of the user determined based on the plurality of images is less than a second threshold;

prioritizing the plurality of images based on at least one of a frame emotion level of the plurality of images, facial features of the user in the plurality of images, or a number of faces in the plurality of images; and

processing the prioritized images to generate an output.

2. The method according to claim 1 , wherein the determining the level of the emotion of the multimedia content comprises:

determining, by using a pre-configured determination model, an emotion probability value based on the emotion identified from the multimedia content; and

determining the level of the emotion based on the emotion probability value.

3. The method according to claim 1 , wherein the determining the adjusted emotion level comprises:

determining an adjustment factor for the emotion based on the user information, wherein the user information comprises at least one of demographic data of the user, a past usage history of the user, or past sensor biological data; and

determining the adjusted emotion level based on the adjustment factor.

4. The method according to claim 1 , wherein the first threshold is determined by retrieving, from an emotion threshold table, a value corresponding to the identified emotion.

5. The method according to claim 1 , wherein the first threshold is a minimum expression intensity for at least one of static and dynamic facial expressions of the emotion to be detected.

6. The method according to claim 1 , wherein the second threshold is determined by:

determining a position of the user in a current captured image and a previous captured image; and

determining a difference in the current captured image and the previous captured image, based on an area corresponding to the position of the user in the current captured image and the previous captured image, and determining a change in the reaction of the user based on the difference, to determine the second threshold.

7. The method according to claim 1 , wherein the prioritizing the plurality of images comprises:

categorizing the plurality of images into at least one set of frames, wherein each set of frames comprises images of the user corresponding to a predefined emotion category;

obtaining, for each of the at least one set of frames, a frame emotion level from images included in each set of frames;

generating a priority value by applying a predetermined function to the obtained frame emotion level in each set of frames, wherein the predetermined function includes weighted summation of the frame emotion level of each set of frames; and

prioritizing the images in each set of frames based on the priority value.

8. The method according to claim 7 , wherein the frame emotion level of a set of frames is obtained by:

determining the frame emotion level based on a weighted sum of emotion values of images included in the set of frames.

9. The method according to claim 7 , wherein the predetermined function further includes determining a facial priority value by:

determining a presence of at least one predefined facial feature in the images of each set of frame, wherein each of the at least one predefined facial feature is assigned a predetermined weight; and

determining the facial priority value based on the presence of the at least one predefined facial feature and the predetermined weight.

10. The method according to claim 1 , wherein the processing of the prioritized images comprises at least one of:

enhancing the prioritized images, wherein enhancing comprises adjusting at least one of blur, lighting, sharpness, or color content in the prioritized images;

generating a new media content from the prioritized images, wherein the new media content comprises at least one of an emoji, an avatar, a video, an audio, or an animated image of the user;

obtaining a health index of the user, wherein the health index indicates a mood of the user; or

editing the multimedia content, wherein editing comprises at least one of zooming, trimming, or animating the multimedia content.

11. The method according to claim 1 , wherein the multimedia content comprises at least one of an audio and video content, a text content, an audio content or a video content.

12. The method according to claim 1 , wherein the multimedia content comprises at least one of a live content or a stored content.

13. An electronic device for automatically capturing and processing an image of a user watching, reading, or listening to a multimedia content, the electronic device comprising at least one processor configured to:

determine a level of an emotion identified from the multimedia content;

determine an adjusted emotion level of the multimedia content by adjusting the level of the emotion based on user information;

capture a plurality of images of the user over a period of time from when the adjusted emotion level of the multimedia content is greater than a first threshold to when a reaction of the user determined based on the plurality of images is less than a second threshold;

prioritize the plurality of images based on at least one of a frame emotion level of the plurality of images, facial features of the user in the plurality of images, or a number of faces in the plurality of images; and

process the prioritized images to generate an output.

14. The electronic device according to claim 13 , wherein the at least one processor is further configured to determine the level of the emotion by:

determining, by using a pre-configured determination model, an emotion probability value based on the emotion identified from the multimedia content; and

determining the level of the emotion based on the emotion probability value.

15. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of an electronic device for automatically capturing and processing an image of a user watching, reading, or listening to a multimedia content, cause the one or more processors to:

determining a level of an emotion identified from the multimedia content;

determining an adjusted emotion level of the multimedia content by adjusting the level of the emotion based on user information;

capturing a plurality of images of the user over a period of time from when the adjusted emotion level of the multimedia content is greater than a first threshold to when a reaction of the user determined based on the plurality of images is less than a second threshold;

prioritizing the plurality of images based on at least one of a frame emotion level of the plurality of images, facial features of the user in the plurality of images, or a number of faces in the plurality of images; and

processing the prioritized images to generate an output.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (48)