Granted Patent B2

US 12,427,377 · App. 18/196,724 · Granted Sep 30, 2025

Inventors: Hyunmi Choi (Suwon-si, KR); Jungmi Park (Seongnam-si, KR)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

A63B24/0075A63B71/0622G06V20/50G06V40/10G06V40/23G16H20/30A63B2024/0009A63B2024/0068A63B2214/00A63B2220/807A63B2230/625

Abstract

Claims (79)

1. An electronic apparatus comprising:

a display;

a camera;

a memory in which reference information associated with a content that provides information is stored; and

a processor configured to:

obtain an image photographing a user through the camera while the content is provided through the display, the content comprising information corresponding to a plurality of steps;

obtain context information associated with the user based on the image;

identify an execution status of the user associated with the information provided by the content by comparing the obtained context information and the reference information; and

control an output state of the content based on the execution status of the user; and

control the display to provide guide information corresponding to a step, requiring a feedback, from among the plurality of steps based on the identified execution status.

2. The electronic apparatus as claimed in claim 1 , wherein the content includes respective sub information corresponding to each of a plurality of steps; and

wherein the processor is configured to;

while the respective sub information corresponding to a first step from among the plurality of steps is provided through the display, identify the execution status of the user associated with the first step by comparing the obtained context information and reference information corresponding to the respective sub information; and

control the output state of the content corresponding to the first step based on the execution status of the user.

3. The electronic apparatus as claimed in claim 2 , wherein the memory stores a type of an output content for each execution level of the user; and

wherein the processor is configured to:

identify an execution level corresponding to the execution status of the user;

identify information associated with the type of the output content corresponding to the identified execution level; and

control the output state of the content based on the type of the output content.

4. The electronic apparatus as claimed in claim 3 , wherein the type of the output content includes at least one of a still image output, repeated output of the respective sub information corresponding to a specific step, output speed adjustment, or enlarged output of contents.

5. The electronic apparatus as claimed in claim 1 , further comprising:

a driving unit,

wherein the processor is configured to:

obtain posture information of the user based on the image obtained while a content that provides health information including an exercise posture is provided through the display;

identify the execution status of the user associated with the exercise posture by comparing the obtained posture information of the user and the posture information included in the reference information; and

control the driving unit to provide a specific feedback for guiding the user's posture based on the execution status of the user.

6. The electronic apparatus as claimed in claim 5 , wherein the electronic apparatus is implemented as a robot that provides a healthcare service; and

wherein the robot performs an operation of correcting the user's posture or provides guide information for guiding the user's posture so that the user executes the exercise posture provided by the content based on the execution status of the user.

7. The electronic apparatus as claimed in claim 5 , wherein the processor is configured to identify an execution level of the user based on at least one of a degree of correspondence between the obtained posture information and the posture information included in the reference information, a frequency of discrepancy between the obtained posture information and the posture information included in the reference information, or complexity of the exercise posture, and provide a feedback corresponding to the identified execution level.

8. The electronic apparatus as claimed in claim 1 , wherein the processor is configured to:

obtain a status information of the user based on the image obtained while the content is provided through the display;

identify the execution status of the user associated with each of the plurality of steps by comparing the obtained status information and the status information included in the reference information; and

control the display to provide the guide information corresponding to the step requiring feedback based on the execution status of the user, wherein the step is from among the plurality of steps.

9. The electronic apparatus as claimed in claim 8 , wherein the processor is configured to:

obtain a first status information of the user by inputting the image to a first neural network model;

obtain a second status information of the user by inputting a voice of the user to a second neural network model; and

control the display to provide the guide information corresponding to the step requiring feedback based on the identified execution status, the first status information, and the second status information.

10. The electronic apparatus as claimed in claim 8 , further comprising:

a speaker,

wherein the processor is configured to, based on the image of the user not being obtained while information associated with one step from among the plurality of steps is provided, control the speaker to output the guide information associated with the content.

11. A controlling method of an electronic apparatus, comprising:

obtaining an image photographing a user through a camera while a content that provides information is displayed, the content comprising information corresponding to a plurality of steps;

obtaining context information associated with the user based on the image;

identifying an execution status of the user associated with the information provided by the content by comparing the obtained context information and the reference information associated with the content;

controlling an output state of the content based on the execution status of the user; and

controlling a display to provide guide information corresponding to a step, requiring a feedback, from among the plurality of steps based on the identified execution status.

12. The method as claimed in claim 11 , wherein the content includes respective sub information corresponding to each of a plurality of steps;

wherein the identifying the execution status of the user comprises, while the respective sub information corresponding to a first step from among the plurality of steps is provided through a display, identifying the execution status of the user associated with the first step by comparing the obtained context information and reference information corresponding to the respective sub information; and

wherein the controlling the output state of the content comprises controlling the output state of the content associated with the first step based on the execution status of the user.

13. The method as claimed in claim 12 , wherein the identifying the execution status of the user comprises identifying an execution level corresponding to the execution status of the user; and

wherein the controlling the output state of the content comprises identifying information associated with a type of an output content corresponding to the identified execution level and controlling the output state of the content based on the type of the output content.

14. The method as claimed in claim 13 , wherein the type of the output content includes at least one of a still image output, repeated output of sub information corresponding to a specific step, output speed adjustment, or enlarged output of contents.

15. The method as claimed in claim 11 , wherein the obtaining context information associated with the user comprises obtaining posture information of the user based on the image obtained while the content that provides health information including an exercise posture is provided through a display; and

wherein the identifying the execution status of the user comprises:

identifying the execution status of the user associated with the exercise posture by comparing the obtained posture information of the user and posture information included in the reference information; and

providing a specific feedback for guiding the user's posture based on the execution status of the user.

16. The method as claimed in claim 15 , wherein the electronic apparatus is implemented as a robot that provides a healthcare service; and

wherein the providing the specific feedback comprises:

performing an operation of correcting the user's posture or providing guide information for guiding the user's posture so that the user executes the exercise posture provided by the content based on the execution status of the user.

17. The method as claimed in claim 15 , wherein the identifying the execution status of the user comprises:

identifying the execution level of the user based on at least one of a degree of correspondence between the obtained posture information and the posture information included in the reference information, a frequency of discrepancy between the obtained posture information and the posture information included in the reference information, or complexity of the exercise posture, and

wherein the providing the specific feedback comprises:

providing a feedback corresponding to the identified execution level.

18. The method as claimed in claim 11 , wherein the obtaining context information associated with the user comprises:

obtaining a status information of the user based on the image obtained while the content is provided through a display;

wherein the identifying the execution status of the user comprises:

identifying the execution status of the user associated with each of the plurality of steps by comparing the obtained status information and the status information included in the reference information; and

providing the guide information corresponding to the step requiring feedback based on the execution status of the user, wherein the step is from among the plurality of steps.

19. The method as claimed in claim 18 , wherein the obtaining the status information of the user comprises:

obtaining a first status information of the user by inputting the image to a first neural network model;

obtaining a second status information of the user by inputting a voice of the user to a second neural network model; and

wherein the providing the guide information comprises:

providing the guide information corresponding to the step requiring feedback based on the identified execution status, the first status information, and the second status information.

20. A non-transitory computer readable recording medium storing computer instructions that cause an electronic apparatus to perform an operation when executed by a processor of the electronic apparatus, wherein the operation comprises;

obtaining an image photographing a user through a camera while a content that provides information is displayed, the content comprising information corresponding to a plurality of steps;

obtaining context information associated with the user based on the image;

identifying an execution status of the user associated with the information provided by the content by comparing the obtained context information and the reference information associated with the content;

controlling an output state of the content based on the execution status of the user; and

controlling a display to provide guide information corresponding to a step, requiring a feedback, from among the plurality of steps based on the identified execution status.

Assignments (1)

Priority Claims (1)

Continuity (2)

References Cited (29)