Electronic apparatus for identifying an operating state of a robot device and controlling method thereof

1 . An electronic apparatus, comprising:

a communication interface configured to receive a beacon from a robot device via a direct communication;

a sensor;

a memory stored with data comprising a plurality of stored movement patterns corresponding to a normal operation of the robot device; and

at least one processor electrically coupled with the memory and configured to cause the electronic apparatus to perform operations to:

obtain a movement pattern of the robot device based on at least one of a plurality of first distances between the electronic apparatus and the robot device or sensing data from the sensor,

identify an operating state of the robot device as the normal operation based on a match between the obtained movement pattern and at least one stored movement pattern among the plurality of stored movement patterns;

identify the operating state of the robot device as an abnormal operation based on a mismatch between the obtained movement pattern and all of the plurality of stored movement patterns; and

output an alarm indicating that the robot device is operating abnormally based on the operating state of the robot device being the abnormal operation,

wherein the plurality of first distances between the electronic apparatus and the robot device is obtained based on a signal strength of the beacon measured at a plurality of intervals.

2 . The electronic apparatus of claim 1 , wherein

the at least one processor is further configured to:

obtain a plurality of clusters by classifying the plurality of stored movement patterns,

identify the operating state of the robot device as the normal operation based on match between a cluster corresponding to the obtained movement pattern and one of the plurality of clusters, and

identify the operating state of the robot device as the abnormal operation based on no match between the cluster corresponding to the obtained movement pattern and one of the plurality of clusters.

3 . The electronic apparatus of claim 2 , wherein

the at least one processor is further configured to:

obtain, based on a plurality of received movement patterns being received from an external electronic apparatus through the communication interface, a plurality of clusters that correspond to a normal operation by coordinating the plurality of received movement patterns and the plurality of stored movement patterns stored in the memory.

4 . The electronic apparatus of claim 1 ,

wherein the at least one processor is further configured to:

identify the operating state as the normal operation or the abnormal operation by inputting the obtained movement pattern in a trained neural network model, and

wherein the trained neural network model is a model trained to identify the operating state according to the obtained movement pattern as the normal operation or the abnormal operation based on the plurality of stored movement patterns corresponding to the normal operation in the data.

5 . The electronic apparatus of claim 1 ,

wherein the at least one processor is further configured to:

obtain a plurality of second distances between an external electronic apparatus and the robot device from the external electronic apparatus through the communication interface, and

obtain the movement pattern further based on the plurality of second distances, and

wherein the plurality of first distances or the plurality of second distances indicates a change in coordinates of the robot device within a space at which the robot device is positioned.

6 . The electronic apparatus of claim 1 , wherein

the at least one processor is further configured to:

identify a change in the plurality of first distances between the electronic apparatus and the robot device based on first sensing data received from the sensor,

identify a change in a plurality of second distances between an external electronic apparatus and the robot device based on second sensing data received from the external electronic apparatus through the communication interface,

identify a change in position of the robot device based on the change in the plurality of first distances and the change in the plurality of second distances, and

obtain the movement pattern according to the change in position.

7 . The electronic apparatus of claim 6 ,

wherein the sensor comprises a microphone,

wherein the first sensing data comprises a magnitude of sound output by the robot device received through the microphone, and

wherein the second sensing data comprises the magnitude of sound output by the robot device received through another microphone provided in the external electronic apparatus.

8 . The electronic apparatus of claim 1 ,

wherein the sensor comprises a camera, and

wherein the at least one processor is further configured to obtain the movement pattern of the robot device based on an image received from the camera.

9 . The electronic apparatus of claim 8 ,

wherein the at least one processor is further configured to:

identify an object from the image,

identify a movement pattern corresponding to the object, and

identify, based on the movement pattern of the robot device being different from a movement pattern corresponding to the object, the operating state as the abnormal operation, and

wherein the movement pattern corresponding to the object comprises a movement pattern for the robot device to avoid the object.

10 . The electronic apparatus of claim 8 , wherein

the at least one processor is further configured to:

determine a color displayed by the robot device from the image captured by the camera;

identify the operating state as the normal operation, based on a determination that the robot device is displaying a first color; and

identify the operating state as the abnormal operation, based on a determination that the robot device is displaying a second color.

11 . The electronic apparatus of claim 1 , wherein the at least one processor is further configured to provide, based on the operating state being identified as the abnormal operation, a notification indicating the operating state of the robot device to a user terminal device through the communication interface.

12 . A control method for an electronic apparatus including a memory storing data including a plurality of stored movement patterns corresponding to a normal operation of a robot device, the method comprising:

causing the electronic apparatus to perform operations comprising:

receiving, through a communication interface of the electronic apparatus, a beacon from the robot device via a direct communication;

obtaining a movement pattern of the robot device based on at least one of a plurality of first distances between the electronic apparatus and the robot device or sensing data of a sensor;

identifying an operating state of the robot device as a normal operation based on a match between the obtained movement pattern and at least one stored movement pattern among the plurality of stored movement patterns;

identifying the operating state of the robot device as an abnormal operation based on a mismatch between the obtained movement pattern and all of the plurality of stored movement patterns; and

outputting an alarm indicating that the robot device is operating abnormally based on the operating state of the robot device being the abnormal operation,

wherein the plurality of first distances between the electronic apparatus and the robot device is obtained based on a signal strength of the beacon measured at a plurality of intervals.

13 . The method of claim 12 , further comprises: obtaining a plurality of clusters by classifying the plurality of stored movement patterns, and

wherein the identifying comprises:

identifying the operating state of the robot device as the normal operation based on match between a cluster corresponding to the obtained movement pattern and one of the plurality of clusters, and

identifying the operating state of the robot device as the abnormal operation based on no match between the cluster corresponding to the obtained movement pattern and one of the plurality of clusters.

14 . The method of claim 13 , wherein

the obtaining the plurality of clusters comprises:

obtaining, based on a plurality of received movement patterns being received from an external electronic apparatus, a plurality of clusters that correspond to a normal operation by coordinating the plurality of received movement patterns and a plurality of stored movement patterns comprised in the electronic apparatus.

15 . The method of claim 12 , wherein the identifying comprises:

identifying the operating state as the normal operation and the abnormal operation by inputting the obtained movement pattern in a trained neural network model, and

wherein the trained neural network model is a model trained to identify the operating state according to the obtained movement pattern as the normal operation or the abnormal operation based on the plurality of stored movement patterns corresponding to the normal operation comprised in the data.

16 . The method of claim 12 , wherein the obtaining comprises:

obtaining a plurality of second distances between an external electronic apparatus and the robot device from the external electronic apparatus through the communication interface of the electronic apparatus, and

obtaining the movement pattern further based on the plurality of second distances, and

wherein the plurality of first distances or the plurality of second distances indicates a change in coordinates of the robot device within a space at which the robot device is positioned.

17 . The method of claim 12 , wherein the obtaining comprises:

identifying a change in the plurality of first distances between the electronic apparatus and the robot device based on first sensing data received from the sensor,

identifying a change in a plurality of second distances between an external electronic apparatus and the robot device based on second sensing data received from the external electronic apparatus through the communication interface,

identifying a change in position of the robot device based on the change in the plurality of first distances and the change in the plurality of second distances, and

obtaining the movement pattern according to the change in position.

18 . The method of claim 17 , wherein the sensor comprises a microphone,

wherein the first sensing data comprises a magnitude of sound output by the robot device received through the microphone, and

wherein the second sensing data comprises the magnitude of sound output by the robot device received through another microphone provided in the external electronic apparatus.

19 . The method of claim 12 , wherein the sensor comprises a camera, and

wherein the obtaining comprises obtaining the movement pattern of the robot device based on an image received from the camera.

20 . A non-transitory computer readable recording medium comprising a program for executing a control method for an electronic apparatus including a memory storing data including a plurality of stored movement patterns corresponding to a normal operation of a robot device, the method comprising:

causing the electronic apparatus to perform operations comprising:

receiving, through a communication interface of the electronic apparatus, a beacon from a robot device via a direct communication;

obtaining a movement pattern of the robot device based on at least one of a plurality of first distances between the electronic apparatus and the robot device or sensing data of a sensor;

identifying an operating state of the robot device as a normal operation based on a match between the obtained movement pattern and at least one stored movement pattern among the plurality of stored movement patterns;

identifying the operating state of the robot device as an abnormal operation based on a mismatch between the obtained movement pattern and all of the plurality of stored movement patterns; and

outputting an alarm indicating that the robot device is operating abnormally based on the operating state of the robot device being the abnormal operation,

wherein the plurality of first distances between the electronic apparatus and the robot device is obtained based on a signal strength of the beacon measured at a plurality of intervals.