Operating mode determining method and operating mode determining device

1. A method of controlling an operation of a clothing treatment apparatus installed indoors, the method comprising:

obtaining first image information obtained by photographing clothing through a camera of the clothing treatment apparatus;

obtaining second image information obtained by photographing an entrance and exit of a user wearing the clothing through a camera for monitoring an entrance and exit of a room of a home; and

determining an operating mode of the clothing treatment apparatus according to an analysis result of both the first image information of the clothing in the clothing treatment apparatus and the second image information of the user wearing the clothing outside of the clothing treatment apparatus when the user enters and exits the home while wearing the clothing,

wherein the first image information and the second image information are obtained from images of identical clothing by different cameras at different times, and

wherein the determining of the operating mode comprises:

determining a clothing material from the first image information by using a first deep neural network that is pre-trained to determine a clothing material from a clothing image;

extracting management guide information of the clothing based on a database in which a management guide according to a clothing material type is stored in advance;

predicting a state of the clothing by extracting weather information between a going out time and a returning time of the user wearing the clothing outside of the home that are included in the second image information; and

determining the operating mode of the clothing treatment apparatus by combining the determined clothing material, the management guide information of the clothing, and the predicted state of the clothing.

2. The method of claim 1 , wherein the obtaining of the second image information comprises obtaining the second image information including a total wearing time between the going out time and the returning time of the user wearing the clothing outside of the home.

3. The method of claim 2 , further comprising, after the obtaining of the second image information and before the determining of the operating mode, obtaining location information for a location of the user wearing the clothing outside of the home between the going out time and the returning time from a terminal of the user.

4. The method of claim 3 , wherein the determining of the operating mode further comprises:

predicting the state of the clothing by extracting weather information of an area according to the location information between the going out time and the returning time included in the second image information.

5. The method of claim 4 , wherein the weather information comprises one or more of temperature, humidity, or fine dust degree of an area according to the location information,

wherein the predicting of the state of the clothing comprises predicting the state of the clothing using a second deep neural network that is pre-trained to predict a state of clothing that changes according to an environmental condition including one or more of temperature, humidity, or fine dust degree and a time when clothing is exposed to a corresponding environment.

6. The method of claim 1 , further comprising, in response to an analysis result of the first image information and the second image information, interworking with the clothing treatment apparatus and setting a washing mode of a laundry apparatus to wash the clothing.

7. A computer-readable recording medium storing a computer program for executing the method of claim 1 using a computer.

8. A device for controlling an operation of a clothing treatment apparatus installed indoors, the device comprising:

one or more processors; and

a memory coupled to the one or more processors,

wherein the one or more processors are configured to:

obtain first image information obtained by photographing clothing through a camera of the clothing treatment apparatus;

obtain second image information obtained by photographing an entrance and exit of a user wearing the clothing through a camera for monitoring an entrance and exit of a room of a home;

determine an operating mode of the clothing treatment apparatus according to an analysis result of both the first image information of the clothing in the clothing treatment apparatus and the second image information of the user wearing the clothing outside of the clothing treatment apparatus when the user enters and exits the home while wearing the clothing,

wherein the first image information and the second image information are obtained from images of identical clothing by different cameras at different times;

determine a clothing material from the first image information by using a first deep neural network that is pre-trained to determine a clothing material from a clothing image;

extract management guide information of the clothing based on a database in which a management guide according to a clothing material type is stored in advance;

predict a state of the clothing by extracting weather information between a going out time and a returning time of the user wearing the clothing outside of the home that are included in the second image information; and

determine the operating mode of the clothing treatment apparatus by combining the determined clothing material, the management guide information of the clothing, and the predicted state of the clothing.

9. The device of claim 8 , wherein the one or more processors are configured to obtain the second image information including total wearing time between the going out time and the returning time of the user wearing the clothing outside of the home.

10. The device of claim 9 , wherein the one or more processors are further configured to obtain location information for a location of the user wearing the clothing outside of the home between the going out time and the returning time from a terminal of the user after the obtaining of the second image information and before the determining of the operating mode.

11. The device of claim 10 , wherein the one or more processors are further configured to:

predict the state of the clothing by extracting weather information of an area according to the location information between the going out time and the returning time included in the second image information.

12. The device of claim 11 , wherein the weather information comprises one or more of temperature, humidity, or fine dust degree of an area according to the location information,

wherein the one or more processors are configured to predict a state of the clothing using a second deep neural network that is pre-trained to predict a state of clothing that changes according to an environmental condition including one or more of temperature, humidity, or fine dust degree and a time when clothing is exposed to a corresponding environment.

13. The device of claim 8 , wherein the one or more processors are further configured to interwork with the clothing treatment apparatus and set a washing mode of a laundry apparatus to wash the clothing in response to an analysis result of the first image information and the second image information.