Refrigerator for managing product stock and method therefor

1 . A method of managing product reception/release data of a refrigerator, the method comprising:

obtaining internal image data of the refrigerator using a camera provided in the refrigerator;

extracting high-reliability data from the internal image data of the refrigerator;

updating product reception/release data based on the high-reliability data;

creating a user database based on the updated product reception/release data; and

generating final product reception/release data by removing temporary reception/release data from the updated product reception/release data, via a processor,

wherein the extracting the high-reliability data from the internal image data of the refrigerator comprises at least removing product covering data,

wherein the removing of the product covering data comprises:

detecting a new product region;

extracting intersection region information of the new product region and an existing product region; and

when intersection region information is larger than a preset value, determining reception/release of an existing product using a similarity between a region other than the intersection region of the existing product region and a region corresponding to the region other than the intersection region of the existing product region before a new product is received, when a size of the new product region is smaller than that of the existing product region, and

wherein the determining of the reception/release of the existing product using the similarity comprises:

maintaining a reception status of the existing product when the similarity is larger than a preset value, and

determining that the existing product is released when the similarity is equal to or less than the preset value.

2 . The method of claim 1 , wherein the obtaining of the internal image data of the refrigerator using the camera provided in the refrigerator comprises:

obtaining the internal image data of the refrigerator using the camera in response to when a shooting command is received from an external device or in response to when a refrigerator door is opened or closed.

3 . The method of claim 1 , wherein the extracting of the high-reliability data from the internal image data of the refrigerator further comprises at least removing low-quality data from the internal image data of the refrigerator, and

wherein the low-quality data comprises image data obtained when a value of at least one of a speed sensor of the refrigerator, an illuminance sensor of the refrigerator and a degree of blur of the internal image data of the refrigerator exceeds a preset criterion.

4 . The method of claim 1 , wherein the extracting of the high-reliability data from the internal image data of the refrigerator further comprises at least removing low confidence data from the internal image data of the refrigerator, via a processor, and

wherein the removing of the low confidence data comprises determining that unspecified data is the low confidence data when a similarity of a recognized product is lower than a preset value through comparison with existing data.

5 . The method of claim 1 , wherein the removing the product covering data further comprises:

detecting a new product;

extracting intersection region information of the new product and an existing product; and

if the intersection region information is larger than a preset value, maintaining a reception status of the existing product and determining that the new product is received when a size of the new product is larger than that of the existing product.

6 . The method of claim 1 , wherein the removing of the product covering data further comprises:

detecting a new product;

extracting an intersection region of the new product and an existing product; and

excluding the intersection region information from the product covering data when the intersection region information is equal to or less than a preset value.

7 . The method of claim 1 , wherein the generating of the final product reception/release data by removing the temporary reception/release data of the updated product reception/release data comprises:

generating a product list included in images taken at at least one specific time using the high-reliability data and shooting time information;

grouping the product list at the at least one specific time into a certain period;

generating reception/release data using the grouped product list; and

correcting the generated reception/release data.

8 . The method of claim 7 , wherein the correcting of the generated reception/release data comprises maintaining the product list and an inventory list at a current time the same as before or after the current time, when the product list and inventory information before the current time and the product list and the inventory information after the current time are the same.

9 . The method of claim 7 , wherein the correcting of the generated reception/release data further comprises determining whether a product is included in a temporary reception/release product list.

10 . The method of claim 1 , further comprising:

generating an inventory list change amount of a product, an average of product change amounts and product consumption information using the user database; and

providing a user-customized shopping information using the generated information.

11 . A refrigerator comprising:

a communication unit;

a memory configured to store product reception/release data of at least one product in the refrigerator;

a camera configured to obtain internal image data of the refrigerator; and

a processor configured to extract high-reliability data from the internal image data of the refrigerator, update the product reception/release data based on the high-reliability data and create a user database based on the updated product reception/release data,

wherein the processor is further configured to generate final product reception/release data by removing temporary reception/release data from the updated product reception/release data, via a processor,

wherein the extracting the high-reliability data from the internal image data of the refrigerator comprises at least removing product covering data,

wherein the removing of the product covering data comprises:

detecting a new product region;

extracting intersection region information of the new product region and an existing product region; and

when intersection region information is larger than a preset value, determining reception/release of an existing product using a similarity between a region other than the intersection region of the existing product region and a region corresponding to the region other than the intersection region of the existing product region before a new product is received, when a size of the new product region is smaller than that of the existing product region, and

wherein the determining of the reception/release of the existing product using the similarity comprises:

maintaining a reception status of the existing product when the similarity is larger than a preset value, and

determining that the existing product is released when the similarity is equal to or less than the preset value.

12 . The refrigerator of claim 11 , wherein the processor is configured to communicate with an external device through the communication unit and obtain the internal image data of the refrigerator using the camera when a shooting command is received from the external device or when a refrigerator door is opened or closed.

13 . The refrigerator of claim 11 , wherein the processor is further configured to extract the high-reliability data by performing at least one of removing low-quality data from the internal image data of the refrigerator and removing low confidence from the internal image data of the refrigerator.

14 . A product reception/release management system comprising:

a refrigerator; and

a server communicating with the refrigerator,

wherein the refrigerator comprises a communication unit, a memory configured to store reception/release data, a camera configured to obtain internal image data of the refrigerator; and a processor,

wherein the server comprises:

a communication unit configured to communicate with the refrigerator; and

a processor configured to obtain the internal image data of the refrigerator through the communication unit and create a user database,

wherein the processor of the server is configured to extract high-reliability data from the internal image data of the refrigerator, update product reception/release data based on the high-reliability data and create a refrigerator reception/release database based on the updated product reception/release data, and

wherein the processor is further configured to generate final product reception/release data by removing temporary reception/release data from the updated product reception/release data, via a processor,

wherein the extracting the high-reliability data from the internal image data of the refrigerator comprises at least removing product covering data,

wherein the removing of the product covering data comprises:

detecting a new product region;

extracting intersection region information of the new product region and an existing product region; and

when intersection region information is larger than a preset value, determining reception/release of an existing product using a similarity between a region other than the intersection region of the existing product region and a region corresponding to the region other than the intersection region of the existing product region before a new product is received, when a size of the new product region is smaller than that of the existing product region, and

wherein the determining of the reception/release of the existing product using the similarity comprises:

maintaining a reception status of the existing product when the similarity is larger than a preset value, and

determining that the existing product is released when the similarity is equal to or less than the preset value.

15 . The product reception/release management system of claim 14 , wherein the processor of the server is further configured to perform at least one of removing low-quality data from the internal image data of the refrigerator and removing low confidence data from the internal image data of the refrigerator to extract the high-reliability data, update product reception/release data based on the high-reliability data and remove temporary reception/release data from the updated product reception/release data to generate final product reception/release data.

16 . The method of claim 1 , wherein the temporary reception/release data of the updated product reception/release data comprises a list of one or more products removed from the refrigerator and restored to the refrigerator within a specific time period.