Refrigerator, operating method thereof and information providing system

1. A refrigerator, comprising:

a camera configured to photograph an item placed in one or more storage spaces;

a communicator configured to communicate with an information providing system;

a memory configured to store item information of the item and text information displayed on an exterior of the item; and

a processor configured to identify the item placed in the one or more storage spaces by recognizing text information included in a photographed image of the item photographed by the camera, and comparing the recognized text information with the text information displayed on the exterior of the item,

wherein in response to the item placed in the one or more storage spaces not identified in the comparison, the processor transmits the photographed image to the information providing system via the communicator, receives information of the item corresponding to the photographed image from the information providing system via the communicator and stores the received information of the item in the memory.

2. The refrigerator of claim 1 , wherein when the item placed in the one or more storage spaces is identified in the comparison, the processor recognizes expiration date information of the identified item based on text, and stores the expiration date information of the identified item in the memory.

3. The refrigerator of claim 2 , wherein the processor distinguishes whether the item identified in the comparison is a previously placed item in the one or more storage spaces or a newly placed item in the one or more storage spaces, based on the expiration date information of the item stored in the memory.

4. The refrigerator of claim 1 , wherein the recognized text information comprises trademark information displayed on the exterior of the item, and the item information comprises at least one of classification information of the item, shape information of the item, or manufacturer information of the item.

5. The refrigerator of claim 4 , wherein the processor receives, from the information providing system via the communicator, at least one of item information of an item that has not been identified, text information comprising expiration date information, or location information of the text information, and stores the received information in the memory.

6. The refrigerator of claim 1 , further comprising:

a main body which contains a storage compartment comprising the one or more storage spaces; and

a first door which is hinged to the main body to rotate in front of one side of the storage compartment,

wherein the one or more storage spaces comprise a storage space located inside the first door.

7. The refrigerator of claim 6 , further comprising a second door which exposes the storage space located inside the first door.

8. The refrigerator of claim 7 , wherein

the second door comprises a transparent display, and

the processor controls a light emitter disposed in the storage space located inside the first door such that the storage space located inside the first door becomes visible when a knocking event occurs on the second door while the second door is closed.

9. The refrigerator of claim 8 , wherein when the knocking event occurs on the second door, the processor outputs, via the transparent display, at least one of item information, text information, or notification information of an expiration date of the item disposed in the storage space located inside the first door, based on augmented reality.

10. The refrigerator of claim 1 , wherein the processor transmits, via the communicator, item information of an item nearing an expiration date to a predetermined mobile terminal.

11. A method for operating a refrigerator, the method comprising:

photographing an item placed in one or more storage spaces;

recognizing text information included in a photographed image of the item placed in the one or more storage spaces;

identifying the item placed in the one or more storage spaces by comparing the recognized text information with previously stored text information;

transmitting the photographed image of the item placed in the one or more storage spaces to an information providing system when the item placed in the one or more storage spaces is not identified in the comparison;

receiving information of the item corresponding to the photographed image from the information providing system; and

storing the received information of the item.

12. The method of claim 11 , further comprising, when the item placed in the one or more storage spaces is identified in the comparison, recognizing expiration date information of the identified item based on text, and storing the expiration date information of the identified item.

13. The method of claim 12 , further comprising identifying whether the item identified in the comparison is a previously placed item in the one or more storage spaces or a newly placed item in the one or more storage spaces, based on previously stored expiration date information of the item.

14. An information providing system configured to communicate with one or more refrigerators, comprising:

a system communicator;

a system memory configured to store item information of one or more items and text information displayed on an exterior of the one or more items; and

a system processor configured to receive, via the system communicator, a photographed image of an item which was not identified by the refrigerator, recognize the item in the photographed image, and transmit, via the system communicator, item information of the item and text information displayed on an exterior of the item to the refrigerator which provided the photographed image

wherein the system processor is further configured to store a first deep neural network model in the system memory, the first deep neural network model being configured to estimate item information of the item and text information displayed on the exterior of the item based on the photographed image of the item.