Display device, remote control device to control display device, method of controlling display device, method of controlling server and method of controlling remote control device

1. A display device comprising:

a wireless communicator configured to transmit and receive a command or data with a remote control device;

a situation determination unit configured to determine a current state of a user based on at least one piece of situation information received from the remote control device using the wireless communicator or collected by an information collector; and

a controller configured to generate control information to provide a sense-based image service to the user according to the current state of the user by transmitting and receiving a command or data with the remote control device using the wireless communicator,

wherein the remote control device outputs a notification signal, and is able to transmit a user command after outputting the notification signal, and

wherein the controller generates the control information after the controller receives the user command from the remote control device.

2. The display device according to claim 1 , further comprising a determination information database to store determination information to determine the current state of the user based on the situation information,

wherein the situation determination unit searches the determination information database for determination information corresponding to the situation information and determines the current state of the user based on the determination information corresponding to the situation information if the determination information corresponding to the situation information is present.

3. The display device according to claim 1 , further comprising a control information database comprising control information of the remote control device or the display device corresponding to the current state of the user.

4. The display device according to claim 3 , wherein the controller detects the control information of the display device corresponding to the current state of the user by referring to the control information database and controls the display device according to the detected control information.

5. The display device according to claim 1 , wherein the controller corrects the generated control information according to the user command, and generates new control information.

6. The display device according to claim 5 , further comprising:

a memory to store the new control information generated by the controller; and

an update unit to analyze a user manipulation pattern based on the stored control information and to update the control information database based on the analyzed result.

7. The display device according to claim 1 , wherein the information collector comprises at least one of an optical sensor to sense light, an acoustic sensor to sense sound, a motion sensor to sense motion, a temperature sensor to sense a temperature, and a biosignal sensor to sense at least one of temperature, pulse, respiratory rate, and heart rate of a person.

8. The display device according to claim 1 , wherein the information collector comprises a face recognizer to detect a face shape from an image, to detect feature points of a face from the detected face shape, to analyze the positions of the feature points of the face, to generate face information and facial expression information, and to collect situation information.

9. The display device according to claim 1 , wherein the notification unit comprises at least one of a light emitting module to emit light, a vibration generation module to generate vibrations, and an acoustic output module to output sound.

10. The display device according to claim 1 , wherein the control information to control the display device performs at least one of:

controlling a camera of the display device to capture an image,

controlling the image captured by the camera of the display device to be displayed on the display device in a picture-in-picture (PIP) manner,

controlling a thumbnail image of at least one image captured by the camera of the display device to be displayed on a screen of the display device, and

controlling information about an image displayed on the screen of the display device to be displayed on the screen of the display device.

11. The display device according to claim 1 , further comprising a beam projector module to project an image onto an external projection surface.

12. The display device according to claim 1 , wherein the situation determination unit aggregates situation information and image information of an image played back by the display device and determines the current state of the user.

13. A remote control device comprising:

a wireless communicator configured to receive a command or data regarding a current state of a user determined based on situation information collected from a display device;

a notification unit configured to output a notification signal corresponding to the received current state of the user based on the command or data received by the wireless communicator; and

a controller configured to be able to transmit a user command to the display device using the wireless communicator after the notification unit outputs the notification signal,

wherein, after the display device receives the user command, the display device generates control information to provide a sense-based image service to the user according to the current state of the user.