IP Library › Granted Patent US 10,423,978

Granted Patent B2

US 10,423,978 · App. 14/808,597 · Granted Sep 24, 2019

Method and device for playing advertisements based on relationship information between viewers

Inventors: Ying Huang (Beijing, CN); Junjun Xiong (Beijing, CN); Yanjun Gao (Beijing, CN); Chao Yan (Beijing, CN); Qiang Wang (Beijing, CN); Yaqin Qiu (Beijing, CN); Yan Chen (Beijing, CN)

Assignee: SAMSUNG ELECTRONICS CO., LTD.

G06Q30/0251 G06Q30/0201 G06Q30/0252 G06Q30/0269

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 10,423,978
App. No.: 14/808,597
Granted: Sep 24, 2019
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

A method of playing advertisements according to exemplary embodiments includes collecting data of at least two viewers near a display, extracting relationship information of the at least two viewers based on the data, determining advertisements to play on the display according to the relationship information, and playing the determined advertisements on the display.

Claims (70)

1. A method of playing advertisements on a display device including a camera and a depth sensor, the method comprising:

generating a first detection box corresponding to a first viewer and a second detection box corresponding to a second viewer, based on image data obtained via the camera;

collecting data of the first viewer and the second viewer based on the image data obtained via the camera;

determining a first distance between the first viewer and the display device via the depth sensor;

determining a second distance between the second viewer and the display device via the depth sensor;

determining a third distance between the first viewer and the second viewer based on a difference between the first distance and the second distance;

determining a fourth distance between the first viewer and the second viewer based on the first detection box and the second detection box;

determining relationship information of the first viewer and the second viewer based on the data, the third distance and the fourth distance; and

displaying advertisements determined according to the relationship information on the display device.

2. The method of claim 1 , wherein the camera comprises at least one selected from a visible light camera, and an infrared camera.

3. The method of claim 1 , wherein the determining of the relationship information comprises extracting body features of the first viewer and the second viewer from the data, and

wherein the body features of the first viewer and the second viewer comprise at least one selected from face features of the first viewer and the second viewer, and body related information of the first viewer and the second viewer.

4. The method of claim 3 , wherein the face features of the first viewer and the second viewer comprise at least one selected from the number of faces, features of the facial organs, face skin color, age, gender, viewpoints, and facial accessories.

5. The method of claim 3 , wherein the body related information of the first viewer and the second viewer comprises at least one selected from the number of bodies, features of body parts, movements of the bodies, hairstyle, clothing, body shapes, and carried items.

6. The method of claim 1 , further comprising storing personal information of the first viewer and the second viewer in advance, and

wherein the determining of the relationship information comprises:

searching for personal information that matches with the data from among the personal information; and

extracting, based on the personal information that matches with the data, the relationship information between the first viewer and the second viewer.

7. The method of claim 1 , wherein the determining of the relationship information comprises determining whether a relationship between the first viewer and the second viewer is family, friends, or colleagues.

8. The method of claim 1 , wherein the determining of the relationship information comprises extracting at least one piece of information from gender, age, skin color, hairstyle, clothing, body shapes, facial accessories, and carried items of the first viewer and the second viewer.

9. A device for playing advertisements, the device comprising:

a display device;

an input interface comprising a camera and a depth sensor; and

at least one processor configured to:

generate a first detection box corresponding to a first viewer and a second detection box corresponding to a second viewer based on image data obtained via the camera,

collect data of the first viewer and the second viewer based on the image data obtained via the camera,

determine a first distance between the first viewer and the display device via the depth sensor,

determine a second distance between the second viewer and the display device via the depth sensor,

determine a third distance between the first viewer and the second viewer based on a difference between the first distance and the second distance,

determine a fourth distance between the first viewer and the second viewer based on the first detection box and the second detection box,

determining relationship information of the first viewer and the second viewer based on the data of the first viewer and the second viewer, the third distance and the fourth distance, and

control the display device to display advertisements determined according to the relationship information.

10. The device of claim 9 , wherein the camera comprises at least one selected from a visible light camera, and an infrared camera.

11. The device of claim 9 , wherein the at least one processor is further configured to extract body features of the first viewer and the second viewer from the data, and determine advertisements to play based on the body features, and

wherein the body features of the first viewer and the second viewer comprise at least one selected from a distance between the first viewer and the second viewer, face features of the first viewer and the second viewer, and body related information of the first viewer and the second viewer.

12. The device of claim 9 , further comprising a memory configured to store personal information of the first viewer and the second viewer in advance, and

wherein the at least one processor is further configured to search for personal information that matches with the data from among the personal information, and extract the relationship information between the first viewer and the second viewer based on the personal information that matches the data.

13. The device of claim 9 , wherein the input interface further comprises a microphone, and

wherein the at least one processor is further configured to:

collect sound information via the microphone, and

determine the relationship information of the first viewer and the second viewer based on the data of the first viewer and the second viewer, the third distance, the fourth distance and the sound information.

14. The device of claim 13 , wherein the at least one processor is further configured to:

identify a threshold value,

identify the sound information as voice information based on a power level a converted power spectrum signal of the sound information exceeding the threshold value,

identify the sound information as noise information based on the power level the converted power spectrum signal of the sound information being less than the threshold value, and

filter the noise information from the sound information used to determine the relationship information of the first viewer and the second viewer.

15. The device of claim 14 , wherein the at least one processor is further configured to identify the power level of the converted power spectrum signal of the sound information using a fast Fourier Transform.

16. The device of claim 9 , wherein the at least one processor is further configured to:

extract body features of the first viewer and the second viewer from the image data, and

17. The device of claim 16 , wherein the at least one processor is further configured to:

identify a first direction corresponding to a first viewpoint of the first viewer,

identify a second direction corresponding to a second viewpoint of the second viewer, and

identify the body features of the first viewer and the second viewer based on the first direction and the second direction.

18. The device of claim 17 , wherein the at least one processor is further configured to:

identify the first direction by mapping an average 3D face model onto the first detection box and comparing the first detection box with the average 3D face model, and

identify the second direction by mapping the average 3D face model onto the second detection box and comparing the second detection box with the average 3D face model.

19. The device of claim 18 , wherein the at least one processor is further configured to:

identify first facial features of the first viewer and second facial features of the second viewer based on a machine learning algorithm,

identify the first direction based on the first facial features, and

identify the second direction based on the second facial features.

20. A non-transitory computer-readable recording medium having recorded thereon a program, which, when executed by a computer including a display device, a camera and a depth sensor, performs a method including: