IP Library Granted Patent US 11,284,020
Granted Patent B2
US 11,284,020 · App. 16/986,518 · Granted Mar 22, 2022

Apparatus and method for displaying graphic elements according to object

Inventors: Kyunghwa Kim (Suwon-si, KR); Saebyuk Sheen (Suwon-si, KR); Chunhyoung Park (Suwon-si, KR)
Assignee: Samsung Electronics Co., Ltd.
H04N5/272G06T11/60H04N5/23219H04N5/232935H04N5/2621
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,284,020
App. No.
16/986,518
Granted
Mar 22, 2022
Kind
B2
Abstract

An electronic device and a method of operating the same are provided. The electronic device includes a camera module, a display, a memory, and a processor configured to be operatively connected to the camera module, the display, and the memory, wherein the processor is further configured to display a preview image obtained from the camera module on the display, to recognize an object from the preview image, to detect a drawing input based on the object, to generate a graphic element by the detected drawing input, and to display the graphic element to correspond to the object by tracking the object. In addition, various embodiments are possible.

Claims (60)

1. An electronic device comprising:

a camera module;

a display;

a memory; and

a processor configured to be operatively connected to the camera module, the display, and the memory,

wherein the processor is further configured to:

display a preview image obtained from the camera module on the display,

recognize an object from the preview image,

detect a drawing input based on the object on the displayed preview image,

generate a graphic element by the detected drawing input,

display the graphic element to correspond to the object by tracking the object on the displayed preview image,

track a change in the object from the preview image obtained in real time while the drawing input is detected or the graphic element is displayed,

track a presence or an absence, a size, an angle, or a location of the object when an attribute of the object is a person, and

track a change in the electronic device when the attribute of the object is a thing.

2. The electronic device of claim 1 ,

wherein the object comprises at least one of the person or the thing, and

wherein the processor is further configured to:

recognize the person as the object when the camera module is driven by a front camera, and

recognize the person or the thing as the object when the camera module is driven by a rear camera.

3. The electronic device of claim 1 , wherein the processor is further configured to display a visual cue guiding that the drawing is possible, on the display based on the recognized object.

4. The electronic device of claim 1 , wherein the processor is further configured to display the drawable region on the display when the attribute of the object is the person.

5. The electronic device of claim 1 , wherein the processor is further configured to:

acquire coordinate information in which the drawing input is detected, based on a feature point of the object when the attribute of the object is the person, and

generate relationship information by mapping the feature point of the object and the coordinate information of the drawing input.

6. The electronic device of claim 5 , wherein the processor is further configured to temporarily store the generated relationship information in the memory in association with graphic information.

7. The electronic device of claim 5 , wherein the processor is further configured to display the graphic element on the object based on the relationship information.

8. The electronic device of claim 1 , wherein the processor is further configured to:

acquire location information or posture information of the electronic device at a time point at which the drawing input is detected as a spatial coordinate value when the attribute of the object is the thing, and

temporarily store the spatial coordinate value in the memory in association with graphic information.

9. The electronic device of claim 8 , wherein the processor is further configured to display the graphic element on the display based on the spatial coordinate value.

10. The electronic device of claim 1 , wherein the processor is further configured to vary and display at least one of a size, an angle, or a location of the graphic element in proportion to at least one of the size, the angle, or the location of the object included in the preview image obtained in real time from the camera module.

11. The electronic device of claim 1 , wherein the processor is further configured to remove the graphic element for the object when the object is not identified in the preview image obtained in real time from the camera module.

12. The electronic device of claim 1 , wherein the processor is further configured to:

display a first graphic element corresponding to a first object and a second graphic element corresponding to a second object in a first preview image,

recognize the first object and the second object from a second preview image obtained after the first preview image, and

remove the first graphic element corresponding to the first object when the first object is not recognized in the second preview image and display the second graphic element corresponding to the second object.

13. A method of operating an electronic device, the method comprising:

displaying a preview image obtained from a camera module of the electronic device on a display of the electronic device;

recognizing an object from the preview image;

detecting a drawing input based on the object;

generating a graphic element by the detected drawing input; and

displaying the graphic element to correspond to the object by tracking the object on the displayed preview image; and

tracking a change in the object from the preview image obtained in real time while the drawing input is detected or the graphic element is displayed,

wherein the tracking the change in the object comprises:

tracking a presence or an absence, a size, an angle, or a location of the object when an attribute of the object is a person, and

tracking a change in the electronic device when the attribute of the object is a thing.

14. The method of claim 13 , further comprising:

displaying a visual cue guiding that the drawing is possible, on the display based on the recognized object.

15. The method of claim 13 , further comprising:

displaying a drawable region on the display when the attribute of the object is the person.

16. The method of claim 13 , wherein the displaying of the graphic element comprises:

acquiring coordinate information in which the drawing input is detected, based on a feature point of the object when the attribute of the object is the person;

generating relationship information by mapping the feature point of the object and the coordinate information of the drawing input; and

displaying the graphic element on the object based on the relationship information.

17. The method of claim 13 , wherein the displaying of the graphic element comprises:

acquiring location information or posture information of the electronic device as a spatial coordinate value at a time point at which the drawing input is detected when the attribute of the object is the thing; and

displaying the graphic element on the display based on the spatial coordinate value.

18. The method of claim 13 , further comprising:

tracking a change in the object from a preview image obtained in real time while the drawing input is detected or the graphic element is displayed; and

controlling a display of the graphic element based on the tracked change in the object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 6, 2020
From: KIM, KYUNGHWA; SHEEN, SAEBYUK; PARK, CHUNHYOUNG
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 053418/0501 →
Priority Claims (1)
KR 10-2019-0095767 · Aug 6, 2019 · national
Continuity (1)
Related Publication 20210044762A1 · Feb 11, 2021