IP Library › Granted Patent US 9,706,256
Granted Patent B2
US 9,706,256 · App. 14/570,866 · Granted Jul 11, 2017

System and method for making semantic annotation for objects in interactive video and interface for the system

Inventor: Geun Sik Jo (Incheon, KR)
Assignee: Geun Sik Jo
H04N21/47205G06K9/00711H04N21/234309H04N21/6125H04N21/8146H04N21/84H04N21/4725
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,706,256
App. No.
14/570,866
Granted
Jul 11, 2017
Kind
B2
Abstract

A system and method for making a semantic annotation for a moving object in an interactive video and an interface for the system is described. The annotation creating system may include an annotation creating unit adapted to create annotation information on a moving object contained in a video, and an annotation replaying unit adapted to execute a preview of the annotation information by the video frame or shot. The video and the annotation information may be registered in a platform for sharing the video.

Claims (60)

1. A video augmented object creating system, comprising:

an augmented object creating unit adapted to;

track a path of movement of an object displayed in a video across a first frame and a second frame of the video, wherein the path of movement of the object is tracked from a first position in the first frame to a second position in the second frame, and wherein the second position is distinct from the first position;

receive an interaction of a user with the object displayed in the video, the interaction comprising the user selecting the object in the video, and the interaction generating an augmented object instruction from the user, the augmented object instruction comprising a designation of the object displayed in the video by the user and a request for an augmented object associated with the object;

identify the selected object by machine object recognition and, in response to the selection of the object in the video, dynamically retrieve information about the object from a semantic web;

create augmented object information for the selected object based on the augmented object instruction and the retrieved information from the semantic web;

generate the augmented object from the augmented object information and display the retrieved information from the semantic web with the generated augmented object; and

associate the generated augmented object with the tracked path of movement of the object, wherein the augmented object tracks with the object displayed in the video; and

an augmented object replaying unit adapted to:

execute a preview of the augmented object information for the video by relocating the augmented object information for the object along the path of movement of the object from the first position in the first frame to the second position in the second frame; and

receive an editing instruction from the user in response to the executed preview, the editing instruction comprising a change in the augmented object information for the video,

wherein the video and the augmented object information are registered in a platform for sharing the video, wherein the user shares the video through the platform, and

wherein the augmented object information for the video is adapted to permit user interaction with the augmented object in an environment of the video, the interaction with the augmented object triggering further interaction with the augmented object in the environment of the video.

2. The video augmented object creating system of claim 1 , further comprising a video management unit adapted to register the video, and

wherein the video is uploaded to an internet or stored in a user terminal.

3. The video augmented object creating system of claim 1 , further comprising a video management unit adapted to register the video, and

wherein the video management unit is adapted to convert a format of the video to a compatible format and register the converted video.

4. The video augmented object creating system of claim 1 , further comprising a tool structured to create the augmented object information, and

wherein the tool is adapted to include functions for selecting the video for augmented object, selecting the object in the video, inputting the augmented object information for the object including an image or text to be annotated to the object, aligning frames or shots of the video, reviewing or modifying the augmented object, and executing the preview of the augmented object.

5. The video augmented object creating system of claim 1 , wherein the augmented object creating unit is adapted to create the augmented object information by means of an image input for the object, and

wherein the image is an image uploaded to an internet or registered in the platform.

6. The video augmented object creating system of claim 1 , wherein the augmented object creating unit is adapted to define a function to be performed when the object is selected, with respect to the augmented object information.

7. The video augmented object creating system of claim 1 , wherein the augmented object information is overlaid on the video based on a frame or time corresponding to the path of the object.

8. The video augmented object creating system of claim 1 , wherein unification of video with the augmented object information on an internet is achieved.

9. The video augmented object creating system of claim 1 , wherein the augmented object information is overlaid as the video frame displays on a video screen and the augmented object information makes the video interactive.

10. The video augmented object creating system of claim 1 , wherein the augmented object information is managed to be separated from the video.

11. A knowledge sharing system, comprising:

a video management unit adapted to register a video selected by an augmented object creating tool, the video including an object that moves along a path of movement from a first position in a first frame of the video to a second position in a second frame of the video, wherein the second position is distinct from the first position;

an augmented object management unit adapted to:

generate, in response to a selection by a user of the object, an augmented object instruction, the augmented object instruction comprising a designation of the object and a request for an augmented object associated with the object;

identify the selected object by machine object recognition and dynamically retrieve information about the object from a semantic web;

create augmented object information for the selected object based on the augmented object instruction and the retrieved information from the semantic web, the retrieved information from the semantic web to be displayed with the augmented object;

associate the augmented object with the path of movement of the object from the first position in the first frame of the video to the second position in the second frame of the video, wherein the augmented object tracks with the object displayed in the video; and

register the augmented object information created for the object displayed in the video by the augmented object creating tool; and

a video replaying unit adapted to:

apply the augmented object information to the video when the video is selected and replay the video including the information retrieved from the semantic web, the replayed video displaying the retrieved information for the object being relocated along the path of movement of the object from the first position in the first frame to the second position in the second frame,

wherein a web service is provided to exchange information with the augmented object creating tool and to search and watch the video,

wherein the user shares the video through the web service, and

wherein the augmented object information for the video is adapted to permit user interaction with the augmented object in an environment of the video, the interaction with the augmented object triggering further interaction with the augmented object in the environment of the video.

12. The knowledge sharing system of claim 11 , wherein the video management unit is adapted to convert a format of the video to a video format supported by a user terminal and register the converted video.

13. The video with knowledge sharing system of claim 11 , wherein the augmented object management unit is adapted to manage an image to be annotated and provide the image to the augmented object creating tool.

14. The video with knowledge sharing system of claim 11 , wherein the augmented object management unit is adapted to register a function, which is defined on the augmented object information by the augmented object creating tool, as a function to be performed when the object is selected, and

wherein the video replaying unit is adapted to overlay the function defined on the augmented object information with the object when the video is replayed.

15. The knowledge sharing system of claim 11 , wherein a semantic web and social network service (SNS) are provided to exchange information with the augmented object creating tool and to implement information push while watching the video.

16. A method for sharing a video with knowledge for a computer, comprising:

registering a video selected by an augmented object creating tool, the video including an object that moves along a path of movement from a first position in a first frame of the video to a second position in a second frame of the video, wherein the second position is distinct from the first position;

generating, in response to a selection by a user of the object, an augmented object instruction, the augmented object instruction comprising a designation of the object and a request for an augmented object associated with the object;

identifying the selected object by machine object recognition and dynamically retrieving information about the object from a semantic web;

creating augmented object information for the selected object based on the augmented object instruction and the retrieved information from the semantic web, the retrieved information from the semantic web to be displayed with the augmented object;

associating the augmented object with the path of movement of the object from the first position in the first frame of the video to the second position in the second frame of the video, wherein the augmented object tracks with the object displayed in the video;

registering the augmented object information created by the augmented object tool for the object that is displayed in the video;

applying the augmented object information to the video when the video is selected; and

replaying the video including the information retrieved from the semantic web, the replayed video displaying the retrieved information for the object being relocated along the path of movement of the object from the first position in the first frame to the second position in the second frame,

wherein a web service is provided to exchange information with the augmented object creating tool and to search and watch the video,

wherein the user shares the video through the web service, and

wherein the augmented object information for the video is adapted to permit user interaction with the augmented object in an environment of the video, the interaction with the augmented object triggering further interaction with the augmented object in the environment of the video.

17. The method of claim 16 , further comprising, before the video and the augmented object information are registered,

creating the augmented object information for the object in the video; and

executing a preview of the augmented object information for the video, the preview including the augmented object information for the object being relocated along the path of movement of the object from the first position in the first frame to the second position in the second frame.

18. The method of claim 16 , wherein a semantic web is provided to exchange information with the augmented object creating tool and to implement information push while watching the video.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 14, 2016
From: INHA INDUSTRY PARTNERSHIP INSTITUTE
To: JO, GEUN SIK
Reel/Frame 037962/0857 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 16, 2014
From: JO, GEUN SIK
To: INHA-INDUSTRY PARTNERSHIP INSTITUTE
Reel/Frame 034514/0956 →
Priority Claims (1)
KR 10-2014-0015962 · Feb 12, 2014 · national
Continuity (1)
Related Publication 20150229996A1 · Aug 13, 2015