Methods and systems for multimedia trajectory annotation
View Patent ↗A system and method of providing annotated trajectories by receiving image frames from a video camera and determining a location based on the image frames from the video camera. The system and method can further include the steps of determining that the location is associated with a preexisting annotation and displaying the preexisting annotation. Additionally or alternatively, the system and method can further include the steps of generating a new annotation automatically or based on a user input and associating the new annotation with the current location.
1. A method of providing annotated trajectories, the method comprising:
receiving at least one image frame from a video camera of a mobile device, wherein the at least one image frame comprises at least one feature point;
determining a location based on the at least one feature point of the at least one image frame;
generating an annotation;
associating the annotation with the location;
determining that a second mobile device is nearby;
determining that the second mobile device is associated with an entry of a contact in an address book of the mobile device;
communicating with the second mobile device;
generating a second annotation automatically as a new event based on the communication with the second mobile device,
determining that a third annotation is associated with the location and an object in the at least one image frame; and
displaying, on a display of the mobile device, a visual representation of the third annotation that overlays a portion of the at least one image frame,
wherein determining the location based on the at least one feature point of the at least one image frame comprises:
identifying the at least one feature point in the at least one image frame from the video camera;
determining feature point information for the at least one feature point;
receiving at least one subsequent image frame from the video camera; and
identifying the at least one feature point in the at least one subsequent image frame from the video camera, wherein the location is determined based on changes in positions of the at least one features point between the at least one image frame and the at least one subsequent image frame.
2. The method of claim 1 , wherein generating the annotation is triggered by audio received via a microphone, a current time matching a pre-specified time stamp, input received from the user, or a combination thereof.
3. The method of claim 1 , wherein the annotation is automatically generated based on the location, audio received via a microphone, a current time, input received from a user, or combinations thereof.
4. The method of claim 1 , wherein:
the annotation is generated based on an input by a user; and
the input by the user is voice, text, a touch, a shake, or a combination thereof.
5. The method of claim 1 , further comprising:
receiving a search query from a user;
determining that the search query matches one or more aspects of the annotation; and
displaying an indication of a direction of and distance to the location.
6. A method of providing annotated trajectories, the method comprising:
receiving at least one image frame from a video camera of a mobile device, wherein the at least one image frame comprises at least one feature point;
determining a location based on the at least one feature point of the at least one image frame;
determining that the location and an object in the at least one image frame is associated with an annotation;
displaying, on a display of the mobile device, a visual representation of the annotation that overlays a portion of the at least one image frame;
determining that a second mobile device is nearby;
determining that the second mobile device is associated with an entry of a contact in an address book of the mobile device;
communicating with the second mobile device; and
generating a second annotation automatically as a new event based on the communication with the second mobile device,
wherein determining the location based on the at least one feature point of the at least one image frame comprises:
identifying the at least one feature point in the at least one image frame from the video camera;
determining feature point information for the at least one feature point;
receiving at least one subsequent image frame from the video camera; and
identifying the at least one feature point in the at least one subsequent image frame from the video camera, wherein the location is determined based on changes in positions of the at least one features point between the at least one image frame and the at least one subsequent image frame.
7. The method of claim 6 , wherein the annotation comprises textual information, an image, an audio file, a video file, or a combination thereof.
8. The method of claim 6 , further comprising:
generating an new annotation; and
associating the new annotation with the location.
9. The method of claim 8 , wherein the new annotation is automatically generated based on the location.
10. The method of claim 8 , wherein:
the annotation is generated based on an input by a user; and
the input by the user is voice, text, a touch, a shake, or a combination thereof.
11. A system for providing annotated trajectories comprising:
a processing system comprising one or more processors; and
a memory system comprising one or more computer-readable media, wherein the one or more computer-readable media contain instructions that, when executed by the processing system, cause the processing system to perform operations comprising:
receiving at least one image frame from a video camera, wherein the at least one image frames comprises at least one feature point;
determining a location based on the at least one feature point of the at least one image frame;
generating an annotation;
associating the annotation with the location;
determining that a mobile device is nearby;
determining that the mobile device is associated with an entry of a contact in an address book of the system;
communicating with the mobile device;
generating a second annotation automatically as a new event based on the communication with the mobile device;
determining that a third annotation is associated with the location and an object in the at least one image frame; and
displaying, on a display, a visual representation of the third annotation that overlays a portion of the at least one image frame;
wherein determining the location based on the at least one feature point of the at least one image frame comprises:
identifying the at least one feature point in the at least one image frame from the video camera;
determining feature point information for the at least one feature point;
receiving at least one subsequent image frame from the video camera; and
identifying the at least one feature point in the at least one subsequent image frame from the video camera, wherein the location is determined based on changes in positions of the at least one features point between the at least one image frame and the at least one subsequent image frame.
12. The system of claim 11 , wherein the annotation is automatically generated.
13. The system of claim 11 , wherein:
the annotation is generated based on an input by a user; and
the input by the user is voice, text, a touch, a shake, or a combination thereof.
14. The method of claim 1 ,
wherein the second annotation indicates a meeting with the contact, an identifier of the contact, a time of the meeting, and a location of the meeting along the trajectory.
15. The method of claim 14 , wherein the information exchanged with the second mobile device includes information describing the trajectory of the mobile device and previous location information of the mobile device.