Embedded metadata engines in digital capture devices
View Patent ↗A digital capture system, such as a digital still camera or digital video recorder, has an embedded real-time content-based analysis function in the capture device to extract metadata from the digital signals. In one embodiment, metadata (descriptive information about the digital content) is formatted and stored separately from the content. In another embodiment, the metadata may be formatted and combined with the digital content in a container format such as MPEG-7, QuickTime, or FlashPix.
1. An integrated data and real-time metadata capture system, comprising:
a digital capture device producing a digital representation of one or more forms of media content;
a feature extraction engine integrated with the digital capture device, the feature extraction engine having a plurality of feature extractors to automatically extract metadata in real-time from the digital content simultaneously with encoding of the content; and
a storage device capable of storing the encoded media content and the metadata, wherein selected portions of the metadata are associated with selected portions of the encoded media content.
2. The system defined in claim 1 , additionally comprising at least one collateral metadata source, wherein the collateral metadata is simultaneously captured with the media content.
3. The system defined in claim 2 , wherein the collateral metadata source includes at least one of the following: user input, device properties, and external sensors.
4. The system defined in claim 2 , wherein the digital capture device comprises a digital video camera capable of marking a video clip, wherein the marking is automatically triggered by a recording process of the camera.
5. The system defined in claim 4 , wherein the collateral metadata is associated with the video clip and is stored on the storage device with the metadata.
6. The system defined in claim 5 , wherein the collateral metadata includes at least one of the following: time/date, location, or scene/take.
7. The system defined in claim 1 , additionally comprising an output interface in data communication with the storage device and generating output data, wherein the output interface is capable of preserving a temporal relationship between a sequence of the metadata and the digital content.
8. The system defined in claim 7 , wherein the digital capture device comprises a digital still camera, and the output data comprises a FlashPix image containing image feature vectors as auxiliary metadata within an image file.
9. The system defined in claim 7 , wherein the digital capture device comprises a digital video camera including an audio capture capability.
10. The system defined in claim 7 , wherein the output data comprises an MPEG-7 digital video file containing a plurality of metadata tracks as embedded metadata within an MPEG-7 metadata framework.
11. The system defined in claim 7 , wherein the output data is transmitted to a computer.
12. The system defined in claim 11 , wherein the computer executes a media processing function.
13. The system defined in claim 1 , wherein the digital capture device comprises a digital still camera.
14. The system defined in claim 13 , wherein the metadata comprises a plurality of feature vectors.
15. The system defined in claim 1 , additionally comprising a format module configured to associate a time code with the encoded media content and the metadata.
16. The system defined in claim 1 , additionally comprising a time code module configured to provide a time code for association with the encoded media content and the metadata.
17. An integrated data and real-time metadata capture method, comprising:
sensing analog signals;
converting the analog signals to a digital representation of one or more forms of media content;
compressing the digital media content;
automatically extracting metadata in real-time from the digital media content simultaneously with the compressing of the digital media content; and
storing the digital media content and the metadata, wherein selected portions of the metadata are associated with selected portions of the digital media content.
18. The method defined in claim 17 , wherein automatically extracting metadata in real-time includes extracting a plurality of feature vectors from the digital media content.
19. The method defined in claim 17 , additionally comprising associating a time code with the digital media content and the metadata.
20. The method defined in claim 17 , additionally comprising generating a time code for association with the digital media content and the metadata.
21. The method defined in claim 17 , wherein the storing includes storing to a single storage device for the digital media content and the metadata.
22. An integrated data and real-time metadata capture system, comprising:
a digital capture device producing a digital representation of one or more forms of media content;
a frame buffer storing a frame of the digital content;
a feature extraction engine integrated with the digital capture device, the feature extraction engine having a plurality of feature extractors to automatically extract metadata in real-time from the digital content stored in the frame buffer simultaneously with encoding of the digital content; and
a storage device capable of storing the digital content and the metadata, wherein selected portions of the metadata are associated with selected portions of the digital content.
23. The system defined in claim 22 , additionally comprising at least one collateral metadata source, wherein the collateral metadata is simultaneously captured with the media content.
24. The system defined in claim 22 , additionally comprising a format module configured to associate a time code with the digital content and the metadata.
25. The system defined in claim 22 , additionally comprising a time code module configured to provide a time code for association with the digital content and the metadata.
26. The system defined in claim 22 , wherein the feature extraction engine generates a plurality of time-based tracks of metadata that index the digital content.
27. An integrated data and real-time metadata capture system, comprising:
a digital capture device producing a digital representation of one or more forms of media content;
a compressor integrated with the digital capture device, the compressor configured to encode the media content;
a feature extraction engine integrated with the digital capture device, the feature extraction engine having a plurality of feature extractors to automatically extract metadata in real-time from the digital content simultaneously with encoding of the content; and
a storage device capable of storing the encoded media content and the metadata, wherein selected portions of the metadata are associated with selected portions of the encoded media content.
28. The system defined in claim 27 , additionally comprising at least one collateral metadata source, wherein the collateral metadata is simultaneously captured with the media content.
29. The system defined in claim 28 , wherein the collateral metadata source includes at least one of the following: user input, device properties, and external sensors.
30. The system defined in claim 28 , wherein the digital capture device comprises a digital video camera capable of marking a video clip, wherein the marking is automatically triggered by a recording process of the camera.
31. The system defined in claim 30 , wherein the collateral metadata is associated with the video clip and is stored on the storage device with the metadata.
32. The system defined in claim 31 , wherein the collateral metadata includes at least one of the following: time/date, location, or scene/take.
33. The system defined in claim 27 , additionally comprising an output interface in data communication with the storage device and generating output data, wherein the output interface is capable of preserving a temporal relationship between a sequence of the metadata and the digital content.
34. The system defined in claim 27 , wherein the digital capture device comprises a digital still camera.
35. The system defined in claim 31 , wherein the metadata comprises a plurality of feature vectors.
36. The system defined in claim 27 , additionally comprising a format module configured to associate a time code with the encoded media content and the metadata.
37. The system defined in claim 27 , additionally comprising a time code module configured to provide a time code for association with the encoded media content and the metadata.
38. An integrated data and real-time metadata capture system, comprising:
a digital capture device producing a digital representation of one or more forms of media content including at least video content;
a feature extraction engine integrated with the digital capture device, the feature extraction engine having a plurality of feature extractors to automatically extract metadata via feature identification in real-time from the video content simultaneously with encoding of the video content; and
a storage device capable of storing the encoded video content and the metadata, wherein selected portions of the metadata are associated with selected portions of the encoded video content.
39. The system of claim 38 , wherein the feature identification includes speaker identification.
40. The system of claim 38 , wherein the feature identification includes face identification.
41. The system of claim 38 , wherein the feature identification includes optical character recognition (OCR).
42. The system of claim 38 , wherein the video content includes audio content.