System and method for object identification and tracking
View Patent ↗What is disclosed is a system and method for identifying materials comprising an object captured in a video and for using the identified materials to track that object as it moves across the captured video scene. In one embodiment, a multi-spectral or hyper-spectral sensor is used to capture a spectral image of an object in an area of interest. Pixels in the spectral planes of the spectral images are analyzed to identify a material comprising objects in that area of interest. A location of each of the identified objects is provided to an imaging sensor which then proceeds to track the objects as they move through a scene. Various embodiments are disclosed.
1. A method for object identification and tracking, the method comprising:
receiving a video comprising a plurality of image frames obtained using an imaging sensor operating in a first mode with high spatial resolution and high frame rate for object tracking;
identifying at least one an area of interest in said video sequence and switching said imaging sensor to operate in a second mode with higher spectral resolution for material analysis;
communicating a location of said area of interest to a spectral sensor;
capturing, by said spectral sensor, a spectral image comprising different spectral planes of said area of interest, each of said spectral planes have pixel locations corresponding to a reflectance at a wavelength band of interest;
analyzing pixel values in said spectral image to identify a material comprising at least one object in said area of interest, said analyzing comprising determining a classification for each pixel location in said spectral image based upon a pixel's intensity value cross-referenced against spectral characteristics of known materials in a materials spectral database;
cross-referencing said identified material of said object with a list of materials of interest; and
in response to said identified material matching any of said list of materials of interest, performing at least one of: continuously tracking said object using said imaging sensor until the occurrence of a terminating event, and initiating an alert.
2. The method of claim 1 , wherein said area of interest is identified by any of: a motion/background detection algorithm, and a user making a selection.
3. The method of claim 1 , wherein said spectral sensor comprises any of: a multi-spectral sensor and a hyper-spectral sensor.
4. The method of claim 1 , wherein said area of interest is identified in response to the occurrence of a triggering event.
5. The method of claim 1 , wherein said imaging sensor is one of: a monochrome video camera, RGB video camera, a multi-spectral video camera, and a hyper-spectral video camera.
6. The method of claim 1 , wherein said image sensor and said spectral sensor is the same sensor.
7. The method of claim 1 , wherein said spectral characteristics are determined using any of: a Linear Discriminant Analysis, and a data clustering method such as K-means, and wherein said spectral characteristics comprise any of: a relevant principal component, and a relevant independent component.
8. The method of claim 1 , further comprising reducing at least one of said captured spectral planes to a subspace to obtain a data reduced domain, said subspace comprising any of: a single channel gray level image, a 3-channel pseudo color image, and a n−1 channel color image where n is the number of channels in said data cube.
9. A system for object identification and tracking, comprising:
an imaging sensor operating in a first mode for capturing high spatial resolution video images and high frame rate for object tracking; and
a processor in communication with said imaging sensor and said controller, said processor executing machine readable instructions for performing the method of:
receiving a video comprising a plurality of image frames obtained using an imaging sensor operating in said first mode;
identifying at least one an area of interest in said video sequence and switching said imaging sensor to operate in a second mode with higher spectral resolution for material analysis;
cross-referencing said identified material of said object with a list of materials of interest; and
in response to said identified material matching any of said list of materials of interest, performing at least one of: continuously tracking said object using said imaging sensor until the occurrence of a terminating event, and initiating an alert.
10. The system of claim 9 , wherein said imaging sensor is one of: a monochrome video camera, a RGB video camera, a multi-spectral video camera, and a hyper-spectral video camera.
11. The system of claim 9 , wherein said image sensor and said spectral sensor are the same sensor.
12. The system of claim 9 , wherein said spectral sensor comprises any of: a multi-spectral sensor and a hyper-spectral sensor.
13. The system of claim 9 , wherein said spectral characteristics are determined using any of: a Linear Discriminant Analysis, and a data clustering method such as K-means, and wherein said spectral characteristics comprise any of: a relevant principal component, and a relevant independent component.
14. The system of claim 9 , further comprising reducing at least one of said spectral planes to a subspace to obtain a data reduced domain, said subspace comprising any of: a single channel gray level image, a 3-channel pseudo color image, and a n−1 channel color image where n is the number of channels in said data cube.