Systems and methods for automatic scale selection in real-time imaging
View Patent ↗A system and method for automatic scale selection in real-time image and video processing and computer vision applications. In one aspect, a non-parametric variable bandwidth mean shift technique, which is based on adaptive estimation of a normalized density gradient, is used for detecting one or more modes in the underlying data and clustering the underlying data. In another aspect, a data-driven bandwidth (or scale) selection technique is provided for the variable bandwidth mean shift method, which estimates for each data point the covariance matrix that is the most stable across a plurality of scales. The methods can be used for detecting modes and clustering data for various types of data such as image data, video data speech data, handwriting data, etc.
1. A method for detecting modes in a set of data, comprising the steps of:
receiving one of image data, video data, speech data and handwriting data, and extracting feature data points from the data;
determining a bandwidth for each data point; and
performing a variable bandwidth mean shift process for each data point using the bandwidth for the data point to detect modes in the data set.
2. The method of claim 1 , wherein the step of performing a variable bandwidth mean shift process comprises adaptively estimating a normalized density gradient of an underlying density to detect a mode in the data set.
3. The method of claim 1 , further comprising the step of clustering the data using the detected modes.
4. The method of claim 1 , wherein the step of performing a variable bandwidth mean shift process comprises iteratively computing a sequence of successive locations of a kernel until convergence at a point of zero gradient.
5. The method of claim 1 , wherein the step of performing a variable bandwidth mean shift process comprises associating a differently scaled kernel to each data point.
6. The method of claim 1 , wherein the step of determining a bandwidth for each data point comprises the steps of:
determining a fixed bandwidth parameter and pilot cstimatc;
determining a proportionality constant using the pilot estimate;
determining an adaptive bandwidth for each data point using the fixed bandwidth, the pilot estimate and the proportionality constant.
7. The method of claim 1 , wherein the step of determining a bandwidth for each data point comprises the steps of:
selecting a plurality of analysis scales;
for each analysis scale, partitioning the data set into one or more modes and determining a mean/covariance pair for each mode, wherein each data point is associated with a detected mode and a corresponding mean/covariance pair of the mode for each of the analysis scales;
for each data point, determining the most stable mean/covariance pair associated with the data point; and
for each data point, selecting for output the covariance of the most stable mean/covariance pair as a variable bandwidth for the data point.
8. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps for detecting modes in a set of data, the method steps comprising:
receiving one of image data, video data, speech data and handwriting data, and extracting feature data points from the data;
determining a bandwidth for each data point; and
performing a variable bandwidth mean shift process for each data point using the bandwidth for the data point to detect modes in the data set.
9. The program storage device of claim 8 , wherein the instructions for performing a variable bandwidth mean shift process comprise instructions for adaptively estimating a normalized density gradient of an underlying density to detect a mode in the data set.
10. The program storage device of claim 8 , further comprising instructions for performing the step of clustering the data using the detected modes.
11. The program storage device of claim 8 , wherein the instructions for performing a variable bandwidth mean shift process comprise instructions for iteratively computing a sequence of successive local ions of a kernel until convergence at a point of zero gradient.
12. The program storage device of claim 8 , wherein the instructions for performing a variable bandwidth mean shift process comprise instructions for associating a differently scaled kernel to each data point.
13. The program storage device of claim 8 , wherein the instructions for determining a bandwidth for each data point comprise instructions for performing the steps of:
determining a fixed bandwidth parameter and pilot estimate;
determining a proportionality constant using the pilot estimate;
determining an adaptive bandwidth for each data point using the fixed bandwidth, the pilot estimate and the proportionality constant.
14. The program storage device of claim 8 , wherein the instructions for determining a bandwidth for each data point comprise instructions for performing the steps of:
selecting a plurality of analysis scales;
for each analysis scale, partitioning the data set into one or more modes and determining a mean/covariance pair for each mode, wherein each data point is associated with a detected mode and a corresponding mean/covariance pair of the mode for each of the analysis scales;
for each data point, determining the most stable mean/covariance pair associated with the data point; and
for each data point, selecting for output the covariance of the most stable mean/covariance pair as a variable bandwidth for the data point.