Robust windshield detection via landmark localization
View Patent ↗A system and method that includes training a classifier using uniquely defined landmark points along the windshield region based on an elastic deformation model. The deformation model uses mixtures of trees with a shared pool of parts and can be globally optimized with dynamic programming and still capture much relevant global elastic structure. Once a candidate area is identified in the scene, a learned threshold is applied to the classification score of the candidate area to determine if the candidate area is a windshield. The identified area is then cropped out for further downstream process.
1. A system configured to perform landmark-based image analysis for localization of a windshield within an image comprising:
an image capture device operably connected to a data processing device that captures an image of a target vehicle; and
a processor-usable medium embodying computer code, said processor-usable medium being coupled to said data processing device, said computer program code comprising instructions executable by said processor and configured for:
identifying a plurality of landmark points associated with a windshield portion of a vehicle, at least one of the landmark points corresponding to at least one side view mirror adjacent the windshield;
organizing each of the landmark points in a tree structure defined by spatial relationships between the landmark points; and
calculating a classification score;
wherein the classification score is generated using:
S(I,L)=App(I,L)+Shape(L), where I is the image of interest, and L={l i =(x i ,y i ), i=1, 2, . . . , N L } is the set of pixel locations for the N L landmarks defined for the windshield, App(I,L) sums the appearance evidence for placing a template for part i at location l i for i=1, 2, . . . , N L , and Shape(L) scores the specific spatial arrangement of parts L.
2. A system as set forth in claim 1 , wherein the landmark points define a tree having a start and end points each on a straight line segment of said windshield thereby minimizing the impact of relative position variation between tree start and end points on windshield identification.
3. A system as set forth in claim 2 , wherein the start and end points of the tree are at a lower center portion of the windshield.
4. A system as set forth in claim 1 , wherein said computer program code comprising instructions executable by said processor is further configured for detecting violations of lane requirements in managed lanes.
5. A system as set forth in claim 1 , wherein said computer program code comprising instructions executable by said processor is further configured for comparing the classification score to a threshold value and, if the classification score exceeds the threshold value, cropping a region identified by the landmark points.
6. A computer implemented method of enhancing an image comprising: using a data processing device to identify a plurality of landmark points associated with a windshield portion of a vehicle;
using a data processing device to organize each of the landmark points in a tree structure defined by spatial relationships between the landmark points and
using a data processing device to calculate a classification score based on the landmark points;
wherein the classification score is generated using:
S(I,L)=App(I,L)+Shape(L), where I is the image of interest, and L={l i =(x i ,y i ), i=1, 2, . . . , N L } is the set of pixel locations for the N L landmarks defined for the windshield, App(I,L) sums the appearance evidence for placing a template for part i at location l i for i=1, 2, . . . , N L , and Shape(L) scores the specific spatial arrangement of parts L.
7. A computer implemented method as set forth in claim 6 , further comprising comparing the classification score to a threshold value and, if the classification score exceeds the threshold value, cropping the region identified by the landmark points.
8. A computer implemented method as set forth in claim 6 , wherein the landmark points define a tree having a start and end points each on a straight line segment of said windshield thereby minimizing the impact of relative position variation between tree start and end points on windshield identification.
9. A computer implemented method as set forth in claim 8 , wherein the start and end points of the tree are at a lower center portion of the windshield.
10. A computer implemented method as set forth in claim 6 , further comprising detecting violations of lane requirements in managed lanes.
11. A non-transitory computer-usable medium for performing landmark-based image analysis for localization of a windshield within an image, said computer-usable medium embodying a computer program code, said computer program code comprising computer executable instructions configured for:
identifying a plurality of landmark points associated with a windshield portion of a vehicle using an elastic deformation model, at least one of the landmark points corresponding to at least one side view mirror adjacent the windshield;
organizing each of the landmark points in a tree structure defined by spatial relationships between the landmark points; and
calculating a classification score based on the landmark points;
wherein the classification score is generated using:
S(I,L)=App(I,L)+Shape(L), where I is the image of interest, and L={l i =(x i ,y i ), i=1, 2, . . . , N L } is the set of pixel locations for the N L landmarks defined for the windshield, App(I,L) sums the appearance evidence for placing a template for part i at location l i for i=1, 2, . . . , N L , and Shape(L) scores the specific spatial arrangement of parts L.
12. A non-transitory computer-usable medium as set forth in claim 11 , wherein the instructions further comprise comparing the classification score to a threshold value and, if the classification score exceeds the threshold value, cropping the region identified by the landmark points.
13. A non-transitory computer-usable medium as set forth in claim 11 , wherein the landmark points define a tree having a start and end points each on a straight line segment of said windshield thereby minimizing the impact of relative position variation between tree start and end points on windshield identification.
14. A non-transitory computer-usable medium as set forth in claim 11 , wherein the start and end points of the tree are at a lower center portion of the windshield.