IP Library Granted Patent US 7,242,810
Granted Patent B2
US 7,242,810 · App. 11/129,090 · Granted Jul 10, 2007

Multimodal high-dimensional data fusion for classification and identification

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,242,810
App. No.
11/129,090
Granted
Jul 10, 2007
Kind
B2
Abstract

A method is provided for evaluating identity of an object, the method including: converting feature information representing the object to a plurality of mathematically defined components; grouping the components into multiple modalities; producing respective first prediction information for each respective modality wherein the respective prediction information for each respective modality is based upon respective components grouped into that respective modality; and producing second prediction information based upon the respective first prediction information produced for the multiple respective modalities.

Claims (132)

1. A method of evaluating identity of an object comprising:

converting feature information representing the object to a plurality of mathematically defined components;

grouping the components into multiple modalities based upon measured correlation among components with each other;

wherein grouping the components involves grouping together within common modalities those components that have higher correlation with each other;

producing respective first prediction information for each respective modality wherein the respective prediction information for each respective modality is based upon respective components grouped into that respective modality; and

producing second prediction information based upon the respective first prediction information produced for the multiple respective modalities.

2. The method of claim 1 ,

wherein converting involves mapping the feature information to components in a mathematically different space from the original feature space.

3. The method of claim 1 ,

wherein converting involves independent component analysis.

4. The method of claim 1 ,

wherein converting involves principal component analysis.

5. The method of claim 1 ,

wherein converting involves independent component analysis; and

wherein converting involves principal component analysis.

6. The method of claim 1 ,

wherein grouping the components involves ensuring that total dimensions within individual modalities is below a prescribed threshold based upon the curse of dimensionality.

7. The method of claim 1 ,

measuring correlation among the components from each other.

8. The method of claim 1 ,

wherein grouping the components involves separating into different modalities those components that have lower correlation with each other.

9. The method of claim 1 ,

measuring correlation among the components; and

wherein grouping the components involves separating into different modalities those components that have lower correlation each other.

10. The method of claim 1 ,

measuring correlation among the components; and

wherein grouping the components involves ensuring that total dimensions within individual modalities is below a prescribed threshold based upon the curse of dimensionality.

11. The method of claim 1 ,

wherein grouping involves grouping the components into multiple modalities so as to minimize correlation among modalities and so as to limit dimensions within individual modalities to below a prescribed threshold based upon the curse of dimensionality.

12. The method of claim 1 ,

wherein producing respective first prediction information involves producing respective first classification prediction information for each respective modality.

13. The method of claim 1 ,

wherein producing respective first prediction information involves producing respective first classification prediction information for each respective modality; and

wherein producing second prediction information involves producing second classification prediction information based upon the respective first classification prediction information.

14. The method of claim 1 ,

wherein producing respective first prediction information involves respectively mapping feature information to respective new modality vectors for each respective modality; further including,

locating in at least one database respective previously stored modality vectors that are similar to the respective new modality vectors; and

wherein producing second prediction information includes predicting identification of the object based upon the located previously stored modality vectors.

15. The method of claim 1 ,

wherein producing respective first prediction information involves respectively mapping feature information to respective new modality vectors for each respective modality; further including,

locating in at least one database respective previously stored modality vectors that are similar to the respective new modality vectors; and

wherein producing second prediction information includes voting among located previously stored modality vectors to predict identification of the object.

16. The method of claim 1 further including:

receiving information concerning the object from multiple information sources; and extracting feature information about the object from the received information.

17. A method of evaluating identity of an object comprising:

converting the feature information to a plurality of mathematically defined components;

grouping the components into multiple modalities based upon measured correlation among components with each other;

wherein grouping the components involves grouping together within common modalities those components that have higher correlation with each other so as to minimize correlation among modalities and so as to limit dimensions within individual modalities to below a prescribed threshold based upon the curse of dimensionality;

producing respective first classification prediction information for each respective modality wherein the respective first classification prediction information for each respective modality is based upon respective components grouped into that respective modality;

producing second classification prediction information based upon a non-linear combination of the respective first classification prediction information produced for the multiple respective modalities.

18. A method of evaluating identity of an object comprising:

converting the feature information to a plurality of mathematically defined components;

grouping the components into multiple modalities based upon measured correlation with each other;

wherein grouping the components involves grouping together within common modalities those components that have higher correlation with each other so as to minimize correlation among modalities and so as to limit dimensions within individual modalities to below a prescribed threshold based upon the curse of dimensionality;

producing respective first prediction information by respectively mapping feature information to respective new modality vectors for each respective modality;

locating in at least one database respective previously stored modality vectors that are similar to the respective new modality vectors; and

producing second prediction information based upon the located previously stored modality vectors.

19. The method of claim 18 ,

wherein producing second prediction information includes voting among located previously stored modality vectors to predict identification of the object.

20. A system for evaluating identity of an object comprising:

means for convening feature information representing the object to a plurality of mathematically defined components;

means for grouping the components into multiple modalities based upon measured correlation of the components with each other;

wherein the means for grouping groups together within common modalities those components that have higher correlation with each other;

multiple first classifiers, each associated with modality, each producing respective first classification prediction information for each respective modality, based upon respective components grouped into that respective modality; and

a second classifier producing second classification prediction information based upon a non-linear combination of the respective first classification prediction information produced for the multiple respective modalities.

21. The system of claim 20 ,

wherein the means for grouping groups the components into multiple modalities so as to minimize correlation among modalities and so as to limit dimensions within individual modalities to below a prescribed threshold based upon the curse of dimensionality.

22. A system for evaluating identity of an object comprising:

means for converting feature information representing the object to a plurality of mathematically defined components;

means for grouping the components into multiple modalities based upon measured correlation among components with each other;

wherein the means for grouping groups together within common modalities those components that have higher correlation with each other;

multiple means for respectively mapping feature information to respective new modality vectors for each respective modality;

means for locating in at least one database respective previously stored modality vectors that are similar to the respective new modality vectors; and

means for producing prediction information based upon the located previously stored modality vectors.

23. The system of claim 22 ,

wherein the means for grouping groups the components into multiple modalities so as to minimize correlation among modalities and so as to limit dimensions within individual modalities to below a prescribed threshold based upon the curse of dimensionality.

24. The system of claim 22 ,

wherein the means for producing prediction information uses voting among the located previously stored modality vectors to produce the second prediction information.

25. Computer readable medium having executable instructions for performing a method comprising:

converting feature information representing an object to a plurality of mathematically defined components;

grouping the components into multiple modalities based upon measured correlation among components with each other;

wherein grouping the components involves grouping together within common modalities those components that have higher correlation with each other;

producing respective first prediction information for each respective modality wherein the respective prediction information for each respective modality is based upon respective components grouped into that respective modality; and

producing second prediction information based upon the respective first prediction information produced for the multiple respective modalities.

26. The computer readable medium of claim 25 ,

wherein converting involves mapping the feature information to components in a mathematically different space from the original feature space.

27. The computer readable medium of claim 25 ,

wherein converting involves independent component analysis.

28. The computer readable medium of claim 25 ,

wherein converting involves principal component analysis.

29. The computer readable medium of claim 25 ,

wherein convening involves independent component analysis; and

wherein convening involves principal component analysis.

30. The computer readable medium of claim 25 ,

wherein grouping the components involves ensuring that total dimensions within individual modalities is below a prescribed threshold based upon the curse of dimensionality.

31. The computer readable medium of claim 25 ,

wherein grouping involves grouping the components into multiple modalities so as to minimize correlation among modalities and so as to limit dimensions within individual modalities to below a prescribed threshold based upon the curse of dimensionality.

32. The computer readable medium of claim 25 ,

wherein producing respective first prediction information involves producing respective first classification prediction information for each respective modality; and

wherein producing second prediction information involves producing second classification prediction information based upon a non-linear combination of the respective first classification prediction information.

33. The computer readable medium of claim 25 ,

wherein producing respective first prediction information involves respectively mapping feature information to respective new modality vectors for each respective modality; further including,

locating in at least one database respective previously stored modality vectors that are similar to the respective new modality vectors; and

wherein producing second prediction information includes voting among located previously stored modality vectors to predict identification of the object.

34. Computer readable medium having executable instructions for performing a method comprising:

converting the feature information representing an object to a plurality of mathematically defined components;

grouping the components into multiple modalities based upon measured correlation among components with each other, so that components that have higher correlation with each other are grouped together within common modalitics so as to minimize correlation among modalities;

producing respective first classification prediction information for each respective modality wherein the respective first classification prediction information for each respective modality is based upon respective components grouped into that respective modality;

producing second classification prediction information based upon a non-linear combination of respective first classification prediction information produced for the multiple respective modalities.

35. Computer readable medium having executable instructions for performing a method comprising:

converting the feature information representing an object to a plurality of mathematically defined components;

grouping the components into multiple modalities based upon measured correlation among components with each other, so that components that have higher correlation with each other are grouped together within common modalities so as to limit dimensions within individual modalities to below a prescribed threshold based upon the curse of dimensionality;

producing respective first classification prediction information for each respective modality wherein the respective first classification prediction information for each respective modality is based upon respective components grouped into that respective modality;

producing second classification prediction information based upon a non-linear combination of the respective first classification prediction information produced for the multiple respective modalities.

36. Computer readable medium having executable instructions for performing a method comprising:

converting the feature information representing an object to a plurality of mathematically defined components;

grouping the components into multiple modalities based upon measured correlation among components with each other, so that components that have higher correlation with each other are grouped together within common modalities so as to minimize correlation among modalities;

producing respective first prediction information by respectively mapping feature information to respective new modality vectors for each respective modality;

locating in at least one database respective previously stored modality vectors that are similar to the respective new modality vectors; and

producing second prediction information based upon the located previously stored modality vectors.

37. Computer readable medium having executable instructions for performing a method comprising:

converting the feature information representing an object to a plurality of mathematically defined components;

grouping the components into multiple modalities based upon measured correlation among components with each other, so that components that have higher correlation with each other are grouped together within common modalities so as to limit dimensions within individual modalities to below a prescribed threshold based upon the curse of dimensionality;

producing respective first prediction information by respectively mapping feature information to respective new modality vectors for each respective modality;

locating in at least one database respective previously stored modality vectors that are similar to the respective new modality vectors; and

producing second prediction information based upon the located previously stored modality vectors.

38. A method of evaluating identity of an object comprising:

converting feature information representing the object to a plurality of mathematically defined components;

grouping the components into multiple modalities based upon measured correlation among components with each other;

wherein grouping the components involves separating into different modalities those components that have lower correlation with each other;

producing respective first prediction information for each respective modality wherein the respective prediction information for each respective modality is based upon respective components grouped into that respective modality; and

producing second prediction information based upon the respective first prediction information produced for the multiple respective modalities.

Assignments (9)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2024
From: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
To: TYCO FIRE & SECURITY GMBH
Reel/Frame 068494/0384 →
NUNC PRO TUNC ASSIGNMENT Recorded Feb 4, 2022
From: JOHNSON CONTROLS, INC.
To: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
Reel/Frame 058955/0472 →
NUNC PRO TUNC ASSIGNMENT Recorded Feb 4, 2022
From: JOHNSON CONTROLS US HOLDINGS LLC
To: JOHNSON CONTROLS, INC.
Reel/Frame 058955/0394 →
NUNC PRO TUNC ASSIGNMENT Recorded Feb 4, 2022
From: SENSORMATIC ELECTRONICS, LLC
To: JOHNSON CONTROLS US HOLDINGS LLC
Reel/Frame 058957/0138 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 30, 2021
From: SENSORMATIC ELECTRONICS LLC
To: JOHNSON CONTROLS US HOLDINGS LLC
Reel/Frame 058600/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 30, 2021
From: JOHNSON CONTROLS US HOLDINGS LLC
To: JOHNSON CONTROLS INC
Reel/Frame 058600/0080 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 30, 2021
From: JOHNSON CONTROLS INC
To: JOHNSON CONTROLS TYCO IP HOLDINGS LLP
Reel/Frame 058600/0126 →
MERGER Recorded Apr 16, 2020
From: PROXIMEX CORPORATION
To: SENSORMATIC ELECTRONICS, LLC
Reel/Frame 052412/0341 →
CORRECTIVE ASSIGNMENT TO CORRECT THE PROPER NAME OF THE ASSIGEE PREVIOUSLY RECORDED ON REEL 016555 FRAME 0334. ASSIGNOR(S) HEREBY CONFIRMS THE CORRECT NAME OF THE ASSIGNEE IS PROXIMEX CORPORATION. Recorded Feb 26, 2014
From: CHANG, EDWARD Y
To: PROXIMEX CORPORATION
Reel/Frame 032600/0225 →