Multi-modal localization
In one implementation, a method of localizing a device is performed by a device including a plurality of sensors, one or more processors, and non-transitory memory. The method includes obtaining a plurality of enrolled location signatures respectively associated with a plurality of locations. The method includes obtaining data from a plurality of sensors. The method includes generating a current location signature based on the data obtained from the plurality of sensors. The method includes determining a location of the device based on the current location signature and the plurality of enrolled location signatures.
1 . A method comprising:
at a device having one or more processors and non-transitory memory;
obtaining a plurality of enrolled location signatures respectively associated with a plurality of locations in an environment, wherein the plurality of enrolled location signatures are generated by the device based on sensor data from sensors captured in the environment during enrollment;
obtaining data from a plurality of sensors;
generating a current location signature based on the data obtained from the plurality of sensors; and
determining a location of the device in the environment by selecting one or more enrolled locations signatures from the plurality of enrolled location signatures closest to the current location signature, and setting the location of the device to a respective location in association with the one or more enrolled locations.
2 . The method of claim 1 , wherein obtaining the data from the plurality of sensors includes obtaining data from one or more sensors of the device.
3 . The method of claim 2 , wherein the one or more sensors of the device include all of the plurality of sensors.
4 . The method of claim 1 , wherein obtaining the data from the plurality of sensors includes obtaining data from one or more sensors of a secondary device.
5 . The method of claim 4 , wherein obtaining the data from the one or more sensors of the secondary device includes determining that the secondary device is within a threshold distance of the device and, in response to determining that the secondary device is within the threshold distance of the device, obtaining the data from the one or more sensors of the secondary device.
6 . The method of claim 4 , wherein the one or more sensors of the secondary device include all of the plurality of sensors.
7 . The method of claim 1 , wherein the plurality of sensors includes a plurality of sensors of different sensor types that generate data based on different physical properties.
8 . The method of claim 1 , wherein obtaining the current location signature includes generating a concatenated vector by concatenating the data obtained from the plurality of sensors.
9 . The method of claim 8 , wherein obtaining the current location signature further includes determining a feature vector indicative of an amount of each of a set of features in the concatenated vector.
10 . The method of claim 1 , wherein obtaining the current location signature includes:
determining, for each of the plurality of sensors, a sensor-specific feature vector indicative of the amount of each of a set of sensor-specific features in the data from the sensor; and
generating a concatenated feature vector by concatenating the sensor-specific feature vectors.
11 . The method of claim 1 , wherein determining the location of the device includes selecting the one or more enrolled locations signatures based on comparing the current location signature to the plurality of enrolled location signatures.
12 . The method of claim 11 , wherein selecting the one or more enrolled locations signatures includes selecting a single location of the plurality of locations as the respective location and wherein setting the location of the device includes setting the location of the device as the single location.
13 . The method of claim 11 , wherein selecting the one or more enrolled locations signatures includes selecting multiple locations of the plurality of locations and wherein setting the location of the device includes setting the location of the device to the respective location between the multiple locations.
14 . The method of claim 1 , wherein determining the location of the device includes determining a regression function based on the plurality of enrolled location signatures and the respectively associated plurality of locations and applying the regression function to the current location signature.
15 . The method of claim 1 , further comprising:
determining a current time; and
selecting a subset of the plurality of enrolled location signatures based on the current time and respective times associated with each of the plurality of enrolled location signatures,
wherein determining the location of the device is based on the current location signature and the subset of the plurality of enrolled location signatures.
16 . A device comprising:
a non-transitory memory; and
one or more processors to:
obtain a plurality of enrolled location signatures respectively associated with a plurality of locations in an environment, wherein the plurality of enrolled location signatures are generated by the device based on sensor data from sensors captured in the environment during enrollment;
obtain data from a plurality of sensors;
generate a current location signature based on the data obtained from the plurality of sensors; and
determine a location of the device in the environment by selecting one or more enrolled locations signatures from the plurality of enrolled location signatures closest to the current location signature, and setting the location of the device to a respective location in association with the one or more enrolled locations.
17 . The device of claim 16 , wherein obtaining the data from the plurality of sensors includes obtaining data from one or more sensors of a secondary device.
18 . The device of claim 16 , wherein the plurality of sensors includes a plurality of sensors of different sensor types that generate data based on different physical properties.
19 . The device of claim 16 , wherein the one or more processors are further to:
determine a current time; and
select a subset of the plurality of enrolled location signatures based on the current time and respective times associated with each of the plurality of enrolled location signatures,
wherein the one or more processors are to determine the location of the device based on the current location signature and the subset of the plurality of enrolled location signatures.
20 . A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device with a plurality of sensors, cause the device to:
obtain a plurality of enrolled location signatures respectively associated with a plurality of locations in an environment, wherein the plurality of enrolled location signatures are generated by the device based on sensor data from sensors captured in the environment during enrollment;
obtain data from a plurality of sensors;
generate a current location signature based on the data obtained from the plurality of sensors; and
determine a location of the device in the environment by selecting one or more enrolled locations signatures from the plurality of enrolled location signatures closest to the current location signature, and setting the location of the device to a respective location in association with the one or more enrolled locations.