System and method to help supplement a vehicles sensor data
One or more embodiments can include identifying a personal sensor on a user device, communicating a library of object recognition profiles for the personal sensor to the user device, capturing personal sensor data related to an environment, using the library of object recognition profiles for the personal sensor to use the captured personal sensor data to identify objects and/or features in the environment, and using the identified objects and/or features in the environment to supplement data collected by the autonomous vehicle's sensors. The library of object recognition profiles can be created by identifying the personal sensor on the user device, training a personal perception module to identify objects and/or other features in an environment setting based on data captured by the personal sensor, and using the training from the personal perception module to create the library of object recognition profiles for the personal sensor.
1 . A method for supplementing a vehicle's sensors with a personal sensor on a user device, the method comprising:
identifying the personal sensor on the user device;
communicating a library of object recognition profiles for the personal sensor to the user device;
selecting one of a plurality of user device holders in the vehicle into which the user device is to be placed;
illuminating a light on the one of the plurality of user device holders to indicate to a user to place the user device into the one of the plurality of user device holders;
subsequent to the user device being placed into the one of the plurality of user device holders, capturing personal sensor data related to an environment;
using the library of object recognition profiles for the personal sensor to use the captured personal sensor data to identify objects and/or features in the environment; and
using the identified objects and/or features in the environment to supplement data collected by the vehicle's sensors.
2 . The method of claim 1 , wherein identifying objects and/or features in the environment includes:
determining a location or orientation of the user device; and
using the location or orientation of the user device to supplement data collected by the vehicle's sensors.
3 . The method of claim 2 , wherein supplementing the data collected by the vehicle's sensors includes translating and rotating spatial data from the personal sensor based on the determined location and orientation of the user device.
4 . The method of claim 1 , wherein the library of object recognition profiles for the personal sensor are stored on the user device and are used by the user device to identify the objects and/or features in the environment based on the captured personal sensor data.
5 . The method of claim 1 , wherein the library of object recognition profiles is created by:
identifying the personal sensor on the user device;
training a personal perception module to identify the objects and/or features in the environment based on data captured by the personal sensor; and
using the training from the personal perception module to create the library of object recognition profiles for the personal sensor.
6 . The method of claim 1 , wherein the library of object recognition profiles are an executable set of machine instructions.
7 . The method of claim 6 , wherein the executable set of machine instructions are in binary code.
8 . The method of claim 1 , wherein the library of object recognition profiles for the personal sensor are stored in a network element.
9 . The method of claim 8 , the library of object recognition profiles for the personal sensor are communicated to the vehicle after the user device that includes the personal sensor is identified.
10 . The method of claim 1 , wherein the identified objects and/or features in the environment were not detected by the sensors on the vehicle.
11 . The method of claim 1 , wherein the personal sensor is a camera or light detection and ranging (LIDAR) sensor.
12 . The method of claim 1 , wherein a user of the user device is requested to position the user device in a certain position to allow the personal sensor on the user device to acquire the personal sensor data.
13 . The method of claim 1 , wherein the user device is a cellular phone or smartphone.
14 . A vehicle for supplementing vehicle sensor data with-supplemental sensor data from one or more sensors on a user device, the vehicle comprising:
a sensor suite including external vehicle sensors to sense an environment and generate sensor data;
a perception module to receive the sensor data and to acquire map data and to use the map data and the sensor data to generate vehicle real world environment data;
a user device interface module to communicate with the user device;
a user device resource usage module to use supplemental sensor data from one or more sensors on the user device to supplement the sensor data; and
a plurality of user device holders, wherein one of the plurality of user device holders is selected for receiving the user device,
wherein a light on the one of the plurality of user device holders is illuminated to indicate to a user to place the user device into the one of the plurality of user device holders, and
wherein the supplemental sensor data is collected subsequent to the user device being placed into the one of the plurality of user device holders.
15 . The vehicle of claim 14 , wherein the vehicle communicates a message to a user of the user device to position the user device in a specific area.
16 . The vehicle of claim 14 ,
wherein the one of the plurality of user device holders fixes a position of the one or more sensors on the user device in a specific location.
17 . At least one non-transitory machine-readable medium comprising one or more instructions that, when executed by at least one processor, causes the at least one processor to:
determine that sensor data from an autonomous vehicle's sensors needs to be supplemented with supplementary sensor data;
identify a supplementary sensor on a user device, wherein the supplementary sensor can provide the supplementary sensor data;
selecting one of a plurality of user device holders in the vehicle into which the user device is to be placed;
generating a sound from the one of the plurality of user device holders to indicate to a user to place the user device into the one of the plurality of user device holders;
if a user of the user device allows the supplementary sensor to be accessed, communicate a library of object recognition profiles for the supplementary sensor to the user device;
capture supplementary sensor data related to an environment in a vicinity of the user device;
use the library of object recognition profiles for the supplementary sensor to use the captured supplementary sensor data to identify objects and/or features in the environment in the vicinity of the user device; and
use the identified objects and/or features in the environment to supplement data collected by the autonomous vehicle's sensors.
18 . The least one machine-readable medium of claim 17 , wherein the library of object recognition profiles for the supplementary sensor are stored in a network element and communicated to the autonomous vehicle after the user device that includes the supplementary sensor is identified.
19 . The least one machine-readable medium of claim 17 , wherein the identified objects and/or features in the environment were not detected by the sensors on the autonomous vehicle.