EMBEDDED AUTOMOTIVE PERCEPTION WITH MACHINE LEARNING CLASSIFICATION OF SENSOR DATA
This application discloses a computing system to implement perception in sensor data for an assisted or automated driving system of a vehicle. The computing system can generate a matchable representation of sensor measurement data collected by sensors mounted in a vehicle, and compare the matchable representation of the sensor measurement data to an object model describing a type of an object capable of being located proximate to the vehicle. Based on the comparison, the computing system can classify the sensor measurement data as corresponding to the type of the object based, at least in part, on the comparison of the matchable representation of the sensor measurement data to the object model. A control system for the vehicle can configured to control operation of the vehicle based, at least in part, on the classified type of the object for the sensor measurement data.
1 . A method comprising:
generating, by a computing system, a matchable representation of sensor measurement data collected by sensors mounted in a vehicle;
comparing, by the computing system, the matchable representation of the sensor measurement data to an object model describing a type of an object capable of being located proximate to the vehicle; and
classifying, by the computing system, the sensor measurement data as corresponding to the type of the object based, at least in part, on the comparison of the matchable representation of the sensor measurement data to the object model, wherein a control system for the vehicle is configured to control operation of the vehicle based, at least in part, on the classified type of the object for the sensor measurement data.
2 . The method of claim 1 , wherein generating the matchable representation of the sensor measurement data further comprises:
extracting lines corresponding to a structure of the sensor measurement data; and
determining relationships and connectivity between the lines, which generates the matchable representation of the sensor measurement data.
3 . The method of claim 1 , wherein generating the matchable representation of the sensor measurement data further comprises:
determining a boundary outline for the sensor measurement data; and
creating surface area data within the boundary outline based, at least in part, on the sensor measurement data, which generates the matchable representation of the sensor measurement data.
4 . The method of claim 1 , wherein comparing the matchable representation of the sensor measurement data to the object model further comprises:
selecting at least one classification graph from a plurality of classification graphs based, at least in part, on one or more detection events in the sensor measurement data, wherein each classification graph includes one or more computational nodes to implement the object model; and
traversing the computational nodes in the selected classification graph, which compares the matchable representation of the sensor measurement data to different characteristics of the object described in the object model associated with the selected classification graph.
5 . The method of claim 4 , wherein each of the computational nodes includes a representation of the object described in the object model with at least one of a pose of the object, a state of the object, an orientation of the object, a textural feature of the object, an inter-frame difference of the object, or one or more deformations for the object.
6 . The method of claim 4 , wherein each of the computational nodes in the selected classification graph is configured to generate a match distance between the matchable representation of the sensor measurement data to the representation of the object included in the corresponding computational node, wherein the traversal of the computational nodes in the selected classification graph is based, at least in part, on the match distance generated by one or more of the computational nodes.
7 . The method of claim 1 , further comprising estimating, by the computing system, a distance of the object associated with the sensor measurement data from the vehicle, wherein comparing the matchable representation of the sensor measurement data to the object models is based, at least in part, on the estimated distance of the object from the vehicle, a center of gravity of the sensor measurement data, or a center of a bounding box corresponding to the sensor measurement data.
8 . An apparatus comprising at least one memory device storing instructions configured to cause one or more processing devices to perform operations comprising:
generating a matchable representation of sensor measurement data collected by sensors mounted in a vehicle;
comparing the matchable representation of the sensor measurement data to an object model describing a type of an object capable of being located proximate to the vehicle; and
classifying the sensor measurement data as corresponding to the type of the object based, at least in part, on the comparison of the matchable representation of the sensor measurement data to the object model, wherein a control system for the vehicle is configured to control operation of the vehicle based, at least in part, on the classified type of the object for the sensor measurement data.
9 . The apparatus of claim 8 , wherein generating the matchable representation of the sensor measurement data further comprises:
extracting lines corresponding to a structure of the sensor measurement data; and
determining relationships and connectivity between the lines, which generates the matchable representation of the sensor measurement data.
10 . The apparatus of claim 8 , wherein generating the matchable representation of the sensor measurement data further comprises:
determining a boundary outline for the sensor measurement data; and
creating surface area data within the boundary outline based, at least in part, on the sensor measurement data, which generates the matchable representation of the sensor measurement data.
11 . The apparatus of claim 8 , wherein comparing the matchable representation of the sensor measurement data to the object model further comprises:
selecting at least one classification graph from a plurality of classification graphs based, at least in part, on one or more detection events in the sensor measurement data, wherein each classification graph includes one or more computational nodes to implement the object model; and
traversing the computational nodes in the selected classification graph, which compares the matchable representation of the sensor measurement data to different characteristics of the object described in the object model associated with the selected classification graph.
12 . The apparatus of claim 11 , wherein each of the computational nodes includes a representation of the object described in the object model with at least one of a pose of the object, a state of the object, an orientation of the object, a textural feature of the object, an inter-frame difference of the object, or one or more deformations for the object.
13 . The apparatus of claim 11 , wherein each of the computational nodes in the selected classification graph is configured to generate a match distance between the matchable representation of the sensor measurement data to the representation of the object included in the corresponding computational node, wherein the traversal of the computational nodes in the selected classification graph is based, at least in part, on the match distance generated by one or more of the computational nodes.
14 . The apparatus of claim 8 , wherein the instructions are further configured to cause the one or more processing devices to perform operations comprising estimating a distance of the object associated with the sensor measurement data from the vehicle, wherein comparing the matchable representation of the sensor measurement data to the object models is based, at least in part, on the estimated distance of the object from the vehicle, a center of gravity of the sensor measurement data, or a center of a bounding box corresponding to the sensor measurement data.
15 . A system comprising:
a memory device configured to store machine-readable instructions; and
a computing system including one or more processing devices, in response to executing the machine-readable instructions, configured to:
generate a matchable representation of sensor measurement data collected by sensors mounted in a vehicle;
compare the matchable representation of the sensor measurement data to an object model describing a type of an object capable of being located proximate to the vehicle; and
classify the sensor measurement data as corresponding to the type of the object based, at least in part, on the comparison of the matchable representation of the sensor measurement data to the object model, wherein a control system for the vehicle is configured to control operation of the vehicle based, at least in part, on the classified type of the object for the sensor measurement data.
16 . The system of claim 15 , wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to:
extract lines corresponding to a structure of the sensor measurement data; and
determine relationships and connectivity between the lines, which generates the matchable representation of the sensor measurement data.
17 . The system of claim 15 , wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to:
determine a boundary outline for the sensor measurement data; and
create surface area data within the boundary outline based, at least in part, on the sensor measurement data, which generates the matchable representation of the sensor measurement data.
18 . The system of claim 15 , wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to:
select at least one classification graph from a plurality of classification graphs based, at least in part, on one or more detection events in the sensor measurement data, wherein each classification graph includes one or more computational nodes to implement the object model; and
traverse the computational nodes in the selected classification graph, which compares the matchable representation of the sensor measurement data to different characteristics of the object described in the object model associated with the selected classification graph.
19 . The system of claim 18 , wherein each of the computational nodes in the selected classification graph is configured to generate a match distance between the matchable representation of the sensor measurement data to the representation of the object included in the corresponding computational node, wherein the traversal of the computational nodes in the selected classification graph is based, at least in part, on the match distance generated by one or more of the computational nodes.
20 . The system of claim 15 , wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to estimate a distance of the object associated with the sensor measurement data from the vehicle, and compare the matchable representation of the sensor measurement data to the object models based, at least in part, on the estimated distance of the object from the vehicle, a center of gravity of the sensor measurement data, or a center of a bounding box corresponding to the sensor measurement data.