DYNAMIC GENERATION OF DATA SETS FOR TRAINING MACHINE-TRAINED NETWORK
Some embodiments of the invention provide a novel method for training a multi-layer node network. Some embodiments train the multi-layer network using a set of inputs generated with random misalignments incorporated into the training data set. In some embodiments, the training data set is a synthetically generated training set based on a three-dimensional ground truth model as it would be sensed by a sensor array from different positions and with different deviations from ideal alignment and placement. Some embodiments dynamically generate training data sets when a determination is made that more training is required. Training data sets, in some embodiments, are generated based on training data sets for which the multi-layer node network has produced bad results.
25 . A method comprising:
capturing first image data comprising a first image captured by a first image capture device and a second image captured by a second image capture device;
computing, using the first image data and a machine-trained (MT) network, first misalignment data, wherein the first misalignment data represents a first misalignment of the first image capture device;
computing, using the first image data and the first misalignment data, first output data comprising a first depth value for a first object; and
storing the first depth value in memory.
26 . The method of claim 25 , wherein the first misalignment data comprises a first vector comprising distance data representing a distance of a first image capture device from a prior position of the first image capture device.
27 . The method of claim 25 , wherein the first misalignment data comprises a second vector comprising orientation data, wherein the orientation data comprises quaternion data representing a change in orientation of the first image capture device from a prior orientation of the first image capture device.
28 . The method of claim 25 , the method comprising:
computing, using the first image data, first disparity data representing a pixel shift between a first location of the first object in the first image and a second location of the first object in the second image.
29 . The method of claim 28 , wherein computing the first output data comprises:
computing, using the first misalignment data, inter-device alignment data comprising distance data representing a distance between the first image capture device and the second image capture device;
computing, using the first disparity data and the inter-device alignment data, the first output data.
30 . The method of claim 25 , comprising:
capturing second image data comprising a third image captured by the first image capture device and a fourth image captured by the second image capture device;
computing, using the second image data and the first misalignment data, second output data, wherein the second output data comprises at least a second depth value for the first object; and
computing, using the first output data and the second output data, a position and speed of the first object.
31 . The method of claim 30 , comprising:
executing, in response to the position and the speed of the first object, a collision avoidance operation, the collision avoidance operation including at least one of (i) outputting a warning, (ii) automatic braking, or (iii) automatic steering.
32 . The method of claim 31 , comprising:
triggering a misalignment data update for the first image capture device based on configuration data specifying trigger events for misalignment data update, wherein updating misalignment data comprises:
capturing, second image data comprising third image captured by the first image capture device and a fourth image captured by the second image capture device; and
computing, using the second image data and the MT network, second misalignment data.
33 . The method of claim 32 , comprising:
determining gyroscopic data;
determining acceleration data;
detecting, based on the gyroscopic data and acceleration data, an impact; and
triggering, based on the impact and the configuration data, an update of the misalignment data.
34 . The method of claim 32 , comprising:
determining a number of times that depth information has been determined using the first misalignment data; and
triggering, based on the number of times that depth information has been determined using the first misalignment data and the configuration data, an update of the misalignment data.
35 . The method of claim 32 , comprising:
capturing ambient condition data;
determining, based on the ambient condition data, a change in ambient conditions; and
triggering, based on the change in ambient conditions and the configuration data, an update of the misalignment data.
36 . A system comprising:
one or more processors;
a plurality of image capture devices;
one or more computer-readable media, the one or more computer-readable media storing processor-executable instructions which, when executed using the one or more processors, perform operations comprising:
capturing, first image data comprising a first image captured by a first image capture device and a second image captured by a second image capture device;
computing, using the first image data and a machine-trained (MT) network, first misalignment data, wherein the first misalignment data represents a first misalignment of the first image capture device;
computing, using the first image data and the first misalignment data, first output data comprising a first depth value for a first object; and
storing the first depth value in memory.
37 . The system of claim 36 , comprising a pre-processor, the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
generating, using the pre-processor and the first image data, a cropped first image and a cropped second image, wherein the first image data is not in a correct format for computations using the MT network.
38 . The system of claim 36 , the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
computing, using the first image data, first disparity data representing a pixel shift between a first location of the first object in the first image and a second location of the first object in the second image.
39 . The system of claim 38 , the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
computing, using the first misalignment data, inter-device alignment data comprising distance data representing a distance between the first image capture device and the second image capture device; and
computing, using the first disparity data and the inter-device alignment data, the first output data.
40 . The system of claim 36 , comprising an orientation sensor, the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
determining, using the orientation sensor, gyroscopic data;
detecting, based on the gyroscopic data, an impact; and
triggering, based on the impact, an update of the first misalignment data.
41 . The system of claim 36 , comprising an accelerometer, the one or more computer-readable media storing processor-executable instructions that, when executed using the one or more processors, perform operations comprising:
determining, using the accelerometer, acceleration data;
detecting, based on the acceleration data, an impact; and
triggering, based on the impact, an update of the first misalignment data.
42 . The system of claim 36 , comprising a sensor array comprising a plurality of image capture devices, wherein the first image capture device and second image capture device positions are determined relative to a center point of the sensor array.
43 . A computer program product comprising at least one non-transitory computer-readable storage medium storing software instructions that, when executed, cause a system to:
capture first image data comprising a first image captured by a first image capture device and a second image captured by a second capture device;
compute, using the first image data and a machine-trained (MT) network, first misalignment data, wherein the first misalignment data represents a first misalignment of the first image capture device;
compute, using the first image data and the first misalignment data, first output data comprising a first depth value for a first object; and
store the first depth value in memory.
44 . The computer program product of claim 43 , the at least one non-transitory computer-readable storage medium storing software instructions that, when executed, cause the system to:
trigger a misalignment data update for the first image capture device based on configuration data specifying trigger events for misalignment data update, wherein updating misalignment data causes the system to:
capture, second image data comprising third image captured by the first image capture device and a fourth image captured by the second capture device; and
compute, using the second image data and the MT network, second misalignment data.