Data augmentation for detour path configuring
This application is directed to augmenting training images used for generating vehicle driving models. A computer system obtains a first image of a road, identifies within the first image a drivable area of the road, obtains an image of a traffic safety object, and determines a detour path on the drivable area. The computer system determines positions of a plurality of traffic safety objects to be placed adjacent to the detour path, and generates a second image from the first image by adaptively overlaying a respective copy of the image of the traffic safety object at each of the positions of the plurality of traffic safety objects on the drivable area within the first image. The second image is added to a corpus of training images to be used by a machine learning system to generate a model for facilitating driving a vehicle (e.g., at least partial autonomously).
1 . A method for augmenting training images used for generating autonomous vehicle driving models, comprising:
at a computer system including one or more processors and memory:
obtaining a first image of a road that depicts a drivable area;
identifying within the first image the drivable area of the road having lane markings defining a lane of the road;
obtaining an image of a traffic safety object;
determining, without user intervention, positions for a plurality of traffic safety objects to be placed along one or more straight or curved lines that start from the lane of the road, wherein the one or more straight or curved lines define one or more vehicle paths that deviate from the lane markings;
generating, without user intervention, a second image from the first image by adaptively overlaying images of the traffic safety object at each of the positions, including overlaying some of the images of the traffic safety objects on the first image of the road at deviations from the determined positions, so as to emulate real-world not perfectly aligned placement of the traffic safety objects; and
adding the second image to a corpus of training images to be used to generate a model for facilitating at least partial autonomous driving of a vehicle.
2 . The method of claim 1 , further comprising:
training, using machine learning, the model using the corpus of training images, including the second image; and
distributing the model to one or more vehicles, including a first vehicle;
wherein, in use, the model is configured to process road images captured by the first vehicle to facilitate at least partially autonomously driving the first vehicle.
3 . The method of claim 1 , wherein adaptively overlaying the images of the traffic safety object comprises scaling a respective size of each image of the traffic safety object based on a respective position where the respective traffic safety object is to be placed.
4 . The method of claim 1 , wherein adaptively overlaying the images of the traffic safety object comprises adjusting an orientation of each image of the traffic safety object based on a direction normal to the drivable area at a corresponding position.
5 . The method of claim 1 , wherein adaptively overlaying the images of the traffic safety object comprises adjusting one or more image properties of each image of the traffic safety object.
6 . The method of claim 1 , further comprising:
obtaining information for a plurality of road features, including one or more of: a vehicle, a lane area, a shoulder area, an edge marking, a lane marking, a shoulder barrier structure, a road divider, a traffic light, a traffic sign, a road sign, a pedestrian, and a bicycle, wherein the images of the traffic safety object are overlaid at each of the positions based on the information for the plurality of road features.
7 . The method of claim 1 , further comprising determining a detour path on the drivable area, wherein the detour path is defined by two substantially parallel detour lane lines, and positions of one or more traffic safety objects are substantially on or near the detour lane lines.
8 . The method of claim 1 , further comprising determining a detour path on the drivable area, including:
determining a first detour lane line that is adjacent to the detour path and configured to define the detour path on the drivable area; and
identifying positions for one or more traffic safety objects on or near the first detour lane line.
9 . The method of claim 8 , wherein the one or more traffic safety objects comprises more than one traffic safety objects and determining the first detour lane line comprises:
determining one or more of: a total number of detour paths, a length of the first detour lane line, a number of objects on the first detour lane line, object spacings between each two immediately adjacent traffic safety objects, curvatures of the first detour lane line at the one or more traffic safety objects, and randomly generated deviations from the first detour lane line.
10 . The method of claim 8 , wherein determining the detour path on the drivable area comprises:
determining a second detour lane line parallel to the first detour lane line, the second detour lane line configured to define the detour path on the road jointly with the first detour lane line; and
identifying positions for a second plurality of traffic safety objects on or near the second detour lane line.
11 . The method of claim 8 , wherein the second image is associated with a ground truth, and the ground truth includes the first detour lane line, the method comprising, iteratively:
recognizing a lane line of the detour path using the model;
comparing the recognized lane line to the first detour lane line of the ground truth; and
adjusting the model to match the recognized detour lane line to the first detour lane line of the ground truth.
12 . The method of claim 1 , wherein determining the positions for the plurality of traffic safety objects further comprises:
selecting the positions in a single file line along the one or more straight or curved lines.
13 . A computer system, comprising:
one or more processors; and
memory storing one or more programs configured for execution by the one or more processors, the one or more programs comprising instructions for:
obtaining a first image of a road that depicts a drivable area;
identifying within the first image the drivable area of the road having lane markings defining a lane of the road;
obtaining an image of a traffic safety object;
determining, without user intervention, positions for a plurality of traffic safety objects to be placed along one or more straight or curved lines that start from the lane of the road, wherein the one or more straight or curved lines define one or more vehicle paths that deviate from the lane markings;
generating, without user intervention, a second image from the first image by adaptively overlaying images of the traffic safety object at each of the positions, including overlaying some of the images of the traffic safety objects on the first image of the road at deviations from the determined positions, so as to emulate real-world not perfectly aligned placement of the traffic safety objects; and
adding the second image to a corpus of training images to be used to generate a model for facilitating at least partial autonomous driving of a vehicle.
14 . The computer system of claim 13 , the one or more programs further comprising instructions for:
determining positions for the plurality of traffic safety objects to be placed adjacent to a detour path, including, for each of the plurality of traffic safety objects:
identifying one or more reference nodes in a respective image of the traffic safety object;
determining a respective location of each of the one or more reference nodes in a first coordinate system corresponding to a first space; and
converting the respective location of each of the one or more reference nodes in the first coordinate system to a respective location near a position of the respective traffic safety object in an image coordinate system.
15 . The computer system of claim 14 , wherein determining the positions further comprises, for each of the plurality of traffic safety objects:
obtaining a camera intrinsic parameter K and a conversion vector between the first coordinate system and a camera coordinate system;
wherein based on the camera intrinsic parameter K and the conversion vector, the respective location of each of the one or more reference nodes in the first coordinate system is converted to the respective location near the position of the respective traffic safety object in the image coordinate system.
16 . The computer system of claim 13 , the one or more programs further comprising instructions for determining a detour path on the drivable area, wherein the detour path is adjacent to a road feature, and the road feature is one of a vehicle, a person, a bike, a motorcycle, a traffic sign, and a road sign.
17 . The computer system of claim 16 , the one or more programs further comprising instructions for:
determining that the road feature is partially occluded by a first traffic safety object based on depth values of the road feature and the first traffic safety object, wherein the first traffic safety object is overlaid partially on the road feature.
18 . A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of a computer system, the one or more programs comprising instructions for:
obtaining a first image of a road that depicts a drivable area;
identifying within the first image the drivable area of the road having lane markings defining a lane of the road;
obtaining an image of a traffic safety object;
determining, without user intervention, positions for a plurality of traffic safety objects to be placed along one or more straight or curved lines that start from the lane of the road, wherein the one or more straight or curved lines define one or more vehicle paths that deviate from the lane markings;
generating, without user intervention, a second image from the first image by adaptively overlaying images of the traffic safety object at each of the positions, including overlaying some of the images of the traffic safety objects on the first image of the road at deviations from the determined positions, so as to emulate real-world not perfectly aligned placement of the traffic safety objects; and
adding the second image to a corpus of training images to be used to generate a model for facilitating at least partial autonomous driving of a vehicle.
19 . The non-transitory computer-readable storage medium of claim 18 , wherein the traffic safety object includes a cone structure.
20 . The non-transitory computer-readable storage medium of claim 18 , the one or more programs further comprising instructions for:
determining positions for the plurality of traffic safety objects to be placed adjacent to a detour path, including:
determining alternative positions of the plurality of traffic safety objects in a first coordinate system; and
converting the alternative positions of the plurality of traffic safety objects in the first coordinate system to the positions of the plurality of traffic safety objects in a camera coordinate system.
21 . The non-transitory computer-readable storage medium of claim 20 , wherein determining the positions further comprises, for each of the plurality of traffic safety objects:
obtaining a camera intrinsic parameter K and a conversion vector between the first coordinate system and the camera coordinate system;
wherein based on the camera intrinsic parameter K and the conversion vector, the alternative positions of the plurality of traffic safety objects in the first coordinate system are converted to the positions of the plurality of traffic safety objects in the camera coordinate system.