Systems and methods for traffic monitoring using monocular depth estimation
Disclosed herein are systems and methods for traffic monitoring using monocular depth estimation. In one example, a system includes a processor and a memory having instructions that, when executed by the processor, cause the processor to generate a point cloud of a scene using a pre-trained monocular depth estimation network that receives an image of the scene as an input and generate traffic information of traffic at the scene using an output head that receives the point cloud and generates the traffic information.
1 . A system comprising:
a processor; and
a memory in communication with the processor, the memory having instructions that, when executed by the processor, cause the processor to:
generate a depth map of a scene using a pre-trained monocular depth estimation network that receives an image of the scene as an input;
generate a point cloud of the scene by back-projecting the depth map into a three-dimensional space; and
generate traffic information of traffic at the scene using an output head that receives the point cloud and generates the traffic information, wherein the traffic information includes the point cloud and points of the point cloud are modified by the output head to include, for each point, an identifier associated with an object, a velocity and direction of the object, and a distance between the object and at least one other object.
2 . The system of claim 1 , wherein the output head is trained separately from the pre-trained monocular depth estimation network.
3 . The system of claim 1 , wherein the traffic information includes static information and dynamic information of vehicles forming the traffic at the scene.
4 . The system of claim 3 , wherein:
the static information includes at least one of: identifiers of the vehicles, locations of the vehicles, a number of the vehicles, distances of the vehicles to a camera that generated the image, and distances between the vehicles; and
the dynamic information includes at least one of velocities of the vehicles and directions of the vehicles.
5 . The system of claim 1 , wherein the memory further includes instructions that, when executed by the processor, cause the processor to:
store a plurality of point clouds generated by the pre-trained monocular depth estimation network of images captured at different times; and
determine one or more temporal characteristics of objects forming points of the plurality of point clouds over time by comparing at least two of the plurality points clouds.
6 . The system of claim 1 , wherein the memory further includes instructions that, when executed by the processor, cause the processor to capture the image using at least one camera mounted on one or more of a vehicle and a fixed location.
7 . A method comprising:
generating a depth map of a scene using a pre-trained monocular depth estimation network that receives an image of the scene as an input;
generating a point cloud of the scene by back-projecting the depth map into a three-dimensional space; and
generating traffic information of traffic at the scene using an output head that receives the point cloud and generates the traffic information, wherein the traffic information includes the point cloud and points of the point cloud are modified by the output head to include, for each point, an identifier associated with an object, a velocity and direction of the object, and a distance between the object and at least one other object.
8 . The method of claim 7 , wherein the output head is trained separately from the pre-trained monocular depth estimation network.
9 . The method of claim 7 , wherein the traffic information includes static information and dynamic information of vehicles forming the traffic at the scene.
10 . The method of claim 9 , wherein:
the static information includes at least one of: identifiers of the vehicles, locations of the vehicles, a number of the vehicles, distances of the vehicles to a camera that generated the image, and distances between the vehicles; and
the dynamic information includes at least one of velocities of the vehicles and directions of the vehicles.
11 . The method of claim 7 , further comprising:
storing a plurality of point clouds generated by the pre-trained monocular depth estimation network of images captured at different times; and
determining one or more temporal characteristics of objects forming points of the plurality of point clouds over time by comparing at least two of the plurality points clouds.
12 . The method of claim 7 , further comprising capturing the image of the scene using at least one camera mounted on one or more of a vehicle and a fixed location.
13 . A non-transitory computer-readable medium including instructions that, when executed by a processor, cause the processor to:
generate a depth map of a scene using a pre-trained monocular depth estimation network that receives an image of the scene as an input;
generate a point cloud of the scene by back-projecting the depth map into a three-dimensional space; and
generate traffic information of traffic at the scene using an output head that receives the point cloud and generates the traffic information, wherein the traffic information includes the point cloud and points of the point cloud are modified by the output head to include, for each point, an identifier associated with an object, a velocity and direction of the object, and a distance between the object and at least one other object.
14 . The non-transitory computer-readable medium of claim 13 , wherein the output head is trained separately from the pre-trained monocular depth estimation network.
15 . The non-transitory computer-readable medium of claim 13 , wherein the traffic information includes static information and dynamic information of vehicles forming the traffic at the scene.
16 . The non-transitory computer-readable medium of claim 15 , wherein:
the static information includes at least one of: identifiers of the vehicles, locations of the vehicles, a number of the vehicles, distances of the vehicles to a camera that generated the image, and distances between the vehicles; and
the dynamic information includes at least one of velocities of the vehicles and directions of the vehicles.
17 . The non-transitory computer-readable medium of claim 13 , further comprising instructions that, when executed by the processor, cause the processor to:
store a plurality of point clouds generated by the pre-trained monocular depth estimation network of images captured at different times; and
determine one or more temporal characteristics of objects forming points of the plurality of point clouds over time by comparing at least two of the plurality points clouds.