Accurate video event inference using 3D information
Techniques for inferring whether an event is occurring in 3D space based on 2D image data and for maintaining a camera's calibration are disclosed. An image of an environment is accessed. Input is received, where the input includes a 2D rule imposed against a ground plane. The 2D rule includes conditions indicative of an event. A bounding box is generated and encompasses a detected object. A point within the bounding box is projected from a 2D-space image plane of the image into 3D space to generate a 3D-space point. Based on the 3D-space point, a 3D-space ground contact point is generated. That 3D-space ground contact point is reprojected onto the ground plane of the image to generate a synthesized 2D ground contact point. A location of the synthesized 2D ground contact point is determined to satisfy the conditions.
1. A computer system comprising:
at least one processor; and
at least one hardware storage device that stores instructions that are executable by the at least one processor to cause the computer system to:
select an image pixel included in an image of a real-world scene, wherein the image pixel corresponds to a real-world object included in the real-world scene;
project the image pixel from an image plane of the image to a three-dimensional (3D) space;
determine, using the projected image pixel as a reference, a first ground point that contacts a ground plane of the 3D space;
reproject a set of points, which includes the first ground point and a second ground point, from the 3D space to a ground plane of the image;
determine locations of the reprojected points relative to the ground plane of the image; and
determine that a specific real-world event is occurring with respect to the real-world object in the real-world scene based on the locations of the reprojected points.
2. The computer system of claim 1 , wherein the locations of the reprojected points are compared against a predetermined rule.
3. The computer system of claim 2 , wherein the predetermined rule includes a line that is drawn on the ground plane of the image.
4. The computer system of claim 3 , wherein a condition of the predetermined rule states that the event is occurring in the scene when the locations of the reprojected points are located on a particular side of the line.
5. The computer system of claim 2 , wherein the predetermined rule includes a shape that is drawn on the ground plane of the image.
6. The computer system of claim 5 , wherein a condition of the predetermined rule states that the event is occurring in the scene when the locations of the reprojected points are located within a space defined by the shape.
7. The computer system of claim 2 , wherein the predetermined rule includes one of a line, an oval, or a polygon.
8. The computer system of claim 1 , wherein the image pixel corresponds to a human represented in the image such that the real-world object is the human.
9. The computer system of claim 1 , wherein the first ground point is at a different location than a location for the projected image pixel in the 3D space.
10. A method that is implemented by a computer system, said method comprising:
selecting an image pixel included in an image of a real-world scene, wherein the image pixel corresponds to a real-world object included in the real-world scene;
projecting the image pixel from an image plane of the image to a three-dimensional (3D) space;
determining, using the projected image pixel as a reference, a first ground point that contacts a ground plane of the 3D space;
reprojecting a set of points, which includes the first ground point and a second ground point, from the 3D space to a ground plane of the image;
determining locations of the reprojected points relative to the ground plane of the image; and
determining that a specific real-world event is occurring with respect to the real-world object in the real-world scene based on the locations of the reprojected points.
11. The method of claim 10 , wherein the image pixel is included in a bounding box that encompasses the real-world object as it is represented in the image.
12. The method of claim 10 , wherein the image pixel corresponds to an abdominal region of a human who is depicted in the image.
13. The method of claim 10 , wherein the image pixel is located above the ground plane of the image.
14. The method of claim 10 , wherein the image pixel corresponds to one of a head of a human who is depicted in the image or an abdominal region of the human.
15. The method of claim 10 , wherein the locations of the reprojected points are compared against a predetermined rule.
16. A method that is implemented by a computer system, said method comprising:
selecting an image pixel included in an image of a real-world scene, wherein the image pixel corresponds to a real-world object included in the real-world scene;
projecting the image pixel to three-dimensional (3D) space;
determining, using the projected image pixel as a reference, a first point that contacts a first plane of the 3D space;
reprojecting a set of points, which includes the first point and a second point that is proximate to the first point, from the 3D space to a second plane of the image;
determining locations of the reprojected points relative to the second plane of the image; and
determining that a specific real-world event is occurring with respect to the real-world object in the real-world scene based on the locations of the reprojected points.
17. The method of claim 16 , wherein the first plane of the 3D space is a ground plane of the 3D space.
18. The method of claim 17 , wherein the second plane of the image is a ground plane of the image.
19. The method of claim 16 , wherein the first point and the second point are both ground contact points that contact a ground plane of the 3D space, the first plane being the ground plane.
20. The method of claim 16 , wherein the locations of the reprojected points are compared against a predetermined rule.