Systems, methods, and computer program products for populating environment models
Systems, methods, and computer program products for managing and populating environment models are described. An environment model is accessed which represents an environment, and the environment model is populated with instances of object models. Locations where the instances of object models should be positioned in the environment model are identified, by determining where in the environment model a respective size of each instance when viewed from a vantage point at the environment model matches a size of the object represented by the respective instance when viewed from a corresponding vantage point at the environment.
1 . A method comprising:
accessing, by at least one processor, an environment model representation of an environment;
accessing, by the at least one processor, a first view of the environment from a first vantage point, the first vantage point having a position and a perspective in relation to the environment, wherein the first view comprises first image data having a first resolution and the first view includes an object in the environment;
accessing, in a library of object models, an object model representation of the object, the object model including dimension data indicative of spatial dimensions of the object; and
populating the environment model with an instance of the object model at a location in the environment model, wherein populating the environment model with the instance of the object model at the location includes:
generating a second view of the environment model from a second vantage point, wherein the second view comprises second image data having a second resolution and a position and a perspective of the second vantage point in relation to the environment model substantially match the position and the perspective of the first vantage point in relation to the environment;
identifying the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data corresponds to a number of pixels occupied by the object in the first image data; and
positioning the instance of the object model at the location.
2 . The method of claim 1 , wherein:
the first resolution is equal to the second resolution; and
identifying the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data corresponds to a number of pixels occupied by the object in the first image data comprises: identifying the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data is equal to a number of pixels occupied by the object in the first image data.
3 . The method of claim 1 , wherein:
the first resolution is different from the second resolution by a fixed ratio;
identifying the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data corresponds to a number of pixels occupied by the object in the first image data comprises: identifying the location in the environment model where a number of pixels occupied by the instance of the object model in the second image data is equal to a number of pixels occupied by the object in the first image data multiplied by the fixed ratio.
4 . The method of claim 1 , further comprising generating the object model representing the object in the library of object models.
5 . The method of claim 4 , wherein generating the object model representing the object comprises generating the object model representing the object, including the dimension data indicative of the spatial dimensions of the object.
6 . The method of claim 4 , further comprising capturing, by at least one image sensor, image data representing the object from multiple viewpoints, wherein generating the object model representing the object in the library of object models comprises generating the object model based on the captured image data from multiple viewpoints.
7 . The method of claim 4 , further comprising capturing, by at least one haptic sensor positioned at an actuatable member which contacts the object, haptic data representing the object, wherein generating the object model representing the object in the library of object models comprises generating the object model based on the captured haptic data.
8 . The method of claim 1 , wherein:
the environment is a three-dimensional environment;
the environment model is a three-dimensional environment model;
the first view comprises first two-dimensional image data representing the environment from the first vantage point;
the second view comprises second two-dimensional image data representing the environment model from the second vantage point; and
populating the environment model with the instance of the object model at the location further comprises, prior to identifying the location, positioning the instance of the object model in the second image data to correspond to a position of the object in the first image data.
9 . The method of claim 8 , wherein populating the environment model with the instance of the object model at the location further comprises, prior to identifying the location, orienting the instance of the object model in the second image data to correspond to an orientation of the object in the first image data.
10 . The method of claim 8 , further comprising determining a distance in the environment model between the second vantage point and the instance of the object model at the location.
11 . The method of claim 1 , wherein the environment is a physical environment, and the environment model is a representation of the physical environment.
12 . The method of claim 1 , wherein the environment is a virtual environment, and the environment model is a representation of the virtual environment.
13 . The method of claim 1 , wherein:
the at least one processor is carried by a robot body positioned at the environment; and
the robot body carries at least one non-transitory processor-readable storage medium which stores the library of object models and the environment model.
14 . The method of claim 1 , wherein:
the at least one processor is positioned at a robot controller remote from the environment; and
the robot controller includes at least one non-transitory processor-readable storage medium which stores the library of object models and the environment model.
15 . The method of claim 1 , wherein:
the at least one processor is carried by a robot body positioned at the environment;
the robot body carries a first at least one non-transitory processor-readable storage medium which stores the environment model;
a robot controller remote from the robot body and operable to provide control data to the robot body, includes a second at least one non-transitory processor-readable storage medium which stores the library of object models;
the robot body includes a communication interface communicatively couplable to the robot controller;
accessing the environment model representation of the environment comprises accessing, by the at least one processor, the environment model stored at the first at least one non-transitory processor-readable storage medium;
accessing the first view of the environment comprises accessing, by the at least one processor, the first view of the environment stored at the first at least one non-transitory processor-readable storage medium;
accessing, in the library of object models, the object model comprises accessing the object model in the library of models stored at the second at least one non-transitory processor-readable storage medium, via the communication interface;
generating a second view of the environment model from the second vantage point comprises generating, by the at least one processor, the second view of the environment model;
identifying the location in the environment model comprises identifying, by the at least one processor, the location in the environment model; and
positioning the instance of the object model at the location comprises updating, by the at least one processor, the environment model stored at the first at least one non-transitory processor-readable storage medium to include the instance of the object model at the location.
16 . The method of claim 1 , further comprising capturing, by at least one image sensor, image data representing the first view of the environment from the first vantage point.
17 . A method comprising:
accessing, by at least one processor, an environment model representation of an environment;
accessing, by the at least one processor, a first view of the environment from a first vantage point, the first vantage point having a position and a perspective in relation to the environment, wherein the first view includes an object in the environment;
capturing, by at least one haptic sensor positioned at an actuatable member which contacts the object, haptic data representing the object;
generating, based on the captured haptic data, an object model representing the object;
accessing, in a library of object models, the object model representation of the object, the object model including dimension data indicative of spatial dimensions of the object; and
populating the environment model with an instance of the object model at a location in the environment model, wherein populating the environment model with the instance of the object model at the location includes:
generating a second view of the environment model from a second vantage point, wherein a position and a perspective of the second vantage point in relation to the environment model substantially match the position and the perspective of the first vantage point in relation to the environment;
identifying the location in the environment model where at least one spatial dimension of the instance of the object model in the second view of the environment model from the second vantage point substantially matches a corresponding spatial dimension of the object in the first view of the environment from the first vantage point; and
positioning the instance of the object model at the location.