Methods and apparatus for autonomous robotic control
Sensory processing of visual, auditory, and other sensor information (e.g., visual imagery, LIDAR, RADAR) is conventionally based on “stovepiped,” or isolated processing, with little interactions between modules. Biological systems, on the other hand, fuse multi-sensory information to identify nearby objects of interest more quickly, more efficiently, and with higher signal-to-noise ratios. Similarly, examples of the OpenSense technology disclosed herein use neurally inspired processing to identify and locate objects in a robot's environment. This enables the robot to navigate its environment more quickly and with lower computational and power requirements.
1. A system for automatically locating and identifying an object in an environment, the system comprising:
at least one sensor to acquire sensor data representing at least a portion of the environment;
at least one processor operably coupled to the at least one sensor; and
at least one memory operably coupled to the at least one processor, the at least one memory storing instructions that, when executed by the at least one processor, cause the at least one processor to implement:
a spatial attention module to produce a foveated representation of the object based at least in part on the sensor data, to track a position of the object within the environment based at least in part on the foveated representation, and to select another portion of the environment to be sensed by the at least one sensor based at least in part on the foveated representation of the object; and
a semantics module to determine an identity of the object based at least in part on the foveated representation of the object,
wherein the spatial attention module comprises a segmentation module to generate at least one contour representation of the object based at least in part on the sensor data,
wherein the spatial attention module further comprises a figure/ground segregation module to determine at least one spatial shroud fitting a form of the object based at least in part on the at least one contour representation of the object, and
wherein the sensor data comprises a plurality of images and the semantics module comprises:
a view layer to group views of the object in the plurality of images based at least in part on the at least one spatial shroud;
an object layer to map the views of the object to an object node associated with the object; and
a name layer to classify the object based at least in part on the object node.
2. The system of claim 1 , wherein the at least one sensor comprises an image sensor to acquire at least one image of the at least a portion of the environment.
3. The system of claim 2 , further comprising:
at least one actuator, operably coupled to the image sensor, to provide sensor position data representative of an orientation and/or a position of the image sensor, and
wherein the spatial attention module is configured to select the other portion of the environment based at least in part on the orientation and/or a position of the image sensor.
4. The system of claim 3 , wherein the
at least one memory is configured to store the sensor position data representing an orientation and/or a position of the image sensor and to store instructions that, when executed by the at least one processor, cause the processor to implement:
an inhibition of return module to inhibit repeated selections of the object based at least in part on the sensor position data stored in the memory.
5. The system of claim 1 , wherein the spatial attention module configured to select the other portion of the environment such that the object appears at or near a center of the other portion of the environment.
6. The system of claim 1 , wherein the spatial attention module comprises:
at least one log-polar transformation module to transform the sensor data into a log-polar representation of the environment so as to provide invariance to translation and/or rotation of the at least one sensor with respect to the object and/or so as to reduce processing time.
7. The system of claim 1 , wherein the view layer is configured to group the views of the object based at least in part on classification of the object by the name layer.
8. The system of claim 1 , wherein the semantics module is configured to learn the identity of the object based at least in part on the location of the object.
9. The system of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to implement:
a teaching module to provide a label for the object, and
wherein the semantics module is configured to assign the label to the object.
10. A method of automatically locating and identifying an object in an environment, the method comprising:
(A) estimating a position and/or an orientation of at least one sensor with respect to the environment;
(B) acquiring, with the at least one sensor, sensor data representing at least a portion of the environment;
(C) producing a foveated representation of the object based at least in part on the sensor data acquired in (B);
(D) determining an identity of the object based at least in part on the foveated representation of the object produced in (C); and
(E) selecting another portion of the environment to be sensed by the at least one sensor based at least in part on the foveated representation of the object produced in (C) and the position and/or the orientation estimated in (A);
(F) acquiring additional sensor data, with the at least one sensor, in response to selection of the other portion of the environment in (D), wherein:
(A) comprises acquiring a plurality of images,
(D) comprises generating at least one contour representation of the object based at least in part on at least one image and determining at least one spatial shroud fitting a form of the object based at least in part on the at least one contour representation of the object, and
(E) comprises:
(E1) grouping views of the object in the plurality of images based at least in part on the at least one spatial shroud;
(E2) mapping the views of the object to an object node associated with the object; and
(E3) classifying the object based at least in part on the object node.
11. The method of claim 10 , wherein (D) comprises selecting the other portion of the environment such that the object appears at or near a center of the other portion of the environment.
12. The method of claim 10 , wherein (D) comprises inhibiting repeated selections of a given portion of the environment based at least in part on the position estimated in (A).
13. The method of claim 10 , wherein (D) comprises transforming the sensor data into a log-polar representation of the environment so as to provide invariance to translation and/or rotation and/or so as to reduce processing time.
14. The method of claim 10 , wherein (E1) comprises grouping the views of the object based at least in part on classification of the object in (E3).