Methods and apparatus for autonomous robotic control
Sensory processing of visual, auditory, and other sensor information (e.g., visual imagery, LIDAR, RADAR) is conventionally based on “stovepiped,” or isolated processing, with little interactions between modules. Biological systems, on the other hand, fuse multi-sensory information to identify nearby objects of interest more quickly, more efficiently, and with higher signal-to-noise ratios. Similarly, examples of the OpenSense technology disclosed herein use neurally inspired processing to identify and locate objects in a robot's environment. This enables the robot to navigate its environment more quickly and with lower computational and power requirements.
1. A method comprising:
acquiring a plurality of images of at least a portion of an environment with an image sensor;
constructing a spatial shroud fitting a form of an object in a first image of the plurality of images from the first image;
identifying the object in the first image based at least in part on the spatial shroud;
translating and/or transforming the spatial shroud based at least in part on a change in a position and/or an orientation of the image sensor;
determining if the spatial shroud fits the form of the object in a second image in the plurality of images; and
in response to determining that the spatial shroud fits the form of the object, identifying a position of the object in the environment based at least in part on the spatial shroud.
2. The method of claim 1 , wherein constructing the spatial shroud comprises segmenting the first image.
3. The method of claim 1 , wherein identifying the object includes creating a shroud-gated representation of a view of the object.
4. The method of claim 1 , further comprising:
extracting a feature from the first image; and
identifying the object in the first image based at least in part on the feature.
5. The method of claim 4 , wherein extracting the feature comprises applying a non-linear transformation to the first image.
6. The method of claim 4 , further comprising:
identifying similarities between the feature and a plurality of features identified in other images in the plurality of images.
7. The method of claim 6 , further comprising:
clustering the images in the plurality of images based at least in part on the similarities.
8. A system comprising:
an image sensor to acquire a plurality of images of at least a portion of an environment;
a processor communicably coupled to the image sensor, the processor to:
construct a spatial shroud fitting a form of an object in a first image of the plurality of images from the first image;
identify the object in the first image based at least in part on the spatial shroud;
translate and/or transform the spatial shroud based at least in part on a change in a position and/or an orientation of the image sensor;
determine if the spatial shroud fits the form of the object in a second image in the plurality of images; and
in response to determining that the spatial shroud fits the form of the object in the second image, identify a position of the object in the environment based at least in part on the spatial shroud.
9. The system of claim 8 , wherein the processor is configured to construct the spatial shroud by segmenting the first image.
10. The system of claim 8 , wherein the processor is configured to identify the object by creating a shroud-gated representation of a view of the object.
11. The system of claim 8 , wherein the processor is further configured to:
extract a feature from the first image; and
identify the object in the first image based at least in part on the feature.
12. The system of claim 11 , wherein the processor is configured to extract the feature by applying a non-linear transformation to the first image.
13. The system of claim 11 , wherein the processor is further configured to:
identify similarities between the feature and a plurality of features identified in other images in the plurality of images.
14. The system of claim 13 , wherein the processor is further configured to:
cluster the images in the plurality of images based at least in part on the similarities.
15. The method of claim 1 , further comprising:
positioning and/or orienting the image sensor with respect to the object based at least in part on the object.
16. The method of claim 1 , further comprising:
adjusting a field of view of the image sensor based at least in part on the change in the position and/or the orientation of the image sensor.
17. The system of claim 8 , further comprising:
an actuator, operably coupled to the processor and to the image sensor, to adjust a field of view of the image sensor based at least in part on the object.
18. The system of claim 17 , wherein the actuator is further configured to adjust the field of view of the image sensor based at least in part on the change in the position and/or the orientation of the image sensor.
19. The method of claim 1 , wherein the object is a previously unknown new object.
20. The method of claim 1 , further comprising:
determining that the spatial shroud does not fit the form of the object in a third image in the plurality of images; and
in response to determining that the spatial shroud does not fit the form of the object in the third image, identifying a second object in the third image different from the object; and
transmitting instructions to the image sensor to capture images of the second object.
21. The method of claim 1 , wherein constructing the spatial shroud occurs without supervision from a user.
22. The method of claim 1 , wherein constructing the spatial shroud occurs without manual labeling of the object.
23. The method of claim 1 , wherein constructing the spatial shroud occurs without corrupting or forgetting previously learned knowledge.