Vision based brain-computer interface systems for performing activities of daily living
In one embodiment, assisting a user in performing an action in relation to an object of interest includes capturing an image of an environment, identifying boundaries of objects within the image, adaptively dividing the image into selectable cells so that edges of the cells generally aligning with boundaries of objects within the image, and detecting selection of one of the cells by the user, the cell encompassing at least part of the object of interest upon which the user wishes an action to be performed.
1. A method for assisting a user in performing an action in relation to an object of interest, the method comprising:
capturing an image of an environment;
identifying boundaries of objects within the image;
adaptively dividing the image into a grid of selectable cells having rows and columns so that edges of the rows and columns generally align with boundaries of objects within the image;
displaying the gridded image to the user; and
detecting selection of one of the cells by the user, the cell encompassing at least part of the object of interest.
2. The method of claim 1 , wherein capturing an image comprises capturing an image with a two-dimensional camera.
3. The method of claim 1 , wherein capturing an image comprises capturing an image with a three-dimensional camera that is configured to determine the depths of objects within the image.
4. The method of claim 1 , wherein the heights of the rows and the widths of the columns are determined by the locations of the boundaries of the objects within the image such that the heights and widths do not have uniform dimensions.
5. The method of claim 4 , wherein each cell of the grid comprises a center marker that acts as a seed point for the cell.
6. The method of claim 1 , wherein detecting selection of one of the cells comprises randomly highlighting the cells and monitoring a brain response of the user to determine the cell upon which the user is concentrating.
7. The method of claim 1 , further comprising:
zooming in on the cell selected by the user to form a zoomed image;
identifying boundaries of objects within the zoomed image;
adaptively dividing the image into selectable cells so that edges of the cells generally align with boundaries of objects within the image; and
detecting selection of one of the cells by the user.
8. The method of claim 1 , further comprising segmenting the selected cell to separate the object of interest from its background.
9. The method of claim 8 , further comprising performing object identification on the object of interest.
10. The method of claim 9 , further comprising determining an action that the user wishes to be performed on the object of interest and performing that action on the object of interest.
11. A non-transitory computer-readable medium that stores a brain-computer interface and control system, comprising:
logic configured to identify boundaries of objects within an image;
logic configured to adaptively divide the image into a grid of selectable cells having rows and columns, edges of the rows and columns generally aligning with boundaries of objects within the image;
logic configured to display the gridded image to the user; and
logic configured to detect selection of a cell by the user, the cell encompassing at least part of an object of interest upon which the user wishes an action to be performed.
12. The computer-readable medium of claim 11 , wherein the heights of the rows and the widths of the columns are determined by the locations of the boundaries of the objects within the image such that the heights and widths do not have uniform dimensions.
13. The computer-readable medium of claim 11 , wherein the logic configured to detect selection of a cell comprises logic configured to randomly highlight cells and monitor a brain response of the user to determine the cell upon which the user is concentrating.
14. The computer-readable medium of claim 11 , further comprising logic configured to zoom in on a selected cell to create a zoomed image to enable adaptive dividing of the zoomed image.
15. The computer-readable medium of claim 11 , further comprising logic configured to segment a selected cell to separate the object of interest from its background.
16. The computer-readable medium of claim 11 , further comprising logic configured to perform object identification on the object of interest.
17. The computer-readable medium of claim 11 , further comprising logic configured to determine an action that the user wishes to be performed on the object of interest and logic configured to control performance of that action on the object of interest.
18. A brain-computer interface system comprising:
a user interface including electrodes configured to detect brain waves;
a camera configured to capture an image of an environment that includes an object of interest upon which a user wishes an action to be performed; and
a computing device that executes:
logic configured to identify boundaries of objects within the image;
logic configured to adaptively divide the image into a grid of selectable cells having rows and columns so that edges of the rows and columns generally align with boundaries of objects within the image;
logic configured to display the gridded image to the user; and
logic configured to detect selection of a cell by the user, the cell encompassing at least part of the object of interest.
19. The system of claim 18 , wherein the heights of the rows and the widths of the columns are determined by the locations of the boundaries of the objects within the image such that the heights and widths do not have uniform dimensions.
20. The system of claim 18 , wherein the computing device further executes logic configured to zoom in on a selected cell to create a zoomed image to enable adaptive dividing of the zoomed image.