Systems and methods for comparing localization and vision data to identify an avoidance region

1. A surgical system, comprising:

a localizer configured to generate localizer data associated with a surgical object;

a vision device configured to generate image data associated with the surgical object and an environment of the surgical object within a field-of-view of the vision device; and

one or more controllers configured to:

associate a first virtual object with the surgical object;

determine a pose of the first virtual object based on the localizer data;

merge the image data and the localizer data in a common coordinate system to identify data points in the image data that have coordinates located outside of the first virtual object; and

associate a second virtual object with the identified data points.

2. The surgical system of claim 1 , wherein:

the first virtual object includes a 3D model of the surgical object;

the image data includes a 3D surface map of the surgical object and the environment of the surgical object; and

the one or more controllers are configured to merge the 3D model with the 3D surface map to identify the data points in the image data that have coordinates located outside of the first virtual object.

3. The surgical system of claim 1 , wherein the second virtual object is configured to constrain movement of a robotically controlled surgical instrument relative to the identified data points.

4. The surgical system of claim 1 , wherein:

the surgical object is a target site of a patient; and

the one or more controllers are configured to associate the first virtual object with a volume of material planned to be removed from the target site.

5. The surgical system of claim 1 , wherein:

the surgical object is an instrument configured to manipulate tissue; and

the one or more controllers are configured to associate the first virtual object with the instrument or a portion of the instrument.

6. The surgical system of claim 1 , wherein:

the surgical object is a robotic manipulator; and

the one or more controllers are configured to associate the first virtual object with the robotic manipulator or a portion of the robotic manipulator.

7. The surgical system of claim 1 , wherein the second virtual object defines a region to be avoided.

8. The surgical system of claim 1 , wherein:

the identified data points include a sensitive anatomical structure to be avoided; and

the one or more controllers are configured to associate the second virtual object with the sensitive anatomical structure to be avoided.

9. The surgical system of claim 1 , wherein:

the identified data points includes a surgical tool to be avoided; and

the one or more controllers are configured to associate the second virtual object with the surgical tool to be avoided.

10. The surgical system of claim 1 , wherein:

a tracker is coupled to the surgical object;

the localizer is configured to cooperate with the tracker to generate the localizer data associated with the surgical object; and

the localizer comprises one of: an optical localizer, an electromagnetic localizer, or a radio frequency localizer.

11. The surgical system of claim 1 , wherein the image data generated by the vision device comprises color image data.

12. The surgical system of claim 1 , wherein the vision device is mounted to an adjustable arm.

13. A method of operating a surgical system, the surgical system including a localizer configured to generate localizer data associated with a surgical object, a vision device configured to generate image data associated with the surgical object and an environment of the surgical object within a field-of-view of the vision device, and one or more controllers for performing the following:

associating a first virtual object with the surgical object;

determining a pose of the first virtual object based on the localizer data;

merging the image data and the localizer data in a common coordinate system for identifying data points in the image data that have coordinates located outside of the first virtual object; and

associating a second virtual object with the identified data points.

14. The method of claim 13 , wherein the first virtual object is a 3D model of the surgical object and the image data includes a 3D surface map of the surgical object and the environment of the surgical object, the method comprising the one or more controllers:

merging the 3D model with the 3D surface map for identifying the data points in the image data that have coordinates located outside of the first virtual object.

15. The method of claim 13 , comprising the one or more controllers:

configuring the second virtual object for constraining movement of a robotically-controlled surgical instrument relative to the identified data points.

16. The method of claim 13 , wherein the surgical object is a target site of a patient, and comprising the one or more controllers:

associating the first virtual object with a volume of material planned to be removed from the target site.

17. The method of claim 13 , wherein the surgical object is an instrument configured to manipulate tissue, and comprising the one or more controllers:

associating the first virtual object with the instrument or a portion of the instrument.

18. The method of claim 13 , wherein the surgical object is a robotic manipulator, and comprising the one or more controllers:

associating the first virtual object with the robotic manipulator or a portion of the robotic manipulator.

19. The method of claim 13 , wherein the identified data points includes a sensitive anatomical structure to be avoided, and comprising the one or more controllers:

associating the second virtual object with the sensitive anatomical structure to be avoided.

20. The method of claim 13 , wherein the identified data points includes a surgical tool to be avoided, and comprising the one or more controllers:

associating the second virtual object with the surgical tool to be avoided.

21. A non-transitory computer readable medium, comprising instructions, which when executed by one or more processors, are configured to:

obtain, from a localizer, localizer data associated with a surgical object;

obtain, from a vision device, image data associated with the surgical object and an environment of the surgical object within a field-of-view of the vision device;

associate a first virtual object with the surgical object;

determine a pose of the first virtual object based on the localizer data;

merge the image data and the localizer data in a common coordinate system to identify data points in the image data that have coordinates located outside of the first virtual object; and

associate a second virtual object with the identified data points.

22. A computer-implemented method comprising:

obtaining, from a localizer, localizer data associated with a surgical object;

obtaining, from a vision device, image data associated with the surgical object and an environment of the surgical object within a field-of-view of the vision device;

associating a first virtual object with the surgical object;

determining a pose of the first virtual object based on the localizer data;

merging the image data and the localizer data in a common coordinate system for identifying data points in the image data that have coordinates located outside of the first virtual object; and

associating a second virtual object with the identified data points.