Positioning a camera for perspective sharing of a surgical site

1 . A mixed reality (MR) surgical system, comprising:

a first MR visualization device configured to be worn by a first user;

a second MR visualization device configured to be worn by a second user;

an intraoperative guidance system comprising one or more processing circuits configured to determine a location for the second user to stand within an operating room, wherein:

the first MR visualization device is further configured to present a first MR visualization that shows one or more images of a surgical site captured by a camera supported by the second user; and

the second MR visualization device is further configured to present a second MR visualization that indicates the location for the second user to stand within the operating room, wherein one or more of: the second MR visualization comprises a first virtual object fixed to a floor of the operating room at the location, the second MR visualization comprises a map of the operating room indicating the location, the second MR visualization comprises a second virtual object that appears to the second user to float in a space above the location, or the second MR visualization includes text instructions that direct the second user to the location.

2 . The MR surgical system of claim 1 , wherein the one or more processing circuits of the intraoperative guidance system are configured to:

obtain a 3-dimensional (3D) virtual model that includes a 3D virtual model of an anatomical component and a 3D virtual line along which a contact component is planned to be inserted into the anatomical component during a surgery involving the anatomical component;

generate registration data that describes a mapping between an actual anatomical component of a patient and the 3D virtual model; and

determine, based on the registration data, the location for the second user to stand within the operating room.

3 . The MR surgical system of claim 2 , wherein the one or more processing circuits of the intraoperative guidance system are configured such that, as part of determining the location for the second user to stand within the operating room, the one or more processing circuits:

determine a plane that is orthogonal to the 3D virtual line and that intersects a first point, the first point being where the 3D virtual line intersects the 3 D virtual model of the anatomical component; and

determine the location for the second user to stand within the operating room such that the camera is within the plane.

4 . The MR surgical system of claim 3 , wherein the one or more processing circuits of the intraoperative guidance system are configured such that, as part of determining the location for the second user to stand within the operating room, the one or more processing circuits:

determine a second point based on the registration data, the second point being within the plane and being at a distance of the camera above the floor of the operating room; and

determine a third point, the third point being on the floor of the operating room directly beneath the second point, wherein the third point is the location for the second user to stand within the operating room.

5 . The MR surgical system of claim 2 , wherein the 3D virtual line corresponds to a reaming axis for a glenoid of the patient.

6 . The MR surgical system of claim 2 , wherein the 3D virtual line corresponds to an axis for pin placement for a humeral cutting frame.

7 . The MR surgical system of claim 2 , wherein the one or more processing circuits of the intraoperative guidance system are configured to:

determine whether a surgical plan specifies that a next step of the surgery is to insert the contact component into the anatomical component; and

in response to determining that the surgical plan specifies that the next step of the surgery is to insert the contact component into the anatomical component, automatically cause the second MR visualization device to indicate within the second MR visualization the location for the second user to stand within the operating room.

8 . The MR surgical system of claim 1 ,

wherein the one or more processing circuits of the intraoperative guidance system are configured to determine a 3D virtual line along which a contact component is planned to be inserted into an anatomical component of a patient during a surgery involving the anatomical component, and

wherein the second MR visualization includes the 3 D virtual line.

9 . The MR surgical system of claim 1 ,

wherein the one or more processing circuits of the intraoperative guidance system are configured to determine a 3D virtual line along which a contact component is planned to be inserted into an anatomical component of a patient during a surgery involving the anatomical component, and

wherein the first MR visualization includes the 3D virtual line.

10 . The MR surgical system of claim 1 ,

wherein the one or more processing circuits of the intraoperative guidance system are configured to determine a 3D virtual line along which a contact component is planned to be inserted into an anatomical component of a patient during a surgery involving the anatomical component, and

wherein the one or more images in the first MR visualization show the 3D virtual line from a perspective of the camera.

11 . The MR surgical system of claim 1 , wherein a perspective of the camera is different from a perspective of the first user.

12 . The MR surgical system of claim 1 , wherein the one or more processing circuits of the intraoperative guidance system are configured to:

generate registration data that describes a mapping between a tip of a contact component and the operating room; and

determine, based on the registration data, the location for the second user to stand within the operating room.

13 . The MR surgical system of claim 12 , wherein the one or more processing circuits of the intraoperative guidance system are configured such that, as part of determining the location for the second user to stand within the operating room, the one or more processing circuits:

determine a plane that is orthogonal to an axis of the contact component and that intersects a first point, the first point being at the tip of the contact component; and

determine the location for the second user to stand within the operating room such that the camera is within the plane.

14 . The MR surgical system of claim 1 , wherein the second MR visualization device is configured to provide feedback to the second user regarding a position of the second user with respect to the location for the second user to stand within the operating room.

15 . The MR surgical system of claim 1 , wherein the one or more processing circuits of the intraoperative guidance system are configured to:

determine a first plane that passes through a point on an anatomical feature of interest at the surgical site;

determine locations where the first plane intersects a second plane, the second plane being a plane parallel to the floor of the operating room at a height of the camera supported by the second user; and

generate the map such that the map indicates places for the second user to stand within the operating room that correspond to locations on the floor of the operating room below the locations where the first plane intersects the second plane.

16 . The MR surgical system of claim 1 , wherein:

the location is a first location;

the first MR visualization device is configured to generate a first point cloud based on first observation data generated by a set of sensors associated with the first user, the first point cloud comprising points having coordinates indicating locations of a first set of landmarks in the operating room;

the one or more processing circuits are configured to determine the first location based on the first point cloud;

the second MR visualization device is configured to generate a second point cloud based on second observation data generated by a set of sensors associated with the second user, the second point cloud comprising points having coordinates indicating locations of a second set of landmarks in the operating room;

the one or more processing circuits of the intraoperative guidance system are further configured to:

generate a merged point cloud based on the first point cloud and the second point cloud; and

register a 3D virtual model with the merged point cloud, and at least one of:

the first MR visualization includes the 3D virtual model registered to a second location in the operating room, or

the one or more images of the surgical site captured by the camera supported by the second user include the 3D virtual model.

17 . A method for augmenting a view of a first user of a first mixed reality (MR) visualization device, the method comprising:

presenting, by a second MR visualization device configured to be worn by a second user, a second MR visualization that indicates a location for the second user to stand within an operating room, wherein one or more of: the second MR visualization comprises a first virtual object fixed to a floor of the operating room at the location, the second MR visualization comprises a map of the operating room indicating the location, the second MR visualization comprises a second virtual object that appears to the second user to float in a space above the location, or the second MR visualization includes text instructions that direct the second user to the location; and

presenting, by the first MR visualization device, a first MR visualization that shows one or more images of a surgical site captured by a camera supported by the second user while the second user is at the location.

18 . The method of claim 17 , further comprising:

obtaining a 3-dimensional (3D) virtual model that includes a 3D virtual model of an anatomical component and a 3D virtual line along which a contact component is planned to be inserted into the anatomical component during a surgery involving the anatomical component;

generating registration data that describes a mapping between an actual anatomical component of a patient and the 3D virtual model; and

determining, based on the registration data, the location for the second user to stand within the operating room.

19 . The method of claim 18 , further comprising:

determining a plane that is orthogonal to the 3D virtual line and that intersects a first point, the first point being where the 3D virtual line intersects the 3D virtual model of the anatomical component; and

determining the location for the second user to stand within the operating room such that the camera is within the plane.

20 . The method of claim 19 , further comprising:

determining a second point based on the registration data, the second point being within the plane and being at a distance of the camera above the floor of the operating room; and

determining a third point, the third point being on the floor of the operating room directly beneath the second point, wherein the third point is the location for the second user to stand within the operating room.

21 . The method of claim 17 , further comprising determining a 3D virtual line along which a contact component is planned to be inserted into an anatomical component of a patient during a surgery involving the anatomical component,

wherein the first MR visualization includes the 3D virtual line.

22 . The method of claim 17 ,

wherein the method further comprises determining a 3D virtual line along which a contact component is planned to be inserted into an anatomical component of a patient during a surgery involving the anatomical component, and

wherein the second MR visualization includes the 3D virtual line.

23 . The method of claim 17 ,

wherein the method further comprises determining a 3D virtual line along which a contact component is planned to be inserted into an anatomical component of a patient during a surgery involving the anatomical component, and

wherein the one or more images in the first MR visualization show the 3D virtual line from a perspective of the camera.

24 . The method of claim 17 , wherein the location is a first location and the method further comprises:

generating, by the first MR visualization device, a first point cloud, the first point cloud comprising points having coordinates indicating locations of a first set of landmarks in the operating room;

generating, by the second MR visualization device, a second point cloud, the second point cloud comprising points having coordinates indicating locations of a second set of landmarks in the operating room;

generating a merged point cloud based on the first point cloud and the second point cloud; and

registering a 3D virtual model with the merged point cloud, and

wherein at least one of:

the first MR visualization includes the 3 D virtual model registered to a second location in the operating room, or

the one or more images of the surgical site captured by the camera supported by the second user include the 3D virtual model.

25 . One or more non-transitory computer-readable data storage media having instructions stored thereon that, when executed, cause:

an interoperative guidance system to determine a location for a second user to stand within an operating room;

a second MR visualization device configured to be worn by the second user to present a second MR visualization that indicates the location for the second user to stand within the operating room, wherein one or more of: the second MR visualization comprises a first virtual object fixed to a floor of the operating room at the location, the second MR visualization comprises a map of the operating room indicating the location, the second MR visualization comprises a second virtual object that appears to the second user to float in a space above the location, or the second MR visualization includes text instructions that direct the second user to the location; and

a first MR visualization device to present a first MR visualization that shows one or more images of a surgical site captured by a camera supported by the second user while the second user is at the location.