Surgical methods using multi-source imaging

1 . A surgical method, comprising:

visualizing, with a rigid scoping device located outside of an organ, a first side of a tissue wall during performance of a surgical procedure, the rigid scoping device being thoracoscopic or laparoscopic;

visualizing, with a flexible scoping device advanced into the organ, a second, opposite side of the tissue wall during the performance of the surgical procedure, the rigid scoping device and the flexible scoping device being unable to directly visualize each other, wherein at least one of the rigid scoping device or the flexible scoping device is configured to emit a structured light pattern via a structured light emitter, emit a spectral light via a spectral light emitter, and detect a reflected structured light pattern and reflected spectral light via an image sensor;

generating a 3D map based on a visualization provided by the rigid scoping device and a visualization provided by the flexible scoping device;

guiding a first surgical instrument based on the visualization provided by the rigid scoping device and the visualization provided by the flexible scoping device, and based on a location of the first surgical instrument on the 3D map; and

guiding a second surgical instrument based on the visualization provided by the flexible scoping device and based on the visualization provided by the rigid scoping device.

2 . The method of claim 1 , wherein at least one of the first surgical instrument or the second surgical instrument identifies a location of the at least one first surgical instrument or the second surgical instrument; and the method further comprises communicating the location to a first imaging system and an endoscopic imaging system so as to provide secondary location verification on the 3D map to the first imaging system and the endoscopic imaging system.

3 . The method of claim 1 , further comprising determining, with a first imaging system that includes the rigid scoping device, a location and orientation of the second surgical instrument relative to the first surgical instrument to facilitate the first surgical instrument being guided based on the visualization provided by the flexible scoping device; and

determining, with an endoscopic imaging system that includes the flexible scoping device, a location and orientation of the first surgical instrument relative to the second surgical instrument to facilitate the second surgical instrument being guided based on the visualization provided by the rigid scoping device.

4 . The method of claim 1 , further comprising visualizing, with each of the rigid scoping device and the flexible scoping device, a common anatomic landmark and thereby facilitate guiding of the first and second surgical instruments relative to one another.

5 . The method of claim 1 , further comprising visualizing, with a computerized tomography (CT) imaging system, the first and second instruments and thereby facilitate guiding of the first and second surgical instruments relative to one another.

6 . The method of claim 1 , further comprising gathering, using at least one of ultrasonic energy and radiofrequency (RF) energy, data indicative of a location of the first and second surgical instruments and thereby facilitate guiding of the first and second surgical instruments relative to one another.

7 . The method of claim 1 , wherein the first and second surgical instruments are each releasably coupled to a robotic surgical system that controls the guiding of each of the first and second surgical instruments.

8 . The method of claim 1 , wherein the rigid scoping device is a thoracoscopic device, the organ is a lung, and the flexible scoping device is advanced transorally into the lung.

9 . The method of claim 1 , wherein the rigid scoping device is a laparoscopic device, the organ is a stomach or an intestine, and the flexible scoping device is advanced transorally into the stomach or the intestine.

10 . The method of claim 1 , wherein the first and second surgical instruments are each releasably coupled to a robotic surgical system operatively coupled to a plurality of surgical hubs; and

one of the surgical hubs controls the guiding of each of the first and second surgical instruments.

11 . A surgical method, comprising:

visualizing, with a first imaging system, a first side of a tissue wall during performance of a surgical procedure, the first imaging system being a thoracoscopic imaging system or a laparoscopic imaging system;

visualizing, with an endoscopic imaging system, a second, opposite side of the tissue wall during the performance of the surgical procedure, the first imaging system and the endoscopic imaging system being unable to directly visualize each other, and the first imaging system and the endoscopic imaging system being operatively coupled together;

guiding a first surgical instrument based on the visualization provided by the first imaging system and the visualization provided by the endoscopic imaging system; and

guiding a second surgical instrument based on the visualization provided by the endoscopic imaging system and based on the visualization provided by the first imaging system;

wherein one of the first imaging system and the endoscopic imaging system includes

a structured light emitter configured to emit a structured light pattern on a surface of an organ,

a spectral light emitter configured to emit spectral light in a plurality of wavelengths capable of penetrating the organ and reaching an internal aspect located below the surface of the organ, and

an image sensor configured to detect reflected structured light pattern, reflected spectral light, and reflected visible light; and

the surgical method further comprises constructing, with a controller, a three-dimensional (3D) digital representation of the organ from the reflected structured light pattern detected by the image sensor,

detecting, with the controller, the internal aspect from the reflected spectral light detected by the image sensor,

integrating, with the controller, the internal aspect with the 3D digital representation of the organ, and

causing, with the controller, a display device to show the internal aspect and a 3D rendering of the organ based on the 3D digital representation of the organ.

12 . The method of claim 11 , wherein at least one of the first and second surgical instruments identifies a location of the at least one first surgical instrument and second surgical instrument; and

the method further comprises communicating the location to the first imaging system and the endoscopic imaging system so as to provide secondary location verification to the first imaging system and the endoscopic imaging system.

13 . The method of claim 11 , further comprising determining, with the first imaging system, a location and orientation of the second surgical instrument relative to the first surgical instrument to facilitate the first surgical instrument being guided based on the visualization provided by the endoscopic imaging system; and

determining, with the endoscopic imaging system, a location and orientation of the first surgical instrument relative to the second surgical instrument to facilitate the second surgical instrument being guided based on the visualization provided by the first imaging system.

14 . The method of claim 11 , further comprising visualizing, with each of the first imaging system and the endoscopic imaging system, a common anatomic landmark and thereby facilitate guiding of the first and second surgical instruments relative to one another.

15 . The method of claim 11 , further comprising visualizing, with a computerized tomography (CT) imaging system, the first and second instruments and thereby facilitate guiding of the first and second surgical instruments relative to one another.

16 . The method of claim 11 , further comprising gathering, using at least one of ultrasonic energy and radiofrequency (RF) energy, data indicative of a location of the first and second surgical instruments and thereby facilitate guiding of the first and second surgical instruments relative to one another.

17 . The method of claim 11 , wherein the first and second surgical instruments are each releasably coupled to a robotic surgical system that controls the guiding of each of the first and second surgical instruments.

18 . The method of claim 11 , wherein the first and second surgical instruments are each releasably coupled to a robotic surgical system operatively coupled to a plurality of surgical hubs; and

one of the surgical instruments controls the guiding of each of the first and second surgical instruments.

19 . The method of claim 11 , wherein the first imaging system includes a rigid scoping device; and

the endoscopic imaging system includes a flexible scoping device.