Coordinated instrument control systems
Surgical systems for operating endoscopically and laparoscopically are provided. The surgical system includes a first scope device configured to transmit image data of a first scene. A second scope device is configured to transmit image data of a second scene. A tracking device associated with the first or second scope device and configured to transmit a signal indicative of a location of the first or second scope device. A first surgical instrument is configured to interact with an internal side of a target tissue structure. A second surgical instrument is configured to interact an external side of the target tissue structure. A controller is configured to receive (i) the transmitted image data and transmitted signal (ii) to determine a first relative distance, a second relative distance, and a third relative distance. Relative movements of the instruments are coordinated based on the determined relative distances.
1 . A surgical controller, comprising:
processor configured to:
receive image data of a first scene within a field of view of a first scope device at least partially disposed within a natural body lumen or organ;
receive image data of a second scene within a field of view of a second scope device disposed within a body cavity and at least partially disposed outside of the natural body lumen or organ simultaneously with the first scope device being at least partially disposed within the natural body lumen or organ, the second scene being different than the first scene;
receive a signal indicative of a location of one of the first scope device or the second scope device relative to the other one of the first scope device or the second scope device;
determine, based on the signal, a first relative distance from the first scope device to the second scope device,
determine, based on the image data of the first scene, a second relative distance from the first scope device to a first surgical instrument at least partially disposed within the natural body lumen or organ, and configured to interact with an internal side of a target tissue structure at a surgical site;
determine, based on the image data of the second scene, a third relative distance from the second scope device to a second surgical instrument at least partially disposed outside of the natural body lumen or organ, and configured to interact with an external side of the target tissue structure, wherein the first surgical instrument and the second surgical instrument are used in a surgical procedure comprising a plurality of surgical steps, and wherein a first surgical step in the plurality of surgical steps comprises the first surgical instrument making an endoscopic incision in the target tissue structure, and a second surgical step in the plurality of surgical steps comprises the second surgical instrument making a laparoscopic incision in the target tissue structure; and
generate a control signal configured to coordinate relative movements of the first surgical instrument and the second surgical instrument at the surgical site based on the determined relative distances, wherein coordinating the relative movements of the first and second surgical instruments at the surgical site based on the determined relative distances comprises coordinating the relative movements of the first and second surgical instruments so that the endoscopic incision and the laparoscopic incision are within a threshold distance from each other.
2 . The surgical controller of claim 1 , wherein the first scope device is an endoscope and the second scope device is a laparoscope.
3 . The surgical controller of claim 1 , wherein the first scene does not include the second scope device, and wherein the second scene does not include the first scope device.
4 . The surgical controller of claim 1 , wherein the first scene does not include the second surgical instrument, and wherein the second scene does not include the first surgical instrument.
5 . The surgical controller of claim 1 , wherein the processor is further configured to receive a signal indicative of an orientation of the first scope device within the natural body lumen or organ.
6 . The surgical controller of claim 1 , wherein the processor is further configured to receive a signal indicative of an orientation of the second scope device positioned outside of the natural body lumen or organ.
7 . The surgical controller of claim 1 , wherein the control signal is further configured to simultaneously move the first surgical instrument and the second surgical instrument relative to each other based on the determined relative distances.
8 . The surgical controller of claim 1 , wherein the control signal is further configured to restrict movement of the first surgical instrument and the second surgical instrument relative to each other at the target tissue structure based on the image data of the first scene, the image data of the second scene, and the signal.
9 . The surgical controller of claim 1 , wherein the processor is further configured to determine an amount of strain that is applied to the target tissue structure by at least one of the first and second surgical instruments based on visual markers associated with the target tissue structure.
10 . The surgical controller of claim 9 , wherein the visual markers comprise at least one of one or more local tissue markings on the target tissue structure, one or more projected light markings on the target tissue structure, and one or more anatomical aspects of the natural body lumen or organ.
11 . The surgical controller of claim 1 , wherein the processor is configured to:
receive, from a first force sensor associated with the first surgical instrument, an indication of a first applied force to the target tissue structure by the first surgical instrument; and
receive, from a second force sensor associated with the second surgical instrument, an indication of a second applied force to the target tissue structure by the second surgical instrument, wherein the control signal configured to coordinate relative movements of the first surgical instrument and the second surgical instrument at the surgical site is further based on the first applied force and the second applied force.
12 . The surgical controller of claim 1 , wherein the controller is further configured to:
merge the image data of the first scene and the image data of the second scene to create a merged image comprising a representative depiction of the first scope device, the second scope device, and one or more structures within a field of view of the first scope device that are not visible via the second scope device due to the target tissue structure; and
generate a control signal configured to display the merged image to a health care provider.
13 . The surgical controller of claim 1 , wherein the controller is further configured to:
generate a control signal configured to display a live image stream of the first scene; and
overlay, on the live image stream of the first scene, a visual representation of locations and orientations of the second surgical instrument and the second scope device.
14 . The surgical controller of claim 1 , wherein a third surgical step in the plurality of surgical steps comprises the first surgical instrument pushing a tumor through the endoscopic incision, and a fourth surgical step in the plurality of surgical steps comprises the second surgical instrument receiving the tumor through the endoscopic incision, wherein coordinating the relative movements of the first and second surgical instruments at the surgical site based on the determined relative distances comprises coordinating the relative movements of the first and second surgical instruments so that the second surgical instrument is aligned with the endoscopic incision such that the second surgical instrument can receive the tumor.
15 . The surgical controller of claim 1 , wherein the relative movements of the first and second surgical instruments at the surgical site are coordinated further based on an anatomical landmark visible in the first scene and the second scene.
16 . The surgical controller of claim 1 , wherein the first surgical instrument comprises an energy device, the energy device is configured to perform an ablation endoscopically, the second scope device is further configured to:
monitor a temperature of the target tissue structure; and
transmit an indication of the temperature to the controller, wherein the controller is further configured to, on a condition that the temperature of the target tissue structure exceeds a threshold, reduce an amount of energy applied by the energy device.
17 . The surgical controller of claim 1 , wherein the first surgical instrument comprises an energy device, the energy device is configured to perform an ablation endoscopically, the second scope device is further configured to:
monitor a temperature of the target tissue structure; and
transmit an indication of the temperature to the controller, wherein the controller is further configured to calculate an insertion depth of the energy device into the target tissue structure based on the temperature.
18 . The surgical controller of claim 1 , wherein the first surgical instrument is inserted into the natural body lumen or organ via a natural orifice of a patient, and the second surgical instrument is inserted into the body cavity via a laparoscopic access port.