Multi-arm robotic systems and methods for identifying a target
Systems and methods for tracking a pose of a target and/or tracking the target is provided. A first robotic arm may be configured to orient a first imaging device and a second robotic arm may be configured to orient a second imaging device. The first robotic arm and the second robotic arm may operate in a shared or common coordinate space. A first image may be received from the first imaging device and a second image may be received from the second imaging device. The first image and the second image may depict at least one target. A pose of the at least one target may be determined in the coordinate space based on the first image, a corresponding first pose, the second image, and a corresponding second pose.
1 . A system for tracking a pose of at least one target comprising:
a first robotic arm including a first sensor configured to orient a first imaging device;
a second robotic arm including a second sensor configured to orient a second imaging device, wherein the first robotic arm and the second robotic arm operate in a shared coordinate space, wherein a first pose of the first robotic arm and a second pose of the second robotic arm in the shared coordinate space is determined based on the first sensor and the second sensor, and wherein the first sensor and the second sensor are position sensors;
at least one processor; and
at least one memory storing instructions for execution by the at least one processor that, when executed, cause the at least one processor to:
receive a first image from the first imaging device, the first image depicting the at least one target;
receive a second image from the second imaging device, the second image depicting the at least one target, wherein at least one of the first robotic arm and the second robotic arm is depicted in at least one of the first image and the second image;
determine the pose of the at least one target in the shared coordinate space based on the first image, the first pose, the second image, and the second pose; and
confirm or update the first pose of the first robotic arm or the second pose of the second robotic arm based on at least one of the first image and the second image.
2 . The system of claim 1 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
calculate a three-dimensional volume of the at least one target based on the first image, the second image, the first pose, and the second pose to identify each surface that defines the at least one target and a pose of each surface.
3 . The system of claim 1 , wherein the first robotic arm is configured to orient a tool in addition to the first imaging device.
4 . The system of claim 1 , wherein the at least one target is an incision, and wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
cause the second imaging device to track the incision formed on a patient.
5 . The system of claim 1 , wherein the first robotic arm is configured to orient a tool, and wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
update a path of the first robotic arm when the second imaging device detects movement of the at least one target.
6 . The system of claim 1 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
cause the second imaging device to obtain a three-dimensional scan of the at least one target.
7 . The system of claim 6 , wherein causing the second imaging device to obtain the three-dimensional scan includes calculating a plurality of poses to obtain the three-dimensional scan and causing the second robotic arm to orient the second imaging device at each pose of the plurality of poses.
8 . The system of claim 1 , wherein the second imaging device comprises a pair of stereoscopic cameras configured to generate three-dimensional images.
9 . The system of claim 1 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
calculate a first set of poses for the first imaging device and a second set of poses for the second imaging device;
cause the first robotic arm to orient the first imaging device at each of the first set of poses to yield a first set of images and the second robotic arm to orient the second imaging device at each of the second set of poses to yield a second set of images, each of the first set of images and the second set of images depicting the at least one target; and
update the pose determination of the at least one target based on the first set of images, the first set of poses, the second set of images, and the second set of poses.
10 . The system of claim 1 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
control the first robotic arm and the second robotic arm in the shared coordinate space to avoid a collision between the first robotic arm and the second robotic arm.
11 . A system for determining a pose of a robotic arm comprising:
a first robotic arm including a first sensor;
a first imaging device and a second imaging device secured to the first robotic arm;
a reference marker disposed on the first robotic arm,
wherein a first pose of the first robotic arm is determined based on the first sensor, and wherein the first sensor is a position sensor;
at least one processor; and
at least one memory storing instructions for execution by the at least one processor that, when executed, cause the at least one processor to:
receive a first image from the first imaging device, the first image depicting the reference marker; and
receive a second image from the second imaging device, wherein the first imaging device captures the first image from a first angle and the second imaging device captures the second image at a same time from a second angle different from the first angle.
12 . The system of claim 11 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
generate a three-dimensional (3D) image from the first image and the second image.
13 . The system of claim 11 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
cause the second imaging device to obtain a three-dimensional scan of at least one target.
14 . The system of claim 13 , wherein the first robotic arm is configured to orient a tool.
15 . The system of claim 11 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
generate a three-dimensional image of at least one target using the first imaging device and the second imaging device.
16 . A system for tracking a target comprising:
a first robotic arm including a first sensor configured to support and operate a tool;
a second robotic arm including a second sensor, wherein the first robotic arm and the second robotic arm are controllable in a common coordinate space, wherein a first pose of the first robotic arm and a second pose of the second robotic arm in the common coordinate space are determined based on the first sensor and the second sensor, and wherein the first sensor and the second sensor are position sensors;
an imaging device secured to the second robotic arm;
at least one processor; and
at least one memory storing instructions for execution by the at least one processor that, when executed, cause the at least one processor to:
receive a surgical plan having information about an initial pose of the target;
receive at least one image from the imaging device, the at least one image depicting the target and at least one of the first robotic arm and the second robotic arm;
update the initial pose of the target based on the at least one image to yield an updated pose of the target; and
confirm or update at least one of the first pose of the first robotic arm and
the second pose of the second robotic arm based on the at least one image.
17 . The system of claim 16 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
determine a boundary of the target based on the at least one image.
18 . The system of claim 16 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
cause the imaging device to identify movement of the target;
cause the first robotic arm to move the tool along a path; and
update the path when the imaging device identifies movement of the target.
19 . The system of claim 16 , wherein the at least one memory stores additional instructions for execution by the at least one processor that, when executed, further cause the at least one processor to:
control the first robotic arm to avoid the target.
20 . The system of claim 16 , wherein the imaging device comprises a pair of stereoscopic cameras configured to generate three-dimensional images.