Granted Patent B2

US 12,299,907 · App. 18/747,172 · Granted May 13, 2025

Inventors: David Julio Colmenares (Seattle, WA); James Andrew Youngquist (Seattle, WA); Adam Gabriel Jones (Seattle, WA); Thomas Ivan Nonn (Kenmore, WA); Jay Peterson (Seattle, WA)

Assignee: PROPRIO, INC.

G06T7/292A61B34/20A61B90/361A61B90/50G06T7/70G06T7/80G06T11/00H04N5/2628H04N5/265H04N23/54H04N23/698H04N23/90A61B2034/2057A61B2090/364A61B2090/371G06T2207/10048G06T2207/30012

Abstract

Claims (38)

1. An imaging system for imaging a scene, comprising:

a sensor array comprising:

a support structure;

a depth sensor fixedly coupled to the support structure and configured to capture depth data of the scene;

a plurality of cameras fixedly coupled to the support structure and configured to capture image data of the scene; and

a plurality of trackers fixedly coupled to the support structure and configured to capture positional data of an instrument within the scene, wherein each of the depth sensor, the cameras, and the trackers has a different field of view that is fixed relative to the support structure; and

an arm coupled to the sensor array, wherein the arm is movable in at least three degrees of freedom to move the sensor array about the scene to position the fields of view of the depth sensor, the cameras, and the trackers at a desired position and orientation relative to the scene while the fields of view of the depth sensor, the cameras, and the trackers remain fixed relative to the support structure.

2. The imaging system of claim 1 wherein the support structure includes a central region, and wherein the depth sensor is fixedly coupled to the support structure at the central region.

3. The imaging system of claim 2 wherein the cameras are fixedly coupled to the support structure outside the central region.

4. The imaging system of claim 3 wherein the trackers are fixedly coupled to the support structure farther outward from the central region than the cameras.

5. The imaging system of claim 4 wherein the fields of view of each of the cameras and the trackers are angled radially inward toward the central region of the support structure.

6. The imaging system of claim 1 , further comprising a processing device communicatively coupled to the depth sensor, the cameras, and the trackers, wherein the processing device is configured to, based on the depth data, the image data, and/or the positional data:

determine a depth of at least a portion of the scene;

determine a position of the instrument within the scene;

synthesize an image corresponding to a selected perspective of the scene.

7. The imaging system of claim 1 wherein the cameras are first cameras, wherein the trackers comprise second cameras, and wherein the first cameras have at least one intrinsic parameter different than the second cameras.

8. The imaging system of claim 1 wherein the fields of view of the cameras at least partially overlap to define a first volume, and wherein the fields of view of the trackers at least partially overlap to define a second volume that is larger than the first volume.

9. The imaging system of claim 1 wherein the trackers comprise infrared cameras, and wherein the cameras comprise RGB cameras.

10. The imaging system of claim 1 wherein the depth sensor has a focal plane, wherein the cameras each have a focal axis, and wherein the focal axes of the cameras converge at a point below the focal plane of the depth sensor.

11. The imaging system of claim 1 wherein the support structure includes a central region, and wherein the first camera is fixedly coupled to the support structure at the central region.

12. The imaging system of claim 11 wherein the second cameras are fixedly coupled to the support structure outside the central region.

13. The imaging system of claim 12 wherein the third cameras are fixedly coupled to the support structure farther outward from the central region than the second cameras.

14. The imaging system of claim 13 wherein the fields of view of each of the second cameras and the third cameras are angled radially inward toward the central region of the support structure.

15. An imaging system for imaging a scene, comprising:

a camera array comprising:

a support structure;

a first camera of a first type fixedly coupled to the support structure and configured to capture first image data of the scene;

a plurality of second cameras of a second type fixedly coupled to the support structure and configured to capture second image data of the scene; and

a plurality of third cameras of a third type fixedly coupled to the support structure and configured to capture third image data the scene, wherein each of the first camera, the second cameras, and the third cameras has a different field of view that is fixed relative to the support structure, wherein the first type is different than the second type and the third type, and wherein the second type is different than the third type; and

an arm coupled to the camera array, wherein the arm is movable in at least three degrees of freedom to move the camera array about the scene to position the fields of view of the first camera, the second cameras, and the third cameras at a desired position and orientation relative to the scene while the fields of view of the first camera, the second cameras, and the third cameras remain fixed relative to the support structure.

16. The imaging system of claim 15 , further comprising a processing device communicatively coupled to the first camera, the second cameras, and the third cameras, wherein the processing device is configured to, based on the first image data, the second image data, and/or the third image data:

determine a depth of at least a portion of the scene;

determine a position of an instrument within the scene;

synthesize an image corresponding to a selected perspective of the scene.

17. The imaging system of claim 15 wherein the second cameras comprise RGB cameras, and wherein the third cameras comprise infrared cameras.

18. The imaging system of claim 15 wherein the fields of view of the second cameras at least partially overlap to define a first volume, and wherein the fields of view of the third cameras at least partially overlap to define a second volume that is larger than the first volume.

19. The imaging system of claim 15 wherein the first camera has a focal plane, wherein the second cameras each have a focal axis, and wherein the focal axes of the second cameras converge at a point below the focal plane of the first camera.

20. The imaging system of claim 15 wherein the second cameras are substantially identical to one another, and wherein the third cameras are substantially identical to one another.

Assignments (2)

Continuity (4)

References Cited (312)