IP Library Granted Patent US 11,636,621
Granted Patent B2
US 11,636,621 · App. 17/120,031 · Granted Apr 25, 2023

Motion capture calibration using cameras and drones

Inventors: Dejan Momcilovic (Wellington, NZ); Jake Botting (Wellington, NZ)
Assignee: UNITY TECHNOLOGIES SF
G06T7/80G05D1/0094G05D1/0253G06T2207/10021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,636,621
App. No.
17/120,031
Granted
Apr 25, 2023
Kind
B2
Abstract

Embodiments facilitate the calibration of cameras in a live action scene using fixed cameras and drones. In some embodiments, a method configures a plurality of reference cameras to observe at least three known reference points located in the live action scene and to observe one or more reference points associated with one or more moving cameras having unconstrained motion. The method further configures the one or more moving cameras to observe one or more moving objects in the live action scene. The method further receives reference point data in association with one or more reference cameras of the plurality of reference cameras, where the reference point data is based on the at least three known reference points and the one or more reference points associated with the one or more moving cameras. The method further computes a location and an orientation of each moving camera of the one or more moving cameras based on one or more of the reference point data and one or more locations of one or more reference cameras of the plurality of reference cameras.

Claims (34)

1. A computer-implemented method for calibrating cameras in a live action scene, the method comprising:

configuring a plurality of reference cameras in the live action scene to observe at least three known reference points located in the live action scene and to observe one or more reference points associated with one or more moving cameras having unconstrained motion, wherein the at least three known reference points are attached to different respective reference cameras of the plurality of reference cameras, wherein the at least three known reference points are in a fixed locations in the live action scene, and wherein the one or more reference points associated with the one or more moving cameras are attached to different respective moving cameras of the one or more moving cameras;

configuring the one or more moving cameras to observe one or more moving objects in the live action scene;

receiving reference point data in association with one or more reference cameras of the plurality of reference cameras, wherein the reference point data is based on the at least three known reference points in their respective fixed locations and the one or more reference points associated with the one or more moving cameras; and

computing a location and an orientation of the one or more moving cameras based on the reference point data and the fixed locations of the known reference points attached to their respective reference cameras.

2. The method of claim 1 , wherein the plurality of reference cameras are at a known location relative to at least a second reference camera of the plurality of reference cameras.

3. The method of claim 1 , wherein a plurality of known cameras is mounted on one or more rigid structures.

4. The method of claim 1 , wherein at least one moving camera is mounted on a mobile apparatus.

5. The method of claim 1 , further comprising computing the location of the one or more moving cameras based at least in part on one or more of global positioning system data, position sensor data, and inertial guide sensor data.

6. The method of claim 1 , wherein at least one moving object of the one or more moving objects is a person.

7. The method of claim 1 , wherein at least a portion of the one or more known reference points are coupled to one or more reference cameras of the plurality of reference cameras.

8. A system for calibrating cameras in a live action scene, the system comprising:

one or more processors; and

logic encoded in one or more non-transitory computer-readable storage media for execution by the one or more processors and when executed operable to cause the one or more processors to perform operations comprising:

configuring a plurality of reference cameras in the live action scene to observe at least three known reference points located in the live action scene and to observe one or more reference points associated with one or more moving cameras having unconstrained motion, wherein the at least three known reference points are attached to different respective reference cameras of the plurality of reference cameras, wherein the at least three known reference points are in a fixed locations in the live action scene, and wherein the one or more reference points associated with the one or more moving cameras are attached to different respective moving cameras of the one or more moving cameras;

configuring the one or more moving cameras to observe one or more moving objects in the live action scene;

receiving reference point data in association with one or more reference cameras of the plurality of reference cameras, wherein the reference point data is based on the at least three known reference points in their respective fixed locations and the one or more reference points associated with the one or more moving cameras; and

computing a location and an orientation of the one or more moving cameras based on the reference point data and the fixed locations of the known reference points attached to their respective reference cameras.

9. The system of claim 8 , wherein the plurality of reference cameras are at a known location relative to at least a second reference camera of the plurality of reference cameras.

10. The system of claim 8 , wherein a plurality of known cameras is mounted on one or more rigid structures.

11. The system of claim 8 , wherein at least one moving camera is mounted on a mobile apparatus.

12. The system of claim 8 , wherein the logic when executed is further operable to cause the one or more processors to perform operations comprising computing the location of the one or more moving cameras based at least in part on one or more of global positioning system data, position sensor data, and inertial guide sensor data.

13. The system of claim 8 , wherein at least one moving object of the one or more moving objects is a person.

14. The system of claim 8 , wherein at least a portion of the one or more known reference points are coupled to one or more reference cameras of the plurality of reference cameras.

15. A non-transitory computer-readable storage medium with program instructions stored thereon, the program instructions when executed by one or more processors are operable to cause the one or more processors to perform operations comprising:

configuring a plurality of reference cameras in a live action scene to observe at least three known reference points located in the live action scene and to observe one or more reference points associated with one or more moving cameras having unconstrained motion, wherein the at least three known reference points are attached to different respective reference cameras of the plurality of reference cameras, wherein the at least three known reference points are in a fixed locations in the live action scene, and wherein the one or more reference points associated with the one or more moving cameras are attached to different respective moving cameras of the one or more moving cameras;

configuring the one or more moving cameras to observe one or more moving objects in the live action scene;

receiving reference point data in association with one or more reference cameras of the plurality of reference cameras, wherein the reference point data is based on the at least three known reference points in their respective fixed locations and the one or more reference points associated with the one or more moving cameras; and

computing a location and an orientation of the one or more moving cameras based on the reference point data and the fixed locations of the known reference points attached to their respective reference cameras.

16. The computer-readable storage medium of claim 15 , wherein at least one reference camera of the plurality of reference cameras is at a known location relative to at least a second reference camera of the plurality of reference cameras.

17. The computer-readable storage medium of claim 15 , wherein a plurality of known cameras is mounted on one or more rigid structures.

18. The computer-readable storage medium of claim 15 , wherein at least one moving camera is mounted on a mobile apparatus.

19. The computer-readable storage medium of claim 15 , wherein the instructions when executed are further operable to cause the one or more processors to perform operations comprising computing the location of the one or more moving cameras based at least in part on one or more of global positioning system data, position sensor data, and inertial guide sensor data.

20. The computer-readable storage medium of claim 15 , wherein at least one moving object of the one or more moving objects is a person.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 9, 2022
From: UNITY SOFTWARE INC.
To: UNITY TECHNOLOGIES SF
Reel/Frame 058980/0369 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 8, 2022
From: WETA DIGITAL LIMITED
To: UNITY SOFTWARE INC.
Reel/Frame 058978/0905 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 17, 2021
From: MOMCILOVIC, DEJAN; BOTTING, JAKE
To: WETA DIGITAL LIMITED
Reel/Frame 057521/0856 →
Continuity (4)
Provisional Application 63072085 · Aug 28, 2020
Provisional Application 63072088 · Aug 28, 2020
Provisional Application 63072092 · Aug 28, 2020
Related Publication 20220067970A1 · Mar 3, 2022