Apparatus and method for performing motion capture using a random pattern on capture surfaces
View Patent ↗A method is described comprising: applying a random pattern to specified regions of an object; tracking the movement of the random pattern during a motion capture session; and generating motion data representing the movement of the object using the tracked movement of the random pattern.
1. A method comprising:
applying a random pattern of material to specified regions of a performer's face, body and/or clothing;
capturing sequences of images of the random pattern with a first plurality of cameras as the performer moves and/or changes facial expressions during a motion capture session;
correlating the random pattern across two or more images captured from two or more different cameras to create a 3-dimensional surface of the specified regions of the performer's face, body, and/or clothing;
generating motion data representing the movement of the 3-dimensional surface across the sequence of images;
strobing a light source on and off, the light source charging the random pattern when on; and
strobing the shutters of a first plurality of cameras synchronously with the strobing of the light source to capture the sequences of images of the random pattern (“glow frames”) as the performer moves or changes facial expressions during a performance, wherein the shutters of the first plurality of cameras are open when the light source is off and the shutters are closed when the light source is on.
2. The method as in claim 1 wherein the material is phosphorescent paint.
3. The method as in claim 1 further comprising:
strobing the shutters of a second plurality of cameras synchronously with the strobing of the light source to capture images of the performer (“lit frames”), wherein the shutters of the second plurality of cameras are open when the light source is on and the shutters of the second plurality of cameras are closed when the light source is off.
4. The method as in claim 3 wherein the first plurality of cameras are grayscale cameras and the second plurality of cameras are color cameras.
5. The method as in claim 3 further comprising:
separating the lit frames from the glow frames to generate two separate sets of image data.
6. The method as in claim 3 wherein cameras capturing the lit frames have a sensitivity which is different from cameras capturing the glow frames.
7. The method as in claim 3 wherein color cameras are used to capture the lit frames and grayscale cameras are used to capture the glow frames.
8. The method as in claim 7 wherein the grayscale cameras have a relatively higher sensitivity than the color cameras.
9. The method as in claim 7 wherein two different synchronization signals are used to control the shutters of the color and grayscale cameras.
10. The method as in claim 9 wherein the different synchronization signals are 180 degrees out of phase.
11. The method as in claim 1 wherein the light source comprises a light emitting diode (LED) array.
12. The method as in claim 1 wherein strobing the shutters comprises opening the shutters for a first period of time and closing the shutters for a second period of time, the second period of time being of a different duration than the first period of time.
13. The method as in claim 12 wherein the first period of time is longer than the second period of time.
14. The method as in claim 1 wherein the camera shutters are controlled by synchronization signals from a computer system.
15. The method as in claim 1 further comprising:
opening the shutters for a first period of time when the light source is on; and
opening the shutters for a second period of time when the light source is off;
wherein the first and second periods of time are unequal.
16. The method as in claim 1 wherein strobing the shutters further comprises:
opening the shutters for a period of time when the light source is on to capture images of the performers face, body, and/or clothing (“lit frame”).
17. The method as in claim 16 wherein after being opened to capture a lit frame, the shutters are closed and then opened again when the light source is off to capture a glow frame, and then closed and then opened again when the light source is on to capture the next lit frame.
18. The method as in claim 16 wherein strobing the shutters comprises opening the shutters for a first period of time and closing the shutters for a second period of time wherein the first period of time is not equal to the second period of time.
19. The method as in claim 18 further comprising:
opening the shutters for a relatively shorter period of time when the light source is on; and
opening the shutters for a relatively longer period of time when the light source is off.
20. The method as in claim 16 further comprising:
separating the lit frames from the glow frames to generate two separate sets of image data.
21. The method as in claim 16 further comprising:
alternating sensitivity, of the cameras between capturing the lit frames and the glow frames.
22. The method as in claim 1 wherein correlating the random pattern further comprises:
performing a first correlation at a first resolution for each of the two or more images using overlapping fields of view of the two or more cameras.
23. The method as in claim 22 wherein correlating the random pattern further comprises:
performing a second correlation at a second resolution for each of the two or more images to render the 3-dimensional surface, wherein the first resolution is lower than the second resolution.
24. The method as in claim 1 wherein the two or more images are captured at substantially the same point in time.
25. The method as in claim 1 wherein the two or more images are captured in sequence at different points in time.
26. The method as in claim 1 further comprising:
capturing a series of images of the performer's face, body, and/or clothing; and
using the images as a texture map corresponding to regions of the random pattern.
27. The method as in claim 1 wherein applying the random pattern comprises:
applying phosphorescent material to a sponge; and
applying the sponge upon the performer's face, body, and/or clothing.
28. The method as in claim 1 wherein applying the random pattern comprises:
spraying the random pattern of material on the performer's face, body, and/or clothing with an airbrush.
29. The method as in claim 1 wherein applying the random pattern comprises:
applying paint to the performer's face, body, and/or clothing through a stencil.
30. The method as in claim 1 wherein applying the random pattern comprises:
flicking a wire brush containing paint such that droplets of paint are splattered onto the surface to be captured.
31. The method as in claim 1 wherein the random pattern is applied with paint viewable in visible light.
32. The method as in claim 31 wherein visible light is used in capturing the images.
33. A method comprising:
applying a random pattern of phosphorescent material to specified regions of a performer's face, body and/or clothing;
strobing a light source on and off, the light source charging the random pattern when on; and
strobing the shutters of the first plurality of cameras synchronously with the strobing of the light source to capture sequences of images of the random pattern (“glow frames”) as the performer moves or changes facial expressions during a performance, wherein the shutters of the first plurality of cameras are open when the light source is off and the shutters are closed when the light source is on.
34. The method as in claim 33 wherein the phosphorescent material is phosphorescent paint.
35. The method as in claim 33 further comprising:
tracking the motion of the phosphorescent paint over time; and
generating motion data representing the movement of the performer's face and/or body using the tracked movement of the phosphorescent paint.
36. The method as in claim 33 wherein the phosphorescent paint is applied as a series of curves on the performer's face.
37. The method as in claim 33 wherein the phosphorescent paint is applied as a series of markers at specified areas of the performer's body.
38. The method as in claim 33 further comprising:
strobing the shutters of a second plurality of cameras synchronously with the strobing of the light source to capture images of the performer (“lit frames”), wherein the shutters of the second plurality of cameras are open when the light source is on and the shutters of the second plurality of cameras are closed when the light source is off.
39. The method as in claim 38 wherein the first plurality of cameras are grayscale cameras and the second plurality of cameras are color cameras.
40. The method as in claim 38 further comprising:
separating the lit frames from the glow frames to generate two separate sets of image data.
41. The method as in claim 38 wherein cameras capturing the lit frames have a sensitivity which is different from cameras capturing the glow frames.
42. The method as in claim 38 further comprising:
opening the shutters for a first period of time when the light source is on; and
opening the shutters for a second period of time when the light source is off;
wherein the first and second periods of time are unequal.
43. The method as in claim 38 wherein color cameras are used to capture the lit frames and grayscale cameras are used to capture the glow frames.
44. The method as in claim 43 wherein the grayscale cameras have a relatively higher sensitivity than the color cameras.
45. The method as in claim 43 wherein two different synchronization signals are used to control the shutters of the color and grayscale cameras.
46. The method as in claim 45 wherein the different synchronization signals are 180 degrees out of phase.
47. The method as in claim 33 wherein the light source comprises a light emitting diode (LED) array.
48. The method as in claim 33 wherein strobing the shutters comprises opening the shutters for a first period of time and closing the shutters for a second period of time, the second period of time being of a different duration than the first period of time.
49. The method as in claim 48 wherein the first period of time is longer than the second period of time.
50. The method as in claim 33 wherein the camera shutters are controlled by synchronization signals from a computer system.
51. The method as in claim 33 wherein strobing the shutters further comprises:
opening the shutters for a period of time when the light source is on to capture images of the performers face and/or body.
52. The method as in claim 51 wherein after being opened to capture a lit frame, the shutters are closed and then opened again when the light source is off to capture the next glow frame, and then closed and then opened again when the light source is on to capture the next lit frame.
53. The method as in claim 52 further comprising:
generating motion data representing the movement of the 3-dimensional surface across the sequence of images.
54. The method as in claim 51 wherein strobing the shutters comprises opening the shutters for a first period of time and closing the shutters for a second period of time wherein the first period of time is not equal to the second period of time.
55. The method as in claim 54 further comprising:
opening the shutters for a relatively shorter period of time when the light source is on; and
opening the shutters for a relatively longer period of time when the light source is off.
56. The method as in claim 51 further comprising:
separating the lit frames from the glow frames to generate two separate sets of image data.
57. The method as in claim 51 further comprising:
alternating sensitivity, of the cameras between capturing the lit frames and the glow frames.
58. The method as in claim 33 further comprising:
correlating the random pattern across two or more images captured from the first plurality of cameras to create a 3-dimensional surface of the specified regions of the performer's face, body, and/or clothing.