Deriving insights into motion of an object through computer vision
Introduced here are computer programs that are able to generate computer vision data through local analysis of image data (also referred to as “raw data” or “input data”). The image data may be representative of one or more digital images that are generated by an image sensor. Also introduced here are apparatuses for generating and handling the image data and computer vision data.
1. An apparatus for generating computer vision data, the apparatus comprising:
a camera configured to generate digital images of an environment in which an individual is situated over an interval of time;
a processor configured to:
generate computer vision data via an analysis of the digital images with a neural network,
wherein the neural network outputs, for each of the digital images, a pose of the individual so as to establish serialized poses of the individual over the interval of time, and
encode the computer vision data into a portable data structure; and
a communication module configured to communicate the digital images and the portable data structure to a second apparatus with a graphics processing unit, at which the digital images are stored and the portable data structure is decoded for analysis of the computer vision data and determination of a health status of the individual.
2. The apparatus of claim 1 , wherein the second apparatus is able to display the computer vision data, or analyses of the computer vision data, so as to visually indicate the health status of the individual.
3. The apparatus of claim 1 , wherein the health status is representative of a musculoskeletal health state.
4. The apparatus of claim 1 , wherein the computer vision data indicates, for each of the digital images, two-dimensional locations of one or more joints of the individual.
5. The apparatus of claim 1 , wherein the computer vision data indicates, for each of the digital images, three-dimensional locations of one or more joints of the individual.
6. The apparatus of claim 1 , wherein the computer vision data indicates, for each of the digital images, three-dimensional rotation of one or more joints of the individual.
7. The apparatus of claim 1 , wherein the computer vision data indicates, for each of the digital images, a location, a size, and/or a shape of one or more muscles of the individual.
8. The apparatus of claim 1 , wherein the computer vision data includes a thermal map that is representative of a surface of a body of the individual.
9. The apparatus of claim 1 , wherein the computer vision data includes a volumetric representation of the individual that is comprised of voxels, each voxel representing a location whose spatial position is determined by the neural network.
10. A method for determining, with a first computing device with a graphics processing unit, a health status of an individual through analysis of computer vision data generated by a second computing device with a central processing unit, the method comprising:
acquiring, by the second computing device, a series of digital images generated by a camera in rapid succession of an environment in which an individual is situated;
applying, by the second computing device, a model to the series of digital images to produce a series of outputs,
wherein each output in the series of outputs is representative of information regarding a spatial position of the individual as determined through analysis of a corresponding digital image of the series of digital images, and
wherein the series of outputs are collectively representative of computer vision data;
populating, by the second computing device, the series of outputs into a data structure that is transmitted to the first computing device;
assessing, by the first computing device based on the computer vision data in the data structure, health of the individual in real time; and
performing, by the first computing device, an action based on the health of the individual.
11. The method of claim 10 , wherein said assessing comprises determining musculoskeletal performance of the individual, and wherein the method further comprises:
receiving, by either the first computing device or the second computing device, input indicative of a request to initiate an exercise therapy session; and
causing, by either the first computing device or the second computing device, presentation of an instruction to the individual to perform an exercise;
wherein the series of digital images are generated by the camera as the individual performs the exercise.
12. The method of claim 11 , wherein in response to a determination that the individual completed the exercise, said performing comprises instructing the individual to perform another exercise.
13. The method of claim 10 , wherein said assessing comprises performing fall detection based on the computer vision data.
14. The method of claim 10 , wherein said assessing comprises performing gait analysis based on the computer vision data.
15. The method of claim 10 , wherein said assessing comprises performing activity analysis based on the computer vision data, the activity analysis indicating an estimated level of effort being employed by the individual.
16. The method of claim 10 , wherein said assessing comprises performing fine motor skill analysis based on the computer vision data.
17. The method of claim 10 , wherein said assessing comprises performing range of motion analysis based on the computer vision data.
18. The method of claim 10 , wherein said assessing comprises performing muscle fatigue analysis based on the computer vision data, the muscle fatigue analysis indicating an estimated level of fatigue being experienced by a muscle of the individual.
19. The method of claim 10 , wherein said assessing comprises performing muscle distribution analysis based on the computer vision data, the muscle distribution analysis indicating an estimated location, size, and/or shape of a muscle of the individual.
20. The method of claim 10 , wherein said assessing comprises performing body mass index (BMI) analysis based on the computer vision data.
21. The method of claim 10 , wherein said assessing comprises performing blood flow analysis based on the computer vision data, the blood flow analysis indicating whether an estimated speed and/or volume of blood flow through the individual is abnormal.
22. The method of claim 10 , wherein said assessing comprises performing temperature analysis based on the computer vision data, the temperature analysis indicating temperature along a surface of a body of the individual in at least two different locations.
23. The method of claim 10 , further comprising:
providing, by the second computing device, the series of digital images to an encoder that produces, as output, an encoded data structure that is transmitted to the first computing device.
24. The method of claim 10 , wherein the series of outputs are encoded into data structure by the second computing device, and wherein the method further comprises:
providing, by the first computing device, the encoded data structure to a decoder that produces, as output, a decoded data structure from which the series of outputs can be extracted for analysis.
25. A system for assessing health of an individual, the system comprising:
a plurality of imaging apparatuses that are deployed in an environment in which an individual is situated,
wherein each imaging apparatus comprises:
an image sensor configured to produce digital images of the individual over an interval of time,
a central processing unit configured to:
generate a dataset that is representative of information related to the individual that is learned through analysis of the digital images, and
populate the dataset into a data structure, and
a communications interface via which the data structure exits the imaging apparatus; and
a processing apparatus that comprises:
a decoder,
a communications interface at which to receive a plurality of data structures from the plurality of imaging apparatuses, and
a graphics processing unit configured to:
provide the plurality of data structures to the decoder, so as to obtain a plurality of datasets, each of which corresponds to a different one of the plurality of data structures, and
assess health of the individual by examining the plurality of datasets that are output by the decoder for the plurality of data structures.
26. The system of claim 25 , wherein the image sensor included in each imaging apparatus is designed to cover the infrared, near infrared, visible, or ultraviolet regions.
27. The system of claim 25 , wherein the communications interface of the processing apparatus is part of a transceiver configured to facilitate wireless communication with each imaging apparatus via a separate communication channel.
28. The system of claim 25 , wherein the central processing unit of each imaging apparatus is further configured to append metadata that identifies the imaging apparatus to the data structure.
29. The system of claim 25 , wherein the plurality of imaging apparatuses are deployed in the environment such that each imaging apparatus produces the digital images of the individual from a different perspective.
30. The system of claim 25 , wherein the central processing unit of each imaging apparatus generates the dataset by applying one or more computer vision algorithms to the digital images.
31. The system of claim 25 , wherein at least one of the plurality of imaging apparatuses and the processing apparatus are representative of a single computing device.
32. The system of claim 25 , wherein the information specifies two- or three-dimensional locations of at least two joints of the individual over the interval of time.