Three-dimensional object tracking using distributed thin-client cameras
View Patent ↗An object tracking system which includes a plurality of camera devices, each of which captures image data, a plurality of thin-client processors, each of which is connected to a respective camera device via a local connection, each thin-client processor hosting a thin-client application that processes the captured image data to obtain two-dimensional foreground region information by using a background subtraction algorithm, and a server hosting an object tracking application that receives the foreground region information from each thin-client processor via a network and generates a three-dimensional visual hull corresponding to each foreground region represented in the received foreground region information, wherein the object tracking application generates identification and position data corresponding to each three-dimensional visual hull. The thin-client application uses two-dimensional object tracking to identify each object in the foreground region information, and sends each object identity to the object tracking application with the foreground region information.
1. A method for tracking objects, the method comprising the steps of:
capturing image data at each of a plurality of camera devices;
processing, in each of a plurality of thin-client processors, the image data to obtain two-dimensional foreground region information by using a background subtraction algorithm, wherein the processing comprises determining background information at each pixel of the image data based on previously-determined foreground region information, and wherein the background subtraction algorithm determines foreground region information for each foreground region in the image data that corresponds to an object and uses a background model to separate the foreground region information for each foreground region from the image data;
facilitating sending of the two-dimensional foreground region information to an apparatus having a processor configured to generate a three-dimensional visual hull;
receiving, in each of the plurality of thin-client processors from the apparatus, background correction information that is generated by the apparatus based on the three-dimensional visual hull; and
updating the background model based on previously-determined foreground region information and an illumination level change, and based on the background correction information received from the apparatus having the processor configured to generate the three-dimensional visual hull.
2. The method according to claim 1 , wherein the background model is updated with the background correction information from the apparatus, wherein the background correction information corresponds to a respective camera device.
3. The method according to claim 1 , wherein the two-dimensional foreground region information includes an identification code corresponding to each foreground region in the image data.
4. The method according to claim 1 , wherein each of the plurality of thin-client processors is remotely controlled and synchronized by the processor of the apparatus.
5. The method according to claim 4 , wherein, in the sending step, the two-dimensional foreground region information is sent to the apparatus from each of the plurality of thin-client processors in response to a request command from the apparatus.
6. The method according to claim 1 , further comprising: generating each three-dimensional visual hull by using a volume reconstruction algorithm which reconstructs a volume for each foreground region represented in the two-dimensional foreground region information from the plurality of thin-client processors.
7. The method according to claim 1 , wherein at least one of the plurality of thin-client processors is disposed within the corresponding camera device.
8. The method according to claim 1 , further comprising:
tracking, at each of the plurality of thin-client processors, each foreground region from one frame to a next frame using two-dimensional object tracking based on two-dimensional motion models.
9. The method according to claim 1 , wherein the two-dimensional foreground region information sent to the apparatus comprises two-dimensional object position data.
10. The method according to claim 1 , wherein the foreground region information comprises only pixels of the image data having an illumination level change greater than a predetermined threshold,
wherein updating the background model based on previously-determined foreground region information and an illumination level change, and based on the background correction information received from the apparatus having the processor configured to generate the three-dimensional visual hull comprises:
updating the background model periodically using an adaptive background estimation algorithm which determines background information at each pixel of the image data based on previously-determined foreground region information and based on an illumination level change at each pixel of the image data and using the adaptive background estimation algorithm based on the background correction information received from the apparatus having the processor configured to generate the three-dimensional visual hull.
11. A non-transitory computer-readable medium encoded with a computer program for processing image data, the computer program comprising code for:
receiving image data;
processing the image data to obtain two-dimensional foreground region information from the image data using a background subtraction algorithm, wherein the processing comprises determining background information at each pixel of the image data based on previously-determined foreground region information, and wherein the background subtraction algorithm determines foreground region information for each foreground region in the image data that corresponds to an object and uses a background model to separate the foreground region information for each foreground region from the image data;
facilitating sending of the two-dimensional foreground region information to an apparatus configured to generate a three-dimensional visual hull;
receiving, in a processor from the apparatus, background correction information that is generated by the apparatus based on the three-dimensional visual hull; and
updating the background model based on previously-determined foreground region information and an illumination level change, and based on the background correction information received from the apparatus having the processor configured to generate the three-dimensional visual hull.
12. An object tracking system, comprising:
a plurality of thin-client processors, each of the plurality of thin client processors configured to receive image data and configured to process the image data to obtain two-dimensional foreground region information by using a background subtraction algorithm in which background information at each pixel of the image data is determined based on previously-determined foreground region information, foreground region information is determined for each foreground region in the image data that corresponds to an object, and a background model is used to separate the foreground region information for each foreground region from the image data, the plurality of thin-client processors configured to facilitate sending of the two-dimensional foreground region information to an apparatus having a processor configured to generate a three-dimensional visual hull, each of the plurality of thin-client processors configured to receive, from the apparatus, background correction information that is generated by the apparatus based on the three-dimensional visual hull, each of the plurality of thin-client processors configured to update the background model based on previously-determined foreground region information and an illumination level change, and based on the background correction information received from the apparatus having the processor configured to generate the three-dimensional visual hull.
13. The object tracking system according to claim 12 , further comprising a plurality of camera devices configured to capture the image data, wherein the background model is configured to be updated with the background correction information received from the apparatus, wherein the background correction information corresponds to a respective camera device.
14. The object tracking system according to claim 12 , wherein the two-dimensional foreground region information includes an identification code corresponding to each foreground region in the image data.
15. The object tracking system according to claim 12 , wherein each of the plurality of thin-client processors is remotely controlled and synchronized by the processor of the apparatus.
16. The object tracking system according to claim 12 , wherein each of the plurality of thin-client processors is configured to send the two-dimensional foreground region information to the apparatus in response to a request command received from the apparatus.
17. The object tracking system according to claim 12 , further comprising the apparatus having the processor, wherein the processor is configured to generate a three-dimensional visual hull corresponding to each foreground region represented in the two-dimensional foreground region information, wherein the processor is configured to generate each three-dimensional visual hull by using a volume reconstruction algorithm which reconstructs a volume for each foreground region represented in the two-dimensional foreground region information from the plurality of thin-client processors, wherein the processor is configured to generate identification and position data corresponding to each three-dimensional visual hull.
18. The object tracking system according to claim 12 , further comprising a plurality of camera devices configured to capture the image data, each of the plurality of thin-client processors connected to a respective one of the plurality of camera devices via a local connection, wherein at least one of the plurality of thin-client processors is disposed within the corresponding camera device.
19. The object tracking system according to claim 12 , wherein each of the plurality of thin-client processors is configured to track each foreground region from one frame to a next frame using two-dimensional object tracking based on two-dimensional motion models.
20. The object tracking system according to claim 12 , wherein the two-dimensional foreground region information sent to the apparatus comprises two-dimensional object position data.
21. The object tracking system according to claim 12 , wherein the foreground region information comprises only pixels of the image data having an illumination level change greater than a predetermined threshold,
wherein each of the plurality of thin-client processors configured to update the background model periodically using an adaptive background estimation algorithm which determines background information at each pixel of the image data based on previously-determined foreground region information and based on an illumination level change at each pixel of the image data and using the adaptive background estimation algorithm based on the background correction information received from the apparatus having the processor configured to generate the three-dimensional visual hull.