IP Library Granted Patent US 8,831,289
Granted Patent B2
US 8,831,289 · App. 13/351,680 · Granted Sep 9, 2014

Identification of objects in a 3D video using non/over reflective clothing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,831,289
App. No.
13/351,680
Granted
Sep 9, 2014
Kind
B2
Abstract

A computing system generates a depth map from at least one image, detects objects in the depth map, and identifies anomalies in the objects from the depth map. Another computing system identifies at least one anomaly in an object in a depth map, and uses the anomaly to identify future occurrences of the object. A system includes a three dimensional (3D) imaging system to generate a depth map from at least one image, an object detector to detect objects within the depth map, and an anomaly detector to detect anomalies in the detected objects, wherein the anomalies are logical gaps and/or logical protrusions in the depth map.

Claims (69)

1. A computing system for processing images, comprising:

a 3-D imaging system, the 3-D imaging system generates at least one depth map from at least one image of a first person in an application during a calibration process of the application;

an object detector, the object detector identifies a body part of the first person in the at least one depth map;

an anomaly detector, the anomaly detector, responsive to the object detector, identifies, from the at least one depth map, an anomaly in the body part of the first person, the anomaly in the body part of the first person comprising: (a) a logical gap anomaly which represents a first object of the body part of the first person appearing to be further from the 3-D imaging system than the first object actually is due to a lower reflective property of the first object, where the first object is identified by pixel depths which are less than pixel depths of an adjacent region of the first object or (b) a logical protrusion anomaly which represents the first object appearing to be closer to the 3-D imaging system than the first object actually is due to a higher reflective property of the first object, where the first object is identified by pixel depths which are greater than pixel depths of an adjacent region of the first object; and

an anomaly marker, the anomaly marker, responsive to the anomaly detector, generates a marker which identifies the anomaly in the body part of the first person, the anomaly marker forwards the marker which identifies the anomaly in the body part of the first person to the object detector and the object detector, in response to the anomaly marker, subsequently uses the marker which identifies the anomaly in the body part of the first person as a template to identify a future occurrence of the body part of the first person in a series of depth maps after the calibration process.

2. The computing system of claim 1 , wherein:

the anomaly marker marks the anomaly in the body part of the first person by an indication of a size and shape of the anomaly in the body part of the first person.

3. The computing system of claim 1 , wherein:

the application comprises a multiplayer application; and

the calibration process occurs prior to a start of the multiplayer application.

4. The computing system of claim 1 , wherein:

the first object comprises hair of the body part of the first person and the body part comprises a head of the first person.

5. The computing system of claim 1 , wherein:

the first object comprises an object worn on the body part of the first person and the body part comprises an arm or wrist of the first person.

6. The computing system of claim 1 , wherein:

the anomaly in the body part of the first person is used to distinguish between left and right paired objects, and the first object is one object of a pair of objects.

7. The computing system of claim 1 , wherein:

the first object comprises at least one of: clothing, a clothing accessory, a jewelry or a medical artifact worn on the body part.

8. The computing system of claim 1 , wherein the anomaly detector, to identify the anomaly in the body part of the first person:

calculates a depth difference by a comparison of depths of at least two regions in the body part of the first person; and

determines that an anomaly exists where an absolute value of the depth difference exceeds a threshold.

9. The computing system of claim 1 , further comprising:

an anomaly compensator, the anomaly compensator compensates for the anomaly in the body part of the first person by a replacement of pixels associated with the anomaly in the body part of first person with pixels of depth similar to that of a region adjacent to the anomaly in the body part of first person.

10. The computing system of claim 1 , wherein:

the at least one image comprises a first image of a scene generated using continuous radiation;

a second image of the scene is generated using pulsed radiation; and

the anomaly detector, to identify the anomaly in the body part of the first person:

finds associated pixels of the first and second images which have intensities below a threshold;

calculates differences between the intensities of the associated pixels in the first and second images; and

determines that an anomaly exists in response to a determination that the calculated differences are less than a pixel intensity threshold.

11. The computing system of claim 10 , wherein the anomaly detector, to identify the anomaly in the body part of the first person:

derives compensated depth values by a division of the intensities of the first image by the intensities of the second image;

determines that the compensated depth values exceed a depth threshold; and

determines that an anomaly exists responsive to the determination that the compensated depth values exceed the depth threshold.

12. The computing system of claim 1 , wherein:

the at least one image comprises a second person in the application during the calibration process of the application;

the object detector identifies a body part of the second person in the at least one depth map;

the anomaly detector, responsive to the object detector, identifies, from the at least one depth map, an anomaly in the body part of the second person, the anomaly in the body part of the second person comprising: (a) a logical gap anomaly which represents a second object of the body part of the second person appearing to be further from a 3-D imaging system than the second object actually is due to a lower reflective property of the second object, where the second object is identified by pixel depths which are less than pixel depths of an adjacent region of the second object or (b) a logical protrusion anomaly which represents the second object appearing to be closer to the 3-D imaging system than the second object actually is due to a higher reflective property of the second object, where the second object is identified by pixel depths which are greater than pixel depths of an adjacent region of the second object; and

the anomaly marker, responsive to the anomaly detector, provides a marker which identifies the anomaly in the body part of the second person, the object detector uses the marker which identifies the anomaly in the body part of the second person as a template to identify a future occurrence of the body part of the second person in the series of depth maps and to differentiate between the first and second persons.

13. A method for processing images, comprising:

performing a calibration process prior to a start of an application, the performing the calibration process comprises:

generating at least one depth map from at least one image of a first person;

detecting a body part of the first person in the at least one depth map;

identifying, from the at least one depth map, responsive to the detecting, an anomaly in the body part of the first person, the anomaly in the body part of the first person comprising: (a) a logical gap anomaly which represents a first object of the body part of the first person appearing to be further from the 3-D imaging system than the first object actually is due to a lower reflective property of the first object, where the first object is identified by pixel depths which are less than pixel depths of an adjacent region of the first object or (b) a logical protrusion anomaly which represents the first object appearing to be closer to the 3-D imaging system than the first object actually is due to a higher reflective property of the first object, where the first object is identified by pixel depths which are greater than pixel depths of an adjacent region of the first object;

responsive to the identifying, generating a marker which identifies the anomaly in the body part of the first person; and

forwarding the marker which identifies the anomaly in the body part of the first person to an object detector; and

after the performing the calibration process, during the application and at the object detector: using the marker which identifies the anomaly in the body part of the first person as template, identifying a future occurrence of the body part of the first person in a series of depth maps.

14. The method of claim 13 , wherein:

the marker which identifies the anomaly in the body part of the first person comprises an indication of a size and shape of the anomaly in the body part of the first person.

15. The method of claim 13 , wherein:

the application comprises a multiplayer application.

16. The method of claim 13 , wherein:

the performing the calibration process comprises prompting the first person to stand in a specific pose or to display the body part.

17. The method of claim 13 , wherein:

the performing the calibration process further comprises:

detecting a body part of a second person in the at least one depth map;

identifying, from the at least one depth map, an anomaly in the body part of the second person, the anomaly in the body part of the second person comprising: (a) a logical gap anomaly which represents a second object of the body part of the second person appearing to be further from the 3-D imaging system than the second object actually is due to a lower reflective property of the second object, where the second object is identified by pixel depths which are less than pixel depths of an adjacent region of the second object or (b) a logical protrusion anomaly which represents the second object appearing to be closer to the 3-D imaging system than the second object actually is due to a higher reflective property of the second object, where the second object is identified by pixel depths which are greater than pixel depths of an adjacent region of the second object; and

responsive to the identifying of the anomaly in the body part of the second person, providing a marker which identifies the anomaly in the body part of the second person; and

after the performing the calibration process, during the application: using the marker which identifies the anomaly in the body part of the second person as a template, identifying a future occurrence of the body part of the second person and differentiating between the body part of the first person and the body part of the second person, in the series of depth maps.

18. The method of claim 17 , further comprising:

the performing the calibration process comprises prompting the first and second persons to stand in specific poses in which their projections as perceived by the 3-D imaging system do not overlap.

19. A computing system for processing images, comprising:

a 3-D imaging system, the 3-D imaging system generates at least one depth map from at least one image of first and second persons in a multiplayer application during a calibration process of the multiplayer application;

an object detector, the object detector identifies a body part of the first person and a body part of the second person in the at least one depth map;

an anomaly detector, the anomaly detector, responsive to the object detector, identifies, from the at least one depth map, an anomaly in the body part of the first person and an anomaly in the body part of the second person, the anomaly in the body part of the first person comprising a logical gap anomaly or a logical protrusion anomaly, and the anomaly in the body part of the second person comprising the logical gap anomaly or the logical protrusion anomaly; and

an anomaly marker, the anomaly marker, responsive to the anomaly detector, generates a marker which identifies the anomaly in the body part of the first person and a marker which identifies the anomaly in the body part of the second person, the object detector uses the marker which identifies the anomaly in the body part of the first person and the marker which identifies the anomaly in the body part of the second person to differentiate between the body part of the first person and the body part of the second person in future occurrences of the body part of the first person and the body part of the second person in a series of depth maps, after the calibration process.

20. The computing system of claim 19 , wherein:

the anomaly in the body part of the first person represents a first object of the body part of the first person appearing to be further from the 3-D imaging system than the first object actually is due to a lower reflective property of the first object, where the first object is identified by pixel depths which are less than pixel depths of an adjacent region of the first object or the first object appearing to be closer to the 3-D imaging system than the first object actually is due to a higher reflective property of the first object, where the first object is identified by pixel depths which are greater than pixel depths of an adjacent region of the first object; and

the anomaly in the body part of the second person represents a second object of the body part of the second person appearing to be further from the 3-D imaging system than the second object actually is due to a lower reflective property of the second object, where the second object is identified by pixel depths which are less than pixel depths of an adjacent region of the second object or the second object appearing to be closer to the 3-D imaging system than the second object actually is due to a higher reflective property of the second object, where the second object is identified by pixel depths which are greater than pixel depths of an adjacent region of the second object.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 24, 2021
From: MICROSOFT CORPORATION
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 057590/0552 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2020
From: MICROSOFT INTERNATIONAL HOLDINGS B.V.
To: MICROSOFT CORPORATION
Reel/Frame 051787/0960 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2012
From: KATZ, SAGI; YAHAV, GIORA
To: 3DV SYSTEMS LTD.
Reel/Frame 027633/0977 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2012
From: 3DV SYSTEMS LTD.
To: MICROSOFT INTERNATIONAL HOLDINGS B.V.
Reel/Frame 027634/0049 →