IP Library › Granted Patent US 11,030,436
Granted Patent B2
US 11,030,436 · App. 16/606,171 · Granted Jun 8, 2021

Object recognition

Inventors: Yang Lei (Palo Alto, CA); Jian Fan (Palo Alto, CA); Jerry Liu (Palo Alto, CA)
Assignee: Hewlett-Packard Development Company, L.P.
G06K9/00214G06K9/4671G06K9/6215G06T7/10G06T7/90G06T2200/04G06T2207/10024G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,030,436
App. No.
16/606,171
Granted
Jun 8, 2021
Kind
B2
Abstract

A method of recognizing an object includes comparing a three-dimensional point cloud of the object to a three-dimensional candidate from a dataset to determine a first confidence score, and comparing color metrics of a two-dimensional image of the object to a two-dimensional candidate from the dataset to determine a second confidence score. The point cloud includes a color appearance calibrated from a white balance image, and the color appearance of the object is compared with the three-dimensional candidate. The first or second confidence score is selected to determine which of the three-dimensional candidate or the two-dimensional candidate corresponds with the object.

Claims (27)

1. A method of recognizing an object, comprising:

comparing a three-dimensional point cloud of the object to a three-dimensional candidate from a dataset to determine a first confidence score, the point cloud including a color appearance calibrated from a white balance image and the comparing including comparing the color appearance of the object with the three-dimensional candidate;

comparing color metrics of a two-dimensional image of the object to a two-dimensional candidate from the dataset to determine a second confidence score; and

selecting one of the first and second confidence scores to determine which of the three-dimensional candidate or the two-dimensional candidate corresponds with the object.

2. The method of claim 1 wherein the selecting includes selecting one of the first and second confidence scores if the three-dimensional candidate and the two-dimensional candidate do not both correspond with the object.

3. The method of claim 1 wherein the selected one of the first and second confidence scores at least meets a threshold.

4. The method of claim 1 wherein the comparing color metrics includes comparing local color keypoints.

5. The method of claim 4 wherein the first and second confidence scores are based on keypoints.

6. The method of claim 1 wherein the comparing the three-dimensional point cloud of the object and comparing color metrics of a two-dimensional image of the object are performed concurrently.

7. A non-transitory computer readable medium to store computer executable instructions to control a processor to:

generate a white balance calibration against a surface;

compare a three-dimensional point cloud of an object to be recognized against a three-dimensional candidate, the point cloud including a color appearance determined from the white balance calibration;

compare color metrics of a two-dimensional image of the object to a two-dimensional candidate; and

select one of the three-dimensional candidate and the two-dimensional candidate to determine which of the three-dimensional candidate or the two-dimensional candidate corresponds with the object.

8. The computer readable medium of claim 7 wherein the surface is a planar mat.

9. The computer readable medium of claim 7 wherein the selected one of the three-dimensional candidate and the two-dimensional candidate is based on confidence scores.

10. The computer readable medium of claim 7 wherein the point cloud includes a segmentation against the surface.

11. The computer readable medium of claim 7 wherein a subset of candidates is generated from a color appearance comparison.

12. The computer readable medium of claim 11 wherein the subset of candidates are ranked in terms of correspondence with the object.

13. A system, comprising:

memory to store a set of instructions; and

a processor to execute the set of instructions to:

compare a three-dimensional point cloud of the object to a three-dimensional candidate from a dataset to determine a first confidence score, the point cloud including a color appearance calibrated from a white balance image and the comparing including comparing the color appearance of the object with the three-dimensional candidate;

compare color metrics of a two-dimensional image of the object to a two-dimensional candidate from the dataset to determine a second confidence score; and

select one of the first and second confidence scores to determine which of the three-dimensional candidate or the two-dimensional candidate corresponds with the object.

14. The system of claim 13 comprising a color camera and color-depth camera operably coupled to the processor.

15. The system of claim 14 wherein the color camera is configured to generate the two-dimensional image of the object and the color-depth camera is configured to generate the three-dimensional point cloud of the object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 26, 2019
From: LEI, YANG; FAN, JIAN; LIU, JERRY
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 051124/0771 →
Continuity (1)
Related Publication 20200125830A1 · Apr 23, 2020