IP Library Granted Patent US 10,936,903
Granted Patent B2
US 10,936,903 · App. 16/370,988 · Granted Mar 2, 2021

Technologies for labeling and validating human-machine interface high definition-map data

Inventors: Domingo Gonzalez (San Jose, CA); Ignacio J. Alvarez (Portland, OR); Mehrnaz Khodam Hazrati (San Jose, CA); Christopher Lopez-Araiza (San Jose, CA)
Assignee: Intel Corporation
G06K9/6254A63F13/23A63F13/46A63F13/533G06K9/00791A63F2300/1025A63F2300/308G06F3/04883
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,936,903
App. No.
16/370,988
Granted
Mar 2, 2021
Kind
B2
Abstract

Techniques are disclosed herein for collecting annotation data via a gamified user interface in a vehicle control system. According to an embodiment disclosed herein, the vehicle control system detects a trigger to initiate an annotation prompt associated with an object classified from an image. The vehicle control system presents, via a user interface, the annotation prompt. The vehicle control system receives, via the user interface, user input indicative of a response to the annotation prompt by a user and updates a confidence score associated with the classified object as a function of one or more metrics associated with the user.

Claims (52)

1. An apparatus, comprising:

a circuitry to:

detect a trigger to initiate an annotation prompt associated with an object classified from an image;

present, via a user interface, the annotation prompt;

receive, via the user interface, user input indicative of a response to the annotation prompt by a user;

update a confidence score associated with the classified object as a function of one or more metrics associated with the user;

generate, via the user interface, a second annotation prompt, the second annotation prompt including a verified object;

receive, via the user interface, user input indicative of a response to the second annotation prompt; and

update a user score, the user score associated with the user as a function of an accuracy of the response to the second annotation prompt relative to the verified object.

2. The apparatus of claim 1 , wherein the circuitry is further to generate, as a function of the trigger, the annotation prompt.

3. The apparatus of claim 2 , wherein to generate, as the function of the trigger, the annotation prompt comprises to determine, as the function of the trigger, a type of the annotation prompt.

4. The apparatus of claim 1 , wherein to detect the trigger to initiate the annotation prompt comprises to determine that the confidence score associated with the classified object is below a specified threshold.

5. The apparatus of claim 1 , wherein to detect the trigger to initiate the annotation prompt comprises to detect anomalous behavior in one or more sensors of the apparatus.

6. The apparatus of claim 1 , wherein to detect the trigger to initiate the annotation prompt comprises to receive, via the user interface, user input indicative of a request to initiate the annotation prompt.

7. The apparatus of claim 1 , wherein to update the confidence score associated with the classified object as the function of one or more metrics associated with the user comprises to update the confidence score as a function of the updated user score.

8. The apparatus of claim 1 , wherein the circuitry is further to determine whether the response matches a label associated with the verified object.

9. The apparatus of claim 1 , wherein the circuitry is further to, upon a determination that the confidence score exceeds a specified threshold, update a high definition-map with the classified object.

10. One or more non-transitory, machine-readable storage media having a plurality of instructions, which, when executed, causes a vehicle control system to:

detect a trigger to initiate an annotation prompt associated with an object classified from an image;

present, via a user interface, the annotation prompt;

receive, via the user interface, user input indicative of a response to the annotation prompt by a user;

update a confidence score associated with the classified object as a function of one or more metrics associated with the user;

generate, via the user interface, a second annotation prompt, the second annotation prompt including a verified object;

receive, via the user interface, user input indicative of a response to the second annotation prompt; and

update a user score, the user score associated with the user as a function of an accuracy of the response to the second annotation prompt relative to the verified object.

11. The one or more non-transitory, machine-readable storage media of claim 10 , wherein the plurality of instructions further causes the vehicle control system to generate, as a function of the trigger, the annotation prompt.

12. The one or more non-transitory, machine-readable storage media of claim 11 , wherein to generate, as the function of the trigger, the annotation prompt comprises to determine, as the function of the trigger, a type of the annotation prompt.

13. The one or more non-transitory, machine-readable storage media of claim 10 , wherein to detect the trigger to initiate the annotation prompt comprises to determine that the confidence score associated with the classified object is below a specified threshold.

14. The one or more non-transitory, machine-readable storage media of claim 10 , wherein to detect the trigger to initiate the annotation prompt comprises to detect anomalous behavior in one or more sensors of the vehicle control system.

15. The one or more non-transitory, machine-readable storage media of claim 10 , wherein to detect the trigger to initiate the annotation prompt comprises to receive, via the user interface, user input indicative of a request to initiate the annotation prompt.

16. The one or more non-transitory, machine-readable storage media of claim 10 , wherein to update the confidence score associated with the classified object as the function of one or more metrics associated with the user comprises to update the confidence score as a function of the updated user score.

17. The one or more non-transitory, machine-readable storage media of claim 10 , wherein the plurality of instructions further causes the vehicle control system to determine whether the response matches a label associated with the verified object.

18. The one or more non-transitory, machine-readable storage media of claim 10 , wherein the plurality of instructions further causes the vehicle control system to, upon a determination that the confidence score exceeds a specified threshold, update a high definition-map with the classified object.

19. A computer-implemented method comprising:

detecting, by a vehicle control system, a trigger to initiate an annotation prompt associated with an object classified from an image;

presenting, by the vehicle control system and via a user interface, the annotation prompt;

receiving, by the vehicle control system and via the user interface, user input indicative of a response to the annotation prompt by a user;

updating, by the vehicle control system, a confidence score associated with the classified object as a function of one or more metrics associated with the user;

generating, by the vehicle control system and via the user interface, a second annotation prompt, the second annotation prompt including a verified object;

receiving, by the vehicle control system and via the user interface, user input indicative of a response to the second annotation prompt; and

updating, by the vehicle control system, a user score, the user score associated with the user as a function of an accuracy of the response to the second annotation prompt relative to the verified object.

20. The computer-implemented method of claim 19 , further comprising generating, by the vehicle control system and as a function of the trigger, the annotation prompt.

21. The computer-implemented method of claim 20 , wherein generating the annotation prompt comprises determining, by the vehicle control system and as the function of the trigger, a type of the annotation prompt.

22. The computer-implemented method of claim 19 , wherein detecting the trigger to initiate the annotation prompt comprises determining, by the vehicle control system, that the confidence score associated with the classified object is below a specified threshold.

23. A vehicle control system comprising:

means for detecting a trigger to initiate an annotation prompt associated with an object classified from an image;

circuitry for presenting, via a user interface, the annotation prompt;

circuitry for receiving, via the user interface, user input indicative of a response to the annotation prompt by a user;

means for updating a confidence score associated with the classified object as a function of one or more metrics associated with the user;

circuitry for generating a second annotation prompt, the second annotation prompt including a verified object;

circuitry for receiving user input indicative of a response to the second annotation prompt; and

means for updating a user score, the user score associated with the user as a function of an accuracy of the response to the second annotation prompt relative to the verified object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2019
From: GONZALEZ, DOMINGO; ALVAREZ, IGNACIO; HAZRATI, MEHMAZ KHODAM; LOPEZ-ARAIZA, CHRISTOPHER
To: INTEL CORPORATION
Reel/Frame 049256/0530 →
Continuity (1)
Related Publication 20190228262A1 · Jul 25, 2019
Cited By (2)
US 12,411,232 US 12,493,576