IP Library Granted Patent US 11,455,750
Granted Patent B2
US 11,455,750 · App. 16/978,602 · Granted Sep 27, 2022

Operator characteristic-based visual overlays

Inventors: Allen Owen Wright (Corvallis, OR); Krishna Prasad Muraleedharan Pillai (Corvallis, OR); Evan Scheessele (Corvallis, OR)
Assignee: Hewlett-Packard Development Company, L.P.
G06T11/00G06F3/013G06Q10/06316G06Q10/06398G06Q10/063112G06Q10/063114G06Q10/20G06Q30/0185G06V20/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,455,750
App. No.
16/978,602
Granted
Sep 27, 2022
Kind
B2
Abstract

A system may include a processing resource, and a computing device comprising instructions executable to: extract data from objects in a digital image of a physical environment; utilize the extracted data to identify information about a task to be performed by an operator at the physical environment; and select, based on a characteristic of the operator, a portion of the identified information about the task to include in a visual overlay to be displayed to the operator at the physical environment.

Claims (32)

1. A system comprising:

a processing resource; and

a computing device comprising instructions executable by the processing resource to:

extract data from objects in a digital image of a physical environment;

utilize the extracted data to identify information about a task to be performed by an operator at the physical environment;

extract, from a digital image of a performance of the task by the operator, a characteristic of the operator indicative of an experience level of the operator; and

select, based on the extracted characteristic of the operator, a portion of the identified information about the task to include in a visual overlay to be displayed to the operator at the physical environment.

2. The system of claim 1 , wherein the characteristic of the operator includes a preference indicated by the operator for a type of information to include in the visual overlay.

3. The system of claim 1 , wherein the characteristic of the operator includes an expertise level associated with the operator.

4. The system of claim 1 , wherein the task includes a maintenance operation involving the objects in the digital image.

5. The system of claim 4 , wherein the selected portion of the identified information includes information for completion of the maintenance operation that is tailored to the operator's expertise level for the maintenance operation.

6. The system of claim 5 , wherein an unselected portion of the identified information includes information that the operator has previously demonstrated a familiarity with.

7. A non-transitory machine-readable medium containing instructions executable by a processor to cause the processor to:

identify markings on a physical object appearing in a digital image of a physical environment where an operator is to perform a task;

utilize the identified markings to identify information about the physical object;

reference a profile of characteristics of the operator, wherein the characteristics are indications of a task experience level of the operator determined from digital images of previous performances of the task by the operator; and

select, based on the experience level of the operator to perform the task as determined from the profile, a portion of the information about the physical object to be included in a visual overlay to be displayed to the operator at the physical environment.

8. The non-transitory machine-readable medium of claim 7 , wherein the instructions to identify the markings include instructions to identify a modification of a characteristic of the marking.

9. The non-transitory machine-readable medium of claim 8 , wherein the instructions to utilize the identified markings to identify information about the physical object include instructions to identify that the physical object has been subjected to a particular environmental condition based on the modification of the characteristic of the marking.

10. The non-transitory machine-readable medium of claim 7 , wherein the instructions to identify the information about the physical object include instructions to determine when the object was installed in a machine present at the physical environment and to select the portion of information including a recommend maintenance to the object based on when the object was installed.

11. The non-transitory machine-readable medium of claim 7 , wherein the instructions to identify the information about the physical object include instructions to determine whether a portion of a machine that the physical object is a component of is outside of a specification for the machine based on a relationship between the physical object and another component of the machine.

12. A method, comprising:

capturing a digital image of a physical environment where an operator is to perform a task;

identifying instructions related to performing the task based on markings recognized in the digital image;

determining, based on a digital image of a prior performance of the task by the operator, a characteristic of the operator associated with a task experience level of the operator;

selecting, based on the determined characteristic of the operator, a first portion of the instructions to be included in a first visual overlay to be displayed to the operator at the physical environment; and

tracking a performance of the task by the operator at the physical environment.

13. The method of claim 12 , comprising modifying the characteristic of the operator based on the tracked performance of the task.

14. The method of claim 12 , comprising selecting, based on the tracked performance, a second portion of the instructions to be included in a second visual overlay to be displayed to the operator at the physical environment.

15. The method of claim 12 , comprising:

tracking an eye gaze of the operator when the first visual overlay is displayed to the operator; and

modifying the first visual overlay based on the tracked eye gaze.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 15, 2020
From: WRIGHT, ALLEN OWEN; MURALEEDHARAN PILLAI, KRISHNA PRASAD; SCHEESSELE, EVAN
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 053772/0722 →
Continuity (1)
Related Publication 20210035333A1 · Feb 4, 2021