Granted Patent B2

US 12,347,060 · App. 17/874,770 · Granted Jul 1, 2025

Inventors: Jared R. Prink (Newberg, OR); Troy D. Driscoll (West Linn, OR)

Assignee: Rockwell Collins, Inc.

G06T3/20G06F3/012G06T11/001

Abstract

Claims (99)

1. A system, comprising:

a head-worn display (HWD) wearable by a user, the HWD comprising:

at least one display unit for presenting output images to the user; and

at least one head tracker configured to determine pose data associated with a head position and a head orientation of the user; and

an image processing module in communication with the HWD, the image processing module comprising:

one or more graphics generation processors configured to:

receive at least one frame of image data corresponding to an output image;

receive from the at least one head tracker the pose data corresponding to the image data;

generate, at least one of 1) a first symbology set associated with the at least one frame, the first symbology set oriented according to a first inertial reference frame and 2) a second symbology set associated with the at least one frame, the second symbology set oriented according to a second inertial reference frame;

determine and assign a hierarchy to the first symbology set and the second symbology set; and

one or more image rendering processors in communication with the graphics generation processors, the image rendering processors configured to:

encode the first symbology set according to a first color range;

encode the second symbology set according to a second color range;

embed the associated pose data in the image data; and

forward to the HWD the image data, the encoded first symbology set, the encoded second symbology set, and embedded pose data;

wherein the HWD further comprises a pose augmentation processor and a tracker processor, the tracker processor configured to:

forward the encoded second symbology set and the embedded pose data to the pose augmentation processor;

produce the output image based on the image data, the encoded first symbology set, a pose-adjusted encoded second symbology set, and the hierarchy previously determined, wherein higher-priority symbology is drawn over lower-priority symbology in the output image;

remove the first and second color range encoding from the output image; and

forward the output image to the display unit; and

the pose augmentation processor configured to produce the pose-adjusted encoded second symbology set by adjusting the encoded second symbology set based on the associated pose data.

2. The system of claim 1 , wherein:

the pose data is head pose data associated with a head-centered inertial reference frame;

the encoded second symbology set is associated with an earth-centered inertial reference frame; and

the pose augmentation processor is configured to adjust the encoded second symbology set according to the head-centered inertial reference frame.

3. The system of claim 1 , wherein:

the pose data is associated with an earth-centered inertial reference frame;

the encoded second symbology set is associated with a head-centered inertial reference frame; and

the pose augmentation processor is configured to adjust the encoded second symbology set according to the earth-centered inertial reference frame.

4. The system of claim 1 , wherein the tracker processor is configured to produce each output image by drawing the encoded first symbology set over the adjusted encoded second symbology set.

5. The system of claim 1 , wherein at least one of the encoded first symbology set or the encoded second symbology set comprises:

at least one first encoded pixel set encoded according to a first color value within the first color range or the second color range; and

at least one second encoded pixel set encoded according to a second color value lower than the first color value.

6. The system of claim 5 , wherein:

the at least one first encoded pixel set corresponds to one or more symbols having a first priority; and

the at least one second encoded pixel set corresponds to one or more symbols having a second priority lower than the first priority.

7. The system of claim 5 , wherein:

the at least one first encoded pixel set corresponds to one or more symbols; and

the at least one second encoded pixel set corresponds to one or more of a mask or a halo associated with the one or more symbols.

8. The system of claim 1 , wherein:

the image rendering processors are configured to encode the embedded pose data according to a third color range; and

the tracker processor is configured to remove the third color range encoding and the embedded pose data from the output image.

9. The system of claim 1 , wherein:

the first color range corresponds to a first color channel; and

the second color range corresponds to a second color channel.

10. The system of claim 1 , wherein:

the first color range corresponds to a first subset of an RGB color channel; and

the second color range corresponds to a second subset of the RGB color channel.

11. The system of claim 1 , wherein:

the head tracker is configured to determine display pose data corresponding to a display time of the output image;

wherein the image processing module is configured to:

receive first image data corresponding to first pose data and to a first time; and

receive second image data corresponding to second pose data and to a second time;

wherein the pose augmentation processor is configured to:

adjust the at least one encoded second symbology set associated with the first image data based on the display pose data and a time difference between the display time and the first time; and

adjust the at least one encoded second symbology set associated with the second image data based on the display pose data and a time difference between the display time and the second time and

wherein the tracker processor is configured to produce a combined output image based on the first image data and the second image data.

12. A method, comprising:

receiving, via an image rendering system in communication with a head-worn display (HWD), at least one frame of image data associated with an output image to be displayed by the HWD;

determining, via a head tracker of the HWD, pose data associated with the at least one frame of image data, the pose data corresponding to a head position and a head orientation of a user;

receiving, via image rendering processors, the pose data associated with the at least one frame of image data;

generating, via the image rendering processors, at least one of 1) a first symbology set associated with the at least one frame, the first symbology set oriented according to a first inertial reference frame and 2) a second symbology set associated with the at least one frame, the second symbology set oriented according to a second inertial reference frame;

encoding, via the image rendering processors, 1) the first symbology set according to a first color range and 2) the second symbology set according to a second color range;

determining and assigning a hierarchy to the first symbology set and the second symbology set;

forwarding to the HWD the image data, the associated pose data, the encoded first symbology set, and the encoded second symbology set;

pose-adjusting, via a pose augmentation processor of the HWD, the encoded second symbology set based on the associated pose data;

producing, via the HWD, the associated output image based on the image data, the encoded first symbology set, the pose-adjusted encoded second symbology set, and the hierarchy previously determined, wherein higher-priority symbology is drawn over lower-priority symbology in the output image; and

removing, via the HWD, the first and second color range encoding from the output image.

13. The method of claim 12 , wherein:

the pose data is head pose data associated with a head-centered inertial reference frame;

the encoded second symbology set is associated with an earth-centered inertial reference frame; and

pose-adjusting, via a pose augmentation processor of the HWD, the encoded second symbology set based on the associated pose data includes pose-adjusting the encoded second symbology set based on the head-centered inertial reference frame.

14. The method of claim 12 , wherein:

the pose data is associated with an earth-centered inertial reference frame;

the encoded second symbology set is associated with a head-centered inertial reference frame; and

pose-adjusting, via a pose augmentation processor of the HWD, the encoded second symbology set based on the associated pose data includes pose-adjusting the encoded second symbology set based on the earth-centered inertial reference frame.

15. The method of claim 12 , wherein producing, via the HWD, the associated output image based on the image data, the encoded first symbology set, and the pose-adjusted encoded second symbology set includes:

drawing the pose-adjusted encoded second symbology set over the image data; and

drawing the encoded first symbology set over the pose-adjusted encoded second symbology set.

16. The method of claim 15 , wherein encoding, via the image rendering processors, 1) the first symbology set according to a first color range and 2) the second symbology set according to a second color range includes:

encoding at least one first pixel subset of the first symbology set according to a first color value within the first color range; and

encoding at least one second pixel subset of the first symbology set according to a second color value lower than the first color value.

17. The method of claim 16 , wherein:

the at least one first pixel subset corresponds to one or more symbols having a first priority; and

the at least one second pixel subset corresponds to one or more symbols having a second priority lower than the first priority.

18. The method of claim 16 , wherein:

the at least one first pixel subset corresponds to one or more symbols; and

the at least one second pixel subset corresponds to one or more of a mask or a halo associated with the one or more symbols.

19. The method of claim 12 , wherein:

encoding, via the image rendering processors, 1) the first symbology set according to a first color range and 2) the second symbology set according to a second color range includes:

embedding the associated pose data in the image data; and

encoding the embedded pose data according to a third color range; and

wherein producing, via the HWD, the associated output image based on the image data, the encoded first symbology set, and the pose-adjusted encoded second symbology set includes removing the third color range encoding and the embedded pose data from the output image.

20. The method of claim 12 , wherein:

receiving, via an image rendering system in communication with a head-worn display (HWD), at least one frame of image data associated with an output image to be displayed by the HWD includes receiving 1) first image data corresponding to first pose data and to a first time and 2) second image data corresponding to second pose data and to a second time;

wherein determining, via a head tracker of the HWD, pose data associated with the at least one frame includes determining display pose data corresponding to a display time of the output image; and

wherein pose-adjusting, via a pose augmentation processor of the HWD, the encoded second symbology set based on the associated pose data includes:

adjusting the at least one encoded second symbology set associated with the first image data based on the display pose data and a time difference between the display time and the first time; and

adjusting the at least one encoded second symbology set associated with the second image data based on the display pose data and a time difference between the display time and the second time.

Assignments (1)

Continuity (1)

References Cited (25)