IP Library Granted Patent US 11,769,262
Granted Patent B2
US 11,769,262 · App. 17/833,196 · Granted Sep 26, 2023

Techniques for accurate pose estimation

Inventors: Alberico Menozzi (Raleigh, NC); Chris F. Argenta (Cary, NC); James B. Cook (Apex, NC)
Assignee: Applied Research Associates, Inc.
G06T7/33G01C21/1654G01C21/1656G02B27/0093G02B27/017G06F1/163G06F3/011G06F3/04817G06F11/00G06T5/20G06T11/00G06T19/006G02B2027/0138G02B2027/0141G02B2027/0181G06F3/01G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,769,262
App. No.
17/833,196
Granted
Sep 26, 2023
Kind
B2
Abstract

The described technology regards an augmented reality system and method for estimating a position of a location of interest relative to the position and orientation of a display, including receiving and selectively filtering a plurality of measurement vectors from a rate-gyroscope. Systems of the described technology include including a plurality of sensors, a processing module or other computation means, and a database. Methods of the described technology use data from the sensor package useful to accurately render graphical user interface information on a display.

Claims (51)

1. One or more non-transitory computer-readable media storing computer executable instructions that, when executed by a processor, perform a method of displaying a 360-degree situational awareness display in a head-mounted display, comprising steps of:

determining a pose azimuth based on updating a state-based estimation of an orientation of a user of the head-mounted display,

wherein the state-based estimation of the orientation of the head-mounted display is periodically updated based upon periodically obtained absolute and relative orientation sensor data at a regular processing interval;

displaying, on a display screen of the head-mounted display, at least a portion of the 360-degree situational awareness display and a plurality of situational awareness icons representing a plurality of points of interest,

wherein each situational awareness icon is positioned on the 360-degree situational awareness display in a position corresponding to a bearing of an associated point of interest of the plurality of points of interest relative to the pose azimuth of the head-mounted display; and

dynamically updating the position of at least a portion of the plurality of situational awareness icons on the 360-degree situational awareness display based on updated bearings to the plurality of points of interest as the plurality of points of interest move relative to the head-mounted display.

2. The media of claim 1 , wherein the 360-degree situational awareness display is one of a situational awareness ring and a situational awareness ellipse.

3. The media of claim 1 , wherein the head-mounted display is an optical see-through display.

4. The media of claim 1 , the method further comprising a step of updating at least one parameter of the 360-degree situational awareness display responsive to a toggle coupled to a motion sensing element of the head-mounted display.

5. The media of claim 1 , wherein at least a portion of the plurality of situational awareness icons are associated with team members of the user of the head-mounted display.

6. The media of claim 1 , the method further comprising a step of displaying at least one absolute compass heading on the 360-degree situational awareness display.

7. The method of claim 1 , further comprising a step of displaying a current position of the user with the 360-degree situational awareness display.

8. The media of claim 1 , wherein the state-based estimation of the orientation of a user of the head-mounted display is an Extended Kalman Filter-based estimation of the orientation of the user of the head-mounted display.

9. A method of providing situation awareness information via a head-mounted display, comprising steps of:

calculating a state-based estimation of an orientation of the head-mounted display,

wherein the state-based estimation of the orientation of the head-mounted display is periodically updated based upon periodically obtained absolute and relative orientation sensor data at a regular processing interval;

determining a plurality of positions associated with a plurality of points of interest;

determining a plurality of relative bearings associated with the plurality of points of interest based on the plurality of positions and the orientation of the head-mounted display;

identifying a plurality of displayable points of interest based of the plurality of points of interest and a field of view of the head-mounted display;

displaying, on a display screen of the head-mounted display, a plurality of geo-registered icons corresponding to the plurality of displayable points of interest, wherein each of the plurality of geo-registered icons is overlaid on an associate displayable point of interest of the plurality of displayable points of interest; and

dynamically updating positions of at least a portion of the plurality of geo-registered icons on the head-mounted display based on updated relative bearings to the plurality of displayable points of interest as the plurality of points of interest move relative to the orientation of the head-mounted display.

10. The method of claim 9 , further comprising steps of:

determining a pose azimuth of the head-mounted display based on the state-based estimation of an orientation of the head-mounted display; and

displaying, on the display screen of the head-mounted display, at least a portion of a 360-degree situational awareness display and a plurality of situational awareness icons representing the plurality of points of interest,

wherein each situational awareness icon is positioned on the 360-degree situational awareness display in a position corresponding to a bearing of an associated point of interest of the plurality of points of interest relative to the pose azimuth of the head-mounted display.

11. The method of claim 9 , wherein the plurality of points of interest includes a movement objective destination.

12. The method of claim 9 , wherein the state-based estimation of the orientation of the head-mounted display incorporates at least one of a forward buffer for predicted states and a rewind buffer for reprocessing late measurements.

13. The method of claim 9 , wherein at least a portion of the plurality of geo-registered icons use MIL-STD-2525C symbology.

14. The method of claim 9 , further comprising a step of displaying range information in association with at least a portion of the plurality of geo-registered icons.

15. A heads-up display system for a user, comprising:

a head-mounted display including a display screen positioned in line-of-sight of the user;

a processor;

at least one absolute orientation sensor;

at least one relative orientation sensor; and

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the processor to perform a method comprising steps of:

periodically receiving absolute orientation data from the at least one absolute orientation sensor;

periodically receiving relative orientation data from the at least one relative orientation sensor;

updating, at a regular processing interval, a state-based estimation of an orientation of the head-mounted display, based upon the absolute orientation data and relative orientation data;

determining an updated pose azimuth of the head-mounted display based on the state-based estimation of the orientation of the head-mounted display, determining a plurality of positions associated with a plurality of points of interest;

displaying, on the display screen, at least a portion of a 360-degree situational awareness display and a plurality of situational awareness icons representing the plurality of points of interest,

wherein each situational awareness icon is positioned on the 360-degree situational awareness display in a position corresponding to a bearing of an associated point of interest of the plurality of points of interest relative to the updated pose azimuth of the head-mounted display;

dynamically updating the position of at least a portion of the plurality of situational awareness icons on the 360-degree situational awareness display based on updated bearings to the plurality of points of interest as the plurality of points of interest move relative to the head-mounted display;

determining a plurality of relative bearings associated with the plurality of points of interest based on the plurality of positions and the orientation of the head-mounted display;

displaying, on a display screen of the head-mounted display, a plurality of geo-registered icons corresponding to at least a portion of the plurality of points of interest,

wherein each of the plurality of geo-registered icons is overlaid on an associated point of interest of the plurality of points of interest; and

dynamically updating positions of at least a portion of the plurality of geo-registered icons on the head-mounted display based on updated relative bearings to the plurality of points of interest as the plurality of points of interest move relative to the orientation of the head-mounted display.

16. The heads-up display system of claim 15 , wherein the plurality of points of interest includes at least one enemy unit of the user.

17. The heads-up display system of claim 15 , wherein a shape of at least one icon of the plurality of geo-registered icons or the plurality of situational awareness icons identifies an associated point of interest.

18. The heads-up display system of claim 15 , further comprising a night-vision camera, and wherein the display screen positioned in line-of-sight of the user is a video see-through display, and wherein the 360-degree situational awareness display, the plurality of situational awareness icons, and the plurality of geo-registered icons are superimposed on night-vision imagery.

19. The heads-up display system of claim 18 , wherein the night-vision camera, the head-mounted display, the processor, the processor; the at least one absolute orientation sensor, the at least one relative orientation sensor, and the one or more non-transitory computer-readable media are integrated into a helmet of the user.

20. The heads-up display system of claim 15 , wherein the method further comprises a step of displaying additional pertinent information in association with at least a portion of the plurality of geo-registered icons.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2022
From: MENOZZI, ALBERICO; ARGENTA, CHRIS F.; COOK, JAMES B.
To: APPLIED RESEARCH ASSOCIATES, INC.
Reel/Frame 060111/0112 →
Continuity (5)
Continuation 16380508 · Apr 10, 2019
Continuation 15704962 · Sep 14, 2017
Continuation 14832492 · Aug 21, 2015
Provisional Application 62040600 · Aug 22, 2014
Related Publication 20220383524A1 · Dec 1, 2022