IP Library › Granted Patent US 11,393,434
Granted Patent B2
US 11,393,434 · App. 17/106,188 · Granted Jul 19, 2022

Method, processing device, and display system for information display

Inventors: Yu-Hsuan Su (Kaohsiung, TW); Tzu-Yi Yu (Kaohsiung, TW); Yu-Hsiang Tsai (Hsinchu County, TW); Kuan-Ting Chen (Yunlin County, TW)
Assignee: Industrial Technology Research Institute
G09G5/38G01S13/06G01S19/01G02B27/0101G06T7/50G06T7/70A61B2034/2051A61B2090/372G02B2027/014G02B2027/0138G02B2027/0141G06T2207/30196G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,393,434
App. No.
17/106,188
Granted
Jul 19, 2022
Kind
B2
Abstract

A method, a processing device and a display system for information display are provided. The system includes a display being light transmissive, at least one first information extraction device, at least one second information extraction device, and a processing device. The processing device is connected to the display, the first information extraction device and the second information extraction device. The first information extraction device is configured to extract a user's position information and posture information. The second information extraction device is configured to extract a target object's position information and posture information. The processing device is configured to perform coordinate transformation according to the user's position information and posture information and the target object's position information and posture information, to generate overlay information between the user and the target object, and display related stereoscopic information of the target object on the display according to the overlay information.

Claims (67)

1. A display system, comprising:

a display being light transmissive;

at least one first information extraction device, configured to extract position information and posture information of a user;

at least one second information extraction device, configured to extract position information and posture information of a target object; and

a processing device, connected to the display, the first information extraction device and the second information extraction device, and configured to:

perform coordinate transformation according to the position information and the posture information of the user as well as the position information and the posture information of the target object, to generate overlay information between the user and the target object; and

display related stereoscopic information of the target object on the display according to the overlay information, wherein the overlay information comprising a display angle of the related stereoscopic information on the display which is associated with a depth distance of the user relative to the display, a depth distance of the target object relative to the display, an offset angle of the user relative to a depth axis of the display, and an offset angle of the target object relative to the depth axis of the display.

2. The display system according to claim 1 , wherein:

the position information and the posture information of the user comprise a user position coordinate and a user orientation angle of the user relative to the first information extraction device, the position information and the posture information of the target object comprise a target object position coordinate and a target object orientation angle of the target object relative to the second information extraction device; and

the processing device transforms the user position coordinate, the user orientation angle, the target object position coordinate and the target object orientation angle to a same coordinate system.

3. The display system according to claim 2 , wherein:

the processing device calculates positions and angles of the user and the target object relative to the display according to the user position coordinate after transformation, the user orientation angle after transformation, the target object position coordinate after transformation and the target object orientation angle after transformation, to generate the overlay information.

4. The display system according to claim 1 , wherein:

the overlay information further comprises a display position and a display size of the related stereoscopic information on the display when the user views the target object.

5. The display system according to claim 4 , wherein:

the display position of the related stereoscopic information on the display is associated with a depth distance of the user relative to the display, a depth distance of the target object relative to the display, an offset angle of the user relative to a depth axis of the display, and an offset angle of the target object relative to the depth axis of the display.

6. The display system according to claim 4 , wherein:

the display size of the related stereoscopic information on the display is associated with a relative distance between the user and the target object.

7. The display system according to claim 1 , wherein:

the first information extraction device comprises at least one image sensor and a first optical positioner, and the second information extraction device comprises a second optical positioner;

the processing device acquires plane position information of the user by the image sensor and acquires depth position information of the user by the first optical positioner;

the processing device acquires the position information and the posture information of the target object by the second optical positioner; and

the processing device performs coordinate transformation according to the plane position information of the user, the depth position information of the user, the position information of the target object and the posture information of the target object, to generate the overlay information.

8. The display system according to claim 7 , wherein:

the user and the target object respectively have thereon a first positioning light ball and a second positioning light ball, wherein the processing device acquires the depth position information of the user by the first positioning light ball through the first optical positioner, and the second optical positioner acquires the position information and the posture information of the target object according to the second positioning light ball.

9. The display system according to claim 1 , wherein:

the first information extraction device comprises at least one image sensor, and the second information extraction device comprises an optical positioner;

the processing device acquires the position information and the posture information of the user by the image sensor;

the processing device acquires the position information and the posture information of the target object by the optical positioner; and

the processing device performs coordinate transformation according to the position information and the posture information of the user as well as the position information and the posture information of the target object, to generate the overlay information.

10. The display system according to claim 9 , wherein:

the target object has thereon a positioning light ball, wherein the processing device acquires the position information and the posture information of the target object by the positioning light ball through the optical positioner.

11. The display system according to claim 1 , wherein:

the first information extraction device comprises at least one image sensor, and the second information extraction device comprises a magnetic positioner;

the processing device acquires the position information and the posture information of the user by the image sensor;

the processing device acquires the position information and the posture information of the target object by the magnetic positioner; and

the processing device performs coordinate transformation according to the position information and the posture information of the user as well as the position information and the posture information of the target object, to generate the overlay information.

12. The display system according to claim 11 , wherein:

the magnetic positioner comprises an electromagnetic navigation probe and an electromagnetic navigation spatial positioning system, wherein the processing device acquires the position information and the posture information of the target object by the electromagnetic navigation probe through the electromagnetic navigation spatial positioning system.

13. The display system according to claim 12 , wherein:

the electromagnetic navigation probe is able to enter the target object.

14. The display system according to claim 1 , wherein:

the first information extraction device comprises at least one first image sensor, and the second information extraction device comprises at least one second image sensor;

the processing device acquires the position information and the posture information of the user by the first image sensor;

the processing device acquires the position information and the posture information of the target object by the second image sensor; and

the processing device performs coordinate transformation according to the position information and the posture information of the user as well as the position information and the posture information of the target object, to generate the overlay information.

15. The display system according to claim 14 , wherein:

the target object further comprises thereon a predetermined pattern; and

the processing device calculates the position information and the posture information of the target object according to the predetermined pattern extracted by the second image sensor.

16. The display system according to claim 1 , wherein:

the second information extraction device comprises a GPS positioner configured to position the target object; and

the processing device acquires the position information and the posture information of the target object by the GPS positioner.

17. The display system according to claim 1 , wherein:

the second information extraction device comprises a radar rangefinder configured to position the target object; and

the processing device acquires the position information and the posture information of the target object by the radar rangefinder.

18. A method for information display, wherein the method is adapted for a display system comprising a display being light transmissive, at least one first information extraction device, at least one second information extraction device and a processing device, wherein the method comprises:

extracting position information and posture information of a user by the first information extraction device;

extracting position information and posture information of a target object by the second information extraction device;

performing coordinate transformation according to the position information and the posture information of the user as well as the position information and the posture information of the target object, to generate overlay information between the user and the target object; and

displaying related stereoscopic information of the target object on the display according to the overlay information, wherein the overlay information comprising a display angle of the related stereoscopic information on the display which is associated with a depth distance of the user relative to the display, a depth distance of the target object relative to the display, an offset angle of the user relative to a depth axis of the display, and an offset angle of the target object relative to the depth axis of the display.

19. A processing device, wherein the processing device is connected to a display being light transmissive, at least one first information extraction device and at least one second information extraction device, wherein the first information extraction device is configured to extract position information and posture information of a user, the second information extraction device is configured to extract position information and posture information of a target object, and the processing device comprises:

a memory, configured to store data; and

a processor, configured to:

acquire the position information and the posture information of the user that are extracted by the first information extraction device;

acquire the position information and the posture information of the target object that are extracted by the second information extraction device;

perform coordinate transformation according to the position information and the posture information of the user as well as the position information and the posture information of the target object, to generate overlay information between the user and the target object; and

display related stereoscopic information of the target object on the display according to the overlay information, wherein the overlay information comprising a display angle of the related stereoscopic information on the display which is associated with a depth distance of the user relative to the display, a depth distance of the target object relative to the display, an offset angle of the user relative to a depth axis of the display, and an offset angle of the target object relative to the depth axis of the display.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 5, 2021
From: SU, YU-HSUAN; YU, TZU-YI; TSAI, YU-HSIANG; CHEN, KUAN-TING
To: INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE
Reel/Frame 055501/0939 →
Continuity (2)
Provisional Application 63050072 · Jul 9, 2020
Related Publication 20220013090A1 · Jan 13, 2022