IP Library Granted Patent US 9,507,147
Granted Patent B2
US 9,507,147 · App. 13/767,360 · Granted Nov 29, 2016

Wearable display system with detached projector

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,507,147
App. No.
13/767,360
Granted
Nov 29, 2016
Kind
B2
Abstract

A display system. A display device has viewports emitting expanded views of an image, an image receiving area receives the image, and optical conduits with exit pupil expanders expand the image through the viewports. A separate tracking projector has an image projector that projects an image along an axis, a marker search illuminator that projects light into a space including the axis, and an image capture component that captures target images that including at least three reflective marker images associated with the image receiving area. An image processor determines, based the marker images, a distance and an orientation of the image receiving area. A projection controller adjusts image data based upon the distance and the orientation to cause projected images to arrive squarely at the image receiving area.

Claims (58)

1. A target tracking projector, comprising:

a marker search illuminator configured to project illuminating light into a view;

an image capture component configured to capture target images of objects within the view, the target image comprising at least three reflective marker images each comprising a respective image of each reflection of the illuminating light by the at least three reflective markers;

an image processor configured to process the target images to determine, based upon locations of each of the at least three reflective marker images identified within the target image, a distance from a projection lens, at least one projection angle from the projection lens, and a pose angle of an image receiving area located within the view, the pose angle comprising a pitch angle, a yaw angle, and a roll angle relative to the projection lens,

wherein the image receiving area comprises a first image receiving area and a second image receiving area that is separate from the first image receiving area,

wherein the image processor is further configured to process the target images to determine, based upon locations of the at least three reflective marker images, at least

a first distance and a first orientation of the first image receiving area relative to the projection lens, and

a second distance and a second orientation of the second image receiving area relative to the projection lens; and

a projection controller configured to create a projected image by adjusting image data based upon the distance and the pose angle of the image receiving area so as to geometrically correct for the pose angle and cause the projected image to be squarely received at the image receiving area,

wherein the projection controller is further configured to create a first sub-image and a second sub-image within the projected image,

the first sub-image being projected to the first image receiving area and created by adjusting image data based upon the first distance and the first orientation of the first image receiving area so as to cause the projected image to arrive squarely at the first image receiving area,

the second sub-image being projected to the second image receiving area and created by adjusting image data based upon the second distance and the second orientation of the image receiving area so as to cause the projected image to arrive squarely at the second image receiving area; and

an image projector configured to project the projected image along the at least one projection angle to the image receiving area.

2. The image projector of claim 1 , wherein the image projector comprises the marker search illuminator and the image projector further emits the illuminating light.

3. The image projector of claim 1 , wherein the image projector emits the illuminating light as an infrared light into the view,

and wherein the image capture component is further configured to capture infrared light reflected by the at least three reflective marker images.

4. The image projector of claim 1 , wherein the target images comprise at least four reflective marker images, each reflective marker image corresponding to a respective reflective marker located at a respective corner of the image receiving area,

and wherein the projection controller is further configured to adjust the image data to create the projected image so as to extend substantially across an area with corners defined by the at least four reflective markers.

5. The image projector of claim 1 , wherein the projection controller is further configured to cease, based on at least one of the distance, the at least one projection angle, and pose angle the pose angle being in excess of a respective threshold, projection of the projected image.

6. The image projector of claim 5 , wherein the projection controller is further configured to progressively dim the projected image based on at least one of the distance, the at least one projection angle, and the pose angle being less than the respective threshold and within a defined respective range of the respective threshold.

7. The image projector of claim 1 , wherein:

the target image comprises two sets of reflective marker images, each set of reflective marker images comprising at least three respective reflective marker images each comprising a respective reflection of the illuminating light by a respective reflective marker,

a first set of reflective marker images within the two sets being associated with the first image receiving area,

a second set of reflective marker images with the two sets being associated with a second image receiving area that is separate from the image receiving area;

the image processor is further configured to:

determine the first distance and the first orientation based upon locations of the first set of reflective marker images; and

determine the second distance and the second orientation based upon locations of the second set of reflective marker images.

8. A method, comprising:

projecting illuminating light into a view of an image capture component;

capturing target images of objects within the view of the image capture component, the target image comprising at least three reflective marker images each comprising a respective image of each reflection of the illuminating light by the at least three reflective markers;

determining, based upon locations of each of the at least three reflective marker images identified within the target image, a distance from a projector, at least one projection angle from the projector, and a pose angle of an image receiving area, the pose angle comprising a pitch angle, a yaw angle, and a roll angle with respect to the projector,

wherein the image receiving area comprises a first image receiving area and a second image receiving area that is separate from the first image receiving area,

wherein the determining further comprises determining, based upon locations of the at least three reflective marker images, at least

a first distance and a first orientation of the first image receiving area relative to the projector, and

a second distance and a second orientation of the second image receiving area relative to the projector;

creating a projected image by adjusting image data based upon the distance and the pose angle of the image receiving area from the projector so as to geometrically correct for the pose angle and cause the projected image to be squarely received at the image receiving area,

wherein the creating further comprises creating a first sub-image and a second sub-image within the projected image,

the first sub-image being projected to the first image receiving area and created by adjusting image data based upon the first distance and the first orientation of the first image receiving area so as to cause the projected image to arrive squarely at the first image receiving area,

the second sub-image being projected to the second image receiving area and created by adjusting image data based upon the second distance and the second orientation of the image receiving area so as to cause the projected image to arrive squarely at the second image receiving area; and

projecting the projected image at the at least one projection angle.

9. The method of claim 8 , wherein the target images comprise at least four reflective marker images, each reflective marker image corresponding to a respective reflective marker located at a respective corner of the image receiving area,

and wherein the creating the projected image further comprises adjusting the image data to create the projected image so as to extend substantially across an area with corners defined by the at least four reflective markers.

10. The method of claim 8 , further comprising ceasing, based on at least one of the distance, the at least one projection angle, and the pose angle being in excess of a respective threshold, projection of the projected image.

11. The method of claim 10 , further comprising progressively dimming the projected image based on at least one of the distance, the at least one projection angle, and the pose angle being less than the respective threshold and within a defined respective range of the respective threshold.

12. A non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising instructions for:

projecting illuminating light into a view of an image capture component;

capturing target images of objects within the view of the image capture component, the target image comprising at least three reflective marker images each comprising a respective image of each reflection of the illuminating light by the at least three reflective markers;

determining, based upon locations of each of the at least three reflective marker images identified within the target image, a distance from a projector, at least one projection angle from the projector, and a pose angle of an image receiving area, the pose angle comprising a pitch angle, a yaw angle, and a roll angle with respect to the projector,

wherein the image receiving area comprises a first image receiving area and a second image receiving area that is separate from the first image receiving area,

wherein the determining further comprises determining, based upon locations of the at least three reflective marker images, at least

a first distance and a first orientation of the first image receiving area relative to the projector, and

a second distance and a second orientation of the second image receiving area relative to the projector;

creating a projected image by adjusting image data based upon the distance and the pose angle of the image receiving area so as to geometrically correct for the pose angle and cause the projected image to be squarely received at the image receiving area,

wherein the creating further comprises creating a first sub-image and a second sub-image within the projected image,

the first sub-image being projected to the first image receiving area and created by adjusting image data based upon the first distance and the first orientation of the first image receiving area so as to cause the projected image to arrive squarely at the first image receiving area, and

the second sub-image being projected to the second image receiving area and created by adjusting image data based upon the second distance and the second orientation of the image receiving area so as to cause the projected image to arrive squarely at the second image receiving area; and

projecting the projected image at the at least one projection angle.

13. The image projector of claim 1 , wherein the image capture component is located separately and remotely from the image projector.

Assignments (5)
NUNC PRO TUNC ASSIGNMENT Recorded Jun 19, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064271/0199 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 16, 2023
From: BLACKBERRY LIMITED
To: MALIKIE INNOVATIONS LIMITED
Reel/Frame 064104/0103 →
CHANGE OF NAME Recorded Mar 15, 2016
From: RESEARCH IN MOTION LIMITED
To: BLACKBERRY LIMITED
Reel/Frame 038087/0963 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 9, 2013
From: RESEARCH IN MOTION DEUTSCHLAND GMBH
To: RESEARCH IN MOTION LIMITED
Reel/Frame 030387/0505 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 4, 2013
From: THEIMER, WOLFGANG MICHAEL; MEIERLING, KLAUS-DIETER
To: RESEARCH IN MOTION DEUTSCHLAND GMBH
Reel/Frame 030148/0496 →