IP Library Granted Patent US 11,099,396
Granted Patent B2
US 11,099,396 · App. 16/996,610 · Granted Aug 24, 2021

Depth map re-projection based on image and pose changes

Inventor: Christopher A. Peri (Mountain View, CA)
Assignee: SAMSUNG ELECTRONICS COMPANY, LTD.
G02B27/0172G02B27/0093G06T7/74G06T15/205G06T19/006G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,099,396
App. No.
16/996,610
Granted
Aug 24, 2021
Kind
B2
Abstract

A method implemented by an extended reality (XR) display device includes rendering a current image frame received from an external electronic device associated with the XR display device. The current image frame is associated with a current pose of the XR display device. The method further includes receiving an updated image frame from the external electronic device, calculating an updated pose based on one or more characteristics of the updated image frame, and determining whether the updated pose is within a pose range with respect to the current pose. The method thus further includes re-rendering, on one or more displays of the XR display device, a previous image frame based on whether the current pose is determined to be within the pose range.

Claims (62)

1. A method comprising, by an extended reality (XR) display device:

rendering a current image frame received from an external electronic device associated with the XR display device, wherein the current image frame is associated with a current pose of the XR display device;

receiving an updated image frame from the external electronic device;

calculating an updated pose based on one or more characteristics of the updated image frame;

determining whether the updated pose is within a pose range with respect to the current pose; and

re-rendering, on one or more displays of the XR display device, a previous image frame based on whether the current pose is determined to be within the pose range.

2. The method of claim 1 , wherein the one or more characteristics comprise one or more of an image content range, a depth density, an image content depth, or a depth continuity.

3. The method of claim 1 , further comprising:

prior to determining whether the updated pose is within the pose range with respect to the current pose, calculating the pose range with respect to the current pose, wherein the pose range is determined based on a target image quality.

4. The method of claim 1 , further comprising:

monitoring for a receipt of a second updated image frame from the external electronic device; and

receiving the second updated image frame from the external electronic device.

5. The method of claim 1 , further comprising, responsive to receiving a second updated image frame:

calculating a second updated pose based on one or more characteristics of the second updated image frame;

determining that the second updated pose is not within the pose range with respect to the current pose; and

determining whether one or more anchors corresponding to the second updated pose are stored on the XR display device.

6. The method of claim 5 , further comprising:

in response to determining that the one or more anchors corresponding to the second updated pose are stored on the XR display device, accessing image data or depth data corresponding to the one or more anchors.

7. The method of claim 6 , further comprising:

determining whether the second updated pose is within an updated pose range corresponding to the one or more anchors; and

rendering, on the one or more displays of the XR display device, the second updated image frame when the second updated pose is determined to be within the updated pose range.

8. The method of claim 1 , wherein re-rendering, on the one or more displays of the XR display device, the previous image frame comprises re-rendering the previous image frame at the current pose.

9. An extended reality (XR) display device comprising:

one or more displays;

one or more non-transitory computer-readable storage media including instructions; and

one or more processors coupled to the storage media, the one or more processors configured to execute the instructions to:

render a current image frame received from an external electronic device associated with the XR display device, wherein the current image frame is associated with a current pose of the XR display device;

receive an updated image frame from the external electronic device;

calculate an updated pose based on one or more characteristics of the updated image frame;

determine whether the updated pose is within a pose range with respect to the current pose; and

re-render, on one or more displays of the XR display device, a previous image frame based on whether the current pose is determined to be within the pose range.

10. The XR display device of claim 9 , wherein the one or more characteristics comprise one or more of an image content range, a depth density, an image content depth, or a depth continuity.

11. The XR display device of claim 9 , wherein the instructions further comprise instructions to:

prior to determining whether the updated pose is within the pose range with respect to the current pose, calculate the pose range with respect to the current pose, wherein the pose range is determined based on a target image quality.

12. The XR display device of claim 9 , wherein the instructions further comprise instructions to:

monitor for a receipt of a second updated image frame from the external electronic device; and

receive the second updated image frame from the external electronic device.

13. The XR display device of claim 9 , wherein, responsive to receiving a second updated image frame, the instructions further comprise instructions to:

calculate a second updated pose based on one or more characteristics of the second updated image frame;

determine that the second updated pose is not within the pose range with respect to the current pose; and

determine whether one or more anchors corresponding to the second updated pose are stored on the XR display device.

14. The XR display device of claim 13 , wherein the instructions further comprise instructions to:

in response to determining that the one or more anchors corresponding to the second updated pose are stored on the XR display device, access image data or depth data corresponding to the one or more anchors.

15. The XR display device of claim 14 , wherein the instructions further comprise instructions to:

determine whether the second updated pose is within an updated pose range corresponding to the one or more anchors; and

render, on the one or more displays of the XR display device, the second updated image frame when the second updated pose is determined to be within the updated pose range.

16. The XR display device of claim 9 , wherein the instructions to re-render, on the one or more displays of the XR display device, the previous image frame comprise instructions to re-rendering the previous image frame at the current pose.

17. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of an extended reality (XR) display device, cause the one or more processors to:

render a current image frame received from an external electronic device associated with the XR display device, wherein the current image frame is associated with a current pose of the XR display device;

receive an updated image frame from the external electronic device;

calculate an updated pose based on one or more characteristics of the updated image frame;

determine whether the updated pose is within a pose range with respect to the current pose; and

re-render, on one or more displays of the XR display device, a previous image frame based on whether the current pose is determined to be within the pose range.

18. The non-transitory computer-readable medium of claim 17 , wherein, responsive to receiving a second updated image frame, the instructions further comprise instructions to:

calculate a second updated pose based on one or more characteristics of the second updated image frame;

determine that the second updated pose is not within the pose range with respect to the current pose; and

determine whether one or more anchors corresponding to the second updated pose are stored on the XR display device.

19. The non-transitory computer-readable medium of claim 18 , wherein the instructions further comprise instructions to:

in response to determining that the one or more anchors corresponding to the second updated pose are stored on the XR display device, access image data or depth data corresponding to the one or more anchors.

20. The non-transitory computer-readable medium of claim 19 , wherein the instructions further comprise instructions to:

determine whether the second updated pose is within an updated pose range corresponding to the one or more anchors; and

render, on the one or more displays of the XR display device, the second updated image frame when the second updated pose is determined to be within the updated pose range.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2020
From: PERI, CHRISTOPHER A.
To: SAMSUNG ELECTRONICS COMPANY, LTD.
Reel/Frame 053529/0918 →
Continuity (2)
Provisional Application 63008592 · Apr 10, 2020
Related Publication 20200379262A1 · Dec 3, 2020
Cited By (1)
US 12,293,549