IP Library Granted Patent US 11,238,834
Granted Patent B2
US 11,238,834 · App. 16/738,324 · Granted Feb 1, 2022

Method, device and system for adjusting image, and computer readable storage medium

Inventor: Xiaofei Ding (Shenzhen, CN)
Assignee: SZ DJI TECHNOLOGY CO., LTD.
G09G5/377G09G5/38G09G2340/12G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,238,834
App. No.
16/738,324
Granted
Feb 1, 2022
Kind
B2
Abstract

A method for adjusting image includes determining, from a background image, a background sub-image corresponding to a foreground sub-image. The foreground sub-image is within a first range of a foreground image, the background image is an image of an external environment, and the foreground image is an image produced by a display device. The method also includes determining a first image attribute of the background sub-image; and adjusting the foreground sub-image based on at least the first image attribute.

Claims (69)

1. A method for adjusting image, comprising:

displaying, to an observer, a foreground image produced by a display device, the foreground image including a foreground sub-image within a first range of the foreground image;

determining a background image, the background image being an image of an external environment; and

after the foreground image is displayed to the observer:

monitoring a line of sight of the observer;

determining whether the line of sight intersects the foreground sub-image;

in response to determining that the line of sight intersects the foreground sub-image:

determining, from the background image, a background sub-image corresponding to the foreground sub-image;

determining a first image attribute of the background sub-image; and

adjusting the foreground sub-image based on at least the first image attribute; and

in response to determining that the line of sight does not intersect the foreground sub-image:

continuing to monitor the line of sight of the observer; and

controlling to not adjust the foreground sub-image based on the background image.

2. The method of claim 1 , wherein determining the background sub-image comprises:

determining an observation position of the observer; and

determining, based on the first range and the observation position, an image within a second range of the background image as the background sub-image.

3. The method of claim 2 , wherein determining the observation position comprises:

obtaining, by using a first image sensor, an observer image containing the observer; and

determining the observation position based on the observer image.

4. The method of claim 1 , wherein determining the line of sight of the observer comprises:

obtaining, by using a first image sensor, an image of an eye of the observer; and

determining the line of sight based on the image of the eye.

5. The method of claim 4 , wherein determining the line of sight of the observer comprises:

determining a pupil position of the eye of the observer according to the eye image; and

determining the line of sight corresponding to the pupil position.

6. The method of claim 5 , wherein determining the line of sight of the observer comprises:

determining the line of sight corresponding to the pupil position using a look-up table.

7. The method of claim 1 , wherein the background image is captured by a second image sensor.

8. The method of claim 1 , wherein:

the foreground image is at least one of: projected on a first transparent or translucent medium, or directly produced on a second transparent or translucent medium.

9. The method of claim 1 , wherein adjusting the foreground sub-image based on at least the first image attribute comprises:

upon determining that a difference between a value of a second image attribute of the foreground sub-image and a value of the first image attribute is not greater than a preset threshold, adjusting the value of the second image attribute, the difference between the adjusted value of the second image attribute and the value of the first image attribute being not less than the preset threshold.

10. The method of claim 1 , wherein adjusting the foreground sub-image based on at least the first image attribute comprises:

upon determining that a difference between a value of a preset image attribute of the foreground image and a value of the first image attribute is not greater than a preset threshold, adjusting a value of a second image attribute of the foreground sub-image, the difference between the adjusted value of the second image attribute and the value of the first image attribute being not less than the preset threshold.

11. The method of claim 1 , wherein adjusting the foreground sub-image based on at least the first image attribute comprises:

adjusting a position of the foreground sub-image in the foreground image based on at least the first image attribute.

12. The method of claim 1 , wherein at least a portion of the foreground image does not intersect the line of sight of the observer.

13. A device for adjusting image, comprising:

a processor;

a first image sensor and a second image sensor, each operably coupled to the processor; and

a memory configured to store instructions, the instructions, when being executed by the processor, cause the processor to:

control displaying a foreground image produced by a display device to an observer, the foreground image including a foreground sub-image within a first range of the foreground image;

determine a background image, the background image being an image of an external environment; and

after the foreground image is displayed to the observer:

monitor a line of sight of the observer;

determine whether the line of sight intersects the foreground sub-image;

in response to determining that the line of sight intersects the foreground sub-image:

determine, from the background image, a background sub-image corresponding to the foreground sub-image;

determine a first image attribute of the background sub-image;

adjust the foreground sub-image based on at least the first image attribute; and

in response to determining that the line of sight does not intersect the foreground sub-image:

continue to monitor the line of sight of the observer; and

control not to adjust the foreground sub-image based on the background image.

14. The device of claim 13 , wherein the instructions further cause the processor to:

determine an observation position of the observer; and

determine, based on the first range and the observation position, an image within a second range of the background image as the background sub-image.

15. The device of claim 14 , wherein the instructions further cause the processor to:

receive, from the first image sensor, an observer image containing the observer, the observer image being captured by the first image sensor; and

determine the observation position based on the observer image.

16. The device of claim 13 , wherein the instructions further cause the processor to:

receive, from the first image sensor, an image of an eye of the observer, the image of the eye being captured by the first image sensor; and

determine the line of sight based on the image of the eye.

17. The device of claim 13 , wherein the background image is captured by the second image sensor; and the instructions further cause the processor to receive the background image from the second image sensor.

18. The device of claim 13 , wherein:

the foreground image is at least one of: projected on a first transparent or translucent medium, or directly produced on a second transparent or translucent medium.

19. The device of claim 13 , wherein the instructions further cause the processor to:

upon determining that a difference between a value of a second image attribute of the foreground sub-image and a value of the first image attribute is not greater than a preset threshold, adjust the value of the second image attribute, the difference between the adjusted value of the second image attribute and the value of the first image attribute being not less than the preset threshold.

20. The device of claim 13 , wherein the instructions further cause the processor to:

upon determining that a difference between a value of a preset image attribute of the foreground image and a value of the first image attribute is not greater than a preset threshold, adjust a value of a second image attribute of the foreground sub-image, the difference between the adjusted value of the second image attribute and the value of the first image attribute being not less than the preset threshold.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 17, 2024
From: SZ DJI TECHNOLOGY CO., LTD.
To: SHENZHEN ZHUOYU TECHNOLOGY CO., LTD.
Reel/Frame 067452/0494 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 9, 2020
From: DING, XIAOFEI
To: SZ DJI TECHNOLOGY CO., LTD.
Reel/Frame 051466/0612 →
Continuity (2)
Continuation PCTCN2017116179 · Dec 14, 2017
Related Publication 20200152156A1 · May 14, 2020