IP Library › Granted Patent US 11,838,494
Granted Patent B2
US 11,838,494 · App. 17/782,385 · Granted Dec 5, 2023

Image processing method, VR device, terminal, display system, and non-transitory computer-readable storage medium

Inventors: Wenyu Li (Beijing, CN); Hao Zhang (Beijing, CN); Lili Chen (Beijing, CN); Jinghua Miao (Beijing, CN)
Assignees: Beijing Boe Optoelectronics Technology Co., LTD.; Boe Technology Group Co., LTD.
H04N13/344H04N13/122H04N13/25H04N13/383
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,838,494
App. No.
17/782,385
Granted
Dec 5, 2023
Kind
B2
Abstract

An image processing method includes: acquiring a fixation point position on a respective screen viewed by each of dominant eye(s); determining a fixation area of a left-eye screen and a fixation area of a right-eye screen according to fixation point position(s) corresponding to the dominant eye(s); rendering a first part of a left-eye image to be displayed on the left-eye screen at a first resolution, and rendering a second part of the left-eye image at a second resolution; rendering a first part of a right-eye image to be displayed on the right-eye screen at a third resolution, and rendering a second part of the right-eye image at a fourth resolution. A resolution of an image to be displayed in a fixation area of the respective screen is greater than resolutions of images to be displayed in other areas of the left-eye screen and the right-eye screen.

Claims (92)

1. An image processing method, comprising:

acquiring a fixation point position on a respective screen viewed by each of at least one dominant eye of a user, the at least one dominant eye including at least one of a left eye and a right eye of the user;

determining a fixation area of a left-eye screen and a fixation area of a right-eye screen according to at least one fixation point position corresponding to the at least one dominant eye;

rendering a first part of a left-eye image to be displayed on the left-eye screen at a first resolution, and rendering a second part of the left-eye image at a second resolution, the first part of the left-eye image being a part of the left-eye image to be displayed in the fixation area of the left-eye screen, the second part of the left-eye image being a part of the left-eye image other than the first part, and the first resolution being greater than the second resolution;

rendering a first part of a right-eye image to be displayed on the right-eye screen at a third resolution, and rendering a second part of the right-eye image at a fourth resolution, the first part of the right-eye image being a part of the right-eye image to be displayed in the fixation area of the right-eye screen, the second part of the right-eye image being a part of the right-eye image other than the first part, and the third resolution being greater than the fourth resolution; wherein

a resolution of an image to be displayed in a fixation area of the respective screen is greater than resolutions of images to be displayed in other areas of the left-eye screen and the right-eye screen;

wherein the at least one dominant eye includes one dominant eye, the dominant eye being the left eye of the user, the image processing method further comprises: reducing the first resolution according to a level of the dominant eye to obtain the third resolution, and reducing the second resolution according to the level of the dominant eye to obtain the fourth resolution; or

the at least one dominant eye includes one dominant eye, the dominant eye being the right eye of the user, the image processing method further comprises: reducing the third resolution according to a level of the dominant eye to obtain the first resolution, and reducing the fourth resolution according to the level of the dominant eye to obtain the second resolution.

2. The image processing method according to claim 1 , wherein the at least one dominant eye includes the left eye of the user; the third resolution is greater than the second resolution and less than the first resolution, and/or the fourth resolution is less than the second resolution; or

the at least one dominant eye includes the right eye of the user; the first resolution is greater than the fourth resolution and less than the third resolution, and/or the second resolution is less than the fourth resolution; or

the at least one dominant eye includes the left eye and the right eye of the user; the third resolution is equal to the first resolution, and the fourth resolution is equal to the second resolution.

3. The image processing method according to claim 1 , further comprising:

displaying the rendered left-eye image on the left-eye screen, and displaying the rendered right-eye image on the right-eye screen; or

sending the rendered left-eye image and the rendered right-eye image to a VR device to be displayed by the VR device.

4. The image processing method according to claim 1 , further comprising:

determining the at least one dominant eye of the user; and

sending a control instruction to a VR device according to the at least one determined dominant eye of the user, so as to control the VR device to acquire at least one eyeball image of the at least one dominant eye, the at least one eyeball image being used to determine the at least one fixation point position.

5. The image processing method according to claim 1 , further comprising:

displaying a dominant eye test image on the left-eye screen and the right-eye screen, respectively; or

sending the dominant eye test image to a VR device to be displayed by the VR device.

6. A terminal, comprising:

a memory configured to store a computer program;

at least one processor configured to execute the computer program stored in the memory when the terminal operates, so as to cause the terminal to perform the image processing method according to claim 1 .

7. The terminal according to claim 6 , further comprising:

the left-eye screen and the right-eye screen, wherein the at least one processor is further configured to control the left-eye screen to display the rendered left-eye image, and control the right-eye screen to display the rendered right-eye image; or

a first communication device configured to send the rendered left-eye image and the rendered right-eye image to a VR device to be displayed by the VR device.

8. The terminal according to claim 6 , wherein the at least one processor is further configured to determine the at least one dominant eye of the user; and

the terminal further comprises:

a second communication device configured to send a control instruction to a VR device according to the determined at least one dominant eye of the user, the control instruction being used to control the VR device to acquire at least one eyeball image of the at least one dominant eye, and the at least one eyeball image being used to determine the at least one fixation point position.

9. A non-transitory computer-readable storage medium storing therein a computer program, wherein

when run on a terminal, the computer program causes the terminal to perform the image processing method according to claim 1 .

10. An image processing method, comprising:

acquiring at least one eyeball image of at least one dominant eye of a user, the at least one dominant eye including at least one of a left eye and a right eye of the user; and

calculating a fixation point position on a respective screen viewed by each of the at least one dominant eye according to the at least one eyeball image;

sending at least one fixation point position corresponding to the at least one dominant eye to a terminal, the at least one fixation point position being used to determine a fixation area of a left-eye screen and a fixation area of a right-eye screen;

determining a fixation area of a left-eye screen and a fixation area of a right-eye screen according to at least one fixation point position corresponding to the at least one dominant eye;

rendering a first part of a left-eye image to be displayed on the left-eye screen at a first resolution, and rendering a second part of the left-eye image at a second resolution, the first part of the left-eye image being a part of the left-eye image to be displayed in the fixation area of the left-eye screen, the second part of the left-eye image being a part of the left-eye image other than the first part, and the first resolution being greater than the second resolution;

rendering a first part of a right-eye image to be displayed on the right-eye screen at a third resolution, and rendering a second part of the right-eye image at a fourth resolution, the first part of the right-eye image being a part of the right-eye image to be displayed in the fixation area of the right-eye screen, the second part of the right-eye image being a part of the right-eye image other than the first part, and the third resolution being greater than the fourth resolution, wherein a resolution of an image to be displayed in a fixation area of the respective screen is greater than resolutions of images to be displayed in other areas of the left-eye screen and the right-eye screen; and

displaying the rendered left-eye image on the left-eye screen, and displaying the rendered right-eye image on the right-eye screen;

wherein the at least one dominant eye includes one dominant eye, the dominant eye being the left eye of the user, the image processing method further comprises: reducing the first resolution according to a level of the dominant eye to obtain the third resolution, and reducing the second resolution according to the level of the dominant eye to obtain the fourth resolution; or

the at least one dominant eye includes one dominant eye, the dominant eye being the right eye of the user, the image processing method further comprises: reducing the third resolution according to a level of the dominant eye to obtain the first resolution, and reducing the fourth resolution according to the level of the dominant eye to obtain the second resolution.

11. The image processing method according to claim 10 , further comprising:

receiving the left-eye image to be displayed on the left-eye screen and the right-eye image to be displayed on the right-eye screen, wherein the left-eye image includes the first part and the second part, a resolution of the first part of the left-eye image is the first resolution, the first part of the left-eye image is used to be displayed in the fixation area of the left-eye screen, a resolution of the second part of the left-eye image is the second resolution, the second part of the left-eye image is a part of the left-eye image other than the first part, and the first resolution is greater than the second resolution; the right-eye image includes a first part and a second part, a resolution of the first part of the right-eye image is the third resolution, the first part of the right-eye image is used to be displayed in the fixation area of the right-eye screen, a resolution of the second part of the right-eye image is the fourth resolution, the second part of the right-eye image is a part of the right-eye image other than the first part, and the third resolution is greater than the fourth resolution; and

displaying the left-eye image on the left-eye screen, and displaying the right-eye image on the right-eye screen.

12. The image processing method according to claim 10 , further comprising:

acquiring the at least one eyeball image of the at least one dominant eye of the user includes:

receiving a control instruction sent by a terminal, and acquiring the at least one eyeball image of the at least one dominant eye of the user according to the control instruction; or

determining the at least one dominant eye of the user, and acquiring the at least one eyeball image of the at least one dominant eye of the user according to a determined result.

13. A VR device, the VR device being used to perform the image processing method according to claim 10 , the VR device comprising:

two cameras in one-to-one correspondence with the left eye and the right eye of the user, the cameras being configured to acquire eyeball images of eyes corresponding to the cameras; and

at least one processor configured to control at least one camera corresponding to the at least one dominant eye of the user to work, and calculate the fixation point position on the respective screen viewed by each of the at least one dominant eye according to at least one eyeball image acquired by the at least one camera, the at least one dominant eye including at least one of the left eye and the right eye of the user;

the VR device further comprising:

a communication device configured to send at least one fixation point position corresponding to the at least one dominant eye to a terminal, the at least one fixation point position being used to determine a fixation area of a left-eye screen and a fixation area of a right-eye screen; or

a left-eye screen and a right-eye screen, wherein

the at least one processor is further configured to:

determine a fixation area of the left-eye screen and a fixation area of the right-eye screen according to at least one fixation point position corresponding to the at least one dominant eye;

render a first part of a left-eye image to be displayed on the left-eye screen at a first resolution, and render a second part of the left-eye image at a second resolution, the first part of the left-eye image being a part of the left-eye image to be displayed in the fixation area of the left-eye screen, the second part of the left-eye image being a part of the left-eye image other than the first part, and the first resolution being greater than the second resolution;

render a first part of a right-eye image to be displayed on the right-eye screen at a third resolution, and render a second part of the right-eye image at a fourth resolution, the first part of the right-eye image being a part of the right-eye image to be displayed in the fixation area of the right-eye screen, the second part of the right-eye image being a part of the right-eye image other than the first part, the third resolution being greater than the fourth resolution, and a resolution of an image to be displayed in a fixation area of the respective screen being greater than resolutions of images to be displayed in other areas of the left-eye screen and the right-eye screen; and

control the left-eye screen to display the rendered left-eye image, and control the right-eye screen to display the rendered right-eye image;

wherein the at least one dominant eye includes one dominant eye, the dominant eye being the left eye of the user, the at least one processor is further configured to:

reduce the first resolution according to a level of the dominant eye to obtain the third resolution, and reducing the second resolution according to the level of the dominant eye to obtain the fourth resolution; or

wherein the at least one dominant eye includes one dominant eye, the dominant eye being the right eye of the user, the at least one processor is further configured to:

reduce the third resolution according to a level of the dominant eye to obtain the first resolution, and reducing the fourth resolution according to the level of the dominant eye to obtain the second resolution.

14. An image processing method, comprising:

determining at least one dominant eye of a user, the at least one dominant eye including at least one of a left eye and a right eye of the user;

acquiring at least one eyeball image of the at least one dominant eye of the user;

obtaining a fixation point position on a respective screen viewed by each of the at least one dominant eye of the user;

determining a fixation area of a left-eye screen and a fixation area of a right-eye screen according to at least one fixation point position corresponding to the at least one dominant eye;

rendering a first part of a left-eye image to be displayed on the left-eye screen at a first resolution, and rendering a second part of the left-eye image at a second resolution, the first part of the left-eye image being a part of the left-eye image to be displayed in the fixation area of the left-eye screen, the second part of the left-eye image being a part of the left-eye image other than the first part, and the first resolution being greater than the second resolution;

rendering a first part of a right-eye image to be displayed on the right-eye screen at a third resolution, and rendering a second part of the right-eye image at a fourth resolution, the first part of the right-eye image being a part of the right-eye image to be displayed in the fixation area of the right-eye screen, the second part of the right-eye image being a part of the right-eye image other than the first part, and the third resolution being greater than the fourth resolution, wherein a resolution of an image to be displayed in a fixation area of the respective screen is greater than resolutions of images to be displayed in other areas of the left-eye screen and the right-eye screen; and

displaying the rendered left-eye image on the left-eye screen, and displaying the rendered right-eye image on the right-eye screen;

wherein the at least one dominant eye includes one dominant eye, the dominant eye being the left eye of the user, and the image processing method further comprises: reducing the first resolution according to a level of the dominant eye to obtain the third resolution; and reducing the second resolution according to the level of the dominant eye to obtain the fourth resolution; or

the at least one dominant eye includes one dominant eye, the dominant eye being the right eye of the user, and the image processing method further comprises: reducing the third resolution according to a level of the dominant eye to obtain the first resolution; and reducing the fourth resolution according to the level of the dominant eye to obtain the second resolution.

15. The image processing method according to claim 14 , wherein

the at least one dominant eye includes the left eye of the user; the third resolution is greater than the second resolution and less than the first resolution, and/or the fourth resolution is less than the second resolution; or

the at least one dominant eye includes the right eye of the user; the first resolution is greater than the fourth resolution and less than the third resolution, and/or the second resolution is less than the fourth resolution; or

the at least one dominant eye includes the left eye and the right eye of the user; the third resolution is equal to the first resolution, and the fourth resolution is equal to the second resolution.

16. A display system, the display system being used to perform one or more steps of the image processing method according to claim 14 , the display system comprising:

two cameras in one-to-one correspondence with the left eye and the right eye of the user, the cameras being configured to acquire eyeball images of eyes corresponding to the cameras;

the left-eye screen and the right-eye screen; and

at least one processor configured to:

control at least one camera corresponding to the at least one dominant eye of the user, and calculate the fixation point position on the respective screen according to the at least one eyeball image;

determine the fixation area of the left-eye screen and the fixation area of the right-eye screen according to the at least one fixation point position;

render the left-eye image to be displayed on the left-eye screen according to the fixation area of the left-eye screen, and render the right-eye image to be displayed on the right-eye screen according to the fixation area of the right-eye screen; and

control the left-eye screen to display the rendered left-eye image, and control the right-eye screen to display the rendered right eye-image; wherein

the at least one processor is configured to render the left-eye image to be displayed on the left-eye screen according to the fixation area of the left-eye screen, and render the right-eye image to be displayed on the right-eye screen according to the fixation area of the right-eye screen, includes the at least one processor is configured to:

render the first part of the left-eye image to be displayed on the left-eye screen at the first resolution, and render the second part of the left-eye image at the second resolution, the first part of the left-eye image being the part of the left-eye image to be displayed in the fixation area of the left-eye screen, the second part of the left-eye image being the part of the left-eye image other than the first part, and the first resolution being greater than the second resolution; and

render the first part of the right-eye image to be displayed on the right-eye screen at the third resolution, and render the second part of the right-eye image at the fourth resolution, the first part of the right-eye image being the part of the right-eye image to be displayed in the fixation area of the right-eye screen, the second part of the right-eye image being the part of the right-eye image other than the first part, the third resolution being greater than the fourth resolution, and the resolution of the image to be displayed in the fixation area of the respective screen being greater than the resolutions of the images to be displayed in the other areas of the left-eye screen and the right-eye screen;

wherein the at least one dominant eye includes one dominant eye, the dominant eye being the left eye of the user, the at least one processor is further configured to:

reduce the first resolution according to a level of the dominant eye to obtain the third resolution, and reducing the second resolution according to the level of the dominant eye to obtain the fourth resolution; or

wherein the at least one dominant eye includes one dominant eye, the dominant eye being the right eye of the user, the at least one processor is further configured to:

reduce the third resolution according to a level of the dominant eye to obtain the first resolution, and reducing the fourth resolution according to the level of the dominant eye to obtain the second resolution.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 3, 2022
From: LI, WENYU; ZHANG, HAO; CHEN, LILI; MIAO, JINGHUA
To: BEIJING BOE OPTOELECTRONICS TECHNOLOGY CO., LTD.; BOE TECHNOLOGY GROUP CO., LTD.
Reel/Frame 060097/0517 →
Priority Claims (1)
CN 202010432109.8 · May 20, 2020 · national
Continuity (1)
Related Publication 20230023903A1 · Jan 26, 2023
Cited By (1)
US 12,355,842