Granted Patent B2

US 12,487,472 · App. 18/832,943 · Granted Dec 2, 2025

Inventors: Xinkai Li (Beijing, CN); Lili Chen (Beijing, CN); Yaoyu Lv (Beijing, CN); Siyan Ma (Beijing, CN); Yan Li (Beijing, CN)

Assignee: Beijing BOE Technology Development Co., Ltd.

G02B30/31G06F3/013G06T7/60G06T7/74G06T7/75G06T17/00G09G3/003G09G3/3406G06T2207/10048G06T2207/30201G09G2354/00

Abstract

Claims (75)

1 . A driving method for a liquid crystal grating, comprising:

determining a real-time position of a center of a pupil;

determining a position of a light-transmitting region, that corresponds to the real-time position, in the liquid crystal grating according to the real-time position, and a pre-established correspondence between positions of the center of the pupil and positions of a light-transmitting region in the liquid crystal grating; and

driving the liquid crystal grating to cause that the liquid crystal grating only transmits light at the position of the light-transmitting region that corresponds to the real-time position;

wherein the determining a real-time position of a center of a pupil comprises: capturing a face image of a user in real time, and determining three dimensional coordinates of the center of the pupil under a camera coordinate system based on the face image;

wherein the capturing a face image of a user in real time, and determining three dimensional coordinates of the center of the pupil under a camera coordinate system based on the face image comprises:

obtaining the face image of the user in real time by using an infrared camera;

extracting a plurality of first edge points of an iris in the face image, performing ellipse fitting on the plurality of first edge points, and using two dimensional coordinates of an ellipse center obtained by fitting under an image coordinate system as two dimensional coordinates of the center of the pupil under the image coordinate system;

obtaining a plurality of face feature points in the face image, and mapping the plurality of face feature points to the same positions on a pre-established three dimensional face model;

adjusting a coordinate system of the three dimensional face model to coincide with the camera coordinate system;

using a mean value or a mode number of depth coordinates of a plurality of second edge points of human eyes in the three dimensional face model under the camera coordinate system as depth coordinates of the center of the pupil under the camera coordinate system; and

transforming the two dimensional coordinates of the center of the pupil under the image coordinate system to be two dimensional coordinates of the center of the pupil in the same dimension under the camera coordinate system, the two dimensional coordinates of the center of the pupil under the camera coordinate system and the depth coordinates of the center of the pupil under the camera coordinate system constituting the three dimensional coordinates of the center of the pupil under the camera coordinate system.

2 . The driving method according to claim 1 , wherein the capturing a face image of a user in real time, and determining three dimensional coordinates of the center of the pupil under a camera coordinate system based on the face image comprises:

capturing face images in real time respectively by using visible light cameras that are located in different positions;

respectively extracting a plurality of first edge points of an iris in the face images captured simultaneously by the visible light cameras, and performing matching on the first edge points at the same positions in the face images;

calculating three dimensional coordinates of successfully matched first edge points under the camera coordinate system by using a triangulation method; and

using the three dimensional coordinates of a center of a mapping space of the first edge points under the camera coordinate system as the three dimensional coordinates of the center of the pupil under the camera coordinate system.

3 . The driving method according to claim 2 , wherein establishing a three dimensional face model comprises:

capturing at least one face image by using the visible light cameras;

obtaining a plurality of face feature points of each face image;

calculating three dimensional coordinates of the plurality of face feature points under the camera coordinate system by using the triangulation method;

using one of the face feature points as an origin of a coordinate system of a to-be-established three dimensional face model, and adjusting the origin of the coordinate system of the to-be-established three dimensional face model to coincide with an origin of the camera coordinate system to cause that three dimensional coordinates of the plurality of face feature points under the camera coordinate system are transformed to be three dimensional coordinates under the coordinate system of the to-be-established three dimensional face model; and

restoring a stereoscopic face represented by the plurality of face feature points according to the three dimensional coordinates of the plurality of face feature points under the coordinate system of the to-be-established three dimensional face model to implement establishment of the three dimensional face model.

4 . The driving method according to claim 1 , wherein establishing correspondence between positions of the center of the pupil and positions of a light-transmitting region in the liquid crystal grating comprises:

establishing a coordinate system of the liquid crystal grating by using a center of the liquid crystal grating as an origin of coordinates, a light emergent direction of the liquid crystal grating as a Z-axis positive direction, and an X-axis negative direction of the camera coordinate system as an X-axis positive direction; and

determining light-transmitting regions in different positions in the liquid crystal grating corresponding to different positions of the center of the pupil under the coordinate system of the liquid crystal grating.

5 . The driving method according to claim 4 , wherein the determining a position of a light-transmitting region, that corresponds to the real-time position, in the liquid crystal grating according to the real-time position, and a pre-established correspondence between positions of the center of the pupil and positions of a light-transmitting region in the liquid crystal grating comprises:

using a negative value of a coordinate x of the center of the pupil under the camera coordinate system determined in real time as a coordinate x of the center of the pupil under the coordinate system of the liquid crystal grating, and a coordinate z of the center of the pupil under the camera coordinate system determined in real time as a coordinate z of the center of the pupil under the coordinate system of the liquid crystal grating; and

judging whether the coordinate z of the center of the pupil under the coordinate system of the liquid crystal grating is equal to a preset best viewing distance; if yes, moving the light-transmitting region in an X-axis direction of the coordinate system of the liquid crystal grating, a size of the light-transmitting region after moving being the same as a size of the light-transmitting region before moving; if not, adjusting a size of the light-transmitting region in the X-axis direction of the coordinate system of the liquid crystal grating, the light-transmitting region after adjusting overlapping partially with the light-transmitting region before adjusting.

6 . The driving method according to claim 5 , wherein the moving the light-transmitting region in an X-axis direction of the coordinate system of the liquid crystal grating comprises:

detecting coordinates of an intersection point between a connecting line extending line of a coordinate point of the center of the pupil and an origin under the coordinate system of the liquid crystal grating, and the liquid crystal grating; and

comparing a coordinate x of the coordinates of the intersection point with a same-side endpoint coordinate of each strip electrode on an X axis in the liquid crystal grating within one period, and determining a region where an i th to [(i+n/2)-1] th strip electrodes within each period is located as a light-transmitting region corresponding to a current center of the pupil in a case that the coordinate x of the coordinates of the intersection point is greater than an endpoint coordinate of an (i−1) th strip electrode within one period and less than or equal to an endpoint coordinate of an i th strip electrode, n being the total number of strip electrodes within one period and being an even number, and i being an integer greater than or equal to 2 and less than or equal to n/2.

7 . The driving method according to claim 5 , wherein the adjusting a size of the light-transmitting region in the X-axis direction of the coordinate system of the liquid crystal grating comprises:

calculating position coordinates of each strip electrode on an X axis within a total of m periods, a coordinate range on the X axis of a light transmittable region corresponding to each left eye pixel in the liquid crystal grating, and a coordinate range on the X axis of a light transmittable region corresponding to each right eye pixel in the liquid crystal grating; and

comparing position coordinates of a b th strip electrode within an a th period with the coordinate range of the light transmittable region corresponding to the left eye pixel and the coordinate range of the light transmittable region corresponding to the right eye pixel, and determining a region where the b th strip electrode within the a th period is located as a light-transmitting region corresponding to a current center of the pupil in the case that the position coordinates of the b th strip electrode within the a th period are located in both the coordinate range of the light transmittable region corresponding to the left eye pixel and the coordinate range of the light transmittable region corresponding to the right eye pixel, m being the total number of grating periods in the liquid crystal grating, a being an integer greater than or equal to 1 and less than or equal to m, and b being an integer greater than or equal to 1 and less than or equal to n.

8 . A display apparatus, comprising: a backlight source, a liquid crystal panel located on a light emergent side of the backlight source, and a liquid crystal grating located between the backlight source and the liquid crystal panel, the liquid crystal grating being driven by using the driving method according to claim 1 .

9 . A display method for the display apparatus according to claim 8 , comprising:

controlling a liquid crystal grating to transmit light completely in a two dimensional display mode; and

controlling the liquid crystal grating to form a light-transmitting region and a light-shading region that are distributed alternately by using a driving method in a three dimensional display mode, wherein the driving method comprises:

determining a real-time position of a center of a pupil;

determining a position of a light-transmitting region, that corresponds to the real-time position, in the liquid crystal grating according to the real-time position, and a pre-established correspondence between positions of the center of the pupil and positions of a light-transmitting region in the liquid crystal grating; and

driving the liquid crystal grating to cause that the liquid crystal grating only transmits light at the position of the light-transmitting region that corresponds to the real-time position;

wherein the determining a real-time position of a center of a pupil comprises: capturing a face image of a user in real time, and determining three dimensional coordinates of the center of the pupil under a camera coordinate system based on the face image;

wherein the capturing a face image of a user in real time, and determining three dimensional coordinates of the center of the pupil under a camera coordinate system based on the face image comprises:

obtaining the face image of the user in real time by using an infrared camera;

extracting a plurality of first edge points of an iris in the face image, performing ellipse fitting on the plurality of first edge points, and using two dimensional coordinates of an ellipse center obtained by fitting under an image coordinate system as two dimensional coordinates of the center of the pupil under the image coordinate system;

obtaining a plurality of face feature points in the face image, and mapping the plurality of face feature points to the same positions on a pre-established three dimensional face model;

adjusting a coordinate system of the three dimensional face model to coincide with the camera coordinate system;

using a mean value or a mode number of depth coordinates of a plurality of second edge points of human eyes in the three dimensional face model under the camera coordinate system as depth coordinates of the center of the pupil under the camera coordinate system; and

transforming the two dimensional coordinates of the center of the pupil under the image coordinate system to be two dimensional coordinates of the center of the pupil in the same dimension under the camera coordinate system, the two dimensional coordinates of the center of the pupil under the camera coordinate system and the depth coordinates of the center of the pupil under the camera coordinate system constituting the three dimensional coordinates of the center of the pupil under the camera coordinate system.

10 . The display method according to claim 9 , wherein while adjusting a size of the light-transmitting region in an X-axis direction of a coordinate system of the liquid crystal grating, the display method further comprises:

determining a total number of strip electrodes corresponding to the light-transmitting region, and adjusting a backlight brightness emitted by a backlight source based on the total number of strip electrodes, and the backlight brightness being in a negative correlation with the total number of strip electrodes.

11 . The display method according to claim 9 , wherein the capturing a face image of a user in real time, and determining three dimensional coordinates of the center of the pupil under a camera coordinate system based on the face image comprises:

capturing face images in real time respectively by using visible light cameras that are located in different positions;

respectively extracting a plurality of first edge points of an iris in the face images captured simultaneously by the visible light cameras, and performing matching on the first edge points at the same positions in the face images;

calculating three dimensional coordinates of successfully matched first edge points under the camera coordinate system by using a triangulation method; and

using the three dimensional coordinates of a center of a mapping space of the first edge points under the camera coordinate system as the three dimensional coordinates of the center of the pupil under the camera coordinate system.

12 . The display method according to claim 11 , wherein establishing a three dimensional face model comprises:

capturing at least one face image by using the visible light cameras;

obtaining a plurality of face feature points of each face image;

calculating three dimensional coordinates of the plurality of face feature points under the camera coordinate system by using the triangulation method;

using one of the face feature points as an origin of a coordinate system of a to-be-established three dimensional face model, and adjusting the origin of the coordinate system of the to-be-established three dimensional face model to coincide with an origin of the camera coordinate system to cause that three dimensional coordinates of the plurality of face feature points under the camera coordinate system are transformed to be three dimensional coordinates under the coordinate system of the to-be-established three dimensional face model; and

restoring a stereoscopic face represented by the plurality of face feature points according to the three dimensional coordinates of the plurality of face feature points under the coordinate system of the to-be-established three dimensional face model to implement establishment of the three dimensional face model.

13 . The display method according to claim 9 , wherein establishing correspondence between positions of the center of the pupil and positions of a light-transmitting region in the liquid crystal grating comprises:

establishing a coordinate system of the liquid crystal grating by using a center of the liquid crystal grating as an origin of coordinates, a light emergent direction of the liquid crystal grating as a Z-axis positive direction, and an X-axis negative direction of the camera coordinate system as an X-axis positive direction; and

determining light-transmitting regions in different positions in the liquid crystal grating corresponding to different positions of the center of the pupil under the coordinate system of the liquid crystal grating.

14 . The display method according to claim 13 , wherein the determining a position of a light-transmitting region, that corresponds to the real-time position, in the liquid crystal grating according to the real-time position, and a pre-established correspondence between positions of the center of the pupil and positions of a light-transmitting region in the liquid crystal grating comprises:

using a negative value of a coordinate x of the center of the pupil under the camera coordinate system determined in real time as a coordinate x of the center of the pupil under the coordinate system of the liquid crystal grating, and a coordinate z of the center of the pupil under the camera coordinate system determined in real time as a coordinate z of the center of the pupil under the coordinate system of the liquid crystal grating; and

judging whether the coordinate z of the center of the pupil under the coordinate system of the liquid crystal grating is equal to a preset best viewing distance; if yes, moving the light-transmitting region in an X-axis direction of the coordinate system of the liquid crystal grating, a size of the light-transmitting region after moving being the same as a size of the light-transmitting region before moving; if not, adjusting a size of the light-transmitting region in the X-axis direction of the coordinate system of the liquid crystal grating, the light-transmitting region after adjusting overlapping partially with the light-transmitting region before adjusting.

15 . The display method according to claim 14 , wherein the moving the light-transmitting region in an X-axis direction of the coordinate system of the liquid crystal grating comprises:

detecting coordinates of an intersection point between a connecting line extending line of a coordinate point of the center of the pupil and an origin under the coordinate system of the liquid crystal grating, and the liquid crystal grating; and

comparing a coordinate x of the coordinates of the intersection point with a same-side endpoint coordinate of each strip electrode on an X axis in the liquid crystal grating within one period, and determining a region where an i th to [(i+n/2)−1] th strip electrodes within each period is located as a light-transmitting region corresponding to a current center of the pupil in a case that the coordinate x of the coordinates of the intersection point is greater than an endpoint coordinate of an (i−1) th strip electrode within one period and less than or equal to an endpoint coordinate of an i th strip electrode, n being the total number of strip electrodes within one period and being an even number, and i being an integer greater than or equal to 2 and less than or equal to n/2.

16 . The display method according to claim 14 , wherein the adjusting a size of the light-transmitting region in the X-axis direction of the coordinate system of the liquid crystal grating comprises:

calculating position coordinates of each strip electrode on an X axis within a total of m periods, a coordinate range on the X axis of a light transmittable region corresponding to each left eye pixel in the liquid crystal grating, and a coordinate range on the X axis of a light transmittable region corresponding to each right eye pixel in the liquid crystal grating; and

comparing position coordinates of a b th strip electrode within an a th period with the coordinate range of the light transmittable region corresponding to the left eye pixel and the coordinate range of the light transmittable region corresponding to the right eye pixel, and determining a region where the b th strip electrode within the a th period is located as a light-transmitting region corresponding to a current center of the pupil in the case that the position coordinates of the b th strip electrode within the a th period are located in both the coordinate range of the light transmittable region corresponding to the left eye pixel and the coordinate range of the light transmittable region corresponding to the right eye pixel, m being the total number of grating periods in the liquid crystal grating, a being an integer greater than or equal to 1 and less than or equal to m, and b being an integer greater than or equal to 1 and less than or equal to n.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (28)