IP Library Granted Patent US 10,997,745
Granted Patent B2
US 10,997,745 · App. 16/713,540 · Granted May 4, 2021

Augmented reality based method and apparatus for spatial positioning

Inventors: Pingping Liu (Shanghai, CN); Yuantong Zhang (Shanghai, CN); Zhenghua Wei (Shanghai, CN)
Assignee: HUAQIN TECHNOLOGY CO., LTD.
G06T7/73G06T15/10G06T19/006G06T2215/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,997,745
App. No.
16/713,540
Granted
May 4, 2021
Kind
B2
Abstract

Embodiments of the present disclosure relate to the field of spatial positioning, and disclose a method and apparatus for spatial positioning based on augmented reality. In the embodiments of the present disclosure, the augmented reality based method for spatial positioning including: acquiring an offset angle of a dual-camera coordinate system relative to a world coordinate system, wherein the world coordinate system is a coordinate system that is preset by using a known target position of a positioning photosphere as a reference coordinate; acquiring an actual position of the positioning photosphere in the world coordinate system; calculating a coordinate value of the dual camera in the world coordinate system in accordance with the offset angle, the actual position of the positioning photosphere, and the target position; and determining a position of a virtual object in a virtual space in the world coordinate system in accordance with the coordinate value of the dual camera in the world coordinate system. Embodiments of the present disclosure further disclose the apparatus for spatial positioning based on augmented reality.

Claims (125)

1. An augmented reality based method for spatial positioning, comprising:

acquiring an offset angle of a dual-camera coordinate system relative to a world coordinate system, wherein the world coordinate system is a coordinate system that is preset by using a known target position of a positioning photosphere as a reference coordinate;

acquiring an actual position of the positioning photosphere in the world coordinate system;

calculating a coordinate value of the dual camera in the world coordinate system in accordance with the offset angle, the actual position of the positioning photosphere, and the target position; and

determining a position of a virtual object in a virtual space in the world coordinate system in accordance with the coordinate value of the dual camera in the world coordinate system;

wherein the step of calculating of a coordinate value of the dual camera in the world coordinate system in accordance with the offset angle, the actual position of the positioning photosphere and the target position, comprises:

establishing a geometric positional relationship in accordance with the offset angle, the actual position of the positioning photosphere and the target position;

establishing a mathematical equation set in accordance with the geometric positional relationship, wherein a coordinate value of the dual camera on each coordinate axis of the world coordinate system is an unknown variable of the mathematical equation set; and

solving the mathematical equation set to acquire the coordinate value of the dual camera on each coordinate axis of the world coordinate system.

2. The augmented reality based method in accordance with claim 1 , wherein the step of establishing the geometric positional relationship in accordance with the offset angle, the actual position of the positioning photosphere and the target position, comprises:

making a projection point B of a target position S on a plane XY of the world coordinate system, making a point A, wherein a line passing through the point A and an actual position C1 of the positioning photosphere is parallel to the Y-axis of a dual-camera coordinate system and the point A is separated from the C1 by a length Y1, making an intersection point N of the CIA and the plane XY of the world coordinate system, wherein the actual position C1 of the positioning photosphere is the offset coordinate (X1, Y1, Z1) in the world coordinate system relative to the target position S;

making an intersection point P of the Y axis in the dual-camera coordinate system and the plane XY of the world coordinate system;

making a perpendicular of the plane XY of the world coordinate system through the position C of the dual camera to have an intersection point H with the plane XY of the world coordinate system;

making an intersection point T by intersecting a line that passes through the S and is parallel to a line NB with a line C1N, wherein a line ST intersects with a preset plane at an intersection point E, and the preset plane is a plane that passes through a line CH and is parallel to a plane YZ of the world coordinate system; and

making the projection G of the S on the preset plane, making the perpendicular of the CH to pass through the C1, and to intersect with the CH at J, making the plane XY of the world coordinate system through the S to intersect with the line CH at C2, making the perpendicular of the line SE to pass through the C2 and to intersect with the line SE at F.

3. The augmented reality based method for spatial positioning based on augmented reality in accordance with claim 2 , wherein the step of establishing a mathematical equation set in accordance with the geometric positional relationship, comprises:

formulating a first equation set in accordance with the geometric figure projection of the geometric positional relationship in the positive direction of the Z-axis of the dual-camera coordinate system:

QB=Y 0*Sin γ+ X 1;

NA =( Y 0*Sin γ+ X 1)*Tan γ+ Y 0*Cos γ,

wherein Y0 is a reference coordinate value of the positioning photosphere, and an angle γ is a rotation angle of the dual-camera coordinate system around the Z-axis relative to the world coordinate system;

formulating a second equation set in accordance with the geometric figure projection of the geometric positional relationship in the negative direction of the X-axis of the dual-camera coordinate system:

N C 1=( Y 0*Cos γ+( X 1+ Y 0*Sin γ)*Tan γ)/Cos α+ Y 1;

CP=Z 1*Tan α+( Y 0*Cos γ+( X 1+ Y 0*Sin γ)*Tan γ)/Cos α+ Y 1;

wherein the α is a rotation angle of the dual-camera coordinate system around the X-axis relative to the world coordinate system;

formulating a third equation set in accordance with the geometric figure projection of the geometric positional relationship in the negative direction of the Z-axis of the world coordinate system:

CH =( Z 1*Tan α+( Y 0*Cos γ+( X 1+ Y 0*Sin γ)*Tan γ)/Cos α+ Y 1)*Cos γ;

solving a coordinate value CH of the dual camera on the Y-axis of the world coordinate system in accordance with the first equation set, the second equation set and the third equation set, when solving the mathematical equation set to acquire coordinate values of the dual camera on each coordinate axis of the world coordinate system;

formulating a fourth equation set in accordance with the geometric figure projection of the geometric positional relationship in the positive direction of the Z-axis of the dual-camera coordinate system:

AT=X 1*Tan γ;

TF =( Y 1+ X 1*Tan γ)*Sin γ;

SF:=X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ;

formulating a fifth equation set in accordance with the geometric figure projection of the geometric positional relationship in the negative direction of the X-axis of the world coordinate system:

C 2 F=Z 1*Cos α;

formulating a sixth equation set in accordance with the geometric figure projection of the geometric positional relationship in the positive direction of the Y-axis of the dual-camera coordinate system:

FE=Z 1*Cos α*Tan β;

KF =( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ)*Tan β;

SE=X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ+ Z 1*Cos α*Tan β;

SG =( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ+ Z 1*Cos α*Tan β)*Cos β;

C 2 K=Z 1*Cos α−( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ)*Tan β;

C 2 G =( Z 1*Cos α−( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ)*Tan β)*Cos β; and

solving the coordinate value SG of the dual camera in the X-axis and the coordinate value C2G of the dual camera in the Z-axis of the world coordinate system in accordance with the fourth equation set, the fifth equation set and the sixth equation set, when solving the mathematical equation set to acquire coordinate values of the dual camera on each coordinate axis of the world coordinate system.

4. An electronic device, comprising:

at least one processor; and,

a memory communicatively connected to the at least one processor, wherein the memory stores instructions which, when executed by the at least one processor, cause the at least one processor to:

acquire an offset angle of a dual-camera coordinate system relative to a world coordinate system, wherein the world coordinate system is a coordinate system that is preset by using a known target position of a positioning photosphere as a reference coordinate;

acquire an actual position of the positioning photosphere in the world coordinate system;

calculate a coordinate value of the dual camera in the world coordinate system in accordance with the offset angle, the actual position of the positioning photosphere and the target position; and

determine a position of a virtual object in a virtual space in the world coordinate system in accordance with the coordinate value of the dual camera in the world coordinate system;

wherein to calculate a coordinate value of the dual camera in the world coordinate system in accordance with the offset angle, the actual position of the positioning photosphere and the target position, the instructions cause the at least one processor to:

establish a geometric positional relationship in accordance with the offset angle, the actual position of the positioning photosphere and the target position;

establish a mathematical equation set in accordance with the geometric positional relationship, wherein a coordinate value of the dual camera on each coordinate axis of the world coordinate system is an unknown variable of the mathematical equation set; and

solve the mathematical equation set to acquire the coordinate value of the dual camera on each coordinate axis of the world coordinate system.

5. The electronic device in accordance with claim 1 , wherein to establish the geometric positional relationship in accordance with the offset angle, the actual position of the positioning photosphere and the target position, the instructions cause the at least one processor to:

make a projection point B of a target position S on a plane XY of the world coordinate system, make a point A, wherein a line passing through the point A and an actual position C1 of the positioning photosphere is parallel to a Y-axis of a dual-camera coordinate system and the point A is separated from the C1 by a length Y1, make an intersection point N of the C1A and the plane XY of the world coordinate system, wherein the actual position C1 of the positioning photosphere is the offset coordinate (X1, Y1, Z1) in the world coordinate system relative to the target position S;

make an intersection point P of the Y axis in the dual-camera coordinate system and the plane XY of the world coordinate system;

make a perpendicular of the plane XY of the world coordinate system through the position C of the dual camera to have an intersection point H with the plane XY of the world coordinate system;

make an intersection point T by intersecting a line that passes through the S and is parallel to a line NB with a line C1N, wherein a line ST intersects with a preset plane at an intersection point E, and the preset plane is a plane that passes through a line CH and is parallel to a plane YZ of the world coordinate system; and

make the projection G of the S on the preset plane, making the perpendicular of the CH to pass through the C1, and to intersect with the CH at J, make the plane XY of the world coordinate system through the S to intersect with the line CH at C2, make the perpendicular of the line SE to pass through the C2 and to intersect with the line SE at F.

6. The electronic device in accordance with claim 5 , wherein to establish a mathematical equation set in accordance with the geometric positional relationship, the instructions cause the at least one processor to:

formulate a first equation set in accordance with the geometric figure projection of the geometric positional relationship in the positive direction of the Z-axis of the dual-camera coordinate system:

QB=Y 0*Sin γ+ X 1;

NA =( Y 0*Sin γ+ X 1)*Tan γ+ Y 0*Cos γ,

wherein Y0 is a reference coordinate value of the positioning photosphere, and an angle γ is a rotation angle of the dual-camera coordinate system around the Z-axis relative to the world coordinate system;

formulate a second equation set in accordance with the geometric figure projection of the geometric positional relationship in the negative direction of the X-axis of the dual-camera coordinate system:

N C 1=( Y 0*Cos γ+( X 1+ Y 0*Sin γ)*Tan γ)/Cos α+ Y;

CP=Z 1*Tan α+( Y 0*Cos γ+( X 1+ Y 0*Sin γ)*Tan γ)/Cos α+ Y 1;

wherein the α is a rotation angle of the dual-camera coordinate system around the X-axis relative to the world coordinate system;

formulate a third equation set in accordance with the geometric figure projection of the geometric positional relationship in the negative direction of the Z-axis of the world coordinate system:

CH =( Z 1*Tan α+( Y 0*Cos γ+( X 1+ Y 0*Sin γ)*Tan γ)/Cos α+ Y 1)*Cos γ;

solve a coordinate value CH of the dual camera on the Y-axis of the world coordinate system in accordance with the first equation set, the second equation set and the third equation set, when solving the mathematical equation set to acquire coordinate values of the dual camera on each coordinate axis of the world coordinate system;

formulate a fourth equation set in accordance with the geometric figure projection of the geometric positional relationship in the positive direction of the Z-axis of the dual-camera coordinate system:

AT=X 1*Tan γ;

TF =( Y 1+ X 1*Tan γ)*Sin γ;

SF=X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ;

formulate a fifth equation set in accordance with the geometric figure projection of the geometric positional relationship in the negative direction of the X-axis of the world coordinate system:

C 2 F=Z 1*Cos α;

formulate a sixth equation set in accordance with the geometric figure projection of the geometric positional relationship in the positive direction of the Y-axis of the dual-camera coordinate system:

FE=Z 1*Cos α*Tan β;

KF =( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ)*Tan β;

SE=X 1/Cos γ−( Y 1+ X *Tan γ)*Sin γ+ Z 1*Cos α*Tan β;

SG =( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ+ Z 1*Cos α*Tan β)*Cos β;

C 2 K=Z 1*Cos α−( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ)*Tan β;

C 2 G =( Z 1*Cos α−( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ)*Tan β)*Cos β; and

solve the coordinate value SG of the dual camera in the X-axis and the coordinate value C2G of the dual camera in the Z-axis of the world coordinate system in accordance with the fourth equation set, the fifth equation set and the sixth equation set, when solving the mathematical equation set to acquire coordinate values of the dual camera on each coordinate axis of the world coordinate system.

7. A non-transitory computer-readable storage medium having machine-readable instructions that, when executed by at least one processor, cause the at least one processor to:

acquire an offset angle of a dual-camera coordinate system relative to a world coordinate system, wherein the world coordinate system is a coordinate system that is preset by using a known target position of a positioning photosphere as a reference coordinate;

acquire an actual position of the positioning photosphere in the world coordinate system;

calculate a coordinate value of the dual camera in the world coordinate system in accordance with the offset angle, the actual position of the positioning photosphere and the target position; and

determine a position of a virtual object in a virtual space in the world coordinate system in accordance with the coordinate value of the dual camera in the world coordinate system;

wherein to calculate a coordinate value of the dual camera in the world coordinate system in accordance with the offset angle, the actual position of the positioning photosphere and the target position, the instructions cause the at least one processor to:

establish a geometric positional relationship in accordance with the offset angle, the actual position of the positioning photosphere and the target position;

establish a mathematical equation set in accordance with the geometric positional relationship, wherein a coordinate value of the dual camera on each coordinate axis of the world coordinate system is an unknown variable of the mathematical equation set; and

solve the mathematical equation set to acquire the coordinate value of the dual camera on each coordinate axis of the world coordinate system.

8. The non-transitory computer-readable storage medium in accordance with claim 7 , wherein to establish the geometric positional relationship in accordance with the offset angle, the actual position of the positioning photosphere and the target position, the instructions cause the at least one processor to:

make a projection point B of a target position S on a plane XY of the world coordinate system, make a point A, wherein a line passing through the point A and an actual position C1 of the positioning photosphere is parallel to a Y-axis of a dual-camera coordinate system and the point A is separated from the C1 by a length Y1, make an intersection point N of the C1A and the plane XY of the world coordinate system, wherein the actual position C1 of the positioning photosphere is the offset coordinate (X1, Y1, Z1) in the world coordinate system relative to the target position S;

make an intersection point P of the Y axis in the dual-camera coordinate system and the plane XY of the world coordinate system;

make a perpendicular of the plane XY of the world coordinate system through the position C of the dual camera to have an intersection point H with the plane XY of the world coordinate system;

make an intersection point T by intersecting a line that passes through the S and is parallel to a line NB with a line C1N, wherein a line ST intersects with a preset plane at an intersection point E, and the preset plane is a plane that passes through a line CH and is parallel to a plane YZ of the world coordinate system; and

make the projection G of the S on the preset plane, making the perpendicular of the CH to pass through the C1, and to intersect with the CH at J, make the plane XY of the world coordinate system through the S to intersect with the line CH at C2, make the perpendicular of the line SE to pass through the C2 and to intersect with the line SE at F.

9. The non-transitory computer-readable storage medium in accordance with claim 8 , wherein to establish a mathematical equation set in accordance with the geometric positional relationship, the instructions cause the at least one processor to:

formulate a first equation set in accordance with the geometric figure projection of the geometric positional relationship in the positive direction of the Z-axis of the dual-camera coordinate system:

QB=Y 0*Sin γ+ X 1;

NA =( Y 0*Sin γ+ X 1)*Tan γ+ Y 0*Cos γ,

wherein Y0 is a reference coordinate value of the positioning photosphere, and an angle γ is a rotation angle of the dual-camera coordinate system around the Z-axis relative to the world coordinate system;

formulate a second equation set in accordance with the geometric figure projection of the geometric positional relationship in the negative direction of the X-axis of the dual-camera coordinate system:

N C 1=( Y 0*Cos γ+( X 1+ Y 0*Sin γ)*Tan γ)/Cos α+ Y 1;

CP=Z 1*Tan α+( Y 0*Cos γ+( X 1+ Y 0*Sin γ)*Tan γ)/Cos α+ Y 1;

wherein the α is a rotation angle of the dual-camera coordinate system around the X-axis relative to the world coordinate system;

formulate a third equation set in accordance with the geometric figure projection of the geometric positional relationship in the negative direction of the Z-axis of the world coordinate system:

CH =( Z 1*Tan α+( Y 0*Cos γ+( X 1+ Y 0*Sin γ)*Tan γ)/Cos α+ Y )*Cos γ;

solve a coordinate value CH of the dual camera on the Y-axis of the world coordinate system in accordance with the first equation set, the second equation set and the third equation set, when solving the mathematical equation set to acquire coordinate values of the dual camera on each coordinate axis of the world coordinate system;

formulate a fourth equation set in accordance with the geometric figure projection of the geometric positional relationship in the positive direction of the Z-axis of the dual-camera coordinate system:

AT=X 1*Tan γ;

TF =( Y 1+ X 1*Tan γ)*Sin γ;

SF=X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ;

formulate a fifth equation set in accordance with the geometric figure projection of the geometric positional relationship in the negative direction of the X-axis of the world coordinate system:

C 2 F=Z 1*Cos α;

formulate a sixth equation set in accordance with the geometric figure projection of the geometric positional relationship in the positive direction of the Y-axis of the dual-camera coordinate system:

FE=Z 1*Cos α*Tan β;

KF =( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ)*Tan β;

SE=X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ+ Z 1*Cos α*Tan β;

SG =( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ+ Z 1*Cos α*Tan β)*Cos β;

C 2 K=Z 1*Cos α−( X 1/Cos γ−( Y 1+ X 1*Tan γ)*Sin γ)*Tan β;

C 2 G =( Z 1*Cos α−( X 1/Cos γ−( Y+X 1*Tan γ)*Sin γ)*Tan α)*Cos β; and

solve the coordinate value SG of the dual camera in the X-axis and the coordinate value C2G of the dual camera in the Z-axis of the world coordinate system in accordance with the fourth equation set, the fifth equation set and the sixth equation set, when solving the mathematical equation set to acquire coordinate values of the dual camera on each coordinate axis of the world coordinate system.

Assignments (2)
CHANGE OF NAME Recorded Feb 5, 2021
From: HUAQIN TELECOM TECHNOLOGY CO.,LTD
To: HUAQIN TECHNOLOGY CO.,LTD
Reel/Frame 055233/0764 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 13, 2019
From: LIU, PINGPING; ZHANG, YUANTONG; WEI, ZHENGHUA
To: HUAQIN TELECOM TECHNOLOGY CO., LTD.
Reel/Frame 051289/0270 →
Priority Claims (1)
CN 201710455706.0 · Jun 16, 2017 · national
Continuity (2)
Continuation PCTCN2017114840 · Dec 6, 2017
Related Publication 20200118290A1 · Apr 16, 2020