Granted Patent B2

US 12,556,816 · App. 18/028,516 · Granted Feb 17, 2026

Inventors: Pengfei Dong (Shenzhen, CN); Cong Chen (Shenzhen, CN)

Assignee: ARASHI VISION INC.

H04N23/683G01P13/04H04N23/6812H04N23/698G06T2207/10016

Abstract

Claims (215)

1 . A method for detensioning a rotation direction of a gyroscope, comprising steps of:

obtaining an acceleration value and an angular velocity value of an IMU in real time, determining the acceleration value as a first acceleration value, and estimating an attitude from an IMU coordinate system to a world coordinate system;

converting the first acceleration value from the IMU coordinate system to the world coordinate system to obtain a second acceleration value;

in the world coordinate system, filtering the second acceleration value to remove a gravitational acceleration to obtain a third acceleration value;

converting the third acceleration value into the IMU coordinate system to obtain acceleration components along an X axis, a Y axis and a Z axis of a fourth acceleration value; and

determining a rotation direction of the gyroscope on the basis of the acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration value,

wherein the step of determining the rotation direction of the gyroscope on the basis of the acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration comprises:

comparing the acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration value to determine an axis corresponding to a maximum acceleration component, and determining the rotation direction of the gyroscope as: a direction of a rotation axis when the axis, corresponding to and in the same direction as the maximum acceleration component, rotates towards a camera.

2 . The method according to claim 1 , wherein the step of estimating the attitude from the IMU coordinate system to the world coordinate system comprises a step of estimating the attitude from the IMU coordinate system to the world coordinate system using a Kalman filter.

3 . The method according to claim 2 , wherein the step of estimating the attitude from the IMU coordinate system to the world coordinate system using the Kalman filter comprises: estimating a rotation amount from the IMU coordinate system to the world coordinate system using an Extended Kalman Filtering and on the basis of the acceleration value and the angular velocity value.

4 . The method according to claim 1 , wherein after the step of determining the acceleration value as the first acceleration value, the method further comprises: a step of performing noise reduction on the first acceleration value using a low-pass filter to obtain the first acceleration value

d

i

′

after low-pass filtering and noise reduction.

5 . The method according to claim 4 , wherein the step of performing noise reduction on the first acceleration value using the low-pass filter to obtain the first acceleration value

d

i

′

after low-pass filtering and noise reduction comprises:

performing low-pass filtering and noise reduction on the first acceleration value based on formula

d

i

′

=

α

·

d

i

+

(

1

-

α

)

·

R

1

·

d

i

-

1

′

,

wherein

,

d

i

′

 represents the first acceleration value after low-pass filtering and noise reduction at time i, d i represents the first acceleration value at time i, R i is a relative rotation amount of the gyroscope at the i th frame in a video, R i =exp[−ω i ·Δt] x , ω i represents the angular velocity value at time i,

d

i

-

1

′

 represents the first acceleration value after low-pass filtering and noise reduction at the time (i−1), α represents a smoothing factor,

α

=

Δ

⁢

t

Rc

+

Δ

⁢

t

,

Rc

=

1

2

⁢

π

⁢

f

c

,

 where, f c represents a cutoff frequency of the low-pass filter, R c represents a time constant, and Δt represents a sampling interval of gyroscope data.

6 . The method according to claim 5 , wherein the step of converting the first acceleration value from the IMU coordinate system to the world coordinate system to obtain the second acceleration value comprises:

according to the formula

A

w

k

=

R

i

⁢

_

⁢

w

k

-

1

*

d

i

′

 at time k, converting the first acceleration value from the IMU coordinate system to the world coordinate system to obtain the second acceleration value

A

w

K

,

where

⁢

R

i

⁢

_

⁢

w

k

 is an estimated rotation amount from the IMU coordinate system to the world coordinate system, and

d

i

′

 is the first acceleration value after low-pass filtering and noise reduction.

7 . The method according to claim 6 , wherein the step of, in the world coordinate system, filtering the second acceleration value to remove the gravitational acceleration to obtain a third acceleration value comprises:

in the world coordinate system, at time k, filtering the second acceleration value

A

w

K

 to remove the gravitational acceleration, and obtaining the third acceleration value

Acc

w

k

.

8 . The method according to claim 7 , wherein before filtering the second acceleration value, the method further comprises a step of:

in the world coordinate system, at time k, performing noise reduction on the second acceleration value

A

w

K

.

9 . The method according to claim 1 , wherein the step of converting the third acceleration value into the IMU coordinate system to obtain acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration value comprises:

calculating the fourth acceleration value

Acc

imu

k

 according to a formula:

Acc

imu

k

=

R

i

⁢

_

⁢

w

k

-

1

⁢

Acc

w

k

,

where

⁢

R

i

⁢

_

⁢

w

k

-

1

 is a rotation matrix from the world coordinate system to the IMU coordinate system at time k, and

A

⁢

c

⁢

c

w

k

 is the third acceleration value.

10 . The method according to claim 1 , wherein after the step of determining the rotation direction of the gyroscope on the basis of the acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration value, the method further comprises steps of:

determining the rotation direction of the gyroscope as a rotation direction of the camera, and determining a positional relationship between the camera and the rotation axis on the basis of the rotation direction of the gyroscope.

11 . A non-transitory computer-readable storage medium, the computer-readable storage medium storing a computer program, which when being executed by a processor, cause the processor to perform the steps of the method of claim 1 .

12 . A computer device, comprising:

one or more processors;

a memory; and

one or more computer programs, the processors and the memory being connected by a bus, wherein the one or more computer programs are stored in the memory, which when being executed by the one or more processors, cause the one or more processors to perform the steps of the method of claim 1 .

13 . A method for obtaining a bullet-time photography effect, comprising steps of:

capturing a panoramic video using a camera to rotate around a shooting target;

obtaining an acceleration value and an angular velocity value of an IMU in real time;

determining the acceleration value as a first acceleration value, and estimating an attitude from an IMU coordinate system to a world coordinate system;

converting the first acceleration value from the IMU coordinate system to the world coordinate system to obtain a second acceleration value;

in the world coordinate system, filtering the second acceleration value to remove a gravitational acceleration to obtain a third acceleration value;

converting the third acceleration value into the IMU coordinate system to obtain acceleration components along the X axis, the Y axis and the Z axis of a fourth acceleration value; and

determining a rotation direction of the gyroscope on the basis of the acceleration components along a X axis, a Y axis and a Z axis of the fourth acceleration value; and

generating images corresponding to the rotation direction of the gyroscope in the panoramic video on the basis of the rotation direction of the gyroscope,

wherein the step of determining the rotation direction of the gyroscope on the basis of the acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration comprises:

comparing the acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration value to determine an axis corresponding to a maximum acceleration component, and determining the rotation direction of the gyroscope as: a direction of a rotation axis when the axis, corresponding to and in the same direction as the maximum acceleration component, rotates towards a camera.

14 . A camera, comprising:

one or more processors;

a memory; and

one or more computer programs, the processors and the memory being connected by a bus, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, which when being executed by the one or more processors, cause the one or more processors to perform a method for determining a rotation direction of a gyroscope, comprising steps of:

obtaining an acceleration value and an angular velocity value of an IMU in real time, determining the acceleration value as a first acceleration value, and estimating an attitude from an IMU coordinate system to a world coordinate system;

converting the first acceleration value from the IMU coordinate system to the world coordinate system to obtain a second acceleration value;

in the world coordinate system, filtering the second acceleration value to remove a gravitational acceleration to obtain a third acceleration value;

converting the third acceleration value into the IMU coordinate system to obtain acceleration components along a X axis, a Y axis and a Z axis of a fourth acceleration value; and

determining a rotation direction of the gyroscope on the basis of the acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration value,

wherein the step of determining the rotation direction of the gyroscope on the basis of the acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration comprises:

comparing the acceleration components along the X axis, the Y axis and the Z axis of the fourth acceleration value to determine an axis corresponding to a maximum acceleration component, and determining the rotation direction of the gyroscope as: a direction of a rotation axis when the axis, corresponding to and in the same direction as the maximum acceleration component, rotates towards a camera.

15 . A non-transitory computer-readable medium having computer programs stored thereon, which when being executed by a processor, cause the processor to perform the steps of the method of claim 13 .

16 . A computer device, comprising:

one or more processors;

a memory; and

one or more computer programs, the processors and the memory being connected by a bus, wherein the one or more computer programs are stored in the memory, which when being executed by the one or more processors, cause the one or more processors to perform the steps of the method of claim 13 .

17 . A camera, comprising:

one or more processors;

a memory; and

one or more computer programs, the processors and the memory being connected by a bus, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, which when being executed by the one or more processors, cause the one or more processors to perform the steps of the method of claim 13 .

Assignments (1)

Priority Claims (1)

Continuity (1)

References Cited (25)