IP Library Granted Patent US 11,120,629
Granted Patent B2
US 11,120,629 · App. 16/508,830 · Granted Sep 14, 2021

Method and device for providing augmented reality, and computer program

Inventors: Sang Jo Choi (Seongnam-si, KR); Hee Cheol Park (Seongnam-si, KR)
Assignee: LINE CORPORATION
G06T19/006G01P15/18G01S7/4802G01S17/08G01S17/42G01S17/86G06F3/011G06F3/0346G06T7/73G01S15/08G06T2219/008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,120,629
App. No.
16/508,830
Granted
Sep 14, 2021
Kind
B2
Abstract

According to at least some example embodiments, a computer-readable medium stores computer-executable program instructions that, when executed by a processor, cause the processor to perform operations including, obtaining plane information of a plane by using first distances from a terminal to a plurality of points on the plane; obtaining a normal vector of the plane by using direction information of the terminal measured by a direction sensor and the plane information; determining, based on the normal vector, a parameter of an object to be displayed on the plane; and displaying, on a display of the terminal, the object according to the determined parameter.

Claims (54)

1. A non-transitory computer-readable medium storing computer-executable program instructions that, when executed by a processor, cause the processor to perform operations including,

obtaining plane information of a plane by using first distances from a terminal to a plurality of points on the plane;

obtaining a normal vector of the plane by using direction information of the terminal measured by a direction sensor and the plane information;

determining, based on the normal vector, a parameter of an object to be displayed on the plane; and

displaying, on a display of the terminal, the object according to the determined parameter.

2. The non-transitory computer-readable medium of claim 1 , wherein the operations further include measuring directions of the plurality of points and the first distances by using an infrared sensor,

wherein the obtaining of the plane information includes obtaining the plane information by using the measured directions and first distances.

3. The non-transitory computer-readable medium of claim 1 , wherein,

the plane information is direction information of the plane based on a direction of the terminal,

the direction sensor includes at least one of an acceleration sensor and a gyro sensor, and

the obtaining of the normal vector includes obtaining the normal vector by converting a reference of the direction information of the plane from the direction of the terminal to a reference direction of the direction sensor, based on the measured direction information of the terminal.

4. The non-transitory computer-readable medium of claim 1 , wherein the determining includes determining an attribute of the plane based on the normal vector and determining the parameter based on the determined attribute.

5. The non-transitory computer-readable medium of claim 4 , wherein,

the plane information is direction information of the plane, and

the determining includes determining a first parameter of the object based on the attribute and determining a second parameter of the object based on the direction information of the plane.

6. The non-transitory computer-readable medium of claim 4 , wherein

the plane information is direction information of the plane,

the determining includes determining a type of the object based on the attribute and determining a rendering direction of the object based on the direction information of the plane, and

the displaying includes two-dimensionally rendering 3-dimensional (3D) information for displaying the object, according to the rendering direction, and two-dimensionally displaying the object.

7. The non-transitory computer-readable medium of claim 1 , wherein the displaying includes overlapping the object and a video captured by a camera provided in the terminal and displaying the video overlapped by the object.

8. The non-transitory computer-readable medium of claim 7 ,

wherein the operations further include measuring the first distances using a distance sensor provided in the terminal, and

wherein the camera and the distance sensor are provided towards a same direction.

9. The non-transitory computer-readable medium of claim 1 , wherein the parameter comprises at least one of a size of the object, direction of the object, color of the object, or an animation applied to the object.

10. An augmented reality providing method comprising:

obtaining plane information of a plane by using first distances from a terminal to a plurality of points on the plane;

obtaining a normal vector of the plane by using the plane information and direction information of the terminal measured by a direction sensor;

determining, based on the normal vector, a parameter of an object to be displayed on the plane; and

displaying, on a display of the terminal, the object according to the determined parameter.

11. An augmented reality providing apparatus comprising:

a distance sensor;

a direction sensor; and

processing circuitry configured to,

obtain plane information of a plane by using first distances measured by the distance sensor, the first distances being distances from a terminal to a plurality of points on the plane,

obtain a normal vector of the plane by using direction information of the terminal measured by the direction sensor and the plane information,

determine based on the normal vector, a parameter of an object to be displayed on the plane, and

display, on a display of the terminal, the object according to the determined parameter.

12. The augmented reality providing apparatus of claim 11 ,

wherein the direction sensor includes at least one of an acceleration sensor or a gyro sensor,

wherein the plane information is direction information of the plane based on a direction of the terminal, and

wherein the processing circuitry is configured to obtain the normal vector by converting a reference of the direction information of the plane from the direction of the terminal to a reference direction of the direction sensor, based on the measured direction information of the terminal.

13. The augmented reality providing apparatus of claim 11 , wherein the processing circuitry is configured to determine an attribute of the plane based on the normal vector and determine the parameter based on the determined attribute.

14. The augmented reality providing apparatus of claim 13 ,

wherein the plane information is direction information of the plane, and

wherein the processing circuitry is configured to determine a first parameter of the object based on the attribute and determine a second parameter of the object based on the direction information of the plane.

15. The augmented reality providing apparatus of claim 13 , wherein,

the plane information is direction information of the plane,

the processing circuitry is configured to determine a type of the object based on the attribute and determine a rendering direction of the object based on the direction information of the plane, and

the processing circuitry is configured to two-dimensionally renders 3-dimensional (3D) information for displaying the object, according to the rendering direction, and two-dimensionally displays 3-dimensional (3D) information for displaying the object.

16. The augmented reality providing apparatus of claim 11 , wherein the processing circuitry is configured to overlap the object and a video captured by a camera provided in the terminal and display the video overlapped by the object.

17. The augmented reality providing apparatus of claim 16 , further comprising:

the camera;

wherein the camera and the distance sensor are provided towards a same direction.

18. The augmented reality providing apparatus of claim 11 , wherein the processing circuitry is configured such that the parameter comprises at least one of a size of the object, direction of the object, color of the object, or an animation applied to the object.

Assignments (7)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 12, 2024
From: Z INTERMEDIATE GLOBAL CORPORATION
To: LY CORPORATION
Reel/Frame 067096/0431 →
CHANGE OF NAME Recorded Apr 10, 2024
From: LINE CORPORATION
To: Z INTERMEDIATE GLOBAL CORPORATION
Reel/Frame 067069/0467 →
CORRECTIVE ASSIGNMENT TO CORRECT THE SPELLING OF THE ASSIGNEES CITY IN THE ADDRESS SHOULD BE TOKYO, JAPAN PREVIOUSLY RECORDED AT REEL: 058597 FRAME: 0303. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 17, 2023
From: A HOLDINGS CORPORATION
To: LINE CORPORATION
Reel/Frame 062401/0490 →
CORRECTIVE ASSIGNMENT TO CORRECT THE THE CITY SHOULD BE SPELLED AS TOKYO PREVIOUSLY RECORDED AT REEL: 058597 FRAME: 0141. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Jan 17, 2023
From: LINE CORPORATION
To: A HOLDINGS CORPORATION
Reel/Frame 062401/0328 →
CHANGE OF NAME Recorded Dec 28, 2021
From: LINE CORPORATION
To: A HOLDINGS CORPORATION
Reel/Frame 058597/0141 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 28, 2021
From: A HOLDINGS CORPORATION
To: LINE CORPORATION
Reel/Frame 058597/0303 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 17, 2019
From: CHOI, SANG JO; PARK, HEE CHEOL
To: LINE CORPORATION
Reel/Frame 049778/0705 →
Continuity (2)
Continuation PCTKR2017002373 · Mar 6, 2017
Related Publication 20190333282A1 · Oct 31, 2019
Cited By (1)
US 12,385,944