IP Library Granted Patent US 11,498,228
Granted Patent B2
US 11,498,228 · App. 16/622,202 · Granted Nov 15, 2022

Method for sensing depth of object by considering external light and device implementing same

Inventors: Changhyeon Lee (Seoul, KR); Dong Seong Kim (Seoul, KR); Byungkon Sohn (Seoul, KR); Seung In Shin (Seoul, KR); Jungmin Shim (Seoul, KR); Jae Hoon Jeong (Seoul, KR)
Assignee: LG ELECTRONICS INC.
B25J19/023B25J9/1697B25J19/022B25J19/026G05D1/02G06T7/50H04N13/128G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,498,228
App. No.
16/622,202
Granted
Nov 15, 2022
Kind
B2
Abstract

The present disclosure relates to a method for sensing the depth of an object by considering external light and a device implementing the same, and a method for sensing the depth of an object by considering external light according to an embodiment of the present disclosure comprises the steps of: storing, in a storage unit, first depth information of an object, which is sensed at a first time point by a depth camera unit of a depth sensing module; storing, in the storage unit, second depth information of the object, which is sensed at a second time point by the depth camera unit; comparing, by a sensing data filtering unit of the depth sensing module, the generated first and second depth information to identify a filtering target region from the second depth information; and adjusting, by a control unit of the depth sensing module, the depth value of the region filtered from the second depth information.

Claims (59)

1. A depth sensing module for a robot, the depth sensing module comprising:

storage configured to store depth information;

a depth camera configured to:

generate first depth information of an object at a first time-point, and

generate second depth information of the object at a second time-point, the second time-point being before or after the first time-point;

a sensed-data filter configured to:

compare the first depth information with the second depth information, and

identify a filtering target region in the second depth information based on comparing the first depth information with the second depth information; and

a controller configured to:

change a depth value of the filtering target region in the second depth information or remove the second depth information from the storage,

wherein when a change in continuously detected depth values is non-linear, the depth sensing module identifies the change in the continuously detected depth values as being due to an influence of external light and filters a corresponding depth value associated with the change in the continuously detected depth values.

2. The depth sensing module of claim 1 , wherein the controller is further configured to:

apply third depth information to the filtering target region in the second depth information to generate updated second depth information or apply the first depth information to the filtering target region in the second depth information to generate updated second depth information, and the third depth information being obtained by applying a weight to the first depth information, and

store the updated second depth information as depth information corresponding to the second time-point into the storage.

3. The depth sensing module of claim 1 , wherein the sensed-data filter is further configured to:

compare a first depth value of a first depth cell in the first depth information with a second depth value of a second depth cell in the second depth information, the second depth information also corresponding to the first depth cell, and

identify the filtering target region based on comparing the first depth value with the second depth value.

4. The depth sensing module of claim 3 , wherein the sensed-data filter is further configured to:

in response to a difference between the first depth value and the second depth value being larger than a difference between a position of the depth sensing module at the first time-point and a position of the depth sensing module at the second time-point or in response to a difference between the first depth value and the second depth value being larger than a predefined reference value, set the second depth value of the second depth cell as a garbage value for nonuse, and

wherein the controller is further configured to:

generate a third depth value using the first depth value based on the difference between the position of the depth sensing module at the first time-point and the position of the depth sensing module at the second time-point, and

set the second depth value of the second depth cell as the third depth value.

5. The depth sensing module of claim 1 , further comprising:

a light information provider configured to store characteristic information of external light at a current position of the depth sensing module,

wherein the controller is further configured to extract the characteristic information of the external light stored in the light information provider, and

wherein the sensed-data filter is further configured to identify the filtering target region based on the extracted characteristic information of the external light.

6. The depth sensing module of claim 5 , wherein the light information provider includes:

map storage configured to store a position of a stationary object, and reflection or transmission information of the external light in a space, wherein the depth sensing module generates depth information in the space;

light-source period storage configured to store period information specifying time periods when the external light is turned on and off; and

context storage configured to store a context in which the external light is turned on and off.

7. A moving robot for sensing a depth of an object based on an effect of external light, the moving robot comprising:

a depth sensing module including:

storage configured to store depth information;

a depth camera configured to:

generate first depth information of an object at a first time-point, and

generate second depth information of the object at a second time-point, the second time-point being before or after the first time-point;

a sensed-data filter configured to:

compare the first depth information with the second depth information, and

identify a filtering target region in the second depth information based on comparing the first depth information with the second depth information;

a controller configured to:

change a depth value of the filtering target region in the second depth information or remove the second depth information from the storage;

an object sensing module configured to sense an object around the moving robot;

a driver configured to move the moving robot; and

a robot controller configured to:

identify the object around the moving robot based on sensing results received from the depth sensing module and the object sensing module, and

control a travel route of the moving robot based on identification of the object identified by the robot controller,

wherein when a change in continuously detected depth values is non-linear, the depth sensing module identifies the change in the continuously detected depth values as being due to an influence of the external light and filters a corresponding depth value associated with the change in the continuously detected depth values.

8. The moving robot of claim 7 , wherein the depth sensing module or the moving robot further includes:

a light information provider configured to store characteristic information of external light at a current position of the depth sensing module,

wherein the controller is further configured to extract the characteristic information of the external light stored in the light information provider, and

wherein the sensed-data filter is further configured to identify the filtering target region based on the characteristic information of the external light extracted from the light information provider.

9. The moving robot of claim 8 , wherein the light information provider includes:

map storage configured to store a position of a stationary object, and reflection or transmission information of the external light in a space, wherein the depth sensing module generates depth information in the space;

light-source period storage configured to store period information specifying time periods when the external light is turned on and off; and

context storage configured to store a context in which the external light is turned on and off.

10. The moving robot of claim 8 , wherein the depth sensing module or the moving robot further includes an optical sensor, and

wherein the robot controller is further configured to:

in response to an intensity of the external light or an on-off time-point of the external light sensed by the optical sensor, while the moving robot travels, being different from the characteristic information of the external light stored in the light information provider, change an intensity or an on-off time-point of external light to generate an adjusted intensity or an adjusted on-off time-point of external light, and

store the adjusted intensity or the adjusted on-off time-point of external light in the light information provider.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 26, 2025
From: LG ELECTRONICS INC.
To: BEAR ROBOTICS KOREA, INC.
Reel/Frame 072964/0006 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 13, 2019
From: LEE, CHANGHYEON; KIM, DONG SEONG; SOHN, BYUNGKON; SHIN, SEUNG IN; SHIM, JUNGMIN; JEONG, JAE HOON
To: LG ELECTRONICS INC.
Reel/Frame 051272/0386 →
Priority Claims (2)
KR 10-2017-0075055 · Jun 14, 2017 · national
KR 10-2017-0095412 · Jul 27, 2017 · national
Continuity (1)
Related Publication 20200122344A1 · Apr 23, 2020