IP Library Granted Patent US 11,937,539
Granted Patent B2
US 11,937,539 · App. 17/003,834 · Granted Mar 26, 2024

Sensor fusion for localization and path planning

Inventors: Mandar N. Kulkarni (Richardson, TX); Yuming Zhu (Plano, TX); Neha Dawar (Plano, TX); Songwei Li (Lewisville, TX); Boon L. Ng (Plano, TX); Jianzhong Zhang (Plano, TX)
Assignee: Samsung Electronics Co., Ltd.
A01D34/008G05D1/0214G05D1/0227G05D1/0274G06F18/22G06V10/803G06V20/58H04W84/18A01D2101/00G05D2201/0208
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,937,539
App. No.
17/003,834
Granted
Mar 26, 2024
Kind
B2
Abstract

An electronic device includes a first set of sensors configured to generate motion information. The electronic device also includes a second set of sensors configured to receive information from multiple anchors. The electronic device further includes a processor configured to generate a path to drive the electronic device within an area. The processor is configured to receive the motion information. The processor is configured to generate ranging information based on the information that is received. While the electronic device is driven along the path, the processor is configured to identify a location and heading direction within the area of the electronic device based on the motion information. The processor is configured to modify the estimate of location and the heading direction of the electronic device based on the ranging information. The processor is configured to drive within the area according to the path, based on the location and heading direction.

Claims (124)

1. An electronic device comprising:

a first set of sensors configured to generate motion information;

a second set of sensors configured to receive information from multiple anchors positioned at fixed locations throughout an area; and

a processor operably connected to the first set of sensors and the second set of sensors, the processor configured to:

generate a path to drive the electronic device within the area, the path configured to substantially cover the area, the path having a start location and an end location,

receive the motion information from the first set of sensors,

determine whether a wheel of the electronic device is slipping based on the motion information,

generate ranging information based on the information that is received, via the second set of sensors, from the multiple anchors,

while the electronic device is driven along the path, identify a location and heading direction within the area of the electronic device based on a subset of the motion information, wherein the subset of the motion information comprises a first subset of the motion information in response to a determination that the wheel of the electronic device is slipping or a second subset of the motion information in response to a determination that the wheel of the electronic device is not slipping,

modify the identified location and heading direction of the electronic device based on the ranging information,

control the electronic device to drive within the area from the start location to the end location according to the path, based on the modified location and heading direction of the electronic device,

upon the electronic device reaching the end location, identify one or more missed portions of the area that were not traversed, and

generate a new path for the electronic device to traverse the one or more missed portions of the area that were not traversed.

2. The electronic device of claim 1 , wherein:

the first set of sensors includes an inertial measurement sensor and a wheel encoder sensor;

the motion information, that is obtained from the first set of sensors, includes linear velocity and angular velocity; and

the processor is further configured to:

identify a first distance traveled based on the motion information from the wheel encoder sensor,

identify a second distance traveled based on the motion information from the inertial measurement sensor,

identify a third distance traveled based on the ranging information from the second set of sensors, and

in response to a determination, over a period of time, that the first distance traveled is larger than either the second distance traveled or the third distance traveled, identify the location and heading direction of the electronic device based on the inertial measurement sensor and the second set of sensors while not using the wheel encoder sensor.

3. The electronic device of claim 1 , wherein the processor is configured to:

identify a first location of the electronic device within the area based on the first set of sensors and the second set of sensors;

while the first location is identified, identify a second location of the electronic device within the area based on the second set of sensors;

compare a difference between a first attribute based on the first location and a second attribute based on the second location to a threshold, wherein the first and second attributes are headings, locations, or times; and

determine whether to use the first location or the second location based on the comparison.

4. The electronic device of claim 1 , wherein:

the electronic device comprises a third set of sensors configured to detect an obstacle; and

the processor is configured to:

while the electronic device is driving along the path, determine whether the obstacle blocks the path,

in response to a determination that the obstacle blocks the path, control the electronic device to stop driving for a predetermined period of time,

in response to a determination that the obstacle did not move after the predetermined period of time, modify the path to avoid the obstacle in order to resume on the path.

5. The electronic device of claim 1 , wherein the processor is configured to:

generate a map of the area, based on the location and a trajectory of the electronic device as the electronic device traverses the area;

in response to the electronic device arriving at the end location, divide the map into a plurality of cells;

provide a first flag to a first portion of the plurality of cells that correspond to portions of the map that the electronic device traversed;

provide a second flag to a second portion of the plurality of cells that correspond to the one or more missed portions of the area that were not traversed, the second flag identifying missed regions;

cluster the missed regions into circles of different radius; and

control the electronic device to revisit the missed regions.

6. The electronic device of claim 5 , wherein the processor is configured to:

generate a list of the missed regions in the area to determine an order in which the missed regions are visited by the electronic device,

control the electronic device to drive to a center position of a first missed region that is included in the list based on the determined order,

in response to a determination that the electronic device is located at the center position of the first missed region, control the electronic device to drive in a predefined trajectory through the first missed region, and

in response to a determination that an obstacle prevents the electronic device from reaching the center position of the first missed region, control the electronic device to drive the electronic device to another missed region of the missed regions included in the list based on the determined order or control the electronic device to drive the electronic device to the end location, upon a detection of a triggering event.

7. The electronic device of claim 1 , wherein:

the electronic device further comprises a third set of sensors that includes at least one of a camera and a grass detector; and

the processor is configured to:

in response to the electronic device approaching a boundary line, determine a boundary type of the boundary line based on the third set of sensors;

in response to a determination that the boundary type is a soft boundary, control the electronic device to drive the electronic device over and along the boundary line; and

in response to a determination that the boundary type is a hard boundary, control the electronic device to drive the electronic device a threshold distance from the boundary line within the area.

8. The electronic device of claim 1 , wherein the processor is configured to:

identify a distance between the second set of sensors and each of the multiple anchors;

determine that the distance between the second set of sensors and a one or more of the multiple anchors is larger than a threshold distance; and

transmit, to at least one of the multiple anchors, a message indicating that the at least one anchor is to enter or exit a power saving mode,

wherein the second set of sensors are positioned on opposite locations on a surface of the electronic device.

9. The electronic device of claim 1 , further comprising:

a drive system configured to drive the electronic device throughout a grass covered region of a lawn; and

a third set of sensors configured to identify a distance to a border of the grass covered region of the lawn,

wherein the processor is configured to:

control the drive system to drive the electronic device along a perimeter of the lawn,

generate a grid map of the lawn,

after the grid map is generated, control the drive system to drive the electronic device within the perimeter of the lawn along a predefined path,

in response to a determination that an obstacle blocks the predefined path, control the drive system to drive the electronic device around the obstacle,

while the drive system drives the electronic device around the obstacle, include the obstacle in the grid map,

after the drive system drives the electronic device around the obstacle, control the drive system to resume driving the electronic device along the predefined path within the perimeter of the lawn, and

update the path while drive system drives the electronic device through the grass covered region of the lawn.

10. The electronic device of claim 1 , wherein the one or more missed portions of the area were missed by the electronic device due to the electronic device moving around an obstacle along the path.

11. A method for controlling an electronic device, the method comprising:

generating a path to drive the electronic device within an area, the path configured to substantially cover the area, the path having a start location and an end location;

receiving motion information from a first set of sensors;

determining whether a wheel of the electronic device is slipping based on the motion information;

generating ranging information based on information that is received, via a second set of sensors, from multiple anchors that are positioned at fixed locations throughout the area;

while the electronic device is driven along the path, identifying a location and heading direction within the area of the electronic device based on a subset of the motion information, wherein the subset of the motion information comprises a first subset of the motion information in response to a determination that the wheel of the electronic device is slipping or a second subset of the motion information in response to a determination that the wheel of the electronic device is not slipping;

modifying the identified location and heading direction of the electronic device based on the ranging information;

controlling the electronic device to drive within the area from the start location to the end location according to the path, based on the modified location and heading direction of the electronic device;

upon the electronic device reaching the end location, identifying one or more missed portions of the area that were not traversed; and

generating a new path for the electronic device to traverse the one or more missed portions of the area that were not traversed.

12. The method of claim 11 , wherein:

the first set of sensors includes an inertial measurement sensor and a wheel encoder sensor;

the motion information, that is obtained from the first set of sensors includes linear velocity and angular velocity; and

the method further comprises:

identifying a first distance traveled based on the motion information from the wheel encoder sensor,

identifying a second distance traveled based on the motion information from the inertial measurement sensor,

identifying a third distance traveled based on the ranging information from the second set of sensors, and

in response to a determination, over a period of time, that the first distance traveled is larger than either the second distance traveled or the third distance traveled, identifying the location and heading direction of the electronic device based on the inertial measurement sensor and the second set of sensors while not using the wheel encoder sensor.

13. The method of claim 11 , further comprising:

identifying a first location of the electronic device within the area based on the first set of sensors and the second set of sensors;

while the first location is identified, identifying a second location of the electronic device within the area based on the second set of sensors;

comparing a difference between a first attribute based on the first location and a second attribute based on the second location to a threshold, wherein the first and second attributes are headings, locations, or times; and

determining whether to use the first location or the second location based on the comparison.

14. The method of claim 11 , further comprising:

while the electronic device is driving along the path, determining whether an obstacle blocks the path,

in response to a determination that the obstacle blocks the path, controlling the electronic device to stop driving for a predetermined period of time,

in response to a determination that the obstacle did not move after the predetermined period of time, modifying the path to avoid the obstacle in order to resume on the path.

15. The method of claim 11 , further comprising:

generating a map of the area, based on the location and a trajectory of the electronic device as the electronic device traverses the area;

in response to the electronic device arriving at the end location, dividing the map into a plurality of cells;

providing a first flag to a first portion of the plurality of cells that correspond to portions of the map that the electronic device traversed;

providing a second flag to a second portion of the plurality of cells that correspond to the one or more missed portions of the area that were not traversed, the second flag identifying missed regions;

clustering the missed regions into circles of different radius; and

controlling the electronic device to revisit the missed regions.

16. The method of claim 15 , further comprising:

generating a list of the missed regions in the area to determine an order in which the missed regions are visited by the electronic device,

controlling the electronic device to drive to a center position of a first missed region that is included in the list based on the determined order,

in response to a determination that the electronic device is located at the center position of the first missed region, controlling the electronic device to drive in a predefined trajectory through the first missed region, and

in response to a determination that an obstacle prevents the electronic device from reaching the center position of the first missed region, controlling the electronic device to drive the electronic device to another missed region of the missed regions included in the list based on the determined order or control the electronic device to drive the electronic device to the end location, upon a detection of a triggering event.

17. The method of claim 11 , further comprising:

in response to the electronic device approaching a boundary line, determining a boundary type of the boundary line based on a third set of sensors that includes at least one of a camera and a grass detector;

in response to a determination that the boundary type is a soft boundary, controlling the electronic device to drive the electronic device over and along the boundary line; and

in response to a determination that the boundary type is a hard boundary, controlling the electronic device to drive the electronic device a threshold distance from the boundary line within the area.

18. The method of claim 11 , further comprising:

identifying a distance between the second set of sensors and each of the multiple anchors;

determining that the distance between the second set of sensors and a one or more of the multiple anchors is larger than a threshold distance; and

transmitting, to at least one of the multiple anchors, a message indicating that the at least one anchor is to enter or exit a power saving mode,

wherein the second set of sensors are positioned on opposite locations on a surface of the electronic device.

19. The method of claim 11 , further comprising:

driving the electronic device along a perimeter of a lawn,

generating a grid map of the lawn,

after the grid map is generated, driving the electronic device within the perimeter of the lawn along a predefined path,

in response to a determination that an obstacle blocks the predefined path, driving the electronic device around the obstacle,

while driving the electronic device around the obstacle, including the obstacle in the grid map,

after driving around the obstacle, resume driving the electronic device along the predefined path within the perimeter of the lawn, and

updating the path while driving the electronic device through a grass covered region of the lawn.

20. The method claim 11 , wherein the one or more missed portions of the area were missed by the electronic device due to the electronic device moving around an obstacle along the path.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 26, 2020
From: KULKARNI, MANDAR N.; ZHU, YUMING; DAWAR, NEHA; LI, SONGWEI; NG, BOON LOONG; ZHANG, JIANZHONG
To: SAMSUNG ELECTRONICS CO., LTD.
Reel/Frame 053607/0916 →
Continuity (5)
Provisional Application 62966769 · Jan 28, 2020
Provisional Application 62941126 · Nov 27, 2019
Provisional Application 62938509 · Nov 21, 2019
Provisional Application 62892762 · Aug 28, 2019
Related Publication 20210064043A1 · Mar 4, 2021
Cited By (3)
US 12,245,050 US 12,429,338 US 12,662,163