IP Library › Granted Patent US 11,410,356
Granted Patent B2
US 11,410,356 · App. 15/931,715 · Granted Aug 9, 2022

Systems and methods for representing objects using a six-point bounding box

Inventor: Daniele Molinari (Sunnyvale, CA)
Assignee: Toyota Research Institute, Inc.
G06T11/203B60W30/09B60W60/001G06T7/10G06T7/70B60W2420/42B60W2420/52G06T2207/30252G06T2210/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,410,356
App. No.
15/931,715
Granted
Aug 9, 2022
Kind
B2
Abstract

System, methods, and other embodiments described herein relate to improving a representation of objects in a surrounding environment. In one embodiment, a method includes, in response to receiving sensor data depicting the surrounding environment including a corridor that defines a left boundary and a right boundary, identifying at least one object from the sensor data. The method includes transforming segmented data from the sensor data that represents the object into a bounding box by defining the bounding box according to six points relative to the corridor. The method includes providing the six points of the bounding box as a reduced representation of the object.

Claims (44)

1. An observation system for improving a representation of objects in a surrounding environment, comprising:

one or more processors;

a memory communicably coupled to the one or more processors and storing:

a detection module including instructions that, when executed by the one or more processors, cause the one or more processors to, in response to receiving sensor data depicting the surrounding environment including a corridor that defines a left boundary and a right boundary, identify at least one object from the sensor data; and

a reference module including instructions that, when executed by the one or more processors, cause the one or more processors to transform segmented data from the sensor data that represents the object into a bounding box by defining the bounding box according to six points relative to the corridor as references of distance within a two-dimensional space to the left boundary and the right boundary,

wherein the reference module further includes instructions to provide the six points of the bounding box as a reduced representation of the object.

2. The observation system of claim 1 , wherein the reference module includes instructions to transform the segmented data into the bounding box including instructions to:

define a reference system using the left boundary, the right boundary, and a centerline that approximately bisects a space in parallel between the left boundary and the right boundary, and

project reference points of the object into the reference system to derive the six points, wherein the reference points are outward boundary minimums and outward boundary maximums relative to the reference system.

3. The observation system of claim 2 , wherein the reference module includes instructions to project the reference points including instructions to project the outward boundary minimums and the outward boundary maximums against the centerline, a left reference that indicates a distance from the left boundary, and a right reference that indicates a distance from the right boundary.

4. The observation system of claim 2 , wherein the six points include right boundary distance points specifying a perpendicular distance of a closest point and a furthest point to the right boundary, left boundary distance points specifying a closest point and a furthest point to the left boundary, and centerline distance points specifying a furthest point and a closest point along the centerline relative to a point of origin.

5. The observation system of claim 1 , wherein the corridor is a roadway,

wherein the left boundary and right boundary are defined relative to a vehicle that acquires the sensor data and a direction of travel of traffic associated with the vehicle, and

wherein the bounding box defines a convex hull.

6. The observation system of claim 1 , wherein the detection module includes instructions to receive the sensor data including instructions to acquire the sensor data from at least one sensor of a vehicle, and

wherein the detection module includes instructions to identify the at least one object from the sensor data including instructions to segment the object from the sensor data using a segmentation model that is a machine-learning model.

7. The observation system of claim 1 , wherein the reference module includes instructions to provide the bounding box including instructions to plan movements of a vehicle using the bounding box as an indicator of a form of the object that is an obstacle for the vehicle to navigate, and controlling the vehicle according to the movements.

8. The observation system of claim 1 , wherein the observation system is integrated within an autonomous vehicle.

9. A non-transitory computer-readable medium for improving a representation of objects in a surrounding environment, and including instructions that, when executed by one or more processors, cause the one or more processors to:

in response to receiving sensor data depicting the surrounding environment including a corridor that defines a left boundary and a right boundary, identify at least one object from the sensor data; and

transform segmented data from the sensor data that represents the object into a bounding box by defining the bounding box according to six points relative to the corridor as references of distance within a two-dimensional space to the left boundary and the right boundary,

provide the six points of the bounding box as a reduced representation of the object.

10. The non-transitory computer-readable medium of claim 9 , wherein the instructions to transform the segmented data into the bounding box include instructions to:

define a reference system using the left boundary, the right boundary, and a centerline that approximately bisects a space in parallel between the left boundary and the right boundary, and

project reference points of the object into the reference system to derive the six points, wherein the reference points are outward boundary minimums and outward boundary maximums relative to the reference system.

11. The non-transitory computer-readable medium of claim 10 , wherein the instructions to project the reference points include instructions to project the outward boundary minimums and the outward boundary maximums against the centerline, a left reference that indicates a distance from the left boundary, and a right reference that indicates a distance from the right boundary.

12. The non-transitory computer-readable medium of claim 10 , wherein the six points include right boundary distance points specifying a perpendicular distance of a closest point and a furthest point to the right boundary, left boundary distance points specifying a closest point and a furthest point to the left boundary, and centerline distance points specifying a furthest point and a closest point along the centerline relative to a point of origin.

13. The non-transitory computer-readable medium of claim 9 , wherein the corridor is a roadway,

wherein the left boundary and right boundary are defined relative to a vehicle that acquires the sensor data and a direction of travel of traffic associated with the vehicle, and

wherein the bounding box defines a convex hull.

14. A method of improving a representation of objects in a surrounding environment, comprising:

in response to receiving sensor data depicting the surrounding environment including a corridor that defines a left boundary and a right boundary, identifying at least one object from the sensor data;

transforming segmented data from the sensor data that represents the object into a bounding box by defining the bounding box according to six points relative to the corridor as references of distance within a two-dimensional space to the left boundary and the right boundary; and

providing the six points of the bounding box as a reduced representation of the object.

15. The method of claim 14 , wherein transforming the segmented data into the bounding box includes:

defining a reference system using the left boundary, the right boundary, and a centerline that approximately bisects a space in parallel between the left boundary and the right boundary; and

projecting reference points of the object into the reference system to derive the six points, wherein the reference points are outward boundary minimums and maximums relative to the reference system.

16. The method of claim 15 , wherein projecting the reference points includes projecting outward boundary minimums and maximums against the centerline, a left reference that indicates a distance from the left boundary, and a right reference that indicates a distance from the right boundary.

17. The method of claim 15 , wherein the six points include right boundary distance points specifying a perpendicular distance of a closest point and a furthest point to the right boundary, left boundary distance points specifying a closest point and a furthest point to the left boundary, and centerline distance points specifying a furthest point and a closest point along the centerline relative to a point of origin.

18. The method of claim 14 , wherein the corridor is a roadway, wherein the left boundary and right boundary are defined relative to a vehicle that acquires the sensor data and a direction of travel of traffic associated with the vehicle, wherein the bounding box defines outward extents using only the six points relative to the corridor, and

wherein the bounding box defines a convex hull.

19. The method of claim 14 , wherein receiving the sensor data includes acquiring the sensor data from at least one sensor of a vehicle, and

wherein identifying the at least one object from the sensor data includes segmenting the object from the sensor data using a segmentation model that is a machine-learning model.

20. The method of claim 14 , wherein providing the bounding box includes planning movements of a vehicle using the bounding box as an indicator of a form of the object that is an obstacle for the vehicle to navigate, and controlling the vehicle according to the movements.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 23, 2022
From: TOYOTA RESEARCH INSTITUTE, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 060866/0439 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 18, 2020
From: MOLINARI, DANIELE
To: TOYOTA RESEARCH INSTITUTE, INC.
Reel/Frame 052692/0533 →
Continuity (1)
Related Publication 20210358184A1 · Nov 18, 2021