IP Library Granted Patent US 11,092,456
Granted Patent B2
US 11,092,456 · App. 16/297,089 · Granted Aug 17, 2021

Object location indicator system and method

Inventors: Nandita Mangal (Palo Alto, CA); Michael H. Laur (Mission Viejo, CA)
Assignee: Aptiv Technologies Limited
G01C21/3632G01S17/08G01S17/89G06K9/00805G06K9/00825H04W4/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,092,456
App. No.
16/297,089
Granted
Aug 17, 2021
Kind
B2
Abstract

A method of operating as object indicator module includes capturing at least one image, said captured image covers at least a portion of an environment of a subject, determining a location of an object relative to the environment of the subject, in response to determining the location of the object, determining modifications of the at least one captured image to generate a modified image that indicates the location of the object relative to the environment, and causing the modified image to be presented on at least one display.

Claims (63)

1. A method comprising:

receiving a first image that is captured by a first perception sensor of an infrastructure device, the first image including a first portion of an environment of a subject, the environment being based on a location of a user device associated with the subject;

determining a location of an object relative to the environment of the subject;

receiving a second image that is captured by a second perception sensor of an infrastructure device, the second image including the object and a second portion of the environment, the second portion of the environment non-overlapping the first portion of the environment;

in response to determining the location of the object and the subject, comparing the content of the first image and the second image, determining a void area based on a previously stored 3D model of the environment, combining the first captured image, the second captured image, and a void area not captured in the first image or the second image to generate a modified image that indicates the location of the object relative to the environment of the subject including a void area depicted by dashed lines that correspond to the curb or boundary of the roadways; and

causing the modified image to be presented on at least one display of a second device located near the subject, the second device being different than the user device.

2. The method of claim 1 , wherein the object is a vehicle and the subject is a pedestrian.

3. The method of claim 1 , wherein the modified image indicates a position of the subject relative to the modified image.

4. The method of claim 1 , wherein generating the modified image includes at least one of:

adjusting a color of a displayed object;

highlighting the displayed object;

generating an arrow pointing towards the object;

generating a distance indicator indicative of a distance between the subject and the object; and

generating a compass indicator indicative of a direction from the subject to the object.

5. The method of claim 1 , wherein determining the location of the object includes at least one of:

receiving global positioning system (GPS) data of the object;

determining a relative location of the object relative to a landmark in the first image or in the second captured image;

receiving a radar data of the object relative to a radar unit; and

receiving a point cloud of the object relative to a lidar unit.

6. The method of claim 1 , wherein the method includes operating a camera to capture the at least one of the first captured image and the second captured image.

7. The method of claim 1 , further comprising:

determining that a distance between the object and the subject is increasing; and in response to determining that the distance is increasing, generating the modified image to convey a notification that the object and subject are moving further apart.

8. A system comprising:

a processor configured to execute an object indicator module that when executed at the processor:

receives a first image that is captured by a first perception sensor of an infrastructure device, the first image including a first portion of an environment of a subject, the environment being based on a location of a user device associated with the subject;

determines a location of an object relative to the environment of the subject;

receives a second image that is captured by a second perception sensor of an infrastructure device, the second image including the object and a second portion of the environment, the second portion of the environment non-overlapping the first portion of the environment;

in response to determining the location of the object and the subject, compares the content of the first image and the second image, determines a void area based on a previously stored 3D model of the environment, and combines the first captured image, the second captured image, and a void area not captured in the first image or the second image to generate a modified image that indicates the location of the object relative to the environment of the subject including a void area depicted by dashed lines that correspond to the curb or boundary of the roadways; and

causes the modified image to be presented on at least one display of a second device located near the subject, the second device being different than the user device.

9. The system of claim 8 , wherein at least one instance of the first perception sensor or the second perception sensor includes a camera.

10. The system of claim 8 , wherein the object is another vehicle and the subject is a pedestrian.

11. The system of claim 8 , wherein the modified image indicates a position of the subject relative to the modified image.

12. The system of claim 8 , wherein the object indicator module, when executed at the processor, further: generates the modified image includes at least one of:

adjusts a color of a displayed object;

highlights the displayed object;

generates an arrow pointing towards the object;

generates a distance indicator indicative of a distance between the subject and the object; and

generates a compass indicator indicative of a direction from the subject to the object.

13. The system of claim 8 , wherein the object indicator module, when executed at the processor, determines the location of the object by at least one of:

receiving global positioning system (GPS) data of the object;

determining a relative location of the object relative to a landmark in the first image or in the second image;

receiving a radar data of the object relative to a radar unit; and

receiving a point cloud of the object relative to a lidar unit.

14. A non-tangible computer readable storage medium that stores instructions that when executed cause a processing device to:

receive a first image that is captured by a first perception sensor of an infrastructure device, the first image including a first portion of an environment of a subject, the environment being based on a location of a user device associated with the subject;

determine a location of an object relative to the environment of the subject;

receives a second image that is captured by a second perception sensor of an infrastructure device, the second image including the object and a second portion of the environment, the second portion of the environment non-overlapping the first portion of the environment;

in response to determining the location of the object and the subject, compare the content of the first image and the second image, determine a void area based on a previously stored 3D model of the environment, and combine the first captured image, the second captured image, and a void area not captured in the first image or the second image to generate a modified image that indicates the location of the object relative to the environment of the subject including a void area depicted by dashed lines that correspond to the curb or boundary of the roadways; and

cause the modified image to be presented on at least one display of a second device located near the subject, the second device being different than the user device.

15. The non-tangible computer readable storage medium of claim 14 , wherein the object is a vehicle and the subject is a pedestrian.

16. The non-tangible computer readable storage medium of claim 14 , wherein the modified image indicates a position of the subject relative to the modified image.

17. The non-tangible computer readable storage medium of claim 14 , wherein the instructions, when executed, further cause the processing device to:

generate the modified image includes at least one of:

adjust a color of a displayed object; highlight the displayed object;

generate an arrow pointing towards the object;

generate a distance indicator indicative of a distance between the subject and the object; and

generate a compass indicator indicative of a direction from the subject to the object.

18. The non-tangible computer readable storage medium of claim 14 , wherein the instructions, when executed, further cause the processing device to determine the location of the object by at least one of:

receiving global positioning system (GPS) data of the object;

determining a relative location of the object relative to a landmark in the first image or in the second image;

receiving a radar data of the object relative to a radar unit; and

receiving a point cloud of the object relative to a lidar unit.

19. The non-tangible computer readable storage medium of claim 14 , wherein at least one instance of the first perception sensor or the second perception sensor includes a camera.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 11, 2024
From: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
To: APTIV TECHNOLOGIES AG
Reel/Frame 066551/0219 →
MERGER Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES (2) S.À R.L.
To: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
Reel/Frame 066566/0173 →
ENTITY CONVERSION Recorded Feb 11, 2024
From: APTIV TECHNOLOGIES LIMITED
To: APTIV TECHNOLOGIES (2) S.À R.L.
Reel/Frame 066746/0001 →
RELEASE OF SECURITY INTEREST Recorded Dec 6, 2021
From: COMERICA BANK
To: AVCTECHNOLOGIES USA INC.; KANDY COMMUNICATIONS LLC
Reel/Frame 058312/0555 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 8, 2019
From: MANGAL, NANDITA; LAUR, MICHAEL H.
To: APTIV TECHNOLOGIES LIMITED
Reel/Frame 048549/0971 →