IP Library › Granted Patent US 11,904,854
Granted Patent B2
US 11,904,854 · App. 16/834,436 · Granted Feb 20, 2024

Systems and methods for modeling pedestrian activity

Inventors: Stephen G. McGill (Broomall, PA); Guy Rosman (Newton, MA); Paul Drews (Watertown, MA)
Assignee: TOYOTA RESEARCH INSTITUTE, INC.
B60W30/0956B60W60/0017G06N3/08G06T7/74G06V10/764G06V10/774G06V20/56G06V20/588G06V40/103B60W2552/45B60W2554/406B60W2554/4029B60W2554/4041B60W2554/4043G06T2207/30256
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,904,854
App. No.
16/834,436
Granted
Feb 20, 2024
Kind
B2
Abstract

A method includes receiving data relating to pedestrian activity at one or more locations outside of a crosswalk, analyzing the data, based on the data, identifying at least one location of the one or more locations as a constructive crosswalk, and controlling operation of an autonomous vehicle based on the at least one location of the constructive crosswalk.

Claims (32)

1. A method comprising:

receiving images of a plurality of road portions;

receiving data relating to pedestrian activity at each of the road portions;

for each of the road portions, determining whether a number of pedestrians that cross a street within the road portion outside of a crosswalk during a predetermined period of time is greater than a predetermined threshold;

labeling each image of a road portion for which the number of pedestrians that cross the street during the predetermined period of time exceeds the predetermined threshold as containing a constructive crosswalk;

labeling each image of a road portion for which the number of pedestrians that cross the street during the predetermined period of time does not exceed the predetermined threshold as not containing a constructive crosswalk;

using the labeled images to train a neural network to receive an input of a road portion and determine a likelihood that the road portion contains a constructive crosswalk where pedestrians are likely to cross a street within the road portion outside of a crosswalk;

capturing an image of a sample road portion with one or more vehicle sensors of an autonomous vehicle;

inputting the image into the trained neural network;

determining a likelihood that one or more locations of the road portion comprise a constructive crosswalk based at least in part on an output of the trained neural network; and

controlling operation of the autonomous vehicle based on the determined likelihood that the one or more locations of the road portion comprise a constructive crosswalk.

2. The method of claim 1 , further comprising determining a pedestrian profile associated with each of the road portions labeled as a constructive crosswalk, the pedestrian profile comprising a most probable path for pedestrians to cross the street at the location of the constructive crosswalk.

3. The method of claim 1 , further comprising determining a pedestrian profile associated with each of the road portions labeled as a constructive crosswalk, the pedestrian profile comprising an average velocity at which pedestrians cross the street at the location of the constructive crosswalk.

4. The method of claim 1 , further comprising determining a pedestrian profile associated with each of the road portions labeled as a constructive crosswalk, the pedestrian profile comprising an average density at which pedestrians cross the street at the location of the constructive crosswalk.

5. The method of claim 1 , further comprising determining a pedestrian profile associated with each of the road portions labeled as a constructive crosswalk, the pedestrian profile comprising a likelihood of pedestrians ceding a right of way to vehicles when crossing the street at the location of the constructive crosswalk.

6. A computing device comprising:

one or more processors;

one or more memory modules; and

machine readable instructions stored in the one or more memory modules that, when executed by the one or more processors, cause the computing device to:

receive data relating to pedestrian activity at each of the road portions;

for each of the road portions, determine whether a number of pedestrians that cross a street within the road portion outside of a crosswalk during a predetermined period of time is greater than a predetermined threshold;

label each image of a road portion for which the number of pedestrians that cross the street during the predetermined period of time exceeds the predetermined threshold as containing a constructive crosswalk;

label each image of a road portion for which the number of pedestrians that cross the street during the predetermined period of time does not exceed the predetermined threshold as not containing a constructive crosswalk;

use the labeled images to train a neural network to receive an input of a road portion and determine a likelihood that the road portion contains a constructive crosswalk where pedestrians are likely to cross a street within the road portion outside of a crosswalk;

capturing an image of a sample road portion with one or more vehicle sensors of an autonomous vehicle;

input the image into the trained neural network;

determine a likelihood that one or more locations of the road portion comprise a constructive crosswalk based at least in part on an output of the trained neural network; and

control operation of the autonomous vehicle based on the determined likelihood that the one or more locations of the road portion comprise a constructive crosswalk.

7. The computing device of claim 6 , wherein the one or more processors further cause the computing device to determine a pedestrian profile associated with each of the road portions labeled as a constructive crosswalk, the pedestrian profile comprising a most probable path for pedestrians to cross the street at the location of the constructive crosswalk.

8. The computing device of claim 6 , wherein the one or more processors further cause the computing device to determine a pedestrian profile associated with each of the road portions labeled as a constructive crosswalk, the pedestrian profile comprising an average velocity at which pedestrians cross the street at the location of the constructive crosswalk.

9. The computing device of claim 6 , wherein the one or more processors further cause the computing device to determine a pedestrian profile associated with each of the road portions labeled as a constructive crosswalk, the pedestrian profile comprising an average density at which pedestrians cross the street at the location of the constructive crosswalk.

10. The computing device of claim 6 , wherein the one or more processors further cause the computing device to determine a pedestrian profile associated with each of the road portions labeled as a constructive crosswalk, the pedestrian profile comprising a likelihood of pedestrians ceding a right of way to vehicles when crossing the street at the location of the constructive crosswalk.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 23, 2024
From: TOYOTA RESEARCH INSTITUTE, INC.
To: TOYOTA JIDOSHA KABUSHIKI KAISHA
Reel/Frame 067189/0563 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 23, 2020
From: MCGILL, STEPHEN G.; ROSMAN, GUY; DREWS, PAUL
To: TOYOTA RESEARCH INSTITUTE, INC.
Reel/Frame 052474/0617 →
Continuity (1)
Related Publication 20210300359A1 · Sep 30, 2021