IP Library › Granted Patent US 12,210,106
Granted Patent B2
US 12,210,106 · App. 17/488,181 · Granted Jan 28, 2025

Method, apparatus, and system for detecting and characterizing parking events based on sensor data

Inventors: Silviu Zilberman (Rishon le-Zion, IL); Harel Primack (Rishon le-Zion, IL)
Assignee: HERE Global B.V.
G01S19/49B60W40/10G01P15/00G07C5/0808G07C5/0816
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,210,106
App. No.
17/488,181
Granted
Jan 28, 2025
Kind
B2
Abstract

An approach is provided for determining a vehicle parking event and respective characteristics using sensor data. The approach, for example, involves receiving sensor data from at least one sensor associated with a mobile device in a vehicle. The approach also involves processing the sensor data to determine a sequence of semantic events. The semantic events respectively indicate a maneuver performed by the vehicle. The approach further involves processing the sensor data to determine a distance estimation over which at least one of the semantic events is performed. The approach further involves detecting a parking event of the vehicle, a characterization of the parking event, or a combination thereof based on the sequence of semantic events and the distance estimation. The approach further involves providing the parking event, the characterization of the parking event, or a combination thereof as an output.

Claims (56)

1. A method comprising:

receiving sensor data from at least one sensor associated with a mobile device in a vehicle, wherein the at least one sensor comprises an accelerometer, a gyroscope, a magnetometer, or a combination thereof that is configured to collect the sensor data for a designated distance or time associated with a parking maneuver performed by the vehicle to minimize an error accumulation in the sensor data over the designated distance or time;

processing the sensor data to determine a sequence of semantic events, wherein the semantic events respectively indicate the parking maneuver performed by the vehicle;

processing the sensor data to determine a distance estimation over which at least one of the semantic events is performed;

detecting a parking event of the vehicle, a characterization of the parking event, or a combination thereof based on the sequence of semantic events and the distance estimation; and

providing the parking event, the characterization of the parking event, or a combination thereof as an output.

2. The method of claim 1 , further comprising:

extracting accelerometer data from the sensor data; and

integrating over the accelerometer data to determine the distance estimation.

3. The method of claim 1 , wherein the detecting of the parking event, the characterization of the parking event, or a combination thereof is based on determining that the distance estimation is below a distance threshold associated with a parking maneuver.

4. The method of claim 3 , wherein the distance threshold is based on a length of the vehicle.

5. The method of claim 3 , wherein the distance threshold is based on a maneuver distance associated with the parking maneuver corresponding to at least one of the semantic events.

6. The method of claim 5 , wherein the parking maneuver corresponding to the at least one of the semantic events includes a forward or reverse maneuver, a left or right turn maneuver, a stopping maneuver, or a combination thereof.

7. The method of claim 1 , wherein the characterization indicates a parking type of the parking event, and wherein the parking type includes a perpendicular parking type, a parallel parking type, an angular parking type, or a combination thereof.

8. The method of claim 1 , further comprising:

extracting gear selection data from the sensor data,

wherein the gear selection data indicates whether the vehicle is in a forward drive gear or a reverse drive gear; and

wherein the semantic events, the sequence of the semantic events, or a combination thereof are further based on the gear selection data.

9. The method of claim 1 , further comprising:

extracting engine activation data from the sensor data,

wherein the engine activation data indicates whether an engine status of the vehicle is on or off; and

wherein the semantic events, the sequence of the semantic events, or a combination thereof are further based on the engine status.

10. The method of claim 1 , further comprising:

extracting air pressure data from the sensor data,

wherein the air pressure data indicates whether a door status of the vehicle is open or close, and

wherein the semantic events, the sequence of the semantic events, or a combination thereof are further based on the door status.

11. The method of claim 1 , wherein the at least one sensor is an inertial measurement unit associated with a mobile device in which a satellite-based location sensor is unavailable or unused.

12. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

receive sensor data from a plurality of sensors associated with at least one mobile device in at least one vehicle, wherein the at least one sensor comprises an accelerometer, a gyroscope, a magnetometer, or a combination thereof that is configured to collect the sensor data for a designated distance or time associated with a parking maneuver performed by the vehicle to minimize an error accumulation in the sensor data over the designated distance or time;

process the sensor data to determine a plurality of parking modes, wherein each of the parking mode includes a time-sequenced semantic events, wherein each of the time-sequenced semantic events indicates the parking maneuver performed by the at least one vehicle;

determine a parking mode of a vehicle based on comparing a time-sequenced semantic events of the vehicle with time-sequenced semantic events of the plurality of parking modes; and

provide the parking mode as an output.

13. The apparatus of claim 12 , wherein the apparatus is further caused to:

extract accelerometer data from the sensor data; and

integrate over the accelerometer data to determine the distance estimation,

wherein the parking mode is further determined based on the distance estimation.

14. The apparatus of claim 13 , wherein the parking mode is based on determining that the distance estimation is below a distance threshold associated with a parking maneuver.

15. The apparatus of claim 14 , wherein the distance threshold is based on a length of the vehicle.

16. The apparatus of claim 12 , wherein the apparatus is further caused to:

extract data of one or more of an accelerometer, a gyroscope, a magnetometer, or a barometer from the sensor data; and

integrate over the data to determine a vehicle idle state, an estimated turning angle, a lane change, a direction of motion, a door state, or a combination thereof of at least one of the time-sequenced semantic events,

wherein the parking mode is further determined based on the vehicle idle state, the estimated turning angle, the lane change, the direction of motion, the door state, or a combination thereof.

17. The apparatus of claim 12 , wherein the plurality of parking modes include one or more of left forward bay parking, right forward bay parking, left reverse bay parking, right reverse bay parking, left forward parallel parking, right forward parallel parking, left reverse parallel parking, right reverse parallel parking, left forward angular parking, right forward angular parking, left reverse angular parking, or right reverse angular parking.

18. A non-transitory computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform:

receiving sensor data from at least one sensor associated with a mobile device in a vehicle, wherein the at least one sensor comprises an accelerometer, a gyroscope, a magnetometer, or a combination thereof that is configured to collect the sensor data for a designated distance or time associated with a parking maneuver performed by the vehicle to minimize an error accumulation in the sensor data over the designated distance or time;

processing the sensor data to determine a sequence of semantic events, wherein the semantic events respectively indicate the parking maneuver performed by the vehicle;

processing the sensor data to determine a distance estimation over which at least one of the semantic events is performed;

detecting a parking event of the vehicle, a characterization of the parking event, or a combination thereof based on the sequence of semantic events and the distance estimation; and

providing the parking event, the characterization of the parking event, or a combination thereof as an output.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the apparatus is caused to further perform:

extracting accelerometer data from the sensor data; and

integrating over the accelerometer data to determine the distance estimation.

20. The non-transitory computer-readable storage medium of claim 18 , wherein the detecting of the parking event, the characterization of the parking event, or a combination thereof is based on determining that the distance estimation is below a distance threshold associated with a parking maneuver.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 29, 2021
From: ZILBERMAN, SILVIU; PRIMACK, HAREL
To: HERE GLOBAL B.V.
Reel/Frame 057643/0143 →
Continuity (1)
Related Publication 20230100346A1 · Mar 30, 2023
References Cited (80)
US 7868784B2 · Kuo · 2011 [cited by examiner]
US 8810434B2 · Groult · 2014 [cited by examiner]
US 8963740B2 · Koukoumidis · 2015 [cited by examiner]
US 9196161B2 · Lai · 2015 [cited by examiner]
US 9448250B2 · Pham et al. · 2016 [cited by applicant]
US 9487101B2 · Kees · 2016 [cited by examiner]
US 9646427B2 · Chen et al. · 2017 [cited by applicant]
US 9812015B1 · Cox · 2017 [cited by examiner]
US 9834222B2 · Shin et al. · 2017 [cited by applicant]
US 10013817B2 · Hasenfratz et al. · 2018 [cited by applicant]
US 10043388B1 · Kahn · 2018 [cited by examiner]
US 10306428B1 · Khanna · 2019 [cited by examiner]
US 10308243B2 · Lavoie · 2019 [cited by examiner]
US 10366610B2 · Ha · 2019 [cited by examiner]
US 10373494B1 · Krysiuk · 2019 [cited by applicant]
US 10410445B2 · Muthusamy · 2019 [cited by examiner]
US 10538149B1 · Dicarlo et al. · 2020 [cited by applicant]
US 10539676B2 · Li et al. · 2020 [cited by applicant]
US 10713943B1 · Mubarek · 2020 [cited by examiner]
US 10733891B2 · Chow · 2020 [cited by examiner]
US 10737690B2 · Lavoie · 2020 [cited by examiner]
US 10814866B2 · Lavoie · 2020 [cited by examiner]
US 10821973B2 · Glebov et al. · 2020 [cited by applicant]
US 10852153B2 · Yun · 2020 [cited by examiner]
US 10974717B2 · Golgiri · 2021 [cited by examiner]
US 11081006B2 · Klochikhin · 2021 [cited by examiner]
US 11373528B2 · Obayashi · 2022 [cited by examiner]
US 11414069B1 · Ko · 2022 [cited by examiner]
US 11482015B2 · Shih · 2022 [cited by examiner]
US 11518464B2 · Begleiter · 2022 [cited by examiner]
US 11676430B2 · Tammali · 2023 [cited by examiner]
US 20070050240A1 · Belani · 2007 [cited by examiner]
US 20110307188A1 · Peng · 2011 [cited by examiner]
US 20120029764A1 · Payne · 2012 [cited by examiner]
US 20130261880A1 · Cho · 2013 [cited by examiner]
US 20130297198A1 · Vande Velde et al. · 2013 [cited by applicant]
US 20140058711A1 · Scofield · 2014 [cited by examiner]
US 20140139359A1 · Paul · 2014 [cited by examiner]
US 20140232569A1 · Skinder · 2014 [cited by examiner]
US 20140278081A1 · Iwuchukwu · 2014 [cited by examiner]
US 20150154711A1 · Christopulos et al. · 2015 [cited by applicant]
US 20150154868A1 · Neuner et al. · 2015 [cited by applicant]
US 20150177362A1 · Gutierrez · 2015 [cited by examiner]
US 20150371541A1 · Korman · 2015 [cited by applicant]
US 20160016590A1 · Fernandez Pozo et al. · 2016 [cited by applicant]
US 20160176408A1 · Lynch · 2016 [cited by examiner]
US 20170118307A1 · Beaurepaire · 2017 [cited by examiner]
US 20180033302A1 · Chan · 2018 [cited by examiner]
US 20180350157A1 · Koreishi · 2018 [cited by examiner]
US 20200132473A1 · Shipley · 2020 [cited by examiner]
US 20200152061A1 · Adelsberger · 2020 [cited by examiner]
US 20200193740A1 · Sabina · 2020 [cited by applicant]
US 20200202209A1 · Mao et al. · 2020 [cited by applicant]
US 20200217669A1 · Hu et al. · 2020 [cited by applicant]
US 20210019671A1 · Cao · 2021 [cited by examiner]
US 20210371030A1 · Begleiter · 2021 [cited by examiner]
US 20220165155A1 · Peng · 2022 [cited by examiner]
US 20220281337A1 · Meessen · 2022 [cited by examiner]
US 20230087202A1 · Lavoie · 2023 [cited by examiner]
US 20230100851A1 · Zilberman · 2023 [cited by examiner]
US 20230104188A1 · Zilberman · 2023 [cited by examiner]
US 20230166725A1 · Tsimhoni · 2023 [cited by examiner]
US 20230185298A1 · Boehm · 2023 [cited by examiner]
US 20230206764A1 · Zilberman · 2023 [cited by examiner]
DE 102012212347A1 · 2014 [cited by applicant]
EP 2302608A2 · 2011 [cited by applicant]
EP 3336488A1 · 2018 [cited by applicant]
EP 3352155A1 · 2018 [cited by applicant]
WO 2013043228A1 · 2013 [cited by applicant]
WO 2018225067A1 · 2018 [cited by applicant]
Naohisa Hashimoto et al., Automated Vehicle Mobile Guidance System for Parking Assistance, 2007, IEEE, pp. 630-635. [cited by examiner]
Massaki Wada et al., Development of Advanced Parking Assistance System, Feb. 2003, IEEE, pp. 1-14 (pdf). [cited by examiner]
Guy Krasner et al., Automatic Parking Identification and Vehicle Guidance with Road Awareness, 2016, ISCEE, pp. 1-5 (pdf). [cited by examiner]
Mahendra B M et al., IoT Based Sensor Enabled Smart Car Parking for Advanced Driver Assistance System, 2017, IEEE, pp. 2188-2193. [cited by examiner]
Liu et al., “A Vehicle Steering Recognition System Based on Low-Cost Smartphone Sensors”, Article, Sensors 2017, 17(3), Published: Mar. 20, 2017, pp. 1-29. [cited by applicant]
Zhao et al., “Smartphone-based Real Time Vehicle Tracking in Indoor Parking Structures”, IEEE Transactions on Mobile Computing, vol. 1, No. 1, Mar. 2017, pp. 1-14. [cited by applicant]
Wikipedia, “Altimeter”, retrieved on Sep. 23, 2021 from https://www.wikiwand.com/en/Altimeter, 3 pages. [cited by applicant]
Srivastava, “Android Activity Recognition API”, Aug. 9, 2020, retrieved from https://abhiappmobiledeveloper.medium.com/android-activity-recognition-api-b7f61847d9dc, 15 pages. [cited by applicant]
Wikipedia, “OpenStreetMap”, retrieved on Sep. 23, 2021 from https://www.wikiwand.com/en/OpenStreetMap, 23 pages. [cited by applicant]
Waze, “Driving Directions. Traffic Reports & Carpool Rideshares by Waze”, retrieved on Sep. 23, 2021 from https://www.waze.com/, 5 pages. [cited by applicant]