IP Library Granted Patent US 12,430,889
Granted Patent B2
US 12,430,889 · App. 16/794,218 · Granted Sep 30, 2025

Distinguishing gesture actions among transport occupants

Inventors: Sachin J. Ahire (Frisco, TX); Manoj Kalamkar (McKinney, TX); Christopher J. Risberg (Flower Mound, TX)
Assignee: TOYOTA MOTOR NORTH AMERICA, INC.
G06V10/765G06F18/2163G06V20/59G06V40/20B60W50/10H04L9/0637
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,430,889
App. No.
16/794,218
Granted
Sep 30, 2025
Kind
B2
Abstract

An example operation includes one or more of detecting a gesture in a transport performed by a transport occupant, determining an occupant status associated with the transport occupant, determining whether the occupant status permits the gesture to be performed, and performing an action associated with the gesture, when the occupant status permits the gesture to be performed.

Claims (66)

1. A method, comprising:

detecting a gesture in a transport performed by a transport occupant;

determining a numerical gesture score associated with the gesture;

determining an occupant status associated with the transport occupant;

determining whether the occupant status permits the gesture to be performed;

determining whether a sum of the numerical gesture score and a numerical current transport operation score exceeds a numerical maximum transport safe operation score threshold, wherein the numerical current transport score is based on one or more components of the transport which are currently being operated and represent one or more numerical scores contributing to the numerical current transport score, and wherein a summed score that exceeds the numerical maximum transport safe operation score threshold is deemed to acceptable to safe transport operation; and

performing an action, by the transport, associated with the gesture, when the occupant status permits the gesture to be performed and the sum of the numerical gesture score and the numerical current transport operation score does not cause the transport to exceed the numerical maximum transport safe operation score threshold indicating the transport can be safely operated.

2. The method of claim 1 , comprising:

identifying a gesture location of the gesture within the transport.

3. The method of claim 1 , comprising:

determining whether the occupant status indicates the occupant is currently operating a mobile device;

when the occupant status indicates the occupant is currently operating a mobile device, cancelling the gesture.

4. The method of claim 2 , comprising:

determining whether the gesture location was within one of a plurality of defined voluminous areas inside the transport where gestures are detected; and

when the gesture location is within one of the plurality of voluminous areas, identifying the action associated with the gesture.

5. The method of claim 2 , comprising:

determining whether the gesture location was within one of a plurality of defined voluminous areas inside the transport where gestures are detected; and

when the gesture location is not within one of the plurality of voluminous areas, cancelling the gesture.

6. The method of claim 2 , wherein identifying the gesture location comprises

determining whether the gesture is performed in at least one priority location within the transport; and

when the gesture is performed in the at least one priority location within the transport, performing the action.

7. The method of claim 2 , wherein identifying the gesture location comprises

determining whether the gesture is performed in at least one priority location within the transport; and

when the gesture is not performed in the at least one priority location within the transport, determining whether the action is a transport operation action or a non-transport operation action, and when the action is not a transport operation action, performing the action.

8. A transport, comprising:

a computing device configured to:

detect a gesture in the transport performed by a transport occupant;

determine a numerical gesture score associated with the gesture;

determine an occupant status associated with the transport occupant;

determine whether the occupant status permits the gesture to be performed;

determine whether a sum of the numerical gesture score and a numerical current transport operation score exceeds a numerical maximum transport safe operation score threshold, wherein the numerical current transport score is based on one or more components of the transport which are currently being operated and represent one or more numerical scores contributing to the numerical current transport score, and wherein a summed score that exceeds the numerical maximum transport safe operation score threshold is deemed to unacceptable to safe transport operation; and

perform an action associated with the gesture, when the occupant status permits the gesture to be performed and the sum of the numerical gesture score and the numerical current transport operation score does not cause the transport to exceed the numerical maximum transport safe operation score threshold indicating the transport can be safely operated.

9. The transport of claim 8 , wherein the computing device is further configured to

identify a gesture location of the gesture within the transport.

10. The transport of claim 8 , wherein the computing device is further configured to

determine whether the occupant status indicates the occupant is currently operating a mobile device; and

when the occupant status indicates the occupant is currently operating a mobile device, the computing device cancels the gesture.

11. The transport of claim 9 , wherein the computing device is further configured to

determine whether the gesture location was within one of a plurality of defined voluminous areas inside the transport where gestures are detected; and

when the gesture location is within one of the plurality of voluminous areas, identify the action associated with the gesture.

12. The transport of claim 9 , wherein the computing device is further configured to

determine whether the gesture location was within one of a plurality of defined voluminous areas inside the transport where gestures are detected; and

when the gesture location is not within one of the plurality of voluminous areas, cancel the gesture.

13. The transport of claim 9 , wherein the computing device identifies the gesture location by being further configured to determine whether the gesture is performed in at least one priority location within the transport; and

when the gesture is performed in the at least one priority location within the transport, the computing device is configured to perform the action.

14. The transport of claim 9 , wherein the computing device identifies the gesture location by the computing device being configured to determine whether the gesture is performed in at least one priority location within the transport; and

when the gesture is not performed in the at least one priority location within the transport, the computing device determines whether the action is a transport operation action or a non-transport operation action, and when the action is not a transport operation action, the computing device performs the action.

15. A non-transitory computer readable storage medium configured to store instructions that when read by a process, cause the processor to perform:

detecting a gesture in a transport performed by a transport occupant;

determining a numerical gesture score associated with the gesture;

determining an occupant status associated with the transport occupant;

determining whether the occupant status permits the gesture to be performed;

determining whether a sum of the numerical gesture score and a numerical current transport operation score exceeds a numerical maximum transport safe operation score threshold, wherein the numerical current of transport score is based on one or more components of the transport which are currently being operated and represent one or more numerical scores contributing to the numerical current transport score, and wherein a summed score that exceeds the numerical maximum transport safe ration score threshold is deemed to unacceptable to safe transport operation; and

performing an action, by the transport, associated with the gesture, when the occupant status permits the gesture to be performed and the sum of the numerical gesture score and the numerical current transport operation score does not cause the transport to exceed the numerical maximum transport safe operation score threshold indicating the transport can be safely operated.

16. The non-transitory computer readable storage medium of claim 15 , wherein the processor is further configured to perform:

identifying a gesture location of the gesture within the transport.

17. The non-transitory computer readable storage medium of claim 16 , wherein the processor is further configured to perform:

determining whether the gesture location was within one of a plurality of defined voluminous areas inside the transport where gestures are detected; and

when the gesture location is within one of the plurality of voluminous areas, identifying the action associated with the gesture.

18. The non-transitory computer readable storage medium of claim 16 , wherein the processor is further configured to perform:

determining whether the gesture location was within one of a plurality of defined voluminous areas inside the transport where gestures are detected; and

when the gesture location is not within one of the plurality of voluminous areas, cancelling the gesture.

19. The non-transitory computer readable storage medium of claim 16 , wherein identifying the gesture location comprises determining whether the gesture is performed in at least one priority location within the transport; and

when the gesture is performed in the at least one priority location within the transport, performing the action.

20. The non-transitory computer readable storage medium of claim 16 , wherein identifying the gesture location comprises determining whether the gesture is performed in at least one priority location within the transport; and

when the gesture is not performed in the at least one priority location within the transport, determining whether the action is a transport operation action or a non-transport operation action, and when the action is not a transport operation action, performing the action.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 19, 2020
From: AHIRE, SACHIN J.; KALAMKAR, MANOJ; RISBERG, CHRISTOPHER J.
To: TOYOTA MOTOR NORTH AMERICA, INC.
Reel/Frame 051851/0768 →
Continuity (1)
Related Publication 20210256316A1 · Aug 19, 2021
References Cited (163)
US 6926374B2 · Dudeck et al. · 2005 [cited by applicant]
US 7110570B1 · Berenz et al. · 2006 [cited by applicant]
US 7130448B2 · Nagaoka et al. · 2006 [cited by applicant]
US 7202776B2 · Breed · 2007 [cited by applicant]
US 7301464B2 · Coulter · 2007 [cited by applicant]
US 7380948B2 · Schofield et al. · 2008 [cited by applicant]
US 7671764B2 · Uyeki et al. · 2010 [cited by applicant]
US 7720580B2 · Higgins-Luthman · 2010 [cited by applicant]
US 7843313B2 · Bazakos et al. · 2010 [cited by applicant]
US 7899616B2 · Breed · 2011 [cited by applicant]
US 7925423B2 · Howarter et al. · 2011 [cited by applicant]
US 7961094B2 · Breed · 2011 [cited by applicant]
US 8027029B2 · Lu et al. · 2011 [cited by applicant]
US 8284041B2 · Cuddihy et al. · 2012 [cited by applicant]
US 8396252B2 · Dokor · 2013 [cited by applicant]
US 8723687B2 · Thomas · 2014 [cited by applicant]
US 8725311B1 · Breed · 2014 [cited by applicant]
US 8744645B1 · Vaghefinazari et al. · 2014 [cited by applicant]
US 8761998B2 · Chen et al. · 2014 [cited by applicant]
US 8849494B1 · Herbach et al. · 2014 [cited by applicant]
US 8855900B2 · Lection et al. · 2014 [cited by applicant]
US 8942881B2 · Hobbs et al. · 2015 [cited by applicant]
US 9152853B2 · Dokor · 2015 [cited by applicant]
US 9182233B2 · Weng et al. · 2015 [cited by applicant]
US 9244527B2 · Lathrop et al. · 2016 [cited by applicant]
US 9332261B2 · Yousefi et al. · 2016 [cited by applicant]
US 9342797B2 · Dokor et al. · 2016 [cited by applicant]
US 9493130B2 · Penilla et al. · 2016 [cited by applicant]
US 9507346B1 · Levinson et al. · 2016 [cited by applicant]
US 9545930B2 · Ricci et al. · 2017 [cited by applicant]
US 9551590B2 · Yan et al. · 2017 [cited by applicant]
US 9711050B2 · Ansari · 2017 [cited by applicant]
US 9714037B2 · Deruyck et al. · 2017 [cited by applicant]
US 9734710B2 · Kang · 2017 [cited by applicant]
US 9761136B2 · Tonguz et al. · 2017 [cited by applicant]
US 9771070B2 · Zagorski et al. · 2017 [cited by applicant]
US 9910441B2 · Levinson et al. · 2018 [cited by applicant]
US 9946262B2 · Ansari · 2018 [cited by applicant]
US 9949107B2 · Gehrling et al. · 2018 [cited by applicant]
US 9952680B2 · Ricci · 2018 [cited by applicant]
US 9964414B2 · Slavin et al. · 2018 [cited by applicant]
US 9965169B2 · Graumann · 2018 [cited by applicant]
US 9994229B2 · Ricci · 2018 [cited by applicant]
US 10120385B2 · Connor · 2018 [cited by applicant]
US 10198619B1 · Zhu et al. · 2019 [cited by applicant]
US 10308258B2 · Fung et al. · 2019 [cited by applicant]
US 10372129B1 · Urmson et al. · 2019 [cited by applicant]
US 10387725B2 · Zink et al. · 2019 [cited by applicant]
US 10409282B2 · Tagnemma et al. · 2019 [cited by applicant]
US 10572717B1 · Zhu et al. · 2020 [cited by applicant]
US 10620232B2 · Tu et al. · 2020 [cited by applicant]
US 10683014B2 · Akaba et al. · 2020 [cited by applicant]
US 10699326B2 · Ricci · 2020 [cited by applicant]
US 10768620B1 · Tran · 2020 [cited by applicant]
US 10850693B1 · Pertsel et al. · 2020 [cited by applicant]
US 10948907B2 · Miller et al. · 2021 [cited by applicant]
US 11067986B2 · Umetani et al. · 2021 [cited by applicant]
US 11068069B2 · Page et al. · 2021 [cited by applicant]
US 11345368B2 · Tian et al. · 2022 [cited by applicant]
US 11347225B2 · Tatourian · 2022 [cited by applicant]
US 11873000B2 · Ahire et al. · 2024 [cited by applicant]
US 20060155427A1 · Yang · 2006 [cited by applicant]
US 20070195997A1 · Paul et al. · 2007 [cited by applicant]
US 20100235034A1 · Higgins · 2010 [cited by applicant]
US 20130155237A1 · Paek et al. · 2013 [cited by applicant]
US 20130162449A1 · Ginsberg · 2013 [cited by applicant]
US 20130204457A1 · King et al. · 2013 [cited by applicant]
US 20130261871A1 · Hobbs et al. · 2013 [cited by applicant]
US 20140223384A1 · Graumann · 2014 [cited by applicant]
US 20140309806A1 · Ricci · 2014 [cited by examiner]
US 20140309871A1 · Ricci · 2014 [cited by applicant]
US 20150131857A1 · Han et al. · 2015 [cited by applicant]
US 20150375756A1 · Do et al. · 2015 [cited by applicant]
US 20160103499A1 · Yamamoto et al. · 2016 [cited by applicant]
US 20160176372A1 · Kim · 2016 [cited by applicant]
US 20160281620A1 · Nakade et al. · 2016 [cited by applicant]
US 20160357187A1 · Ansari · 2016 [cited by applicant]
US 20160357262A1 · Ansari · 2016 [cited by applicant]
US 20160362117A1 · Kaufmann et al. · 2016 [cited by applicant]
US 20170006151A1 · Doorandish · 2017 [cited by applicant]
US 20170010679A1 · Hillis et al. · 2017 [cited by applicant]
US 20170017392A1 · Castaneda et al. · 2017 [cited by applicant]
US 20170068438A1 · Ricci · 2017 [cited by applicant]
US 20170120930A1 · Ling · 2017 [cited by examiner]
US 20170123428A1 · Levinson et al. · 2017 [cited by applicant]
US 20170161575A1 · Banno et al. · 2017 [cited by applicant]
US 20170269695A1 · Tokish et al. · 2017 [cited by applicant]
US 20170282717A1 · Jang et al. · 2017 [cited by applicant]
US 20170315824A1 · Ochiai et al. · 2017 [cited by applicant]
US 20180061237A1 · Erickson et al. · 2018 [cited by applicant]
US 20180155967A1 · Peng et al. · 2018 [cited by applicant]
US 20180210463A1 · Switkes et al. · 2018 [cited by applicant]
US 20180210464A1 · Switkes et al. · 2018 [cited by applicant]
US 20180275652A1 · Kuwahara et al. · 2018 [cited by applicant]
US 20180288586A1 · Tran et al. · 2018 [cited by applicant]
US 20180342036A1 · Zachary · 2018 [cited by applicant]
US 20190004614A1 · Mori et al. · 2019 [cited by applicant]
US 20190025815A1 · Saadoun et al. · 2019 [cited by applicant]
US 20190052747A1 · Breaux · 2019 [cited by examiner]
US 20190065873A1 · Wang et al. · 2019 [cited by applicant]
US 20190070963A1 · Jang et al. · 2019 [cited by applicant]
US 20190217868A1 · Kim et al. · 2019 [cited by applicant]
US 20190302895A1 · Jiang et al. · 2019 [cited by applicant]
US 20190372986A1 · Singh · 2019 [cited by applicant]
US 20200208460A1 · Ma et al. · 2020 [cited by applicant]
US 20200334477A1 · Aoi et al. · 2020 [cited by applicant]
US 20200339133A1 · Olsson et al. · 2020 [cited by applicant]
US 20200349937A1 · Li · 2020 [cited by applicant]
US 20210118060A1 · Floyd et al. · 2021 [cited by applicant]
US 20210182019A1 · Li · 2021 [cited by applicant]
US 20210197856A1 · Hoshina et al. · 2021 [cited by applicant]
US 20210248359A1 · Du et al. · 2021 [cited by applicant]
US 20210253120A1 · Ahire et al. · 2021 [cited by applicant]
US 20210253135A1 · Ahire et al. · 2021 [cited by applicant]
US 20210256316A1 · Ahire et al. · 2021 [cited by applicant]
US 20210293567A1 · Bill et al. · 2021 [cited by applicant]
US 20210300423A1 · Ahire et al. · 2021 [cited by applicant]
US 20210311554A1 · McFarland, Jr. · 2021 [cited by applicant]
US 20220031197A1 · McFarland, Jr. · 2022 [cited by applicant]
US 20220126864A1 · Moustafa et al. · 2022 [cited by applicant]
US 20220135050A1 · Egashira et al. · 2022 [cited by applicant]
US 20220331028A1 · Sternitzke et al. · 2022 [cited by applicant]
US 20230053103A1 · Roberts et al. · 2023 [cited by applicant]
US 20230194663A1 · Zhang · 2023 [cited by applicant]
CN 101414407A · 2009 [cited by applicant]
CN 101281685B · 2010 [cited by applicant]
CN 101475002B · 2014 [cited by applicant]
CN 102866803B · 2016 [cited by applicant]
CN 105313835A · 2016 [cited by applicant]
CN 106406514A · 2017 [cited by applicant]
CN 104271420B · 2017 [cited by applicant]
CN 103998316B · 2017 [cited by applicant]
CN 104620257B · 2017 [cited by applicant]
CN 107851395A · 2018 [cited by applicant]
CN 108248406A · 2018 [cited by applicant]
CN 109895788A · 2019 [cited by applicant]
CN 110045825A · 2019 [cited by applicant]
CN 113696904A · 2021 [cited by applicant]
CN 115352450A · 2022 [cited by applicant]
DE 60106296 · 2004 [cited by applicant]
DE 102010013232 · 2016 [cited by applicant]
DE 102016219795A1 · 2018 [cited by applicant]
DE 102017216000A1 · 2019 [cited by applicant]
EP 1542194A1 · 2005 [cited by applicant]
EP 1271179B1 · 2006 [cited by applicant]
EP 2216225A1 · 2010 [cited by applicant]
EP 2228779A2 · 2010 [cited by applicant]
EP 2673156A1 · 2013 [cited by applicant]
ES 2391556 · 2012 [cited by applicant]
FR 3095063A1 · 2020 [cited by applicant]
JP 4028135B2 · 2007 [cited by applicant]
JP 5016022B2 · 2012 [cited by applicant]
JP 2013544696A · 2013 [cited by applicant]
JP 2015125778A · 2015 [cited by applicant]
JP 6051162B2 · 2016 [cited by applicant]
JP 6126199B2 · 2017 [cited by applicant]
JP 6218200B2 · 2017 [cited by applicant]
KR 101210875B1 · 2012 [cited by applicant]
KR 20150073857A · 2015 [cited by applicant]
KR 101542502B1 · 2015 [cited by applicant]
KR 101614677B1 · 2016 [cited by applicant]
KR 20180009795A · 2018 [cited by applicant]
WO 2012002645A2 · 2012 [cited by applicant]