Method and system for predicting gesture of subjects surrounding an autonomous vehicle
Embodiments of present disclosure relates to method and gesture prediction system for predicting gesture of subjects for controlling AV during navigation. The gesture prediction system obtains data from sensors and generates context information of environment of AV. The gesture prediction system generates parameters for sampling frames for determining gesture of subjects. The gesture prediction system estimates current pose, and subsequent poses by extrapolating current pose and subsequent poses using deep learning techniques. Further, the gesture prediction system infers current behaviour of subjects by classifying current pose and subsequent poses into one of predefined gesture categories. The gesture prediction system predicts gesture of subjects based on current behaviour, context information and location information of subjects for controlling AV during navigation. Thus, the present disclosure forecast gestures of subjects at a faster rate and controls the AV during the navigation.
1 . A method of predicting a gesture of subjects for controlling an Autonomous Vehicle (AV) during navigation, the method comprising:
obtaining, by a gesture prediction system, data associated with a current environment of the AV from one or more sensors associated with the AV, wherein the data comprises a plurality of frames and location information of the AV;
generating, by the gesture prediction system, context information associated with the current environment of the AV based on the obtained data, wherein the context information comprises information associated with the current environment and information associated with one or more subjects present in the current environment;
generating, by the gesture prediction system, a plurality of parameters based on the location information and the context information for sampling the plurality of frames;
estimating, by the gesture prediction system, a current pose and one or more subsequent poses of each of one or more subjects from each sampled frames using a deep learning technique, wherein estimating the one or more subsequent poses comprising:
predicting the one or more subsequent poses to be performed by each of the one or more subjects based on one or more factors associated with corresponding previous poses of the one or more subjects using the deep learning technique;
inferring, by the gesture prediction system, a gesture category from a plurality of predefined gesture categories, for the one or more subsequent poses of each of the one or more subjects, based on corresponding current pose, the one or more subsequent poses and pre-existing gesture data; and
predicting, by the gesture prediction system, a gesture to be performed by each of the one or more subjects based on respective gesture category, the information associated with the one or more subjects, and the context information, for controlling the AV during navigation.
2 . The method as claimed in claim 1 , wherein the context information further comprises, number of objects present around the AV and information related to the objects, and wherein the information associated with the one or more subjects comprises a category of the one or more subjects, location of the one or more subjects, and movement information of the one or more subjects.
3 . The method as claimed in claim 1 , wherein the plurality of parameters comprises relational parameters and temporal parameters configured based on the context information and location information for sampling the plurality of frames by controlling number of the plurality of frames and a time interval between each of the plurality of frames, respectively.
4 . The method as claimed in claim 1 , wherein predicting the one or more subsequent poses of the one or more subjects comprises:
estimating, by the gesture prediction system, motion associated with the current pose and the one or more subsequent poses estimated corresponding to the one or more subjects; and
predicting, by the gesture prediction system, a subsequent pose of the one or more subjects by extrapolating the respective motion using the deep learning technique.
5 . The method as claimed in claim 1 , wherein the one or more factors for the one or more subjects comprise a number of one or more estimated poses and time interval between each of the one or more estimated poses.
6 . The method as claimed in claim 1 , wherein controlling the AV during navigation comprises:
determining, by the gesture prediction system, a navigational path for the AV dynamically based on a priority of the gesture associated with the one or more subjects surrounding the AV, and the location information.
7 . A gesture prediction system for predicting a gesture of subjects for controlling an Autonomous Vehicle (AV) during navigation, comprising:
a processor; and
a memory communicatively coupled to the processor, wherein the memory stores processor-executable instructions, which, on execution, cause the processor to:
obtain data associated with a current environment of the AV from one or more sensors associated with the AV, wherein the data comprises a plurality of frames and location information of the AV;
generate context information associated with the current environment of the AV based on the obtained data, wherein the context information comprises information associated with the current environment and information associated with one or more subjects present in the current environment;
generate a plurality of parameters based on the location information and the context information for sampling the plurality of frames;
estimate a current pose and one or more subsequent poses of each of one or more subjects from each sampled frames using a deep learning technique, wherein the processor is configured to estimate the one or more subsequent poses by:
predicting the one or more subsequent poses to be performed by each of the one or more subjects based on one or more factors associated with corresponding previous poses of the one or more subjects using the deep learning technique;
infer a gesture category from a plurality of predefined gesture categories, for the one or more subsequent poses of each of the one or more subjects, based on corresponding current pose, the one or more subsequent poses and pre-existing gesture data; and
predict a gesture to be performed by each of the one or more subjects based on respective gesture category, the information associated with the one or more subjects, and the context information, for controlling the AV during navigation.
8 . The gesture prediction system as claimed in claim 7 , wherein the context information further comprises number of objects present around the AV and information related to the objects, and, wherein the information associated with the one or more subjects comprises a category of the one or more subjects, location of the one or more subjects, and movement information of the one or more subjects.
9 . The gesture prediction system as claimed in claim 7 , wherein the plurality of parameters comprises relational parameters and temporal parameters configured based on the context information and location information for sampling the plurality of frames by controlling number of the plurality of frames and a time interval between each of the plurality of frames, respectively.
10 . The gesture prediction system as claimed in claim 7 , wherein the processor is configured to predict the one or more subsequent poses of the one or more subjects by:
estimating motion associated with the current pose and the one or more subsequent poses estimated corresponding to each of the one or more subjects; and
predicting a subsequent pose of each of the one or more subjects by extrapolating the respective motion using the deep learning technique.
11 . The gesture prediction system as claimed in claim 7 , wherein the one or more factors for the one or more subjects comprise a number of one or more estimated poses and time interval between each of the one or more estimated poses.
12 . The gesture prediction system as claimed in claim 7 , wherein the processor controls the AV during navigation by:
determining a navigational path for the AV dynamically based on a priority of the gesture associated with the one or more subjects surrounding the AV, and the location information.
13 . A non-transitory computer readable medium including instruction stored thereon that when processed by at least one processor cause a gesture prediction system to perform operation comprising:
obtaining data associated with a current environment of the AV from one or more sensors associated with the AV, wherein the data comprises a plurality of frames and location information of the AV;
generating context information associated with the current environment of the AV based on the obtained data, wherein the context information comprises information associated with the current environment and information associated with one or more subjects present in the current environment;
generating a plurality of parameters based on the location information and the context information for sampling the plurality of frames;
estimating a current pose and one or more subsequent poses of each of one or more subjects from each sampled frames using a deep learning technique, wherein estimating the one or more subsequent poses comprising:
predicting the one or more subsequent poses to be performed by each of the one or more subjects based on one or more factors associated with corresponding previous poses of the one or more subjects using the deep learning technique;
inferring a gesture category from a plurality of predefined gesture categories, for the one or more subsequent poses of each of the one or more subjects, based on corresponding current pose, the one or more subsequent poses and pre-existing gesture data; and
predicting a gesture to be performed by each of the one or more subjects based on respective gesture category, the information associated with the one or more subjects, and the context information, for controlling the AV during navigation.