Method and system for ambient intelligence based user interaction
This disclosure relates generally to system and method for ambient intelligence based user interaction. Prior methods for touchless user interaction are sensitive to ambient temperature in a lab environment, susceptible to noise from metallic surfaces and ambient radio waves and are dependent on ambient lighting. Embodiments of the present disclosure provides a multi-modal sensor fusion method which captures touchless gestures from a user or a group of users with their physical context information fused and tagged to these gestures for user interaction. Further pose graphs are generated for user interaction systems using a data association technique and Gaussian mixture model technique. The disclosed method provides a hands-free interface to operate instruments in a smart space, using principles of ambient intelligence.
1. A processor implemented method comprising the steps of:
receiving, via one or more hardware processors, (i) a set of raw point cloud from a Light Detection and Ranging (LiDAR) sensor (ii) a set of XY coordinates and (iii) a set of tag IDs corresponding to a set of users captured from a real time locating system sensor in an ambient environment for user interaction;
obtaining, via the one or more hardware processors, a set of point cloud by filtering the set of raw point cloud;
computing, via the one or more hardware processors, a set of point clusters from the set of point cloud using a segmentation algorithm, wherein each point cluster comprises a cluster center and an XY coordinate;
converting, via the one or more hardware processors, the set of point clusters to a set of tagged sub point clouds based on a cost matrix and a data association algorithm;
segmenting, via the one or more hardware processors, the set of tagged sub point clouds to obtain a set of tagged sub point cloud segments corresponding to a plurality of body parts of the set of users; and
generating, via the one or more hardware processors, a set of pose graphs corresponding to the set of tagged sub point cloud segments using a Gaussian Mixture Model technique, wherein the pose graph is being utilized for user interaction.
2. The method of claim 1 , wherein the step of converting the set of point clusters to the set of tagged sub point clouds comprises:
computing a Euclidean distance vector between the set of XY coordinates and the set of cluster centers;
obtaining the cost matrix from the Euclidean distance vector and the set of tag IDs; and
obtaining the set of tagged sub point clouds by applying the data association algorithm on the cost matrix.
3. The method of claim 1 , wherein the set of raw point cloud is filtered by removing noise and background information from the set of raw point cloud.
4. The method of claim 1 , wherein the user interaction is a touchless user interaction.
5. The method of claim 1 , wherein the step of generating the set of pose graphs comprises,
obtaining a set of first point cloud corresponding to the set of tagged sub point cloud segments by projecting the XY coordinate of the set of tagged sub point cloud segments;
down sampling the set of first point cloud to obtain a set of second point cloud;
applying the Gaussian mixture model technique on the set of second point cloud to obtain a set of gaussian clusters and a set of gaussian means; and
performing a topological based data analysis on the set of gaussian clusters and the set of gaussian means to generate the set of pose graphs.
6. A system, comprising:
a memory storing instructions;
one or more communication interfaces; and
one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
receive (i) a set of raw point cloud from a Light Detection and Ranging (LiDAR) sensor (ii) a set of XY coordinates and (iii) a set of tag IDs corresponding to a set of users captured from a real time locating system sensor in an ambient environment for user interaction;
obtain a set of point cloud by filtering the set of raw point cloud;
compute a set of point clusters from the set of point cloud using a segmentation algorithm, wherein each point cluster comprises a cluster center and an XY coordinate;
convert the set of point clusters to a set of tagged sub point clouds based on a cost matrix and a data association algorithm;
segment the set of tagged sub point clouds to obtain a set of tagged sub point cloud segments corresponding to a plurality of body parts of the set of users; and
generate a set of pose graphs corresponding to the set of tagged sub point cloud segments using a Gaussian Mixture Model technique, wherein the pose graph is being utilized for user interaction.
7. The system of claim 6 , wherein the one or more hardware processors are configured to convert the set of point clusters to the set of tagged sub point clouds by:
computing a Euclidean distance vector between the set of XY coordinates and the set of cluster centers;
obtaining the cost matrix from the Euclidean distance vector and the set of tag IDs; and
obtaining the set of tagged sub point clouds by applying the data association algorithm on the cost matrix.
8. The system of claim 6 , wherein the set of raw point cloud is filtered by removing noise and background information from the set of raw point cloud.
9. The system of claim 6 , wherein the user interaction is a touchless user interaction.
10. The system of claim 6 , wherein the one or more hardware processors are configured to generate the set of pose graphs by,
obtaining a set of first point cloud corresponding to the set of tagged sub point cloud segments by projecting the XY coordinate of the set of tagged sub point cloud segments;
down sampling the set of first point cloud to obtain a set of second point cloud;
applying the Gaussian mixture model technique on the set of second point cloud to obtain a set of gaussian clusters and a set of gaussian means; and
performing a topological based data analysis on the set of gaussian clusters and the set of gaussian means to generate the set of pose graphs.
11. One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
receiving, (i) a set of raw point cloud from a Light Detection and Ranging (LiDAR) sensor (ii) a set of XY coordinates and (iii) a set of tag IDs corresponding to a set of users captured from a real time locating system sensor in an ambient environment for user interaction;
obtaining a set of point cloud by filtering the set of raw point cloud;
computing a set of point clusters from the set of point cloud using a segmentation algorithm, wherein each point cluster comprises a cluster center and an XY coordinate;
converting the set of point clusters to a set of tagged sub point clouds based on a cost matrix and a data association algorithm;
segmenting the set of tagged sub point clouds to obtain a set of tagged sub point cloud segments corresponding to a plurality of body parts of the set of users; and
generating a set of pose graphs corresponding to the set of tagged sub point cloud segments using a Gaussian Mixture Model technique, wherein the pose graph is being utilized for user interaction.
12. The one or more non-transitory machine-readable information storage mediums of claim 11 , wherein the step of converting the set of point clusters to the set of tagged sub point clouds comprises:
computing a Euclidean distance vector between the set of XY coordinates and the set of cluster centers;
obtaining the cost matrix from the Euclidean distance vector and the set of tag IDs; and
obtaining the set of tagged sub point clouds by applying the data association algorithm on the cost matrix.
13. The one or more non-transitory machine-readable information storage mediums of claim 11 , wherein the set of raw point cloud is filtered by removing noise and background information from the set of raw point cloud.
14. The one or more non-transitory machine-readable information storage mediums of claim 11 , wherein the user interaction is a touchless user interaction.
15. The one or more non-transitory machine-readable information storage mediums of claim 11 , wherein the step of generating the set of pose graphs comprises,
obtaining a set of first point cloud corresponding to the set of tagged sub point cloud segments by projecting the XY coordinate of the set of tagged sub point cloud segments;
down sampling the set of first point cloud to obtain a set of second point cloud;
applying the Gaussian mixture model technique on the set of second point cloud to obtain a set of gaussian clusters and a set of gaussian means; and
performing a topological based data analysis on the set of gaussian clusters and the set of gaussian means to generate the set of pose graphs.