Tracking, analysing and assessment of human body movements using a subject-specific digital twin model of the human body
The invention relates to systems and methods for the capture, tracking, analysis and assessment of human movements, action or behavior using novel digital quantum twin model of the human. The model is integrated and reinforced by novel representations and novel techniques in computer vision, machine learning, speech processing, sport science, exercise and health. The aim is to achieve optimal analysis and assessment of human motion and other impacting internal and external forces using valid quantum physics-based model of the human combining movements, behaviors, and other health info. Unlike existing approaches derived from two-or three-dimensional landmarks or just the shape or composition e.g. those extracted from images, videos or sensors, this invention develops an accurate finite element-like quantum representations of human-specific body combining shape features, anatomical structure, internal particles, their intensity, classifications and is constraint by clinical, physical, and biomechanical characteristics of the body and forces affecting each particle.
1 . A device for tracking, modelling, analysing and assessment of a human body movement or a specific human body part; the device comprising:
a storage storing a dataset of diverse human data comprising one or more of imageries, videos, shapes, movements, motions, forces, moments, and relevant human health, physical, anatomical, clinical or biomechanical data of a full human body or a part thereof;
a computing controller integrated with storage for storing electronic programs and instructions for operating the controller and executing specific applications and processes to process one or more of the human data and creating a deformable, dynamic, scalable, digital twin model of the human body co-registered shape and composition and its movement patterns and movements limitations constrained by the human body anatomical, physical and biomechanical characteristics, to form a digital quantum model of a human and a quantum human graph;
the step of creating the deformable, dynamic, scalable, digital twin model of the human body comprising a computer integrated with;
(i) computer vison, statistical and machine learning models developed to facilitate slicing, matching and alignment of the human body shape, parts, data and images and its corresponding medical images, scans, particles, forces, moments, kinetic, kinematics and external environmental affecting factors,
(ii) a storage storing the developed digital quantum human twin and;
(iii) input device operable to capture or receive one or more input data of the human movement data, human body photographs, frames, videos including audio, depth, static and dynamic shapes, or other sensory data or predicted human data;
the developed digital quantum human twin is digitized into particles and is optimized in a form able to facilitate its integration and application in desktops, smart devices and systems-on-chips, process the captured or received input together with the developed digital human twin and an offline dataset to facilitate accurate tracking, analyses and assessment of the human body movement or a specific human body part and generate an output;
the output can be real-time or non-real-times and includes evaluation, analyses or assessment of one or more human movement patterns and any driven criteria, score, indicator, performance, forces, inertia, angles, stress, twist, speed, acceleration, orientation, momentum and risk of the body as a whole or in-parts; communicate the output;
wherein the data is pre-processed and inspected subjected to standardised clinical, physical, anatomical, biomedical and biomechanical constraints and limits relevant to the specific human body and its digital twin using advance statistical techniques and/or machine learning techniques and/or AI techniques to score each specific data type and its contribution in the analysis and movement of the human body or part thereof.
2 . A device according to claim 1 , wherein the input data comprises one or more of marker or markerless data, information, formulas or features or a mathematical representation extracted from imageries, shapes or videos of the human body or a part thereof and the output can be the human body or part thereof, speed, acceleration, angles, joint angles, forces, joint forces, moments, joint moments kinetics, kinematics, or other analysis or assessment derived from one or more resource and include physical, clinical, health and sport performance assessment.
3 . A device according to claim 1 , wherein the input data comprises one or more of invasive, non-invasive or other external input or subjects or weight or forces or collision affecting the human body and its movement and contributing to the analysis and assessment of the human body, or a part thereof.
4 . A device according to claim 1 , wherein the assessment and analysis processes comprises output performed by at least one computer vision (CV) approach, machine learning (ML), and/or artificial intelligence (AI) model.
5 . A device according to claim 1 , wherein the movement of the human body being assessed belongs to a category or a group or a class of bodies and movement patterns, and the database comprises details of a plurality of different movements of the same or different human bodies belonging to the same, and/or a similar, category or group or class to that of the human body movement being analysed and assessed using the digital human twin.
6 . A device according to claim 5 , wherein the human body and movement data and/or information comprises one or more of human motion or movement marker and markerless data, videos, photos, multiviews, full and/or partial body shapes or 3D surface scans, anthropometry, characteristics, attributes medical body composition imaging and health data, medical epidemiological and physiology information.
7 . A device according to claim 1 , comprising a display for displaying a user interface, wherein the controller is operable and guided by electronic program instructions, to communicate the output by displaying the output via the display depicting a visualization of the analysis and assessment of the human body movement pattern and positions via at least one of text, images, graphs, spreadsheets, 3D or multi-layered meshes, landmarks, avatar, or 3D pointclouds, heatmaps, videos, or virtual reality, and finite element analysis of particles.
8 . A device according to claim 1 , wherein the human body or the part thereof is that of an individual person, and the output comprises an estimate of the individual person's: movements, motion, speed, acceleration, angles, joint angles, forces, joint forces, moments, joint moments kinetics, kinematics, or other analysis or assessment derived from one or more of these including physical, clinical, health and sport performance assessment, functional movement assessment or musculoskeletal assessment, clinical biomechanics assessment including pathological populations, injury prevention, motor development assessment including monitoring motor skill development in children or assessing people with Parkinson's disease, measurement of physical activity and automatically monitoring mentally ill patient or elders and other wellness and other relevant health and risk indicators.
9 . A method for tracking, modelling, analysing and assessment of a human body movement or a specific human body part; the method comprising:
a storage storing in a storage a dataset of diverse human data comprising one or more of imageries, videos, shapes, movements, motions, forces, moments, and relevant human health, physical, anatomical, clinical or biomechanical data of a full human body or a part thereof;
operating a controller and executing specific applications, and processes to process one or more of the human data and create a deformable, dynamic, scalable, digital twin module of the human body co-registered shape and composition and its movement patterns and movements limitations constrained by the human body anatomical, physical and biomechanical characteristics, to form a digital quantum model of a human and a quantum human graph;
creating the digital quantum human twin comprising;
(iv) using computer vison, statistical and machine learning models developed to facilitate slicing, matching and alignment of the human body shape, parts, data and images and its corresponding medical images, scans, particles, forces, moments, kinetic, kinematics and external environmental affecting factors,
(v) storing the developed digital quantum human twin and;
(vi) capturing or receiving with an input one or more of the human movement data, human body photographs, frames, videos including audio, depth, static and dynamic shapes, or other sensory data or predicted human data by existing technologies;
digitizing the developed digital quantum human twin into particles in an optimized form able to facilitate its integration and application in desktops, smart devices and systems-on-chips, process the captured or received input together with the developed digital human twin and an offline dataset to facilitate accurate tracking, analyses and assessment of the human body movement or a specific human body part and generate an output;
the output can be real-time or non-real-times and includes evaluation, analyses or assessment of one or more human movement patterns and any driven criteria, score, indicator, performance, forces, inertia, angles, stress, twist, speed, acceleration, orientation, momentum and risk of the body as a whole or in-parts;
communicating the output;
wherein the data is pre-processed and inspected subjected to standardised clinical, physical, anatomical, biomedical and biomechanical constraints and limits relevant to the specific human body and its digital twin using advance statistical techniques and/or machine learning techniques and/or AI techniques to score each specific data type and its contribution in the analysis and movement of the human body or part thereof.
10 . A method according to claim 9 , wherein the input can be marker or markerless data, information, formulas or features or a mathematical representation extracted from imageries, shapes or videos of the human body or a part thereof and the output can be the human body or part thereof, speed, acceleration, angles, joint angles, forces, joint forces, moments, joint moments kinetics, kinematics, or other analysis or assessment derived from one or more resource and include physical, clinical, health and sport performance assessment.
11 . A method according to claim 9 , wherein the input can be invasive, non-invasive or external input or subjects or weight or forces or collision affecting the human body and its movement and contributing to the analysis and assessment of the human body, or a part thereof.
12 . A method according to claim 9 , wherein the assessment and analysis processes comprises output performed by at least one computer vision (CV) approach, machine learning (ML), and/or artificial intelligence (AI) model.
13 . A method according to claim 9 , wherein the movement of the human body being assessed belongs to a category or a group or a class of bodies and movement patterns, and the database comprises details of a plurality of different movements of the same or different human bodies belonging to the same, and/or a similar, category or group or class to that of the human body movement being analysed and assessed using the digital human twin.
14 . A method according to claim 13 , wherein the human body and movement data and/or information comprises one or more of human motion or movement marker and markerless data, videos, photos, multiviews, full and/or partial body shapes or 3D surface scans, anthropometry, characteristics, attributes medical body composition imaging and health data, medical epidemiological and physiology information.
15 . A method according to claim 9 , comprising a display for displaying a user interface, wherein the controller is operable and guided by electronic program instructions, to communicate the output by displaying the output via the display depicting a visualization of the analysis and assessment of the human body movement pattern and positions via at least one of text, images, graphs, spreadsheets, 3D or multi-layered meshes, avatar, or 3D pointclouds, heatmaps, videos, or virtual reality or finite element analysis of particles.
16 . A method according to claim 9 , wherein the human body or the part thereof is that of an individual person, and the output comprises an estimate of the individual person's: movements, motion, speed, acceleration, angles, joint angles, forces, joint forces, moments, joint moments kinetics, kinematics, or other analysis or assessment derived from one or more of these including physical, clinical, health and sport performance assessment, functional movement assessment or musculoskeletal assessment, clinical biomechanics assessment including pathological populations, injury prevention, motor development assessment including monitoring motor skill development in children or assessing people with Parkinson's disease, measurement of physical activity and automatically monitoring mentally ill patient or elders and other wellness and other relevant health and risk indicators.
17 . A non-transitory computer-readable storage medium on which is stored instructions that, when executed by a processor, causes the processor to perform a method according to claim 9 .
18 . A system for analysing a body movement comprising a device according claim 1 wherein the at least one input representation of the human body and its movement is in the form of numbers and/or text and/or data and/or images and/or videos of any type or format.
19 . A method of production of a digital twin of an individual person, comprising:
receiving a 3D representation of the individual person;
receiving medical images of the individual person;
forming a representation of the individual person from the 3D representation and the medical images as a model of a body shape and anatomical structure of the individual person;
segmenting the body shape and anatomical structure into slices and into positions in each slice;
allocating a composition type to each position in each slice.
20 . The method according to claim 19 , wherein the model comprises bone structure, density scores, hierarchy, connectivity, joints, joint rotation and allowable movements.
21 . A method of performing human movement analysis comprising:
receiving an input of a representation of an individual human;
fitting a skeleton to the representation;
using the skeleton to align a digital twin model of the individual human;
evaluating the human movement using anatomical structure of the digital twin model.
22 . The method according to claim 21 , wherein the digital twin model is produced by:
receiving a 3D representation of the individual person;
receiving medical images of the individual person;
forming a representation of the individual person from the 3D representation and the medical images as a model of a body shape and anatomical structure of the individual person;
segmenting the body shape and anatomical structure into slices and into positions in each slice;
allocating a composition type to each position in each slice.