Action recognition system and method

1 . An action recognition system for recognising actions performed by a subject, the action recognition system comprising:

a computing apparatus comprising a processor, a memory unit, wherein the memory unit comprises a non-transitory computer readable medium comprising computer executable instructions stored thereon that, upon execution of the instructions by the processor, the processor configured to:

receive one or more input images of a subject captured by an image capture device,

generate a wire three dimensional (3D) skeletal model of the subject identified within the one or more input images,

determine an observation viewpoint for each image,

apply a transform to the skeletal model based on the identified viewpoint, and

process the skeletal model to identify an actions;

wherein the processor is further configured to repeatedly, for each image of the one or more images, determine an observation viewpoint for each image and transform the skeletal model based on the identified viewpoint;

wherein the processor is configured to apply a transformation function that defines parameters related to an optimal viewpoint;

wherein the transformation function, when executed by the processor, transforms each coordinate of the skeleton model and/or the coordinate system defining the position of the skeleton model, based on the transformation function, such that the skeleton model is arranged as viewed from an optimal viewpoint;

wherein the transformation function comprises a transformation matrix and a translation vector, the transformation matrix defines a plurality of transformation values, wherein at least one transformation value corresponds to a transformation of x coordinates of the skeleton model, at least one transformation value corresponds to a transformation of y coordinates of the skeleton model, and at least one transformation value that corresponds to a transformation value of z coordinates of the skeleton model;

and

wherein the transformation function is defined as

=

(

x

t

,

α

,

b

t

)

⁢

(

y

t

,

β

,

b

t

)

⁢

(

z

t

,

γ

,

b

t

)

,

wherein x, y, z correspond to coordinates, α=radians of x coordinate transformation, β=radians of y coordinate transformation and Y=radians of z coordinate transformation, wherein the transformation function defines the transformation for each image.

2 . The action recognition system in accordance with claim 1 , wherein the transformation matrix is defined as

t

=

[

x

t

,

α

,

y

t

,

β

,

z

t

,

γ

]

and wherein the translation vector is defined as b∈ 3 .

3 . The action recognition system in accordance with claim 2 , wherein each joint of the skeleton model in each image shares the same transformation parameters, wherein the transformation parameters are defined as (α, β, γ, b).

4 . An action recognition system for recognising actions performed by a subject, the action recognition system comprising:

a computing apparatus comprising a processor, a memory unit, wherein the memory unit comprises a non-transitory computer readable medium comprising computer executable instructions stored thereon that, upon execution of the instructions by the processor, the processor configured to:

receive one or more input images of a subject captured by an image capture device,

generate a wire three dimensional (3D) skeletal model of the subject identified within the one or more input images,

determine an observation viewpoint for each image,

apply a transform to the skeletal model based on the identified viewpoint, process the skeletal model to identify an action,

wherein the processor is configured to apply a transformation function that defines associated transformation parameters related to an optimal viewpoint;

wherein the processor is configured to automatically learn an optimal viewpoint when the processor executes the step of determining an observation viewpoint; and the processor further configured to automatically learning and determining an observational viewpoint with the associated transformation parameters,

wherein the processor is configured to implement and execute a neural network, the neural network may be defined by the computer executable instructions stored in the memory unit, and wherein the neural network comprises a view adaption block, and the view adaption block is configured to apply an unsupervised learning algorithm as part of the step of learning an observation viewpoint with associated transformation parameters.

5 . The action recognition system in accordance with claim 4 , wherein the learning algorithm comprises applying a learning function to each of the received images.

6 . The action recognition system in accordance with claim 5 , wherein the learning algorithm comprises applying a learning function, the learning function comprises

[

α

t

,

β

t

,

γ

t

]

T

=

r

k

t

r

+

c

r

,

wherein r ∈ 3×n denotes the weight matrix,

k

t

r

∈

ℝ

n

×

1

defines a vector for the linear layer of the output with n number of neurons within the neural network and c r ∈ 3×1 defines an offset vector of the linear layer.

7 . An action recognition system for recognising actions performed by a subject, the action recognition system comprising:

a computing apparatus comprising a processor, a memory unit, wherein the memory unit comprises a non-transitory computer readable medium comprising computer executable instructions stored thereon that, upon execution of the instructions by the processor, the processor configured to:

receive one or more input images of a subject captured by an image capture device,

generate a wire three dimensional (3D) skeletal model of the subject identified within the one or more input images,

determine an observation viewpoint for each image,

apply a transform to the skeletal model based on the identified viewpoint, and

process the skeletal model to identify an action by:

converting the transformed skeleton model, of each image, into a graph representation, and

processing each graph representation to identify an action.

8 . The action recognition system in accordance with claim 7 , wherein the processor is configured to implement an adaptive convolution graph neural network, and wherein the steps of converting the transformed skeleton model into a graph representation and processing each graph representation is performed by an adaptive convolution graph neural network.

9 . The action recognition system in accordance with claim 8 , wherein the processor, as part of the step of converting the transformed skeleton into a graph representation, is further configured to execute computer readable instructions causing the processor to perform the additional steps of:

representing each set of joints of the skeleton model as a total number of nodes within the graphrepresentation,

applying a graph convolution to each node within the graph representation by:

i) transforming node representations to a learnable parameter matrix defined as ∈ (l+1)×l , and

ii) applying a nonlinear ReLU function (σ) to the transformed node representations, and

processing each graph representation comprises identifying relationships between two or more nodes that represent joints in the skeleton model; and determining an action based on the relationship between two or more nodes.

10 . The action recognition system in accordance with claim 9 , wherein the convolution graph neural network further comprises:

at least a first adaptive convolution block; and

at least a second adaptive convolution block,

wherein the first adaptive convolution block comprises at least a 2D convolution layer, a first batch normalisation layer, and a first rectified linear unit activation layer, a graph convolution block with 128 channels, wherein the graph convolution block with 128 channels is followed by a second batch normalisation layer and a second rectified linear unit activation layer, and;

wherein the second adaptive convolution block comprises at least a 2D convolution layer, a first batch normalisation layer, and a first rectified linear unit activation layer, a graph convolution block with 192 channels, wherein the graph convolution block with 192 channels is followed by a second batch normalisation layer and a second rectified linear unit activation layer.