Granted Patent B2

US 12,670,740 · App. 18/380,591 · Granted Jun 30, 2026

Inventors: Razwan Ghafoor (Sutton Coldfield, GB); Peter Rennert (Dunblane, GB); Hichame Moriceau (London, GB)

Assignee: STANDARD COGNITION, CORP.

G06V40/103

Abstract

Claims (24)

1 . A computer-implemented method of performing image analysis to identify human actions represented in a sequence of images, the method comprising:

analyzing the sequence of images depicting one or more people using a first computational neural network to determine a set of joint candidates for the one or more people depicted in the sequence of images;

deriving pose estimates from the set of joint candidates that estimate a body configuration for the one or more people depicted in the sequence of images; and

identifying an action performed by a person depicted in the sequence of images according to a region of interest within the sequence of images identified from the derived pose estimates that is analyzed using a recursive neural network.

2 . The computer-implemented method of claim 1 , further comprising:

extracting, from each derived pose estimate, values for a set of one or more parameters characterizing the pose estimate; and

using the recursive neural network to identify an action performed by a person depicted in the one or more images in dependence on the extracted parameter values.

3 . The computer-implemented method of claim 2 , wherein the set of one or more parameters comprises at least one of: joint position for specified joints; joint angles between specified connected joints; joint velocity for specified joints; and a distance between specified pairs of joints.

4 . The computer-implemented method of claim 3 , wherein the one or more images is a series of multiple images, and wherein the action performed by the person depicted in the series of images is identified from changes in their derived pose estimate over the series of images.

5 . The computer-implemented method of claim 4 , further comprising identifying the action performed by the person depicted in the series of images from changes in their derived pose estimate over the series of images, wherein the recursive neural network identifies an action performed by the person from the change in the extracted parameter values characterizing their derived pose estimate over the series of images.

6 . The computer-implemented method of claim 1 , wherein each image of the one or more images depicts a plurality of people, and derived pose estimates are analyzed for each of the plurality of people for the one or more images using the recursive neural network and to identify an action performed by each person depicted in the one or more images.

7 . The computer-implemented method of claim 1 , wherein the first computational neural network analyzes the region of interest within the sequence of images on a pixel-by-pixel basis.

8 . The computer-implemented method of claim 7 , wherein the first computational neural network generates an output vector containing a number of elements, each element within the number of elements storing a probability that a respective pixel within the image depicts a joint, and wherein the output vector is evaluated to determine the set of candidate joints.

9 . The computer-implemented method of claim 1 , wherein the region of interest defines a sub-region of the sequence of images, and wherein only the sub-region of the sequence of images is analyzed using the recursive neural network.

10 . The computer-implemented method of claim 9 , wherein the sub-region of the sequence of images is analyzed on a pixel-by-pixel basis.

11 . The computer-implemented method of claim 1 , wherein the region of interest within the sequence of images is analyzed to identify objects of a specified object class, and to identify the action in response to detecting an object of the specified class in the region of interest.

12 . The computer-implemented method of claim 1 , further comprising identifying the region of interest within the sequence of images from the derived pose estimates.

13 . The computer-implemented method of claim 12 , wherein the region of interest bounds a specified subset of joints of a derived pose estimate.

14 . The computer-implemented method of claim 12 , wherein the region of interest bounds one or more derived pose estimates.

15 . The computer-implemented method of claim 12 , further comprising:

identifying a region of interest within the sequence of images that bounds terminal ends of a derived pose estimate; and

analyzing the identified region of interest that bounds terminal ends of the derived pose estimate using the recursive neural network to identify whether the person depicted in the sequence of images is holding an object of a specified class or not.

16 . The computer-implemented method of claim 1 , further comprising identifying the action from a class of actions including: scanning an item at a point-of-sale; and selecting an item for purchase.

17 . The computer-implemented method of claim 1 , wherein the first computational neural network is a convolutional neural network.

Assignments (1)

Priority Claims (1)

Continuity (3)

References Cited (367)