Granted Patent B2

US 12,639,824 · App. 18/189,447 · Granted May 26, 2026

Inventors: Niko Nevatie (Helsinki, FI); Erkki Parkkulainen (Helsinki, FI)

Assignee: TGI SPORT VIRTUAL TECHNOLOGIES LIMITED

G06T7/194G06V10/751G06T2207/20081

Abstract

Claims (68)

1 . A computer-implemented training method for training a segmentation model to segment an image, the method comprising receiving a plurality of first training data sets, the first training data sets each comprising:

first image data representing a first image comprising a foreground and a background, the first image data being captured by at least one visible electromagnetic radiation imaging device;

wherein the method comprises, for each first training data set:

generating a first target segmentation for differentiating between the foreground and the background of the first image, the first target segmentation being generated in dependence on first segmentation data captured by at least one non-visible electromagnetic radiation imaging device, the first segmentation data identifying one or more subjects of interest occurring in the background of the first image; and

processing the first image data using the segmentation model, the processing of the first image data comprising:

processing the first image data to segment the first image and generate a first candidate segmentation;

comparing the first candidate segmentation and the first target segmentation to determine a first error; and

updating the segmentation model in dependence on the determined first error.

2 . A computer-implemented training method as claimed in claim 1 , wherein the first image comprises at least one foreground area occurring in the foreground of the first image, the segmentation model being configured to process the first image data to segment the first image to determine the at least one foreground area.

3 . A computer-implemented training method as claimed in claim 1 , wherein the first image comprises at least one background area occurring in the background of the first image, the segmentation model being configured to process the first image data to segment the first image to determine the at least one background area.

4 . A computer-implemented training method as claimed in claim 1 , wherein the first image comprises at least one mixed image area occurring in the foreground and the background of the first image, the segmentation model being configured to process the first image data to segment the first image to determine the at least one mixed image area.

5 . A computer-implemented training method as claimed in claim 1 , wherein the method comprises receiving a plurality of second training data sets, the second training data sets each comprising:

second image data, the second image data representing a second image consisting of either a foreground or a background, the second image data being captured by at least one visible electromagnetic radiation imaging device.

6 . A computer-implemented training method as claimed in claim 5 , wherein the method comprises, for each second training data set:

generating a second target segmentation for differentiating between the foreground and the background of the second image;

processing the second training data set using the segmentation model, the processing of the second training data set comprising:

processing the second image data to segment the second image to generate a second candidate segmentation;

comparing the second candidate segmentation and the second target segmentation to determine a second error; and

updating the segmentation model in dependence on the determined second error.

7 . A computer-implemented training method as claimed in claim 5 , wherein the second training data sets each comprise annotation data associated with the second image data, the annotation data identifying the presence solely of the foreground or the background in the second image;

wherein the method comprises, for each second training data set:

processing the second training data set using the segmentation model, the processing of the second training data set comprising:

processing the second image data to segment the second image to generate a second candidate segmentation;

using the annotation data to determine a second error for the second candidate segmentation; and

updating the segmentation model in dependence on the determined second error.

8 . A computer-implemented training method as claimed in claim 1 , wherein the method comprises:

processing the first image data to determine at least one segment of the first image;

augmenting the first image data by applying a first supplementary image data to the at least one determined segment to generate first augmented image data;

generating a first augmented training data set comprising the first augmented image data; and

introducing the first augmented training data set as one of the first training data sets.

9 . A non-transitory computer-readable medium having a set of instructions stored therein which, when executed, cause a processor to perform the method claimed in claim 1 .

10 . An image segmentation system for segmenting an image, the image segmentation system comprising one or more processors; wherein the one or more processors is configured to implement a segmentation model trained according to the method claimed in claim 1 .

11 . A content replacement system for replacing the content of an image, the content replacement system comprising an image segmentation system as claimed in claim 10 .

12 . A system for training a segmentation model to segment an image, the system comprising at least one processor and at least one memory device, the at least one processor comprising at least one input configured to receive a plurality of first training data sets, the first training data sets each comprising:

first image data representing a first image comprising a foreground and a background, the first image data being captured by at least one visible electromagnetic radiation imaging device;

wherein, in respect of each of the plurality of first training data sets, the at least one processor is configured to:

determine a first target segmentation for differentiating between the foreground and the background of the first image, the first target segmentation being generated in dependence on first segmentation data captured by at least one non-visible electromagnetic radiation imaging device, the first segmentation data identifying one or more subjects of interest occurring in the background of the first image; and

process the first image data using the segmentation model, the processing of the first image data comprising:

processing the first image data to segment the first image to generate a first candidate segmentation;

comparing the first candidate segmentation and the first target segmentation to determine a first error; and

updating the segmentation model in dependence on the determined first error.

13 . A system as claimed in claim 12 , wherein the first image comprises at least one foreground area occurring in the foreground of the first image, the segmentation model being configured to process the first image data to segment the first image to determine the at least one foreground area.

14 . A system as claimed in claim 12 , wherein the first image comprises at least one background area occurring in the background of the first image, the segmentation model being configured to process the first image data to segment the first image to determine the at least one background area.

15 . A system as claimed in claim 12 , wherein the first image comprises at least one mixed image area occurring in the foreground and the background of the first image, the segmentation model being configured to process the first image data to segment the first image to determine the at least one mixed image area in the foreground and the background of the first image.

16 . A system as claimed in claim 12 , wherein the first target segmentation data identifies at least one subject of interest in the background of the first image.

17 . A system as claimed in claim 12 , wherein the at least one input is configured to receive a plurality of second training data sets, the second training data sets each comprising:

second image data, the second image data representing a second image consisting of either a foreground or a background, the second image data being captured by at least one visible electromagnetic radiation imaging device.

18 . A system as claimed in claim 17 , wherein, in respect of each of the plurality of second training data sets, the at least one processor is configured to:

determine a second target segmentation for differentiating between the foreground and the background of the second image;

process the second training data set using the segmentation model, the processing of the second training data set comprising:

processing the second image data to segment the second image to generate a second candidate segmentation;

comparing the second candidate segmentation and the second target segmentation to determine a second error; and

updating the segmentation model in dependence on the determined second error.

19 . A system as claimed in claim 17 , wherein the second training data sets each comprise annotation data associated with the second image data, the annotation data identifying the presence solely of the foreground or the background in the second image;

wherein, in respect of each of the plurality of second training data sets, the at least one processor is configured to:

process the second training data set using the segmentation model, the processing of the second training data set comprising:

processing the second image data to segment the second image to generate a second candidate segmentation;

using the annotation data to determine a second error for the second candidate segmentation; and

updating the segmentation model in dependence on the determined second error.

20 . A system as claimed in claim 12 , wherein, in respect of each of the plurality of second training data sets, the at least one processor is configured to:

process the first image data to determine at least one segment of the first image;

augment the first image data by applying a first supplementary image data to the at least one determined segment to generate first augmented image data;

generate a first augmented training data set comprising the first augmented image data; and

introduce the first augmented training data set as one of the first training data sets.

21 . An image processing system for processing an image, the image processing system comprising one or more processors having an electrical input for receiving first image data captured by at least one visible electromagnetic radiation imaging device and representing a first image comprising a foreground and a background;

wherein the one or more processors is configured to implement a segmentation model trained in accordance with the computer-implemented training method claimed in claim 1 ; the segmentation model being configured to segment the first image to differentiate between the foreground and the background of the first image.

22 . A system as claimed in claim 12 , wherein the first segmentation data is captured by at least one non-visible electromagnetic radiation imaging device configured to capture electromagnetic radiation in the near-infrared spectrum.

23 . A computer-implemented training method as claimed in claim 1 , wherein the first segmentation data is captured by at least one non-visible electromagnetic radiation imaging device configured to capture electromagnetic radiation in the near-infrared spectrum.

Assignments (2)

Priority Claims (1)

Continuity (1)

References Cited (90)