IP Library Granted Patent US 12,651,350
Granted Patent B2
US 12,651,350 · App. 18/189,486 · Granted Jun 9, 2026

Image processing method and apparatus

Inventors: Niko Nevatie (Helsinki, FI); Erkki Parkkulainen (Helsinki, FI)
Assignee: TGI SPORT VIRTUAL TECHNOLOGIES LIMITED
G06T7/194G06T2207/20021G06T2207/20081G06T2207/20084
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,651,350
App. No.
18/189,486
Granted
Jun 9, 2026
Kind
B2
Abstract

Aspects of the present invention relate to a computer-implemented training method for training a segmentation model to segment an image. The method includes receiving a plurality of training data sets each including image data representing an image comprising a foreground and a background; and sample image data comprising one or more sample image occurring in the background of the image. The image data is captured by at least one visible electromagnetic radiation imaging device. The method includes processing each training data set using the segmentation model. The processing of each training data includes supplying the sample image data to the segmentation model; and segmenting the image data to generate a candidate segmentation in dependence on the sample image data. An error is determined for the candidate segmentation. The segmentation model is updated in dependence on the determined error. According to a further aspect of the present invention there is provided a system for training a segmentation model to segment an image. Aspects of the present invention also relate to an image processing system and method.

Claims (38)

1 . A computer-implemented training method for training a segmentation model to segment an image, the method comprising receiving a plurality of training data sets, the training data sets each comprising:

image data representing an image comprising a foreground and a background, the image data being captured by at least one visible electromagnetic radiation imaging device; and

sample image data comprising one or more sample image occurring in the background of the image, wherein one or more advertising boards is present in the background of the image represented by the image data, the sample image data comprising a library of images displayed on the one or more advertising boards, the library of images being updated dynamically by identifying and extracting an image displayed on the advertising board;

wherein the method comprises:

processing each training data set using the segmentation model, the processing of each training data set comprising:

supplying the sample image data to the segmentation model;

segmenting the image data to generate a candidate segmentation in dependence on the sample image data;

determining an error of the candidate segmentation; and

updating the segmentation model in dependence on the determined error.

2 . A computer-implemented training method as claimed in claim 1 , wherein the sample image data comprises a plurality of sample images, one or more of the sample images appearing in the background of the image.

3 . A computer-implemented training method as claimed in claim 1 , comprising supplying background composition data defining a composition of at least a portion of the background of the image represented by the image data.

4 . A computer-implemented training method as claimed in claim 3 , wherein the segmentation of the image by the segmentation model is performed in dependence on the background composition data.

5 . A computer-implemented training method as claimed in claim 1 , wherein the image comprises at least one background area occurring in the background of the image, the segmentation model being configured to process the image data to segment the image to determine the at least one background area.

6 . A computer-implemented training method as claimed in claim 1 , wherein the image comprises at least one mixed area occurring in the foreground and the background of the first image, the segmentation model being configured to process the image data to segment the first image to determine the at least one mixed area.

7 . A non-transitory computer-readable medium having a set of instructions stored therein which, when executed, cause a processor to perform the method claimed in claim 1 .

8 . An image segmentation system for segmenting an image, the image segmentation system comprising one or more processors; wherein the one or more processors is configured to implement a segmentation model trained according to the method claimed in claim 1 .

9 . A system for training a segmentation model to segment an image, the system comprising at least one processor and at least one memory device, the at least one processor comprising at least one input configured to receive a plurality of training data sets, the training data sets each comprising:

image data representing an image comprising a foreground and a background, the image data being captured by at least one visible electromagnetic radiation imaging device; and

sample image data comprising one or more sample image occurring in the background of the image, wherein one or more advertising boards is present in the background of the image represented by the image data, the sample image data comprising a library of images displayed on the one or more advertising boards, the library of images being updated dynamically by identifying and extracting an image displayed on the advertising board;

wherein, in respect of each of the plurality of first training data sets, the at least one processor is configured to:

process each training data set using the segmentation model, the processing of each training data set comprising:

supplying the sample image data to the segmentation model;

segmenting the image data to generate a candidate segmentation in dependence on the sample image data;

determining an error of the candidate segmentation; and

updating the segmentation model in dependence on the determined error.

10 . A system as claimed in claim 9 , wherein the sample image data comprises a plurality of sample images, one or more of the sample images appearing in the background of the image.

11 . A system as claimed in claim 9 , wherein the at least one processor is configured to receive background composition data defining a composition of at least a portion of the background of the image represented by the image data.

12 . A system as claimed in claim 11 , wherein the segmentation of the image by the segmentation model is performed in dependence on the background composition data.

13 . A system as claimed in claim 9 , wherein the image comprises at least one background area occurring in the background of the image, the segmentation model being configured to process the image data to segment the image to determine the at least one background area.

14 . A system as claimed in claim 9 , wherein the image comprises at least one mixed area occurring in the foreground and the background of the first image, the segmentation model being configured to process the image data to segment the first image to determine the at least one mixed area.

15 . An image processing system for processing an image, the image processing system comprising one or more processors having at least one electrical input for receiving:

first image data captured by at least one visible electromagnetic radiation imaging device and representing a first image comprising a foreground and a background; and

receiving sample image data comprising one or more sample image occurring in the background of the image, wherein one or more advertising boards is present in the background of the image represented by the image data, the sample image data comprising a library of images displayed on the one or more advertising boards, the library of images being updated dynamically by identifying and extracting an image displayed on the advertising board;

wherein the one or more processors is configured to implement a segmentation model; the segmentation model being configured to segment the first image to determine the foreground and the background of the first image, the segmentation model accessing the sample image data to determine the one or more sample image occurring in the background of the image.

16 . An image processing system for processing an image, the image processing system comprising one or more processors having an electrical input for receiving image data captured by at least one visible electromagnetic radiation imaging device and representing a image comprising a foreground and a background;

wherein the one or more processors is configured to implement a segmentation model trained using the computer-implemented training method claimed in claim 1 ; the segmentation model being configured to segment the image to differentiate between the foreground and the background of the image.

17 . A computer-implemented method of processing a first image, the method comprising receiving image data captured by at least one visible electromagnetic radiation imaging device and representing an image comprising a foreground and a background;

wherein the method comprises implementing a segmentation model to segment the image to differentiate between the foreground and the background of the first image, the segmentation model being trained using the computer-implemented training method claimed in claim 1 .

Assignments (2)
CHANGE OF NAME Recorded Dec 3, 2025
From: SUPPONOR TECHNOLOGIES LIMITED
To: TGI SPORT VIRTUAL TECHNOLOGIES LIMITED
Reel/Frame 073104/0398 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 19, 2023
From: NEVATIE, NIKO; PARKKULAINEN, ERKKI
To: SUPPONOR TECHNOLOGIES LIMITED
Reel/Frame 063987/0042 →
Priority Claims (1)
GB 2204205 · Mar 24, 2022 · national
Continuity (1)
Related Publication 20230306613A1 · Sep 28, 2023
References Cited (85)
US 9965865B1 · Agrawal et al. · 2018 [cited by applicant]
US 10242449B2 · Liston et al. · 2019 [cited by applicant]
US 10255681B2 · Price et al. · 2019 [cited by applicant]
US 10713794B1 · He et al. · 2020 [cited by applicant]
US 10860836B1 · Tyagi et al. · 2020 [cited by applicant]
US 11341653B2 · Kim · 2022 [cited by examiner]
US 12086995B2 · Benou et al. · 2024 [cited by applicant]
US 12165284B2 · Zhang et al. · 2024 [cited by applicant]
US 20050001852A1 · Dengler et al. · 2005 [cited by applicant]
US 20070269108A1 · Steinberg et al. · 2007 [cited by applicant]
US 20150163416A1 · Nevatie · 2015 [cited by applicant]
US 20160028968A1 · Affaticati · 2016 [cited by applicant]
US 20170324889A1 · Nevatie et al. · 2017 [cited by applicant]
US 20180176484A1 · Nevatie · 2018 [cited by applicant]
US 20180189951A1 · Liston et al. · 2018 [cited by applicant]
US 20180253865A1 · Price et al. · 2018 [cited by applicant]
US 20180324382A1 · Rantalainen · 2018 [cited by applicant]
US 20190220983A1 · Price et al. · 2019 [cited by applicant]
US 20190222777A1 · Lovemelt et al. · 2019 [cited by applicant]
US 20200267334A1 · Cyrus · 2020 [cited by applicant]
US 20200364877A1 · Bradski et al. · 2020 [cited by applicant]
US 20210034901A1 · Chen · 2021 [cited by applicant]
US 20210166393A1 · Ramani et al. · 2021 [cited by applicant]
US 20210279503A1 · Ql et al. · 2021 [cited by applicant]
US 20210366128A1 · Kim · 2021 [cited by examiner]
US 20220044365A1 · Zhang et al. · 2022 [cited by applicant]
US 20220044366A1 · Zhang et al. · 2022 [cited by applicant]
US 20220058369A1 · Alahmari · 2022 [cited by examiner]
US 20220222791A1 · Zhang · 2022 [cited by applicant]
US 20220262009A1 · Yu et al. · 2022 [cited by applicant]
US 20220272245A1 · Chu et al. · 2022 [cited by applicant]
US 20230010947A1 · Wang et al. · 2023 [cited by applicant]
US 20230064450A1 · Hiskens et al. · 2023 [cited by applicant]
US 20230306611A1 · Nevatie et al. · 2023 [cited by applicant]
US 20230306612A1 · Nevatie et al. · 2023 [cited by applicant]
US 20230326030A1 · Nevatie et al. · 2023 [cited by applicant]
US 20230326031A1 · Nevatie et al. · 2023 [cited by applicant]
CN 108229478A · 2018 [cited by applicant]
CN 108596184A · 2018 [cited by applicant]
CN 110188760A · 2019 [cited by applicant]
CN 110189341A · 2019 [cited by applicant]
CN 110246145A · 2019 [cited by applicant]
CN 111260679A · 2020 [cited by examiner]
CN 111667027A · 2020 [cited by applicant]
CN 112116625A · 2020 [cited by examiner]
CN 112232364A · 2021 [cited by applicant]
CN 112489047A · 2021 [cited by examiner]
CN 113192072A · 2021 [cited by examiner]
CN 110189341B · 2021 [cited by examiner]
CN 113515968A · 2021 [cited by examiner]
CN 114782460B · 2022 [cited by examiner]
GB 2560219A · 2018 [cited by applicant]
WO 2013186278A1 · 2013 [cited by applicant]
WO 2020021067A1 · 2020 [cited by applicant]
WO 2020156303A1 · 2020 [cited by applicant]
WO 2022048151A1 · 2022 [cited by applicant]
“Patents Act 1977: Combined Search and Examination Report under Sections 17 and 18(3)”, GB Application No. GB2304305.2, Oct. 10, 2023, 5 pp. [cited by applicant]
“Patents Act 1977: Combined Search and Examination Report under Sections 17 and 18(3)”, GB Application No. GB2304308.6, Oct. 10, 2023, 6 pp. [cited by applicant]
“Patents Act 1977: Combined Search and Examination Report under Sections 17 and 18(3)”, GB Application No. GB2304294.8, Oct. 10, 2023, 9 pp. [cited by applicant]
“Patents Act 1977: Combined Search and Examination Report under Sections 17 and 18(3)”, GB Application No. GB2304303.7, Oct. 24, 2023, 12 pp. [cited by applicant]
“Patents Act 1977: Combined Search and Examination Report under Sections 17 and 18(3)”, GB Application No. GB2304297.1, Oct. 24, 2023, 9 pp. [cited by applicant]
Hou, Qiqi , et al., “Context-Aware Image Matting for Simultaneous Foreground and Alpha Estimation”, 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Soul, South Korea, Oct. 27, 2019, pp. 4129-4138. [cited by applicant]
“Patents Act 1977: Search Report under Section 17(5)”, GB Application No. GB2204198.2, Sep. 13, 2022, 3 pp. [cited by applicant]
“Patents Act 1977: Search Report under Section 17(5)”, GB Application No. 2204204.8, Sep. 20, 2022, 4 pp. [cited by applicant]
“Patents Act 1977: Search Report under Section 17(5)”, GB Application No. GB2204205.5, Sep. 20, 2022, 4 pp. [cited by applicant]
“Patents Act 1977: Search Report under Section 17(5)”, GB Application No. 2204202.2, Sep. 22, 2022, 3 pp. [cited by applicant]
“Patents Act 1977: Search Report under Section 17(5)”, GB Application No. GB2204196.6, Oct. 19, 2022, 6 pp. [cited by applicant]
“International Search Report and Written Opinion of the International Searching Authority”, International Application No. PCT/EP2023/057625, Jul. 3, 2023, 17 pp. [cited by applicant]
“International Search Report and Written Opinion of the International Searching Authority”, International Application No. PCT/EP2023/057628, Jul. 3, 2023, 17 pp. [cited by applicant]
“International Search Report and Written Opinion of the International Searching Authority”, International Application No. PCT/EP2023/057618, Jul. 3, 2023, 18 pp. [cited by applicant]
“International Search Report and Written Opinion of the International Searching Authority”, International Application No. PCT/EP2023/057624, Jul. 3, 2023, 18 pp. [cited by applicant]
“International Search Report and Written Opinion of the International Searching Authority”, International Application No. PCT/EP2023/057626, Jul. 3, 2023, 18 pp. [cited by applicant]
Arabi, Hossein , et al., “Does Prior Knowledge Enhance Accuracy of Deep Learning-assisted Semantic Segmentation?”, 2021 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), Piscataway, NJ, USA, Oct. … [cited by applicant]
Chen, Yuhua, et al., “Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach”, arXiv:1812.05040v2, arxiv.org, Cornell University, Jan. 13, 2019, 10 pp. [cited by applicant]
Chen, Yuhua, et al., “Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach”, ArXiv:1812.05040v1, arxiv.org, Cornell University Library, Dec. 12, 2018, 10 pp. [cited by applicant]
Cheng, Gong , et al., “Prototype-CNN for Few-Shot Object Detection in Remote Sensing Images”, IEEE Transactions on Geoscience and Remote Sensing, vol. 60, May 19, 2021, 10 pp. [cited by applicant]
Dmitriev, Konstantin , et al., “Learning Multi-Class Segmentations From Single-Class Datasets”, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, Jun. 15, 2020, pp. 9493-95… [cited by applicant]
El Jurdi, Rosana , et al., “High-level prior-based loss functions for medical image segmentation: A survey”, Computer Vision and Image Understanding, vol. 210, Jul. 21, 2021. [cited by applicant]
Ghassab, Vahid Khorasani, et al., “REP-Model: A deep learning framework for replacing ad billboards in soccer videos”, 2020 IEEE International Symposium on Multimedia (ISM), Naples Italy, Dec. 2, 2020, pp. 149-153. [cited by applicant]
Hou, Qiqi , et al., “Context-Aware Image Matting for Simultaneous Foreground and Alpha Estimation”, 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea, Oct. 27, 2019, pp. 4129-4138. [cited by applicant]
Naveed, Humza , “Survey: Image Mixing and Deleting for Data Augmentation”, arXiv:2106.07085v2, arxiv.org, Cornell University Library, Nov. 1, 2021, 15 pp. [cited by applicant]
No Name , “Using FCNN for multi-class semantic segmentation trained on single class labeled image data”, Retrieved from the Internet: URL:https://stats.stackexchange.com/questions/502896/using-fcnn-for-multi-class-seman… [cited by applicant]
Wang, Kaixin , et al., “PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment”, 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea, Oct. 27, 2019, pp. 9196-9205. [cited by applicant]
Zhang, Jiawei , et al., “ObjectAug: Object-level Data Augmentation for Semantic Image Segmentation”, 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China, Jul. 18, 2021, 8 pp. [cited by applicant]
Hu, et al., “Instance Segmentation based Semantic Matting for Compositing Applications”, 2019 16th Conference on Computer and Robot Vision (CRV), Kingston, QC, Canada, doi: 10.1109/CRV.2019.00026, 2019, pp. 135-142. [cited by applicant]