IP Library Granted Patent US 12,412,377
Granted Patent B2
US 12,412,377 · App. 17/888,572 · Granted Sep 9, 2025

Validation of AI-based result data

Inventors: Andreas Fieselmann (Erlangen, DE); Christian Huemmer (Lichtenfels, DE); Ramyar Biniazan (Nuremberg, DE); Ludwig Ritschl (Buttenheim, DE)
Assignee: Siemens Healthineers AG
G06V10/776G06T5/50G06T5/70G06V10/751
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,412,377
App. No.
17/888,572
Granted
Sep 9, 2025
Kind
B2
Abstract

In a method, comparison features are extracted from labeled reference image data. Features are also extracted from the image data. A statistical comparison of the comparison features with the features then takes place. On the basis of the statistical comparison and a quality criterion, the quality of the AI-based result data is determined. A method for correcting result data is additionally described. Furthermore, a method for AI-based acquisition of result data on the basis of measured examination data is described. Also described is a validation entity. An entity for correcting result data is additionally described. Furthermore, an entity for acquiring result data is described. Also described is a medical imaging entity.

Claims (79)

1. A method for validating result data acquired via artificial intelligence (AI), the result data being based on image data, and the method comprising:

extracting comparison features from reference result data in a rule-based manner such that at least one of

the comparison features indicate at least one of a physical or anatomical characteristic of an examination object, or

the comparison features indicate a known characteristic present in an image of a given type, the result data being based on at least one image of the given type;

extracting features from the result data in the rule-based manner;

statistically comparing the comparison features with the features; and

determining a quality of the result data based on the statistical comparison and a quality threshold value.

2. The method as claimed in claim 1 , further comprising:

acquiring the result data.

3. The method as claimed in claim 1 , wherein the result data comprises:

image segmentation data;

detected objects;

classification data; or

regression data.

4. The method as claimed in claim 1 , wherein, in the event that the quality of the result data is below the quality threshold value, the method comprises:

outputting information indicating that a reliability of the result data is not sufficient, or

correcting the result data based on the comparison features and rules for the extracting of the comparison features.

5. The method as claimed in claim 1 , wherein

the comparison features include a reference image noise, and

the method further includes

comparing, as part of a procedure for reducing image noise of image data, the image noise of the image data with the reference image noise.

6. The method as claimed in claim 5 , wherein

the reference image noise is broken down into a number of frequency components, and

the statistical comparison of the comparison features with the features includes comparing frequency components of the reference image noise with frequency components of the image noise of the image data.

7. A method for correcting result data, the method comprising:

determining a validation result based on a method for validating result data as claimed in claim 1 ; and

generating corrected result data in the event that the validation result is negative.

8. The method for correcting result data as claimed in claim 7 , wherein the generating of the corrected result data comprises at least one of:

capturing substitute result data and replacing the result data with the substitute result data, or

correcting the result data directly based on the statistical comparison of the comparison features with the features, wherein the correcting includes

correcting an extracted feature such that the extracted feature becomes valid in a statistical comparison with the comparison features, and

generating corrected result data based on rules applied during the extracting of the comparison features.

9. A method for AI-based acquisition of result data based on measured examination data, the method comprising:

capturing the measured examination data from an examination object;

determining result data using AI based on the measured examination data;

determining a validation result based on a method as claimed in claim 1 ; and

determining the result data, which has been corrected based on the validation result.

10. A validation entity comprising:

at least one memory; and

processing circuitry configured to, with the at least one memory, cause the validation entity to

extract comparison features from labeled reference result data in a rule-based manner such that at least one of

the comparison features indicate at least one of a physical or anatomical characteristic of an examination object, or

the comparison features indicate a known characteristic present in an image of a given type, wherein result data is based on image data of at least one image of the given type,

extract features from the result data in the rule-based manner, the result data acquired via artificial intelligence (AI);

statistically compare the comparison features with the features; and

determine a quality of the result data based on the statistical comparison and a quality threshold value.

11. The validation entity as claimed in claim 10 , wherein the processing circuitry is further configured to cause the validation entity to:

determine a validation result based on the quality of the result data; and

generate corrected result data in the event that the validation result is negative.

12. The validation entity as claimed in claim 10 , wherein the processing circuitry is further configured to cause the validation entity to:

capture measured examination data from the examination object;

determine the result data based on the measured examination data;

determine a validation result based on the result data; and

generate corrected result data in the event that the validation result is negative.

13. The validation entity as claimed in claim 10 , wherein the processing circuitry is further configured to cause the validation entity to:

generate image data from an examination region;

determine the result data based on the image data; and

validate the result data.

14. A non-transitory computer program product including a computer program having program sections that, when executed by at least one processor at a medical imaging entity, cause the medical imaging entity to perform the method of claim 1 .

15. A non-transitory computer-readable medium storing program sections that, when executed by at least one processor of a computer unit, cause the computer unit to perform the method of claim 1 .

16. The method of claim 9 , wherein the measured examination data is medical image data.

17. A method for AI-based acquisition of result data based on measured examination data, the method comprising:

capturing the measured examination data from an examination object;

determining result data using AI based on the measured examination data;

determining a validation result for the result data; and

determining corrected result data based on the validation result, according to the method of claim 7 .

18. The validation entity as claimed in claim 11 , wherein the processing circuitry is further configured to cause the validation entity to:

generate image data from an examination region;

determine the result data based on the image data;

validate the result data; and

correct the result data.

19. The validation entity as claimed in claim 12 , wherein the processing circuitry is further configured to cause the validation entity to:

generate image data from an examination region;

determine the result data based on the image data; and

validate the result data.

20. The method as claimed in claim 4 , further comprising:

using, as part of a procedure for reducing image noise of image data, a reference image noise as a comparison feature; and

comparing the image noise of the image data with the reference image noise.

21. The method as claimed in claim 1 , wherein the comparison features indicate a distance between landmarks of the examination object.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2023
From: SIEMENS HEALTHCARE GMBH
To: SIEMENS HEALTHINEERS AG
Reel/Frame 066267/0346 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2022
From: FIESELMANN, ANDREAS; HUEMMER, CHRISTIAN; BINIAZAN, RAMYAR; RITSCHL, LUDWIG
To: SIEMENS HEALTHCARE GMBH
Reel/Frame 061449/0907 →
Priority Claims (1)
DE 10 2021 209 169.5 · Aug 20, 2021 · national
Continuity (1)
Related Publication 20230054619A1 · Feb 23, 2023
References Cited (17)
US 10474934B1 · Cosic · 2019 [cited by examiner]
US 10970577B1 · Song · 2021 [cited by examiner]
US 20080292194A1 · Schmidt · 2008 [cited by examiner]
US 20140341471A1 · Ono et al. · 2014 [cited by applicant]
US 20170211150A1 · Breen · 2017 [cited by examiner]
US 20170372155A1 · Odry et al. · 2017 [cited by applicant]
US 20210118197A1 · Gao · 2021 [cited by examiner]
US 20220096034A1 · Herbst et al. · 2022 [cited by applicant]
US 20220139531A1 · Wang · 2022 [cited by examiner]
US 20220139552A1 · Binder · 2022 [cited by examiner]
US 20220277452A1 · Vaidyanathan · 2022 [cited by examiner]
US 20220351530A1 · Zaritsky · 2022 [cited by examiner]
EP 2762072A1 · 2014 [cited by applicant]
(Lak17) Lakshminarayanan B. et al.:“Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles”, 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA. [cited by applicant]
(Man17) Mandelbaum A. et al.“Distance-based Confidence Score for Neural Network Classifiers”, School of Computer Science and Engineering Hebrew University of Jerusalem, Israel, Sep. 2017. [cited by applicant]
(Arun20) Arun N. et al.:“Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging”, Aug. 2020 [Arun20] https://arxiv.org/abs/2008.02766. [cited by applicant]
Subramanian V. et al.:“Automated Detection and Type Classification of Central Venous Catheters in Chest X-Rays”, Jul. 2019, [Sub19] https://www.researchgate.net/publication/334223370_Automated_Detection_and_Type_Classif… [cited by applicant]