Object keypoint detection

1 . A system for obtaining object keypoints for an object in a medical scanner, wherein the object keypoints are three dimensional (3D) coordinates with respect to the medical scanner of pre-determined object parts, and wherein the object keypoints are defined inside the object parts, the system comprising:

a camera system for obtaining two dimensional (2D) images of the object in the medical scanner, wherein the camera system comprises:

one or more cameras;

a processor configured to:

obtain scanner variables from at least one of the medical scanner or the camera system, wherein the medical scanner contains a first moving part and/or wherein the camera system contains a second moving part, and wherein the scanner variables include a position of the first moving part of the medical scanner and/or a position of the second moving part of the camera system;

determine object keypoint projections based on inputting the 2D images into a neural network trained to learn spatial relationships between object keypoint projections corresponding to different parts of the object, wherein the neural network is configured to output the object keypoint projections in each one of the input 2D images, wherein the object keypoint projections are 2D coordinates projected on to the 2D images, respectively;

determine the object keypoints in 3D coordinates by triangulating the object keypoint projections with respect to the camera system based on the scanner variables; and

determine imaging parameters for the medical scanner based on the object keypoints for imaging a subject.

2 . The system of claim 1 , wherein the neural network is based on a U-Net neural network architecture.

3 . The system of claim 1 , wherein the camera system comprises exactly one camera.

4 . The system of claim 1 , wherein the medical scanner is a magnetic resonance imaging (MRI) scanner comprising:

a table for positioning the subject; and

an imaging section for obtaining medical images, and

wherein the scanner variables include a position of the table relative to the imaging section.

5 . The system of claim 1 , wherein determining the object keypoints is further based on triangulating the object keypoint projections in a first image taken before a movement and the object keypoint projections in a second image taken after the movement based on distance covered during the movement.

6 . A method for obtaining object keypoints for an object in a medical scanner, wherein the object keypoints are three dimensional (3D) coordinates with respect to the medical scanner and wherein the object keypoints are defined inside object parts of the object, the method comprising:

obtaining two dimensional (2D) images of the object in the medical scanner from a camera system, wherein the camera system comprises one or more cameras;

obtaining scanner variables from at least one of the medical scanner or the camera system, wherein the medical scanner contains a first moving part and/or wherein the camera system contains a second moving part, and wherein the scanner variables include a position of the first moving part of the medical scanner and/or a position of the second moving part of the camera system;

determining object keypoint projections based on inputting the 2D images into a neural network trained to learn spatial relationships between object keypoint projections corresponding to different parts of the object, wherein the neural network is configured to output the object keypoint projections in each one of the input 2D images, wherein the object keypoint projections are 2D coordinates projected on to the 2D images, respectively;

determining the object keypoints in 3D coordinates by triangulating the object keypoint projections with respect to the camera system based on the scanner variables; and

determining imaging parameters for the medical scanner based on the object keypoints for imaging a subject.

7 . The method of claim 6 , wherein the neural network is based on a U-Net neural network architecture.

8 . The method of claim 6 , wherein the camera system comprises exactly one camera.

9 . The method of claim 6 , wherein the medical scanner is a magnetic resonance imaging (MRI) scanner comprising:

a table for positioning the subject; and

an imaging section for obtaining medical images, and

wherein the scanner variables include a position of the table relative to the imaging section.

10 . The method of claim 6 , wherein determining the object keypoints is further based on triangulating the object keypoint projections in a first image taken before a movement and the object keypoint projections in a second image taken after the movement based on distance covered during the movement.

11 . A non-transitory computer readable medium storing computer program code for obtaining object keypoints for an object in a medical scanner, wherein when executed on a computing device having at least one processor, the computer program code causes the at least one processor to:

obtain two dimensional (2D) images of the object in the medical scanner from a camera system, wherein the camera system comprises one or more cameras;

obtain scanner variables from at least one of the medical scanner or the camera system, wherein the medical scanner contains a first moving part and/or wherein the camera system contains a second moving part, and wherein the scanner variables include a position of the first moving part of the medical scanner and/or a position of the second moving part of the camera system;

determine object keypoint projections based on inputting the 2D images of the object into a neural network trained to learn spatial relationships between object keypoint projections corresponding to different parts of the object, wherein the neural network is configured to output the object keypoint projections in each one of the input 2D images, wherein the object keypoint projections are 2D coordinates projected on to the 2D images, respectively;

determine the object keypoints by triangulating the object keypoint projections with respect to the camera system based on the scanner variables, wherein the object keypoints are three dimensional (3D) coordinates with respect to the medical scanner and wherein the object keypoints are defined in an object part of the object; and

determine imaging parameters for the medical scanner based on the object keypoints for imaging a subject.

12 . The non-transitory computer readable medium of claim 11 , wherein the neural network is based on a U-Net neural network architecture.

13 . The non-transitory computer readable medium of claim 11 , wherein the camera system comprises exactly one camera.

14 . The non-transitory computer readable medium of claim 11 , wherein the medical scanner is a magnetic resonance imaging (MRI) scanner comprising:

a table for positioning the subject; and

an imaging section for obtaining medical images, and

wherein the scanner variables include a position of the table relative to the imaging section.

15 . The non-transitory computer readable medium of claim 11 , wherein the computer program code causes the at least one processor to determine the object keypoints further based on triangulating the object keypoint projections in a first image taken before a movement and the object keypoint projections in a second image taken after the movement based on distance covered during the movement.

16 . The non-transitory computer readable medium of claim 11 , wherein the imaging parameters comprise at least one of an imaging plane or a positioning of the medical scanner.