Method and system of determining operation pathway based on image matching

1. A system to determine an operation pathway for a patient, comprising:

a processor; and

a non-transitory computer-readable medium having stored thereon program code that, upon being executed by the processor, causes the processor to:

construct a three-dimensional model based on a medical image scan of the patient;

obtain image information of the patient with a set of two-dimensional optical devices;

select a first set of two-dimensional feature points associated with the three-dimensional model;

select a second set of two-dimensional feature points associated with the image information of the patient;

transform the first set of two-dimensional feature points to a first set of three-dimensional feature points;

transform the second set of two-dimensional feature points to a second set of three-dimensional feature points;

match between the first set of three-dimensional feature points and the second set of three-dimensional feature points to determine a relationship that aligns the first set of three-dimensional feature points and the second set of three-dimensional feature points;

transform coordinates from a first coordinate system associated with the three-dimensional model to a second coordinate system associated with the set of two-dimensional optical devices based on the relationship; and

determine the operation pathway in a third coordinate system associated with a robotic arm based on the transformed coordinates in the second coordinate system;

wherein the program code, upon being executed by the processor, causes the processor further to generate a two-dimensional snapshot of the three-dimensional model and select the first set of two-dimensional feature points in the snapshot.

2. The system of claim 1 , wherein transforming the first set of two-dimensional feature points to the first set of three-dimensional feature points is based on a reverse operation associated with the snapshot.

3. The system of claim 1 , wherein the image information of the patient includes two-dimensional image information of the patient and a depth information associated with the patient.

4. The system of claim 3 , wherein the second set of two-dimensional feature points is selected in the two-dimensional image information of the patient.

5. The system of claim 4 , wherein transforming the second set of two-dimensional feature points to the second set of three-dimensional feature points is based on the depth information.

6. A method to determine an operation pathway for a patient, comprising:

constructing a three-dimensional model based on a medical image scan of the patient;

obtaining image information of the patient with a set of two-dimensional optical devices;

selecting a first set of two-dimensional feature points associated with the three-dimensional model;

selecting a second set of two-dimensional feature points associated with the image information of the patient;

transforming the first set of two-dimensional feature points to a first set of three-dimensional feature points;

transforming the second set of two-dimensional feature points to a second set of three-dimensional feature points;

matching between the first set of three-dimensional feature points and the second set of three-dimensional feature points to determine a relationship that aligns the first set of three-dimensional feature points and the second set of three-dimensional feature points;

transforming coordinates from a first coordinate system associated with the three-dimensional model to a second coordinate system associated with the set of two-dimensional optical devices based on the relationship;

determining the operation pathway in a third coordinate system associated with a robotic arm based on the transformed coordinates in the second coordinate system;

generating a two-dimensional snapshot of the three-dimensional model; and

selecting the first set of two-dimensional feature points in the snapshot.

7. The method of claim 6 , wherein transforming the first set of two-dimensional feature points to the first set of three-dimensional feature points is based on a reverse operation associated with the snapshot.

8. The method of claim 6 , wherein the image information of the patient includes two-dimensional image information of the patient and a depth information associated with the patient.

9. The method of claim 8 , wherein the second set of two-dimensional feature points is selected in the two-dimensional image information of the patient.

10. The method of claim 9 , wherein transforming the second set of two-dimensional feature points to the second set of three-dimensional feature points is based on the depth information.

11. The method of claim 6 , wherein any of the first set of two-dimensional feature points and the second set of two-dimensional feature points is substantially planar on a face of the patient.

12. A non-transitory computer-readable storage medium that includes a set of instructions which, in response to execution by a processor, cause the processor to perform a method to determine an operation pathway for a patient, wherein the method comprises:

constructing a three-dimensional model based on a medical image scan of the patient;

obtaining image information of the patient with a set of two-dimensional optical devices;

selecting a first set of two-dimensional feature points associated with the three-dimensional model;

selecting a second set of two-dimensional feature points associated with the image information of the patient;

transforming the first set of two-dimensional feature points to a first set of three-dimensional feature points;

transforming the second set of two-dimensional feature points to a second set of three-dimensional feature points;

matching between the first set of three-dimensional feature points and the second set of three-dimensional feature points to determine a relationship that aligns the first set of three-dimensional feature points and the second set of three-dimensional feature points;

transforming coordinates from a first coordinate system associated with the three-dimensional model to a second coordinate system associated with the set of two-dimensional optical devices based on the relationship;

determining the operation pathway in a third coordinate system associated with a robotic arm based on the transformed coordinates in the second coordinate system;

generating a two-dimensional snapshot of the three-dimensional model; and

selecting the first set of two-dimensional feature points in the snapshot.

13. The non-transitory computer-readable storage medium of claim 12 , wherein transforming the first set of two-dimensional feature points to the first set of three-dimensional feature points is based on a reverse operation associated with the snapshot.

14. The non-transitory computer-readable storage medium of claim 12 , wherein the image information of the patient includes two-dimensional image information of the patient and a depth information associated with the patient.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the second set of two-dimensional feature points is selected in the two-dimensional image information of the patient.

16. The non-transitory computer-readable storage medium of claim 15 , wherein transforming the second set of two-dimensional feature points to the second set of three-dimensional feature points is based on the depth information.