Methods and systems for processing an image

1. A computer-implemented method for processing an image of a machine-readable code affixed to an item of a surgical system, the surgical system including a display and one or more processors, the method comprising:

accessing, by the one or more processors, the image that depicts the machine-readable code;

determining, with the one or more processors, at least one corner of the machine-readable code is occluded, false, or blurry such that a machine-readable code region of the image is not locatable;

providing, with the one or more processors, the image as an input to a trained neural network;

receiving, with the one or more processors and as an output from the trained neural network, a cleaned image of the machine-readable code;

locating, with the one or more processors, the machine-readable code region of the cleaned image of the machine-readable code;

decoding, with the one or more processors, coded information contained within the machine-readable code region of the cleaned image;

outputting, with the one or more processors, the decoded information; and

displaying, on the display, patient information generated by the surgical system based on the decoded information.

2. The method of claim 1 , wherein the step of locating the machine-readable code region further comprises performing at least one of corner detection or edge detection by:

applying a corner detection algorithm to the cleaned image to generate a heatmap including heatmap values corresponding to each coordinate within the image; and

detecting the corners of the machine-readable code in the cleaned image based on the heatmap generated by the corner detect algorithm.

3. The method of claim 1 , further comprising, with the trained neural network:

identifying a set of coordinates in the image as likely corresponding to the machine-readable code;

identifying at least one outlier coordinate of the set of coordinates that is unlikely to correspond to the machine-readable code; and

removing the at least one outlier coordinate to generate a cleaned set of coordinates.

4. The method of claim 3 , wherein the step of decoding the coded information further comprises:

identifying the cleaned set of coordinates; and

analyzing a portion of the cleaned image which corresponds to the cleaned set of coordinates.

5. The method of claim 1 , further comprising decreasing high frequency noise of the image to generate the cleaned image.

6. The method of claim 5 , further comprising applying at least one smoothing algorithm to decrease the high frequency noise of the image.

7. The method of claim 5 , further comprising applying a bilateral filter on the image to decrease the high frequency noise of the image while preserving high frequency signals.

8. The method of claim 1 , further comprising increasing a signal to noise ratio of the image to generate the cleaned image.

9. The method of claim 8 , further comprising applying at least one deblurring algorithm to the image to increase the signal to noise ratio.

10. The method of claim 1 , wherein the step of determining that at least one corner of the machine-readable code is occluded, false, or blurry further comprises:

applying a corner detection algorithm to the image; and

identifying the at least one corner of the machine-readable code as occluded, false, or blurry by analyzing an output of the corner detection algorithm.

11. The method of claim 1 , wherein the trained neural network is trained on manually cleaned images of machine-readable code and/or images of obscured machine-readable codes.

12. A computer-implemented method for processing an image of a machine-readable code affixed to an item of a surgical system, the surgical system including a display and one or more processors, the method comprising:

accessing, by one or more processors, an image that depicts a machine-readable code;

determining, by the one or more processors, that at least one corner of the machine-readable code is occluded, false, or blurry;

providing, by the one or more processors, the image as an input to a trained neural network;

receiving, by the one or more processors and as an output from the trained neural network, a cleaned image of the machine-readable code;

applying a corner detection algorithm to the cleaned image to generate a heatmap including heatmap values corresponding to each coordinate within the image;

detecting the corners of the machine-readable code in the cleaned image based on the heatmap generated by the corner detection algorithm;

decoding, with the one or more processors, coded information contained within the machine-readable code of the cleaned image;

outputting, with the one or more processors, the decoded information; and

displaying, on the display, patient information generated by the surgical system based on the decoded information.

13. The method of claim 12 , further comprising decreasing high frequency noise of the image to generate the cleaned image.

14. The method of claim 12 , further comprising increasing a signal to noise ratio of the image to generate the cleaned image.

15. The method of claim 12 , further comprising, with the trained neural network:

identifying a set of coordinates in the image as likely corresponding to the machine-readable code;

identifying outlier coordinates of the set of coordinates that are unlikely to correspond to the machine-readable code; and

removing the outlier coordinates to generate a cleaned set of coordinates.

16. A computer-implemented method for processing an image of a machine-readable code affixed to an item of a surgical system, the surgical system including a display and one or more processors, the method comprising:

accessing, by one or more processors, an image that depicts a machine-readable code;

determining, by the one or more processors, that at least one corner of the machine-readable code is occluded, false, or blurry;

providing, by the one or more processors, the image as an input to a trained neural network;

processing, with the trained neural network, at least a region of the image to generate a cleaned image by:

identifying a set of coordinates in the image that likely correspond to the machine-readable code;

removing outlier coordinates from the set of coordinates to generate a cleaned set of coordinates which correspond to an estimated position of the machine-readable code;

receiving, by the one or more processors and as an output from the trained neural network, the cleaned image of the machine-readable code;

decoding, with the one or more processors, coded information contained within the machine-readable code of the cleaned image;

outputting, with the one or more processors, the decoded information; and

displaying, on the display, patient information generated by the surgical system based on the decoded information.

17. The method of claim 16 , wherein the step of decoding the coded information further comprises:

identifying the cleaned set of coordinates; and

analyzing a portion of the cleaned image which corresponds to the cleaned set of coordinates.

18. The method of claim 16 , further comprising decreasing high frequency noise of the image to generate the cleaned image.

19. The method of claim 18 , further comprising applying at least one smoothing algorithm to decrease the high frequency noise of the image.

20. The method of claim 16 , wherein the step of locating the machine-readable code further comprises performing at least one of corner detection or edge detection by:

applying a corner detection algorithm to the cleaned image to generate a heatmap including heatmap values corresponding to each coordinate within the image; and

detecting the corners of the machine-readable code in the cleaned image based on the heatmap generated by the corner detect algorithm.