Self-service checkout counter checkout

1. A computer-implemented method for self-service checkout, comprising:

obtaining, by using a camera, an image, wherein the image is of at least one product placed on a checkout counter;

performing image segmentation on the image to obtain at least one image region;

identifying a product code included in a code region in an image region of the at least one image region;

determining that a product category of a product associated with the product code fails to be identified;

in response to determining that the product category fails to be identified, determining the product category by using an object detection model obtained by pre-training using labeled training sample images, each labeled training sample image comprising a rectangle that frames the product code to label the code region, wherein the object detection model generates a candidate image region in the image, performs target identification within the candidate image region, performs bounding box regression in the candidate image region, and corrects the code region by using a perspective transformation relative to position markers to obtain a corrected code region, and determines the product category based on the corrected code region; and

determining a price of the product based on the product category.

2. The computer-implemented method of claim 1 , wherein obtaining the image comprises:

controlling the camera to face the at least one product to obtain the image.

3. The computer-implemented method of claim 1 , wherein obtaining the image comprises receiving the image from a self-service checkout counter.

4. The computer-implemented method of claim 3 , wherein the image is taken from one of a top direction, a front direction, a left direction, a right direction, a rear direction, and an oblique direction of the at least one product, and an angle between the oblique direction and a direction vertical to a countertop of the self-service checkout counter is between 30 degrees to 60 degrees.

5. The computer-implemented method of claim 1 , wherein the image segmentation is performed by using an image segmentation model, and the image segmentation model is obtained by pre-training segmentation sample images comprising product images and annotations of silhouettes of products.

6. The computer-implemented method of claim 1 , wherein the product code is a barcode, and identifying the product code comprises:

detecting a barcode region in the image region by using an object detection model, the object detection model is obtained by pre-training training sample images comprising product images and annotations for identifying barcode regions in the product images.

7. The computer-implemented method of claim 6 , wherein identifying the product code comprises:

adjusting the barcode region based on the perspective transformation to obtain an adjusted barcode region; and

identifying the barcode in the adjusted barcode region.

8. The computer-implemented method of claim 1 , wherein the product code is a two-dimensional code, and identifying the product code comprises:

detecting position markers of the two-dimensional code in the image region; and

determining that a two-dimensional code region is detected in response to detecting at least two positioning markers.

9. The computer-implemented method of claim 8 , wherein identifying the product code comprises:

adjusting the two-dimensional code region based on the perspective transformation to obtain an adjusted two-dimensional code;

determining a corner-module relationship in the adjusted two-dimensional code based on the at least two positioning markers; and

identifying the two-dimensional code based on the corner-module relationship.

10. The computer-implemented method of claim 1 , wherein identifying the product category of the product associated with the product further comprises:

determining the product category based on an object detection model, wherein the object detection model is obtained by pre-training training sample images comprising product images and annotations for identifying products and product categories.

11. The computer-implemented method of claim 1 , wherein the image is a first image, the camera is a first camera, the at least one image region is a first at least one image region, the product is a first product, and the method further comprising:

obtaining, by using a second camera, a second image, wherein the second image is of the at least one product;

performing image segmentation on the second image to obtain a second at least one image region;

determining a product category of a second product associated with the second at least one image region based on a product code or visual identification; and

determining, based on a relative position between the first camera and the second camera, that the first at least one image region and the second at least one image region are associated with an identical product.

12. A computer-implemented system for self-service checkout, comprising:

one or more computers; and

one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations, comprising:

obtaining, by using a camera, an image, wherein the image is of at least one product placed on a checkout counter;

performing image segmentation on the image to obtain at least one image region;

identifying a product code included in a code region in an image region of the at least one image region;

determining that a product category of a product associated with the product code fails to be identified;

in response to determining that the product category fails to be identified, determining the product category by using an object detection model obtained by pre-training using labeled training sample images, each labeled training sample image comprising a rectangle that frames the product code to label the code region, wherein the object detection model generates a candidate image region in the image, performs target identification within the candidate image region, performs bounding box regression in the candidate image region, and corrects the code region by using a perspective transformation relative to position markers to obtain a corrected code region, and determines the product category based on the corrected code region; and

determining a price of the product based on the product category.

13. The computer-implemented system of claim 12 , wherein obtaining the image comprises:

controlling the camera to face the at least one product to obtain the image.

14. The computer-implemented system of claim 12 , wherein obtaining the image comprises receiving the image from a self-service checkout counter.

15. The computer-implemented system of claim 14 , wherein the image is taken from one of a top direction, a front direction, a left direction, a right direction, a rear direction, and an oblique direction of the at least one product, and an angle between the oblique direction and a direction vertical to a countertop of the self-service checkout counter is between 30 degrees to 60 degrees.

16. The computer-implemented system of claim 12 , wherein the image segmentation is performed by using an image segmentation model, the image segmentation model is obtained by pre-training segmentation sample images comprising product images and annotations of silhouettes of products.

17. The computer-implemented system of claim 12 , wherein the product code is a barcode, and identifying the product code comprises:

detecting a barcode region in the image region by using an object detection model, the object detection model is obtained by pre-training training sample images comprising product images and annotations for identifying barcode regions in the product images.

18. The computer-implemented system of claim 17 , wherein identifying the product code comprises:

adjusting the barcode region based on the perspective transformation to obtain an adjusted barcode region; and

identifying the barcode in the adjusted barcode region.

19. The computer-implemented system of claim 12 , wherein the product code is a two-dimensional code, and identifying the product code comprises:

detecting position markers of the two-dimensional code in the image region; and

determining that a two-dimensional code region is detected in response to detecting at least two positioning markers.

20. The computer-implemented system of claim 19 , wherein identifying the product code comprises:

Adjusting the two-dimensional code region based on the perspective transformation to obtain an adjusted two-dimensional code;

determining a corner-module relationship in the adjusted two-dimensional code based on the at least two positioning markers; and

identifying the two-dimensional code based on the corner-module relationship.

21. A non-transitory, computer-readable medium storing one or more instructions executable by a computer-implemented system to perform one or more operations for self-service checkout comprising:

obtaining, by using a camera, an image, wherein the image is of at least one product placed on a checkout counter;

performing image segmentation on the image to obtain at least one image region;

identifying a product code included in a code region in an image region of the at least one image region;

determining that a product category of a product associated with the product code fails to be identified;

in response to determining that the product category fails to be identified, determining the product category by using an object detection model obtained by pre-training using labeled training sample images, each labeled training sample image comprising a rectangle that frames the product code to label the code region, wherein the object detection model generates a candidate image region in the image, performs target identification within the candidate image region, performs bounding box regression in the candidate image region, and corrects the code region by using a perspective transformation relative to position markers to obtain a corrected code region, and determines the product category based on the corrected code region; and

determining a price of the product based on the product category.

22. The non-transitory, computer-readable medium of claim 21 , wherein obtaining the image comprises:

controlling the camera to face the at least one product placed on the checkout counter to obtain the image.

23. The non-transitory, computer-readable medium of claim 21 , wherein obtaining the image comprises receiving the image from a self-service checkout counter.

24. The non-transitory, computer-readable medium of claim 23 , wherein the image is taken from one of a top direction, a front direction, a left direction, a right direction, a rear direction, and an oblique direction of the at least one product placed on the self-service checkout counter, and an angle between the oblique direction and a direction vertical to a countertop of the self-service checkout counter is between 30 degrees to 60 degrees.

25. The non-transitory, computer-readable medium of claim 21 , wherein the image segmentation is performed by using an image segmentation model, the image segmentation model is obtained by pre-training segmentation sample images comprising product images and annotations of silhouettes of products.

26. The non-transitory, computer-readable medium of claim 21 , wherein the product code is a barcode, and identifying the product code comprises:

detecting a barcode region in the image region by using an object detection model, the object detection model is obtained by pre-training training sample images comprising product images and annotations for identifying barcode regions in the product images.

27. The non-transitory, computer-readable medium of claim 26 , wherein identifying the product code comprises:

adjusting the barcode region based on the perspective transformation to obtain an adjusted barcode region; and

identifying the barcode in the adjusted barcode region.

28. The non-transitory, computer-readable medium of claim 21 , wherein the product code is a two-dimensional code, and identifying the product code comprises:

detecting position markers of the two-dimensional code in the image region; and

determining that a two-dimensional code region is detected in response to detecting at least two positioning markers.

29. The non-transitory, computer-readable medium of claim 28 , wherein identifying the product code comprises:

adjusting the two-dimensional code region based on the perspective transformation to obtain an adjusted two-dimensional code;

determining a corner-module relationship in the adjusted two-dimensional code based on the at least two positioning markers; and

identifying the two-dimensional code based on the corner-module relationship.

30. The non-transitory, computer-readable medium of claim 21 , wherein identifying the product category of the product associated with the product further comprises:

determining the product category based on an object detection model, wherein the object detection model is obtained by pre-training training sample images comprising product images and annotations for identifying products and product categories.