IP Library Granted Patent US 9,607,199
Granted Patent B2
US 9,607,199 · App. 15/008,876 · Granted Mar 28, 2017

Decoding machine-readable optical codes with aesthetic component

Inventor: Andrew Stein (San Francisco, CA)
Assignee: Anki, Inc.
G06K7/1443G06K7/12G06K7/1417G06K7/1447G06K19/0614G06K19/06037G06K19/06056G06K19/06103
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,607,199
App. No.
15/008,876
Granted
Mar 28, 2017
Kind
B2
Abstract

Techniques are provided for decoding machine-readable optical codes that have an aesthetic component that is integrated into the codes themselves. In this manner, the machine-readable optical codes can be designed to be aesthetically pleasing and/or can convey information to human viewers, and can even be disguised so that they do not appear to be machine-readable optical codes at all. Such information can be (but need not be) distinct from the information encoded for reading by a machine, even when the information is integrated into the code itself. The techniques described herein can be applied to any type of machine-readable optical code.

Claims (135)

1. A computer-implemented method for detecting a fiducial containing a machine-readable optical code, comprising:

at an image capture device, capturing an input image;

at a processor, generating a binary image from the input image;

at the processor, identifying connected components within the binary image;

at the processor, identifying quadrilaterals within the connected components;

at the processor, filtering the identified quadrilaterals to generate a set of at least one quadrilateral; and

at an output device, outputting the set of at least one quadrilateral as a set of at least one detected fiducial containing a machine-readable optical code;

wherein generating a binary image from the input image comprises:

determining a local scale for each of a plurality of locations within the image, to generate a locally averaged image; and

at each location, comparing the locally averaged image to a threshold.

2. The computer-implemented method of claim 1 , wherein the machine-readable optical code comprises an aesthetic component.

3. The computer-implemented method of claim 1 , further comprising, prior to identifying quadrilaterals, filtering the connected components to determine which regions are above a threshold metric to represent meaningful data.

4. The computer-implemented method of claim 3 , wherein the threshold metric comprises at least one of size and solidity.

5. The computer-implemented method of claim 1 , further comprising, prior to outputting the set of at least one quadrilateral, refining the at least one quadrilateral using image-based alignment.

6. A computer-implemented method for determining a match for a machine-readable optical code, comprising:

at an image capture device, capturing an input image;

at a processor, identifying connected components within the input image;

at the processor, identifying quadrilaterals within the connected components;

at the processor, filtering the identified quadrilaterals to generate a set of at least one detected fiducial containing a machine-readable optical code;

at the processor, obtaining a canonical fiducial containing canonical probe locations;

at the processor, determining a perspective transformation of the at least one detected fiducial as compared with the canonical fiducial;

at the processor, applying the perspective transformation to the canonical probe locations to obtain transformed probe locations;

at the processor, determining at least one probe value for each transformed probe location;

at the processor, determining binary probe values from the determined probe values;

at the processor, comparing the determined binary probe values with a code library to identify a matched code; and

at an output device, outputting the matched code.

7. The computer-implemented method of claim 6 , wherein determining at least one probe value for each transformed probe location comprises, for each transformed probe location:

determining a plurality of probe values; and

averaging the probe values.

8. The computer-implemented method of claim 6 , wherein comparing the determined binary probe values with a code library to identify a matched code comprises determining an orientation based on at least one orientation marker.

9. A system for detecting a fiducial containing a machine-readable optical code, comprising:

an image capture device, configured to capture an input image;

a hardware processor, communicatively coupled to the image capture device, configured to perform the steps of:

generating a binary image from the input image;

identifying connected components within the binary image;

identifying quadrilaterals within the connected components; and

filtering the identified quadrilaterals to generate a set of at least one quadrilateral; and

an output device, communicatively coupled to the hardware processor, configured to output the set of at least one quadrilateral as a set of at least one detected fiducial containing a machine-readable optical code;

wherein the hardware processor generating a binary image from the input image comprises:

the hardware processor determining a local scale for each of a plurality of locations within the image, to generate a locally averaged image; and

at each location, the hardware processor comparing the locally averaged image to a threshold.

10. The system of claim 9 , wherein the machine-readable optical code comprises an aesthetic component.

11. The system of claim 9 , wherein the hardware processor is further configured to, prior to identifying quadrilaterals, filter the connected components to determine which regions are above a threshold metric to represent meaningful data.

12. The system of claim 11 , wherein the threshold metric comprises at least one of size and solidity.

13. The system of claim 9 , wherein the hardware processor is further configured to, prior to the output device outputting the set of at least one quadrilateral, refine the at least one quadrilateral using image-based alignment.

14. A system for determining a match for a machine-readable optical code, comprising:

an image capture device, configured to capture an input image;

a hardware processor, communicatively coupled to the image capture device, configured to perform the steps of:

identifying connected components within the input image;

identifying quadrilaterals within the connected components; and

filtering the identified quadrilaterals to generate a set of at least one detected fiducial containing a machine-readable optical code;

obtaining a canonical fiducial containing canonical probe locations;

determining a perspective transformation of the detected fiducial as compared with the canonical fiducial;

applying the perspective transformation to the canonical probe locations to obtain transformed probe locations;

determining at least one probe value for each transformed probe location;

determining binary probe values from the determined probe values; and

comparing the determined binary probe values with a code library to identify a matched code; and

an output device, communicatively coupled to the hardware processor, configured to output the matched code.

15. The system of claim 14 , wherein the hardware processor determining at least one probe value for each transformed probe location comprises, for each transformed probe location:

determining a plurality of probe values; and

averaging the probe values.

16. The system of claim 14 , wherein the hardware processor comparing the determined binary probe values with a code library to identify a matched code comprises determining an orientation based on at least one orientation marker.

17. A non-transitory computer-readable medium for detecting a fiducial containing a machine-readable optical code, comprising instructions stored thereon, that when executed on a processor, perform the steps of:

causing an image capture device to capture an input image;

generating a binary image from the input image;

identifying connected components within the binary image;

identifying quadrilaterals within the connected components;

filtering the identified quadrilaterals to generate a set of at least one quadrilateral; and

causing an output device to output the set of at least one quadrilateral as a set of at least one fiducial containing a machine-readable optical code;

wherein generating a binary image from the input image comprises:

determining a local scale for each of a plurality of locations within the image, to generate a locally averaged image; and

at each location, comparing the locally averaged image to a threshold.

18. The non-transitory computer-readable medium of claim 17 , wherein the machine-readable optical code comprises an aesthetic component.

19. The non-transitory computer-readable medium of claim 17 , further comprising instructions that when executed on the processor, perform the step of, prior to identifying quadrilaterals, filtering the connected components to determine which regions are above a threshold metric to represent meaningful data.

20. The non-transitory computer-readable medium of claim 19 , wherein the threshold metric comprises at least one of size and solidity.

21. The non-transitory computer-readable medium of claim 17 , wherein the non-transitory computer-readable medium further comprises instructions stored thereon, that when executed on a processor, perform the step of, prior to outputting the set of at least one quadrilateral, refining the at least one quadrilateral using image-based alignment.

22. A non-transitory computer-readable medium for determining a match for a machine-readable optical code, comprising instructions stored thereon, that when executed on a processor, perform the steps of:

causing an image capture device to capture an input image;

identifying connected components within the input image;

identifying quadrilaterals within the connected components;

filtering the identified quadrilaterals to generate a set of at least one detected fiducial containing a machine-readable optical code;

obtaining a canonical fiducial containing canonical probe locations;

determining a perspective transformation of the detected fiducial as compared with the canonical fiducial;

applying the perspective transformation to the canonical probe locations to obtain transformed probe locations;

determining at least one probe value for each transformed probe location;

determining binary probe values from the determined probe values;

comparing the determined binary probe values with a code library to identify a matched code; and

causing an output device to output the matched code.

23. The non-transitory computer-readable medium of claim 22 , wherein determining at least one probe value for each transformed probe location comprises, for each transformed probe location:

determining a plurality of probe values; and

averaging the probe values.

24. The non-transitory computer-readable medium of claim 22 , wherein comparing the determined binary probe values with a code library to identify a matched code comprises determining an orientation based on at least one orientation marker.

25. A computer-implemented method for decoding a machine-readable optical code contained in a detected fiducial, comprising:

at an image capture device, capturing an input image;

at a processor, detecting a fiducial within the input image, the fiducial comprising a machine-readable optical code;

at the processor, obtaining a canonical fiducial containing canonical probe locations;

at the processor, based on a comparison of the detected fiducial with the canonical fiducial, obtaining transformed probe locations;

at the processor, determining at least one probe value for each transformed probe location;

at the processor, determining binary probe values from the determined probe values;

at the processor, comparing the determined binary probe values with a code library to identify a matched code; and

an output device, outputting the matched code.

26. The computer-implemented method of claim 25 , wherein determining at least one probe value for each transformed probe location comprises, for each transformed probe location:

determining a plurality of probe values; and

averaging the probe values.

27. The computer-implemented method of claim 25 , wherein comparing the determined binary probe values with a code library to identify a matched code comprises determining an orientation based on at least one orientation marker.

28. A system for decoding a machine-readable optical code contained in a detected fiducial, comprising:

an image capture device, configured to capture an input image;

a hardware processor, communicatively coupled to the image capture device, configured to perform the steps of:

detecting a fiducial within the input image, the fiducial comprising a machine-readable optical code;

obtaining a canonical fiducial containing canonical probe locations;

based on a comparison of the detected fiducial with the canonical fiducial, obtaining transformed probe locations;

determining at least one probe value for each transformed probe location;

determining binary probe values from the determined probe values; and

comparing the determined binary probe values with a code library to identify a matched code; and

an output device, communicatively coupled to the hardware processor, configured to output the matched code.

29. The system of claim 28 , wherein the hardware processor determining at least one probe value for each transformed probe location comprises, for each transformed probe location:

determining a plurality of probe values; and

averaging the probe values.

30. The system of claim 28 , wherein the hardware processor comparing the determined binary probe values with a code library to identify a matched code comprises determining an orientation based on at least one orientation marker.

31. A non-transitory computer-readable medium for decoding a machine-readable optical code contained in a detected fiducial, comprising instructions stored thereon, that when executed on a processor, perform the steps of:

causing an image capture device to capture an input image;

detecting a fiducial within the input image, the fiducial comprising a machine-readable optical code;

obtaining a canonical fiducial containing canonical probe locations;

based on a comparison of the detected fiducial with the canonical fiducial, obtaining transformed probe locations;

determining at least one probe value for each transformed probe location;

determining binary probe values from the determined probe values;

comparing the determined binary probe values with a code library to identify a matched code; and

causing an output device to output the matched code.

32. The non-transitory computer-readable medium of claim 31 , wherein determining at least one probe value for each transformed probe location comprises, for each transformed probe location:

determining a plurality of probe values; and

averaging the probe values.

33. The non-transitory computer-readable medium of claim 31 , wherein comparing the determined binary probe values with a code library to identify a matched code comprises determining an orientation based on at least one orientation marker.

34. The non-transitory computer-readable medium of claim 31 , wherein the non-transitory computer-readable medium further comprises instructions stored thereon, that when executed on a processor, perform the step of, prior to obtaining the transformed probe locations:

determining a perspective transformation of the detected fiducial as compared with the canonical fiducial;

and wherein obtaining transformed probe locations comprises applying the perspective transformation to the canonical probe locations to obtain the transformed probe locations.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 28, 2022
From: DIGITAL DREAM LABS, LLC
To: DIGITAL DREAM LABS, INC.
Reel/Frame 059819/0720 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2020
From: DSI ASSIGNMENTS, LLC
To: DIGITAL DREAM LABS, LLC
Reel/Frame 052211/0235 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2020
From: ANKI, INC.
To: DSI ASSIGNMENTS, LLC
Reel/Frame 052190/0487 →
RELEASE OF SECURITY INTEREST Recorded Jan 6, 2020
From: SILICON VALLEY BANK
To: ANKI, INC.
Reel/Frame 051485/0600 →
SECURITY INTEREST Recorded Apr 4, 2018
From: ANKI, INC.
To: SILICON VALLEY BANK
Reel/Frame 046231/0312 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 28, 2016
From: STEIN, ANDREW
To: ANKI, INC.
Reel/Frame 037609/0639 →
Continuity (3)
Continuation 14682483 · Apr 9, 2015
Provisional Application 61978113 · Apr 10, 2014
Related Publication 20160162720A1 · Jun 9, 2016