IP Library Granted Patent US 12,131,297
Granted Patent B2
US 12,131,297 · App. 16/710,320 · Granted Oct 29, 2024

Automatic detection and recognition of visual tags

Inventors: Ryan R. Fink (Vancouver, WA); Sean M. Adkinson (North Plains, OR)
Assignee: STREEM, LLC
G06Q10/20G06F16/903G06T7/73G06V20/20G06V30/1444G06V30/412G06V30/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,131,297
App. No.
16/710,320
Granted
Oct 29, 2024
Kind
B2
Abstract

Methods and systems for automatic detection and recognition of visual tags on equipment are disclosed. The make and model of an object such as an appliance or consumer device is recognized from an image or video using object detection. This make and model information may then be used to direct a user to locate an equipment information tag that includes model and serial number information. Object recognition and optical character recognition can then be employed to extract the model and serial number from the tag, along with any other relevant information. The extracted information may then be used to locate service and/or operation information.

Claims (49)

1. A method comprising:

receiving, from a user device, an image of an object where an equipment information is not visible from an external view of the object, the equipment information comprising at least make, model, and serial number information for the object;

determining, from the image of the object, a location of the equipment information on the object and a layout of the equipment information, wherein the equipment information comprises a visual tag, and determining the location of the equipment information from the image of the object comprises:

detecting, in the image of the object, a shape and position of one or more features of the object;

searching, with the detected shape and position of the one or more features of the object, into a database to determine a likely make and model of the object; and

obtaining, with the likely make and model of the object, instructions for locating the equipment information;

providing, to the user device, the instructions to assist a user of the user device in locating the location on the object of the equipment information, the instructions comprising a silhouette or shape of the visual tag to be presented on a display of the user device;

receiving, from a user device, an image of the equipment information;

extracting, from the image of the equipment information, identifying information about the object;

using the identifying information to query into a database containing service information about the object; and

transmitting, to a second user device in two-way communication with the user device, the service information about the object.

2. The method of claim 1 , wherein determining a layout of the equipment information further comprises:

determining, from the make and/or model of the object, a format of the visual tag; and

using the format of the visual tag to extract the identifying information from the image of the equipment information.

3. The method of claim 1 , wherein extracting the identifying information about the object comprises:

performing character recognition on the image of the equipment information; and

parsing any recognized characters to obtain the identifying information.

4. The method of claim 1 , further comprising transmitting, to the user device, service information obtained in response to the query into the database.

5. The method of claim 1 , wherein providing instructions for locating the location of the equipment information comprises providing one or more augmented reality objects for display on the user device that visually indicate the location of the equipment information.

6. A non-transitory computer-readable medium (CRM) comprising instructions that, when executed by an apparatus, cause the apparatus to:

receive, from a user device, an image of an object where an equipment information is not visible from an external view of the object, the equipment information comprising at least make, model, and serial number information for the object;

determine, from the object image, a location of the equipment information on the object and a layout of the equipment information, wherein the equipment information comprises a visual tag, and to determine the location of the equipment information from the image of the object, the instructions further cause the apparatus to:

detect, in the image of the object, a shape and position of one or more features of the object;

search, with the detected shape and position of the one or more features of the object, into a database to determine a likely make and model of the object; and

obtain, with the likely make and model of the object, instructions for locating the equipment information;

transmit, to the user device, the instructions to assist a user of the user device in locating the location on the object of the equipment information, wherein the instructions comprise a silhouette or shape of the visual tag to be presented on a display of the user device;

receive, from the user device in response to transmission of the location of the equipment information, an image of the equipment information; and

extract, from the image of the equipment information, identifying information about the object;

use the identifying information to query into a database containing service information about the object; and

transmit, to a second user device in two-way communication with the user device, the service information about the object.

7. The CRM of claim 6 , wherein the instructions are to further cause the apparatus to:

determine, from the make and/or model of the object, a format of the visual tag; and

use the format of the visual tag to extract the identifying information from the image of the equipment information.

8. The CRM of claim 6 , wherein the instructions are to further cause the apparatus to:

perform character recognition on the image of the equipment information; and

parse any recognized characters to obtain the identifying information.

9. The CRM of claim 6 , wherein the instructions are to further cause the apparatus to transmit, to the user device, service information obtained in response to the query into the database.

10. The CRM of claim 6 , wherein the user device is a smartphone, and the apparatus is a server remote from the smartphone.

11. A non-transitory computer-readable medium (CRM) comprising instructions that, when executed by an apparatus, cause the apparatus to:

initiate, with a remote device, a two-way communications session;

transmit, to a remote server, an image of an object where an equipment information is not visible from an external view of the object, the equipment information comprising at least make, model, and serial number information for the object;

receive, from the remote server in response to transmission of the image of the object, instructions to assist a user of the apparatus in locating a location on the object of the equipment information, the instructions based at least in part on the image of the object, wherein the equipment information comprises a visual tag and the instructions comprise at least a silhouette or shape of the visual tag, and the location of the equipment information is determined from the image of the object by detection of, and search of a database with, a shape and position of one or more features of the object;

overlay, on a display of the apparatus, the instructions;

transmit, to the remote server, an image of the equipment information; and

receive, from the remote server, service information about the object, the service information based at least in part on the equipment information, wherein the service information about the object is also provided by the remote server to the remote device.

12. The CRM of claim 11 , wherein the instructions are to further cause the apparatus to:

receive from the remote server, in response to transmission of the image of the object, a list of one or more makes and/or models of the object; and

transmit, to the remote server, a selection of one of the makes and/or models of the object from a last of one or more makes and/or models,

wherein the location of the equipment information and the service information is based at least in part on the selected make and/or model.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2024
From: FINK, RYAN R.; ADKINSON, SEAN M.
To: STREEM, INC.
Reel/Frame 067647/0485 →
MERGER Recorded Jun 6, 2024
From: STREEM, INC.
To: STREEM, LLC
Reel/Frame 067647/0827 →
Continuity (2)
Provisional Application 62778630 · Dec 12, 2018
Related Publication 20200193203A1 · Jun 18, 2020