Granted Patent B2

US 12,380,159 · App. 18/122,328 · Granted Aug 5, 2025

Inventors: Kirk Ouimet (Orem, UT); Rong Yan (Marina Del Rey, CA); Yantao Zheng (Sunnyvale, CA)

Assignee: Snap Inc.

G06F16/5866G06F16/9038G06F18/24G06N3/04G06N3/045G06N3/08G06T1/0007G06V10/751G06V10/764G06V10/82H04N23/63G06N5/022G06V2201/09

Abstract

Claims (46)

1. A method comprising:

causing to be displayed, by a client device, a user interface (UI) comprising indications of a plurality of system actions;

causing, by one or more processors, a plurality of input devices to capture a plurality of scan data, the plurality of input devices comprising a camera and a microphone;

analyzing, using one or more processors and a neural network, the plurality of scan data to identity one or more objects in the plurality of scan data;

matching the identified one or more objects with a plurality of data patterns to identify a plurality of identified data patterns of the plurality of data patterns;

selecting a first identified data pattern of the plurality of identified data patterns based on a pattern type ranking, wherein the plurality of identified data patterns is associated with the plurality of system actions, and wherein the first identified data pattern is associated with a first system action of the plurality of system actions, and wherein the pattern type ranking is based on a priority of the plurality of system actions;

adjusting the plurality of system actions based on the first system action; and

causing to be displayed, by the client device, the UI comprising the adjusted plurality of system actions comprising an indication of the first system action.

2. The method of claim 1 wherein the plurality of scan data comprises an image comprising a scan code, and the pattern type ranking comprises a first level ranking for a scan code pattern.

3. The method of claim 1 wherein the plurality of input devices further comprise one or more of: a position sensor, a temperature sensor, a motion sensor, or a wireless communication device.

4. The method of claim 1 wherein causing to be displayed further comprises:

causing to be displayed on a display of the client device an interface including an option to perform one of the adjusted plurality of system actions or the first system action.

5. The method of claim 1 wherein the plurality of data patterns comprises one or more of: a scan code pattern, an audio signature pattern, a logo pattern, a register trademark pattern, a pattern associated with a third-party registered keyword, and a pattern associated with a system keyword.

6. The method of claim 1 wherein the plurality of scan data comprises one or more of: wireless access point name data, image data, video data, audio data, location data, temperature data, and motion data.

7. The method of claim 1 further comprising:

in response to the matching the identified one or more objects with the plurality of data patterns failing to find a match, communicating the plurality of scan data to a third-party search server, andreceiving the first system action from the third-party search server.

8. The method of claim 1 wherein the first system action comprises performing an image search using keywords associated with the one or more objects, and wherein the causing further comprises: causing to be displayed an image search interface on a display of the client device with images from an image gallery.

9. The method of claim 8 , wherein the matching further comprising: matching the identified one or more objects with the plurality of data patterns to identify the plurality of identified data patterns of the plurality of data patterns, wherein the one or more objects are associated with one or more keywords.

10. The method of claim 1 wherein the matching further comprises:

identifying keywords associated with the identified one or more objects; andmatching the identified one or more objects with the plurality of data patterns based on the identified keywords.

11. The method of claim 10 wherein the first system action comprises an image search and wherein before the causing the method further comprises:

communicating, to a network computer, a request to perform the image search with a keyword of the identified keywords; and

receiving a plurality of search results from the networked computer in response to the image search request.

12. The method of claim 11 wherein the causing to be displayed further comprises:

causing to be displayed, by the client device, the indication of the first system action, wherein the indication of the first system action is the plurality of search results.

13. An apparatus of a client device comprising:

one or more processors; and

a memory storing instructions that, when executed by the one or more processors, configure the one or more processors to perform operations comprising:

causing to be displayed, by a client device, a user interface (UI) comprising indications of a plurality of system actions;

causing, by one or more processors, a plurality of input devices to capture a plurality of scan data, the plurality of input devices comprising a camera and a microphone;

analyzing, using one or more processors and a neural network, the plurality of scan data to identity one or more objects in the plurality of scan data;

matching the identified one or more objects with a plurality of data patterns to identify a plurality of identified data patterns of the plurality of data patterns;

selecting a first identified data pattern of the plurality of identified data patterns based on a pattern type ranking, wherein the plurality of identified data patterns is associated with the plurality of system actions, and wherein the first identified data pattern is associated with a first system action of the plurality of system actions, and wherein the pattern type ranking is based on a priority of the plurality of system actions;

adjusting the plurality of system actions based on the first system action; and

causing to be displayed, by the client device, the UI comprising the adjusted plurality of system actions comprising an indication of the first system action.

14. The apparatus of claim 13 wherein the scan data comprises an image comprising a scan code, and the pattern type ranking comprises a first level ranking for a scan code pattern.

15. The apparatus of claim 13 wherein the plurality of scan data comprises an image comprising a scan code, and the pattern type ranking comprises a first level ranking for a scan code pattern.

16. The apparatus of claim 13 wherein the plurality of input devices further comprise one or more of: a position sensor, a temperature sensor, a motion sensor, or a wireless communication device.

17. A non-transitory computer-readable storage medium, the non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of an apparatus of a client device, cause the at least one processor to perform operations comprising:

causing to be displayed, by a client device, a user interface (UI) comprising indications of a plurality of system actions; causing, by one or more processors, a plurality of input devices to capture a plurality of scan data, the plurality of input devices comprising a camera and a microphone; analyzing, using one or more processors and a neural network, the plurality of scan data to identity one or more objects in the plurality of scan data;

matching the identified one or more objects with a plurality of data patterns to identify a plurality of identified data patterns of the plurality of data patterns;

selecting a first identified data pattern of the plurality of identified data patterns based on a pattern type ranking, wherein the plurality of identified data patterns is associated with the plurality of system actions, and wherein the first identified data pattern is associated with a first system action of the plurality of system actions, and wherein the pattern type ranking is based on a priority of the plurality of system actions;

adjusting the plurality of system actions based on the first system action; and

causing to be displayed, by the client device, the UI comprising the adjusted plurality of system actions comprising an indication of the first system action.

18. The non-transitory computer-readable storage medium of claim 17 wherein the plurality of scan data comprises an image comprising a scan code, and the pattern type ranking comprises a first level ranking for a scan code pattern.

19. The non-transitory computer-readable storage medium of claim 17 wherein the plurality of input devices further comprise one or more of: a position sensor, a temperature sensor, a motion sensor, or a wireless communication device.

Assignments (1)

Continuity (5)

References Cited (400)