IP Library Granted Patent US 10,157,307
Granted Patent B2
US 10,157,307 · App. 15/299,353 · Granted Dec 18, 2018

Accessibility system

Inventors: Paul Alexander Dow (Redwood City, CA); Brett Alden Lavalla (San Carlos, CA); Jeffrey C. Wieland (Mountain View, CA); Shaomei Wu (Mountain View, CA); Yuhang Zhao (Menlo Park, CA); Lindsay Reynolds (San Carlos, CA); Matt King (Palo Alto, CA)
Assignee: Facebook, Inc.
G06K9/00275G06F3/167G06F17/30247G06K9/00288G06T11/00H04L67/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,157,307
App. No.
15/299,353
Granted
Dec 18, 2018
Kind
B2
Abstract

In one embodiment, a method includes capturing one or more images of a physical environment surrounding the first user; determining that one or more of the images comprises a face; generating a facial image by cropping a region of the image comprising the face; sending instructions to perform facial recognition on the facial image; receiving a name associated with a second user of the online social network, wherein the face is associated with the second user, wherein the first user is represented by a first node on a social graph maintained by the social-networking system and the second user is represented by a second node on the social graph, and the first node and the second node are connected by at least one edge; and providing an audible indication of the name associated with the second user.

Claims (67)

1. A method comprising:

by a computing device associated with a first user of an online social network hosted by a social-networking system, capturing one or more images of a physical environment surrounding the first user;

by the computing device, determining that one or more of the images comprises a face;

by the computing device, generating a facial image by cropping a region of the image comprising the face;

by the computing device, sending instructions to perform facial recognition on the facial image;

by the computing device, receiving a name associated with a second user of the online social network, wherein the face is associated with the second user, wherein the first user is represented by a first node on a social graph maintained by the social-networking system and the second user is represented by a second node on the social graph, and the first node and the second node are connected by at least one edge;

by the computing device, providing an audible indication of the name associated with the second user;

comparing the facial image to one or more cached images on the computing device;

determining that the facial image does not match any of the one or more cached images; and

sending, to a server, a request comprising the facial image to perform facial recognition.

2. The method of claim 1 , wherein one or more cameras of the computing device continuously capture the images in response to detecting an audible command.

3. The method of claim 1 , further comprising:

determining that one or more of the images comprises an object;

generating an object image by cropping a region of the image comprising the object;

sending instructions to perform object recognition on the object image;

receiving a description associated with the object; and

providing an audible indication of the description associated with the object.

4. The method of claim 1 , wherein the audible indication comprises an approximate location of the second user.

5. The method of claim 1 , further comprising:

computing a path comprising an end point associated with a person corresponding to the face in the one or more images, wherein a starting point for the path is determined with respect to the computing device; and

providing navigation information to the first user in relation to the path.

6. The method of claim 5 , wherein the computing the path comprises identifying one or more obstacles between the starting point and the end point, and wherein the navigation information comprises instructions to direct the first user to the end point while avoiding the obstacles.

7. The method of claim 1 , wherein capturing one or more images comprises:

identifying one or more enhancements that may be applied to one or more of the images;

providing information regarding the one or more enhancements; and

receiving user input in response to the provided information.

8. The method of claim 1 , further comprising:

determining that an affinity between the first user and the second user exceeds a threshold affinity; and

storing one or more pictures of the second user in a local cache on the computing device.

9. The method of claim 1 , further comprising:

by the computing device, determining that the one or more images comprise a portion of an entire face; and

by the computing device, providing instructions to reposition the computing device to capture the entire face.

10. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

capture one or more images of a physical environment surrounding a first user of an online social network hosted by a social-networking system;

determine that one or more of the images comprises a face;

generate a facial image by cropping a region of the image comprising the face;

send instructions to perform facial recognition on the facial image;

receive a name associated with a second user of the online social network, wherein the face is associated with the second user, wherein the first user is represented by a first node on a social graph maintained by the social-networking system and the second user is represented by a second node on the social graph, and the first node and the second node are connected by at least one edge;

provide an audible indication of the name associated with the second user;

compute a path comprising an end point associated with a person corresponding to the face in the one or more images, wherein a starting point for the path is determined with respect to the computing device; and

provide navigation information to the first user in relation to the path.

11. The media of claim 10 , wherein one or more cameras of the computing device continuously capture the images in response to detecting an audible command.

12. The media of claim 10 , wherein the software is further operable when executed to:

determine that one or more of the images comprises an object;

generate an object image by cropping a region of the image comprising the object;

send instructions to perform object recognition on the object image;

receive a description associated with the object; and

provide an audible indication of the description associated with the object.

13. The media of claim 10 , wherein the computing the path comprises identifying one or more obstacles between the starting point and the end point, and wherein the navigation information comprises instructions to direct the first user to the end point while avoiding the obstacles.

14. The media of claim 10 , wherein the software is further operable when executed to:

identify one or more enhancements that may be applied to one or more of the images;

provide information regarding the one or more enhancements; and

receive user input in response to the provided information.

15. A system comprising: one or more processors; and a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to:

capture one or more images of a physical environment surrounding a first user of an online social network hosted by a social-networking system;

determine that one or more of the images comprises a face;

generate a facial image by cropping a region of the image comprising the face;

send instructions to perform facial recognition on the facial image;

receive a name associated with a second user of the online social network, wherein the face is associated with the second user, wherein the first user is represented by a first node on a social graph maintained by the social-networking system and the second user is represented by a second node on the social graph, and the first node and the second node are connected by at least one edge;

provide an audible indication of the name associated with the second user;

compute a path comprising an end point associated with a person corresponding to the face in the one or more images, wherein a starting point for the path is determined with respect to the computing device; and

provide navigation information to the first user in relation to the path.

16. The system of claim 15 , wherein the computing the path comprises identifying one or more obstacles between the starting point and the end point, and wherein the navigation information comprises instructions to direct the first user to the end point while avoiding the obstacles.

17. The system of claim 15 , wherein the processors are further operable when executing the instructions to:

identify one or more enhancements that may be applied to one or more of the images;

provide information regarding the one or more enhancements; and

receive user input in response to the provided information.

Assignments (2)
CHANGE OF NAME Recorded Dec 20, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058553/0802 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 14, 2016
From: DOW, PAUL ALEXANDER; LAVALLA, BRETT ALDEN; WIELAND, JEFFREY C.; WU, SHAOMEI; ZHAO, YUHANG; REYNOLDS, LINDSAY; KING, MATT
To: FACEBOOK, INC.
Reel/Frame 040313/0099 →
Continuity (1)
Related Publication 20180114054A1 · Apr 26, 2018