IP Library › Granted Patent US 11,025,815
Granted Patent B2
US 11,025,815 · App. 16/586,852 · Granted Jun 1, 2021

Devices, methods, and graphical user interfaces for assisted photo-taking

Inventors: Christopher B. Fleizach (Gilroy, CA); Darren C. Minifie (Santa Cruz, CA); Eryn R. Wells (Redwood City, CA); Nandini Kannamangalam Sundara Raman (San Jose, CA)
Assignee: APPLE INC.
H04N5/23222H04N5/23219H04N5/232935
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,025,815
App. No.
16/586,852
Granted
Jun 1, 2021
Kind
B2
Abstract

An electronic device obtains one or more images of a scene. After obtaining the one or more images of the scene, the electronic device detects a plurality of objects within the scene, provides a first audible description of the scene, and detects a user input that selects a respective object of the plurality of objects within the scene. The first audible description of the scene provides information corresponding to the plurality of objects as a group. In response to the user input selecting the respective object within the scene, the electronic device provides a second audible description of the respective object. The second audible description is distinct from the first audible description and includes a description of one or more characteristics specific to the respective object.

Claims (150)

1. A method, comprising:

at an electronic device:

obtaining one or more images of a scene;

after obtaining the one or more images of the scene:

detecting a plurality of objects within the scene; and

providing a first audible description of the scene, wherein the first audible description provides information corresponding to the plurality of objects as a group, wherein:

in accordance with a determination that a number of objects in the scene is below a threshold, the first audible description includes a first summarization of the scene; and

in accordance with a determination that the number of objects in the scene is above the threshold, the first audible description includes a second summarization of the scene, different from the first summarization of the scene.

2. The method of claim 1 , further including:

detecting a user input that selects a respective object of the plurality of objects within the scene; and

in response to the user input selecting the respective object within the scene, providing a second audible description of the respective object, wherein the second audible description is distinct from the first audible description and includes a description of one or more characteristics specific to the respective object.

3. The method of claim 2 , including:

detecting a second user input; and

in accordance with a determination that a first mode is active on the electronic device:

selecting a second object of the plurality of objects;

providing a third audible description of the second object, the third audible description including a description of one or more characteristics specific to the second object; and

in accordance with a determination that the first mode is not active on the electronic device, performing a respective operation that does not include providing a description of the second object.

4. The method of claim 3 , wherein the electronic device includes a camera and the respective operation includes focusing the camera at a location corresponding to the second user input.

5. The method of claim 3 , including:

while the first mode is active on the electronic device:

detecting a predefined user input for performing the respective operation when the electronic device is in the first mode; and

in response to detecting the predefined user input, performing the respective operation.

6. The method of claim 2 , wherein:

the electronic device includes a touch-sensitive display; and

the method further includes:

displaying a preview of the scene on the touch-sensitive display, wherein:

the user input includes a contact on the touch-sensitive display; and

selecting the respective object of the plurality of objects within the scene includes detecting a contact over a location of the respective object in the preview of the scene.

7. The method of claim 2 , wherein selecting the respective object of the plurality of objects within the scene includes detecting a predefined user input for selecting, in sequence, respective objects in the plurality of objects.

8. The method of claim 1 , wherein the electronic device includes a camera and the method further includes:

detecting, using one or more sensors, an orientation of a first axis of the electronic device relative to a respective vector; and

in accordance with a determination that the orientation of the first axis of the electronic device has moved within a predefined range of the respective vector, providing a first audible and/or tactile indication that the camera is obtaining level images of the scene.

9. The method of claim 8 , further including:

in accordance with a determination that the orientation of the first axis of the electronic device has moved outside of the predefined range of the respective vector, providing a second audible and/or tactile indication, wherein the second audible and/or tactile indication indicates that the camera is not obtaining level images of the scene.

10. The method of claim 8 , further including:

in accordance with a determination that the orientation of the first axis of the electronic device meets movement threshold criteria upon moving within the predefined range of the respective vector, forgoing provision of the first audible and/or tactile indication.

11. The method of claim 2 , further including:

accessing a multimedia collection that includes one or more photographs and/or videos that have been tagged with people; and

determining that the respective object corresponds to a person that has been tagged in the multimedia collection;

wherein at least one of the first audible description and the second audible description identifies the person.

12. The method of claim 11 , wherein the second audible description identifies one or more visible characteristics of the person.

13. The method of claim 2 , further including:

identifying one or more prominent objects of the plurality of objects; and

determining that the respective object is a respective prominent object of the one or more prominent objects;

wherein the second audible description identifies the respective object as a prominent object.

14. The method of claim 1 , including:

after obtaining the one or more images of the scene:

detecting a plurality of people in the scene;

in accordance with a determination that a number of people in the scene is less than a threshold number of people, identifying individual people of the plurality of people in the scene; and

in accordance with a determination that the number of people in the scene is more than the threshold number of people, identifying the number of people in the scene.

15. An electronic device, comprising:

one or more processors; and

memory storing one or more programs, wherein the one or more programs are configured to be executed by the one or more processors, the one or more programs including instructions for:

obtaining one or more images of a scene;

after obtaining the one or more images of the scene:

detecting a plurality of objects within the scene; and

providing a first audible description of the scene, wherein the first audible description provides information corresponding to the plurality of objects as a group, wherein:

in accordance with a determination that a number of objects in the scene is below a threshold, the first audible description includes a first summarization of the scene; and

in accordance with a determination that the number of objects in the scene is above the threshold, the first audible description includes a second summarization of the scene, different from the first summarization of the scene.

16. The electronic device of claim 15 , wherein the one or more programs include instructions for:

detecting a user input that selects a respective object of the plurality of objects within the scene; and

in response to the user input selecting the respective object within the scene, providing a second audible description of the respective object, wherein the second audible description is distinct from the first audible description and includes a description of one or more characteristics specific to the respective object.

17. The electronic device of claim 16 , wherein the one or more programs include instructions for:

detecting a second user input; and

in accordance with a determination that a first mode is active on the electronic device:

selecting a second object of the plurality of objects;

providing a third audible description of the second object, the third audible description including a description of one or more characteristics specific to the second object; and

in accordance with a determination that the first mode is not active on the electronic device, performing a respective operation that does not include providing a description of the second object.

18. The electronic device of claim 17 , wherein the electronic device includes a camera and the respective operation includes focusing the camera at a location corresponding to the second user input.

19. The electronic device of claim 17 , wherein the one or more programs include instructions for:

while the first mode is active on the electronic device:

detecting a predefined user input for performing the respective operation when the electronic device is in the first mode; and

in response to detecting the predefined user input, performing the respective operation.

20. The electronic device of claim 16 , wherein:

the electronic device includes a touch-sensitive display; and

the one or more programs include instructions for:

displaying a preview of the scene on the touch-sensitive display, wherein:

the user input includes a contact on the touch-sensitive display; and

selecting the respective object of the plurality of objects within the scene includes detecting a contact over a location of the respective object in the preview of the scene.

21. The electronic device of claim 16 , wherein selecting the respective object of the plurality of objects within the scene includes detecting a predefined user input for selecting, in sequence, respective objects in the plurality of objects.

22. The electronic device of claim 15 , wherein the electronic device includes a camera, and the one or more programs include instructions for:

detecting, using one or more sensors, an orientation of a first axis of the electronic device relative to a respective vector; and

in accordance with a determination that the orientation of the first axis of the electronic device has moved within a predefined range of the respective vector, providing a first audible and/or tactile indication that the camera is obtaining level images of the scene.

23. The electronic device of claim 22 , wherein the one or more programs include instructions for:

in accordance with a determination that the orientation of the first axis of the electronic device has moved outside of the predefined range of the respective vector, providing a second audible and/or tactile indication, wherein the second audible and/or tactile indication indicates that the camera is not obtaining level images of the scene.

24. The electronic device of claim 22 , wherein the one or more programs include instructions for:

in accordance with a determination that the orientation of the first axis of the electronic device meets movement threshold criteria upon moving within the predefined range of the respective vector, forgoing provision of the first audible and/or tactile indication.

25. The electronic device of claim 16 , wherein the one or more programs include instructions for:

accessing a multimedia collection that includes one or more photographs and/or videos that have been tagged with people; and

determining that the respective object corresponds to a person that has been tagged in the multimedia collection;

wherein at least one of the first audible description and the second audible description identifies the person.

26. The electronic device of claim 25 , wherein the second audible description identifies one or more visible characteristics of the person.

27. The electronic device of claim 16 , wherein the one or more programs include instructions for:

identifying one or more prominent objects of the plurality of objects; and

determining that the respective object is a respective prominent object of the one or more prominent objects;

wherein the second audible description identifies the respective object as a prominent object.

28. The electronic device of claim 15 , wherein the one or more programs include instructions for:

after obtaining the one or more images of the scene:

detecting a plurality of people in the scene;

in accordance with a determination that a number of people in the scene is less than a threshold number of people, identifying individual people of the plurality of people in the scene; and

in accordance with a determination that the number of people in the scene is more than the threshold number of people, identifying the number of people in the scene.

29. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device, cause the electronic device to:

obtain one or more images of a scene;

after obtaining the one or more images of the scene:

detect a plurality of objects within the scene; and

provide a first audible description of the scene, wherein the first audible description provides information corresponding to the plurality of objects as a group, wherein:

in accordance with a determination that a number of objects in the scene is below a threshold, the first audible description includes a first summarization of the scene; and

in accordance with a determination that the number of objects in the scene is above the threshold, the first audible description includes a second summarization of the scene, different from the first summarization of the scene.

30. The non-transitory computer readable storage medium of claim 29 , wherein the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

detect a user input that selects a respective object of the plurality of objects within the scene; and

in response to the user input selecting the respective object within the scene, provide a second audible description of the respective object, wherein the second audible description is distinct from the first audible description and includes a description of one or more characteristics specific to the respective object.

31. The non-transitory computer readable storage medium of claim 30 , wherein the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

detect a second user input; and

in accordance with a determination that a first mode is active on the electronic device:

select a second object of the plurality of objects;

provide a third audible description of the second object, the third audible description including a description of one or more characteristics specific to the second object; and

in accordance with a determination that the first mode is not active on the electronic device, perform a respective operation that does not include providing a description of the second object.

32. The non-transitory computer readable storage medium of claim 31 , wherein the electronic device includes a camera and the respective operation includes focusing the camera at a location corresponding to the second user input.

33. The non-transitory computer readable storage medium of claim 31 , wherein the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

while the first mode is active on the electronic device:

detecting a predefined user input for performing the respective operation when the electronic device is in the first mode; and

in response to detecting the predefined user input, performing the respective operation.

34. The non-transitory computer readable storage medium of claim 30 , wherein:

the electronic device includes a touch-sensitive display; and

the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

display a preview of the scene on the touch-sensitive display, wherein:

the user input includes a contact on the touch-sensitive display; and

selecting the respective object of the plurality of objects within the scene includes detecting a contact over a location of the respective object in the preview of the scene.

35. The non-transitory computer readable storage medium of claim 30 , wherein selecting the respective object of the plurality of objects within the scene includes detecting a predefined user input for selecting, in sequence, respective objects in the plurality of objects.

36. The non-transitory computer readable storage medium of claim 29 , wherein the electronic device includes a camera, and the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

detect, using one or more sensors, an orientation of a first axis of the electronic device relative to a respective vector; and

in accordance with a determination that the orientation of the first axis of the electronic device has moved within a predefined range of the respective vector, provide a first audible and/or tactile indication that the camera is obtaining level images of the scene.

37. The non-transitory computer readable storage medium of claim 36 , wherein the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

in accordance with a determination that the orientation of the first axis of the electronic device has moved outside of the predefined range of the respective vector, provide a second audible and/or tactile indication, wherein the second audible and/or tactile indication indicates that the camera is not obtaining level images of the scene.

38. The non-transitory computer readable storage medium of claim 36 , wherein the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

in accordance with a determination that the orientation of the first axis of the electronic device meets movement threshold criteria upon moving within the predefined range of the respective vector, forgo provision of the first audible and/or tactile indication.

39. The non-transitory computer readable storage medium of claim 30 , wherein the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

access a multimedia collection that includes one or more photographs and/or videos that have been tagged with people; and

determine that the respective object corresponds to a person that has been tagged in the multimedia collection;

wherein at least one of the first audible description and the second audible description identifies the person.

40. The non-transitory computer readable storage medium of claim 39 , wherein the second audible description identifies one or more visible characteristics of the person.

41. The non-transitory computer readable storage medium of claim 30 , wherein the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

identify one or more prominent objects of the plurality of objects; and

determine that the respective object is a respective prominent object of the one or more prominent objects;

wherein the second audible description identifies the respective object as a prominent object.

42. The non-transitory computer readable storage medium of claim 29 , wherein the one or more programs further comprise instructions, which, when executed by the electronic device, cause the electronic device to:

after obtaining the one or more images of the scene:

detect a plurality of people in the scene;

in accordance with a determination that a number of people in the scene is less than a threshold number of people, identify individual people of the plurality of people in the scene; and

in accordance with a determination that the number of people in the scene is more than the threshold number of people, identify the number of people in the scene.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 18, 2020
From: FLEIZACH, CHRISTOPHER B.; MINIFIE, DARREN C.; WELLS, ERYN R.; SUNDARA RAMAN, NANDINI KANNAMANGALAM
To: APPLE INC.
Reel/Frame 051849/0954 →
Continuity (2)
Provisional Application 62739177 · Sep 29, 2018
Related Publication 20200106955A1 · Apr 2, 2020