IP Library › Granted Patent US 12,101,567
Granted Patent B2
US 12,101,567 · App. 18/228,591 · Granted Sep 24, 2024

User interfaces for altering visual media

Inventors: Johnnie B. Manzari (San Francisco, CA); Jeffrey A. Brasket (Honolulu, HI); Graham R. Clarke (Scotts Valley, CA); Saumitro Dasgupta (Redwood City, CA); Mikko Berggren Ettienne (Santa Clara, CA); Toke Jansen (Cupertino, CA); Wayne Loofbourrow (San Jose, CA); Joseph A Malia (San Francisco, CA); Seyyedhossein Mousavi (Millbrae, CA); Jens Jacob Pallisgaard (Santa Clara, CA); Paul Thomas Schneider (Palo Alto, CA); Joshua Blake Shagam (Redwood City, CA); William A. Sorrentino, III (Mill Valley, CA); Andre Souza Dos Santos (San Jose, CA); Piotr J. Stanczyk (San Francisco, CA)
Assignee: Apple Inc.
H04N23/959G06T5/73H04N5/2226H04N23/69
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,101,567
App. No.
18/228,591
Filed
Jul 31, 2023
Granted
Sep 24, 2024
Kind
B2
Art Unit
2637
USPC
348/222.1
Abstract

The present disclosure generally relates to user interfaces for altering visual media. In some embodiments, user interfaces capturing visual media (e.g., via a synthetic depth-of-field effect), playing back visual media (e.g., via a synthetic depth-of-field effect), editing visual media (e.g., that has a synthetic depth-of-field effect applied), and/or managing media capture.

Claims (141)

1. A computer system configured to communicate with one or more input devices, the computer system comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

detecting, via the one or more input devices, a request to apply a synthetic depth-of-field effect to a plurality of frames of a video that have been captured; and

in response to detecting the request:

in accordance with a determination that the request was detected based on a first type of gesture being detected, applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with a first type of tracking; and

in accordance with a determination that the request was detected based on a second type of gesture being detected, applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with a second type of tracking, wherein the second type of tracking is different from the first type of tracking.

2. The computer system of claim 1 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes emphasizing a first subject relative to a second subject until a first set of criteria are met; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking includes emphasizing the first subject relative to the second subject until a second set of criteria are met, wherein the second set of criteria are different from the first set of criteria.

3. The computer system of claim 1 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes emphasizing a first subject relative to a second subject for a first set of frames that occur for a first duration; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking emphasizing the first subject relative to the second subject for a second set of frames that occur for a second duration, wherein the second duration is different from the first duration.

4. The computer system of claim 1 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes applying the synthetic depth-of-field effect to temporarily emphasize a first subject relative to a second subject in the plurality of frames; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking includes applying the synthetic depth-of-field effect to non-temporarily emphasize the first subject relative to the second subject.

5. The computer system of claim 4 , the one or more programs further including instructions for:

while applying the synthetic depth-of-field effect to non-temporarily emphasize the first subject relative to the second subject, detecting that the first subject is not within visual content of the plurality of frames; and

in response to detecting that the first subject is not within the visual content of the plurality of frames, applying the synthetic depth-of-field effect to temporarily emphasize the second subject.

6. The computer system of claim 1 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes temporarily applying the synthetic depth-of-field effect to emphasize a first subject relative to a plane in the plurality of frames; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking includes applying the synthetic depth-of-field effect to emphasize the plane relative to the first subject.

7. The computer system of claim 1 , wherein the request includes an input directed to visual content of the plurality of frames.

8. The computer system of claim 1 , wherein the request is based on a gesture that is not directed to one or more subjects in the plurality of frames.

9. The computer system of claim 1 , wherein the request is based on a gesture that is directed to a subject in the plurality of frames.

10. The computer system of claim 1 , the one or more programs further including instructions for:

in response to detecting the request:

in accordance with the determination that the request was detected based on the first type of gesture being detected, displaying, via a display generation component, a first user interface object indicating a subject is being emphasized using the first type of tracking; and

in accordance with the determination that the request was detected based on a second type of gesture being detected, displaying, via the display generation component, a second user interface object, different from the first user interface object, indicating the subject is being emphasized using the second type of tracking.

11. The computer system of claim 1 , the one or more programs further including instructions for:

in response to detecting the request:

in accordance with a determination that the request was detected based on a third type of gesture being detected, applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with a third type of tracking, wherein the third type of tracking is different from the first type of tracking and the second type of tracking.

12. The computer system of claim 1 , wherein:

the first type of gesture is selected from the group consisting of: a single-tap gesture, a multi-tap gesture, and a press-and-hold gesture; and

the second type of gesture is selected from the group consisting of: a single-tap gesture, a multi-tap gesture, and a press-and-hold gesture.

13. The computer system of claim 1 , wherein the computer system is configured to communicate with one or more cameras, the one or more programs further including instructions for:

prior to capturing the plurality of frames the video, detecting, via the one or more input devices, a request to capture the plurality of frames of the video, wherein the plurality of frames of the video is representative of a field-of-view of the one or more cameras;

in response to detecting the request to capture the video:

capturing the video over a first capture duration, wherein the video includes the plurality of frames that are captured over the first capture duration, wherein the plurality of frames represent a first subject in the field-of-view of the one or more cameras and a second subject in the field-of-view of the one or more cameras, and wherein, in the plurality of frames, the first subject is moving relative to the field-of-view of the one or more cameras over the first capture duration; and

applying, to the plurality of frames of the video, the synthetic depth-of-field effect, wherein the synthetic depth-of-field effect alters visual information captured by the one or more cameras to emphasize the first subject in the plurality of frames of the video relative to the second subject in the plurality of frames of the video, wherein the synthetic depth-of-field effect changes over time as the first subject moves within the field-of-view of the one or more cameras.

14. The computer system of claim 1 , wherein the computer system is configured to communicate with one or more cameras, the one or more programs further including instructions for:

displaying, via a display generation component, a user interface that includes:

a representation of the plurality of frames of the video that has been captured, the representation including a first subject and a second subject; and

a first user interface object indicating that the first subject is being emphasized by the synthetic depth-of-field effect, wherein the synthetic depth-of-field effect alters visual information captured by the one or more cameras to emphasize the first subject in the plurality of frames relative to the second subject;

while displaying the user interface that includes the representation of the video and the first user interface object, detecting, via the one or more input devices, a gesture that corresponds to selection of the second subject in the representation; and

in response to detecting the gesture that corresponds to selection of the second subject in the representation:

changing the synthetic depth-of-field effect to alter the visual information captured by the one or more cameras to emphasize the second subject in the plurality of frames relative to the first subject; and

displaying a second user interface object indicating that the second subject is being emphasized by the changed synthetic depth-of-field effect that alters the visual information captured by the one or more cameras to emphasize the second subject in the plurality of frames relative to the first subject.

15. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with one or more input devices, the one or more programs including instructions for:

detecting, via the one or more input devices, a request to apply a synthetic depth-of-field effect to a plurality of frames of a video that have been captured; and

in response to detecting the request:

in accordance with a determination that the request was detected based on a first type of gesture being detected, applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with a first type of tracking; and

in accordance with a determination that the request was detected based on a second type of gesture being detected, applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with a second type of tracking, wherein the second type of tracking is different from the first type of tracking.

16. The non-transitory computer-readable storage medium of claim 15 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes emphasizing a first subject relative to a second subject until a first set of criteria are met; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking includes emphasizing the first subject relative to the second subject until a second set of criteria are met, wherein the second set of criteria are different from the first set of criteria.

17. The non-transitory computer-readable storage medium of claim 15 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes emphasizing a first subject relative to a second subject for a first set of frames that occur for a first duration; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking emphasizing the first subject relative to the second subject for a second set of frames that occur for a second duration, wherein the second duration is different from the first duration.

18. The non-transitory computer-readable storage medium of claim 15 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes applying the synthetic depth-of-field effect to temporarily emphasize a first subject relative to a second subject in the plurality of frames; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking includes applying the synthetic depth-of-field effect to non-temporarily emphasize the first subject relative to the second subject.

19. The non-transitory computer-readable storage medium of claim 18 , the one or more programs further including instructions for:

while applying the synthetic depth-of-field effect to non-temporarily emphasize the first subject relative to the second subject, detecting that the first subject is not within visual content of the plurality of frames; and

in response to detecting that the first subject is not within the visual content of the plurality of frames, applying the synthetic depth-of-field effect to temporarily emphasize the second subject.

20. The non-transitory computer-readable storage medium of claim 15 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes temporarily applying the synthetic depth-of-field effect to emphasize a first subject relative to a plane in the plurality of frames; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking includes applying the synthetic depth-of-field effect to emphasize the plane relative to the first subject.

21. The non-transitory computer-readable storage medium of claim 15 , wherein the request includes an input directed to visual content of the plurality of frames.

22. The non-transitory computer-readable storage medium of claim 15 , wherein the request is based on a gesture that is not directed to one or more subjects in the plurality of frames.

23. The non-transitory computer-readable storage medium of claim 15 , wherein the request is based on a gesture that is directed to a subject in the plurality of frames.

24. The non-transitory computer-readable storage medium of claim 15 , the one or more programs further including instructions for:

in response to detecting the request:

in accordance with the determination that the request was detected based on the first type of gesture being detected, displaying, via a display generation component, a first user interface object indicating a subject is being emphasized using the first type of tracking; and

in accordance with the determination that the request was detected based on a second type of gesture being detected, displaying, via the display generation component, a second user interface object, different from the first user interface object, indicating the subject is being emphasized using the second type of tracking.

25. The non-transitory computer-readable storage medium of claim 15 , the one or more programs further including instructions for:

in response to detecting the request:

in accordance with a determination that the request was detected based on a third type of gesture being detected, applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with a third type of tracking, wherein the third type of tracking is different from the first type of tracking and the second type of tracking.

26. The non-transitory computer-readable storage medium of claim 15 , wherein:

the first type of gesture is selected from the group consisting of: a single-tap gesture, a multi-tap gesture, and a press-and-hold gesture; and

the second type of gesture is selected from the group consisting of: a single-tap gesture, a multi-tap gesture, and a press-and-hold gesture.

27. The non-transitory computer-readable storage medium of claim 15 , wherein the computer system is configured to communicate with one or more cameras, the one or more programs further including instructions for:

prior to capturing the plurality of frames the video, detecting, via the one or more input devices, a request to capture the plurality of frames of the video, wherein the plurality of frames of the video is representative of a field-of-view of the one or more cameras;

in response to detecting the request to capture the video:

capturing the video over a first capture duration, wherein the video includes the plurality of frames that are captured over the first capture duration, wherein the plurality of frames represent a first subject in the field-of-view of the one or more cameras and a second subject in the field-of-view of the one or more cameras, and wherein, in the plurality of frames, the first subject is moving relative to the field-of-view of the one or more cameras over the first capture duration; and

applying, to the plurality of frames of the video, the synthetic depth-of-field effect, wherein the synthetic depth-of-field effect alters visual information captured by the one or more cameras to emphasize the first subject in the plurality of frames of the video relative to the second subject in the plurality of frames of the video, wherein the synthetic depth-of-field effect changes over time as the first subject moves within the field-of-view of the one or more cameras.

28. The non-transitory computer-readable storage medium of claim 15 , wherein the computer system is configured to communicate with one or more cameras, the one or more programs further including instructions for:

displaying, via a display generation component, a user interface that includes:

a representation of the plurality of frames of the video that has been captured, the representation including a first subject and a second subject; and

a first user interface object indicating that the first subject is being emphasized by the synthetic depth-of-field effect, wherein the synthetic depth-of-field effect alters visual information captured by the one or more cameras to emphasize the first subject in the plurality of frames relative to the second subject;

while displaying the user interface that includes the representation of the video and the first user interface object, detecting, via the one or more input devices, a gesture that corresponds to selection of the second subject in the representation; and

in response to detecting the gesture that corresponds to selection of the second subject in the representation:

changing the synthetic depth-of-field effect to alter the visual information captured by the one or more cameras to emphasize the second subject in the plurality of frames relative to the first subject; and

displaying a second user interface object indicating that the second subject is being emphasized by the changed synthetic depth-of-field effect that alters the visual information captured by the one or more cameras to emphasize the second subject in the plurality of frames relative to the first subject.

29. A method, comprising:

at a computer system that is in communication with one or more input devices:

detecting, via the one or more input devices, a request to apply a synthetic depth-of-field effect to a plurality of frames of a video that have been captured; and

in response to detecting the request:

in accordance with a determination that the request was detected based on a first type of gesture being detected, applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with a first type of tracking; and

in accordance with a determination that the request was detected based on a second type of gesture being detected, applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with a second type of tracking, wherein the second type of tracking is different from the first type of tracking.

30. The method of claim 29 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes emphasizing a first subject relative to a second subject until a first set of criteria are met; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking includes emphasizing the first subject relative to the second subject until a second set of criteria are met, wherein the second set of criteria are different from the first set of criteria.

31. The method of claim 29 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes emphasizing a first subject relative to a second subject for a first set of frames that occur for a first duration; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking emphasizing the first subject relative to the second subject for a second set of frames that occur for a second duration, wherein the second duration is different from the first duration.

32. The method of claim 29 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes applying the synthetic depth-of-field effect to temporarily emphasize a first subject relative to a second subject in the plurality of frames; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking includes applying the synthetic depth-of-field effect to non-temporarily emphasize the first subject relative to the second subject.

33. The method of claim 32 , further comprising:

while applying the synthetic depth-of-field effect to non-temporarily emphasize the first subject relative to the second subject, detecting that the first subject is not within visual content of the plurality of frames; and

in response to detecting that the first subject is not within the visual content of the plurality of frames, applying the synthetic depth-of-field effect to temporarily emphasize the second subject.

34. The method of claim 29 , wherein:

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the first type of tracking includes temporarily applying the synthetic depth-of-field effect to emphasize a first subject relative to a plane in the plurality of frames; and

applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with the second type of tracking includes applying the synthetic depth-of-field effect to emphasize the plane relative to the first subject.

35. The method of claim 29 , wherein the request includes an input directed to visual content of the plurality of frames.

36. The method of claim 29 , wherein the request is based on a gesture that is not directed to one or more subjects in the plurality of frames.

37. The method of claim 29 , wherein the request is based on a gesture that is directed to a subject in the plurality of frames.

38. The method of claim 29 , further comprising:

in response to detecting the request:

in accordance with the determination that the request was detected based on the first type of gesture being detected, displaying, via a display generation component, a first user interface object indicating a subject is being emphasized using the first type of tracking; and

in accordance with the determination that the request was detected based on a second type of gesture being detected, displaying, via the display generation component, a second user interface object, different from the first user interface object, indicating the subject is being emphasized using the second type of tracking.

39. The method of claim 29 , further comprising:

in response to detecting the request:

in accordance with a determination that the request was detected based on a third type of gesture being detected, applying the synthetic depth-of-field effect to the plurality of frames of the video that have been captured with a third type of tracking, wherein the third type of tracking is different from the first type of tracking and the second type of tracking.

40. The method of claim 29 , wherein:

the first type of gesture is selected from the group consisting of: a single-tap gesture, a multi-tap gesture, and a press-and-hold gesture; and

the second type of gesture is selected from the group consisting of: a single-tap gesture, a multi-tap gesture, and a press-and-hold gesture.

41. The method of claim 29 , wherein the computer system is configured to communicate with one or more cameras, wherein the method further comprises:

prior to capturing the plurality of frames the video, detecting, via the one or more input devices, a request to capture the plurality of frames of the video, wherein the plurality of frames of the video is representative of a field-of-view of the one or more cameras;

in response to detecting the request to capture the video:

capturing the video over a first capture duration, wherein the video includes the plurality of frames that are captured over the first capture duration, wherein the plurality of frames represent a first subject in the field-of-view of the one or more cameras and a second subject in the field-of-view of the one or more cameras, and wherein, in the plurality of frames, the first subject is moving relative to the field-of-view of the one or more cameras over the first capture duration; and

applying, to the plurality of frames of the video, the synthetic depth-of-field effect, wherein the synthetic depth-of-field effect alters visual information captured by the one or more cameras to emphasize the first subject in the plurality of frames of the video relative to the second subject in the plurality of frames of the video, wherein the synthetic depth-of-field effect changes over time as the first subject moves within the field-of-view of the one or more cameras.

42. The method of claim 29 , wherein the computer system is configured to communicate with one or more cameras, wherein the method further comprises:

displaying, via a display generation component, a user interface that includes:

a representation of the plurality of frames of the video that has been captured, the representation including a first subject and a second subject; and

a first user interface object indicating that the first subject is being emphasized by the synthetic depth-of-field effect, wherein the synthetic depth-of-field effect alters visual information captured by the one or more cameras to emphasize the first subject in the plurality of frames relative to the second subject;

while displaying the user interface that includes the representation of the video and the first user interface object, detecting, via the one or more input devices, a gesture that corresponds to selection of the second subject in the representation; and

in response to detecting the gesture that corresponds to selection of the second subject in the representation:

changing the synthetic depth-of-field effect to alter the visual information captured by the one or more cameras to emphasize the second subject in the plurality of frames relative to the first subject; and

displaying a second user interface object indicating that the second subject is being emphasized by the changed synthetic depth-of-field effect that alters the visual information captured by the one or more cameras to emphasize the second subject in the plurality of frames relative to the first subject.

Continuity (7)
Continuation 17944765 · Sep 14, 2022
Continuation 17483684 · Sep 23, 2021
Provisional Application 63244213 · Sep 14, 2021
Provisional Application 63243724 · Sep 13, 2021
Provisional Application 63197460 · Jun 6, 2021
Provisional Application 63182751 · Apr 30, 2021
Related Publication 20230388665A1 · Nov 30, 2023
Cited By (1)
US 12,744,990