IP Library Granted Patent US 12,455,673
Granted Patent B2
US 12,455,673 · App. 17/947,029 · Granted Oct 28, 2025

Devices, methods, and graphical user interfaces for dynamically adjusting presentation of audio outputs

Inventors: Marcos Alonso Ruiz (Oakland, CA); David C. Graham (Columbus, OH); Freddy A. Anzures (San Francisco, CA); Hugo D. Verweij (Portola Valley, CA); Afrooz Family (Emerald Hills, CA); Matthew I. Brown (New York, NY)
Assignee: APPLE INC.
G06F3/0482G06F3/0488G06F3/04883G06F3/04886G06F3/165
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,455,673
App. No.
17/947,029
Granted
Oct 28, 2025
Kind
B2
Abstract

An electronic device detects a first input directed to a first affordance in a set of one or more affordances displayed on a display. In response to detecting the first input, the electronic device initiates presentation of a first audio output having a first audio profile. The electronic device later detects a second input directed to a second affordance in the set. In response to detecting the second input and if audio alteration criteria are satisfied, the electronic device causes: (i) presentation of altered first audio output having an altered audio profile and (ii) presentation of a second audio output having a second audio profile. In response to detecting the second input and if the audio alteration criteria are not satisfied, the electronic device causes: (i) continued presentation of the first audio output and (ii) presentation of a third audio output having a third audio profile.

Claims (76)

1 . A method, comprising:

at an electronic device with one or more processors and memory, wherein the electronic device is in communication with a display and an audio system:

while a user interface that includes a set of one or more affordances is displayed on the display:

detecting, at a first point in time, a first input directed to a first affordance in the set of one or more affordances;

in response to detecting the first input directed to the first affordance, initiating providing, to the audio system, first sound information to present a first audio output that corresponds to the first affordance, wherein the first audio output has a first audio profile that governs a change in one or more respective audio characteristics of the first audio output over time;

while continuing to provide the first sound information in response to the first input, detecting, at a second point in time that is after the first point in time, a second input directed to a second affordance in the set of one or more affordances; and

in response to detecting the second input directed to the second affordance:

while continuing to provide the first sound information in response to the first input and in accordance with a determination that audio alteration criteria are satisfied:

 causing the audio system to present altered first audio output that corresponds to the first affordance, instead of continuing to present the first audio output with the first audio profile, wherein the altered first audio output has an altered audio profile that is distinct from the first audio profile, wherein having the altered audio profile includes changing the one or more respective audio characteristics of the first audio output over time in a different manner than the one or more respective audio characteristics of the first audio output would have been changed over time if the first audio profile were to have been used; and

 providing, to the audio system, second sound information to present a second audio output that corresponds to the second affordance, wherein the second audio output has a second audio profile that governs a change in one or more respective audio characteristics of the second audio output over time; and

in accordance with a determination that the audio alteration criteria are not satisfied:

 causing the audio system to continue presenting the first audio output that corresponds to the first affordance and that has the first audio profile; and

 providing, to the audio system, third sound information to present a third audio output that corresponds to the second affordance, wherein the third audio output has a third audio profile that governs a change in one or more respective audio characteristics of the third audio output over time.

2 . The method of claim 1 , including:

in response to detecting the second input directed to the second affordance, determining whether the first audio output presented via the audio system by the second point in time satisfies the audio alteration criteria.

3 . The method of claim 1 , wherein the audio alteration criteria include a criterion that is satisfied when an amount of time elapsed between the first point in time and the second point in time is less than a predetermined amount of time.

4 . The method of claim 1 , wherein the audio alteration criteria include a criterion that is satisfied when a time elapsed since the first audio output was initiated is less than a predetermined amount of time.

5 . The method of claim 1 , wherein the audio alteration criteria include a criterion that is satisfied when an amplitude of the first audio output has dropped below a predetermined amplitude at the second point in time.

6 . The method of claim 1 , including:

while the audio system is presenting the first audio output, causing the display to present a visual effect that corresponds to the first audio output.

7 . The method of claim 6 , wherein the visual effect includes one or more graphics that extend outward, away from the first affordance.

8 . The method of claim 1 , wherein the first affordance and the second affordance are the same affordance.

9 . The method of claim 1 , wherein the first affordance is different from the second affordance.

10 . The method of claim 1 , wherein the displayed user interface includes a keypad and the first affordance and the second affordance correspond to a same key on the keypad.

11 . The method of claim 1 , wherein the displayed user interface includes a keypad and the first affordance and the second affordance are different keys on the keypad.

12 . The method of claim 1 , wherein the third audio output comprises the second audio output and the third audio profile comprises the second audio profile.

13 . The method of claim 1 , wherein the one or more respective audio characteristics for which changes are governed by the first audio profile and the second audio profile are selected from the group consisting of audio envelope, pitch, timbre, left-right balance, reverberation, and/or frequency filtering characteristics.

14 . An electronic device that is in communication with a display and an audio system, the electronic device comprising:

one or more processors;

memory; and

one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

while a user interface that includes a set of one or more affordances is displayed on the display:

detecting, at a first point in time, a first input directed to a first affordance in the set of one or more affordances;

in response to detecting the first input directed to the first affordance, initiating providing, to the audio system, first sound information to present a first audio output that corresponds to the first affordance, wherein the first audio output has a first audio profile that governs a change in one or more respective audio characteristics of the first audio output over time;

while continuing to provide the first sound information in response to the first input, detecting, at a second point in time that is after the first point in time, a second input directed to a second affordance in the set of one or more affordances; and

in response to detecting the second input directed to the second affordance:

while continuing to provide the first sound information in response to the first input and in accordance with a determination that audio alteration criteria are satisfied:

 causing the audio system to present altered first audio output that corresponds to the first affordance, instead of continuing to present the first audio output with the first audio profile, wherein the altered first audio output has an altered audio profile that is distinct from the first audio profile, wherein having the altered audio profile includes changing the one or more respective audio characteristics of the first audio output over time in a different manner than the one or more respective audio characteristics of the first audio output would have been changed over time if the first audio profile were to have been used; and

 providing, to the audio system, second sound information to present a second audio output that corresponds to the second affordance, wherein the second audio output has a second audio profile that governs a change in one or more respective audio characteristics of the second audio output over time; and

in accordance with a determination that the audio alteration criteria are not satisfied:

 causing the audio system to continue presenting the first audio output that corresponds to the first affordance and that has the first audio profile; and

 providing, to the audio system, third sound information to present a third audio output that corresponds to the second affordance, wherein the third audio output has a third audio profile that governs a change in one or more respective audio characteristics of the third audio output over time.

15 . The electronic device of claim 14 , wherein the one or more programs include instructions for:

in response to detecting the second input directed to the second affordance, determining whether the first audio output presented via the audio system by the second point in time satisfies the audio alteration criteria.

16 . The electronic device of claim 14 , wherein the audio alteration criteria include a criterion that is satisfied when an amount of time elapsed between the first point in time and the second point in time is less than a predetermined amount of time.

17 . The electronic device of claim 14 , wherein the audio alteration criteria include a criterion that is satisfied when a time elapsed since the first audio output was initiated is less than a predetermined amount of time.

18 . The electronic device of claim 14 , wherein the audio alteration criteria include a criterion that is satisfied when an amplitude of the first audio output has dropped below a predetermined amplitude at the second point in time.

19 . The electronic device of claim 14 , wherein the one or more programs include instructions for:

while the audio system is presenting the first audio output, causing the display to present a visual effect that corresponds to the first audio output.

20 . The electronic device of claim 14 , wherein the first affordance and the second affordance are the same affordance.

21 . The electronic device of claim 14 , wherein the first affordance is different from the second affordance.

22 . The electronic device of claim 14 , wherein the third audio output comprises the second audio output and the third audio profile comprises the second audio profile.

23 . The electronic device of claim 14 , wherein the one or more respective audio characteristics for which changes are governed by the first audio profile and the second audio profile are selected from the group consisting of audio envelope, pitch, timbre, left-right balance, reverberation, and/or frequency filtering characteristics.

24 . A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions that, when executed by an electronic device in communication with a display and an audio system, cause the electronic device to:

while a user interface that includes a set of one or more affordances is displayed on the display:

detect, at a first point in time, a first input directed to a first affordance in the set of one or more affordances;

in response to detecting the first input directed to the first affordance, initiate providing, to the audio system, first sound information to present a first audio output that corresponds to the first affordance, wherein the first audio output has a first audio profile that governs a change in one or more respective audio characteristics of the first audio output over time;

while continuing to provide the first sound information in response to the first input, detect, at a second point in time that is after the first point in time, a second input directed to a second affordance in the set of one or more affordances; and

in response to detecting the second input directed to the second affordance:

while continuing to provide the first sound information in response to the first input and in accordance with a determination that audio alteration criteria are satisfied:

cause the audio system to present altered first audio output that corresponds to the first affordance, instead of continuing to present the first audio output with the first audio profile, wherein the altered first audio output has an altered audio profile that is distinct from the first audio profile, wherein having the altered audio profile includes changing the one or more respective audio characteristics of the first audio output over time in a different manner than the one or more respective audio characteristics of the first audio output would have been changed over time if the first audio profile were to have been used; and

provide, to the audio system, second sound information to present a second audio output that corresponds to the second affordance, wherein the second audio output has a second audio profile that governs a change in one or more respective audio characteristics of the second audio output over time; and

in accordance with a determination that the audio alteration criteria are not satisfied:

cause the audio system to continue presenting the first audio output that corresponds to the first affordance and that has the first audio profile; and

provide, to the audio system, third sound information to present a third audio output that corresponds to the second affordance, wherein the third audio output has a third audio profile that governs a change in one or more respective audio characteristics of the third audio output over time.

25 . The non-transitory computer-readable storage medium of claim 24 , wherein the one or more programs include instructions that when executed by the electronic device cause the electronic device to:

in response to detecting the second input directed to the second affordance, determining whether the first audio output presented via the audio system by the second point in time satisfies the audio alteration criteria.

26 . The non-transitory computer-readable storage medium of claim 24 , wherein the audio alteration criteria include a criterion that is satisfied when an amount of time elapsed between the first point in time and the second point in time is less than a predetermined amount of time.

27 . The non-transitory computer-readable storage medium of claim 24 , wherein the audio alteration criteria include a criterion that is satisfied when a time elapsed since the first audio output was initiated is less than a predetermined amount of time.

28 . The non-transitory computer-readable storage medium of claim 24 , wherein the audio alteration criteria include a criterion that is satisfied when an amplitude of the first audio output has dropped below a predetermined amplitude at the second point in time.

29 . The non-transitory computer-readable storage medium of claim 24 , wherein the one or more programs include instructions that when executed by the electronic device cause the electronic device to:

while the audio system is presenting the first audio output, causing the display to present a visual effect that corresponds to the first audio output.

30 . The non-transitory computer-readable storage medium of claim 24 , wherein the first affordance and the second affordance are the same affordance.

31 . The non-transitory computer-readable storage medium of claim 24 , wherein the first affordance is different from the second affordance.

32 . The non-transitory computer-readable storage medium of claim 24 , wherein the third audio output comprises the second audio output and the third audio profile comprises the second audio profile.

33 . The non-transitory computer-readable storage medium of claim 24 , wherein the one or more respective audio characteristics for which changes are governed by the first audio profile and the second audio profile are selected from the group consisting of audio envelope, pitch, timbre, left-right balance, reverberation, and/or frequency filtering characteristics.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2023
From: VERWEIJ, HUGO D.; FAMILY, AFROOZ; BROWN, MATTHEW I.
To: APPLE INC.
Reel/Frame 062857/0699 →
Continuity (3)
Continuation 15620698 · Jun 12, 2017
Provisional Application 62349056 · Jun 12, 2016
Related Publication 20230080470A1 · Mar 16, 2023
References Cited (103)
US 9798512B1 · Faaborg et al. · 2017 [cited by applicant]
US 20060074649A1 · Pachet · 2006 [cited by examiner]
US 20060256131A1 · Trepess · 2006 [cited by applicant]
US 20070097151A1 · Rosenberg · 2007 [cited by applicant]
US 20070294297A1 · Kesteloot et al. · 2007 [cited by applicant]
US 20080297475A1 · Woolf et al. · 2008 [cited by applicant]
US 20090013059A1 · Partaker et al. · 2009 [cited by applicant]
US 20090061824A1 · Neelakantan et al. · 2009 [cited by applicant]
US 20100053348A1 · Yoshomoto et al. · 2010 [cited by applicant]
US 20100302179A1 · Ahn et al. · 2010 [cited by applicant]
US 20110050653A1 · Miyazawa et al. · 2011 [cited by applicant]
US 20110196520A1 · Peterson · 2011 [cited by examiner]
US 20110239155A1 · Christie · 2011 [cited by applicant]
US 20120038580A1 · Sasaki · 2012 [cited by applicant]
US 20120062491A1 · Coni · 2012 [cited by examiner]
US 20120105367A1 · Son et al. · 2012 [cited by applicant]
US 20120139844A1 · Ramstein et al. · 2012 [cited by applicant]
US 20120274662A1 · Kim et al. · 2012 [cited by applicant]
US 20130022218A1 · Miyazawa et al. · 2013 [cited by applicant]
US 20130044061A1 · Ashbrook · 2013 [cited by applicant]
US 20130077804A1 · Glebe et al. · 2013 [cited by applicant]
US 20130141396A1 · Lynn et al. · 2013 [cited by applicant]
US 20130169531A1 · Jahnke · 2013 [cited by examiner]
US 20130300590A1 · Dietz · 2013 [cited by examiner]
US 20140185834A1 · Fromel et al. · 2014 [cited by applicant]
US 20140214927A1 · Garmark et al. · 2014 [cited by applicant]
US 20140258853A1 · Raman · 2014 [cited by examiner]
US 20140267363A1 · Kocienda · 2014 [cited by examiner]
US 20150062052A1 · Berstein et al. · 2015 [cited by applicant]
US 20150143248A1 · Beechuk et al. · 2015 [cited by applicant]
US 20150200641A1 · Cho et al. · 2015 [cited by applicant]
US 20150234493A1 · Parivar et al. · 2015 [cited by applicant]
US 20150348168A1 · Quinn · 2015 [cited by applicant]
US 20160188181A1 · Smith · 2016 [cited by applicant]
US 20160342200A1 · Dziuk et al. · 2016 [cited by applicant]
US 20160345116A1 · Yen et al. · 2016 [cited by applicant]
US 20170068511A1 · Brown et al. · 2017 [cited by applicant]
US 20170177135A1 · Sarin · 2017 [cited by applicant]
US 20200363915A1 · Alonso Ruiz et al. · 2020 [cited by applicant]
AU 2016100246A4 · 2016 [cited by applicant]
AU 2016100293A4 · 2016 [cited by applicant]
AU 2016102063A4 · 2017 [cited by applicant]
CN 101072257A · 2007 [cited by applicant]
CN 101159437A · 2008 [cited by applicant]
CN 102098606A · 2011 [cited by applicant]
CN 102843640A · 2012 [cited by applicant]
CN 103928037A · 2014 [cited by applicant]
CN 104169848A · 2014 [cited by applicant]
CN 104487929A · 2015 [cited by applicant]
CN 104903834A · 2015 [cited by applicant]
CN 105117131A · 2015 [cited by applicant]
CN 105163186A · 2015 [cited by applicant]
CN 105185401A · 2015 [cited by applicant]
CN 105389078A · 2016 [cited by applicant]
EP 2463861A1 · 2012 [cited by applicant]
EP 2680121A1 · 2014 [cited by applicant]
EP 2808870A1 · 2014 [cited by applicant]
GB 2324642A · 1998 [cited by applicant]
WO WO2006085265A2 · 2006 [cited by applicant]
WO WO2014065846A1 · 2014 [cited by applicant]
WO WO2014105274A1 · 2014 [cited by applicant]
McGookin et al., “Pressure Player: Combined Pressure and Audio Interaction”, https://davidmcgookin.files.wordpress.com/2013/03/pressure_player_final.pdf, Aug. 18, 2008, 2 pages. [cited by applicant]
Office Action, dated Nov. 21, 2017, received in U.S. Appl. No. 15/620,698 (7373), 13 pages. [cited by applicant]
Final Office Action, dated Jul. 27, 2018, received in U.S. Appl. No. 15/620,698 (7373), 13 pages. [cited by applicant]
Office Action, dated May 3, 3019, received in U.S. Appl. No. 15/620,698 (7373), 13 pages. [cited by applicant]
Final Office Action, dated Jul. 25, 2019, received in U.S. Appl. No. 15/620,698 (7373), 14 pages. [cited by applicant]
Office Action, dated Jun. 22, 2020, received in U.S. Appl. No. 15/620,698 (7373), 29 pages. [cited by applicant]
Final Office Action, dated Apr. 6, 2021, received in U.S. Appl. No. 15/620,698 (7373), 30 pages. [cited by applicant]
Office Action, dated May 27, 2019, received in Australian Patent Application No. 2017286013 (7373AU), which corresponds with U.S. Appl. No. 15/620,698, 4 pages. [cited by applicant]
Notice of Acceptance, dated Aug. 15, 2019, received in Australian Patent Application No. 2017286013 (7373AU), which corresponds with U.S. Appl. No. 15/620,698, 3 pages. [cited by applicant]
Certificate of Grant, dated Dec. 19, 2019, received in Australian Patent Application No. 2017286013 (7373AU), which corresponds with U.S. Appl. No. 15/620,698, 4 pages. [cited by applicant]
Office Action, dated Sep. 19, 2018, received in Chinese Patent Application No. 201710364610.3 (7373CN), which corresponds with U.S. Appl. No. 15/620,698, 6 pages. [cited by applicant]
Office Action, dated Apr. 15, 2019, received in Chinese Patent Application No. 201710364610.3 (7373CN), which corresponds with U.S. Appl. No. 15/620,698, 3 pages. [cited by applicant]
Office Action, dated Sep. 4, 2019, received in Chinese Patent Application No. 201710364610.3 (7373CN), which corresponds with U.S. Appl. No. 15/620,698, 3 pages. [cited by applicant]
Notice of Allowance, dated Jan. 2, 2020, received in Chinese Patent Application No. 201710364610.3 (7373CN), which corresponds with U.S. Appl. No. 15/620,698, 5 pages. [cited by applicant]
Patent, dated Mar. 27, 2020, received in Chinese Patent Application No. 201710364610.3 (7373CN), which corresponds with U.S. Appl. No. 15/620,698, 6 pages. [cited by applicant]
Office Action, dated Jul. 2, 2018, received in Chinese Patent Application No. 201810369048.8 (7373CN01), which corresponds with U.S. Appl. No. 15/620,698, 2 pages. [cited by applicant]
Office Action, dated May 15, 2019, received in Chinese Patent Application No. 201810369048.8 (7373CN01), which corresponds with U.S. Appl. No. 15/620,698, 6 pages. [cited by applicant]
Office Action, dated Nov. 21, 2019, received in Chinese Patent Application No. 201810369048.8 (7373CN01), which corresponds with U.S. Appl. No. 15/620,698, 6 pages. [cited by applicant]
Office Action, dated Jun. 9, 2020, received in Chinese Patent Application No. 201810369048.8 (7373CN01), which corresponds with U.S. Appl. No. 15/620,698, 7 pages. [cited by applicant]
Notice of Allowance, dated Nov. 18, 2020, received in Chinese Patent Application No. 201810369048.8 (7373CN01), which corresponds with U.S. Appl. No. 15/620,698, 5 pages. [cited by applicant]
Patent, dated Jan. 8, 2021, received in Chinese Patent Application No. 201810369048.8 (7373CN01), which corresponds with U.S. Appl. No. 15/620,698, 6 pages. [cited by applicant]
Office Action, dated Oct. 21, 2016, received in Danish Patent Application No. 201670597 (7373DK), which corresponds with U.S. Appl. No. 15/620,698, 7 pages. [cited by applicant]
Notice of Allowance, dated May 5, 2017, received in Danish Patent Application No. 201670597 (7373DK), which corresponds with U.S. Appl. No. 15/620,698, 2 pages. [cited by applicant]
Office Action, dated Sep. 29, 2016, received in Danish Patent Application No. 201670599 (7373DK01), which corresponds with U.S. Appl. No. 15/620,698, 8 pages. [cited by applicant]
Office Action, dated Jul. 25, 2018, received in European Patent Application No. 17163558.4 (7373EP), which corresponds with U.S. Appl. No. 15/620,698, 8 pages. [cited by applicant]
Summons, dated Jan. 28, 2019, received in European Patent Application No. 17163558.4 (7373EP), which corresponds with U.S. Appl. No. 15/620,698, 12 pages. [cited by applicant]
Intention to Grant, dated Jul. 9, 2019, received in European Patent Application No. 17163558.4 (7373EP), which corresponds with U.S. Appl. No. 15/620,698, 5 pages. [cited by applicant]
Summons, dated Sep. 27, 2019, received in European Patent Application No. 17163558.4 (7373EP), which corresponds with U.S. Appl. No. 15/620,698, 4 pages. [cited by applicant]
Intention to Grant, dated Mar. 2, 2020, received in European Patent Application No. 17163558.4 (7373EP), which corresponds with U.S. Appl. No. 15/620,698, 5 pages. [cited by applicant]
Decision to Grant, dated Jul. 23, 2020, received in European Patent Application No. 17163558.4 (7373EP), which corresponds with U.S. Appl. No. 15/620,698, 2 pages. [cited by applicant]
Patent, dated Aug. 19, 2020, received in European Patent Application No. 17163558.4 (7373EP), which corresponds with U.S. Appl. No. 15/620,698,3 pages. [cited by applicant]
Office Action, dated Jul. 28, 2020, received in Australian Patent Application No. 2019257439 (7648AU), which corresponds from U.S. Appl. No. 16/985,128, 4 pages. [cited by applicant]
Patent, dated Feb. 25, 2021, received in Australian Patent Application No. 2019257439 (7648AU), which corresponds from U.S. Appl. No. 16/985,128, 3 pages. [cited by applicant]
Extended European Search Report, dated Sep. 29, 2017, received in European Patent Application No. 17163558.4 (7373EP), which corresponds with U.S. Appl. No. 15/620,698, 9 pages. [cited by applicant]
International Search Report and Written Opinion, dated Sep. 22, 2017, received in International Patent Application No. PCT/US2017/027451 7373WO), which corresponds with U.S. Appl. No. 15/620,698, 14 pages. [cited by applicant]
International Preliminary Report on Patentability, dated Dec. 18, 2018, received in International Patent Application No. PCT/US2017/027451 7373WO), which corresponds with U.S. Appl. No. 15/620,698, 9 pages. [cited by applicant]
Extended European Search Report, dated Oct. 19, 2020, received in European Patent Application No. 20185278.7 (7648EP), which corresponds with U.S. Appl. No. 16/985,128, 8 pages. [cited by applicant]
Office Action, dated Nov. 9, 2022, received in U.S. Appl. No. 16/985,128 (7648), 23 pages. [cited by applicant]
Notice of Allowance, dated Mar. 29, 2023, received in U.S. Appl. No. 16/985,128 (7648), 25 pages. [cited by applicant]
Office Action, dated Jan. 12, 2023, received in Chinese Patent Application No. 202010072444.1 (7648CN), which corresponds from U.S. Appl. No. 16/985,128, 4 pages. [cited by applicant]
Patent, dated Jul. 25, 2023, received in Chinese Patent Application No. 202010072444.1 (7648CN), which corresponds from U.S. Appl. No. 16/985,128, 6 pages. [cited by applicant]
Office Action, dated Apr. 22, 2024, received in European Patent Application 20189278.7 (7648EP), which corresponds with U.S. Appl. No. 16/985,128, 4 pages. [cited by applicant]