Granted Patent B2

US 12,711,329 · App. 17/833,537 · Granted Aug 18, 2026

Inventors: Ngoc H. Pham (San Francisco, CA); Vincent M. Lane (San Francisco, CA); Grant R. Paul (San Francisco, CA); Chen Ye (Seattle, WA)

Assignee: Apple Inc.

G06F40/58G06F3/167G06F40/103G06F40/169G06F40/171G06F40/35G10L13/086G10L15/005G10L15/22G10L15/26

Abstract

Claims (129)

1 . A computer system configured to communicate with a display generation component and one or more input devices, comprising:

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:

displaying, via the display generation component, a translation user interface, wherein a first region of the translation user interface corresponds to a first language and a second region of the translation user interface different from the first region corresponds to a second language different from the first language, wherein displaying the translation user interface comprises:

in accordance with a determination that an automatic language detection setting is disabled, concurrently displaying:

a first audio input selectable object corresponding to the first language that is selectable to engage a first language listening mode of the computer system in which the computer system is configured to receive an audio input in the first language; and

a second audio input selectable object corresponding to the second language that is selectable to engage a second language listening mode of the computer system in which the computer system is configured to receive an audio input in the second language; and

in accordance with a determination that the automatic language detection setting is enabled, displaying a single audio input selectable object that is selectable to engage a general listening mode of the computer system in which the computer system is configured to receive an audio input in a plurality of languages;

while displaying the translation user interface, receiving, via the one or more input devices, an input, wherein the input is an audio input; and

in response to receiving the input:

in accordance with a determination that the input is in the first language, concurrently displaying, via the display generation component, in the first region of the translation user interface:

a transcription of the input in the first language, and

a translation of the input in the second language; and

in accordance with a determination that the input is in the second language, concurrently displaying, via the display generation component, in the second region of the translation user interface:

a transcription of the input in the second language, and

a translation of the input in the first language.

2 . The computer system of claim 1 , wherein:

the translation user interface comprises an audio input selectable object that is selectable to engage or disengage a listening mode of the computer system, wherein when the listening mode of the computer system is engaged, the computer system is configured to receive an audio input, and when the listening mode of the computer system is disengaged, the computer system is not configured to receive an audio input; and

the one or more programs further include instructions for:

while displaying the translation user interface and the listening mode of the computer system is disengaged, and prior to receiving the audio input, detecting, via the one or more input devices, one or more user inputs corresponding to selection of the audio input selectable object; and

in response to detecting the one or more user inputs corresponding to selection of the audio input selectable object, engaging the listening mode of the computer system, wherein:

receiving the audio input is performed while the listening mode of the computer system is engaged.

3 . The computer system of claim 2 , the one or more programs further including instructions for:

after receiving the audio input:

in accordance with a determination that a continuous listening setting is enabled, maintaining the listening mode of the computer system in the engaged state until one or more user inputs are received indicative of a request to disengage the listening mode of the computer system; and

in accordance with a determination that the continuous listening setting is disabled, automatically disengaging the listening mode of the computer system after receiving the audio input.

4 . The computer system of claim 1 , wherein:

when the automatic language detection setting is enabled and the general listening mode of the computer system is engaged, the computer system is configured to receive an audio input in the first language and/or in the second language.

5 . The computer system of claim 1 , the one or more programs further including instructions for:

in response to receiving the input:

in accordance with a determination that the input is in the first language:

in accordance with a determination that an autoplay setting is enabled, outputting an audio output of the translation of the input in the second language; and

in accordance with a determination that the autoplay setting is disabled, forgoing outputting the audio output of the translation of the input in the second language.

6 . The computer system of claim 1 , the one or more programs further including instructions for:

displaying, via the display generation component, a display mode option that is selectable to initiate a process for switching the translation user interface between a first display mode and a second display mode, wherein:

in the first display mode:

the first region of the translation user interface displays transcriptions in the first language and translations in the second language in a first orientation, and

the second region of the translation user interface displays transcriptions in the second language and translations in the first language in the first orientation; and

in the second display mode:

a third region of the translation user interface displays transcriptions in the first language and translations in the first language in a second orientation; and

a fourth region of the translation user interface different from the third region displays transcriptions in the second language and translations in the second language in a third orientation different from the second orientation.

7 . A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with a display generation component and one or more input devices, the one or more programs including instructions for:

displaying, via the display generation component, a translation user interface, wherein a first region of the translation user interface corresponds to a first language and a second region of the translation user interface different from the first region corresponds to a second language different from the first language, wherein displaying the translation user interface comprises:

in accordance with a determination that an automatic language detection setting is disabled, concurrently displaying:

a first audio input selectable object corresponding to the first language that is selectable to engage a first language listening mode of the computer system in which the computer system is configured to receive an audio input in the first language; and

a second audio input selectable object corresponding to the second language that is selectable to engage a second language listening mode of the computer system in which the computer system is configured to receive an audio input in the second language; and

in accordance with a determination that the automatic language detection setting is enabled, displaying a single audio input selectable object that is selectable to engage a general listening mode of the computer system in which the computer system is configured to receive an audio input in a plurality of languages;

while displaying the translation user interface, receiving, via the one or more input devices, an input, wherein the input is an audio input; and

in response to receiving the input:

in accordance with a determination that the input is in the first language, concurrently displaying, via the display generation component, in the first region of the translation user interface:

a transcription of the input in the first language, and

a translation of the input in the second language; and

in accordance with a determination that the input is in the second language, concurrently displaying, via the display generation component, in the second region of the translation user interface:

a transcription of the input in the second language, and

a translation of the input in the first language.

8 . The non-transitory computer-readable storage medium of claim 7 , wherein:

the translation user interface comprises an audio input selectable object that is selectable to engage or disengage a listening mode of the computer system, wherein when the listening mode of the computer system is engaged, the computer system is configured to receive an audio input, and when the listening mode of the computer system is disengaged, the computer system is not configured to receive an audio input; and

the one or more programs further include instructions for:

while displaying the translation user interface and the listening mode of the computer system is disengaged, and prior to receiving the audio input, detecting, via the one or more input devices, one or more user inputs corresponding to selection of the audio input selectable object; and

in response to detecting the one or more user inputs corresponding to selection of the audio input selectable object, engaging the listening mode of the computer system, wherein:

receiving the audio input is performed while the listening mode of the computer system is engaged.

9 . The non-transitory computer-readable storage medium of claim 8 , the one or more programs further including instructions for:

after receiving the audio input:

in accordance with a determination that a continuous listening setting is enabled, maintaining the listening mode of the computer system in the engaged state until one or more user inputs are received indicative of a request to disengage the listening mode of the computer system; and

in accordance with a determination that the continuous listening setting is disabled, automatically disengaging the listening mode of the computer system after receiving the audio input.

10 . The non-transitory computer-readable storage medium of claim 7 , wherein:

when the automatic language detection setting is enabled and the general listening mode of the computer system is engaged, the computer system is configured to receive an audio input in the first language and/or in the second language.

11 . The non-transitory computer-readable storage medium of claim 7 , the one or more programs further including instructions for:

in response to receiving the input:

in accordance with a determination that the input is in the first language:

in accordance with a determination that an autoplay setting is enabled, outputting an audio output of the translation of the input in the second language; and

in accordance with a determination that the autoplay setting is disabled, forgoing outputting the audio output of the translation of the input in the second language.

12 . The non-transitory computer-readable storage medium of claim 7 , the one or more programs further including instructions for:

displaying, via the display generation component, a display mode option that is selectable to initiate a process for switching the translation user interface between a first display mode and a second display mode, wherein:

in the first display mode:

the first region of the translation user interface displays transcriptions in the first language and translations in the second language in a first orientation, and

the second region of the translation user interface displays transcriptions in the second language and translations in the first language in the first orientation; and

in the second display mode:

a third region of the translation user interface displays transcriptions in the first language and translations in the first language in a second orientation; and

a fourth region of the translation user interface different from the third region displays transcriptions in the second language and translations in the second language in a third orientation different from the second orientation.

13 . A method, comprising:

at a computer system that is in communication with a display generation component and one or more input devices:

displaying, via the display generation component, a translation user interface, wherein a first region of the translation user interface corresponds to a first language and a second region of the translation user interface different from the first region corresponds to a second language different from the first language, wherein displaying the translation user interface comprises:

in accordance with a determination that an automatic language detection setting is disabled, concurrently displaying:

a first audio input selectable object corresponding to the first language that is selectable to engage a first language listening mode of the computer system in which the computer system is configured to receive an audio input in the first language; and

a second audio input selectable object corresponding to the second language that is selectable to engage a second language listening mode of the computer system in which the computer system is configured to receive an audio input in the second language; and

in accordance with a determination that the automatic language detection setting is enabled, displaying a single audio input selectable object that is selectable to engage a general listening mode of the computer system in which the computer system is configured to receive an audio input in a plurality of languages;

while displaying the translation user interface, receiving, via the one or more input devices, an input, wherein the input is an audio input; and

in response to receiving the input:

in accordance with a determination that the input is in the first language, concurrently displaying, via the display generation component, in the first region of the translation user interface:

a transcription of the input in the first language, and

a translation of the input in the second language; and

in accordance with a determination that the input is in the second language, concurrently displaying, via the display generation component, in the second region of the translation user interface:

a transcription of the input in the second language, and

 a translation of the input in the first language.

14 . The method of claim 13 , wherein:

the translation user interface comprises an audio input selectable object that is selectable to engage or disengage a listening mode of the computer system, wherein when the listening mode of the computer system is engaged, the computer system is configured to receive an audio input, and when the listening mode of the computer system is disengaged, the computer system is not configured to receive an audio input; and

the method further comprises:

while displaying the translation user interface and the listening mode of the computer system is disengaged, and prior to receiving the audio input, detecting, via the one or more input devices, one or more user inputs corresponding to selection of the audio input selectable object; and

in response to detecting the one or more user inputs corresponding to selection of the audio input selectable object, engaging the listening mode of the computer system, wherein:

receiving the audio input is performed while the listening mode of the computer system is engaged.

15 . The method of claim 14 , further comprising:

after receiving the audio input:

in accordance with a determination that a continuous listening setting is enabled, maintaining the listening mode of the computer system in the engaged state until one or more user inputs are received indicative of a request to disengage the listening mode of the computer system; and

in accordance with a determination that the continuous listening setting is disabled, automatically disengaging the listening mode of the computer system after receiving the audio input.

16 . The method of claim 13 , wherein:

when the automatic language detection setting is enabled and the general listening mode of the computer system is engaged, the computer system is configured to receive an audio input in the first language and/or in the second language.

17 . The method of claim 13 , further comprising:

in response to receiving the input:

in accordance with a determination that the input is in the first language:

in accordance with a determination that an autoplay setting is enabled, outputting an audio output of the translation of the input in the second language; and

in accordance with a determination that the autoplay setting is disabled, forgoing outputting the audio output of the translation of the input in the second language.

18 . The method of claim 13 , further comprising:

displaying, via the display generation component, a display mode option that is selectable to initiate a process for switching the translation user interface between a first display mode and a second display mode, wherein:

in the first display mode:

the first region of the translation user interface displays transcriptions in the first language and translations in the second language in a first orientation, and

the second region of the translation user interface displays transcriptions in the second language and translations in the first language in the first orientation; and

in the second display mode:

a third region of the translation user interface displays transcriptions in the first language and translations in the first language in a second orientation; and

a fourth region of the translation user interface different from the third region displays transcriptions in the second language and translations in the second language in a third orientation different from the second orientation.

19 . The computer system of claim 1 , the one or more programs further including instructions for:

while displaying the translation of the input in the second language, detecting, via the one or more input devices, one or more user inputs corresponding to selection of the translation of the input in the second language; and

in response to detecting the one or more user inputs corresponding to selection of the translation of the input in the second language, displaying, via the display generation component, one or more selectable options pertaining to the translation of the input in the second language.

20 . The non-transitory computer-readable storage medium of claim 7 , the one or more programs further including instructions for:

while displaying the translation of the input in the second language, detecting, via the one or more input devices, one or more user inputs corresponding to selection of the translation of the input in the second language; and

in response to detecting the one or more user inputs corresponding to selection of the translation of the input in the second language, displaying, via the display generation component, one or more selectable options pertaining to the translation of the input in the second language.

21 . The method of claim 13 , further comprising:

while displaying the translation of the input in the second language, detecting, via the one or more input devices, one or more user inputs corresponding to selection of the translation of the input in the second language; and

in response to detecting the one or more user inputs corresponding to selection of the translation of the input in the second language, displaying, via the display generation component, one or more selectable options pertaining to the translation of the input in the second language.

Assignments (1)

Continuity (2)

References Cited (168)