IP Library › Granted Patent US 12,217,346
Granted Patent B2
US 12,217,346 · App. 18/308,321 · Granted Feb 4, 2025

Integrated input/output (I/O) for a three-dimensional (3D) environment

Inventors: David B. Baszucki (Portola Valley, CA); Philippe Clavel (Belmont, CA); Morgan Clemens Tucker (San Francisco, CA)
Assignee: Roblox Corporation
G06T13/40G06T13/205G06T19/20G06F3/04815G06F3/167G06F2203/04802G06T2219/2004H04L65/1089H04L65/403H04N7/15H04N7/157
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,217,346
App. No.
18/308,321
Granted
Feb 4, 2025
Kind
B2
Abstract

Various input modes and output modes may be used for a three-dimensional (3D) environment. A user may use a particular input mode (e.g., text, audio, video, etc.) for animating a 3D avatar of the user in the 3D environment. The user may use a particular output mode (e.g., text, audio, 3D animation, etc.) in the presentation of the 3D environment. The input/output modes may change based on conditions such as a location of the user.

Claims (55)

1. A method to operate a three-dimensional (3D) environment, the method comprising:

receiving a first input signal from a first user, and using the first input signal for animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user;

presenting the 3D environment to the first user and to a second user during a 3D session, wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user, wherein a first appearance of the first avatar during the 3D session is based on a setting chosen by the first user for presentation of the first avatar in the 3D environment to the second user, and wherein the first output mode selected by the second user overrides the setting chosen by the first user for presentation of the first avatar, so as to present the first avatar to the second user with a second appearance during the 3D session that is different from the first appearance;

in response to a change in a physical location of the first user, changing an input mode of the first user from the first input mode to a second input mode;

in response to the change in the physical location of the first user, changing the first output mode selected by the first user from the first output mode to a second output mode that is usable in the changed physical location while the first output mode is disabled; and

in response to a change in a physical location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed physical location of the second user and further based on capabilities of a device of the second user.

2. The method of claim 1 , wherein the other output mode disables a visual presentation of the second appearance of the first avatar and of the 3D environment on the device of the second user, and presents output from the first user to the second user as text-only or audio-only on the device of the second user.

3. The method of claim 1 , wherein:

the first input mode of the first user is text input,

the second input mode which has been changed from the first input mode in response to the change in the physical location of the first user, is video input or audio input, and

the first input mode is disabled in response to the change in the physical location of the first user.

4. The method of claim 1 , wherein:

the first input mode of the first user is audio input, and

the second input mode, which has been changed from the first input mode in response to the change in the physical location of the first user, is text input.

5. The method of claim 1 , wherein the change in the physical location of the first user is detected by a global positioning system (GPS) tool on a device of the first user.

6. The method of claim 1 , wherein the animation of the first avatar is further based in part on emotion of the first user as determined by a sensor on a device of the first user.

7. An apparatus, comprising:

a processor; and

a non-transitory computer-readable medium coupled to the processor and having instructions stored thereon that, responsive to execution by the processor, cause the processor to perform or control performance of operations for a three-dimensional (3D) environment, wherein the operations comprise:

receive a first input signal from a first user, and use the first input signal for animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user;

present the 3D environment to the first user and to a second user during a 3D session, wherein presentation of the 3D environment during the 3D session to the first user and to the second user is based on respective first output modes selected by the first user and by the second user, wherein a first appearance of the first avatar during the 3D session is based on a setting chosen by the first user for presentation of the first avatar in the 3D environment to the second user, and wherein the first output mode selected by the second user overrides the setting chosen by the first user for presentation of the first avatar, so as to present the first avatar to the second user with a second appearance during the 3D session that is different from the first appearance;

in response to a change in a physical location of the first user, change an input mode of the first user from the first input mode to a second input mode;

in response to the change in the physical location of the first user, change the first output mode selected by the first user from the first output mode to a second output mode that is usable in the changed physical location while the first output mode is disabled; and

in response to a change in a physical location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed physical location of the second user and further based on capabilities of a device of the second user.

8. The apparatus of claim 7 , wherein the other output mode disables a visual presentation of the second appearance of the first avatar and of the 3D environment on the device of the second user, and presents output from the first user to the second user as text-only or audio-only on the device of the second user.

9. The apparatus of claim 7 , wherein:

the first input mode of the first user is text input,

the second input mode which has been changed from the first input mode in response to the change in the physical location of the first user, is video input or audio input, and

the first input mode is disabled in response to the change in the physical location of the first user.

10. The apparatus of claim 7 , wherein:

the first input mode of the first user is audio input, and

the second input mode, which has been changed from the first input mode in response to the change in the physical location of the first user, is text input.

11. The apparatus of claim 7 , wherein the change in the physical location of the first user is detected by a global positioning system (GPS) tool on a device of the first user.

12. The apparatus of claim 7 , wherein the animation of the first avatar is further based in part on emotion of the first user as determined by a sensor on a device of the first user.

13. A method to operate a three-dimensional (3D) environment, the method comprising:

receiving a first input signal from a first user, and using the first input signal for animation of a first avatar representative of the first user in the 3D environment, wherein the animation of the first avatar is based at least in part on a first input mode through which the first input signal is received from the first user;

presenting the 3D environment to the first user and to second, third, and fourth users at runtime during a 3D session, wherein the runtime presentation of the 3D environment during the 3D session to the first user and to the second, third, and fourth users is based on respective first output modes selected by the first user and by the second, third, and fourth users,

wherein a first appearance of the first avatar during the 3D session is based on a setting chosen by the first user for presentation of the first avatar in the 3D environment to the second, third, and fourth users,

wherein the first output mode selected by the second user overrides the setting chosen by the first user for presentation of the first avatar, so as to present the first avatar to the second user with a second appearance during the 3D session that is different from the first appearance,

wherein the first output mode selected by the third user overrides the setting chosen by the first user for presentation of the first avatar, so as to present the first avatar to the third user with a third appearance during the 3D session that is different from the first and second appearances,

wherein the first output mode selected by the fourth user presents the first avatar to the fourth user with the first appearance during the 3D session, and

wherein the first, second, and third appearances of the first avatar are presented concurrently, in the runtime presentation of the 3D environment during the 3D session, to the fourth, second, and third users, respectively;

in response to a change in a location of the first user, changing an input mode of the first user from the first input mode to a second input mode;

in response to the change in the location of the first user, changing the first output mode selected by the first user from the first output mode to a second output mode that is usable in the changed location while the first output mode is disabled; and

in response to a change in a location of the second user, changing the first output mode selected by the second user from the first output mode selected by the second user to some other output mode that is usable in the changed location of the second user and further based on capabilities of a device of the second user.

14. The method of claim 13 , wherein the animation of the first avatar is further based in part on emotion of the first user as determined by a sensor on a device of the first user.

15. The method of claim 13 , wherein the change in the location of the first user is a change in a physical location of the first user.

16. The method of claim 13 , wherein:

the first input mode of the first user is text input,

the second input mode which has been changed from the first input mode in response to the change in the location of the first user, is video input or audio input, and

the first input mode is disabled in response to the change in the location of the first user.

17. The method of claim 13 , wherein:

the first input mode of the first user is audio input, and

the second input mode, which has been changed from the first input mode in response to the change in the location of the first user, is text input.

18. The method of claim 13 , wherein the change in the location of the first user is detected by a global positioning system (GPS) tool on a device of the first user.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 27, 2023
From: BASZUCKI, DAVID B.; CLAVEL, PHILIPPE; TUCKER, MORGAN CLEMENS
To: ROBLOX CORPORATION
Reel/Frame 063466/0278 →
Continuity (3)
Continuation 17375062 · Jul 14, 2021
Provisional Application 63155064 · Mar 1, 2021
Related Publication 20230334743A1 · Oct 19, 2023
References Cited (44)
US 5598209A · Cortjens · 1997 [cited by examiner]
US 9930470B2 · Reilly · 2018 [cited by examiner]
US 10521946B1 · Roche et al. · 2019 [cited by applicant]
US 11140360B1 · Kasaba · 2021 [cited by applicant]
US 20080155632A1 · Marilly et al. · 2008 [cited by applicant]
US 20090254843A1 · Van Wie · 2009 [cited by examiner]
US 20100045697A1 · Reville · 2010 [cited by examiner]
US 20100083140A1 · Dawson et al. · 2010 [cited by applicant]
US 20110072367A1 · Bauer · 2011 [cited by examiner]
US 20120204120A1 · Lefar et al. · 2012 [cited by applicant]
US 20120209933A1 · Ridges · 2012 [cited by examiner]
US 20150133197A1 · Kwak et al. · 2015 [cited by applicant]
US 20160134840A1 · McCulloch · 2016 [cited by applicant]
US 20170134821A1 · D'Amelio · 2017 [cited by examiner]
US 20170337360A1 · Gaspard et al. · 2017 [cited by applicant]
US 20180098059A1 · Valdivia · 2018 [cited by examiner]
US 20190313059A1 · Agarawala et al. · 2019 [cited by applicant]
US 20190321725A1 · Zimring et al. · 2019 [cited by applicant]
US 20200368616A1 · Delamont · 2020 [cited by applicant]
US 20210012770A1 · Choudhary et al. · 2021 [cited by applicant]
US 20210256261A1 · Wang et al. · 2021 [cited by applicant]
US 20220150083A1 · Faulkner · 2022 [cited by applicant]
CN 104995662 · 2015 [cited by applicant]
JP 2003037826 · 2003 [cited by applicant]
JP 2004266746 · 2004 [cited by applicant]
JP 2004274550 · 2004 [cited by applicant]
JP 2012063921 · 2012 [cited by applicant]
KR 1020080058281 · 2008 [cited by applicant]
EPO, Extended European Search Report for European Patent Application No. 22155039.5, Jul. 27, 2022, 10 pages. [cited by applicant]
IPO, First Examination Report for Indian Patent Application No. 202214010731, Sep. 12, 2022, 6 pages. [cited by applicant]
JPO, Notice of Reasons for Rejection (with English translation) for Japanese Patent Application No. 2022-29442, Jun. 26, 2023, 10 pages. [cited by applicant]
Nvidia, “AI-Powered Video Conferencing with NVIDIA Maxine”, Video available at https://www.youtube.com/watch?v=eFK7ly8enqM, Oct. 5, 2020, 3 pages. [cited by applicant]
USPTO, Non-final Office Action for U.S. Appl. No. 17/375,062, Jan. 19, 2022, 19 pages. [cited by applicant]
USPTO, Non-final Office Action for U.S. Appl. No. 17/375,062, Aug. 26, 2022, 27 pages. [cited by applicant]
USPTO, Final Office Action for U.S. Appl. No. 17/375,062, May 13, 2022, 29 pages. [cited by applicant]
USPTO, Notice of Allowance for U.S. Appl. No. 17/375,062, Jan. 12, 2023, 9 pages. [cited by applicant]
Vincent James, “Nvidia says its AI can fix some of the biggest problems in video calls”, The Verge, available at https://www.theverge.com/2020/10/5/21502003/nvidia-ai-videoconferencing-maxine-platform-face-gaze-alignmen… [cited by applicant]
CNIPA, First Office Action (with English translation) for Chinese Patent Application No. 202210182907.9, Sep. 11, 2023, 16 pages. [cited by applicant]
KIPO, Office Action (with English translation) for Korean Patent Application No. 10-2022-0019520, Mar. 20, 2024, 13 pages. [cited by applicant]
JPO, Notice of Reasons for Rejection (with English translation) for Japanese Patent Application No. 2022-29442, Dec. 18, 2023, 11 pages. [cited by applicant]
CNIPA, Second Office Action (with English translation) for Chinese Patent Application No. 202210182907.9, Apr. 10, 2024, 16 pages. [cited by applicant]
JPO, Notice of Allowance (with English translation) for Japanese Patent Application No. 2022-29442, May 13, 2024, 6 pages. [cited by applicant]
CNIPA, Decision of Rejection (with English translation) for Chinese Patent Application No. 202210182907.9, Jul. 15, 2024, 10 pages. [cited by applicant]
KIPO, Notice of Allowance (with English translation) for Korean Patent Application No. 10-2022-0019520, Oct. 31, 2024, 7 pages. [cited by applicant]