IP Library Granted Patent US 12,656,997
Granted Patent B2
US 12,656,997 · App. 18/082,925 · Granted Jun 16, 2026

Virtual sound engineer system and method

Inventors: Wallace Levi Coleman (Wilmington, DE); Charles Homer (Lawrenceville, GA)
Assignee: VIRTUAL SOUND ENGINEER, INC.
G06F3/165G06F3/04815G06F3/04847G06T15/10G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,656,997
App. No.
18/082,925
Granted
Jun 16, 2026
Kind
B2
Abstract

A virtual sound engineer system includes a mobile device having a processor connected to an interface, the processor being configured to initiate a first remote access digital mixing session to remotely access a first digital mixing console communicatively coupled to a first plurality of peripheral devices disposed at a first location, initiate a second remote access digital mixing session to remotely access a second digital mixing console communicatively coupled to a second plurality of peripheral devices disposed at a second location different from the first, such that remotely accessing the first console and the second console includes adjusting sound output by at least one of the peripheral devices of the first console and at least one of the peripheral devices of the second console and at least a portion of the first mixing session and at least a portion of the second mixing session occur concurrently.

Claims (70)

1 . A virtual sound engineer system comprising a mobile device coupled to a head-mounted display, the mobile device configured to:

in response to receipt of a first access code including authorization information, initiate a first remote access digital mixing session by generating a virtual sound engineering dashboard to remotely access a first digital mixing console disposed at a first location, and wherein the first digital mixing console is communicatively coupled to a first plurality of peripheral devices disposed at the first location;

receive real-time audio and video data captured by at least one of the peripheral devices of the first digital mixing console at the first location in response to initiating the first remote access digital mixing session;

display, with the head-mounted display at a second location different from the first location, the virtual sound engineering dashboard in a virtual reality interface based on the real-time audio and video data; and

control the first digital mixing console with the virtual sound engineering dashboard in the virtual reality interface;

wherein the mobile device is further configured to:

in response to receiving a second access code including authorization information, initiate a second remote access digital mixing session by generating the virtual sound engineering dashboard to remotely access a second digital mixing console disposed at a third location, and wherein the second digital mixing console is communicatively coupled to a second plurality of peripheral devices disposed at the third location; and

display, with the head-mounted display at the second location different from the first location and the third location, the virtual sound engineering dashboard in the virtual reality interface;

wherein the first remote access digital mixing session and the second remote access digital mixing session occur concurrently at the first digital mixing console and the second digital mixing console,

wherein:

the received real-time audio and video data corresponds with real-time audio and video events at the first location which corresponds with the first digital mixing console;

real-time modifications are made to the received real-time audio and video data at the third location which corresponds to the second digital mixing console; and

the real-time modifications made to the received real-time audio and video data at the third location are concurrently broadcast back to the first location as the real-time modifications to the received real-time audio and video data are made at the third location.

2 . The system of claim 1 , wherein to display the virtual sound engineering dashboard in the virtual reality interface comprises to render a three-dimensional representation of the first digital mixing console.

3 . The system of claim 1 , wherein to display the virtual sound engineering dashboard in the virtual reality interface comprises to render a three-dimensional representation of the first location based on the real-time audio and video data captured by the at least one of the peripheral devices.

4 . The system of claim 1 , wherein the first digital mixing console comprises a plurality of user controls corresponding to the plurality of peripheral devices communicatively coupled thereto, and wherein to display the virtual sound engineering dashboard in the virtual reality interface comprises to render a plurality of virtual user controls corresponding to the plurality of user controls.

5 . The system of claim 4 , wherein:

the mobile device is further configured to receive user input indicating a change in position of a virtual user control associated with at least one of the plurality of peripheral devices; and

to control the first digital mixing console comprises to control real-time audio output by the at least one of the plurality of peripheral devices in response to receipt of the user input.

6 . The system of claim 1 , wherein:

to receive the real-time audio and video data comprises to receive first real-time audio data indicative of an environment of the first location and second real-time audio data indicative of audio output by at least one of the plurality of peripheral devices communicatively coupled to the first digital mixing console; and

to display the virtual sound engineering dashboard in the virtual reality interface comprises to provide output of the first real-time audio data or the second real-time audio data.

7 . The system of claim 1 , wherein the mobile device is further configured to:

display, with the head-mounted display, a notification indicative of a request to initiate the first remote access digital mixing session;

generate the first access code including the authorization information in response to display of the notification; and

transmit the first access code including the authorization information to a first user device disposed at the first location and coupled to the first digital mixing console;

wherein the first access code is received from the first user device after transmission of the first access code.

8 . The system of claim 1 , wherein the mobile device is further configured to control an audiovisual presentation device disposed at the first location with the virtual sound engineering dashboard in the virtual reality interface.

9 . The system of claim 1 , wherein the mobile device is further configured to establish a two-way communication channel between the first location and the second location with the head-mounted display in the virtual reality interface.

10 . The system of claim 9 , wherein the two-way communication channel comprises text chat or voice communication.

11 . One or more non-transitory, computer-readable storage media comprising a plurality of instructions that in response to being executed cause a mobile device to:

in response to receiving a first access code including authorization information, initiate a first remote access digital mixing session by generating a virtual sound engineering dashboard to remotely access a first digital mixing console disposed at a first location, and wherein the first digital mixing console is communicatively coupled to a first plurality of peripheral devices disposed at the first location;

receive real-time audio and video data captured by at least one of the peripheral devices of the first digital mixing console at the first location in response to initiating the first remote access digital mixing session;

display, with a head-mounted display at a second location different from the first location, the virtual sound engineering dashboard in a virtual reality interface based on the real-time audio and video data; and

control the first digital mixing console with the virtual sound engineering dashboard in the virtual reality interface;

wherein the mobile device is further configured to:

in response to receiving a second access code including authorization information, initiate a second remote access digital mixing session by generating the virtual sound engineering dashboard to remotely access a second digital mixing console disposed at a third location, and wherein the second digital mixing console is communicatively coupled to a second plurality of peripheral devices disposed at the third location; and

display, with the head-mounted display at the second location different from the first location and the third location, the virtual sound engineering dashboard in the virtual reality interface;

wherein the first remote access digital mixing session and the second remote access digital mixing session occur concurrently at the first digital mixing console and the second digital mixing console,

wherein:

the received real-time audio and video data corresponds with real-time audio and video events at the first location which corresponds with the first digital mixing console;

real-time modifications are made to the received real-time audio and video data at the third location which corresponds to the second digital mixing console; and

the real-time modifications made to the received real-time audio and video data at the third location are concurrently broadcast back to the first location as the real-time modifications to the received real-time audio and video data are made at the third location.

12 . The one or more computer-readable storage media of claim 11 , wherein to display the virtual sound engineering dashboard in the virtual reality interface comprises to render a three-dimensional representation of the first digital mixing console.

13 . The one or more computer-readable storage media of claim 11 , wherein to display the virtual sound engineering dashboard in the virtual reality interface comprises to render a three-dimensional representation of the first location based on the real-time audio and video data captured by the at least one of the peripheral devices.

14 . The one or more computer-readable storage media of claim 11 , wherein:

to receive the real-time audio and video data comprises to receive first real-time audio data indicative of an environment of the first location and second real-time audio data indicative of audio output by at least one of the plurality of peripheral devices communicatively coupled to the first digital mixing console; and

to display the virtual sound engineering dashboard in the virtual reality interface comprises to provide output of the first real-time audio data or the second real-time audio data.

15 . The one or more computer-readable storage media of claim 11 , further comprising a plurality of instructions that in response to being executed cause the mobile device to control an audiovisual presentation device disposed at the first location with the virtual sound engineering dashboard in the virtual reality interface.

16 . A method for a virtual sound engineer system, the method comprising:

in response to receiving a first access code including authorization information, by a mobile device, initiating a first remote access digital mixing session by generating a virtual sound engineering dashboard to remotely access a first digital mixing console disposed at a first location, and wherein the first digital mixing console is communicatively coupled to a first plurality of peripheral devices disposed at the first location;

receiving, by the mobile device, real-time audio and video data captured by at least one of the peripheral devices of the first digital mixing console at the first location in response to initiating the first remote access digital mixing session;

displaying, by the mobile device, with a head-mounted display at a second location different from the first location, the virtual sound engineering dashboard in a virtual reality interface based on the real-time audio and video data; and

controlling, by the mobile device, the first digital mixing console with the virtual sound engineering dashboard in the virtual reality interface;

wherein the mobile device is further configured to:

in response to receiving a second access code including authorization information, initiate a second remote access digital mixing session by generating the virtual sound engineering dashboard to remotely access a second digital mixing console disposed at a third location, and wherein the second digital mixing console is communicatively coupled to a second plurality of peripheral devices disposed at the third location; and

display, with the head-mounted display at the second location different from the first location and the third location, the virtual sound engineering dashboard in the virtual reality interface;

wherein the first remote access digital mixing session and the second remote access digital mixing session occur concurrently at the first digital mixing console and the second digital mixing console,

wherein:

the received real-time audio and video data corresponds with real-time audio and video events at the first location which corresponds with the first digital mixing console;

wherein real-time modifications are made to the received real-time audio and video data at the third location which corresponds to the second digital mixing console; and

the real-time modifications made to the received real-time audio and video data at the third location are concurrently broadcast back to the first location as the real-time modifications to the received real-time audio and video data are made at the third location.

17 . The method of claim 16 , wherein displaying the virtual sound engineering dashboard in the virtual reality interface comprises rendering a three-dimensional representation of the first location based on the real-time audio and video data captured by the at least one of the peripheral devices.

18 . The method of claim 16 , wherein:

receiving the real-time audio and video data comprises receiving first real-time audio data indicative of an environment of the first location and second real-time audio data indicative of audio output by at least one of the plurality of peripheral devices communicatively coupled to the first digital mixing console; and

displaying the virtual sound engineering dashboard in the virtual reality interface comprises providing output of the first real-time audio data or the second real-time audio data.

19 . The method of claim 16 , wherein the first digital mixing console comprises a plurality of user controls corresponding to the plurality of peripheral devices communicatively coupled thereto, and wherein displaying the virtual sound engineering dashboard in the virtual reality interface comprises to render a plurality of virtual user controls corresponding to the plurality of user controls.

20 . The method of claim 19 , wherein:

the mobile device is further configured to receive user input indicating a change in position of a virtual user control associated with at least one of the plurality of peripheral devices; and

controlling the first digital mixing console comprises controlling real-time audio output by the at least one of the plurality of peripheral devices in response to receipt of the user input.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 19, 2025
From: VIRTUAL SOUND ENGINEER INC.
To: 302609 LLC
Reel/Frame 070554/0223 →
ASSIGNEE'S ADDRESS HAS CHANGED. Recorded Apr 15, 2024
From: VIRTUAL SOUND ENGINEER, INC.
To: VIRTUAL SOUND ENGINEER, INC.
Reel/Frame 067100/0150 →
ENTITY CONVERSION Recorded Jan 17, 2024
From: VIRTUAL SOUND ENGINEER, LLC
To: VIRTUAL SOUND ENGINEER, INC.
Reel/Frame 066340/0484 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 23, 2023
From: COLEMAN, WALLACE LEVI; HOMER, CHARLES
To: VIRTUAL SOUND ENGINEER, LLC
Reel/Frame 062454/0919 →
Continuity (3)
Continuation In Part 17217906 · Mar 30, 2021
Provisional Application 63064095 · Aug 11, 2020
Related Publication 20230123726A1 · Apr 20, 2023
References Cited (42)
US 6061458A · East et al. · 2000 [cited by applicant]
US 6653545B2 · Redmann · 2003 [cited by applicant]
US 6898637B2 · Curtin · 2005 [cited by applicant]
US 7245727B2 · Cresci et al. · 2007 [cited by applicant]
US 7742609B2 · Yeakel et al. · 2010 [cited by applicant]
US 8028323B2 · Weel · 2011 [cited by applicant]
US 8491386B2 · Reiss et al. · 2013 [cited by applicant]
US 9078091B2 · Lehtiniemi · 2015 [cited by applicant]
US 9686030B2 · Holtman et al. · 2017 [cited by applicant]
US 9933991B2 · Olesh et al. · 2018 [cited by applicant]
US 10263656B2 · Fathollahi et al. · 2019 [cited by applicant]
US 10469965B2 · Heineman · 2019 [cited by applicant]
US 10514885B2 · Leppänen et al. · 2019 [cited by applicant]
US 11079918B2 · Rotta · 2021 [cited by applicant]
US 11432097B2 · Munoz · 2022 [cited by examiner]
US 11528307B2 · Winsor · 2022 [cited by examiner]
US 20020026256A1 · Hilton · 2002 [cited by applicant]
US 20030229900A1 · Reisman · 2003 [cited by examiner]
US 20100303261A1 · Stieler von Heydekampf · 2010 [cited by applicant]
US 20120195445A1 · Inlow · 2012 [cited by applicant]
US 20130262687A1 · Avery et al. · 2013 [cited by applicant]
US 20140006483A1 · Garmark · 2014 [cited by applicant]
US 20140064519A1 · Silfvast · 2014 [cited by examiner]
US 20140328485A1 · Saulters · 2014 [cited by applicant]
US 20150234530A1 · Lengling · 2015 [cited by applicant]
US 20150293649A1 · Sukumar · 2015 [cited by applicant]
US 20160266867A1 · Olesh · 2016 [cited by examiner]
US 20170364321A1 · Chavez et al. · 2017 [cited by applicant]
US 20180054487A1 · Hebsur · 2018 [cited by applicant]
US 20180190250A1 · Hiskey et al. · 2018 [cited by applicant]
US 20180226084A1 · Lin · 2018 [cited by applicant]
US 20180308518A1 · Leonard · 2018 [cited by applicant]
US 20180349088A1 · Leppänen · 2018 [cited by examiner]
US 20190215540A1 · Nicol · 2019 [cited by applicant]
US 20200042284A1 · Milne · 2020 [cited by examiner]
US 20220050659A1 · Coleman et al. · 2022 [cited by applicant]
WO 2022207821A1 · 2022 [cited by applicant]
U.S. Appl. No. 17/217,906, filed Mar. 30, 2021. [cited by applicant]
Korff, “Tablet-controlled Mixers,” accessed on-line at: https://www.soundonsound.com/reviews/tablet-controlled-mixers, (Jun. 11, 2020). [cited by applicant]
Mixoline, “iOS Applications for Console Control,” accessed on-line at: https://www.mixonline.com/technology/ios-applications-console-control-423069 (Nov. 24, 2014). [cited by applicant]
Sounddevices, “Requirements for Remote Control of 6-Series,” accessed on-line at: https://www.sounddevices.com/6-series-remote-ctrl-requirements/ (Sep. 13, 2019). [cited by applicant]
International Search & Written Opinion dated Mar. 13, 2024—PCT Application No. PCT/US/2023/81568. [cited by applicant]