Granted Patent B2

US 11,669,155 · App. 17/517,976 · Granted Jun 6, 2023

Inventors: Chase Bowman (Media, PA); Seth Johnson (Pasadena, CA)

Assignee: Light Wand LLC

G06F3/011G06F3/017G06F3/165G06T19/006G02B27/0172G02B2027/014

Abstract

Claims (67)

1. A system for controlling the operation of a secondary device in a mixed reality environment, the system comprising:

a server comprising at least one processor programmed to:

receive user positional data from a mixed reality user device;

determine gestural data based on the user positional data;

generate a virtual object corresponding to the generated gestural data;

generate secondary device settings for a secondary device communicatively coupled to the server via a network;

cause the mixed reality user device to display the virtual object; and

transmit the secondary device settings to the secondary device to effectuate a change in a configuration of the secondary device;

wherein the secondary device comprises a motorized dolly and the secondary device settings comprises a location of the motorized dolly in space.

2. The system of claim 1 , wherein the at least one processor is programmed to:

receive second user positional data from the mixed reality user device, wherein the second user positional data indicates a user requested adjustment to the virtual object;

determine adjustment gestural data corresponding to the received second user positional data;

regenerate the virtual object to correspond to the adjustment gestural data;

transmit the regenerated virtual object to the mixed reality user device for display;

regenerate secondary device settings for the secondary device based on the adjustment gestural data; and

update the secondary device based on the regenerated secondary device settings.

3. The system of claim 1 , wherein the at least one processor is programmed to:

generate one or more pre-configured virtual objects for display by the mixed reality user device, wherein each of the pre-configured virtual objects corresponds to a parameter of the secondary device; and

update the secondary device based on the parameter corresponding to the pre-configured virtual object responsive to the virtual object interacting with the one or more pre-configured virtual objects in a mixed reality environment generated by the mixed reality user device.

4. A method for controlling the operation of a secondary device in a mixed reality environment, the method comprising:

receiving user positional data from a mixed reality user device;

generating, at a server communicatively coupled to the mixed reality user device via a network, gestural data based on the user positional data;

generating a virtual object corresponding to the generated gestural data, wherein the virtual object is configured for display in the mixed reality environment;

generating secondary device settings for a secondary device communicatively coupled to the server via the network;

transmitting the generated virtual object to the mixed reality user device; and

transmitting the secondary device settings to the secondary device to effectuate a change in the settings of the secondary device;

wherein the secondary device comprises a motorized dolly and the secondary device settings comprises a location of the motorized dolly in space.

5. The method of claim 4 , further comprising:

receiving second user positional data from the mixed reality user device, wherein the second user positional data indicates a user requested adjustment to the virtual object;

determining adjustment gestural data corresponding to the received second user positional data;

regenerating the virtual object to correspond to the adjustment gestural data;

transmitting the regenerated virtual object to the mixed reality user device for display;

regenerating secondary device settings for the secondary device based on the adjustment gestural data; and

updating the secondary device based on the regenerated secondary device settings.

6. The method of claim 4 further comprising:

generating a confirmation virtual object for display by the mixed reality user device responsive to the secondary device settings being applied to the secondary device.

7. The method of claim 4 further comprising:

generating one or more pre-configured virtual objects for display by the mixed reality user device, wherein each of the pre-configured virtual objects corresponds to a parameter of the secondary device; and

updating the secondary device based on the parameter corresponding to the pre-configured virtual object responsive to the virtual object interacting with the one or more pre-configured virtual objects in a mixed reality environment generated by the mixed reality user device.

8. A system comprising:

a server comprising at least one processor programmed to:

receive user positional data from a mixed reality user device;

determine gestural data based on the user positional data;

generate a virtual object corresponding to the generated gestural data;

generate secondary device settings for a secondary device communicatively coupled to the server via a network;

cause the mixed reality user device to display the virtual object; and

transmit the secondary device settings to the secondary device to effectuate a change in a configuration of the secondary device; and

a mixed reality user device comprising:

a sensor configured to determine user positional data; and

a display configured to generate a mixed reality environment including the virtual object; and

the secondary device configured to receive secondary device settings from the server and update a configuration of the secondary device;

wherein the secondary device comprises a motorized dolly and the secondary device settings comprises a location of the motorized dolly in space.

9. The system of claim 8 , wherein the server is configured to:

receive second user positional data from the mixed reality user device, wherein the second user positional data indicates a user requested adjustment to the virtual object;

determine adjustment gestural data corresponding to the received second user positional data;

regenerate the virtual object to correspond to the adjustment gestural data;

transmit the regenerated virtual object to the mixed reality user device for display;

regenerate secondary device settings for the secondary device based on the adjustment gestural data; and

update the secondary device based on the regenerated secondary device settings.

10. The system of claim 8 , wherein the server is configured to:

generate one or more pre-configured virtual objects for display by the mixed reality user device, wherein each of the pre-configured virtual objects corresponds to a parameter of the secondary device; and

update the secondary device based on the parameter corresponding to the pre-configured virtual object responsive to the virtual object interacting with the one or more pre-configured virtual objects in a mixed reality environment generated by the mixed reality user device.

11. The system of claim 8 , wherein the server is configured to:

generate a confirmation virtual object for display by the mixed reality user device responsive to the secondary device settings being applied to the secondary device.

12. The system of claim 8 wherein the server is configured to update the secondary device via transmitting at least one of an open sound control (OSC), digital multiplex (DMX), remote device management (RDM), or musical instrument digital interface (MIDI) file to the secondary device.

13. The system of claim 8 , wherein the mixed reality environment comprises at least one of an image from the real-world environment of a user of the mixed reality device, a virtual representation of real-world objects from the real-world environment of the user and virtual objects generated by the server.

14. The system of claim 8 , wherein the mixed reality device comprises a head-mounted display device.

Assignments (1)

Continuity (2)

Cited By (37)

US 12,236,515 US 12,236,546 US 12,242,706 US 12,283,020 US 12,287,921 US 12,288,285 US 12,307,580 US 12,333,083 US 12,437,494 US 12,443,273 US 12,456,271 US 12,461,641 US 12,475,635 US 12,511,847 US 12,524,142 US 12,524,956 US 12,524,977 US 12,535,931 US 12,536,762 US 12,541,280 US 12,566,532 US 12,596,472 US 12,608,115 US 12,608,877 US 12,608,890 US 12,620,187 US 12,625,608 US 12,632,157 US 12,632,170 US 12,633,044 US 12,663,907 US 12,675,958 US 12,681,587 US 12,687,922 US 12,688,660 US 12,737,093 US 12,749,272