IP Library Granted Patent US 11,467,656
Granted Patent B2
US 11,467,656 · App. 16/806,162 · Granted Oct 11, 2022

Virtual object control of a physical device and/or physical device control of a virtual object

Inventor: Nova Spivack (Redmond, WA)
Assignee: Magical Technologies, LLC
G06F3/011G06F3/017G06F21/32G06T19/006G06T19/20G06V20/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,467,656
App. No.
16/806,162
Filed
Mar 2, 2020
Granted
Oct 11, 2022
Kind
B2
Art Unit
2674
USPC
345/633
Abstract

Systems, methods and apparatuses of virtual object control of a physical device and/or physical device control of a virtual object are disclosed. In one aspect, embodiments of the present disclosure include a method, which may be implemented on a system, to present a depiction of a virtual object in the augmented reality environment. In the method, one or more functions of the virtual object accessible in the augmented reality environment are used for control of a physical device.

Claims (77)

1. A method of an augmented reality environment, the method, comprising:

presenting a depiction of a virtual object in the augmented reality environment, the depiction of the virtual object being observable in the augmented reality environment;

wherein, one or more functions of the virtual object accessible in the augmented reality environment are used for control of a physical device;

responsive to detecting action performed on the physical device, identifying a function of the one or more functions of the virtual object corresponding to the action performed on the physical device;

updating the depiction of the virtual object in the augmented reality environment based on the function of the one or more functions of the virtual object that has been identified;

wherein,

the function of the one or more functions is accessible by a human user via the depiction of the virtual object, for the human user to interact with the physical device.

2. The method of claim 1 , further comprising:

identifying the physical device in a physical location that is controlled by the virtual object;

ascertaining functions of the physical device;

rendering the one or more functions of the virtual object accessible for use in control of the physical device, based on the functions of the physical device.

3. The method of claim 1 , wherein:

the virtual object is used to adjust a setting of the physical device to a value amongst a set of values.

4. The method of claim 1 , wherein:

the virtual object is depicted in the augmented reality environment in response to the action performed on the physical device.

5. The method of claim 1 , wherein:

an instance of the virtual object is launched in the augmented reality environment in response to the action detected by the physical device;

further wherein, the physical device is also used to access the one or more functions of the virtual object.

6. The method of claim 1 , wherein

the virtual object is accessed to control the physical device through one or more of, mechanical actuation, electrical actuation, magnetic actuation, acoustic actuation, biometric and biochemical actuation.

7. A method of an augmented reality environment in a real world environment, the method, comprising:

causing to be perceptible or imperceptible, to a human user, a virtual object in the augmented reality environment;

wherein, the virtual object is associated with a physical device in a physical location;

wherein, the augmented reality environment includes:

the real world environment or a representation of the real world environment associated with the physical location in which the physical device is physically located;

the virtual object;

wherein, the virtual object is usable to control the physical device;

further wherein, the virtual object is able to control multiple physical devices in the physical location;

further wherein, a function of the virtual object in the augmented reality environment is governed by a function or an attribute of the physical device;

further wherein, a rendering of the virtual object in the augmented reality environment is determined based on the function or the attribute of the physical device which the virtual object controls.

8. The method of claim 7 , wherein:

the physical device associated with the virtual object, is used to control the virtual object.

9. The method of claim 7 , wherein:

the virtual object includes a biometric sensor to authenticate the human user to access the physical device.

10. The method of claim 7 , wherein:

the virtual object is usable to control multiple functions or attributes of the physical device;

the virtual object is depicted as an instrument panel having multiple virtual control objects in the augmented reality environment;

wherein, the multiple virtual control objects are operable to access or control the multiple functions or attributes of the physical device.

11. The method of claim 7 , wherein:

the virtual object is adjustable to control another physical device different from the physical device in the physical location;

further wherein, a representation of the virtual object is updated in the augmented reality environment, to depict a change in its control from the physical device to the other physical device.

12. The method of claim 7 , wherein:

the virtual object is depicted as an instrument panel having multiple virtual control objects in the augmented reality environment;

wherein, the multiple virtual control objects are operable to access or control the multiple physical devices in the physical location.

13. A system to facilitate control of a physical object of a real world environment by a virtual object in an augmented reality environment, the system, comprising:

a processor;

memory having stored thereon instructions, which when executed by a processor, cause the processor to;

render perceptible or imperceptible, to a human user, a virtual object in the augmented reality environment,

wherein, the virtual object is associated with a physical device in a physical location;

wherein, the augmented reality environment includes:

the real world environment or a representation of the real world environment associated with the physical location in which the physical device is physically located;

the virtual object;

wherein, the virtual object is made perceptible or imperceptible to the human user in the augmented reality environment depending on a rule set;

wherein, the rule set includes a proximity parameter;

wherein, the proximity parameter includes an indication of a physical distance between the human user and the physical device;

wherein, the augmented reality environment is rendered at least in part in a hologram, wherein the hologram is accessible in 3D and in 360 degrees;

wherein, perceptibility of the virtual object and perceptibility of the representation of the real environment is configurable or adjustable by the human user.

14. A system to facilitate control of a physical object by a virtual object in an augmented reality environment, the system, comprising:

a processor;

memory having stored thereon instructions, which when executed by a processor, cause the processor to:

present a depiction of a virtual object in the augmented reality environment, the depiction of the virtual object being observable in the augmented reality environment;

wherein, one or more functions of the virtual object accessible in the augmented reality environment are used for control of the physical object;

identify the physical object in a physical location that is controlled by the virtual object;

wherein, perceptibility of the virtual object and perceptibility of a representation of the physical location in the augmented reality environment is configurable or adjustable.

15. The system of claim 14 , wherein, the processor is further operable to:

determine functions of the physical object;

render the one or more functions of the virtual object accessible for use in control of the physical object, based on the functions of the physical object.

16. The system of claim 14 ,

wherein, the physical object is also used to access the one or more functions of the virtual object;

wherein, the processor is further operable to:

detect a gesture performed in relation to the physical object;

identify a function of the one or more functions of the virtual object corresponding to the gesture performed;

update the depiction of the virtual object in the augmented reality environment, based on the function of the one or more functions of the virtual object that has been identified.

17. The system of claim 14 , further comprising,

a sensor coupled to the processor;

wherein, the sensor detects actuation of the physical device to access the one or more functions of the virtual object.

18. The system of claim 14 , wherein, the sensor includes one or more of, a heat sensor, a proximity sensor, a motion sensor, a pressure sensor, a light sensor, a temperature, and an acoustic sensor.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 13, 2026
From: ZAMBALA LLC
To: ADEIA MEDIA HOLDINGS INC.
Reel/Frame 075639/0144 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2026
From: MAGICAL TECHNOLOGIES, LLC
To: ZAMBALA LLC
Reel/Frame 074887/0861 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 14, 2021
From: SPIVACK, NOVA
To: MAGICAL TECHNOLOGIES, LLC
Reel/Frame 057790/0813 →
Continuity (2)
Provisional Application 62813216 · Mar 4, 2019
Related Publication 20210026441A1 · Jan 28, 2021
Cited By (8)
US 12,249,034 US 12,277,654 US 12,548,245 US 12,572,253 US 12,613,938 US 12,633,068 US 12,670,001 US 12,681,561