IP Library › Granted Patent US 11,609,345
Granted Patent B2
US 11,609,345 · App. 16/796,641 · Granted Mar 21, 2023

System and method to determine positioning in a virtual coordinate system

Inventors: Thong T. Nguyen (New Berlin, WI); Paul D. Schmirler (Glendale, WI); Alex L. Nicoll (Brookfield, WI)
Assignee: Rockwell Automation Technologies, Inc.
G01S19/51G01S19/396G06F16/248G06F16/29
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,609,345
App. No.
16/796,641
Granted
Mar 21, 2023
Kind
B2
Abstract

A system includes a computing system configured to communicatively couple to a database configured to store a virtual coordinate system and a plurality of features associated with a representative environment associated with the virtual coordinate system. The computing system is configured to receive a first input indicative of a physical positioning of a user in a physical environment, determine a virtual positioning of the user in the virtual coordinate system based on the first input, receive a second input indicative of an updated physical positioning of the user in the physical environment, determine an updated virtual positioning of the user in the virtual coordinate system based on the second input, and output a first signal to a computing device in response to determining the updated virtual positioning of the user in the virtual coordinate system.

Claims (60)

1. A system, comprising:

a physical marker in a physical environment, wherein the physical marker is associated with a virtual location in a virtual coordinate system stored in a database, a physical direction in the physical environment, and a virtual direction in the virtual coordinate system, and the physical marker is configured to receive an initial input from a computing device at an initial orientation between the computing device and the physical marker;

a computing system configured to communicatively couple to the database, the database being configured to store a plurality of features associated with a representative environment associated with the virtual coordinate system, wherein the computing system is configured to:

receive a first input indicative of a physical positioning of a user in the physical environment;

determine a first distance between the physical positioning of the user and the physical marker in the physical environment;

establish a first orientation of the physical positioning of the user relative to the physical direction of the physical marker in the physical environment as the initial orientation between the computing device and the physical marker in response to the physical marker receiving the initial input;

map the physical positioning of the user to a virtual positioning of the user in the virtual coordinate system based on:

a second distance between the virtual positioning of the user and the virtual location of the physical marker in the virtual coordinate system corresponding to the first distance; and

a second orientation of the virtual positioning of the user relative to the virtual direction of the physical marker in the virtual coordinate system corresponding to the first orientation;

receive a second input indicative of an updated physical positioning of the user in the physical environment;

determine an updated virtual positioning of the user in the virtual coordinate system based on the second input; and

output a first signal to the computing device based on the virtual positioning, the updated virtual positioning of the user in the virtual coordinate system, or both.

2. The system of claim 1 , wherein the first signal is configured to cause the computing device to present a feature of the plurality of features associated with the representative environment based on the virtual positioning of the user in the virtual coordinate system via an electronic display.

3. The system of claim 2 , wherein the computing system is configured to output a second signal to the computing device in response to determining the updated virtual positioning of the user in the virtual coordinate system, and wherein the second signal is configured to cause the computing device to modify a presentation of the feature of the plurality of features associated with the representative environment, present an additional feature of the plurality of features associated with the representative environment, or both.

4. The system of claim 1 , wherein the computing system is configured to receive sensor data from the computing device, wherein the sensor data is indicative of movement of the user relative to the physical positioning of the user in the physical environment, wherein the second input comprises the sensor data, and wherein the computing system is configured to determine the updated physical positioning based on the movement of the user relative to the physical positioning of the user as indicated by the sensor data.

5. The system of claim 4 , wherein the computing system is configured to:

determine corresponding movement of the user relative to the virtual positioning of the user in the virtual coordinate system based on the movement of the user relative to the physical positioning of the user in the physical environment; and

determine the updated virtual positioning of the user in the virtual coordinate system based on the corresponding movement of the user relative to the virtual positioning of the user.

6. The system of claim 1 , wherein the plurality of features comprises an image, an audio output, haptic feedback, or any combination thereof, associated with the representative environment.

7. The system of claim 1 , wherein the representative environment is associated with a virtual environment, the physical environment, a mixed environment, an additional physical environment, or any combination thereof.

8. The system of claim 1 , wherein the first orientation comprises a first angle between the physical direction of the physical marker and an additional physical direction of the physical positioning of the user, the second orientation comprises a second angle between the virtual direction of the physical marker and an additional virtual direction of the virtual positioning of the user, and the first angle and the second angle correspond to one another.

9. The system of claim 1 , wherein the physical marker comprises a sensor configured to receive the initial input from the computing device.

10. A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by a processor, are configured to cause the processor to:

receive a first input indicative of a first physical positioning of a first user in a first physical environment;

determine a first distance between the first physical positioning of the first user and a physical marker in the first physical environment, wherein the physical marker is associated with a virtual location in a virtual coordinate system, a physical direction in the first physical environment, and a virtual direction in the virtual coordinate system;

determine a first orientation of the first physical positioning of the first user relative to the physical direction of the physical marker in the first physical environment, wherein the first orientation comprises a first angle between the physical direction of the physical marker and an additional physical direction of the first physical positioning of the first user;

map the first physical positioning of the first user to a first virtual positioning of the first user in the virtual coordinate system based on:

a second distance between the first virtual positioning of the first user and the virtual location of the physical marker in the virtual coordinate system corresponding to the first distance; and

a second orientation of the first virtual positioning of the first user relative to the virtual direction of the physical marker in the virtual coordinate system corresponding to the first orientation, wherein the second orientation comprises a second angle between the virtual direction of the physical marker and an additional virtual direction of the virtual positioning of the first user, and the first angle and the second angle correspond to one another;

receive a second input indicative of a second physical positioning of the first user in the first physical environment;

determine a second virtual positioning of the first user in the virtual coordinate system based on the second input, wherein the second virtual positioning corresponds to the second physical positioning; and

output a first signal to a first computing device based on the second virtual positioning of the first user in the virtual coordinate system.

11. The non-transitory computer-readable medium of claim 10 , wherein the computer-executable instructions, when executed by the processor, are configured to cause the processor to:

receive a third input indicative of a third physical positioning of a second user in a second physical environment; and

determine a third virtual positioning of the second user in the virtual coordinate system based on the third input, wherein the third virtual positioning corresponds to the third physical positioning.

12. The non-transitory computer-readable medium of claim 11 , wherein the computer-executable instructions, when executed by the processor, are configured to cause the processor to output the first signal configured to cause the first computing device to display a first image of the second user to the first user based on the third virtual positioning of the second user relative to the second virtual positioning of the first user in the virtual coordinate system, output a second signal configured to cause a second computing device to display a second image of the first user to the second user based on the second virtual positioning of the first user relative to the third virtual positioning of the second user in the virtual coordinate system, or both.

13. The non-transitory computer-readable medium of claim 11 , wherein the first signal is configured to cause the first computing device to present a first feature associated with a first representative environment associated with the virtual coordinate system via an electronic display, and wherein the computer-executable instructions, when executed by the processor, are configured to cause the processor to output a second signal to a second computing device associated with the second user, wherein the second signal is configured to cause the second computing device to present a second feature associated with a second representative environment associated with the virtual coordinate system via an additional electronic display.

14. The non-transitory computer-readable medium of claim 10 , wherein the computer-executable instructions, when executed by the processor, are configured to cause the processor to receive a third input indicative of a third virtual positioning of a feature in the virtual coordinate system, and wherein the first signal is configured to cause the first computing device of the first user to present the feature to the first user based on the second virtual positioning of the first user relative to the third virtual positioning of the feature in the virtual coordinate system.

15. The non-transitory computer-readable medium of claim 14 , wherein the computer-executable instructions, when executed by the processor, are configured to cause the processor to:

receive a fourth input indicative of movement of the feature from the third virtual positioning to a fourth virtual positioning in the virtual coordinate system; and

transmit a third signal configured to cause the first computing device of the first user to modify a presentation of the feature based on the fourth virtual positioning of the feature in the virtual coordinate system.

16. The non-transitory computer-readable medium of claim 15 , wherein the fourth input is received from a second computing device of a second user.

17. The non-transitory computer-readable medium of claim 10 , wherein the computer-executable instructions, when executed by the processor, are configured to cause the processor to:

determine a representative positioning of the first user in a representative environment based on the second virtual positioning of the first user in the virtual coordinate system, wherein the representative environment is associated with the virtual coordinate system; and

output a second signal to instruct a mobile device of a second user to present an image of the first user in the representative environment based on the representative positioning of the first user in the representative environment.

18. A method comprising:

receiving, via a processor, a first input indicative of a physical positioning of a user in a physical environment;

determining, via the processor, receipt of a second input at a device in the physical environment, wherein the device is associated with a virtual location in a virtual coordinate system, a physical direction in the physical environment, and a virtual direction in the virtual coordinate system, and the device is configured to receive the second input from a computing device at an initial orientation between the device and the computing device;

determining, via the processor, a first distance between the physical positioning of the user and the device in the physical environment;

setting, via the processor, a first orientation of the physical positioning of the user relative to the physical direction of the device in the physical environment as the initial orientation in response to the device receiving the second input;

mapping, via the processor, the physical positioning of the user to a virtual positioning of the user in the virtual coordinate system based on:

a second distance between the virtual positioning of the user and the virtual location of the device in the virtual coordinate system corresponding to the first distance; and

a second orientation of the virtual positioning of the user relative to the virtual direction of the device in the virtual coordinate system corresponding to the first orientation;

determining, via the processor, a representative positioning of the user in a representative environment based on the virtual positioning of the user in the virtual coordinate system, wherein the representative environment is associated with the virtual coordinate system; and

outputting, via the processor, a signal configured to cause a computing device to present a feature associated with the representative environment via an electronic display based on the virtual positioning of the user in the virtual coordinate system.

19. The method of claim 18 , comprising:

receiving, via the processor, an additional input indicative of an additional physical positioning of the user in the physical environment;

determining, via the processor, a first relationship between the additional physical positioning and the physical positioning; and

determining, via the processor, an additional virtual positioning of the user relative to the virtual positioning of the user in the virtual coordinate system based on a second relationship between the virtual positioning and the additional virtual positioning corresponding to the first relationship between the additional physical positioning and the physical positioning.

20. The method of claim 18 , wherein the device comprises a scanner.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 20, 2020
From: NGUYEN, THONG T.; SCHMIRLER, PAUL D.; NICOLL, ALEX L.
To: ROCKWELL AUTOMATION TECHNOLOGIES, INC.
Reel/Frame 051880/0407 →
Continuity (1)
Related Publication 20210263168A1 · Aug 26, 2021