IP Library › Granted Patent US 12682584
Granted Patent B2
US 12682584 · App. 18/658,554 · Granted Jul 14, 2026

Augmented reality device operation with robotic total station

Inventors: Mark Nichols (Christchurch, NZ); Thomas Peterson (Christchurch, NZ); Adam Bursill (Christchurch, NZ)
Assignee: Trimble Inc.
G06T19/006G06T5/50G06T19/20G06T2207/20221G06T2219/2004G06T2219/2016
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12682584
App. No.
18/658,554
Granted
Jul 14, 2026
Kind
B2
Abstract

Techniques for operating an augmented reality (AR) device using a robotic total station (RTS) are described. An RTS reference frame is correlated with an AR reference frame. A set of RTS measurements are received from the RTS including a distance and an angle from the RTS to a reflector of the AR device. An RTS point within the RTS reference frame is determined based on the distance and the angle. An AR point is determined within the AR reference frame based on a set of vision-based or inertia-based measurements. The RTS reference frame and/or the AR reference frame is shifted to align the RTS point with the AR point. The RTS reference frame and/or the AR reference frame is rotated to align an RTS vector formed using the RTS point with an AR vector formed using the AR point.

Claims (66)

1 . A method of operating an augmented reality (AR) device using a robotic total station (RTS), the method comprising:

correlating an RTS reference frame with an AR reference frame, the RTS reference frame for tracking positions of the AR device with RTS measurements and the AR reference frame for tracking positions of the AR device with vision-based or inertia-based measurements;

receiving a set of RTS measurements from the RTS including a distance and an angle from the RTS to a reflector of the AR device;

determining an RTS point within the RTS reference frame based on the distance and the angle;

determining an AR point within the AR reference frame based on a set of vision-based or inertia-based measurements;

shifting the RTS reference frame and/or the AR reference frame to align the RTS point with the AR point;

rotating the RTS reference frame and/or the AR reference frame to align an RTS vector formed using the RTS point with an AR vector formed using the AR point;

determining whether the RTS is tracking the reflector of the AR device at a first time;

in response to determining that the RTS is not tracking the reflector of the AR device at the first time:

determining an untracked AR point within the AR reference frame; and

sending an instruction to the RTS to cause the RTS to rotate toward the untracked AR point.

2 . The method of claim 1 , wherein the RTS vector is formed between the RTS point and a previous RTS point, and wherein the AR vector is formed between the AR point and a previous AR point.

3 . The method of claim 1 , further comprising:

generating a model image based on model data, the shifting of the RTS reference frame and/or the AR reference frame, and the rotating of the RTS reference frame and/or the AR reference frame; and

generating a superimposed image by superimposing the model image onto a camera image generated by a camera; and

displaying the superimposed image.

4 . The method of claim 1 , further comprising:

determining whether the RTS is tracking the reflector of the AR device at a second time after the first time; and

in response to determining that the RTS is tracking the reflector of the AR device at the second time, determining the RTS point and the AR point.

5 . The method of claim 1 , wherein the AR device comprises a camera component including a camera and a target component including the reflector, and wherein the camera component is separable from and configured to removably attach to the target component.

6 . The method of claim 1 , further comprising:

calculating an angle formed by the RTS vector and the AR vector, wherein the RTS reference frame and/or the AR reference frame is rotated by the angle.

7 . A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations for operating an augmented reality (AR) device using a robotic total station (RTS), the operations comprising:

correlating an RTS reference frame with an AR reference frame, the RTS reference frame for tracking positions of the AR device with RTS measurements and the AR reference frame for tracking positions of the AR device with vision-based or inertia-based measurements;

receiving a set of RTS measurements from the RTS including a distance and an angle from the RTS to a reflector of the AR device;

determining an RTS point within the RTS reference frame based on the distance and the angle;

determining an AR point within the AR reference frame based on a set of vision-based or inertia-based measurements;

shifting the RTS reference frame and/or the AR reference frame to align the RTS point with the AR point;

rotating the RTS reference frame and/or the AR reference frame to align an RTS vector formed using the RTS point with an AR vector formed using the AR point;

determining whether the RTS is tracking the reflector of the AR device at a first time;

in response to determining that the RTS is not tracking the reflector of the AR device at the first time:

determining an untracked AR point within the AR reference frame; and

sending an instruction to the RTS to cause the RTS to rotate toward the untracked AR point.

8 . The non-transitory computer-readable medium of claim 7 , wherein the RTS vector is formed between the RTS point and a previous RTS point, and wherein the AR vector is formed between the AR point and a previous AR point.

9 . The non-transitory computer-readable medium of claim 7 , wherein the operations further comprise:

generating a model image based on model data, the shifting of the RTS reference frame and/or the AR reference frame, and the rotating of the RTS reference frame and/or the AR reference frame; and

generating a superimposed image by superimposing the model image onto a camera image generated by a camera; and

displaying the superimposed image.

10 . The non-transitory computer-readable medium of claim 7 , wherein the operations further comprise:

determining whether the RTS is tracking the reflector of the AR device at a second time after the first time; and

in response to determining that the RTS is tracking the reflector of the AR device at the second time, determining the RTS point and the AR point.

11 . The non-transitory computer-readable medium of claim 7 , wherein the AR device comprises a camera component including a camera and a target component including the reflector, and wherein the camera component is separable from and configured to removably attach to the target component.

12 . The non-transitory computer-readable medium of claim 7 , wherein the operations further comprise:

calculating an angle formed by the RTS vector and the AR vector, wherein the RTS reference frame and/or the AR reference frame is rotated by the angle.

13 . A system comprising:

one or more processors; and

a non-transitory computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for operating an augmented reality (AR) device using a robotic total station (RTS), the operations comprising:

correlating an RTS reference frame with an AR reference frame, the RTS reference frame for tracking positions of the AR device with RTS measurements and the AR reference frame for tracking positions of the AR device with vision-based or inertia-based measurements;

receiving a set of RTS measurements from the RTS including a distance and an angle from the RTS to a reflector of the AR device;

determining an RTS point within the RTS reference frame based on the distance and the angle;

determining an AR point within the AR reference frame based on a set of vision-based or inertia-based measurements;

shifting the RTS reference frame and/or the AR reference frame to align the RTS point with the AR point; and

rotating the RTS reference frame and/or the AR reference frame to align an RTS vector formed using the RTS point with an AR vector formed using the AR point;

determining whether the RTS is tracking the reflector of the AR device at a first time;

in response to determining that the RTS is not tracking the reflector of the AR device at the first time:

determining an untracked AR point within the AR reference frame; and

sending an instruction to the RTS to cause the RTS to rotate toward the untracked AR point.

14 . The system of claim 13 , wherein the RTS vector is formed between the RTS point and a previous RTS point, and wherein the AR vector is formed between the AR point and a previous AR point.

15 . The system of claim 13 , wherein the operations further comprise:

generating a model image based on model data, the shifting of the RTS reference frame and/or the AR reference frame, and the rotating of the RTS reference frame and/or the AR reference frame; and

generating a superimposed image by superimposing the model image onto a camera image generated by a camera; and

displaying the superimposed image.

16 . The system of claim 13 , wherein the operations further comprise:

determining whether the RTS is tracking the reflector of the AR device at a second time after the first time; and

in response to determining that the RTS is tracking the reflector of the AR device at the second time, determining the RTS point and the AR point.

17 . The system of claim 13 , wherein the AR device comprises a camera component including a camera and a target component including the reflector, and wherein the camera component is separable from and configured to removably attach to the target component.