IP Library › Granted Patent US 12711713
Granted Patent B2
US 12711713 · App. 18/529,924 · Granted Aug 18, 2026

Reactive augmented reality

Inventor: Stephen Wilkes (Westport, CT)
G06T19/006G06T15/205G06T15/50G06T17/05G06T2215/12G06T2215/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12711713
App. No.
18/529,924
Granted
Aug 18, 2026
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating composite images. One of the methods includes maintaining first data associating each location within an environment with a particular time; obtaining an image depicting the environment from a point of view of a display device; obtaining second data characterizing one or more virtual objects; and processing the obtained image and the second data to generate a composite image depicting the one or more virtual objects at respective locations in the environment from the point of view of the display device, wherein the composite image depicts each virtual object according to the particular time that the first data associates with the location of the virtual object in the environment.

Claims (48)

1 . A computer-implemented method comprising:

obtaining an image depicting a physical environment from a point of view of a display device;

obtaining data characterizing one or more virtual objects;

generating, using the image and the data, a composite image depicting the one or more virtual objects at respective locations in the composite image for display from the point of view of the display device;

determining a common coordinate system in the physical environment for the composite image, the common coordinate system having a first dimension and a second dimension;

determining whether a change in a position associated with the point of view of the display device along the first dimension or the second dimension satisfies a first threshold;

in response to determining that the change in position associated with the point of view of the display device along the first dimension satisfies the first threshold, modifying the one or more virtual objects with a first modification; and

in response to determining that the change in position associated with the point of view of the display device along the second dimension satisfies a second threshold, modifying the composite image with a second modification, wherein the first modification and the second modification are different.

2 . The method of claim 1 , wherein modifying the composite image with the first modification or modifying the composite image with the second modification comprises changing a time of day in which one or more virtual objects are represented in the composite image.

3 . The method of claim 1 , wherein the first modification or the second modification are a time of day or weather condition.

4 . The method of claim 1 , wherein the first modification or the second modification is an animation or a sound that is directional and originating from a particular direction.

5 . The method of claim 1 , wherein the first modification or the second modification is a selection of one or more modifications.

6 . The method of claim 1 , comprising:

modifying a first or second virtual object in response to the respective virtual object changing position relative to the first dimension or the second dimension.

7 . The method of claim 1 , wherein the first modification causes a first virtual object to react to the position associated with the point of view of the display device.

8 . The method of claim 1 , wherein the composite image comprises a representation of the physical environment at a time other than a current time.

9 . The method of claim 1 , wherein the composite image is scalable to the physical environment using the common coordinate system.

10 . The method of claim 1 comprising:

determining a distance between the position associated with the point of view of the display device and a first physical object; and

modifying the composite image with an additional virtual object responsive to the distance between the position associated with the point of view of the display device and the first physical object.

11 . The method of claim 1 , comprising:

modifying the composite image to depict objects in the composite image according to a light source along the first dimension or the second dimension in the composite image.

12 . The method of claim 1 , wherein generating, using the image and the data, the composite image depicting the one or more virtual objects at respective locations in the composite image for display from the point of view of the display device comprises:

generating for use in the composite image a spatial boundary indicating a trigger point for a change in depiction of the composite image.

13 . The method of claim 1 , wherein generating, using the image and the data, the composite image depicting the one or more virtual objects at respective locations in the composite image for display from the point of view of the display device comprises:

generating for use in the composite image a visual indication indicating a trigger point for a change in depiction of the composite image.

14 . The method of claim 1 , wherein the first modification or the second modification are a time of year.

15 . One or more non-transitory computer storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

obtaining an image depicting a physical environment from a point of view of a display device;

obtaining data characterizing one or more virtual objects;

generating, using the image and the data, a composite image depicting the one or more virtual objects at respective locations in the composite image for display from the point of view of the display device;

determining a common coordinate system in the physical environment for the composite image, the common coordinate system having a first dimension and a second dimension;

determining whether a change in a position associated with the point of view of the display device along the first dimension or the second dimension satisfies a first threshold;

in response to determining that the change in position associated with the point of view of the display device along the first dimension satisfies the first threshold, modifying the one or more virtual objects with a first modification; and

in response to determining that the change in position associated with the point of view of the display device along the second dimension satisfies a second threshold, modifying the composite image with a second modification, wherein the first modification and the second modification are different.

16 . A system comprising one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

obtaining an image depicting a physical environment from a point of view of a display device;

obtaining data characterizing one or more virtual objects;

generating, using the image and the data, a composite image depicting the one or more virtual objects at respective locations in the composite image for display from the point of view of the display device;

determining a common coordinate system in the physical environment for the composite image, the common coordinate system having a first dimension and a second dimension;

determining whether a change in a position of the point of view of the display device along the first dimension or the second dimension satisfies a first threshold;

in response to determining that the change in position associated with the point of view of the display device along the first dimension satisfies the first threshold, modifying the one or more virtual objects with a first modification; and

in response to determining that the change in position associated with the point of view of the display device along the second dimension satisfies a second threshold, modifying the composite image with a second modification, wherein the first modification and the second modification are different.

17 . The system of claim 16 , wherein:

the first dimension along which depictions of the one or more virtual objects change is defined by a light source within the composite image.

18 . The system of claim 16 , wherein the first modification causes a first virtual object to react to the position associated with the point of view of the display device.

19 . The system of claim 16 , wherein the composite image comprises a representation of the physical environment at a time other than a current time.

20 . The system of claim 16 , wherein the composite image is scalable to the physical environment using the common coordinate system.