IP Library Granted Patent US 11,176,746
Granted Patent B2
US 11,176,746 · App. 16/679,005 · Granted Nov 16, 2021

Modulated display AR tracking systems and methods

Inventors: Michael Romaszewicz (Brooklyn, NY); Andrew Plourde (New York, NY)
Assignee: MSG Entertainment Group, LLC
G06T19/006G02B27/0101G06K9/00671G06K19/06037G02B2027/014
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,176,746
App. No.
16/679,005
Granted
Nov 16, 2021
Kind
B2
Abstract

Modulated display AR tracking systems and methods include a display configured to transmit first image content, the image content comprising a first image content frame and a second image content frame and a device comprising an image sensor, a processor and a memory. The device identifies the first image content frame and the second image content frame by distinguishing at least one feature of the first image content frame. The device recognizes at least one tracking pattern embedded of the second image content frame and retrieves a graphical element based on the at least one tracking pattern of the second image content frame. The device displays augmented image content including the first image content frame and the graphical element.

Claims (74)

1. A system, comprising:

a display configured to display first image content, the first image content including a first image content frame and a second image content frame, the first image content frame being displayed at a first frame rate that is perceivable to a human eye and the second image content frame being displayed at a second frame rate that is not perceivable to the human eye; and

a device configured to:

analyze the second image content frame as the second image content frame is being displayed by the display to identify at least one tracking pattern embedded in the second image content frame and a relative position of the at least one tracking pattern relative to the device or a relative orientation of the at least one tracking pattern relative to the device,

retrieve second image content based on the at least one tracking pattern and the relative position or the relative orientation, and

display the second image content.

2. The system of claim 1 , wherein the at least one tracking pattern includes one or more geometric shapes.

3. The system of claim 1 , wherein the second image content does not include the second image content frame.

4. The system of claim 1 , wherein the at least one tracking pattern comprises at least one pre-determined tracking pattern.

5. The system of claim 4 , wherein the device is further configured to:

retrieve the at least one pre-determined tracking pattern; and

compare the second image content frame to the at least one pre-determined tracking pattern to identify the at least one tracking pattern.

6. The system of claim 5 , wherein the device is further configured to:

identify at least one distortion in the at least one tracking pattern relative to the at least one pre-determined tracking pattern; and

identify the relative position or the relative orientation based upon the at least one distortion.

7. The system of claim 6 , wherein the at least one distortion results from a curvature of a venue.

8. The system of claim 1 , wherein the display is configured to display the second image content frame within the first image content, and

wherein the display is configured to display the second image content frame at or greater than a frame rate of 60 frames per second.

9. The system of claim 1 , wherein the device is further configured to determine the relative position or the relative orientation from the second image content frame.

10. The system of claim 1 , wherein the at least one tracking pattern is associated with a graphical element, and

wherein the device is configured to:

identify a position or an orientation of the graphical element content to be superimposed within the first image content based upon the relative position or the relative orientation;

overlay the graphical element onto the first image content based upon the position or the orientation of the graphical element content; and

display the first image content.

11. The system of claim 1 , wherein the device is further configured to:

analyze the second image content frame in response to the device being at a first location;

analyze the second image content frame to identify a second relative position of the at least one tracking pattern relative to the device or a second relative orientation of the at least one tracking pattern relative to the device in response to the device being at a second location;

retrieve third image content based on the at least one tracking pattern and the second relative position or the second relative orientation; and

display the third image content.

12. The system of claim 1 , wherein the display and the device are from among a plurality of displays and a plurality of devices, respectively, and

wherein the system further comprises:

a second display from among the plurality of displays configured to display third image content, the third image content including a third image content frame and a fourth image content frame, the third image content frame being displayed at the first frame rate and the fourth image content frame being displayed at the second frame rate; and

a second device from among the plurality of devices configured to:

analyze the fourth image content frame as the fourth image content frame is being displayed by the second display to identify at least one second tracking pattern embedded in the fourth image content frame and a second relative position of the at least one second tracking pattern relative to the second device or a second relative orientation of the at least one second tracking pattern relative to the second device,

retrieve fourth image content, different from the second image content, based on the at least one second tracking pattern and the second relative position or the second relative orientation, and

display the fourth image content.

13. The system of claim 1 , wherein the device is further configured to compare the relative position or the relative orientation to a model of a venue to identify a position of the device within the venue.

14. A method, comprising:

receiving, by a user device, first image content from a video display, the first image content comprising a first image content frame and a second image content frame, the first image content frame being displayed at a first frame rate that is perceivable to a human eye and the second image content frame being displayed at a second frame rate that is not perceivable to the human eye;

analyzing, by the user device, the second image content frame as the second image content frame is being displayed to identify at least one tracking pattern embedded in the second image content frame and a relative position of the at least one tracking pattern relative to the user device or a relative orientation of the at least one tracking pattern relative to the user device;

retrieving, by the user device, second image content based on the at least one tracking pattern in the second image content and the relative position or the relative orientation; and

displaying, by the user device, the second image content.

15. The method of claim 14 , wherein the second content frame is not displayed by the user device.

16. The method of claim 14 , wherein the analyzing comprises analyzing the second image content frame to identify at least one pre-determined tracking pattern in the second image content frame.

17. The method of claim 16 , wherein the analyzing further comprises:

retrieving the at least one pre-determined tracking pattern from a memory; and

comparing the second image content frame to the at least one pre-determined tracking pattern to identify the at least one pre-determined tracking pattern embedded in the second image content frame.

18. The method of claim 14 , wherein the receiving comprises:

receiving the second image content frame at or above 60 frames per second.

19. A mobile device, comprising:

an image sensor configured to receive first image content, the first image content comprising a first image content frame and a second image content frame, the first image content frame being displayed at a first frame rate that is perceivable to a human eye and the second image content frame being displayed at a second frame rate that is not perceivable to the human eye;

a processor; and

at least one memory configured to store a plurality of instructions that, when executed by the processor, cause the processor to perform operations, the operations comprising:

analyzing a second image content frame as the second image content frame is being displayed to identify at least one tracking pattern embedded in the second image content frame and a relative position of the at least one tracking pattern relative to the mobile device or a relative orientation of the at least one tracking pattern relative to the mobile device;

retrieving second image content based on the at least one tracking pattern in the second image content frame and the relative position or the relative orientation; and

displaying the second image content.

20. The mobile device of claim 19 , wherein the displaying the second image content does not include displaying the second image content frame.

21. The mobile device of claim 19 , wherein the analyzing comprises:

analyzing the second image content frame to identify at least one pre-determined tracking pattern in the second image content frame.

22. The mobile device of claim 21 , wherein the analyzing further comprises:

retrieving the at least one pre-determined tracking pattern; and

comparing the second image content frame to the at least one pre-determined tracking pattern to identify the at least one pre-determined tracking pattern embedded in the second image content frame.

23. The mobile device of claim 19 , wherein the image sensor is configured to receive the second image content frame within the first image content at a greater frame rate than the first image content frame within the first image content, and

wherein the greater frame rate comprises a frame rate at or above 60 frames per second.

24. The mobile device of claim 19 , wherein displaying the second image content comprises displaying one or more graphical objects superimposed on the second image content.

25. A system, comprising:

a server configured to provide first image content to a display, the first image content including a first image content frame and a second image content frame,

wherein the display is configured to display the first image content, wherein the first image content frame is displayed at a first frame rate that is perceivable to a human eye and the second image content frame is displayed at a second frame rate that is not perceivable to the human eye; and

a device configured to:

analyze the second image content frame as the second image content frame is being displayed by the display to identify at least one tracking pattern embedded in the second image content frame and a relative position of the embedded tracking pattern relative to the device or a relative orientation of the embedded tracking pattern relative to the device, and

communicate the at least one tracking pattern and the relative position or the relative orientation to the server,

wherein the server is further configured to:

retrieve second image content based on the at least one tracking pattern in the second image content frame and the relative position or the relative orientation to the device, and

provide the second image content to the device for display.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2020
From: ROMASZEWICZ, MICHAEL; PLOURDE, ANDREW
To: MSG SPORTS AND ENTERTAINMENT, LLC
Reel/Frame 054151/0418 →
CHANGE OF NAME Recorded Oct 23, 2020
From: MSG SPORTS AND ENTERTAINMENT, LLC
To: MSG ENTERTAINMENT GROUP, LLC
Reel/Frame 054194/0991 →
Continuity (1)
Related Publication 20210142569A1 · May 13, 2021
Cited By (1)
US 12,244,886