IP Library Granted Patent US 10,933,316
Granted Patent B1
US 10,933,316 · App. 15/953,213 · Granted Mar 2, 2021

Object track for physical interaction and virtualization, and physical object detection and virtualization

Inventors: Jerome Scholler (San Francisco, CA); Arnaud Brejeon (Yorba Linda, CA); Andrew Berwald (Palo Alto, CA); Mark Solomon (Palo Alto, CA)
Assignee: Tangible Play, Inc.
A63F13/245A63F13/65A63F13/803A63F13/98
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,933,316
App. No.
15/953,213
Granted
Mar 2, 2021
Kind
B1
Abstract

An example method for virtualizing physical objects into a virtual environment includes capturing a video stream of a racing field component, displaying a graphical user interface embodying a virtual game, detecting a placement of a physical vehicle at a starting portion of a channel of the racing field component, identifying one or more characteristics of the physical vehicle at the starting portion, generating a virtual vehicle representation of the physical vehicle using the one or more characteristics of the physical vehicle, and displaying the virtual vehicle representation of the physical vehicle.

Claims (108)

1. A method comprising:

capturing, using a video capture device, a video stream of a physical scene including a racing field component having a track extending from a starting portion to an end portion;

displaying, on a display, a graphical user interface embodying a virtual game;

detecting, in the video stream, a placement of a physical vehicle in the starting portion of the track of the racing field component;

identifying, using a processor of a computing device, one or more characteristics of the physical vehicle;

generating, using the processor of the computing device, a virtual vehicle representation of the physical vehicle using the one or more characteristics of the physical vehicle;

displaying, on the display of the computing device, the virtual vehicle representation of the physical vehicle;

displaying, in the graphical user interface of the virtual game, a starting prompt;

detecting a movement of the physical vehicle responsive to the starting prompt being displayed, the movement of the physical vehicle caused by a vehicle retaining mechanism releasing the physical vehicle from a starting position;

determining, using the processor of the computing device, whether the movement is within a start release period; and

executing, in the graphical user interface of the virtual game, a starting action associated with the virtual vehicle representation based on the determination of whether the movement is within the start release period.

2. The method of claim 1 , further comprising:

detecting, in the video stream, a token placed in the physical scene;

identifying a virtual action based on one or more characteristics of the token; and

executing the virtual action in the virtual game.

3. The method of claim 2 , wherein executing the virtual action in the virtual game includes changing a virtual movement of the virtual vehicle representation in the virtual game.

4. The method of claim 2 , wherein the virtual action is one of a speed increase, a speed decrease, a lane change, a jump, and an attack.

5. The method of claim 1 , further comprises:

detecting, using the processor of the computing device, a first quantity of a visual cue surface of the racing field component that is depicted in the video stream at a first time;

determining a first sound based on the first quantity of the visual cue surface of the racing field component that is depicted; and

emitting, using an audio reproduction device, the first sound.

6. The method of claim 5 , further comprising:

detecting, using the processor of the computing device, a second quantity of the visual cue surface of the racing field component that is depicted in the video stream at a second time;

determining a second sound based on the second quantity of the visual cue surface of the racing field component that is depicted; and

emitting, using the audio reproduction device, the second sound.

7. The method of claim 1 , wherein:

the track is a first track,

the starting portion is a first starting portion,

the end portion is a first end portion,

the placement is a first placement,

the physical vehicle is a first physical vehicle,

the virtual vehicle representation is a first virtual vehicle representation,

the video stream of the physical scene includes a second track extending from a second starting portion to a second end portion, and

the method further comprises:

detecting, in the video stream, a second placement of a second physical vehicle in the second starting portion of the second track;

identifying, using the processor of the computing device, one or more characteristics of the second physical vehicle at the second starting portion;

generating, using the processor of the computing device, a second virtual vehicle representation of the second physical vehicle using the one or more characteristics of the second physical vehicle; and

displaying, on the display of the computing device, the second virtual vehicle representation of the second physical vehicle in the graphical user interface of the virtual game, the second virtual vehicle representation displayed adjacent to the first virtual vehicle representation.

8. The method of claim 7 , wherein the first physical vehicle is different from the second physical vehicle and the method further comprises:

associating, using the processor of the computing device, the first physical vehicle with a first user of the virtual game and the second physical vehicle with a second user of the virtual game.

9. The method of claim 7 , wherein the movement is a first movement, the vehicle retaining mechanism is a first vehicle retaining mechanism, and the starting action is a first starting action, the method further comprising:

detecting a second movement of the second physical vehicle responsive to the starting prompt being displayed, the second movement of the second physical vehicle caused by a second vehicle retaining mechanism releasing the second physical vehicle from the second starting portion;

determining, using the processor of the computing device, whether the second movement is within the start release period; and

executing, in the graphical user interface of the virtual game, a second starting action associated with the second virtual vehicle representation based on the determination of whether the second movement is within the start release period.

10. A virtualization system comprising:

a video capture device coupled to a computing device, the video capture device being adapted to capture a video stream of a physical scene that includes a racing field component having a track extending from a starting portion to an end portion;

a display coupled to the computing device and adapted to display a graphical user interface embodying a virtual game;

a detector coupled to the computing device and adapted to detect, in the video stream, a placement of a physical vehicle in the starting portion of the track of the racing field component and detect a movement of the physical vehicle responsive to a starting prompt being displayed and a vehicle retaining mechanism releasing the physical vehicle from a starting position; and

a processor of the computing device, the processor being adapted to:

identify one or more characteristics of the physical vehicle;

generate a virtual vehicle representation of the physical vehicle using the one or more characteristics of the physical vehicle; display, on the display of the computing device, the virtual vehicle representation of the physical vehicle;

display, in the graphical user interface of the virtual game, the starting prompt;

determine whether the movement is within a start release period; and

execute, in the graphical user interface, a starting action associated with the virtual vehicle representation based on the determination of whether the movement is within the start release period.

11. The virtualization system of claim 10 , wherein:

the detector coupled to the computing device is adapted to detect, in the video stream, a token placed in the physical scene; and

the processor of the computing device is adapted to:

identify a virtual action based on one or more characteristics of the token; and

execute the virtual action in the virtual game.

12. The virtualization system of claim 11 , wherein executing the virtual action in the virtual game includes changing a virtual movement of the virtual vehicle representation in the virtual game.

13. The virtualization system of claim 11 , wherein the virtual action is one of a speed increase, a speed decrease, a lane change, a jump, and an attack.

14. The virtualization system of claim 10 , wherein the processor of the computing device is adapted to:

detect a first quantity of a visual cue surface of the racing field component that is depicted in the video stream at a first time;

determine a first sound based on the first quantity of the visual cue surface of the racing field component that is depicted; and

emit, using an audio reproduction device, the first sound.

15. The virtualization system of claim 14 , wherein the processor of the computing device is adapted to:

detect a second quantity of the visual cue surface of the racing field component that is depicted in the video stream at a second time;

determine a second sound based on the second quantity of the visual cue surface of the racing field component that is depicted; and

emit, using the audio reproduction device, the second sound.

16. The virtualization system of claim 10 , wherein:

the track is a first track,

the starting portion is a first starting portion,

the end portion is a first end portion,

the placement is a first placement,

the physical vehicle is a first physical vehicle,

the virtual vehicle representation is a first virtual vehicle representation,

the video stream of the physical scene includes a second track extending from a second starting portion to a second end portion,

the detector coupled to the computing device is adapted to detect, in the video stream, a second placement of a second physical vehicle in the second starting portion of the second track, and

the processor of the computing device is adapted to:

identify one or more characteristics of the second physical vehicle at the second starting portion;

generate a second virtual vehicle representation of the second physical vehicle using the one or more characteristics of the second physical vehicle; and

display, on the display of the computing device, the second virtual vehicle representation of the second physical vehicle in the graphical user interface of the virtual game, the second virtual vehicle representation displayed adjacent to the first virtual vehicle representation.

17. The virtualization system of claim 16 , wherein the first physical vehicle is different from the second physical vehicle and the processor of the computing device is adapted to:

associate the first physical vehicle with a first user of the virtual game and the second physical vehicle with a second user of the virtual game.

18. The virtualization system of claim 16 , wherein, the movement is a first movement, the vehicle retaining mechanism is a first vehicle retaining mechanism, the starting action is a first starting action, and wherein:

the detector coupled to the computing device is adapted to:

detect a second movement of the second physical vehicle responsive to the starting prompt being displayed, the second movement of the second physical vehicle caused by a second vehicle retaining mechanism releasing the second physical vehicle from the second starting portion; and

the processor of the computing device is adapted to:

determine whether the second movement is within the start release period; and

execute, in the graphical user interface of the virtual game, a second starting action associated with the second virtual vehicle representation based on the determination of whether the second movement is within the start release period.

19. A method comprising:

capturing, using a video capture device, a video stream of a physical scene including a racing field component having a track extending from a starting portion to an end portion;

displaying, on a display, a graphical user interface embodying a virtual game;

detecting, in the video stream, a placement of an object in the starting portion of the track of the racing field component;

identifying, using a processor of a computing device, one or more characteristics of the object;

generating, using the processor of the computing device, a virtual representation of the object using the one or more characteristics of the object;

displaying, on the display of the computing device, the virtual representation of the object;

displaying, on the display, a starting prompt;

detecting a movement of the object responsive to the starting prompt being displayed, the movement of the object caused by a retaining mechanism releasing the object from the starting portion;

determining, using the processor of the computing device, whether the movement is within a start release period;

executing, a starting action associated with the virtual representation based on the determination that the movement is within the start release period;

detecting, in the video stream, a token placed in the physical scene;

identifying a virtual action based on one or more characteristics of the token; and

applying the virtual action to the virtual representation of the object in the virtual game.

20. The method of claim 19 , further comprising:

detecting, using the processor of the computing device, a quantity of a visual cue surface of the racing field component depicted in the video stream;

determining a sound based on the quantity of the visual cue surface; and

emitting, using an audio reproduction device, the sound.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 30, 2026
From: TANGIBLE PLAY, INC.
To: PLAY OSMO INC.
Reel/Frame 074224/0665 →
RELEASE OF SECURITY INTEREST Recorded May 30, 2025
From: GLAS TRUST COMPANY LLC
To: CLAUDIA Z. SPRINGER, AS CHAPTER 11 TRUSTEE OF TANGIBLE PLAY, INC., ON BEHALF OF THE ESTATE OF DEBTOR TANGIBLE PLAY, INC.
Reel/Frame 071473/0897 →
SECURITY INTEREST Recorded Nov 1, 2024
From: CLAUDIA Z. SPRINGER, AS CHAPTER 11 TRUSTEE OF EPIC! CREATIONS, INC., ON BEHALF OF THE ESTATE OF DEBTOR EPIC! CREATIONS, INC.; CLAUDIA Z. SPRINGER, AS CHAPTER 11 TRUSTEE OF NEURON FUEL, INC., ON BEHALF OF THE ESTATE OF DEBTOR NEURON FUEL, INC.; CLAUDIA Z. SPRINGER, AS CHAPTER 11 TRUSTEE OF TANGIBLE PLAY, INC., ON BEHALF OF THE ESTATE OF DEBTOR TANGIBLE PLAY, INC.
To: GLAS TRUST COMPANY LLC
Reel/Frame 069290/0456 →
SECURITY INTEREST Recorded May 16, 2022
From: TANGIBLE PLAY, INC.
To: GLAS TRUST COMPANY LLC
Reel/Frame 060257/0811 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 13, 2019
From: SCHOLLER, JEROME; BREJEON, ARNAUD; BERWALD, ANDREW; SOLOMON, MARK
To: TANGIBLE PLAY, INC
Reel/Frame 048320/0402 →
Continuity (2)
Provisional Application 62564150 · Sep 27, 2017
Provisional Application 62484906 · Apr 13, 2017