Granted Patent B2

US 12,251,643 · App. 17/635,416 · Granted Mar 18, 2025

Inventors: Morgan James Walker (Billund, DK); Henrik Munk Storm (Billund, DK)

Assignee: LEGO A/S

A63H3/003A63H5/00A63H2200/00

Abstract

Claims (55)

1. An interactive music play system, comprising:

a plurality of toy elements, an image capturing device, a processing unit and a music rendering device,

wherein the processing unit is configured to:

recognize a user-selected subset of toy elements in one or more captured images, captured by the image capturing device, of the user-selected subset of said toy elements arranged in a field of view of the image capturing device;

create a musical presentation and cause the music rendering device to present the created musical presentation to the user;

cause the music rendering device to provide, responsive to one or more of the recognized toy elements, one or more user-activatable user-interface elements, and

cause the music rendering device to alter, responsive to a user-activation of one or more of the user-interface elements, the presented musical presentation.

2. The system according to claim 1 , wherein:

the plurality of toy elements comprises a plurality of tokens configured to be individually recognizable and distinguishable from each other by the processing unit in an image depicting at least one or more of the tokens, and

the processing unit is adapted to:

cause the music rendering device to provide, responsive to recognizing one or more of the tokens in the one or more captured images, one or more user-activatable user-interface elements, and

cause the music rendering device to alter, responsive to a user-activation of one or more of the user-interface elements, the presented musical presentation.

3. The system according to claim 1 , wherein:

the plurality of toy elements comprises one or more figurines, each representing an artist or music performer, and the processing unit is configured, responsive to recognizing the one or more figurines in

the one or more captured images, to select a virtual character associated with the recognized figurine and to create an audio-visual presentation showing the virtual character performing a piece of music.

4. The system according to claim 1 , wherein:

the plurality of toy elements comprises one or more figurines, each representing an artist or music performer, and

the processing unit is configured to create an interactive audio-visual presentation showing a virtual character performing a piece of music and, responsive to recognizing the one or more figurines as corresponding to the virtual character, to modify the created interactive audio-visual presentation.

5. The system according to claim 1 , wherein the processing unit is configured to create, from one or more images, captured by the image capturing device, of a real-world scene or of a representation of a real-world scene, an audio-visual presentation showing a virtual character moving about a representation of the real-world scene and performing a piece of music.

6. The system according to claim 5 , wherein the created audio-visual presentation includes an interactive augmented-reality presentation of the real-world scene augmented with virtual characters performing the piece of music.

7. The system according claim 1 , wherein the processing unit is configured to recognize a reference toy in a captured live video, captured by the image capture device, to track the position and orientation of the reference toy in the captured live video and to create an augmented-reality presentation where one or more virtual characters are shown as moving about the reference toy and performing a selected piece of music.

8. The system according to claim 7 , wherein the processing unit is configured to detect at least one user-modifiable feature of the reference toy and to create or modify the augmented-reality presentation responsive to the detected one or more user-modifiable features.

9. The system according to claim 7 , wherein:

at least one of the toy elements is a figurine, and

creating the musical presentation comprises:

selecting at least one virtual character responsive to recognizing a figurine by the processing unit in one or more images, captured by the image capturing device, of a real-world scene,

creating an interactive audio-visual musical presentation from the one or more captured images of the real-world scene and including a representation of the selected at least one virtual character within the scene, and

animating the representation of the virtual character to represent performance of a piece of music by said virtual character within the scene.

10. The system according to claim 9 , wherein animating the representation of the virtual character includes scaling the representation of the virtual character relative to the scene.

11. The system according to claim 7 , further comprising a storage container for storing the toy elements of the system described herein, the storage container including a support structure for supporting the toy elements when an image is captured of the toy elements arranged in a predetermined spatial configuration relative to each other by the image capture device.

12. The system according to claim 7 , wherein the system is configured to provide functionality allowing a user to define movements to be performed by one or more virtual character, and wherein the processing unit is configured to create an audio-visual presentation showing the virtual character performing a piece of music; wherein the character performs the user defined movements.

13. The system according to claim 12 , wherein:

the processing unit is configured to process a captured video, captured by the image capturing device, of a person performing a movement, and to detect the performed movement, and to map the detected movement onto the virtual character.

14. A computer-implemented method of operating a music play system, the method comprising:

recognizing a user-selected subset of toy elements in one or more captured images of the user-selected subset of said toy elements arranged in a field of view of an image capturing device;

creating a musical presentation and presenting the created musical presentation to the user;

providing, responsive to one or more of the recognized toy elements, one or more user-activatable user-interface elements; and

altering, responsive to a user-activation of one or more of the user-interface elements, the presented musical presentation.

15. A music play system comprising:

a plurality of toy elements, wherein at least one of the toy elements is a figurine;

a camera configured to capture images;

a musical rendering device module; and

a processing unit configured to:

receive images from the camera;

recognize a user-selected subset of the toy elements in one or more of the images; and

create a musical presentation and cause the music rendering device to play the musical presentation,

wherein the musical presentation includes: selecting a virtual character based on the figurine, including a representation of the virtual character in the musical presentation, and animating the representation of the virtual character to represent performance of a piece of music by the virtual character.

16. A music play system comprising:

a plurality of toy elements, wherein at least one of the toy elements is a figurine;

a camera configured to capture images;

a musical rendering device module; and

a processing unit configured to:

receive images from the camera;

recognize a user-selected subset of the toy elements in one or more of the images; and

create a musical presentation responsive to the recognized user-selected subset of the toy elements, and cause the music rendering device to play the musical presentation.

Assignments (1)

Continuity (1)

References Cited (31)