IP Library Granted Patent US 11,688,079
Granted Patent B2
US 11,688,079 · App. 17/218,331 · Granted Jun 27, 2023

Digital representation of multi-sensor data stream

Inventor: Patrick Soon-Shiong (Los Angeles, CA)
Assignee: Nant Holdings IP, LLC
G06T7/246G06F16/70G09B19/0038H04N7/181G06T2207/30221
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,688,079
App. No.
17/218,331
Granted
Jun 27, 2023
Kind
B2
Abstract

A digital content production center and method are described. An illustrative method includes receiving object information from a plurality of different sensors, where a first sensor in the plurality of sensors provides first sensor data to describe the object information and where a second sensor in the plurality of sensors provides second sensor data to describe the object information. The method further includes determining that the first sensor data and not the second sensor data, thereby differentiating sensor data information, is to be used in connection with building an image of an object described by the object information, generating a set of prioritized data streams that include digital representations of the first sensor data and not necessarily the second sensor data, and transmitting the data streams across a communication network, according to a prioritization.

Claims (33)

1. A computer-based broadcast system comprising:

a set of sensors disposed proximal to a physical environment;

at least one non-transitory, computer readable memory storing data; and

at least one processor communicatively coupled with the set of sensors and with the memory, and which performs the following operations upon processing the data:

obtaining a set of sensor data from the set of sensors;

compiling a set of digital data streams, each having a different priority assigned thereto, from the set of sensor data according to a prioritization policy that provides prioritized layer rendering instructions on how each different digital data stream in the set of digital data streams is instantiated in the memory relative to each other based on the different priority assigned thereto and that binds priorities of each digital data stream to how each stream's layer of renderable digital content derived from the set of sensor data is rendered in a layered fashion;

determining, during the compiling, that a conflict exists between two or more digital data streams in the set of digital data streams;

referencing a prioritization policy to resolve the conflict between the two or more digital data streams, wherein the prioritization policy comprises one or more of an object-specific rule and conflict-specific rule by which to resolve the conflict and further indicates which particular object and/or action takes priority and will be included in the compiled set of digital data streams; and

transmitting over a network, according to each stream's different priority assigned thereto, the set of digital data streams to at least one endpoint device such that the at least one endpoint device is enabled to produce an image representing two or more digital data streams from the set of digital data streams by rendering the layer of renderable digital content of each of the two or more digital data streams in the layered fashion based on the priorities bound thereto by the prioritization.

2. The system of claim 1 , wherein the physical environment comprises at least one of the following: a stage, a sound stage, a scene, an outdoor setting, a sporting arena, an eSporting arena, a court, a field, and a raceway.

3. The system of claim 1 , wherein the set of sensors comprise a heterogeneous mix of sensor types.

4. The system of claim 1 , wherein the sensor data comprises at least one real-time sensor feed.

5. The system of claim 4 , wherein at least one data stream of the set of data streams comprises a real-time stream representing the at least one real-time sensor feed.

6. The system of claim 1 , wherein the layer of renderable content comprises rendering instructions.

7. The system of claim 1 , wherein the layer of renderable content comprises a wireframe model having a highest priority as compared to other layers of renderable content associated with other digital data streams in the set of digital data streams.

8. The system of claim 1 , wherein the layer of renderable content comprises a polygon model.

9. The system of claim 1 , wherein the layer of renderable content comprises digital textures.

10. The system of claim 1 , wherein the operations further include storing the prioritization policy in the at least one non-transitory, computer readable memory.

11. The system of claim 1 , wherein the prioritization policy comprises rendering parameters of the at least one endpoint device.

12. The system of claim 1 , wherein the prioritization policy comprises a subscription level.

13. The system of claim 12 , wherein the prioritization of the set of digital data streams depends on the subscription level.

14. The system of claim 13 , wherein the prioritization of the prioritized layer rendering instructions depends on the subscription level.

15. The system of claim 12 , wherein the subscription level comprises a fee.

16. The system of claim 1 , wherein the layers of the renderable content comprises at least two layers, each corresponding to a different digital data stream from the set of digital data streams.

17. The system of claim 1 , wherein the layer of renderable content comprises at least one of the following types of digital content: an image, a video, a 3D model, an animation, audio data, haptic data, movie data, and game data.

18. The system of claim 1 , wherein the set of sensors include at least two of the following: a microphone, a camera, a laser sensor, a biometric sensor, and a depth sensor.

19. The system of claim 1 , wherein the data comprises at least one of an Artificial Neural Network (ANN) and an instruction set.

20. A broadcast method, comprising:

obtaining a set of sensor data from a set of sensors;

compiling a set of digital data streams, each having a different priority assigned thereto, from the set of sensor data according to a prioritization policy that provides prioritized layer rendering instructions on how each different digital data stream in the set of digital data streams is instantiated in the memory relative to each other based on the different priority assigned thereto and that binds priorities of each digital data stream to how each stream's layer of renderable digital content derived from the set of sensor data is rendered in a layered fashion;

determining, during the compiling, that a conflict exists between two or more digital data streams in the set of digital data streams;

referencing a prioritization policy to resolve the conflict between the two or more digital data streams, wherein the prioritization policy comprises one or more of an object-specific rule and conflict-specific rule by which to resolve the conflict and further indicates which particular object and/or action takes priority and will be included in the compiled set of digital data streams; and

transmitting over a network, according to each stream's different priority assigned thereto, the set of digital data streams to at least one endpoint device such that the at least one endpoint device is enabled to produce an image representing two or more digital data streams from the set of digital data streams by rendering the layer of renderable content of each of the two or more digital data streams in the layered fashion based on the priorities bound thereto by the prioritization policy.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 7, 2021
From: SOON-SHIONG, PATRICK
To: NANT HOLDINGS IP, LLC
Reel/Frame 055850/0607 →
Continuity (2)
Provisional Application 63002708 · Mar 31, 2020
Related Publication 20210304418A1 · Sep 30, 2021
Cited By (3)
US 12,437,422 US 12,675,109 US 12,710,751