IP Library Granted Patent US 10,586,124
Granted Patent B2
US 10,586,124 · App. 15/668,510 · Granted Mar 10, 2020

Methods and systems for detecting and analyzing a region of interest from multiple points of view

Inventors: Richard Oesterreicher (Johns Creek, GA); Jonathan Hessing (Johns Creek, GA); Zunair Ukani (Johns Creek, GA); Austin Schmidt (Johns Creek, GA)
Assignee: Streaming Global, Inc.
G06K9/3241G06K9/00718G06K9/481G06T7/292G06T7/521G06T7/55G06T7/62G06T7/73G06T2207/10016G06T2207/30241G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,586,124
App. No.
15/668,510
Granted
Mar 10, 2020
Kind
B2
Abstract

Disclosed are devices, systems, methods, techniques, and computer program products for estimating a Region Of Interest (ROI) corresponding to a plurality of content streams. The method may include receiving, by a system having at least one processor, a plurality of sensor data associated with a plurality of mobile devices. The plurality of sensor data corresponds to a plurality of content streams captured by the plurality of mobile devices. Further, each of the plurality of mobile devices may include at least one recorder configured to capture a corresponding content steam. Further, a sensor data associated with a mobile device may include a location data and an orientation data of the mobile device during capturing of a content stream. The method may further include, analyzing, by the system, the plurality of sensor data and determining, by the system, a ROI based on at least the plurality of sensor data, thereby generating an estimation of the ROI.

Claims (65)

1. A method of estimating a Region of Interest (ROI) corresponding to a plurality of content streams, the method comprising:

receiving, by a system having at least one processor, a plurality of sensor data associated with a plurality of mobile devices, wherein the plurality of sensor data corresponds to the plurality of content streams captured by the plurality of mobile devices, wherein each of the plurality of mobile devices comprises at least one recorder configured to capture a corresponding content steam, wherein a sensor data associated with a mobile device comprises a location data and an orientation data of the mobile device during capturing of a content stream;

analyzing, by the system, the plurality of sensor data;

determining, by the system, a ROI based on at least the plurality of sensor data, thereby generating an estimation of the ROI; and

receiving, by the system, a plurality of optical characteristics corresponding to the plurality of mobile devices, wherein determining the ROI is based at least on the plurality of optical characteristics.

2. The method of claim 1 , wherein an optical characteristic of the mobile device comprises a Field of View (FOV) of a lens comprised in the mobile device.

3. The method of claim 1 , further comprising receiving a plurality of device identifiers from the plurality of mobile devices, wherein the plurality of optical characteristics is identified based on at least the plurality of device identifiers.

4. The method of claim 1 , wherein analyzing the plurality of sensor data comprises:

determining a plurality of origin points based at least on a plurality of location data corresponding to the plurality of mobile devices; and

determining a plurality of vectors associated with the plurality of origin points based at least on a plurality of orientation data corresponding to the plurality of mobile devices, a vector associated with an origin point originating at the origin point.

5. The method of claim 4 , wherein determining the ROI is based at least on the plurality of origin points and the plurality of vectors.

6. The method of claim 5 , wherein the plurality of origin points is configured to be used as coordinates in a mapping coordinate system.

7. The method of claim 6 , further comprising determining a plurality of vector intersections of the plurality of vectors; and wherein determining the ROI is based at least on the plurality of vector intersections.

8. The method of claim 1 , further comprising:

receiving a plurality of Field of Views (FOVs) associated with the plurality of mobile devices, a Field of View (FOV) associated with the mobile device being associated with a lens comprised in the mobile device;

determining a plurality of origin points based at least on a plurality of location data corresponding to the plurality of mobile devices, the plurality of origin points configured as coordinates in a mapping coordinate system;

determining a plurality of vectors associated with the plurality of origin points based at least on a plurality of orientation data corresponding to the plurality of mobile devices, a vector associated with an origin point originating at the origin point;

generating a plurality of viewing frustums based at least on the plurality of FOVs, wherein an apex of a viewing frustum associated with the mobile device is coincident with the origin point associated with the mobile device, wherein a view direction of the viewing frustum is aligned with the vector associated with the mobile device; and

determining a plurality of intersections of the plurality of viewing frustums, wherein determining the ROI is based at least on the plurality of intersections of the plurality of viewing frustums.

9. The method of claim 8 , wherein determining the plurality of origin points comprises:

converting the plurality of location data corresponding to the plurality of mobile devices into the plurality of origin points in the mapping coordinate system; and

wherein determining the plurality of vectors comprises converting the plurality of orientation data corresponding to the plurality of mobile devices into a plurality of vectors associated with the plurality of origin points, wherein the vector associated with an origin point originates at the origin point.

10. The method of claim 1 , further comprising:

receiving the plurality of content streams from the plurality of mobile devices;

analyzing the plurality of content streams;

determining at least one object associated with the ROI; and

determining at least one spatial dimension associated with the at least one object;

wherein determining the ROI is based at least on the at least one spatial dimension.

11. The method of claim 1 further comprising:

receiving the plurality of content streams from the plurality of mobile devices;

analyzing the plurality of content streams; determining at least one object associated with the ROI based at least on the plurality of content streams;

determining at least one spatial dimension associated with the at least one object based at least on a predetermined association between a plurality of objects and a plurality of spatial dimensions; and

determining a plurality of distances corresponding to the plurality of mobile devices and the ROI, wherein a distance between the mobile device and the ROI is estimated based on a spatial dimension of the at least one object determined in the content stream captured by the mobile device;

wherein determining the ROI is based at least on the plurality of distances.

12. The method of claim 11 , wherein determining the plurality of distances comprises determining the plurality of distances corresponding to the plurality of mobile devices, wherein the distance between the mobile device and the ROI is determined based at least on the spatial dimension of the at least one object determined in the content stream captured by the mobile device.

13. The method of claim 1 , wherein the sensor data further comprises a distance data representing a distance of the mobile device from at least one object comprised in the ROI, wherein the mobile device comprises a ranging sensor configured to determine the distance.

14. The method of claim 13 , wherein the ranging sensor comprises at least one of a LIDAR sensor and an image sensor.

15. The method of claim 1 , further comprising:

executing a search on a venue database based on the location data;

determining a venue from the venue database based at least on the location data; and

retrieving at least one spatial characteristic of the venue;

wherein determining the ROI is based at least on the at least one spatial characteristic.

16. The method of claim 15 , further comprising determining at least one trajectory associated with at least one mobile device based at least on analysis of the plurality of sensor data; wherein determining the ROI is based at least on the at least one trajectory.

17. The method of claim 1 , wherein the plurality of mobile devices comprises a drone configured for flight.

18. The method of claim 1 , further comprising:

generating a visualization of the ROI and the plurality of mobile devices; and

transmitting the visualization to at least one mobile device of the plurality of mobile devices.

19. The method of claim 18 , further comprising:

receiving a selection associated with the visualization;

identifying one or more mobile devices associated with the selection; and

transmitting one or more content streams captured by the plurality of mobile devices.

20. A method of estimating a Region of Interest (ROI) corresponding to a plurality of content streams, the method comprising:

receiving, by a system having at least one processor, a plurality of sensor data associated with a plurality of mobile devices, wherein the plurality of sensor data corresponds to the plurality of content streams captured by the plurality of mobile devices, wherein each of the plurality of mobile devices comprises at least one recorder configured to capture a corresponding content steam, wherein a sensor data associated with a mobile device comprises a location data and an orientation data of the mobile device during capturing of a content stream;

analyzing, by the system, the plurality of sensor data;

determining, by the system, a ROI based on at least the plurality of sensor data, thereby generating an estimation of the ROI;

receiving the plurality of content streams from the plurality of mobile devices;

analyzing the plurality of content streams;

determining at least one object associated with the ROI; and

determining at least one spatial dimension associated with the at least one object.

21. A method of estimating a Region of Interest (ROI) corresponding to a plurality of content streams, the method comprising:

receiving, by a system having at least one processor, a plurality of sensor data associated with a plurality of mobile devices, wherein the plurality of sensor data corresponds to a plurality of content streams captured by the plurality of mobile devices, wherein each of the plurality of mobile devices comprises at least one recorder configured to capture a corresponding content steam, wherein a sensor data associated with a mobile device comprises a location data and an orientation data of the mobile device during capturing of a content stream;

analyzing, by the system, the plurality of sensor data, wherein analyzing the plurality of sensor data comprises:

determining a plurality of origin points based at least on a plurality of location data corresponding to the plurality of mobile devices; and

determining a plurality of vectors associated with the plurality of origin points based at least on a plurality of orientation data corresponding to the plurality of mobile devices, a vector associated with an origin point originating at the origin point; and

determining, by the system, a ROI based on at least the plurality of sensor data, thereby generating an estimation of the ROI.

Assignments (3)
ASSIGNMENT CONVERSION Recorded Jan 7, 2020
From: STREAMING GLOBAL, LLC
To: STREAMING GLOBAL, INC.
Reel/Frame 051496/0304 →
CHANGE OF NAME Recorded Aug 2, 2019
From: SHOUTY, LLC
To: STREAMING GLOBAL, LLC
Reel/Frame 049950/0656 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 15, 2018
From: OESTERREICHER, RICHARD; HESSING, JONATHAN; UKANI, ZUNAIR; SCHMIDT, AUSTIN
To: SHOUTY, LLC
Reel/Frame 047516/0932 →
Continuity (1)
Related Publication 20190042868A1 · Feb 7, 2019