IP Library Granted Patent US 10,628,706
Granted Patent B2
US 10,628,706 · App. 16/408,937 · Granted Apr 21, 2020

Systems and methods for intelligent and interpretive analysis of sensor data and generating spatial intelligence using machine learning

Inventors: Shikhar Shrestha (Palo Alto, CA); Vikesh Khanna (Palo Alto, CA)
Assignee: Ambient AI, Inc.
G06K9/6232G06K9/00362G06K9/72G06N5/02G06N20/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,628,706
App. No.
16/408,937
Granted
Apr 21, 2020
Kind
B2
Abstract

Systems and methods for augmenting real-time semantic information to a spatial rendering of a predefined space and providing a real-time situational awareness feed.

Claims (21)

1. A method for augmenting real-time semantic information to a spatial rendering, comprising: with a comprehension system that is coupled to a plurality of image data sources arranged within a predefined space:

identifying at least one blind spot within the predefined space based on information that identifies a field-of-view (FOV) for each of the plurality of image data sources;

providing, to a user interface system, a user interface that includes a visual representation of a spatial rendering of the predefined space, the user interface including a visual representation of each identified blind spot;

using a semantic module to generate semantic information that identifies a plurality of entities within the predefined space in-real time from image data generated by the plurality of image data sources;

for each identified entity:

tracking a location of the entity in real-time within a coordinate space of the spatial rendering by using contextual metadata stored in a knowledge graph, and

augmenting the visual representation of the spatial rendering with a visual representation of the tracked location, and

for at least one blind spot:

estimating location of at least one identified entity within the blind spot based on a trajectory of the at least one identified entity determined by using image data generated by at least one of the plurality of image data sources, and

augmenting the visual representation of the spatial rendering with a visual representation of each estimated location within the blind spot.

2. The method of claim 1 , wherein the contextual metadata includes, for each image data source, a location and orientation of the image data source within the coordinate space of the spatial rendering and information identifying a field-of-view (FOV) of the image data source.

3. The method of claim 2 , wherein the contextual information includes, for each image data source, a mapping of coordinates in a coordinate space of image frames of the image data source to coordinates in the coordinate space of the spatial rendering, wherein tracking a location of the entity comprises using a mapping of at least one image data source to track the location of the entity.

4. The method of claim 3 , wherein the contextual information includes semantic information generated by the semantic module from at least one of the plurality of image data sources.

5. The method of claim 4 , wherein generating semantic information comprises: generating semantic information based on image data of the plurality of image data sources and the contextual metadata.

6. The method of claim 5 , further comprising: with an event detection module, detecting at least one event by using the contextual metadata and the semantic information.

7. The method of claim 6 , further comprising: with the event detection module, identifying an event entity associated with the detected event, and tracking location of the event entity across FOV's of different image data sources by comparing semantic information generated by the semantic module for at least two of the plurality of image data sources.

8. The method of claim 7 , further comprising: with the comprehension system: updating the user interface to include a live situational awareness feed that includes semantic information generated by the semantic module in real-time for the detected event by using the image data of the plurality of image capture devices.

9. The method of claim 8 , further comprising: with the comprehension system:

receiving sensor information from the user interface system and storing the sensor information in the knowledge graph, wherein the received sensor information includes information identifying a location, orientation and field-of-view (FOV) of at least one image data source; and

receiving from the user interface system a description for at least one location and storing the description in the knowledge graph in association with the respective location.

10. The method of claim 9 , further comprising: updating the visual representation of the spatial rendering to display a view that corresponds to a FOV of an image data source identified by user-selection received via the user interface.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 28, 2019
From: SHRESTHA, SHIKHAR; KHANNA, VIKESH
To: AMBIENT AI, INC.
Reel/Frame 049290/0434 →
Continuity (2)
Provisional Application 62670616 · May 11, 2018
Related Publication 20190347518A1 · Nov 14, 2019
Cited By (1)
US 12,560,926