Ingesting streaming signals
The present invention extends to methods, systems, and computer program products for ingesting streaming signals. Signal ingestion modules sample a frame from a raw streaming signal. A preliminary severity or possible event type is computed from the sampled frame. A deeper inspection request is triggered of the raw streaming signal or of another raw streaming signal. Segments of content from the raw streaming signal or the other raw streaming signal are inspected. An actual severity or actual event type is computed from the inspected content. The actual severity or actual event type can be included in a normalized signal.
1. A method comprising:
ingesting a raw streaming signal, including:
sampling a plurality of single frames from the raw streaming signal; and
storing a plurality of segments separated from at least a portion of the raw streaming signal;
analyzing the plurality of single frames at a first resource allocation level;
determining, based on the analysis at the first resource allocation level, that the raw streaming signal indicates an occurrence of a possible event of a real-world event type;
triggering, based on the determining that the raw streaming signal indicates an occurrence of the possible event, an inspection of at least a portion of the stored plurality of segments utilizing an increased resource allocation level as compared to the first resource allocation level;
verifying that the possible event of the real-world event type is a real-world event of the real-world event type based on the possible event and the inspection of the stored plurality of segments; and
updating a context dimension of a normalized version of the raw streaming signal to include the verified real-world event and real-world event type.
2. The method of claim 1 , wherein ingesting the raw streaming signal comprises ingesting one or more of a video bitstream or an audio bitstream.
3. The method of claim 1 , wherein sampling the plurality of single frame comprises sampling the raw streaming signal in accordance with a sampling frequency.
4. The method of claim 1 , wherein storing the plurality of segments comprises storing the plurality of segments in a memory buffer.
5. The method of claim 1 , wherein storing the plurality of segments comprises storing the plurality of segments at a storage device configured to provide multi-consumer access by time and range indexing.
6. The method of claim 1 , further comprising:
playing a stored segment that is related to the possible event of the real-world event type, from among the plurality of segments; and
inspecting the content of the stored segment during play of the stored segment.
7. The method of claim 6 , further comprising:
feeding the segment to a content classifier;
receiving a context annotation from the content classifier; and
looking up a classification tag corresponding to the context annotation.
8. The method of claim 7 , further comprising normalizing the raw streaming signal into a Time, Location, Context (TLC) signal, including:
determining a time associated with the raw streaming signal;
determining a location associated with the raw streaming signal;
including the time, the location, the classification tag, and the plurality of segments in the TLC signal; and
further comprising sending the TLC signal to an event detection infrastructure.
9. The method of claim 1 , wherein preliminarily classifying the raw streaming signal as indicative of a real-world event type comprises preliminarily classifying the raw streaming signal based on metadata associated with the raw streaming signal.
10. The method of claim 1 , wherein ingesting the raw streaming signal comprises ingesting a raw streaming signal selected from among: a social broadcast or a camera signal.
11. The method of claim 1 , further comprising:
determining an at least approximate likelihood that the raw streaming signal indicates the occurrence of the possible event of the real-world event type; and
determining that the at least approximate likelihood indicates a chance of an actual event that exceeds a threshold.
12. A computer system comprising:
a processor;
system memory coupled to the processor and storing instructions configured to cause the processor to:
ingest a raw streaming signal;
sample a plurality of single frames from the raw streaming signal at a first resource allocation level;
determine, based on analysis of the plurality of single frames, that the raw streaming signal indicates an occurrence of a possible event of a real-world event type;
trigger, based on determining that the raw streaming signal indicates an occurrence of the possible event, an inspection of at least a portion of the stored plurality of segments of the raw streaming signal at an increased resource allocation level as compared to the first resource allocation level;
verify that the possible event of the real-world event type is a real-world event of the real-world event type based on the possible event and the inspected content; and
update a dimension of a normalized signal corresponding to the raw streaming signal to include the verified real-world event and real-world event type.
13. The computer system of claim 12 , wherein the instructions configured to sample the plurality of single frames comprises instructions configured to sample a video bitstream frame; and
wherein the instructions configured to determine that the raw streaming signal indicates an occurrence of a possible event of a real-world event type comprises instructions configured to analyze pixels in the video bitstream frame.
14. The computer system of claim 13 , wherein the instructions configured to analyze pixels in the video bitstream frame comprise instructions configured to use artificial intelligence to analyze the pixels.
15. The computer system of claim 12 , further comprising instructions configured to store a plurality of segments at a storage device providing multi-consumer access by time and range indexing.
16. The computer system of claim 12 , further comprising instructions configured to:
store a plurality of segments separated from at least a portion of the raw streaming signal;
play a stored segment that is related to the possible event of the real-world event type, from among the plurality of segments; and
inspect the content of the stored segment during play of the stored segment.
17. The computer system of claim 16 , further comprising instructions configured to:
feed the stored segment to a content classifier;
receive a context annotation from the content classifier; and
look up a classification tag corresponding to the context annotation.
18. The computer system of claim 17 , further comprising instructions configured to normalize the raw streaming signal into a TLC signal, including causing the processor to:
determine a time associated with the raw streaming signal;
determine a location associated with the raw streaming signal; and
include the time, the location, the classification tag, and the plurality of segments in the TLC signal; and
further comprising instructions configured to send the TLC signal to an event detection infrastructure.
19. The computer system of claim 12 , further comprising instructions configured to preliminarily classify the raw streaming signal as indicative of the real-world event type, including instructions configured to:
determine an at least approximate likelihood that the raw streaming signal indicates the occurrence of the possible event of the real-world event type;
determine that the at least approximate likelihood indicates a chance of an actual event that exceeds a threshold; and
preliminarily classify the raw streaming signal based on metadata associated with the raw streaming signal.
20. The computer system of claim 12 , wherein instructions configured to ingest the raw streaming signal comprises instructions configured to ingest a raw streaming signal selected from among: a social broadcast or a camera signal.