Digital imagery, audio, and meta-data
View Patent ↗Methods, systems, and apparatus, including computer programs encoded on computer storage media, to process audio and visual data. One of the methods includes receiving, from a mobile device, image data and audio data, wherein the image data and audio data were obtained within the same time frame. The method includes processing the image data to identify a first set of facts. The method includes processing the audio data to identify a second set of facts. The method includes processing meta-data associated with the image data to identify a third set of facts. The method also includes performing at least one process using the first set of facts, the second set of facts, and the third set of facts.
1. A system, comprising:
a memory configured to store instructions; and
one or more processors communicatively coupled to the memory, wherein the one or more processors are configured to execute the instructions and perform operations comprising:
receiving, from a mobile device, a data package comprising image data and audio data, wherein the image data and the audio data are acquired during the same time period by the mobile device, and wherein the audio data comprises speech describing a set of facts regarding an accident, and one or more weather conditions;
extracting metadata from the image data and the audio data, wherein the metadata comprises one or more time stamps and global positioning system (GPS) data;
identifying and detecting one or more objects and one or more object conditions within images, wherein the one or more object conditions comprises the one or more weather conditions and one or more road conditions;
deriving weather data based on the extracted metadata and the image data;
determining, via one or more machine learning algorithms, whether the image data comprises the one or more objects and the one or more object conditions representative of one or more components of the set of facts described in the speech, wherein the one or more objects comprise rain, snow, ice, or any combination thereof; and
authenticating an insurance claim based on the derived weather data corroborating the set of facts using speech recognition, the image data, and the metadata to confirm that the one or more objects and the one or more object conditions corroborate the weather data.
2. The system of claim 1 , wherein the one or more components comprises a person, a road, a building, or any combination thereof.
3. The system of claim 1 , the operations further comprising determining a likelihood of fraud based on the authentication.
4. The system of claim 3 , the operations further comprising generating an alert in response to the likelihood of fraud exceeding a threshold value.
5. The system of claim 3 , wherein corroborating the set of facts comprises:
generating a transcription of the speech;
extracting the set of facts from the transcription, wherein the set of facts is associated with the one or more time stamps, the GPS data, or both; and
determining that the set of facts corresponds to the one or more components.
6. The system of claim 1 , the operations further comprising identifying a location of an event associated with the insurance claim based on the GPS data extracted from the metadata.
7. The system of claim 1 , wherein the image data comprises the images, and wherein the images are representative of a vehicle, a damage to the vehicle, a location, a road, or any combination thereof.
8. The system of claim 1 , the operations further comprising providing a set of questions to the mobile device, wherein the set of questions is associated with the audio data.
9. The system of claim 1 , the operations further comprising providing a set of instructions to the mobile device, wherein the set of instructions is associated with the image data.
10. The system of claim 1 , wherein the audio data is acquired simultaneously with the image data.
11. A method, comprising:
receiving, via a processing system, a data package comprising image data and audio data, wherein the audio data comprises speech describing a set of components, wherein the image data and the audio data are acquired by a mobile device during the same time period, and wherein the audio data comprises speech describing a set of facts regarding an accident and one or more weather conditions;
extracting, via the processing system, metadata from the image data and the audio data, wherein the metadata comprises one or more time stamps and global positioning systems (GPS) data;
identifying and detecting, via the processing system, one or more objects and one or more object conditions within images, wherein the one or more object conditions comprises the one or more weather conditions and one or more road conditions;
deriving, via the processing system, weather data based on the extracted metadata and the image data;
determining, via one or more machine learning algorithms, whether the image data comprises the one or more objects and the one or more object conditions representative of one or more components of the set of facts described in the speech, wherein the one or more objects comprise rain, snow, ice, or any combination thereof; and
authenticating, via the processing system, an insurance claim in response to the derived weather data corroborating the set of facts using speech recognition, the image data, and the metadata to confirm that the one or more objects and the one or more object conditions corroborate the weather data.
12. The method of claim 11 , wherein the audio data is acquired simultaneously with the image data.
13. The method of claim 11 , wherein the image data comprises a video recording.
14. The method of claim 11 , further comprising receiving, via the processing system, a request to perform an activity based on the derived weather data.
15. The method of claim 11 , further comprising determining, via the processing system, an activity to be performed based on the derived weather data.
16. The method of claim 15 , wherein the activity comprises determining a likelihood of fraud based on the authentication.
17. A non-transitory computer storage medium comprising computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving, from a mobile device, a data package comprising image data and audio data, wherein the audio data comprises speech describing a set of facts regarding an accident and one or more weather conditions, wherein the image data and the audio data are acquired by the mobile device during the same time period;
extracting metadata from the image data and the audio data, wherein the metadata comprises one or more time stamps and global positioning system (GPS) data;
identifying and detecting one or more objects and one or more object conditions within images, wherein the one or more object conditions comprises the one or more weather conditions and one or more road conditions;
deriving weather data based on the extracted metadata and the image data;
determining, via one or more machine learning algorithms, whether the image data comprises the one or more objects and the one or more objects conditions representative of one or more components of the set of facts described in the speech, wherein the one or more objects comprise rain, snow, ice, or any combination thereof; and
authenticating an insurance claim based on the derived weather data corroborating the set of facts using speech recognition, the image data, and the metadata to confirm that the one or more objects and the one or more object conditions corroborate the weather data.
18. The non-transitory computer storage medium of claim 17 , wherein authenticating the insurance claim comprises determining that a location of an event associated with the insurance claim matches the GPS data extracted from the metadata.