Method and Apparatus for Providing Information Via Collected and Stored Metadata Using Inferred Attentional Model
A method and/or system is able to improve vehicle safety via metadata extraction by an inferred attentional (“IA”) model trained by a virtuous cycle containing sensors, machine learning center (“MLC”), and cloud based network (“CBN”). In one aspect, the system includes a set of outward facing camera, inward facing cameras, and vehicle onboard computer (“VOC”). The outward facing cameras collect external images representing a surrounding environment in which the vehicle operates. The inward facing cameras collect internal images including operator facial expression representing at least operator's attention. The VOC is configured to identify operator's attention in response to the collected internal images and the collected external images.
1 . A system for enhancing safety via metadata extraction managed by an inferred attentional model, comprising:
a plurality of outward facing cameras mounted on a vehicle collecting external images representing a surrounding environment in which the vehicle operates;
a plurality of inward facing cameras mounted in the vehicle collecting internal images including operator facial expression representing at least operator's attention; and
a vehicle onboard computer (“VOC”), coupled to the plurality of outward facing cameras and the plurality of inward facing cameras, configured to identify operator's attention in response to the collected internal images and the collected external images.
2 . The system of claim 1 , further comprising a plurality of audio sensors coupled to the VOC and configured to provide metadata relating to audio data.
3 . The system of claim 2 , wherein the plurality of audio sensors includes exterior audio sensors collecting exterior sound outside of the vehicle and interior audio sensors collecting interior sound inside of the vehicle.
4 . The system of claim 1 , wherein the collecting external images includes obtaining real-time images relating to at least one of road, buildings, traffic lights, pedestrian, and retailers.
5 . The system of claim 1 , wherein the collecting internal images includes a set of interior sensors obtaining data relating to at least one of operator's eyes, facial expression, driver, and passage.
6 . The system of claim 5 , wherein the obtaining data relating to at least one of operator's eyes, facial expression, driver, and passage includes at least one camera is capable of detecting direction of where the operator is looking.
7 . The system of claim 1 , wherein the VOC is wirelessly connected to Internet capable of communicating with cloud data.
8 . The system of claim 1 , wherein the VOC is configured to communicate with virtuous cycle to modify the inferred attention model.
9 . The system of claim 1 , wherein the VOC configured to identify operator's attention includes a pipeline processor capable of identify what the operator is looking based on direction the operator is looking collected by interior cameras and exterior objects captured by the exterior cameras.
10 . The system of claim 9 , wherein the VOC provides a warning component capable of issuing a warning sound based on exterior data obtained by exterior cameras and interior data collected by interior cameras.
11 . A method for assisting vehicle operation via metadata extraction processed by an inferred attentional (“IA”) model, comprising:
activating a set of outward facing cameras mounted on a vehicle for recording external surrounding images representing a geographic environment in which the vehicle operates;
selectively enabling at least one of a set of inward facing cameras mounted in the vehicle for collecting interior images of the vehicle;
identifying targeted direction focused by operator eyes in accordance with the interior images and stored data managed by the IA model;
determining external target in response to the external surrounding images and the targeted direction; and
identifying an object that the operator eyes are looking based on the external target and information supplied by the IA model.
12 . The method of claim 11 , further comprising tracking surrounding environmental event in accordance with the external surrounding images and information provided by the IA model trained by a virtuous cycle.
13 . The method of claim 12 , further comprising identifying potential vehicle collision in response to the surrounding environmental event.
14 . The method of claim 13 , further comprising providing a spatially correlated audio warning to the operator if the potential vehicle collision is determined.
15 . The method of claim 14 , further comprising providing automatic adjustment of interior entertainment volume when the spatially correlated audio warning is activated.
16 . The method of claim 13 , further comprising providing a spatially correlated visual warning to the operator if the potential vehicle collision is determined.
17 . The method of claim 13 , further comprising adjusting moving direction of the vehicle to avoid potential collision in accordance with a set of predefined collision avoidance policy facilitated by the IA model which is trained by a virtuous cycle via a cloud computing.
18 . A method for facilitating a prediction of imminent machine failure via metadata generated by embedded sensors, comprising:
activating at least one embedded sensor attached to a mechanical component in a vehicle for collecting metadata from the mechanical component;
processing the metadata by a local pipelined processing unit to convert metadata format to a transferrable metadata format;
uploading the converted metadata to cloud via a wireless communication network for training a failure prediction model including analyzing the converted metadata according to a plurality of failure samples relating to the mechanical component; and
pushing a failure predication to the vehicle indicating an imminent machine failure.
19 . The method of claim 18 , further comprising:
activating a plurality of outward facing video cameras for aggregating a predefined set of images relating to external geographic environment in which the vehicle operates; and
uploading aggregated images to a cloud via a wireless communications network for detecting required maintenance.
20 . A method for facilitating identification of attentional state, comprising:
presenting to a first software application layer configured to build a first real-time object model representing a first contextual state of exterior of a vehicle;
presenting to a second software application layer configured to build a second real-time object model representing a second contextual state of interior of the vehicle;
generating a third software application to build a third real-time object model representing attentional state of occupants of the vehicle; and
determining an attentional state in accordance with relationships between the first and second contextual states and the attentional state of occupants.