Intelligent video conference cropping based on audio and vision
A video conferencing system for a vehicle includes a display device disposed in a passenger compartment of the vehicle, the display device comprising a screen, a first imager configured to capture a first image data corresponding to a field of view of a passenger compartment of the vehicle, at least one microphone configured to collect audio data from within the passenger compartment, and a controller. The controller is configured detect a display prompt within the image data corresponding to a portion of the image data received from the first imager, detect an audio prompt within a predetermined zone of the passenger compartment of the vehicle, and selectively crop the first image data according to the portion of the image data corresponding with the display prompt in response to determining a correlation between the portion of the image data and the zone of the passenger department associated with the audio prompt.
1. A video conferencing system for a vehicle comprising:
a first imager configured to capture a first image data corresponding to a field of view of a passenger compartment of the vehicle;
a plurality of microphones in respective predetermined locations corresponding with respective ones of a plurality of seating locations within the passenger compartment and configured to collect audio data from within the passenger compartment, wherein one of the plurality of seating locations corresponds with a driver seat location; and
a controller in communication with a vehicle network and configured to:
receive information related to a vehicle motion state from the vehicle network and determine that the vehicle is not in a motion state;
monitor for a display prompt within the image data corresponding to a portion of the image data received from the first imager;
monitor for an audio prompt at least primarily from one of the plurality of microphones within one of the seating locations;
determine a correlation between the portion of the image data and the seating area of the passenger compartment associated with the audio prompt, including if the seating area of the passenger compartment associated with the audio prompt is the driver seating location;
responsive to the vehicle being in a motion state, selectively crop the first image data according to the portion of the image data corresponding with the display prompt, only if the seating area of the passenger compartment associated with the audio prompt is not the driver seating location; and
responsive to the vehicle not being in a motion state, selectively crop the first image data according to the portion of the image data corresponding with the display prompt.
2. The video conferencing system of claim 1 , further comprising a display device disposed in a passenger compartment of the vehicle, the display device comprising a screen, wherein:
the controller is further configured to transmit the cropped portion of the image data and at least a portion of the audio data to an external conference server; and
the controller is further configured to receive video and audio data from the conference server.
3. The video conferencing system of claim 1 , wherein the display prompt corresponds to a detection of a movement of one or more features of an occupant of the vehicle identified in the first image data.
4. The video conferencing system of claim 1 , wherein the audio prompt corresponds to a detection of a sound identifiable as speech.
5. The video conferencing system of claim 1 , wherein the controller is further configured to detect at least one of a gesture and a motion of an occupant in the portion of image data to identify the display prompt.
6. The video conferencing system of claim 1 , wherein the portion of the image data and includes a facial region of the occupant.
7. The video conferencing system of claim 6 , wherein the portion of the image data includes an area surrounding the facial region according to one of a predetermined fixed border or a proportion of recognized features defining the facial region.
8. The video conferencing system of claim 1 , wherein the controller is further configured to digitally scale the selectively cropped first image data to an image size on an order of the first image data before the first image data is selectively cropped.