Capturing and processing content for in-vehicle and extended display, curated journaling, and real-time assistance during vehicular travel
Methods and systems are described for imaging and content generation. During a road trip, a driver or passenger observes an object or scene from a vehicle. From a perspective of the driver or passenger, the object or scene flows out of view in a relatively short amount of time. Images and video of the object or scene are captured with cameras. A field of view is extended beyond what the driver or passenger can easily observe. The capture is configurable to focus on particular types of objects or scenes. Recordings of billboards and road signs are easily displayed after passing the object. Object identification, gaze determination, and interest determination are provided. Curated content is generated. Applications to extended reality environments are provided. Artificial intelligence systems, including neural networks, and models are utilized to improve the imaging and content generation. Related apparatuses, devices, techniques, and articles are also described.
1 . A method for controlling media acquisition by an imaging device, the imaging device positioned to capture media external to a vehicle, the method comprising:
instructing the imaging device to acquire the media;
identifying an object in the acquired media;
determining whether the identified object visually conveys information;
determining whether the information is of interest to a person in the vehicle; and
in response to determining the information is of interest to the person in the vehicle:
generating content related to the identified object, wherein:
the related content comprises an image overlay to be displayed on a smart rear-view mirror and display (SRVMD) of the vehicle,
the image overlay comprises an inverted view of the identified object, and
the inverted view is generated based on one or more images of the identified object captured by one or more front-facing cameras; and
generating the related content for display on the SRVMD of the vehicle.
2 . The method of claim 1 , comprising:
determining a gaze of the person in the vehicle; and
determining whether the gaze of the person is focused on the identified object.
3 . The method of claim 2 , comprising:
based at least in part on determining that the gaze of the person is focused on the identified object:
performing the generation of the related content; and
performing the generation of the related content for display on the device SRVMD of the vehicle.
4 . The method of claim 1 , comprising:
acquiring a current location of the vehicle;
acquiring, from an external source, a location of the identified object; and
in response to determining that the location of the identified object is proximate to the current location of the vehicle, generating the related content by accessing, from the external source, information related to the identified object.
5 . The method of claim 1 , comprising:
generating for display a user-selectable icon, wherein the user-selectable icon is generated for display on or near the display of the generated related content;
in response to user selection of the user-selectable icon on or near the display of the generated related content, generating for display at least one of:
additional content related to the generated related content;
interactive material related to the generated related content; or
turn-by-turn directions to a real-world location associated with the identified object.
6 . The method of claim 1 , comprising:
identifying in real time or in near real time a safety-related object in the acquired media; and
determining whether the display of the related content obscures the identified safety-related object.
7 . The method of claim 6 , comprising:
in response to determining that the display of the related content obscures the identified safety-related object, altering the generating content related to the at least one of the identified object, or the information, and altering the generating the related content for display on the display device in the vehicle for viewing by the person in the vehicle.
8 . The method of claim 1 , wherein the identified object comprises at least one of a license plate, a road sign, a traffic light, a street name sign, a neon sign, a digital sign, a building sign, a sandwich board, a vehicle wrap, a banner, a billboard, or building wrap.
9 . The method of claim 1 , wherein the related content displayed on the SRVMD comprises an user-selectable icon.
10 . The method of claim 1 , wherein the SRVMD is configured to display a video stream of one or more cameras to reproduce the scene around the vehicle.
11 . A system for controlling media acquisition by an imaging device, the imaging device positioned to capture media external to a vehicle, and a display device, the system comprising:
control circuitry configured to:
instruct the imaging device to acquire the media;
identify an object in the acquired media;
determine whether the identified object visually conveys information;
determine whether the information is of interest to a person in the vehicle; and
in response to determining the information is of interest to the person in the vehicle:
generate content related to the identified object, wherein:
the related content comprises an image overlay to be displayed on a smart rear-view mirror and display (SRVMD) of the vehicle,
the image overlay comprises an inverted view of the identified object, and
the inverted view is generated based on one or more images of the identified object captured by one or more front-facing cameras; and
generate the related content for display on the SRVMD of the vehicle.
12 . The system of claim 11 , wherein the control circuitry is configured to:
determine a gaze of the person in the vehicle; and
determine whether the gaze of the person is focused on the identified object.
13 . The system of claim 12 , wherein the control circuitry is configured to:
based at least in part on determining that the gaze of the person is focused on the identified object:
perform the generation of the related content; and
perform the generation of the related content for display on the SRVMD of the vehicle.
14 . The system of claim 11 , comprising:
an external source,
wherein the control circuitry is configured to:
acquire a current location of the vehicle;
acquire, from an external source, a location of the identified object; and
in response to determining that the location of the identified object is proximate to the current location of the vehicle, generate the related content by accessing, from the external source, information related to the identified object.
15 . The system of claim 11 , wherein the control circuitry is configured to:
generate for display a user-selectable icon, wherein the user-selectable icon is generated for display on or near the display of the generated related content;
in response to user selection of the user-selectable icon on or near the display of the generated related content, generate for display at least one of:
additional content related to the generated related content;
interactive material related to the generated related content; or
turn-by-turn directions to a real-world location associated with the identified object.
16 . The system of claim 11 , wherein the control circuitry is configured to:
identify in real time or in near real time a safety-related object in the acquired media; and
determine whether the display of the related content obscures the identified safety-related object.
17 . The system of claim 16 , wherein the control circuitry is configured to:
in response to determining that the display of the related content obscures the identified safety-related object, alter the generating content related to the at least one of the identified object, or the information, and alter the generating the related content for display on the display device in the vehicle for viewing by the person in the vehicle.
18 . The system of claim 11 , wherein the identified object comprises at least one of a license plate, a road sign, a traffic light, a street name sign, a neon sign, a digital sign, a building sign, a sandwich board, a vehicle wrap, a banner, a billboard, or building wrap.
19 . The system of claim 11 , wherein the related content displayed on the SRVMD comprises an user-selectable icon.
20 . The system of claim 11 , wherein the SRVMD is configured to display a video stream of one or more cameras to reproduce the scene around the vehicle.