System and method for insight-triggered opportunistic imaging in a vehicle
A system and method for insight-triggered opportunistic imaging in a vehicle are provided. In one embodiment, a plurality of images are captured opportunistically using an image capture device in a vehicle. A compressive sensing sampling technique is used to store a subset of image pixels of at least some of the plurality of images. A retention filter is applied to the plurality of images to select a subset of the plurality of images to retain based on a signal from a sensor in the vehicle. The retained image information can be sent to an entity external to the vehicle for processing.
1 . A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by one or more processors in a vehicle comprising an image capture device, cause the one or more processors to:
cause the image capture device to capture a plurality of images at a regular frequency;
cause the image capture device to capture a plurality of additional images in response to available processing time of the one or more processors being above a threshold; and
implement a retention filter to select a subset of the plurality of additional images to retain based on a signal from at least one sensor in the vehicle.
2 . The non-transitory computer-readable storage medium of claim 1 , wherein the plurality of images captured at the regular frequency are compressed using a lossy compression technique and the plurality of additional images are compressed using a lossless compression technique.
3 . The non-transitory computer-readable storage medium of claim 1 , wherein the at least one sensor comprises a lane departure sensor and/or a yaw sensor.
4 . The non-transitory computer-readable storage medium of claim 1 , wherein the at least one sensor comprises one or more of the following: a driver-facing image capture device, a radar, a deceleration sensor, a steering angle sensor, a wheel speed sensor, a brake pressure sensor, a vehicle load sensor, a tire pressure sensor, an image analyzer, and a location sensor.
5 . A non-transitory computer-readable storage medium storing computer-readable instructions that, when executed by one or more processors in a vehicle comprising an image capture device, cause the one or more processors to:
cause the image capture device to capture a plurality of images at a regular frequency;
cause the image capture device to capture a plurality of additional images in response to available processing time of the one or more processors being above a threshold; and
select a subset of the plurality of additional images to retain based on at least one signal from at least one sensor in the vehicle,
wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
store only a subset of image pixels of each image in the subset of the plurality of additional images.
6 . The non-transitory computer-readable storage medium of claim 5 , wherein the subset of image pixels of each image is sufficient to reconstruct each image from its subset of image pixels.
7 . The non-transitory computer-readable storage medium of claim 5 , wherein different subsets of image pixels are stored for at least some of the images in the subset of the plurality of additional images.
8 . The non-transitory computer-readable storage medium of claim 7 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
read a look-up table to identify which image pixels to store for each image of the at least some of the images; and
for each image of the at least some of the images, store an identifier that identifies which image pixels were stored.
9 . The non-transitory computer-readable storage medium of claim 5 , wherein the subset of image pixels of each image in the subset of the plurality of additional images is stored prior to selecting the subset of the plurality of additional images.
10 . The non-transitory computer-readable storage medium of claim 5 , wherein the subset of image pixels of each image in the subset of the plurality of additional images is stored after selecting the subset of the plurality of additional images.
11 . The non-transitory computer-readable storage medium of claim 5 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
store only a subset of image pixels of every image in the plurality of additional images.
12 . A method comprising:
performing in one or more processors in a vehicle:
capturing a plurality of images using an image capture device in the vehicle based on an available processing time of the one or more processors;
using a compressive sensing sampling technique to store a subset of image pixels of at least some of the plurality of images; and
applying a retention filter to the plurality of images to select a subset of the plurality of images to retain based on a signal from a sensor in the vehicle.
13 . The method of claim 12 , wherein the plurality of images are stored in a volatile buffer and the subset of the plurality of images are retained in non-volatile memory.
14 . The method of claim 12 , further comprising transmitting at least some of the subset of the plurality of images to an external processor for processing.
15 . The method of claim 12 , wherein the sensor in the vehicle comprises a lane departure sensor or a yaw sensor.
16 . The method of claim 12 , wherein the sensor in the vehicle comprises a driver-facing image capture device, a radar, a deceleration sensor, a steering angle sensor, a wheel speed sensor, a brake pressure sensor, a vehicle load sensor, a tire pressure sensor, an image analyzer, or a location sensor.
17 . The method of claim 12 , further comprising:
reading a look-up table to identify which image pixels to store for each image in the at least some of the plurality of images; and
for each image of the at least some of the plurality of images, storing an identifier that identifies which image pixels were stored.
18 . The method of claim 12 , further comprising:
capturing a plurality of images at a regular frequency as part of an event detection system.
19 . The method of claim 18 , wherein the plurality of images captured at the regular frequency are compressed using a lossy compression technique, and wherein the compressive sensing sampling technique provides lossless compression.
20 . A system comprising:
means for causing an image capture device in a vehicle to capture a plurality of images based on an available processing time of one or more processors that capture an image;
means for sampling a subset of image pixels of at least some of the plurality of images; and
means for filtering the plurality of images by selecting a subset of the plurality of images to retain based on a signal from a sensor in the vehicle.
21 . An opportunistic data capture system with subsequent curation, the system comprising:
a memory; and
one or more processors configured to:
capture data in response to available processing time; and
apply a retention filter to the captured data to select a subset of the captured data to retain in the memory based on a signal from a sensor in a vehicle.
22 . The system of claim 21 , wherein the data comprises image data.