Image generation devices and image processing systems for converting resolutions of image data captured by image sensors mounted on movable-body apparatuses

1. An image processing system comprising:

an image generation device including:

a receiver which, in operation, receives travel information about a travel state of a movable-body apparatus; and

a controller which, in operation, selects, based on the travel information received by the receiver, a first partial region from an entirety of a plurality of pixels of an image sensor or an entirety of image data captured by the image sensor, and generates image data in which a region other than the first partial region has a resolution lower than a resolution of the first partial region, the image sensor being configured to be mounted to the movable-body apparatus and being configured to capture an area in a traveling direction of the movable-body apparatus; and

an image processing device which, in operation, converts the resolution of the image data,

wherein the image processing device is configured to be connected to an active sensor which detects an object around the active sensor based on a return wave resulting from a predetermined wave emitted from the active sensor and then returning to the active sensor, the image processing device being operable to communicate with the active sensor, and

when the image processing device detects that a predetermined object is captured in the image data having the resolution converted by the image processing device, the image processing device controls the active sensor based on a detection result.

2. The image processing system according to claim 1 ,

wherein each of the plurality of pixels of the image sensor belongs to one of a plurality of segments that are predetermined or the entirety of image data is divided into the plurality of segments that are predetermined,

the controller determines a resolution of each of the plurality of segments based on the travel information to make a resolution of at least a part of the region other than the first partial region lower than the resolution of the first partial region,

the plurality of segments includes one or more segments belonging to the region other than the first partial region and one or more segments belonging to the first partial region, and

the controller generates image data in which a resolution of the one or more segments belonging to the region other than the first partial region is lower than a resolution of the one or more segments belonging to the first partial region.

3. The image processing system according to claim 2 ,

wherein the controller further generates image format information of the image data, and

the image format information includes at least a resolution of an image obtained from the image sensor, the resolution of the image data, a total number of the plurality of segments, and the resolution of each of the plurality of segments.

4. The image processing system according to claim 3 ,

wherein the image processing device converts the resolution of the image data based on the image format information.

5. The image processing system according to claim 3 ,

wherein the image generation device transmits the image format information to a transmission path in a vertical blanking period for the image data.

6. The image processing system according to claim 5 ,

wherein in the image data, at least two of the plurality of segments respectively have a first resolution and a second resolution that are different from each other; and

after performing image processing to emphasize a predetermined characteristic portion in the image data, the image processing device converts the resolution of each of the plurality of segments of the image data into an intermediate resolution having an intermediate value between the first resolution and the second resolution.

7. The image processing system according to claim 3 ,

wherein the image processing device includes deep learning layers corresponding to resolutions of the plurality of segments and converts the resolution of the image data based on the image format information while optimizing the deep learning layers by learning at least changes in the resolutions of the plurality of segments based on the travel information of the movable-body apparatus.

8. An image generation device comprising:

a receiver which, in operation, receives travel information about a travel state of a movable-body apparatus; and

a controller which, in operation, selects, based on the travel information received by the receiver, a first partial region from an entirety of a plurality of pixels of an image sensor or an entirety of image data captured by the image sensor, and generates image data in which a region other than the first partial region has a resolution lower than a resolution of the first partial region, the image sensor being configured to be mounted to the movable-body apparatus and being configured to capture an area in a traveling direction of the movable-body apparatus,

wherein each of the plurality of pixels of the image sensor belongs to one of a plurality of segments that are predetermined or the entirety of image data is divided into the plurality of segments that are predetermined,

the controller determines a resolution of each of the plurality of segments based on the travel information to make a resolution of at least a part of the region other than the first partial region lower than the resolution of the first partial region,

the plurality of segments includes one or more segments belonging to the region other than the first partial region and one or more segments belonging to the first partial region,

the controller generates the image data in which a resolution of the one or more segments belonging to the region other than the first partial region is lower than a resolution of the one or more segments belonging to the first partial region, and

when the controller determines, based on the travel information, that the movable-body apparatus has almost reached one of a top of an upslope and a bottom of a downslope, the controller sets, as the first partial region, one of the plurality of segments that includes a pixel receiving light at one of a downward angle and an upward angle of the movable-body apparatus among the plurality of pixels.

9. The image generation device according to claim 8 ,

wherein, when the controller determines, based on the travel information, that the movable-body apparatus is traveling straight, the controller sets, as the first partial region, one of the plurality of segments that includes a pixel receiving at least light from straight ahead among the plurality of pixels.

10. The image generation device according to claim 8 ,

wherein, when the controller determines, based on the travel information, that the movable-body apparatus is steering through a turn, the controller sets, as the first partial region, one of the plurality of segments that includes a pixel receiving light from a steering direction of the movable-body apparatus among the plurality of pixels.

11. The image generation device according to claim 8 ,

wherein the controller further includes a memory and develops image data of a plurality of frames in the memory, and the image data of the plurality of frames have total resolutions equal to one another.

12. An image generation device comprising:

a receiver which, in operation, receives travel information about a travel state of a movable-body apparatus; and

a controller which, in operation, selects, based on the travel information received by the receiver, a first partial region from an entirety of a plurality of pixels of an image sensor or an entirety of image data captured by the image sensor, and generates image data in which a region other than the first partial region has a resolution lower than a resolution of the first partial region, the image sensor being configured to be mounted to the movable-body apparatus and being configured to capture an area in a traveling direction of the movable-body apparatus,

wherein each of the plurality of pixels of the image sensor belongs to one of a plurality of segments that are predetermined or the entirety of image data is divided into the plurality of segments that are predetermined,

the controller determines a resolution of each of the plurality of segments based on the travel information to make a resolution of at least a part of the region other than the first partial region lower than the resolution of the first partial region,

the plurality of segments includes one or more segments belonging to the region other than the first partial region and one or more segments belonging to the first partial region,

the controller generates the image data in which a resolution of the one or more segments belonging to the region other than the first partial region is lower than a resolution of the one or more segments belonging to the first partial region,

the controller further generates image format information of the image data, and

the image format information includes at least a resolution of an image obtained from the image sensor, the resolution of the image data, a total number of the plurality of segments, and the resolution of each of the plurality of segments.

13. The image generation device according to claim 12 ,

wherein, when the controller determines, based on the travel information, that the movable-body apparatus is traveling straight, the controller sets, as the first partial region, one of the plurality of segments that includes a pixel receiving at least light from straight ahead among the plurality of pixels.

14. The image generation device according to claim 12 ,

wherein, when the controller determines, based on the travel information, that the movable-body apparatus is steering through a turn, the controller sets, as the first partial region, one of the plurality of segments that includes a pixel receiving light from a steering direction of the movable-body apparatus among the plurality of pixels.

15. The image generation device according to claim 12 ,

wherein the controller further includes a memory and develops image data of a plurality of frames in the memory, and the image data of the plurality of frames have total resolutions equal to one another.

16. An image processing system comprising:

an image generation device including:

a receiver which, in operation, receives travel information about a travel state of a movable-body apparatus; and

a controller which, in operation, selects, based on the travel information received by the receiver, a first partial region from an entirety of a plurality of pixels of an image sensor or an entirety of image data captured by the image sensor, and generates image data in which a region other than the first partial region has a resolution lower than a resolution of the first partial region, the image sensor being configured to be mounted to the movable-body apparatus and being configured to capture an area in a traveling direction of the movable-body apparatus; and

an image processing device which, in operation, converts the resolution of the image data,

wherein each of the plurality of pixels of the image sensor belongs to one of a plurality of segments that are predetermined or the entirety of image data is divided into the plurality of segments that are predetermined,

the controller determines a resolution of each of the plurality of segments based on the travel information to make a resolution of at least a part of the region other than the first partial region lower than the resolution of the first partial region,

the plurality of segments includes one or more segments belonging to the region other than the first partial region and one or more segments belonging to the first partial region,

the controller generates image data in which a resolution of the one or more segments belonging to the region other than the first partial region is lower than a resolution of the one or more segments belonging to the first partial region,

the controller further generates image format information of the image data, and

the image format information includes at least a resolution of an image obtained from the image sensor, the resolution of the image data, a total number of the plurality of segments, and the resolution of each of the plurality of segments.

17. The image processing system according to claim 16 ,

wherein the image processing device converts the resolution of the image data based on the image format information.

18. The image processing system according to claim 16 ,

wherein the image generation device transmits the image format information to a transmission path in a vertical blanking period for the image data.

19. The image processing system according to claim 18 ,

wherein in the image data, at least two of the plurality of segments respectively have a first resolution and a second resolution that are different from each other; and

after performing image processing to emphasize a predetermined characteristic portion in the image data, the image processing device converts the resolution of each of the plurality of segments of the image data into an intermediate resolution having an intermediate value between the first resolution and the second resolution.

20. The image processing system according to claim 16 ,

wherein the image processing device includes deep learning layers corresponding to resolutions of the plurality of segments and converts the resolution of the image data based on the image format information while optimizing the deep learning layers by learning at least changes in the resolutions of the plurality of segments based on the travel information of the movable-body apparatus.