Learning system, learning method, learning program, and learning and recognition system for image recognition
An information processing apparatus according to an embodiment includes a generation part that generates a signal corresponding to a first signal read from a first sensor. The signal is generated based on a second signal read from a second sensor different from the first sensor in at least one of a reading unit, a pixel characteristic, and a signal characteristic.
1 . An information processing apparatus comprising:
circuitry including a CPU that is configured to
generate a signal corresponding to a first signal read from a first image sensor that has a first reading unit, a first pixel characteristic, and a first signal characteristic, the signal being generated based on a second signal read from a second image sensor that has a second reading unit, a second pixel characteristic, and a second signal characteristic, wherein
the first reading unit is one frame and the second reading unit is smaller than the one frame, and
the circuitry is further configured to
accumulate the second signal for a period of the one frame,
generate a first generated signal corresponding to the first signal based on the second signal that was accumulated,
generate a second generated signal corresponding to the first signal by combining the second signal acquired in the period of the one frame, and
generate the signal corresponding to the first signal by integrating the first generated signal and the second generated signal with a weight, the weight being determined according to at least one of resolution, reliability, and processing delay, relating to the first generated signal and the second generated signal.
2 . The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to generate the signal corresponding to the first signal by converting the second signal according to the first reading unit.
3 . The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to generate the signal corresponding to the first signal when a predetermined amount of the second signal is accumulated.
4 . The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to generate control information for controlling a first recognizer that is configured to perform a recognition process based on the first signal, based on reference information for controlling a second recognizer that is configured to perform a recognition process based on the second signal, the reference information being output according to the first signal from the first recognizer.
5 . The information processing apparatus according to claim 4 , wherein
the reference information is information indicating an attention region extracted based on a feature amount output from a predetermined layer included in the first recognizer, and
the circuitry is further configured to generate the control information based on information extracted from the attention region in a dataset for the second recognizer to perform the recognition process based on the reference information.
6 . The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to convert a second dataset including second training data for training a second recognizer configured to perform a recognition process based on the second signal into a first dataset including first training data for training a first recognizer that is configured to perform a recognition process based on the first signal by approximating the second pixel characteristic or the second signal characteristic to the first pixel characteristic or the first signal characteristic when there is a lack of information on the second pixel characteristic or the second signal characteristic with respect to the first pixel characteristic or the first signal characteristic.
7 . The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to convert a second dataset including second training data for training a second recognizer that is configured to perform a recognition process based on the second signal into a first dataset including first training data for training a first recognizer that is configured to perform a recognition process based on the first signal by estimating missing information due to a lack of information when there is the lack of information on the second pixel characteristic or the second signal characteristic with respect to the first pixel characteristic or the first signal characteristic.
8 . The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to convert the second pixel characteristic or the second signal characteristic into the first pixel characteristic or the first signal characteristic based on preset information when a correspondence relationship between the first pixel characteristic or the first signal characteristic and the second pixel characteristic or the second signal characteristic is unknown.
9 . The information processing apparatus according to claim 8 , wherein
the circuitry is further configured to use at least one of a noise characteristic and a signal processing pipeline as the preset information.
10 . The information processing apparatus according to claim 1 , wherein
the circuitry is further configured to estimate the first pixel characteristic or the first signal characteristic into which the second pixel characteristic or the second signal characteristic is converted when a correspondence relationship between the first pixel characteristic or the first signal characteristic and the second pixel characteristic or the second signal characteristic is unknown.
11 . The information processing apparatus according to claim 10 , wherein
the circuitry is further configured to estimate a noise characteristic, and convert the second pixel characteristic or the second signal characteristic into the first pixel characteristic or the first signal characteristic by using the noise characteristic estimated.
12 . The information processing apparatus according to claim 10 , wherein
the circuitry is further configured to estimate a signal processing pipeline, and convert the second pixel characteristic or the second signal characteristic into the first pixel characteristic or the first signal characteristic by using the signal processing pipeline estimated.
13 . The information processing apparatus according to claim 1 , wherein
the second pixel characteristic and the first pixel characteristic are at least one of optical linearity, a noise characteristic, a bit length, presence or absence of high dynamic range composition, a static gradation characteristic, and a shading characteristic of the second signal and the first signal.
14 . An information processing method implemented by a processor, the information processing method comprising:
generating a signal corresponding to a first signal read from a first image sensor that has a first reading unit, a first pixel characteristic, and a first signal characteristic, the signal being generated based on a second signal read from a second image sensor that has a second reading unit, a second pixel characteristic, and a second signal characteristic, wherein
the first reading unit is one frame and the second reading unit is smaller than the one frame, and
the method further comprises,
accumulating the second signal for a period of the one frame,
generating a first generated signal corresponding to the first signal based on the second signal that was accumulated,
generating a second generated signal corresponding to the first signal by combining the second signal acquired in the period of the one frame, and
generating the signal corresponding to the first signal by integrating the first generated signal and the second generated signal with a weight, the weight being determined according to at least one of resolution, reliability, and processing delay, relating to the first generated signal and the second generated signal.
15 . A non-transitory computer-readable medium storing an information processing program causing a processor to implement:
generating a signal corresponding to a first signal read from a first image sensor that has a first reading unit, a first pixel characteristic, and a first signal characteristic, the signal being generated based on a second signal read from a second image sensor that has a second reading unit, a second pixel characteristic, and a second signal characteristic, wherein
the first reading unit is one frame and the second reading unit is smaller than the one frame, and
the information processing program further causing the processor to implement,
accumulating the second signal for a period of the one frame,
generating a first generated signal corresponding to the first signal based on the second signal that was accumulated,
generating a second generated signal corresponding to the first signal by combining the second signal acquired in the period of the one frame, and
generating the signal corresponding to the first signal by integrating the first generated signal and the second generated signal with a weight, the weight being determined according to at least one of resolution, reliability, and processing delay, relating to the first generated signal and the second generated signal.
16 . An information processing system comprising:
learning circuitry including a CPU that is configured to generate a signal corresponding to a first signal read from a first image sensor that has a first reading unit, a first pixel characteristic, and a first signal characteristic, the signal being generated based on a second signal read from a second image sensor that has a second reading unit, a second pixel characteristic, and a second signal characteristic; and
recognition circuitry including a recognizer configured to perform a recognition process based on the signal corresponding to the first signal generated by the circuitry, wherein
the first reading unit is one frame and the second reading unit is smaller than the one frame, and
the circuitry is further configured to
accumulate the second signal for a period of the one frame,
generate a first generated signal corresponding to the first signal based on the second signal that was accumulated,
generate a second generated signal corresponding to the first signal by combining the second signal acquired in the period of the one frame, and
generate the signal corresponding to the first signal by integrating the first generated signal and the second generated signal with a weight, the weight being determined according to at least one of resolution, reliability, and processing delay, relating to the first generated signal and the second generated signal.