Image signal processing

1. A method of optimising an image signal processor (ISP), which is to be used to process sensor image data generating output image data, the method comprising:

receiving test image data,

processing the test image data using a simulator to generate simulated sensor image data, wherein the simulator simulates a physical capture process of a scene by a sensor;

processing the simulated sensor image data according to one or more ISP settings to produce output image data;

producing quality metric data associated with the output image data by performing a comparison between the output image data and the test image data; and

optimising the one or more ISP settings based on the quality metric data.

2. The method of claim 1 , wherein the method comprises:

inputting the output image data into a computer vision algorithm;

processing the output image data using the computer vision algorithm to generate one or more output labels; and

comparing test labels associated with the test image data to the output labels.

3. The method of claim 2 , wherein the method comprises processing the output image data using a classification neural network.

4. The method of claim 1 , wherein the method comprises optimising a plurality of the ISP settings using at least one of:

a single setting at a time; and

a group of related settings at a time.

5. The method of claim 1 , wherein the method comprises optimising one or more ISP parameters using at least one of:

a line search; and

a Bayesian optimisation method.

6. A method of optimising a simulator, which is to be used to process test image data to generate simulated sensor image data, the method comprising:

receiving test image data consistent with a scene;

processing the test image data according to one or more simulator settings to produce simulated sensor image data, wherein the simulator simulates a physical capture process of a scene by a sensor;

capturing the scene using the sensor to produce sensor image data;

performing a comparison between the simulated sensor image data and sensor image data;

producing quality metric data associated with the comparison; and

optimising one or more simulator settings based on the quality metric data.

7. The method of claim 6 , wherein the method comprises optimising a plurality of the simulator settings using at least one of:

a single setting at a time; and

a group of related settings at a time.

8. The method of claim 6 , wherein the method comprises optimising the one or more simulator settings using at least one of:

a line search method; and

a Bayesian optimisation method.

9. The method of claim 6 , wherein the method comprises at least one of:

performing an alignment between the simulated sensor image data and sensor image data; or

performing the comparison using a cost function, and determining a cost associated with the cost function.

10. An image processing system comprising:

memory circuitry arranged to store test image data, one or more settings associated with an ISP and output image data; and

processing circuitry configured to:

receive test image data;

process the test image data using a simulator to generate simulated sensor image data, wherein the simulator simulates a physical capture process of a scene by a sensor;

process the simulated sensor image data according to one or more ISP settings to produce output image data;

produce a quality metric associated with the output image data by performing a comparison between the output image data and the test image data; and

optimise one or more settings associated with the ISP based on the quality metric.

11. The system of claim 10 , wherein the processing circuitry is arranged to:

input the output image data into a computer vision algorithm;

process the output image data using the computer vision algorithm to generate one or more output labels; and

compare test labels associated with the test image data to the output labels.

12. An image processing system comprising:

memory circuitry arranged to store test image data, one or more settings associated with a simulator, simulated sensor image data and sensor image data; and

processing circuitry configured to:

receive test image data consistent with a scene;

process the test image data according to one or more simulator settings to produce simulated sensor image data, wherein the simulator simulates a physical capture process of a scene by a sensor;

capture the scene using the sensor to produce sensor image data;

perform a comparison between the simulated sensor image data and sensor image data;

produce quality metric data associated with the comparison; and

optimise one or more simulator settings based on the quality metric data.

13. The system of claim 12 , wherein the processing circuitry is arranged to optimise at least one of:

a plurality of the simulator settings using at least one of a single setting at a time, or a group of related settings at a time; or

the one or more simulator settings using at least one of a line search method, or a Bayesian optimisation method.

14. The system of claim 12 , wherein the processing circuitry is arranged to at least one of:

perform an alignment between the simulated sensor image data and sensor image data; or

perform the comparison using a cost function, and determine a cost associated with the cost function.

15. A non-transitory computer readable storage medium comprising instructions which, when executed by a computer, cause the computer to:

receive test image data,

process the test image data using a simulator to generate simulated sensor image data, wherein the simulator simulates a physical capture process of a scene by a sensor;

process the simulated sensor image data according to one or more ISP settings to produce output image data;

produce quality metric data associated with the output image data by performing a comparison between the output image data and the test image data; and

optimise the one or more ISP settings based on the quality metric data.

16. A non-transitory computer readable storage medium comprising instructions which, when executed by a computer, cause the computer to:

receive test image data consistent with a scene;

process the test image data according to one or more simulator settings to produce simulated sensor image data, wherein the simulator simulates a physical capture process of a scene by a sensor;

capture the scene using the sensor to produce sensor image data;

perform a comparison between the simulated sensor image data and sensor image data;

produce quality metric data associated with the comparison; and

optimise one or more simulator settings based on the quality metric data.