Systems, methods, and apparatuses for implementing an image sensor reconfiguration framework for seamless resolution-based tradeoffs

1. A method for performing a resolution reconfiguration of an image sensor hardware without dropping frames, wherein the method comprises:

accessing image device circuitry having the image sensor hardware embedded therein;

setting the image sensor hardware to operate utilizing a first sensor format;

requesting and mapping buffers for a vision data stream from the image sensor hardware in accordance with the first sensor format;

streaming the vision data stream from the image sensor hardware in accordance with the first sensor format;

receiving a resolution reconfiguration request to modify operation of the image sensor hardware from the first sensor format to a second sensor format; and

performing a parallel reconfiguration of the image sensor hardware, wherein the image device circuitry continues to process the vision data stream from the image sensor hardware in accordance with the first sensor format in parallel with the image device circuitry resetting the image sensor hardware to the second sensor format.

2. The method of claim 1 :

wherein performing the parallel reconfiguration of the image sensor hardware comprises transitioning streaming of the vision data stream from the image sensor hardware in accordance with the first sensor format to streaming the vision data stream from the image sensor in accordance with the second sensor format without dropping a single frame.

3. The method of claim 1 :

wherein performing the parallel reconfiguration of the image sensor hardware comprises transitioning streaming of the vision data stream via the first sensor format to streaming of the vision data stream via the second sensor format with frame-to-frame resolution reconfiguration latency being fully hidden from any application receiving the streaming of the vision data stream from the image sensor hardware.

4. The method of claim 1 , wherein performing the parallel reconfiguration of the image sensor hardware comprises:

executing a main thread for application processing of the vision data stream from the image sensor hardware;

spawning a new reconfiguration thread to process reconfiguration requests received; and

joining the main thread and the new reconfiguration thread before a capture thread for the image sensor hardware wakes to resume capturing frames from the image sensor.

5. The method of claim 1 , wherein performing the parallel reconfiguration of the image sensor hardware further comprises:

prohibiting reconfiguration of the image sensor hardware while a capture thread is actively capturing frames from the image sensor hardware.

6. The method of claim 5 , further comprising:

defining T interval as an interval between consecutive frame captures;

identifying T capture as the time required for each frame capture;

defining a reconfiguration timing budget T budget as a function of a frame interval T interval , where T budget is equal to frame interval T interval minus the required capture time T capture ; and

requiring the parallel reconfiguration of the image sensor hardware to complete within the defined reconfiguration timing budget T budget .

7. A mobile computing device, comprising:

a memory to store instructions;

a processor to execute instructions stored within the memory

an image device having an image sensor embedded therein;

wherein the instructions cause the mobile device to perform operations including:

opening the image device having the image sensor embedded therein;

setting the image sensor to a first sensor format;

requesting and mapping buffers for a vision data stream from the image sensor in accordance with the first sensor format;

streaming the vision data stream from the image sensor in accordance with the first sensor format;

receiving a resolution reconfiguration request to change the image sensor from the first sensor format to a second sensor format;

performing a parallel reconfiguration of the image sensor, wherein the image device continues to process the vision data stream from the image sensor in accordance with the first sensor format in parallel with the image device resetting the image sensor to the second sensor format; and

wherein performing the parallel reconfiguration of the image sensor comprises transitioning streaming of the vision data stream from the image sensor hardware in accordance with the first sensor format to streaming the vision data stream from the image sensor in accordance with the second sensor format without dropping a single frame.

8. The mobile computing device of claim 7 :

wherein performing the parallel reconfiguration of the image sensor comprises transitioning streaming of the vision data stream via the first sensor format to streaming of the vision data stream via the second sensor format with frame-to-frame resolution reconfiguration latency being fully hidden from any application receiving the streaming of the vision data stream from the image sensor.

9. The mobile computing device of claim 7 , wherein performing the parallel reconfiguration of the image sensor comprises:

executing a main thread for application processing of the vision data stream from the image sensor;

spawning a new reconfiguration thread to process reconfiguration requests received; and

joining the main thread and the new reconfiguration thread before a capture thread for the image sensor wakes to resume capturing frames from the image sensor.

10. The mobile computing device of claim 7 , wherein performing the parallel reconfiguration of the image sensor further comprises:

prohibiting reconfiguration of the image sensor while a capture thread is actively capturing frames from the image sensor.

11. The mobile computing device of claim 10 , wherein the instructions cause the mobile device to perform operations further including:

defining T interval as an interval between consecutive frame captures;

identifying T capture as the time required for each frame capture;

defining a reconfiguration timing budget T budget as a function of a frame interval T interval , where T budget is equal to frame interval T interval minus the required capture time T capture ; and

requiring the parallel reconfiguration of the image sensor to complete within the defined reconfiguration timing budget T budget .

12. Non-transitory computer readable storage media having instructions stored thereupon that, when executed by a processor of a image device, the instructions cause the image device to perform operations including:

opening the image device having an image sensor embedded therein;

setting the image sensor to a first sensor format;

requesting and mapping buffers for a vision data stream from the image sensor in accordance with the first sensor format;

streaming the vision data stream from the image sensor in accordance with the first sensor format;

receiving a resolution reconfiguration request to change the image sensor from the first sensor format to a second sensor format;

performing a parallel reconfiguration of the image sensor, wherein the image device continues to process the vision data stream from the image sensor in accordance with the first sensor format in parallel with the image device resetting the image sensor to the second sensor format; and

wherein performing the parallel reconfiguration of the image sensor comprises transitioning streaming of the vision data stream from the image sensor hardware in accordance with the first sensor format to streaming the vision data stream from the image sensor in accordance with the second sensor format without dropping a single frame.

13. The non-transitory computer readable storage media of claim 12 :

wherein performing the parallel reconfiguration of the image sensor comprises transitioning streaming of the vision data stream via the first sensor format to streaming of the vision data stream via the second sensor format with frame-to-frame resolution reconfiguration latency being fully hidden from any application receiving the streaming of the vision data stream from the image sensor.

14. The non-transitory computer readable storage media of claim 12 , wherein performing the parallel reconfiguration of the image sensor comprises:

executing a main thread for application processing of the vision data stream from the image sensor;

spawning a new reconfiguration thread to process reconfiguration requests received; and

joining the main thread and the new reconfiguration thread before a capture thread for the image sensor wakes to resume capturing frames from the image sensor.

15. The non-transitory computer readable storage media of claim 12 , wherein performing the parallel reconfiguration of the image sensor further comprises:

prohibiting reconfiguration of the image sensor while a capture thread is actively capturing frames from the image sensor.

16. The non-transitory computer readable storage media of claim 15 , wherein the instructions cause the image device to perform operations further comprising:

defining T interval as an interval between consecutive frame captures;

identifying T capture as the time required for each frame capture;

defining a reconfiguration timing budget T budget as a function of a frame interval T interval , where T budget is equal to frame interval T interval minus the required capture time T capture ; and

requiring the parallel reconfiguration of the image sensor to complete within the defined reconfiguration timing budget T budget .