IP Library › Granted Patent US 10,785,419
Granted Patent B2
US 10,785,419 · App. 16/257,277 · Granted Sep 22, 2020

Light sensor chip, image processing device and operating method thereof

Inventor: Guo-Zhen Wang (Hsin-Chu County, TW)
Assignee: PIXART IMAGING INC.
H04N5/2353G06T5/002H04N5/2351H04N5/2355H04N5/23229H04N5/357H04N5/35581G06N20/00G06T2207/20182H04N5/374
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,785,419
App. No.
16/257,277
Granted
Sep 22, 2020
Kind
B2
Abstract

There is provided an image processing device including a light sensor and a processor. The light sensor is used to detect light and output an image frame. The processor identifies intensity of ambient light according to an image parameter associated with the image frame. When the ambient light is identified to be strong enough, the processor performs an object identification directly using the image frame. When the ambient light is identified to be not enough, the processor firstly converts the image frame to a converted image using a machine learning model, and then performs the object identification using the converted image.

Claims (46)

1. A light sensor chip, comprising:

a light sensor configured to detect light using a first exposure time to output a first image; and

a processor electrically connected to the light sensor to receive the first image, and configured to identify an operating mode according to an image parameter associated with the first image, wherein

when identifying that the operating mode is a strong light mode, the processor is configured to output the first image, and

when identifying that the operating mode is a weak light mode, the processor is configured to

convert the first image into a converted image using a pre-stored learning model, and then output the converted image,

control the light sensor to detect light using a second exposure time to output a second image, wherein the second exposure time is longer than the first exposure time, and

convert, using the pre-stored learning model, the second image into another converted image, and then output the another converted image.

2. The light sensor chip as claimed in claim 1 , wherein the learning model is trained by a data network architecture based on a ground truth image.

3. The light sensor chip as claimed in claim 1 , wherein

an image quality, a contrast or a clarity of the converted image is higher than that of the first image, or

a blurring of the converted image is lower than that of the first image.

4. The light sensor chip as claimed in claim 1 , wherein the image parameter comprises at least one of an image brightness, a gain value, a convergence time of auto exposure and an image quality.

5. The light sensor chip as claimed in claim 1 , wherein the processor does not output the first image in the weak light mode.

6. The light sensor chip as claimed in claim 1 , wherein when a blurring of the second image is higher than a blur threshold, the processor is further configured to shorten the second exposure time.

7. An image processing device, comprising:

a light sensor chip configured to detect light using a first exposure time to output a first image; and

an electronic device coupled to the light sensor chip, and comprising a processor configured to identify an operating mode according to an image parameter associated with the first image, wherein

when identifying that the operating mode is a strong light mode, the processor is configured to perform an object identification using the first image, and

when identifying that the operating mode is a weak light mode, the processor is configured to

convert the first image into a converted image using a pre-stored learning model, and then perform the object identification using the converted image,

when an image feature of the first image is lower than a predetermined threshold, the processor is further configured to

control the light sensor chip to detect light using a second exposure time to output a second image, wherein the second exposure time is longer than the first exposure time, and

convert, using the pre-stored learning model, the second image into another converted image, and then perform the object identification using the another converted image.

8. The image processing device as claimed in claim 7 , wherein the learning model is trained by a data network architecture based on a ground truth image.

9. The image processing device as claimed in claim 7 , wherein

an image quality, a contrast or a clarity of the converted image is higher than that of the first image, or

a blurring of the converted image is lower than that of the first image.

10. The image processing device as claimed in claim 7 , wherein the image parameter comprises at least one of an image brightness, a gain value, a convergence time of auto exposure and an image quality.

11. The image processing device as claimed in claim 7 , wherein the processor does not use the first image to perform the object identification in the weak light mode.

12. The image processing device as claimed in claim 7 , wherein when a blurring of the second image is higher than a blur threshold, the processor is further configured to shorten the second exposure time.

13. An operating method of an image processing device, the image processing device comprising a light sensor and a processor coupled to each other, the method comprising:

detecting, by the light sensor, light using a first exposure time to output a first image;

comparing, by the processor, an image parameter associated with the first image with a parameter threshold;

directly using the first image to perform an object identification when the image parameter exceeds the parameter threshold;

converting, using a pre-stored learning model, the first image into a converted image and then performing the object identification using the converted image when the image parameter does not exceed the parameter threshold; and

when an image feature of the first image is lower than a predetermined threshold,

controlling the light sensor to detect light using a second exposure time to output a second image, wherein the second exposure time is longer than the first exposure time, and

converting, using the pre-stored learning model, the second image into another converted image, and then performing the object identification using the another converted image.

14. The operating method as claimed in claim 13 , wherein the learning model is trained by a data network architecture based on a ground truth image.

15. The operating method as claimed in claim 13 , wherein

an image quality, a contrast or a clarity of the converted image is higher than that of the first image, or

a blurring of the converted image is lower than that of the first image.

16. The operating method as claimed in claim 13 , wherein the image parameter comprises at least one of an image brightness, a gain value, a convergence time of auto exposure and an image quality.

17. The operating method as claimed in claim 13 , further comprising:

not using the first image to perform the object identification when the image parameter does not exceed the parameter threshold.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 25, 2019
From: WANG, GUO-ZHEN
To: PIXART IMAGING INC.
Reel/Frame 048133/0879 →
Continuity (1)
Related Publication 20200244861A1 · Jul 30, 2020