IP Library › Granted Patent US 12,430,869
Granted Patent B2
US 12,430,869 · App. 17/742,694 · Granted Sep 30, 2025

Image sensor and method of operating the same

Inventors: Chih-Cheng Hsieh (Hsinchu, TW); Min-Yang Chiu (Taichung, TW)
Assignees: TAIWAN SEMICONDUCTOR MANUFACTURING COMPANY, LTD.; NATIONAL TSING HUA UNIVERSITY
G06V10/25G06T7/11G06T7/20H04N19/17H04N25/40
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,430,869
App. No.
17/742,694
Granted
Sep 30, 2025
Kind
B2
Abstract

A method is provided to operate an image sensor, including the following steps: comparing a first threshold with multiple first frame-difference signals to generate a first region of interest (ROI) address, wherein each of multiple first frame-difference signals is generated by a corresponding pixel, in multiple pixels, operating during a first time frame and a second time frame, wherein multiple pixels are arranged in multiple columns and multiple rows; identifying edge columns and edge rows in the first ROI address; adjusting, according to the edge columns and the edge rows, the first ROI address to generate a second ROI address; and outputting a ROI image according to the second ROI address and multiple second frame-difference signals generated by multiple pixels operating during the second time frame and a third time frame after the second time frame.

Claims (71)

1. A method, comprising:

comparing a first threshold with a plurality of first frame-difference signals to generate a first region of interest (ROI) address, wherein each of the plurality of first frame-difference signals is generated by a corresponding pixel, in a plurality of pixels, operating during a first time frame and a second time frame, wherein the plurality of pixels are arranged in a plurality of columns and a plurality of rows;

identifying edge columns and edge rows in the first ROI address, wherein identifying the edge columns and the edge rows comprises:

identifying a first column and a second column, as the edge columns, in a first column address of M bits in the first ROI address, wherein the first column corresponds to a K-th bit and the second column corresponding to a L-th bit in the first column address,

wherein a first bit to a (K−1)-th bit and a (L+1)-th bit to a M-th bit in the first column address have a first bit number different from a second bit number of the K-th and L-th bits; and

identifying a first row and a second row, as the edge rows, in a first row address of N bits in the first ROI address, wherein the first row corresponds to an I-th bit and the second row corresponding to a J-th bit in the first row address,

wherein a first bit to an (I−1)-th bit and a (J+1)-th bit to a N-th bit in the first row address have the first bit number, and the I-th and J-th bits have the second bit number;

adjusting, according to the edge columns and the edge rows, the first ROI address to generate a second ROI address; and

outputting a ROI image according to the second ROI address and a plurality of second frame-difference signals generated by the plurality of pixels operating during the second time frame and a third time frame after the second time frame.

2. The method of claim 1 , wherein the plurality of first frame-difference signals are pulse-width modulation (PWM) signals.

3. The method of claim 1 , further comprising:

generating a first value in response to the first threshold being less than a corresponding one of the plurality of first frame-difference signals for the corresponding pixel.

4. The method of claim 3 , further comprising:

generating each one of a plurality of column sum values by adding up the first value corresponding to pixels arranged in a corresponding column of the plurality of columns, and

generating each one of a plurality of row sum values by adding up the first value corresponding to pixels arranged in a corresponding row of the plurality of rows.

5. The method of claim 4 , further comprising:

comparing the plurality of column sum values with a second threshold to generate a column address in the first ROI address, and

comparing the plurality of row sum values with a third threshold to generate a row address in the first ROI address.

6. The method of claim 1 , wherein adjusting the first ROI address to generate the second ROI address comprises:

replacing bit numbers of a (K+1)-th bit to a (L−1)-th bit in the first column address with the second bit number to generate a second column address in the second ROI address, and

replacing bit numbers of an (I+1)-th bit to a (J−1)-th bit in the first row address with the second bit number to generate a second row address in the second ROI address.

7. The method of claim 1 , further comprising:

increasing the first threshold in response to the ROI image.

8. A method, comprising:

comparing a first threshold with a plurality of first frame-difference signals each generated by a corresponding pixel, in a plurality of pixels, operating during a first time frame and a second time frame, wherein the plurality of pixels are arranged in a plurality of columns and a plurality of rows;

in response to the comparison, generating a plurality of values corresponding to the plurality of pixels separately;

accumulating the plurality of values in a column-by-column manner to generate a column matrix, and accumulating the plurality of values a row-by-row manner to generate a row matrix;

quantizing the column matrix to generate a column binary sequence, and quantizing the row matrix to generate a row binary sequence; and

generating, according to the column binary sequence and the row binary sequence, a region of interest (ROI) image that corresponds to the second time frame and a third time frame after the second time frame.

9. The method of claim 8 , wherein generating the plurality of values comprises:

outputting a first value in response to the first threshold being less than a corresponding one of the plurality of first frame-difference signals for the corresponding pixel, and

outputting a second value in response to the first threshold being greater than a corresponding one of the plurality of first frame-difference signals for the corresponding pixel.

10. The method of claim 8 , wherein quantizing the column matrix and quantizing the row matrix comprise:

comparing values in the column matrix with a second threshold to generate the column binary sequence, and

comparing values in the row matrix with a third threshold to generate the row binary sequence,

wherein the second threshold and the third threshold are different from each other.

11. The method of claim 10 , further comprising:

adjusting the first threshold, the second threshold, and the third threshold according to the ROI image.

12. The method of claim 8 , further comprising:

adjusting the column binary sequence by identifying a first column and a second column in the column binary sequence of M bits,

wherein the first column corresponds to a K-th bit and the second column corresponding to a L-th bit in the column binary sequence, and

a first bit to a (K−1)-th bit and a (L+1)-th bit to a M-th bit in the column binary sequence have a first bit number different from a second bit number of the K-th and L-th bits; and

replacing bit numbers of a (K+1)-th bit to a (L−1)-th bit in the column binary sequence to the second bit number.

13. The method of claim 12 , further comprising:

adjusting the row binary sequence by identifying a first row and a second row in the row binary sequence of N bits,

wherein the first row corresponds to an I-th bit and the second row corresponding to a J-th bit in the row binary sequence, and

a first bit to an (I−1)-th bit and a (J+1)-th bit to an N-th bit in the row binary sequence have the first bit number, and the I-th and J-th bits have the second bit number; and

replacing bit numbers of an (I+1)-th bit to a (J−1)-th bits in the row binary sequence to the second bit number.

14. The method of claim 8 , further comprising:

adjusting the column binary sequence by identifying a first column and a second column in the column binary sequence of M bits,

wherein the first column corresponds to a (K−3)-th bit and the second column corresponding to a (L+3)-th bit in the column binary sequence, and

a first bit to a (K−1)-th bit and a (L+1)-th bit to an M-th bit in the column binary sequence have a first bit number different from a second bit number of K-th and L-th bits; and

replacing bit numbers of a (K−2)-th bit to a (L+2)-th bit in the column binary sequence with the second bit number.

15. A method, comprising:

generating a first region of interest (ROI) image according to a plurality of first frame-difference signals and a first ROI address;

generating a second ROI address based on the plurality of first frame-difference signals, wherein generating the second ROI address comprises:

comparing a first threshold with the plurality of first frame-difference signals to generate a plurality of values corresponding to a plurality of pixels separately;

accumulating the plurality of values in a column-by-column manner to generate a column matrix, and quantizing the column matrix to generate a column binary sequence of the second ROI address; and

accumulating the plurality of values in a row-by-row manner to generate a row matrix, and quantizing the row matrix to generate a row binary sequence of the second ROI address; and

generating a second ROI image according to a plurality of second frame-difference signals and the second ROI address.

16. The method of claim 15 , wherein the plurality of first frame-difference signals are generated by a plurality of pixels operating during a first time frame and a second time frame, and

the plurality of second frame-difference signals are generated by the plurality of pixels operating during the second time frame and a third time frame after the second time frame.

17. The method of claim 15 , further comprising:

adjusting the second ROI address based on the first ROI image.

18. The method of claim 15 , further comprising:

identifying a first column and a second column, as edge columns, in a first column address of M bits in the first ROI address, wherein the first column corresponds to a (K−q)-th bit and the second column corresponding to a (L+q)-th bit in the first column address,

wherein a first bit to a (K−1)-th bit and a (L+1)-th bit to an M-th bit in the first column address have a first bit number different from a second bit number of the K-th and L-th bits; and

adjusting, according to the edge columns, the first ROI address.

19. The method of claim 18 , wherein q is a positive number.

20. The method of claim 15 , further comprising:

increasing the first threshold in response to the second ROI image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 12, 2022
From: HSIEH, CHIH-CHENG; CHIU, MIN-YANG
To: TAIWAN SEMICONDUCTOR MANUFACTURING COMPANY, LTD.; NATIONAL TSING HUA UNIVERSITY
Reel/Frame 060021/0194 →
Continuity (1)
Related Publication 20230368486A1 · Nov 16, 2023
References Cited (8)
US 6718055B1 · Suri · 2004 [cited by examiner]
US 11551099B1 · Senthil · 2023 [cited by examiner]
US 20020141615A1 · Mcveigh · 2002 [cited by examiner]
US 20140368661A1 · Angot · 2014 [cited by examiner]
US 20220141401A1 · Kim · 2022 [cited by examiner]
Choi et al., “A Spatial-Temporal Multi-Resolution CMOS Image Sensor with Adaptive Frame Rates for Moving Objects in the Region-of-Interest”, 2007 IEEE International Solid-State Circuits Conference, Total 3 pages. [cited by applicant]
Lin et al., “A Dual-Mode CMOS Imager for Free-Space Optical Communication with Signal Light Source Tracking and Background Cancellation”, 2015 International Image Sensor Workshop (IISW). Total 5 pages. [cited by applicant]
Verdant et al., “A 120 μW 240×110@25fps vision chip with ROI detection SIMD processing unit”, 2013 IEEE International Symposium on Circuits and Systems (ISCAS) pp. 2412-2415. [cited by applicant]