IP Library Granted Patent US 11,076,071
Granted Patent B2
US 11,076,071 · App. 16/406,194 · Granted Jul 27, 2021

Device having a camera overlaid by display and method for implementing same

Inventors: John Weldon Nicholson (Cary, NC); Daryl Cromer (Cary, NC); Howard J. Locker (Cary, NC)
Assignee: LENOVO (SINGAPORE) PTE. LTD.
H04N5/2252G06F3/04812G06F9/3004H01L27/14603H04N5/23229
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,076,071
App. No.
16/406,194
Granted
Jul 27, 2021
Kind
B2
Abstract

A device, method and computer program product are provided. The device includes a housing having display and back surfaces. A digital camera unit (DCU) is held in the housing and is oriented to face the display surface. The DCU includes sensor pixels to capture image data for a scene in a field of view of the DCU. The sensor pixels are arranged in a sensor matrix with a sensor pixel density. A display layer is held in the housing proximate to the display surface. The display layer includes display pixels arranged in a display matrix having a first pixel density. The display pixels are separated by gaps arranged in rows and columns. The display layer overlays the DCU such that a first subset of the sensor pixels aligns with, and are covered by, the display pixels and a second subset of the sensor pixels align with, and are exposed through, the gaps between the display pixels. The device includes a processor and memory storing program instructions accessible by the processor. Responsive to execution of the program instructions, the processor performs graphical user interface (GUI) related operations to display content on the display layer and performs an image capture operation to obtain image data utilizing the DCU.

Claims (36)

1. A device, comprising:

a housing having display and back surfaces;

a digital camera unit (DCU) held in the housing and oriented to face the display surface, the DCU including sensor pixels to capture image data for a scene in a field of view of the DCU, the sensor pixels arranged in a sensor matrix with a sensor pixel density;

a display layer held in the housing proximate to the display surface, the display layer including display pixels arranged in a display matrix having a first pixel density, the display pixels separated by gaps arranged in rows and columns;

the display layer overlaying the DCU such that a first subset of the sensor pixels align with, and are covered by, the display pixels and a second subset of the sensor pixels align with, and are exposed through, the gaps between the display pixels;

a processor; and

memory storing program instructions accessible by the processor, wherein, responsive to execution of the program instructions, the processor performs graphical user interface (GUI) related operations to display content on the display layer and performs an image capture operation to obtain image data utilizing the DCU, wherein, during the image capture operation, the first subset of sensor pixels are blocked by the display pixels and do not capture the image data for the scene;

the display matrix having the first pixel density in a region overlaying the DCU and having a second pixel density greater than the first pixel density in a region that does not overlay the DCU; and

wherein and the DCU has a first sensor pixel density in a region overlayed by the region having the first pixel density of the display matrix, and a second sensor pixel density in a region not overlayed by the region having the first pixel density of the display matrix.

2. The device of claim 1 , wherein, during the image capture operation, the second subset of sensor pixels capture the image data for a portion of the scene visible to the second subset through the gaps, the image data representing vertical and horizontal strips of the scene corresponding to the rows and columns of the gaps.

3. The device of claim 1 , wherein the sensor matrix includes a third subset of sensor pixels that are partially covered by the display pixels and partially exposed through the gaps, the third subset of pixels to capture the image data for a portion of the scene visible to the third subset through the gaps.

4. The device of claim 1 , wherein the display pixels comprise a grid of subpixel color elements with the gaps arranged in the rows and columns between the subpixel color elements.

5. The device of claim 1 , wherein the sensor pixel density is at least twice the first pixel density of the display matrix.

6. The device of claim 1 , wherein the gaps include a first row that overlays and exposes at least two rows of the sensor pixels and include a first column that overlays and exposes at least two columns of the sensor pixels.

7. The device of claim 1 , wherein the display matrix comprises a first arrangement of display pixels in a region overlaying the DCU and a second arrangement of display pixels in a region that does not overlay the DCU, the first arrangement different from the second arrangement.

8. The device of claim 1 , wherein the first sensor pixel density is greater than the second sensor pixel density.

9. A computer implemented method, the method comprising:

under control of one or more processors configured with specific executable program instructions,

capturing image data for a scene in a field of view of a digital camera unit (DCU), the DCU located below a display layer of an electronic device and oriented to face the display surface, the DCU including sensor pixels to capture image data for a scene, the sensor pixels arranged in a sensor matrix with a sensor pixel density;

displaying content on the display layer of the electronic device, the display layer including display pixels arranged in a display matrix having a first pixel density, the display pixels separated by gaps arranged in rows and columns, the display layer overlaying the DCU, such that a first subset of the sensor pixels align with, and are covered by, the display pixels and a second subset of the sensor pixels align with, and are exposed through, the gaps between the display pixels, wherein, during capturing, the first subset of sensor pixels are blocked by the display pixels and do not capture the image data for the scene; and

wherein the display matrix has the first pixel density in a region overlaying the DCU and has a second pixel density greater than the first pixel density in a region that does not overlay the DCU; and

wherein and the DCU has a first sensor pixel density in a region overlayed by the region having the first pixel density of the display matrix, and a second sensor pixel density in a region not overlayed by the region having the first pixel density of the display matrix.

10. The method of claim 9 , wherein, during capturing, the second subset of sensor pixels capture the image data for a portion of the scene visible to the second subset through the gaps, the image data representing vertical and horizontal strips of the scene corresponding to the rows and columns of the gaps.

11. The method of claim 9 , wherein the sensor matrix includes a third subset of sensor pixels that are partially covered by the display pixels and partially exposed through the gaps; and

wherein, during capturing, the third subset of pixels capture the image data for a portion of the scene visible to the third subset through the gaps.

12. The method of claim 9 , wherein the sensor pixel density is at least twice the first pixel density of the display matrix.

13. The method of claim 9 , wherein, during capturing, the gaps include a first row that overlays and exposes at least two rows of the sensor pixels and include a first column that overlays and exposes at least two columns of the sensor pixels.

14. The method of claim 9 , wherein the first sensor pixel density is greater than the second sensor pixel density.

15. A computer program product comprising a non-signal computer readable storage medium comprising computer executable code to:

capture image data for a scene in a field of view of a digital camera unit (DCU), the DCU located below a display layer of an electronic device and oriented to face the display surface, the DCU including sensor pixels to capture image data for a scene, the sensor pixels arranged in a sensor matrix with a sensor pixel density;

display content on the display layer of the electronic device, the display layer including display pixels arranged in a display matrix having a first pixel density, the display pixels separated by gaps arranged in rows and columns, the display layer overlaying the DCU, such that a first subset of the sensor pixels align with, and are covered by, the display pixels and a second subset of the sensor pixels align with, and are exposed through, the gaps between the display pixels, wherein, during the capture, the computer executable code does not capture image data for the scene from the first subset of sensor pixels, the first subset of sensor pixels being blocked by the display pixels; and

wherein the display matrix has the first pixel density in a region overlaying the DCU and having a second pixel density greater than the first pixel density in a region that does not overlay the DCU; and

wherein and the DCU has a first sensor pixel density in a region overlayed by the region having the first pixel density of the display matrix, and a second sensor pixel density in a region not overlayed by the region having the first pixel density of the display matrix.

16. The computer program product of claim 15 , wherein the first sensor pixel density is greater than the second sensor pixel density.

17. The computer program product of claim 15 , wherein, during the capture, the computer executable code captures image data for a portion of the scene visible to the second subset of sensor pixels through the gaps between the display pixels, the image data representing vertical and horizontal strips of the scene corresponding to the rows and columns of the gaps.

18. The computer program product of claim 15 , wherein the sensor matrix includes a third subset of sensor pixels that are partially covered by the display pixels and partially exposed through the gaps; and wherein, during the capture, the third subset of pixels capture the image data for a portion of the scene visible to the third subset through the gaps.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 10, 2025
From: LENOVO PC INTERNATIONAL LIMITED
To: LENOVO SWITZERLAND INTERNATIONAL GMBH
Reel/Frame 069870/0670 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2022
From: LENOVO (SINGAPORE) PTE LTD
To: LENOVO PC INTERNATIONAL LIMITED
Reel/Frame 060638/0044 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 8, 2019
From: NICHOLSON, JOHN WELDON; CROMER, DARYL; LOCKER, HOWARD J.
To: LENOVO (SINGAPORE) PTE. LTD.
Reel/Frame 049112/0058 →