IP Library › Granted Patent US 12,197,646
Granted Patent B2
US 12,197,646 · App. 18/569,785 · Granted Jan 14, 2025

Electronic device

Inventors: Hiromichi Godo (Isehara, JP); Yoshiyuki Kurokawa (Sagamihara, JP); Seiko Inoue (Atsugi, JP); Kazuaki Ohshima (Atsugi, JP); Shunpei Yamazaki (Setagaya, JP)
Assignee: Semiconductor Energy Laboratory Co., Ltd.
G06F3/013G06T1/20G06T7/75G06V10/141G06V40/193H04N23/611H04N23/617H04N23/90G06T2207/10048G06T2207/20081G06T2207/30041G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,197,646
App. No.
18/569,785
Filed
Dec 13, 2023
Granted
Jan 14, 2025
Kind
B2
Examiner
AHN, SEJOON
Art Unit
2628
USPC
345/156
Abstract

An electronic device that enables smooth communication is provided. The electronic device includes a display portion including a first camera; a second camera; and an image processing portion. The second camera is positioned in a region not overlapping with the display portion. The first camera has a function of generating a first image of a subject, and the second camera has a function of generating a second image of the subject. The image processing portion includes a generator that performs learning using training data. The training data includes an image including a person's face. The image processing portion has a function of making the first image clear when the first image is input to the generator and a function of tracking the gaze of the subject on the basis of the second image.

Claims (32)

1. An electronic device comprising:

a display portion comprising a first camera; a second camera; and an image processing portion,

wherein the second camera is positioned in a region not overlapping with the display portion,

wherein the first camera is configured to generate a first image of a subject,

wherein the second camera is configured to generate a second image of the subject,

wherein the image processing portion comprises a generator performing learning using training data,

wherein the training data comprises an image comprising a face of a person, and

wherein the image processing portion is configured to make the first image clear when the first image is input to the generator and track a gaze of the subject on the basis of the second image.

2. The electronic device according to claim 1 , further comprising a light source emitting infrared light,

wherein the light source is positioned in a region not overlapping with the display portion,

wherein the light source is used for detection of the gaze of the subject, and

wherein the gaze of the subject is tracked by repeating the detection of the gaze of the subject.

3. An electronic device comprising:

a display portion comprising a first camera; a second camera; a third camera; and an image processing portion,

wherein the second camera and the third camera are each independently positioned in a region not overlapping with the display portion,

wherein the first camera is configured to generate a first image of a subject,

wherein the second camera is configured to generate a second image of the subject,

wherein the third camera is configured to generate a third image of the subject, and

wherein the image processing portion is configured to make the first image clear, recognize a face of the subject from the clear first image, recognize a three-dimensional shape of the face of the subject from the second image and the third image, and create an avatar from the face of the subject and the three-dimensional shape of the face of the subject.

4. The electronic device according to claim 1 , wherein the first camera is positioned behind a pixel in the display portion when seen from the subject.

5. The electronic device according to claim 1 , wherein the first camera is positioned in a region comprising a pixel in the display portion when seen from the subject.

6. An electronic device comprising:

a display portion comprising a first camera and a fourth camera; and an image processing portion,

wherein the first camera is configured to generate a first image of a subject,

wherein the fourth camera is configured to generate a fourth image of the subject,

wherein the image processing portion comprises a generator performing learning using training data,

wherein the training data comprises an image comprising a face of a person, and

wherein the image processing portion is configured to activate one of the first camera and the fourth camera and make the first image or the fourth image clear when the first image or the fourth image generated using the activated one of the first camera and the fourth camera is input to the generator.

7. The electronic device according to claim 6 , wherein the first camera and the fourth camera are positioned behind a pixel in the display portion when seen from the subject.

8. The electronic device according to claim 6 , wherein the first camera and the fourth camera are positioned in a region comprising a pixel in the display portion when seen from the subject.

9. The electronic device according to claim 3 , wherein the first camera is positioned behind a pixel in the display portion when seen from the subject.

10. The electronic device according to claim 3 , wherein the first camera is positioned in a region comprising a pixel in the display portion when seen from the subject.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 13, 2023
From: GODO, HIROMICHI; KUROKAWA, YOSHIYUKI; INOUE, SEIKO; OHSHIMA, KAZUAKI; YAMAZAKI, SHUNPEI
To: SEMICONDUCTOR ENERGY LABORATORY CO., LTD.
Reel/Frame 065860/0013 →
Priority Claims (1)
JP 2021-109366 · Jun 30, 2021 · national
Continuity (1)
Related Publication 20240256033A1 · Aug 1, 2024
References Cited (69)
US 7948481B2 · Vilcovsky · 2011 [cited by applicant]
US 8624883B2 · Vilcovsky · 2014 [cited by applicant]
US 8970569B2 · Vilcovsky et al. · 2015 [cited by applicant]
US 8976160B2 · Vilcovsky et al. · 2015 [cited by applicant]
US 8982109B2 · Vilcovsky et al. · 2015 [cited by applicant]
US 8982110B2 · Saban et al. · 2015 [cited by applicant]
US 9269157B2 · Saban et al. · 2016 [cited by applicant]
US 9369638B2 · Saban et al. · 2016 [cited by applicant]
US 10109315B2 · Saban et al. · 2018 [cited by applicant]
US 10682038B1 · Zhang et al. · 2020 [cited by applicant]
US 10789714B2 · Kang et al. · 2020 [cited by applicant]
US 11288546B2 · Lee et al. · 2022 [cited by applicant]
US 11513409B2 · Toyotaka et al. · 2022 [cited by applicant]
US 11631180B2 · Kang et al. · 2023 [cited by applicant]
US 11647152B2 · Kokura · 2023 [cited by applicant]
US 20140055342A1 · Kamimura et al. · 2014 [cited by applicant]
US 20140225977A1 · Vilcovsky et al. · 2014 [cited by applicant]
US 20150049165A1 · Choi · 2015 [cited by applicant]
US 20180069060A1 · Rappoport et al. · 2018 [cited by applicant]
US 20180245840A1 · Chen · 2018 [cited by examiner]
US 20180246566A1 · Bitauld · 2018 [cited by examiner]
US 20180246567A1 · Campbell · 2018 [cited by examiner]
US 20180246568A1 · Holz · 2018 [cited by examiner]
US 20180246846A1 · Takimoto · 2018 [cited by examiner]
US 20190197690A1 · Kang et al. · 2019 [cited by applicant]
US 20190370608A1 · Lee et al. · 2019 [cited by applicant]
US 20200219947A1 · Yang et al. · 2020 [cited by applicant]
US 20200396415A1 · Kokura · 2020 [cited by applicant]
US 20210026176A1 · Toyotaka et al. · 2021 [cited by applicant]
US 20210176383A1 · Kim et al. · 2021 [cited by applicant]
US 20220197582A1 · Yokoi et al. · 2022 [cited by applicant]
US 20230006010A1 · Abe et al. · 2023 [cited by applicant]
US 20230088632A1 · Toyotaka et al. · 2023 [cited by applicant]
CN 105556508A · 2016 [cited by applicant]
CN 106653802A · 2017 [cited by applicant]
CN 109951698A · 2019 [cited by applicant]
CN 111047507A · 2020 [cited by applicant]
CN 111902856A · 2020 [cited by applicant]
CN 113011271A · 2021 [cited by applicant]
EP 2884738A · 2015 [cited by applicant]
EP 3404619A · 2018 [cited by applicant]
EP 3502959A · 2019 [cited by applicant]
EP 3588388A · 2020 [cited by applicant]
EP 4184443A · 2023 [cited by applicant]
JP 11122544A · 1999 [cited by applicant]
JP 2014039617A · 2014 [cited by applicant]
JP 2016532197 · 2016 [cited by applicant]
JP 2018124457A · 2018 [cited by applicant]
JP 2019115037A · 2019 [cited by applicant]
JP 2020201823A · 2020 [cited by applicant]
JP 2021503613 · 2021 [cited by applicant]
JP 2021045990A · 2021 [cited by applicant]
KR 20160041965A · 2016 [cited by applicant]
KR 20190075501A · 2019 [cited by applicant]
KR 20190136833A · 2019 [cited by applicant]
KR 20200139701A · 2020 [cited by applicant]
WO WO2006092793 · 2006 [cited by applicant]
WO WO2014100250 · 2014 [cited by applicant]
WO WO2015020703 · 2015 [cited by applicant]
WO WO2016112346 · 2016 [cited by applicant]
WO WO2018086353 · 2018 [cited by applicant]
WO WO2019186339 · 2019 [cited by applicant]
WO WO2021054222 · 2021 [cited by applicant]
WO WO2021124449 · 2021 [cited by applicant]
International Search Report (Application No. PCT/IB2022/055685) Dated Sep. 27, 2022. [cited by applicant]
Written Opinion (Application No. PCT/IB2022/055685) Dated Sep. 27, 2022. [cited by applicant]
Slobodin.D, “Displays with Integrated Microcamera Arrays for Image Capture and Sensing”, SID Digest '21 : SID International Symposium Digest of Technical Papers, May 17, 2021, pp. 745-748. [cited by applicant]
Feng.Y et al., “Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network”, arXiv.org: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR), Mar. 21, 2018, pp. 1-18, Cornell Univ… [cited by applicant]
Karras.T et al., “Analyzing and Improving the Image Quality of StyleGAN”, arXiv.org: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Image and Video … [cited by applicant]