IP Library › Granted Patent US 12,348,850
Granted Patent B2
US 12,348,850 · App. 17/853,731 · Granted Jul 1, 2025

Integrating electronic and physical workspaces

Inventors: Nigel Clarke (Mountain View, CA); Fulya Yilmaz (San Francisco, CA); Curtis Aumiller (San Jose, CA); Mohammad Abu Saude (Sunnyvale, CA)
Assignee: Samsung Electronics Co., Ltd.
H04N23/56G06V20/52H04N7/18H04N23/74
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,348,850
App. No.
17/853,731
Granted
Jul 1, 2025
Kind
B2
Abstract

In one embodiment, an apparatus includes a head unit that includes one or more image sensors and one or more illumination sources. Both an intensity level of the one or more illumination sources and a position or an orientation of the head unit are adjustable. The apparatus also includes one or more processors and one or more memory elements that can execute instructions to capture, by the one or more image sensors, an image of a user's physical workspace, and provide the captured image for display to one or more users.

Claims (78)

1. A system comprising one or more processors and one or more non-transitory computer readable storage media embodying instructions coupled to the one or more processors, the one or more processors operable to execute the instructions to:

capture, by one or more image sensors of an apparatus, an initial image of a user's physical workspace, wherein:

the apparatus comprises a head unit comprising a plurality of illumination sources and the one or more image sensors; and

an intensity level of each of the plurality of illumination sources is separately adjustable and a position or an orientation of the head unit is adjustable; and

determine a brightness-related visual artifact in the captured initial image of the user's physical workspace;

automatically adjust an intensity level of at least one of the plurality of separately adjustable illumination sources based on the determined brightness-related visual artifact in the initial captured image;

automatically capture, by the one or more image sensors, a subsequent image of the user's physical workspace as illuminated by the adjusted intensity level of the at least one of the plurality of separately adjustable illumination sources;

compare the initial image to the subsequent image to determine the effect of the intensity-level adjustment on the brightness-related visual artifact;

further automatically adjust the intensity level of at least one of the plurality of separately adjustable illumination sources based on the comparison; and

further capture images at different adjusted intensity levels and compare captured images at different levels to determine further automatic adjustments to the intensity level of at least one of the plurality of separately adjustable illumination sources, until a stopping condition regarding the brightness-related visual artifact is met.

2. The system of claim 1 , wherein:

the plurality of separately adjustable illumination sources define a boundary within the head unit; and

the one or more image sensors are disposed within the boundary.

3. The system of claim 1 , wherein the one or more image sensors are disposed at a geometric center of the plurality of separately adjustable illumination sources.

4. The system of claim 1 , wherein the processors are further operable to execute the instructions to:

turn on at least one of the plurality of separately adjustable illumination sources to a preset intensity level;

determine, based on a preliminary image of the user's physical workspace, one or more image conditions;

adjust, based on the determined image conditions, an intensity of the at least one of the plurality of separately adjustable illumination sources; and

set the at least one of the plurality of separately adjustable illumination sources to illuminate at the adjusted intensity level while the one or more image sensors capture the image of the user's physical workspace.

5. The system of claim 1 , wherein the processors are further operable to execute the instructions to:

identify a physical object in a field of view of the one or more image sensors; and

crop the captured image to the physical object.

6. The system of claim 1 , wherein the processors are further operable to execute the instructions to create from the captured image a processed image that provides a top-down perspective of at least part of the user's physical workplace.

7. The system of claim 1 , wherein the processors are further operable to execute the instructions to:

receive, from a user, an annotation to the captured image; and

provide the captured image as annotated for display to one or more users.

8. The system of claim 1 , wherein the processors are further operable to execute the instructions to:

identify the apparatus as a webcam associated with a client computing device;

process the captured image; and

provide the processed captured image webcam output for use by one or more applications executing on the client computing device.

9. The system of claim 1 , wherein the one or more processors are further operable to execute the instructions to automatically capture, based on one or more criteria, the image of the user's physical workspace.

10. The system of claim 9 , wherein the one or more criteria comprise one or more of:

a motion of an object in the user's physical workspace;

an amount of time since a previous image of the user's physical workspace was captured; or

one or more changes in a lighting of the user's physical workspace or in a location of an object in the user's physical workspace.

11. The system of claim 1 , wherein the one or more processors are further operable to execute the instructions to automatically provide the capture image to a clipboard of computing device.

12. An apparatus comprising:

a head unit comprising one or more image sensors and a plurality of illumination sources, wherein an intensity level of each of the plurality of illumination sources is separately adjustable and a position or an orientation of the head unit is adjustable; and

one or more processors and a non-transitory computer readable storage media embodying instructions coupled to the one or more processors, the one or more processors operable to execute the instructions to:

capture, by the one or more image sensors, an initial image of a user's physical workspace;

determine a brightness-related visual artifact in the captured initial image of the user's physical workspace;

automatically adjust an intensity level of at least one of the plurality of separately adjustable illumination sources based on the determined brightness-related visual artifact in the initial captured image;

automatically capture, by the one or more image sensors, a subsequent image of the user's physical workspace as illuminated by the adjusted intensity level of the at least one of the plurality of separately adjustable illumination sources;

compare the initial image to the subsequent image to determine the effect of the intensity-level adjustment on the brightness-related visual artifact;

further automatically adjust the intensity level of at least one of the plurality of separately adjustable illumination sources based on the comparison; and

further capture images at different adjusted intensity levels and compare captured images at different levels to determine further automatic adjustments to the intensity level of at least one of the plurality of separately adjustable illumination sources, until a stopping condition regarding the brightness-related visual artifact is met.

13. The apparatus of claim 12 , wherein:

the plurality of separately adjustable illumination sources define a boundary within the head unit; and

the one or more image sensors are disposed within the boundary.

14. The apparatus of claim 12 , wherein the plurality of separately adjustable image sensors are disposed at a geometric center of the one or more illumination sources.

15. The apparatus of claim 12 wherein the processors are further operable to execute the instructions to:

turn on at least one of the plurality of separately adjustable illumination sources to a preset intensity level;

determine, based on a preliminary image of the user's physical workspace, one or more image conditions;

adjust, based on the determined image conditions, an intensity of the at least one of the plurality of separately adjustable illumination sources; and

set the at least one of the plurality of separately adjustable illumination sources to illuminate at the adjusted intensity level while the one or more image sensors capture the image of the user's physical workspace.

16. The apparatus of claim 12 , wherein the processors are further operable to execute the instructions to create from the captured image a processed image that provides a top-down perspective of at least part of the user's physical workplace.

17. The apparatus of claim 12 , wherein the processors are further operable to execute the instructions to:

identify the apparatus as a webcam associated with a client computing device;

process the captured image; and

provide the processed captured image webcam output for use by one or more applications executing on the client computing device.

18. A method comprising:

capturing, by one or more image sensors of an apparatus, an initial image of a user's physical workspace, wherein:

the apparatus comprises a head unit comprising a plurality of illumination sources and the one or more image sensors; and

an intensity level of each of the plurality of illumination sources is separately adjustable and a position or an orientation of the head unit is adjustable;

determining a brightness-related visual artifact in the captured initial image of the user's physical workspace;

automatically adjusting an intensity level of at least one of the plurality of separately adjustable illumination sources based on the determined brightness-related visual artifact in the initial captured image;

automatically capture, by the one or more image sensors, a subsequent image of the user's physical workspace as illuminated by the adjusted intensity level of the at least one of the plurality of separately adjustable illumination sources; and

comparing the initial image to the subsequent image to determine the effect of the intensity-level adjustment on the brightness-related visual artifact;

further automatically adjusting the intensity level of at least one of the plurality of separately adjustable illumination sources based on the comparison; and

further capturing images at different adjusted intensity levels and compare captured images at different levels to determine further automatic adjustments to the intensity level of at least one of the plurality of separately adjustable illumination sources, until a stopping condition regarding the brightness-related visual artifact is met.

19. The method of claim 18 , wherein:

the plurality of separately adjustable illumination sources define a boundary within the head unit; and

the one or more image sensors are disposed within the boundary.

20. The method of claim 18 , further comprising:

turning on at least one of the plurality of separately adjustable illumination sources to a preset intensity level;

determining, based on a preliminary image of the user's physical workspace, one or more image conditions;

adjusting, based on the determined image conditions, an intensity of the at least one of the plurality of separately adjustable illumination sources; and

setting the at least one of the plurality of separately adjustable illumination sources to illuminate at the adjusted intensity level while the one or more image sensors capture the image of the user's physical workspace.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 29, 2022
From: CLARKE, NIGEL; YILMAZ, FULYA; AUMILLER, CURTIS; ABU SAUDE, MOHAMMAD
To: SAMSUNG ELECTRONICS COMPANY, LTD.
Reel/Frame 060360/0140 →
Continuity (1)
Related Publication 20240007730A1 · Jan 4, 2024
References Cited (9)
US 9747499B2 · Kim · 2017 [cited by applicant]
US 10187546B2 · Liu · 2019 [cited by applicant]
US 20040021834A1 · Satomi · 2004 [cited by examiner]
US 20050265014A1 · Matsui · 2005 [cited by examiner]
US 20230090743A1 · Pinto · 2023 [cited by examiner]
US 20240249490A1 · Hashimoto · 2024 [cited by examiner]
KR 1020130118704A · 2013 [cited by applicant]
KR 102054552B1 · 2019 [cited by applicant]
KR 1020210001371A · 2021 [cited by applicant]