IP Library Granted Patent US 9,591,149
Granted Patent B2
US 9,591,149 · App. 13/459,825 · Granted Mar 7, 2017

Generation of a combined image of a presentation surface

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,591,149
App. No.
13/459,825
Granted
Mar 7, 2017
Kind
B2
Abstract

Example embodiments relate to generating a combined image of a presentation surface. In example embodiments, an image including a portion of a presentation surface is captured using a camera. A location identifier is then received, where the location identifier specifies a location of a digital version of the content displayed on the presentation surface. Next, the digital version of the content displayed on the presentation surface is retrieved from the location specified by the location identifier. Finally, a combined image is generated by combining the captured image and the retrieved digital version of the content displayed on the presentation surface.

Claims (67)

1. A computing device for obtaining images of a presentation surface, the computing device comprising:

a camera;

a communication module configured to enable the computing device to engage in wired or wireless communication;

a display; and

a processor communicatively coupled with the camera, the communication module, and the display, the processor configured to:

capture an image using the camera, a portion of the captured image including the presentation surface;

obtain, from the image, a location identifier specifying a location, on a device separate from the computing device, at which a digital version of content displayed on the presentation surface is stored;

retrieve, via the communication module, the digital version of the content displayed on the presentation surface from the location specified by the location identifier;

generate a combined image by:

identifying a target position within the captured image, and

inserting, at the target position, at least a portion of the retrieved digital version of the content displayed on the presentation surface; and cause the display to show the combined image.

2. The computing device of claim 1 , wherein the presentation surface is an electronic whiteboard, a projection screen, or an electronic display.

3. The computing device of claim 1 , wherein, to obtain the location identifier, the processor is additionally configured to:

scan the captured image to identify a fiducial marker included on the presentation surface; and

extract the location identifier by decoding the fiducial marker.

4. The computing device of claim 3 , wherein the processor is configured to cause the computing device to determine a length and width of the presentation surface based on information encoded in the fiducial marker.

5. The computing device of claim 1 , wherein the processor is further configured to:

receive an updated digital version of the content displayed on the presentation surface when the content on the presentation surface is modified; and

generate a new combined image using the captured image and the updated digital version of the content displayed on the presentation surface.

6. The computing device of claim 5 , wherein the processor is further configured to:

cause the display to show a user interface element that enables a user to switch a currently-displayed image between the combined image, the new combined image, and any subsequently-created combined images.

7. The computing device of claim 1 , wherein the processor is configured to insert the at least a portion of the retrieved digital version of the content displayed on the presentation surface by replacing the portion of the captured image that includes the presentation surface with the retrieved digital version.

8. The computing device of claim 7 , wherein, to generate the combined image, the processor is additionally configured to:

detect a size and orientation of the presentation surface within the captured image; and

adjust a size and orientation of the retrieved digital version to match the size and orientation of the presentation surface prior to replacing the portion of the captured image that includes the presentation surface.

9. The computing device of claim 7 , wherein, to generate the combined image, the processor is additionally configured to:

identify a subset of the presentation surface that is included within the captured image; and

replace the subset of the presentation surface in the captured image with a corresponding subset of the retrieved digital version.

10. The computing device of claim 1 , wherein the processor is configured to determine whether the entire presentation surface is included in the image or whether only a portion of the presentation surface is in the image.

11. The computing device of claim 10 , wherein the processor is configured to, when determining only a portion of the presentation surface is in the image, further determine which portion of the presentation surface is in the image.

12. The computing device of claim 1 , wherein the processor is configured to insert the at least a portion of the retrieved digital version of the content displayed on the presentation surface by further truncating the retrieved digital version of the content displayed on the presentation surface.

13. A non-transitory machine-readable storage medium encoded with instructions executable by a processor of a computing device for obtaining images of a presentation surface, the machine-readable storage medium comprising:

instructions for capturing an image using a camera of the computing device; wherein a portion of the captured image includes the presentation surface;

instructions for obtaining, from the image, a location identifier specifying a location, on a device separate from the computing device, at which a digital version of content displayed on the presentation surface is stored;

instructions for retrieving, via a communication module, the digital version of the content displayed on the presentation surface from the location specified by the location identifier;

instructions for generating a combined image by:

identifying a target position within the captured image, and

inserting, at the target position, at least a portion of the retrieved digital version of the content displayed on the presentation surface; and

instructions for causing a display of the computing device to show the combined image.

14. The machine-readable storage medium of claim 13 , wherein the instructions for obtaining the location identifier comprise:

instructions for analyzing the captured image to identify a fiducial marker included on the presentation surface; and

instructions for extracting the location identifier by decoding the fiducial marker.

15. A method for obtaining images of a presentation surface using a computing device, the method comprising:

capturing an image using a camera of the computing device, a portion of the captured image including the presentation surface;

obtaining, from the image, a location identifier specifying a location, on a device separate from the computing device, at which a digital version of content displayed on the presentation surface is stored;

retrieving, via a communication module of the computing device, the digital version of the content displayed on the presentation surface from the location specified by the location identifier; and

generating, with a processing unit of the computing device, a combined image by:

identifying a target position within the captured image, and

inserting, at the target position, at least a portion of the retrieved digital version of the content displayed on the presentation surface; and

causing a display of the computing device to show the combined image.

16. The method of claim 15 , wherein inserting the at least a portion of the retrieved digital version of the content displayed on the presentation surface comprises:

replacing the portion of the captured image that includes the presentation surface with the retrieved digital version.

17. The method of claim 16 , wherein generating the combined image further comprises:

detecting a size and orientation of the presentation surface within the captured image; and

adjusting a size and orientation of the retrieved digital version to match the size and orientation of the presentation surface prior to replacing the portion of the captured image that includes the presentation surface.

18. An apparatus comprising:

means for capturing an image, wherein a portion of the captured image includes a presentation surface;

means for obtaining, from the image, a location identifier specifying a location, on a device separate from the apparatus, at which a digital version of content displayed on the presentation surface is stored;

means for retrieving the digital version of the content displayed on the presentation surface from the location specified by the location identifier;

means for generating a combined image by: combining the captured image and the retrieved digital version of the content displayed on the presentation surface

identifying a target position within the captured image, and

inserting, at the target position, at least a portion of the retrieved digital version of the content displayed on the presentation surface; and

means for displaying the combined image.

19. The apparatus of claim 18 , wherein the means for generating the combined image comprise means for replacing the portion of the captured image that includes the presentation surface with the retrieved digital version.

20. The apparatus of claim 19 , wherein the means for generating the combined image further comprise:

means for detecting a size and orientation of the presentation surface within the captured image; and

means for adjusting a size and orientation of the retrieved digital version to match the size and orientation of the presentation surface prior to replacing the portion of the captured image that includes the presentation surface.

Assignments (6)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 28, 2014
From: HEWLETT-PACKARD COMPANY; HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.; PALM, INC.
To: QUALCOMM INCORPORATED
Reel/Frame 032177/0210 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2013
From: PALM, INC.
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 031837/0239 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2013
From: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
To: PALM, INC.
Reel/Frame 031837/0544 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 18, 2013
From: PALM, INC.
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 031837/0659 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 3, 2013
From: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
To: PALM, INC.
Reel/Frame 030341/0459 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 1, 2012
From: DI CENSO, DAVIDE; MARTI, STEFAN J.; KIM, SEUNG WOOK
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 028302/0171 →