IP Library › Granted Patent US 11,790,572
Granted Patent B1
US 11,790,572 · App. 17/557,719 · Granted Oct 17, 2023

Digitization of whiteboards

Inventors: Guillaume Oulès (Bordeaux, FR); Daryl Stimm (Encinitas, CA)
Assignee: GoPro, Inc.
G06T11/00G06T1/60G06T7/11G06T7/50G06V10/225G06V10/255H04N7/15G06T2207/20021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,790,572
App. No.
17/557,719
Granted
Oct 17, 2023
Kind
B1
Abstract

Depiction of a physical whiteboard may be captured by an image capture device. The depiction of the physical whiteboard may be detected and used to generate a virtual whiteboard. A change in the depiction of the physical whiteboard may be detected and used to change the virtual whiteboard. A virtual change to the virtual whiteboard may be received and used to change the virtual whiteboard. The change to the virtual whiteboard based on the virtual change may be projected on top of the physical whiteboard.

Claims (52)

1. A system for digitizing whiteboards, the system comprising:

one or more physical processors configured by machine-readable instructions to:

obtain visual information, the visual information defining visual content captured by an image capture device during a capture duration, the visual content including depiction of a physical whiteboard;

detect the depiction of the physical whiteboard within the visual content;

generate a virtual whiteboard based on the depiction of the physical whiteboard detected within the visual content;

receive a virtual change to the virtual whiteboard;

change the virtual whiteboard based on the virtual change to the virtual whiteboard;

project the change to the virtual whiteboard on top of the physical whiteboard, the projection of the change to the virtual whiteboard on top of the physical whiteboard including projection of a sequence of images on top of the physical whiteboard, the sequence of images including images with the change to the virtual whiteboard and images without the change to the virtual whiteboard, wherein the projection of the images without the change to the virtual whiteboard facilitates detection of physical changes to the physical whiteboard;

detect a change in the depiction of the physical whiteboard within the visual content captured during the projection, on top of the physical whiteboard, of the images without the change to the virtual whiteboard, the change in the depiction of the physical whiteboard corresponding to a physical change to the physical whiteboard; and

change the virtual whiteboard based on the change in the depiction of the physical whiteboard within the visual content.

2. The system of claim 1 , wherein the change in the depiction of the physical whiteboard within the visual content is detected based on:

segmentation of the depiction of the physical whiteboard from other depiction within the visual content to identify a target zone of the visual content that includes the depiction of the physical whiteboard; and

detection of the change within the target zone of the visual content.

3. The system of claim 2 , wherein the target zone of the visual content is reidentified based on movement of the image capture device.

4. The system of claim 1 , wherein the sequence of images includes more of the images with the change to the virtual whiteboard than the images without the change to the virtual whiteboard.

5. The system of claim 1 , wherein the virtual change to the virtual whiteboard includes addition of a virtual media item to the virtual whiteboard.

6. The system of claim 5 , wherein the virtual media item includes a virtual document, a virtual image, or a virtual video.

7. The system of claim 1 , wherein the images with the change to the virtual whiteboard include one or more markers, the one or more markers indicating that the marked images include the change to the virtual whiteboard and are not to be used to detect physical changes to the physical whiteboard.

8. The system of claim 1 , wherein the images without the change to the virtual whiteboard include one or more markers, the one or more markers indicating that the marked images do not include the change to the virtual whiteboard and are to be used to detect physical changes to the physical whiteboard.

9. The system of claim 1 , wherein the visual content is stored within a video file, the visual content having a progress length, and a moment within the progress length corresponding to the change in the depiction of the physical whiteboard is flagged as a highlight moment.

10. A method for digitizing whiteboards, the method performed by a computing system including one or more processors, the method comprising:

obtaining, by the computing system, visual information, the visual information defining visual content captured by an image sensor of an image capture device during a capture duration, the visual content including depiction of a physical whiteboard;

detecting, by the computing system, the depiction of the physical whiteboard within the visual content;

generating, by the computing system, a virtual whiteboard based on the depiction of the physical whiteboard detected within the visual content;

receiving, by the computing system, a virtual change to the virtual whiteboard;

changing, by the computing system, the virtual whiteboard based on the virtual change to the virtual whiteboard;

projecting, by the computing system, the change to the virtual whiteboard on top of the physical whiteboard, the projection of the change to the virtual whiteboard on top of the physical whiteboard including projection of a sequence of images on top of the physical whiteboard, the sequence of images including images with the change to the virtual whiteboard and images without the change to the virtual whiteboard, wherein the projection of the images without the change to the virtual whiteboard facilitates detection of physical changes to the physical whiteboard;

detecting, by the computing system, a change in the depiction of the physical whiteboard within the visual content captured during the projection, on top of the physical whiteboard, of the images without the change to the virtual whiteboard, the change in the depiction of the physical whiteboard corresponding to a physical change to the physical whiteboard; and

changing, by the computing system, the virtual whiteboard based on the change in the depiction of the physical whiteboard within the visual content.

11. The method of claim 10 , wherein the change in the depiction of the physical whiteboard within the visual content is detected based on:

segmentation of the depiction of the physical whiteboard from other depiction within the visual content to identify a target zone of the visual content that includes the depiction of the physical whiteboard; and

detection of the change within the target zone of the visual content.

12. The method of claim 11 , wherein the target zone of the visual content is reidentified based on movement of the image capture device.

13. The method of claim 10 , wherein the sequence of images includes more of the images with the change to the virtual whiteboard than the images without the change to the virtual whiteboard.

14. The method of claim 10 , wherein the virtual change to the virtual whiteboard includes addition of a virtual media item to the virtual whiteboard.

15. The method of claim 14 , wherein the virtual media item includes a virtual document, a virtual image, or a virtual video.

16. The method of claim 10 , wherein the images with the change to the virtual whiteboard include one or more markers, the one or more markers indicating that the marked images include the change to the virtual whiteboard and are not to be used to detect physical changes to the physical whiteboard.

17. The method of claim 10 , wherein the images without the change to the virtual whiteboard include one or more markers, the one or more markers indicating that the marked images do not include the change to the virtual whiteboard and are to be used to detect physical changes to the physical whiteboard.

18. The method of claim 10 , wherein the visual content is stored within a video file, the visual content having a progress length, and a moment within the progress length corresponding to the change in the depiction of the physical whiteboard is flagged as a highlight moment.

19. A system for digitizing whiteboards, the system comprising:

one or more physical processors configured by machine-readable instructions to:

obtain visual information, the visual information defining visual content captured by an image capture device during a capture duration, the visual content including depiction of a physical whiteboard;

detect the depiction of the physical whiteboard within the visual content;

generate a virtual whiteboard based on the depiction of the physical whiteboard detected within the visual content;

receive a virtual change to the virtual whiteboard;

change the virtual whiteboard based on the virtual change to the virtual whiteboard;

project the change to the virtual whiteboard on top of the physical whiteboard, the projection of the change to the virtual whiteboard on top of the physical whiteboard including projection of a sequence of images on top of the physical whiteboard, the sequence of images including images with the change to the virtual whiteboard and images without the change to the virtual whiteboard, the sequence of images including more of the images with the change to the virtual whiteboard than the images without the change to the virtual whiteboard, wherein the projection of the images without the change to the virtual whiteboard facilitates detection of physical changes to the physical whiteboard;

detect a change in the depiction of the physical whiteboard within the visual content captured during the projection, on top of the physical whiteboard, of the images without the change to the virtual whiteboard, the change in the depiction of the physical whiteboard corresponding to a physical change to the physical whiteboard; and

change the virtual whiteboard based on the change in the depiction of the physical whiteboard within the visual content.

20. The system of claim 19 , wherein:

the images with the change to the virtual whiteboard include one or more first markers, the one or more first markers indicating that the marked images include the change to the virtual whiteboard and are not to be used to detect physical changes to the physical whiteboard; or

the images without the change to the virtual whiteboard include one or more second markers, the one or more second markers indicating that the marked images do not include the change to the virtual whiteboard and are to be used to detect physical changes to the physical whiteboard.

Assignments (3)
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: FARALLON CAPITAL MANAGEMENT, L.L.C., AS AGENT
Reel/Frame 072340/0676 →
SECURITY INTEREST Recorded Aug 4, 2025
From: GOPRO, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION, AS AGENT
Reel/Frame 072358/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 21, 2021
From: OULÈS, GUILLAUME; STIMM, DARYL
To: GOPRO, INC.
Reel/Frame 058447/0717 →
Cited By (1)
US 12,284,228