IP Library › Granted Patent US 11,792,355
Granted Patent B1
US 11,792,355 · App. 17/874,784 · Granted Oct 17, 2023

Using synchronous recording by front and back smartphone cameras for creating immersive video content and for video conferencing

Inventors: Phil Libin (San Francisco, CA); Roelof F. Botha (Los Altos Hills, CA)
Assignee: mmhmm inc.
H04N5/272H04N5/2628H04N5/772H04N23/698
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,792,355
App. No.
17/874,784
Granted
Oct 17, 2023
Kind
B1
Abstract

Creating a video presentation includes obtaining a first front-camera video stream and a first back-camera video stream that are recorded synchronously using a first device having a front-camera and a back-camera and extracting a first image of a first presenter from the first front-camera video stream. Creating a video presentation also includes obtaining a second front-camera video stream and a second back-camera video stream that are recorded synchronously and extracting a second image of a second presenter from the second front-camera video stream. It is determined whether that the first back-camera video stream and the second back-camera video stream are complimentary views. Creating a video presentation also includes constructing a panoramic view by combining the first back-camera video stream and the second back-camera video stream and superimposing the first image of the first presenter and the second image of the second presenter onto to the panoramic view.

Claims (32)

1. A method of creating a video presentation, comprising:

obtaining a first front-camera video stream and a first back-camera video stream that are recorded synchronously using a first device having a front-camera and a back-camera;

extracting a first image of a first presenter from the first front-camera video stream by removing background objects from the first front-camera video stream;

obtaining a second front-camera video stream and a second back-camera video stream that are recorded synchronously using a second device having a front-camera and a back-camera;

extracting a second image of a second presenter from the second front-camera video stream by removing background objects from the second front-camera video stream;

determining that the first back-camera video stream and the second back-camera video stream are complimentary views;

constructing a panoramic view by combining the first back-camera video stream and the second back-camera video stream; and

superimposing the first image of the first presenter and the second image of the second presenter onto to the panoramic view to provide the video presentation.

2. The method, according to claim 1 , further comprising:

providing the video presentation to the first device of the first presenter and to the second device of the second presenter to allow the presenters to view the video presentation and to interact.

3. The method, according to claim 1 , wherein each of the images are at least one of: semi-transparent, an outline, a silhouette, or a stylized/enhanced graphical version of a corresponding one of the presenters.

4. The method, according to claim 1 , wherein the first presenter controls size and position of the image of the first presenter superimposed onto the panoramic view and the second presenter controls size and position of the image of the second presenter superimposed onto the panoramic view.

5. The method, according to claim 4 , wherein the presenters control the size and position of the images of the presenters using at least one of: a remote control with one-handed manipulation, eye-tracking, sensors that track 3D trajectory and gesture recognition made with a free hand of the presenter with a wearable device, or a camera-based 3D gesture recognition mechanism.

6. The method, according to claim 1 , wherein the devices are smartphones.

7. The method, according to claim 1 , wherein the devices use at least one of: GPS technology or IMU (Inertial Measurement Unit) technology.

8. The method, according to claim 7 , wherein the at least one of GPS technology or IMU technology is used in connection with determining that the first back-camera video stream and the second back-camera video stream are complimentary views.

9. The method, according to claim 7 , wherein the at least one of GPS technology or IMU technology is used in connection with constructing the panoramic view.

10. A non-transitory computer readable medium containing software that creates a video presentation from a first front-camera video stream and a first back-camera video stream that are recorded synchronously using a first device having a front-camera and a back-camera and a second front-camera video stream and a second back-camera video stream that are recorded synchronously using a second device having a front-camera and a back-camera, the software comprising:

executable code that extracts a first image of a first presenter from the first front-camera video stream by removing background objects from the first front-camera video stream;

executable code that extracts a second image of a second presenter from the second front-camera video stream by removing background objects from the second front-camera video stream;

executable code that determines that the first back-camera video stream and the second back-camera video stream are complimentary views;

executable code that constructs a panoramic view by combining the first back-camera video stream and the second back-camera video stream; and

executable code that superimposes the first image of the first presenter and the second image of the second presenter onto to the panoramic view to provide the video presentation.

11. The non-transitory computer readable medium, according to claim 10 , further comprising:

executable code that provides the video presentation to the first device of the first presenter and to the second device of the second presenter to allow the presenters to view the video presentation and to interact.

12. The non-transitory computer readable medium, according to claim 10 , wherein each of the images are at least one of: semi-transparent, an outline, a silhouette, or a stylized/enhanced graphical version of a corresponding one of the presenters.

13. The non-transitory computer readable medium, according to claim 10 , wherein the first presenter controls size and position of the image of the first presenter superimposed onto the panoramic view and the second presenter controls size and position of the image of the second presenter superimposed onto the panoramic view.

14. The non-transitory computer readable medium, according to claim 13 , wherein the presenters control the size and position of the images of the presenters using at least one of: a remote control with one-handed manipulation, eye-tracking, sensors that track 3D trajectory and gesture recognition made with a free hand of the presenter with a wearable device, or a camera-based 3D gesture recognition mechanism.

15. The non-transitory computer readable medium, according to claim 10 , wherein the devices are smartphones.

16. The non-transitory computer readable medium, according to claim 10 , wherein the devices use at least one of: GPS technology or IMU (Inertial Measurement Unit) technology.

17. The non-transitory computer readable medium, according to claim 16 , wherein the at least one of GPS technology or IMU technology is used in connection with determining that the first back-camera video stream and the second back-camera video stream are complimentary views.

18. The non-transitory computer readable medium, according to claim 16 , wherein the at least one of GPS technology or IMU technology is used in connection with constructing the panoramic view.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2022
From: LIBIN, PHIL; BOTHA, ROELOF F.
To: MMHMM INC.
Reel/Frame 060750/0771 →
Continuity (1)
Provisional Application 63228437 · Aug 2, 2021
Cited By (2)
US 12,262,068 US 12,477,078