IP Library Granted Patent US 8,947,506
Granted Patent B2
US 8,947,506 · App. 13/077,899 · Granted Feb 3, 2015

Method and system for utilizing depth information for generating 3D maps

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,947,506
App. No.
13/077,899
Granted
Feb 3, 2015
Kind
B2
Abstract

A mobile device, which comprises a monoscopic three-dimensional (3D) video generation device, one or more depth sensors and one or more devices operable to determine distance, may be operable to capture two-dimensional (2D) video image data and corresponding depth information of surroundings of the mobile device. The mobile device may capture distance information via the one or more devices operable to determine distance. A 3D map may be rendered by the mobile device utilizing the captured 2D video image data, the captured corresponding depth information and/or the captured distance information. The monoscopic 3D video generation device may comprise one or more image sensors. The one or more devices operable to determine distance may comprise an optical and/or electromagnetic emitting device. The 3D map may be generated by the mobile device. The 3D map may also be generated by the mobile device based on information received from a location server.

Claims (44)

1. A method for communication, the method comprising:

capturing by a mobile device, two-dimensional (2D) video image data and corresponding depth information of surroundings of said mobile device, wherein said mobile device comprises a monoscopic three-dimensional (3D) video generation device with one or more image sensors and one or more depth sensors;

capturing distance information via one or more devices operable to determine distance, wherein said one or more devices comprise a monoscopic three-dimensional (3D) video generation device and said

distance information indicates distance between said mobile device and one or more other monoscopic 3D video generation devices, said distance information being different than said depth information captured by said mobile device;

communicating said captured 2D video image data, said corresponding depth information, and said distance information to a location server, wherein said location server determines a location of said mobile device based on contents of said 2D video image data, said captured corresponding depth information, and said distance information;

rendering, by said location server, a 3D map of a surrounding of said mobile device by utilizing said captured 2D video image data, said corresponding depth information, and said location of said mobile device;

receiving, by said mobile device, said location of said mobile device and said 3D map from said location server.

2. The method according to claim 1 , wherein said one or more devices operable to determine distance comprises an optical and/or electromagnetic emitting device.

3. The method according to claim 1 , comprising generating said 3D map by said mobile device.

4. The method according to claim 1 , comprising:

capturing said 2D video Image data via said one or more image sensors of said monoscopic 3D video generation device; and

capturing said corresponding depth information via said one or more depth sensors.

5. The method according to claim 1 , comprising emitting one or more signals, via said one or more devices operable to determine said distance, to said one or more other monoscopic 3D video generation devices surrounding said mobile device, wherein each of said one or more other monoscopic 3D video generation devices determines said distance information for said mobile device based on said emitted one or more signals.

6. The method according to claim 5 , comprising:

receiving said distance information and a known location associated with each of said one or more other monoscopic 3D video generation devices from each of said one or more other monoscopic 3D video generation devices; and

receiving said 2D video image data and said corresponding depth information, which are captured by each of said one or more other monoscopic 3D video generation devices, from each of said one or more other monoscopic 3D video generation devices.

7. The method according to claim 6 , comprising determining said location of said mobile device based on said received distance information and said known location associated with each of said one or more other monoscopic 3D video generation devices.

8. A system for communication, the system comprising:

a mobile device which comprises a monoscopic three-dimensional (3D) video generation device with one or more image sensors and one or more depth sensors, capturing two-dimensional (2D) video image data and corresponding depth information of surroundings of said mobile device;

one or more devices, which comprise a monoscopic three-dimensional (3D) video generation device with one or more image sensors and one or more depth sensors, determining distance information indicating distance between said mobile device and said one or more other monoscopic 3D video generation devices, wherein said distance information being different than said depth information captured by said mobile device;

a location server communicating said captured 2D video image data, said captured corresponding depth information, and said distance information to determine a location of said mobile device based on contents of said 2D video image data, said corresponding depth information, and said distance information and rendering a 3D map of a surrounding of said mobile device by utilizing said captured 2D video image data, said captured corresponding depth information and said location of said mobile device, said mobile device further receiving said location of said mobile device and said 3D map from said location server.

9. The system according to claim 8 , wherein said one or more devices operable to determine distance comprises an optical and/or electromagnetic emitting device.

10. The system according to claim 8 , wherein said one or more processors and/or circuits are operable to generate said 3D map by said mobile device.

11. The system according to claim 8 , wherein said one or more processors and/or circuits are operable to generate said 3D map by said mobile device based on said location information of said mobile device received from said location server.

12. The system according to claim 11 , wherein said one or more processors and/or circuits are operable to:

capture said 2D video image data via said one or more image sensors of said monoscopic 3D video generation device; and

capture said corresponding depth information via said one or more depth sensors.

13. The system according to claim 11 , wherein each of said one or more other monoscopic 3D video generation devices determines said distance information for said mobile device based on said emitted one or more signals, and said one or more processors and/or circuits are further operable to:

receive said distance information and a known location associated with each of said one or more other monoscopic 3D video generation devices from each of said one or more other monoscopic 3D video generation devices; and

receive said 2D video image data and said corresponding depth information, which are captured by each of said one or more other monoscopic 3D video generation devices, from each of said one or more other monoscopic 3D video generation devices.

14. The system according to claim 13 , wherein said mobile device is operable to determine said location of said mobile device based on said received distance information and said known location associated with each of said one or more other monoscopic 3D video generation devices.

15. The system according to claim 14 , wherein said mobile device is operable to:

receive said 3D map from said location server, wherein said location server generates said 3D map associated with said location of said mobile device based on said 2D video image data, said corresponding depth information, and said distance information.

16. A mobile device comprising:

an image sensor configured to capture two-dimensional (2D) image data of surroundings of the mobile device;

a depth sensor configured to capture depth information of surroundings of the mobile device corresponding to the 2D image data;

a distance measurement device configured to emit a signal to a device at a known location, the distance measurement device further configured to receive, from the device at the known location, a distance information including the distance between the mobile device and the device at the know location, the distance being different than the depth information captured by the depth sensor;

an interface configured to communicate captured information to a server, the captured information comprising the 2D image data, the depth information, and the distance information, wherein the server determines a location of the mobile device based on the communicated captured information;

the interface further configured to receive the location from the server; and

a processor configured to render a 3D map of the surroundings and indicate the location of the mobile device on the 3D map using the 2D image data, the depth information, and the received distance information.

17. The mobile device of claim 16 , wherein the processor is further configured to generate the 3D map of the surroundings using the captured information.

18. The mobile device of claim 16 , wherein the 3D map of the surroundings is received from the server, wherein the server generates the 3D map using the captured information communicated by the mobile device.

19. The mobile device of claim 16 , wherein the captured information further comprises image data and depth information captured by the device at the known location.

20. The mobile device of claim 19 , wherein the image data and the depth information captured by the device at the known location is used to generate the 3D map.

Assignments (7)
CORRECTIVE ASSIGNMENT TO CORRECT THE PATENT NUMBER 9,385,856 TO 9,385,756 PREVIOUSLY RECORDED AT REEL: 47349 FRAME: 001. ASSIGNOR(S) HEREBY CONFIRMS THE MERGER. Recorded Mar 22, 2019
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 051144/0648 →
CORRECTIVE ASSIGNMENT TO CORRECT THE EFFECTIVE DATE PREVIOUSLY RECORDED ON REEL 047229 FRAME 0408. ASSIGNOR(S) HEREBY CONFIRMS THE THE EFFECTIVE DATE IS 09/05/2018. Recorded Oct 29, 2018
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 047349/0001 →
MERGER Recorded Oct 4, 2018
From: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
To: AVAGO TECHNOLOGIES INTERNATIONAL SALES PTE. LIMITED
Reel/Frame 047229/0408 →
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS Recorded Feb 3, 2017
From: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
To: BROADCOM CORPORATION
Reel/Frame 041712/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2017
From: BROADCOM CORPORATION
To: AVAGO TECHNOLOGIES GENERAL IP (SINGAPORE) PTE. LTD.
Reel/Frame 041706/0001 →
PATENT SECURITY AGREEMENT Recorded Feb 11, 2016
From: BROADCOM CORPORATION
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 037806/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 22, 2012
From: KARAOGUZ, JEYHAN; SESHADRI, NAMBI; CHEN, XUEMIN; BOROSS, CHRIS
To: BROADCOM CORPORATION
Reel/Frame 027743/0604 →