IP Library Granted Patent US 11,146,727
Granted Patent B2
US 11,146,727 · App. 17/200,659 · Granted Oct 12, 2021

Method and device for generating a panoramic image

Inventors: Tong Rao (Beijing, CN); Yonglin Yang (Beijing, CN); Mingyuan Wang (Beijing, CN); Yutong Chen (Beijing, CN)
Assignee: Ke.com (Beijing) Technology Co., Ltd.
H04N5/23238H04N5/265H04N9/0451
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,146,727
App. No.
17/200,659
Granted
Oct 12, 2021
Kind
B2
Abstract

A method for generating a panoramic image is disclosed. The method includes receiving a three-dimensional model of a target space and an initial panoramic image of the target space, the initial panoramic image having a latitude span less than a preset latitude span; determining a first set of coordinate parameters of a camera associated with the initial panoramic image and with respect to a reference frame associated with the three-dimensional model; mapping, based on the first set of coordinate parameters of the camera, data points of the three-dimensional model to a camera coordinate system associated with the camera to obtain an intermediate panoramic image; and obtaining a final panoramic image by merging the initial panoramic image and the intermediate panoramic image, the final panoramic image having a latitude span greater than or equal to the preset latitude span.

Claims (29)

1. A method for generating a panoramic image, the method comprising:

receiving a three-dimensional model of a target space and an initial panoramic image of the target space, the initial panoramic image having a latitude span less than a preset latitude span;

determining a first set of coordinate parameters of a camera associated with the initial panoramic image and with respect to a reference frame associated with the three-dimensional model;

mapping, based on the first set of coordinate parameters of the camera, data points of the three-dimensional model to a camera coordinate system associated with the camera to obtain an intermediate panoramic image;

determining a plurality of pixels in the intermediate panoramic image that correspond to the plurality of pixels in the initial panoramic image;

determining, based on a distance between the position of a pixel in the initial panoramic image and the center position of the initial panoramic image, a first weight;

determining, based on a distance between the position of a corresponding pixel in the intermediate panoramic image that corresponds to the pixel in the initial panoramic image and the center position of the intermediate panoramic image, a second weight;

based on the first weight and the second weight, obtaining fused color values of a plurality of pixels associated with a final panoramic image by performing a weighted summation on the color values of the plurality of pixels in the initial panoramic image and the color values of the corresponding plurality of pixels in the intermediate panoramic image that correspond to the plurality of pixels in the initial panoramic image; and

obtaining the final panoramic image by using the fused color values of the plurality of pixels associated with the final panoramic image, the final panoramic image having a latitude span greater than or equal to the preset latitude span.

2. A device for generating a panoramic image, the device comprising:

a non-transitory memory storage comprising instructions; and

a processor coupled with the non-transitory memory storage, wherein the processor executes the instructions to:

receive a three-dimensional model of a target space and an initial panoramic image of the target space, the initial panoramic image having a latitude span less than a preset latitude span;

determine a first set of coordinate parameters of a camera associated with the initial panoramic image and with respect to a reference frame associated with the three-dimensional model;

map, based on the first set of coordinate parameters of the camera, data points of the three-dimensional model to a camera coordinate system associated with the camera to obtain an intermediate panoramic image;

determine a plurality of pixels in the intermediate panoramic image that correspond to the plurality of pixels in the initial panoramic image;

determine, based on a distance between the position of a pixel in the initial panoramic image and the center position of the initial panoramic image, a first weight;

determine, based on a distance between the position of a corresponding pixel in the intermediate panoramic image that corresponds to the pixel in the initial panoramic image and the center position of the intermediate panoramic image, a second weight;

based on the first weight and the second weight, obtain fused color values of a plurality of pixels associated with a final panoramic image by performing a weighted summation on the color values of the plurality of pixels in the initial panoramic image and the color values of the corresponding plurality of pixels in the intermediate panoramic image that correspond to the plurality of pixels in the initial panoramic image; and

obtain the final panoramic image by using the fused color values of the plurality of pixels associated with the final panoramic image, the final panoramic image having a latitude span greater than or equal to the preset latitude span.

3. A non-transitory computer-readable medium, having computer-executable instructions stored thereon, the computer-executable instructions, when executed by one or more processor, cause a processor to facilitate:

receiving a three-dimensional model of a target space and an initial panoramic image of the target space, the initial panoramic image having a latitude span less than a preset latitude span;

determining a first set of coordinate parameters of a camera associated with the initial panoramic image and with respect to a reference frame associated with the three-dimensional model;

mapping, based on the first set of coordinate parameters of the camera, data points of the three-dimensional model to a camera coordinate system associated with the camera to obtain an intermediate panoramic image;

determining a plurality of pixels in the intermediate panoramic image that correspond to the plurality of pixels in the initial panoramic image;

determining, based on a distance between the position of a pixel in the initial panoramic image and the center position of the initial panoramic image, a first weight;

determining, based on a distance between the position of a corresponding pixel in the intermediate panoramic image that corresponds to the pixel in the initial panoramic image and the center position of the intermediate panoramic image, a second weight;

based on the first weight and the second weight, obtaining fused color values of a plurality of pixels associated with a final panoramic image by performing a weighted summation on the color values of the plurality of pixels in the initial panoramic image and the color values of the corresponding plurality of pixels in the intermediate panoramic image that correspond to the plurality of pixels in the initial panoramic image; and

obtaining the final panoramic image by using the fused color values of the plurality of pixels associated with the final panoramic image, the final panoramic image having a latitude span greater than or equal to the preset latitude span.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 14, 2022
From: KE.COM (BEIJING) TECHNOLOGY CO., LTD.
To: REALSEE (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 061426/0564 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 15, 2021
From: RAO, TONG; YANG, YONGLIN; CHEN, YUTONG; WANG, MINGYUAN
To: KE.COM (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 055596/0301 →
Priority Claims (2)
CN 202010180842.5 · Mar 16, 2020 · national
CN 202010196117.7 · Mar 19, 2020 · national
Continuity (1)
Related Publication 20210289135A1 · Sep 16, 2021