IP Library Granted Patent US 11,032,571
Granted Patent B2
US 11,032,571 · App. 16/739,568 · Granted Jun 8, 2021

Image processing method, terminal, and server

Inventors: Yi Song (Shenzhen, CN); Qingpeng Xie (Shenzhen, CN); Peiyun Di (Shenzhen, CN)
Assignee: HUAWEI TECHNOLOGIES CO., LTD.
H04N19/597G06T3/0037H04N19/115H04N19/132H04N19/162H04N19/176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,032,571
App. No.
16/739,568
Granted
Jun 8, 2021
Kind
B2
Abstract

An image processing method includes performing horizontal division and vertical division on a longitude-latitude map or a sphere map of a to-be-processed image to obtain sub-areas of the longitude-latitude map or the sphere map, where a division location of the horizontal division is a preset latitude, a division location of the vertical division is determined by a latitude, there are at least two types of vertical division intervals in an area formed by adjacent division locations of the horizontal division, and a vertical division interval is a distance between adjacent division locations of the vertical division, and encoding images of the sub-areas.

Claims (60)

1. An image processing method implemented by a server, the image processing method comprising:

determining a division location of vertical division using a latitude;

performing horizontal division and the vertical division on a longitude-latitude map or a sphere map of a to-be-processed image to obtain sub-areas of the longitude-latitude map or the sphere map, wherein a division location of the horizontal division is a preset latitude, wherein at least two types of vertical division intervals in an area are formed by adjacent division locations of the horizontal division, and wherein a vertical division interval is a distance between adjacent division locations of the vertical division;

sampling an image of a sub-area in a horizontal direction at a first sampling interval, wherein a higher latitude corresponding to the sub-area indicates a larger first sampling interval:

encoding, sampled images of sampled sub-areas; and

encoding images of the sub-areas after sampling the image of the sub-area in the horizontal direction at the first sampling interval and after encoding the sampled images of the sampled sub-areas.

2. The image processing method of claim 1 , wherein a higher latitude of a division location of the vertical division indicates a larger vertical division interval.

3. The image processing method of claim 1 , wherein before encoding the images of the sub-areas, the image processing method further comprises sampling the image of the sub-area in a vertical direction at a second sampling interval.

4. The image processing method of claim 1 , further comprising:

obtaining the sub-area by performing horizontal division and vertical division on the sphere map of the to-be-processed image;

mapping the image of the sub-area to a two-dimensional planar image based on a preset size before sampling the image of the sub-area in the horizontal direction at the first sampling interval; and

sampling, at the first sampling interval in the horizontal direction, the two-dimensional planar image.

5. The image processing method of claim 1 , wherein before encoding the images of the sampled sub-areas, the image processing method further comprises adjusting locations of the sampled sub-areas such that a horizontal edge and a vertical edge of a spliced image spliced by images of adjusted sub-areas are respectively aligned.

6. The image processing method of claim 5 , further comprising encoding a tile of the spliced image.

7. The image processing method of claim 1 , wherein after encoding the images of the sub-areas, the image processing method further comprises:

independently encapsulating bitstreams corresponding to the images of the sub-areas; and

encoding location information of the sub-areas to obtain encoded location information of each of the sub-areas, wherein:

the encoded location information of each of the sub-areas and a bitstream of each of the sub-areas exist in a same track;

the encoded location information of each of the sub-areas and the bitstream of each of the sub-areas respectively exist in a track of location information and a track of the bitstream of each of the sub-areas;

the encoded location information of each of the sub-areas exists in a media presentation description (MPD);

the encoded location information of each of the sub-areas exists in a private file, wherein an address of the private file exists in the MPD; or

the encoded location information of each of the sub-areas exists in supplemental enhancement information (SEI) of the bitstream of each of the sub-areas.

8. The image processing method of claim 7 , further comprising:

forming, by sampled sub-areas, a sampled longitude-latitude map when the sub-areas are obtained by performing the horizontal division and the vertical division on the longitude-latitude map of the to-be-processed image, wherein the location information comprises either:

a location and a size of each of the sub-areas in the longitude-latitude map and a location and a size of each of sub-areas in the sampled longitude-latitude map; or

the location and the size of each of the sub-areas in the longitude-latitude map and a location and a size of each of sub-areas in a spliced image; and

forming, by the sampled sub-areas, a sampled sphere map when the sub-areas are obtained by performing the horizontal division and the vertical division on the sphere map of the to-be-processed image, wherein the location information comprises either:

a location and a latitude-longitude range of each of sub-areas in an image of the sphere map and a location and a size of each of sub-areas in an image of the sampled sphere map; or

the location and the latitude-longitude range of each of the sub-areas in the image of the sphere map and the location and the size of each of the sub-areas in the spliced image.

9. The image processing method of claim 8 , further comprising:

sending, to a terminal, the bitstreams corresponding to the images of the sub-areas;

receiving visual angle information from the terminal, obtaining, based on the visual angle information, a sub-area corresponding to the visual angle information, and sending a bitstream of the sub-area corresponding to the visual angle information to the terminal; or

receiving a number of a sub-area from the terminal, and sending a bitstream corresponding to the number of the sub-area to the terminal.

10. The image processing method of claim 7 , wherein the private file further comprises information representing a correspondence between a user viewpoint and a number of a sub-area covered by a visual angle of the user viewpoint.

11. The image processing method of claim 7 , wherein the private file further comprises information representing a quantity of sub-areas that to be displayed in a sub-area covered by a visual angle of a user, information about a number of the sub-area that to be displayed, and information about a number of a sub-area that is secondarily displayed.

12. The image processing method of claim 1 , wherein the longitude-latitude map comprises a longitude-latitude map corresponding to a left eye and a longitude-latitude map corresponding to a right eye, and wherein before performing the horizontal division and the vertical division on the longitude-latitude map or the sphere map of the to-be-processed image, the image processing method further comprises:

separating the longitude-latitude map corresponding to the left eye from the longitude-latitude map corresponding to the right eye;

performing the horizontal division and the vertical division on the longitude-latitude map corresponding to the left eye; and

performing the horizontal division and the vertical division on the longitude-latitude map corresponding to the right eye.

13. The image processing method of claim 1 , wherein the longitude-latitude map is a longitude-latitude map of a 360-degree panorama video image or a part of the longitude-latitude map of the 360-degree panorama video image, and wherein the sphere map is a sphere map of the 360-degree panorama video image or a part of the sphere map of the 360-degree panorama video image.

14. The image processing method of claim 1 , wherein after encoding the images of the sub-areas, the image processing method further comprises:

independently encapsulating bitstreams corresponding to the images of the sub-areas; and

encoding location information of the sub-areas to obtain encoded location information of each of the sub-areas, wherein the encoded location information of each of the sub-areas and a bitstream of each of the sub-areas exist in a same track.

15. An image processing method implemented by a terminal, the image processing method comprises:

determining first location information of each of a plurality of sub-areas of a panorama image by:

receiving a media presentation description (MPD) from a server, wherein the MPD comprises either the first location information or an address of a private file, and wherein the private file comprises the first location information; and

parsing, the MPD to obtain the first location information;

determining, based on the first location information, second location information of a first sub-area of the plurality of sub-areas covered by a current visual angle in the panorama image;

determining a first sampling interval of the first sub-area;

obtaining, a bitstream corresponding to the first sub-area based on the second location information;

decoding the bitstream to obtain an image of the first sub-area;

resampling the image based on the second location information and the first sampling interval to obtain a resampled image; and

playing the resampled image.

16. The image processing method of claim 15 , further comprising:

receiving first information from the server, wherein the first information comprises a track of each of the sub-areas of the panorama image and a bitstream of each of the sub-areas, and wherein the track of each of the sub-areas comprises the first location information; and

obtaining the first location information based on the track of each of the sub-areas.

17. The image processing method of claim 15 , wherein the first location information exists in supplemental enhancement information (SEI) of a bitstream corresponding to each of the sub-areas.

18. The image processing method of claim 15 , further comprising determining a preset sampling interval as the first sampling interval.

19. The image processing method of claim 15 , further comprising obtaining the first location information based on the track of each of the sub-areas.

20. The image processing method of claim 15 , wherein the MPD comprises the address of the private file, and wherein the private file stores description information of sub-area division.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 2, 2021
From: SONG, YI; XIE, QINGPENG; DI, PEIYUN
To: HUAWEI TECHNOLOGIES CO., LTD.
Reel/Frame 055457/0363 →
Priority Claims (1)
CN 201710645108.X · Jul 31, 2017 · national
Continuity (2)
Continuation PCTCN2018081177 · Mar 29, 2018
Related Publication 20200154138A1 · May 14, 2020
Cited By (1)
US 12,238,394