Media file unpacking method and apparatus, device, and storage medium

1 . A media file unpacking method, performed by a file unpacking device having at least one processor, the method comprising:

receiving first indication information transmitted by a file packing device, the first indication information indicating metadata of a recommended viewport, the metadata of the recommended viewport comprising feature information of immersive media corresponding to the recommended viewport, and the recommended viewport having been determined according to content of the immersive media;

determining a media file of the immersive media corresponding to the recommended viewport by associating the recommended viewport with the feature information of the immersive media corresponding to the recommended viewport;

based on the first indication information, determining whether to request the metadata of the recommended viewport, and determining whether to request the media file of the immersive media corresponding to the recommended viewport;

based on determining to request the metadata of the recommended viewport, transmitting first request information to the file packing device, the first request information requesting the metadata of the recommended viewport;

receiving a metadata track of the recommended viewport transmitted by the file packing device, and unpacking and then decoding the metadata track of the recommended viewport to obtain the metadata of the recommended viewport; and

unpacking and then decoding, according to the feature information of the immersive media corresponding to the recommended viewport, the media file of the immersive media corresponding to the recommended viewport to obtain the content of the immersive media corresponding to the recommended viewport.

2 . The method according to claim 1 , wherein the immersive media comprises at least one of multi-view video media and point cloud media.

3 . The method according to claim 2 , further comprising:

transmitting second request information to the file packing device according to the feature information of the immersive media corresponding to the recommended viewport in the metadata of the recommended viewport, wherein the second request information requests the media file of the immersive media corresponding to the recommended viewport; and

receiving the media file of the immersive media corresponding to the recommended viewport transmitted by the file packing device, and unpacking and then decoding the media file of the immersive media corresponding to the recommended viewport, to obtain content of the immersive media corresponding to the recommended viewport.

4 . The method according to claim 3 , wherein based on the immersive media being multi-view video media, a video track of the immersive media being divided according to viewpoints or viewpoint groups, and the recommended viewport being associated with view information of the multi-view video media corresponding to the recommended viewport, the transmitting the second request information to the file packing device comprises:

transmitting the second request information to the file packing device according to the view information of the multi-view video media corresponding to the recommended viewport, the second request information comprising the view information of the multi-view video media corresponding to the recommended viewport.

5 . The method according to claim 4 , wherein before transmitting the second request information to the file packing device, the method further comprises:

determining whether the metadata of the recommended viewport comprises a first flag indicating that the recommended viewport is associated with the view information of the multi-view video media corresponding to the recommended viewport; and

transmitting the second request information to the file packing device according to the view information of the multi-view video media corresponding to the recommended viewport comprises:

based on determining that the metadata of the recommended viewport comprises the first flag, transmitting the second request information to the file packing device according to the view information of the multi-view video media corresponding to the recommended viewport.

6 . The method according to claim 3 , wherein based on the immersive media being the point cloud media, the point cloud media being packed according to point cloud tiles, the point cloud tiles of the point cloud media having different quality rankings, the recommended viewport being associated with alternative group selection information of the point cloud tiles corresponding to the recommended viewport, and the alternative group selection information of the point cloud tiles comprising at least one of identification information of component tracks corresponding to the point cloud tiles and quality rankings corresponding to the point cloud tiles, the transmitting second request information to the file packing device according to the feature information of the immersive media corresponding to the recommended viewport in the metadata of the recommended viewport comprises:

transmitting the second request information to the file packing device according to the alternative group selection information of the point cloud tiles corresponding to the recommended viewport, the second request information comprising the alternative group selection information of the point cloud tiles.

7 . The method according to claim 6 , wherein before transmitting the second request information to the file packing device according to the alternative group selection information of the point cloud tiles corresponding to the recommended viewport, the method further comprises:

determining whether the metadata of the recommended viewport comprises a second flag indicating that the recommended viewport is associated with the alternative group selection information of the point cloud tiles corresponding to the recommended viewport; and

transmitting the second request information to the file packing device according to the alternative group selection information of the point cloud tiles corresponding to the recommended viewport comprises:

based on determining that the metadata of the recommended viewport comprises the second flag, transmitting the second request information to the file packing device according to the alternative group selection information of the point cloud tiles corresponding to the recommended viewport.

8 . The method according to claim 2 , further comprising:

transmitting third request information that requests the media file of the immersive media to the file packing device; and

receiving the media file of the immersive media transmitted by the file packing device.

9 . The method according to claim 8 , wherein based on the immersive media being the point cloud media, the point cloud media being packed according to point cloud tiles, the point cloud tiles of the point cloud media having different quality rankings, the recommended viewport being associated with alternative group selection information of the point cloud tiles corresponding to the recommended viewport, and the alternative group selection information of the point cloud tiles comprising at least one of identification information of component tracks corresponding to the point cloud tiles and quality rankings corresponding to the point cloud tiles, the unpacking and then decoding, according to the feature information of the immersive media corresponding to the recommended viewport, the media file of the immersive media corresponding to the recommended viewport to obtain the content of the immersive media corresponding to the recommended viewport comprises:

querying, according to alternative group selection information of the point cloud tiles corresponding to the recommended viewport, a media file corresponding to the alternative group selection information in received media files of the immersive media; and

unpacking and then decoding a queried media file corresponding to the alternative group selection information, to obtain the content of the immersive media corresponding to the recommended viewport.

10 . The method according to claim 9 , wherein before the querying, the method further comprises:

determining whether the metadata of the recommended viewport comprises a second flag indicating that the recommended viewport is associated with the alternative group selection information of the point cloud tiles corresponding to the recommended viewport; and

the querying comprises:

based on determining that the metadata of the recommended viewport comprises the second flag, querying, according to alternative group selection information of the point cloud tiles corresponding to the recommended viewport, the media file corresponding to the alternative group selection information in the received media files of the immersive media.

11 . A media file unpacking apparatus comprising:

at least one memory configured to store computer program code; and

at least one processor configured to access the at least one memory and operate according to the computer program code, the computer program code including:

first receiving code configured to cause at least one of the at least one processor to receive first indication information transmitted by a file packing device, the first indication information indicating metadata of a recommended viewport, the metadata of the recommended viewport comprising feature information of immersive media corresponding to the recommended viewport, and the recommended viewport having been determined according to content of the immersive media;

first determining code configured to cause at least one of the at least one processor to determine a media file of the immersive media corresponding to the recommended viewport by associating the recommended viewport with the feature information of the immersive media corresponding to the recommended viewport;

second determining code configured to cause at least one of the at least one processor to determine, based on the first indication information, whether to request the metadata of the recommended viewport, and determining whether to request the media file of the immersive media corresponding to the recommended viewport; and

first transmitting code configured to cause at least one of the at least one processor to transmit, based on determining to request the metadata of the recommended viewport, first request information to the file packing device requesting the metadata of the recommended viewport;

second receiving code configured to cause at least one of the at least one processor to receive a metadata track of the recommended viewport transmitted by the file packing device, and unpacking and then decoding the metadata track of the recommended viewport to obtain the metadata of the recommended viewport; and

unpacking and decoding code configured to cause at least one of the at least one processor to unpack and then decode, according to the feature information of the immersive media corresponding to the recommended viewport, the media file of the immersive media corresponding to the recommended viewport to obtain the content of the immersive media corresponding to the recommended viewport.

12 . The apparatus according to claim 11 , wherein the immersive media comprises at least one of multi-view video media and point cloud media, further comprising:

second transmitting code configured to cause at least one of the at least one processor to transmit second request information to the file packing device according to the feature information of the immersive media corresponding to the recommended viewport in the metadata of the recommended viewport, wherein the second request information requests the media file of the immersive media corresponding to the recommended viewport; and

third receiving code configured to cause at least one of the at least one processor to receive the media file of the immersive media corresponding to the recommended viewport transmitted by the file packing device, and unpacking and then decoding the media file of the immersive media corresponding to the recommended viewport, to obtain content of the immersive media corresponding to the recommended viewport.

13 . The apparatus according to claim 12 , wherein based on the immersive media being multi-view video media, a video track of the immersive media being divided according to viewpoints or viewpoint groups, and the recommended viewport being associated with view information of the multi-view video media corresponding to the recommended viewport, the second transmitting code is further configured to cause at least one of the at least one processor to:

transmit the second request information to the file packing device according to the view information of the multi-view video media corresponding to the recommended viewport, the second request information comprising the view information of the multi-view video media corresponding to the recommended viewport.

14 . The apparatus according to claim 13 , wherein the first determining code is further configured to cause at least one of the at least one processor to:

determine whether the metadata of the recommended viewport comprises a first flag indicating that the recommended viewport is associated with the view information of the multi-view video media corresponding to the recommended viewport; and

the second transmitting code is further configured to cause at least one of the at least one processor to:

transmit, based on determining that the metadata of the recommended viewport comprises the first flag, the second request information to the file packing device according to the view information of the multi-view video media corresponding to the recommended viewport.

15 . The apparatus according to claim 12 , wherein based on the immersive media being the point cloud media, the point cloud media being packed according to point cloud tiles, the point cloud tiles of the point cloud media having different quality rankings, the recommended viewport being associated with alternative group selection information of the point cloud tiles corresponding to the recommended viewport, and the alternative group selection information of the point cloud tiles comprising at least one of identification information of component tracks corresponding to the point cloud tiles and quality rankings corresponding to the point cloud tiles, the second transmitting code is further configured to cause at least one of the at least one processor to:

transmit the second request information to the file packing device according to the alternative group selection information of the point cloud tiles corresponding to the recommended viewport, the second request information comprising the alternative group selection information of the point cloud tiles.

16 . The apparatus according to claim 15 , wherein second determining code is further configured to:

determine whether the metadata of the recommended viewport comprises a second flag indicating that the recommended viewport is associated with the alternative group selection information of the point cloud tiles corresponding to the recommended viewport; and

the second transmitting code is further configured to cause at least one of the at least one processor to:

transmit, based on determining that the metadata of the recommended viewport comprises the second flag, the second request information to the file packing device according to the alternative group selection information of the point cloud tiles corresponding to the recommended viewport.

17 . The apparatus according to claim 12 , wherein the first transmitting code is further configured to cause at least one of the at least one processor to:

third transmitting code configured to cause at least one of the at least one processor to transmit third request information that requests the media file of the immersive media to the file packing device; and

fourth receiving code configured to cause at least one of the at least one processor to receive the media file of the immersive media transmitted by the file packing device.

18 . The apparatus according to claim 17 , wherein based on the immersive media being the point cloud media, the point cloud media being packed according to point cloud tiles, the point cloud tiles of the point cloud media having different quality rankings, the recommended viewport being associated with alternative group selection information of the point cloud tiles corresponding to the recommended viewport, and the alternative group selection information of the point cloud tiles comprising at least one of identification information of component tracks corresponding to the point cloud tiles and quality rankings corresponding to the point cloud tiles, the unpacking and then decoding of the third receiving code causes the at least one processor to:

query, according to alternative group selection information of the point cloud tiles corresponding to the recommended viewport, a media file corresponding to the alternative group selection information in received media files of the immersive media; and

unpack and then decode a queried media file corresponding to the alternative group selection information, to obtain the content of the immersive media corresponding to the recommended viewport.

19 . The apparatus according to claim 18 , wherein the third receiving code is further configured to cause at least one of the at least one processor to:

before querying, determine whether the metadata of the recommended viewport comprises a second flag indicating that the recommended viewport is associated with the alternative group selection information of the point cloud tiles corresponding to the recommended viewport; and

wherein querying comprises:

based on determining that the metadata of the recommended viewport comprises the second flag, querying, according to alternative group selection information of the point cloud tiles corresponding to the recommended viewport, the media file corresponding to the alternative group selection information in the received media files of the immersive media.

20 . A non-transitory computer readable storage that stores media file unpacking program code which, when executed by at least one processor, causes the at least one processor to at least:

receive first indication information transmitted by a file packing device, the first indication information indicating metadata of a recommended viewport, the metadata of the recommended viewport comprising feature information of immersive media corresponding to the recommended viewport, and the recommended viewport having been determined according to content of the immersive media;

determine a media file of the immersive media corresponding to the recommended viewport by associating the recommended viewport with the feature information of the immersive media corresponding to the recommended viewport;

determine whether to request the metadata of the recommended viewport based on the first indication information;

determine whether to request the media file of the immersive media corresponding to the recommended viewport;

based on determining to request the metadata of the recommended viewport, transmit first request information to the file packing device, the first request information requesting the metadata of the recommended viewport;

receive a metadata track of the recommended viewport transmitted by the file packing device, and unpack and then decode the metadata track of the recommended viewport to obtain the metadata of the recommended viewport; and

unpack and then decode, according to the feature information of the immersive media corresponding to the recommended viewport, the media file of the immersive media corresponding to the recommended viewport to obtain the content of the immersive media corresponding to the recommended viewport.