IP Library › Granted Patent US 10,609,419
Granted Patent B2
US 10,609,419 · App. 15/735,514 · Granted Mar 31, 2020

Encoding apparatus, decoding apparatus and transmission control method

Inventor: Yuji Fujimoto (Kanagawa, JP)
Assignee: SONY CORPORATION
H04N19/89H04N19/105H04N19/162H04N19/164H04N19/174H04N19/55H04N19/65H04N19/70H04N19/895
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,609,419
App. No.
15/735,514
Granted
Mar 31, 2020
Kind
B2
Abstract

The present invention makes it possible to easily implement a mechanism to recover an appropriately decoded video in a situation where encoded information for decoding the video has been lost or is absent. The present invention provides an encoding apparatus including: a setting section configured to partition each of images included in a video to be encoded into a plurality of tiles and set a partial region including one or more of the plurality of tiles for the image; an encoding section configured to encode each image on a per-tile basis to generate an encoded stream; and a transmission control section configured to control transmission of the encoded stream to a decoding apparatus that decodes the video. The setting section is configured to set out-of-tile reference for motion compensation for the tiles within the partial region to be prohibited, and the transmission control section is configured to restrict, upon detection of a loss or an absence of encoded information at the decoding apparatus, the transmission such that only an encoded stream corresponding to the tiles within the partial region is transmitted.

Claims (72)

1. An encoding apparatus, comprising:

a processor configured to:

partition each image of a plurality of images included in a video into a plurality of tiles;

set a first partial region that includes at least one tile of the plurality of tiles for an image of the plurality of images;

encode each image of the plurality of images on a per-tile basis to generate an encoded stream;

control transmission of the encoded stream to a decoding apparatus that decodes the video;

set out-of-tile reference for motion compensation to be prohibited for the at least one tile within the first partial region;

restrict, based on detection of one of a loss or an absence of encoded information at the decoding apparatus, the transmission of the encoded stream such that only a first part of the encoded stream corresponding to the at least one tile within the first partial region is transmitted;

progressively extend the first partial region tile by tile during a time of the restriction of the transmission of the encoded stream; and

encode, as an intra tile, a tile corresponding to a newly extended part of the first partial region.

2. The encoding apparatus according to claim 1 , wherein the processor is further configured to:

encode, as the intra tile, the at least one tile within the first partial region, wherein the at least one tile within the first partial region is non-decodable due to one of the loss or the absence of the encoded information, and

transmit a second part of the encoded stream corresponding to the intra tile to the decoding apparatus during the time of the restriction of the transmission of the encoded stream.

3. The encoding apparatus according to claim 1 , wherein

a set of tiles of the plurality of tiles outside the first partial region is non-decodable due to one of the loss or the absence of the encoded information, and

the set of tiles of the plurality of tiles outside the first partial region is decodable through the progressive extension of the first partial region.

4. The encoding apparatus according to claim 1 , wherein the processor is further configured to reset the first partial region after all tiles of the plurality of tiles have become decodable through the progressive extension of the first partial region.

5. The encoding apparatus according to claim 1 , wherein the processor is further configured to:

dynamically determine a timing based on availability of transmission bandwidth during the time of the restriction of the transmission of the encoded stream; and

extend the first partial region at the determined timing.

6. The encoding apparatus according to claim 1 , wherein the processor is further configured to set, in a case where the first partial region is non-rectangular, each tile of the plurality of tiles such that the image of the plurality of images decoded by the decoding apparatus is displayed rectangularly.

7. The encoding apparatus according to claim 1 , wherein the processor is further configured to set the first partial region for the image of the plurality of images based on one of specific region information, region information input by a user, or analysis of the video.

8. The encoding apparatus according to claim 1 , wherein the processor is further configured to set the first partial region for the image of the plurality of images based on reception of region information from the decoding apparatus.

9. The encoding apparatus according to claim 1 , wherein the processor is further configured to detect the loss of the encoded information based on a loss of a packet in the transmission of the encoded stream to the decoding apparatus.

10. The encoding apparatus according to claim 1 , wherein the processor is further configured to detect the absence of the encoded information based on a request of a cut-in reproduction of the video.

11. The encoding apparatus according to claim 1 , wherein the processor is further configured to:

determine a scene change based on analysis of the video; and

restrict, based on the determined scene change, the transmission of the encoded stream such that only the first part of the encoded stream corresponding to the at least one tile within the first partial region is transmitted.

12. The encoding apparatus according to claim 1 , wherein the processor is further configured to:

set a second partial region and a third partial region for the image of the plurality of images, wherein the third partial region is smaller than the second partial region;

restrict, based on a loss of a packet in the transmission of the encoded stream, the transmission of the encoded stream such that only a second part of the encoded stream corresponding to a first set of tiles of the plurality of tiles within the second partial region is transmitted; and

restrict, based on a request of a cut-in reproduction of the video, the transmission of the encoded stream such that only a third part of the encoded stream corresponding to a second set of tiles of the plurality of tiles within the third partial region is transmitted.

13. The encoding apparatus according to claim 1 , wherein

the processor is further configured to encode each image of the plurality of images based on high efficiency video coding (HEVC) scheme, and

a supplemental enhancement information (SEI) message includes a parameter that indicates the set out-of-tile reference.

14. A transmission control method, comprising:

in an encoding apparatus:

partitioning each image of a plurality of images included in a video into a plurality of tiles;

setting a partial region that includes at least one tile of the plurality of tiles for an image of the plurality of images;

encoding each image of the plurality of images on a per-tile basis to generate an encoded stream;

controlling transmission of the encoded stream to a decoding apparatus;

setting out-of-tile reference for motion compensation to be prohibited for the at least one tile within the partial region;

restricting, based on detection of one of a loss or an absence of encoded information at the decoding apparatus, the transmission of the encoded stream such that only a part of the encoded stream corresponding to the at least one tile within the partial region is transmitted;

progressively extending the partial region tile by tile during a time of the restriction of the transmission of the encoded stream; and

encoding, as an intra tile, a tile corresponding to a newly extended part of the partial region.

15. A decoding apparatus, comprising:

a processor configured to:

transmit, to an encoding apparatus, region information regarding a partial region that includes at least one tile of a plurality of tiles of an image included in a video;

receive, from the encoding apparatus, an encoded stream of the video,

wherein, in a normal operation, the encoded stream of the video corresponds to each of the plurality of tiles;

decode the encoded stream of the video to obtain the video; and

receive, based on detection of one of a loss or an absence of encoded information, only a first part of the encoded stream corresponding to the at least one tile within the partial region, wherein

out-of-tile reference for motion compensation is prohibited for the at least one tile within the partial region,

the partial region is progressively extended, by the encoding apparatus, tile by tile during a time of restriction of transmission of the encoded stream, and

a tile corresponding to a newly extended part of the partial region is encoded, by the encoding apparatus, as an intra tile.

16. The decoding apparatus according to claim 15 , wherein

the at least one tile of the plurality of tiles within the partial region is encoded as the intra tile,

the at least one tile within the partial region is non-decodable due to one of the loss or the absence of the encoded information, and

the processor is further configured to receive, from the encoding apparatus, a second part of the encoded stream corresponding to the intra tile during a time when a target of transmission of the encoded stream is restricted to the partial region.

17. The decoding apparatus according to claim 15 , wherein the processor is further configured to:

control reproduction of the decoded stream of the video; and

scale an image of the partial region to fit with a frame size, wherein the image of the partial region is partially decoded based on the detection of one of the loss or the absence of the encoded information.

18. A transmission control method, comprising:

in a decoding apparatus:

transmitting, to an encoding apparatus, region information regarding a partial region that includes at least one tile of a plurality of tiles of an image included in a video;

receiving, from the encoding apparatus, an encoded stream of the video,

wherein, in a normal operation, the encoded stream of the video corresponds to each of the plurality of tiles;

decoding the encoded stream of the video to obtain the video; and

receiving, based on detection of one of a loss or an absence of encoded information, only a part of the encoded stream corresponding to the at least one tile within the partial region, wherein

out-of-tile reference for motion compensation is prohibited for the at least one tile within the partial region,

the partial region is progressively extended, by the encoding apparatus, tile by tile during a time of restriction of transmission of the encoded stream, and

a tile corresponding to a newly extended part of the partial region is encoded, by the encoding apparatus, as an intra tile.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 11, 2017
From: FUJIMOTO, YUJI
To: SONY CORPORATION
Reel/Frame 044357/0131 →
Priority Claims (1)
JP 2015-126486 · Jun 24, 2015 · national
Continuity (1)
Related Publication 20180176603A1 · Jun 21, 2018