IP Library Granted Patent US 11,361,471
Granted Patent B2
US 11,361,471 · App. 15/931,349 · Granted Jun 14, 2022

Point cloud occupancy map compression

Inventors: Khaled Mammou (Vancouver, CA); Yeping Su (Cupertino, CA); David W. Singer (San Francisco, CA); Alexandros Tourapis (Los Gatos, CA); Fabrice A. Robinet (Sunnyvale, CA); Hsi-Jung Wu (San Jose, CA)
Assignee: Apple Inc.
G06T9/00A01C3/02G06T1/00G06T7/20G06T7/50G06T7/70G06T9/001G08B1/00G09B1/00G06T2207/10028G06T2213/00G11B7/002
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,361,471
App. No.
15/931,349
Granted
Jun 14, 2022
Kind
B2
Abstract

A system comprises an encoder configured to compress attribute information and/or spatial for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. To compress the attribute and/or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. A block/sub-block organization scheme is used to encode blocks and sub-blocks of an occupancy map used in compressing the point cloud. Binary values are assigned to blocks/sub-blocks based on whether they contain patches projected on the point cloud. A traversal path is chosen that takes advantage of run-length encoding strategies to reduce a size of an encoded occupancy map. Also, auxiliary information is used to further improve occupancy map compression.

Claims (79)

1. A non-transitory computer-readable medium storing program instructions, that when executed by one or more processors, cause the one or more processors to:

decompress compressed occupancy information for an image frame comprising packed patches, wherein the compressed occupancy information comprises binary values corresponding to pixels of the image frame that indicate whether the corresponding pixels are occupied with one or more of the packed patches or are unoccupied, and

wherein to decompress the compressed occupancy information, the program instructions cause the one or more processors to:

identify, based on the compressed occupancy information, non-empty blocks of pixels of the image frame; and

for each non-empty block:

determine a traversal order for traversing sub-blocks of the non-empty block; and

assign respective ones of the binary values to the sub-blocks of the non-empty block according to the determined traversal order to indicate whether the pixels of the sub-blocks are occupied or unoccupied, wherein the respective binary values are assigned based on an order of the binary values included in the compressed occupancy information.

2. The non-transitory computer-readable medium of claim 1 , wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

receive an encoded version of the occupancy information that has been encoded using an arithmetic encoding technique or an entropy encoding technique; and

decode the encoded version of the occupancy information to generate the compressed occupancy information that is to be decompressed.

3. The non-transitory computer-readable medium of claim 2 , wherein a bit stream comprising the encoded version of the occupancy information also comprises indicators of respective traversal orders to be used to traverse the sub-blocks of the respective non-empty blocks.

4. The non-transitory computer-readable medium of claim 3 , wherein the respective traversal orders comprise one or more of:

a diagonal traversal order across the sub-blocks of a given block;

a top-to-bottom traversal order across the sub-blocks of a given block;

a bottom-to-top traversal order across the sub-blocks of a given block; or

a side-to-side traversal order across the sub-blocks of a given block.

5. The non-transitory computer-readable medium of claim 4 , wherein the binary values included in the compressed occupancy information comprise:

encoded multi-bit words for each non-empty block, wherein each multi-bit word comprises a grouping of occupied or unoccupied binary values for the sub-blocks of a given non-empty block when traversed according to the determined traversal order for the given non-empty block.

6. A device, comprising:

a memory storing program instructions for decompressing compressed occupancy information for an image frame comprising packed patches, wherein the compressed occupancy information comprises binary values corresponding to pixels of the image frame that indicate whether the corresponding pixels are occupied with one or more of the packed patches or are unoccupied; and

one or more processors, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

determine non-empty blocks of pixels of the image frame;

for each non-empty block:

determine a traversal order for traversing sub-blocks of the non-empty block; and

assign respective ones of the binary values to the sub-blocks of the non-empty block according to the determined traversal order to indicate whether the pixels of the sub-blocks are occupied or unoccupied, wherein the binary values are assigned based on an order of binary values included in the compressed occupancy information.

7. The device of claim 6 , wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

receive an encoded bit stream comprising:

the binary values that correspond to the non-empty blocks; and

indicators of respective traversal orders to be used to traverse the sub-blocks of the respective non-empty blocks.

8. The device of claim 7 , wherein the respective traversal orders comprise one or more of:

a diagonal traversal order across the sub-blocks of a given block;

a top-to-bottom traversal order across the sub-blocks of a given block;

a bottom-to-top traversal order across the sub-blocks of a given block; or

a side-to-side traversal order across the sub-blocks of a given block.

9. The device of claim 8 , wherein the encoded bit stream further comprises:

indicators of respective run lengths corresponding to the respective traversal orders.

10. The device of claim 6 , wherein a resolution of the occupancy information for the image frames is less than a resolution of the image frame, such that a given binary value of the occupancy information corresponds to two or more pixels of the image frame.

11. A non-transitory computer-readable medium storing program instructions, that when executed by one or more processors, cause the one or more processors to:

compress occupancy information for an image frame comprising packed patches, wherein the occupancy information comprises binary values corresponding to pixels of the image frame, and wherein the binary values indicate pixels of the image frame that are occupied by one or more of the packed patches and other pixels of the image frame that are unoccupied by the packed patches,

wherein to compress the occupancy information, the program instructions cause the one or more processors to:

identify non-empty blocks of pixels of the image frame that include at least one occupied pixel;

assign binary values to sub-blocks of each non-empty block of the image frame, wherein occupied sub-blocks comprising at least one occupied pixel and unoccupied sub-blocks that do not comprise an occupied pixel are assigned different binary values;

select a traversal order for the sub-blocks of each respective non-empty block such that sub-blocks assigned a same one of the binary values are traversed sequentially; and

encode, for each non-empty block, the traversal order and sequential binary values for the sub-blocks traversed according to the selected traversal order.

12. The non-transitory computer-readable medium of claim 11 , wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

identify full blocks of the image frame that do not include unoccupied pixels; and

encode, for each block of the image frame, a first binary value if the block is full and a second binary value if the block is non-empty, wherein a non-empty block includes at least one unoccupied pixel.

13. The non-transitory computer-readable medium of claim 12 , wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

select a traversal order for the blocks of the image frame such that blocks assigned a same one of the binary values are traversed sequentially; and

include in the compressed occupancy information, the selected traversal order for the blocks of the image frame.

14. The non-transitory computer-readable medium of claim 13 , wherein the traversal order for the blocks of the image frame or the respective traversal orders for the sub-blocks of respective ones of the non-empty blocks of the image frame comprise one or more of:

a diagonal traversal order;

a top-to-bottom traversal order;

a bottom-to-top traversal order; or

a side-to-side traversal order.

15. The non-transitory computer-readable medium of claim 14 , wherein:

sequential binary values for the blocks traversed according to the selected traversal order for the blocks; and

sequential binary values for respective ones of the non-empty sub-blocks traversed according to the respective selected traversal orders for the non-empty sub-blocks,

are encoded as respective multi-bit words, wherein each multi-bit word comprises:

the sequential binary values for the blocks of the image frame; or

the sequential binary values for the sub-blocks of a given non-empty block.

16. The non-transitory computer-readable medium of claim 15 , wherein the program instructions, when executed by the one or more processors, cause the one or more processors to encode the multi-bit words using an arithmetic encoding technique or an entropy encoding technique.

17. A device, comprising:

a memory storing program instructions for compressing occupancy information for an image frame comprising packed patches; and

one or more processors, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

identify non-empty blocks of pixels of the image frame that include at least one occupied pixel;

assign binary values to sub-blocks of each non-empty block of the image frame, wherein occupied sub-blocks and unoccupied sub-blocks are assigned different binary values;

select a traversal order for the sub-blocks of each respective non-empty block such that sub-blocks assigned a same one of the binary values are traversed sequentially; and

signal, for each non-empty block the sequential binary values for the sub-blocks traversed according to the traversal order.

18. The device of claim 17 , wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

signal in a bit stream comprising the sequential binary values, the selected traversal orders for the non-empty blocks of the image frame.

19. The device of claim 17 , wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

identify full blocks of the image frame that do not include unoccupied pixels; and

signal, for each block of the image frame, a first binary value if the block is full and a second binary value if the block is non-empty, wherein a non-empty block includes at least one unoccupied pixel.

20. The device of claim 17 , wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

select a traversal order for the blocks of the image frame such that blocks assigned a same one of the binary values are traversed sequentially

signal a multi-bit word for the sequential binary values for the blocks of the image frame traversed according to the selected traversal order for the blocks of the image frame;

signal respective multi-bit words for the sequential binary values for the sub-blocks of respective ones of the non-empty blocks of the image frame traversed according to the respective selected traversal orders for the non-empty blocks; and

encode the multi-bit words using an arithmetic encoding technique or an entropy encoding technique.

Continuity (3)
Continuation 16198635 · Nov 21, 2018
Provisional Application 62590206 · Nov 22, 2017
Related Publication 20200273208A1 · Aug 27, 2020
Cited By (5)
US 12,243,197 US 12,327,381 US 12,439,083 US 12,452,427 US 12,688,619