Granted Patent B2

US 12,348,774 · App. 18/471,753 · Granted Jul 1, 2025

Inventors: Pranav Aniruddha Kadam (Sunnyvale, CA); Alexandre Zaghetto (San Jose, CA); Danillo Bracco Graziosi (Flagstaff, AZ); Ali Tabatabai (Cupertino, CA)

Assignees: SONY GROUP CORPORATION; SONY CORPORATION OF AMERICA

H04N19/597H04N19/105H04N19/124H04N19/30H04N19/503H04N19/96

Abstract

Claims (61)

1. A first electronic device, comprising:

a memory configured to store a first neural network predictor; and

circuitry configured to:

receive a three-dimensional (3D) point cloud sequence that includes a set of reference 3D point cloud frames and a current 3D point cloud frame that is to be encoded;

generate reference frame data comprising a feature set associated with 3D points of each reference 3D point cloud frame of the set of reference 3D point cloud frames;

generate current frame data associated with 3D points of the current 3D point cloud frame,

wherein the current frame data comprises a first set of features associated with an occupancy of the 3D points in the current 3D point cloud frame;

predict a second set of features associated with the 3D points of the current 3D point cloud frame based on application of the first neural network predictor on the reference frame data;

compute a set of residual features based on the first set of features and the second set of features;

generate a set of quantized residual features based on application of a quantization scheme on the set of residual features; and

generate a bitstream of encoded point cloud data for the current 3D point cloud frame based on application of an encoding scheme on the set of quantized residual features.

2. The first electronic device according to claim 1 , wherein the second set of features are predicted further based on coordinate information associated with the 3D points of the current 3D point cloud frame.

3. The first electronic device according to claim 2 , wherein the circuitry is further configured to encode the coordinate information based on an application of an octree-based encoder on the coordinate information, and

the bitstream of encoded point cloud data includes the encoded coordinate information.

4. The first electronic device according to claim 1 , wherein

the memory is further configured to store a first Point Cloud Compression (PCC) encoder,

the reference frame data is generated based on an application of the first PCC encoder on the set of reference 3D point cloud frames, and

the current frame data is generated based on application of the first PCC encoder on the current 3D point cloud frame.

5. The first electronic device according to claim 1 , wherein the feature set associated with 3D points of each reference 3D point cloud frame of the set of reference 3D point cloud frames comprises:

reference features associated with an occupancy of the 3D points in a corresponding reference 3D point cloud frame of the set of reference 3D point cloud frames, and

reference coordinate information associated with the 3D points of the corresponding reference 3D point cloud frame.

6. The first electronic device according to claim 1 , wherein the set of reference 3D point cloud frames includes at least one reference 3D point cloud frame that precedes the current 3D point cloud frame or at least one reference 3D point cloud frame that succeeds the current 3D point cloud frame.

7. The first electronic device according to claim 1 , wherein the circuitry is further configured to down sample the feature set associated with 3D points of each reference 3D point cloud frame of the set of reference 3D point cloud frames by at least one scaling factor.

8. A second electronic device, comprising:

a memory configured to store a second neural network predictor; and

circuitry configured to:

receive a three-dimensional (3D) point cloud sequence that includes a set of reference 3D point cloud frames;

generate reference frame data comprising a feature set associated with 3D points of each reference 3D point cloud frame of the set of reference 3D point cloud frames;

receive a bitstream of encoded point cloud data associated with a current 3D point cloud frame that is to be decoded;

predict a third set of features associated with 3D points of the current 3D point cloud frame based on an application of the second neural network predictor on the reference frame data;

generate a fourth set of features associated with the 3D points of the current 3D point cloud frame based on the received bitstream of encoded point cloud data and the predicted third set of features; and

reconstruct the current 3D point cloud frame based on application of a decoding scheme on the generated fourth set of features.

9. The second electronic device according to claim 8 , wherein the received bitstream of encoded point cloud data further comprises encoded coordinate information associated with the 3D points of the current 3D point cloud frame.

10. The second electronic device according to claim 9 , wherein the circuitry is further configured to generate coordinate information associated with the 3D points of the current 3D point cloud frame based on application of an octree-based decoder on the encoded coordinate information.

11. The second electronic device according to claim 10 , wherein the third set of features are predicted further based on the generated coordinate information.

12. The second electronic device according to claim 8 , wherein the memory is further configured to store a Point Cloud Compression (PCC) encoder,

wherein the reference frame data is generated based on an application of the PCC encoder on the set of reference 3D point cloud frames.

13. The second electronic device according to claim 8 , wherein the feature set associated with 3D points of each reference 3D point cloud frame of the set of reference 3D point cloud frames comprises:

reference features associated with an occupancy of the 3D points in a corresponding reference 3D point cloud frame of the set of reference 3D point cloud frames, and

reference coordinate information associated with the 3D points of the corresponding reference 3D point cloud frame.

14. A method, comprising:

in a first electronic device:

receiving a three-dimensional (3D) point cloud sequence that includes a set of reference 3D point cloud frames and a current 3D point cloud frame that is to be encoded;

generating reference frame data comprising a feature set associated with 3D points of each reference 3D point cloud frame of the set of 3D point cloud reference frames;

generating current frame data associated with 3D points of the current 3D point cloud frame,

wherein the current frame data comprises a first set of features associated with an occupancy of the 3D points in the current 3D point cloud frame;

predicting a second set of features associated with the 3D points of the current 3D point cloud frame based on application of a first neural network predictor on the reference frame data;

computing a set of residual features based on the first set of features and the second set of features;

generating a set of quantized residual features based on application of a quantization scheme on the set of residual features; and

generating a bitstream of encoded point cloud data for the current 3D point cloud frame based on an application of an encoding scheme on the set of quantized residual features.

15. The method according to claim 14 , wherein the second set of features are predicted further based on coordinate information associated with the 3D points of the current 3D point cloud frame.

16. The method according to claim 15 , further comprising encoding the coordinate information based on application of an octree-based encoder on the coordinate information,

wherein the bitstream of encoded point cloud data includes the encoded coordinate information.

17. The method according to claim 14 , wherein,

the reference frame data is generated based on an application of a first Point Cloud Compression (PCC) encoder on the set of reference 3D point cloud frames, and

the current frame data is generated based on application of the first PCC encoder on the current 3D point cloud frame.

18. The method according to claim 14 , wherein the feature set associated with 3D points of each reference 3D point cloud frame of the set of reference 3D point cloud frames comprises:

reference features associated with an occupancy of the 3D points in a corresponding reference 3D point cloud frame of the set of reference 3D point cloud frames, and

reference coordinate information associated with the 3D points of the corresponding reference 3D point cloud frame.

19. The method according to claim 14 , wherein the set of reference 3D point cloud frames includes at least one reference 3D point cloud frame that precedes the current 3D point cloud frame or at least one reference 3D point cloud frame that succeeds the current 3D point cloud frame.

20. The method according to claim 14 , further comprising down sampling the feature set associated with 3D points of each reference 3D point cloud frame of the set of reference 3D point cloud frames by at least one scaling factor.

Assignments (1)

Continuity (3)

References Cited (20)