IP Library Granted Patent US 12,243,197
Granted Patent B2
US 12,243,197 · App. 18/352,250 · Granted Mar 4, 2025

Image processing apparatus and image processing method for suppressing an increase in processing time of a filter process for point cloud data

Inventors: Koji Yano (Tokyo, JP); Tsuyoshi Kato (Tokyo, JP); Satoru Kuma (Tokyo, JP); Ohji Nakagami (Tokyo, JP)
Assignee: SONY GROUP CORPORATION
G06T5/70G06T5/20G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,243,197
App. No.
18/352,250
Granted
Mar 4, 2025
Kind
B2
Abstract

A filter process is performed on point cloud data using a representative value of the point cloud data for each local region obtained by dividing a three-dimensional space. A two-dimensional plane image on which the point cloud data subjected to the filter process is projected is encoded, and a bitstream is generated. The filter process can be performed by, for example, an information processing apparatus, an image processing apparatus, electronic equipment, an information processing method, a program, or the like.

Claims (48)

1. An image processing apparatus comprising:

circuitry configured to

perform a filter process on point cloud data using a representative value of the point cloud data for each local region obtained by dividing a three-dimensional space;

encode a two-dimensional plane image on which the point cloud data subjected to the filter process is projected;

generate a bitstream including information regarding the local region that includes information regarding a size or a size and a shape of the local region;

determine whether a current position in an occupancy map of the point cloud data is located at an end of a patch in the occupancy map;

set, on a basis of determination that the current position is located at the end of the patch, a processing range of the filter process to the current position; and

exclude, on a basis of determination that the current position is not located at the end of the patch, the current position from the filter process.

2. The image processing apparatus according to claim 1 , wherein

the local region includes a cube region having a predetermined size.

3. The image processing apparatus according to claim 1 , wherein

the local region includes a rectangular parallelepiped region having a predetermined size.

4. The image processing apparatus according to claim 1 , wherein

the local region includes a region obtained by dividing the three-dimensional space such that each of the regions contains a predetermined number of points of the point cloud data.

5. The image processing apparatus according to claim 1 , wherein

the information regarding the local region includes information regarding a shape of the local region.

6. The image processing apparatus according to claim 1 , wherein

the representative value includes an average of the point cloud data contained in the local region.

7. The image processing apparatus according to claim 1 , wherein

the representative value includes a median of the point cloud data contained in the local region.

8. The image processing apparatus according to claim 1 , wherein

the filter process includes a smooth process that smooths data of a processing target point in the point cloud data using the representative value of the local region around the processing target point.

9. The image processing apparatus according to claim 1 , wherein

the circuitry is configured to perform the filter process on position information on a point of the point cloud data.

10. The image processing apparatus according to claim 1 , wherein

the circuitry is configured to perform the filter process on attribute information on a point of the point cloud data.

11. An image processing method comprising:

performing a filter process on point cloud data using a representative value of the point cloud data for each local region obtained by dividing a three-dimensional space;

encoding a two-dimensional plane image on which the point cloud data subjected to the filter process is projected, and generating a bitstream including information regarding the local region that includes information regarding a size or a size and a shape of the local region;

determining whether a current position in an occupancy map of the point cloud data is located at an end of a patch in the occupancy map;

setting, on a basis of determination that the current position is located at the end of the patch, a processing range of the filter process to the current position; and

excluding, on a basis of determination that the current position is not located at the end of the patch, the current position from the filter process.

12. An image processing apparatus comprising:

circuitry configured to

a decode a bitstream;

generate coded data of a two-dimensional plane image on which point cloud data is projected;

perform a filter process on the point cloud data restored from the two-dimensional plane image, using a representative value of the point cloud data for each local region obtained by dividing a three-dimensional space, wherein the bitstream includes information regarding the local region that includes information regarding a size or a size and a shape of the local region;

determine whether a current position in an occupancy map of the point cloud data is located at an end of a patch in the occupancy map;

set, on a basis of determination that the current position is located at the end of the patch, a processing range of the filter process to the current position; and

exclude, on a basis of determination that the current position is not located at the end of the patch, the current position from the filter process.

13. The image processing method according to claim 11 , wherein

the local region includes a cube region having a predetermined size.

14. The image processing method according to claim 11 , wherein

the local region includes a rectangular parallelepiped region having a predetermined size.

15. The image processing apparatus according to claim 12 , wherein

the local region includes a cube region having a predetermined size.

16. The image processing apparatus according to claim 12 , wherein

the local region includes a rectangular parallelepiped region having a predetermined size.

Assignments (1)
CHANGE OF NAME Recorded Jul 14, 2023
From: SONY CORPORATION
To: SONY GROUP CORPORATION
Reel/Frame 064274/0083 →
Priority Claims (1)
JP 2018-131295 · Jul 11, 2018 · national
Continuity (2)
Continuation 15733894
Related Publication 20230377100A1 · Nov 23, 2023
References Cited (37)
US 11182612B2 · Liu · 2021 [cited by examiner]
US 11348284B2 · Tourapis · 2022 [cited by examiner]
US 11361471B2 · Mammou · 2022 [cited by examiner]
US 11367224B2 · Mammou · 2022 [cited by examiner]
US 20130181983A1 · Kitamura · 2013 [cited by applicant]
US 20170116487A1 · Yamazaki · 2017 [cited by examiner]
US 20170214943A1 · Cohen et al. · 2017 [cited by applicant]
US 20170219336A1 · Kurtz · 2017 [cited by examiner]
US 20170220887A1 · Fathi · 2017 [cited by examiner]
US 20170347100A1 · Chou · 2017 [cited by examiner]
US 20170347120A1 · Chou · 2017 [cited by examiner]
US 20190018730A1 · Charamisinau · 2019 [cited by examiner]
US 20190087979A1 · Mammou · 2019 [cited by examiner]
US 20190311500A1 · Mammou · 2019 [cited by examiner]
US 20190313110A1 · Mammou · 2019 [cited by examiner]
CN 107578391A · 2018 [cited by applicant]
JP 2008283540A · 2008 [cited by applicant]
KR 101666937B1 · 2016 [cited by applicant]
WO WO2017126314A · 2017 [cited by applicant]
Chen, Jiawei, Cheng Zhang, and Pingbo Tang. “Geometry-based optimized point cloud compression methodology for construction and infrastructure management.” Computing in Civil Engineering (2017): 377-385. (Year: 2017). [cited by examiner]
R. L. de Queiroz and P. A. Chou, “Compression of 3D Point Clouds Using a Region-Adaptive Hierarchical Transform,” in IEEE Transactions on Image Processing, vol. 25, No. 8, pp. 3947-3956, Aug. 2016, doi: 10.1109/TIP.2016… [cited by examiner]
Ochotta1 et al, Image-Based Surface Compression, Computer Graphics forum, vol. 27 (2008), No. 6 pp. 1647-1663 ( Year: 2008). [cited by examiner]
Zewei et al, Vehicle Recognition and Classification Method Based on Laser Scanning Point Cloud Data, The 3rd International Conference on Transportation Information and Safety, Jun. 25-Jun. 28, 2015, Wuhan, P. R. China (… [cited by examiner]
Krivokuc et al, A Volumetric Approach to Point Cloud Compression, 81 Technical Report, Sep. 2018 (Year: 2018). [cited by examiner]
International Search Report and Written Opinion mailed on Sep. 10, 2019, received for PCT Application No. PCT/JP2019/025557 filed on Jun. 27, 2019, 6 pages. [cited by applicant]
Golla et al., “Real-time Point Cloud Compression”, IEEE, 2015, 6 pages. [cited by applicant]
Mekuria et al., “Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video”, IEEE, pp. 1-14. [cited by applicant]
Lanyi He, et al., “Best-Effort Projection Based Attribute Compression for 3D Point Cloud”, Cooperative MediaNet Innovation Center, Shanghai Jiao Tong University, Shanghai, China, Sep. 19, 2019, 7 pages. [cited by applicant]
Khaled Mammou, et al., “Video-based and Hierarchical Approaches Point Cloud Compression”, Apple Inc., ISO/IEC JTC1/SC29/WG11 m41649, Macau, China, Oct. 2017, 3 pages. [cited by applicant]
Khaled Mammou, et al., “PCC Test Model Category 2 v0”, 3DG, ISO/IEC JTC1/SC29/WG11 N17248, Macau, China, Oct. 2017, 11 pages. [cited by applicant]
Ohji Nakagami (Sony) et al: “Point cloud compression technology proposal by Sony”, 120.MPEG Meeting;Oct. 23, 2017-Oct. 27, 2017;Macau; (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11),No. m41665 Oct. 21, 2017(Oct… [cited by applicant]
Schwarz (Nokia)Setal:“Nokia's response to CIP for Point Cloud Compression(Category 2)”, 120.MPEG Meeting;Oct. 23, 2017-Oct. 27, 2017;Macau;(Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), No. m4177917 Oct. 10, 2… [cited by applicant]
Sung-Bum Park et al:“Multiscale Representation and Compression of 3-D Point Data”, IEEE Transactions on Multimedia, IEEE Service Center, US, vol. 11,No. 1, Jan. 1, 2009 (Jan. 1, 2009), pp. 177-183, XP011346575,ISSN:1520… [cited by applicant]
Nakagami O.: PCC TMC2 low complexity geometry smoothing, ISO/IEC JTC1/SC29/WG11 M43501, 2018 (Year: 2018). [cited by applicant]
Graziosi, D., et al. “An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-Pcc).” APSIPA Transactions on Signal and Information Processing 9 (2020): e13. (… [cited by applicant]
Iso et al, ISO/IEC JTC 1/SC 29/WG 11 Coding of moving pictures and audio Convenorship: UNI (Italy), https://mpeg-pcc.org/wp-content/uploads/2019/11/w18892_ VPCC_ TMv8_AlgorithmDescription_d1 .docx, 2019 (Year: 2019). [cited by applicant]
K. Mammou, PCC Test Model Category 2 v1, Iso/Iec JTC1/SC29/WG11 N17348 Ver.1, Jan. 2018, pp. 1-11. [cited by applicant]