IP Library › Granted Patent US 12,327,382
Granted Patent B2
US 12,327,382 · App. 18/112,589 · Granted Jun 10, 2025

Method, electronic device, and computer program product for model processing

Inventors: Zijia Wang (Weifang, CN); Zhisong Liu (Shenzhen, CN); Zhen Jia (Shanghai, CN)
Assignee: Dell Products L.P.
G06T9/001G06T9/002
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,327,382
App. No.
18/112,589
Granted
Jun 10, 2025
Kind
B2
Abstract

Methods, electronic devices and computer program products for model processing are disclosed in embodiments herein. A method in an illustrative embodiment includes encoding first data of a point cloud model to obtain a first matrix, and encoding second data of the point cloud model to obtain a second matrix, where the first data and the second data are data of the point cloud model acquired from different angles. The method further includes respectively decomposing, by an equivariant encoder, the first matrix and the second matrix into a first equivariant matrix and a second equivariant matrix, and respectively decomposing, by an invariant encoder, the first matrix and the second matrix into a first invariant matrix and a second invariant matrix. The method further includes training the equivariant encoder and the invariant encoder based on the first equivariant matrix, the second equivariant matrix, the first invariant matrix, and the second invariant matrix.

Claims (55)

1. A method comprising:

encoding first data of a point cloud model to obtain a first matrix;

encoding second data of the point cloud model to obtain a second matrix, wherein the first data and the second data are data of the point cloud model acquired from different angles;

respectively decomposing, by an equivariant encoder, the first matrix and the second matrix into a first equivariant matrix and a second equivariant matrix, wherein the equivariant encoder is configured to encode data inside the point cloud model;

respectively decomposing, by an invariant encoder, the first matrix and the second matrix into a first invariant matrix and a second invariant matrix, wherein the invariant encoder is configured to encode data of the point cloud model relative to a physical space; and

training the equivariant encoder and the invariant encoder based on the first equivariant matrix, the second equivariant matrix, the first invariant matrix, and the second invariant matrix.

2. The method according to claim 1 , wherein training the equivariant encoder based on the first equivariant matrix and the second equivariant matrix comprises:

adjusting, through linear transformation, elements in the first equivariant matrix to correspond to elements in the second equivariant matrix, so as to make a similarity between the first equivariant matrix and the second equivariant matrix conform to a similarity threshold.

3. The method according to claim 1 , wherein training the invariant encoder based on the first invariant matrix and the second invariant matrix comprises:

adjusting a Frobenius norm between the first invariant matrix and the second invariant matrix, so as to make the first invariant matrix consistent with the second invariant matrix.

4. The method according to claim 1 , further comprising:

reconstructing the point cloud model based on one or more matrices in the first invariant matrix, the second invariant matrix, the first equivariant matrix, and the second equivariant matrix; and

training a decoder in reconstructing the point cloud model, so as to make a similarity between a point cloud model reconstructed by the decoder and the point cloud model conform to a similarity threshold.

5. The method according to claim 1 , wherein the first invariant matrix and the second invariant matrix represent an absolute position of the point cloud model relative to the physical space.

6. The method according to claim 1 , wherein the first equivariant matrix and the second equivariant matrix represent relative positions between the data in the point cloud model.

7. The method according to claim 1 , wherein when the method is applied to generating a simulated face of a character, the first invariant matrix and the second invariant matrix represent an overall face outline of the simulated face of the character, and the first equivariant matrix and the second equivariant matrix represent facial expressions of the simulated face of the character.

8. The method according to claim 1 , further comprising:

using the invariant encoder to decompose data of one or more point cloud models into one or more invariant matrices after the invariant encoder is trained; and

classifying the one or more point cloud models based on the one or more invariant matrices.

9. The method according to claim 1 , further comprising:

using the equivariant encoder to decompose data of one or more point cloud models into one or more equivariant matrices after the equivariant encoder is trained;

dividing the one or more point cloud models based on the one or more equivariant matrices; and

identifying one or more elements of the one or more point cloud models based on divided parts of the one or more point cloud models.

10. An electronic device comprising:

at least one processor; and

a memory coupled to the at least one processor and having instructions stored therein, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform actions comprising:

encoding first data of a point cloud model to obtain a first matrix;

encoding second data of the point cloud model to obtain a second matrix, wherein the first data and the second data are data of the point cloud model acquired from different angles;

respectively decomposing, by an equivariant encoder, the first matrix and the second matrix into a first equivariant matrix and a second equivariant matrix, wherein the equivariant encoder is configured to encode data inside the point cloud model;

respectively decomposing, by an invariant encoder, the first matrix and the second matrix into a first invariant matrix and a second invariant matrix, wherein the invariant encoder is configured to encode data of the point cloud model relative to a physical space; and

training the equivariant encoder and the invariant encoder based on the first equivariant matrix, the second equivariant matrix, the first invariant matrix, and the second invariant matrix.

11. The electronic device according to claim 10 , wherein training the equivariant encoder based on the first equivariant matrix and the second equivariant matrix comprises:

adjusting, through linear transformation, elements in the first equivariant matrix to correspond to elements in the second equivariant matrix, so as to make a similarity between the first equivariant matrix and the second equivariant matrix conform to a similarity threshold.

12. The electronic device according to claim 10 , wherein training the invariant encoder based on the first invariant matrix and the second invariant matrix comprises:

adjusting a Frobenius norm between the first invariant matrix and the second invariant matrix, so as to make the first invariant matrix consistent with the second invariant matrix.

13. The electronic device according to claim 10 , further comprising:

reconstructing the point cloud model based on one or more matrices in the first invariant matrix, the second invariant matrix, the first equivariant matrix, and the second equivariant matrix; and

training a decoder in reconstructing the point cloud model, so as to make a similarity between a point cloud model reconstructed by the decoder and the point cloud model conform to a similarity threshold.

14. The electronic device according to claim 10 , wherein the first invariant matrix and the second invariant matrix represent an absolute position of the point cloud model relative to the physical space.

15. The electronic device according to claim 10 , wherein the first equivariant matrix and the second equivariant matrix represent relative positions between the data in the point cloud model.

16. The electronic device according to claim 10 , wherein when the electronic device is applied to generating a simulated face of a character, the first invariant matrix and the second invariant matrix represent an overall face outline of the simulated face of the character, and the first equivariant matrix and the second equivariant matrix represent facial expressions of the simulated face of the character.

17. The electronic device according to claim 10 , further comprising:

using the invariant encoder to decompose data of one or more point cloud models into one or more invariant matrices after the invariant encoder is trained; and

classifying the one or more point cloud models based on the one or more invariant matrices.

18. The electronic device according to claim 10 , further comprising:

using the equivariant encoder to decompose data of one or more point cloud models into one or more equivariant matrices after the equivariant encoder is trained;

dividing the one or more point cloud models based on the one or more equivariant matrices; and

identifying one or more elements of the one or more point cloud models based on divided parts of the one or more point cloud models.

19. A computer program product that is tangibly stored on a non-transitory computer-readable medium and comprises machine-executable instructions, wherein the machine-executable instructions, when executed by a machine, cause the machine to perform a method, the method comprising:

encoding first data of a point cloud model to obtain a first matrix;

encoding second data of the point cloud model to obtain a second matrix, wherein the first data and the second data are data of the point cloud model acquired from different angles;

respectively decomposing, by an equivariant encoder, the first matrix and the second matrix into a first equivariant matrix and a second equivariant matrix, wherein the equivariant encoder is configured to encode data inside the point cloud model;

respectively decomposing, by an invariant encoder, the first matrix and the second matrix into a first invariant matrix and a second invariant matrix, wherein the invariant encoder is configured to encode data of the point cloud model relative to a physical space; and

training the equivariant encoder and the invariant encoder based on the first equivariant matrix, the second equivariant matrix, the first invariant matrix, and the second invariant matrix.

20. The computer program product according to claim 19 , wherein when the method is applied to generating a simulated face of a character, the first invariant matrix and the second invariant matrix represent an overall face outline of the simulated face of the character, and the first equivariant matrix and the second equivariant matrix represent facial expressions of the simulated face of the character.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 22, 2023
From: WANG, ZIJIA; LIU, ZHISONG; JIA, ZHEN
To: DELL PRODUCTS L.P.
Reel/Frame 062764/0312 →
Priority Claims (1)
CN 202310077123.4 · Jan 16, 2023 · national
Continuity (1)
Related Publication 20240242388A1 · Jul 18, 2024
References Cited (39)
US 12015776B2 · Besenbruch · 2024 [cited by examiner]
US 12022077B2 · Besenbruch · 2024 [cited by examiner]
US 12095994B2 · Besenbruch · 2024 [cited by examiner]
US 12160579B2 · Besenbruch · 2024 [cited by examiner]
US 12165366B2 · Zhang · 2024 [cited by examiner]
US 12256075B2 · Besenbruch · 2025 [cited by examiner]
US 20200228836A1 · Schwarz · 2020 [cited by examiner]
US 20230388502A1 · Besenbruch · 2023 [cited by examiner]
US 20240078711A1 · Dean · 2024 [cited by examiner]
US 20240244242A1 · Lasserre · 2024 [cited by examiner]
L. Nanbo et al., “Object-Centric Representation Learning with Generative Spatial-Temporal Factorization,” arXiv:2111.05393v1, Nov. 9, 2021, 24 pages. [cited by applicant]
B, H. Zhang et al., “Mitigating Unwanted Biases with Adversarial Learning,” arXiv: 1801.07593v1, Jan. 22, 2018, 7 pages. [cited by applicant]
J. C. Duchi et al., “Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach,” arXiv:1610.03425v3, Jun. 30, 2018 48 pages. [cited by applicant]
E. Creager et al., “Exchanging Lessons Between Algorithmic Fairness and Domain Generalization,” ICLR 2021 Conference Blind Submission, Sep. 28, 2020, 19 pages. [cited by applicant]
C. R. Qi et al., “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation,” arXiv:1612.00593v2, Apr. 10, 2017, 19 pages. [cited by applicant]
M. Aubry et al., “The Wave Kernel Signature: A Quantum Mechanical Approach to Shape Analysis,” International Conference on Computer Vision Workshops, Nov. 2011, 8 pages. [cited by applicant]
M. M. Bronstein et al., “Scale-invariant Heat Kernel Signatures for Non-rigid Shape Recognition,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Jun. 2010, 8 pages. [cited by applicant]
J. Sun et al., “A Concise and Provably Informative Multi-Scale Signature Based on Heat Diffusion,” Eurographics Symposium on Geometry Processing, Jul. 15, 2009, 10 pages. [cited by applicant]
D.-Y. Chen et al., “On Visual Similarity Based 3D Model Retrieval,” Computer Graphics Forum, vol. 22, No. 3, Sep. 2003, pp. 223-232. [cited by applicant]
A. E. Johnson et al., “Using Spin Images for Efficient Object Recognition in Cluttered 3D Scenes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, No. 5, May 1999, pp. 433-449. [cited by applicant]
H. Ling et al., “Shape Classification Using the Inner-Distance,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Jan. 2007, 35 pages. [cited by applicant]
R. B. Rusu et al., “Fast Point Feature Histograms (FPFH) for 3D Registration,” IEEE International Conference on Robotics and Automation, May 2009, 6 pages. [cited by applicant]
R. B. Rusu et al., “Aligning Point Cloud Views using Persistent Feature Histograms,” IEEE/RSJ International Conference on Intelligent Robots and Systems, Sep. 2008, 8 pages. [cited by applicant]
D. Maturana et al., “VoxNet: A 3D Convolutional Neural Network for Real-Time Object Recognition,” IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. 2015, 7 pages. [cited by applicant]
C. R. Qi et al., “Volumetric and Multi-View CNNs for Object Classification on 3D Data,” arXiv: 1604.03265v2, Apr. 29, 2016, 14 pages. [cited by applicant]
Z. Wu et al., “3D ShapeNets: A Deep Representation for Volumetric Shapes,” arXiv: 1406.5670v3, Apr. 15, 2015, 9 pages. [cited by applicant]
Y. Li et al., “FPNN: Field Probing Neural Networks for 3D Data,” arXiv:1605.06240v3, Oct. 25, 2016, 10 pages. [cited by applicant]
D. Z. Wang et al., “Voting for Voting in Online Point Cloud Object Detection,” Robotics: Science and Systems, vol. 1, No. 3, Jul. 2015, 9 pages. [cited by applicant]
H. Su et al., “Multi-view Convolutional Neural Networks for 3D Shape Recognition,” arXiv:1505.00880v3, Sep. 27, 2015, 12 pages. [cited by applicant]
Y. Fang et al., “3D Deep Shape Descriptor,” IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2015, pp. 2319-2328. [cited by applicant]
K. Guo et al., “3D Mesh Labeling via Deep Convolutional Neural Networks,” ACM Transactions on Graphics, vol. 35, No. 1, Dec. 2015, pp. 3:1-3:12. [cited by applicant]
N. Guan et al., “MahNMF: Manhattan Non-negative Matrix Factorization,” arXiv:1207.3438v1, Jul. 14, 2012, 43 pages. [cited by applicant]
Z. Geng et al., “Is Attention Better Than Matrix Decomposition?” arXiv:2109.04553v2, Dec. 28, 2021, 24 pages. [cited by applicant]
G. Gafni et al., “Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction,” IEEE/CVF Conf. on Computer Vision and Pattern Recognition, Jun. 2021, pp. 8649-8658. [cited by applicant]
B. Gecer et al., “Synthesizing Coupled 3D Face Modalities by Trunk-Branch Generative Adversarial Networks,” arXiv:1909.02215v3, Dec. 2, 2020, 19 pages. [cited by applicant]
W. Sun et al., “Canonical Capsules: Self-Supervised Capsules in Canonical Pose,” arXiv:2012.04718v2, Nov. 24, 2021, 16 pages. [cited by applicant]
T. Deprelle et al., “Learning Elementary Structures for 3d Shape Generation and Matching,” arXiv:1908.04725v2, Aug. 14, 2019, 11 pages. [cited by applicant]
Y. Zhao et al., “3D Point Capsule Networks,” arXiv:1812.10775v2, Jul. 11, 2019, 14 pages. [cited by applicant]
E. Creager et al., “Environment Inference for Invariant Learning,” arXiv:2010.07249v5, Jul. 15, 2021, 23 pages. [cited by applicant]