IP Library › Granted Patent US 12,374,089
Granted Patent B2
US 12,374,089 · App. 17/890,391 · Granted Jul 29, 2025

Method, device, and computer program product for image processing

Inventors: Zijia Wang (WeiFang, CN); Zhisong Liu (Shenzhen, CN); Zhen Jia (Shanghai, CN)
Assignee: Dell Products L.P.
G06V10/7747G06T7/0002G06V10/82
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,374,089
App. No.
17/890,391
Granted
Jul 29, 2025
Kind
B2
Abstract

Embodiments of the present disclosure relate to a method, a device, and a computer program product for image processing. The method includes determining a group of composite images, a composite image in the group of composite images being composited from images of the same type in a plurality of images. The method further includes determining an image quality of an acquired target image based on the group of composite images. The method further includes processing the acquired target image based on the image quality. Through the method, the storage and management of image data are improved, the effective utilization rate of data is improved, and the consumption of resources is reduced.

Claims (66)

1. A method for image processing, comprising:

determining, in a processor-based machine learning system, a group of composite images, a composite image in the group of composite images being composited from images of the same type in a plurality of images, the group of composite images comprising at least a first composite image generated from a first subset of the plurality of images each having a first classification, and a second composite image, different than the first composite image and generated from a second subset of the plurality of images each having a second classification different than the first classification, wherein the first and second subsets of the plurality of images are part of a sample image set;

determining, in the processor-based machine learning system, an image quality of an acquired target image, for potential addition to the sample image set, based on the group of composite images, the image quality being determined at least in part based on similarity metrics computed utilizing pixel values of the acquired target image and pixel values of respective ones of the composite images;

processing, in the processor-based machine learning system, the acquired target image based on the image quality, wherein processing the acquired target image based on the image quality further comprises controlling inclusion of the acquired target image in the sample image set at least in part as a function of the image quality, to provide an updated sample image set; and

training, in the processor-based machine learning system, at least one machine learning model utilizing the updated sample image set.

2. The method according to claim 1 , wherein determining the group of composite images comprises:

acquiring the plurality of images; and

performing data distillation on the plurality of images to obtain the group of composite images.

3. The method according to claim 2 , wherein performing the data distillation on the plurality of images comprises:

acquiring a capsule network model; and

obtaining the group of composite images by applying the capsule network model to the plurality of images.

4. The method according to claim 1 , wherein determining the image quality comprises:

determining a similarity between the acquired target image and each composite image in the group of composite images;

performing an addition operation on a group of similarities between the target image and the group of composite images; and

determining a result of the addition operation as the image quality of the target image with respect to the group of composite images.

5. The method according to claim 1 , wherein the image quality is expressed as a quality level; and processing the acquired target image comprises:

comparing the quality level with a threshold level; and

retaining the acquired target image responsive to determining that the quality level exceeds the threshold level.

6. The method according to claim 5 , wherein processing the acquired target image further comprises:

discarding the acquired target image responsive to determining that the quality level is lower than or equal to the threshold level.

7. The method according to claim 5 , further comprising:

determining a ranking of the acquired target images based on the quality level;

comparing the ranking with a threshold ranking; and

determining that the acquired target image is able to be used to train the at least one machine learning model responsive to determining that the ranking precedes the threshold ranking.

8. The method according to claim 7 , further comprising:

training the at least one machine learning model online by using the acquired target image to reduce a loss value of the at least one machine learning model.

9. The method according to claim 7 , further comprising:

storing the acquired target image and the corresponding quality level according to the ranking.

10. An electronic device, comprising:

at least one processor; and

a memory coupled to the at least one processor and having instructions stored therein, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform actions comprising:

determining, in a processor-based machine learning system, a group of composite images, a composite image in the group of composite images being composited from images of the same type in a plurality of images, the group of composite images comprising at least a first composite image generated from a first subset of the plurality of images each having a first classification, and a second composite image, different than the first composite image and generated from a second subset of the plurality of images each having a second classification different than the first classification, wherein the first and second subsets of the plurality of images are part of a sample image set;

determining, in the processor-based machine learning system, an image quality of an acquired target image, for potential addition to the sample image set, based on the group of composite images, the image quality being determined at least in part based on similarity metrics computed utilizing pixel values of the acquired target image and pixel values of respective ones of the composite images;

processing, in the processor-based machine learning system, the acquired target image based on the image quality, wherein processing the acquired target image based on the image quality further comprises controlling inclusion of the acquired target image in the sample image set at least in part as a function of the image quality, to provide an updated sample image set; and

training, in the processor-based machine learning system, at least one machine learning model utilizing the updated sample image set.

11. The electronic device according to claim 10 , wherein determining the group of composite images comprises:

acquiring the plurality of images; and

performing data distillation on the plurality of images to obtain the group of composite images.

12. The electronic device according to claim 11 , wherein performing data distillation on the plurality of images comprises:

acquiring a capsule network model; and

obtaining the group of composite images by applying the capsule network model to the plurality of images.

13. The electronic device according to claim 10 , wherein determining the image quality comprises:

determining a similarity between the acquired target image and each composite image in the group of composite images;

performing an addition operation on a group of similarities between the target image and the group of composite images; and

determining a result of the addition operation as the image quality of the target image with respect to the group of composite images.

14. The electronic device according to claim 10 , wherein the image quality is expressed as a quality level; and processing the acquired target image comprises:

comparing the quality level with a threshold level; and

retaining the acquired target image responsive to determining that the quality level exceeds the threshold level.

15. The electronic device according to claim 14 , wherein processing the acquired target image further comprises:

discarding the acquired target image responsive to determining that the quality level is lower than or equal to the threshold level.

16. The electronic device according to claim 14 , wherein the actions further comprise:

determining a ranking of the acquired target images based on the quality level;

comparing the ranking with a threshold ranking; and

determining that the acquired target image is able to be used to train the at least one machine learning model responsive to determining that the ranking precedes the threshold ranking.

17. The electronic device according to claim 16 , wherein the actions further comprise:

training the at least one machine learning model online by using the acquired target image to reduce a loss value of the at least one machine learning model.

18. The electronic device according to claim 16 , wherein the actions further comprise:

storing the acquired target image and the corresponding quality level according to the ranking.

19. A computer program product comprising a non-transitory computer-readable medium having machine-executable instructions stored therein, wherein the machine-executable instructions, when executed by a machine, cause the machine to perform a method for image processing, the method comprising:

determining, in a processor-based machine learning system, a group of composite images, a composite image in the group of composite images being composited from images of the same type in a plurality of images, the group of composite images comprising at least a first composite image generated from a first subset of the plurality of images each having a first classification, and a second composite image, different than the first composite image and generated from a second subset of the plurality of images each having a second classification different than the first classification, wherein the first and second subsets of the plurality of images are part of a sample image set;

determining, in the processor-based machine learning system, an image quality of an acquired target image, for potential addition to the sample image set, based on the group of composite images, the image quality being determined at least in part based on similarity metrics computed utilizing pixel values of the acquired target image and pixel values of respective ones of the composite images;

processing, in the processor-based machine learning system, the acquired target image based on the image quality, wherein processing the acquired target image based on the image quality further comprises controlling inclusion of the acquired target image in the sample image set at least in part as a function of the image quality, to provide an updated sample image set; and

training, in the processor-based machine learning system, at least one machine learning model utilizing the updated sample image set.

20. The computer program product according to claim 19 , wherein determining the group of composite images comprises:

acquiring the plurality of images; and

performing data distillation on the plurality of images to obtain the group of composite images.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2022
From: WANG, ZIJIA; LIU, ZHISONG; JIA, ZHEN
To: DELL PRODUCTS L.P.
Reel/Frame 060841/0445 →
Priority Claims (1)
CN 202210874440.4 · Jul 22, 2022 · national
Continuity (1)
Related Publication 20240029416A1 · Jan 25, 2024
References Cited (46)
US 9922261B2 · Isler · 2018 [cited by examiner]
US 10891550B2 · Kapur · 2021 [cited by examiner]
US 11568634B2 · Jonnalagedda · 2023 [cited by examiner]
US 11616988B2 · Dinh · 2023 [cited by examiner]
US 12119088B2 · Jaganathan · 2024 [cited by examiner]
US 20160210529A1 · Nishiyuki · 2016 [cited by examiner]
US 20180189602A1 · Hellier · 2018 [cited by examiner]
US 20180204111A1 · Zadeh · 2018 [cited by examiner]
US 20180260665A1 · Zhang · 2018 [cited by examiner]
US 20200302224A1 · Jaganathan · 2020 [cited by examiner]
US 20210097699A1 · Li · 2021 [cited by examiner]
US 20210390696A1 · Iwase · 2021 [cited by examiner]
US 20210398259A1 · Yamazoe · 2021 [cited by examiner]
US 20240404004A1 · Kim · 2024 [cited by examiner]
CA 3225437A1 · 2022 [cited by examiner]
Wang, Tongzhou, et al. “Dataset distillation.” arXiv preprint arXiv:1811.10959 (Year: 2018). [cited by examiner]
Tung, Frederick, and Greg Mori. “Similarity-preserving knowledge distillation.” Proceedings of the IEEE/CVF international conference on computer vision. (Year: 2019). [cited by examiner]
M. Aumueller et al., “ANN-Benchmarks,” http://ann-benchmarks.com/, Accessed Jul. 21, 2022, 25 pages. [cited by applicant]
O. Zendel et al., “Analyzing Computer Vision Data—The Good, the Bad and the Ugly,” IEEE Conference on Computer Vision and Pattern Recognition, Jul. 1, 2017, pp. 1980-1990. [cited by applicant]
D. J. Butler et al., “A Naturalistic Open Source Movie for Optical Flow Evaluation,” European Conference on Computer Vision, Part IV, LNCS 7577, Oct. 2012, pp. 611-625. [cited by applicant]
C. Kaner, “What Is a Good Test Case?” Star East, May 2003, 16 pages. [cited by applicant]
P. Dollar et al., “Pedestrian Detection: A Benchmark,” Conference on Computer Vision and Pattern Recognition, Jun. 2009, 8 pages. [cited by applicant]
V. Haltakov et al., “Framework for Generation of Synthetic Ground Truth Data for Driver Assistance Applications,” German Conference on Pattern Recognition, Sep. 2013, 10 pages. [cited by applicant]
M. Kristan et al., “A Novel Performance Evaluation Methodology for Single-Target Trackers,” IEEE Transactions on Pattern Analysis and Machine Intelligence, arXiv:1503.01313v3, Jan. 8, 2016, 21 pages. [cited by applicant]
A. Li et al., “NUS-PRO: A New Visual Tracking Challenge,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, No. 2, Feb. 2016, pp. 335-347. [cited by applicant]
N. Onkarappa et al., “Synthetic Sequences and Ground-truth Flow Field Generation for Algorithm Validation,” Multimedia Tools and Applications, vol. 74, No. 9, May 2015, pp. 3121-3135. [cited by applicant]
G. Ros et al., “The Synthia Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes,” Conference on Computer Vision and Pattern Recognition. Jun. 2016, pp. 3234-3243. [cited by applicant]
G. S. Semwezi, “Automation of Negative Testing,” Master of Science Thesis, Chalmers University of Technology, Jun. 2012, 69 pages. [cited by applicant]
A. W. M. Smeulders et al., “Visual Tracking: An Experimental Survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, No. 7, Jul. 2014, pp. 1442-1468. [cited by applicant]
A. Torralba et al., “Unbiased Look at Dataset Bias,” Conference on Computer Vision and Pattern Recognition, Jun. 2011, pp. 1521-1528. [cited by applicant]
O. Zendel et al., “CV-HAZOP: Introducing Test Data Validation for Computer Vision,” International Conference on Computer Vision, Dec. 2015, pp. 2066-2074. [cited by applicant]
D. Biedermann et al., “Evaluating Visual ADAS Components on the Congrats Dataset,” 2016 IEEE Intelligent Vehicles Symposium (IV), Jun. 2016, 6 pages. [cited by applicant]
N. Schneider et al., “Pedestrian Path Prediction with Recursive Bayesian Filters: A Comparative Study,” German Conference on Pattern Recognition Proceedings, Sep. 2013, 10 pages. [cited by applicant]
Github, “jsubercaze/simhashdb,” https://github.com/jsubercaze/simhashdb, Accessed Apr. 22, 2022, 3 pages. [cited by applicant]
Wikipedia, “Capsule Neural Network,” https://en.wikipedia.org/wiki/Capsule_neural_network, Feb. 28, 2022, 8 pages. [cited by applicant]
V. Kurama, “Capsule Networks: A Quick Primer,” https://blog.paperspace.com/capsule-networks/, Jul. 17, 2020, 13 pages. [cited by applicant]
Geeks for Geeks, “Residual Networks (ResNet)—Deep Learning,” https://www.geeksforgeeks.org/residual-networks-resnet-deep-learning/, Jan. 27, 2022, 16 pages. [cited by applicant]
T. Wang et al., “Dataset Distillation,” arXiv:1811.10959v3, Feb. 24, 2020, 14 pages. [cited by applicant]
G. Hinton et al., “Distilling the Knowledge in a Neural Network,” arXiv:1503.02531v1, Mar. 9, 2015, 9 pages. [cited by applicant]
S. Sabour et al., “Dynamic Routing Between Capsules,” arXiv:1710.09829v2, Nov. 7, 2017, 11 pages. [cited by applicant]
Y. Lecun et al., “Gradient-Based Learning Applied to Document Recognition,” Proceedings of the IEEE, Nov. 1998, 46 pages. [cited by applicant]
E. Strubell et al., “Energy and Policy Considerations for Deep Learning in NLP,” Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Jul. 28-Aug. 2, 2019, pp. 3645-3650. [cited by applicant]
R. Haeusler et al., “Synthesizing Real World Stereo Challenges,” German Conference on Pattern Recognition Proceedings, Sep. 2013, pp. 164-173. Abstract Only. [cited by applicant]
U.S. Appl. No. 17/569,030 filed in the name of Zijia Wang et al. on Jan. 5, 2022, and entitled “Label Inheritance for Soft Label Generation in Information Processing System.”. [cited by applicant]
U.S. Appl. No. 17/829,508 filed in the name of Zhen Jia et al. on Jun. 1, 2022, and entitled “Content Analysis and Retrieval Using Machine Learning.”. [cited by applicant]
R. Haeusler et al., “Synthesizing Real World Stereo Challenges,” German Conference on Pattern Recognition Proceedings, Sep. 2013, pp. 164-173. [cited by applicant]
Cited By (1)
US 12,548,309