IP Library › Granted Patent US 12,744,014
Granted Patent B2
US 12,744,014 · App. 18/150,334 · Granted Sep 22, 2026

Selecting frames during augmented reality reconstruction

Inventors: Xiang Yu Yang (Xi'an, CN); Yong Wang (Xi'an, CN); Jun Guo (Xi'an, CN); Zhi Yong Jia (Xian, CN); Yu Pan (Shanghai, CN); Zhong Fang Yuan (Xi'an, CN)
Assignee: International Business Machines Corporation
G09G3/3607G09G2340/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,744,014
App. No.
18/150,334
Granted
Sep 22, 2026
Kind
B2
Abstract

Performing a three-dimensional (3D) reconstruction process of a set of selected image frames. In some instances, a plurality of sequential image frames that are extracted from a video file (including video content created through the use of Augmented Reality) are converted into a grayscale format. Once these image files are converted to a grayscale format, a two-dimensional (2D) entropy process is performed that is ultimately used to create a set of user-defined clusters, with each of these user-defined clusters being associated with at least a sub-set set of the grayscale frames. After a cluster is selected, a 3D reconstruction process is performed on the frames associated with the selected cluster.

Claims (47)

1 . A computer-implemented method (CIM) comprising:

receiving a plurality of image frames, with each given image frame of the plurality of image frames including an image, and with each given image frame of the plurality of image frames being organized in a chronological sequence;

converting each given image of the plurality of images from its original image format into a grayscale image format to obtain a grayscale image frame sequence having a plurality of grayscale images;

for each given grayscale image of the grayscale image frame sequence, calculating a two-dimensional image entropy in order to obtain a set of image entropy data;

clustering the set of image entropy data in order to obtain a plurality of clusters, with a number of clusters in the plurality of clusters being defined by a user;

selecting grayscale image frames that are associated with a first cluster of the plurality of clusters; and

performing a three-dimensional image reconstruction of the selected grayscale frames that are associated with the first cluster.

2 . The CIM of claim 1 wherein the calculation of the two-dimensional image entropy is based, at least in part, upon a set of pixel density values.

3 . The CIM of claim 1 wherein the calculation of the two-dimensional image entropy is based, at least in part, upon a gray value at a defined pixel position.

4 . The CIM of claim 1 wherein the calculation of the two-dimensional image entropy is based, at least in part, upon a distribution of gray pixels that are proximate to a defined gray value.

5 . The CIM of claim 1 wherein the plurality of image frames includes image frames that are taken from video data of multiple angles of a target object.

6 . The CIM of claim 1 further comprising:

for each given grayscale image of the grayscale image frame sequence, calculating a one-dimensional entropy in order to obtain a set of image entropy values; and

selecting the grayscale image that corresponds to a largest image entropy value.

7 . A computer program product comprising:

a machine readable storage device; and

computer code stored on the machine readable storage device, with the computer code including instructions and data for causing a processor(s) set to perform operations including the following:

receiving a plurality of image frames, with each given image frame of the plurality of image frames including an image, and with each given image frame of the plurality of image frames being organized in a chronological sequence,

converting each given image of the plurality of images from its original image format into a grayscale image format to obtain a grayscale image frame sequence having a plurality of grayscale images,

for each given grayscale image of the grayscale image frame sequence, calculating a two-dimensional image entropy in order to obtain a set of image entropy data,

clustering the set of image entropy data in order to obtain a plurality of clusters, with a number of clusters in the plurality of clusters being defined by a user,

selecting grayscale image frames that are associated with a first cluster of the plurality of clusters, and

performing a three-dimensional image reconstruction of the selected grayscale frames that are associated with the first cluster.

8 . The CPP of claim 7 wherein the calculation of the two-dimensional image entropy is based, at least in part, upon a set of pixel density values.

9 . The CPP of claim 7 wherein the calculation of the two-dimensional image entropy is based, at least in part, upon a gray value at a defined pixel position.

10 . The CPP of claim 7 wherein the calculation of the two-dimensional image entropy is based, at least in part, upon a distribution of gray pixels that are proximate to a defined gray value.

11 . The CPP of claim 7 wherein the plurality of image frames includes image frames that are taken from video data of multiple angles of a target object.

12 . The CPP of claim 7 further comprising:

for each given grayscale image of the grayscale image frame sequence, calculating a one-dimensional entropy in order to obtain a set of image entropy values; and

selecting the grayscale image that corresponds to a largest image entropy value.

13 . A computer system (CS) comprising:

a processor(s) set;

a machine readable storage device; and

computer code stored on the machine readable storage device, with the computer code including instructions and data for causing the processor(s) set to perform operations including the following:

receiving a plurality of image frames, with each given image frame of the plurality of image frames including an image, and with each given image frame of the plurality of image frames being organized in a chronological sequence,

converting each given image of the plurality of images from its original image format into a grayscale image format to obtain a grayscale image frame sequence having a plurality of grayscale images,

for each given grayscale image of the grayscale image frame sequence, calculating a two-dimensional image entropy in order to obtain a set of image entropy data,

clustering the set of image entropy data in order to obtain a plurality of clusters, with a number of clusters in the plurality of clusters being defined by a user,

selecting grayscale image frames that are associated with a first cluster of the plurality of clusters, and

performing a three-dimensional image reconstruction of the selected grayscale frames that are associated with the first cluster.

14 . The CS of claim 13 wherein the calculation of the two-dimensional image entropy is based, at least in part, upon a set of pixel density values.

15 . The CS of claim 13 wherein the calculation of the two-dimensional image entropy is based, at least in part, upon a gray value at a defined pixel position.

16 . The CS of claim 13 wherein the calculation of the two-dimensional image entropy is based, at least in part, upon a distribution of gray pixels that are proximate to a defined gray value.

17 . The CS of claim 13 wherein the plurality of image frames includes image frames that are taken from video data of multiple angles of a target object.

18 . The CS of claim 13 further comprising:

for each given grayscale image of the grayscale image frame sequence, calculating a one-dimensional entropy in order to obtain a set of image entropy values; and

selecting the grayscale image that corresponds to a largest image entropy value.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2023
From: YANG, XIANG YU; WANG, YONG; GUO, JUN; JIA, ZHI YONG; PAN, YU; YUAN, ZHONG FANG
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 062282/0654 →
Continuity (1)
Related Publication 20240233665A1 · Jul 11, 2024
References Cited (18)
US 8913503B2 · Chen · 2014 [cited by applicant]
US 9076059B2 · Leung · 2015 [cited by applicant]
US 12531016B2 · Lin · 2026 [cited by examiner]
US 20130148851A1 · Leung · 2013 [cited by examiner]
US 20130235157A1 · Chen · 2013 [cited by examiner]
US 20170347120A1 · Chou · 2017 [cited by applicant]
US 20190088187A1 · Yan · 2019 [cited by examiner]
US 20240078969A1 · Lin · 2024 [cited by examiner]
US 20260004706A1 · Zhang · 2026 [cited by examiner]
CN 103996220A · 2014 [cited by applicant]
KR 20110064161A · 2011 [cited by applicant]
JP 2024091108 Welding Monitoring System, Welding Monitoring Method and Welding Monitoring Program Jul. 4, 2024 Yamabuki Manabu (Year: 2024). [cited by examiner]
Dasgupta, Saumitro, “Indoor Augmented Reality using 3D Scene Reconstruction”, Stanford University, Stanford, CA, printed on Oct. 10, 2022, 5 pages. [cited by applicant]
Doulamis, Anastasios, “Automatic 3D Reconstruction From Unstructured Videos Combining Video Summarization and Structure From Motion”, Frontiers in ICT | www.frontiersin.org, Nov. 2018 | vol. 5 | Article 29, 13 pages. [cited by applicant]
Koschel et al., “Selection of Key Frames for 3D Reconstruction in Real Time”, Algorithms 2021, 14, 303, MDPI, 16 pages. [cited by applicant]
Pavoni et al., “Automatic selection of video frames for path regularization and 3D reconstruction”, EUROGRAPHICS Workshop on Graphics and Cultural Heritage (2016) C. E. Catalano and L. De Luca (Editors), 10 pages. [cited by applicant]
Wang et al., Video Key Frame Monitoring Algorithm and Virtual Reality Display Based on Motion Vector, Special Section on Gigapixel Panoramic Video With Virtual Reality, IEEE Access, date of current version Sep. 11, 2020… [cited by applicant]
Xie et al., “Hierarchical Clustering-Aligning Framework Based Fast Large-Scale 3D Reconstruction Using Aerial Imagery”, remote sensing MDPI, Published: Feb. 5, 2019, 20 pages. [cited by applicant]