IP Library › Granted Patent US 12,131,421
Granted Patent B2
US 12,131,421 · App. 18/083,796 · Granted Oct 29, 2024

Cropping for efficient three-dimensional digital rendering

Inventors: Kowsheek Mahmood (North York, CA); Kevin Roan (Amsterdam, NL); David McKinley Cardwell (Toronto, CA)
Assignee: Adobe Inc.
G06T15/08G06T15/50G06T19/00G06T2210/22G06T2219/004
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,131,421
App. No.
18/083,796
Granted
Oct 29, 2024
Kind
B2
Abstract

A method for generating a volume for three-dimensional rendering extracts a plurality of images from a source image input, normalizes the extracted images to have a common pixel size, and determines a notional camera placement for each normalized image to obtain a plurality of annotated normalized images, each annotated with a respective point of view through the view frustum of the notional camera. From the annotated normalized images, the method generates a first volume encompassing a first three-dimensional representation of the target object and selects a smaller subspace within the first volume that encompasses the first three-dimensional representation of the target object. The method generates, from the annotated normalized images, a second volume overlapping the first volume, encompassing a second three-dimensional representation of the target object and having a plurality of voxels, and crops the second volume to limit the second volume to the subspace.

Claims (71)

1. A method for generating a volume for three-dimensional rendering, the method comprising:

receiving a plurality of images, wherein each image of the plurality of images depicts a target object from a particular view angle of a plurality of view angles;

for each of the plurality of images, determining a notional camera placement for the image to obtain a plurality of annotated images, wherein each annotated image of the plurality of annotated images is annotated with a respective point of view of the target object based on the corresponding notional camera placement;

generating, from the plurality of annotated images, a first volume encompassing a first three-dimensional representation of the target object;

selecting a subspace within the first volume, wherein the subspace is smaller than the first volume and the subspace encompasses the first three-dimensional representation of the target object;

generating, from at least a subset of the plurality of annotated images, a second volume, the second volume overlapping the first volume and encompassing a second three-dimensional representation of the target object; and

cropping the second volume to limit the second volume to the subspace to obtain a cropped volume encompassing the second three-dimensional representation of the target object, wherein the cropped volume is rendered at a computing device.

2. The method of claim 1 , wherein the plurality of images are extracted from a source image input, and wherein:

the source image input is a CAD file; and

using a path tracing system, a plurality of high-fidelity images are generated, wherein the plurality of images are selected from the plurality of high-fidelity images.

3. The method of claim 1 , wherein the plurality of images are extracted from a source image input, and wherein:

the source image input is a video file, wherein the video file captures the target object from the plurality of view angles; and

the plurality of images are selected from images extracted from the video file, wherein each image extracted from the video file is associated with the particular view angle of the plurality of view angles.

4. The method of claim 1 , wherein generating, from at least a subset of the plurality of annotated images, the second volume encompassing the second three-dimensional representation of the target object comprises:

providing at least a subset of the plurality of annotated images to a view generation engine, wherein the view generation engine generates a plurality of annotated synthetic images, wherein each annotated synthetic image is annotated with a respective point of view; and

extrapolating the second volume from the plurality of annotated synthetic images.

5. The method of claim 1 , wherein the first volume is a point cloud.

6. The method of claim 1 , wherein selecting the subspace of the first volume is responsive to a signal received from a user computing device representing a user delineation of the subspace.

7. The method of claim 1 , wherein determining the notional cameral placement of the image includes:

transforming the image to have a specified width and height to generate a normalized image; and

determining the notional cameral placement from the normalized image.

8. The method of claim 1 , wherein the second volume includes a plurality of voxels each having a color and an opacity.

9. A method for generating a volume for three-dimensional rendering, the method comprising:

transmitting, from a user computing device, a plurality of images, wherein each image of the plurality of images depicts a target object from a particular view angle of a plurality of view angles;

receiving, at the user computing device, a first volume encompassing a first three-dimensional representation of the target object, wherein the first volume is obtained by:

for each of the plurality of images, determining a notional camera placement for the image to obtain a plurality of annotated images, wherein each annotated image of the plurality of annotated images is annotated with a respective point of view of the target object based on the corresponding notional camera placement; and

generating the first volume from the plurality of annotated images;

presenting, on a display of the user computing device, a visualization of the first volume;

transmitting, from the user computing device, a delineation of a subspace within the first volume, wherein the subspace is smaller than the first volume and the subspace encompasses the first three-dimensional representation of the target object;

receiving, at the user computing device, a cropped volume encompassing a second three-dimensional representation of the target object, wherein the cropped volume is limited to the subspace; and

rendering, at the user computing device, the cropped volume.

10. The method of claim 9 , wherein the cropped volume is obtained by:

generating, from at least a subset of the plurality of annotated images, a second volume encompassing the target object, wherein the second volume overlapping the first volume and encompassing the second three-dimensional representation of the target object; and

cropping the second volume to limit the second volume to the subspace to obtain the cropped volume.

11. The method of claim 10 , wherein generating, from at least a subset of the plurality of annotated images, the second volume encompassing of the target object, comprises:

providing at least a subset of the plurality of annotated images to a view generation engine, wherein the view generation engine generates a plurality of annotated synthetic images, wherein each annotated synthetic image is annotated with a respective point of view; and

extrapolating the second volume from the plurality of annotated synthetic images.

12. The method of claim 11 , wherein the second volume includes a plurality of voxels each having a color and an opacity.

13. The method of claim 9 , wherein the plurality of images are extracted from a source image input, and wherein:

the source image input is a CAD file; and

using a path tracing system, 200 to 400 high-fidelity images are generated, wherein the plurality of images are selected from the 200 to 400 high-fidelity images.

14. The method of claim 9 , wherein the plurality of images are extracted from a source image input, and wherein:

the source image input is a video file, wherein the video file captures the target object from the plurality of view angles; and

the plurality of images are selected from images extracted from the video file, wherein each image extracted from the video file is associated with the particular view angle of the plurality of view angles.

15. A data processing system comprising at least one processor and a memory coupled to the at least one processor, the memory containing instructions which, when executed by the at least one processor, cause the at least one processor to perform:

receiving a source image input, wherein the source image input is selected from the group consisting of:

an image set comprising a plurality of images; and

a CAD file;

wherein the source image input captures a target object;

extracting the plurality of images from the source image input to obtain extracted images;

normalizing the extracted images to obtain normalized images, wherein the normalized images have a common pixel size;

for each of the normalized images, determining a notional camera placement for that respective normalized image to obtain a plurality of annotated normalized images, wherein each annotated normalized image is annotated with a respective point of view through a view frustum of a notional camera;

generating, from the annotated normalized images, a first volume encompassing a first three-dimensional representation of the target object;

selecting a subspace within the first volume, wherein the subspace is smaller than the first volume and the subspace encompasses the first three-dimensional representation of the target object;

generating, from at least a subset of the annotated normalized images, a second volume, the second volume overlapping the first volume and encompassing a second three-dimensional representation of the target object;

wherein the second volume includes a plurality of voxels each having a color and an opacity; and

cropping the second volume to limit the second volume to the subspace to obtain a cropped volume encompassing the second three-dimensional representation of the target object.

16. The data processing system of claim 15 , wherein:

the source image input is a CAD file; and

using a path tracing system, 200 to 400 high-fidelity images are generated.

17. The data processing system of claim 15 , wherein:

the source image input is a video file, and wherein:

the video file is of at least 1 minute but less than 3 minutes in length;

the video file captures the target object from a plurality of view angles; and

the video file has a frame rate of X frames per second; and

images are extracted at every Y th frame, where Y<X, to extract a total of 200 to 400 images.

18. The data processing system of claim 15 , wherein generating, from at least a subset of the plurality of annotated normalized images, the second volume encompassing the second three-dimensional representation of the target object comprises:

providing at least a subset of the plurality of annotated normalized images to a view generation engine, wherein the view generation engine generates a plurality of annotated synthetic images, wherein each annotated synthetic image is annotated with a respective point of view; and

extrapolating the second volume from the plurality of annotated synthetic images.

19. The data processing system of claim 15 , wherein the first volume is a point cloud.

20. The data processing system of claim 15 , wherein selecting the subspace of the first volume is responsive to a signal received from a user computing device representing a user delineation of the subspace.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 19, 2022
From: MAHMOOD, KOWSHEEK; ROAN, KEVIN; CARDWELL, DAVID MCKINLEY
To: ADOBE INC.
Reel/Frame 062140/0434 →
Continuity (2)
Provisional Application 63314169 · Feb 25, 2022
Related Publication 20230274494A1 · Aug 31, 2023