IP Library › Granted Patent US 11,557,077
Granted Patent B2
US 11,557,077 · App. 15/136,949 · Granted Jan 17, 2023

System and method for retexturing of images of three-dimensional objects

Inventors: Joshua D. I. Distler (Brooklyn, NY); Michael Litvin (Tel Aviv, IL); Ariel Shamir (Jerusalem, IL)
Assignee: LIVESURFACE INC.
G06T15/04G06T11/001G06T17/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,557,077
App. No.
15/136,949
Granted
Jan 17, 2023
Kind
B2
Abstract

Systems and methods are described for retexturing portions of surface in a 2-D image, where the surface is an image of a 3-D object. The systems and methods analyze, with user input, the 2-D image and then, in computer memory, generate a 3-D model of the imaged surface. The surface may then be retextured, that is, for example, artwork may be added to the 2-D image in a realistic manner, taking into account a 3-D geometry in the scene of the 2-D image.

Claims (53)

1. A method implemented on a computer having memory, a processor, and a display, where the method comprises:

accepting a file containing a 2-D image of a 3-D scene into memory, where the 2-D image includes a 2-D surface which is an image of a 3-D surface of an object in the 3-D scene;

forming a first perspective model including a mathematical transformation re-projecting from the 2-D image to the 3-D scene, where said re-projecting includes using geometric information related to the object in the scene;

identifying segments of a silhouette of the 2-D surface, where the segments include a first silhouette segment, a second silhouette segment, a third silhouette, and a fourth silhouette segment, where the first silhouette segment is opposite the second silhouette segment;

generating a 3-D model using segments of a silhouette of the 2-D surface and said first perspective model, where said generating a 3-D model includes

forming the 3-D spine, using the first perspective model, by re-projecting a 2-D spine disposed between the identified first silhouette segment and the identified second silhouette segment,

forming one or more 3-D profile curves, using the first perspective model, by re-projecting one or more corresponding 2-D profile curves extending from the identified first silhouette segment to the identified second silhouette segment, where one of the one or more 2-D profile curves is the third silhouette segment, and

interpolating the one or more 3-D profile curves along the 3-D spine;

forming a second perspective model for projecting points from the 3-D model to the 2-D image; and

projecting the 3-D model on the display using said second perspective model.

2. The method of claim 1 , where said geometric information is two physical measurements of the object.

3. The method of claim 1 , where said geometric information is an indication of the geometric shape of the 3-D surface.

4. The method of claim 3 , where the indication is the geometric shape of the 3-D surface.

5. The method of claim 3 , where the indication is of the cross-section of the 3-D surface.

6. The method of claim 1 , where said obtaining geometric information includes obtaining user input.

7. The method of claim 6 , where said generating the 3-D model includes iteratively repeating the steps of:

generating the 3-D model;

accepting user input of geometric information; and

displaying the 3-D model on the display,

such that the user provides user input to produce an acceptable 3-D model on the display.

8. The method of claim 1 , where said generating the 3-D model further includes accepting user input defining one or more of the first silhouette segment, the second silhouette segment, one or more 2-D profile curves, or the 2-D spine.

9. The method of claim 1 , where said generating the 3-D model includes iteratively repeating the steps of:

identifying the first silhouette segment of the 2-D surface by accepting user input,

identifying the second silhouette segment of the 2-D surface, and either

determining a 3-D spine by re-projecting a 2-D spine identified by accepting user input; or

determining one or more 3-D profile curves by re-projecting the one or more 2-D profiles identified by accepting user input, and

generating the 3-D model by interpolating the one or more 3-D profile curves along the 3-D spine,

such that the user provides user input to produce an acceptable 3-D model on the display.

10. The method of claim 1 , where the image is obtained from a camera trained on the scene.

11. The method of claim 1 , where said first perspective model or said second perspective model includes the camera field of view or EXIF data.

12. The method of claim 1 , where said identifying the first silhouette segment on the 2-D surface includes defining the first corner point and defining the second corner point.

13. The method of claim 1 , where said identifying the second silhouette segment on the 2-D surface includes defining the third corner point and defining the fourth corner point.

14. The method of claim 1 , where one of the one or more 2-D profile curves includes the intersection of the 2-D profile curve and the 2-D spine.

15. The method of claim 1 , where defining one or more 2-D profile curves includes defining one 2-D profile curve.

16. The method of claim 1 , where defining one or more 2-D profile curves includes defining two 2-D profile curves.

17. The method of claim 1 , where said defining the 2-D spine includes defining two points of the 2-D spine.

18. The method of claim 1 , where said re-projecting points from the 2-D image to the 3-D scene is accurate to within a scale factor around the camera.

19. The method of claim 1 , where said re-projecting points from the 2-D image to the 3-D scene re-projects points onto a pre-determined 3-D plane.

20. The method of claim 1 , where a silhouette of the 3-D model projected onto to the 2-D image includes the first silhouette segment and the first silhouette segment.

21. The method of claim 1 , further comprising:

accepting a texture; and

where said associating said feature includes applying the accepted texture to the generated mesh.

22. The method of claim 1 , where said first perspective model or said second perspective model is a camera model.

23. The method of claim 1 , further comprising:

generating a mesh on the 3-D model,

associating a feature with the generated mesh; and

projecting the feature on the display.

24. The method of claim 1 , where said projecting includes the computer automatically scaling the one or more 3-D profile curves to fit between the first silhouette segment and the second silhouette segment.

25. The method of claim 23 , further comprising:

accepting artwork; and

where said associating said feature includes applying the accepted artwork to the generated mesh.

26. A system comprising:

a computer having memory, a processor, and a display, where the memory includes instructions for the processor perform the steps of claim 1 .

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2016
From: LITVIN, MICHAEL; SHAMIR, ARIEL; DISTLER, JOSHUA
To: LIVESURFACE INC.
Reel/Frame 039885/0045 →
Continuity (3)
Provisional Application 62152401 · Apr 24, 2015
Provisional Application 62152403 · Apr 24, 2015
Related Publication 20170032563A1 · Feb 2, 2017