IP Library › Granted Patent US 11,307,731
Granted Patent B2
US 11,307,731 · App. 16/888,760 · Granted Apr 19, 2022

Computer implemented platform, software, and method for drawing using augmented reality

Inventors: Andrey Drobitko (Kaliningrad, RU); Aleksandr Danilin (Penzenskaya oblast, RU); Mikhail Kopeliovich (Rostov, RU); Mikhail Petrushan (Rostov, RU)
Assignee: SKETCHAR, UAV
G06F3/04815G06T7/248G06T7/35G06T7/74G06T19/006G06V10/757G06V20/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,307,731
App. No.
16/888,760
Granted
Apr 19, 2022
Kind
B2
Abstract

A computer-implemented method of the present invention allows the user to draw images using augmented reality. The method is implemented by a processor executing a marker-less tracking algorithm stored in a memory of a mobile computing device. The method comprises detecting, by a page detector, an image of a drawing area; initializing a marker-less tracker, wherein said initializing comprises (a) capturing, via the page detector, a frame of the drawing area, (b) displaying, via a graphical user interface (GUI) of the mobile computing device, the frame of the drawing area, and (c) uniformly distributing on the GUI a template patch over the frame of the drawing area, and wherein the template patch is a fragment of texture of a template image used for surface tracking. The method further comprises executing a template patch tracking, wherein a perspective transformation of the template image to the frame as a current image is evaluated in video streaming on the GUI of the mobile computing device.

Claims (17)

1. A non-transitory computer-readable medium storing a code with computer-readable instructions for drawing using augmented reality that when executed by a processor, causes the processor to execute a marker-less tracking algorithm stored in a memory of a mobile computing device to perform the steps of:

detecting, by a page detector, an image of a drawing area;

initializing a marker-less tracker, wherein said initializing comprises (a) capturing, via the page detector, a frame of the drawing area, (b) displaying, via a graphical user interface (GUI) of the mobile computing device, the frame of the drawing area, and (c) uniformly distributing on the GUI a template patch over the frame of the drawing area, wherein the template patch is a fragment of texture of a template image used for surface tracking; and

executing a template patch tracking, wherein a perspective transformation of the template image to the frame as a current image is evaluated in video streaming on the GUI of the mobile computing device;

wherein the processor is configured to detect a correspondence between one or more coordinates of the template image and one or more coordinates of the current image; and wherein the processor is configured to evaluate the perspective transformation based on the correspondence by random sample consensus (RANSAC);

estimate a confidence level of the perspective transformation as a percentage based on the one or more coordinates of the current image having an under-threshold difference from one or more coordinates predicted by said evaluating the perspective transformation by RANSAC;

switch, utilizing the percentage confidence level as a criterion to a re-initialization mode;

update an initial template patch with an initial descriptor to an updated template patch with a new descriptor;

monitor the updated template patch to validate it and responsive to a failure to validate the updated patch, switching back to the initial template patch with the initial descriptor;

enhance the texture of the template image through template patches accumulation based on temporal averaging of binarized patches; and

prepare a rough estimation of the perspective transformation by computing, via an inverse matrix, the perspective transformation from the template image to the current image.

2. The non-transitory computer-readable medium of claim 1 , wherein the processor is configured to: estimate a confidence level of the perspective transformation after said template patch tracking; and in response to the confidence level being low, switch from said template patch tracking to a re-initialization mode.

3. The non-transitory computer-readable medium of claim 1 , wherein the processor is configured to use an inverse matrix of the perspective transformation.

4. The non-transitory computer-readable medium of claim 3 , wherein the processor is configured to use inverse matrix to transform, via a rough estimation, the current image of the perspective transformation to the template image.

5. The non-transitory computer-readable medium of claim 3 , wherein the processor is configured to enhance a local texture contrast by performing image binarization with an adaptive local binarization threshold, wherein the local binarization threshold is calculated for each pixel as a function of intensities in a local surrounding region.

6. The non-transitory computer-readable medium of claim 1 , wherein the processor is configured to detect an out-of-frame or an obscured template patch.

7. The non-transitory computer-readable medium of claim 6 , wherein the processor is configured to filter the out-of-the-frame or obscured template patch; and update a contour of the out-of-the-frame or the obscured template patch, wherein said template patch tracking is based on searching of one or more template patches as the current image transformed by a roughly estimated perspective transformation.

Continuity (2)
Continuation In Part 16412462 · May 15, 2019
Related Publication 20200363931A1 · Nov 19, 2020