IP Library Granted Patent US 11,574,467
Granted Patent B2
US 11,574,467 · App. 16/690,265 · Granted Feb 7, 2023

Document augmented auto complete

Inventors: Sougata Mukherjea (New Delhi, IN); Saiprasad Kolluri Venkata Sesha (Bengaluru, IN); Seema Nagar (Bangalore, IN); Kuntal Dey (New Delhi, IN)
Assignee: KYNDRYL, INC.
G06V20/20G06F3/0482G06F40/166G06F40/274G06T7/13G06V30/40G06F40/211G06F40/284G06F40/30G06T2207/30176
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,574,467
App. No.
16/690,265
Granted
Feb 7, 2023
Kind
B2
Abstract

A field-of-view of a scene is scanned by an augmented reality device. The scene includes one or more objects including a first computing device. A portion of an electronic document is detected based on the scanned field-of-view. The portion of the electronic document is rendered on a display of the first computing device. A content element of the electronic document that is rendered on the display is captured. A second computing device determines an incomplete portion of the content element. A suggestion to complete the incomplete portion is provided by the augmented reality device.

Claims (50)

1. A method of document manipulation, the method comprising:

receiving, from an augmented reality device, a scanned field-of-view of a scene, wherein the scene includes one or more objects including a first computing device, and the scanned field-of-view includes a display monitor of the first computing device;

detecting, in the scanned field-of-view, a portion of an electronic document being rendered on the display monitor and one or more edges of the display monitor;

capturing a content element of the electronic document rendered on the display monitor;

determining, by a second computing device, that the content element rendered on the display monitor incomplete, wherein the incomplete content element is an incomplete sentence, an incomplete figure, or both the incomplete sentence and the incomplete figure, wherein the incomplete content element rendered on the display monitor does not include a missing portion of the content element that is not rendered on the display monitor, and wherein the content element consists of the incomplete element and the missing portion; and

providing, to the augmented reality device, a suggestion to identify the missing portion to complete the incomplete content element for rendering the completed content element on the augmented reality device, the suggestion being displayed adjacent to the one or more edges of the display monitor.

2. The method of claim 1 , wherein:

the first computing device includes a first data store,

the second computing device includes a second data store,

the second data store contains more information than the first data store, and

the first computing device does not have access to the second data store.

3. The method of claim 1 , wherein the determining further comprises:

performing an image analysis operation on the electronic document to determine the missing portion of the content element.

4. The method of claim 1 , wherein the scanned field-of-view that includes the display monitor includes a bezel of the display monitor;

and wherein the providing the suggestion further comprises:

generating, for display on the augmented reality device, an augmented view that includes the suggestion at the one or more edges at the bezel of the display monitor, wherein the augmented view is aligned with the one or more edges of the display monitor, and wherein the suggestion is located adjacent to the incomplete content element.

5. The method of claim 1 , the method further comprising:

receiving, by the augmented reality device and in response to the suggestion to complete the incomplete content element, an autocomplete indication regarding the suggestion.

6. The method of claim 5 , the method further comprising:

establishing a network connection between the augmented reality device and the first computing device; and

providing, through the established network connection, the missing portion of the content element of the electronic document.

7. A system, said system comprising: one or more memories containing one or more instructions; and one or more processors communicatively coupled to the one or more memories, said one or more processors, in response to reading the one or more instructions, configured to:

receive, from an augmented reality device, a scanned field-of-view of a scene, wherein the scene includes one or more objects including a first computing device, and the scanned field-of-view includes a display monitor of the first computing device;

detect, based on the scanned field-of-view, a portion of an electronic document being rendered on the display monitor and one or more edges of the display monitor;

capture a content element of the electronic document rendered on the display monitor;

determine, by a second computing device, that the content element rendered on the display monitor is incomplete, wherein the incomplete content element is an incomplete sentence, an incomplete figure, or both the incomplete sentence and the incomplete figure, wherein the incomplete content element rendered on the display monitor does not include a missing portion of the content element that is not rendered on the display monitor, and wherein the content element consists of the incomplete element and the missing portion; and

provide, to the augmented reality device, a suggestion to identify the missing portion to complete the incomplete content element for rendering the completed content element on the augmented reality device, the suggestion being displayed adjacent to the one or more edges of the display monitor.

8. The system of claim 7 , wherein:

the first computing device includes a first data store,

the second computing device includes a second data store,

the second data store contains more information than the first data store, and

the first computing device does not have access to the second data store.

9. The system of claim 7 , wherein the processor is further configured to:

perform an image analysis operation on the electronic document to determine the missing portion of the content element.

10. The system of claim 7 , wherein the scanned field-of-view that includes the display monitor includes a bezel of the display monitor;

and wherein said provide the suggestion further comprises:

generate, for display on the augmented reality device, an augmented view that includes the suggestion at the one or more edges at the bezel of the display monitor, wherein the augmented view is aligned with the one or more edges of the display monitor, and wherein the suggestion is located adjacent to the incomplete content element.

11. A computer program product, said computer program product comprising a computer readable storage medium having one or more program instructions embodied therewith, said program instructions configured to:

receive, from an augmented reality device, a scanned field-of-view of a scene, wherein the scene includes one or more objects including a first computing device, and the scanned field-of-view includes a display monitor of the first computing device;

detect, based on the scanned field-of-view, a portion of an electronic document being rendered on the display monitor and one or more edges of the display monitor;

capture a content element of the electronic document rendered on the display monitor;

determine, by a second computing device, that the content element rendered on the display monitor is incomplete, wherein the incomplete content element is an incomplete sentence, an incomplete figure, or both the incomplete sentence and the incomplete figure, wherein the incomplete content element rendered on the display monitor does not include a missing portion of the content element that is not rendered on the display monitor, and wherein the content element consists of the incomplete element and the missing portion; and

provide, by the augmented reality device, a suggestion to complete the incomplete content element for rendering the completed content element on the augmented reality device, the suggestion being displayed adjacent to the one or more edges of the display monitor.

12. The computer program product of claim 11 , wherein the program instructions are further configured to:

receive, by the augmented reality device and in response to the suggestion to complete the incomplete content element, an autocomplete indication regarding the suggestion.

13. The computer program product of claim 11 , wherein the program instructions are further configured to:

establish a network connection between the augmented reality device and the first computing device; and

provide, through the established network connection, the missing portion of the content element of the electronic document.

14. The computer program product of claim 11 , wherein the scanned field-of-view that includes the display monitor includes a bezel of the display monitor; and wherein said provide the suggestion further comprises:

generate, for display on the augmented reality device, an augmented view that includes the suggestion at the one or more edges at the bezel of the display monitor, wherein the augmented view is aligned with the one or more edges of the display, and wherein the suggestion is located adjacent to the incomplete content element.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 18, 2021
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: KYNDRYL, INC.
Reel/Frame 058213/0912 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 21, 2019
From: MUKHERJEA, SOUGATA; KOLLURI VENKATA SESHA, SAIPRASAD; NAGAR, SEEMA; DEY, KUNTAL
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 051073/0025 →