IP Library › Granted Patent US 10,303,755
Granted Patent B2
US 10,303,755 · App. 14/878,482 · Granted May 28, 2019

Enhanced documentation validation

Inventors: Nathan Cullen (Cork, IE); Paul B. French (Cork, IE); James Young (Cork, IE)
Assignee: INTERNATIONAL BUSINESS MACHINES CORPORATION
G06F17/24G06F17/2765G06K9/00483
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,303,755
App. No.
14/878,482
Granted
May 28, 2019
Kind
B2
Abstract

Updating a document relating to updates of a graphical user interface (GUI) is provided. An example method includes identifying one or more images in a screenshot of an existing graphical user interface (GUI) and in a screenshot of a new GUI modified from the existing GUI; extracting image text associated with one or more component images from the existing GUI and the new GUI; identifying, in the new GUI, text and image differences between the component images from the existing GUI and the new GUI; identifying, in a document, document text associated with the text and image differences; and modifying the associated document text according to the identified text and image differences.

Claims (38)

1. A computer system comprising:

a memory; and

a processing device in communication with the memory, wherein the computer system is configured to perform a method comprising:

identifying one or more component images in a screenshot of an existing graphical user interface (GUI) and in a screenshot of a new GUI modified from the existing GUI, the one or more component images each being an image of a respective GUI element of the existing GUI or the new GUI;

extracting image text associated with each component image of the one or more component images from the existing GUI and the new GUI, the image text associated with each component image being text appearing in the component image with the respective GUI element of which the component image is an image;

identifying, based on the one or more component images, one or more image text differences between the new GUI and the existing GUI, the one or more image text differences comprising one or more differences in text appearing within a GUI element of the new GUI as compared to the existing GUI, wherein the extracting recognizes alphanumeric characters forming the text appearing within the GUI element of the new GUI, and wherein the identifying the one or more image text differences comprises identifying that the alphanumeric characters forming the text appearing within the new GUI element differ from a text appearing within a corresponding GUI element of the existing GUI;

based on identifying the image text differences, identifying, in a document, document text associated with the identified image text differences; and

modifying the document text according to the identified image text differences, the modifying comprising including in the document the alphanumeric characters forming the text appearing within the GUI element of the new GUI.

2. The computer system of claim 1 , wherein the modifying comprises adding new image text where new image text is identified in the new GUI.

3. The computer system of claim 1 , wherein the modifying makes an equivalent text modification when document text is the same as unmodified image text.

4. The computer system of claim 1 , wherein, when document text is different from the unmodified image text, the modifying modifies with an indication of a potential manual modification.

5. The computer system of claim 1 , wherein the modifying provides modifications, and wherein a modification of the modifications is marked up to reference the modification and a description of the modifications.

6. The computer system of claim 5 , wherein a list of modification descriptions is made available.

7. The computer system of claim 1 , wherein the document further comprises tags linking document text to image text.

8. The computer system of claim 1 , wherein at least one globally unique identifier of a user interface element is embedded in the document to create an association with its related document text element.

9. The computer system of claim 1 , wherein GUI images are generated directly from a first and second version of an application and stored as metadata in application documentation.

10. The computer system of claim 1 , wherein the one or more component images include at least one selected from the group consisting of: (i) a GUI button, (ii) a GUI menu, and (iii) a GUI field, and wherein the one or more component images are identified by edge detection.

11. A computer-implemented method comprising:

identifying one or more component images in a screenshot of an existing graphical user interface (GUI) and in a screenshot of a new GUI modified from the existing GUI, the one or more component images each being an image of a respective GUI element of the existing GUI or the new GUI;

extracting image text associated with each component image of the one or more component images from the existing GUI and the new GUI, the image text associated with each component image being text appearing in the component image with the respective GUI element of which the component image is an image;

identifying, based on the one or more component images, one or more image text differences between the new GUI and the existing GUI, the one or more image text differences comprising one or more differences in text appearing within a GUI element of the new GUI as compared to the existing GUI, wherein the extracting recognizes alphanumeric characters forming the text appearing within the GUI element of the new GUI, and wherein the identifying the one or more image text differences comprises identifying that the alphanumeric characters forming the text appearing within the new GUI element differ from a text appearing within a corresponding GUI element of the existing GUI;

based on identifying the image text differences, identifying, in a document, document text associated with the identified image text differences; and

modifying the document text according to the identified image text differences, the modifying comprising including in the document the alphanumeric characters forming the text appearing within the GUI element of the new GUI.

12. The method of claim 11 , wherein the modifying comprises adding new image text where new image text is identified in the new GUI.

13. The method of claim 11 , wherein the modifying makes an equivalent text modification when document text is the same as unmodified image text.

14. The method of claim 11 , wherein, when document text is different from the unmodified image text, the modifying modifies with an indication of a potential manual modification.

15. The method of claim 11 , wherein the modifying provides modifications, and wherein a modification of the modifications is marked up to reference the modification and a description of the modifications.

16. The method of claim 15 , wherein a list of modification descriptions is made available.

17. The method of claim 11 , wherein the document further comprises tags linking document text to image text.

18. The method of claim 11 , wherein at least one globally unique identifier of a user interface element is embedded in the document to create an association with its related document text element.

19. A computer program product comprising:

a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising:

identifying one or more component images in a screenshot of an existing graphical user interface (GUI) and in a screenshot of a new GUI modified from the existing GUI, the one or more component images each being an image of a respective GUI element of the existing GUI or the new GUI;

extracting image text associated with each component image of the one or more component images from the existing GUI and the new GUI, the image text associated with each component image being text appearing in the component image with the respective GUI element of which the component image is an image;

identifying, based on the one or more component images, one or more image text differences between the new GUI and the existing GUI, the one or more image text differences comprising one or more differences in text appearing within a GUI element of the new GUI as compared to the existing GUI, wherein the extracting recognizes alphanumeric characters forming the text appearing within the GUI element of the new GUI, and wherein the identifying the one or more image text differences comprises identifying that the alphanumeric characters forming the text appearing within the new GUI element differ from a text appearing within a corresponding GUI element of the existing GUI;

based on identifying the image text differences, identifying, in a document, document text associated with the identified image text differences; and

modifying the document text according to the identified image text differences, the modifying comprising including in the document the alphanumeric characters forming the text appearing within the GUI element of the new GUI.

20. The computer program product claim 19 , wherein the modifying comprises adding new image text where new image text is identified in the new GUI.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 8, 2015
From: CULLEN, NATHAN; FRENCH, PAUL B.; YOUNG, JAMES
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 036759/0524 →
Priority Claims (1)
GB 1417948.5 · Oct 10, 2014 · national
Continuity (1)
Related Publication 20160103811A1 · Apr 14, 2016