Document scanner
Systems as described herein may include document scanning to identify problems with the documents. A document scanning system may determine a scope of a document scan. The system may initiate a silent mode of a browser application and send a request to access an enterprise document repository. The system may be granted read-only access and determine page elements associated with a document within the scope. Based on the page elements, the system may use a machine learning model to determine an indication of one or more problems with a document. Accordingly, the system may generate a health report for each document within the scope of the document scan.
1 . A computer-implemented method comprising:
determining a plurality of data objects that fall within a scope of a document scan of an enterprise document repository, wherein the plurality of data objects comprise document identifiers associated with a plurality of documents in the document scan;
initiating, by a computing device, a silent mode of a browser application;
sending, by the computing device, via the silent mode of the browser application, a request to access the enterprise document repository;
receiving, by the computing device and from a cyber vault, an encryption key granting the computing device read-only access to the enterprise document repository;
determining, based on scanning a Document Object Model (DOM) representing a document within the scope of the document scan, a plurality of page elements associated with the document;
providing, as input to a machine learning model, the plurality of page elements associated with the document, wherein the plurality of page elements comprises one or more Uniform Resource Locators (URLs), a section that contains the one or more URLs, a parent document, and a document scan status;
receiving, as output from the machine learning model, an indication of whether the one or more URLs in the document are likely to be broken;
generating, based on the indication, a health report for each document of the plurality of documents that were within the scope of the document scan; and
after generating the health report, terminating, by the computing device, the silent mode of the browser application.
2 . The computer-implemented method of claim 1 , wherein the plurality of page elements further comprises a rating and a page view associated with the document.
3 . The computer-implemented method of claim 1 , wherein the document scan status comprises:
a date and time of a current document scan;
a date and time of a previous document scan;
a status indicating whether the previous document scan has completed successfully; and
an indication of whether the previous document scan has identified broken links in the document.
4 . The computer-implemented method of claim 1 , wherein the health report comprises an indication of at least one of: a broken link, a dead link, a blank page, a stale page, or an anti-pattern.
5 . The computer-implemented method of claim 1 , further comprising:
prior to the providing the plurality of page elements, training the machine learning model based on historical scan data comprising at least one of:
dates and time of historical document scans;
pre-labeled historical documents comprising broken links; or
pre-labeled historical documents comprising stale pages, blank pages and anti-patterns.
6 . The computer-implemented method of claim 1 , further comprising:
prior to the providing the plurality of page elements, training the machine learning model based on real time data streamed by producers via a Session Description Protocol (SDP), wherein the real time data comprises at least one of:
identified documents in the current document scan comprising broken links; or
identified documents in the current document scan comprising stale pages, blank pages and anti-patterns.
7 . The computer-implemented method of claim 1 , wherein the plurality of data objects comprises a URL to access a document within the scope of the document scan and a declaration whether the corresponding document is accessible.
8 . The computer-implemented method of claim 1 , further comprising:
after receiving indications that the one or more URLs are likely to be broken, determining, based on historical scan data and using a second machine learning model, a recommendation for alternative links to replace the one or more URLs that are likely to be broken.
9 . The computer-implemented method of claim 1 , further comprising:
providing, as input to a second machine learning model, the scope of the document scan, wherein the second machine learning model is trained based on historical loads on the enterprise document repository and schedules of historical document scans to the plurality of documents in the document scan; and
receiving, from the second machine learning model and based on the input, a recommendation of a time to execute the document scan.
10 . A system comprising:
one or more processors; and
memory storing instructions that, when executed by the one or more processors, cause the system to:
determine a plurality of data objects that fall within a scope of a document scan of an enterprise document repository, wherein the plurality of data objects comprise document identifiers associated with a plurality of documents within the scope of the document scan;
initiate a silent mode of a browser application;
send, via the silent mode of the browser application, a request to access the enterprise document repository;
receive authorization to access the enterprise document repository;
determine, based on scanning a Document Object Model (DOM) representing a document within the scope of the document scan, a plurality of page elements associated with the document;
provide, as input to a machine learning model, the plurality of page elements associated with the document, wherein the plurality of page elements comprises one or more Uniform Resource Locators URLs, a section that contains the one or more URLs, a parent document, and a document scan status;
receive, as output from the machine learning model, an indication of whether the one or more URLs are likely to be broken;
generate, based on the indication, a health report for each document of the plurality of documents that were within the scope of the document scan; and
after generating the health report, terminate the silent mode of the browser application.
11 . The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the system to:
after receiving indications that the one or more URLs are likely to be broken, determine, based on historical scan data and using a second machine learning model, a recommendation for alternative links to replace the one or more URLs that are likely to be broken.
12 . The system of claim 10 , wherein the plurality of page elements further comprises a rating and a page view associated with the document.
13 . The system of claim 10 , wherein the document scan status comprises:
a date and time of a current document scan;
a date and time of a previous document scan;
a status indicating whether the previous document scan has completed successfully; and
an indication of whether the previous document scan identified broken links in the document.
14 . The system of claim 10 , wherein the health report comprises an indication of at least one of: a broken link, a dead link, a blank page, a stale page, or an anti-pattern.
15 . The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the system to:
prior to the providing the plurality of page elements, train the machine learning model based on historical scan data comprising at least one of:
dates and time of historical document scans;
pre-labeled historical documents comprising broken links; or
pre-labeled historical documents comprising stale pages, blank pages and anti-patterns.
16 . The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the system to receive the authorization by:
receiving, from a cyber vault, an encryption key granting the system read-only access to the enterprise document repository.
17 . The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the system to:
prior to the providing the plurality of page elements, train the machine learning model based on real time data streamed by producers via a Session Description Protocol (SDP) comprising at least one of:
identified documents in the current document scan comprising broken links; or
identified documents in the current document scan comprising stale pages, blank pages and anti-patterns.
18 . The system of claim 10 , wherein the instructions, when executed by the one or more processors, cause the system to:
provide, as input to a second machine learning model, the scope of the document scan, wherein the second machine learning model is trained based on historical loads on the enterprise document repository and schedules of historical document scans to the plurality of documents in the document scan; and
receive, from the second machine learning model and based on the input, a recommendation of a time to execute the document scan.
19 . One or more non-transitory media storing instructions that when executed by one or more processors, cause a computing device to:
determine a plurality of data objects that fall within a scope of a document scan of an enterprise document repository, wherein the plurality of data objects comprise document identifiers associated with a plurality of documents in the document scan;
initiate a silent mode of a browser application;
send, via the silent mode of the browser application, a request to access an enterprise document repository;
receive, from a cyber vault, an encryption key granting the computing device read-only access to the enterprise document repository;
determine, based on the scope of the document scan, a plurality of page elements associated with the document;
provide, as input to a machine learning model, the plurality of page elements associated with the document, wherein the plurality of page elements comprises one or more Uniform Resource Locators URLs, a section that contains the one or more URLs, a parent document, and a document scan status;
receive, as output from the machine learning model, an indication of whether the one or more URLs are likely to be broken;
generate, based on the indication, a health report for each document of the plurality of documents that were within the scope of the document scan; and
after generating the health report, terminate the silent mode of the browser application.
20 . The non-transitory media of claim 19 , wherein the instructions, when executed by the one or more processors, cause the computing device to:
after the receiving indications that the one or more URLs are likely to be broken, determine, based on historical scan data and using a second machine learning model, a recommendation for correct links to replace the one or more URLs that are likely to be broken.