Flexible framework for secure search
A flexible and extensible architecture allows for secure searching across an enterprise. Such an architecture can provide a simple Internet-like search experience to users searching secure content inside (and outside) the enterprise. The architecture allows for the crawling and searching of a variety of sources across an enterprise, regardless of whether any of these sources conform to a conventional user role model. The architecture further allows for security attributes to be received at query time, for example, in order to provide real-time secure access to enterprise resources. The user query also can be transformed to provide for dynamic querying that provides for a more current result list than can be obtained for static queries.
1. A method of generating a user-specific search index across a plurality of data repositories, the method comprising:
providing a template source for crawling a target data repository, wherein the template source comprises instructions for crawling the target data repository, but the template source does not include user-specific security credentials for crawling the target data repository;
receiving a request to subscribe a user to the template source;
receiving security credentials from the user for the target data repository;
creating a user-subscribed source for the template source using the security credentials from the user for the target data repository;
launching a crawler on the target data repository, wherein the user-subscribed source with the security credentials from the user inherits the instructions for crawling the target data repository from the template source;
causing the crawler to log into the target data repository using the security credentials from the user; and
causing the crawler to crawl the target data repository using the instructions for crawling the target data repository.
2. The method of claim 1 , wherein the user-subscribed source with the security credentials from the user inherits the instructions for crawling the target data repository from the template source at a crawl time.
3. The method of claim 2 , further comprising:
receiving changes to the template source after the creating the user-subscribed source, wherein the changes to the template source are inherited by the user-subscribed source at the crawl time.
4. The method of claim 2 , wherein the crawl time is coordinated to prevent denial-of-service attacks against the target data repository.
5. The method of claim 1 , wherein the target data repository comprises:
one or more documents to which the user has access; and
one or more documents to which the user does not have access.
6. The method of claim 5 , wherein the one or more documents to which the user has access are stamped with an identity of the user.
7. The method of claim 6 , wherein documents in the target data repository that are stamped with the identity of the user are only available for search to the user.
8. A system comprising:
one or more processors; and
one or more memory devices comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations configured to:
provide a template source for crawling a target data repository, wherein the template source comprises instructions for crawling the target data repository, but the template source does not include user-specific security credentials for crawling the target data repository;
receive a request to subscribe a user to the template source;
receive security credentials from the user for the target data repository;
create a user-subscribed source for the template source using the security credentials from the user for the target data repository;
launch a crawler on the target data repository, wherein the user-subscribed source with the security credentials from the user inherits the instructions for crawling the target data repository from the template source;
cause the crawler to log into the target data repository using the security credentials from the user; and
cause the crawler to crawl the target data repository using the instructions for crawling the target data repository.
9. The system of claim 8 , wherein the target data repository is selected from among a plurality of target data repositories.
10. The system of claim 9 , wherein the plurality of target data repositories reside both inside of an enterprise and outside of the enterprise.
11. The system of claim 9 , wherein the plurality of target data repositories comprises one or more selections from the group consisting of: a webpage, a database, a mail server, a document repository, a file server, and a desktop.
12. The system of claim 8 , wherein the operations are further configured to:
generate an index from crawling the target data repository; and
assign an access control list to the index.
13. The system of claim 12 , wherein the operations are further configured to:
receive a search query from the user;
determine that the user is on the access control list; and
return individual search results for the user comprising only documents that the user is authorized to access.
14. The system of claim 12 , wherein a group of users is assigned to the access control list.
15. The system of claim 8 , wherein the instructions for crawling the target data repository in the template source comprises one or more selections from the group consisting of: a seed Uniform Resource Locator, a boundary rule, a document type, an attribute mapping, and a crawling parameter.
16. A non-transitory, computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
providing a template source for crawling a target data repository, wherein the template source comprises instructions for crawling the target data repository, but the template source does not include user-specific security credentials for crawling the target data repository;
receiving a request to subscribe a user to the template source;
receiving security credentials from the user for the target data repository;
creating a user-subscribed source for the template source using the security credentials from the user for the target data repository;
launching a crawler on the target data repository, wherein the user-subscribed source with the security credentials from the user inherits the instructions for crawling the target data repository from the template source;
causing the crawler to log into the target data repository using the security credentials from the user; and
causing the crawler to crawl the target data repository using the instructions for crawling the target data repository.
17. The non-transitory, computer-readable medium of claim 16 , wherein the security credentials for the user are obtained from an identity management system associated with the target data repository.
18. The non-transitory, computer-readable medium of claim 16 , wherein the operations further comprise:
providing a customized page that, when executed, allows the user to modify an aspect or attribute of the user-subscribed source.
19. The non-transitory, computer-readable medium of claim 16 , wherein the template source designates subscription parameters to be provided by the user.
20. The non-transitory, computer-readable medium of claim 19 , wherein the subscription parameters comprise specifying a subset of the target data repository to be crawled.