Distributed data collection in an enterprise network
In some implementations a starter task to gather data associated with at least a portion of a communications system may be received. A plurality of targets associated with the starter task may be determined. A portion of the plurality of targets may be grouped, based on grouping criteria, to create a collection task. A plurality of bids may be received from a plurality of collection agents, where each bid of the plurality of bids includes a cost estimate to perform the collection task. A winning bid of the plurality of bids may be selected. A particular collection agent associated with the winning bid may perform the collection task.
1. A method comprising:
receiving a starter task to gather data associated with at least a portion of a communications system;
determining a plurality of targets associated with the starter task;
grouping, based on grouping criteria, a portion of the plurality of targets to create a collection task;
receiving a plurality of bids from a plurality of collection agents, wherein each bid of the plurality of bids comprises a numerical representation of an amount of resources that the respective collection agent is expected to use to perform the collection task; and
selecting a winning bid of the plurality of bids.
2. The method of claim 1 , wherein the communications system comprises at least:
an email system;
a productivity suite that comprises a word processor application and a spreadsheet application;
a videoconferencing system; and
a collaboration suite.
3. The method of claim 1 , wherein the plurality of targets include at least one of:
mailboxes for email;
tracking logs associated with email;
logs associated with data loss prevention events;
logs associated with particular types of events;
server properties;
domain controllers;
databases storing information associated with video conferencing activities;
configuration and user policies associated with a video conferencing system;
a call data record (CDR) database associated with a collaboration suite; or
user subscriptions associated with a productivity suite.
4. The method of claim 1 , wherein the grouping criteria includes at least one of:
a number of messages in a target mailbox;
a number of events in an activity log database associated with a target;
whether more than one target is located on a same server;
an amount of effort associated with extracting data from the target; or
a number of application programming interface (API) calls to extract data from the target.
5. The method of claim 1 , wherein selecting the winning bid of the plurality of bids comprises:
determining a lowest cost bid of the plurality of bids; and
selecting the lowest cost bid as the winning bid.
6. The method of claim 1 , wherein selecting the winning bid of the plurality of bids comprises:
determining a low cost bid of the plurality of bids, wherein the low cost bid is:
lower than at least ninety percent of the plurality of bids; and
associated with a first collection agent of the plurality of collection agents; and
selecting the low cost bid as the winning bid based at least in part on a length of time from when the first collection agent performed a previous task.
7. The method of claim 1 , wherein each bid of the plurality of bids is created based on at least one of:
an analysis of historical costs;
an analysis of a status of one or more servers associated with the collection task;
an analysis of a configuration of the portion of the plurality of targets associated with the collection task; and
a cost analysis of a recently performed task.
8. A computing device comprising:
one or more processors;
one or more memory storage devices storing instructions that are executable by the one or more processors to perform operations comprising:
creating a starter task to gather data associated with at least a portion of a communications system;
determining a plurality of targets associated with the starter task;
grouping, based on grouping criteria, a portion of the plurality of targets to create a collection task;
receiving a plurality of bids from a plurality of collection agents, wherein each bid of the plurality of bids comprises a numerical representation of an amount of resources that the respective collection agent is expected to use to perform the collection task by a particular collection agent of the plurality of collection agents; and
selecting a winning bid of the plurality of bids to perform the collection task.
9. The computing device of claim 8 , wherein the operations further comprise:
creating a storage task to transfer gathered data collected as a result of performing the collection task to a central storage facility.
10. The computing device of claim 9 , wherein the operations further comprise:
receiving a plurality of storage bids from a plurality of storage agents, wherein each storage bid of the plurality of storage bids comprises a storage cost estimate to perform the storage task by a particular storage agent of the plurality of storage agents; and
selecting a winning storage bid of the plurality of storage bids to perform the storage task.
11. The computing device of claim 9 , wherein the operations further comprise:
updating a status of the storage task completed after the collection task has been performed.
12. The computing device of claim 8 , wherein the operations further comprise:
updating a status of the collection task completed after the collection task has been performed.
13. The computing device of claim 8 , wherein the operations further comprise:
grouping, based on the grouping criteria, a second portion of the plurality of targets to create a second collection task;
receiving a plurality of additional bids from the plurality of collection agents to perform the second collection task; and
selecting an additional winning bid of the plurality of additional bids to perform the second collection task.
14. One or more non-transitory computer-readable media storing instructions that are executable by one or more processors to perform operations comprising:
receiving a starter task to gather data associated with at least a portion of a communications system;
determining a plurality of targets associated with the starter task;
grouping, based on grouping criteria, a portion of the plurality of targets to create a collection task;
receiving a plurality of bids from a plurality of collection agents, wherein each bid of the plurality of bids comprises a numerical representation of an amount of resources that the respective collection agent is expected to use to perform the collection task; and
selecting a winning bid of the plurality of bids, wherein the winning bid is associated with a particular collection agent of the plurality of collection agents.
15. The one or more non-transitory computer-readable media of claim 14 , wherein a particular bid of the plurality of bids comprises a numerical estimate of an amount of time for a particular agent to perform the collection task.
16. The one or more non-transitory computer-readable media of claim 14 , wherein a particular bid of the plurality of bids comprises a numerical estimate of an amount of resources for a particular agent to perform the collection task, wherein the resources include one or more of central processing unit (CPU) cycles, memory usage, or bandwidth usage.
17. The one or more non-transitory computer-readable media of claim 14 , wherein the grouping criteria includes at least one of:
a number of messages in a target mailbox;
a number of events in an activity log database associated with a target;
whether more than one target is located on a same server;
an amount of effort associated with extracting data from the target; or
a number of application programming interface (API) calls to extract data from the target.
18. The one or more non-transitory computer-readable media of claim 14 , the operations further comprising:
collecting performance data associated with the particular collection agent performing the collection task; and
creating a second bid for a second collection task based at least in part on the performance data.
19. The one or more non-transitory computer-readable media of claim 14 , wherein selecting the winning bid of the plurality of bids comprises:
determining a low cost bid of the plurality of bids, wherein the low cost bid is lower than at least ninety percent of the plurality of bids; and
selecting the low cost bid as the winning bid based at least in part on a length of time from when the particular collection agent performed a previous task.
20. The one or more non-transitory computer-readable media of claim 14 , wherein each bid of the plurality of bids is created based on at least one of:
an analysis of historical costs;
an analysis of a status of one or more servers associated with the collection task;
an analysis of a configuration of the portion of the plurality of targets associated with the collection task; and
a cost analysis of a recently performed task.