SYSTEM AND METHOD FOR RECOMMENDING A GRAMMAR FOR A MESSAGE CAMPAIGN USED BY A MESSAGE OPTIMIZATION SYSTEM

1 . A method performed by a computer system for recommending a grammar for a message campaign used by a message optimization system, the method comprising:

providing a user interface that enables a campaign manager to specify one or more parameters for a new campaign;

recommending at least one grammar structure from one or more past campaigns that are based on the campaign parameters or from a default grammar in the event that none of the past campaigns has a grammar that complies with the campaign parameters, the grammar structure specifying a plurality of message component types;

providing a user interface that enables a campaign manager to select one of the recommended grammar structures for the new campaign;

for each message component type in the selected grammar structure, generating a ranked list of previously-used values for the component type in the one or more past campaigns, wherein the previously-used values are each associated with a semantic tag and generating the ranked list comprises:

identifying the semantic tags associated with the previously-used values in the one or more past campaigns, wherein each semantic tag identifies the semantic meaning of the associated value,

creating a list of the previously-used values in the one or more past campaigns grouped by semantic tag,

ranking groups of semantic tags based on performance in the one or more past campaigns of the previously-used values within a tag group versus other tag groups, and

ordering the previously-used values first by their ranked tag group and second, within each tag group, by the number of times an individual value has been identified as the winning value in the one or more past campaigns;

for each message component type, selecting a plurality of values to recommend testing based at least in part on the ranked list of previously-used values;

enabling the campaign manager to reject one or more of the recommended values;

in response to the campaign manager rejecting one or more of the recommended values, providing alternate recommended values for the rejected values; and

generating variations of a message to test based on the grammar structure and values accepted by the campaign manager.

2 . The method of claim 1 , wherein the recommended values are selected in part from the ranked list of previously-used values and in part from a ranked list of untested values.

3 . The method of claim 2 , wherein a plurality of untested values is grouped by semantic tag and ranked according to their semantic tag, wherein within a tag group an untested value is randomly ranked.

4 . The method of claim 2 , wherein the percentage of recommended untested values depends on a degree of exploration/conservatism indicated by the campaign manager.

5 . The method of claim 1 , wherein the recommended values are also selected in part from a group of untested values having an associated untested/unranked semantic tag.

6 . The method of claim 5 , wherein each untested value in the group of untested values is associated with a confidence level based on the global ranking of the untested value across all campaigns.

7 . The method of claim 1 , wherein one or more parameters in the user interface is associated with a drop-down menu that lists the options available to the campaign manager for the parameter.

8 . The method of claim 1 , wherein the parameters comprise campaign duration, audience size and characteristics, expected response rate, effect size, constraints, and objectives of the campaign.

9 . The method of claim 8 , wherein the campaign manager selects the objectives of the campaign via a drop-down menu, from a sliding scale, or by inputting a value.

10 . The method of claim 1 , wherein if the campaign manager does not specify certain parameters, default values, empirically determined based on past campaigns, are used.

11 . The method of claim 1 , further comprising evaluating the recommended grammar based on various computed metrics.

12 . The method of claim 1 , wherein providing alternate recommended values for the rejected values comprises choosing values from the next best performing tag group or another candidate value associated with the same semantic tag as the previously rejected value.

13 . A non-transitory computer-readable medium comprising code that, when executed by a computer system, enables the computer system to perform the following method for recommending a grammar for a message campaign used by a message optimization system, the method comprising:

enabling a campaign manager to specify one or more parameters for a new campaign;

recommending at least one grammar structure from one or more past campaigns that are based on the campaign parameters or from a default grammar in the event that none of the past campaigns has a grammar that complies with the campaign parameters, the grammar structure specifying a plurality of message component types;

enabling a campaign manager to select one of the recommended grammar structures for the new campaign;

for each message component type in the selected grammar structure, generating a ranked list of previously-used values for the component type in the one or more past campaigns, wherein the previously-used values are each associated with a semantic tag and generating the ranked list comprises:

identifying the semantic tags associated with the previously-used values in the one or more past campaigns, wherein each semantic tag identifies the semantic meaning of the associated value,

creating a list of the previously-used values in the one or more past campaigns grouped by semantic tag,

ranking groups of semantic tags based on performance in the one or more past campaigns of the previously-used values within a tag group versus other tag groups, and

ordering the previously-used values first by their ranked tag group and second, within each tag group, by the number of times an individual value has been identified as the winning value in the one or more past campaigns;

for each message component type, selecting a plurality of values to recommend testing based at least in part on the ranked list of previously-used values;

enabling the campaign manager to reject one or more of the recommended values;

in response to the campaign manager rejecting one or more of the recommended values, providing alternate recommended values for the rejected values; and

generating variations of a message to test based on the grammar structure and values accepted by the campaign manager.

14 . The non-transitory computer-readable medium of claim 13 , wherein the recommended values are selected in part from the ranked list of previously-used values and in part from a ranked list of untested values.

15 . The non-transitory computer-readable medium of claim 14 , wherein a plurality of untested values is grouped by semantic tag and ranked according to their semantic tag, wherein within a tag group an untested value is randomly ranked.

16 . The non-transitory computer-readable medium of claim 14 , wherein the percentage of recommended untested values depends on a degree of exploration/conservatism indicated by the campaign manager.

17 . The non-transitory computer-readable medium of claim 13 , wherein the recommended values are also selected in part from a group of untested values having an associated untested/unranked semantic tag.

18 . The non-transitory computer-readable medium of claim 17 , wherein each untested value in the group of untested values is associated with a confidence level based on the global ranking of the untested value across all campaigns.

19 . The non-transitory computer-readable medium of claim 13 , wherein enabling a campaign manager to specify parameters for a new campaign comprises providing a user interface wherein the campaign manager is prompted to enter parameters for the campaign.

20 . The non-transitory computer-readable medium of claim 19 , wherein one or more parameters in the user interface is associated with a drop-down menu that lists the options available to the campaign manager for the parameter.

21 . The non-transitory computer-readable medium of claim 13 , wherein the parameters comprise campaign duration, audience size and characteristics, expected response rate, effect size, constraints, and objectives of the campaign.

22 . The non-transitory computer-readable medium of claim 21 , wherein the campaign manager selects the objectives of the campaign via a drop-down menu, from a sliding scale, or by inputting a value.

23 . The non-transitory computer-readable medium of claim 13 , wherein if the campaign manager does not specify certain parameters, default values, empirically determined based on past campaigns, are used.

24 . The non-transitory computer-readable medium of claim 13 , further comprising evaluating the recommended grammar based on various computed metrics.

25 . The non-transitory computer-readable medium of claim 13 , wherein providing alternate recommended values for the rejected values comprises choosing values from the next best performing tag group or another candidate value associated with the same semantic tag as the previously rejected value.

26 . A computer system for recommending a grammar for a message campaign used by a message optimization system, the system comprising:

a processor;

a memory coupled to the processor, wherein the memory stores instructions that, when executed by the processor, causes the system to perform the operations of:

enabling a campaign manager to specify one or more parameters for a new campaign;

recommending at least one grammar structure from one or more past campaigns that are based on the campaign parameters or from a default grammar in the event that none of the past campaigns has a grammar that complies with the campaign parameters, the grammar structure specifying a plurality of message component types;

enabling a campaign manager to select one of the recommended grammar structures for the new campaign;

for each message component type in the selected grammar structure, generating a ranked list of previously-used values for the component type in the one or more past campaigns, wherein the previously-used values are each associated with a semantic tag and generating the ranked list comprises:

identifying the semantic tags associated with the previously-used values in the one or more past campaigns, wherein each semantic tag identifies the semantic meaning of the associated value,

creating a list of the previously-used values in the one or more past campaigns grouped by semantic tag,

ranking groups of semantic tags based on performance in the one or more past campaigns of previously-used values within a tag group versus other tag groups, and

ordering the previously-used values first by their ranked tag group and second, within each tag group, by the number of times an individual value has been identified as the winning value in the one or more past campaigns;

for each message component type, selecting a plurality of values to recommend testing based at least in part on the ranked list of previously-used values;

enabling the campaign manager to reject one or more of the recommended values;

in response to the campaign manager rejecting one or more of the recommended values, providing alternate recommended values for the rejected values; and

generating variations of a message to test based on the grammar structure and values accepted by the campaign manager.