Systems and methods for discovery of data that needs improving or authored using user search results diagnostics

1. A method for analyzing user behavior data gathered from a user search session using a search mechanism, the method comprising:

employing a processor executing computer executable instructions to implement the following acts:

tracking user behavior data related to a failed query during the user search session, the tracking comprising monitoring whether:

a content desired by a user exists, but is not returned in a search result list when a query is made by the user with intent to find the desired content,

the content desired by the user does not exist in which the user searched for said content by the query made by the user with intent to find the desired content,

the content desired by the user exists, but the content desired is not recognized by the user in the search result list,

the content desired by the user exists, but the content desired is below a set threshold in the search result list, and

the user has accepted the search result list within the search session, wherein a number of queries that took the user within the session to accept the search result list within the session is determined and only content that has been tracked over a threshold number of times due to falling within certain criteria is reported; and

providing the user behavior data corresponding to whether the content desired by the user exists to an intent determination diagnostic that indicates the desired content exists for the failed query if a confidence score is over a threshold value, and providing the user behavior data corresponding to whether the content desired by the user does not exist to a content quantity diagnostic that indicates the desired content does not exist for the failed query if the confidence score is below the threshold value.

2. The method of claim 1 wherein the results of the analyzing are ordered by how often the content is identified as that which is tracked according to a certain criteria.

3. A method for analyzing user behavior data gathered from a user search session using a search mechanism, the method comprising:

employing a processor executing computer executable instructions to implement the following acts:

tracking implicit user behavior data when a user search session ends without indication of a satisfying search query result, the tracking comprising monitoring whether:

a content desired by a user exists, but the content desired is not returned in a search result list when a query is made by the user with intent to find the desired content,

the content desired by the user does not exist in which the user searched for said content by the query made by the user with intent to find the desired content,

the content desired by the user exists, but the content desired is not recognized by the user in the search result list,

the content desired by the user exists, but the content desired is below a set threshold in the search result list, and

the user has accepted the search result list within the session, wherein a number of queries that took the user within the session to accept the search result list within the session is determined and only content that has been tracked over a threshold number of times due to falling within certain criteria is reported;

providing the implicit user behavior data corresponding to whether the desired content of the search query exists to an intent determination diagnostic that indicates the desired content exists if a confidence score is over a threshold value, and providing the implicit user behavior data corresponding to whether the desired content of the search query does not exist to a content quantity diagnostic that indicates the desired content does not exist if the confidence score is below the threshold value; and

providing a diagnostic report with the intent determination diagnostic and the content quantity diagnostic for the user search session, wherein the diagnostic report identifies one or more results that are candidates for the desired content for the query made by the user that should have a search result with the desired content, and wherein the diagnostic report includes at least one of a content clarity diagnostic, a misleading result diagnostic, and a misranked result diagnostic,

wherein content clarity diagnostic identifies results that evoked divergent results from a plurality of users submitting the same query,

wherein the misleading result diagnostic identifies misleading results including at least one of results that are viewed frequently and have a high dissatisfaction, results that are viewed seldom and have high satisfaction are listed low in a ranked results list and have high click-through, and results that are listed high in a ranked results list have a low click-through, and

wherein the misranked results diagnostic provides a recommendation for a results position for at least one misranked result.

4. A computer storage medium readable by a computer having a memory and a processor, the medium tangibly embodying one or more programs of instructions executable by the computer to implement steps for analyzing user behavior data gathered from a user search session using a search mechanism, comprising:

tracking user behavior data related to a failed search query during the user search session, the tracking comprising monitoring whether:

a content desired by a user exists, but the content desired is not returned in a search result list when a query is made by the user with intent to find the desired content,

the content desired by the user does not exist in which the user searched for said content by the query made by the user with intent to find the desired content,

the content desired by the user exists, but the content desired is not recognized by the user in the search result list,

the content desired by the user exists, but the content desired is below a set threshold in the search result list, and

the user has accepted the search result list within the session, wherein a number of queries that took the user within the session to accept the search result list within the session is determined and only content that has been tracked over a threshold number of times due to falling within certain criteria is reported;

providing the user behavior data corresponding to whether the content desired by the user exists to an intent determination diagnostic that indicates the desired content exists for the failed query if a confidence score is over a threshold value, and providing the user behavior data corresponding to whether the content desired by the user does not exist to a content quantity diagnostic that indicates the desired content does not exist for the failed query if the confidence score is below the threshold value; and

generating a diagnostic report including the intent determination diagnostic and the content quantity diagnostic, wherein the diagnostic report identifies one or more results that are candidates for the desired content for the query made by the user that should have a search result with the desired content, and wherein the diagnostic report includes at least one of a content clarity diagnostic, a misleading result diagnostic, and a misranked result diagnostic,

wherein content clarity diagnostic identifies results that evoked divergent results from a plurality of users submitting the same query,

wherein the misleading result diagnostic identifies misleading results including at least one of results that are viewed frequently and have a high dissatisfaction, results that are viewed seldom and have high satisfaction are listed low in a ranked results list and have high click-through, and results that are listed high in a ranked results list have a low click-through, and

wherein the misranked results diagnostic provides a recommendation for a results position for at least one misranked result.

5. The computer storage medium of claim 4 , wherein the confidence score is based upon the number of queries in the user search session.

6. The computer storage medium of claim 4 , further comprising tracking which content has divergent user responses to the content over a set threshold, said responses coming from multiple users.