IP Library Granted Patent US 7,376,620
Granted Patent B2
US 7,376,620 · App. 09/911,839 · Granted May 20, 2008

System and method for measuring the quality of information retrieval

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 7,376,620
App. No.
09/911,839
Filed
Jul 23, 2001
Granted
May 20, 2008
Kind
B2
Art Unit
3692
USPC
705/40
Abstract

In an information retrieval application, a system and method for detecting content holes. A content body is parsed into a plurality of concepts nodes, including a first concept node. A percentage of successful service interactions is determined as a function of concept node and, if the percentage of successful service interactions at the first concept node is below a predefined threshold, a content hole is flagged.

Claims (42)

1. A computer-assisted method for detecting content holes, comprising:

accessing a content body organized into a plurality of concept nodes to which content is selectively pre-tagged before receiving a user query as part of a service interaction,

wherein the content body includes a first concept node;

determining a percentage of successful service interactions as a function of concept nodes; and

if the percentage of successful service interactions related to the first concept node is below a predefined threshold, flagging a content hole and presenting an indication of the flagged content hole to a user or a computerized process.

2. A computer-readable medium having instructions that, when executed in a computer, detects content holes-by:

accessing a content body organized into a plurality of concept nodes to which content is selectively pre-tagged before receiving; a user query as part of a service interaction,

wherein the content body includes a first concept node;

determining a percentage of successful service interactions as a function of concept nodes; and

if the percentage of successful service interactions related to the first concept node is below a predefined threshold, flagging a content hole and presenting an indication of the flagged content hole to a user or a computerized process.

3. A computer-assisted method of charging for services, comprising:

determining a percentage of successful service interactions in a first information retrieval system;

determining a percentage of successful service interactions for services provided in a second information retrieval system; and

billing as a function of the difference between the percentage of successful service interactions in the first information retrieval system and the percentage of successful service interactions for services provided in the second information retrieval system.

4. The computer-assisted method according to claim 3 , wherein determining a percentage of successful service interactions for services provided in the second information retrieval system includes:

accessing a content body organized into a plurality of concept nodes to which content is selectively pre-tagged before receiving a user query as part of a service interaction, wherein the content body includes a first concept node;

determining a percentage of successful service interactions as a function of concept nodes; and

wherein billing as a function of the difference between the percentage of successful service interactions in the first information retrieval system and the percentage of successful service interactions for services provided in the second information retrieval system includes weighting successful interactions as a function of concept nodes.

5. A computer-readable medium having instructions when executed in a computer, charges for services by:

accessing a content body organized into a plurality of concept nodes to which content is selectively pre-tagged before receiving a user query as part of a service interaction, wherein the content body includes a first concept node;

determining a percentage of successful service interactions in a first information retrieval system;

determining a percentage of successful service interactions for services provided in a second information retrieval system; and

billing as a function of the difference between the percentage of successful service interactions in the first information retrieval system and the percentage of successful service interactions for services provided in the second information retrieval system, wherein the billing includes weighting successful interactions as a function of concept nodes.

6. A computer-assisted method for detecting content holes, comprising:

(a) accessing a content body organized into a plurality of concept nodes to which content is selectively pre-tagged before receiving a user query as part of a service interaction, wherein the content body includes a first concept node;

(b) determining a percentage of successful service interactions (SSIs) as a function of concept nodes;

(c) determining a percentage of queries as a function of concept nodes;

(d) determining a percentage of documents as a function of concept nodes;

(e) computing a content hole score for the first concept node as a function of at least one of (b), (c), and (d); and

(f) flagging a content hole if the content hole score is below a predefined threshold and and presenting an indication of the flagged content hole to a user or a computerized process.

7. A computer-assisted method of charging for services, comprising:

determining a percentage of successful service interactions in an information retrieval system over a period of time; and

billing as a function of the percentage of successful service interactions in the information retrieval system over the period of time.

8. The computer-assisted method of claim 1 , wherein each concept node represents a concept in the content body.

9. The computer-assisted method of claim 1 , wherein the successful service interaction comprises a query from a user for which returned content matches that user's intent.

10. The computer-assisted method of claim 3 , wherein the successful service interaction comprises a query from a user for which returned content matches that user's intent.

11. The computer-assisted method of claim 4 , wherein each concept node represents a concept in the content body.

12. The computer-assisted method of claim 4 , wherein the successful service interaction comprises a query from a user for which returned content matches that user's intent.

13. The computer-assisted method of claim 6 , wherein each concept node represents a concept in the content body.

14. The computer-assisted method of claim 6 , wherein the successful service interaction comprises a query from a user for which returned content matches that user's intent.

15. The computer-assisted method of claim 7 , wherein the successful service interaction comprises a query from a user for which returned content matches that user's intent.

16. The computer-assisted method of claim 6 , wherein the acts (a)-(f) are performed in the order presented in claim 6 .

Assignments (15)
RELEASE OF SECURITY INTEREST Recorded Feb 10, 2026
From: JPMORGAN CHASE BANK, N.A.
To: AVOLIN, LLC
Reel/Frame 073742/0694 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 6, 2019
From: APTEAN, INC.
To: APTEAN SAPPHIRE, LLC
Reel/Frame 049395/0150 →
CHANGE OF NAME Recorded Jun 6, 2019
From: APTEAN SAPPHIRE, LLC
To: AVOLIN, LLC
Reel/Frame 049395/0337 →
RELEASE OF SECURITY INTEREST Recorded Oct 31, 2018
From: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
To: APTEAN, INC.; APTEAN SAPPHIRE, LLC
Reel/Frame 047375/0049 →
RELEASE OF SECURITY INTEREST Recorded Oct 31, 2018
From: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
To: APTEAN, INC.; APTEAN SAPPHIRE, LLC
Reel/Frame 047375/0234 →
SECURITY INTEREST Recorded Oct 30, 2018
From: APTEAN SAPPHIRE, LLC
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 047348/0813 →
SECOND LIEN SECURITY AGREEMENT Recorded Dec 22, 2016
From: APTEAN, INC.; ACTIVPLANT CORPORATION; APTEAN SYSTEMS, LLC; GQ LIFE SCIENCES, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
Reel/Frame 041175/0370 →
FIRST LIEN SECURITY AGREEMENT Recorded Dec 21, 2016
From: APTEAN, INC.; ACTIVPLANT CORPORATION; APTEAN SYSTEMS, LLC; GQ LIFE SCIENCES, INC.
To: MORGAN STANLEY SENIOR FUNDING, INC., AS COLLATERAL AGENT
Reel/Frame 041085/0752 →
RELEASE OF SECOND LIEN SECURITY INTEREST Recorded Dec 20, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: APTEAN, INC.; ACTIVPLANT CORPORATION; APTEAN SYSTEMS, LLC; GQ LIFE SCIENCES, INC.; ADVANCED PUBLIC SAFETY, INC.
Reel/Frame 041035/0631 →
RELEASE OF FIRST LIEN SECURITY INTEREST Recorded Dec 20, 2016
From: MORGAN STANLEY SENIOR FUNDING, INC.
To: APTEAN, INC.; ACTIVPLANT CORPORATION; APTEAN SYSTEMS, LLC; GQ LIFE SCIENCES, INC.; ADVANCED PUBLIC SAFETY, INC.
Reel/Frame 041035/0395 →
FIRST LIEN PATENT SECURITY AGREEMENT Recorded Feb 27, 2014
From: APTEAN, INC.; ACTIVPLANT CORPORATION
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 032366/0564 →
SECOND LIEN PATENT SECURITY AGREEMENT Recorded Feb 27, 2014
From: APTEAN, INC.; ACTIVPLANT CORPORATION
To: MORGAN STANLEY SENIOR FUNDING, INC.
Reel/Frame 032366/0634 →
RELEASE OF SECURITY INTEREST Recorded Feb 26, 2014
From: BANK OF MONTREAL
To: APTEAN, INC., SUCCESSOR-IN-INTEREST TO CONSONA CORPORATION AND TRADEBEAM, INC.
Reel/Frame 032307/0045 →
RELEASE OF SECURITY INTEREST Recorded Aug 6, 2012
From: WELLS FARGO CAPITAL FINANCE, LLC
To: CONSONA CORPORATION, CONSONA ERP, INC., CAPRI CORP., CIMNET SYSTEMS, INC., CIMNET SYSTEMS,SERVICES, LLC, CSI-1, LLC, CSI-2, INC.AND CONSONA CRM INC.
Reel/Frame 028733/0464 →
SECURITY AGREEMENT Recorded Aug 6, 2012
From: CONSONA CORPORATION; TRADEBEAM, INC.
To: BANK OF MONTREAL, AS ADMINISTRATIVE AGENT
Reel/Frame 028734/0560 →