Granted Patent B1

US 11,741,145 · App. 16/147,822 · Granted Aug 29, 2023

Inventors: Bhushan Pandit (Maharashtra, IN); Surashree Kane (Maharashtra, IN); Abhishek Shinde (Maharashtra, IN)

Assignee: Veritas Technologies LLC

G06F16/355G06F16/313

Abstract

Claims (145)

1. A computer-implemented method, implemented in a computer system, comprising:

producing an item identifier corresponding to each unstructured data item of a plurality of unstructured data items, wherein

the producing comprises

performing a hashing operation on information associated with each of the plurality of unstructured data items;

storing each unstructured data item and its corresponding item identifier in association with one another in a storage device of the computer system;

for each item of a plurality of items,

 determining whether a backup operation should be performed on an item of the plurality of items, wherein

the item and each unstructured data item comprises unstructured data, and

the determining comprises

ingesting the item into a classification engine, wherein

the classification engine is implemented in the computer system, and

the ingesting comprises

 generating an item identifier for the item, at least in part, by performing the hash operation on information associated with the item, and

 storing the item identifier and item in association with one another in the storage device,

performing term processing, wherein

the performing the term processing comprises

 determining a first number of occurrences of each term of a plurality of terms in the item, comprising

 identifying at least one term of the plurality of terms in the item by determining a term frequency of each of the plurality of terms in the item, and

 determining an inverse document frequency of the at least one term with respect to the plurality of unstructured data items, and

 determining a second number of occurrences of each term of the plurality of terms in a reference item of unstructured data,

generating a similarity index, comprising

producing a first list of ranking values by ranking the plurality of terms in the item based on the first number of occurrences,

producing a second list of ranking values by ranking the plurality of terms in the reference item based on the second number of occurrences, and

determining a number of common ranking values, wherein

 each ranking value is in the first list of ranking values and in the second list of ranking values, and

responsive to a size of the similarity index meeting a threshold,

determining a relational similarity index, wherein

 the relational similarity index is based, at least in part, on a subset of the first list of ranking values and another subset of a list of ranking values for an unstructured data item of the plurality of unstructured data items, and

 the relational similarity index represents a similarity between the item and the unstructured data item, and

in response to the relational similarity index indicating that the item and the unstructured data item are sufficiently similar, associating a classification tag with the item; and

performing a backup operation on one or more items of the plurality of items that are associated with the classification tag.

2. The method of claim 1 , wherein

the information associated with each unstructured data item comprises at least one of

a data item title,

data item contents, or

data item metadata,

the hash operation comprises a two-level hash operation

the first list of ranking values and the second list of ranking values are stored in a classification matrix, and

the term processing comprises

performing noise filtering on the at least one term.

3. The method of claim 1 , wherein the determining the relational similarity index comprises:

identifying an item pair in a pair list; and

generating a plurality of relational similarity indexes for a plurality of item pairs.

4. The method of claim 3 , wherein the determining the relational similarity index further comprises:

determining whether an item pair list is empty;

in response to determining that the item pair list is empty, determining whether another item has been ingested; and

in response to determining that another item has been ingested, retrieving a plurality of terms of the another item.

5. The method of claim 3 , wherein the determining the relational similarity index further comprises:

enumerating a global item pair list; and

retrieving a plurality of terms of each item in the global item pair list.

6. The method of claim 3 , wherein the identifying the item pair in the pair list comprises:

identifying a plurality of common terms, wherein

the plurality of common terms are common as between the item and the reference item, and

the item and the reference item form a current pair; and

generating a similarity index using the plurality of common terms.

7. The method of claim 6 , wherein the identifying the item pair in the pair list further comprises:

determining whether the similarity index indicates that the current pair should be inserted in a global pair list; and

in response to a determination that the similarity index indicates that the current pair should be inserted in the global pair list, inserting the current pair in the global pair list.

8. The method of claim 6 , wherein the identifying the item pair in the pair list further comprises:

determining whether the similarity index indicates that the current pair should be inserted in a pair list of the item; and

in response to a determination that the similarity index indicates that the current pair should be inserted in the pair list of the item, inserting the current pair in the pair list of the item.

9. The method of claim 3 , wherein the generating the plurality of relational similarity indexes comprises:

determining a maximum number of terms to include from a pair list of the item;

identifying a plurality of common terms in the pair list of the item, wherein

a number of the plurality of common terms in the pair list of the item is less than the maximum number of terms; and

storing the number in a relational similarity index.

10. The method of claim 2 , further comprising:

generating a plurality of item identifiers, wherein

the ranking values of the first list of ranking values and the ranking values of the second list of ranking values are stored in a data structure that associates one or more terms of the plurality of terms of the item with one or more terms of the plurality of terms in the reference item, and

the generating comprises

the performing the hash operation on each item of a plurality of items; and

grouping a plurality of items into one or more groups of items, using classification information generated by the classification engine, wherein

the plurality of items comprises the item.

11. A non-transitory computer-readable storage medium, comprising program instructions, which, when executed by one or more processors of a computing system, perform a method comprising:

producing an item identifier corresponding to each unstructured data item of a plurality of unstructured data items, wherein

the producing comprises

performing a hashing operation on information associated with each of the plurality of unstructured data items;

storing each unstructured data item and its corresponding item identifier in association with one another in a storage device of the computer system;

for each item of a plurality of items,

 determining whether a backup operation should be performed on an item of the plurality of items, wherein

the item and each unstructured data item comprises unstructured data, and

the determining comprises

ingesting the item into a classification engine, wherein

the classification engine is implemented in the computer system, and the ingesting comprises

 generating an item identifier for the item, at least in part, by performing the hash operation on information associated with the item, and

 storing the item identifier and item in association with one another in the storage device,

performing term processing, wherein

the performing the term processing comprises

 determining a first number of occurrences of each term of a plurality of terms in the item, comprising

 identifying at least one term of the plurality of terms in the item by determining a term frequency of each of the plurality of terms in the item, and

 determining an inverse document frequency of the at least one term with respect to the plurality of unstructured data items, and

 determining a second number of occurrences of each term of the plurality of terms in a reference item, wherein the reference item comprises unstructured data,

generating a similarity index, comprising

producing a first list of ranking values by ranking the plurality of terms in the item based on the first number of occurrence,

producing a second list of ranking values by ranking the plurality of terms in the reference item based on the second number of occurrences, and

determining a number of common ranking values, wherein

 each ranking value is in the first list of ranking values and in the second list of ranking values, and

responsive to a size of the similarity index meeting a threshold,

determining a relational similarity index, wherein

 the relational similarity index is based, at least in part, on processing a subset of the first list of ranking values and processing another subset of a list of ranking values for an unstructured data item of the plurality of unstructured data items, and

 the relational similarity index represents a similarity between the item and the unstructured data item, and

 in response to the relational similarity index indicating that the item and the unstructured data item are sufficiently similar, associating a classification tag with the item; and

performing a backup operation on one or more items of the plurality of items that are associated with the classification tag.

12. The non-transitory computer-readable storage medium of claim 11 , wherein

the information associated with each unstructured data item comprises at least one of

a data item title,

data item contents, or

data item metadata,

the hash operation comprises a two-level hash operation

the first list of ranking values and the second list of ranking values are stored in a classification matrix, and

the term processing comprises

performing noise filtering on the at least one term.

13. The non-transitory computer-readable storage medium of claim 11 , wherein the determining the relational similarity index comprises:

identifying an item pair in a pair list; and

generating a plurality of relational similarity indexes for a plurality of item pairs.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the determining the relational similarity index further comprises:

determining whether an item pair list is empty;

in response to determining that the item pair list is empty, determining whether another item has been ingested; and

in response to determining that another item has been ingested, retrieving a plurality of terms of the another item.

15. The non-transitory computer-readable storage medium of claim 13 , wherein the determining the relational similarity index further comprises:

enumerating a global item pair list; and

retrieving a plurality of terms of each item in the global item pair list.

16. The non-transitory computer-readable storage medium of claim 13 , wherein the identifying the item pair in the pair list comprises:

identifying a plurality of common terms, wherein

the plurality of common terms are common as between the item and the reference item, and

the item and the reference item form a current pair; and

generating a similarity index using the plurality of common terms.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the identifying the item pair in the pair list further comprises:

determining whether the similarity index indicates that the current pair should be inserted in a global pair list; and

in response to a determination that the similarity index indicates that the current pair should be inserted in the global pair list, inserting the current pair in the global pair list.

18. The non-transitory computer-readable storage medium of claim 16 , wherein the identifying the item pair in the pair list further comprises:

determining whether the similarity index indicates that the current pair should be inserted in a pair list of the item; and

in response to a determination that the similarity index indicates that the current pair should be inserted in the pair list of the item, inserting the current pair in the pair list of the item.

19. The non-transitory computer-readable storage medium of claim 13 , wherein the generating the plurality of relational similarity indexes comprises:

determining a maximum number of terms to include from a pair list of the item;

identifying a plurality of common terms in the pair list of the item, wherein

a number of the plurality of common terms in the pair list of the item is less than the maximum number of terms; and

storing the number in a relational similarity index.

20. The non-transitory computer-readable storage medium of claim 12 , wherein the method further comprises:

generating a plurality of item identifiers, wherein

the ranking values of the first list of ranking values and the ranking values of the second list of ranking values are stored in a data structure that associates one or more terms of the plurality of terms of the item with one or more terms of the plurality of terms in the reference item, and

the generating comprises

the performing the hash operation on each item of a plurality of items; and

grouping a plurality of items into one or more groups of items, using classification information generated by the classification engine, wherein

the plurality of items comprises the item.

Assignments (14)

Cited By (1)

US 12,332,751