System, method and computer program product for protecting derived metadata when updating records within a search engine
View Patent ↗A search engine may maintain a list of derived metadata. When an event occurs that requires updating a search index, the search engine can determine which metadata is derived metadata and take appropriate actions with respect to the derived metadata. For example, if a request is received to update the index for a particular object, the search engine may protect the derived metadata from change while updating the other metadata in the index. As another example, if a request is received to update the text content for the object, the search engine may change the text content and the derived metadata. By identifying derived metadata, the search engine can protect the derived metadata from change when a request is received that otherwise causes metadata to change and can change the derived metadata when a request is received that would otherwise not change the metadata portion of the index.
1. A computerized search system, comprising:
a search engine embodied on non-transitory computer-readable memory;
at least one processor;
a storage unit storing a search index and instructions; and
stored instructions, wherein the stored instructions when translated by the at least one processor instruct the search engine to:
receive an indexing request associated with an object;
identify any derived metadata fields in the search index that would be affected by the indexing request associated with the object;
determine whether any of the identified derived metadata fields in the search index that would be affected by the indexing request associated with the object are within a list or record of protected metadata fields;
retain any of the identified derived metadata fields in the search index that would be affected by the indexing request associated with the object and that are determined to be within the list or record of protected metadata fields such that any of the identified derived metadata fields in the search index thus retained are protected from change caused by the indexing request associated with the object; and
execute an indexing command to update the search index with data associated with the object and without any of the identified derived metadata fields in the search index thus retained by the search engine such that the search index is updated without the search engine deleting or changing any value of the derived metadata fields in the search index that have been identified by the search engine as would be affected by the indexing request associated with the object and that have been determined by the search engine to be associated with the list or record of protected metadata fields, the data comprising text, metadata, or both.
2. The computerized search system of claim 1 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by analyzing at least one of a file format, mime type, or file name extension to determine a file type for the object.
3. The computerized search system of claim 1 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by applying a filter to parse a known file type to extract metadata that is included in the object.
4. The computerized search system of claim 1 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by:
analyzing the text to determine at least one language of the text; and
calculating a confidence score for the determined language.
5. The computerized search system of claim 1 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by applying language rules to determine whether language in the text is positive, negative, or neutral.
6. The computerized search system of claim 1 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by identifying keywords or concepts in the text using semantic or statistical methods.
7. The computerized search system of claim 1 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by applying language rules or known lists to identify significant people, places, or events in the data.
8. The computerized search system of claim 1 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by creating a list of geospatial coordinates that represent probable locales in the data.
9. The computerized search system of claim 1 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by applying business rules and patterns to identify a most likely classification of the object.
10. The computerized search system of claim 1 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by generating a code that can be used to match documents that have similar or identical sections.
11. A computer program product comprising at least one non-transitory computer-readable medium storing instructions translatable by at least one processor of a computerized search system for instructing a search engine of the computerized search system to:
receive an indexing request associated with an object;
identify any derived metadata fields in a search index that would be affected by the indexing request associated with the object, the search index stored in a storage unit of the computerized search system;
determine whether any of the identified derived metadata fields in the search index that would be affected by the indexing request associated with the object are within a list or record of protected metadata fields;
retain any of the identified derived metadata fields in the search index that would be affected by the indexing request associated with the object and that are determined to be within the list or record of protected metadata fields such that any of the identified derived metadata fields in the search index thus retained are protected from change caused by the indexing request associated with the object; and
execute an indexing command to update the search index with data associated with the object and without any of the identified derived metadata fields in the search index thus retained by the search engine such that the search index is updated without the search engine deleting or changing any value of the derived metadata fields in the search index that have been identified by the search engine as would be affected by the indexing request associated with the object and that have been determined by the search engine to be associated with the list or record of protected metadata fields, the data comprising text, metadata, or both.
12. The computer program product of claim 11 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by analyzing at least one of a file format, mime type, or file name extension to determine a file type for the object.
13. The computer program product of claim 11 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by applying a filter to parse a known file type to extract metadata that is included in the object.
14. The computer program product of claim 11 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by:
analyzing the text to determine at least one language of the text; and
calculating a confidence score for the determined language.
15. The computer program product of claim 11 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by applying language rules to determine whether language in the text is positive, negative, or neutral.
16. The computer program product of claim 11 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by identifying keywords or concepts in the text using semantic or statistical methods.
17. The computer program product of claim 11 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by applying language rules or known lists to identify significant people, places, or events in the data.
18. The computer program product of claim 11 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by creating a list of geospatial coordinates that represent probable locales in the data.
19. The computer program product of claim 11 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by applying business rules and patterns to identify a most likely classification of the object.
20. The computer program product of claim 11 , wherein the list or record of protected metadata fields that are to be protected from change includes a metadata field derived by generating a code that can be used to match documents that have similar or identical sections.