IP Library Granted Patent US 12,645,994
Granted Patent B2
US 12,645,994 · App. 18/147,648 · Granted Jun 2, 2026

Method and system for implementing machine learning analysis of documents

Inventors: Divya Jain (San Jose, CA); Adelbert Chang (Cupertino, CA); Lance Co Ting Keh (Tujunga, CA); Shivani Rao (Mountain View, CA); Sivaramakrishnan Subramanian (San Jose, CA)
Assignee: Box, Inc.
G06N20/00G06F16/35G06F16/383G06F40/242G06F40/284
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,645,994
App. No.
18/147,648
Granted
Jun 2, 2026
Kind
B2
Abstract

Disclosed is an approach for performing auto-classification of documents. A machine learning framework is provided to analyze the document, where labels associated with certain documents can be propagated to other documents.

Claims (44)

1 . A computer implemented method for determining a confidentiality status for documents using label propagation, comprising:

storing a plurality of documents in a document repository on at least one storage device of a cloud-based storage system, wherein a first subset of documents of the plurality of documents are labeled non-confidential, a second subset of documents of the plurality of documents are labeled confidential, and a third subset of documents of the plurality of documents are to be labeled; and

processing, using the cloud-based storage system, individual documents of the third subset of documents by:

accessing a document in the document repository on the at least one storage device of the cloud-based storage system of the third subset of documents to generate a feature vector representing the document of the third subset of documents and to store the feature vector in a database of feature vectors at the cloud-based storage system;

accessing the database of feature vectors at the cloud-based storage system to identify one or more documents of the first or second subsets of documents as one or more neighbor documents to the document based on at least a corresponding feature vector stored in the database of feature vectors; and

storing, at the cloud-based storage system, a recommended label for the document based on one or more labels for the one or more neighbor documents, and wherein the recommended label corresponds to at least one of confidential or non-confidential and the recommended label is generated using one or more rules for label propagation comprising a first rule that specifies propagation of a label when no conflicting labels are assigned to the one or more neighbor documents, a second rule that specifies propagation of a label when a majority of the one or more neighbor documents with labels have the same label, or a third rule that specifies propagation of a label only for a nearest neighbor document of the one or more neighbor documents.

2 . The computer implemented method of claim 1 , wherein the feature vector comprises a term-frequency vector.

3 . The computer implemented method of claim 2 , wherein the feature vector is generated by tokenizing the respective document, and counting a number of occurrences of respective tokens.

4 . The computer implemented method of claim 3 , wherein the feature vector is generated based on at least a local dictionary, and the local dictionary corresponds to a respective customer of a plurality of customers that used the cloud-based storage system.

5 . The computer implemented method of claim 4 , wherein one or more newly identified tokens are added to the local dictionary.

6 . The computer implemented method of claim 1 , wherein documents of the plurality of documents are clustered into a plurality of groups having a centroid feature vector and are represented in a graph maintaining connections between centroids representing relationships between the centroids.

7 . The computer implemented method of claim 1 , wherein the recommended label is presented to a user for acceptance of the label, and only accepted labels are used for label propagation.

8 . The computer implemented method of claim 1 , wherein accessing the database of feature vectors at the cloud-based storage system to identify documents of the plurality of documents as one or more neighbor documents to the respective document is further based on at least metadata comprising a folder name, a file name, or a document owner.

9 . The computer implemented method of claim 8 , wherein a weight is applied to the feature vector or the metadata that increases or decreases a significance of the feature vector or the metadata.

10 . A system, comprising:

a memory storing a set of instructions;

a processor that executes the set of instructions to cause a set of acts for determining a confidentiality status for documents using label propagation, the set of acts comprising:

storing a plurality of documents in a document repository on at least one storage device of a cloud-based storage system, wherein a first subset of documents of the plurality of documents are labeled non-confidential, a second subset of documents of the plurality of documents are labeled confidential, and a third subset of documents of the plurality of documents are to be labeled; and

processing, using the cloud-based storage system, individual documents of the third subset of documents by:

accessing a document in the document repository on the at least one storage device of the cloud-based storage system of the third subset of documents to generate a feature vector representing the document of the third subset of documents and to store the feature vector in a database of feature vectors at the cloud-based storage system;

accessing the database of feature vectors at the cloud-based storage system to identify one or more documents of the first or second subsets of documents as one or more neighbor documents to the document based on at least a corresponding feature vector stored in the database of feature vectors; and

storing, at the cloud-based storage system, a recommended label for the document based on one or more labels for the one or more neighbor documents, and wherein the recommended label corresponds to at least one of confidential or non-confidential and the recommended label is generated using one or more rules for label propagation comprising a first rule that specifies propagation of a label when no conflicting labels are assigned to the one or more neighbor documents, a second rule that specifies propagation of a label when a majority of the one or more neighbor documents with labels have the same label, or a third rule that specifies propagation of a label only for a nearest neighbor document of the one or more neighbor documents.

11 . The system of claim 10 , wherein the feature vector comprises a term-frequency vector.

12 . The system of claim 11 , wherein the feature vector is generated by tokenizing the respective document, and counting a number of occurrences of respective tokens.

13 . The system of claim 12 , wherein the feature vector is generated based on at least a local dictionary, and the local dictionary corresponds to a respective customer of a plurality of customers that used the cloud-based storage system.

14 . The system of claim 13 , wherein one or more newly identified tokens are added to the local dictionary.

15 . The system of claim 10 , wherein documents of the plurality of documents are clustered into a plurality of groups having a centroid feature vector and are represented in a graph maintaining connections between centroids representing relationships between the centroids.

16 . The system of claim 10 , wherein the recommended label is presented to a user for acceptance of the label, and only accepted labels are used for label propagation.

17 . The system of claim 10 , wherein accessing the database of feature vectors at the cloud-based storage system to identify documents of the plurality of documents as one or more neighbor documents to the respective document is further based on at least metadata comprising a folder name, a file name, or a document owner.

18 . The system of claim 17 , wherein a weight is applied to the feature vector or the metadata that increases or decreases a significance of the feature vector or the metadata.

19 . A computer program product embodied on a non-transitory computer usable medium having stored thereon a sequence of instructions which, when executed by a processor causes a set of acts for determining a confidentiality status for documents using label propagation, the set of acts comprising:

storing a plurality of documents in a document repository on at least one storage device of a cloud-based storage system, wherein a first subset of documents of the plurality of documents are labeled non-confidential, a second subset of documents of the plurality of documents are labeled confidential, and a third subset of documents of the plurality of documents are to be labeled; and

processing, using the cloud-based storage system, individual documents of the third subset of documents by:

accessing a document in the document repository on the at least one storage device of the cloud-based storage system of the third subset of documents to generate a feature vector representing the document of the third subset of documents and to store the feature vector in a database of feature vectors at the cloud-based storage system;

accessing the database of feature vectors at the cloud-based storage system to identify one or more documents of the first or second subsets of documents as one or more neighbor documents to the document based on at least a corresponding feature vector stored in the database of feature vectors; and

storing, at the cloud-based storage system, a recommended label for the document based on one or more labels for the one or more neighbor documents, and wherein the recommended label corresponds to at least one of confidential or non-confidential and the recommended label is generated using one or more rules for label propagation comprising a first rule that specifies propagation of a label when no conflicting labels are assigned to the one or more neighbor documents, a second rule that specifies propagation of a label when a majority of the one or more neighbor documents with labels have the same label, or a third rule that specifies propagation of a label only for a nearest neighbor document of the one or more neighbor documents.

20 . The computer program product of claim 19 , wherein the feature vector comprises a term-frequency vector.

21 . The computer program product of claim 20 , wherein the feature vector is generated by tokenizing the respective document, and counting a number of occurrences of respective tokens.

22 . The computer program product of claim 21 , wherein the feature vector is generated based on at least a local dictionary, and the local dictionary corresponds to a respective customer of a plurality of customers that used the cloud-based storage system.

23 . The computer program product of claim 22 , wherein one or more newly identified tokens are added to the local dictionary.

24 . The computer program product of claim 19 , wherein documents of the plurality of documents are clustered into a plurality of groups having a centroid feature vector and are represented in a graph maintaining connections between centroids representing relationships between the centroids.

25 . The computer program product of claim 19 , wherein the recommended label is presented to a user for acceptance of the label, and only accepted labels are used for label propagation.

26 . The computer program product of claim 19 , wherein accessing the database of feature vectors at the cloud-based storage system to identify documents of the plurality of documents as one or more neighbor documents to the respective document is further based on at least metadata comprising a folder name, a file name, or a document owner.

27 . The computer program product of claim 26 , wherein a weight is applied to the feature vector or the metadata that increases or decreases a significance of the feature vector or the metadata.

Assignments (2)
SECURITY INTEREST Recorded Jul 26, 2023
From: BOX, INC.
To: WELLS FARGO BANK, NATIONAL ASSOCIATION
Reel/Frame 064389/0686 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 28, 2022
From: JAIN, DIVYA; CHANG, ADELBERT; KEH, LANCE CO TING; RAO, SHIVANI; SUBRAMANIAN, SIVARAMAKRISHNAN
To: BOX, INC.
Reel/Frame 062229/0093 →
Continuity (3)
Continuation 15016682 · Feb 5, 2016
Provisional Application 62113288 · Feb 6, 2015
Related Publication 20230214723A1 · Jul 6, 2023
References Cited (47)
US 8774515B2 · Mensink · 2014 [cited by examiner]
US 9058327B1 · Lehrman · 2015 [cited by examiner]
US 9092802B1 · Akella · 2015 [cited by applicant]
US 9928295B2 · Lambert et al. · 2018 [cited by applicant]
US 9953075B1 · Reiner · 2018 [cited by examiner]
US 10157347B1 · Kasturi et al. · 2018 [cited by applicant]
US 10354187B2 · Kasravi · 2019 [cited by examiner]
US 10963810B2 · Dirac et al. · 2021 [cited by applicant]
US 11562286B2 · Jain · 2023 [cited by examiner]
US 20030061201A1 · Grefenstette et al. · 2003 [cited by applicant]
US 20040205448A1 · Grefenstette et al. · 2004 [cited by applicant]
US 20060123083A1 · Goutte · 2006 [cited by examiner]
US 20090089078A1 · Bursey · 2009 [cited by applicant]
US 20100312769A1 · Bailey · 2010 [cited by examiner]
US 20110137898A1 · Gordo · 2011 [cited by examiner]
US 20120143797A1 · Wang · 2012 [cited by examiner]
US 20120296891A1 · Rangan · 2012 [cited by examiner]
US 20140304821A1 · Chari · 2014 [cited by examiner]
US 20150046779A1 · Akselrod et al. · 2015 [cited by applicant]
US 20150078672A1 · Eguro et al. · 2015 [cited by applicant]
US 20150324459A1 · Chhichhia et al. · 2015 [cited by applicant]
US 20150379427A1 · Dirac et al. · 2015 [cited by applicant]
US 20150379428A1 · Dirac et al. · 2015 [cited by applicant]
US 20150379429A1 · Lee et al. · 2015 [cited by applicant]
US 20160034757A1 · Chhichhia et al. · 2016 [cited by applicant]
US 20160070731A1 · Chang et al. · 2016 [cited by applicant]
US 20160147891A1 · Chhichhia et al. · 2016 [cited by applicant]
US 20160232456A1 · Jain · 2016 [cited by examiner]
Final Office Action for U.S. Appl. No. 15/016,682 dated Feb. 4, 2021. [cited by applicant]
Final Office Action for U.S. Appl. No. 15/016,682 dated Jul. 29, 2019. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 15/016,682 dated Feb. 1, 2022. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 15/016,682 dated Dec. 13, 2019. [cited by applicant]
Non-Final Office Action for U.S. Appl. No. 15/016,682 dated Jan. 10, 2019. [cited by applicant]
Notice of Allowance for U.S. Appl. No. 15/016,682 dated Sep. 6, 2022. [cited by applicant]
Krishna et al.—“A New Approach to Mining Fuzzy Databases Using Nearest Neighbor Classification by Exploiting Attribute Hierarchies”—2004—https://onlinelibrary.wiley.com/doi/pdf/10.1 002/int.20048 (Year: 2004). [cited by applicant]
Kalhori, Sharareh R. Niakan. “Improvement the accuracy of six applied classification algorithms through Integrated Supervised and Unsupervised Learning Approach.” Journal of Computer and Communications 2.04 (2014): 201. [cited by applicant]
Ayodele, Taiwo Oladipupo. “Types of machine learning algorithms.” New advances in machine learning (2010): 19-48. [cited by applicant]
Boinee, Praveen. “Insights into machine learning: Data clustering and classification algorithms for astrophysical experiments.” (2006). [cited by applicant]
Kotsiantis, Sotiris B., Ioannis D. Zaharakis, and Panayiotis E. Pintelas. “Machine learning: a review of classification and combining techniques.” Artificial Intelligence Review 26.3 (2006): 159-190. [cited by applicant]
Zhu, Xiaojin, and Zoubin Ghahramanih. “Learning from labeled and unlabeled data with label propagation.” (2002). [cited by applicant]
Lance Co Ting Keh, “Evaluating Apache Spark and Twitter Scalding”, Box Blogs, URL:“https://blog.box.com/evaluating-apache-spark-and-twitter-scalding”, Jul. 24, 2014. [cited by applicant]
Fujiwara, Yasuhiro, and Go Irie. “Efficient label propagation.” International Conference on Machine Learning. 2014. [cited by applicant]
Saumitra Srivastav, “Scalable machine learning with apache spark and mlbase”, glassbeam.com, URL: https://www.glassbeam.com/blog/scalable-machine-learning-apache-spark-and-mlbase, Jul. 11, 2014. [cited by applicant]
“Hadoop MapReduce vs. Apache Spark Who Wins the Battle?”, dezyre.com, URL: https://www.dezyre.com/article/hadoop-mapreduce-vs-apache-spark-who-wins-the-battle/83, Nov. 12, 2014. [cited by applicant]
Dean, Jeffrey, and Sanjay Ghemawat. “MapReduce: Simplified data processing on large clusters.” (2004). [cited by applicant]
“Twitter's Scalding—Scala and Hadoop hand in hand”, URL:“https://bighadoop.wordpress.com/2012/04/29/twitter-scalding-scala-and-hadoop-hand-in-hand/”, Apr. 29, 2012. [cited by applicant]
David Matuszek, “Understanding the Scala API”, University URL:“https://www.cis.upenn.edu/˜matuszek/cis554-2011/Pages/scala-api.html”, 2011. [cited by applicant]