IP Library Granted Patent US 12,579,368
Granted Patent B2
US 12,579,368 · App. 18/583,631 · Granted Mar 17, 2026

System, device, and method to provide generalized knowledge routing utilizing machine learning to a user within the system

Inventor: Dayne Freitag (Descanso, CA)
G06F40/30G06Q10/101H04L67/55
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,579,368
App. No.
18/583,631
Granted
Mar 17, 2026
Kind
B2
Abstract

The machine learning in the neural networks module can analyze an annotation and its metadata on the annotation made by a first user on a first computing device to make an embedding regarding the annotation and then cooperate with the persistence knowledge store to store the embedding of the machine learning's understanding of the annotation and its metadata. The delivery module can proactively push a notice regarding a potentially related embedding out to a second user on a second computing device based on a threshold amount of relatedness between one or more factors of i) a first task undertaken by the first user and a second task undertaken by the second user, ii) a role of the first user and a role of the second user, and iii) a subject matter of the embedding to a subject matter of a task undertaken by the second user.

Claims (39)

1 . An apparatus, comprising:

a delivery module that is configured to cooperate with a neural networks model, and a persistence knowledge store,

where machine learning in the neural networks module is configured to analyze an annotation and its metadata on the annotation made by a first user on a first computing device to make an embedding regarding the annotation and then cooperate with the persistence knowledge store to store the embedding of the machine learning's understanding of the annotation and its metadata, and

where the delivery module is further configured to facilitate a collaborative process between users of a system by sending a push notice regarding a potentially related embedding to a second user on a second computing device based on a threshold amount of relatedness between one or more factors selected from a group consisting of i) a first task undertaken by the first user and a second task undertaken by the second user, ii) a role of the first user and a role of the second user, and iii) a subject matter of the embedding to a subject matter of a task undertaken by the second user, where portions of the delivery module, the neural networks model, and the persistence knowledge store implemented in software instructions are stored in an executable format in one or more memories to be executed by one or more processing units.

2 . The apparatus of claim 1 , further comprising:

a collections module is configured to use one or more sensors to capture and understand content in the annotation and portions of a file corresponding to the annotation, and then after a semantic analysis and parsing of the content in the portion of the file and its corresponding annotation, then the collections module is configured to supply the portion of the file and its corresponding annotation for the analysis using the machine learning in the neural networks model to create the embedding.

3 . The apparatus of claim 1 , where the delivery module is configured to cooperate with at least one of the persistence knowledge store and the neural networks model to monitor activities of the second user in order to trigger a proactive push of the notice regarding the potentially related embedding.

4 . The apparatus of claim 1 , where the neural networks model is configured to use synthetic data coming from a generative large language model to generate synthetic versions of the annotations with the synthetic data in an initial training process of the neural networks model.

5 . The apparatus of claim 1 , where the delivery module is further configured to cooperate with at least one of the persistence knowledge store and the neural networks model, where the delivery module is configured to make queries back and forth with at least one of the neural networks module and the persistence knowledge store in order to retrieve information regarding the embedding on the annotation its meta data and a portion of a file corresponding to the annotation, when making a determination to proactively push the notice regarding the potentially related embedding out to the second user on the second computing device.

6 . The apparatus of claim 1 , further comprising:

an evaluation module configured to track whether the second user consumed the proactively delivered notice.

7 . The apparatus of claim 6 , where the evaluation module is configured to cooperate with a feedback module such that the consumption event of the proactively delivered notice is then supplied by the feedback module to at least one of i) the persistence knowledge store to adjust a distance between stored embeddings in a common shared embedding space relative to other embeddings based upon whether the proactively delivered notice was consumed by the second user or not, and ii) to the neural networks model to refine the machine learning's understanding of the embedding with additional information about the embedding based upon whether the proactively delivered notice was consumed by the second user or not and then store the refined machine learning's understanding of the embedding in the common shared embedding space relative to other embeddings in the persistence knowledge store.

8 . The apparatus of claim 1 , where the collection module and the neural networks model are configured to cooperate with the persistence knowledge store to create a shared embedding space for the embedding relative to other embeddings, as well as the persistence knowledge store has a search function for all of the embeddings stored in that persistence knowledge store, where a corpus of all of the embeddings were produced by the neural networks model.

9 . The apparatus of claim 1 , where the delivery module is further configured to proactively push the notice regarding the potentially related embedding out to the second user on the second computing device based on the threshold amount of relatedness between all three factors selected from the group consisting of i) the first task undertaken by the first user and the second task undertaken by the second user, ii) the role of the first user and the role of the second user, and iii) the subject matter of the embedding to the subject matter of task undertaken by the second user.

10 . The apparatus of claim 1 , where the neural networks model is composed of at least one or more graph neural networks.

11 . A method to provide generalized knowledge routing utilizing machine learning to users within a system, comprising:

providing a delivery module that cooperates with a neural networks model, and a persistence knowledge store,

providing machine learning in the neural networks module to analyze an annotation and its metadata on the annotation made by a first user on a first computing device to make an embedding regarding the annotation and then cooperate with the persistence knowledge store to store the embedding of the machine learning's understanding of the annotation and its metadata, and

providing the delivery module to proactively push a notice regarding a potentially related embedding to a second user on a second computing device based on a threshold amount of relatedness between one or more factors selected from a group consisting of i) a first task undertaken by the first user and a second task undertaken by the second user, ii) a role of the first user and a role of the second user, and iii) a subject matter of the embedding to a subject matter of a task undertaken by the second user, where portions of the delivery module, the neural networks model, and the persistence knowledge store implemented in software instructions are stored in an executable format in one or more memories to be executed by one or more processing units.

12 . The method of claim 11 , further comprising:

providing a collections module to use one or more sensors to capture and understand content in the annotation and portions of a file corresponding to the annotation, and then after a semantic analysis and parsing of the content in the portion of the file and its corresponding annotation, then the collections module to supply the portion of the file and its corresponding annotation for the analysis using the machine learning in the neural networks model to create the embedding.

13 . The method of claim 11 , further comprising:

providing the delivery module to cooperate with at least one of the persistence knowledge store and the neural networks model to monitor activities of the second user in order to proactively push the notice regarding the potentially related embedding.

14 . The method of claim 11 , further comprising:

providing the neural networks model to use synthetic data coming from a generative large language model to generate synthetic versions of the annotations with the synthetic data in an initial training process of the neural networks model.

15 . The method of claim 11 , further comprising:

providing the delivery module to cooperate with at least one of the persistence knowledge store and the neural networks model, where the delivery module is configured to make queries back and forth with at least one of the neural networks module and the persistence knowledge store in order to retrieve information regarding the embedding on the annotation its meta data and a portion of a file corresponding to the annotation, when making a determination to proactively push the notice regarding the potentially related embedding out to the second user on the second computing device.

16 . The method of claim 11 , further comprising:

providing an evaluation module to track whether the second user finds the proactively delivered notice on the annotation useful by determining whether the second user consumed the proactively delivered notice.

17 . The method of claim 16 , further comprising:

providing the evaluation module to cooperate with a feedback module such that the consumption event of the proactively delivered notice is then supplied by the feedback module to at least one of i) the persistence knowledge store to adjust a distance between stored embeddings in a common shared embedding space relative to other embeddings based upon whether the proactively delivered notice was consumed by the second user or not, and ii) to the neural networks model to refine the machine learning's understanding of the embedding with additional information about the embedding based upon whether the proactively delivered notice was consumed by the second user or not and then store the refined machine learning's understanding of the embedding in the common shared embedding space relative to other embeddings in the persistence knowledge store.

18 . The method of claim 11 , further comprising:

providing the delivery module and the neural networks model to cooperate with the persistence knowledge store to create a shared embedding space for the embedding relative to other embeddings, as well as the persistence knowledge store has a search function for all of the embeddings stored in that persistence knowledge store, where a corpus of all of the embeddings were produced by the neural networks model.

19 . The method of claim 11 , further comprising:

providing the delivery module to proactively push the notice regarding the potentially related embedding out to the second user on the second computing device based on the threshold amount of relatedness between all three factors selected from the group consisting of i) the first task undertaken by the first user and the second task undertaken by the second user, ii) the role of the first user and the role of the second user, and iii) the subject matter of the embedding to the subject matter of task undertaken by the second user.

20 . A non-transitory machine-readable medium, which stores further instructions in an executable format by one or more processors to cause operations as follows, comprising:

using a delivery module that cooperates with a neural networks model, and a persistence knowledge store,

using machine learning in the neural networks module to analyze an annotation and its metadata on the annotation made by a first user on a first computing device to make an embedding regarding the annotation and then cooperate with the persistence knowledge store to store the embedding of the machine learning's understanding of the annotation and its metadata, and

using the delivery module to proactively push a notice regarding a potentially related embedding out to a second user on a second computing device based on a threshold amount of relatedness between one or more factors selected from a group consisting of i) a first task undertaken by the first user and a second task undertaken by the second user, ii) a role of the first user and a role of the second user, and iii) a subject matter of the embedding to a subject matter of a task undertaken by the second user, where portions of the delivery module, the neural networks model, and the persistence knowledge store implemented in software instructions are stored in an executable format in one or more memories to be executed by one or more processing units.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 8, 2024
From: FREITAG, DAYNE
To: SRI INTERNATIONAL
Reel/Frame 066706/0683 →
Continuity (2)
Provisional Application 63447561 · Feb 22, 2023
Related Publication 20240281610A1 · Aug 22, 2024
References Cited (37)
US 8135669B2 · Olstad · 2012 [cited by examiner]
US 9110873B2 · Woodall et al. · 2015 [cited by applicant]
US 9348815B1 · Estes · 2016 [cited by examiner]
US 10163043B2 · Eigen et al. · 2018 [cited by applicant]
US 11397462B2 · Eledath et al. · 2022 [cited by applicant]
US 20190129749A1 · White · 2019 [cited by examiner]
US 20190325342A1 · Sikka et al. · 2019 [cited by applicant]
US 20200293975A1 · Faulkner · 2020 [cited by examiner]
US 20220058489A1 · Volkovs · 2022 [cited by examiner]
US 20220405709A1 · Nouri · 2022 [cited by examiner]
Bansal, A., Sikka, K., Sharma, G., Chellapa, R. and Divakaran, A. Zero-Shot Object Detection, European Conference on Computer Vision, 2018. [cited by applicant]
Datta, S., Sikka, K., Roy, A., Ahuja, K., Parikh, D. and Divakaran, A. Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment. Intl. Conference on Computer Vision, 2019. [cited by applicant]
Devlin, J., Chang, M. W., Lee, K. and Toutanova, K., BERT: Pre-training of deep bidirectional transformers for language understanding, 2018. arXiv. [cited by applicant]
Elizalde, B., Zarar, S. and Raj, B. Cross Modal Audio Search and Retrieval with Joint Embeddings Based on Text and Audio. Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019. [cited by applicant]
Fox., M. and Long, D., PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains. Journal of Artificial Intelligence Research, vol. 20, pp. 61-124, 2003. [cited by applicant]
Freed, M., Yarnall, L., Spaulding, A. and Gervasio, M. A Mobile Strategy for Self-Directed Learning in the Workplace. Interservice/Industry Training, Simulation, and Education Conference, 2017. [cited by applicant]
Gervasio, M. and Myers, K. Learning procedures by augmenting sequential pattern mining with planning knowledge. Advances in Cognitive Systems, 2020. [cited by applicant]
Li, X., Yin X., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Dong, L., Wei, F., Choi, Y., Gao, J., Oscar: object-semantics aligned pre-training for vision-language tasks, ECCV 2020. [cited by applicant]
Myers, K., Kolojejchick, J., Angiolillo, C., Cummings, T., Garvey, T., Gervasio, M., Haines, W., Jones, C., Knittel, J., Morley, D., Ommert, W. and Potter, S. Learning by Demonstration Technology for Military Planning a… [cited by applicant]
Myers, K. and Gervasio, M. Solution authoring via demonstration and annotation: An empirical study. Proceedings of the IEEE 16th International Conference on Advanced Learning Technologies (ICALT), 2016. [cited by applicant]
Peintner, B., Dinger, J., Rodriguez, A. and Myers, K. Task Assistant: Personalized Task Management for Military Environments. Innovative Applications of Artificial Intelligence, 2009. [cited by applicant]
Qiu, Q., Lezama, J., Bronstein, A. and Sapiro, G. ForestHash: Semantic Hashing with Shallow Random Forests and Tiny Convolutional Networks, European Conference on Computer Vision, 2018. [cited by applicant]
Quirós, L. and Vidal, E., Evaluation of a Region Proposal Architecture for Multi-task Document Layout Analysis.,2021. [cited by applicant]
Roundtree, K. A., Goodrich, M. A. and Adams, J. A. Transparency: Transitioning from human-machine systems to human-swarm systems. Journal of Cognitive Engineering and Decision Making, vol. 13(3), (pp. 171-195), 2019. [cited by applicant]
Sahu, P., Cogswell, M., Rutherford-Quach, S. and Divakaran, A. Comprehension Based Question Answering Using Bloom's Taxonomy, arXiv, 2021. [cited by applicant]
Sahu, P., Sikka, K. and Divakaran, A. Towards Solving Multimodal Comprehension, arXiv, 2021. [cited by applicant]
Shwartz, V., West, P., Le Bras, R., Bhagavatula, C., and Choi, Y., Unsupervised commonsense question answering with self-talk, EMNLP 2021. [cited by applicant]
Smith, R.G., Schoen, E.J., Eckroth, J., Endres, D.M., Florez, S., Elliott, J. and Buchanan, B.G., SPE Research Portal: How SPE Uses Artificial Intelligence to Help You Find Technical Information. The Way Ahead, Society … [cited by applicant]
Trabasso, T. and Suh, S. Understanding text: Achieving explanatory coherence through online inferences and mental operations in working memory. Discourse processes, vol. 16(1-2), (pp. 3-34), 1993. [cited by applicant]
St. John, M. F. and Smallman, H. S. Staying up to speed: Four design principles for maintaining and recovering situation awareness. Journal of Cognitive Engineering and Decision Making, vol. 2, (pp. 118-139), 2008. [cited by applicant]
Xie, H. and Virtanen, T. Zero-Shot Audio Classification Via Semantic Embeddings, in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29,1242), 2021. [cited by applicant]
Yang, Z., Garcia, N., Chu, C., Otani, M., Nakashima, Y. and Takemura, H. BERT representations for video question answering. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 1556-1565… [cited by applicant]
Yang, F., Matei. B. and Davis, L. S. Re-Ranking by Multi-Feature Fusion with Diffusion for Image Retrieval. IEEE Workshop on Applications of Computer Vision, 2015. [cited by applicant]
Yorke-Smith, N., Saadati, S., Myers, K. and Morley, D. Like an Intuitive and Courteous Butler: A Proactive Personal Agent for Task Management. Autonomous Agents and Multi Agent Systems, vol. 1, (pp. 337-344), 2009. [cited by applicant]
Zhong, X., Tang, J. and Yepes, A. J. PubLayNet: largest dataset ever for document layout analysis. In 2019 International Conference on Document Analysis and Recognition (ICDAR) (pp. 1015-1022), 2019. https://arxiv.org/a… [cited by applicant]
Zhu, X., Singla, A., Zilles, S. and Rafferty, A. N. An overview of machine teaching, 2018. arXiv preprint arXiv:1801.05927. [cited by applicant]
Zhang, P., Li, X., Hu, X., Yang, J., Zhang,L., Wang, L., Choi,Y. and Gao, J., VinVL: revisiting visual representations in vision-language models, Proc. of CVPR 2021. https://arxiv.org/abs/2101.00529. [cited by applicant]