IP Library › Granted Patent US 12,530,619
Granted Patent B2
US 12,530,619 · App. 17/812,605 · Granted Jan 20, 2026

Feature-aware open set multi-model for trajectory classification in mobile edge devices

Inventors: Paulo Abelha Ferreira (Rio de Janeiro, BR); Vinicius Michel Gottin (Rio de Janeiro, BR); Pablo Nascimento da Silva (Niteroi, BR)
Assignee: Dell Products L.P.
G06N20/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,530,619
App. No.
17/812,605
Filed
Jul 14, 2022
Granted
Jan 20, 2026
Kind
B2
Art Unit
2166
USPC
706/12
Abstract

One example method includes training an event predictor, of a machine learning model, wherein the training includes training an event predictor of the machine learning model, and training open set trajectory classifier, of the machine learning model. After the event predictor is trained, the event predictor is operable to receive an unknown trajectory class and predict an event class for the unknown trajectory class. Further, after the open set trajectory classifier is trained, the open set trajectory classifier is operable to receive an unknown trajectory and classify the unknown trajectory with a predicted trajectory classification. Finally, during training, the output of the open-set trajectory classifier may be used as an input to the event predictor.

Claims (34)

1 . A method, comprising:

training an event predictor, of a machine learning model, wherein the training comprises:

providing, to the event predictor, a predicted trajectory classification that is unknown to the event predictor component; and

providing, to the event predictor, a trajectory feature that is associated with the predicted trajectory classification,

wherein after the event predictor is trained with the predicted trajectory classification and the trajectory feature, the event predictor is operable to receive an unknown trajectory class and predict an event class for the unknown trajectory class, or predict that it is in fact an unknown trajectory class; and

training an open set trajectory classifier, of the machine learning model, wherein the training of the open set trajectory classifier comprises:

performing a closed set training process using a labeled dataset that maps a trajectory and its trajectory features to a trajectory class; and

performing an open set training process using the labeled dataset, and performing the open set training process using the labeled dataset comprises performing a first reconstruction of the trajectories in the labeled dataset for those trajectories in the labeled dataset that are in a correct class, and performing a second reconstruction of the trajectories in the labeled dataset for those trajectories in the labeled dataset that are in an incorrect class,

wherein after the open set trajectory classifier is trained, the open set trajectory classifier is operable to receive an unknown trajectory and classify the unknown trajectory with the predicted trajectory classification.

2 . The method as recited in claim 1 , wherein the predicted trajectory classification is provided by the open set trajectory classifier to the event predictor.

3 . The method as recited in claim 1 , wherein the unknown trajectory comprises information about a movement and/or position of a mobile edge device, in which the machine learning model is deployed, in a physical operating domain.

4 . The method as recited in claim 1 , wherein the open set training process for the open set trajectory classifier accounts for a class imbalance across trajectory classes that are known to the open set trajectory classifier.

5 . The method as recited in claim 1 , wherein the machine learning model is operable to detect abnormal events occurring during operation of a mobile edge device in which the machine learning model is deployed.

6 . The method as recited in claim 1 , wherein the event predictor comprises an auto-encoder that receives, as inputs, the predicted trajectory classification and the trajectory feature.

7 . The method as recited in claim 1 , wherein the open set trajectory classifier comprises an auto-encoder that receives, as input, trajectory features of the unknown trajectory.

8 . The method as recited in claim 1 , wherein training of the machine learning model is automatically triggered as a result of a monitoring process that tracks performance of the machine learning model.

9 . The method as recited in claim 1 , further comprising deploying the machine learning model, after the machine learning model is trained, to a group of mobile edge devices.

10 . A non-transitory storage medium having stored therein instructions that are executable by one or more hardware processors to perform operations comprising:

training an event predictor, of a machine learning model, wherein the training comprises:

providing, to the event predictor, a predicted trajectory classification that is unknown to the event predictor component; and

providing, to the event predictor, a trajectory feature that is associated with the predicted trajectory classification,

wherein after the event predictor is trained with the predicted trajectory classification and the trajectory feature, the event predictor is operable to receive an unknown trajectory class and predict an event class for the unknown trajectory class, or predict that it is in fact an unknown trajectory class; and

training an open set trajectory classifier, of the machine learning model, wherein the training of the open set trajectory classifier comprises:

performing a closed set training process using a labeled dataset that maps a trajectory and its trajectory features to a trajectory class; and

performing an open set training process using the labeled dataset, and performing the open set training process using the labeled dataset comprises performing a first reconstruction of the trajectories in the labeled dataset for those trajectories in the labeled dataset that are in a correct class, and performing a second reconstruction of the trajectories in the labeled dataset for those trajectories in the labeled dataset that are in an incorrect class,

wherein after the open set trajectory classifier is trained, the open set trajectory classifier is operable to receive an unknown trajectory and classify the unknown trajectory with the predicted trajectory classification.

11 . The non-transitory storage medium as recited in claim 10 , wherein the predicted trajectory classification is provided by the open set trajectory classifier to the event predictor.

12 . The non-transitory storage medium as recited in claim 10 , wherein the unknown trajectory comprises information about a movement and/or position of a mobile edge device, in which the machine learning model is deployed, in a physical operating domain.

13 . The non-transitory storage medium as recited in claim 10 , wherein the open set training process for the open set trajectory classifier accounts for a class imbalance across trajectory classes that are known to the open set trajectory classifier.

14 . The non-transitory storage medium as recited in claim 10 , wherein the machine learning model is operable to detect abnormal events occurring during operation of a mobile edge device in which the machine learning model is deployed.

15 . The non-transitory storage medium as recited in claim 10 , wherein the event predictor comprises an auto-encoder that receives, as inputs, the predicted trajectory classification and the trajectory feature.

16 . The non-transitory storage medium as recited in claim 10 , wherein the open set trajectory classifier comprises an auto-encoder that receives, as input, trajectory features of the unknown trajectory.

17 . The non-transitory storage medium as recited in claim 10 , wherein training of the machine learning model is automatically triggered as a result of a monitoring process that tracks performance of the machine learning model.

18 . The non-transitory storage medium as recited in claim 10 , wherein the operations further comprise deploying the machine learning model, after the machine learning model is trained, to a group of mobile edge devices.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 14, 2022
From: FERREIRA, PAULO ABELHA; GOTTIN, VINICIUS MICHEL; SILVA, PABLO NASCIMENTO DA
To: DELL PRODUCTS L.P.
Reel/Frame 060509/0128 →
Continuity (1)
Related Publication 20240020571A1 · Jan 18, 2024
References Cited (105)
US 6393362B1 · Burns · 2002 [cited by applicant]
US 8169335B2 · Frederick et al. · 2012 [cited by applicant]
US 9547969B2 · Beggs et al. · 2017 [cited by applicant]
US 9576480B1 · Shoshan · 2017 [cited by applicant]
US 9671777B1 · Aichele · 2017 [cited by examiner]
US 9674498B1 · Mukherjee et al. · 2017 [cited by applicant]
US 9764472B1 · Ebrahimi et al. · 2017 [cited by applicant]
US 10525882B2 · Bernstein et al. · 2020 [cited by applicant]
US 10661795B1 · Li et al. · 2020 [cited by applicant]
US 10885775B2 · Herson et al. · 2021 [cited by applicant]
US 11086328B2 · Keivan et al. · 2021 [cited by applicant]
US 11165954B1 · Beach et al. · 2021 [cited by applicant]
US 11232294B1 · Banerjee et al. · 2022 [cited by applicant]
US 11577741B1 · Reschka et al. · 2023 [cited by applicant]
US 12012318B2 · Gottin et al. · 2024 [cited by applicant]
US 12093815B2 · Otsuki · 2024 [cited by examiner]
US 12094451B1 · Zhang et al. · 2024 [cited by applicant]
US 20090109049A1 · Frederick et al. · 2009 [cited by applicant]
US 20100208064A1 · Liu et al. · 2010 [cited by applicant]
US 20100228427A1 · Anderson et al. · 2010 [cited by applicant]
US 20110010023A1 · Kunzig et al. · 2011 [cited by applicant]
US 20110093134A1 · Emanuel et al. · 2011 [cited by applicant]
US 20110279261A1 · Gauger et al. · 2011 [cited by applicant]
US 20120025964A1 · Beggs et al. · 2012 [cited by applicant]
US 20120083947A1 · Anderson et al. · 2012 [cited by applicant]
US 20140191869A1 · Frederick · 2014 [cited by applicant]
US 20140324336A1 · Prokhorov et al. · 2014 [cited by applicant]
US 20150145661A1 · Beggs et al. · 2015 [cited by applicant]
US 20160129592A1 · Saboo et al. · 2016 [cited by applicant]
US 20160180713A1 · Bernhardt et al. · 2016 [cited by applicant]
US 20170297569A1 · Nilsson et al. · 2017 [cited by applicant]
US 20170337813A1 · Taylor · 2017 [cited by examiner]
US 20170337820A1 · Glatfelter et al. · 2017 [cited by applicant]
US 20170356596A1 · Frederick · 2017 [cited by applicant]
US 20180203727A1 · Jarvis et al. · 2018 [cited by applicant]
US 20190043351A1 · Yang et al. · 2019 [cited by applicant]
US 20190138895A1 · Jin et al. · 2019 [cited by applicant]
US 20190179329A1 · Keivan et al. · 2019 [cited by applicant]
US 20190268722A1 · Elswick et al. · 2019 [cited by applicant]
US 20190317219A1 · Smith · 2019 [cited by examiner]
US 20200225673A1 · Ebrahimi et al. · 2020 [cited by applicant]
US 20200257310A1 · Du et al. · 2020 [cited by applicant]
US 20200311559A1 · Chattopadhyay et al. · 2020 [cited by applicant]
US 20200334837A1 · Feigl et al. · 2020 [cited by applicant]
US 20200380406A1 · Dickens et al. · 2020 [cited by applicant]
US 20200410281A1 · Goel · 2020 [cited by examiner]
US 20210089762A1 · Rahimi · 2021 [cited by examiner]
US 20210192810A1 · Paysan · 2021 [cited by examiner]
US 20210273858A1 · Radovanovic et al. · 2021 [cited by applicant]
US 20210295154A1 · Groh · 2021 [cited by examiner]
US 20220066456A1 · Ebrahimi et al. · 2022 [cited by applicant]
US 20220089372A1 · Phillips et al. · 2022 [cited by applicant]
US 20220138943A1 · Zhu · 2022 [cited by examiner]
US 20220231919A1 · Pandey et al. · 2022 [cited by applicant]
US 20220253647A1 · Perkins · 2022 [cited by applicant]
US 20220330896A1 · Min et al. · 2022 [cited by applicant]
US 20220394531A1 · Jeong et al. · 2022 [cited by applicant]
US 20220409991A1 · Ohashi et al. · 2022 [cited by applicant]
US 20230030039A1 · Drapeko et al. · 2023 [cited by applicant]
US 20230068523A1 · Santra · 2023 [cited by examiner]
US 20230159027A1 · Pronovost · 2023 [cited by applicant]
US 20230177951A1 · Mubarek · 2023 [cited by examiner]
US 20230221942A1 · Zhang et al. · 2023 [cited by applicant]
US 20230305564A1 · Gottin et al. · 2023 [cited by applicant]
US 20230368129A1 · Gottin et al. · 2023 [cited by applicant]
US 20230409881A1 · Heitzmann · 2023 [cited by examiner]
US 20240003235A1 · Bruns et al. · 2024 [cited by applicant]
US 20240020571A1 · Ferreira · 2024 [cited by examiner]
US 20240190473A1 · Akella · 2024 [cited by examiner]
US 20240214314A1 · Zhao et al. · 2024 [cited by applicant]
US 20240278797A1 · Agrawal et al. · 2024 [cited by applicant]
US 20240311731A1 · Caron et al. · 2024 [cited by applicant]
US 20250077942A1 · Chaabane · 2025 [cited by examiner]
CN 106651884B · 2019 [cited by applicant]
CN 112613586A · 2021 [cited by applicant]
CN 112945227A · 2021 [cited by applicant]
CN 114460537A · 2022 [cited by applicant]
CN 114527676A · 2022 [cited by applicant]
DE 102021200452A1 · 2022 [cited by applicant]
WO 2011092229A1 · 2011 [cited by applicant]
WO 2022156947A1 · 2022 [cited by applicant]
WO 2023287176A1 · 2023 [cited by applicant]
U.S. Appl. No. 17/663,423, titled, “Unsupervised Learning For Real-Time Detection of Far Edge Mobile Device Trajectories”, filed May 14, 2022. [cited by applicant]
U.S. Appl. No. 17/585,055, titled, Edge-Enabled Trajectory Map Generation, filed Jan. 26, 2022. [cited by applicant]
Oza, P., & Patel, V. M. (2019). “C2ae: Class Conditioned Auto-Encoder for Open-set Recognition.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2307-2316). [cited by applicant]
“How Accurate Are RFID Systems?”—RFID Journal; (Accessed Oct. 28, 2022). [cited by applicant]
B. Kim, C. Kang, J. Kim, S. Lee, C. Chung and J. Choi, “Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network,” in 20th International Conference on Intelligent Transportation S… [cited by applicant]
C. Hegde, S. Dash and P. Agarwal, “Vehicle Trajectory Prediction using GAN,” in Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC) , 2020. [cited by applicant]
Gal, Yarin, and Zoubin Ghahramani. “Dropout as a Bayesian approximation: Representing model uncertainty in deep learning.” international conference on machine learning. PMLR, 2016. [cited by applicant]
Inpixon—Bluetooth Location Tracking & Positioning, https://www.inpixon.com/technology/standards/bluetooth-low-energy (Accessed Oct. 28, 2022). [cited by applicant]
Link Labs—“The Complete Active RFID Overview”, Mar. 14, 2018. [cited by applicant]
S. Srikanth, J. Ansari, R. Ram, S. Sharma, J. Murthy and K. Krishna, “Infer: Intermediate representations for future prediction,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2019. [cited by applicant]
Wikipedia, Autoencoder, https://en.wikipedia.org/wiki/Autoencoder, Jan. 10, 2021. [cited by applicant]
B. Alexe, T. Deselaers and V. Ferrari, “Measuring the objectness of image windows,” IEEE transactions on pattern analysis and machine intelligence, vol. 11, No. 34, pp. 2189-2202, 2021. [cited by applicant]
B. McMahan, E. Moore, D. Ramage, S. Hampson and B. y Arcas, “Communication-efficient learning of deep networks from decentralized data.,” In Artificial Intelligence and Statistics, pp. 1273-1282, 2017. [cited by applicant]
C. L. Zitnick and p. Dollár, “Edge boxes: Locating object proposals from edges.,” European conference on computer vision, pp. 391-405, 2014. [cited by applicant]
J. Bernstein, Y. X. Wang, K. Azizzadenesheli and A. Anandkumar, “signSGD: compressed optimisation for non-convex problems,” CoRR, vol. abs/1802.04434, 2021. [cited by applicant]
K. M. Rashid and A. H. Behzadan, “Risk Behavior-Based Trajectory Prediction for Construction Site Safety Monitoring,” Journal of construction engineering and management, 2018. [cited by applicant]
Li, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine, vol. 37, No. 3. pp. 50-60, 2020. [cited by applicant]
P. Tang, X. Wang, A. Wang, Y. Yan, W. Liu, J. Huang and A. Yuille, “Weakly supervised region proposal network and object detection,” Proceedings of the European conference on computer vision (ECCV), 2018. [cited by applicant]
He, Chaoyang, et al. “Federated Learning for Internet of Things” IEEE Internet of Things Magazine 5.1 (2021): 24-29. (Year: 2021). [cited by applicant]
Liu et al., “T-C3D: Temporal Convolutional 3D Network for Real-Time Action Recognition” AAAI, 2018, (Year: 2018). [cited by applicant]
W.-H. Chung, G. Chakraborty, R.-C. Chen and C. Bornand, “Object Dynamics from Video Clips using YOLO Framework,” 2019 IEEE 10th International Conference on Awareness Science and Technology (iCAST), Morioka, Japan, Oct. … [cited by applicant]
Mirus et al., “Detection of abnormal driving situations using distributed representations and unsupervised learning”, Oct. 2020, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine L… [cited by applicant]
Shekhar, “What Are L1 and L2 Loss Functions?”, Aug. 1, 2019, Outcome School, p. 1-4 (Year: 2019). [cited by applicant]