IP Library › Granted Patent US 12,499,711
Granted Patent B2
US 12,499,711 · App. 18/276,633 · Granted Dec 16, 2025

Identification of workers using personal protective equipment and worker movement characteristic analysis

Inventors: Marie D. Manner (St. Paul, MN); Kiran S. Kanukurthy (Woodbury, MN)
Assignee: 3M Innovative Properties Company
G06V40/25G06Q10/0639G06V20/41G06V20/52G06V40/10G06V40/28
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,499,711
App. No.
18/276,633
Granted
Dec 16, 2025
Kind
B2
Abstract

Aspects of the present disclosure relate to a method of identifying a worker, a system that performs the method, or non-transitory computer-readable storage medium that performs the method. The method can include acquiring, by a computer having one or more processors, a video of at least part of a worker from a camera system. The method also includes identifying, with the computer, a first worker profile based on personal protective equipment of the worker in the video. The method includes identifying, with the computer, a second worker profile based on a worker movement characteristic of the worker in the video. The method also includes determining, with the computer, an identity of the worker within the video based on the correspondence between the first worker profile and the second worker profile. The method also includes performing at least one action in response to determining the identity of the worker.

Claims (55)

1 . A computer-implemented method of identifying a worker, comprising:

acquiring, by a computer having one or more processors, a video of at least part of a worker from a camera system;

identifying, with the computer, a first worker profile based on personal protective equipment of the worker in the video, wherein the identifying the first worker profile comprises:

identifying personal protective equipment worn by the worker in the video;

associating identified personal protective equipment with a first transient worker profile;

comparing the identified personal protective equipment in the first transient worker profile with a plurality of personal protective equipment records from a plurality of stored worker profiles; and

identifying the first worker profile at least partially based on a correspondence of the identified personal protective equipment in the first transient worker profile with a personal protective equipment record associated with the first worker profile from the plurality of stored worker profiles;

adding the first transient worker profile to the plurality of stored worker profiles based on a weak correspondence of the identified personal protective equipment with any of the plurality of personal protective equipment records;

identifying, with the computer, a second worker profile based on a worker movement characteristic of the worker in the video;

determining, with the computer, an identity of the worker within the video based on a correspondence between the first worker profile and the second worker profile; and

performing at least one action in response to determining the identity of the worker.

2 . The method of claim 1 , further comprising:

determining whether the identified personal protective equipment matches a pre-determined personal protective equipment for the first worker profile;

performing at least one action in response to the pre-determined personal protective equipment matching all identified personal protective equipment for the first worker profile.

3 . The method of claim 1 , wherein the identifying the second worker profile comprises:

determining a worker movement characteristic of the worker in the video;

associating the worker movement characteristic with a second transient worker profile;

determining a worker movement characteristic metric for the worker movement characteristic of the worker;

comparing the worker movement characteristic metric in the second transient worker profile with a plurality of worker movement characteristic metric records from the plurality of stored worker profiles.

4 . The method of claim 3 , further comprising:

identifying the second worker profile at least partially based on a correspondence of the worker movement characteristic metric from the second transient worker profile with a worker movement characteristic metric record for a second worker profile from the plurality of stored worker profiles.

5 . The method of claim 4 , further comprising:

adding second transient worker profile to the plurality of stored worker profiles based on a weak correspondence of the worker movement characteristic metric with any of the pluralities of worker movement characteristic metric records.

6 . The method of claim 1 , wherein the worker movement characteristic is an arm movement pattern of the worker, the activity is determined use arm movements of the worker,

further comprising:

determining the arm movement pattern of the worker in the video, wherein the arm movement pattern is associated with the activity;

associating the arm movement pattern with the second transient worker profile;

determining an identity of the worker within the video based on the worker movement characteristic, the identified personal protective equipment of the worker, and the arm movement pattern of the worker.

7 . The method of claim 1 , wherein the worker movement characteristic is a gait of the worker.

8 . A non-transitory computer-readable storage medium including instructions that, when processed by a computer, configure the computer to perform the method of claim 1 .

9 . A computing apparatus comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the apparatus to:

acquire, by a computer having one or more processors, a video showing at least part of a worker from a camera system;

identify, with the computer, a first worker profile based on personal protective equipment of the worker in the video, wherein the identifying the first worker profile comprises:

identifying personal protective equipment worn by the worker in the video;

associating identified personal protective equipment with a first transient worker profile;

comparing the identified personal protective equipment in the first transient worker profile with a plurality of personal protective equipment records from a plurality of stored worker profiles; and

identifying the first worker profile at least partially based on a correspondence of the identified personal protective equipment in the first transient worker profile with a personal protective equipment record associated with a first worker profile from the plurality of stored worker profiles;

add the first transient worker profile to the plurality of stored worker profiles based on a weak correspondence of the identified personal protective equipment with any of the plurality of personal protective equipment records;

identify, with the computer, a second worker profile based on a worker movement characteristic of the worker in the video;

determine, with the computer, an identity of the worker within the video based on the correspondence between the first worker profile and the second worker profile; and

perform at least one action in response to determining the identity of the worker.

10 . The computing apparatus of claim 9 , wherein the personal protective equipment record is based on pre-determined personal protective equipment for the worker.

11 . The computing apparatus of claim 9 , wherein the instructions further configure the apparatus to:

determine whether the identified personal protective equipment matches a pre-determined personal protective equipment for the first worker profile;

perform at least one action in response to the pre-determined personal protective equipment matching all identified personal protective equipment for the first worker profile.

12 . The computing apparatus of claim 9 , wherein the identifying the second worker profile comprises:

determining a worker movement characteristic of the worker in the video;

associating the worker movement characteristic with a second transient worker profile;

determining a worker movement characteristic metric for the worker movement characteristic of the worker;

comparing the worker movement characteristic metric in the second transient worker profile with a plurality of worker movement characteristic metric records from the plurality of stored worker profiles.

13 . The computing apparatus of claim 12 , wherein the instructions further configure the apparatus to:

identify the second worker profile at least partially based on a correspondence of the worker movement characteristic metric from the second transient worker profile with a worker movement characteristic metric record for a second worker profile from the plurality of stored worker profiles.

14 . The computing apparatus of claim 12 , wherein the worker movement characteristic is a gait of the worker.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2023
From: MANNER, MARIE D.; KANUKURTHY, KIRAN S.
To: 3M INNOVATIVE PROPERTIES COMPANY
Reel/Frame 064543/0263 →
Continuity (2)
Provisional Application 63151420 · Feb 19, 2021
Related Publication 20240104960A1 · Mar 28, 2024
References Cited (55)
US 7761310B2 · Rodgers · 2010 [cited by applicant]
US 7987069B2 · Rodgers et al. · 2011 [cited by applicant]
US 8675059B2 · Johnson et al. · 2014 [cited by applicant]
US 9633268B1 · Ma et al. · 2017 [cited by applicant]
US 9934427B2 · Derenne et al. · 2018 [cited by applicant]
US 10147210B1 · Desai et al. · 2018 [cited by applicant]
US 10387720B2 · Johnson et al. · 2019 [cited by applicant]
US 10565439B2 · Wang et al. · 2020 [cited by applicant]
US 20190095601A1 · Chan et al. · 2019 [cited by applicant]
US 20200005079A1 · Bapat et al. · 2020 [cited by applicant]
US 20200279116A1 · Ylitalo et al. · 2020 [cited by applicant]
US 20200410444A1 · Bohannon · 2020 [cited by examiner]
AU 2011101355A4 · 2011 [cited by applicant]
CN 101226597B · 2010 [cited by applicant]
CN 101807245B · 2013 [cited by applicant]
CN 106529499A · 2017 [cited by applicant]
CN 107590452A · 2018 [cited by applicant]
CN 105844746B · 2018 [cited by applicant]
CN 109359580B · 2021 [cited by applicant]
CN 108563939B · 2022 [cited by applicant]
CN 108960072B · 2022 [cited by applicant]
WO 2012077142A1 · 2012 [cited by applicant]
WO 2019160601A1 · 2019 [cited by applicant]
E. Hossain and G. Chetty, “Multimodal feature learning for gait biometric based human identity recognition,” in Neural Information Processing—ICONIP (Lecture Notes in Computer Science), vol. 8227, M. Lee, A. Hirose, Z. … [cited by applicant]
W. Kusakunniran, “Recognizing gaits on spatio-temporal feature domain,” IEEE Trans. Inf. Forensics Security, vol. 9, No. 9, pp. 1416-1423, Sep. 2014. doi: 10.1109/TIFS.2014.2336379. [cited by applicant]
Z. Wu, Y. Huang, L. Wang, X. Wang, and T. Tan, “A comprehensive study on cross-view gait based human identification with deep CNNs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, No. 2, pp. 209-226, Feb. 2016. doi: … [cited by applicant]
C. Yan, B. Zhang, and F. Coenen, “Multi-attributes gait identification by convolutional neural networks,” 2015 8 [cited by applicant]
C. Li, X. Min, S. Sun, W. Lin, and Z. Tang, “DeepGait: A learning deep convolutional representation for view-invariant gait recognition using joint Bayesian,” App. Sci., vol. 7, No. 3, p. 15, 2017. doi: 10.3390/app70302… [cited by applicant]
T. Wolf, M. Babaee, and G. Rigoll, “Multi-view gait recognition using 3D convolutional neural networks,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2016, pp. 4165-4169. doi: 10.1109/ICIP.2016.7533144. [cited by applicant]
F. M. Castro, M. J. M. Jiménez, N. Guil and N. P. D. L. Blanca “Automatic learning of gait signatures for people identification,” in Advances in Computational Intelligence, I. Rojas et al., Eds. IWANN 2017, Part II, Lec… [cited by applicant]
Yeoh et al., Clothing-Invariant Gait Recognition Using Convolutional Neural Network, Authorized licensed use limited to: University of Minnesota Duluth. Downloaded on Dec. 18, 2023 at 14:23:28 UTC from IEEE Eplore. [cited by applicant]
J. Frank, S. Mannor, and D. Precup, “Activity and gait recognition with time-delay embeddings.” in AAAI, 2010. [cited by applicant]
J. Lu, G. Wang, and P. Moulin, “Human identity and gender recognition from gait sequences with arbitrary walking directions,” IEEE T-IFS, vol. 9, No. 1, pp. 51-61, 2014. [cited by applicant]
L. Wang, T. Tan, H. Ning and W. Hu, “Silhouette analysis-based gait recognition for human identification”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, No. 12, pp. 1505-1518, Dec. 2003. [cited by applicant]
J. Han and B. Bhanu, “Individual recognition using gait energy image”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, No. 2, pp. 316-322, Feb. 2006. [cited by applicant]
C. Chen, J. Liang, H. Zhao, H. Hu and J. Tian, “Frame difference energy image for gait recognition with incomplete silhouettes”, Pattern Recognit. Lett., vol. 30, No. 11, pp. 977-984, Aug. 2009. [cited by applicant]
E. Zhang, Y. Zhao and W. Xiong, “Active energy image plus 2DLPP for gait recognition”, Signal Process., vol. 90, No. 7, pp. 2295-2302, Jul. 2010. [cited by applicant]
A. Roy, S. Sural and J. Mukherjee, “Gait recognition using pose kinematics and pose energy image”, Signal Process., vol. 92, No. 3, pp. 780-792, 2012. [cited by applicant]
J. Han and B. Bhanu, “Individual recognition using gait energy image,” IEEE T-PAMI, vol. 28, No. 2, pp. 316-322, 2006. [cited by applicant]
L. Lee and W. E. L. Grimson, “Gait analysis for recognition and classification,” in AFGR, 2002, pp. 155-162. [cited by applicant]
C. Nakajima, M. Pontil, B. Heisele, and T. Poggio, “Full-body person recognition system,” Pattern Recognition, vol. 36, No. 9, pp. 1997-2006, 2003. [cited by applicant]
E. P. Ijjina and C. K. Mohan, “One-shot periodic activity recognition using convolutional neural networks,” in ICMLA, 2014, pp. 388-391. [cited by applicant]
M. Alotaibi and A. Mahmood, “Improved gait recognition based on specialized deep convolutional neural network,” CVIU, vol. 164, pp. 103-110, 2017. [cited by applicant]
N. Takemura, Y. Makihara, D. Muramatsu, T. Echigo, and Y. Yagi, “On input/output architectures for convolutional neural network-based cross-view gait recognition,” IEEE Transactions on Circuits and Systems for Video Tec… [cited by applicant]
W. Yuan and L. Zhang, “Gait classification and identity authentication using CNN,” in Asian Simulation Conference, 2018, pp. 119-128. [cited by applicant]
Y. Zhao and S. Zhou, “Wearable device-based gait recognition using angle embedded gait dynamic images and a convolutional neural network,” Sensors, vol. 17, No. 3, p. 478, 2017. [cited by applicant]
Yeoh, “Clothing-Invariant Gait Recognition Using Convolutional Neural Network,” Downloaded on Dec. 18, 2023 UTC from IEEE Xplore. [cited by applicant]
Batchuluun “Body-movement-based human identification using convolutional neural network”, Expert Systems with Applications, Jul. 2018, vol. 101, pp. 56-77. [cited by applicant]
Chen, “A Vision-Based Approach for Ensuring Proper Use of Personal Protective Equipment (PPE) in Decommissioning of Fukushima Daiichi Nuclear Power Station”, Applied Sciences, Jul. 2020, vol. 10, No. 15, pp. 5129(1-14). [cited by applicant]
Harrou, “An Integrated Vision-Based Approach for Efficient Human Fall Detection in a Home Environment,” IEEE Access, Aug. 2019, vol. 7, pp. 114966-114974. [cited by applicant]
International Search Report for PCT International Application No. PCT/IB2022/051344, mailed on May 17, 2022, 5 pages. [cited by applicant]
Video: “Microsoft CVPR Demo—Time 7:47” Fireside Chat with Nadella-ComputerVisionFoundation Videos, Nov. 2022, URL <https://www.youtube.com/watch?v=6ml8kr1M1Pw>, 2 pages. [cited by applicant]
Wu, “A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, vol. 39, No. 2, pp. 209-226. [cited by applicant]
Yan, “Multi-attributes Gait Identification by Convolutional Neural Networks,” 2015 8th International Congress on Image and Signal Processing (CISP), 2015, p. 642-647. [cited by applicant]
Yuan, “A Convolutional Neural Network Based on TensorFlow for Face Recognition”, 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Mar. 2017, IEEE Xplore, pp. 525-529. [cited by applicant]