Granted Patent B2

US 12,412,262 · App. 18/581,590 · Granted Sep 9, 2025

Inventors: Ali Osman (Houston, TX); Akhil Dharamprakash Soni (Pune, IN); Pragati Kumar Singh (Pune, IN); Swayan Jeet Mishra (Sundergarh, IN); Anshuman Chakravarty (Pune, IN); Abhishek Dubey (Pune, IN); Aakash Aakash (Pune, IN); Anurag Chandrakar (Pune, IN); Sanjyot Samarth Zade (Pune, IN); Abhishek Kumar (Pune, IN); Hardik Tejani (Pune, IN)

Assignee: Schlumberger Technology Corporation

G06T7/0004G06T7/215G06V10/30G06V10/70G06V20/52G06V40/10G06T2207/20056G06T2207/30232G06T2207/30242

Abstract

Claims (118)

1. A computing system for detecting safety non-compliance and monitoring performance in a workshop, the computer system comprising:

one or more processors; and

a memory system comprising one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations, the operations comprising:

receiving data, wherein the data comprises image data, video data, or both, and wherein the data is received from the workshop;

executing artificial intelligence processes to perform mechanical lifting non-compliance detection, generic load lifting non-compliance detection, or personnel proximity to moving vehicle detection by processing the data to produce processed image data;

filtering the processed image data using a plurality of filters to produce filtered data, wherein the plurality of filters:

removes false positive examples based on pre-defined object metrics, aspect ratio, intersection over union, and dice coefficient, and

includes slicing aided hyper-interfacing enabling objects and confluence wherein the slicing aided hyper-interfacing enables objects within a pre-selected distance from a collection device to be detected, and the confluence enables overlapping object types to be detected;

aggregating the filtered data at various levels in the workshop;

executing data correction processes on the aggregated filtered data over a plurality of time stamps producing corrected data;

determining an estimate of the safety non-compliance from the corrected data;

providing alerts associated with the estimate;

determining the performance for workflows associated with the workshop based on the corrected data, the estimate, or both;

displaying the performance and personnel during the safety non-compliance; and

generating an instruction to implement behavior to correct the safety non-compliance.

2. The computing system as in claim 1 wherein the data comprise:

images from use cases, wherein

the use cases include personal protective equipment detection, tracking of personnel, workshop utilization, technician count, asset identification, mechanical lifting detection, tool detection, red zone detection, tagline detection, personnel anonymization, generic load detection, proximity to moving vehicle detection, and performance indicator determination.

3. The computing system as in claim 1 wherein the artificial intelligence processes comprise:

object tracking, scene subtraction, pose estimation, and key-point estimation, wherein

the scene subtraction includes detection of moving objects and segmentation of objects, wherein

the moving objects include tools, the personnel, slings, vehicles, and taglines, and

the segmentation includes tool and hand segmentation.

4. The computing system as in claim 1 wherein

the artificial intelligence processes execute on edge devices or devices remote to the workshop or both.

5. The computing system as in claim 1 wherein

the artificial intelligence processes include models, wherein

the models include a light model and a heavy model, wherein

the light model detects events, wherein

the events comprise workshop-related events, and

the heavy model processes the data when the light model detects the workshop-related events.

6. The computing system as in claim 1 wherein

the filtered data are aggregated across workshop bays, floors, buildings, and locations, and time periods.

7. The computing system as in claim 3 wherein

the aggregated filtered data are mapped from a 2D mapping projecting camera viewpoint onto a 2D map using homographic transformation, wherein

the aggregated filtered data preserve a consistent person identification during the personnel tracking to accurately determine a count of the personnel,

the homographic transformation protects the objects from over/under count, and

massaging the aggregated filtered data, wherein massaging comprises executing operations to update a fidelity and an accuracy of the aggregated filtered data.

8. The computing system as in claim 1 wherein:

the performance is computed at a plurality of the time stamps,

the performance includes a count of the personnel in a location, a personal protective equipment usage non-compliance percentage, a vehicle encounter non-compliance percentage, a mechanical lifting non-compliance percentage, an amount of a wrench time, an effective capacity, an amount of technician hours, an amount of data entry time, an amount of bench utilization, and an amount of tool on bench time,

a state of the performance is changed based on the corrected data that are gathered at different of the time stamps, and

a display of the personnel includes anonymizing the personnel in the display, wherein the anonymizing includes blurring the display.

9. A computing system for detecting safety non-compliance and monitoring performance in a workshop, the computer system comprising:

one or more processors; and

a memory system comprising one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations, the operations comprising:

receiving data, wherein the data comprises image data, video data, or both, and wherein the data is received from the workshop;

executing artificial intelligence processes to perform mechanical lifting non-compliance detection, generic load lifting non-compliance detection, or personnel proximity to moving vehicle detection by processing the data to produce processed image data;

filtering the processed image data using a plurality of filters to produce filtered data;

aggregating the filtered data at various levels in the workshop;

removing random noisy data from the aggregated filtered data producing intermediate data, wherein the random noisy data include discrepancies in account a count of the personnel;

executing data correction processes on the aggregated filtered data over a plurality of time stamps producing corrected data;

filtering the corrected data using a low pass filter and/or a time series filter, wherein

the low pass filter includes a moving average filter, wherein

the moving average filter removes jitters in the intermediate data, wherein

 the jitters arise from the count of the personnel in a bay or

the personnel passing other personnel who are handling the tools;

performing a Fast Fourier Transform (FFT) using FFT convolution on the intermediate data to produce time series data;

detecting anomalies/outliers in the time series data by identifying trends, seasonality, and noise in the time series data to produce the corrected data;

determining an estimate of the safety non-compliance from the corrected data;

providing alerts associated with the estimate;

determining the performance for workflows associated with the workshop based on the corrected data, the estimate, or both;

displaying the performance and personnel during the safety non-compliance; and

generating an instruction to implement behavior to correct the safety non-compliance.

10. The computing system as in claim 9 further comprising:

counting the personnel from frame to frame of the intermediate data, wherein

the personnel are counted as they pass a location in the workshop, and

the personnel are counted as they are working on assets.

11. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations for detecting safety non-compliance and monitoring performance in a workshop, the operations comprising:

receiving data, wherein the data comprises image data, video data, or both, and wherein the data is received from the workshop, wherein

the data include images from use cases, wherein

the use cases include personal protective equipment non-compliance detection, tracking of personnel, workshop utilization, technician count, asset identification, mechanical lifting non-compliance detection, tool non-compliance detection, red zone non-compliance detection, tagline non-compliance detection, personnel anonymization, generic load non-compliance detection, proximity to moving vehicle detection, and performance indicator determination;

executing artificial intelligence processes to perform mechanical lifting non-compliance detection, generic load lifting non-compliance detection, or personnel proximity to moving vehicle detection by processing the data to produce processed image data, wherein

the artificial intelligence processes include object tracking, scene subtraction, pose estimation, and key-point estimation, wherein

the scene subtraction includes detection of moving objects and segmentation of objects, wherein

the moving objects include tools, the personnel, slings, vehicles, and taglines, and

the segmentation includes tool and hand segmentation;

the artificial intelligence processes execute on edge devices or devices remote to the workshop or both, and

the artificial intelligence processes include models, wherein

the models include a light model and a heavy model, wherein

the light model detects events, wherein

 the events comprise workshop-related events, and

 the heavy model processes the data when the light model detects the workshop-related events;

filtering the processed image data using a plurality of filters to produce filtered data, wherein

the plurality of filters removes false positive examples based on pre-defined object metrics, aspect ratio, intersection over union, and dice coefficient, and

the plurality of filters includes slicing aided hyper-inferencing and confluence, wherein

the slicing aided hyper-inferencing enables the objects within a pre-selected distance from a collection device to be detected, and

the plurality of filters includes confluence, wherein

the confluence enables overlapping object types to be detected;

aggregating the filtered data at various levels in the workshop, wherein

the filtered data are aggregated across workshop bays, floors, buildings, and locations, and time periods,

the aggregated data are mapped from a 2D mapping projecting camera viewpoint onto a 2D map using homographic transformation, wherein

the aggregated data preserve a consistent person identification during the personnel tracking to accurately determine a count of the personnel,

the homographic transformation protects the object from over/under count, and

massaging the aggregated data, wherein massaging comprises executing operations to update a fidelity and an accuracy of the aggregated data;

executing data correction processes on the aggregated filtered data over a plurality of time stamps producing corrected data, including:

removing random noisy data from the aggregated filtered data producing intermediate data, wherein

the random noisy data include discrepancies in the count of the personnel;

counting the personnel from frame to frame of the intermediate data, wherein

the personnel are counted as they pass a location in the workshop, and

the personnel are counted as they are working on assets;

filtering the corrected data using a low pass filter and/or a time series filter, wherein

the low pass filter includes a moving average filter, wherein

the moving average filter removes jitters in the intermediate data, wherein

the jitters arise from the count of the personnel in a bay or the personnel passing other personnel who are handling the tools;

performing a Fast Fourier Transform (FFT) using FFT convolution on the intermediate data to produce time series data; and

detecting anomalies/outliers in the time series data by identifying trends, seasonality, and noise in the time series data to produce the corrected data;

determining an estimate of the safety non-compliance in the use cases from the corrected data, wherein

the estimate is based on pre-selected thresholds for the use cases;

providing alerts associated with the estimate;

determining the performance for workflows associated with the workshop based on the corrected data, the estimate, or both;

displaying the performance and the personnel during the safety non-compliance, wherein

the performance is computed at a plurality of the time stamps,

the performance includes the count of the personnel in a location, a personal protective equipment usage non-compliance percentage, a vehicle encounter non-compliance percentage, a mechanical lifting non-compliance percentage, an amount of a wrench time, an effective capacity, an amount of technician hours, an amount of data entry time, an amount of bench utilization, and an amount of tool on bench time,

a state of the performance is changed based on the corrected data that are gathered at different of the time stamps, and

a display of the personnel includes anonymizing the personnel in the display, wherein

the anonymizing includes blurring the display; and

generating an instruction to implement behavior to correct the safety non-compliance.

Assignments (1)

Continuity (2)

References Cited (14)