IP Library Granted Patent US 12,525,017
Granted Patent B2
US 12,525,017 · App. 18/141,904 · Granted Jan 13, 2026

Object imaging and detection systems and methods

Inventors: Francois Ruel (Victoriaville, CA); Martin E. Oehlbeck (Cohocton, NY); Deep Rajendrakumar Joshi (Houston, TX); Calvin Stuart Holt (Houston, TX)
G06V20/52G06T7/0004G06T7/20G06T7/70G06V10/25G06V10/764G06V10/82G06V20/41H04N7/188G06T2207/10016G06T2207/20084G06T2207/30108
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,525,017
App. No.
18/141,904
Granted
Jan 13, 2026
Kind
B2
Abstract

A method including selecting image data of a mechanical mud separation machines (“MMSM”) to detect objects in an object flow and other operational conditions at the MMSM. The image data may be processed by a Deep Neural Network to identify objects in the object flow, operational parameters of the MMSM, and environmental conditions. Additional image data may be selected for additional processing based on the results of the analysis.

Claims (47)

1 . A computer-implemented method comprising:

receiving image data from at least one imaging device imaging at least one mechanical mud separation machine (“MMSM”);

selecting, from the image data, at least one Region of Interest (“ROI”);

analyzing, using a Deep Neural Network (“DNN”), the at least one ROI to identify at least one image aspect in the ROI, wherein the image aspect is at least one of an object in an object flow, signal noise, or another physical object;

based on results from the analyzing operation, selecting at least one additional ROIs from the image data, wherein the at least one image aspect in the at least one ROI is an object in an object flow and the at least one additional ROIs comprises additional image data of the object in the object flow at a falling zone of a first MMSM of the at least one MMSMs; and

analyzing the at least one additional ROIs using the DNN.

2 . The computer implemented method of claim 1 , wherein the selecting operation comprises:

associating the at least one ROI with a first-time frame;

identifying the falling zone of the first MMSM;

determining a second time frame and location within a field of view of the at least one imaging device at which the object will likely be present at a falling zone of an MMSM;

selecting additional image data corresponding to the second time frame and location to form one additional ROI.

3 . The computer implemented method of claim 2 , wherein identifying a falling zone of the first MMSM comprises using a DNN.

4 . The computer implemented method of claim 2 , wherein selecting additional image data further comprises:

determining a size of the additional ROI to capture the entire object.

5 . The computer implemented method of claim 4 , wherein the size of the additional ROI is 224×224 pixels.

6 . The method of claim 2 , wherein the second time frame occurs earlier in time than the first time frame.

7 . A system comprising at least one computer processor in electronic communication with at least one computer readable storage device storing instructions that, when executed, performs a method, the method comprising:

receiving image data from at least one imaging device imaging at least one mechanical mud separation machine (“MMSM”);

selecting, from the image data, at least one Region of Interest (“ROI”);

analyzing, using a Deep Neural Network (“DNN”), the at least one ROI to identify at least one image aspect in the ROI, wherein the image aspect is at least one of an object in an object flow, signal noise, or another physical object;

based on results from the analyzing operation, selecting at least one additional ROIs from the image data, wherein the at least one image aspect in the at least one ROI is an object in an object flow and the at least one additional ROIs comprises additional image data of the object in the object flow at a falling zone of a first MMSM of the at least one MMSMs; and

analyzing the at least one additional ROIs using the DNN.

8 . The system of claim 7 , wherein the selecting operation comprises:

associating the at least one ROI with a first-time frame;

identifying the falling zone of the first MMSM;

determining a second time frame and location within a field of view of the at least one imaging device at which the object will likely be present at a falling zone of an MMSM;

selecting additional image data corresponding to the second time frame and location to form one additional ROI.

9 . The system of claim 8 , wherein identifying a falling zone of the first MMSM comprises using a DNN.

10 . The system of claim 9 , wherein selecting additional image data further comprises:

determining a size of the additional ROI to capture the entire object.

11 . The system of claim 10 , wherein the size of the additional ROI is 224×224 pixels.

12 . The system of claim 8 , wherein the second time frame occurs earlier in time than the first time frame.

13 . A non-transitory computer readable medium storage device storing instructions that, when executed, performs a method, the method comprising:

receiving image data from at least one imaging device imaging at least one mechanical mud separation machine (“MMSM”);

selecting, from the image data, at least one Region of Interest (“ROI”);

analyzing, using a Deep Neural Network (“DNN”), the at least one ROI to identify at least one image aspect in the ROI, wherein the image aspect is at least one of an object in an object flow, signal noise, or another physical object;

based on results from the analyzing operation, selecting at least one additional ROIs from the image data, wherein the at least one image aspect in the at least one ROI is an object in an object flow and the at least one additional ROIs comprises additional image data of the object in the object flow at a falling zone of a first MMSM of the at least one MMSMs; and

analyzing the at least one additional ROIs using the DNN.

14 . The non-transitory computer readable medium storage device of claim 13 , wherein the selecting operation comprises:

associating the at least one ROI with a first-time frame;

identifying the falling zone of the first MMSM;

determining a second time frame and location within a field of view of the at least one imaging device at which the object will likely be present at a falling zone of an MMSM;

selecting additional image data corresponding to the second time frame and location to form one additional ROI.

15 . The non-transitory computer readable medium storage device of claim 14 , wherein identifying a falling zone of the first MMSM comprises using a DNN.

16 . The non-transitory computer readable medium storage device of claim 15 , wherein selecting additional image data further comprises:

determining a size of the additional ROI to capture the entire object.

17 . The non-transitory computer readable medium storage device of claim 16 , wherein the size of the additional ROI is 224×224 pixels.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 9, 2023
From: RUEL, FRANCOIS; OEHLBECK, MARTIN E.; JOSHI, DEEP RAJENDRAKUMAR; HOLT, CALVIN STUART
To: DRILLDOCS COMPANY
Reel/Frame 064537/0091 →
Continuity (3)
Continuation 17917782
Provisional Application 63188107 · May 13, 2021
Related Publication 20230298353A1 · Sep 21, 2023
References Cited (73)
US 7933018B2 · Vannuffelen et al. · 2011 [cited by applicant]
US 8818779B2 · Sadlier et al. · 2014 [cited by applicant]
US 9046509B2 · Dvorkin et al. · 2015 [cited by applicant]
US 9651468B2 · Rowe et al. · 2017 [cited by applicant]
US 9670775B2 · Tjhang et al. · 2017 [cited by applicant]
US 9912918B2 · Samuel · 2018 [cited by applicant]
US 9951607B2 · Wessling et al. · 2018 [cited by applicant]
US 10019875B1 · Alghooneh et al. · 2018 [cited by applicant]
US 10151677B2 · Kulkarni et al. · 2018 [cited by applicant]
US 10633941B2 · Teodorescu · 2020 [cited by applicant]
US 10643322B2 · Holton et al. · 2020 [cited by applicant]
US 10683178B2 · Lanoue · 2020 [cited by examiner]
US 10781692B2 · Francois et al. · 2020 [cited by applicant]
US 10796424B2 · Parmeshwar et al. · 2020 [cited by applicant]
US 10927671B1 · Tonner et al. · 2021 [cited by applicant]
US 10954729B2 · Torrione et al. · 2021 [cited by applicant]
US 20060127298A1 · Westerberg · 2006 [cited by examiner]
US 20060162962A1 · Koederitz et al. · 2006 [cited by applicant]
US 20100044034A1 · Bailey · 2010 [cited by applicant]
US 20100193249A1 · Saiz · 2010 [cited by examiner]
US 20110210083A1 · Scott et al. · 2011 [cited by applicant]
US 20110220410A1 · Aldred · 2011 [cited by applicant]
US 20110308784A1 · Ollerenshaw · 2011 [cited by examiner]
US 20140020954A1 · Pelletier et al. · 2014 [cited by applicant]
US 20140214325A1 · Wessling et al. · 2014 [cited by applicant]
US 20140333754A1 · Graves et al. · 2014 [cited by applicant]
US 20150013448A1 · Smith · 2015 [cited by applicant]
US 20150235544A1 · Hernandez et al. · 2015 [cited by applicant]
US 20160090799A1 · Geehan et al. · 2016 [cited by applicant]
US 20160130928A1 · Torrione · 2016 [cited by applicant]
US 20160370274A1 · Rowe et al. · 2016 [cited by applicant]
US 20170044844A1 · Ollerenshaw · 2017 [cited by examiner]
US 20170056928A1 · Torrione · 2017 [cited by applicant]
US 20170056929A1 · Torrione · 2017 [cited by applicant]
US 20170058620A1 · Torrione · 2017 [cited by applicant]
US 20170058621A1 · Bailey · 2017 [cited by applicant]
US 20170314369A1 · Rosano · 2017 [cited by applicant]
US 20180130215A1 · Schulter · 2018 [cited by examiner]
US 20190302310A1 · Fox et al. · 2019 [cited by applicant]
US 20190368287A1 · Shekhar et al. · 2019 [cited by applicant]
US 20190368347A1 · Kulkarni et al. · 2019 [cited by applicant]
US 20200004225A1 · Buller · 2020 [cited by examiner]
US 20200126386A1 · Michakopulos et al. · 2020 [cited by applicant]
US 20200157929A1 · Torrione · 2020 [cited by applicant]
US 20200284145A1 · Elgamal · 2020 [cited by applicant]
US 20200299145A1 · Ning · 2020 [cited by examiner]
US 20200326322A1 · Farrukh Hamza · 2020 [cited by applicant]
US 20200332654A1 · Rowe et al. · 2020 [cited by applicant]
US 20200363289A1 · Shekhar et al. · 2020 [cited by applicant]
US 20210008467A1 · Wang · 2021 [cited by examiner]
US 20210019875A1 · Parmeshwar et al. · 2021 [cited by applicant]
US 20210027144A1 · Madasu et al. · 2021 [cited by applicant]
US 20210090239A1 · Pattnaik et al. · 2021 [cited by applicant]
US 20210172251A1 · Leung · 2021 [cited by examiner]
US 20210189813A1 · Torrione · 2021 [cited by applicant]
US 20220017393A1 · Li · 2022 [cited by examiner]
US 20230298353A1 · Ruel · 2023 [cited by examiner]
WO 2021054840 · 2021 [cited by applicant]
International Search Report and Written Opinion for PCT/US2022/034092, mailed Nov. 4, 2022, 15 pgs. [cited by applicant]
Astrebov, “Conceptual Design of Drilling Cuttings Analysis System Based on Machine Learning Techniques”, Diss. University of Leoben, Jul. 2020, 44 pgs. [cited by applicant]
Kunsheng et al., “Classifying Cutting, Volume at Shale Shakers in Real-Time via Video Streaming Using Deep- Learning Techniques”, SPE Drilling & Completion, 2020, 12 pgs. [cited by applicant]
Marana, et al., “An Intelligent System for Petroleum Well Drilling Cutting Analysis”, International Conference on Adaptive and Intelligent Systems, 2009, 6 pgs. [cited by applicant]
Guilherme, et al., “Fast Petroleum Well Drilling Monitoring Through Optimum-Path Forest”, Journal of Next Generation Information Technology, vol. 1, No. 1, May 2010, 9 pgs. [cited by applicant]
Marana, et al., “An Intelligent System to Detect Drilling Problems Through Drilled Cuttings Return Analysis”, IADC/SPE Drilling Conference and Exhibition, Feb. 2010, 8 pgs. [cited by applicant]
Han, et al., “Real-time 3d Computer Vision Shape Analysis of Cuttings and Cavings”, SPE Annual Technical Conference and Exhibition 2018: SPE-191634-MS, Sep. 24, 2018, 16 pgs. [cited by applicant]
Covar Applied Technologies, “Video Processing for Improved Drilling”, DEA, Mar. 20, 2013, 25 pgs. [cited by applicant]
Schuurmans, “Classification of Images in Real Time with Image Analytics”, Halliburton Landmark Innovation Forum & Expo 2019, Aug. 28, 2019, 23 pgs. [cited by applicant]
Han, “Automated Computer Vision System for Real-Time Drilling Cuttings Monitoring”, Thesis, The Graduate School of The University of Texas at Austin, Aug. 2016, 112 pgs. [cited by applicant]
Ripperger, et al., “Safer and Faster Drilling Through AI Driven Cuttings Monitoring—An Operators Approach”, ADIPEC, Abu Dhabi, UAE, SPE-211759-MS, Oct. 31, 2022 - Nov. 3, 2022, 18 pgs. [cited by applicant]
PCT International Search report and Written Opinion in International Application PCT/US2022/029230, mailed Sep. 27, 2022, 9 pgs. [cited by applicant]
Non-Final Office Action, U.S. Appl. No. 17/190,397, mailed Feb. 23, 2023, 16 pgs. [cited by applicant]
Extended European Search Report, Europe, Application No. 22808420.8-1207, mailed Mar. 10, 2025, 15 pgs. [cited by applicant]
Substantive Examination Report, Saudi Arabia, Application No. 523450988, mailed Sep. 19, 2024, 10 pgs. [cited by applicant]