IP Library Granted Patent US 12,592,313
Granted Patent B2
US 12,592,313 · App. 18/196,338 · Granted Mar 31, 2026

Analyzing surgical videos to identify a billing coding mismatch

Inventors: Tamir Wolf (Palo Alto, CA); Dotan Asselman (Holon, IL)
Assignee: Theator Inc.
G16H40/20G06Q10/10G06Q30/018G06T7/0012G06V20/41G06V20/44G06V20/46G06V20/52G16H10/60G16H15/00G16H30/40G16H40/67G16H70/20G06F40/20G06Q40/08G06T2207/10016G06V10/70G06V10/774G06V2201/03G06V2201/034
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,592,313
App. No.
18/196,338
Granted
Mar 31, 2026
Kind
B2
Abstract

Disclosed herein are apparatus, system, method, and computer-readable medium aspects for using surgical video analysis for improving compliance with medical guidelines and improving processing of medical bills, medical malpractice claims, and insurance claims. Aspects disclosed herein utilize intracorporeal video footage, image analysis, and notifications to optimize correspondences among medical procedure information, medical guidelines, and medical transaction information.

Claims (74)

1 . A non-transitory computer readable medium containing instructions that, when executed by at least one processor, cause the at least one processor to execute operations to identify a billing coding mismatch, the operations comprising:

receiving a medical reimbursement code associated with a surgical procedure;

receiving surgical video of the surgical procedure;

performing image analysis on the surgical video to determine whether a match exists between the medical reimbursement code and the surgical video, the image analysis further comprising:

analyzing the surgical video to detect an adverse event in the surgical procedure,

calculating a result value by executing a convolution of at least part of the surgical video, and

determining whether the match exists based on the detected adverse event and the result value; and

when the match is determined not to exist, outputting an indicator of a lack of support in the surgical video for the medical reimbursement code.

2 . The non-transitory computer readable medium of claim 1 , wherein the performing the image analysis comprises analyzing the surgical video to determine a level of complexity of the surgical procedure, and determining whether an alleged reimbursable event associated with the medical reimbursement code took place in the surgical procedure based on the determined level of complexity.

3 . The non-transitory computer readable medium of claim 1 , wherein the performing the image analysis comprises analyzing the surgical video to determine a level of guideline compliance during the surgical procedure, and determining whether an alleged reimbursable event associated with the medical reimbursement code took place in the surgical procedure based on the determined level of guideline compliance.

4 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise accessing a medical record of the patient, and determining whether an alleged reimbursable event associated with the medical reimbursement code took place in the surgical procedure based on the medical record of the patient.

5 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise accessing an audio recording captured during the surgical procedure, and determining whether an alleged reimbursable event associated with the medical reimbursement code took place in the surgical procedure based on the audio recording.

6 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

analyzing the surgical video to detect a surgical tool; and

basing the determination of whether the match exists between the medical reimbursement code and the surgical video on the detected surgical tool.

7 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

analyzing the surgical video to detect an anatomical structure;

analyzing the surgical video to determine a condition of the anatomical structure; and

basing the determination of whether the match exists between the medical reimbursement code and the surgical video on the condition of the anatomical structure.

8 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

analyzing the surgical video to detect an interaction between a surgical tool and an anatomical structure; and

basing the determination of whether the match exists between the medical reimbursement code and the surgical video on the detected interaction between the surgical tool and the anatomical structure.

9 . The non-transitory computer readable medium of claim 8 , wherein the operations further comprise:

analyzing the surgical video to determine a characteristic of the interaction between the surgical tool and the anatomical structure; and

basing the determination of whether the match exists between the medical reimbursement code and the surgical video on the characteristic of the interaction between the surgical tool and the anatomical structure.

10 . The non-transitory computer readable medium of claim 9 , wherein the characteristic is a duration of the interaction.

11 . The non-transitory computer readable medium of claim 9 , wherein the characteristic is a type of the interaction.

12 . The non-transitory computer readable medium of claim 9 , wherein the characteristic is a state of the surgical tool during the interaction.

13 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

calculating a convolution of at least part of the surgical video to obtain a result value; and

basing the determination of whether the match exists between the medical reimbursement code and the surgical video on the result value.

14 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise using a machine learning model to analyze the surgical video to determine whether the match exists between the medical reimbursement code and the surgical video.

15 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

analyzing the surgical video to classify a mismatch between the medical reimbursement code and the surgical video; and

basing the indicator on a classification of the mismatch between the medical reimbursement code and the surgical video.

16 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

analyzing the surgical video to determine an alternative medical reimbursement code; and

outputting an indication of the alternative medical reimbursement code.

17 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

analyzing the surgical video to detect in the surgical procedure a first surgical event and a second surgical event; and

further basing the determination of whether the match exists between the medical reimbursement code and the surgical video on whether the first surgical event precedes the second surgical event in the surgical procedure.

18 . The non-transitory computer readable medium of claim 1 , wherein the operations further comprise:

training a machine learning model using training examples to analyze the surgical video, the training to determine whether particular actions occurred.

19 . A method for performing intracorporeal video analysis operations for identifying a billing coding mismatch, comprising:

receiving a medical reimbursement code associated with a surgical procedure;

receiving surgical video of the surgical procedure;

performing image analysis on the surgical video to determine whether a match exists between the medical reimbursement code and the surgical video, the image analysis further comprising:

analyzing the surgical video to detect an adverse event in the surgical procedure,

calculating a result value by executing a convolution of at least part of the surgical video, and

determining whether the match exists based on the detected adverse event and the result value; and

when the match is determined not to exist, outputting an indicator of a lack of support in the surgical video for the medical reimbursement code.

20 . A system for identifying a billing coding mismatch, the system comprising a processor configured to perform steps comprising:

receiving a medical reimbursement code associated with a surgical procedure;

receiving surgical video of the surgical procedure;

performing image analysis on the surgical video to determine whether a match exists between the medical reimbursement code and the surgical video, the image analysis further comprising:

analyzing the surgical video to detect an adverse event in the surgical procedure,

calculating a result value by executing a convolution of at least part of the surgical video, and

determining whether the match exists based on the detected adverse event and the result value; and

when the match is determined not to exist, outputting an indicator of a lack of support in the surgical video for the medical reimbursement code.

21 . A non-transitory computer readable medium containing instructions that, when executed by at least one processor, cause the at least one processor to execute operations to identify a billing coding mismatch and an alleged reimbursable event, the operations comprising:

receiving a medical reimbursement code associated with a surgical procedure;

receiving surgical video of the surgical procedure;

performing image analysis on the surgical video to determine whether a match exists between the medical reimbursement code and the surgical video, the image analysis further comprising:

calculating a result value by executing a convolution of at least part of the surgical video;

when the match is determined not to exist, outputting an indicator of a lack of support in the surgical video for the medical reimbursement code, the match determined not to exist based on the result value;

accessing a medical record of a patient; and

determining whether the alleged reimbursable event associated with the reimbursement code took place in the surgical procedure based on the medical record of the patient.

22 . A non-transitory computer readable medium containing instructions that, when executed by at least one processor, cause the at least one processor to execute operations to identify a billing coding mismatch using a surgical video and an audio recording captured during a surgical procedure, the operations comprising:

receiving a medical reimbursement code associated with the surgical procedure;

receiving the surgical video of the surgical procedure;

performing image analysis on the surgical video to determine whether a match exists between the medical reimbursement code and the surgical video, the image analysis further comprising:

calculating a result value by executing a convolution of at least part of the surgical video;

accessing an audio recording capturing during the surgical procedure; and

when the match is determined not to exist, outputting an indicator of a lack of support in the surgical video for the medical reimbursement code, the matched determined not to exist based on the result value and the audio recording.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 11, 2023
From: WOLF, TAMIR; ASSELMANN, DOTAN
To: THEATOR INC.
Reel/Frame 063617/0874 →
Continuity (5)
Provisional Application 63346987 · May 30, 2022
Provisional Application 63389130 · Jul 14, 2022
Provisional Application 63399698 · Aug 21, 2022
Provisional Application 63411758 · Sep 30, 2022
Related Publication 20230386651A1 · Nov 30, 2023
References Cited (155)
US 5791907A · Ramshaw et al. · 1998 [cited by applicant]
US 8886577B2 · Mangione-Smith · 2014 [cited by applicant]
US 9788907B1 · Alvi et al. · 2017 [cited by applicant]
US 9836654B1 · Alvi et al. · 2017 [cited by applicant]
US 9877633B2 · Zhao et al. · 2018 [cited by applicant]
US 10058393B2 · Bonutti et al. · 2018 [cited by applicant]
US 10387720B2 · Johnson et al. · 2019 [cited by applicant]
US 10646156B1 · Schnorr · 2020 [cited by applicant]
US 10791301B1 · Garcia Kilroy et al. · 2020 [cited by applicant]
US 10803538B2 · Nichols · 2020 [cited by examiner]
US 11495336B2 · Piron · 2022 [cited by examiner]
US 20040062381A1 · Shambaugh et al. · 2004 [cited by applicant]
US 20040078236A1 · Stoodley et al. · 2004 [cited by applicant]
US 20040125121A1 · Pea et al. · 2004 [cited by applicant]
US 20050149361A1 · Saus et al. · 2005 [cited by applicant]
US 20060053249A1 · Yamaki et al. · 2006 [cited by applicant]
US 20060143060A1 · Conry et al. · 2006 [cited by applicant]
US 20060159325A1 · Zeineh et al. · 2006 [cited by applicant]
US 20070156344A1 · Sender et al. · 2007 [cited by applicant]
US 20070238981A1 · Zhu et al. · 2007 [cited by applicant]
US 20080243064A1 · Stahler et al. · 2008 [cited by applicant]
US 20090192823A1 · Hawkins et al. · 2009 [cited by applicant]
US 20090300507A1 · Raghavan et al. · 2009 [cited by applicant]
US 20100001149A1 · Song et al. · 2010 [cited by applicant]
US 20100036676A1 · Safdi et al. · 2010 [cited by applicant]
US 20100097398A1 · Tsurumi · 2010 [cited by applicant]
US 20100134609A1 · Johnson · 2010 [cited by applicant]
US 20110046476A1 · Cinquin et al. · 2011 [cited by applicant]
US 20110090367A1 · White et al. · 2011 [cited by applicant]
US 20110225000A1 · Selim · 2011 [cited by applicant]
US 20110264528A1 · Whale · 2011 [cited by applicant]
US 20110276340A1 · DeBoer et al. · 2011 [cited by applicant]
US 20110276348A1 · Ahn et al. · 2011 [cited by applicant]
US 20110306985A1 · Inoue et al. · 2011 [cited by applicant]
US 20120035963A1 · Qian et al. · 2012 [cited by applicant]
US 20120177256A1 · Keefe et al. · 2012 [cited by applicant]
US 20120179061A1 · Ramanan et al. · 2012 [cited by applicant]
US 20130132117A1 · Barsoum et al. · 2013 [cited by applicant]
US 20130297343A1 · Hirose et al. · 2013 [cited by applicant]
US 20140031659A1 · Zhao et al. · 2014 [cited by applicant]
US 20140081659A1 · Nawana et al. · 2014 [cited by applicant]
US 20140101550A1 · Zises · 2014 [cited by applicant]
US 20140220527A1 · Li et al. · 2014 [cited by applicant]
US 20140226888A1 · Skidmore · 2014 [cited by applicant]
US 20140270711A1 · Maser et al. · 2014 [cited by applicant]
US 20140276940A1 · Seo · 2014 [cited by applicant]
US 20140286533A1 · Luo et al. · 2014 [cited by applicant]
US 20140297301A1 · Rock · 2014 [cited by applicant]
US 20150046182A1 · Kinney et al. · 2015 [cited by applicant]
US 20150119705A1 · Tochterman et al. · 2015 [cited by applicant]
US 20150190208A1 · Silveira · 2015 [cited by applicant]
US 20160004911A1 · Cheng et al. · 2016 [cited by applicant]
US 20160067007A1 · Piron et al. · 2016 [cited by applicant]
US 20160191887A1 · Casas · 2016 [cited by applicant]
US 20160239617A1 · Farooq et al. · 2016 [cited by applicant]
US 20160253472A1 · Pedersen et al. · 2016 [cited by applicant]
US 20160259888A1 · Liu et al. · 2016 [cited by applicant]
US 20170046835A1 · Tajbakhsh et al. · 2017 [cited by applicant]
US 20170053543A1 · Agrawal et al. · 2017 [cited by applicant]
US 20170071679A1 · Weir et al. · 2017 [cited by applicant]
US 20170112577A1 · Bonutti et al. · 2017 [cited by applicant]
US 20170119258A1 · Kotanko et al. · 2017 [cited by applicant]
US 20170132785A1 · Wshah et al. · 2017 [cited by applicant]
US 20170177806A1 · Fabian · 2017 [cited by applicant]
US 20170249432A1 · Grantcharov · 2017 [cited by applicant]
US 20170300651A1 · Strobridge · 2017 [cited by applicant]
US 20170312031A1 · Amanatullah et al. · 2017 [cited by applicant]
US 20180065248A1 · Barral et al. · 2018 [cited by applicant]
US 20180078195A1 · Sutaria et al. · 2018 [cited by applicant]
US 20180110398A1 · Schwartz et al. · 2018 [cited by applicant]
US 20180122506A1 · Grantcharov et al. · 2018 [cited by applicant]
US 20180158361A1 · Goll et al. · 2018 [cited by applicant]
US 20180174311A1 · Kluckner et al. · 2018 [cited by applicant]
US 20180174616A1 · Aguilar et al. · 2018 [cited by applicant]
US 20180197624A1 · Robaina et al. · 2018 [cited by applicant]
US 20180247024A1 · Divine et al. · 2018 [cited by applicant]
US 20180247128A1 · Alvi et al. · 2018 [cited by applicant]
US 20180247560A1 · Mackenzie et al. · 2018 [cited by applicant]
US 20180303552A1 · Ryan et al. · 2018 [cited by applicant]
US 20180322949A1 · Mohr et al. · 2018 [cited by applicant]
US 20180350144A1 · Rathod · 2018 [cited by applicant]
US 20180366231A1 · Wolf et al. · 2018 [cited by applicant]
US 20180368930A1 · Esterberg et al. · 2018 [cited by applicant]
US 20190005848A1 · Garcia Kilroy et al. · 2019 [cited by applicant]
US 20190006047A1 · Gorek et al. · 2019 [cited by applicant]
US 20190060893A1 · Evans et al. · 2019 [cited by applicant]
US 20190122330A1 · Saget et al. · 2019 [cited by applicant]
US 20190125361A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201136A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190201141A1 · Shelton, IV et al. · 2019 [cited by applicant]
US 20190206565A1 · Shelton, IV · 2019 [cited by applicant]
US 20190223961A1 · Barral et al. · 2019 [cited by applicant]
US 20190272917A1 · Couture et al. · 2019 [cited by applicant]
US 20190279765A1 · Giataganas et al. · 2019 [cited by applicant]
US 20190286652A1 · Habbecke et al. · 2019 [cited by applicant]
US 20190340963A1 · Kishima et al. · 2019 [cited by applicant]
US 20190347557A1 · Khan · 2019 [cited by applicant]
US 20190362834A1 · Venkataraman et al. · 2019 [cited by applicant]
US 20190362859A1 · Bhat · 2019 [cited by applicant]
US 20190365252A1 · Fernald et al. · 2019 [cited by applicant]
US 20190371456A1 · Page et al. · 2019 [cited by applicant]
US 20190380792A1 · Poltaretskyi et al. · 2019 [cited by applicant]
US 20200030044A1 · Wang et al. · 2020 [cited by applicant]
US 20200082934A1 · Venkataraman · 2020 [cited by applicant]
US 20200168334A1 · Mowery · 2020 [cited by applicant]
US 20200194111A1 · Venkataraman et al. · 2020 [cited by applicant]
US 20200211720A1 · Goldberg · 2020 [cited by applicant]
US 20200226751A1 · Jin et al. · 2020 [cited by applicant]
US 20200237452A1 · Wolf et al. · 2020 [cited by applicant]
US 20200258616A1 · Likosky et al. · 2020 [cited by applicant]
US 20200268457A1 · Wolf et al. · 2020 [cited by applicant]
US 20200268469A1 · Wolf et al. · 2020 [cited by applicant]
US 20200268472A1 · Wolf et al. · 2020 [cited by applicant]
US 20200273548A1 · Wolf et al. · 2020 [cited by applicant]
US 20200273552A1 · Wolf · 2020 [cited by examiner]
US 20200273557A1 · Wolf et al. · 2020 [cited by applicant]
US 20200273561A1 · Wolf et al. · 2020 [cited by applicant]
US 20200273575A1 · Wolf et al. · 2020 [cited by applicant]
US 20200273581A1 · Wolf et al. · 2020 [cited by applicant]
US 20200337648A1 · Saripalli et al. · 2020 [cited by applicant]
US 20210006752A1 · Kilroy et al. · 2021 [cited by applicant]
US 20210012868A1 · Wolf et al. · 2021 [cited by applicant]
US 20210015560A1 · Boddington et al. · 2021 [cited by applicant]
US 20210043308A1 · Sugie et al. · 2021 [cited by applicant]
US 20210158845A1 · Sethi et al. · 2021 [cited by applicant]
US 20210290046A1 · Nazareth et al. · 2021 [cited by applicant]
US 20210298869A1 · Wolf et al. · 2021 [cited by applicant]
US 20210307840A1 · Wolf et al. · 2021 [cited by applicant]
US 20210307864A1 · Wolf et al. · 2021 [cited by applicant]
US 20210313052A1 · Makrinich et al. · 2021 [cited by applicant]
US 20220115099A1 · Vollrath · 2022 [cited by examiner]
US 20220165403A1 · Asselmann et al. · 2022 [cited by applicant]
US 20220301674A1 · Wolf et al. · 2022 [cited by applicant]
US 20230094690A1 · Sharma · 2023 [cited by examiner]
US 20230385945A1 · Wolf · 2023 [cited by examiner]
US 20240212012A1 · Wang · 2024 [cited by examiner]
CA 2575759A1 · 2006 [cited by applicant]
CA 3049148A1 · 2018 [cited by applicant]
CN 109447723A · 2019 [cited by examiner]
WO 2015066565A1 · 2015 [cited by applicant]
WO 2017031175A1 · 2017 [cited by applicant]
WO 2018089816A2 · 2018 [cited by applicant]
WO 2019040705A1 · 2019 [cited by applicant]
WO 2019139478A1 · 2019 [cited by applicant]
WO 2019226182A1 · 2019 [cited by applicant]
Oukas, C. Video content analysis of surgical procedures. Surg Endosc 32, 553-568 (2018) . https://doi.org/10.1007/$00464-017-5878-1 (Year: 2018). [cited by applicant]
Hall, Dr. Jena Mae, “Enhancing surgical education using video playback: A case study on the influence of video playback on the nature and experience of feedback between supervising surgeons and surgical residents”, Quee… [cited by applicant]
Twinanda et al., RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations, 38(4) IEEE Trans Med Imaging 1069-1078 (Oct. 25, 2018) (Year: 2018). [cited by applicant]
Bhatia et al., Real-Time Identification of Operating Room State from Video, Conference: Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence 1761-1766 (Jul. 2007) (Year: 2007). [cited by applicant]
“Video Analysis: An Approach for Use in Health Care”; Mackenzie et al.; Handbook of Human Factors and Ergonomics in Health Care and Patient Safety; Aug. 3, 2011 (Year: 2011). [cited by applicant]
Handbook of Human Factors Cover Pages (with copyright date) (Year: 2011). [cited by applicant]
Bernd Munzer et al. (2017). “EndoXplore: A Web-Based Video Explorer for Endoscopic Videos”. 2017 IEEE International Symposium on Multimedia (ISM), p. 366-367. (Year: 2017). [cited by applicant]
Niitsu H, Hirabayashi N, Yoshinnitsu M, et al. Using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg Today. 201… [cited by applicant]
A. Jin et al., “Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks,” 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), 2018, pp. 691-69… [cited by applicant]
Hassan, A., Ghafoor, M., Tariq, S.A. et al. High Efficiency Video Coding (HEVC)-Based Surgical Telennentoring System Using Shallow Convolutional Neural Network. J Digit Imaging 32, 1027-1043 (2019). https://doi.org/10.1… [cited by applicant]