IP Library › Granted Patent US 12,718,567
Granted Patent B2
US 12,718,567 · App. 18/035,078 · Granted Aug 25, 2026

Systems and methods for surgical data censorship

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,718,567
App. No.
18/035,078
Filed
May 2, 2023
Granted
Aug 25, 2026
Kind
B2
Examiner
ZHAO, LEI
Art Unit
2668
USPC
382/128
Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 2, 2023
From: WANG, ZIHENG; BHATTACHARYYA, KIRAN; BRETZ, SAMUEL; JARC, ANTHONY; LIU, XI; VILLA, ANDREA; ZIA, ANEEQ
To: INTUITIVE SURGICAL OPERATIONS, INC.
Reel/Frame 063513/0642 →
Continuity (2)
Provisional Application 63116906 · Nov 22, 2020
Related Publication 20230316756A1 · Oct 5, 2023
References Cited (47)
US 9283675B2 · Hager et al. · 2016 [cited by applicant]
US 10799090B1 · Venkataraman · 2020 [cited by examiner]
US 20110026840A1 · Tao · 2011 [cited by examiner]
US 20200154080A1 · Miller et al. · 2020 [cited by applicant]
US 20200372180A1 · Venkataraman · 2020 [cited by examiner]
Viola, Robust Real-Time Face Detection, International Journal of Computer Vision 57(2), 137-154, 2004 (Year: 2004). [cited by examiner]
Louizos, Multiplicative Normalizing Flows for Variational Bayesian Neural Networks, Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, PMLR 70, 2017 (Year: 2017). [cited by examiner]
“Advantech—Enabling an Intelligent Planet—Accelerating the Intelligent Video Infrastructure” webpage, as of Oct. 20, 2020. Retrieved from Internet Archive Wayback Machine [URL: https://web.archive.org/web/20201020234151… [cited by applicant]
Brynolfsson, P., et al., “Haralick Texture Features from Apparent Diffusion Coefficient (ADC) MRI Images Depend on Imaging and Pre-processing Parameters,” Scientific Reports, Jun. 2017, vol. 7(1): 4041. Retrieved from [… [cited by applicant]
“Caffe”, framework webpage [URL: https://caffe.berkeleyvision.org/] as of Nov. 18, 2020. Retrieved from Internet Archive Wayback Machine [URL: https://web.archive.org/web/20201118052534/ http://caffe.berkeleyvision.org/… [cited by applicant]
Carreira, J., et al., “Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset,” Computer Vision and Pattern Recognition, 2017, pp. 6299-6308. Retrieved from [URL: https://openaccess.thecvf.com/content_cvpr_… [cited by applicant]
Castellano, G., et al., “Texture Analysis of Medical Images,” indicated as corresponding to Clinical Radiology, Dec. 2004, vol. 59 (12), pp. 1061-1069. Retrieved from [URL: http://facweb.cs.depaul.edu/research/vc/medix/… [cited by applicant]
Chawla, N.V., et al. “SMOTE: Synthetic Minority Over-sampling Technique.” arXiv preprint arXiv:1106.1813 (2011), though indicated as appearing earlier in Journal of Artificial Intelligence Research. 2002, pp. 321-357. R… [cited by applicant]
“Scikit-learn Machine Learning in Python” toolset webpage [URL: https://scikit-learn.org/stable/] as of Nov. 19, 2020. Retrieved from Internet Archive Wayback Machine [URL: https://web.archive.org/web/20201119151816/htt… [cited by applicant]
“DMLC Decord v0.4.2” GitHub Repository Webpage, indicated as published Nov. 13, 2020. Retrieved from [URL: https://github.com/dmlc/decord/tree/v0.4.2] on May 9, 2023, 03 pages. [cited by applicant]
“Entropy (information theory)—Wikipedia” webpage, indicales last edit as Feb. 6, 2022, as received with the International Search Report for PCT/US2021/05992 as reference XP55889861AI, which indicated as retrieved from t… [cited by applicant]
Flouty, E., et al., “FaceOff: Anonymizing Videos in the Operating Rooms.” arXiv preprint arXiv:1808.04440 (2018). Retrieved from [URL: https://arxiv.org/pdf/1808.04440] on Jul. 6, 2023, 08 pages. [cited by applicant]
Haralick, R.M., et al., “Textural Features for Image Classification,” 1973, indicated as reprinted from IEEE Transactions on Systems, Man, and Cybernetics, Nov. 1973, vol. 3(6), pp. 610-621. Retrieved from [URL: http://… [cited by applicant]
He, K., et al. “Identity Mappings in Deep Residual Networks.” Sep. 2016, In European Conference on Computer Vision (pp. 630-645). Springer. Retrieved from [URL: https://link.springer.com/content/pdf/10.1007/978-3-319-46… [cited by applicant]
Hu, M-K., “Visual Pattern Recognition by Moment Invariants,” indicated as appearing in IRE Transactions on Information Theory, 1962, vol. 8(2), pp. 179-187. Retrieved from [URL: https://www.cs.utexas.edu/~grauman/course… [cited by applicant]
“Imbalanced-learn v0.7.0” package repository webpage, Jun. 9, 2020. Retrieved from [URL: https://pypi.org/project/imbalanced-learn/0.7.0/] on May 9, 2023, 03 pages. [cited by applicant]
International Preliminary Report on Patentability for Application No. PCT/US2021/059952, mailed Jun. 1, 2023, 10 pages. [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US2021/059952, mailed Mar. 4, 2022, 13 pages. [cited by applicant]
Issenhuth, T., et al. “Face detection in the operating room: Comparison of state-of-the-art methods and a self-supervised approach,” International journal of computer assisted radiology and surgery 14 (2019): 1049-1058.… [cited by applicant]
Jia, Y., et al. “Caffe: Convolutional Architecture for Fast Feature Embedding.” arXiv preprint arXiv:1408.5093 (2014) Retrieved from [URL: https://arxiv.org/pdf/1408.5093] on Jul. 6, 2023, 04 pages. [cited by applicant]
“Keras” Application Programming Interface webpage [URL: https://keras.io/], as of Nov. 19, 2020. Retrieved from Internet Archive Wayback Machine [URL: http://web.archive.org/web/20201119075317/https://keras.io/] on May … [cited by applicant]
Lemaitre, G., et al., “Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Leaming,” Journal of Machine Learning Research, 2017, vol. 18, pp. 1-5. Retrieved from [URL: https://www.jm… [cited by applicant]
Li, J., et al., “A Comprehensive Review of Current Local Features for Computer Vision,” indicated as appearing in Neurocomputing, Jun. 2008, vol. 71(10-12), pp. 1771-1787. Retrieved from [URL: https://www.labxing.com/fi… [cited by applicant]
“LIBSVM—A Library for Support Vector Machines” library webpage [URL: https://www.csie.ntu.edu.tw/~cjlin/libsvm/], as of Nov. 9, 2020. Retrieved from Internet Archive Wayback Machine [URL: https://web.archive.org/web/202… [cited by applicant]
Maier-Hein L., et al., “Surgical data science: enabling next-generation surgery.” arXiv preprint arXiv:1701.06482 (2017). Retrieved from [URL: https://arxiv.org/pdf/1701.06482] on Jul. 6, 2023, 10 pages. [cited by applicant]
Munzer, B., et al., “Content-based Processing and Analysis of Endoscopic Images and Videos: A Survey,” Multimedia Tools and Applications, 2018, vol. 77(1), pp. 1323-1362. Retrieved from [URL: https://link.springer.com/c… [cited by applicant]
Munzer, B., et al., “Relevance Segmentation of Laparoscopic Videos,” IEEE International Symposium on Multimedia, 2013, pp. 84-91. Retrieved from an account with “Reprints Desk” [URL: https://www.researchsolutions.com/] … [cited by applicant]
“Read the Docs imblearn.over_sampling.SMOTE” API webpage, [URL: https://imbalanced-learn.readthedocs.lo/en/stable/generated/imblearn.over_sampling.SMOTE.html] as of Nov. 11, 2020. Retrieved from Internet Archive Wayback… [cited by applicant]
Sandler, M., et al., “MobileNetV2: Inverted Residuals and Linear Bottlenecks.” arXiv preprint arXiv:1801.04381 (2019). Retrieved from [URL: https://arxiv.org/pdf/1801.04381] on Jul. 6, 2023, 14 pages. [cited by applicant]
Simonyan, K., et al., “Very deep convolutional networks for large-scale image recognition.” arXiv preprint arXiv:1409.1556 (2015). Indicated as being published as a conference paper at ICLR, 2015. Retrieved from [URL: h… [cited by applicant]
Stanek, S.R., et al., “Automatic Real-time Detection of Endoscopic Procedures Using Temporal Features,” Computer Methods and Programs in Biomedicine, Nov. 2012, vol. 108(2), pp. 524.535. Retrieved from an account with “… [cited by applicant]
Tao, L., et al., “Surgical Gesture Segmentation and Recognition,” Medical Image Computing and Computer-assisted Intervention, 2013, vol. 16 (Part III), pp. 339-346. Retrieved from [URL: https://link.springer.com/content… [cited by applicant]
“Tensorflow” platform webpage [URL: https://www.tensorflow.org/] as of Nov. 18, 2020. Retrieved from Internet Archive Wayback Machine [URL: https://web.archive.org/web/20201118224711/https://www.tensorflow.org/] on May … [cited by applicant]
“Torch, A Scientific Computing Framework for LUAJIT” framework webpage [URL: http://torch.ch/] as of Nov. 18, 2020. Retrieved from Internet Archive Wayback Machine [URL: https://web.archive.org/web/20201118030716/ http:… [cited by applicant]
Twinanda, A., “EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos,” in IEEE Transactions on Medical Imaging, vol. 36, No. 1, pp. 86-97. Jan. 2017. Retrieved from an account with “Reprints Desk” [U… [cited by applicant]
Vedula, S.S., et al., “Objective Assessment of Surgical Technical Skill and Competency in the Operating Room,” Annual Review of Biomedical Engineering, Jun. 2017, vol. 19, pp. 301-325. Retrieved from [URL: https://www.n… [cited by applicant]
Vedula, S.S., et al., “Surgical Data Science: the New Knowledge Domain,” Innovative Surgical Sciences, Apr. 2017, vol. 2(3), pp. 109-121. Retrieved from https://www.ncbi.nim.nih.gov/pmc/articles/PMC5602563/pdf/iss-2-201… [cited by applicant]
“VEGA-6301 Datasheet” [online], indicated as last updated Jan. 10, 2017. Retrieved from [URL: https://advdownload.advantech.com/productfile/PIS/VEGA- 6301/Product%20-%20Datasheet/VEGA-6301_OS(Oct. 1, 2017) 2017011609283… [cited by applicant]
Vertut, J., and Coiffet, P., “Robot Technology: Teleoperation and Robotics Evolution and Development,” English translation, Prentice-Hall, Inc., Inglewood Cliffs, NJ, USA 1986, vol. 3A, 332 pages. [cited by applicant]
Wang, Z., et al., “Automated Video Scrubbing When Endoscopes Are Out of Body during Minimally Invasive Surgery,” submitted Feb. 7, 2020 to the 2020 International Symposium on Medical Robotics (ISMR2020), 01 page. [cited by applicant]
Zia, A., et al. “Surgical activity recognition in robot-assisted radical prostatectomy using deep learning.” Medical Image Computing and Computer Assisted Intervention—MICCAI 2018: 21st International Conference, Granada… [cited by applicant]
Zohar, M., et al., “Accurate Detection of Out of Body Segments in Surgical Video using Semi-Supervised Learning.” Proceedings of Machine Learning Research, Jul. 2020, vol. 121, pp. 923-936 (“Jul. 6-8, 2020” conference d… [cited by applicant]