IP Library Granted Patent US 12,658,293
Granted Patent B2
US 12,658,293 · App. 19/295,129 · Granted Jun 16, 2026

Healthcare object recognition, systems and methods

Inventors: Patrick Soon-Shiong (Los Angeles, CA); John Zachary Sanborn (Santa Cruz, CA); Stephen Charles Benz (Santa Cruz, CA); Charles Joseph Vaske (Santa Cruz, CA)
Assignee: Nant Holdings IP, LLC
G16H10/60G16H40/67
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,658,293
App. No.
19/295,129
Granted
Jun 16, 2026
Kind
B2
Abstract

Healthcare object (HCO) discriminator systems and methods are presented. Systems can obtain a digital representation of a scene via a sensor interface. An HCO discriminator platform analyzes the digital representation to discriminate objects within the scene as being associated with a type of HCO or as being unrelated to a type of HCO. Once the HCO recognition platform determines that a type of HCO is relevant, it instantiates an actual HCO. The HCO can be routed to one or more destinations based on routing rules generated from a template or based on the manner in which the objects in the scene were discriminated.

Claims (49)

1 . A computer-based method of routing healthcare objects, the method comprising:

receiving, by at least one processor from at least one sensor, a digital representation of a scene, wherein the at least one sensor includes at least one electrocardiogram (EKG) medical sensor;

deriving, by the at least one processor, a set of discriminator characteristics via execution of one or more discriminator algorithms on the digital representation according to data modalities within the digital representation, the set of discriminator characteristics comprises a vector of discriminator characteristics, the vector of discriminator characteristics comprises multiple vector members, and each vector member corresponds to a different modality of data;

discriminating, by the at least one processor, objects in the scene by recognizing an object as a healthcare object type based on the set of discriminator characteristics;

instantiating, by the at least one processor, a healthcare object from the recognized object according to the healthcare object type;

routing, by the at least one processor, the healthcare object to a destination device according to routing rules associated with the healthcare object; and

causing, by the at least one processor, the destination device to enable rendering of a user interface on a display in response to receiving the healthcare object.

2 . The method of claim 1 , wherein the digital representation of the scene includes heart rate of a person associated with the at least one EKG medical sensor.

3 . The method of claim 1 , wherein routing the healthcare object to a destination device includes generating an alert that a person associated with the at least one EKG medical sensor is having a heart condition.

4 . The method of claim 1 , wherein the user interface is rendered on at least one of a cell phone or a browser device.

5 . The method of claim 1 , further comprising requesting, via the at least one processor, additional medically-relevant information via the user interface.

6 . The method of claim 5 , wherein requesting additional medically-relevant information via the user interface includes requesting a user to take a pulse of a person associated with the at least one EKG medical sensor.

7 . The method of claim 5 , further comprising, in response to requesting additional medically-relevant information via the user interface, receiving an input of additional medically-relevant information via the user interface.

8 . The method of claim 1 , further comprising transmitting a notification to a remote device of a physician associated with a person wearing the at least one EKG medical sensor.

9 . The method of claim 1 , wherein:

instantiating the healthcare object includes provisioning the healthcare object with destination attributes;

the destination attributes include the routing rules; and

the destination attributes comprise at least one destination address.

10 . The method of claim 9 , wherein the at least one destination address comprises a document object identifier.

11 . The method of claim 1 , wherein the modality of data comprises one of the following:

image features, audio features, and OCRed text.

12 . The method of claim 1 , wherein:

the set of discriminator characteristics comprises an N-tuple of characteristics; and

the N-tuple of characteristics comprises attribute names along with one or more corresponding values.

13 . The method of claim 1 , wherein the one or more discriminator algorithms include at least one of the following: SIFT, BRISK, GLOW, SURF, SLAM, vSLAM, a hidden markov model, optical character recognition, a support vector machine, and dynamic time warping.

14 . The method of claim 1 , further comprising recognizing objects in the scene as non-healthcare objects.

15 . The method of claim 1 , wherein:

instantiating the healthcare object includes populating fields of a healthcare object template; and

fields of the healthcare object template include required fields or optional fields.

16 . The method of claim 1 , wherein:

instantiating the healthcare object includes causing a mechanical turk computer system to construct a mechanical turk task; and

the method further incudes populating fields of the healthcare object based on information resulting from the mechanical turk task.

17 . The method of claim 1 , further comprising encrypting, by the at least one processor, the healthcare object.

18 . A system for routing healthcare objects, the system comprising:

memory configured to store computer-executable instructions; and

at least one processor configured to execute the computer-executable instructions to:

receive, from at least one sensor, a digital representation of a scene, wherein the at least one sensor includes at least one electrocardiogram (EKG) medical sensor;

derive a set of discriminator characteristics via execution of one or more discriminator algorithms on the digital representation according to data modalities within the digital representation, the set of discriminator characteristics comprises a vector of discriminator characteristics, the vector of discriminator characteristics comprises multiple vector members, and each vector member corresponds to a different modality of data;

discriminate objects in the scene by recognizing an object as a healthcare object type based on the set of discriminator characteristics;

instantiate a healthcare object from the recognized object according to the healthcare object type;

route the healthcare object to a destination device according to routing rules associated with the healthcare object; and

cause the destination device to enable rendering of a user interface on a display in response to receiving the healthcare object.

19 . A non-transitory computer-readable media configured to store computer-executable instructions which, when executed by at least one processor, cause the at least one processor to:

receive, from at least one sensor, a digital representation of a scene, wherein the at least one sensor includes at least one electrocardiogram (EKG) medical sensor;

derive a set of discriminator characteristics via execution of one or more discriminator algorithms on the digital representation according to data modalities within the digital representation, the set of discriminator characteristics comprises a vector of discriminator characteristics, the vector of discriminator characteristics comprises multiple vector members, and each vector member corresponds to a different modality of data;

discriminate objects in the scene by recognizing an object as a healthcare object type based on the set of discriminator characteristics;

instantiate a healthcare object from the recognized object according to the healthcare object type;

route the healthcare object to a destination device according to routing rules associated with the healthcare object; and

cause the destination device to enable rendering of a user interface on a display in response to receiving the healthcare object.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 11, 2025
From: SOON-SHIONG, PATRICK; SANBORN, JOHN ZACHARY; BENZ, STEPHEN; VASKE, CHARLES JOSEPH
To: NANT HOLDINGS IP, LLC
Reel/Frame 071983/0166 →
Continuity (10)
Continuation 19079691 · Mar 14, 2025
Continuation 18821076 · Aug 30, 2024
Continuation 18371885 · Sep 22, 2023
Continuation 18123578 · Mar 20, 2023
Continuation 17494608 · Oct 5, 2021
Continuation 16460325 · Jul 2, 2019
Continuation 15818249 · Nov 20, 2017
Continuation 14350048
Provisional Application 61543922 · Oct 6, 2011
Related Publication 20250364095A1 · Nov 27, 2025
References Cited (76)
US 7016532B2 · Boncyk et al. · 2006 [cited by applicant]
US 7403652B2 · Boncyk et al. · 2008 [cited by applicant]
US 7477780B2 · Boncyk et al. · 2009 [cited by applicant]
US 7565008B2 · Boncyk et al. · 2009 [cited by applicant]
US 7627334B2 · Cohen et al. · 2009 [cited by applicant]
US 7680324B2 · Boncyk et al. · 2010 [cited by applicant]
US 7881529B2 · Boncyk et al. · 2011 [cited by applicant]
US 7899243B2 · Boncyk et al. · 2011 [cited by applicant]
US 7899252B2 · Boncyk et al. · 2011 [cited by applicant]
US 7996186B2 · Nakagawa et al. · 2011 [cited by applicant]
US 8218873B2 · Boncyk et al. · 2012 [cited by applicant]
US 8224077B2 · Boncyk et al. · 2012 [cited by applicant]
US 8224078B2 · Boncyk et al. · 2012 [cited by applicant]
US 9824184B2 · Soon-Shiong et al. · 2017 [cited by applicant]
US 10388409B2 · Soon-Shiong et al. · 2019 [cited by applicant]
US 11170882B2 · Soon-Shiong et al. · 2021 [cited by applicant]
US 12087412B1 · Sorkey et al. · 2024 [cited by applicant]
US 20010051879A1 · Johnson et al. · 2001 [cited by applicant]
US 20050149360A1 · Galperin · 2005 [cited by applicant]
US 20050158767A1 · Haskell et al. · 2005 [cited by applicant]
US 20050210044A1 · Hills et al. · 2005 [cited by applicant]
US 20060195339A1 · Backhaus et al. · 2006 [cited by applicant]
US 20060226957A1 · Miller et al. · 2006 [cited by applicant]
US 20060293925A1 · Flom · 2006 [cited by applicant]
US 20070065017A1 · Kotwaliwale et al. · 2007 [cited by applicant]
US 20070072250A1 · Kim et al. · 2007 [cited by applicant]
US 20070239482A1 · Finn et al. · 2007 [cited by applicant]
US 20070265533A1 · Tran · 2007 [cited by applicant]
US 20070273504A1 · Tran · 2007 [cited by applicant]
US 20070276270A1 · Tran · 2007 [cited by applicant]
US 20080001735A1 · Tran · 2008 [cited by applicant]
US 20080004904A1 · Tran · 2008 [cited by applicant]
US 20080077604A1 · Bharara · 2008 [cited by applicant]
US 20080147554A1 · Stevens et al. · 2008 [cited by applicant]
US 20090005650A1 · Angell et al. · 2009 [cited by applicant]
US 20090006125A1 · Angell et al. · 2009 [cited by applicant]
US 20090227876A1 · Tran · 2009 [cited by applicant]
US 20090227877A1 · Tran · 2009 [cited by applicant]
US 20090318779A1 · Tran · 2009 [cited by applicant]
US 20100105034A1 · Hutton et al. · 2010 [cited by applicant]
US 20100172555A1 · Hasezawa et al. · 2010 [cited by applicant]
US 20100309198A1 · Kauffmann · 2010 [cited by examiner]
US 20110105034A1 · Senders et al. · 2011 [cited by applicant]
US 20110115624A1 · Tran · 2011 [cited by applicant]
US 20110122138A1 · Schmidt et al. · 2011 [cited by applicant]
US 20110125519A1 · Dhoble · 2011 [cited by applicant]
US 20110125521A1 · Dhoble · 2011 [cited by applicant]
US 20110133903A1 · Alsafadi · 2011 [cited by applicant]
US 20110181422A1 · Tran · 2011 [cited by applicant]
US 20130046547A1 · Drucker et al. · 2013 [cited by applicant]
US 20130314522A1 · Ravid · 2013 [cited by examiner]
US 20140114675A1 · Soon-Shiong · 2014 [cited by applicant]
US 20220028508A1 · Soon-Shiong et al. · 2022 [cited by applicant]
US 20240013874A1 · Soon-Shiong et al. · 2024 [cited by applicant]
US 20240420814A1 · Soon-Shiong et al. · 2024 [cited by applicant]
US 20250210164A1 · Soon-Shiong et al. · 2025 [cited by applicant]
GB 2403041A · 2004 [cited by applicant]
WO 2005076772A2 · 2005 [cited by applicant]
Gossow, David, Peter Decker, and Dietrich Paulus, “An evaluation of open source SURF implementations.” Robot Soccer World Cup. Springer, Berlin, Heidelberg, 2010, pp. 169-179. [cited by applicant]
Leutenegger, Stefan, Margarita Chli, and Roland Y. Siegwart, “BRISK: Binary robust invariant scalable keypoints.” IEEE, 2011, (8 pages). [cited by applicant]
Lowe, David G., “Object recognition from local scale-invariant features.” iccv. Ieee, 1999 (8 pages). [cited by applicant]
Mikolajczyk, Krystian, and Cordelia Schmid, “A performance evaluation of local descriptors.” IEEE transactions on pattern analysis and machine intelligence 27.1O (2005): (34 pages). [cited by applicant]
Mountney, Peter, et al., “Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Be… [cited by applicant]
Office Action issued in Canadian Application No. 2,851,426 dated Apr. 27, 2018, 4 pages. [cited by applicant]
Chow et al., “Controlling Data in the Cloud: Outsourcing Computation without Outsourcing Control,” CCSW 2009, 6 pages. [cited by applicant]
Li et al., “Fuzzy Keyword Search over Encrypted Data in Cloud Computing,” INFOCOM, 2010 Proceedings IEEE, Mar. 14-19, 2010, 5 pages. [cited by applicant]
“NTT and Mitsubishi Electric Develop Advanced Encryption Scheme to Increase Cloud Computing Security,” Nippon Telegraph and Telephone Corporation, NTT News Release, http:i/wv,Nv'.nttco.jpinews20i0/1OO?e/100728a.html, Ju… [cited by applicant]
European Search Report issued in European Application No., 12838194.4 dated Jul. 7, 2015, 6 pages. [cited by applicant]
International Search Report and Written Opinion issued in International Application No. PCT/US2012/059155 dated Dec. 27, 2012, 8 pages. [cited by applicant]
Office Action from corresponding U.S. Appl. No. 15/818,249 dated Nov. 27, 2018. [cited by applicant]
Office Action from corresponding U.S. Appl. No. 15/818,249 dated Jun. 19, 2018. [cited by applicant]
Office Action from corresponding U.S. Appl. No. 15/818,249 dated Feb. 20, 2018. [cited by applicant]
Office Action from corresponding U.S. Appl. No. 14/350,048 dated Feb. 22, 2017. [cited by applicant]
Office Action from corresponding U.S. Appl. No. 14/350,048 dated Nov. 3, 2016. [cited by applicant]
Office Action from corresponding U.S. Appl. No. 14/350,048 dated May 20, 2016. [cited by applicant]
Office Action from corresponding U.S. Appl. No. 16/460,325 dated Oct. 27, 2020. [cited by applicant]