IP Library Granted Patent US 11,829,892
Granted Patent B2
US 11,829,892 · App. 18/181,529 · Granted Nov 28, 2023

Detecting and correcting anomalies in computer-based reasoning systems

Inventor: Christopher James Hazard (Raleigh, NC)
Assignee: Diveplane Corporation
G06N5/025G06F16/2465
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,829,892
App. No.
18/181,529
Granted
Nov 28, 2023
Kind
B2
Abstract

Techniques for detecting and correcting anomalies in computer-based reasoning systems are provided herein. The techniques can include obtaining current context data and determining a contextually-determined action based on the obtained context data and a reasoning model. The reasoning model may have been determined based on one or more sets of training data. The techniques may cause performance of the contextually-determined action and, potentially, receiving an indication that performing the contextually-determined action in the current context resulted in an anomaly. The techniques include determining a portion of the reasoning model that caused the determination of the contextually-determined action based on the obtained context data and causing removal of the portion of the model that caused the determination of the contextually-determined action, to produce a corrected reasoning model. Subsequently, second context data is obtained, a second action is determined based on that data and the corrected reasoning model, and the second contextually-determined action can be performed.

Claims (35)

1. A method comprising:

determining, using the one or more computing devices, a contextually-determined action for a control system based on obtained context data and a reasoning model, wherein the reasoning model was determined based on one or more sets of training data and the reasoning model relates to the control system, wherein the control system is a self-driving vehicle control system, wherein the one or more sets of training data include multiple context data and action data pairings, and wherein determining the contextually-determined action for the control system comprises determining, using a premetric, closest context data in the one or more sets of training data that is closest to a current context based on the premetric and determining an action paired with the closest context data as the contextually-determined action for the control system, wherein the premetric is a Minkowski distance measure of order less than one;

determining, using the one or more computing devices, whether performance of the contextually-determined action would result in an indication of an anomaly for the control system;

updating, using the one or more computing devices, a portion of the reasoning model associated with the closest context data in the one or more sets of training data that is closest to the current context based on the premetric in order to produce a corrected reasoning model, wherein updating the portion of the reasoning model comprises changing the action paired with the closest context data, wherein updating the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly to produce the corrected reasoning model comprises removing an association between the previously-identified closest context data and the action paired with the closest context data;

obtaining, using the one or more computing devices, subsequent contextual data for a second context for the control system;

determining, using the one or more computing devices, a second contextually-determined action for the control system based on the obtained subsequent contextual data and the corrected reasoning model; and

causing performance, using the one or more computing devices, of the second contextually-determined action for the control system, wherein the second contextually-determined action is a contextually-determined vehicle action to be taken by the self-driving vehicle control system.

2. The method of claim 1 , wherein the reasoning model is a case-based reasoning model.

3. The method of claim 1 , further comprising determining the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly by determining the previously-identified closest context data in the one or more sets of training data.

4. The method of claim 1 , wherein updating the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly to produce the corrected reasoning model comprises removing the closest context data and the paired action.

5. The method of claim 1 , wherein the method additionally comprises determining additional portions of the reasoning model that would cause the performance of the contextually-determined action that resulted in the indication of the anomaly in the current context and updating the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly to produce the corrected reasoning model comprises removing the additional portions of the reasoning model.

6. The method of claim 1 , further comprising determining whether performance of the contextually-determined action would result in the indication of the anomaly for the control system based at least in part on a receipt of an anomaly indication from a system separate from the control system.

7. A system for performing a machine-executed operation involving instructions, wherein said instructions are instructions which, when executed by one or more computing devices, cause performance of certain steps including:

determining, using the one or more computing devices, a contextually-determined action for a control system based on obtained context data and a reasoning model, wherein the reasoning model was determined based on one or more sets of training data and the reasoning model relates to the control system, wherein the control system is a self-driving vehicle control system, wherein the one or more sets of training data include multiple context data and action data pairings, and wherein determining the contextually-determined action for the control system comprises determining, using a premetric, closest context data in the one or more sets of training data that is closest to a current context based on the premetric and determining an action paired with the closest context data as the contextually-determined action for the control system, wherein the premetric is a Minkowski distance measure of order less than one;

determining whether performance of the contextually-determined action would result in an indication of an anomaly for the control system;

updating a portion of the reasoning model associated with the closest context data in the one or more sets of training data that is closest to the current context based on the premetric in order to produce a corrected reasoning model, wherein updating the portion of the reasoning model comprises changing the action paired with the closest context data, wherein updating the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly to produce the corrected reasoning model comprises removing an association between the previously-identified closest context data and the action paired with the closest context data;

obtaining subsequent contextual data for a second context for the control system;

determining a second contextually-determined action for the control system based on the obtained subsequent contextual data and the corrected reasoning model; and

causing performance of the second contextually-determined action for the control system, wherein the second contextually-determined action is to be taken by the self-driving vehicle control system.

8. The system of claim 7 , wherein the performed steps additionally comprise determining the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly by determining the previously-identified closest context data in the one or more sets of training data.

9. The system of claim 7 , wherein the performed steps additionally comprise determining additional portions of the reasoning model that would cause the performance of the contextually-determined action that resulted in the indication of the anomaly and updating the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly to produce the corrected reasoning model comprises removing the additional portions of the reasoning model.

10. The system of claim 7 , wherein the reasoning model is a case-based reasoning model.

11. The system of claim 7 , wherein the performed steps additionally comprise determining whether performance of the contextually-determined action results in the indication of the anomaly for the control system based at least in part on a receipt of an anomaly indication from a system separate from the control system.

12. One or more non-transitory computer readable media storing instructions which, when executed by one or more computing devices, cause performance of certain steps including:

determining, using the one or more computing devices, a contextually-determined action for a control system based on obtained context data and a reasoning model, wherein the reasoning model was determined based on one or more sets of training data and the reasoning model relates to the control system, wherein the control system is a self-driving vehicle control system, wherein the one or more sets of training data include multiple context data and action data pairings, and wherein determining the contextually-determined action for the control system comprises determining, using a premetric, closest context data in the one or more sets of training data that is closest to a current context based on the premetric and determining an action paired with the closest context data as the contextually-determined action for the control system, wherein the premetric is a Minkowski distance measure of order less than one;

determining whether performance of the contextually-determined action would result in an indication of an anomaly for the control system;

updating a portion of the reasoning model associated with the closest context data in the one or more sets of training data that is closest to the current context based on the premetric in order to produce a corrected reasoning model, wherein updating the portion of the reasoning model comprises changing the action paired with the closest context data, wherein updating the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly to produce the corrected reasoning model comprises removing an association between the previously-identified closest context data and the action paired with the closest context data;

obtaining subsequent contextual data for a second context for the control system;

determining a second contextually-determined action for the control system based on the obtained subsequent contextual data and the corrected reasoning model; and

causing performance of the second contextually-determined action for the control system, wherein the second contextually-determined action is a contextually-determined vehicle action to be taken by the self-driving vehicle control system.

13. The one or more non-transitory computer readable media of claim 12 , wherein the performed steps additionally comprise determining the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly by determining the previously-identified closest context data in the one or more sets of training data.

14. The one or more non-transitory computer readable media of claim 12 , wherein the performed steps additionally comprise determining additional portions of the reasoning model that would cause the performance of the contextually-determined action that resulted in the indication of the anomaly and updating the portion of the reasoning model that caused the determining of the contextually-determined action that resulted in the indication of the anomaly to produce the corrected reasoning model comprises removing the additional portions of the reasoning model.

15. The one or more non-transitory computer readable media of claim 12 , wherein the reasoning model is a case-based reasoning model.

16. The one or more non-transitory computer readable media of claim 12 , wherein the performed steps additionally comprise determining whether performance of the contextually-determined action would result in the indication of the anomaly for the control system based at least in part on a receipt of an anomaly indication from a system separate from the control system.

17. The one or more non-transitory computer readable media of claim 12 , wherein the performed steps additionally comprise determining whether performance of the contextually-determined action results in the indication of the anomaly for the control system based at least in part on a receipt of an anomaly indication from a system separate from the control system.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 20, 2023
From: HAZARDOUS SOFTWARE INC.
To: DIVEPLANE CORPORATION
Reel/Frame 065298/0678 →
CHANGE OF NAME Recorded Sep 28, 2023
From: DIVEPLANE CORPORATION
To: HOWSO INCORPORATED
Reel/Frame 065081/0559 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 21, 2023
From: HAZARD, CHRISTOPHER JAMES
To: HAZARDOUS SOFTWARE INC.
Reel/Frame 063044/0991 →
Continuity (4)
Continuation 17346583 · Jun 14, 2021
Continuation 15900398 · Feb 20, 2018
Continuation 15681219 · Aug 18, 2017
Related Publication 20230281481A1 · Sep 7, 2023